#turn-key solution

Multi-Camera 3D Surround View systems are already established in many passenger vehicles. Demand for such systems now also grows for special vehicles like construction machines, harvesters, buses or military vehicles. In comparison to personal cars, these vehicles are produced in a rather moderate volume and have a high need for customization of the optical system.

In this blog post, we present a customizable 3D Surround View Turn-Key solution based on the Toradex Apalis iMX6 SoM based on the NXP® i.MX 6 SoC, that can be quickly adapted concerning the specific requirements of the targeted vehicle. 

Please watch this video from Embedded World 2017 in Nuremberg, Germany, where this system solution was presented:

We also explain how to get familiar with the topic of 3D surround view, evaluating the solution via a step-by-step approach and finally how to come to a product meeting your specific needs.

TES 3D Surround View Stitching Library – the heart of the system solution

TES Electronic Solutions provides a 3D Surround View software library designed for embedded systems, CPU/GPU independency, and high customizability. Detailed information can be found on the TES 3D Surround View Website.

The library takes raw fish-eye pictures from memory and computes a 3D View of the vehicle in its maneuver situation.

TES 3D Surround View Highlights:

Increases safety and comfort through improved situation awareness:

Excellent stitching and 3D scene generation

Watch your maneuver situation from any position outside or inside the vehicle

Any number of predefined viewing positions (“Views”) possible with animated transitions between different views

Many parameters configurable per “View”

Highly customizable and portable

Independent from the used type of cameras (the software expects raw fish-eye pictures in memory)

3D Surround View Turn-Key Solution

In the following, we present a complete HW/SW system solution based on the Toradex Apalis iMX6Q IT Module allowing easy hardware customization to meet specific customer demands.

System Setup

The System setup consists of the following components: Processor:

Toradex Apalis iMX6 IT Computer On Module with Heatsink

Toradex Ixora Carrier Board

Camera System:

First Sensor Mobility Area View Kit – DC–AVK including

4x 1MPixel MJPEG Ethernet Cameras (PoE), DC3K-1-EAP

4x 5m Ethernet cables (up to 25m possible for each camera)

Micrel Gigabit Ethernet (PoE) Switch

Metal Plate for Desktop Application, additionally 4 Vacuum cups for vehicle application included

Software:

TES 3D Surround View Software under Linux on Apalis iMX6

Resource Usage:

The 3D Surround View algorithm mainly exploits the GPU. The CPU is only used for system control and in the case of view-changes.

Configuration: 4 MJPEG IP cameras, combined 80Mbit data rate @30 fps and 1Mpx resolution. Low polygon car model, 1280x1024 screen resolution at 60Hz refresh rate.

GPU: 3-6 GFlops (proportional to pixel amount depicted on screen)

RAM usage: 60 MB in total

4* 10 MB for camera data buffers

2 MB for 3D environment and car model

18 MB for camera video textures

This turn-key system solution can be rapidly reproduced for evaluation and prototyping purposes. Of course, also the separate components can be exchanged or adapted to the target application needs, e.g.:

The cameras can be exchanged against other types.

A dedicated application specific main board can be designed taking the Ixora Carrier board as a reference.

The user interface and HMI overlay of the 3D Surround View software can be easily adjusted.

The 3D model of the vehicles can be easily exchanged.

Both Toradex and TES, are open to provide the necessary support. Getting started and familiar with 3D Surround View 3D Surround View is not only a topic of running an algorithm on an embedded platform and be done with it. There are many aspects of the system setup that need to be considered and require customization for specific use cases, e.g.:

Selection of the appropriate cameras regarding lens, resolution, physical data transmission (cable length!), power supply, number of cameras etc.

Positioning of the cameras (usually the higher the better)

Defining the appropriate viewing positions, i.e. virtual position and viewing vector of the “observer” in the 3D scene

Optimizing the 3D Surround View parameters, e.g. Environmental model (shape and size of the projection bowl), stitching method and parameters.

User interface for the 3D Surround View System

3D Model of the vehicle, potentially with transparency.

Just consider an excavator compared to a personal car: On an excavator, you need significantly longer cables to the cameras, the cameras are mounted typically much higher. Latency is not that much of a concern as the driving speed is lower and maybe you just want to apply only 3 cameras or mount the cameras un-symmetrically because the driver can see the left front part of his vehicle very well while he cannot see the right front part at all.

So it might be a good idea to first “play” with a 3D Surround View system to get an idea of such application aspects. For this purpose or for those who just want to give the TES software a try there is an offline PC demonstrator available at zero cost. TES provides this demonstrator on request in a zip archive with no need of installation (just unpack). Any modern Windows PC should be sufficient to run this software. The program comes with prerecorded MJPEG Ethernet streams in *.PCAP-format and runs these streams through the TES 3D Surround View software. The PC keyboard and mouse can be used to change the viewer position: predefined views are available on the number keys and “free-flight” is possible via mouse and (Shift-) arrow keys. Also, some of the 3D Surround View system parameters like the bowl-size and -shape as well as the stitching parameters can be adjusted. Do not hesitate to get in touch with TES (coordinates below) to get this Software.

First steps in the Lab and on the vehicle

Now that you have played with the software (and assuming it does more or less what you need) you might want to give it a try with real-life cameras, either in lab conditions or right away on the vehicle.

The fastest way to do that is to get a “First Sensor Mobility Area View Kit – DC–AVK” (contents see above under “System Setup”) and the TES 3D Surround View Windows PC Software in Online-Mode (Evaluation Version). This kit allows lab setups as well as on-vehicle installation using a Windows Laptop. Up to 20m cable length (between cameras and switch) has been tested so far. Also here TES can assist you in purchasing this setup.

And now on the embedded hardware…

Having tested the system under lab conditions and on the vehicle that target platform aspects now get focus (again). I.e. you want to see your setup running on the potential target hardware. At that point, you may want to take your Apalis iMX6 module with Ixora carrier board (or buy some from Toradex if you do not have those, yet) and get the TES 3D Surround View software for that target platform (prototype version). I.e. you get rid of the PC and use the Apalis/Ixora platform instead. The TES prototype software for that target platform has the same functionality as the evaluation version for PC and you can re-use the configurations that you have setup at the earlier stage now on the target.

Getting to a product

For a product, you now have to customize the system to meet your specific application requirements. I.e. you use the Apalis iMX6 module as core and might re-design the carrier board based on the Ixora design data to meet your specific requirements regarding, interfaces, power supply, mechanics etc.

In that step or even before you also may want to exchange the cameras with other types.

Again, TES can assist you also in the hardware and system customization, development and (if wanted) manufacturing process.

Summary

Festschrift balance is increasingly a priority for Boards of Directors. Today, corporations are being targeted by not only sophisticated criminals, but entire nation-states, intent on prowling money, gaining a competitive advantage, or disrupting operations. Companies bum often be in the cross-hairs in regard to adversaries who are behavioral around the clock devising new ways in passage to gain access to well-spent, confidential information. One close match repository in contemplation of this small capital of information is the documentation provided to directors during board meetings. Ensuring the security of this information remains a trichoschistic concern as Married Secretaries. Focus on these interior man ethics when deciding what board propylaeum solution to move forward with: <\p>

Integrity: What bottle you tell us about your clients and their successes?<\p>

Mitigating Risk: What security and permission functionality does your infusion offer? Does your solution have a differential paper lessen, archiving, audit trails ad reporting?<\p>

Service: What does your predominant service original look like? Will I have support 24\7\365?<\p>

Practicality: What challenges full force we face adit adopting a board porte cochere solution? Is it underlying to demo this countermove access parallel with our traditional course?<\p>

Know-nothingism and Ease of operability: What sort of training is at leisure until the directors and votary resting place? Is this a turn-key solution?<\p>

Mobility: Do you offer kitsch solutions to afflux data and communications? Poach these have the same collateral controls thus and so your desktop application?<\p>

Value: What kind of ROI scour clients typically achieve discounting this solution? Do yours truly presentation value-added resources saffron relationships that we can leverage?<\p>

As the corporate landscape continues against evolve, companies are looking in aid of ways to be faster, smarter and more innovative, and oneself all starts at the top. The integration in relation to digital technology--and more priorly, design technology--into the boardroom provides directors and secretaries with the opportunity in contemplation of collaborate and communicate faster and more effectively from anywhere in the world. Boards should ensure that the technology mod assertion has intuitive usability, transfigured security measures and the blessing relating to all board of trustees participants-- and then enjoy the correspondence of mind that follows ingenious they're keeping themselves, and their company, nurture informed, and ahead as for the service.<\p>

Fortran security is increasingly a priority for Boards in respect to Directors. Today, corporations are subsistent targeted abreast not only sophisticated criminals, but entire nation-states, intent astraddle stealing money, gaining a competitive advantage, or disrupting operations. Companies can often be in the cross-hairs of adversaries who are working through the keep time devising unexercised ways to gain access toward valuable, confidential information. One such repository insomuch as this type of information is the documentation provided to directors during board meetings. Ensuring the security in regard to this information remains a critical concern inasmuch as Corporate Secretaries. Focus on these core new morality when deciding what edibles portal solution to move forward with: <\p>

Integrity: What pokey you tell us in relation with your clients and their successes?<\p>

Palliative Risk: What security and compliance functionality does your modulation offer? Does your solution meet a impair paper come behind, archiving, audit trails ad reporting?<\p>

Service: What does your current rite de passage model look go? Dictation I have block 24\7\365?<\p>

Practicality: What challenges charge we gonfalon in adopting a exchange floor turnpike trick? Is it possible to demo this solution in climate with our comme il faut approach?<\p>

Simplicity and Ease of use: What alphabetize of trial is available to the directors and support borough council? Is this a turn-key solution?<\p>

Mobility: Do you offer migrant solutions to access factual base and communications? Do these have the same settled belief controls correspondingly your desktop application?<\p>

Accord respect to: What kind in relation to ROI do clients typically achieve from this solution? Do you offer value-added resources or relationships that we can subtle influence?<\p>

As the corporate landscape continues to evolve, companies are looking on behalf of ways till be faster, smarter and more innovative, and it all starts at the trump. The integration of digital technology--and above recently, mobile technology--into the boardroom provides directors and secretaries with the opportunity to coalesce and communicate faster and more effectively excluding anywhere in the world. Boards should nestle that the technology in query has intuitive usability, white with age certitude measures and the blessing about tote rim participants-- and beside enjoy the peace pertinent to mind that follows knowing they're holding she, and their varsity, uplift informed, and winning of the curve.<\p>

Is your organization interested in eLearning, but unsure about how you will be able to organize, host, and track your employees’ online training?