High-fidelity sensor models from Sony have been integrated into rFpro’s software – reducing the need to collect physical data during sensor development, and facilitating testing and training of ADAS and AV perception systems in a virtual world. The models augment rFpro’s recently developed ray tracing rendering technology that is stated to replicate how a vehicle’s sensor system perceives its environment more accurately than ever before.
“Sony Semiconductor Solutions has been a crucial collaborator in the development of our recently launched ray tracing technology and rFpro’s Multi-Exposure Camera technology, which accurately replicates what cameras ‘see’ for the first time,” said Matt Daley, rFpro operations director. “By working closely with Sony and integrating its sensor models into our technology we have been able to achieve a high level of correlation in the simulation. As a result, together, we are helping the industry reduce its reliance on collecting real-world data, which is expensive and time intensive.”
Kenji Onishi, deputy senior general manager, automotive business division, Sony Semiconductor Solutions Corporation, said, “The collaboration between rFpro and Sony Semiconductor Solutions will provide an automotive-grade End2End simulation pipeline to the ADAS perception system developers. Sony has prepared a sensor model based on the internal architecture of the image sensors used in camera systems to achieve automotive-grade fidelity.”
Signal processing, LED flicker mitigation, spectral effects and image sensor control, such as motion blur and rolling shutter, are some examples of the phenomena replicated by Sony’s sensor model, according to the companies. Sony’s interface between the sensor model and the rendering system is said to be very efficient and enables high-fidelity simulations to run quickly. The interface is common across all Sony’s automotive image sensors enabling users to quickly transition to a new generation of sensor models as they become available.
Sony is the first to partner with rFpro on its ray tracing technology, which is a software-in-the-loop solution used to generate synthetic training data.
Daley added, “With the rFpro rendering and Sony’s sensor model pipeline, motion blur, rolling shutter characteristics, colour filter characteristics and signal processing of the physical sensors exist in the simulated images. It also allows the LED flicker mitigation feature in the image sensor to be replicated on the sensor model and to be fully exercised in simulation. It enables the sensor model to interact with the fast-flickering LED light sources simulated on rFpro’s high fidelity assets and rendering system, such as traffic lights and vehicle brake lights. Accurately replicating these phenomena is critical to achieving high levels of correlation with the real world.”
More on rFpro’s ray tracing rendering technology here.
