Recently the simulation of sensor data of advanced driver assistance systems (ADAS) becomes more and more important. In  a theoretical approach for the approval of autonomous driving functions is given. It is presented using the example of an autonomous highway pilot. Deadly accidents are happening every 662 Million kilometers on German motorways. It is said that this situation needs to be reconstructed ten times to approve the driving function. This results in a theoretical demand of 6.62 Billion test kilometers. It is obvious that it is not possible to cover that many kilometers. Hence, it is necessary to find new approaches for testing. The solution can be found in virtual testing of the overall system. Different integration levels can be found for that. A promising approach is testing in virtual environments in combination with hardware in the loop (HIL) like presented in . A real vehicle is placed on a testbench. The simulation of the environment is done in virtual reality. Thus, it is possible to integrate the real vehicle into the virtual environment. For this, interfaces of the surrounding sensors are needed. In the given example, target simulators are used to stimulate the real radar sensor. In the literature, there are further approaches to completely substitute the real sensors by models. These models can be found in different complexity levels. Probabilistic models like  and  aim to simulate the phenomena of real radar sensors based on object lists generated in virtual environments, without a physical correct reproduction of the radar. These models are used to test the robustness of automated driving algorithms. A higher level of complexity can be found in physical simulation approaches, based on the concept of shooting and bouncing rays. An example for that is given in . Virtual rays are shot into the scene and reflected by objects. According to the authors, the model has a real-time capability. However, simple algorithms are used, that do not sufficiently simulate the microwaves of the radar. Compared to this, more complex approaches like     and  are using realistic sending, reflection and receiving models to accurately simulate the radar. But these models do not aim to do the simulation in real-time. Hence, they are not usable for the above mentioned HIL-Testing of automated driving functions. Thus, there is a need for real-time capable simulation models that provide radar raw data in a sufficient quality for the virtual testing of such systems.
In the present work, investigations on this topic are conducted. The aim is to implement a physical correct radar model for ADAS functionalities within a virtual reality environment. For this, the overall radar model is separated into sub-models according the physical steps: emit, reflect and receive. Therefore, in Section 2 the theoretical basics of the used models are provided. Section 3 describes the implementation process. Section 4 provides a validation method for the model. The results of this validation are presented and discussed in Section 5 by using real measured values and analytical calculations. Finally, a conclusion for the potentials and further work to be done is given.
2. Raytracing Based Radar Model
At the following chapter the overall radar model is introduced. It is separated into a sending antenna model, a scattering model and a receiving antenna model. Whereby the scattering model is additionally divided into more detailed models for specular reflection, diffuse reflection and directive reflection. Figure 1 shows the used model structure for the overall radar model.
2.1. Sending Antenna Model
In common automotive radar sensors different types of sending and receiving antennas can be found. They are divided into single beam, multibeam and planar array concepts. For the control of planar array antennas complex algorithms are used. The objective of this work is to model radar antennas and propagations and not to model these algorithms. Hence a single beam antenna approach is chosen, based at  and modified for application in VR. Such single beam antennas show typical radiation patterns. Usually one main lobe and multiple symmetrical sidelobes can be found. It shows that a function of third degree, as shown in (1), works best to describe a typical radiation pattern. The sign stands for the azimuth angle of an antenna. It is defined in Equation (2) and represented by the relation of the antenna aperture to the wavelength multiplied with the sine of , the azimuth angle in degree. represents the lateral shift of each lobe. The parameters a and b describe elongation and the vertical shift of the lobes, respectively.
Figure 2 shows the radiation intensity relative to the maximum intensity in decibels. Subsequently the pattern will be used to launch rays with a clearly defined power. Hence the values for the radiation intensity need to be absolute. In order to compute these absolute values Equation (3) is used.
A further problem is that each ray needs a specified power to be send with, not a power density as defined in Equation (3). The concept provides an approach defining the power of one ray by integrating the power density up to the next. Regarding to Figure 3 this means the ray in the azimuth angle represents the field up to the next in the azimuth angle . Since the model will be implemented in a Virtual Reality engine a numerical estimation for the integral is needed. For the visualized example of Figure 3 Equation (4) gives a simple method to integrate the field. The equation is based on the assumptions that a high resolution of the azimuth angle leads to small errors in the result.
Table 1. Parameters for a typical single lobe radiation pattern of a radar antenna.
Figure 1. Model-structure of the developed raytracing based radar model.
Figure 2. Typical wavelength and aperture size dependent radiation pattern of a single beam automotive radar.
Figure 3. Visualization of the approach to integrate the power density from one ray to the next.
With this assumption, it is possible to compute a defined sending power for each discrete ray of the antenna pattern in a two-dimensional case. To include the elevation angle, the propagation of the field in this direction is assumed constant. In this way, the model can be modified to a three-dimensional case through dividing the power of each ray in the azimuth direction by the number of rays in the elevation direction. This method allows to easily rebuild a radiation pattern of an automotive radar sending antenna by a defined number of rays.
2.2. Scattering Model
To simulate the propagation of the microwaves emitted by the sending antenna in a virtual reality environment, scattering models are necessary. In this case three models are conceived. A specular model simulates the reflection of microwaves on perfectly smooth surfaces. A diffuse Lambertian model represents the opposite, perfectly rough surfaces. A third directive model represents the cases in between, where a scattered lobe is mainly reflected towards the direction of the specular case.
2.2.1. Specular Reflection Model
The specular reflection is a model for simulating microwave bounces on perfectly smooth surfaces. It uses a simple approach. Each ray impinging on a surface with an angle of is mirrored on the surface normal at the impact point, as shown in Figure 4. This results in the same value for exit angle as has. Even if the microwave power is not scattered, only a part of it gets reflected. The other part gets absorbed by the surface and is converted into heat. The proportion of the reflected power can be computed by using (5), where is the scattering factor representing the reflected part of the incoming power . The Reflection of the rays is done multiple times, until either a maximum number of hits is reached, or a ray leaves the virtual area. Hence this model can be used to simulate the multipath propagation of microwaves.
2.2.2. Diffuse Reflection Model
As mentioned above a Lambertian model is used to represent the case of reflection on perfectly rough surfaces. The approach is based on the work  but needs to be changed for the implementation into a Virtual Reality engine. The resulting model is visualized in Figure 5. It shows that a ray impinging on a surface with an angle of is always reflected with its maximum power density towards the direction of the surface normal. From there the field intensity decreases by the cosine of the angle . The resulting field can be computed by the Equations (6) and (7). These are adapted and modified from . The scattering factor again represents the proportion of the scattered power to the incoming power. K represents a correction value.
Figure 4. Visualization of the specular reflection model.
Figure 5. Visualization of the diffuse Lambertian reflection model.
These equations provide the possibility to compute the diffuse scattered field of an impinging ray. However, as with the antenna model, the problem arises that the resulting field needs to be discrete. Equation (8) shows an approach similar to the one used in the antenna model to numerically integrate the power density.
This assumes that a ray represents the power integrated by the density up to the next ray. So far, a two-dimensional case for the diffuse scattering of a ray is described. To reproduce a three-dimensional scenario, for each discrete position of a defined number of rays rotated around the surfaces normal is created. To get the power of the single rays, the power computed by (8) needs to be divided by the number of new rays. Now it is possible to describe a diffuse scattered field by a defined number of reflected rays. To allow a theoretically multipath propagation each outgoing ray needs to be reflected with the same model at its next bounce. This would result in a high number of rays. The impact of this on the implementation will be clarified in the further course of this study.
2.2.3. Directive Reflection Model
Just like the diffuse model the directive reflection model is adopted from  and modified for the implementation into Virtual Reality. The approach is very similar to the Lambertian model, even if the mathematical calculation is more complex. This model is visualized in Figure 6. It shows, that the maximum intensity of the scattered field is steered towards the direction of the specular reflection. The field in the so-called scattering lobe is described by the angle between the specular direction and the currently considered field direction. The field can be computed by the Equations given in (9) to (12), which are adapted and modified from . Here is a parameter to steer the directivity of the outgoing field. By rising its value, the scattered field gets slimmer and more orientated towards the direction of specular reflection. Hence this parameter allows to steer the field depending on the hit surface. The other used parameters are already known from the Lambertian model.
As with the Lambertian model an approach is needed to describe the scattered field by a defined number of rays. Equivalent to (8) Equation (13) is used to integrate the power for the rays, with the difference that the angle of the field is used. This approach based on  and is modified for application at VR. It is again approximated that one ray integrates the power density up to the next one.
Also, as before this model only covers the two-dimensional case. To derive the three-dimensional scenario each discrete ray needs to be rotated around the vector of the specular direction. The power of each ray can be computed by dividing the result of the two-dimensional case of (13) by the number of rays around the specular direction. For a theoretical multipath propagation each outgoing ray also needs to be reflected with the same model at its next bounce. Again, the impact of this on the implementation will be clarified later. This model allows to simulate cases of reflections that are not perfectly rough or smooth.
Figure 6. Visualization of the directive reflection model.
2.3. Receiving Antenna Model
To model a receiving antenna a simplified approach is used. A primitive geometry bounded to the position of the virtual sensor represents the antenna. Each ray that hits this geometry on the way through the virtual space passes its data to a memory. To do that a request is done on each bounce of a ray whether the body hit was the antenna. If the request is true, the ray is not reflected anymore. The data that need to be stored are depending on the application case of the model. For example the rays powers, their impinging angle or also metadata like the last body hit can be stored. With this model it is not relevant whether a ray was directly reflected or received multiple bounces. The advantage is that the antenna geometry can be scaled up to perceive lower field strengths, represented by less impinging rays. Figure 7 visualizes the idea of the receiving antenna model.
3. Implementation into a Virtual Reality
For the implementation into a Virtual Reality the Unreal Engine by Epic Games is used. It is a VR engine whose original purpose is the development of computer games. The programming can either be done graphically by using so called blueprint scripts or by textual C++ coding. The classical textual coding shows a much higher performance. With a regular state of the art gaming computer setup it is possible to generate 1.5 million rays per second. Based on this the implementation of the models as defined in the section 2 takes place. For this a simple testing environment is created. In the environment a character carries the model. This allows the user to move the virtual sensor within the scene. The sending and the receiving antenna models as well as the total reflection model can be implemented into the engine without any difficulties. For the diffuse and the directive model a problem arises. Because of the limited number of rays per second the reflection of the rays can only be done once. With these models it is not possible to do multiple bounces. Because of that only the specular and the directive models are implemented. The effect of this must be clarified by the validation of the model. The storage of the generated raw data of each ray is done as explained in Section 2.3. Each timestep arrays with the data of the impinging rays are stored into a matrix column by column. To export the data the MAT file format is used. This allows further processing of the raw data by the using the software MATLAB. For this the third-party C++ library “MAT-File API Library” is included into the Unreal Engine. This makes it possible to process and visualize the results. Thus, the validation of the model can be started.
4. Methodical Model Validation
To validate the implemented model a two-stepped validation concept with an increasing complexity of the considered models is conceived. The benefit of this is, that inadequacies can be systematically detected and analyzed. The individual steps are explained in the following sections.
Figure 7. Visualization of the diffuse Lambertian reflection model.
4.1. Validation Scenario 1: Simple Geometries
In the first validation step, simple geometries are used to check the basic functionality of the models and to identify the parameters for further steps. A sphere with a cross section of one square meter and a square plate with the dimensions of 0.7 m × 1 m are used. The advantage if the spherical shape is, that its radar cross section (RCS) corresponds with its physical cross section. Hence a sphere with a cross-sectional area of one square meter is modeled and implemented into the virtual environment. In addition to the sphere a virtual rectangle plate with the given measurements is implemented into the virtual reality. The approximated RCS of this object is known from a measurement campaign with a real radar sensor. The measurement setup of this campaign is shown in Figure 8.
The RCS value of the used plate was determined to be 15.5 m2. To compare the measured and the simulated results a simple form of the radar equation is used (13). It represents the theoretically impinging power on the receiving antenna in connection with sending power of the radar , the RCS value of the considered object and the distance to the object R. The parameters and are describing the antenna gain of the sending antenna and the effective area of the receiving antenna respectively. As the receiving antenna is loss-free the effective area corresponds to the actual area of the antenna.
By Equation (14), which based on , it is now possible to compare the theoretically determined results of the objects represented by their RCS value with the simulated results from the Virtual Reality.
4.2. Validation Scenario 2: Human Dummys
The second validation scenario uses 3D-models of different pedestrians and cyclists to simulate their radar signature and compares it with measured results. The measured data are taken from . In the paper a reference library for radar signatures of pedestrian dummies in the frequency bands of 24 GHz and 77 GHz are given. Figure 9 shows the measurement setup that was used in the paper to determine the radar signature of the dummys.
The pedestrian is placed on a rotatable table. The signature of the dummy is recorded on every rotation step and visualized in a heatmap plot. This heatmap shows die RCS value of the object applied in color. Its X-Axis represents the rotation of the dummy, the Y-axis displays the distance on the reflection, whose the origin lies in the rotation center of the rotatable table. To generate comparable results within the Virtual Reality an equivalent test setup and a similar visualization is implemented. The test objects are rotated by one degree on every simulation step. The results of each on the receiving antenna impinging ray are stored into matrices and exported into a Mat-file. In MATLAB the results of the rays are sorted into range gates. Then the RCS value is calculated for each area. So, the results can be plotted as a heatmap, comparable to the paper. In the last step equivalent 3D-models for measured dummies from the paper must be found. For comparison a pedestrian model and a cyclist model are chosen. The selected models are shown in Figure 10.
Even if the poses of the Models are very similar, small differences can be found. For example, the physique of the digital walking pedestrian is stronger, and it has a more open body posture. The physical dummy is smaller. The cyclist and the cycle also show small inadequacies. The frontal wheel fork is not given in the physical arrangement. Also, the posture of the digital cyclist is more open again. The model quality is considered sufficient for a comparison of the simulated data with measured results. Errors due to simplifications are to be expected on the physical measurement side, not on the simulation side.
Figure 8. Measurement setup to determine the radar cross section of physical test objects.
Figure 9. Measurement setup to determine the radar signature used in .
Figure 10. Selected models for the comparison of measured radar signatures with simulated results; (a) Used virtual 3D-models; (b) Measured Models from .
5. Simulation Results and Analysis
Based on the in section 4 conceived validation scenarios now the results of the simulations are presented and discussed. This is done individually for each scenario according to the same structure as used before.
5.1. Validation Results 1: Simple Geometries
The surveying of the simple sphere and plate geometries in the virtual environment is done individually for both implemented reflection models, the total and the directive reflection. The results of the measurement with the total reflection model immediately show its shortcomings. For the sphere geometry no incoming power is registered on the receiving antenna. On closer examination it can be determined that this reflection model is sensitive to small geometric errors. Figure 11 shows this recognition by the visualization of the microwave paths. All rounded shapes are represented by straight segments in virtual environments. Hence the sphere is modeled by a limited number of facets. Each facet acts like a mirror, so the rays are reflected away in bundles. Hence, this model generates inadequate results for geometries represented by a low number of facets.
For the rectangular plate, the results of the total reflection are also to be discussed. With the measured RCS value and the usage of the radar Equation (13) the theoretically impinging power results in 0.0162 W. The simulated impinging power is 0.0434 W. This large deviation can only be reduced by an artificial reduction of the scattering parameter . Summarizing the total reflection model shows weaknesses caused by geometric inadequacies as well as physically incorrect simulations. In conclusion the total reflection model is not valid.
The directive reflection model shows much better results. In contrast to the previous model it is insensitive to geometric inaccuracies. The results for the simple test geometries are summarized in Table 2. For the sphere geometry the relative deviation is only 1.1% to the theoretically computed results. For the rectangle plate the deviation is higher. This may be due to measurement errors of the real test. Nevertheless, the directive reflection model can be considered as valid for the simulation of the reflected power for simple geometries.
Figure 11. Visualization of the reflected rays by a spherical shape with the total reflection model in the virtual environment; green: emitted rays; red: reflected rays.
Table 2. Results of the simulation with the directive reflection model for simple geometries.
5.2. Validation Results 2: Human Dummys
Based on the previous results and parameters determined, the simulation for the human models is carried out. As it has been shown the total reflection model does not provide valid results. Hence the following simulations are done with the directive reflection model. The execution of the simulation is done as described in Section 4.2. The simulated results for the pedestrian and the cyclist dummy are shown in Figure 12 and Figure 13 respectively. In both figures it is directly noticeable that the resolution of the simulated result is much higher than the measured one. This is based on the setup of the range gates, the impinging signals are sorted in. To reduce the resolution the range gates should be chosen larger. For the validation the higher resolution does not bring any disadvantages. The comparison of the measured and the simulated results for the pedestrian model shows a very similar radar signature. The significant RCS hotspots are noticeable. For example both results show a rising RCS value in the range around 0˚ and around 180˚, where the pedestrian provides a large target surface. Also, the reflection of the legs and the arms can be seen as arches in the negative range area over the rotation. The separation of these arches from the torso reflection is sharper in the simulated results. On the one hand this may be due to higher resolution of simulated results on the other hand the digital pedestrian has a more open body posture that may cause the deviation. For the cyclist dummy the results are similar. The significant parts of the bike and the RCS hotspots can be read out of the simulation as well as the measurement. Here the small inadequacies may also be caused by the different postures and the differences of the bicycle model. The missing multiple reflection of the directive scattering model does not have any influence on the results of this tests. So, the model is valid for the carried-out tests. The results show small deviations, but still represent radar signature of the dummies sufficient. The impact of the missing multiple reflections on the data processing with the information of complex scenes needs to be done in further studies. This is not part of the present work.
The final step in the validation process is to assess performance of the implemented directive reflection model. The previous validation tests have been done without considering the real-time capability. Assuming that a framerate of at least 30 FPS is needed for further use cases of the model, this framerate is selected as the lower limit. The tests are done on a state-of-the-art gaming computer. It shows that it is possible to do 1.5 Million ray traces per second. The practice shows that 34,400 rays per frame lead to a framerate of 40 FPS. This brings the advantage of having a reserve for drops in the framerate. Figure 14 shows the results of the simulation with the reduced framerate compared to the previous test for the cyclist dummy. It shows that the signature still can be recognized, but the contend of information is reduced. A look into the raw data shows, that the absolute RCS value for an angle segment remains approximately the same. The parameter is represented by fewer points with a higher value, but the average RCS stays the same. Summarizing the model is real time capable and still gives valid results with a reduced content of information. The influence of this reduction of information on further data processing needs to be analyzed in later researches depending on the models use case.
Figure 12. Comparison of the radar signatures of a pedestrian dummy; (a) Measured from ; (b) Simulated in Virtual Reality.
Figure 13. Comparison of the radar signatures of a cyclist dummy; (a) Measured from ; (b) Simulated in Virtual Reality.
Figure 14. Comparison of the radar signatures of a cyclist dummy with reduced framerate; (a) Aprox. 3 FPS; (b) Aprox. 40 FPS.
The present work shows a new approach for the generation of automotive radar raw data in a virtual environment. The implemented model is divided into three sub-models, a sending antenna, different scattering models and a receiving antenna model. Thereby the overall scattering model includes the cases specular reflection, diffuse reflection and directive reflection. To decrease numerical effort and validate the early stage of the model, multiple reflections for the directive reflection model were neglected. Therefore, a two stepped validation process was conducted. The first step shows a general functionality of the total reflection model, but caused of individual issues of the modelling method of geometries in VR, the overall functionality derivate from physical reality. It is sensitive against small geometric errors and does not validly simulate the microwave physics. The directive scattering model shows much better accordance for the simulation of the reflected power of simple geometries as well as the simulation of the radar signatures of typical road user models. The results turn out to be valid, only the performance causes issues. With a rising framerate the content of information in the virtual raw data is reduced. The influence of this lack of information on further algorithms needs to be clarified regarding the final use case. Summarizing the implemented models give new opportunities for the simulation of radar raw data in virtual environments.
The Project is supported by the Ministry of Economic Affairs, Innovation, Digitization and Energy of North Rhine-Westphalia and by HH Vision, Hoersch und Hennrich Achitekten GbR.
 Wachenfeld, W. and Winner, H. (2015) Die Freigabe des autonomen Fahrens. In: Fahren, A., Maurer, M., et al., Eds., Springer Vieweg, Berlin, Heidelberg, 439-464.
 Muckenhuber, S., Holzer, H., Rübsam, J. and Stettinger, G. (2019) Object-Based Sensor Model for Virtual Testing of ADAS/AD Functions. IEEE ICCVE, Graz.
 Bühren, M. and Yang, B. (2006) Automotive Radar Target List Simulation Based on Reflection Center Representation of Objects. Proceedings of the 3rd International Workshop on Intelligent Transportation (WIT), Hamburg, 161-166.
 Weiskopf, M., Wohlfahrt, Ch. and Schmidt, A. (2015) Integrationslösung zur Absicherung eines realenRadarsensors im Systemverbund mit der Hardware-in-the Loop Testtechnologie. Automotive-Safety and Security, 240, 29-40.
 Skidmore, G., Chawla, T. and Bedrosian, G. (2019) Combining Physical Optics and Method of Equivalent Currents to Create Unique Near-Field Propagation and Scattering Technique for Automotive Radar Applications. IEEE COMCAS, Tel-Aviv.
 Chipengo, U., Krenz, P. and Carpenter, S. (2018) From Antenna Design to High Fidelity, Full Physics Automotive Radar Sensor Corner Case Simulation. Modelling and Simulation in Engineering, 2018, 1-19.
 Dudek, M., Wahl, R., Kissinger, D., Weigel, R. and Fischer, G. (2010) Millimeter Wave FMCW Radar System Simulations including a 3D Ray Tracing Channel Simulator. IEEE Asia-Pacific Microwave Conference, Yokohama.
 Degli-Esposti, V., Fuschini, F., Vitucci, E. and Falciasecca, G. (2007) Measurement and Modelling of Scattering from Buildings. IEEE Transactions on Antennas and Propagation, 55, 143-153.
 Fortuny-Guasch, J. and Chareau, J.M. (2013) Radar Cross Section Measurements of Pedestrian Dummies and Humans in the 24/77 GHz Frequency Bands. European Commission, Joint Research Centre, Institute for the Protection and Security of the Citizen, Luxembourg.