Introduction

The development and evaluation of automated driving functions has become common within the automotive industry. In particular, automated parking functions, with their low speeds and small range of operation make it easier to implement them in an automated manner. However, to satisfy the user, these functions must be able to be executed smoothly and at least as fast as a human driver. In this paper, we will present a real vehicle-in-the-loop (ViL) approach for laboratory development, jointly implemented by MdynamiX Germany and its partners, to support the consistent development and evaluation of automated driving functions.

Application of the ViL method

The level of automation of parking functions, whether partially automated or fully automated, is constantly increasing. Testing automated parking functions is complex and costly because of the large variety of parking scenarios, despite the low speeds at which the vehicles are parked. In order to ensure the safety of all traffic participants, parking in different vehicles, in different environments, and in different situations, such as reverse parking and side parking, poses many challenges. If one wanted to test all these scenarios, the test would be complex. For example, test scenarios in which children appear in the blind spot of the field of view would be difficult to properly conduct a real-world test.

For these reasons, it is necessary to find suitable solutions through virtual scenarios to validate and evaluate the relevant functionality in all development phases at a reasonable economic cost.

The Vehicle-in-the-Loop (ViL) approach offers a viable solution.The ViL approach combines the advantages of real vehicle testing and computer simulation. The motion of the real vehicle (the vehicle under test), including all subcomponents, is transferred to the simulation, allowing the driving of the vehicle under test in the virtual world to mimic driving in the real physical world without the need to parameterize complex vehicle models. The simulated virtual environment will be injected into the real sensors of the vehicle under test, allowing the vehicle under test to perceive and respond to the virtual environment. Thus, the interaction between the automatic parking control function and the vehicle actuators is fully tested. Since ViL uses virtual environments, the design scenarios are highly flexible, which facilitates the design of safety-critical scenarios.

Scenario Demonstration for Testing Security-Related Tests Using the ViL Approach

The ViL system in this article tests the hardware configuration of the automatic parking function

MXeval - KPI evaluation

In order to comprehensively compare manual and automatic parking operations (partially or fully automatic), a standardized evaluation method is needed - a stage model approach, which divides the parking operation into independent parts. In addition to the stage model, objective Key Performance Indicators (KPIs) are required for the evaluation. the MXeval analysis software from MdynamiX fulfills just such a requirement, not only being able to be used for the evaluation of vehicle dynamics and automated lateral and longitudinal guided driving functionality, but also containing a test library for automated parking. Specific evaluation criteria and target values to be pursued can be identified and incorporated into the evaluation program. The evaluation of the measured data is carried out efficiently in fully automated mode and, depending on requirements, can be carried out in a live vehicle environment or by remote connection. In addition to the evaluation charts, the KPIs are also visible at a glance. All evaluation results are eventually saved in a report. The overall evaluation process is shown in the diagram below.

MXeval Evaluation Process

Results

In this test, the vehicle under test was operated in both real and virtual environments. In Figures 4 and 5, a comparison of eight sets of measurements is shown, each obtained by performing side parking for a real parking space, and its corresponding virtual space. Figure 4 shows that the measurements exhibit a consistent trend. Further comparisons show that the measurements are very similar, especially at the moment of initial motion, with only minor differences. The instantaneous speed KPI, for example, is almost the same in the real vehicle and in the simulation.

In addition, Figure 4 shows the similarity of the velocity profiles at subsequent moments of the initial motion. The maximum and instantaneous speeds of the initial motion are relatively similar. The average speed is 3.43 km/h in the real environment and 3.49 km/h in the virtual environment.

Eight Groups of Side Parking Vehicle Speed Comparisons

Notably, comparing the steering wheel corner inputs throughout the maneuver, in both real and virtual environments, revealed four sets of measurements, which were nearly identical for the initial move (time points 0 to 16s), as shown in Figure 5. After 16s, however, significant differences in steering wheel angle inputs were observed. These differences can be attributed to the absence of a steering robot for manual steering wheel turning in the real vehicle - it is common for manual steering wheel turning to have a large baseline difference. Especially for parking assistance systems, even for the same vehicle, the starting position plays a decisive role, leading to different parking strategies and trajectories. In this case, the use of a steering robot may be advantageous.

Comparison of steering wheel corner inputs for side parking in eight groups

There are various KPIs used to evaluate parking assistance systems. Some of them are decisive, especially for the initial motion. Some of the KPIs are illustrated in Table 1.Again, it is evident that the KPIs are very similar for both real vehicle and simulation environments.

Parking Assistance System Partial KPI

Overall, the ViL approach, not only is simulation for validation purposes increasingly important due to regulatory requirements, but also provides an effective complement and/or alternative to traditional real-world vehicle testing.

Summary and Outlook

This test scenario presented in this article by MdynamiX and its partners can be used as a basis for most automated driving functions, and is able to assess the maturity of the function development and validate the realization of the function goals during all development phases. It enables efficient comparison of results between different real vehicles and simulations and validates the simulation quality. This solution makes it possible to achieve a high degree of consistency between real and simulated vehicle behavior.

In addition, the ViL approach is also applicable to other sensor-based advanced driver assistance systems. Especially in safety-related tests, such as NCAP tests, the ViL approach shows great potential for cost reduction. As requirements become more complex and new, high-risk operations will emerge, the benefits of the ViL approach are likely to be even more far-reaching.

Note: The translation of this article is excerpted from Consistent Evaluation of Automated Driving Function Based on Vehicle-in-the-Loop, ATZ Worldwide 01/2024