Transfer of IT test practices into safety-critical vehicle environments – CI/CD pipelines, large-scale automation and coverage-driven validation – often fails.
Automotive development programs are executing more tests than ever – across SIL, HIL and full-vehicle levels. Regression suites are expanding, automation rates are increasing and coverage metrics continue to improve. Yet confidence in system behavior is not increasing at the same rate. In many programs, the opposite is observed: testing effort grows but release decisions remain difficult, late defects persist and uncertainty in safety-critical functions remains high. This raises a fundamental question: what if the limitation is not test execution but the way testing is defined?
With the transition toward software-defined vehicles, many organizations have adopted testing approaches from IT environments: CI/CD pipelines, large-scale automation and coverage-driven validation. These approaches are effective in domains where failures are recoverable and can be addressed post-release. Automotive systems, however, operate under fundamentally different constraints. As such, functional safety (ISO 26262), SOTIF considerations and increasing cybersecurity and homologation requirements demand a different level of assurance.
But this distinction is increasingly blurred. Modern vehicle architectures combine IT-based systems, such as infotainment, with safety-critical domains on shared platforms. Testing must therefore reconcile both worlds: coverage-driven validation and risk-driven validation. In IT environments, increasing test coverage is often a reliable proxy for confidence. In automotive systems, this assumption breaks down.
It is entirely possible to achieve high traceability, extensive regression coverage and broad validation across SIL, HIL and vehicle levels – and still miss critical scenarios. Based on patterns observed across multiple large-scale integration programs, regression suites can grow by 30-40% within a single release cycle, while defect detection shifts toward later vehicle-level validation phases.
The issue is not a lack of testing. It is a lack of risk relevance. Specification is not reality. Many automotive systems behave correctly under specification, yet still fail in real-world conditions due to timing dependencies, sensor ambiguity or unforeseen combinations of valid system states. This is particularly evident in SOTIF-related scenarios, where systems operate as designed but still produce unsafe outcomes.
However, not all critical scenarios can be derived up front. Hazardous behavior often emerges from previously unknown combinations, requiring large-scale simulation, data-driven scenario discovery and long-tail validation.
Testing confirms expected behavior, but not necessarily that expectations are complete. It is still widely treated as an execution function. In safety-critical systems, this reaches
its limits. A more effective model shifts testing toward risk ownership – prioritizing system-relevant scenarios, focusing on interactions and integrating safety, SOTIF and cybersecurity early. In practice, however, regulatory constraints remain.
Test cases linked to safety requirements cannot simply be removed without affecting traceability and safety cases. As a result, extensive regression suites persist – not by choice, but by necessity. What changes, therefore, is not the volume of testing but the way its value is defined. In practice, this requires a structured approach to identifying system-relevant scenarios – something traditional requirement-based methods cannot provide.
The real shift is not in tools but in thinking: from coverage to risk relevance, from execution to system understanding, and from compliance to assurance.
Automotive systems are becoming more complex, connected and safety critical. Testing must evolve accordingly, not by doing more but by redefining its purpose. Risk-based prioritization, scenario-driven validation and large-scale simulation must work together, because in practice, organizations do not struggle with too much testing, they struggle with identifying which scenarios truly define system behavior. Until this problem is addressed, increasing test volume will remain the most defensible – but not necessarily the most effective – strategy. And those who solve this problem will define the next generation of automotive testing.
