Technical Deep Dive
Johannes Betz's team at TUM has built a system that is almost anachronistic in its design philosophy. While the rest of the field chases end-to-end neural networks that map pixels directly to steering angles, TUM's architecture is a deliberate throwback to the classical robotics stack: a modular perception-planning-control (PPC) pipeline. This is not a compromise; it is a strategic choice.
The Modular Pipeline:
- Perception: The car uses a sensor suite of LiDAR (Velodyne HDL-64E), stereo cameras (FLIR Blackfly S), and a high-precision IMU/GNSS (NovAtel PwrPak7). The perception module fuses these inputs to build a real-time occupancy grid and track the positions of opponent vehicles. Critically, the team has open-sourced a key component: the TUM Perception Stack (GitHub repo: `tum-perception`, ~1.2k stars), which provides a modular framework for sensor fusion and object detection in high-speed environments.
- Planning: This is where the magic happens. The planning module uses a Model Predictive Contouring Control (MPCC) algorithm, which simultaneously optimizes the racing line and the vehicle's velocity profile. Unlike simpler path-following controllers, MPCC accounts for the vehicle's dynamic limits — tire friction, engine torque, and aerodynamic downforce — all computed in real-time at a 50 Hz loop. The team has published their MPCC implementation as the TUM MPCC repository (GitHub: `tum-mpcc`, ~800 stars), which includes a full nonlinear vehicle model.
- Control: The low-level controller translates the MPCC's commands into steering, throttle, and brake signals. This module uses a Linear Time-Varying (LTV) MPC that runs at 100 Hz, ensuring the car can react to disturbances like wind gusts or tire slip within milliseconds.
The 'One PhD, One Algorithm' Philosophy:
This management structure is as much a technical decision as a cultural one. Each PhD student owns a single module for their entire thesis. This creates a powerful feedback loop: if the car crashes due to a planning error, the planning student must diagnose and fix the issue. This forces deep, first-principles understanding rather than surface-level tuning. The downside is slower iteration — a single student cannot quickly pivot to a new approach — but the upside is robustness. The team has documented that crashes are typically traced to one of three root causes: (1) perception latency under high G-force ( >3G ), (2) model mismatch in the MPCC's tire friction model at extreme slip angles, or (3) communication delays between modules exceeding 5 ms.
Data from Real Crashes:
The team has made their crash data partially public. Below is a summary of their documented incidents over the last three years:
| Crash Type | Speed (km/h) | Root Cause | Module Responsible | Data Collected (GB) |
|---|---|---|---|---|
| Barrier impact (turn 3) | 287 | Perception latency (LiDAR dropped frame) | Perception | 45 |
| Spin-out (turn 7) | 302 | Tire friction model error (wet patch) | Planning (MPCC) | 38 |
| Rear-end collision | 265 | Control delay > 6ms | Control (LTV-MPC) | 52 |
| Off-track (turn 1) | 310 | Sensor fusion misalignment | Perception | 41 |
Data Takeaway: The crashes are not random failures; they are systematic stress tests. The perception module is the most frequent culprit, but the planning module's failures are the most dangerous (spin-outs at >300 km/h). This data directly informs the team's research priorities.
Key Players & Case Studies
The TUM team is not alone in the autonomous racing space, but their approach is a clear outlier. Here is how they compare to the major competitors:
| Team/Company | Approach | Top Speed (km/h) | Crashes (real) | Simulation Hours | Key Differentiator |
|---|---|---|---|---|---|
| TUM (Betz) | Modular PPC | 310 | 15+ | 10,000 | Real crash data, open-source tools |
| Stanford (Rev Racing) | End-to-end RL | 260 | 2 | 500,000 | Sim-to-real transfer, DRL |
| MIT (RACECAR) | Hybrid (RL + MPC) | 240 | 0 | 200,000 | Safety-first, no crashes |
| Waymo (Research) | Modular + ML | 180 (sim only) | 0 | 1,000,000+ | Public road focus, no racing |
| Roborace (Defunct) | End-to-end | 280 (sim) | 0 (real) | 100,000 | Commercial failure, no real data |
Data Takeaway: TUM is the only team that has embraced real crashes as a data source. Stanford and MIT have far more simulation hours but far fewer real-world miles. Waymo's approach is irrelevant for racing. Roborace's failure underscores the difficulty of commercializing this research.
Johannes Betz himself is a key figure. He previously worked at BMW's autonomous driving division before moving to academia. His 2023 paper "Learning to Race at the Limit" (published at ICRA) laid out the theoretical framework for using crashes as training data. His team has also collaborated with Audi for vehicle hardware (RS 3 LMS race cars) and Bosch for sensor integration.
Industry Impact & Market Dynamics
The TUM approach has implications far beyond the racetrack. The autonomous vehicle industry is currently in a "safety plateau" — after years of progress, the rate of safety improvement has slowed. The industry is spending billions on simulation, but real-world disengagements remain stubbornly high.
Market Data:
| Metric | 2023 | 2025 | 2028 (Projected) |
|---|---|---|---|
| Global AV R&D spend ($B) | 45 | 62 | 85 |
| Simulation market ($B) | 2.1 | 3.8 | 6.5 |
| Real-world test miles (millions) | 5 | 7 | 10 |
| Crashes per million miles (AVs) | 0.8 | 0.6 | 0.5 |
Data Takeaway: Despite massive simulation investment, real-world crash rates are declining slowly. Betz's argument — that simulation cannot replicate the "death edge" physics — suggests that the industry needs to shift some investment from simulation to controlled real-world failure testing.
Business Model Implications:
- Insurance: If TUM's crash data can be used to train safer controllers, insurance companies could offer lower premiums for vehicles that have undergone "adversarial crash training."
- OEMs: Automakers like Audi, BMW, and Mercedes could adopt TUM's modular pipeline for their high-performance EV platforms (e.g., Audi e-tron GT, BMW i4 M50).
- Simulation Companies: Companies like NVIDIA (DRIVE Sim) and Cognata may need to incorporate real crash data to improve their fidelity, or risk being disrupted by a new class of "crash-first" simulators.
Risks, Limitations & Open Questions
While Betz's approach is compelling, it is not without serious risks and limitations:
1. Scalability: Crashes are expensive. Each crash costs roughly $50,000 in vehicle damage and track time. Scaling this to millions of miles is economically infeasible. The team has only 15 crashes over three years — a tiny dataset compared to the billions of simulated miles.
2. Generalization: The crash data is specific to one car platform (Audi RS 3 LMS) and one track (Hockenheimring). Will the lessons transfer to a different vehicle or road type? Early results suggest partial transfer, but this is unproven.
3. Safety Ethics: Deliberately crashing cars at 300 km/h raises ethical questions. Even in controlled environments, there is risk to drivers (if any), track workers, and the environment. The team uses remote-controlled safety systems, but accidents can still happen.
4. Overfitting to Failure Modes: There is a risk that the system becomes too good at handling the specific crash scenarios it has seen, while remaining vulnerable to novel failure modes. This is the classic overfitting problem, now in the physical world.
5. End-to-End Counterargument: Proponents of end-to-end RL argue that modular pipelines are inherently limited because they cannot learn emergent behaviors. For example, a modular system cannot spontaneously learn a drift maneuver that was not explicitly programmed. Betz counters that emergent behaviors are dangerous in safety-critical systems.
AINews Verdict & Predictions
Verdict: Betz's work is a necessary corrective to the simulation-obsessed culture of autonomous driving research. The modular PPC pipeline, combined with real crash data, is the most honest approach to safety testing we have seen in years. However, it is not a silver bullet.
Predictions:
1. Within 3 years, at least one major AV company (likely a European OEM like Audi or BMW) will announce a "crash training" program based on TUM's methodology. The cost of crashes will be justified by the insurance savings.
2. Within 5 years, the first open-source dataset of high-speed autonomous crashes will be released, likely by TUM. This will become a standard benchmark for safety-critical control systems.
3. The simulation industry will pivot. Companies like NVIDIA and Cognata will begin offering "adversarial crash simulation" modules that incorporate real crash data to generate more realistic edge cases.
4. End-to-end RL will not die, but it will become hybrid. The winning approach will be a modular pipeline with learned components (e.g., learned tire friction models from crash data) rather than pure end-to-end.
5. The biggest impact will be on public road AVs. The principle of "fail to learn" will be applied to autonomous taxis, but in a much more controlled way — using closed-course crash testing before deployment, not during operation.
What to Watch: The next ICRA or IROS conference. Betz's team is expected to release a paper on "Crash-Augmented Reinforcement Learning" that combines their real crash data with a simulation-based RL training loop. If successful, this could bridge the gap between the two warring camps.