Tesla Autopilot Crashes: Risks and Safety Concerns

Tesla's Autopilot system, relying heavily on camera vision, poses significant safety risks due to its limitations in obstacle detection, resulting in numerous crashes.

### The Risks of Tesla's Autopilot Technology: A Fatal Incident and Ongoing Concerns
* 🚗 On May 5th, 2021, Steven Hendrickson tragically lost his life while driving his Tesla Model 3 on Autopilot, leaving behind a **wife and two children**.
* 📊 Since 2021, Tesla has reported **over a thousand crashes** to federal regulators, yet many details remain undisclosed to the public, raising **safety concerns** about the technology.
* ⚠ Experts, including Missy Cummings, have warned that the reliance on camera-based systems in Tesla's Autopilot could lead to fatal accidents, stating, "**There is no question that someone is going to die using this technology**."
* 🔍 Investigations by the National Highway Traffic Safety Administration (NHTSA) into Autopilot have revealed that many crashes stem from **failures to recognize obstacles**, with 222 crashes analyzed showing **alarming patterns of sudden veering and failure to stop**.
* 📉 Tesla claims their Autopilot is **safer than human driving**, but experts argue that its machine learning systems are **inadequately trained on certain scenarios**, leading to **critical failures**.
* 📽 The difficulty in accessing detailed crash data from Tesla complicates investigations, as families like **Janell Hendrickson's** struggle to **obtain information about their loved ones' incidents**.
* 🔒 The Department of Justice is currently investigating Tesla's marketing of Autopilot, as there are concerns about **misleading claims regarding the technology's capabilities**.
* ⚠ As the public continues to place trust in Autopilot, experts warn that this **overconfidence may lead to more accidents**, highlighting the **urgent need for improved safety measures and transparency** in autonomous driving technology.

(brooding music) — [Narrator] On the morning of May 5th, 2021, Steven Hendrickson was driving his Tesla Model 3 to work. His car was on Autopilot as he drove through Fontana, California.

At about 2:30 AM, an overturned semi-truck appeared in front of him. Moments later, he was killed. (brooding music) The 35-year-old left behind his wife and two children. This crash is one of more than a thousand

that Tesla has submitted to federal regulators since 2021. The details of which have been hidden from the public. Video and data gathered from these crashes by the Wall Street Journal shows that Tesla's heavy reliance on cameras for its Autopilot technology,

which differs from the rest of the industry, is putting the public at risk. (tires screeching) Teslas operating on Autopilot have been involved in hundreds of crashes across US roads and highways since 2016.

Over the years, Tesla CEO Elon Musk has maintained the technology is safe. — I do think that, long-term, it can reduce accidents by a factor of 10. We're solving a very important part of AI and one that can ultimately save millions of lives

and prevent tens of millions of serious injuries. The safety per mile is better than human driving. — [Narrator] Leading expert on autonomous driving, Missy Cummings, who has been critical of Tesla, warned of the risks of semi-autonomous vehicles in 2016.

- There is no question that someone is going to die using this technology. The question is when. - [Narrator] Cummings has also worked as a safety advisor for the federal regulator, the National Highway Traffic Safety Administration, or NHTSA.

- I am besieged with requests from families of people who have been killed in Tesla crashes. It's really tough to explain to them that, you know, this is how the tech was designed. - [Narrator] Since 2021,

NHTSA has launched a series of investigations into Autopilot. They've released little information about them, and the cars remain on the road. Also, in 2021, NHTSA ordered car makers to report all serious crashes involving semi-autonomous driving systems like Autopilot.

But much of the data Tesla submitted has been hidden from public view because the company considers it proprietary. To unlock it, the Journal gathered reports from individual states and matched them with the crash data Tesla submitted to NHTSA and found that longstanding concerns

about the Autopilot technology are showing up on America's roads. Of the 1000-plus crashes Tesla submitted so far, the Journal pieced together 222 and found that 44 of them occurred when Teslas in Autopilot veered suddenly.

31 occurred when a Tesla in Autopilot failed to stop or yield for an obstacle in front of it. (brooding music) Orlando, Florida. A Model 3 drove into the back of a stopped police car that was attending to a disabled vehicle.

Guadalupe County, Texas. A Model 3 plowed through an intersection and off the road.  These failures to stop, like the one that killed Steven Hendrickson, account for the most serious injuries and deaths in the cases the Journal unlocked.

- I had to hang up the phone and look at both my kids as they're crying and telling me to tell them it's not true, that their dad was in a car accident and he won't be coming home. - [Narrator] Tesla has said drivers operating

in Autopilot need to be ready to take control at all times. They say Steven Hendrickson was warned to keep his hands on the wheel 19 times before he crashed, and that his car initiated braking before impact. But whether or not a driver is alert,

it's important to know how Autopilot performed, and that can only be seen through the large amounts of internal video and data Teslas record. The Journal obtained video and partial data from the Hendrickson crash and asked experts to analyze it.

- The kind of things that tend to go wrong with these systems are things like it was not trained on pictures of an overturned double trailer. It just didn't know what it was. There were some lights there, but the lights were in unusual positions.

A person would have clearly said, 'Something big is in the middle of the road.' But the way machine learning works is it trains on a bunch of examples, and if it encounters something it doesn't have a bunch of examples for, it may have no idea what's going on.

- [Narrator] Experts say this illustrates a fundamental flaw in the Autopilot technology. It relies mainly on cameras, or computer vision, with radar as a backup in some models. - Other driver-assist cars use radar, computer vision, and lidar (laser ranging)

that helps detect obstacles. But lidar is very expensive. - Expensive sensors are unnecessary. It's like having a whole bunch of expensive appendices. - [Narrator] Instead, Tesla's camera-based system relies on humans to train it on obstacles it may encounter.

- Even though people think these cars are learning while they're driving, they aren't. They're only learning at the rate that the companies decide to retrain the algorithms. (brooding music) - So, I'm going to turn Autopilot on right now.

I'll just pull my finger down like that once. And as you can see on the screen, the visualizations have started. - [Narrator] John Bernal is a former Tesla employee who was fired in 2022 after posting videos of Autopilot failures.

Tesla has not commented publicly on his dismissal. - So, as we go through this downtown area, you'll see a lot of cars visualized, just, you know, kind of parked on the side of the road. You'll see pedestrians walking on the sidewalk.

I originally started at Tesla in August 2020 as a data annotation specialist. That role primarily involved taking image data and labeling it to train the technology on what a car was, what a pedestrian was, maybe an animal, or a lane line,

or what a red light was. That way, with this image data training, the vehicle would then know how to operate in a real-world situation when that data came through the cameras and through the car's computer. The car has eight cameras, specifically three right here in the front.

And so, as that truck just went across us, it's actually going through the plane of multiple different cameras. In one camera, it may have looked perfectly on the ground and centered, but in another camera, it could have been six feet ahead

or six feet behind, or maybe five feet floating in the air. And those are things I've noticed while labeling these image clips; these cameras are not calibrated properly. So, what looks true in one camera will not be true in another camera.

- [Narrator] If the cameras aren't seeing the same thing, they can encounter significant challenges identifying obstacles. We can see this in an Autopilot crash, similar to Steven Hendrickson's, near Los Angeles in 2021.

The Journal obtained a rare set of raw data for the Tesla in this crash from an anonymous hacker. When a crashed pickup truck becomes visible on the road, one of the car's cameras sees it, but the other doesn't.

Then, as the obstacle gets closer, the Tesla doesn't recognize it and it crashes at full speed. Getting this degree of information from Tesla after it crashes is nearly impossible for the average driver. Tesla says they provide drivers with hundreds of data points

with granularity down to the millisecond, in a user-friendly format. (drill whirring) But that information doesn't show the exact decisions Autopilot made. Those details need to be extracted from the car's internal computer.  A Tesla's pretty much a computer on wheels.

I work on salvage Teslas, usually from accidents and wrecks. I've seen a lot of videos where drivers are being reckless, and I've seen videos where it leaves you wondering, 'How did that happen?'

And it makes you curious: was it the driver or was it Autopilot? - [Narrator] The video only tells part of the story. The data on the car's computer gives a fuller picture. - The computers all report data

and link to the Tesla mothership. And we have the computer off. - [Narrator] Diaz sends Autopilot computers he doesn't need across the country to a hacker who then extracts the information. Janell Hendrickson has been unable

to get data from her husband's car. Her lawyers requested it over a year ago. Tesla says they're in the process of gathering it. - It's actually been really difficult to get any information about this. We know that Tesla itself has all the video

and all the information, but they will not share it. - [Narrator] One thing not reflected in the car's data is the driver's confidence in the technology. - He trusted that Autopilot, obviously with his life.

He trusted it with my kids' lives. He said that it let him be a little less worried when he was driving so much. He'd leave about 3 AM to get to work. By six, all over SoCal.  If the cars do a pretty good job, not perfect,

but pretty good, we develop a false sense of trust in them. And this overconfidence is coming from the levers of the tech, but it's also coming from the top down. The Model S and Model X at this point

can drive autonomously with greater safety than a person. We've gone over this multiple times, like, "Are we sure we have the right sensor suite? Should we add anything more?" No. The car currently drives me around Austin most of the time without interventions.

- [Narrator] The Department of Justice is investigating Tesla for its marketing of Autopilot. Tesla disputes claims that they misled the public about the car's capabilities. Janell Hendrickson's case against Tesla is set to go to trial in 2025.

- People are still going to buy Tesla. They're still going to support Elon Musk.  Doesn't matter how many accidents there are, but at least understand your car and understand your car's capabilities before you just put your entire life behind it.

(brooding music) Computer vision is such a deeply flawed technology, and we're not doing enough to determine how to fix its gaps and how to make it recognize objects. Like I look into my crystal ball, you know,

"What do I see coming?" I see that having the car do most of the driving for you, and requiring you to pay attention to make sure nothing bad happens, I don't think this is going to be a long-term technology that we're going to keep in cars.

The Hidden Autopilot Data That Reveals Why Teslas Crash | WSJ

The Wall Street Journal takes you inside carefully selected stories and events in a visually captivating way so you can dig deeper into the news and insights that matter to you. 

The Wall Street Journal

Fatal Tesla Autopilot Crash and Subsequent Investigations

Steven Hendrickson's death in a Tesla Autopilot crash on May 5th, 2021.

Over 1000 Autopilot-related crashes submitted to federal regulators since 2021, details largely hidden from the public.

Wall Street Journal investigation reveals Tesla's heavy reliance on cameras for Autopilot poses risks.

Hundreds of Autopilot crashes since 2016, despite Elon Musk's claims of safety and accident reduction.

Expert Missy Cummings' warnings about Autopilot risks and her experience advising NHTSA.

NHTSA investigations into Autopilot and limited information released to the public.

Journal analysis of 222 crashes reveals Autopilot failures, including sudden veering and failure to stop.

Examples of crashes involving failure to stop, resulting in serious injuries and deaths.

Tesla's statement on driver responsibility and warnings given to Hendrickson.

Importance of accessing internal video and data for understanding Autopilot performance.

Expert analysis of Hendrickson crash highlighting Autopilot's limitations in recognizing unusual obstacles.

Technological Flaws in Tesla's Autopilot System

Comparison of Tesla's camera-based system with other systems using radar and lidar.

Tesla's reliance on human training for its camera system and limitations in real-time learning.

Former Tesla employee John Bernal's insights into Autopilot failures and data annotation processes.

Issues with camera calibration affecting object recognition.

Analysis of a Los Angeles crash highlighting inconsistent camera readings and failure to recognize obstacles.

Challenges in obtaining data from Tesla's internal computer after a crash.

Difficulty in determining driver vs. Autopilot responsibility in accidents.

Process of obtaining data from salvaged Tesla computers.

Hendrickson family's struggle to obtain data from the crashed vehicle.

Hendrickson's trust in Autopilot and the development of overconfidence in the technology.

Musk's claims of autonomous driving capabilities.

Department of Justice investigation into Tesla's marketing of Autopilot.

Hendrickson's lawsuit against Tesla set for trial in 2025.

Call for driver awareness of Autopilot limitations.

Concerns about the flaws of computer vision technology.

Doubt about the long-term viability of current Autopilot systems.