How this Santa Clara professor is using AI to predict disaster before it happens

Of all the applications of AI, one of the most intriguing is what it can do to make our daily commutes or travel or workplaces safer. So says Santa Clara associate professor of computer science and engineering David Anastasiu, who is developing AI that reviews hours upon hours of videos of car crashes and other accidents to potentially predict how and when they happen.

This summer, he was awarded a nearly $600,000 grant from the National Science Foundation to develop real-time models for predicting out-of-the-ordinary occurrences that are captured on video, known as video anomaly anticipation. Things like aircraft collisions around airports or a car driving off the street onto a sidewalk full of pedestrians, which are captured on security or street cameras.

AI models based on deep neural networks will then be created to analyze the video data and produce text descriptions of what occurred in any given scenario and, potentially, identify how to avoid disaster in the future.

“Our goal is to develop AI that can predict a potential accident with enough lead time to prevent it from ever occurring,” Anastasiu says.

The models they train will be tasked with two things. First, they need to anticipate the accident, or predict it with enough time and enough confidence to sound an alarm. Second, if an alarm is triggered and an event prevented, the models will need to explain what could have happened if the incident occurred. His team is working with NVIDIA to create datasets that will train these models, as there is a severe lack of such data in video anomaly detection and anticipation.

Currently, Anastasiu and his students use AI in video analysis to detect anomalies after they happen. That is, they flood their models with information from previous accidents and near-misses to produce descriptions of what occurred in any given scenario. This kind of work is still useful in a number of different ways. For example, Anastasiu says, descriptions of a car crash can be “forwarded on to first responders so they’re better equipped to handle the situation rather than just getting a 911 call or hearing an alarm without details,” he says. “This way, they’ll know exactly what happened before they arrive on scene.” Such descriptions can also be used by insurance companies to expedite claims and lower the potential for fraud, or by police when reviewing video to determine if anyone is at fault.

This summer, a team of two students in Anastasiu’s lab, Ridham Kachhadiya M.S. ’24, Ph.D. ’28 and Dhanishtha Patil Ph.D. ’28, put their work to the test at the AI City Challenge, hosted by the 2025 International Conference on Computer Vision. At the event, hundreds of teams from around the world tested their AI models in different competitions that utilized video data sets from physical settings like warehouse environments or traffic systems. Using the video description work from the lab, Kachhadiya and Patil competed in the traffic safety description and analysis track to develop an AI model that used multi-camera footage of staged traffic scenarios based on actual past pedestrian-car incidents provided by a subsidiary of Toyota. Their model then produced descriptions of various traffic incidents and captured details about pedestrian and vehicle behavior, and they were scored based on the accuracy and quality of those descriptions.

They won second place, just behind Chunghwa Telecom, the largest telecom company in Taiwan, and outperforming other teams from major research universities and industry leaders, such as NVIDIA’s Metropolis Video Intelligence team.

Fresh off this summer’s victory, Anastasiu is looking forward to expand the scope of his lab to focus on predictive machine learning.  He’s excited to develop “algorithms that can best solve interesting problems,” he says. And what makes a problem interesting? “In general, I choose problems that could have a meaningful impact — that could benefit the world.”