Data Engineering & Machine Learning
PotholeVision
Built an end-to-end data pipeline that transformed street footage into geolocated pothole detections.
Graduate Team Project · MSBA Big Data Analytics
Problem
Manual road inspection does not scale.
Municipalities often depend on manual inspection and citizen reports to locate road damage, requiring additional time and resources before repairs can even begin.
Proposed Use
Collect road footage during normal city operations.
Road-facing cameras could be mounted on municipal fleet vehicles such as police, emergency-response, or public works vehicles, allowing roadway footage to be collected passively while those vehicles perform their normal duties.
Outcome
Turn footage into prioritized repair intelligence.
Likely potholes are detected, geolocated, and surfaced in ArcGIS alongside traffic-volume information so maintenance teams can identify and prioritize high-impact locations.
System Architecture
From road video to mapped pothole detections
Technical Deep Dive
Three core pieces of the detection pipeline
Frame Extraction
Raw road video was ingested into Databricks and processed with OpenCV. The pipeline sampled one image per second while preserving the frame index and video timestamp before writing extracted-frame metadata to a Silver Delta table.
CNN Classification
A ResNet18-based CNN classified each extracted frame as pothole or non-pothole. ResNet18 was selected to balance predictive performance with the project's compute and memory constraints.
Detection Logic
The model returned a pothole probability rather than only a class label. Adjustable probability thresholds controlled which frames became positive detections and were passed into the downstream geolocation workflow.
Model Development
The main challenge was generalization.
Early testing showed that the model could learn visual characteristics associated with a particular dataset rather than the actual characteristics of a pothole. The modeling process therefore expanded beyond a single train/test split.
→ Tested external positive and negative datasets to identify domain shift.
→ Built an alligator-cracking challenge set to stress-test visually similar road damage.
→ Experimented with hard-negative mining, class weighting, and probability thresholds.
→ Added more diverse pothole training examples and validated the final model on an independent external benchmark.
Independent External Validation
Earlier Recall
66.3%
→
Final Recall
90.6%
90.6%
Recall
88.2%
Accuracy
9.4%
Miss Rate
Why prioritize recall?
Questionable detections can be reviewed and removed from the ArcGIS dashboard. A missed pothole, however, never enters the repair workflow at all.
Decision Support
The model is only useful if someone can act on it.
Positive detections were designed to feed a geospatial workflow in ArcGIS. Nearby detections could be consolidated into map points and combined with traffic-volume information, giving municipalities an intuitive way to prioritize road repairs rather than treating every detected pothole equally.
Project Takeaway
The strongest improvement did not come from simply increasing model complexity. It came from identifying domain shift, refining the classification boundary, testing across multiple datasets, and deliberately balancing false positives against missed potholes.
Explore the full project
View the team repository, notebooks, model artifacts, and supporting project documentation.