Problem and solution
Problem
Safety teams need a way to review missing helmets and safety vests across camera feeds, distinguish persistent violations from transient detections, and retain evidence for follow-up without treating every frame as a separate incident.
Solution
A camera-to-review pipeline combines person and PPE detection with zone-specific rules, multi-frame confirmation, evidence capture, and role-controlled alert handling. The system supports human safety supervision rather than replacing it.
Technical approach
- Detects person, helmet, and safety-vest using an Ultralytics YOLO PPE model with a person-assist model, optional low-light enhancement, and person-crop refinement when running on GPU.
- Confirms violations in 4 of 5 frames by default, clears the violation state after 3 consecutive compliant frames, and applies a 60-second cooldown to reduce repeated events.
- Applies polygon-based zone rules for required helmets and vests, with low, medium, high, or critical risk levels. Active zone rules take precedence over personal settings.
- Stores event snapshots and buffered clips, with default windows of 5 seconds before and 10 seconds after an event. Camera evidence receives best-effort head blurring before storage.
- Broadcasts real-time updates through WebSocket and sends camera-event email alerts when SMTP is configured, with up to 3 delivery attempts.
- Combines shared dashboard analytics, alert review, protected detection media, and browser-generated PDF reports with JWT authentication and role-based access control.
Implementation details
Developed by Nicky and Krit, this academic prototype connects computer vision with a practical safety-review workflow. A React and TypeScript frontend provides a shared dashboard, camera monitoring, a Reports & Alerts review center, personal detection settings, and administrator-managed user accounts. A FastAPI backend handles authentication, inference, camera processing, event confirmation, protected evidence, and notifications, with PostgreSQL for persistent records. Image, video, and individual-frame detection are also available through authenticated APIs for safety officers and administrators; the legacy upload page is not exposed in the current interface.
Architecture and camera workflow
USB or RTSP source → frame capture → person and PPE inference → zone rules → temporal confirmation → detection, event, and alert creation → privacy-filtered evidence → WebSocket update → optional SMTP delivery. The browser communicates with FastAPI through HTTP, protected media endpoints, MJPEG previews, and WebSocket connections.
Camera runtime, inference coordination, rate limiting, and WebSocket connection state run inside the API process. This architecture targets a single-instance edge pilot. Multiple API replicas require camera workers and shared coordination or a message broker; camera inference currently uses one shared asynchronous lock.
The runtime selects CUDA when available and falls back to CPU, disabling person-crop refinement on CPU to reduce latency. Active cameras attempt to restart with the backend, and reconnect uses exponential backoff up to 30 seconds. These are runtime behaviors and configuration defaults, not measured performance guarantees.
Roles and review workflow
Viewer: reads system-wide dashboards, detection history, alerts, events, and protected evidence, and exports PDF reports. Viewer access is shared across the system rather than limited to personally created records; viewers cannot open unblurred live previews, control cameras, or modify events.
Safety Officer: additionally performs detection, opens live previews, tests and starts or stops registered cameras, acknowledges or resolves incidents, and updates personal detection settings. Administrator: additionally manages users, roles, account status, camera registration, and zone configuration.
Reports & Alerts share a review interface. Events without an associated alert do not yet appear as a separate event list. Camera and zone administration screens do not expose every operation supported by the backend APIs.
Evidence and privacy
The system saves event evidence rather than continuously recording camera video. A JPEG ring buffer retains pre-event frames in memory. Snapshots and clips from backend cameras receive best-effort head blurring; this does not guarantee anonymization and does not cover original uploaded files. The system performs no face recognition or employee identification.
Unblurred live previews stay in memory and are restricted to safety officers and administrators. Detection results, snapshots, and clips are served through authenticated endpoints. Evidence and upload cleanup is configurable, with a default 30-day file retention period; detection and event rows remain, and metadata-row retention is not yet enforced.
Delivery and validation
Docker Compose coordinates PostgreSQL, the Python backend, and the frontend, waits for database health, and runs Alembic migrations before starting the API. The standard backend image uses CPU inference; native execution is recommended for USB cameras where Docker Desktop device passthrough is unsuitable.
The documented Pytest suite covers authentication, role permissions, user administration, camera runtime, detection, alerts, evidence privacy, retention, temporal tracking, and model-evaluation tools. Frontend validation uses lint, strict TypeScript checks, builds, and browser smoke tests; a separate frontend unit or end-to-end suite is not configured.
The project includes model-training, evaluation, and pilot-monitoring tools. Pilot recall, precision, false-alert, throughput, latency, and soak-test targets are acceptance criteria, not achieved results. No production accuracy or performance claim is made here.
Prototype scope and limitations
This is an academic prototype and decision-support system, not a certified safety-control device. It cannot replace safety-officer supervision, may produce false positives or false negatives, and should not be used for automatic disciplinary decisions.
Deployment requires site-specific testing of cameras, network conditions, lighting, hardware, and throughput. The system has no tenant or site isolation, high availability, comprehensive administrator audit log, or formal privacy certification. Model weights, datasets, and the Ultralytics runtime require separate license review before commercial use; the source-code MIT license does not automatically cover them.