Yoloserv 2026 Project

Project Objectives

Repo to large for GitLab / Streamline the repo for Deployment

Move camera_stream from core to YOLO

Problems with camera_stream

Camera stream is a proccess that provides a live video feed to the UI. It has led to better success with the face matching proccess and has allowed for a more seamless user experience. Right now the camera_stream functionalty is in the CORE repo and runs and runs contiously in the background. There are a few issues with this:

It occupies and holds the camera would cause issues with other applications that use core and not the camera_stream
It uses a full CPU core to run which as our applications scale is not maintainable.

Solution

The solution was to move the camera_stream functionality to the YOLO repo and make it a UKDI variable that can be enabled or disabled. However this failed as it added to much delay and lagging when loading the stream and moving frames back and forth from yolo to core back to yolo. The decision was made to keep the camera_stream in core for now but init it from a UKDI var.

Flow Diagram

The Flow has been broken into two parts to show the flow of the camera stream.

flowchart LR
    UI[UI]
    CORE[Core]
    YOLO[YOLO]
    CAM[Camera]

    UI --> | Open Camera Stream | CORE

    CORE --> | Open Camera Stream | YOLO
    YOLO --> | Open Camera Stream | CAM
    CAM --> | Frames | YOLO
    YOLO --> | Frames | CORE

This flow below can be viewed within the method cam_livefeed in the file yoloserv.py.

flowchart LR
    UI[UI]
    CORE[CORE]
    CAM[Camera]

    UI --> | Open Camera Stream | CORE
    CORE --> | Open Camera Stream | CAM
    CAM --> | Frames | CORE
    CORE --> | Camera Stream | UI

Camera Stream Symlink

Dispension has moved hardware conenctions to symlinks after dealing with unreliable USB hubs in the field. Below is how to setup a symlink for a camera stream.

Do not use /dev/video instead tun this command ls -l /dev/v4l/by-id/ to find the correct camera. In the case where the device provides two /dev/video devies we can run v4l2-ctl --list-formats-ext -d /dev/video0 to find the correct device you'll see the formats it provides and also the resolution for those formats.

So I ran ls -l /dev/v4l/by-id/ and v4l2-ctl --list-formats-ext -d /dev/video0 to find the correct device. I then created a symlink to the device with sudo ln -s /dev/v4l/by-id/usb-046d_HD_Pro_Webcam_C920_628DABFF-video-index0 /dev/facematch_camera. I confirmed this worked by running ls -l /dev/facematch_camera and v4l2-ctl --list-formats-ext -d /dev/facematch_camera.

Yoloserv pipeline

sequenceDiagram
    participant UI
    participant CORE
    participant CAM
    participant YOLO
    participant para

    rect rgba(200, 220, 255, 0.35)
        UI->>CORE: pic_still
        CORE->>CAM: capture still
        CAM-->>CORE: still image
        CORE-->>UI: still image
    end

    rect rgba(220, 255, 220, 0.35)
        UI->>CORE: facematch (regula)
        CORE->>YOLO: facematch
        YOLO->>para: facematch
        para-->>YOLO: result
        YOLO-->>CORE: result
        CORE-->>UI: result
    end

Implement 2D detection

Dispension faced the problem of needing a detection model in place for fully autonomous operation. To resolve this we implemented a 2D detection model that can be used to detect faces in a video stream. We implemented paravisions 2D model detection. Refer to para.md for more information on installing models.

Here are the results from a live user test:

 <ValidnessResult (isValid=0, sharpness=-1.000000, quality=-1.000000, acceptability=-1.000000, frontality=-1.000000, mask=-1.000000, image_avg_pixel=115.375671, image_bright_pixel_pct=0.145549, image_dark_pixel_pct=0.030642, face_height_pct=0.318821, face_width_pct=0.208163, face_width=133.224304, face_height=153.033981, face_roll_angle=3.102657, face_positions=[0.377501,-0.001564,0.585664,0.317257])>
HERE IS THE LIVENESS RESULT <LivenessResult (liveProbability=0.999998, spoofProbability=0.000002)>

HERE IS THE VALIDNESS RESULT <ValidnessResult (isValid=0, sharpness=0.023801, quality=-1.000000, acceptability=-1.000000, frontality=97.000000, mask=0.008516, image_avg_pixel=58.403454, image_bright_pixel_pct=0.000000, image_dark_pixel_pct=0.000000, face_height_pct=0.551221, face_width_pct=0.579525, face_width=373.213989, face_height=446.488739, face_roll_angle=0.489146, face_positions=[0.182151,0.242310,0.761676,0.793531])>
HERE IS THE LIVENESS RESULT <LivenessResult (liveProbability=0.057056, spoofProbability=0.830750)>

Improvements

Ideally we should be using the first frame to complete the facematch process and not for rendering. This would speed things up by a few seconds but is something for the future.

Add cleanup functionality to remove localcam files after facematch is complete.

4.9 KiB Raw Permalink Blame History