coxner/yoloserv
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

added docs inital push for ox_Server

Browse Source
This commit is contained in:
ox 2026-01-06 13:43:11 -04:00
parent 3c73c8eb4c
commit 3c4cbfd300
3 changed files with 288 additions and 70 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

113
doc/fm_project2026.md Normal file
View File

@ -0,0 +1,113 @@
# Yoloserv 2026 Project
## Project Objectives
### Repo to large for GitLab / Streamline the repo for Deployment
### Move camera_stream from core to YOLO
#### Problems with camera_stream
Camera stream is a proccess that provides a live video feed to the UI. It has led to better success with the face matching proccess and has allowed for a more seamless user experience. Right now the camera_stream functionalty is in the CORE repo and runs and runs contiously in the background. There are a few issues with this:
- It occupies and holds the camera would cause issues with other applications that use core and not the camera_stream
- It uses a full CPU core to run which as our applications scale is not maintainable.
#### Solution
The solution was to move the camera_stream functionality to the YOLO repo and make it a UKDI variable that can be enabled or disabled. However this failed as it added to much delay and lagging when loading the stream and moving frames back and forth from yolo to core back to yolo. The decision was made to keep the camera_stream in core for now but init it from a UKDI var.
#### Flow Diagram
The Flow has been broken into two parts to show the flow of the camera stream.
```mermaid
flowchart LR
UI[UI]
CORE[Core]
YOLO[YOLO]
CAM[Camera]
UI --> | Open Camera Stream | CORE
CORE --> | Open Camera Stream | YOLO
YOLO --> | Open Camera Stream | CAM
CAM --> | Frames | YOLO
YOLO --> | Frames | CORE
```
This flow below can be viewed within the method `cam_livefeed` in the file `yoloserv.py`.
```mermaid
flowchart LR
UI[UI]
CORE[CORE]
CAM[Camera]
UI --> | Open Camera Stream | CORE
CORE --> | Open Camera Stream | CAM
CAM --> | Frames | CORE
CORE --> | Camera Stream | UI
```
### Yoloserv pipeline
```mermaid
sequenceDiagram
participant UI
participant CORE
participant CAM
participant YOLO
participant para
rect rgba(200, 220, 255, 0.35)
UI->>CORE: pic_still
CORE->>CAM: capture still
CAM-->>CORE: still image
CORE-->>UI: still image
end
rect rgba(220, 255, 220, 0.35)
UI->>CORE: facematch (regula)
CORE->>YOLO: facematch
YOLO->>para: facematch
para-->>YOLO: result
YOLO-->>CORE: result
CORE-->>UI: result
end
```
### Implement 2D detection
Dispension faced the problem of needing a detection model in place for fully autonomous operation. To resolve this we implemented a 2D detection model that can be used to detect faces in a video stream. We implemented paravisions 2D model detection. Refer to para.md for more information on installing models.
Here are the results from a live user test:
```bash
<ValidnessResult (isValid=0, sharpness=-1.000000, quality=-1.000000, acceptability=-1.000000, frontality=-1.000000, mask=-1.000000, image_avg_pixel=115.375671, image_bright_pixel_pct=0.145549, image_dark_pixel_pct=0.030642, face_height_pct=0.318821, face_width_pct=0.208163, face_width=133.224304, face_height=153.033981, face_roll_angle=3.102657, face_positions=[0.377501,-0.001564,0.585664,0.317257])>
HERE IS THE LIVENESS RESULT <LivenessResult (liveProbability=0.999998, spoofProbability=0.000002)>
HERE IS THE VALIDNESS RESULT <ValidnessResult (isValid=0, sharpness=0.023801, quality=-1.000000, acceptability=-1.000000, frontality=97.000000, mask=0.008516, image_avg_pixel=58.403454, image_bright_pixel_pct=0.000000, image_dark_pixel_pct=0.000000, face_height_pct=0.551221, face_width_pct=0.579525, face_width=373.213989, face_height=446.488739, face_roll_angle=0.489146, face_positions=[0.182151,0.242310,0.761676,0.793531])>
HERE IS THE LIVENESS RESULT <LivenessResult (liveProbability=0.057056, spoofProbability=0.830750)>
```
## Improvements
Ideally we should be using the first frame to complete the facematch process and not for rendering. This would speed things up by a few seconds but is something for the future.

37
doc/gitea_setup.md Normal file
View File

@ -0,0 +1,37 @@
# Gitea Setup
## Table of Contents
- [Gitea Setup](#gitea-setup)
- [Table of Contents](#table-of-contents)
- [Installation](#installation)
- [Configuration](#configuration)
- [Usage](#usage)
- [Troubleshooting](#troubleshooting)
The Problem we faced is that yoloserv was to big to be hosted on Gitlab. Gitlab would also suspend the repo if bills were not paid and user seats were maxed. Obviously this was not a viable solution. Explored maybe self hosting without an intreface just to have repos but when I read Gitea did not need docker and could be run as a system service I tried it out.
## Setting up the Server
First step was setting up the server to run gitea. I scanned my local netowrk with ```bash arp-scan --localnet``` to find the IP address of the server. I could nto find my server running I had to reset the DHCP client with these commands. Use ```nmcli connection show``` to find the connection name and then ```nmcli connection down <connection name>``` and ```nmcli connection up <connection name>``` to reset the DHCP client. Was able to succesfully ```ping 8.8.8.8``` to confirm the server was running.
## Installation
I remoted onto the server and installed gitea with the following commands:
Ran ```free -h``` to check memory and ```df -h``` to check disk space. Was recommended to have at least 8GB of RAM and 2.5 repo size of disk space. These seem a bit much.
Depending on what distro you run commands may vary but I:
- Updated and upgraded the system
- Installed git, openssh-server, ufw and fail2ban
- sudo ufw enable, sudo ufw allow ssh, sudo ufw allow 3000
#### SSH Hardening
Can harden the SSH to be keys only should be done will be a future addition. Done in ```/etc/ssh/sshd_config```. Run ```sudo systemctl restart sshd``` to apply changes.
```bash
PasswordAuthentication no
PermitRootLogin no
```

198
src/paravisionx.py
View File

@ -1,139 +1,207 @@
#
# Paravision based face matcher
#
from paravision.recognition.sdk import SDK
from paravision.recognition.types import Settings, ValidnessCheck
from paravision.recognition.exceptions import ParavisionException
from paravision.liveness2d import SDK as Liveness2DSDK
from paravision.liveness2d.types import (
Settings as Liveness2DSettings,
ValidnessSettings,
)
import paravision.recognition.utils as pru
import json
import sys
import os
import cv2
import torchvision.transforms
### export PYTHONPATH=/wherever/yoloserv/modules ... as long as "paravision/.../" is in there
from paravision.recognition.exceptions import ParavisionException
from paravision.recognition.sdk import SDK
from paravision.recognition.engine import Engine
import paravision.recognition.utils as pru
#from openvino.inference_engine import Engineq
#from deepface.basemodels import VGGFace, OpenFace, Facenet, FbDeepFace, DeepID
from faceclass import FaceClass
# Paravision is quite unlike all the other open source img recognition packages
# which cause problems when image formats move between the two.
# So this class has quite a few differences.
class Paravision(FaceClass):
models = {}
scores = {}
quality = {}
sdk = None
match_score = 0
def init(self,backend=None,model=None):
print("@@@ initialising paravision")
def init(self):
try:
self.sdk = SDK(engine=Engine.AUTO)
except ParavisionException:
pass
print("INIT FROM PARAVISION")
settings = Settings()
# 1 is the default
settings.worker_count = 4
settings.detection_model = "default"
settings.validness_checks = ValidnessCheck.LIVENESS
self.sdk = SDK(settings=settings)
liveness2d_settings = Liveness2DSettings()
self.liveness2d_sdk = Liveness2DSDK(settings=liveness2d_settings)
except ParavisionException as e:
# error handling logic
print("Exception:", e)
# @doc Load a pic using the device label
def load1(self, name,fname):
print(" Loading image '%s' from file %s" % (name, fname))
def load1(self, name, fname):
print(" Loading image '%s' from file %s" % (name, fname))
if not os.path.isfile(fname):
print(" * file not found: %s" % (fname))
return '{ "status":442565, "remark":"file name not found", "guilty_param":"fname", "guilty_value":"%s" }' % (fname)
print(" * file not found: %s" % (fname))
return (
'{ "status":442565, "remark":"file name not found", "guilty_param":"fname", "guilty_value":"%s" }'
% (fname)
)
self.files[name] = fname
self.imgs[name] = pru.load_image(fname)
print(" Loaded %s from file %s" % (name, fname))
return '{ "status":0, "remark":"OK", "name":"%s", "fname":"%s" }' % (name,fname)
print(" Loaded %s from file %s" % (name, fname))
return '{ "status":0, "remark":"OK", "name":"%s", "fname":"%s" }' % (
name,
fname,
)
# @doc find all the faces in the named image
def detect(self, name):
boxes = []
self.boxes = []
print("** face_recognition::detect ... %s" % name)
try:
# Get all faces from images with qualities, landmarks, and embeddings
faces = self.sdk.get_faces([self.imgs[name]], qualities=True, landmarks=True, embeddings=True)
faces = self.sdk.get_faces(
[self.imgs[name]], qualities=True, landmarks=True, embeddings=True
)
print("HERE IS THE FACES %s" % faces)
inferences = faces.image_inferences
print("HERE IS THE INFERENCE %s" % inferences)
ix = inferences[0].most_prominent_face_index()
face = inferences[0].faces[ix]
print("HERE IS THE FACE %s" % face)
self.models[name] = inferences[0].faces[ix].embedding
self.quality[name] = round(1000*inferences[0].faces[ix].quality)
self.boxes = [ (0,0,0,0) ]
self.quality[name] = round(1000 * inferences[0].faces[ix].quality)
self.boxes = [(0, 0, 0, 0)]
except Exception as ex:
self.errstr = "image processing exception at get_faces: "+str(ex)
return '{ "status":222310, "remark":"image processing exception", "guilty_param":"error", "guilty_value":"%s" }' % str(ex)
return '{ "status":0, "remark":"OK", "faces":%d, "boxes":%s }' % (len(self.boxes), json.dumps(self.boxes))
self.errstr = "image processing exception at get_faces: " + str(ex)
return (
'{ "status":222310, "remark":"image processing exception", "guilty_param":"error", "guilty_value":"%s" }'
% str(ex)
)
return '{ "status":0, "remark":"OK", "faces":%d, "boxes":%s }' % (
len(self.boxes),
json.dumps(self.boxes),
)
def detect_liveness(self, name):
self.boxes = []
print("** face_recognition::detect ... %s" % name)
try:
# Get all faces from images with qualities, landmarks, and embeddings
faces = self.sdk.get_faces(
[self.imgs[name]], qualities=True, landmarks=True, embeddings=True
)
print("HERE IS THE FACES %s" % faces)
inferences = faces.image_inferences
print("HERE IS THE INFERENCE %s" % inferences)
ix = inferences[0].most_prominent_face_index()
face = inferences[0].faces[ix]
print("HERE IS THE FACE %s" % face)
liveness_result = self.validness(face, self.sdk)
live_prob = getattr(liveness_result, "live_prob", None)
print("HERE IS THE LIVE PROB %s" % live_prob)
if live_prob < 0.5:
return (
'{ "status":222310, "remark":"image processing exception", "guilty_param":"error", "guilty_value":"%s" }'
% live_prob
)
self.models[name] = inferences[0].faces[ix].embedding
self.quality[name] = round(1000 * inferences[0].faces[ix].quality)
self.boxes = [(0, 0, 0, 0)]
except Exception as ex:
self.errstr = "image processing exception at get_faces: " + str(ex)
return (
'{ "status":222310, "remark":"image processing exception", "guilty_param":"error", "guilty_value":"%s" }'
% str(ex)
)
return '{ "status":0, "remark":"OK", "faces":%d, "boxes":%s }' % (
len(self.boxes),
json.dumps(self.boxes),
)
# @doc This does everything for you.
# If you are smartserv, "crowd" means cam and "govid" means regula pic
def crowd_vs_govid(self, name1,file1,scale1, name2,file2,scale2):
print("##1##")
def crowd_vs_govid(self, name1, file1, scale1, name2, file2, scale2):
print("##1## DETECTING FROM IMG")
if self.json2obj(self.load1(name1, file1))["status"] != 0:
return self.jsonx
if self.json2obj(self.detect(name1))["status"] != 0:
if self.json2obj(self.detect_liveness(name1))["status"] != 0:
return self.jsonx
self.save(name1,"/tmp")
self.save(name1, "/tmp")
print("##2##")
print("##2## DETECTING FROM ID")
if self.json2obj(self.load1(name2, file2))["status"] != 0:
return self.jsonx
self.save(name2,"/tmp")
if self.json2obj(self.detect(name2))["status"]!=0:
self.save(name2, "/tmp")
if self.json2obj(self.detect(name2))["status"] != 0:
return self.jsonx
self.save(name2,"/tmp")
self.save(name2, "/tmp")
print("##R##")
jsonstr = self.compare(name1,name2)
jsonstr = self.compare(name1, name2)
print(jsonstr)
return jsonstr
# @doc compare two named images, previously loaded
def compare(self, name1, name2):
print("** face_recognition::compare ... %s vs %s" % (name1,name2))
print("** face_recognition::compare ... %s vs %s" % (name1, name2))
try:
res = self.sdk.get_match_score(self.models[name1], self.models[name2])
print("Match is ",res)
print("Match is ", res)
self.match_score = res
except Exception as ex:
print("** paravision::compare exception ... " + str(ex) )
self.errstr = "image comparison exception at compute_scores: "+str(ex)
print("** paravision::compare exception ... " + str(ex))
self.errstr = "image comparison exception at compute_scores: " + str(ex)
return '{ "status":332410, "remark":"%s" }' % self.errstr
return '{ "status":0, "threshold": 500, "device1_qual": 0.5, "device2_qual": 0.5, "remark":"OK", "score":%d }' % self.match_score
return (
'{ "status":0, "threshold": 500, "device1_qual": 0.5, "device2_qual": 0.5, "remark":"OK", "score":%d }'
% self.match_score
)
def validness(self, face, sdk):
validness_settings = ValidnessSettings(face)
validness_result = self.liveness2d_sdk.check_validness(
face, validness_settings, sdk
)
print("HERE IS THE VALIDNESS RESULT %s" % validness_result)
liveness_result = self.liveness2d_sdk.get_liveness(face)
print("HERE IS THE LIVENESS RESULT %s" % liveness_result)
return liveness_result
def scores(self):
return '{ "status":0, "threshold": 500, "device1_qual": 0.5, "device2_qual": 0.5, "remark":"OK", "score":%d }' % self.match_score
return (
'{ "status":0, "threshold": 500, "device1_qual": 0.5, "device2_qual": 0.5, "remark":"OK", "score":%d }'
% self.match_score
)
if __name__ == "__main__":
if __name__ == '__main__':
d = Paravision()
d = Paravisionox()
d.init()
if sys.argv[1]=="messia":
if sys.argv[1] == "messia":
jsonstr = d.load1("pic1", "testimg/messi4.jpg")
print(jsonstr)
jsonstr = d.detect("pic1")
print(jsonstr)
if sys.argv[1]=="test":
if sys.argv[1] == "test":
d.load1("pic1", "testimg/ox.jpg")
d.detect("pic1")
if sys.argv[1]=="kiosk":
jsonstr = d.crowd_vs_govid("pic1", "testimg/ox.jpg", 0, "pic2", "testimg/ox_govid.jpg", 0.25)
if sys.argv[1] == "kiosk":
jsonstr = d.crowd_vs_govid(
"pic1", "testimg/ox.jpg", 0, "pic2", "testimg/ox_govid.jpg", 0.25
)
print(jsonstr)
if sys.argv[1]=="messi":
jsonstr = d.crowd_vs_govid("pic1", "testimg/messi4.jpg", 0, "pic2", "testimg/messi2.jpg", 0)
if sys.argv[1] == "messi":
jsonstr = d.crowd_vs_govid(
"pic1", "testimg/messi4.jpg", 0, "pic2", "testimg/messi2.jpg", 0
)
print(jsonstr)
if sys.argv[1]=="maiden":
jsonstr = d.crowd_vs_govid("pic1", "testimg/ironmaiden.jpg", 0, "pic2", "testimg/davemurray.jpg", 0)
if sys.argv[1] == "maiden":
jsonstr = d.crowd_vs_govid(
"pic1", "testimg/ironmaiden.jpg", 0, "pic2", "testimg/davemurray.jpg", 0
)
print(jsonstr)