Milyonus
Optic Tracking 2.4

Real-time, fully on-device facial optic tracking for the browser: a 478-point face mesh, 3D head pose, per-face confidence scoring, multi-face tracking with persistent IDs, and a focus-tracking system that samples attention once per second. Nothing leaves the device.

View on GitHub

Quick Start

Overview

On-device, private by design

Two MediaPipe Tasks Vision graphs run on every video frame entirely in the browser via WebAssembly/WebGL: FaceLandmarker for a 478-point mesh, 52 blendshape coefficients and a facial transformation matrix, and FaceDetector for a genuine per-face confidence score. Camera frames are processed locally and never uploaded — the only network requests are the one-time model downloads, cached by the browser afterward.

Capabilities

Everything the tracker reports

478-point face mesh

Full tesselation, contours and iris landmarks rendered live over the mirrored camera feed

3D head pose

Yaw, pitch and roll extracted from the raw rotation matrix, plus a relative depth estimate visualized as an RGBaxis gizmo at the nose tip.

Confidence score

A real per-face detection-confidence value read from a dedicated face detector and matched to each tracked face.

Multi-face & Track ID

Tracks up to 4 faces per frame; a nearest-centroid tracker keeps a stable ID per face across frames.

Focus Tracking Score

A continuous 0-1 score from head yaw/pitch, plus a gaze label (center / left / right / up / down / none) and a looking-away alert.

Per-second & per-minute reports

Confidence and gaze are sampled on a clean 1Hz timer and aggregated into a live-updating per-minute table.

How it works

Two graphs, four purpose-built modules

On top of raw model output, dedicated modules turn landmarks into meaning.

facePose.ts

Decomposes the column-major transformation matrix into yaw / pitch / roll, plus a depth proxy from its translation.

faceTracker.ts

A minimal greedy nearest-centroid tracker that assigns a stable ID to each face across frames.

blendshapeEmotion.ts

Heuristically combines 52 blendshape coefficients into 8 expression scores.

focusTracking.ts

Turns head pose into a gaze label and a 0-1 focus score, records one sample per second, and aggregates into per-minute reports.

Quick start

Running in under a minute

Requires Node.js 18+ (or Bun). Models load from the network on first run and are cached by the browser after that.

git clone https://github.com/ayazelrico/Milyonus-Optic-Tracking-2.4
cd milyonus-optic-tracking
npm install
npm

Privacy & models

All inference runs locally via WebAssembly/WebGL — no video frame or image is ever sent to a server. The only network requests fetch the MediaPipe WASM runtime and the two model files on first load.

MIT License

Free to use, modify and distribute. Built on MediaPipe Face Landmarker (Google AI Edge), used under its own license.