Squeeze your thumb across your palm as hard as you can and, by the finger thresholds, it is only half closed. The thumb isn’t a finger with a shorter bone — it’s a different joint. Thumb curled, fingers open. Only pull_thumb goes negative (−19.5 N, flexion 0.69); every other motor pushes open. Hinges versus a saddle # Index to pinky are chains of hinge joints. A fist rolls them into a tight spiral, each knuckle approaching 90°. The thumb hangs from the carpometacarpal (CMC) saddle joint at the wrist. It sweeps across the palm — opposition — instead of simply curling. Much of “closing the thumb” is rotation of the whole thumb, not bending at its knuckles. What MediaPipe measures because of it # Interior angles come from 3D landmark positions, so the sweep only partly shows up as knuckle bend:
Open Fully closed Travel Finger — mean of MCP/PIP/DIP ≈ 3.10 rad ≈ 1.60 rad 1.50 rad Thumb — mean of CMC/MCP/IP ≈ 2.90 rad ≈ 2.30 rad 0.60 rad A fully closed thumb stops near 2.30 rad (~132°). Even its open angle sits below a finger’s — a relaxed thumb is never in line with the wrist.
Twenty-one points per frame, on a laptop CPU, from a flat RGB image. That is what MediaPipe hands to a robot. It is easy to treat as a black box — until you compute angles from it and discover the box stretched your coordinate space. Where MediaPipe sits # flowchart TB CAM["USB webcam
640×480 @ 30 fps · YUYV"] -->|"BGR frame"| RGB["cv2.cvtColor
BGR → RGB"] RGB --> MP["MediaPipe Hands
palm detector + landmark model"] MP -->|"21 × (x, y, z)"| ANG["Triplet angles
Part 5"] ANG --> AVG["Average per finger
Part 6"] AVG --> NORM["Flexion 0..1
Parts 7–8"] NORM -->|"/hand/target_flexions"| TWIN["MuJoCo twin"] The vision layer knows nothing about MuJoCo. Its entire output is five numbers between 0.0 (open) and 1.0 (closed).
Two networks, not one # MediaPipe Hands is a cascade:
BlazePalm, a single-shot detector, finds a palm bounding box in the full frame. Palms, not hands: a palm is close to a rigid square; a hand with moving fingers is not. A landmark model crops that region and regresses 21 keypoints, a hand-presence score and handedness. In video mode (static_image_mode=False) the detector barely runs. Landmarks from frame t define the crop for frame t+1, and the detector wakes only when tracking confidence drops. That shortcut is why this pipeline holds 30 fps on a CPU.
The shipped thresholds were measured on one hand with one webcam. Here is how to measure yours — and what a live recording revealed about how far off “good enough” can be while the twin still looks perfect. Symptoms that call for calibration # What you see in the published flexions Cause Change A fist, but flexion stays below 1.0 your fist angle is above the curled limit raise *_CURLED_ANGLE to your fist reading Flexion hits 1.0 with the hand half closed curled limit too high lower *_CURLED_ANGLE Flexion above 0 with a relaxed open hand your open angle is below the straight limit lower *_STRAIGHT_ANGLE to your open reading The thumb flickers thumb window too narrow for your jitter widen it (Part 8) Calibrate against the numbers, not the render. With the current force-control tuning, the simulated finger snaps shut past flexion ≈ 0.51 (Part 11), so most calibration errors are invisible in the viewer. Watch ros2 topic echo /hand/target_flexions. Real data: what a live session produced # MuJoCo’s viewer has a Control panel showing the live force of every motor. Since the force is \(F = 50 - 100 \cdot \text{flexion}\), every recorded frame gives back the exact flexion the tracker published: flexion = (50 − F) / 100.
Twenty-one points, thirty frames a second, on a CPU, from a flat RGB image with no depth information. This is how that is possible — and the four failure modes you will meet the first time you rely on it. Every vision-driven robotics project has a moment where the camera stops being a camera and starts being a sensor. For this one, that moment is MediaPipe Hands: a webcam frame goes in, and 21 numbered points in space come out.
It is easy to treat that as a black box. It is also a mistake, because the box has a specific shape, and its failure modes follow directly from how it was built.
It is two networks, not one # The single most useful thing to know about MediaPipe Hands is that it is a cascade: a detector and a regressor, with completely different jobs.
flowchart TB A["📷 Full frame
e.g. 640 × 480"] --> B["Stage 1 — BlazePalm
SSD detector"] B --> C["Oriented palm crop
256 × 256"] C --> D["Stage 2 — Landmark regressor
MobileNetV2-style encoder"] D --> E["63 floats
21 landmarks × (x, y, z)"] D --> F["Presence score"] D --> G["Handedness
left / right"] F -->|"confidence ≥ 0.5"| C F -->|"confidence < 0.5"| B The regressor’s output, drawn back onto the frame: 21 points and the connections between them. Stage 1 — BlazePalm detects palms, never fingers # This is the design decision the whole system rests on.
