Ros2 on Mulham Fetna

Docker Compose architecture for ROS 2: dependency-only images and mounted code

contact@mulhamfetna.com (Mulham Fetna) — Wed, 16 Sep 2026 00:00:00 +0000

Both images hold dependencies and nothing else. The code and the robot model are mounted from your checkout at runtime — so an edit is a five-second restart, not a five-minute rebuild.

The big picture
#

flowchart TB
    subgraph HOST["🐧 Linux host"]
        CAMDEV["/dev/video0"]
        GPU["/dev/dri · Intel iGPU"]
        X11["/tmp/.X11-unix
XWayland :0"]
        SHM["/dev/shm
Fast DDS segments"]
        NET["host network
UDP multicast · domain 42"]
        REPO["repository checkout"]
        subgraph VT["🐳 vision_tracker · 2.9 GB image"]
            VN["vision_tracker_node.py
mediapipe 0.10.14 · OpenCV"]
        end
        subgraph MT["🐳 mujoco_twin · 1.7 GB image"]
            MN["mujoco_twin_node.py
mujoco 3.13.0 · GLFW"]
        end
    end
    CAMDEV --> VN
    GPU --> VN
    GPU --> MN
    X11 <--> VN
    X11 <--> MN
    VN <--> SHM <--> MN
    VN <--> NET <--> MN
    REPO -. "bind mount .:/workspace:ro" .-> VN
    REPO -. "bind mount .:/workspace:ro" .-> MN

Repository layout
#

.
├── docker-compose.yml          # both services, shared namespaces
├── setup_host.sh               # xhost + device checks, once per login
├── vision_tracker/
│   ├── Dockerfile              # ros:jazzy + mediapipe==0.10.14
│   ├── .dockerignore           # src/ is mounted, so keep it out of the build context
│   └── src/vision_tracker_node.py
├── mujoco_twin/
│   ├── Dockerfile              # ros:jazzy + mujoco==3.13.0
│   ├── .dockerignore           # src/ and model/ are mounted
│   ├── src/mujoco_twin_node.py
│   └── model/                  # scene.xml → robot.xml (+ tendons.xml), assets/, config.json
├── standalone/main.py          # the same pipeline, one process
└── docs/

Each service folder is its own build context: editing the vision Dockerfile never invalidates the twin’s image cache, and neither build uploads the 13 MB of meshes it doesn’t need.

From tutorial project to production robot: a roadmap for the tendon hand twin

contact@mulhamfetna.com (Mulham Fetna) — Wed, 16 Sep 2026 00:00:00 +0000

The system works, and it teaches well. Getting it to drive real servos safely is a sequence of well-scoped upgrades — each one grounded in a limitation measured earlier in this series.

Where it stands
#

Area	Today	Production target
Hand tracking	image-normalized landmarks, one global calibration	metric world landmarks, per-finger calibration, a temporal filter
Command mapping	flexion → ±50 N; the twin is a switch	flexion → tendon length; proportional curl
Physics model	force motors, decorative horns, no self-contact	position servos on horns, tuned stiffness, contacts for grasping
Middleware	one topic, default QoS, open on the LAN	parameters, explicit QoS, a watchdog, SROS 2
Containers	privileged, host namespaces, root, `xhost`	least privilege, non-root, optional headless
Code	classes copied into three files, no tests	one shared package, tests, CI
Hardware	simulation only	a servo driver on the same topic

Stage 1 · Correctness

Measure the right thing, command the right quantity

World landmarks. Image-normalized coordinates bend angles by up to 16° with hand orientation (Part 4). Read multi_hand_world_landmarks instead, then recalibrate.

GUI, webcam and GPU passthrough for ROS 2 and MuJoCo containers

contact@mulhamfetna.com (Mulham Fetna) — Wed, 16 Sep 2026 00:00:00 +0000

Containers are headless by design. This project needs two windows, a webcam and a GPU — so the Compose file spends most of its lines punching carefully chosen holes back through the isolation.

Two containers, two windows, one X display: the Qt/OpenCV tracker (vision_tracker) and the GLFW MuJoCo viewer (mujoco_twin), recorded on Wayland through XWayland.

Windows: pass the X11 socket through
#

environment:
  DISPLAY: ${DISPLAY:-:0}
  QT_X11_NO_MITSHM: 1
volumes:
  - /tmp/.X11-unix:/tmp/.X11-unix:rw

Piece	What it does
`/tmp/.X11-unix` mount	the X server listens on a Unix socket here (`X0` for `:0`); mounting it gives the container a line to it
`DISPLAY`	tells X clients (GLFW, Qt) which display to use; defaults to `:0`
`QT_X11_NO_MITSHM=1`	stops Qt — OpenCV’s `imshow` backend — from using MIT-SHM, which fails across containers and gives blank or garbled windows
`xhost +local:root` (in `setup_host.sh`)	the X server refuses untrusted clients; this admits local root, which is who the containers run as

On Wayland
#

Wayland sessions (KDE Plasma, GNOME) still run XWayland on :0, and both windows open through it — the recording above was made exactly that way. Running MuJoCo natively on a Wayland host, GLFW may warn Wayland: The platform does not provide the window position; it’s harmless.

How a webcam moves a simulated tendon-driven hand

contact@mulhamfetna.com (Mulham Fetna) — Wed, 16 Sep 2026 00:00:00 +0000

One vision container turns webcam frames into five numbers. One simulation container turns those numbers into tendon forces. Everything else in this series is detail inside one of those two boxes — or the pipe between them.

Left to right, three layers in one frame: vision (landmarks), simulation (the twin), actuation (live motor forces from MuJoCo’s Control panel).

End to end
#

flowchart LR
    subgraph VISION["🐳 vision_tracker container"]
        direction TB
        A["Webcam frame
640×480 BGR"] --> B["MediaPipe Hands
21 landmarks"]
        B --> C["3 knuckle angles / finger
dot product"]
        C --> D["mean → 1 curl angle
(underactuation)"]
        D --> E["normalize + clip
flexion 0..1"]
    end
    subgraph TWIN["🐳 mujoco_twin container"]
        direction TB
        F["lerp
+50 N … −50 N"] --> G["data.ctrl on
pull_{finger} motor"]
        G --> H["spatial tendon
through 6 sites"]
        H --> I["3 passive hinge joints
curl"]
        I --> J["MuJoCo viewer"]
    end
    E -- "ROS 2 · /hand/target_flexions
sensor_msgs/JointState" --> F

Stage	What comes out	Deep dive
MediaPipe Hands	21 (x, y, z) landmarks per frame	Part 4
Triplet angles	3 interior angles per finger, in radians	Part 5
Averaging	1 curl angle per finger	Part 6
Normalization	flexion 0..1 (the thumb has its own window)	Parts 7–8
ROS 2 topic	`JointState`: names are fingers, positions are flexions	Part 15
Lerp	force in newtons per tendon	Part 10
Tendon physics	joint angles	Part 11

Why the pipe carries flexions, not angles or forces
#

The contract between the containers is five unitless numbers: 0.0 is an open finger, 1.0 is a closed one. That choice is the architecture.

ROS 2 across Docker containers: shared network, discovery and Fast DDS shared memory

contact@mulhamfetna.com (Mulham Fetna) — Wed, 16 Sep 2026 00:00:00 +0000

“The topic shows up in ros2 topic list, but echo prints nothing” — the classic ROS-in-Docker symptom. Here’s why it happens, how two containers in this project share a network and a block of memory instead, and what that costs.

How ROS 2 nodes find each other
#

ROS 2 has no master. Nodes discover each other through DDS — in Jazzy, eProsima Fast DDS by default — using the SPDP protocol:

Each participant announces itself over UDP multicast (239.255.0.1) on ports derived from the domain ID $d$: discovery multicast on $7400 + 250d$, unicast on $7410 + 250d + 2p$ for participant $p$. For domain 42: UDP 17900 and 17910+.
Peers exchange their topic endpoints; matching publishers and subscribers connect.
Data flows over the best transport both support — shared memory when they share a host and /dev/shm, UDP otherwise.

Why Docker’s default network breaks it
#

Compose attaches services to a bridge network — a private NATed subnet. Multicast isn’t reliably routed across it, and machines on your LAN can’t reach container IPs at all.

Run the tendon-driven hand digital twin in 10 minutes

contact@mulhamfetna.com (Mulham Fetna) — Wed, 16 Sep 2026 00:00:00 +0000

Clone, run one setup script, run one Compose command — and a webcam window and a MuJoCo viewer open side by side, with a simulated hand that closes when you close yours. Here are four ways in, from the full stack down to the bare model.

The goal of this article: both windows open, and the twin follows your hand.

What you need
#

Requirement	Check with	Notes
Linux desktop, X11 or XWayland	`echo $DISPLAY` → `:0`	Wayland sessions work through XWayland
Docker Engine + Compose v2	`docker compose version`	Built with Docker 29.8 / Compose 5.5
A webcam	`ls /dev/video0`	Close any other app using it
GPU device nodes	`ls /dev/dri`	Intel/AMD out of the box; NVIDIA needs the container toolkit (Part 18)
~5 GB of disk		images: vision 2.9 GB, twin 1.7 GB

No ROS installation is needed on the host — ROS 2 Jazzy lives inside the containers.

Four ways to run it
#

git clone https://github.com/mulhamfetna/ros2-tendon-driven-hand-mujoco-digital-twin-vision-teleoperation.git
cd ros2-tendon-driven-hand-mujoco-digital-twin-vision-teleoperation

./setup_host.sh              # X11 access for the containers + camera/GPU checks (once per login)
docker compose up --build    # builds both images, starts vision_tracker and mujoco_twin

Stop with Ctrl+C, then docker compose down to release the camera.

The ROS 2 topic contract: one JointState topic, two nodes, and rclpy inside a render loop

contact@mulhamfetna.com (Mulham Fetna) — Wed, 16 Sep 2026 00:00:00 +0000

The whole ROS 2 layer is one topic carrying five numbers. The interesting decisions are what those numbers mean, which message type carries them, and how to run a ROS subscriber when a 3D viewer owns your main thread.

Why ROS 2 between vision and physics at all?
#

The single-process script works well. Splitting it across ROS 2 buys:

Benefit	Concretely
Process isolation	a MediaPipe crash doesn’t kill the simulator, and vice versa
Independent environments	vision and physics get their own container, dependencies and restarts
Swappable endpoints	replace the twin with a servo driver, or the tracker with a data glove
Free observability	`ros2 topic echo`, `hz`, `bag record` on the live stream
Network transparency	any machine on the LAN can subscribe

The costs — a bigger stack, DDS configuration, one more hop — are small next to 19–85 ms of inference (Part 19).

The node graph
#

flowchart LR
    subgraph C1["🐳 vision_tracker"]
        V["/vision_tracker_node
timer · 30 Hz"]
    end
    subgraph C2["🐳 mujoco_twin"]
        T["/mujoco_twin_node
spin_once in viewer loop"]
    end
    V -- "/hand/target_flexions
sensor_msgs/JointState · depth 10" --> T
    V -. "any LAN subscriber
ROS_DOMAIN_ID=42" .-> X["ros2 topic echo · rosbag
future servo driver"]

Every frame of the twin is driven by one JointState message like the one below.

The contract
#

Field	Value
Topic	`/hand/target_flexions`
Type	`sensor_msgs/msg/JointState`
Publisher	`vision_tracker_node`, timer at 30 Hz — effective rate bounded by inference
Subscriber	`mujoco_twin_node`
QoS	default reliable, keep-last 10
`header.stamp`	publisher clock at publish time
`name`	`["thumb", "index", "middle", "ring", "pinky"]`
`position`	flexion per finger, 0.0 open … 1.0 closed, same order as `name`
`velocity`, `effort`	empty

A real message, captured with ros2 topic echo during testing (published by hand with ros2 topic pub, hence the zero stamp):

Troubleshooting and FAQ: MuJoCo, MediaPipe and ROS 2 in Docker

contact@mulhamfetna.com (Mulham Fetna) — Wed, 16 Sep 2026 00:00:00 +0000

Every entry here was hit, or deliberately checked, while building this project. Error messages are quoted exactly so a search for the message lands on the fix.

Camera and windows
#

`RuntimeError: Failed to open camera at index 0`
#

Something else holds the webcam — the standalone script, a previous container, a browser tab. Run docker compose down and close video apps. A camera opens in one process at a time.
The camera is another node. ls /dev/video*, then map that device and set CAMERA_INDEX (many webcams expose /dev/video0 for frames and /dev/video1 for metadata — use the first).
The device isn’t mapped. Check with docker compose config | grep video.

Windows don’t open — `cannot connect to X server`, `could not connect to display`
#

Run ./setup_host.sh (it runs xhost +local:root). The permission resets when you log out.
Make sure echo $DISPLAY on the host prints :0, or export DISPLAY before docker compose up.
On Wayland, confirm XWayland is running: ls /tmp/.X11-unix/ should list X0.

Black, blank or garbled OpenCV window
#

QT_X11_NO_MITSHM=1 must reach the container. It’s in the shared Compose environment — if a service defines its own environment:, it must merge the shared anchor with <<: *ros-env rather than replace it (Part 16).

A webcam, some vector geometry, and a hand that moves

contact@mulhamfetna.com (Mulham Fetna) — Tue, 08 Sep 2026 00:00:00 +0000

Everything in this series in one read: how a $20 webcam ends up driving a 15-DOF CAD model in real time, why every step is deliberately explicit rather than learned, and what broke along the way.

You hold your hand up to a laptop camera. On the other half of the screen, a robotic hand — designed in CAD, never manufactured — closes its fingers at the same moment yours do.

There is no glove, no marker, no depth sensor. Just an RGB webcam, two small neural networks, about forty lines of vector geometry, and a middleware stack that thinks it is talking to a real robot.

All of it is open source under AGPL-3.0 and archived with a DOI: 10.5281/zenodo.22658556.

flowchart LR
    A["📷 Webcam
/dev/video0"] --> B["BlazePalm
palm detector"]
    B --> C["Landmark regressor
21 × (x, y, z)"]
    C --> D["Dot-product geometry
15 interior angles"]
    D --> E["Normalize → flexion
0.0 straight · 1.0 curled"]
    E --> F["Lerp onto the URDF's
mechanical limits"]
    F --> G["/joint_states"]
    G --> H["robot_state_publisher
→ /tf"]
    H --> I["🖥️ RViz digital twin"]

The rule that shaped the build
#

There is an easier version of this project. Collect a few thousand frames of a hand next to the corresponding CAD poses, train a network to map one to the other, and let gradient descent work out the relationship.

Containerizing ROS 2 without losing the hardware

contact@mulhamfetna.com (Mulham Fetna) — Tue, 08 Sep 2026 00:00:00 +0000

Web developers containerize to isolate processes. Roboticists containerize and then spend the rest of the day punching holes back through the isolation — for the camera, the GPU, the display server and the network.

ROS 2 Jazzy is hard-locked to Ubuntu 24.04. That single fact is reason enough to containerize: without it, adopting a ROS distribution means adopting an operating system version, on every machine that will ever run the code.

But a ROS container is a strange artifact. A web service container wants isolation — that is the product. A robotics container needs a webcam, a GPU, a display server and multicast networking, all of which live outside it. You end up building a box and then carefully cutting four holes in it.

Here is every hole in this project’s compose file and what it is for.

Cold start to live tracking. Note hand_msgs being compiled separately inside two different containers — that duplication is deliberate and load-bearing.

flowchart TB
    subgraph HOST["🖥️ Host — Ubuntu / Kubuntu"]
        X11["X11 socket
/tmp/.X11-unix"]
        CAM["/dev/video0"]
        GPU["/dev/dri"]
        NET["host network
UDP multicast"]
    end
    subgraph C["Active containers · ROS_DOMAIN_ID=42"]
        HT["hand_tracker"]
        RV["ros_rviz"]
        TS["topic_sniffer"]
    end
    GZ["gazebo_sim
(commented out)"]:::parked
    classDef parked stroke-dasharray: 5 5,opacity:0.55
    CAM --> HT
    GPU --> HT
    GPU --> RV
    X11 --> HT
    X11 --> RV
    NET <--> HT
    NET <--> RV
    NET <--> TS

Hole 1 — the network wall has to come down
#

network_mode: "host"
ipc: host
pid: host

This is the one that breaks stacks silently, so it is worth understanding rather than copying.

From camera coordinates to mechanical radians

contact@mulhamfetna.com (Mulham Fetna) — Tue, 08 Sep 2026 00:00:00 +0000

Twenty-one points in a camera’s coordinate system on one side. A CAD assembly with hard mechanical stops on the other. This is the arithmetic that makes them agree — and the one row where it currently does not.

The vision layer gives you 21 points floating in a normalized coordinate space that has no physical units and a faked depth axis. The mechanical layer gives you fifteen revolute joints, each with a lower and upper bound in radians that came out of a CAD mate.

Nothing connects them. Building that connection is the actual work of this project, and it happens in about forty lines of Python.

flowchart LR
    A["21 landmarks
(x, y, z) normalized"] --> B["Triplet selection
15 × (p₁, p₂, p₃)"]
    B --> C["Two vectors per joint
v₁ = p₁ − p₂ · v₂ = p₃ − p₂"]
    C --> D["Dot product → arccos
θ in radians"]
    D --> E["Normalize
flexion ∈ [0, 1]"]
    E --> F["Lerp onto URDF limits
θ_urdf"]
    F --> G["JointState
names + positions"]

Step 1 — pick three points
#

To measure a joint you need the joint itself and the two bones meeting at it. In landmark terms: the vertex, plus its two neighbours.

Why ROS 2 earns its complexity — and how this graph is wired

contact@mulhamfetna.com (Mulham Fetna) — Tue, 08 Sep 2026 00:00:00 +0000

Every mechatronics engineer hits the wall where ROS 2 feels like an enormous tax just to move a few servos. Here is when that instinct is right, when it stops being right, and what this project’s graph actually looks like.

Writing raw sockets on an ESP32 is cleaner on day one. It is genuinely simpler, genuinely faster to get moving, and for a single microcontroller driving a handful of servos it is often the correct engineering decision.

ROS 2 is not a plug-and-play convenience layer. It is distributed middleware built to solve problems you do not have yet — time-synchronizing asynchronous nodes, standardizing message types across C++ and Python, managing coordinate transform trees that nest six levels deep. Adopting it before you have those problems is pure overhead.

The question is when you cross over. For this project, the crossing point was concrete: the moment a second consumer needed the same hand data. One script drawing on a frame is trivial. One script producing angles, another rendering a kinematic tree, a third logging telemetry, all needing the same data at the same instant without knowing about each other — that is when the framework starts paying rent.

Ros2 on Mulham Fetna

Docker Compose architecture for ROS 2: dependency-only images and mounted code

The big picture #

Repository layout #

From tutorial project to production robot: a roadmap for the tendon hand twin

Where it stands #

Stage 1 · Correctness

Measure the right thing, command the right quantity

GUI, webcam and GPU passthrough for ROS 2 and MuJoCo containers

Windows: pass the X11 socket through #

On Wayland #

How a webcam moves a simulated tendon-driven hand

End to end #

Why the pipe carries flexions, not angles or forces #

ROS 2 across Docker containers: shared network, discovery and Fast DDS shared memory

How ROS 2 nodes find each other #

Why Docker’s default network breaks it #

Run the tendon-driven hand digital twin in 10 minutes

What you need #

Four ways to run it #

The ROS 2 topic contract: one JointState topic, two nodes, and rclpy inside a render loop

Why ROS 2 between vision and physics at all? #

The node graph #

The contract #

Troubleshooting and FAQ: MuJoCo, MediaPipe and ROS 2 in Docker

Camera and windows #

RuntimeError: Failed to open camera at index 0 #

Windows don’t open — cannot connect to X server, could not connect to display #

Black, blank or garbled OpenCV window #

A webcam, some vector geometry, and a hand that moves

The rule that shaped the build #

Containerizing ROS 2 without losing the hardware

Hole 1 — the network wall has to come down #

From camera coordinates to mechanical radians

Step 1 — pick three points #

Why ROS 2 earns its complexity — and how this graph is wired

The big picture
#

Repository layout
#

Where it stands
#

Windows: pass the X11 socket through
#

On Wayland
#

End to end
#

Why the pipe carries flexions, not angles or forces
#

How ROS 2 nodes find each other
#

Why Docker’s default network breaks it
#

What you need
#

Four ways to run it
#

Why ROS 2 between vision and physics at all?
#

The node graph
#

The contract
#

Camera and windows
#

`RuntimeError: Failed to open camera at index 0`
#

Windows don’t open — `cannot connect to X server`, `could not connect to display`
#

Black, blank or garbled OpenCV window
#

The rule that shaped the build
#

Hole 1 — the network wall has to come down
#

Step 1 — pick three points
#