Domino Hands

Overview

Domino Hands is a fully autonomous robot system that detects, picks up, and precisely places dominoes into configurable patterns using a Franka Emika robot arm. The project was developed as a final project at Northwestern University's MSR program by Gregory Aiosa, Daniel Augustin, Michael Jenz, and Chenyu Zhu.

Key Contributions

• Autonomous Perception: Developed an RGB-D pose estimation pipeline for small-scale object detection.
• Collision-Aware Planning: Implemented a staged manipulation strategy to navigate tight spatial constraints.
• Force-Feedback Control: Integrated joint effort monitoring to achieve robust placement on non-uniform surfaces.
• ROS 2 Ecosystem: Orchestrated a multi-node system involving MoveIt, RealSense, and custom CV nodes.

System Architecture

1. Domino Vision Algorithm

The vision pipeline identifies both the position and orientation of dominoes to generate actionable TF frames. Position is determined using color filtering and depth data to compute 3D coordinates. Orientation uses OpenCV bounding boxes to extract the yaw (z-axis rotation) and convert it to a quaternion.

2. Domino Movement Algorithm

To avoid collisions between the bulky Franka Emika gripper and dense domino patterns, a three-stage manipulation pipeline was used:

1. 1Initial Pickup: Retrieval from the detected starting pose.
2. 2Staging (Reorientation): The robot stands the domino up vertically. This top-down grip is essential for high-density placement.
3. 3Final Placement: Precise insertion into the goal pattern (Straight, Circle, or Squiggle).

3. Force Sensing Placement

To compensate for uneven workspace surfaces, a force-feedback loop monitors Joint 2 effort on the Franka. The robot moves in 1mm increments — if the effort threshold is exceeded, contact is confirmed. Collision objects in the MoveIt planning scene are dynamically added/removed during this phase to prevent planning errors.

Quickstart

# 1. Launch the system
ros2 launch place_dominoes domino.launch.xml open_cv:=true demo:=false pattern:=straight_line.yaml

# 2. Trigger placement
ros2 service call /place_dominoes std_srvs/srv/Empty

# 3. Topple
ros2 service call /topple_dominoes std_srvs/srv/Empty