Research question

How can a robot link an instruction to the right object on a table, plan a grasp for it, and use feedback to adjust while it acts?

Approach under investigation

This work develops research prototypes for tabletop manipulation on the Unitree G1 with a Dex3 dexterous hand. The prototypes are under active development.

  • Object recognition in tabletop scenes.
  • Language targets — resolving which object an instruction refers to.
  • Grasp planning for a dexterous hand.
  • Visual and execution feedback used to adjust the motion during the grasp.

Diagrams and media

Concept: object perception and a language instruction identify a target, followed by grasp planning and execution with visual and execution feedback.

    • Object perception Tabletop scene
    • Language instruction Which object, what to do
  1. Target grounding Link the words to an object
  2. Grasp planning Dexterous hand
  3. Execution Feedback control

↺ Visual and execution feedback

Concept illustration

Language-guided closed-loop manipulation

Concept diagram of the approach under investigation, not an experimental result.

Physical robot Unitree G1

G1 tabletop grasping test

A tabletop grasping test on Unitree G1 with a Dex3 dexterous hand, recorded in the lab. Silent clip.

What this shows: G1 reaches for a red pepper on the table, lifts it and moves it while held by a safety gantry.

Investigator: Haonan Zhou

Recorded 17 November 2026

Related projects

All research

Ongoing research Task-Grounded Robot Learning

Progress-Aware Long-Horizon Robot Tasks

We investigate how robots can track task progress, maintain useful context and select subsequent actions during extended manipulation tasks.

Project investigator: Haonan Zhou