Research question

During an extended manipulation task, how can a robot judge where it is in the task, decide which context to keep, and choose what to do next — including after a step has not gone to plan?

Approach under investigation

Many useful tasks take more than a single action. Each step changes the scene, and the robot needs to know whether the step worked before it moves on. We study the loop between observing, judging progress and acting.

  • Observation and memory — which observations and past events to retain as task context.
  • Task-stage estimation — judging which stage of a task has been reached.
  • Outcome verification — checking whether an action achieved its intended effect.
  • Next actions and recovery — selecting subsequent actions, and studying how a robot can recover after a failed step.

Diagrams and media

Concept loop: observation and memory, task context, action selection and outcome verification, then re-observation or recovery.

  1. Observation & memory What the robot sees and keeps
  2. Task context Which stage has been reached
  3. Action selection What to do next
  4. Outcome verification Did the step work?

↺ Re-observe, then continue or recover

Concept illustration

Progress-aware task loop

Concept diagram of the approach under investigation, not an experimental result.

Related projects

All research

Ongoing research Task-Grounded Robot Learning

Language-Guided Closed-Loop Manipulation

We develop research prototypes that connect object perception, language instructions, grasp planning and feedback control for tabletop manipulation.

Project investigator: Haonan Zhou