Research question

Which parts of a robot's observations need to cross a wireless link for a task to succeed, and how should they be represented when bandwidth is limited and the link is unreliable?

Approach under investigation

We study the path from what a robot observes to what a model or controller at the other end of a link needs in order to act. Rather than treating every pixel or measurement as equally important, we ask which information the task depends on.

  • Visual and semantic representations that keep task-relevant content and leave out detail the task does not use.
  • Compression and coding of these representations for links with limited bandwidth.
  • Link impairments — delay, loss and fluctuating capacity — and how they carry through to the receiving end.
  • Receiver-side inference and its relationship to task performance, so that task feedback can inform what is encoded and sent.

Diagrams and media

Concept: observation, representation and encoding, a constrained wireless link, and receiver-side inference, with task feedback informing what is encoded.

  1. Observation Camera views and robot state
  2. Representation & encoding Keep what the task needs
  3. Receiver-side inference Model or controller acts on what arrives
  4. Task performance

↺ Task feedback informs what is encoded and sent

Concept illustration

Task-aware communication pipeline

Concept diagram of the approach under investigation, not an experimental result.