LangWBC:
Language-directed Humanoid Whole Body Control via End-to-end Learning

Yiyang Shao, Xiaoyu Huang, Bike Zhang, Qiayuan Liao, Yuman Gao,
Yufeng Chi, Zhongyu Li, Sophia Shao, Koushil Sreenath

TLDR

An end-to-end method enabling humanoid robots to interpret and execute diverse whole-body motions directly from natural language commands.

Diverse Motions

A person is moving forward briskly

A person walking backwards slowly

A man waves his right hand

A person shuffles from the left to the right, then shuffles back to the left

A person is standing and making a strumming gesture

A person claps his hands

A man pats his left hand with his right hand

A person raised both their hands

Smooth Transitions

Wave -> Run -> Wave (Side View)

Wave -> Run -> Wave (Front View)

Walk -> Run -> Wave

Command Loop Over: A person walks forward then turns around and walks back

Adapt to Variations

A person walks forward

A person moves ahead

A person moves forward

Generating from Interpolation

Interpolation of A person is moving forward briskly and A person shuffles from the left to the right, then shuffles back to the left

Side View

Front View

Integration with LLM

LLM translating There is a friend 3 meters in front of you, what should you do? into a sequence of text commands

Robustness to Disturbances

Recover from Kicking

Recover from Pushing