Kaiming Yang 杨铠铭

Dream.exe

Can Video Generation Models Dream Executable Robot Manipulation?

Rui Zhao*, Kaiming Yang*, Jifeng Zhu, Siyang Chen, Ziqi Wang, Weijia Wu, Kevin Qinghong Lin, Heng Wang, Mike Zheng Shou

* Equal contribution

Can a convincing robot video become an executable action? Dream.exe recovers robot trajectories from generated videos and tests them in a physics simulator, assessing physical plausibility and task completion. Across 101 manipulation tasks and 8 models, it asks what visual quality alone cannot tell us: can an imagined future actually be carried out?

WorldGUI

An Interactive Benchmark for Desktop GUI Automation from Any Starting Point

Henry Hengyuan Zhao, Kaiming Yang, Wendi Yu, Difei Gao, Mike Zheng Shou

Real software tasks often begin halfway through a workflow, rather than on a clean starting screen. WorldGUI tests whether GUI agents can recognize progress, adapt their plans and continue from diverse initial states across 10 applications and 611 task instances. Its Plan-Act-Critic framework integrates reflection into planning and execution so agents can detect errors and adjust their next steps.

CriticGUI

Evaluating Multimodal LLMs as GUI Critic Models

Can a multimodal LLM serve as a reliable critic for GUI agents? CriticGUI evaluates whether models can understand interface states and transitions, judge whether an action fulfills the current instruction, and explain why it succeeds or fails. Using hierarchical instructions, action code, before-and-after screenshots and action videos, the benchmark assesses critic judgments and explanations independently of action generation, focusing on state-aware, instruction-grounded multimodal understanding.

nl2spec++

From language to temporal reasoning and formal specifications

Temporal requirements are often scattered across sentences or left implicit. nl2spec++ links relations through shared entities, reasons about hidden ordering and dependencies, and translates them into parseable linear temporal logic (LTL). It makes constraints in natural-language descriptions explicit and ready for subsequent verification.

AdaHC

Accelerating Multi-Token Prediction with Adaptive Head Chunking with Pipeline Parallelism

Yan Wang, Chang Si, Kaiming Yang, Zhipeng Zhang, Weijian Liu, Man Yuan, Mingzhen Li, Yong Li, Weile Jia

AdaHC addresses the computational overhead of multi-token prediction in LLM training, combining adaptive head chunking with pipeline parallelism to make additional prediction capacity more efficient.

Libra

Taming Attention Workload Skew in Long-Context LLM Training with Bounded Sequence Pool

Yan Wang, Xiulong Yuan, Kaiming Yang, et al.

Long-context training can distribute attention computation unevenly across devices. Libra uses a bounded sequence pool to reorganize workloads, addressing this imbalance so distributed training can use compute resources more effectively.