Inducing Task Models from Computer-Use Traces
Yucheng Jiang, Zora Zhiruo Wang, Ruishi Chen, Diyi Yang
arXiv:2608.20319v1A lot of useful work happens on computers in a messy, interleaved way: people answer email, edit documents, browse files, and switch tasks constantly. That makes it hard for AI systems to learn what a task really is, or for organizations to audit how work gets done, because all they see are low-level traces like screenshots and mouse clicks. This paper tackles that problem with Task Model Induction, a method that turns passive computer-use traces into structured task models. The key idea is to first discover the hidden tasks inside an unconstrained trace, even when multiple goals are happening at once, and then build two kinds of models for each task: a hierarchical description of the goals, and a procedure model of the steps and control flow. The result is more than a summary; it is reusable, symbolic knowledge about work. That matters for training computer-use agents, improving task performance, and making AI behavior easier to inspect and trust.
Also spotted that day
Previous daily papers