Inducing Task Models from Computer-Use Traces
Yucheng Jiang, Zora Zhiruo Wang, Ruishi Chen, Diyi Yang
arXiv:2608.20319v1Today’s standout is Inducing Task Models from Computer-Use Traces. The problem it tackles is surprisingly practical: as AI agents start operating real software, we need ways to understand what people actually do on computers, not just summarize screenshots or click logs. But real work is messy, with overlapping tasks, interruptions, and hidden goals, so simple step-by-step tracing is not enough. The core idea is to learn a structured task model directly from naturalistic traces: first discover the latent tasks inside an interleaved session, then build a hierarchy of goals and a procedure model that explains how each task unfolds. That makes the result more than a transcript; it becomes something symbolic, auditable, and reusable. Why does this matter? Because it could help train better computer-use agents, make their behavior easier to inspect, and let organizations reuse human workflows instead of starting from scratch. In short, it turns raw desktop activity into actionable knowledge.
Also spotted that day
Previous daily papers