Trace-ing the Path to Self-adapting AI Agents - Sep 30, 2024

​

Speakers: Ching-An Cheng

​

Biography of the speakers:

Ching-An Cheng is a Senior Researcher in MSR AI Frontiers. He received PhD in Robotics in 2020 from Georgia Tech, where he was advised by Byron Boots at Institute for Robotics and Intelligent Machines. He’s a practical theoretician who is interested in developing foundations for designing principled algorithms that can tackle real-world challenges. Ching-An’s research studies structural properties in sequential decision making problems, especially in robotics, and aims to improve the learning efficiency of autonomous agents. His recent works focus on developing agents that can learn from general feedback, which unifies Learning from Language Feedback (LLF), reinforcement learning (RL), and imitation learning (IL). Ching-An’s research has received several awards, including Outstanding Paper Award, Runner-Up (ICML 2022) and Best Paper (AISTATS 2018).

​

Abstract:

What is Trace? Trace is a new AutoDiff-like framework for training AI workflows end-to-end with general feedback (like numerical rewards or losses, natural language text, compiler errors, etc.). Trace generalizes the back-propagation algorithm by capturing and propagating an AI workflow execution trace and applies LLM-based optimization to improve the workflow’s performance.Trace is implemented as a PyTorch-like Python library and is compatible with any Python workflow. Users write Python code directly and can use Trace primitives to optimize certain parts (like codes, prompts, etc.), just like training neural networks! In this talk, I will discuss insights behind designing Trace and showcase what Trace can do in training AI agents.