How Can AI Learn with Fewer Tokens?

AI agents often hit a wall when it comes to multi-step tasks - it's not that they don't know what to do, it's just that they can't seem to apply that knowledge in a reliable way. That's where systems like ACE and ALTK-Evolve come in, allowing agents to learn from their own experiences and essentially creating a kind of memory for them. What's interesting is that both systems reject the idea of boiling an agent's lessons down to a simple summary, instead opting for a detailed, itemized record of everything the agent has been through.

The main difference between ACE and ALTK-Evolve lies in how they build and deliver these memories to the agent. ACE takes a more comprehensive approach, generating a kind of playbook through a process of reflection and curation, and then delivering the whole thing to the model every step of the way. ALTK-Evolve, on the other hand, clusters similar lessons together, merges them, and extracts guidelines with causal attribution - and only delivers a select few to the model when it's actually making a decision.

This difference in approach has a pretty significant impact on efficiency - on AppWorld, for example, ALTK-Evolve manages to achieve similar accuracy to ACE using about 40% of the tokens. And when you're working with weaker models, ALTK-Evolve actually edges out ACE in terms of accuracy, all while using about one-seventh the tokens. The secret to ALTK-Evolve's efficiency is that it only serves up the context that's actually needed, rather than dumping the entire playbook on the model every step of the way - which has some big implications for developing more efficient AI systems, and making AI agents more reliable and cost-effective in real-world scenarios.

Source: Hugging Face