Overview

JAZ is a stripped-down agent framework developed by researchers at MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL). In benchmark tests, it outperformed Letta and ACE on recall-heavy tasks.

Design and Architecture

At its core, JAZ relies on a single LLM-based primitive called invoke. The agent's history and prompt are exposed as variables inside a code environment, allowing the model to write executable code to inspect, slice, or manipulate its own history directly.

Benchmark Results

The researchers tested JAZ on two fronts: long-term recall and self-improvement. On the StuLife benchmark, JAZ scored 70% and Letta scored 62%. On the AppWorld benchmark, JAZ scored 74%, finishing 4 percentage points ahead of ACE.

Cost Comparison

JAZ reportedly got to the top spot at approximately half the cost of Letta. The cost side makes the result more interesting, as it highlights the potential for significant cost savings in agent development.

Implications for Agent Builders

The most practical takeaway for developers building AI agents is the cost savings. Getting a higher score at roughly half the price is a result that gets attention in engineering budget meetings. However, there are also engineering questions to consider, such as sandboxing, error handling, and predictability when the agent is effectively programming its own memory access.

Availability

The framework and its evaluation code have been published on GitHub, allowing other researchers to poke at the results themselves.