my notes from Shengran Hu’s appearance on TWIML

mindmap transcript research paper

• they embrace an open-ended design patter to the agentic system, to give them room to discover freely. in so doing they find interesting complex solutions even starting with a simple initial scenario. and interesting design patterns emerge; sequences of “stepping stones” to reach greater complexity

  • they would observe that sub-optimal agents would be created (one of the stepping stones) but then it would be leveraged later in a particular system design that yielded better results then initially achieved

• he speaks about the tradeoff between exploration and exploitation (going back to the archive to try to improve). in their case, they let the meta-agent choose which to do for each iteration

• in the reflection stage, they prompt the meta-agent, is this newly discovered agent you’ve designed novel? is it interesting? and then they observe it reason why. this is how they encourage it.

• the boundaries of what is considered a single agent in a multi-agent system can be interpreted in multiple ways. in one case, you may consider that agent to still have further modules within: decision making, memory, etc.

• with diverse LLMs in your system, it’s less likely they all make the same mistake

The components to automated design of an agentic system (ADAS)

• the search space. a code space. since this is where the agents live
• search algorithm. ..give a meta-agent control that has knowledge of the design of a particular model and the system itself. a self referential method
• evaluation function - how can we judge them good or bad? how do we improve? test the performance and get a scalar reward (eg tasks, games)

building blocks of the system

• prompt, control flow, interactions between models, tool use, memory, RAG

For an AI generating algorithm, you could say there are three main parts: