Do you need a multiagent setup or just an LLM-calling tool?

Do you need a multiagent setup or just an LLM-calling tool?

Thursday, August 29, 2024

Is your LLM-related task too complex to fit nicely into a single prompt (almost all real-world tasks are)? Is it not linear enough to implement as a simple pipeline (for example, does it need a validate-retry loop, which many real-world tasks do)? Does this mean you have to go multi-agent?

This may be the case, but many use cases can be solved with a simpler setup. Namely, you could have a single agent with several tools, some of which would also call an LLM as part of their logic. Each tool would have its own simple, granular prompt, making it much more likely that the LLM will follow it faithfully.

In the simplest case, all the tool might need to do is call the LLM. For example, the agent might be writing a story, and the tool could be the critic, making sure the story doesn't sound too much like AI has written it :). The prompt of the tool would then contain all the telltale patterns of AI writing to guard against, and the agent's prompt could focus on the story itself. In such a case, you could use motleycrew's LLM tool directly, calling its methods to_llamaindex_tool() , to_langchain_tool() , to_autogen_tool() , and to_crewai_tool() to use it in your favorite framework.

In a more complicated case, the tool could be a database querying specialist, whose LLM prompt would contain all the information about the database schema, and the tool would first translate a text query from the agent into SQL and then evaluate that SQL against the database. However, the SQL might be malformed or unable to be executed for some other reason. To handle that, you'd want a retry loop - and the simplest way to do it is to make that tool an agent (that would still take the natural language query as an input), with the raw SQLQuery tool given to it. In motleycrew, you could simply pass that agent as a tool to the primary agent; but also in other frameworks, only a couple lines of code are necessary to wrap an agent as a tool.

So as a general rule, when should you use an LLMTool, and when a full agent-as-a-tool? You should use an agent-as-a-tool in two cases: firstly, when it needs a validate-retry loop, like in the example above; secondly, when it needs to make a choice inside itself that needs an LLM, for example which data source to query. 

For all simpler cases, an LLM-calling tool will do the trick!

Search