Hi, I’m spending around $0.8 per session - any patterns that I can look for optimising this?
This is actually something I’m interested in researching - What can we learn from current management theory (derived and applied to humans) that can make AI agents more efficient (less expensive)?
One way to reduce the context history that I’ve tried is to reduce knowledge carry-over to maybe a table or single paragraph. The concept is from several management theorists, such as Henri Fayol (employee tenure over task iteration - the need for history) and Herbert Simon (bounded rationality - people don’t utilize full history). The bottom line is to reduce the number of tokens with small memory aids.
That’s one idea. Would love to hear what you got to work. Your set up and your fixes.
Of course, there are other ideas - limit the number of retries, if you allow rework. And use cheaper models, and so on.
There could be quite a few causes, such as token bloat, or retry loops, etc - but how do we detect and identify these in an already running Agent, and how to fix them.