CrewAI has incorporated the Patronus evaluation system into its platform. The documentation on the CrewAI docs page is very sparse. Has anyone used this platform in CrewAI and if so, do you have a simple example of how to run this platform with your crew and get the agents to automatically evaluate their outputs using this new tool?
Thx
Hey @Moe have you found any answer to this?
Hey Evan, I got the following link from Patronus:
maybe this can help.
The Patronus docs are outdated - refer to Client Challenge or their GitHub. Once I saw this, using it was a breeze.
Do you have an example implementation using CrewAI agents with tasks? I still don’t quite get how to implement it all properly.
For example in the crewai docs they show the example there of where the Patronus tool is being used. However how can the evaluation pass_ score and explanation etc. be extracted once the entire thing has been run in the code, rather than having to log into the platform to get all that.
also is there a list of evaluators and their associated criteria anywhere. the documentation on the Patronus site is very poor!