OLLAMA and max_tokens etc

CrewAI Community Support
, ,
1

I got a basic news report crew running after using this tutorial: https://www.youtube.com/watch?v=ONKOXwucLvE and it seems to be working somewhat but with some major issues.

Token limit of 2048
It looks like there is a max_token limit somewhere as I get this message in the server.log:

time=2024-12-29T12:00:38.943+01:00 level=WARN source=runner.go:129 msg="truncating input prompt" limit=2048 prompt=2527 keep=4 new=2048

I tried this: ollama_llm = LLM(model='ollama/granite3.1-dense',base_url='http://localhost:11434', max_tokens=8192)

and this:

	@agent
	def file_writer(self) -> Agent:
		return Agent(
			config=self.agents_config['file_writer'],
			 tools=[FileWriterTool()], 
			verbose=True,
      		max_tokens=8192,  
   			llm=self.ollama_llm # ollama_llm,  # tilføjet for Olama,
		)

But i still get that same token limit of 2048, where else do I need to set it… perhaps it’s some basic OLLAMA limit but where do I set it?

The 2048 tolken limit seems to limit the amount of output a lot, links are cut off and

FileWriterTool
As you can see I also use the FileWriterTool, for some reason it does not fully work. Often it will create the file but with some short random text other times it will write a report, but it looks very cut off. Also there is a issue regarding the directory often the FileWriterTool will put the file in a directory called ‘None’ and always requires an existing sub directory (in my case one called ‘news’).

This is the agent and the task:

file_writer:
  role: >
File Writer
 goal: >
Write the report from the previous agents to a file using FileWriterTool
   backstory: >
You'er a skilled file writer, an expert in writing unmodified reports to a file.

file_writer_task:
  description: >
    Using FileWriterTool write the output to a file '{date}_CAPnews.md' in the 'news' directory 
  expected_output: >
    Write the output to a new file named '{date}_CAPnews.md' in the 'news' directory 
  agent: file_writer
2

… with regards to the FileWriterTool this is what I get - it looks as-if the agent do not realize what to output…:

# Agent: File Writer
## Thought: Thought:
To write the content to the specified file, I need to use the File Writer Tool with the appropriate arguments. The filename is '2024-12-29_13-53-17_CAPnews.md', the directory is 'news', and there's no need for overwriting as we don't know if a file with that name already exists. The content to be written is the context provided about the Common Agricultural Policy (CAP) of the European Union.
## Using tool: File Writer Tool
## Tool Input:
"{\"filename\": \"2024-12-29_13-53-17_CAPnews.md\", \"directory\": \"news\", \"overwrite\": \"False\", \"content\": \"context\"}"
## Tool Output:
Content successfully written to news\2024-12-29_13-53-17_CAPnews.md
3

If i’m not mistaken, all the ollama models has a 2048 limit by default when you download a model. If you want to change that, change it with ollama create and set num_ctx to a higher value.