Feature Request
We can improve the AI requests with a few steps:
- Translation seems to be an easy task, we can use
meta-llama/Llama-3.1-8B-Instruct, the request will be faster
- add a
maxToken to the call API, to do so we have to calculate the amount of token in our request, see https://github.com/belladoreai/llama-tokenizer-js or tiktoken (python)
- we can improve the frontend message errors by displaying what is the real problem ?
- We can stream the response, by streaming the response we will be able to display content the the user gradually.
Feature Request
We can improve the AI requests with a few steps:
meta-llama/Llama-3.1-8B-Instruct, the request will be fastermaxTokento the call API, to do so we have to calculate the amount of token in our request, see https://github.com/belladoreai/llama-tokenizer-js or tiktoken (python)