Custom LLM bots
Virtual agents are currently in open beta. Bot active time is limited to 24 hours per month. Please contact us at [email protected] if you need more information or need a quota raise.
What are Custom LLM bots?
Custom LLM bots are powered by self-hosted LLM engines (like Ollama) or external LLM services. They can provide answers to questions, engage in conversations, and provide information based on the capabilities of the underlying model. Depending on the model you use, they can speak in multiple languages and can even be used as translators.
Configuring your bot
Your Custom LLM bot must be compatible with the OpenAI client. This means that the API endpoint must adhere to the specifications required by the OpenAI client. If you are using a self-hosted LLM engine, you must ensure that it is compatible with the OpenAI client. Ollama is an example of an external LLM service that is compatible with the OpenAI client. see Ollama OpenAi compatibility for more information.
Connect to your WorkAdventure room.
Click on the menu icon (the coffee cup), and then click on "Bots" menu item:
Click on the "Create new bot" button:
You can give the bot any name and choose any avatar you like (please note that since your bot has a "Bot" tag, it can select any Woka avatar that is reserved for bots).
Select "Custom LLM" as the bot type.
A Custom LLM bot requires specific configurations:
- The API URL: This is the URL of the API endpoint compatible with an OpenAI client. It is crucial to ensure this endpoint adheres to the required specifications.
- The Model: Specify the model to be used. This should correspond to the model available at the provided API URL.
- The API Key (optional): If the API used requires an API key for authentication, you can provide it here. This parameter is optional and should be left empty if the API does not require it.
- The chat prompt is the prompt that will be used to generate the bot's answers. Use this prompt to explain to the bot who it is, what it is supposed to do, and how it should behave.
Done? Save the bot.
You can now start the bot using the "Start" button.
When you start the bot, a headless browser containing the bot starts on the WorkAdventure servers. It will take between 1 and 2 minutes for the bot to be fully operational. If the servers need to scale, it could take as long as 10 minutes.
Your bot capabilities
Any Custom LLM bot has the following capabilities:
Discussing with users
The bot can engage in conversations with users. It can answer questions, provide information, and discuss various topics.
Limitations
Currently, most models do not support function calling the way OpenAI does. Consequently, the bot cannot move, read the map, or even send an email. These functionalities are dependent on advanced function calling capabilities which are not available in most Custom LLM models at this time.
Fill as much information as possible in your profile to help the bot provide more accurate answers. The 'information' field is particularly useful for the bot.