Integrating AI models into Zulip bots using FastAPI - Part 1 - HedgeDoc
180 views
<center> # Integrating AI models into Zulip bots using FastAPI: Part 1 <big> **Migrating the Zulip’s bot server from Flask to FastAPI.** </big> *Written by [Cesar Uribe](https://github.com/curibe) and [Juan David Arias](https://github.com/juanArias8). Originally published 2023-01-31 on the [Monadical blog](https://monadical.com/blog.html).* </center> Robotic Process Automation is a term for when virtual robots (bots) are used within the tech infrastructure of a company to perform automated, digital tasks. Bots are like little computer soldiers that are recruited to perform the repetitive and tedious tasks that we all dread. The appeal of bots is obvious – who wouldn’t want to be like Iron Man, offloading all the boring crap to J.A.R.V.I.S., leaving you free to unlock your true potential as a gadget-building genius? Of course, a fully automated, intelligent, world-saving butler-bot (botler?) like J.A.R.V.I.S isn’t quite within our reach just yet, but we imagine a pared-down corporate J.A.R.V.I.S as bot that can do stuff like: * Provide a succinct summary of what was discussed in a particular stream from a chat platform -- no more reading through 2,300 inane messages to catch up just because you took a couple days off! * Translate content in a simple and integrated way, making communication smoother for multinational teams – this one’s like C3PO, but without the crippling neuroses. * Generate accurate quotes for clients based on the number of available staff and the skills required for a particular project. * Autocompleting or fully writing messages and emails based on what’s needed in a particular context. Sounds awesome, doesn’t it? Well, it’s possible to create a bot that can complete these tasks, if we integrate it with artificial intelligence. Over the next three posts, we’ll show you how to do that by making use of the FastAPI framework for creating bots on the Zulip messaging platform. You’ll start by implementing your own bot in FastAPI. Then, you’ll implement an N-tiers architecture that will make the integration of new features easier. And finally, you’ll use the HuggingFace platform to integrate different AI models to the bot. This post is the first of the series. Here, you’ll learn how to create a Zulip bot following its documentation and tools. Once you understand how to test and use a Zulip bot, you’ll learn how to quickly migrate it from Flask to FastAPI. Let’s begin! ## Setting up an example bot Before creating the bot using FastAPI, let’s test one of the example bots found in [Zulip's repository](https ://github.com/zulip/python-zulip-api.git). First, you’ll need to set up your development environment. According to the [documentation](https://zulip.com/api/running-bots): 1. Add a new bot to your Zulip account by going to your Zulip client app > settings icon > personal settings > Bots (https://zulip.com/help/add-a-bot-or-integration) > click on the “Add new bot” button 2. Select the type of bot (generic, incoming or outgoing webhook). In this case, an outgoing webhook. Feel free to play around with a generic type bot, but be aware that for the example with FastAPI, we’re going to use an outgoing webhook type. You can find more information on the difference between the options [here](https://zulip.com/help/bots-and-integrations#bot-type). 3. Set a name and an email to the new bot. The email doesn’t have to correspond to a real email; it can be fictitious. 4. Set the Endpoint URL field as https://localhost:5000. We’ll change it later. 5. Add an avatar image (optional). ![](https://docs.monadical.com/uploads/a5817479-884c-441f-a552-f6335279f99b.png) 6. After creating the bot, download the zuliprcfile by clicking the download icon under the bot card. This file contains the information about the bot and an API key that will be used as the authentication token between the Zulip client and the bot server. 7. Clone the [bot’s examples repository](https://github.com/zulip/python-zulip-api/tree/main/zulip_bots/zulip_bots/bots). From a terminal, run the command git clone [email protected]:zulip/python-zulip-api.git.This will create a new directory python-zulip-api with a list of example bots that you can refer to, and use as samples to build your own. 8. Move the zuliprc file you downloaded in step 5 to the python-zulip-api directory with the command mv /path/to/zuliprc python-zulip-api. Now let’s move to the python-zulip-api folder with cd python-zulip-api. 9. Install the requirements and dependencies inside the zulip-python-api. To do so, run the command python3 ./tools/provision from a terminal. This will install all dependencies in a virtual environment. Dependencies include the following tools: zulip, zulip_bots and zulip_botserver. The zulip_botserver is a Flask server that’ll allow us to run the server bot and expose it to the clients. 10. Activate the virtual environment. Once the installation of the dependencies is complete, execute the command source ./zulip-api-py3-venv/bin/activate to activate the virtual environment. ## Testing an example bot Now it’s time to run a test example. Let’s start by testing the classic [helloworld example](https://github.com/zulip/python-zulip-api/tree/main/zulip_bots/zulip_bots/bots/helloworld) which which consists of the following code: python from typing import Any, Dict from zulip_bots.lib import BotHandler class HelloWorldHandler: def usage(self) -> str: return """ This is a boilerplate bot that responds to a user query with "beep boop", which is a robot for "Hello World". This bot can be used as a template for other, more sophisticated, bots. """ def handle_message(self, message: Dict[str, Any], bot_handler: BotHandler) -> None: content = "beep boop" # type: str bot_handler.send_reply(message, content) emoji_name = "wave" # type: str bot_handler.react(message, emoji_name) return handler_class = HelloWorldHandler  The bot above receives any message as input and responds by adding a reaction to the sent message and replying with the message beep boop. To run it locally, run the command zulip-run-bot helloworld --config-file zuliprc from a terminal: ![](https://docs.monadical.com/uploads/acaf22b3-cb07-48b5-88a5-3b952991ec22.png) Or interact with it locally without having to create a server, like so: zulip-bot-shell -b zuliprc helloworld ![](https://docs.monadical.com/uploads/8711a72f-b3c7-4994-a280-b48a924e8695.png) We recommend checking out the different bots that the Zulip team offers, in order to familiarize yourself with bot coding and to see what’s already available. You can find them [here](https://github.com/zulip/python-zulip-api/tree/main/zulip_bots/zulip_bots/bots). There are converters, games, translators, search engines, and more. As you can see, the only thing that’s needed to create a Zulip bot is to define a bot handler with the methods usage and handle_message. The remaining code corresponds to the business logic of the functionality you want to add to the bot. To run the bot in a self-hosted server or in any cloud provider, execute the command zulip-botserver -c zuliprc -b helloworld --port 8000 --host 0.0.0.0 ![](https://docs.monadical.com/uploads/1ed3b173-a932-4632-8794-d9d0c082c8cd.png) With the zulip-botserver, you can run more than one bot at the same time using only the botserverrc file. This is important to know in case you need multiple bots running on the same server. If that‘s the case, then you can deploy as many bots as you want on your own server by just using a good architecture in your system. If you’re interested, you can find more information on that [here](https://zulip.com/api/deploying-bots#running-multiple-bots-using-the-zulip-bot-server). This zulip-botserveris the same Zulip file that you downloaded in the beginning. You just need to specify the bot’s name or the bot’s file’s custom path that you want to run, for example, [helloworld] or [/your/bot/file/location/bot-file.py]: bash [helloworld] # or the custom path of your external bot: ~/Documents/my_bot.py [email protected] key=xxxxxkeyxxxxx site=https://zulip.domain.com token=xxxxxtokenxxxxx  Once you’ve run a couple of examples and feel comfortable by using them, you're ready to implement a Zulip bot with FastAPI. ## Setting up a bot with FastAPI So far, we've used the initial Zulip example bot repository and executed one of their bots. The repository code that Zulip provides contains the structure to create and integrate a new bot, as well as several test examples, and a series of libraries that allow for the creation and management of the bots. The libraries also add functionalities for things like local tests, deployments, pull requests, etc. This structure, while useful, is overly complex if what we want is to integrate a simple bot. So, we’ll only maintain some key concepts and redefine the structure in the new project we’re going to create. In the next section, you’ll learn how to write your own Zulip bot server in FastAPI by migrating some Flask files to be FastAPI compatible. The bot we’re going to create is named Delphos, this in reference to the city that housed [The Delphi Oracle](https://blogs.unimelb.edu.au/sciencecommunication/2021/08/15/the-delphi-or/) in Greek mythology, because our goal is for Delphos to be the source of all knowledge within our company. ## Migrating from Flask to FastAPI While it's possible to implement the project from scratch, for ease, we're going to reuse some files from the Flask project and rewrite a couple of functions to be FastAPI compliant. The zulip-botserver tool has two essential files that we’re going to use: * [input_parameters.py](https://github.com/zulip/python-zulip-api/blob/main/zulip_botserver/zulip_botserver/input_parameters.py) is an implementation of [argparse](https://docs.python.org/3/library/argparse.html) which allows the server to receive configuration arguments when running the application. argparse is a Python module that is compatible with both Flask and FastAPI, so it’s not necessary to make changes to this file. * [server.py](https://github.com/zulip/python-zulip-api/blob/main/zulip_botserver/zulip_botserver/server.py) is the entrypoint for the bot application. This file contains the code that creates the application, defines the POST endpoint, runs the application, and loads the system configurations. The file was initially written to run on Flask, but we’re going to change it to be FastAPI compatible. To do that, follow the next steps. ### Step 1: Init the server To start with the FastAPI project, create a new directory and a Python file that will be the entry point to the server. From a terminal, run the following command: shell mkdir delphos &&\ cd delphos &&\ touch server.py &&\ pipenv install fastapi uvicorn &&\ pipenv shell  After opening the delphos directory with your favorite editor, enter the following content inside the main.py file: python import uvicorn from fastapi import FastAPI app = FastAPI() @app.get("/") async def main(): return {"success": True, "message": "Hello Delphos"} if __name__ == "__main__": uvicorn.run("server:app", host="127.0.0.1", port=5000, reload=True)  The above script creates a basic FastAPI server with a GET endpoint that returns a JSON with the message "Hello Delphos". To test that everything works, run the python server.py command from a terminal and go to the url http://127.0.0.1:5000/ in a browser. Once there, you should see the following JSON: {"success":true, "message": "Hello Delphos"}. To start with the migration, first copy the contents of [server.py](https://github.com/zulip/python-zulip-api/blob/main/zulip_botserver/zulip_botserver/server.py) into the server.py file in the main directory of the delphos project. Then copy the contents of [input_parameters.py](https://github.com/zulip/python-zulip-api/blob/main/zulip_botserver/zulip_botserver/input_parameters.py) into src/config/input_parameters.py. ### Step 2: Create the request schemas With the files moved to the right directories, let’s create the [Pydantic Schemas](https://pydantic-docs.helpmanual.io/usage/schema), that are based on the JSON’s content sent from the Zulip client through a POST method. See [here](https://zulip.com/api/outgoing-webhooks) for more info. By using Pydantic to create the schemas, you can make use of the default typing validations that Pydantic provides. To define the schemas, create a new file src/models/schemas.py and then add the following content: python from typing import List, Any, Union from pydantic import BaseModel class Message(BaseModel): id: int sender_id: int content: str recipient_id: int timestamp: int client: str is_me_message: bool sender_full_name: str sender_email: str sender_realm_str: str subject: str = None topic_links: List[Any] = None last_edit_timestamp: int = None edit_history: Any = None reactions: List[Any] = None submessages: List[Any] = None flags: List[str] = None display_recipient: Union[Any, str] = None type: str = None stream_id: int = None avatar_url: str = None content_type: str = None rendered_content: str = None class ZulipEvent(BaseModel): data: str message: Message bot_email: str trigger: str token: str bot_full_name: str  The bot endpoint will always receive an object of type ZulipEvent as the body of the request. ZulipEvent contains the event’s metadata and a field of type Message that describes the message information. For instance, the Message object indicates the content, the time, the message’s detailed information, and the list of Recipients which are representations of Zulip users. ### Step 3: Create the config module Now create a new file src/config/config.py and add the following content: python from types import ModuleType from typing import Any, Dict, Optional from pydantic import BaseModel from zulip_bots.lib import ExternalBotHandler class Settings(BaseModel): bots_lib_modules: Optional[Dict[str, ModuleType]] bot_handlers: Optional[Dict[str, ExternalBotHandler]] message_handlers: Optional[Dict[str, Any]] class Config: arbitrary_types_allowed = True  This settings model will replace the Flask app.config for the FastAPI version, and define the attributes of the server bot configuration. This is a global model that contains a mapping system between the bots and their respective configurations. How it works: * bots_lib_module is a dictionary that maps each of the available bots with the file as a module containing the bot source code. * bots_handler is a dictionary that maps each of the bots with an object of type ExternalBotHandler containing the connection to the Zulip API client and other properties. The connection is established based on the bot credentials exposed in the zuliprc file. * message_handlers is another dictionary that’s responsible for mapping each of the bots with the BotHandler functionality. The BotHandler functionality is then responsible for operating the bot logic and sending responses to the client, this by using the handle_message function that each bot must implement. Once the server starts up, the system loads the bots configuration. To do this, the system iterates over each of the entries found in the zuliprc file and builds a Settings type object. Each of the attributes of the Settings object will have an entry for each bot found in the zuliprc file. shell Settings = ( bots_lib_modules: { "delphos": src/delphos/delphos.py } bot_handlers: { "delphos": ExternalBotHandler = client: ( email="[email protected]", api_key="delphos_api_key", site="https://zulip.site.com", ) ... } message_handlers: { "delphos": DelphosHandler.handle_message() })  The above is a preview of what the Settings object will look like once theDelphos server is initialized. ### Step 4: Import third-party library FastAPI is a framework that works asynchronously, which means that FastAPI needs to run on an asynchronous server engine. You’ll use uvicorn, an Asynchronous Server Gateway Interface (ASGI) for Python. In the server.py file, let’s import the uvicorn library for serving the files, also import the main FastAPI object, which will be used to define the application instance in later steps. ![](https://docs.monadical.com/uploads/26ec986b-3e7c-4c01-9d04-cbf0e77a9bd8.png) Finally, update your project dependencies by installing the required libraries. From a terminal, let’s run the command pipenv install importlib_metadata pydantic werkzeug zulip zulip_bots && pipenv update. ### Step 5: Import the local modules Now, import the classes previously created. These are the Settings model to map the bot with its controllers, and the ZulipEvent schema to represent the body when making a POST request from the Zulip client. ![](https://docs.monadical.com/uploads/512a5825-8037-4472-85cc-7c81e8acd5d5.png) You also need to change the function read_config_section to update the module_name. ### Step 6: Initialize the FastAPI app Change the app instance so that it 's an object of type FastAPI. Additionally, define an initdata variable of type Settings that will serve to store the initial configuration of the bot server as a global state of the app. ![](https://docs.monadical.com/uploads/764f1f24-57bc-4c47-a976-24ecca3fa458.png) ### Step 7: Adapt the path operation to FastAPI Adapt the POST entrypoint so that it’s compatible with FastAPI. Here, the ZulipEvent model is used, which is the representation of the request body. ![](https://docs.monadical.com/uploads/f248b2f6-9613-4ec5-b14f-4ebd5382f921.png) ### Step 8: Update the handle_bot function Change the way the configuration’s attributes are obtained. Instead of using the app.config of Flask, use the values of the initdata dictionary that represents Settings. In the return, the json.dumps() call is eliminated, because FastAPI takes care of the response encoding process internally. ![](https://docs.monadical.com/uploads/434e257d-b321-42e6-8390-ec74127f1525.png) ### Step 9: Change the main function on_startup event FastAPI works by default in an asynchronous and multithreaded way, so it’s necessary that the parameters are initialized in the main function executed at the beginning of the application. For this,use the [on_startup event](https://fastapi.tiangolo.com/advanced/events/#startup-event) offered by FastAPI. ![](https://docs.monadical.com/uploads/e2c8cdc6-0149-4b90-8e19-32983cb225fd.png) ### Step 10: Run the server with uvicorn Finally, make use of uvicorn to run the application: ![](https://docs.monadical.com/uploads/6a5ffcd1-d3d0-4990-819e-7e066083c448.png) Note that some of the parameters of the run command are passed as environment variables arguments when running the application from the console. Up to this point, you’ve managed to migrate the zulip-botserver from Flask to FastAPI changing only what’s necessary. With the content of the server.py file, you can start the server that’ll be executing the bot and receiving requests from a Zulip client. But before that, let’s add a basic functionality to the bot. ### Step 11: Add your bot functionality To run your own functionalities, you need to add commands to the bot. The following is an example of the process, with a command to get a city’s weather specified as a parameter. For example, @delphos weather Medellin. To achieve this, we’ll make use of the public API **www.weatherapi.com**. First, register on its page to obtain an API KEY. Once you have the API KEY, you can make requests to its endpoint. Attach the API KEY and specify the name of the city as a parameter. Something like http://api.weatherapi.com/v1/current.json?key=API_KEY&q=Medellin. Let's see the complete example. Create a new file file src/bots/delphos/delphos.py and add the following content: python from typing import Any, Dict import requests from zulip_bots.lib import BotHandler def get_weather(city_name: str): API_KEY = "37bf4285515446549a4222213231001" url = f"https://api.weatherapi.com/v1/current.json?key={API_KEY}&q={city_name}" return requests.get(url).json() class DelphosHandler: ERROR_MESSAGE = """ Sorry, No command/city found. You need to write the command and city: '@weather weather <city>' or 'weather <city>' in DM """ def usage(self) -> str: return """ This is a weather report system, Please specify the city name to obtain detailed info. """ def handle_message(self, message: Dict[str, Any], bot_handler: BotHandler) -> None: content = message.get("content", None) response = self.ERROR_MESSAGE try: content = content.strip().split()[-2:] command = content[0] city = content[1] except IndexError as e: command = None city = None if command == "weather" and city: response = get_weather(city) bot_handler.send_reply(message, response) return handler_class = DelphosHandler  ### Step 12: Run the bot Download the zuliprc file as zuliprc-delphos again, and put it inside the delphos project folder. Remember this file will be used as the credential when authenticating the bot against Zulip. Also, you need to update the file with the custom path: bash [./src/bots/delphos/delphos.py] [email protected] key=xxxxxkeyxxxxx site=https://zulip.domain.com token=xxxxxtokenxxxxx  Now you can run the application with the following command from a terminal: shell python3 server.py --config-file zuliprc-delphos --hostname="0.0.0.0" --port=8000  Here, the bot(s)’ credential’s file, hostname, port, and name that matches the key of the credentials file are specified. ![](https://docs.monadical.com/uploads/8fb862bc-fa28-42b5-9f2d-b305421ef815.png) Finally, to test the success of the bot’s operation and integration with Zulip, we’ll expose our local machine as a secure server with the help of [ngrok](https://ngrok.com/). Then, we’ll expose the local machine as a secure server. From a new terminal, run the command ngrok http 8000. If successful, you should have an image similar to: ![](https://docs.monadical.com/uploads/cf4d68e1-d569-4af8-a84c-59cc0c15f170.png) Once ngrok has completed the setup, replace the URL in the Endpoint URL field of the bot configuration form with the value returned: ![](https://docs.monadical.com/uploads/3735e588-02b9-43d0-97b2-2ea32df813bb.png) To Deploy your bot in the cloud, deploy the project as a microservice in a cloud provider and run the same command as before. When the app is running, replace the URL in the Endpoint URL field with the domain of your microservice. ### Step 13: Playing with the bot Once the bot is running and integrated with Zulip, test it by writing something like weather medellin to Delphos by DM. ![](https://docs.monadical.com/uploads/08557440-ae34-4f2b-8266-e81c5c35f747.png) If you write an incomplete command, you’ll receive a help message like this: ![](https://docs.monadical.com/uploads/28d228be-6f32-4cb0-9020-8cebd5422044.png) You’ve just run a Zulip bot with a FastAPI server 🎉! You can also find all the source code at https://github.com/juanArias8/zulip-delphos-bot. ### Conclusions Instead of creating a bot server from scratch, we’ve adapted Zulip’s base repository from Flask to FastAPI. At first, you may have questioned the adaptation of the bot. After reading this blog post, we hope you’ve learned how FastAPI offers some cool tools that make the developer experience easier. This is because of its asynchronous process handling, automatic integration with OpenAPI, Swagger and Redoc for documentation, strong typing and automatic validations with pydantic. With this adaptation, you can use the FastAPI server to run your bots. Finally, we’ve implemented a simple project structure without going into detail. In the next post, you’ll learn how to structure your project with a more elaborate architecture. Once you understand the architecture, you’ll be able to quickly integrate new features into your bot and easily scale it. ### References * https://www.youtube.com/watch?v=ZEyKxsWjwNM * https://github.com/zhanymkanov/fastapi-best-practices * https://github.com/dev-courses/fastapi-backend * https://zulip.com/api/running-bots * https://fastapi.tiangolo.com * https://www.cosmicpython.com/ * https://testdriven.io/blog/moving-from-flask-to-fastapi/

Recent posts: