<center> Drone Web Platform === <big> **A Platform for Operating Drones Through the Internet** </big> *Written by [Jose](https://twitter.com/yojosebenitez). Originally published 2021-02-22 on the [Monadical blog](https://monadical.com/blog.html).* </center> Have you ever piloted a quadcopter? How about a [rover](https://ardupilot.org/rover/)? Or an [ROV](https://en.wikipedia.org/wiki/Remotely_operated_underwater_vehicle)? You might have seen people using drones[^1] to take photos and videos of nature, or to race remote controlled cars, but they can also be used for other tasks, like agriculture, civil inspection, exploring nature, rescue, or security. In fact, once you've mastered flying/driving a drone, the only limits are your budget and your imagination! Some time ago, a friend came to me with a project: a research group wanted to take measurements of the air quality in the city. Our first idea was to use an airship, but handling helium turned out to be very complicated. So, we decided to take the measurements with drones. Normally only the pilot of the drone has access to flight data in real time (pose, path, mission, etc…)--data is usually downloaded after the flight. For this project, though, I needed to set things up so that the whole team could see the drone mission and measurements live.  The research project was cut short, but I kept thinking about the idea. Wouldn’t it be cool if, as well as the flight data, the whole operation was accessible from a web browser, kind of like a game? There would need to be a map, controls, and a video view. Users could check the system status, create the mission, and fly their drone--all from their browser! Well, last year I finally decided to give the project a shot. In this post, I’ll explain some of the challenges of the build, what I’ve come up with, and why (I hope!) the platform will be useful. ## The main idea Drones are mainly composed of two things: the vehicle and a remote controller or station. So, first of all, I needed to be able to interact with the drone from a server as if the server were the control station. This meant that the drone had to be able to connect to the internet! Nice, two problems at once: connect to the internet and create a web control station. I took a drone (a quadcopter to be precise), connected it to a mini PC (a Raspberry Pi), connected my mini PC to the internet through a 4G dongle, and started building the most freaking awesome web platform to manage drones 🥳. What about the control station? Let me introduce you to the wonderful world of Mavlink! ✨ Mavlink is a message protocol, which means it’s a standard set of commands, messages and definitions that drones use to communicate with the external world. For example, you can send a message to the drone that says “MOVE FORWARD 10 METERS”, and the drone will understand. In the same way, a drone can say, “I'M AT COORDINATES X,Y” and you will know where the drone is. This communication can occur through many channels: WiFi, 4G, Serial USB, etc. So, voilà! I had a way to connect the drone to the internet, and the tools to control the drone. Now it was just a matter of programming skills to bring those things together and create a usable web-based platform.  ## From Mavlink to the internet Drones that use Mavlink have so many capabilities--the number of messages available is huge[^2]--so I just started with the basics: taking off, landing, moving to points, and tracking the drone on a map. This is where I started communicating with the drone from my mini PC, and guess what? It worked! I could command the drone to take off and move around, and could see where it was, where it was heading, etc. The next step was to connect a 4G dongle to the mini PC to establish a communication channel to a server. This was straightforward, since the problem has already been solved--4G is what allows mobile networks to provide internet to devices (using a SIM card, like with your cell phone). There’s a small piece of software written in [Python](https://www.python.org/) that runs on the mini PC which simply serves as a bridge in the communication channel. There is no magic in it--it’s just a script to pass messages from the drone’s onboard controller to the server and vice versa. This is all done using [websockets](https://en.wikipedia.org/wiki/WebSocket). All that was left to build was the server and client software... ## Cloud control station Web applications use [client-server architecture](https://cio-wiki.org/wiki/Client_Server_Architecture), composed mainly of two parts called the 'backend' and the 'frontend'.[^3] For this project, I wrote the backend in Python using a framework called [Django](https://www.djangoproject.com/), and the frontend in [Javascript](https://www.javascript.com/) using a library called [‘React’](https://reactjs.org/)[^4]. It’s really important to remember that the communication here happens through websockets, so once the connection is established, the mini PC, server and client can communicate in real time. The backend handles two major endpoints, one for drones and one for users (frontend/client). After the whole communication channel was up--i.e, the drone talks to the mini PC, the mini PC talks to the server, and the client talks to the server--it was time to start sending real commands and messages to control, and to manage the drone from a web browser. To give you an example of how this works, the drone sends a Mavlink message containing GPS coordinates to the mini PC, the mini PC sends a GPS message to the server using websockets, the server conveys the message to the client, and the frontend renders a map with an icon at the received coordinates.  ## The magic land of React With the communication infrastructure built (at least the basics), the next step was to create a beautiful graphical user interface. I wanted this to look like a game interface, with panels, buttons, a main view, controls, info sections, and indicators. I made everything using React components. [ReactJS](https://reactjs.org/) is a tool created by Facebook to build user interfaces--with ReactJS, it was just a matter of doing some frontend design to place the components and interconnect them, including the websocket handler and the action of each button/click. Each [React component](https://reactjs.org/docs/react-component.html) has two main elements: [props and state](https://flaviocopes.com/react-state-vs-props/). These control the life and behavior of every component, allowing it to interact with Mavlink messages. GUI quickly became complex, so another cool tool arose--say hi to [React Redux](https://react-redux.js.org/). Without going into too much detail, basically Redux facilitates the interaction between components. This way, I can send a message through the websocket from any component and vice versa, and the interested components are notified and are passed the message data (and I don’t have to micro-manage those interactions). ## What about measured data? Remember the air quality project? We wanted to have on-screen readings of the sensors mounted on the drone. Mavlink’s set of messages has the ability to define, compile and include custom messages for custom purposes like this one! If you want to read more about it, check out [this guide](https://mavlink.io/en/mavgen_python/). After adding the custom messages, you can tweak the mini PC script a little bit to grab the data from the extra sensors or actuators on board, and send that data to the client using custom Mavlink messages. ## How does it look so far?  The image above shows the current interface. The base is a map where the position of the drone is represented by an icon, the blue line indicates the programmed route to follow, and the red line is the traveled path. On the left is a pane with some buttons for things like takeoff, land, and return to launch. There are also some information sections with telemetry data. This is what a user would experience from a web browser while piloting their drone. One important component that is still a work in progress is the video streaming pane. The idea is to have it floating around, with the controls to move the gimbal, take pictures and/or record video. In the long-term future, I also plan to add some AI to it. Eventually, I want to create an authentication system, link drones to users, build flight rooms and scale the infrastructure to handle multiple connections. The platform will have a flight log, a dashboard, and a mission creator/editor. ## What’s next? I think there are some really exciting applications for this project... **Remote tourism** - There's been an increase in online tourism recently, as travel restrictions persist and people are missing travel and culture. We can take a tour of the [Louvre](https://www.louvre.fr/en/visites-en-ligne), the [British Museum](https://blog.britishmuseum.org/how-to-explore-the-british-museum-from-home/), and [MoMA](https://www.moma.org/calendar/groups/58) from our living rooms. But these are all compilations of pictures taken inside the buildings in advance. Imagine being able to get a sweeping view of rivers, forests, lakes, deserts, canyons and beaches live with a drone?  *The Egyptian Sculpture Gallery* During the pandemic, we’ve experienced what it's like to have our movement restricted and this has sparked a welcome increase in accessibility. With a web platform like this, everyone could visit the world from their phones or laptops. It wouldn’t even be necessary to own a drone: people could rent drones housed as sites of interest and explore places they’d otherwise never get a chance to see. **Scheduled flights** - Drones are already being used for monitoring purposes, e.g., you can fly one around a farm to monitor the growing process of crops. However, you have to be there in person to do this. Having a docking station in place would allow you to remotely schedule flight routines and record (or livestream) what you have programmed the drone to see. With AI integrations, it could even identify intruders, sound alarms, or count trees or animals. This technology could also be useful on solar farms, where scheduled flights with AI integrations could inspect and detect faulty solar panels. **Special payloads** - As I mentioned above, this platform was born from a data collection project where we mounted air quality sensors on a drone and monitored the results live. The platform can interact with a number of other payloads too, including cameras and any type of sensor or actuator that can be carried onboard. This could be useful for detecting forest fires with thermal cameras, checking for crop disease with multispectral cameras, package delivery, and even remote sampling from tree tops with robotic arms! **Fleets of drones** - Another important application for this technology is in the coordination of a group of drones. You may have already seen light shows, where hundreds of drones move together to produce images and words in the sky. Small groups of drones can also be programmed to perform tasks together, like searching for missing people or monitoring wildfires. This kind of platform is especially useful here as it allows for flight data to be seen live.  ## Concerns These possibilities are naturally going to raise privacy concerns for us. The prospect of a dystopian drone-surveilled future is not something to take lightly! But, as drones become more and more accessible, we need to think about how to legislate around their use. One way to control drone use could be to create a public database with designated airspace available to drones. Drones would have to make contact with the database to get permissions and flying space instructions. As long as the drone is connected to the internet, it could be constrained to a certain airspace (the angle of its camera could also be constrained :+1:), and could be tracked and monitored. ## Conclusion There are already a few web drone platforms out there[^5]. However, these platforms are still expensive and require some technical knowledge to get your drone connected. I hope this project can serve as a free, more user-friendly platform for remote drone use. I’ve had a lot of fun building this platform and testing drones with it: being able to manage remote drones through the internet is becoming a reality and is an exciting use for these amazing machines! [^1]: Drones are any unmanned vehicle controlled remotely (cars, boats, airplanes, etc.). [^2]: Check out the full list of Mavlink messages [here](https://mavlink.io/en/messages/common.html). [^3]: The backend is what happens in a server (away from you) and the frontend is whatever code is run on the client’s side (your own machine: phone, PC, tablet, tv). [^4]: I chose these technologies because I’d seen how powerful they were when we used them to build [Oddslingers](https://oddslingers.com/), a super cool free poker site (now open source!)--go check out the site, it’s browser native and works wonderfully! [^5]: Check out https://flytnow.com/.