Inside Skydio's Autonomy Stack: From a Live Remote Flight Deck to Multi-Drone Orchestration

The talk opened with the presenter thanking the seminar and then showing what he called a remote flight deck, the company's main user interface. Anyone with login access can use it to see the company's systems across an organization. In the demonstration, docking stations were displayed on a map, and the presenter selected one at the company's headquarters in San Mateo, close to the venue.

The map also shows where drones can and cannot fly. The red areas mark limits on maximum altitude or no-fly zones, which the presenter said exist mainly because of San Francisco airport and other locations with altitude limits. The presenter stressed that the demonstration was live, not a video, and then took off one of the drones from the headquarters roof, flying it with a touchpad while expressing the hope that the underlying autonomy system would make the piloting efficient.

The presenter said Skydio is best known to many people from its consumer products, which began more than ten years ago with the Skydio R1 and Skydio 2. Those drones were mainly used for filming people doing sports, tracking them closely while avoiding obstacles and keeping them in line of sight. The presenter described that tracking capability as the foundation the company has built on for a long time.

The company's approach, as described, is to take off-the-shelf commodity mobile CPUs and imaging sensors, make them custom and more efficient, and mount them on four motors and propellers together with AI technology. The presenter said Skydio was founded in 2014, has shipped more than 60,000 drones and serves more than 4,000 customers, with around 1,000 employees, about 50 to 60 of them on the autonomy team. Skydio builds the dock hardware, drone hardware, low-level vehicle software, autonomy software and cloud software itself, and manufactures in the US.

The presenter said that as usage grows, the reliability required to deliver useful work rises very quickly. He showed an image of a deployment in Las Vegas, which he said was not AI-generated, where multiple docking stations, which the company calls hives, are placed across the city. Pilots sit in a real-time crime center and can coordinate and teleoperate the drones from one location.

He described a Bernalillo County Sheriff's Office case in which officers were called about someone shooting a gun. Deploying a drone and viewing the scene from a different perspective revealed that the people involved were children who had taken a gun from their parents. That information was relayed to officers on site, who adjusted their response, disarmed the children and de-escalated the situation. The presenter quoted the officers as saying the incident could have turned much worse without that information.

A case from OKC PD involved a call that a person had been hit by a train. The presenter said a drone flown along the tracks helped locate the person, and paramedics were able to save that person's life. He also referred to a case in which a stolen car was tracked for the whole incident with multiple drones, and said the suspect swapped license plates, which the drone captured.

The presenter said that Skydio serves what he called the backbone of civilization and that the company has demonstrated life-saving impact across many industries. He argued that these systems are no longer toys. Once a system becomes infrastructure, it must be available 24/7 and 365 days a year and function whenever a pilot needs it, which he said marks a shift from the early era when these devices were mainly cameras for cool shots and videos.

He said the company's flight count has grown sharply since 2021, driven mainly by the introduction of the X1 and, more specifically, the Dock Plus X1, which he described as the key enabler that lets systems be deployed fully autonomously and launched by pressing a button. He said police officers now fly more than 1 million times a year, while noting that this is still a small fraction of all 911 calls. The presenter also described about 610 docks located near distribution poles and said there are 180 million distribution poles in the US that could be inspected. He mentioned that around 3,000 X10 units were delivered under a Ukraine contract, roughly equal to one week of usage there.

To test whether the concept can scale, the presenter described a Clark County simulation built from about 500,000 calls in the company's data set. The simulation starts by placing docks in the most densely populated areas with the most 911 calls, beginning with 15 docks and 15 operators. Covering all of them would require 160 docks and 160 humans if each drone were paired one-to-one with a pilot.

The presenter said finding high-quality pilots and training them to be proficient is extremely difficult and is one of the key things the company must overcome. The company has introduced a capability that lets one pilot operate at least four drones at the same time, but he said that still means 42 operators. The stated goal is to get as close to zero as possible, so that pilots become high-level orchestrators who give high-level commands and assign executive missions that the drones perform together.

The presenter added that the company does not move from zero to 100 immediately.

The presenter said that in dense cities such as downtown San Francisco, GPS tends to be very unreliable. He attributed this mainly to satellites being occluded and to multipath effects, which make the measurements unreliable.

Because the drone's sensor measurements and camera information are both available, the company can fuse them. The system identifies which satellites are occluded in the image space and rejects them, which the presenter said produces a GPS pose estimate that is substantially more reliable and accurate in exactly the scenarios where it matters most.

For inspecting critical infrastructure, the presenter said users load all assets into the software, select which ones to inspect and choose a shot type. The software then generates the shots needed for the best coverage of the things that matter most, and the mission runs autonomously. The drones carry an RTK attachment, which provides very reliable pose estimation, and humans review the resulting shots for anomalies.

The presenter said the biggest problem in deployment is the location of the assets. Some utility customers have very precise locations because they created high-precision LiDAR point clouds of their entire grid, but this is not true across all utility customers. The company therefore has to build a way to obtain precise asset locations.

For security and monitoring, the presenter described docking stations placed across sites such as universities, headquarters and utility customers that fly autonomous patrols around the clock. Operators draw areas on the map where no one should be. Onboard people and car detection then sends alerts to pilots only when something is detected in those areas, so one pilot can oversee fleets in completely different regions of the country.

The presenter said the reliability of these models is very important. With tens or hundreds of drones across a deployment, frequent alerts would make it impossible to keep up, so the false positive rate really matters. He also said people or cars alone are sometimes not enough, so the company is moving toward open-vocabulary models in which users can type a target of interest. His example was wooden pallets, with doors possibly to follow, and the aim is to build models that generalize as much as possible using the latest foundation models.

The presenter said the most important thing these systems give customers is real-time video showing where the drone is. Because the compute cannot run on board, the company streams footage from the drone through the cloud to each user's computer. He called this a very important fundamental part of the architecture and product, and said the company invests substantial resources in it.

The connection is built from multiple sources. Every docking station has an external radio that talks to the drone, and in many deployments the drones also have SIM cards for 5G. The company fuses these links into one connection that keeps latency low and video resolution high. The presenter said this is the piece that allows very large models to be trained and then deployed in the cloud.

The presenter explained a visibility problem with fixed-wing flight. If a vehicle loiters around a point, a part of the wing can occlude the view of the subject. If the subject is tracked at high frequency, losing it even for a fraction of a second can mean losing the subject. The company responded by mounting the gimbal on the left wing tip, so that the aircraft keeps line of sight to a point while orbiting.

The aircraft's design, as described, counterbalances the fuselage on the right and includes two control surfaces, two flaps and two motors, with two nav cameras at the front for obstacle avoidance. The MegaDock is a container with a shelf where multiple aircraft charge and are stored, plus a robotic arm that takes a vehicle from the shelf, launches it, catches it on return and puts it back. Because one dock can handle several aircraft, a single operator can run multiple vehicles from one dock. The presenter said the goal is that takeoff and landing are fully autonomous and that the flying experience is a single click at a point.

The presenter said that Skydio has a mature autonomy stack able to control its vehicles across the whole portfolio, and that the fundamental enabler is a loop: how quickly the company can deploy to the real world, get data back, retrain models and evaluate them. He said that how quickly this loop turns determines how quickly the company can move.

He said the company believes it now has the largest data set of flying robots. Some customers choose to give access to their data, which Skydio aggregates and de-identifies to train models at scale. He said more robots will fly in the future and that one next step is for the drone to become the user's body, controlled through natural language, which he said would enable a transition beyond tasking drones at intermediate levels.

The presenter said police departments define their own policies on how and when they use the drones and manage them themselves, but that transparency is perhaps the most important factor. The company has invested heavily in a transparency dashboard, announced at a recent event, on which every police department publishes every flight it conducts. Anyone interested can look at each time the drones fly and see how they are used.

In a question about partnerships with companies such as Amazon, the presenter said Skydio does not do deliveries. He said its work with Amazon and other companies such as Zipline is to make sure the systems never crash into each other, through direct collaboration and information sharing. Because Skydio's flights generally respond to 911 calls and carry higher priority, the delivery companies can adjust their paths to avoid collisions.

The presenter described a guiding principle: anything that requires high-frequency commands, or has to work without connectivity, runs on the vehicle. Obstacle avoidance, subject tracking and sky masking, which identifies where the satellites are, all run onboard. He said connectivity is very good but is still occasionally lost, so the drone must be able to avoid crashing into buildings if that happens. Higher-level reasoning, such as open-vocabulary detection, can run in the cloud.

Drones never communicate with each other directly; all communication goes through the cloud. The presenter said coordination between drones could be a future use case but is not something the company is actively pursuing.

The presenter said that in thunderstorms and extremely high winds, the system will not move forward, and the vehicle generally cannot fly. Every docking station has a thermal station, so if conditions are harsh the system prompts the user not to fly. The user can override this, but the company says they should not. In rain, the devices can still fly because they are waterproof, but the company turns obstacle avoidance off, since rain produces many spurious voxels that would cause unwanted vehicle movement.

He said flights generally take place beyond visual line of sight, so operators do not see the drones directly, and that wind is the most important specification for the vehicle. In answer to a question on speed, he said the fastest drone reaches about 100 miles per hour, shown in a sprint at the company's Ascent conference, and that flight time is much higher than a typical quadcopter, though he did not recall the exact figure.

The presenter said Skydio works closely with the FAA and helps customers acquire regulatory permissions. All of these customers and their pilots have beyond-visual-line-of-sight permissions, and some have permission to operate multiple drones simultaneously. Some areas require flying below certain altitudes, and the company enforces some of these restrictions in the underlying software, which pilots cannot necessarily override. Pilots are also trained and certified under Part 107.

In answer to a question on tracking speed, the presenter said that as an engineer he did not know the figure in miles per hour, but that the maximum safe speed is 20 meters per second. The drone has obstacle avoidance, and the company does not currently change the models for vehicles versus cars, using a unified approach.

Asked about vision-language-action models, the presenter said the answer depends on the use case. Some functions, such as transcribing a call, need not be VLA. He said VLA as a general solution is very interesting to the company and an area it is actively working on, though it will take time to mature. In the meantime the company will build intermediate, tailor-made solutions that enable parts of the functionality.

He said the models are multi-billion-parameter, so the initial focus will be the cloud side, with the aim of making it work there first. Models are becoming more efficient and faster, and over time they may be distilled into smaller, targeted problem sets, which he said he sees as a definite path to running on the drone, though not as the first application. He described the round-trip latency as roughly 100 milliseconds.