HABMC

The Stanford Student Space Initiative has launched dozens of high altitude balloons. These balloons can go up to 100,000 feet or travel as far as Morocco. Yet these flights would hardly be valuable without the ability to read, parse, and visualize the data streaming back to the ground.

The data comes from sensors on board the balloon, which is then heavily compressed and sent over the Iridium satellite network. Naturally, what exactly that data is varies with each flight, which means that HABMC has to be extremely flexible. In addition to visualizing the telemetry data sent on every flight, HABMC has to be capable of everything from displaying ozone concentrations to detecting voltage spikes -- all in real time.

How to get involved

At a balloons meeting, talk to Kai, or just talk to him on slack @{{{display-name}}} . You can also check out our tutorials [1]

Public Page

The public page of HABMC shows the position and altitude of the balloon, as provided by the ValBal communications system. This allows us to provide detailed information to the FAA and other viewers without overwhelming them with a vast quantity of potentially misleading data.

SSI Interface

If you're logged in as a member of the space initiative or have an access code, you can view the full SSI page, jam-packed with features.

Dashboard

The dashboard provides a quick overview of the flight. Fully customizable for each payload, it displays a series of graphs to understand recent trends, underneath which is a map of the flight and charts detailing remaining quantities such as battery and ballast. Below the map lies a table of the configuration on the payload itself that gives exact numbers about how the payload is set to behave.

Maps, Charts, & Graphs

We often want finer control over how we delve into and analyze the data. To that end, we built the charts utility, capable of graphing an arbitrary number of variables against each other. The user may zoom into the data by time period, or look at the whole mission.

Naturally, to see position data we have maps as well. On most flights we have three redundant positioning systems: a primary GPS baked into the board, an external GPS, and position estimates given by the Iridium satellite network. Each of these systems has their own foibles; for example, HABMC is capable of filtering out erroneous Iridium position estimates and displaying error bounds on the map itself.

Raw Data Viewing

Sometimes charts, maps, and graphs aren’t enough and the flight controllers need to read the data directly. To that end, HABMC provides tables of all transmissions, the raw, compressed details, and every other piece of data the suite has access to. To make some of this data more readable, it shows the most recent transmission’s data grouped and sorted appropriately; one can also browse through old transmissions in this mode. Finally, should the flight controllers want to access the data directly, HABMC offers a full API.

Messages

HABMC can also send messages back to the balloon, in addition to getting data from it. These messages, however, are far from human readable. HABMC builds these messages through a more intuitive interface, then verifies that they’re in the correct format before ultimately sending them to our payload. It will also store these messages and notify the flight controllers so as to ensure that everyone stays informed.

Configuration and permissions

From the start, HABMC was designed to handle a wide variety of flight profiles. It’s able to understand three different parsing formats; much of the UI is even customizable. HABMC can even be set to notify the flight controllers via Slack or SMS when it detects a suspicious transmission. To safeguard these features, HABMC has a sophisticated user management system to grant select permissions only to those who need them.

Prediction & Footprint Calculator

We almost always want to know where the balloon will go. To that end, we have a full predictor utility. Although it currently relies on the GFS weather model, it’s designed to be compatible with the more-accurate ECMWF model should we gain access. The predictor, written from the group up in Rust, takes this data and turns it into real predictions.

Of course, there are a lot of uncertainties when it comes to prediction, such as descent rate and burst altitude. Thus, when we want to know where the payload will land, we use HABMC’s footprint calculator. This system generates a probability distribution about where the balloon will ultimately come down.

Navigation

One of the core features of our novel altitude stabilization platform, ValBal, is that it’s capable of flying at whatever altitude we tell it to, and adjusting that altitude mid-flight. Since there are different wind patterns at different altitudes, by strategically adjusting altitudes the balloon can optimize ground distance or even aim for a specific location. For example, the balloon might fly at 13km to travel west, then ascend to 15km briefly to steer north toward its final destination.

In this screenshot, the naive option of flying at the single altitude with the fastest winds is shown in blue; active guidance is shown in green. In the same total flight time, with active guidance the balloon can fly over twice as far.

Commands Module

On the launch site, we often suffer from an unreliable internet connection -- yet we still need to know how our payload is performing. To that end, HABMC also hosts a natural language processing module. While we may not have internet, we almost always have SMS service: the commands module allows us to text commands and queries to HABMC. Example messages include “When was the last transmission?” or “Push the launch time back by 15 minutes”, to which HABMC will respond appropriately.

Livestreaming

Sending live video down through SSTV can be displayed on HABMC as well. Piping the data through ffmpeg on the client to Youtube Live lets us maintain maximum compatibility with inputs and deliver the video at maximum spee

High-bandwidth radio backend

Capable of storing data and delivering high-bandwidth transmissions as fast as the database can handle, the radio code turns HABMC into a truly real time control suite. With the system in place, it can display the data from multiple transmissions per second.

Technical Facts

HABMC is written with Ruby on Rails and React.js, hosted on Heroku. External services include a prediction and navigation server (written in Rust), an InfluxDB for storing high-bandwidth radio transmissions, a Postgres primary database, writer and followers for radio transmission (written in Node), Newrelic for analytics, and Cloudflare as a CDN.

History

Origin

Kirill Safin first created HABMC, designed as a way of understanding ValBal launches. At this stage, functionality was minimal; it had the skeleton of the entire site, written with angular. The only pages that had any content were the SPOT GPS pages with their embedded maps displaying where the balloons were. Beyond that, the core achievement was the skeleton itself: it was well designed and provided an excellent foundation for what was to come.

Version 1.0

The next stage of HABMC, when it was still written with PHP, involved fleshing out the skeleton. We built the dashboard, which contains graphs and indicators of the flight vitals. The full map, which provided a larger view of the flight path, the transmissions page, which listed the parsed transmissions, and a utility to send messages to the balloon through RockBlock were all added as well. We created a couple pages to see what was happening at a lower level: a page to show us the raw, unparsed transmissions and a page that mapped the position estimate based on the iridium satellite network. One of the more exciting things that we added was the predictor utility, which made an educated guess as to where the balloon would go next, based off NOAA data. And to delve deeper into the data, we created the charts and graphs page, which graphed arbitrary data streams against each other. Finally, to control it all, we had a configuration page.

To power all of this, we had a complex backend, written at this point in PHP. When a transmission came in, it would be stored in a SQL database. Then, to parse and understand it, the transmission would be compared with the parser configuration we had set from the frontend. The backend was also responsible for keeping track of that configuration and making calculations (such as velocity) that were not in the transmission.

The data would also update in realtime, but it did so much less efficiently than it does now. Pages would repeatedly ping the backend and update their data if necessary. While this worked well from the user's perspective, as they never had to reload the page to see new data, it was terrible from the server's perspective. Each chart or graph, of which there were nine on the dashboard alone, had to make a request to the server every few seconds, meaning that the backend was bombarded with requests, most of which did nothing.

In this stage too a Github was created for easier code maintenance. While the deploy process still involved manually uploading the files via FTP, this simplified the challenges of working with multiple people on one project.

Version 2.0

The release of version 2 involved turning HABMC from a clunky interface to a more elegant bundle of code. To start, we rewrote the entire backend in Ruby on Rails. As GoDaddy does not support Rails apps, we transitioned to hosting HABMC on Heroku. Advantages here included better load monitoring and simpler deploys. The real advantages, though, came from the updated backend: we had a much more MVC oriented architecture, maintenance was easier, and security was stronger. Further, this allowed us to refactor the JavaScript that powered the core of HABMC. Prior to this, all of the JavaScript lived in one behemoth of a file. Taking advantage of the asset pipeline meant that we could separate this out into a couple dozen distinct files, making the codebase as a whole much simpler and understandable. Routing these files through the asset pipeline additionally meant that we could first minify and then gzip them (and all other static assets such as stylesheets), meaning that the end files were about a tenth the size.

A host of new features arrived between Version 1 and Version 2, like a login system and plotting FAA zones on maps. It also included an architectural change to reduce server load: rather than pinging the server constantly, we moved to a model with push notifications, based on websockets. To further reduce the load, we also batched requests when a change was triggered.

Version 3.0

The frontend underwent a complete overhaul, getting rewritten from the ground up in React to be as efficient as possible. This made all of HABMC snappier -- and also way more maintainable. While we were at it, we redid the styles, switching to a dark theme.

Future Plans

At this point, the core functionality of HABMC has stabilized, but there's always more to be done. Throughout the pipeline, HABMC can improve; old features need to be made more efficient, and new features need to be built. Many features, like navigation and radio integrations, need to mature to support the rigour of flight.

Above all, going into the future HABMC will continue the relentless cycle of refinement and improvement to build it into an ever more powerful mission control suite.