High-Level Project Summary
Our team developed an application, which, given a set of data similar to that provided by the ISS, maps a high-resolution spacial image onto the earth, indicating electron density of the earth's ionosphere across the globe. This solves our challenge by providing a platform for amateur radio operators and citizen scientists, to provide data (station-to-station, or ISS-to-station) in order to generate a more robust model. The input of data is accessible and open to any party that would wish to contribute to mapping the most accurate image of Earth's ionosphere. This high-resolution image of the ionosphere can provide us with key information on space weather research and forecast predictions.
Link to Final Project
Link to Project "Demo"
Detailed Project Description
This web application maps electron density in the ionosphere on a 3D interactive globe, by utilizing raw data collected by the Amateur Radio on the International Space Station. To launch the application, please follow the instructions in the README of the repository.
The strength of this application is its visual intuitiveness and interactiveness, which lends itself to a good application that can be used by enthusiasts around the world to learn more about Heliophysics. Currently, in its form, it only maps ionosphere data collected from 2021, but the goal is to be able to show ionospheric data from any year for which there exists data.
Firstly, data is downloaded from NASA's open source datasets of the Amateur Radio on the ISS, in the format of a CDF file. This data is then processed through a Python script which extracts the useful data and reformats the data into a JSON object. Each data point in the JSON encodes information about longitude, latitude, time, and electron density. This JSON object is written to a data file held locally for later retrieval. We use the React Javascript front-end framework to construct the user interface. Particularly, we make use of ThreeJS and GlobeGL libraries within React to construct the 3D interactive model of a globe. Within React, it takes the local data file which encodes ionospheric information, and uses it to place data over top of the earth's surface with various colors corresponding to the intensity of electron density.
Space Agency Data
Initially, teams were unsure of which dataset would provide the most accurate visualization of the Earth's ionosphere. Many datasets were linked in the resources tab, but it was unclear whether there needed to be a concrete file format, or amount of parameters found within, that would allow us to correctly map the ionosphere. As we waited on subject matter experts to narrow down a definitive answer, it was recommended that teams do their best to continue with the rest of their projects. However, we quickly realized it would be challenging to further develop our application without the use of any data, so it was on us to determine the most appropriate data for use in our application.
From NASA's datasets, we found Amateur Radio ISS data which we were able to use to find electron density in the ionosphere. The datasets we used can be found here. Ultimately, these data sets contained exactly the information we needed to probe into the ionosphere and learn about heliophysics. This NASA data was directly used in our visualization application.
Hackathon Journey
The Signal Processing Lads entered NASA's Space Apps challenge with excitement and anticipation, eager to take our passion for both signal processing and physics/astronomy, and apply it to a unique software application. The open-ended nature of a problem such as this provided many learning opportunities for us. Lots of time was taken to fully grasp the scope and true nature of this problem; the Earth's ionosphere, and the extensive research accompanying it made for quite the rabbit hole to delve into. As we parsed through online articles, research papers, and space agency databases, our team took pride in strengthening our background knowledge in astronomy and signal processing before writing any code.
An immediate challenge arose when teams realized that no explicit dataset was mentioned in the resource tab for teams to be using. Upon contacting the administrative team, we were informed that we would have to wait on them to determine a concrete set for teams to work with, and that in the meantime we could continue developing other aspects of our project. Data can be challenging to work with, and being unsure of the format and parameters of the data we would be provided with, made planning the rest of the project a bit more challenging. Fortunately, through different linked resources, our team managed to find and parse data from the CASSIOPE Satellite, operated by the University of Calgary. This satellite utilizes the Enhanced Polar Outflow Probe (e-POP) suite of scientific instruments to study the ionosphere, and published its findings to the University of Calgary website. From here, we could view its radio receiver instrument (RRI) data, which provided quicklook plots of the Satellite's recordings every day between 2013-2022. We used this data as a benchmark for what we could expect from the ISS data: frequencies, voltages, geographical data (latitude, longitude, altitude), time, and angle. We then got to work on attempting to process these signals, and learning how one could use them to determine electron density.
As the physics behind this challenge grew increasingly more difficult, we received word that a dataset from the ISS had been linked to us in the competition's discord page. This data was saved within CDF files, which required us to write a python file in order to open and read what the ISS had recorded. As we looked over it, we found that it provided all of the same information we gathered from the CASSIOPE Satellite (so our predictions were correct!), however, it also included with it an accompanying electron density for every day that data was taken. This confused us, as we were unsure what to do with this information. If we simply took it and mapped it to a 2D or 3D rendering of the Earth, it felt like half the challenge was being done for us. The data we were given was pre-processed, nullifying any work on processing the signal from CASSIOPE that we assumed would need to be done for a challenge such as this. After speaking with an SME in discord, we confirmed that this was the case. It was difficult to let go of the work we had done up to this point, trying to learn how to extract more information out of these signals; we had worked hard and were passionate about what we had done up until this point. However, we did come to accept that in order to measure electron densities, given our current data, we would have had to make many leaps in our physical assumptions that may have invalidated the physics work done up until this point. Thus, we opted to take the data we were given, and begin mapping it.
I'd like to that everyone on this team for dedicating their weekends to learning about modern data radio astronomy and employing their technical backgrounds in pursuit of deploying a software solution.
References
On how ionospheric weather can affect forecasts: https://research.cornell.edu/news-features/weather-ionosphere
React Globe Rendering: https://github.com/vasturiano/react-globe.gl#paths-layer
CASSIOPE RRI Data: https://epop-data.phys.ucalgary.ca/2016/
ISS Data: https://spdf.gsfc.nasa.gov/pub/data/international_space_station_iss/sp_fpmu/
NASA Article explaining ionosphere: https://science.nasa.gov/heliophysics/focus-areas/ionosphere_thermosphere_mesosphere
How to read CDF files with Python: https://towardsdatascience.com/cdfs-using-spacepy-to-read-and-write-your-data-conveniently-9490f0bfee4a
Tags
#ionosphere #radio #signalprocessing #space #data #datavisualization