High-Level Project Summary
Solar storm prediction is important for many reasons. A storm can disrupt power grids, satellites, GPS, and communications. It is also a major threat to astronauts in space. If we can predict when a storm is coming, we can be better prepared for it. Using satellite data, we trained a neural network to predict solar storms. Our goal is to provide an early warning of a storm that could potentially be as strong as the Carrington Event of 1859.
Link to Final Project
Link to Project "Demo"
Detailed Project Description
The project is divided into three subprojects:
• Carrington Detection AI Model
• Carrington Detection AI Website
Carrington Detection Model:
Before developing any solution, it is necessary to understand what is the problem.
Specific objectives:
- Read the data from the FC (Faraday Cup) instrument of the DSCOVR.
- Design a Machine Learning algorithm to map the raw data of the Faraday Cup (FC) of the DSCOVR in magnitudes of the solar wind (density, temperature, speed), based on the training of the network with the data of the Faraday Cup of the WIND in the periods of “ground truth”.
How is the data from the DSCOVR structured?
- Spectra/cycles are taken as the minimum unit in time (data points).
- In each spectrum/cycle, an upward sweep of voltages is made.
- Sensors A, B, and C detect the current with respect to the upward sweep of voltages.
- In each spectrum/cycle, there can be a different number of steps in the voltage sweep, from 19 to 60 steps. That is, in each spectrum/cycle a total of 19 to 60 measurements can be made, distributed among sensors A, B, and C.
- Current values are not given in physical quantities but in simple units, ranging from 0 to 3072.
- The voltage values range from 0 to 63 units (no physical magnitudes are specified)
What data are expected to be obtained?
- Solar ion wind density vector (n)
- Solar ion wind temperature (w)
- Solar ion wind speed (v)
Current problem
Raw FC data from the DSCOVR spacecraft, launched in 2015, is processed into useful physical variables using classical mathematical and physical algorithms. Subsequently, with these physical variables, Carrington events can be detected.
The problem is that using classical mathematical and physical methods, it is difficult to eliminate noise, harmonics, transients, ground problems, and other details that make the measured signals incorrect.
Proposed solution.
The WIND spacecraft, launched in 1994, has the most calibrated Faraday Cup (FC) instrument ever launched on any other mission.
So, it is common for any Faraday Cup (FC) instrument of any mission to be calibrated by making validations against the data obtained in the WIND.
However, WIND and DSCOVR have differences in electrical grounds, which can result in errors when performing calibration.
It is proposed that by the magnetic field data of both the WIND and the DSCOVR, a correlation between them can be made, and when there is a maximum correlation close to 1, it is taken as the “ground truth”.
The magnetic field data is found in the datasets:
- DSCOVR_H0_MAG.B1F1 for the DSCOVR spacecraft.
- WI_H2_MFI.BF1 for the WIND spacecraft.
The problem is that to make the correlation, it must be taken into account that the x-axis of the graphs corresponds to different time series in the WIND and DSCOVR. For a time t in the DSCOVR, the corresponding measurement in the WIND occurs at t' = D(t).
For this reason, it is necessary to carry out an analysis of time series (according to the proposal), using the Dynamic Time Warping (DTW) algorithm, which consists of an unsupervised learning algorithm to measure the similarity between two temporal sequences.
After knowing the periods of “ground truth”, solar wind data of the WIND spacecraft must be obtained from the SWE_H1 dataset, filtering only those that correspond to the periods in which there is a “ground truth”.
Subsequently, the raw data of the Faraday Cup (FC) of the DSCOVR must be prepared, in such a way that they can be entered into a neural network. It is necessary to remember that the DSCOVR does not offer already processed data like the ones that WIND offers in its SWE_H1 dataset. This is precisely the goal: from the raw data get data like SWE_H1.
Then, the raw FC data from the DSCOVR and the SWE_H1 data from the WIND for the periods in which there was “ground truth” (correlation close to 1), are fed to a neural network for training.
That is, the neural network is being trained with correct data, by introducing data from periods in which there was “true electrical ground”, which were determined with the DTW algorithm.
In the end, the neural network, once trained, must map the raw data of the FC DSCOVR and calculate an output value for each of the three requested variables (which are similar to those of the SWE_H1 WIND), that is, the vector of solar ion wind density (n), solar ion wind temperature (w), solar ion wind speed (v).
Training stage:
Execution stage:
Because of deadlines, only a preliminary recurrent neural network was trained with the magnetic field data, this is helpful considering this is one of the biggest changes that we can measure when a solar storm is coming to earth.
The trained neural network gave us the following results:
Carrington Detection AI website:
The web side of the project is aimed to let the public access to the AI. It was developed mainly using JavaScript as the front-end language, with react as the main framework. At the top of the landing page, can be found our team logo with the name of our solution. After, context is provided about the challenge and also a brief description of how the AI was trained. The link to the drive that contents the code is placed at the page footer.
Space Agency Data
🛰️Datasets for neural network training
We used the following data:
https://cdaweb.gsfc.nasa.gov/pub/data/wind/mfi/mfi_h2/2022/
https://cdaweb.gsfc.nasa.gov/pub/data/dscovr/h0/mag/2022/
https://cdaweb.gsfc.nasa.gov/pub/data/wind/swe/swe_h1/2022/
By changing the year at the end of the link, we were able to use more data from different dates.
Also for data exploration, we used CDAWeb:
To get data from CDAWeb, we used the API cdasws:
Hackathon Journey
The space apps event is the first hackathon for most of the members who participated and we are happy with this experience, during these days we were forced to learn new programming paradigms, work as a team, and demonstrate that together we can achieve our goals. Our registration to the challenge was peculiar and thanks to the research center where we are doing an intership we decided to try to solve one of the proposed challenges, being our first contest of this type, we decided to go for a problem in which we could apply our programming, machine learning knowledge, and that had an interesting problem for us, so we chose Save the Earth from Another Carrington Event!, this problem has multiple approaches, however, we emphasize on what we believe is the most important, the scientific dissemination, now, the fundamental thing is to identify the positive and negative aspects of the problem. As for the project process, we solved setbacks and problems as we went along, always being empathetic and responsible with the team members and with the project.
References
Orozco-Del-Castillo, M. G., Ortiz-Alemán, J. C., Couder-Castañeda, C., Hernández-Gómez, J. J., & Solís-Santomé, A. (2017). High solar activity predictions through an artificial neural network. International Journal of Modern Physics. C, Physics and Computers, 28(06), 1750075. https://doi.org/10.1142/s0129183117500759
Owens, M. J., & Nichols, J. D. (2021). Using in situ solar-wind observations to generate inner-boundary conditions to outer-heliosphere simulations – I. Dynamic time warping applied to synthetic observations. Monthly Notices of the Royal Astronomical Society, 508(2), 2575–2582. https://doi.org/10.1093/mnras/stab2512
Vech, D., Stevens, M. L., Paulson, K. W., Malaspina, D. M., Case, A. W., Klein, K. G., & Kasper, J. C. (2021). A powerful machine learning technique to extract proton core, beam, and α-particle parameters from velocity distribution functions in space plasmas. Astronomy and Astrophysics, 650, A198. https://doi.org/10.1051/0004-6361/202141063
UNAM, Servicio de Clima Espacial México SCiESMEX. 2018 [Online]. Available at: http://www.sciesmex.unam.mx/ [Consultado: 01-oct-2022]
Okeleke, K. J. J. (s. f.). Region in Focus: Latin America, Q1 2022. [Online]. Available at: https://data.gsmaintelligence.com/research/research/research-2022/region-in-focus-latin-america-q1-2022 [Consultado: 01-oct-2022]
Siddiqi, A. A. Beyond Earth: A Chronicle of Deep Space Exploration, 1958-2016. NASA History Program Office, 2018. Available at: https://www.nasa.gov/sites/default/files/atoms/files/beyond-earth-tagged.pdf
Images
Manuel Ballester Sánchez, "Virgin Express Boeing 737-36N.jpg". Wikimedia Commons. Licenses: https://creativecommons.org/licenses/by-sa/3.0/deed.en and https://creativecommons.org/licenses/by-sa/2.0/deed.en
NASA, "Wind probe.jpg". Wikimedia Commons. Public Domain.
Tags
#CDA #Carrington #AI #Science #Pyhisics #AerospacialScience