High-Level Project Summary
We have developed a visualization application for the Jupiter system using AI.On the NASA website, Junocam raw images are available for public viewing and volunteers can post and view the results of processing the raw images.We developed an app that allows users to easily try processing these raw images using AI technology.Generating beautiful images from raw images is not easy because image processing requires knowledge.The goal of this project is to promote interest in scientific observation of Jupiter's system, AI technology, and image processing through the experience of using AI to easily generate beautiful and artistic images by oneself.
Link to Final Project
Link to Project "Demo"
Detailed Project Description
We have developed an AI-based visualization application for the Jupiter system.
We developed an application that can generate, transform, and analyze images from artistic and scientific perspectives using three machine learning models.
① Image generation by Stable Diffusion
② Image translation by CycleGAN
③ Semantic segmentation by Deeplabv3
①Image generation by Stable Diffusion
Stable Diffusion can generate images that could be used in the field of art. By inputting a junocam image, transformation theme, and transformation strength, the image is generated according to the theme. The created images may be used as materials for game production, etc.
②Image translation by CycleGAN
With CycleGAN, you can perform image transformations like those of the most skilled image processors.
Even those who are not confident in their image processing skills can easily process.
③Semantic segmentation by Deeplabv3
This theme is still in the provisional implementation stage. We are aiming for something like a color-coded display of what material each pixel is composed of.
If successfully implemented in the future, it may be useful for independent control of spacecraft and selection of data to be sent to the earth. For example, the spacecraft could autonomously select a landing site, or preferentially transmit images of areas of interest to scientists to Earth.
Space Agency Data
Raw images were downloaded from the NASA juno cam website and processed using AI.
Hackathon Journey
We chose this challenge because we wanted to complete some cohesive deliverables in a short period of time.
We are mainly composed of members of a university laboratory, specializing in image processing and AI technologies.Therefore, we aimed to apply these technologies to implement something useful in space.
Although I had some prior knowledge of the implementation of AI itself, there were many aspects of application development that were a new challenge for me, and I learned many new things.
I would like to thank the mentors at the Toyohashi venue for their cooperation.
It was also my first experience to keep looking at planetary system images seriously, and I learned to think about processing related to images that I am not exposed to on a daily basis.
References
- Stable diffusion
https://karaage.hatenadiary.jp/entry/2022/09/26/073000
https://github.com/cedro3/others2/blob/main/Stable_Diffusion2.ipynb
- CycleGAN
https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix/blob/master/train.py
- Deeplabv3
https://colab.research.google.com/github/pytorch/pytorch.github.io/blob/master/assets/hub/pytorch_vision_deeplabv3_resnet101.ipynb