Awards & Nominations
Argonauts has received the following awards and nominations. Way to go!
Global Finalists Honorable Mentions
NASART - by Argonauts
High-Level Project Summary
NASART by Argonauts is a prototype text-to-image generating application created using Machine Learning/Artificial Intelligence techniques for displaying creative artwork using official NASA imagery. NASART retrieves an image from the NASA APOD (Astronomy Picture of the Day) API based on a user input date and utilizes it as the base to generate an artistic image corresponding to the user’s text prompt. Thus, NASART enables a broad public audience to engage with NASA data creatively and artistically.
Link to Final Project
Link to Project "Demo"
Detailed Project Description
Not long ago generative arts and NFTs took the world by storm. This was made possible after OpenAI made large strides in text-to-image generation and announced DALL-E, a powerful text-to-image generator. However DALL-E was not released to the public and non-commercial text-to-image generation was nothing but a daydream for a long time. At least until OpenAI decided to publicly release the neural network known as CLIP.
What tools, hardware, or software were used to develop the project?
CLIP for Image Training
CLIP (Contrastive Language-Image Pre-training) is a neural network that efficiently learns visual concepts from natural language supervision. In short, CLIP can compare an image and a text prompt and give a score based on how similar the image is to the text prompt. DALL-E, the current leading text-to-image generator, also uses CLIP to rank the generated images and output the image with the highest similarity score to the text prompt.
VQGAN for Image Generation
VQGAN (Vector Quantized GAN) is a publicly available application, also known as taming transformers for high-resolution image synthesis. VQGAN uses a massive library of images to learn a codebook of context-rich visual parts from each image. Thus, VQGAN can mix, merge and adjust these individual parts to generate complex and widely different images.
How does NASART work?
VQGAN+CLIP
The basic concept behind NASART’s image generation is a combination of VQGAN and CLIP. VQGAN will generate an image and send it over to CLIP who will give the image a score compared to the text prompt. If the score is too low, VQGAN will modify the image based on its library of image parts and send over the new image to CLIP. This process will continue until CLIP gives a sufficiently high score.
Combining VQGAN+CLIP with APOD API
Astronomy Picture of the Day (APOD) is one of NASA’s most popular websites, presenting a new astronomical picture every single day. The APOD API contains these images in a vast repository corresponding to each date. NASART will access the APOD API and retrieve the image corresponding to the user’s input data. This image will then be sent to VQGAN to be used as the initial image in the generation process.
What exactly does NASART do?
NASART generates images and art, limited only by the text prompt and the NASA imagery. Consider NASART to be a household artist, that is inspired by NASA imagery to create and imagine the artwork that you, the user wish to see.
Due to the training model of CLIP and VQGAN, NASART’s image generation can be highly specialized. Specific hashtags can be used along with the primary text prompt to give exciting and creative results.
Some particular hashtags such as #artstation #deviantart would limit the image generator to art analyzed from those websites and thus generate visually similar artwork. #4k or #8k would improve the quality of the image by using higher quality models in the generation process. #painting #oil painting #pencil sketch #photo are some hashtags that can completely change the medium of the generated artwork. The users can test and experiment with various hashtags of choice and observe how it affects the image generation.
Final Output of the Project
Running on Google Colab
The Google Colab notebook link: https://colab.research.google.com/drive/1dgVZQaFwC__gTVg8leJFUJDOlxvw1Nzh?usp=sharing
NASART can be run on the Google Colab servers with their GPUs, by following the included instructions. This was done with intention of making NASART more widely accessible as the program requires a high capacity GPU for the image generation.
Using GRADIO for the UI
The Google Colab notebook also comes with a code to run NASART on Gradio, a free hosting service for machine learning applications . This makes NASART even more accessible to a general audience. Besides the input of the text prompt and APOD date, the Gradio UI of NASART comes with the feature to adjust the quality as well as the size of the image. It also comes with a screenshot feature to share the image once the generation is completed. However, since the hosting temporary it is limited in a way.
What do you hope to achieve?
Our objective is to allow everyone to create images and artwork without having to know technical drawing skills and to visualize the concepts related to space in an artistic manner. NASART can produce high-quality art and realistic images much faster than humans. Therefore it can help artists to develop new ideas for their work.
Future of our project
Given a chance and longer development time, a few improvements we would like to make are,
- Optimizing the image generation procedure to be faster and more efficient.
- Adding more changeable settings to the image generation process.
- Hosting NASART on an actual website with dedicated GPU servers allowing seamless and concurrent image generation. Currently, only one image can be generated at a time, and the following user’s image will only be generated after it is completed.
- Improving the website UI to be more attractive and including a gallery for previously created images.
Space Agency Data
We made use of the Astronomy Picture of the Day API based on the official website at: https://apod.nasa.gov/apod/astropix.html
NASA's github link for the APOD API used in the notebook: https://github.com/nasa/apod-api
The specific data used were images from the APOD API taken based on the user's input regarding the date.
Hackathon Journey
We had a fantastic time completing the SpaceApps challenge this year. Although we went through numerous obstacles, we faced them successfully as a team. We spent countless sleepless nights coding and designing our project. We knew nothing about Machine Learning and Artificial Intelligence techniques, but we challenged ourselves not to limit ourselves and give our best shot at it. It was satisfying reaching the end when we finally got our platform to produce fantastic results. We are also taking this opportunity to thank all our local leads, mentors, friends and family members for giving us their fullest support with our project. At the end of the day, it’s been a delightful journey to work with everyone.
References
Link to official CLIP blog: https://openai.com/blog/clip/
Link to official VQGAN code: https://compvis.github.io/taming-transformers/
NASART refers to PIXRAY by dribnet: https://github.com/dribnet/pixray
NASA's Github link for the APOD API: https://github.com/nasa/apod-api
Tags
#art #ArtificialIntelligence #machinelearning