Awards & Nominations
Artti has received the following awards and nominations. Way to go!
Artti: Generating Space Art for NFTs
High-Level Project Summary
ML/AI advancement has led to many scientific breakthroughs in Healthcare, Language, and even Arts! With breakthroughs like Stable Diffusion and Dall-e, we've been able to generate text from images, and even images from the text! Artworks like the popular monkey face art, are now being stored as NFTs. We built an app called Artti, that helps you bring your dreams and imaginations about space to life and store them as digital assets like NFTs. It only needs you to put your “If I were in space, I would” wishes and it turns them into artistic pictures. While we wait for technology that’ll take people to mars, Artti helps you explore possibilities in space from images gotten from NASA
Link to Final Project
Link to Project "Demo"
Detailed Project Description
What is Artti?
Artti is an android app that allows users to input texts and returns images that visualize the text. It uses an AI model trained on images and the corresponding text description to generate images based on text.
Why Artti?
- Artti allows you to generate images that represent your imagination of activities you'll love to do in space.
- It allows you to share these pictures with your friends or store them as digital assets known as NFTs
How was it made?
The project consists of three main parts:
- The Data gathering process: The data used were images scrapped using BeautifulSoup (python) from NASA Art Program, European Space Agency Space Art, and Astroanimation websites. About 100 images were scrapped with Python.
- The AI model and API: The AI model is a finetuned pre-trained Dall-e mini, recreated and open source by LAION. Dall-e is a State-Of-The-Art AI model used for generating images from text. This model was fine-tuned with the images gotten above and deployed to an API. This API houses the model and the functions needed to receive input(text) and return output(image). The API and AI model were built with Python.
- The Android app: Artti app was built with Kotlin and the Android SDK tool and it is the only part visible to the user. The user can see a search box asking for input and it displays the result gotten from the AI model after a text input is entered.
The project was built with two programming languages, Python and Kotlin. Python was used to build the AI model and gather the data while Kotlin was used to develop the mobile app.
How does it work?
Artti Interface:
Artti hopes to be an app that explores imaginations and paints them in pictures, creates artistic AR/VR experiences, and stores them as NFTs.
Space Agency Data
To train the AI model, we used art images from European Space Agency and NASA
European Space Agency - Space Art: https://esahubble.org/products/art/
NASA - Space Art: https://www.flickr.com/photos/nasacommons/sets/72157633977913266/with/8973492146/
We also got inspired by NASA's Mars Rover Photos data to recreate what it's like to live on mars.
NASA's Mars Rover: https://api.nasa.gov/
Hackathon Journey
Overall, it was thrilling, exciting, challenging, and fun. It was a time of immense learning and collaboration. We had a great time hacking. We chose this challenge because text-generating images have been trending for a while and we wanted to see how we could use them to generate space art and convert them to digital assets. We learned how to finetune and implement a SOTA model in less than 2 days!
When we chose this challenge, it took brainstorming sessions for us to figure out how the project can make an impact. We needed to first understand what the problem was and how we were going to approach it. We then did brainstorm how we could channel the idea to make an impact apart from just building something cool.
Some of the challenges we faced were:
- Finding a suitable open source pre-trained model architecture since we can’t possibly train our own from scratch because we don’t have the data size, time, or computational power.
- The large models like Dall-e are not open-sourced so we could not customize them. Fortunately, a group of scientists recreated Dall-e and made it open source so we only need to finetune it with the space data.
- The model we found was very large, about 5.5GB and so it was difficult to upload to a cloud server to consume over an API for free. We could only make inferences when the codebase is active, which means that it can't be accessible every time unless the code is running as we were unable to deploy due to the model size. We also had to figure out how the project worked because the documentation wasn’t so clear.
- An alternative that we could use are Diffusion Models, but the open-sourced version was specifically stated to be used for research purposes only, so we couldn't use it.
So we decided to make use of the resources available to us, which was a ready-made Inference API from Crayon to generate images from text.
References
GitHub Project Link:
https://github.com/feranmi2002/Artti
Inference
Model Building
https://github.com/lucidrains/DALLE-pytorch/discussions/362
https://openai.com/blog/dall-e/
https://www.astroanimation.org/
Android App:
https://www.iconpacks.net/free-icon/rocket-12245.html
https://kotlinlang.org/
Tags
#art #AI #NFTs