Aussie Astronauts

https://callumw0906.wixsite.com/aussie-astronauts

https://github.com/daniel-lyon/Aussie-Astronauts-Art-Exhibition/blob/f34d7f7bc30b8afb39343f5208089493570b56e8/demo.pdf

We created a gallery of artificial intelligence (AI) generated art using DALL-E 2 – an AI system that can create realistic images and art from a description in natural language. DALL-E 2 is a machine learning model that has learned the relationship between images and the text used to describe them. Specifically, the model uses a process called "diffusion", to progressively learn the patterns of colours and features within an image and its associated text description. DALL-E 2 uses over 3.5 billion parameters, trained on text-image pairs from the Internet.

Our gallery was inspired by the most recent photography captured by NASA Mars Perseverance Rover. We obtained the photographs by accessing the NASA Mars Rover API through a Jupyter Notebook in an Anaconda Virtual Environment. We then provided DALL-E 2 with a combination of different subject descriptions, using specific keywords and terms relating to space exploration, NASA, and science. We also included various art styles, including Indigenous Australian Art, Abstract, Impressionist, Oil Paintings, and many more. All of our text descriptions are visible when you hover over the images in our website gallery (created using Wix free website builder).

We strongly believe that art and science should be accessible to everyone. That's why we started developing an image-to-music app, to communicate the unique signatures of our AI-generated art through sound. We created a Public GitHub Repository for our project, and then set out to create a simple Python script in Visual Studio Code, that mapped the red, green, and blue (RGB) colour values of our images to parameters constituting a sound wave. For example, the red values were mapped to the note duration, the green values were mapped to the note frequency and the blue values were mapped to the volume. We then output a .midi audio file to Anvil Studio, where we converted the .midi files to .wav format, before editing and compiling the music tracks using Audacity. We hope to continue developing this project, and eventually create an interactive web application for users to generate their own artwork and associated music tracks.

We started our NASA Space Apps journey by searching through the NASA APIs for inspiration. We discovered the NASA Mars Rover API, which gave us access to the most recent photographs captured from the Perseverance Rover Cameras, in addition to some of the older Mars Rovers. We collected a large sample of photographs of the surface of Mars, including close-up shots of the surface details, as well as some great desert landscape shots. These photographs were taken from three cameras on the Perseverance Rover - Mastcam-Z, Navcam-left, and Sherloc-Watson. Soon enough, we found ourselves imagining what the human experience on Mars might be. These photographs made us feel very strongly about three aspects of the human experience:

1. The potential impact of human activity on the climate of Mars.
2. The feelings of loneliness and isolation that would be felt by the first astronauts and explorers to reach Mars.
3. The importance of diversity and collaboration as part of humanity's global effort to address climate change on Earth, as well as achieving human exploration and habitation of Mars.

We tried our best to capture these aspects of the human condition in every piece of our AI-generated artwork.

Message from our team:

"The Space Apps experience was our very first hackathon. We formed a team of two from our QUT astrophysics class and set out to learn more about science communication. We were drawn to the challenge – “The Art in Our Worlds” – because its objective strongly reflected our motivation and skillset. We initially brainstormed ideas around creating an application that would use AI to generate images. However, we were pleasantly surprised when, only two days before the challenge commenced, OpenAI released DALL-E 2 to the public. We were so excited by this news that we decided to focus the main part of our project on using DALL-E 2 to generate a powerful gallery of diverse and unique artworks. We drew upon our inspiration from our initial research into the NASA Mars Rover API, and even produced some incredible images using photographs captured from the Perseverance Rover Cameras with the DALL-E 2 outpainting feature. We also began developing a multi-media/multi-sensory application with image-to-music, to help share our work with a wider audience and promote our personal values of diversity, inclusion, and transparency."

---

Callum Williams - EDA using NASA Mars Rover API, image generation using DALL-E 2, and website design.

"Daniel and I worked alongside each other to achieve our collective vision for the project. The greatest challenge we faced was not having enough time to implement a web application interface for our image-to-music feature. However, we temporarily resolved this setback by generating a series of soundtracks for our AI-generated images and embedding them on the website for users to play and listen. In the future, we hope to optimize our program and implement additional functionality for users to upload their own images and generate their unique musical soundtrack."

---

Daniel Lyon - Image generation using DALL-E 2, image processing in Python, and soundtrack generation.

"This was my first hackathon ever, let alone Space Apps participation. It was actually a lot of fun! The submitted project has real tangible results that can be greatly improved upon in the future. I'll definitely be coming back to the github repo and making improvements, and actually, I already have plans on how to advance the project and where it's going. I learnt a lot, especially relating to project scope size given the time constraints. Tinkering with code and learning the documentation takes quite a bit of time, which meant we had to reduce the scope of our project a little bit. Relating to music theory, I knew almost nothing before starting this project, but now, I learned a lot about things like pitch, velocity, and more. Our team was inspired to take on this challenge from sonifications we'd seen else where, such as from the James Webb Space Telescope https://www.youtube.com/watch?v=urY6VCAC8ps. We we're also very interested in AI image creation, and as such, led us to choosing our challenge: The Art in Our World."

Data

Photographs that inspired us were obtained from the NASA Mars Rover API on the NASA Open APIs Page: https://api.nasa.gov/

Note: We selected photos from the Perseverance Rover from three cameras - Mastcam-Z, Navcam-left, and Sherloc-Watson.

Code

GitHub Repo for the music app that we started developing:

https://github.com/daniel-lyon/Aussie-Astronauts-Art-Exhibition

We used python with the following modules:

numpy, matplotlib, music21, skimage.

Tools

Anvil Studio was used to edit the midi files https://anvilstudio.com/

Audacity was used to edit the wav files https://www.audacityteam.org/download/Audacity ®

DALL-E 2 by OpenAI (public access) to generate the images and art. Website: https://openai.com/dall-e-2/

Visual Studio Code and Juypyter Notebook (Anaconda) to write the code.

Wix Website Editor to make our gallery website: https://www.wix.com/

#art, #OpenAI, #science, #space, #mars, #NASA, #astronaut, #Perseverance, #diversity, #spaceexploration, #climate