Space Dolphins

https://github.com/jprinaldi/starsense

https://starsense.earth

What does our project do?

Our project aims to allow users to enjoy NASA's amazing images in a new, enhanced way. This is done by stimulating multiple senses (namely vision, touch and hearing) all at the same time.

How does our project work?

Our project works by giving the user an interface where they can input any text they want, and we use this text to query NASA's Image API to retrieve relevant images. Then, we use a different API to obtain a soundscape that matches the image, and we give users an interface where they can touch the screen of their device and feel different levels of vibration depending on the area they are touching.

It will apply AI techniques in several different ways. It will intelligently match the user input to NASA's images instead of just doing it based on keywords. In addition, it will generate soundscapes based on the input image. Furthermore, user feedback will be used to adjust the experience.

Currently, this app can be accessed via https://starsense.earth. In the future, it may be interesting to explore creating native versions of this app for different operating systems in order to have greater control of each devices features. This is especially important given the fact that, as of the time of writing, the Vibration API is not supported by any of the most popular web browser on iOS. However, since we didn't want to leave iOS users out of the experience entirely, we introduced sounds that play at different volumes depending on the area of the screen being touched, similar to how the vibration feature works.

Starsense: The Universe at your fingerprints. Sun image: https://starsense.earth/sense/PIA17669

What benefits does our project have?

Our project aims to give people a new, different and hopefully exciting way to experience NASA's amazing images.

What do we hope to achieve?

Our hope is that people who maybe don't find this content interesting initially might find a different way to appreciate it by presenting it in a creative and artistic way.

What tools, coding languages, hardware and software did we use to develop our project?

This is detailed in the references section.

We integrated NASA's Image API, accessible through https://images-api.nasa.gov. This API exposed 4 endpoints, and we used 2 of these:

GET /search?q={q}

We used this endpoint to retrieve a list of images using a query provided by the user.

GET /asset/{nasa_id}

We used this endpoint to retrieve data for a particular asset. This was useful to be able to create a URL that points to a specific image, so that users could share interesting experiences with others without having the latter perform any type of search themselves.

Speaking of this API, we actually believe we found a mistake in the official documentation (https://images.nasa.gov/docs/images.nasa.gov_api_docs.pdf). It is mentioned that the endpoint used to perform a search supports an optional parameter named media_type which supposedly accepts 2 possible values: "image" and "audio". However, our tests indicate that, while "image" is indeed an accepted value, "audio" is not. Instead, it seems the accepted values are "image" and "video".

We are a group of three professionals from different types of studies, a Computer Scientific, passionate about AI and Machine Learning, an Industrial Engineer who works on industrial processes automation and a Doctor who works with people with disabilities. We put a good effort on building a strong diverse team, all of us share the will to create tools or solutions for people who most need them.

This was our first Space Apps experience and we all agree it was wonderful. From the amazing projects that were presented by the other teams to the lovely people that we had the fortune to meet, including participants and organizers, we truly think this is an experience that cannot be missed.

We were able to learn how people with very different skillsets can successfully work together towards a common goal. We learned the importance of divide & conquer: no matter how complex the project is, if you can divide it into manageable tasks, you can still make great progress, regardless of how little time you have.

We started developing the project by defining very exactly what was our objective, then talking to mentors and people in general to validate the idea as well as getting feedback. We developed a proof of concept and tackled many technical issues in record-time thanks to combining best expertise from each member. When this version was working, we were so happy to see that the first impression of everyone who we explained our Project and then were able to test it!

Hardware

MacBook Pro (M1, 13-inch, Late 2020): Main development machine

iPhone 12 Pro Max: Main device for testing iOS compatibility

Moto g200 5G: Main device for testing Android compatibility

Software

Visual Studio Code: Code editor

Languages/Frameworks

HTML5, CSS3, TypeScript, Svelte

Libraries

Tailwind CSS: UI framework

daisyUI: UI components

ESLint: Code linting

Prettier: Code formatting

Services

Porkbun: Domain registrar (starsense.earth)

Cloudflare: DNS provider, application hosting, HTTPS support, DDoS protection, HTML/CSS/JS minification & more

GitHub: Code hosting & version control

SVG Repo: SVG icon provider, we got our app's logo from here

APIs

NASA Images API: Used to retrieve list of images given a specific query and to retrieve a particular asset given its NASA ID

Freesound API: Used to retrieve a list of URLs that point to audio files that match a specific query

#art #creativity #soundscape #accessibility #inclusion #curiosity #experience #ai #artificial-intelligence #machine-learning