Stargazer

https://github.com/dXdR-AI/Stargazer-Space-Apps-2022

https://drive.google.com/file/d/1VahLsYo31LMGS5LJCHyqHgAGGBWZ5WFg/view?usp=sharing and https://nice-meadow-0f677b610.2.azurestaticapps.net/

"Remember to look up at the stars and not down at your feet. Try to make sense of what you see and wonder about what makes the universe exist." ― Stephen Hawking

dXdR's search engine - Stargazer 1.0 - uses ML and 3D Force Graphs to visualize your query of NASA images in a dynamic, vivid, and customizable 3D projection.

Below, we explain how we implement clustering and BERT-like embedding. The clustering algorithm is designed to group together NASA images that have commonalities between them. For example, if you have two picture of nebulae that are both identified as "blue", this would be their common feature and a cluster could be formed. We use BERT-like embeddings as our search algorithm; the efficacy of this simple method is surprisingly high.

The data is queried using our backend, and visualized using the React 3D Force Graph in our Frontend.

Frontend

• Display formats: VR (AFrame), AR (AR.js), 2D (2D HTML Canvas), and 3D (WebGL/ThreeJS)
• We use the3D Force Graph, an open-source tool
• Future Stargazer versions can quickly and easily implement VR and AR display
• Linking & clustering
• Allows for linking and clustering different NASA images (“nodes”) in a space-resembling manner

Backend

• Stargazer utilizes the lightweight backend framework Flask to provide a singular API endpoint for the frontend interface to query user input
• The backend server facilitates an encapsulated model packaged by the Pickle module to leverage the overall searchability of the system with ML techniques
• The entire system was deployed on Microsoft Azure within a unified resource group
• The front-end application applies Azure Static App as a hosting option, and the backend server additionally is hosted by the Microsoft Azure App Server with a built-in CI/CD pipeline from GitHub

AI

• Coding language: Python
• Coding platform: Google Colab Pro
• Data Cleaning: eliminating misfit feature candidates
• Texthero and Scikit-Learn are applied for cleaning all data
• Removing HTML tags, links, and extraneous phrases/sentences, e.g. “Follow us on Instagram”
• Typical data science packages applied: Numpy, Matplotlib, csv, pandas, json
• No AI applied for case of less than 100 results returned from NASA API

Clustering

• How we organize groups of data along with their interconnected networks
• K-means Clustering: Classify data into K clusters where K is determined by search results and cluster’s centroid is determined by mean
• K determined by Elbow Method or Silhouette Method
• Otherwise, default K set to 20 for analyzing 20 potential characteristic commonalities
• Hierarchical Clustering: after research, we eventually opt not to use this method

BERT-like embedding

• We first scrape the Astronomy Picture of the Day
• We then extract the image descriptions
• A BERT model from HuggingFace is then used to compute embeddings for the image descriptions
• Finally, the same BERT model is used to compute an embedding for the user’s query and perform maximum inner product search to find the best results

We believe that Stargazer 1.0 and improved versions could be used as a powerful educational tool to teach students and aspiring scientists about the universe.

We collect NASA data using the provided NASA API. Some of this data includes images of NASA meetings and events as well as R&D processes. However, we try to filter out this type of data as our search engine is intended to exclusively display images of the cosmos.

We also extract the NASA Astronomy Picture of the Day.

We're a team of young developers. This Space Apps Challenge was the first hackathon event that either dXdR or XPACE have ever participated in as companies. In addition, this was the first hackathon that Mark or Astrid have participated in. We started brainstorming our project idea several days in advance. We wanted to have a collaboration between dXdR, XPACE, and lab developers that would be meaningful and impactful for both the companies, NASA, and the general public.

The vast majority of our code was finished on the first day (10/1) including both the UI and backend.

Setbacks

• In training our embedded model overnight, we found that a pickle file was corrupt. We decided to switch to a faster model in order to catch up.
• We found that the intended 3D Force Graph Bloom effect was not working.
• Data cleaning proved problematic as we largely had to develop our own customized algorithms.

Contributions

• Cindy was the developer that created the entire concept for our project in the first place. She also singlehandedly wrote about half of all the code we used in this project.
• Seth helped to divide the work for implementing Cindy's awesome idea. He contributed numerous ideas for the frontend and he also coded certain data science & data analytics processes such as data collection.
• Astrid created awesome-looking, comprehensive slides for the team and further contributed to our project idea.
• Alex provided the backend code.
• Ryan created a new NLP approach for collecting the most relevant search results.
• Mark singlehandedly developed the entire front-end for our project.

We've gone through both high pressure moments and moments of victory. The greater the struggle, the more glorious the triumph.

We believe that Stargazer 1.0 is built on a powerful idea, and when further improved and fully integrated may prove a useful tool even for young astronomers.

Thank you to Cindy for sacrificing much of your time into this. Thank you to Astrid for helping and teaching me as a leader. Thank you to Ryan and Alex for being willing to join last minute and getting so much done under a pressured time constraint. Thank you to Mark for coding the entire frontend even though you had an exam today.

1. Common Space Apps resources: Chat guide, Team formation guide, Project submission guide, Judging and awards guide, FAQ
2. NASA APIs
3. NASA Image and Video Library
4. Clustering images (Towards Data Science)
5. NASA Es2vec: Earth Science Metadata Keyword Assignment Using Domain-Specific Word Embeddings
6. Force Graph
7. Texthero Implementation
8. Determining the optimal K value for K-means clustering
9. K-means Clustering
10. K-means Clustering vs. Hierarchical Clustering
11. Scibert
12. Our previous lab project on News Cosine Similarity
13. KeyBERT
14. Image Labeler: A Web Interface To Catalog Earth Science Events
15. Machine Learning Prototyping ML and AI Research
16. UTF-8 Everywhere
17. Flask
18. Deploy Azure (Python) Flask
19. Astronomy Picture of the Day archive
20. Sbert

#AI, #NLP, #memorization, #art, #3D, #exploration