Dark Matter

https://github.com/CharbelSahlani/the_art_in_our_worlds

https://github.com/CharbelSahlani/the_art_in_our_worlds/tree/master/Pitch

Features:

1.      Straightforward and user-friendly GUI (Graphical User Interface)

2.      Various options to tweak, enhance, modify, beautify the original Image from NASA

3.       Exceptional image generation via NST (Neural Style Transfer) of the NASA image in the style of an image provided by the user

4.      Optimized image search solution with the help of NLP (Natural Language Processing) and CLIP (Contrastive Language-Image Pre-training)

5.      Options to share your masterpiece with the world on your favorite social media apps

6.      Availability in multiple languages

Execution:

1.      The UI is programmed using Flask (python framework), Javascript(ReactJs), HTML and CSS to give a lively and interactive experience to the user

2.      The options to tweak and beautify the image will be programmed using python and image processing libraries, namely OpenCV and Pillow, which are out of the box open source tools

3.      The NST is programed in python with the help of open source libraries namely tensorflow, matplotlib, scipy and numpy. The NST algorithm uses the convolution neural networks technique to drastically reduce the number of parameters in the neural network, hence reducing the training and image generation time. The shallower layers of a ConvNet tend to detect lower-level features such as edges and simple textures. The deeper layers tend to detect higher-level features such as more complex textures and object classes

4.      The idea behind CLIP is to take an Image and predict the text from this image. Simply by pushing the unrelated features away from the dimensional space, and pushing the similar feature together (we can think of it like k-nearest-neighborhood) this idea although it was successfully executed, we found that we didn't really need it, as the provided dataset from NASA didn't require it

5.      To be able to share the final image with the world we are using an open source module called Shareon that is used in the HTML script

6.      For the application to work in multiple languages, the Google Cloud Translation API module takes care of this by translating the web application and the user input to always receive the desired results

Equipment:

All we need for the completion of this project is a decent laptop and a stable internet connection.

Future Works:

1.      New feature to generate Images related to earth and space from user text by using NLP through the GAN (Generative Adversarial Networks) algorithm

2.      Short story generation based on the generated images

3.      Integrate augmented reality features to make the application even more immersive

4.      Make an IOS and Android version of the application

Screenshots:

Neural Style Transfer

Image Editor

Arabic Translation

French Translation

Trivia Game

In our project we made use of the NASA image and video library API, which takes the user text as input and then searches the database for media that match the user's request. One thing that is remarkable about this API is that there are many options to filter out the images to get the ones that you need with a very high accuracy.

Another type of data that we gathered from NASA is the facts for the trivia game, where the user is asked about various space events, scientific discoveries and many more. All the data were gathered from an official NASA website.

Screenshots:

NASA Image and Video Library/API

Trivia Game Fact

This is the second hackathon for team Dark Matter at NASA Space Apps and the experience was outstanding like last time. We have met a lot of very passionate people about space and technology, people of all ages and backgrounds and we find that truly amazing. The entire journey of the hackathon is such a pleasant experience, especially that the Local lead Dr. Antoine and his team of experts made sure to provide us with the right environment for us to work and bring great results to the table.

As team Dark Matter, we learn new things every single day. And what we learned at Nasa Space Apps, and by listening to local experts in technology, is that for every great idea to come to life there's consistency, discipline and commitment. It's not enough to have a great idea, but it's what you do with your idea is what makes a difference.

Everything in life can be inspiring, but what really inspired team Dark Matter to choose the Arts in Our Worlds challenge is the way that we can combine technology and creativity to make a masterpiece. The idea of our solution, is to provide a space for everyone to get creative and try different things, and monitor the results. Everyone can be an artist, when provided with the right tools.

The approach to tackle this project was to divide it into multiple mini projects and then piece it together to get the end result. So one member was working on the front end of the web application which consists of the user interface, the image processing editor window, and the various tools and features for the user to interact with. One other member was working on a machine learning algorithm to provide the option to regenerate the image in the style of a different image or painting. And lastly, one member was also working on mini games to be integrated in the web application as well, to keep the user entertained as they wait for their image to be generated. Not to mention the various various other cool image filters that were developed.

The team faced many setbacks, most notably the restriction on a server host that we were relying on to run our application, but the host wouldn't run on the provided network so we had to think of a different solution to this problem. Luckily, we were able to make the web application work on our local machine regardless of the limited processing power that it had, we managed t get the job done.

Team Dark Matter would like to thank the local lead Dr. Antoine and his team of experts, for all the effort that they put, for going beyond to deliver a great environment for us to hack and develop our ideas. Their constant support and motivation played a huge role in our lives and we hope that our solution would play a huge role in your lives as well.

Screenshots:

Images Generated With GAN

Pixel Effect

Watercolor Effect

Sketch Effect

Space Adventure Game

Papers:

The Neural Style Transfer algorithm was due to Gatys et al. (2015). Harish Narayanan and Github user "log0" also have highly readable write-ups this lab was inspired by. The pre-trained network used in this implementation is a VGG network, which is due to Simonyan and Zisserman (2015). Pre-trained weights were from the work of the MathConvNet team.

• Leon A. Gatys, Alexander S. Ecker, Matthias Bethge, (2015). A Neural Algorithm of Artistic Style
• Harish Narayanan, Convolutional neural networks for artistic style transfer.
• Log0, TensorFlow Implementation of "A Neural Algorithm of Artistic Style".
• Karen Simonyan and Andrew Zisserman (2015). Very deep convolutional networks for large-scale image recognition
• MatConvNet.

Repos:

• https://github.com/gxercavins/image-api
• https://codeberg.org/kytta/shareon
• https://github.com/gerchristko/Machine-Learning-Web-App
• https://github.com/CompVis/stable-diffusion?utm_campaign=The%20Batch&utm_medium=email&_hsmi=226067301&utm_content=226055109&utm_source=hs_email
• 
  

Drivers:

• https://developer.nvidia.com/cudnn
• https://developer.nvidia.com/cuda-downloads?target_os=Windows&target_arch=x86_64&target_version=11&target_type=exe_local

Documentation:

• https://www.tensorflow.org/install/pip#windows-native
• https://cloud.google.com/python/docs/setup
• chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://images.nasa.gov/docs/images.nasa.gov_api_docs.pdf

Useful Links:

• https://www.nasa.gov/stem-ed-resources/Extreme_Space_Facts.html

#spaceappslebanon #art #ai #deeplearning #technology #nasa #space #world