Nowadays, it’s even easier to get started with machine learning and art. The Martinez Group recently had an internal competition using the neural_style package and a pre-trained VGG net to transfer art styles onto pictures of molecules we are studying.
I chose this image of myoglobin I generated for a benchmark paper I am writing as it is one of the most composed molecular images I have generated so far. The actual structure is shown in the transparent ball-and-stick style, with the heme group (an iron atom in a porphyrin ring) displayed as thicker and more opaque. Due to the size of proteins, it is common to visualize a cartoon-y ribbon style that shows secondary structure, such as the various alpha helices shown in dark gray.
I wouldn’t say I know a lot about art or art history, but you don’t need an encyclopedic knowledge of artists and art styles to try and make something cool. In this case, the ribbon representation of the alpha helices reminded me of the stairs looping around in Relativity by M.C. Escher. I thought it would be cool if the stair pattern could be imprinted into the alpha helices of the myoglobin, so I chose this piece for the style.
In the end, the net was fairly simple to run, assuming you have Anaconda installed on a machine with a GPU and are somewhat comfortable on the terminal:
git clone https://github.com/anishathalye/neural-style.git conda create -n neural_style conda activate neural_style cd neural-style pip install -r requirements.txt curl http://www.vlfeat.org/matconvnet/models/imagenet-vgg-verydeep-19.mat -o imagenet-vgg-verydeep-19.mat python neural_style.py --content myoglobin.png --styles escher.jpg --content-weight-blend 0.2 --pooling avg --style-scales 0.2 --preserve-colors --output stairway_to_heme.png
Only the content, styles, and output keywords are needed for the net to run. However, I played with a few other keywords to get something closer to what I was looking for:
- I lowered content-weight-blend to transfer more of the style since I didn’t care about the fine detail structure of the myoglobin
- I lowered style-scales to try and match the size of the stairs in Relativity with the alpha helices
- I switched pooling from max to avg to go for a more abstract look
- I used preserve-colors so that the nitrogen (blue), oxygen (red), and iron (orange) have a chance to pop out
I didn’t get exactly what I was imagining, but for half an hour of playing with some command line arguments, I’m pleasantly surprised by the results. I call it Stairway to Heme, and challenge you to try it for yourself!