Stairway to Heme

Nowadays, it’s even easier to get started with machine learning and art. The Martinez Group recently had an internal competition using the neural_style package and a pre-trained VGG net to transfer art styles onto pictures of molecules we are studying.

Myoglobin, with secondary structure (dark gray) and heme group (center) emphasized.

I chose this image of myoglobin I generated for a benchmark paper I am writing as it is one of the most composed molecular images I have generated so far. The actual structure is shown in the transparent ball-and-stick style, with the heme group (an iron atom in a porphyrin ring) displayed as thicker and more opaque. Due to the size of proteins, it is common to visualize a cartoon-y ribbon style that shows secondary structure, such as the various alpha helices shown in dark gray.

Relativity by M.C. Escher (1953)

I wouldn’t say I know a lot about art or art history, but you don’t need an encyclopedic knowledge of artists and art styles to try and make something cool. In this case, the ribbon representation of the alpha helices reminded me of the stairs looping around in Relativity by M.C. Escher. I thought it would be cool if the stair pattern could be imprinted into the alpha helices of the myoglobin, so I chose this piece for the style.

In the end, the net was fairly simple to run, assuming you have Anaconda installed on a machine with a GPU and are somewhat comfortable on the terminal:

git clone
conda create -n neural_style
conda activate neural_style
cd neural-style
pip install -r requirements.txt
curl -o imagenet-vgg-verydeep-19.mat

python --content myoglobin.png --styles escher.jpg --content-weight-blend 0.2 --pooling avg --style-scales 0.2 --preserve-colors --output stairway_to_heme.png

Only the content, styles, and output keywords are needed for the net to run. However, I played with a few other keywords to get something closer to what I was looking for:

  • I lowered content-weight-blend to transfer more of the style since I didn’t care about the fine detail structure of the myoglobin
  • I lowered style-scales to try and match the size of the stairs in Relativity with the alpha helices
  • I switched pooling from max to avg to go for a more abstract look
  • I used preserve-colors so that the nitrogen (blue), oxygen (red), and iron (orange) have a chance to pop out

I didn’t get exactly what I was imagining, but for half an hour of playing with some command line arguments, I’m pleasantly surprised by the results. I call it Stairway to Heme, and challenge you to try it for yourself!

Stairway to Heme
%d bloggers like this: