Using math to study a masterpiece

Combining the mathematics of digital image processing with the history, craftsmanship and science of art conservation, the Bass Connections Image Processing Algorithms for Art Conservation team spent last year working with the NC Museum of Art to study, restore and exhibit a 14th-century altarpiece that hadn't been displayed in its entirety for over a century.

Crucial to this project's success was the work of Rachel Yin, a doctoral student in applied mathematics, who helped develop an image processing algorithm that removed the artifacts of previous preservation efforts from X-ray images to allow contemporary conservators to better see and study the intricacies of the painting's texture, detail and age.

Using Yin's algorithm, the team developed and deployed a new image processing software, Platypus, which is now being used by a growing community of art conservators to help them preserve and restore works of art.

Yin reflects on tackling her first collaborative applied math project at the graduate level, making friends with collaborators in the arts and humanities and the importance of this Bass Connections project to her own research goals.

What drew you to this project?

My doctoral supervisor, Ingrid Daubechies, introduced me to this project. She has many good connections with art conservators at the NC Museum of Art and was very interested in collaborating with them on an art conservation project. My interest is in general image processing, so we thought there might be some very fruitful connections between our work and that of art conservators at the museum.

Can you describe the other team members?

The Platypus team included Ingrid Daubechies, who supervised my work as well as the work of a postdoc in Belgium, who was responsible for connections with art conservators in Europe. We also had another graduate student in Belgium who helped us do some software development. Also, there were a couple of undergrads who worked on color remapping and rejuvenation of the paintings.

On the North Carolina Museum of Art side, we had conservator Noelle Ocon. She worked very closely with the team to help us understand the exact problems and issues we needed to look into, and she helped with the eventual deployment of the image processing software.

What was your role on the team?

In this project, I worked on the cradle removal on X-ray images of panel paintings. Eighteenth-century conservators used a technique called cradling to preserve old European panel paintings. Cradles of wood were attached to the back of many panel paintings to stabilize or reinforce the surface of the work.

Typically when you visit the museum you don't see these cradles because they're on the back of the artwork, but if you take X-ray images of those panel paintings, they show up in the X-ray.

The problem is that these cradle artifacts are blocking the sight of conservators, who are more interested in seeing and studying the structure of the paint and the other details of the panel painting itself, not the cradling. So, we wanted to see if we could remove those artifacts from the X-ray images to aid in the conservation process.

My contribution was to design an algorithm that automatically detected those cradle artifacts in the X-ray images and separated them from the original raw X-ray image. That way, the art conservators could have a cradle-free X-ray image to work with. It is much easier for art conservators to read X-ray images after these artifacts are removed.

Noelle Ocon from the NCMA then helped us take this algorithm and deploy it to art conservators through a software package, so anyone would be able to get their hands on it.

Why did your team work on this specific piece of art?

The St. John Altarpiece, which was created in the 14th century by Francescuccio Ghissi, has a very interesting history. There is a missing panel in this set of panel paintings. That one missing panel was reconstructed by contemporary artist and conservator Charlotte Caspers so the museum could display the entire panel painting together. Usually it is displayed in pieces at different museums, so this was a unique event.

Charlotte is a very good friend of Professor Ingrid Daubechies, and later, she and I became good friends as well. So, I have some special connections to this panel, and I really wanted to figure out how to help the artist and art conservators better reconstruct the missing panel and study the existing panels.

How did this project connect to your own research?

This collaboration with art conservators was very unique for me in that it's an interdisciplinary project. Usually I'm not looking into images related to the arts, so the mission of this project was very exciting for me. What I found was that there are a lot of interesting math problems associated with art conservation, which is why I became very interested in this project research-wise. I figured out that I can apply and develop new mathematical models to help art conservators better understand and better manipulate art images to serve conservation purposes. So, this was a very valuable project for me in terms of understanding what applied mathematics can do in the real world.

What was the most surprising thing about your experience?

The most surprising thing for me was figuring out that I could collaborate not only with people outside my discipline but even with people in humanities. This was a very unusual experience for me, because people in science and arts often think about things in very different ways. Figuring out efficient ways to communicate with our collaborators was pretty unexpected, but I developed very good friendships with art conservators and every collaborator we had in the project. That was the surprising extra gift I got from this project.

How has this experience played into your future goals?

I think this project has a unique position in my graduate studies. It helped me realize that in real life problems there is amazing math to be explored. This gets me very excited. Although later on I'll move more to the theoretical side of image processing, the applications of this project were important and motivating for me. I really want to do something that is applicable and valuable and has real-world use.

I think Bass Connections provides unique opportunities for students and faculty to hit the ground running with really interesting interdisciplinary projects that they might not get to do otherwise. The collaborative environment promotes fruitful results and real-world applications.