Science Gets Down With Miles Davis and Bernini

This month, in our article, “Literature by the Numbers,” we introduced you to scholars using digital tools to uncover fresh historical and critical insights into fiction, poetry, and plays. To the chagrin of their old-school colleagues, the IBM Watson-era scholars are showing how computer analysis can uncover new meanings in the works of masters like Shakespeare. Digital research, though, doesn’t stop with literature. This week we reached out to scholars in music and sculpture for further insights into how digital tools illuminate art, music, and creativity.

Anna Jordanous, research associate at the Center for e-Research at King’s College London, explained how she programmed a computer “to be creative” and “compose” music. Tony Sigel, conservator of objects and sculpture at the Straus Center for Conservation and Technical Studies, the conservation wing of Harvard University’s art museums, informed us how science is shining new light on Italian artist Gian Lorenzo Bernini, whose magnificent sculptures, such as The Fountain of the Four Rivers in Piazza Navona, have enraptured anybody who has been to Rome.

How can a computer be programmed to be creative?

Jordanous: One approach to creative computing is to give a computer a whole set of rules to follow, and parameters for what is acceptable and what is not. When we are learning to be creative with music, we learn rules. We play according to musical theory, which is a set 

behavior in. Instead you feed in all sorts of tiny things, small bits of information that create certain conditions, and then a much more sophisticated behavior can emerge out of that. You aren’t telling them how to be creative, no one can just tell someone or something that, but you are telling them good conditions or behaviors for creativity.

How did you develop computer protocols for creativity?

Jordanous: To get the protocols I needed, I did a computational linguistics collaborative analysis with Bill Keller, a computational linguist at Sussex. We analyzed articles on creativity published in English over the past 60 years. We looked at what words kept coming up in the context of creativity, and got 14 different themes that appeared to be important semantically for creativity. To turn that into something useful for music, I spoke to a collection of people, about 30, all involved in music, and asked them what did creativity mean to them, when improvising music. I analyzed what they said according to the 14 themes we had already generated, noticing which were most relevant for music. That gave me an idea of what was going to be important in creating a “good” piece of music, and thus I could set a standard on my improvisation program.

Then what did you do?

Jordanous: I use a thing called “genetic algorithms.” Basically, I would have the computer generate music, pick what was best, and then have it generate more music on the basis of the ones I had already evaluated to be more creative than the others. This would then be repeated, and every time you get through the process, you refine the music. Eventually the music becomes more and more creative. The program I used generated what can only best be described as random sounding noise, but it generated noise that had obviously been fine-tuned by other things. I had different parameters that I had set on my improviser. Different ranges of notes, different harmonies. The program started as random horrible nonsense, but by the end of the many iterations I put it through to keep honing the composition, it eventually generated something that was vaguely listenable to.

So the computer is “creating” its own music?

Jordanous: Right. There have been examples of compositions that have been done in electronic music, for example, which are all algorithm-based, and have the computer acting as a creative force. And just as we don’t have a set of rules to being creative, the fun of my model is getting a computer to be creative and not give it a set of rules. That opens up a lot of things in terms of how we experience creativity, but it opens up our own creativity as well in the sense that we can learn from the process and learn from the computers’ mistakes. We can see the steps and stages that the creative process goes through, and then analyze that. When a symphony is drafted, it goes through many different draft versions. Well, imagine this is a kind of computational draft and redraft process.

You play the flute, saxophone, and piano. Has computational analysis helped you as a musician?

Jordanous: Oh, yes. In my research, I had to ask myself: How on earth do you justify saying this computer is more creative than that one? I realized the generated noises that were truly successful were those that took into account other sounds and reacted to them. So understanding what made one system better than another, or more creative than another, was to realize the importance of communication. That made me think a lot about the communication aspect of improvisation. Personally, when I improvise, I think about scales, chords, or the theme of the music, not the people playing around me, or the audience I am playing to. Through the computational model, I found that when you listen to what is going on around you, listen to the others playing with you, it really opens up doors as far as your ability goes. I wouldn’t say it has made me Miles Davis, but it has definitely helped!

As an art conservator, what are some of the scientific tools of your trade?

Sigel: One is hand-held X-ray fluorescence spectrometry. It became more available and affordable following 9/11. Homeland Security needed them for on-the-spot analysis at airports. 

brightness, and sharpen features allows us to enhance details less visible than X-rays on film. For instance, joins in the clay can be seen from air trapped between pieces. This lets you see if a head, arm, or other feature was added, or modeled from a larger piece of clay, and reveals an artist’s individual technique.

Infrared reflectography imaging is sensitive to carbon and illuminates anything drawn in ink, pencil, or charcoal underneath the paint, known as an underdrawing. It lets us see what the artist first drew on the blank wood panel or canvas, before painting over it. To get a sense of the chemical composition of the paint, there’s gas chromatography-mass spectrometry. Paint is made up of the binding material such as oil or egg tempera, and the pigment that gives it color. To get a sense of the chemical composition of the paint, there’s gas chromatography-mass spectrometry. 

Recently you’ve been studying Bernini’s clay models. Why are clay models important?

Sigel: These models are the preparatory sketches that sculptors made in inexpensive, easy to model clay before cutting into expensive, difficult to carve marble. In the 17th century there was no digital computer modeling, so the only way that an artist could conceptualize a three-dimensional object was to make one. For Bernini, who lived and worked in Rome, clay was readily available. It would take him from minutes to a few hours to model clay into the likeness of an angel, saint, or a cardinal. He would often make numerous versions before embarking on the real thing; over 20 clay sketches for a single marble were seen in his studio. Bernini’s prolific and creative use of models elevated their value. They began to be collected by other sculptors, patrons, and art collectors. Today they are invaluable “sketchbooks” of a great artist. To a technical art historian like myself, these small models reveal a great deal about how Bernini thought, approached art, and also the nuts and bolts of how he worked.

What kind of things did you learn?

Sigel: Bernini was the dominant artistic personality of his time, and his patrons were the Popes and the royalty of Europe. To meet demand for his work as his fame grew, Bernini established a large workshop of the most talented sculptors in Rome. Besides making models for his own art, Bernini needed a means to pass detailed instructions to his assistants. The models served that purpose, and the marks that give proportional measurements for the full-size marble still remain in the clay.

Having now examined more than 70 terracottas that are attributed to Bernini, I’ve come to understand his personal modeling techniques—and how to differentiate him from others in his circle. He “built” his sculptures, adding limbs, drapery, wings, and other features to a central core of clay. He detailed the faces with particular tools making specific gestures, and used his fingers to mold and pinch the clay in repeated, characteristic ways. He did not hollow his sketch models, and rather than using an internal framework known as an armature, as other sculptors did, he used wood props and formed temporary buttresses of clay behind his figures to hold them up while drying. This is an indication that he was a virtuoso with little patience for details in what he considered a disposable sketch. 

What were some of the other ways you established Bernini’s authorship?

Sigel: Because there are many fingerprints left in the clay, I initiated a campaign of fingerprint analysis. From the 70-odd clay models I studied, 100 fingerprints were collected. Matches were found, confirming Bernini’s authorship of five works, some of which that had previously been in doubt. Attribution is important to art historical scholarship—it lets us see how a sculptor evolved as an artist, and establish authenticity and value.

What else did the X-rays show?

Sigel: As clay shrinks nearly 10 percent as it dries, the smaller arms and legs on a model would dry much faster than the larger torso, thus forming cracks at the shoulders and hips. The X-rays showed that Bernini usually did not bother to hollow out his clay models as other sculptors did, allowing his to develop cracks during drying. This suggests that Bernini was less concerned with the preservation of these rough sketches. He considered them disposable, and this tells us something about his artistic spirit.

How, ultimately, do the tools of science serve Bernini’s art—and our appreciation of it?

Sigel: These tools, coupled with close visual examination, help us understand how these works were made, of what materials, and how best to preserve them for the future. Combined with art historical scholarship, they can help to reveal something of the intentions and personality of the artist.

Get the Nautilus newsletter

Cutting-edge science, unraveled by the very brightest living thinkers.

NL – In Page Mobile

Email: *

Captcha

Sign up for free