It is the invisible sound of music that people have produced and listened to since the beginning of time. But have you ever wondered what it would look like if music manifested itself physically? What strokes and landscapes would musicians create if their instruments created illustrations? “Stories in Sounds” is a stunning contemporary book that answers these questions and allows its readers to immerse themselves in the visual experience of making music.
Created to celebrate the centenary of JVLMA, the book features interviews with people who make up the institution as it is today – its students, tutors and graduates. Organized in a number of chapters, eight sections feature an abstract drawing at the start. On closer inspection, each illustration turns out to be a creative merging of paint, musicians and musical instruments. An augmented reality overlay allows the readers to explore the-making-of each artwork.
Released in a set number of copies, “Stories in Sounds” is a gift to the institution’s personnel, closest supporters, friends and international partners. It was launched at a special event in the final stretch of the academy’s centenary celebrations. The event was attended by 60 people, and the eight AR markers featured within the book and exhibited as large format artworks at JVLMA have received more than 600 scans since its launch.
It took two years from the initial concept to the launch of the book, and it is an excellent example of how to allow the process to guide, and inspire you, reveals “Stories in Sounds” designer, Maija Rozenfelde. “The key design and content ideas were clear from the beginning, we knew we were not going to chronologically look through the academy’s historic milestones. We instead wanted to capture today and look to the future. We also wanted to reveal the present through personal stories that represent diversity and individuality within the institution.
“It was clear from the get-go that the book would feature intimate portraits of the interviewees, revealing their characteristics and uniqueness. The other must-have was the visual capture of the moments when the sound is created. Sound is our sense of experiencing music, but it doesn’t allow us to understand what work goes into creating it. I wanted to reveal the physical aspects of the melodies we hear. But the question was – how do we capture something we cannot see?” reflects Maija.
As the text for the book was coming together, Maija came up with an experiment of using paint to capture the creation of sound. “The rector and his team put great trust in this idea as we decided to start each chapter with these captures, but no one knew what the end result was going to look like. I dressed up students in painter’s wear, covered their instruments with paper, provided paint, and musicians created music while splattering color. Giving into the unknown was the success factor to it all as the artworks came out so interesting that it was hard to choose the ones to feature within the book,” she tells.
The musical paintings created throughout the experiments now mark the start of each new chapter of the book. But while the design looks effective, the paintings on their own do not reveal the creative process, which is where augmented reality came into the picture, explains Maija. “Although there is a description of the process at the end of the book, it still doesn’t quite showcase what the musicians did to create the illustrations. We did after all tape felt-tip pens to their fingers, dipped their hands in paint and that’s just some of the creative work. It isn’t something you just write up, you have to see it to believe it. Luckily, we had captured the process in videos for reference. The next step was to figure out how to incorporate these videos in line with the book launch.
“Of course, we could’ve shown a short video at the launch event, but it would really narrow down the audience that would know about this. Another option was social media, but still, the audience wouldn’t be so relevant. And then I remembered Overly and the picture was completed in my head. We already had the videos, we just had to bring the illustrations to life and complete the story. Now everyone who owns the book can use the Overly app to scan the illustrations and see how they were created.”
“It was at the book’s launch event when we finally saw how people take to the idea. Augmented reality was greatly received by the public and proved to add great value to “Stories in Sounds”. I think a prerequisite of effective design today is interaction — the freedom for people to participate in a given experience. Augmented reality provides this opportunity,” adds Maija.
