Augmented reality brings sheet music to life

Augmented reality awakens musicians’ drawings on sheet music, giving a sneak peek of upcoming performances at the Latvian National Opera and Ballet.


1. Get Overly app for mobile


2. Scan this image with the Overly app

3. Experience magic

Scan this image with the free Overly app

Project overview

Overly app was recruited to deliver a super-cool augmented reality idea courtesy of the Latvian National Opera and Ballet (LNOB). Turns out that in-between the production rehearsals, some orchestra musicians release their creativity through sketching on their music score. Artists draw caricatures, write jokes or illustrate operas and ballets.

The team running the sheet music storage and distribution library at the LNOB had put aside all scores that had once returned to them featuring some new looks. With the help of the Overly app, some of the characters from musicians’ drawings were brought to life to launch the 101st season of the LNOB.

Results: multiple creative uses for the same markers

In total seven pieces of sheet music were turned into AR markers with embedded animations showcasing the upcoming repertoire. To launch the campaign, the augmented reality music sheets were turned into human-size banners and placed outside the opera and ballet house for a month, as well as utilized as bus stop advertisements across the capital of Latvia, Riga. The same markers were also a key element of the LNOB’s annual opera and ballet book, which is available throughout the year.

Touching all senses

An evening at the opera or the ballet is an experience for all the senses and it is difficult to be captured in just a photograph or traditional media advertisement. It is well suited for the LNOB to awaken motion on sound when sharing the upcoming performances, tells Sanita Duka, Head of Advertising and Print at Latvian National Opera Ballet. ‘If our advertisements come to life, play music and move around, people are given a true sense of what they are signing up for when coming to one of our productions.”

Augmented reality outdoor banners outside the Latvian National Opera and Ballet

Conducting music with augmented reality

“Notes are just pieces of paper with writing on them when sheet music lies on a shelf. They only become music when the score is in the hands of musicians. Through scanning the sheet music with the Overly app, our visitors can experience exactly how it is to bring notes to life and create a musical experience. And they can do it by using something they have in their hands every day — their mobile devices,” says Sanita.

Other case studies

Augmented reality wedding invites wow guests

Augmented reality aids 5G storytelling efforts

Augmented reality art exhibition brings children’s poems to life