LEARNING MUSICAL CONCEPTS THROUGH EXPLORATION AND CO-DISCOVERY

My master’s thesis - Piano Play. An augmented piano experience for kids.

The Concept

Inspired by my own passion for music and experience learning to play piano as a kid - a process that was dry, rote, and completely un-creative - Piano Play is a fun and playful way for kids to learn to play the piano. I wanted to create an experience that felt truly magical and inspired kids to want to learn about music and to create it.

The system combines projection mapping onto piano keys as well as onto the surface of the tangible interface - a magic book. The platform concept provides the opportunity for an entirely new learning experience; one that encourages play and exploration alone or in groups, and fosters musical creativity.

This project was a 16-week independent masters degree project completed with the help and collaboration of many, many inspiring and knowledgable people, and inspiration and feedback from 2 first grade classes in Umeå, Sweden.

Project Background

The piano is one of the most popular instruments, and is also well-suited for learning young. But the age at which piano education should begin is a widely debated subject. Many child development specialists agree that learning should be done primarily through play in early childhood, but piano is almost always taught using a rigid method that leads beginners through a progression of lessons that ultimately teach them to play the right note at the right time. This project aims to address these issues; inspiring children in early childhood but allowing them to learn through play.

There were many themes that I wanted to explore with this project. I've always been inspired by the intersection between nature and technology, and the blending of old an new. I wanted to see if it was possible to augment and old piano and make it enticing to kids in an digital age. I was also drawn to the idea of allowing kids to discover musical concepts through play - and especially through co-play. Can kids learn concepts like chords, major and minor keys and improvisation through a series of exploratory games? And perhaps more importantly, can kids be inspired and excited about making music if they are encouraged to play rather than to practice to play existing songs in a prescribed form.

During the development of this project, I interviewed many music teachers and game developers, as well as creators of existing piano learning software. I had brainstorm sessions with musicians who had learned music at a young age. And I worked closely with 2 first-grade classes testing several rounds of iterations and refining based on observing them play.

Alice Explains Piano play in her own words

Alice was a student in one of the classes that tested 3 rounds of piano play prototypes. She had never played any musical instruments before, and in this video, she invited her friend Hannah to try it out with her. This was Hannah's first time ever playing a piano.

Prototypes

This video shows the testing of 3 different iterations of prototypes with 2 first grade classes. Only a few of these kids had ever played a piano before.

Demo

Piano Play is platform on which you could have many games of increasing difficulty, but i ended up creating 5 games as a proof of concept and to demonstrate the philosophies of learning through exploration and co-discovery. Here is a demo of the games.

00 Getting started

Open the book to turn on the system. The keys turn into rainbows. Each player taps on a color to select it. Anytime that color appears, the player knows it’s their turn to play.

01 The Shuffle

Core concepts: Player 1 - The bassline : rhythm, learning by ear. Player 2 - The improviser : improvising to a beat, playing two hands together.

The boogie shuffle is a classic blues piano bassline. The system leads one player through this bassline by playing it once, then highlighting the keys for the second cycle. The bassline begins slow and speeds up with time. While one player tries to keep the beat, the other player can try to improvise along in rhythm on highlighted keys, in a key that matches the bassline.

Changes based on testing: The bassline began a bit fast and sped up a bit fast for the children to keep up to, so this was slowed down a bit. The player on the improvisation side often had trouble improvising, as in playing the notes in different orders and different rhythms. In an effort to inspire them, a video showing hands playing the piano was added to the screen. The thought was that if they children saw the fingers on these hands playing at different times and in different orders, then maybe they would try to do the same. In an effort to encourage them to experiment with tempo rhythm variations, the speed of the video plays relative to the speed at which the child is playing. Further changes included moving the highlighted keys closer together on the improviser’s side, using the player’s colors and switching roles after each cycle, and beginning the bassline part with a single key, and adding a second key later.

02 The Elephant and the Bird

Core concepts: High/low notes, dynamics, guided improvisation, major / minor keys

This game is a conversations between and elephant and a bird who speak to each other in musical notes. The elephant speaks in low minor keys and the bird speaks in high major keys. At the start of the game, five keys are highlighted on the low side of the keyboard in the color that one of the players has chosen. There is nothing saying that the player has to press those keys, but if they do, then the notes will sound good together because they are all in the same key. As keys are pressed on that side of the keyboard, a note appears in the elephant’s speech bubble and both note and bubble grow in size relative to the loudness of the note played. After a certain number of notes have been played on that half of the keyboard, notes in a certain major key on the high side of the keyboard are highlighted in the other player’s color. The last key on each side of the keyboard glows white. Pressing either key toggles on or off a simple drum beat.

Changes based on testing: Highlighted keys are now closer together so they can be played by a single hand in a static hand position. In order to get the children comfortable playing on both white and black keys, the keys in the first scale highlighted are all on white notes, in the second round, they are all on black notes, and after that they use a combination of both.

03 Broken Chords

Core concepts: Playing with all fingers, alternating hands, playing on different places on the keyboard, turn-taking.

The game: The notes of a certain chord are highlighted from one side of the keyboard to the other in one player’s color. Two hand outlines stretch across three notes each. The notes can be ‘turned off’ as the keys are pressed in order beginning on the low end of the keyboard. After the first three notes are turned off, that hand outline crosses over the other one onto the next three notes.

Changes based on testing: This game was really well-received by the children; they didn’t know what to do at first but once they figured out the pattern, quite a few of them giggled or smiled. They also seemed to really enjoy lining up their hands with the outlines. So the only thing that was changed was the addition of a video on the book surface with hands playing broken chords up a keyboard.

04 Meerkat Strut

Core concepts: rhythm, learning a new song, dynamics.

This is the simples of all the games, and is not creative in any way, but the children enjoyed it because it got them playing a melody right away. A simple beat is played, and a single key is highlighted on the keyboard. When that key is pressed, another key lights up. At first the highlighted keys alternate between the two players, then both at the same time, and eventually two keys at a time for each player. If the highlighted keys are followed, then a song is played. In this game the song is Stray Cat Strut, a good, funky tune, but in theory it could be any song.

Changes based on testing: A second iteration of this game featured two meerkats on the screen, one in each player’s color, and whichever player plays a note, their meerkat jumps in the air at a height relative to the volume of the note played. Future versions might give some incentive for NOT jumping too high (or playing too loud).

05 Rainbow Notes

Core concepts: exploration of note names, key / note relationship, introduction to the musical staff, note-reading.

The game: A musical staff appears on the pages of the book with colored notes and a box around the notes furtherst to the left on the staff. As keys are pressed on the keyboard, they light us in the colors of the rainbow. The corresponding note appears on the staff in the same color and with it’s note name above or below it. If the note is played that matched the note on the staff, then the box slides over to the next note on the staff. If all the notes on the staff are played, then new ones appear, and a song can be learned. If the entire song is played, then the color is removed from the notes and keys. The note played still appears on the staff with the note name, but not color.

Changes based on testing: This game was adapted to work better for 2 players. Hand outlines were added in each players color that line up with the notes in the song. At first, either player can play the note on the staff and it will move forward, so there is no pressure on either player, but later in the game, each player gets their own solos to play one at a time. As well, the hand prints are switched part of the way through the game from the right hand to the left hand so the children can get used to playing with either hand.

Recording

Any game can be recorded. Simply flip over the recording tab on the edge of the book and the recorder will fill up. Flip the play tab to hear your creation!