Project
| # | Title | Team Members | TA | Documents | Sponsor |
|---|---|---|---|---|---|
| 98 | Real Time Piano Input Visualizer For Learning |
Jay Park Nuwan Singhal Sarayu Suresh |
Wenjing Song | proposal1.pdf |
|
| Team Members: - Nuwan Singhal (nuwans2) - Jay Park (jaypark3) - Sarayu Suresh (sarayus2) # Problem Learning to play the piano, especially if self-taught, comes with many difficulties. Two of the main ones are learning how to read sheet music, as well as knowing if your timing is accurate. These hurdles can be difficult to overcome, and lead to people giving up as they have to put in a lot of preparation before they can start playing songs. # Solution Create a hardware solution that controls an RGB LED Matrix which responds to MIDI input from a piano. This can be used to learn songs by preloading data through an SD card and having it so visual cues tell the user when to press keys and which keys to press, waiting for users to press the correct key before moving on. Other features such as controlling the speed of the song and working with only one hand can be used to incrementally learn. It could also be used when teaching piano by instead outputting what key is currently being pressed, allowing students to have a better understanding of what their teacher is playing. # Solution Components ## Subsystem 1 - Led Matrix board to display which keys to press and user interface The LED board shows which keys should be pressed by the user who is trying to learn how to play the song. It lights up when the user needs to press the key for playing the song that is stored in the SD card. There can be multiple modes for playing the music as well as options related to speed and hand used. We plan on using the RGB LED Matrix 1528-2094-ND as it offers multiple colours and enough space to display a few octaves of the piano. We plan on designing this project to mainly work with 4 octaves as those are the most commonly used ones, but with this LED Matrix, extending it to more octaves remains as an option. For the MCU, we found that a STM32F446RET6 would be a possible option mainly due to the high speed which is needed to keep the LED Matrix persistent for vision without major flickering. Additionally, we already have a Nucleo-F446RE, which means it can be used as a dev board and part of it can later be used as an ST-Link. ## Subsystem 2 - SD card for storing and loading songs Our PCB should have an SD card reader which will store MIDI files of various songs which can be read by the microcontroller and loaded onto the LED board so that users can play along visually to learn the track. We can use Micro SD Card Reader Module TS-891 for the SD card Reader, and we can use Sandisk ImageMate Sdxc Flash Memory for storing songs as MIDI Files. ## Subsystem 3 - USB input for reading and registering piano input in real time The keyboard sends messages whenever a key is pressed or released. The microcontroller reads these messages and extracts the note number and timing information. This data is compared with the expected notes from the SD card to determine whether the user played the correct key. We can use a MIDI Jack (SDS-50J) to take in Midi input from a piano, this is a better option than using USB A since that will require our microcontroller to act as a USB host, while using MIDI input directly uses UART which is all we need for our use case. ## Subsystem 4 - User interface to control the system Users need to be able to control certain parts of the system such as what song to play, which mode to operate in, the speed of the playback and actually starting and turning on and off the system. We will use external buttons and switches for these parts of the system as well as the LED matrix to display text related to the interface. We can use simple buttons since the main controls we need are ‘left’, ‘right’ and ‘select’. (Omrom B3F-4055) ## Subsystem 5 - Power Management Outlet power supply can be used for our project to power up our project. We can convert AC current to DC current using an external 5V adapter barrel plug that we used in previous labs to convert mains to 5V DC. We can use a low dropout regulator (AMS1117-3.3) to convert it from 5V to 3.3V for the MCU. The LED Matrix we plan to use uses 5 Volts, so no converter is necessary for that, the LED Matrix uses bare wires to get powered, so we could use a 2 input screw terminal block (TB002-500-02BE). For the data of the LED board, we need a 16 pin header (900-0702461602-ND). # Criterion For Success The system successfully loads a song from the SD card and begins playback. The LED board correctly displays which piano keys should be pressed for the selected song. A user interface allows users to interact with the system and choose the song they wish to play as well as details such as the speed, which hand and switch from input to output mode. USB MIDI input accurately detects pressed keys in real time and matches them to expected notes. Input detects the length of the time that the key was pressed. Also displays multiple notes when multiple notes need to be pressed because of the song. The system measures timing differences between expected notes and user input. The system should have the option to wait for the user to press the correct key before moving forward with the song The system should have options to control the speed of playback of a song At the end of a song, the system reports basic performance metrics such as number of correct notes and average timing error. # Alternatives Currently three main similar solutions exist. The first of which are software solutions like Synthesia, but these require an internet connection, a smart device which runs supported operating systems and the software itself costs money. Our solution is a separate device from phones or laptops making it more accessible to younger and older people, more affordable, and not requiring an internet connection to function. The other solution is a one dimensional LED strip that sits on piano keys, such as “The ONE Piano Hi-Lite”, these solutions also require a smart device to function, but more importantly only offer one dimensional lighting which means that users see which keys to press in advance which is an important feature for harder songs and for learning the timing. The last option would be products similar to the “PopuPiano” which is essentially a combination of a led strip and a piano, but we aim to offer a separate device that piano owners can use rather than a piano itself. Also this solution comes with many of the same drawbacks as the other two. |
|||||