Project

# Title Team Members TA Documents Sponsor
59 Virtual Synthesizer using MIDI Keyboard
 Connor Barker
Dylan Pokorny
Patrick Ptasznik
 Eric Tang design_document1.pdf
final_paper1.pdf
proposal1.pdf
video
# Virtual Synthesizer using MIDI Keyboard

Team Members:
- Connor Barker (cbarker4)
- Patrick Ptasznik (pptas2)
- Dylan Pokorny (dylangp2)

# Problem
The high cost of professional-grade virtual studio technology (VSTs) and digital audio workstations (DAWs) presents a significant barrier to entry for aspiring music producers. Many individuals, specifically those just starting out, lack the financial resources to gather the necessary equipment, limiting their ability to explore the world of music. This project aims to address this problem by creating an affordable, standalone hardware synthesizer that replicates VST functionality, making music production more accessible for the average music enthusiast.


# Solution
This project aims to create a low-cost hardware synthesizer, making music production accessible to a wider audience. The design centers around an ESP32-S3 microcontroller which acts as the brain, which processes input from a MIDI keyboard to generate sounds through speakers. Power is supplied through a wall adapter and a buck converter to ensure proper voltage levels for all components. The generated audio is then outputted to a speaker for real-time sound production. A user interface consisting of a potentiometer for volume control and buttons for instrument selection along with an LCD screen for displaying information such as wave type allows for intuitive interaction. This standalone device bypasses the need for a computer and complex software, significantly reducing the financial cost.



# Solution Components
## Subsystem 1: Microcontroller / Software (ESP32)
We plan on using the ESP32-S3 microcontroller because it has a few main features that greatly help our project. First, it has USB hosting that can turn some of its GPIO pins into pins that support reading directly from our midi keyboard’s usb port with a USB adapter. Additionally, it has multiple I2C ports for us to connect to the LCD screen as well as multiple I2S ports that can output audio data to the speaker. Finally, it is powerful with more cores than some of its counterparts that can allow multiple processes to be running when looking for user input and generating the sound wave (sine, saw, etc).

## Subsystem 2: Power (wall outlet)
We will draw power from an AC 120V 60Hz wall outlet using an adapter to convert it to DC 5V. The DC 5V supply will power the LCD screen and the MIDI keyboard through a USB-A adapter. We will use a buck converter to step down the voltage to 3.3V for use with the microcontroller and speakers.
Wall outlet adapter 120V 60Hz AC to 5V DC
Buck converter components
Transistors
Diode
Capacitors
Inductors

## Subsystem 3: Speakers

We plan on using a 4-Ohm speaker with a power range between 3-5 Watts in order to have sufficient sound quality while limiting power demand. We have listed some example options below at different ranges of power.


## Subsystem 4: User Control

Volume control will be handled by a potentiometer that connects to a GPIO pin on the ESP that will control the output signal in software. Additionally, we will use simple mechanical buttons connected to GPIO pins so the user can cycle through available instruments.


## Subsystem 5: LCD screen

We will display the name of the sound currently selected on an LCD screen controlled by the microcontroller through I2C protocol. The display will visually assist users in selecting the sound they want to play.



## Subsystem 6: MIDI Keyboard

We plan on using the MIDIPLUS AKM320 as our piano input that outputs midi data via USB type A. We will have a USB-A connector that splits into VCC, D+ , D-, and GND so that we can use the USB host mode to then connect to the ESP32.


# Criterion For Success


The synthesizer will be capable of switching between several sounds of different waveforms that are clearly distinguishable.

Volume control potentiometer can raise and lower the volume of speaker output in a continuous manner.

Supports multiple notes being played at the same time (Chords).

Supports a range of at least 3 octaves of notes to be output on speaker.

Sound features must be adjustable in real time, as the synthesizer is in use.



# Resources & Citations

**Microcontroller / Software (ESP32)**:
https://www.amazon.com/Espressif-ESP32-DevKitC-VE-Development-Board/dp/B087TNPQCV?source=ps-sl-shoppingads-lpcontext&ref_=fplfs&smid=A33XZ36WFNH796&gQT=2&th=1

LCD Screen: https://www.amazon.com/GeeekPi-Character-Backlight-Raspberry-Electrical/dp/B07S7PJYM6?crid=3NFE1JY7T1MDW&dib=eyJ2IjoiMSJ9.3LG-rdQyBtOaCRNH2P5W1gbZ0fmHmFZQ9pHUMksSeyRTMIO-_dFWjwM5dELoTud6V_NowIFGdGGkOcVWORnhcPIu2jGzKywg_-0sluGTvejwLetYOb44z6zOB2wjYhh4r2w7umgCugyzyDLOEyJa7JYFfm7lbD0HnLQN4wgbOWSkLDwhAqS8Z-__CkpfdozsjuaDIEInA5Z64L0Wzp20CMMDfx2oz_9hkgdhBOMHWaebiTp2HxdOnCEikWO_XFQDGeQrIvo6K64-ZDbe0OmUf8RzQnFAAFKPXG6WEq2TYUoh3gfP9mySKIdCHB3rw4Zw3ff-yNT244T6Jo4X5fq-mbNkaL08CNzNgrmgK3ZBlu8.Pi6n6hRDZvfI_iccKXpOIpZVY0Q-vsD9BjD9otaEsJk&dib_tag=se&keywords=lcd+screen+i2s&qid=1738196187&s=electronics&sprefix=lcd+screen+i2%2Celectronics%2C142&sr=1-4

Speaker Options:
https://www.amazon.com/Gikfun-Speaker-Stereo-Loudspeaker-Arduino/dp/B01CHYIU26/ref=sr_1_5?crid=H3YZHZ7EW1LD&dib=eyJ2IjoiMSJ9.JxfX0DtoMc3EK4kMjWnChI0FreS6wWoy9zEvJmvhcjj-UTOBNjy4oEsL_4rq1b3hge0U0YyboxhnX-h-FQe3nFRhVbOICJDh88talb83w61MyBHqj9GONi-uylmW7PQ71P_gCSX2skcK4eX_s2fvjz5qMBYPI5kpEDOHIjXlPpaxd1TALGcSZdGKOupGIm7FhsglNMLOKX_jMSx3Y_OCDbvstR2fvILpAWEHm5uS7B0.XcpkmIU-GtrD8iRgeiyV2xOXJEMB9xLfhKBddBAjjQs&dib_tag=se&keywords=circuit+loudspeaker&qid=1738196030&sprefix=circuit+loudspeake%2Caps%2C109&sr=8-5

https://www.amazon.com/Gikfun-Speaker-Loudspeaker-Arduino-Replacement/dp/B081169PC5/ref=sr_1_1?crid=H3YZHZ7EW1LD&dib=eyJ2IjoiMSJ9.JxfX0DtoMc3EK4kMjWnChI0FreS6wWoy9zEvJmvhcjj-UTOBNjy4oEsL_4rq1b3hge0U0YyboxhnX-h-FQe3nFRhVbOICJDh88talb83w61MyBHqj9GONi-uylmW7PQ71P_gCSX2skcK4eX_s2fvjz5qMBYPI5kpEDOHIjXlPpaxd1TALGcSZdGKOupGIm7FhsglNMLOKX_jMSx3Y_OCDbvstR2fvILpAWEHm5uS7B0.XcpkmIU-GtrD8iRgeiyV2xOXJEMB9xLfhKBddBAjjQs&dib_tag=se&keywords=circuit+loudspeaker&qid=1738196030&sprefix=circuit+loudspeake%2Caps%2C109&sr=8-1

Power Adapter, 120V 60 Hz AC to 5V DC 15W:
https://www.amazon.com/MTDZKJG-Adapter-100V-240V-Transformer-Security/dp/B0BZP65GRW/ref=sr_1_8?dib=eyJ2IjoiMSJ9.lt4Dgb27bTajkIeDcd8swsiOjzJ1W2QmIfdBQ7_ahaAwoZQW7WZT5-8AAq5eO-U3gPg7JLb7gG5ApYMsSGhn1URvtswbMboxyXNguxbZp9x8vo-XKVhFeYR718fDVvqt5pq8Fm69GqbQcccbft7M2FIN5mx-wSvo81yy8O-vkdiITNwAqmRbwcdA-aLqEeghpkxNBbo6j4YeaQV-XAnYrKYwaAvx15HuXzDKm35MaTMQN0lhteHusMF8TQp_oZvaKlfOphY4AJMI20KQTlm8nyCyNAt7phcz6irY1BdM-99ZCwEv2LpjeK-jcJOBBF26QSp5H0I9qG4lq_Mb6l-NVVxCE_5YrAUNsBm5j_fXqy0.YoKiwCFxh_6txGCrj5XQvP6w7R17ZPkm87osANvsZfw&dib_tag=se&keywords=120v+to+5v&qid=1738197561&s=electronics&sr=1-8


MIDI Keyboard:
https://www.amazon.com/midiplus-32-Key-Midi-Controller-AKM320/dp/B00VHKMK64/ref=sr_1_1?crid=19Z5UVHJGE6MO&dib=eyJ2IjoiMSJ9.B3fOhHaP4O1-06iJF-1ObLtOnDzngOUeP1gjPnLKux8F-oCAti98qP5_9hxSh3xXi34fWhRQLeZFHMQQtj_HZiJxdDVbdczE6f6u6-TvAaCz6bvXD1t2vbNnFTN-Nf2NWRaVr5BM8IWNMJDoqouDdxHyRDn9abehbUR-an58-oj5K5mOA1opEmGjvoHeit2b04v9ehE0842C8DKo0yppB4qpp3icjy5IgsC1RDlcbvXs_GCHzerrx2XiPcJwtzhOk5-6MWAZ8YB0vf7lO62AhQQJpIF0Vcm019Jpt_I3D6bAR2DTWmNdikYfCFw4z-5Kb9EcRF49MTHNKLxTwHV0zzqfnjJd2pOaz5LzexPNCbjTz3b32f9KCotyeP5L_s5lHni3peR32R6jAi2IWb24NM304vJ0_cjZLNlbY-uAb_2cYIluJ7ljKLcFs6-q1_P9.2k1JoRB3bVdFtLBBRn1p1PAaxmC4y8WTYLLVdzy9kKA&dib_tag=se&keywords=midiplus%2Bakm320%2Busb%2Bmidi%2Bkeyboard%2Bcontroller%2C%2Bblack%2C%2B32-key&qid=1738199055&sprefix=%2Caps%2C102&sr=8-1&th=1

USB 2.0 Type A Female SMT Connector:
https://www.mouser.com/ProductDetail/TE-Connectivity/292303-7?qs=e6gk%2FTaAuqWZCg5WWmtijA%3D%3D

Iron Man Mouse

Jeff Chang, Yayati Pahuja, Zhiyuan Yang

Featured Project

# Problem:

Being an ECE student means that there is a high chance we are gonna sit in front of a computer for the majority of the day, especially during COVID times. This situation may lead to neck and lower back issues due to a long time of sedentary lifestyle. Therefore, it would be beneficial for us to get up and stretch for a while every now and then. However, exercising for a bit may distract us from working or studying and it might take some time to refocus. To control mice using our arm movements or hand gestures would be a way to enable us to get up and work at the same time. It is similar to the movie Iron Man when Tony Stark is working but without the hologram.

# Solution Overview:

The device would have a wrist band portion that acts as the tracker of the mouse pointer (implemented by accelerometer and perhaps optical sensors). A set of 3 finger cots with gyroscope or accelerometer are attached to the wrist band. These sensors as a whole would send data to a black box device (connected to the computer by USB) via bluetooth. The box would contain circuits to compute these translational/rotational data to imitate a mouse or trackpad movements with possible custom operation. Alternatively, we could have the wristband connected to a PC by bluetooth. In this case, a device driver on the OS is needed for the project to work.

# Solution Components:

Sensors (finger cots and wrist band):

1. 3-axis accelerometer attached to the wrist band portion of the device to collect translational movement (for mouse cursor tracking)

2. gyroscope attached to 3 finger cots portion to collect angular motion when user bend their fingers in different angles (for different clicking/zoom-in/etc operations)

3. (optional) optical sensors to help with accuracy if the accelerometer is not accurate enough. We could have infrared emitters set up around the screen and optical sensors on the wristband to help pinpoint cursor location.

4. (optional) flex sensors could also be used for finger cots to perform clicks in case the gyroscope proves to be inaccurate.

Power:

Lithium-ion battery with USB charging

Transmitter component:

1. A microcontroller to pre-process the data received from the 4 sensors. It can sort of integrate and synchronize the data before transmitting it.

2. A bluetooth chip that transmits the data to either the blackbox or the PC directly.

Receiver component:

1. Plan A: A box plugged into USB-A on PC. It has a bluetooth chip to receive data from the wristband, and a microcontroller to process the data into USB human interface device signals.

2. Plan B: the wristband is directly connected to the PC and we develop a device driver on the PC to process the data.

# Criterion for Success:

1. Basic Functionalities supported (left click, right click, scroll, cursor movement)

2. Advanced Functionalities supported(zoom in/out, custom operations eg. volume control)

3. Performance (accuracy & response time)

4. Physical qualities (easy to wear, durable, and battery life)