Project
| # | Title | Team Members | TA | Documents | Sponsor |
|---|---|---|---|---|---|
| 20 | Air Guitar |
Arturo Arroyo Valencia Miaomiao Jin Youngmin Jeon |
Eric Tang | design_document1.pdf final_paper1.pdf presentation1.pptx proposal1.pdf video |
|
| # Title Team Members: - Miaomiao Jin (mj47) - Youngmin Jeon (yj21) - Arturo Arroyo Valencia (aarro6) # Problem Traditional guitars are bulky and non-portable, making it difficult for musicians to practice or perform in mobile environments. While software-based "virtual guitars" exist, they lack the tactile "muscle memory" of fretting with one hand and strumming with the other. There is a need for a wearable system that captures the physical kinetics of guitar playing without the physical footprint of the instrument. # Solution Project: Air Guitar is a dual-wearable sensor system that mimics the ergonomics of a real guitar. The left hand captures "fretting" finger patterns to determine chords, while the right hand captures "strumming" velocity and timing. By fusing these two data streams wirelessly, the system generates real-time MIDI audio. The design focuses on low-latency wireless communication and precise gesture recognition, allowing the user to play music anywhere without being tethered to a physical instrument or a power outlet. # Solution Components ## Subsystem 1: The Left-Hand "Fret" Controller This subsystem identifies the chord the user is trying to play. It maps the curvature of each finger to a specific digital profile (e.g., specific bend angles = C Major). - Flex Sensors (4x) [P/N: FS-L-0054-103-ST]: These are long, thin strips placed along the fingers. As the user curls their fingers to form a chord shape, the resistance changes. We use these to measure the degree of flexion for each finger. - Voltage Divider Network: A series of precision resistors used to convert the changing resistance of the flex sensors into a measurable voltage that the microcontroller's ADC (Analog-to-Digital Converter) can read. ## Subsystem 2: The Right-Hand "Strum" Controller This subsystem acts as the "trigger." It determines when a sound should be played and how loud it should be based on the intensity of the movement. - 9-Axis IMU [P/N: BNO055]: This contains an accelerometer and a gyroscope. It tracks the rapid "up and down" motion of a strum. We chose the BNO055 because it has an on-board processor that handles "Sensor Fusion," giving us clean orientation data without taxing our main CPU. - Backup IMU (Plan B): InvenSense MPU-6050. It is widely available and has extensive library support. While it only offers 6-axis sensing (no magnetometer) and requires the ESP32 to handle the Kalman filtering or Complementary filtering in code, it is a highly reliable fallback if the BNO055 has procurement delays or I2C clock-stretching issues. - Force Sensitive Resistor (FSR) [P/N: FSR 402]: A small pressure sensor placed on the thumb. This allows the user to simulate "holding a pick." The sound only triggers when the user "squeezes" the virtual pick while strumming. ## Subsystem 3: Processing & Wireless Communication This is the "Brain" of the system. It collects data from both hands and converts it into music. - ESP32 Microcontroller (2x) [P/N: ESP32-WROOM-32E]: One for each hand. These chips are powerful and have built-in Bluetooth and Wi-Fi. - ESP-NOW Protocol: We will use this specialized low-latency wireless protocol to send data from the "Strum" hand to the "Fret" hand in less than 5ms, ensuring the two hands are perfectly in sync. - BLE MIDI: The final output is sent via Bluetooth Low Energy MIDI to a phone or laptop, allowing the glove to work with any professional music software (like GarageBand or Ableton). ## Subsystem 4: Power Management Since we want the project to be wearable and "Cyberpunk" in style, the power system must be compact and efficient. - LiPo Batteries (2x): Small 3.7V rechargeable batteries tucked into the wrist straps. - TP4056 Charging Modules: To allow the gloves to be recharged via a standard USB-C cable. - Buck-Boost Converters: To ensure the ESP32 and sensors receive a steady, clean 3.3V even as the battery voltage drops during use. # Criterion For Success - Latency: The total "Motion-to-Sound" delay must be under 30ms. Anything higher is noticeable to a musician. **Test Method:** We will program a "Test Mode" where a physical button press on the Strum hand toggles a GPIO pin (HIGH) and simultaneously sends the wireless strum packet. Using an oscilloscope, we will measure the delta (t) between the GPIO HIGH signal and the arrival of the MIDI Note On message at the receiver's serial port. - Chord Recognition: The system must accurately distinguish between at least 5 different chord shapes with a success rate of >90%. Dynamic Range: The system must be able to distinguish between a "Soft Strum" and a "Hard Strum," translating that into different MIDI volume levels. - Battery Life: The device must operate continuously for at least 2 hours on a single charge. - Wireless Stability: The ESP-NOW link between hands must maintain a Packet Delivery Ratio (PDR) of ≥ 99%within a 2-meter radius (the typical wingspan of a human) over a continuous 10-minute testing window. **Test Method:** The Right-Hand unit will send 1,000 packets at the target rate (e.g., 100Hz). The Left-Hand unit will log the sequence numbers; a successful test results in ≤ 10 missed packets. |
|||||