Project

# Title Team Members TA Documents Sponsor
44 Voice-Activated Geographic Reference Globe
 Mahathi Jayaraman
Rijul Roy
Varsha Mullangi
 Chihun Song design_document1.pdf
final_paper1.pdf
photo1.webp
photo2.webp
photo3.webp
presentation1.pptx
proposal2.pdf
video
Team Members:
Mahathi Jayaraman (mj45)
Rijul Roy (rijulr2)
Varsha Mullangi (varsham3)

Problem
Many kids these days, especially American kids, don’t know their geography that well. In addition, many kids are spending a lot of time on screens and online, which is taking them out of the real world. We want to create a solution where kids can learn geography in a manner that does not need them to be connected to the internet or on a screen. This solution should be able to be used in classrooms for kids to learn from, as well as be able to rotate to accommodate the shorter height of kids.

Solution
Our proposed solution is to build a globe that is screen-free and interactive. Rather than manually rotating a globe and having to search for where a certain country is, kids can now simply push a button to activate a microphone and say a country name out loud. The globe will rotate automatically to a designated front marker of the globe and light up the specified countries with LEDs. This will help kids feel more engaged with learning.

Solution Components

Subsystem 1: Speech Recognition with a Push to Talk Mechanism

This subsystem will implement the speech recognition mechanism of the globe. A simple push button and microphone will be used, connected to the GPIO pins of the ESP32-S3 MCU. While the button is pressed, the microphone will collect audio from the user, capturing the specified country the user wants to find. The MCU uses this audio to run an offline, on-device speech recognition software (ESP-SR) to determine which country the user wants to find, which will be used to handle the motor control logic and LEDs.

Components:
ESP32-S3 MCU and ESP-SR Package
I2S Digital Microphone (INMP441)






Subsystem 2: Software-Driven Motor Control

This subsystem controls how the globe physically rotates to face the input country. A low speed DC gear motor will be driven by the ESP32-S3 through a motor driver, allowing the MCU to control both the direction and speed of rotation on the axis. A separate motor will be used to tilt the globe up and down, with the globe sitting in a ring with a ball bearing track. Based on the target country’s stored position and the current angle of the globe, the software will calculate the direction of rotation and the number of turns needed for the globe to rotate to align the country with the front marker. Feedback from a magnetic angle sensor will be used to track the globe’s position and stop rotation at the correct point. This makes the rotation more reliable and prevents the globe from rotating too far past the target.

Components:
22 RPM 24 Volt DC Globe Inline Gearmotor [500635]


Subsystem 3: LED Outline/Markers

This subsystem is responsible for the physical identification of countries using LEDs. We will use a LED grid placed behind the globe, ensuring that that LEDS line the borders and corners of countries. If its a smaller country, making it harder to border, we will use the center point of the country, lighting up only one LED to indicate the location of that country. Since we will be using addressable LEDs, we will be able to assign LEDS to countries, so that when a country is chosen, the logic can quickly determine which LEDS to turn on. We will also use one LED near the button that captures audio, helping the user know when audio is being recorded.

Components:
LED strips (WS2812B)


Subsystem 4: Front Marker Reference
This subsystem is responsible for rotating the globe to face a designed front marker. This marker will be a point on a ring around the globe. This will designate where the user of this globe will be positioned, so that when the globe rotates to allow the country to face this marker, the country will also be facing the user. The globe will also rotate on multiple axes to face this, which can help accommodate the shorter height of kids by making the globe rotate down to make areas near the north pole (such as Iceland or the North Pole) visible to kids who may not be tall enough to see the top of the globe. Every time a country is detected through the microphone, that country will automatically rotate to this marker. The slip ring will be used to ensure that the internal components do not get caught in each other as the globe rotates, and the limit switches will make sure the globe does not rotate too much in any direction.

Components:
ESP32-S3 MCU (controller)
Adafruit AS5600 Magnetic Angle Sensor - rotation position sensor
Slip Ring (because it is a rotating system)
Optional Limit Switches to prevent overrotation
The Motor System (subsystem 2)


Criteria for Success:
The system can use the microphone to accurately identify spoken words, and check if the word is in the database of country names.
When a country name is spoken, the system can light up the country on the globe.
When a country name is spoken, the globe can rotate to display the lit country in front of the user.
When the word “reset” is provided as an input, the globe moves back to its default position and all LEDs are turned off.
The globe will correctly detect the spoken country name and rotate automatically so the specified country is facing the front marker

Bracelet Aid for deaf people/hard of hearing

Aarushi Biswas, Yash Gupta, Anit Kapoor

Bracelet Aid for deaf people/hard of hearing

Featured Project

# PROJECT TITLE: Bracelet Aid for deaf people/hard of hearing

# TEAM MEMBERS:

- Aarushi Biswas (abiswas7)

- Anit Kapoor (anityak3)

- Yash Gupta (yashg3)

# PROBLEM

We are constantly hearing sounds around us that notify us of events occurring, such as doorbells, fire alarms, phone calls, alarms, or vehicle horns. These sounds are not enough to catch the attention of a d/Deaf person and sometimes can be serious (emergency/fire alarms) and would require the instant attention of the person. In addition, there are several other small sounds produced by devices in our everyday lives such as washing machines, stoves, microwaves, ovens, etc. that cannot be identified by d/Deaf people unless they are observing these machines constantly.

Many people in the d/Deaf community combat some of these problems such as the doorbell by installing devices that will cause the light in a room to flicker. However, these devices are generally not installed in all rooms and will also obviously not be able to notify people if they are asleep. Another common solution is purchasing devices like smartwatches that can interact with their mobile phones to notify them of their surroundings, however, these smartwatches are usually expensive, do not fulfill all their needs, and require nightly charging cycles that diminish their usefulness in the face of the aforementioned issues.

# SOLUTION

A low-cost bracelet aid with the ability to convert sounds into haptic feedback in the form of vibrations will be able to give d/Deaf people the independence of recognizing notification sounds around them. The bracelet will recognize some of these sounds and create different vibration patterns to catch the attention of the wearer as well as inform them of the cause of the notification. Additionally, there will be a visual component to the bracelet in the form of an OLED display which will provide visual cues in the form of emojis. The bracelet will also have buttons for the purpose of stopping the vibration and showing the battery on the OLED.

For instance, when the doorbell rings, the bracelet will pick up the doorbell sound after filtering out any other unnecessary background noise. On recognizing the doorbell sound, the bracelet will vibrate with the pattern associated with the sound in question which might be something like alternating between strong vibrations and pauses. The OLED display will also additionally show a house emoji to denote that the house doorbell is ringing.

# SOLUTION COMPONENTS

Based on this solution we have identified that we need the following components:

- INMP441 (Microphone Component)

- Brushed ERM (Vibration Motor)

- Powerboost 1000 (Power subsystem)

- 1000 mAh LiPo battery x 2 (hot swappable)

- SSD1306 (OLED display)

## SUBSYSTEM 1 → SOUND DETECTION SUBSYSTEM

This subsystem will consist of a microphone and will be responsible for picking up sounds from the environment and conducting a real-time FFT on them. After this, we will filter out lower frequencies and use a frequency-matching algorithm to infer if a pre-programmed sound was picked up by the microphone. This inference will be outputted to the main control unit in real-time.

## SUBSYSTEM 2 → VIBRATION SUBSYSTEM

This subsystem will be responsible for vibrating the bracelet on the wearer’s wrist. Using the vibration motor mentioned above, we should have a frequency range of 30Hz~500Hz, which should allow for the generation of a variety of distinguishable patterns. This subsystem will be responsible for the generation of the patterns and control of the motor, as well as prompting the Display subsystem to visualize the type of notification detected.

## SUBSYSTEM 3 → DISPLAY SUBSYSTEM

The Display subsystem will act as a set of visual cues in addition to the vibrations, as well as a visual feedback system for user interactions. This system should not draw a lot of power as it will be active only when prompted by user interaction or by a recognized sound. Both of these scenarios are relatively uncommon over the course of a day, which means that the average power draw for our device should still remain low.

## SUBSYSTEM 4 → USER INTERACTION SUBSYSTEM

This subsystem is responsible for the interaction of the user with the bracelet. This subsystem will include a set of buttons for tasks such as checking the charge left on the battery or turning off a notification. Checking the charge will also display the charge on the OLED display thus interacting and controlling the display subsystem as well.

## SUBSYSTEM 5 → POWER SUBSYSTEM

This subsystem is responsible for powering the device. One of our success criteria is that we want long battery life and low downtime. In order to achieve this we will be using a power boost circuit in conjunction with two rechargeable 1000 mAh batteries. While one is charging the other can be used so the user doesn’t have to go without the device for more than a few seconds at a time. We are expecting our device to use anywhere from 20-50mA which would mean we get an effective use time of more than a day. The power boost circuit and LiPo battery’s JST connector allow the user to secure and quick battery swaps as well.

# CRITERION FOR SUCCESS

- The bracelet should accurately identify only the crucial sounds in the wearer’s environment with each type of sound having a fixed unique vibration + LED pattern associated with it

- The vibration patterns should be distinctly recognizable by the wearer

- Should be relatively low cost

- Should have prolonged battery life (so the power should focus on only the use case of converting sound to vibration)

- Should have a small profile and a sleek form factor

Project Videos