Project

# Title Team Members TA Documents Sponsor
25 Building Interior Reconnaissance Drone (BIRD)
 Jack Lavin
Jacob Witek
Mark Viz
 Shiyuan Duan design_document1.pdf
proposal1.pdf
# Building interior reconnaissance drone proposal

Team Members:
- Mark Viz (markjv2)
- Jack Lavin (jlavin4)
- Jacob Witek (witek5)

# Problem

There are many situations when law enforcement or emergency medical service professionals need quick, real-time, useful information about a non-visible location without sending in a human to gather this information due to present risks. One of the most important things to know in these situations is if there are people in a room or area, and if so, where they are located. While there are current promising solutions used by these professionals, they can rarely be operated by one person and take away time and manpower from situations which usually greatly require both. Our solution attempts to address these issues while providing an easy-to-use interface with critical information.

# Solution

Our solution to this issue is to use a reconnaissance drone equipped with a camera and other sensing components and simple autonomous behavior capabilities, and process the video feed on a separate laptop to determine an accurate location of all people in view of the drone relative to the location of a phone or viewing device nearby. This phone or viewing device would run an augmented-reality application using position information from the drone system to overlay the positions of people near the drone over first-person perspective video. The end result would allow someone to slide/toss the drone into a room, and after a second or two, be able to "see through the wall" where anyone in the room is.

# Solution Components

## Drone and Sensors

The drone itself will be a basic lightweight quadcopter design. The frame will be constructed using a 2D design cut from a sheet of carbon fiber and assembled with aluminum hardware and thread locks. The total volume including the rotor blades should not exceed 4" H by 8" W by 8" L at maximum (ideally much less). This simple frame will consist of a rectangular section to mount the PCB and a 2S (7.4 V) LiPo pack of about 2" x 2" or less, and four identical limbs mounted to the corners. On each of the four limbs will be brushless DC motors (EMAX XA2212 2-3S) driven by electronic speed controllers from the PCB (assuming they can't be pre-purchased). The PCB will have a two-pin DuPont/JST connectors for battery leads, a TP4056 LiPo discharging circuit, and buck converters for necessary voltage(s) all on the underside. On top, the PCB will house an ESP32-S3 microcontroller, an IMU with decent accuracy, a set of mmWave 24 GHz human presence sensor (like the LD2410) and ultrasonic transducers to form a phase array sensor with an accurate, narrow beam to scan for human presence with range. These components will allow the drone to be programmed with very simple and limited autonomous flight behaviors (fly up 5 feet, spin 360 degrees, land) and properly/safely control itself. The ultrasonic transducers and human sensing radars will be the primary method of determining human presence and mostly calculated on the ESP-32, however additional calculation will need to be made on the AR end with the received data. If time and budget allow, we may also include a small 2 MP or 5 MP camera for WiFi video stream or a composite video camera for an analog video stream as a backup/failsafe to the other sensors.

A working rough breakdown of the expected mass of each component will go as follows:

- 4 hobby motors: ~ 50 grams (based on consumer measurements)
- Carbon fiber frame: ~ 40 grams (estimate based on similar style and sized frames)
- 2S 500 mAh battery: ~30 grams (based on common commercial LiPo product info)
- PCB with MCU & peripherals: ~50 grams (based on measurements of similar boards)
- 10-20 ultrasonic transducers: ~50 grams (based on commercial component info)
- Metal hardware/fasteners & miscellaneous: ~25 grams (accounting for error as well)
- Total mass: ~255 grams
- Total thrust (at 7.6 V 7.3 A): ~2000 grams (from manufacturer ratings)
- Thrust/weight is well over 2.0 and should allow for quick movement and considerable stability along with the improved frame considerations, and also extra room for more weight if needed.

## AR Viewer or Headset

To create a useful augmented-reality display of the collected position data, the simplest way will be to write an app that uses the digital camera and gyroscope/IMU API's of a smart phone to overlay highlighted human position data on a live camera view. We would use the android studio platform to create this custom app which would interface with the data incoming from the drone. Building upon the android API's we would overlay the data to the phone camera. If we have more time to develop one, a headset or AR glasses could make the experience more useful (hands-free) and immersive. We may also use a laptop at this stage to run a server alongside the app for better processing.

# Working Supply List

*some can be found in student self-service, some need to be ordered
- Carbon fiber sheet (find appropriate size and 2-3 mm thick)
- Aluminum machine screws with lock-tite or bolt/nut with locking washer
- 4 EMAX brushless DC motors and mounting hardware
- 4 quadcopter rotor blades
- 2S (7.6 V) 500 mAh LiPo battery
- Custom PCB
- ESP32-S3 chip w/ PCB antenna
- 20 ultrasonic (40 kHz) transducer cans
- 4 mmWave 24 GHz human presence radar sensors
- TP 4056 LiPo Charging IC (find other necessary SMD components)
- DuPont two-pin connector for LiPo charging/discharging (choose whether removable battery design)
- Various SMD LEDs to indicate functionalities or states on PCB
- Voltage buck converter circuit components
- ESC circuit components
- Adafruit Accelerometer

# Criterion For Success

The best criteria for the success of this project is whether our handheld device or headset can effectively communicate human position data of a visually obstructed location to a nearby user within an accuracy of 1-2 meters while still allowing the user to carry out personal tasks. The video feed should be stable with minimal latency as to be effective and usable, and estimated human positions should be updated only when they are positively in view and information about the recency of data should be apparent (maybe a red highlight on new people, yellow on a stale location, and green for a newly updated position).

Bracelet Aid for deaf people/hard of hearing

Aarushi Biswas, Yash Gupta, Anit Kapoor

Bracelet Aid for deaf people/hard of hearing

Featured Project

# PROJECT TITLE: Bracelet Aid for deaf people/hard of hearing

# TEAM MEMBERS:

- Aarushi Biswas (abiswas7)

- Anit Kapoor (anityak3)

- Yash Gupta (yashg3)

# PROBLEM

We are constantly hearing sounds around us that notify us of events occurring, such as doorbells, fire alarms, phone calls, alarms, or vehicle horns. These sounds are not enough to catch the attention of a d/Deaf person and sometimes can be serious (emergency/fire alarms) and would require the instant attention of the person. In addition, there are several other small sounds produced by devices in our everyday lives such as washing machines, stoves, microwaves, ovens, etc. that cannot be identified by d/Deaf people unless they are observing these machines constantly.

Many people in the d/Deaf community combat some of these problems such as the doorbell by installing devices that will cause the light in a room to flicker. However, these devices are generally not installed in all rooms and will also obviously not be able to notify people if they are asleep. Another common solution is purchasing devices like smartwatches that can interact with their mobile phones to notify them of their surroundings, however, these smartwatches are usually expensive, do not fulfill all their needs, and require nightly charging cycles that diminish their usefulness in the face of the aforementioned issues.

# SOLUTION

A low-cost bracelet aid with the ability to convert sounds into haptic feedback in the form of vibrations will be able to give d/Deaf people the independence of recognizing notification sounds around them. The bracelet will recognize some of these sounds and create different vibration patterns to catch the attention of the wearer as well as inform them of the cause of the notification. Additionally, there will be a visual component to the bracelet in the form of an OLED display which will provide visual cues in the form of emojis. The bracelet will also have buttons for the purpose of stopping the vibration and showing the battery on the OLED.

For instance, when the doorbell rings, the bracelet will pick up the doorbell sound after filtering out any other unnecessary background noise. On recognizing the doorbell sound, the bracelet will vibrate with the pattern associated with the sound in question which might be something like alternating between strong vibrations and pauses. The OLED display will also additionally show a house emoji to denote that the house doorbell is ringing.

# SOLUTION COMPONENTS

Based on this solution we have identified that we need the following components:

- INMP441 (Microphone Component)

- Brushed ERM (Vibration Motor)

- Powerboost 1000 (Power subsystem)

- 1000 mAh LiPo battery x 2 (hot swappable)

- SSD1306 (OLED display)

## SUBSYSTEM 1 → SOUND DETECTION SUBSYSTEM

This subsystem will consist of a microphone and will be responsible for picking up sounds from the environment and conducting a real-time FFT on them. After this, we will filter out lower frequencies and use a frequency-matching algorithm to infer if a pre-programmed sound was picked up by the microphone. This inference will be outputted to the main control unit in real-time.

## SUBSYSTEM 2 → VIBRATION SUBSYSTEM

This subsystem will be responsible for vibrating the bracelet on the wearer’s wrist. Using the vibration motor mentioned above, we should have a frequency range of 30Hz~500Hz, which should allow for the generation of a variety of distinguishable patterns. This subsystem will be responsible for the generation of the patterns and control of the motor, as well as prompting the Display subsystem to visualize the type of notification detected.

## SUBSYSTEM 3 → DISPLAY SUBSYSTEM

The Display subsystem will act as a set of visual cues in addition to the vibrations, as well as a visual feedback system for user interactions. This system should not draw a lot of power as it will be active only when prompted by user interaction or by a recognized sound. Both of these scenarios are relatively uncommon over the course of a day, which means that the average power draw for our device should still remain low.

## SUBSYSTEM 4 → USER INTERACTION SUBSYSTEM

This subsystem is responsible for the interaction of the user with the bracelet. This subsystem will include a set of buttons for tasks such as checking the charge left on the battery or turning off a notification. Checking the charge will also display the charge on the OLED display thus interacting and controlling the display subsystem as well.

## SUBSYSTEM 5 → POWER SUBSYSTEM

This subsystem is responsible for powering the device. One of our success criteria is that we want long battery life and low downtime. In order to achieve this we will be using a power boost circuit in conjunction with two rechargeable 1000 mAh batteries. While one is charging the other can be used so the user doesn’t have to go without the device for more than a few seconds at a time. We are expecting our device to use anywhere from 20-50mA which would mean we get an effective use time of more than a day. The power boost circuit and LiPo battery’s JST connector allow the user to secure and quick battery swaps as well.

# CRITERION FOR SUCCESS

- The bracelet should accurately identify only the crucial sounds in the wearer’s environment with each type of sound having a fixed unique vibration + LED pattern associated with it

- The vibration patterns should be distinctly recognizable by the wearer

- Should be relatively low cost

- Should have prolonged battery life (so the power should focus on only the use case of converting sound to vibration)

- Should have a small profile and a sleek form factor

Project Videos