Project

# Title Team Members TA Documents Sponsor
33 HelpMeRecall
 Michael Jiang
Sravya Davuluri
William Li
# HelpMeRecall

Team Members:
- Sravya Davuluri (sravyad2)
- William Li (wli202)
- Michael Jiang (mbjiang2)

# Problem

Many individuals have difficulty remembering recent activities and completing routine tasks like eating or taking medication.

# Solution

A standalone assistive device that supports activity recall using sensor-gated voice interaction. It allows users to verbally log activities they have completed, and later query if a specific activity has been performed. It uses an onboard microphone and on-device audio processing on a microcontroller to perform keyword detection.

This device is always on and will be verifiable with an LED, but the voice input is only accepted if the device is worn (capacitive touch sensor) and specific words from a limited vocabulary is said to avoid accidental logging. To address the possibility of reduced correct detection of supported keywords, we will have various keywords targeted for an activity. So in the case of taking medicine, it might be medicine, medication, pill, drug, and prescription. This also simplifies the problem and prevents confidence rate issues. To validate a completed action, the action is logged only if an accelerometer detects physical movement around the time in order to reduce false logging. If a voice log is accepted, haptic feedback is provided by the device. Activities are also timestamped and stored in local memory. If the device notes that a specific activity has been completed, it affirms it including the timestamp using an integrated speaker.

The logs reset at midnight automatically since the activities repeat on the daily. There is also an option of a hard reset button to clear logs. There will also be a button to delete the latest log in case of a logging mistake by the user.

# Solution Components

## Subsystem 1: Microcontroller Unit and Controls

Acts as the central unit for logic. Manages the sensor inputs, and executes a finite state machine. The FSM states are start, idle, listening, logging, and replying.

Components: ESP32-S3-WROOM-1

## Subsystem 2: Audio input processing unit

Captures the voice input from the user and performs keyword detection on a limited vocabulary, where each action can be mapped to multiple set keywords to improve detection.

Components: Digital MEMS microphone (INMP441), ESP32-S3-WROOM-1

## Subsystem 3: Sensor gating and activity validation

Uses a capacitive touch sensor and an accelerometer to detect motion, which ensures that voice input is only received and accepted if the device is worn and recent movement is detected by the accelerometer instead of continuous voice recognition. A "cooldown" period is enforced where the microphone will be disabled for 10 seconds if there's motion but no logging during the listening period multiple times in a row to help conserve some battery.

Components: Capacitive touch sensor (AT42QT1010), Accelerometer (MPU-6050)

## Subsystem 4: Feedback and Output

Uses a speaker for audio feedback as a response to the user’s query. This subsystem also provides haptic feedback as an indication of an accepted user voice log. To indicate if the device is on, the LED is green. If the device is listening, the LED is yellow. If the device is low on power, the LED will be red.

Components: Speaker (8 ohm speaker), amplifier (MAX98357A), coin vibration motor, transistor (2N3904), RGB LED

## Subsystem 5: Time logging and local storage

Stores the activity voice logs along with timestamps. Allows automatic reset at midnight to support daily repetitive tasks. Timekeeping is done using ESP32’s internal RTC.

Components: ESP32-S3-WROOM-1

## Subsystem 6: Power

Supplies power to the device.

Components: Battery (Li-Po battery)

# Criterion For Success
- Correctly detects supported keywords with an accuracy of at least 80% in a quiet environment
- Device will only log upon verifying physical activity and hearing a keyword from the user within a 5 second window
- Upon successful logging, the speaker will output audibly and haptic feedback can be felt by the user with a 2 second vibration
- While querying logs, speaker will output and LED will be solid
- Logs will be automatically cleared at midnight and can be manually reset with the reset button
- Latest log will be deleted upon pushing a separate button
- LED stays solid while device is powered
- False log rate < 1 per hour in normal conversation when worn.

Economic Overnight Outlet

Chester Hall, Sabrina Moheydeen, Jarad Prill

Featured Project

**Team**

- Chester Hall (chall28), Sabrina Moheydeen (sabrina7), Jarad Prill (jaradjp2)

**Title**

- Economic Overnight Outlet

**Problem**

- Real-time pricing in ISOs, such as the Midwest, California, New England, and New York, provides differentials in electricity prices throughout the day that can be taken advantage of. The peak price of electricity compared to the minimum prices can feature variations of up to 70%. With price agnostic charging, this results in unnecessary costs for those who charge devices (see attached spreadsheet). This same principle can thus be scaled for large commercialized applications requiring high-capacity batteries, resulting in a higher savings potential to be taken advantage of.

- Calcs: https://docs.google.com/spreadsheets/d/1JBzt2xm0Ue4a_teosdak623h0zSP5nHRKi7Wi8rMcPo/edit?usp=sharing

**Solution Overview**

- We will create a device that can fetch real-time prices from regional ISOs and enable charging when prices are lowest. Our primary application will be centered towards warehouse electric vehicles using high-capacity, fast-charging lithium ion batteries. Such vehicles include forklifts, cleaning machines, and golf carts.

**Solution Components**

- [ISO LMP API] - Through use of a WiFi-enabled microcontroller we can fetch real-time prices and build our control system around these values.

- [Passive High Performance Protection] - In order to provide downstream safety to the loads, we will ensure the device features surge protection and is rated for the high current of fast charging. The switching of the connection will be done with a contactor whose coil is energized according to the microcontroller.

- [Device Display] - LCD display to show information about the current energy price and the current day’s savings.

- [Manual User Override] - The device will feature a manual toggle switch to either enable or disable the cost-optimized charging feature allowing users to charge loads at any time, not necessarily the cheapest.

- [User Interface] - Software application to allow for user input regarding the time of day the device must be charged by. The application will also display information about total savings per week, month, or year and savings over the device’s lifetime.

- [Control Power Converter] - In order to run the low voltage control systems from the outlet, either 120VAC or 3-phase 480VAC, we will need to step this down to a low DC voltage of around 3.3VDC.

- [Memory System] - Microcontroller capable of performing control function within user specified parameters.

- [Device Connection] - Connectivity to the battery of the device being charged so that current state of charge (SoC) information can be used. Potential experimental filter algorithms will be used in order to estimate the SoC automatically, without requiring the user to input the specific data of the device being used.

**Criterion for Success**

- Able to charge devices at lowest cost times of the day and display current pricing and savings information. The upfront cost of a large-scale reproducible product must be less than the lifetime savings incurred by purchasing the product. Users without an engineering background can easily analyze their savings to visually recognize the device’s benefit.