Project

# Title Team Members TA Documents Sponsor
5 Navigation Vest Suite For People With Eye Disability
 Haoming Mei
Jiwoong Jung
Pump Vanichjakvong
# Navigation Vest Suite For People With Eye Disability


Team Members & Experiences:
- Jiwoong Jung (jiwoong3): Experienced in Machine Learning, and some Embedded programming. Worked on many research and internships that requires expertise in Machine Learning, Software Engineering, Web Dev., and App Dev. Had some experience with Embedded programming for Telemetry.
- Haoming Mei (hmei7): Experienced in Embedded programming and PCB design. Worked with projects like lights, accelerometer, power converter, High-Fet Board, and motor control for a RSO that involve understanding of electronics, PCB design, and programming with STM32 MCUs.
- Pump Vanichjakvong (nv22): Experienced with Cloud, Machine Learning, and Embedded programming. Done various internships and classes that focuses on AI, ML, and Cloud. Experience with Telemetry and GPS system from RSO that requires expertises in SPI, UART, GPIOs, and etc with STM32 MCUs.

# Problem

People with Eye Disability often face significant challenges navigating around in their daily lives. Currently, most available solutions ranges like white canes and guide dogs to AI-powered smart glasses, many of which are difficult to use and can cost as much as $3,000. Additionally, problems arises for people with disability, especially in crowded/urban areas, and that includes injuries from collision with obstacles, person, or from terrains. According to the U.S department of Transportation's 2021 Crash Report , 75% of pedestrian fatalities occurred at locations that were not intersections. Thus we aim to design a navigation vest suite to help people with eye disability to deal with these issues.

https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/813458.pdf

# Solution
We have devised a solution which helps ease visually impaired individuals in daily activities such as walking from two places, or navigating around a building with multiple obstacles. Our focus will for out-door navigation in urban areas, where obstacles, terrain, and pedestrians. But, if time permits we will also deal with traffics and crosswalks.




In order to achieve this, we will be utilizing 3 main components:
- Lidar sensors to help the wearer with depth perception tasks
- Vibration Motors to aid navigation (turning left/right)
- Magnetometer to enable more accurate GPS coordination

All the above components will contribute to the sensory fusion algorithm.

# Solution Components

## Subsystem 1
### Microcontroller System
We are planning to use a STM32 microcontroller as main processing unit for sensory data from lidar sensors (magnetometer and GPS if time permits) and object detection data from the **machine learning system**, and direction data from navigation app (our design on phone). We will use these information to generate vibration in the direction the wearer should navigate.

### Power Systems
The whole system will be battery-powered by a battery module, which contains 5V battery-cells. It will be connected to the **Microcontroller System**, which will also supply it to the **Machine Learning System**. We will also implement the necessary power protection, buck converter, regulator, and boost converters as necessary per sensors or components.
- Battery Module Pack
- Buck Converter (Step-Down)
- Boost Converter (Step-Up)
- Voltage Regulator
- Reverse Polarity Protection
- BMS

## Subsystem 2
### Navigation Locator Systems
Our navigation system will consist of an App which directly connects to the Google Maps API, paired with our existing sensors. We plan to utilize a magnetometer sensor, which will indicate the direction the user is facing (North, South, East, West, .etc). In order to pinpoint which direction the wearer needs to be heading, our built-in LiDAR sensors will enable us to create a SLAM (Simultaneous Localization and Mapping) to build a map of the environment. With these systems in place, we would be able to assist users in navigation. To deal with Terrain hazards, we will use the LiDAR to sensors to assist us in dealing with elevation changes the user needed to make.

- LiDAR
- Android App (Connected to Google API)
- Magnetometer
- Vibration Motors

Extra Features (if time permits):
- Audio Output (Text to Speech Generation from Raspberry PI 5 sends to microcontroller through AUX cable )
## Subsystem 3
### Machine Learning Systems

- We plan to employ a object detection model on a 16GB Raspberry PI 5 (already) along with a PI camera to detect objects, signs, and people on the road, which will be feed to the microcontroller
- Raspberry PI 5
- PI Camera


a) The image video model will be expected to be less than 5 billion parameters with convolutional layers, running on device in raspberry pi Obviously the processing power on the raspberry pi is expected to be limited, however we are planning to accept the challenge and find out ways to improve the model with limited hardware capabilities.

b) If the overall project for subtask a) becomes arduous or time consuming, we can utilize api calls or free open source models to process the image/video in real time if the user wants the feature enabled. The device is paired with the phone via the wifi chip on the raspberry pi to enable the API call. Some of the best candidates we can think of are the YOLO family models, MMDetection and MMTracking toolkit, or Detectron2 model that is developed by Facebook AI Research that supports real time camera feedbacks.

# Criterion For Success


### Navigational Motor/Haptic Feedback
1) The Haptic feedback (left/right vibration) should perfectly match with the navigation directions received from the app (turn left/right)

2) Being able to Detect Obstacles, Stairs, Curbs, and people.

3) Being able to detect intersections infront and the point of turn through the lidar sensory data.

4) Being able to obey the haptic feedback patterns that is designed. (tap front for walking forward, tap right to go right etc...)

### Object Detection
1) Using the Illinois Rules of the Road and the Federal Manual on Uniform Traffic Control Device Guidelines, we will be using total of 10-30 distinct pedestrian road signs to test the object detection capability. We will be using a formal ML testing methods like geometric transformations, photometric transformations, and background clutter. Accuracy will be measured by the general equation (Total Number of correctly classified Datasets)/(Total Number of Datasets)

2) The ML Model should be able to detect potential environmental hazards including but not limited to Obstacles, Stairs, Curbs, and people. We are planning onto gather multiple hazard senarios via online research, surveys, and in-person interviews. Based on the collected research, we will be building solid test cases to ensure that our device can reliably identify potential hazards. More importantly, we are planning design strict timing and accuracy measures metrics.


3) The ML model should be able to detect additional road structures such as curbs, crosswalks, and stairs to provide comprehensive environment awareness. We will be utilizing different crosswalks located on north quad and utilize the accuracy measurement techniques mentioned in 1)



### Power and Battery Life

1) The device should support at least 3 hours of battery life.

2) The device should obey the IEC 62368-1 safety standard. IEC 62368-1 safety standard lays out different classes such as ES1, ES2, and ES3 that considers electrical and flame



Economic Overnight Outlet

Chester Hall, Sabrina Moheydeen, Jarad Prill

Featured Project

**Team**

- Chester Hall (chall28), Sabrina Moheydeen (sabrina7), Jarad Prill (jaradjp2)

**Title**

- Economic Overnight Outlet

**Problem**

- Real-time pricing in ISOs, such as the Midwest, California, New England, and New York, provides differentials in electricity prices throughout the day that can be taken advantage of. The peak price of electricity compared to the minimum prices can feature variations of up to 70%. With price agnostic charging, this results in unnecessary costs for those who charge devices (see attached spreadsheet). This same principle can thus be scaled for large commercialized applications requiring high-capacity batteries, resulting in a higher savings potential to be taken advantage of.

- Calcs: https://docs.google.com/spreadsheets/d/1JBzt2xm0Ue4a_teosdak623h0zSP5nHRKi7Wi8rMcPo/edit?usp=sharing

**Solution Overview**

- We will create a device that can fetch real-time prices from regional ISOs and enable charging when prices are lowest. Our primary application will be centered towards warehouse electric vehicles using high-capacity, fast-charging lithium ion batteries. Such vehicles include forklifts, cleaning machines, and golf carts.

**Solution Components**

- [ISO LMP API] - Through use of a WiFi-enabled microcontroller we can fetch real-time prices and build our control system around these values.

- [Passive High Performance Protection] - In order to provide downstream safety to the loads, we will ensure the device features surge protection and is rated for the high current of fast charging. The switching of the connection will be done with a contactor whose coil is energized according to the microcontroller.

- [Device Display] - LCD display to show information about the current energy price and the current day’s savings.

- [Manual User Override] - The device will feature a manual toggle switch to either enable or disable the cost-optimized charging feature allowing users to charge loads at any time, not necessarily the cheapest.

- [User Interface] - Software application to allow for user input regarding the time of day the device must be charged by. The application will also display information about total savings per week, month, or year and savings over the device’s lifetime.

- [Control Power Converter] - In order to run the low voltage control systems from the outlet, either 120VAC or 3-phase 480VAC, we will need to step this down to a low DC voltage of around 3.3VDC.

- [Memory System] - Microcontroller capable of performing control function within user specified parameters.

- [Device Connection] - Connectivity to the battery of the device being charged so that current state of charge (SoC) information can be used. Potential experimental filter algorithms will be used in order to estimate the SoC automatically, without requiring the user to input the specific data of the device being used.

**Criterion for Success**

- Able to charge devices at lowest cost times of the day and display current pricing and savings information. The upfront cost of a large-scale reproducible product must be less than the lifetime savings incurred by purchasing the product. Users without an engineering background can easily analyze their savings to visually recognize the device’s benefit.