My Role
Timeline
(12 weeks)
Tools
TL;DR
Project Overview
- In collaboration with Georgia Tech Sonification Lab and Microsoft Mixed Reality, this project investigated how head-worn devices (HWD) can support knowledge acquisition for blind or low vision (BLV) individuals when learning and performing daily tasks
Contribution
- Led semi-structured interviews with 9 BLV participants and synthesized insights through thematic analysis
Impact
- Established HWD as a probable form factor for BLV assistive technology, highlighting 3 key characteristics, including portable, real-time, and complementary, and raising awareness for unintended consequences of existing technologies
Learning
- Negotiated with stakeholders to align research objectives and refine scope with a realistic timeline.
Blind and low-vision (BLV) individuals experience unique daily challenges during knowledge acquisition.
This encompasses learning new skills (e.g., code, sports, dance) or performing everyday activities (e.g., cooking, cleaning, traveling). The complexity of these tasks arises from their inability to rely on visual information.
Understand how head-worn devices (HWDs) such as smart glasses can support daily knowledge acquisition for BLV individuals.
Questions to be answered through research
- Portable & lightweight HWDs (e.g., smart glasses) may be an optimal form factor for BLV assistive technologies
- The device should support real-time dialogue with users, enriching BLV individuals’ perceptions of their immediate surroundings
- HWDs are complementary tools supporting learning by providing supplemental information where accessible materials may fall short and should not substitutes for accessible material development
- Unintended consequences of existing devices underscore the importance of careful design process and continuous issue monitoring
We conducted semi-structured interviews with completely or legally blind participants (N = 9) recruited through the National Federation of Blind (NFB), American Council for Blind (ACB) email lists, and Veterans Affairs (VA). Thematic analysis was then used to analyze the interview transcripts.
Some questions we discussed with our participants
- Top challenges: navigating unfamiliar areas, learning physical activities, recovering from technical glitches, and needing to use multiple devices simultaneously
- Key characteristics of a perfect assistive device: portable, versatile, hands-free, supporting voice interaction, and real-time feedback
- Main motivation for using HWDs: increased efficiency and independence, with primary use cases such as navigation, reading labels/signs, learning movements, finding objects, and recognizing people
- Main concerns for using HWDs: Accuracy, privacy, connectivity, and isolation from the surrounding environment
1. Navigation Challenges
- Outdoor navigation issues of finding new routes, exploring new places, requiring alternative means of transportation, etc.
“There's times when I wanna go train my dog [in] places, but I'm unsure because I am not familiar with the place. and my dog is new, you know, so I'm training her so it's like new for both of us.” — P3
- Bumping into things, retrieving lost objects in indoor environments.
“[My legs] are scarred. I literally come back and kick that door that's open, like I'm trying to kick a field goal, cause I don't see it.” — P9
- "Not enough hands"
“Anybody that's going blind is eventually gonna lose the use of both their hands when they're in public” — P9
2. Physical Activities & Materials Challenges
- Self-paced online learning programs allow BLV individuals to take as much time as needed, yet these programs have many limitations
“One thing I do miss about being at in-person [dance] workshops is like [...] people can kind of correct the little tiny things such as “bend your arm” or “oh wait, not that far” or “stay, hold it there” you know things like that, like the little intricacies of motion.” — P3
- Often missing cues of non-verbal communication
"Up to 70% of communication can be non-verbal. People that are sighted have deciding advantages over me when we’re discussing things. They can see me. I can’t see them.” — P4
- Filling out paperwork or physical documents is difficult
“[...] for a lot of visual tasks such as filing out paperwork, you need someone to help you read and interpret the information. I need someone to show me where to fill information in and or essentially fill it out for me” — P6
3. Digital Screens Challenges
- Inaccessible materials & tools
“This CAD software. The person on a YouTube video will say, Oh, in order to enlarge this, you just move them as you click here on the file menu. And he’s doing about 3 to 5 things when he’s explaining one thing . . . I have to go over those videos 10 times before I even start to grasp what’s going on. It’s all due to visual impairment.” — P8
- Screen reader failures/glitches
“There’s a lot of instances where my screen reader has basically glitched and gone away for like a bit. And then I have to figure out a way to get back.” — P1
- Remote assistance apps (Aira, Be My Eyes) lack immediate support, and are usually treated as a last resort
“I tried using the apps to get help with the soda machine but never got anybody. To rely on someone else [...] it is important to rely on yourself and be independent.” — P8
4. Portable & Hands-free Form Factor is Preferred
- Small, Portable, Hands-free
“I don't expect it to be really big. I don't expect it to be human size. I think you know something that is around the size of a bigger insect that is able to maneuver the place and maybe climb on things and stuff like that [...] I definitely want it to be more of a handsfree experience where I just direct it to go to different places without having to pick it up” — P5
- Mainstream look
“I hope they wouldn't make me look like someone in space or look like a pair of workshield glasses. I hope they look like normal regular glasses similar to Ray Bans.” — P7
5. Real-time Voice Interaction & Feedback as Primary Modality
- Voice as the primary mode of interaction
“Totally voice, totally voice, like how I interact with my phone if I say ‘Hey, Siri’.” — P2
- Real-time feedback, direct dialogue
“To take a picture, I can say photo, I can say camera, and when I go to the camera it should be able to tell me if the face is aligned on the camera or tell me how to go left or right with the glasses to get whatever I'm trying to take a picture centered. Then I should be able to tap it and take the picture with the glasses” — P7
- Remained a bit skeptical & conservative of gestural interactions
“I feel like for myself, [gestures] would be kind of distracting to have.” — P1
6. Seamless Integration with Existing Devices
- Integration with phones, laptops, etc.
"[…] particularly if I was using the device to read things to me. I might want to be able to copy the text and put it places. So yeah, integration with smartphone would be important to me." — P3
- An ecosystem for software development to foster an expansive application
"I think eventually, for wearable. There needs to be a way for people to create these different apps, whether it's for golf, whether it's for finding things, whether it's for detection of any kind that people can then use" — P9
7. Expectations for HWDs for BLVs
- Screenless form factor
“You would not need that screen which, just like all the rest, takes up space and power.” — P4
- Support independence and efficiency
“[will] probably go more places [with HWDs] because I could use it to hopefully navigate, or at least tell what was in front of me in a grander sense than my cane [and dog] can do.” — P3
- Versatile use cases
“[About using kitchen appliances] [...] just to be able to see what was on the touch screen and to be able to determine if the device is almost at temperature would have been helpful.” — P3
“If you do have a screen on your computer or laptop, [HWDs] can help you navigate or like, give you guidance on what to do, or some shortcut keys.” — P1
8. Concerns for HWDs for BLVs
- Accuracy
“When there is a barrier by the time you realize that there is a barrier, you would have already collided with that barrier. I think it still has to improve quite a bit.” — P2
- Affordability
“I'm not really ready to invest in one of those yet, because they're they are really pricey. And …even though it's slightly less convenient, I feel like the apps that I have on my phone are enough for me to get by with.” — P5
- Dependence on internet connectivity & extra costs for services
“[…] subscription for their services in order for them to work, even though it’s paired with your phone [...] and their subscription was very high [in price].” — P9
- Privacy
“[know] where my data is being stored” — P3
- Public usage: isolation from surroundings
“I'm not really fond of headphones. I believe in hearing, you know, outside sound, so headphone becomes a barrier in that sense. So, any device that you know does not have headphone usage as a prerequisite. I'm all good for it.” — P2
- Discomfort of wearing
"I don't personally like wearing glasses. So for me, I feel like it would just be another like just an accessory that I wouldn't want on my head when I'm cooking like I want to stay focused on what I'm trying to do with my non-visual skills." — P1
conclusion
This work was presented to Microsoft Mixed Reality User Research. In this research, we suggested a new way of viewing "learning" as to increase perceptions of immediate surroundings. This broadens the scope of learning activities to things like navigation, which was often not considered a form of learning but a separate activity. This re-categorization of activities can hopefully benefit future works in exploring BLVs' needs.
The current research also establishes HWDs as a probable form factor for BLV assistive technologies. Key features highlighted in this study could provide groundwork for more prototype attempts in the near future. We encourage future research to recruit a more diverse group of participants, including those with more experiences using HWDs, gather a larger data set (e.g., survey), and conduct usability tests of existing/prototyped HWDs to holistically understand BLV needs.
learnings
- Negotiated with stakeholders to align research objectives and refine scope with a realistic timeline. We narrowed the target population from BLV and hard of hearing population to BLV alone to allow more time for recruitment and more focused, in-depth insights.
- Employed strategic pauses to encourage deeper, more reflective responses. Through this interview-intensive project, I enhanced my skills in facilitating in-depth discussions, recognizing that while silence may initially feel uncomfortable, it proves invaluable in allowing participants to fully explore and articulate their thoughts.
- Ensured accessible recruitment communications and research materials (e.g., screen reader-compatible, clear language) to foster rapport-building and enhance participant experience.