Workgreen2019

Auto-Suggesting Browsing Actions for Personalized Web Screen Reading

Vikas Ganjigunte Ashok · Syed Masum Billah · Yevgen Borodin · I. V. Ramakrishnan

19 citations0/0 files storedACM Conference on User Modeling, Adaptation and Personalization (UMAP)

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Digital Accessibility for Disabilities Usability and User Interface Design Assistive Technology in Communication and Mobility

Abstract

Web browsing has never been easy for blind people, primarily due to the serial press-and-listen interaction mode of screen readers -- their "go-to'' assistive technology. Even simple navigational browsing actions on a page require a multitude of shortcuts. Auto-suggesting the next browsing action has the potential to assist blind users in swiftly completing various tasks with minimal effort. Extant auto-suggest feature in web pages is limited to filling form fields; in this paper, we generalize it to any web screen-reading browsing action, e.g., navigation, selection, etc. Towards that, we introduce SuggestOmatic, a personalized and scalable unsupervised approach for predicting the most likely next browsing action of the user, and proactively suggesting it to the user so that the user can avoid pressing a lot of shortcuts to complete that action. SuggestOmatic rests on two key ideas. First, it exploits the user's Action History to identify and suggest a small set of browsing actions that will, with high likelihood, contain an action which the user will want to do next, and the chosen action is executed automatically. Second, the Action History is represented as an abstract temporal sequence of operations over semantic web entities called Logical Segments - a collection of related HTML elements, e.g., widgets, search results, menus, forms, etc.; this semantics-based abstract representation of browsing actions in the Action History makes SuggestOmatic scalable across websites, i.e., actions recorded in one website can be used to make suggestions for other similar websites. We also describe an interface that uses an off-the-shelf physical Dial as an input device that enables SuggestOmatic to work with any screen reader. The results of a user study with 12 blind participants indicate that SuggestOmatic can significantly reduce the browsing task times by as much as 29% when compared with a hand-crafted macro-based web automation solution.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 19 citations exist upstream.

References

2007What Frustrates Screen Reader Users on the Web: A Study of 100 Blind Users 2010More than meets the eye 2009Trailblazer 2017Web Screen Reading Automation Assistance Using Semantic Abstraction 2018SteeringWheel 2013Predictive web automation assistant for people with vision impairments 2014Widget Classification with Applications to Web Accessibility 2017Speed-Dial

Cited by

2023Speaking with My Screen Reader: Using Audio Fictions to Explore Conversational Access to Interfaces 2020A Web Augmentation Framework for Accessibility Based on Voice Interaction 2022InSupport: Proxy Interface for Enabling Efficient Non-Visual Interaction with Web Data Records 2025Inaccessible and Deceptive: Examining Experiences of Deceptive Design with People Who Use Visual Accessibility Technology 2023Enabling Efficient Web Data-Record Interaction for People with Visual Impairments via Proxy Interfaces 2025Augmenting Tourism Websites With Voice Commands for Increasing Accessibility in Smart Tourism Destinations

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.