Automated Accessibility Analysis of Dynamic Content Changes on Mobile Apps

With mobile apps playing an increasingly vital role in our daily lives, the importance of ensuring their accessibility for users with disabilities is also growing. Despite this, app developers often overlook the accessibility challenges encountered by users of assistive technologies, such as screen readers. Screen reader users typically navigate content sequentially, focusing on one element at a time, unaware of changes occurring elsewhere in the app. While dynamic changes to content displayed on an app's user interface may be apparent to sighted users, they pose significant accessibility obstacles for screen reader users. Existing accessibility testing tools are unable to identify challenges faced by blind users resulting from dynamic content changes. In this work, we first conduct a formative user study on dynamic changes in Android apps and their accessibility barriers for screen reader users. We then present TIMESTUMP, an automated framework that leverages our findings in the formative study to detect accessibility issues regarding dynamic changes. Finally, we empirically evaluate TIMESTUMP on real-world apps to assess its effectiveness and efficiency in detecting such accessibility issues. TIMESTUMP is able to detect inaccessible dynamic content changes with a precision of 94% and a recall of 92%.

How does TIMESTUMP work?

Relying on the insights gained from formative interviews with screen reader users, we developed an automated framework called TIMESTUMP, designed to identify accessibility issues associated with dynamic content changes. In the initial phase, we install an Android app on a Virtual Machine~(VM) and utilize a GUI crawler to automatically explore the app, generating a diverse set of states in an app. The Snapshot Recorder tracks app states and records snapshots of distinct screens. In the second phase, we extract the list of actionable elements in each recorded snapshot. Then, the Interaction Automator systematically executes each action, capturing information before, during, and after the action. This rich dataset is passed to the third phase, where the Localizer component assesses the gathered information, precisely flagging accessibility issues stemming from dynamic content changes.

[TIMESTUMP picture]

Artifacts

The artifacts are publicly available here.

Publications

More details about can be found in our publication below:


[seal's logo]
[uci's logo]