Team

Dr. Alissa N. Antle (PI, research and design lead)
Dr. Min Fan (Communication University of China, Beijing, China)
Dr. Victor Cheung (post-doc)

Graduate students:
Shubhra Sarker
Jianyu Fan
Boxiao Gong

Learn more here about original Tangible PhonoBlocks system.

Project Summary

AR PhonoBlocks was an exploration of technical variations using everyday technologies (e.g. iPad + fridge magnets) in order to make the effective tangible-PhonoBlocks system (see PhonoBlocks: Tangible System Windows) more accessible for broader deployment into schools and families. We created three AR versions of the system using different technical approaches to sense physical letters and provide digital visual overlays that can provide colour cues, including: 1) camera sensing of fiducial markers using Vuforial and AR Toolkit; 2) custom-created machine learning for optical character recognition via a trained convolutional neural network; and 3) capacitive sensing of custom letters with 3D laser printed metal pin patterns.

Research Objectives

Explore how to develop and evaluate augmented reality (AR) multimodal reading systems for children at-risk for dyslexia and other English language learners, including English as a foreign language learners.

Social Impact Goals

Enable children, aged 6-7 with reading and spelling challenges, to learn the alphabetic principle more effectively and efficiently using a mobile tablet system.

Keywords

Augmented reality, dyslexia, children, design for vulnerable populations, literacy, reading, spelling, English language learners (ELL), field studies.

Technology

Fridge magnet letters, 3D printed capacitive strips, AR Toolkit, Vuforial plugin, custom convolutional neural network, Unity, Processing, Android and Apple tablets.

Community Partners

Beizuo Region (背坐村), Guizhou, China.

Funders

SSHRC Insight