Team

Alissa N. Antle (PI, research and design lead)
Dr. Min Fan (Communication University of China, Beijing, China)
Dr. Victor Cheung (post-doc)

Graduate students:
Alexandra Kitson (post-doc)
Zoe Dao-Kroeker (NSERC USRA, VPR USRA)

Learn more here

Project Summary

AR PhonoBlocks was an exploration of technical variations using everyday technologies (e.g. ipad + fridge magnets) in order to make the effective tangible-PhonoBlocks system (see PhonoBlocks: Tangible System Windows) more accessible for broader deployment into schools and families. We created three AR versions of the system using different technical approaches to sense physical letters and provide digital visual overlays that can provide colour cues, including: 1) camera sensing of fiducial markers using Vuforial and AR Toolkit; 2) custom-created machine learning for optical character recognition via a trained convolutional neural network; and 3) capacitive sensing of custom letters with 3D laser printed metal pin patterns.

Research Objectives

Design, develop and evaluate an augmented reality (AR) multimodal reading system for children at-risk for dyslexia and other English language learners, including English as a foreign language learners.

Social Impact Goals

Enable children, aged 6-7 with reading and spelling challenges, to learn the alphabetic principle more effectively and efficiently with a less instructor intensive intervention using tablet technology.

Keywords

Augmented reality, embodied interaction design, design for vulnerable populations, children, literacy, reading, spelling, dyslexia, English as a foreign language training.

Technology

Fridge magnet letters, 3D printed capacitive strips, AR Toolkit, Vuforial plugin, custom convolutional neural network, Unity, Processing, Android and Apple tablets.

Community Partners

Beizuo Region (背坐村), Guizhou, China.

Funders

SSHRC Insight