2020
Victor Cheung, Alissa N. Antle, Shubhra Sarker, Min Fan, Jianyu Fan, Philippe Pasquier. 2020. Techniques for Augmented-Tangibles on Mobile Devices for Early Childhood Learning Proceedings Article . In Proceedings of the Interaction Design and Children Conference, IDC '20 Association for Computing Machinery, London, United Kingdom, 589–601, .
Abstract | BibTeX | Tags: augmented reality, early childhood learning, education mobile apps, tablets, tangible interaction
@inproceedings{10.1145/3392063.3394412,
title = {Techniques for Augmented-Tangibles on Mobile Devices for Early Childhood Learning},
author = {Victor Cheung and Alissa N. Antle and Shubhra Sarker and Min Fan and Jianyu Fan and Philippe Pasquier},
url = {https://doi.org/10.1145/3392063.3394412},
doi = {10.1145/3392063.3394412},
isbn = {9781450379816},
year = {2020},
date = {2020-01-01},
booktitle = {Proceedings of the Interaction Design and Children Conference},
pages = {589–601},
publisher = {Association for Computing Machinery},
address = {London, United Kingdom},
series = {IDC '20},
abstract = {Integrating physical learning materials with mobile device applications may have benefits for early childhood learning. We present three techniques for creating a hybrid tangible-augmented reality (T-AR) enabling technology platform. This platform enables researchers to develop applications that use readily available physical learning materials, such as letters, numbers, symbols or shapes. The techniques are visual marker-based; computer-vision and machine-learning; and capacitive touches. We describe details of implementation and demonstrate these techniques through a use case of a reading tablet app that uses wooden/plastic letters for input and augmented output. Our comparative analysis revealed that the machine-learning technique most flexibly sensed different physical letter sets but had variable accuracy impacted by lighting and tracking lag at this time. Lastly, we demonstrate how this enabling technology can be generalized to a variety of early learning apps through a second use case with physical numbers.},
keywords = {augmented reality, early childhood learning, education mobile apps, tablets, tangible interaction},
pubstate = {published},
tppubtype = {inproceedings}
}
Integrating physical learning materials with mobile device applications may have benefits for early childhood learning. We present three techniques for creating a hybrid tangible-augmented reality (T-AR) enabling technology platform. This platform enables researchers to develop applications that use readily available physical learning materials, such as letters, numbers, symbols or shapes. The techniques are visual marker-based; computer-vision and machine-learning; and capacitive touches. We describe details of implementation and demonstrate these techniques through a use case of a reading tablet app that uses wooden/plastic letters for input and augmented output. Our comparative analysis revealed that the machine-learning technique most flexibly sensed different physical letter sets but had variable accuracy impacted by lighting and tracking lag at this time. Lastly, we demonstrate how this enabling technology can be generalized to a variety of early learning apps through a second use case with physical numbers.
2015
Augusto Esteves, Saskia Bakker, Alissa N. Antle, Aaron May, Jillian Warren, Ian Oakley. 2015. The ATB Framework: Quantifying and Classifying Epistemic Strategies in Tangible Problem-Solving Tasks Proceedings Article . In Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction, TEI '15 Association for Computing Machinery, Stanford, California, USA, 13–20, .
@inproceedings{10.1145/2677199.2680546,
title = {The ATB Framework: Quantifying and Classifying Epistemic Strategies in Tangible Problem-Solving Tasks},
author = {Augusto Esteves and Saskia Bakker and Alissa N. Antle and Aaron May and Jillian Warren and Ian Oakley},
url = {https://doi-org.proxy.lib.sfu.ca/10.1145/2677199.2680546},
doi = {10.1145/2677199.2680546},
isbn = {9781450333054},
year = {2015},
date = {2015-01-01},
booktitle = {Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction},
pages = {13–20},
publisher = {Association for Computing Machinery},
address = {Stanford, California, USA},
series = {TEI '15},
abstract = {In task performance, pragmatic actions refer to behaviors that make direct progress, while epistemic actions involve altering the world so that cognitive processes are faster, more reliable or less taxing. Epistemic actions are frequently presented as a beneficial consequence of interacting with tangible systems. However, we currently lack tools to measure epistemic behaviors, making substantiating such claims highly challenging. This paper addresses this problem by presenting ATB, a video-coding framework that enables the identification and measurement of different epistemic actions during problem-solving tasks. The framework was developed through a systematic literature review of 78 papers, and analyzed through a study involving a jigsaw puzzle -- a classical spatial problem -- involving 60 participants. In order to assess the framework's value as a metric, we analyze the study with respect to its reliability, validity and predictive power. The broadly supportive results lead us to conclude that the ATB framework enables the use of observed epistemic behaviors as a performance metric for tangible systems. We believe that the development of metrics focused explicitly on the properties of tangible interaction are currently required to gain insight into the genuine and unique benefits of tangible interaction. The ATB framework is a step towards this goal.},
keywords = {epistemic actions, tangible interaction, video-coding},
pubstate = {published},
tppubtype = {inproceedings}
}
In task performance, pragmatic actions refer to behaviors that make direct progress, while epistemic actions involve altering the world so that cognitive processes are faster, more reliable or less taxing. Epistemic actions are frequently presented as a beneficial consequence of interacting with tangible systems. However, we currently lack tools to measure epistemic behaviors, making substantiating such claims highly challenging. This paper addresses this problem by presenting ATB, a video-coding framework that enables the identification and measurement of different epistemic actions during problem-solving tasks. The framework was developed through a systematic literature review of 78 papers, and analyzed through a study involving a jigsaw puzzle -- a classical spatial problem -- involving 60 participants. In order to assess the framework's value as a metric, we analyze the study with respect to its reliability, validity and predictive power. The broadly supportive results lead us to conclude that the ATB framework enables the use of observed epistemic behaviors as a performance metric for tangible systems. We believe that the development of metrics focused explicitly on the properties of tangible interaction are currently required to gain insight into the genuine and unique benefits of tangible interaction. The ATB framework is a step towards this goal.
2014
Augusto Esteves, Saskia Bakker, Alissa N. Antle, Aaron May, Jillian Warren, Ian Oakley. 2014. Classifying Physical Strategies in Tangible Tasks: A Video-Coding Framework for Epistemic Actions Proceedings Article . In CHI '14 Extended Abstracts on Human Factors in Computing Systems, CHI EA '14 Association for Computing Machinery, Toronto, Ontario, Canada, 1843–1848, .
@inproceedings{10.1145/2559206.2581185,
title = {Classifying Physical Strategies in Tangible Tasks: A Video-Coding Framework for Epistemic Actions},
author = {Augusto Esteves and Saskia Bakker and Alissa N. Antle and Aaron May and Jillian Warren and Ian Oakley},
url = {https://doi.org/10.1145/2559206.2581185},
doi = {10.1145/2559206.2581185},
isbn = {9781450324748},
year = {2014},
date = {2014-01-01},
booktitle = {CHI '14 Extended Abstracts on Human Factors in Computing Systems},
pages = {1843–1848},
publisher = {Association for Computing Machinery},
address = {Toronto, Ontario, Canada},
series = {CHI EA '14},
abstract = {Tangible interaction is a compelling interface paradigm that elegantly merges the fluency of physical manipulation with the flexibility of digital content. However, it is currently challenging to understand the real benefits and advantages of tangible systems. To address this problem, this paper argues that we need new evaluation techniques capable of meaningfully assessing how users perform with tangible, physical objects. Working towards this aim, it presents a video-coding framework that supports the granular identification of epistemic actions (physical actions that are made to simplify cognitive work) during tangible tasks. The framework includes 20 epistemic actions, identified through a systematic literature review of 77 sources. We argue that data generated by applying this process will help us better understand epistemic behavior and, ultimately, lead to the generation of novel, grounded design insights to support physically-grounded cognitive strategies in tangible tasks.},
keywords = {epistemic action, tangible interaction, video coding},
pubstate = {published},
tppubtype = {inproceedings}
}
Tangible interaction is a compelling interface paradigm that elegantly merges the fluency of physical manipulation with the flexibility of digital content. However, it is currently challenging to understand the real benefits and advantages of tangible systems. To address this problem, this paper argues that we need new evaluation techniques capable of meaningfully assessing how users perform with tangible, physical objects. Working towards this aim, it presents a video-coding framework that supports the granular identification of epistemic actions (physical actions that are made to simplify cognitive work) during tangible tasks. The framework includes 20 epistemic actions, identified through a systematic literature review of 77 sources. We argue that data generated by applying this process will help us better understand epistemic behavior and, ultimately, lead to the generation of novel, grounded design insights to support physically-grounded cognitive strategies in tangible tasks.
2013
Alissa N. Antle, Sijie Wang. 2013. Comparing Motor-Cognitive Strategies for Spatial Problem Solving with Tangible and Multi-Touch Interfaces Proceedings Article . In Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction, TEI '13 Association for Computing Machinery, Barcelona, Spain, 65–72, .
Abstract | BibTeX | Tags: epistemic actions, hands-on interaction, interactive tabletops, multi-touch interaction, tangible interaction
@inproceedings{10.1145/2460625.2460635,
title = {Comparing Motor-Cognitive Strategies for Spatial Problem Solving with Tangible and Multi-Touch Interfaces},
author = {Alissa N. Antle and Sijie Wang},
url = {https://doi-org.proxy.lib.sfu.ca/10.1145/2460625.2460635},
doi = {10.1145/2460625.2460635},
isbn = {9781450318983},
year = {2013},
date = {2013-01-01},
booktitle = {Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction},
pages = {65–72},
publisher = {Association for Computing Machinery},
address = {Barcelona, Spain},
series = {TEI '13},
abstract = {We present the results from a mixed methods comparison of a tangible and a multi-touch interface for a spatial problem solving task. We applied a modified version of a previous framework to code video of hand-based events. This enabled us to investigate motor-cognitive strategies as well as traditional performance and preference constructs. Sixteen adult participants completed jigsaw puzzles using both interfaces. Our results suggest that the 3D manipulation space, eyes-free tactile feedback, and the offline workspace afforded by the tangible interface enabled more efficient and effective motor-cognitive strategies. We discuss the implications of these findings for interface design; including suggestions for spatial and visual structures that may support epistemic strategies, and hybrid interfaces where tangible handles may be used as structural anchors as well as controls and representational objects.},
keywords = {epistemic actions, hands-on interaction, interactive tabletops, multi-touch interaction, tangible interaction},
pubstate = {published},
tppubtype = {inproceedings}
}
We present the results from a mixed methods comparison of a tangible and a multi-touch interface for a spatial problem solving task. We applied a modified version of a previous framework to code video of hand-based events. This enabled us to investigate motor-cognitive strategies as well as traditional performance and preference constructs. Sixteen adult participants completed jigsaw puzzles using both interfaces. Our results suggest that the 3D manipulation space, eyes-free tactile feedback, and the offline workspace afforded by the tangible interface enabled more efficient and effective motor-cognitive strategies. We discuss the implications of these findings for interface design; including suggestions for spatial and visual structures that may support epistemic strategies, and hybrid interfaces where tangible handles may be used as structural anchors as well as controls and representational objects.
2009
Saskia Bakker, Alissa N. Antle, Elise Hoven. 2009. Identifying Embodied Metaphors in Children's Sound-Action Mappings Proceedings Article . In Proceedings of the 8th International Conference on Interaction Design and Children, IDC '09 Association for Computing Machinery, Como, Italy, 140–149, .
Abstract | BibTeX | Tags: auditory interaction, children, design research, embodied metaphor, embodied schemas, image schemas, learning, music, sound enactment, tangible interaction, Tangible User Interfaces
@inproceedings{10.1145/1551788.1551812,
title = {Identifying Embodied Metaphors in Children's Sound-Action Mappings},
author = {Saskia Bakker and Alissa N. Antle and Elise Hoven},
url = {https://doi-org.proxy.lib.sfu.ca/10.1145/1551788.1551812},
doi = {10.1145/1551788.1551812},
isbn = {9781605583952},
year = {2009},
date = {2009-01-01},
booktitle = {Proceedings of the 8th International Conference on Interaction Design and Children},
pages = {140–149},
publisher = {Association for Computing Machinery},
address = {Como, Italy},
series = {IDC '09},
abstract = {Physical activity and manipulating physical objects can be beneficial for learning. Earlier studies [2] have shown that interaction models that rely on unconscious and embodied knowledge (based on embodied metaphors) can benefit the learning process. However, more than one embodied metaphor might be applicable. In this paper, we present the results of a user study (n=65) designed to identify embodied metaphors seven to nine year old children use when enacting abstract concepts related to musical sound. The results provide evidence that multiple different embodied metaphors can unconsciously be used to structure the understanding of these concepts. In addition, we have identified and categorized commonly used metaphors based on the children's enactments of changing sound concepts.},
keywords = {auditory interaction, children, design research, embodied metaphor, embodied schemas, image schemas, learning, music, sound enactment, tangible interaction, Tangible User Interfaces},
pubstate = {published},
tppubtype = {inproceedings}
}
Physical activity and manipulating physical objects can be beneficial for learning. Earlier studies [2] have shown that interaction models that rely on unconscious and embodied knowledge (based on embodied metaphors) can benefit the learning process. However, more than one embodied metaphor might be applicable. In this paper, we present the results of a user study (n=65) designed to identify embodied metaphors seven to nine year old children use when enacting abstract concepts related to musical sound. The results provide evidence that multiple different embodied metaphors can unconsciously be used to structure the understanding of these concepts. In addition, we have identified and categorized commonly used metaphors based on the children's enactments of changing sound concepts.
Alissa N. Antle, Milena Droumeva, Daniel Ha. 2009. Hands on What? Comparing Children's Mouse-Based and Tangible-Based Interaction Proceedings Article . In Proceedings of the 8th International Conference on Interaction Design and Children, IDC '09 Association for Computing Machinery, Como, Italy, 80–88, .
Abstract | BibTeX | Tags: children, comparative experiment, digital tabletop, embodied interaction, evaluation, input methods, interaction styles, jigsaw puzzle, Methodology, object manipulation, tangible computing, tangible interaction, video analysis
@inproceedings{10.1145/1551788.1551803,
title = {Hands on What? Comparing Children's Mouse-Based and Tangible-Based Interaction},
author = {Alissa N. Antle and Milena Droumeva and Daniel Ha},
url = {https://doi-org.proxy.lib.sfu.ca/10.1145/1551788.1551803},
doi = {10.1145/1551788.1551803},
isbn = {9781605583952},
year = {2009},
date = {2009-01-01},
booktitle = {Proceedings of the 8th International Conference on Interaction Design and Children},
pages = {80–88},
publisher = {Association for Computing Machinery},
address = {Como, Italy},
series = {IDC '09},
abstract = {We investigate the similarities and differences -- in terms of quantitative performance and qualitative behaviors -- between how children solve an object manipulation task using mouse-based input versus tangible-based input. This work examines the assumption common in tangible computing that direct physical manipulation is beneficial for certain spatial tasks. We describe an ecologically valid comparison of mouse-based versus tangible-based input for a jigsaw puzzle task in order to better understand the tradeoffs in choosing input and interaction styles. We include a traditional cardboard puzzle for comparative purposes. The results of an experiment with 132 children indicate children are more successful and faster at solving puzzles using a tangible-based approach. Detailed temporal analysis indicates that pairs in the tangible group spend most of their time using a combination of epistemic and pragmatic actions which support mental problem solving. Conversely, pairs in the mouse group use an ineffective trial and error strategy.},
keywords = {children, comparative experiment, digital tabletop, embodied interaction, evaluation, input methods, interaction styles, jigsaw puzzle, Methodology, object manipulation, tangible computing, tangible interaction, video analysis},
pubstate = {published},
tppubtype = {inproceedings}
}
We investigate the similarities and differences -- in terms of quantitative performance and qualitative behaviors -- between how children solve an object manipulation task using mouse-based input versus tangible-based input. This work examines the assumption common in tangible computing that direct physical manipulation is beneficial for certain spatial tasks. We describe an ecologically valid comparison of mouse-based versus tangible-based input for a jigsaw puzzle task in order to better understand the tradeoffs in choosing input and interaction styles. We include a traditional cardboard puzzle for comparative purposes. The results of an experiment with 132 children indicate children are more successful and faster at solving puzzles using a tangible-based approach. Detailed temporal analysis indicates that pairs in the tangible group spend most of their time using a combination of epistemic and pragmatic actions which support mental problem solving. Conversely, pairs in the mouse group use an ineffective trial and error strategy.
Alissa N. Antle, Nima Motamedi, Karen Tanenbaum, Zhen Lesley Xie. 2009. The EventTable Technique: Distributed Fiducial Markers Proceedings Article . In Proceedings of the 3rd International Conference on Tangible and Embedded Interaction, TEI '09 Association for Computing Machinery, Cambridge, United Kingdom, 307–313, .
Abstract | BibTeX | Tags: distributed markers, fiducial markers, reacTIVision, tangible interaction, visual marker tracking
@inproceedings{10.1145/1517664.1517728,
title = {The EventTable Technique: Distributed Fiducial Markers},
author = {Alissa N. Antle and Nima Motamedi and Karen Tanenbaum and Zhen Lesley Xie},
url = {https://doi-org.proxy.lib.sfu.ca/10.1145/1517664.1517728},
doi = {10.1145/1517664.1517728},
isbn = {9781605584935},
year = {2009},
date = {2009-01-01},
booktitle = {Proceedings of the 3rd International Conference on Tangible and Embedded Interaction},
pages = {307–313},
publisher = {Association for Computing Machinery},
address = {Cambridge, United Kingdom},
series = {TEI '09},
abstract = {The EventTable technique is a tangible object tracking technique implemented on a camera vision based tabletop platform. The technique supports an event-driven -- rather than object centric -- tracking technique. Fiducial markers are distributed between objects. When objects are brought into a proximal or connected relationship, a whole marker is formed and recognized by the tracking system. Thus, rather than tracking each individual object, the system tracks user-driven events that occur when two or more objects are proximal. The technique can be used in addition to individual object tracking and touch tracking. This approach provides a reliable and flexible approach to tabletop object tracking for a wide variety of tabletop activities. We describe three prototype applications to illustrate how the distributed marker technique can be applied. We describe the advantages and limitations of this approach. We conclude with a brief discussion of how the EventTable technique enables a shift in human computer interaction research from an information-centric to an action-centric epistemological view on how users' create meaning.},
keywords = {distributed markers, fiducial markers, reacTIVision, tangible interaction, visual marker tracking},
pubstate = {published},
tppubtype = {inproceedings}
}
The EventTable technique is a tangible object tracking technique implemented on a camera vision based tabletop platform. The technique supports an event-driven -- rather than object centric -- tracking technique. Fiducial markers are distributed between objects. When objects are brought into a proximal or connected relationship, a whole marker is formed and recognized by the tracking system. Thus, rather than tracking each individual object, the system tracks user-driven events that occur when two or more objects are proximal. The technique can be used in addition to individual object tracking and touch tracking. This approach provides a reliable and flexible approach to tabletop object tracking for a wide variety of tabletop activities. We describe three prototype applications to illustrate how the distributed marker technique can be applied. We describe the advantages and limitations of this approach. We conclude with a brief discussion of how the EventTable technique enables a shift in human computer interaction research from an information-centric to an action-centric epistemological view on how users' create meaning.