Foundations of Research Design for Human-Centred Design of Interactive Technologies

The following course schedule for IAT 804 is subject to change.

Week 1

Tues Sept 14, 2021

09:30-12:20

Room: SUR 2750

Introduction

We will go over the course outline, web syllabus, texts/readings, expectations about preparation and workload, assessment, academic honesty policy.

We will talk about research approaches as a combination of worldview, research designs and methodologies.

Key terms: paradigm (worldview), epistemology, ontology, axiology, rhetoric, methodology, research design, and measuring humans, concepts, factors, constructs and variables (independent, dependent, random, controlled) .

Over the course of the term, you should learn what all of the words in this Terminology Document mean. You should add definitions/descriptions to each term as the course progresses.

Lecture Slides I will provide slides after the first lecture. Note taking helps you learn!

Required Reading

Creswell, J.W. Research Design: Qualitative, Quantitative, and Mixed Methods Approaches (2018, 5th edition):
Chapter 1 – The Selection of a Research Design (20 pages)

Olson and Kellogg, Ways of Knowing in HCI, 2014, Springer:  Prologue only

Week 2

Tues Sept 21, 2021

09:30-12:20

Room: SUR 2750

Strategies of Inquiry I

We will explore how to write a research question from different perspectives, including quantitative, qualitative, and mixed methods approaches.

We will cover an overview of quantitative methods with some details on the scientific method. These details are elaborated on in subsequent weeks when we cover controlled experiments.

Activity – Research Topic with PACT

Discuss research topic areas of interest in small groups and the perspectives/backgrounds people come from. Discuss as a large group.

Note: A research topic in SIAT typically is in the area of human-centred design of interactive technology. As such it often involves the investigation, creation/design and/or evaluation of a phenomena comprised of four elements (PACT): 1) a specific subgroup of people 2) doing an activity (typically with some goal) 3) in a context/environment(s) 4) using a form of interactive technology.

Activity – Writing a Research Question (Quant):

1. as a group, identify a research topic in the form of a phenomena with four elements: people, activity, context, interactive technology as well as identifying factors (IV) and constructs (DV) of interest as appropriate
2. write a quantitative research question about this topic on the board. be minimalistic and be specific
3. in groups ask yourselves: can it be answered? is it clear what the four elements are? constructs of interest are?
4. decide how would you answer the research question, what data would you collect using what method, etc.

Required Reading

Creswell, J.W. Research Design: Qualitative, Quantitative, and Mixed Methods Approaches :

Chapter 7 – Research Questions and Hypotheses (24 pages)
Chapter 8 – Quantitative Methods (26 pages)

Highly recommended but optional: 

ACM Interactions article “Types of Contributions in HCI"

Week 3

Tues Sept 28, 2021

09:30-12:20

Room: SUR 2750

Research Ethics

Special guest presentation from SFU’s Office of Research Ethics (ORE), 10:30am for 60-90 minutes.

ORE Slides PDF.

The Government of Canada TCPS 2 online tutorial on research ethics is to be done outside of class time.

Discuss Assignment 1: Ethics

Strategies of Inquiry II

We will work more on how to write a research question, adding to the types of questions we can ask from a quantitative or qualitative approach.

We will cover qualitative and creation methodologies. These details are iteratively elaborated on in subsequent weeks when we cover observational studies, interviews, qualitative surveys, and design ethnography. These seminars and the readings will give you the basics to do Assignment 2.

Activity – Writing a Research Question (Qual):

1. as a group, identify a research topic in the form of a phenomena with four elements: people, activity, context, interactive technology
2. write a qualitative research question about this topic on the board. be minimalistic and be specific
3. in groups ask yourselves: can it be answered? is it clear what the four elements are? constructs of interest are?
4. decide how would you answer the research question, what data would you collect using what method, etc.

Required Reading

Creswell, J.W. Research Design: Qualitative, Quantitative, and Mixed Methods Approaches:
Chapter 9 – Qualitative Methods (30 pages)
Chapter 10 – Mixed Methods (23 pages)

Optional Reading

Stappers & Giacarddi, Research Through Design from HCI Encyclopedia on web.

Week 4

Tues Oct 5, 2021

09:30-12:20

Room: SUR 2750

Three Lenses for Research at SIAT

We will explore an overview of three main ways that research occurs at SIAT, including:

effective – experimental
experience – observational
creation – artifact/design-based approaches

Try to prepare in advance — for each approach, we will discuss and clarify:

1. general practices
2. the epistemological commitments
3. areas of use
4. strengths of the approach
5. weaknesses of the approach
6. Uses of Theory We will explore the ways that theory can be used as a part of research. (Creswell Research Design – Ch 3) 

Uses of Theory: We will explore the ways that theory can be used as a part of research. (Creswell Research Design – Ch 3)

Activity

We will split into small groups to discuss the papers for Assignment 2: Short Paper. You need to have read one of these papers BEFORE class. After you discuss them in small groups, you will informally present your groups’ main findings to the class.

Due today

TCPS 2 Ethics Tutorial. Show your ethics certificate in class.

Assignment 1: Ethics in class.

Required Reading

Creswell, J.W. Research Design: Qualitative, Quantitative, and Mixed Methods Approaches:  Chapter 3 – The Use of Theory (25 pages)

Read “lightly" — Olson and Kellogg, Ways of Knowing in HCI, 2014, Springer:  Science and Design: The Implications of Different Forms of Accountability (p143-165)

Read one of the papers for Assignment 2: Short Paper. You should choose the tradition that you know the least about.

Optional If you have no background in design evaluation, you might want to read this:

Dix, A., Finlay, J., Abowd, G., Beale, R. Human-Computer Interaction, Third Edition. Prentice Hall, 2004:
Chapter 9 – Design Evaluation (44 pages)

Week 5

Tues Oct 12, 2021

09:30-12:20

Room: SUR 2750

Effective: Experiments

We will introduce the scientific method, hypothesis generation, controls, independent variables, dependent variables, etc. We will talk about between and within subject designs, recruitment and sampling procedures, validity threats (internal, external), descriptive statistics, and basic inferential statistics.

Assignments

Talk about Assignment 3a: Methodological Traditions Assignment for Effective. [Skim this in advance of class].

Activity

Small group experiment on mice.

Outside of Class Workshops:  TA will hold workshops outside of class time to demo how to run statistical testing in JMP (previously was SPSS), which you will need for Effective Assignment . You should attend if you are not familiar with inferential analysis using JMP/SPSS.  Workshop Demo Resources

Due Today

Assignment 2: Short Paper in class.

Required Reading

Olson and Kellogg, Ways of Knowing in HCI, 2014, Springer: Chapter: Experimental Research in HCI (36 pages)  Available at: SFU Library Link

Week 6

Tues Oct 19, 2021

09:30-12:20

Room: SUR 2750

Experience: Observation

Observational research is a loose class of research, traditionally undertaken in the field. In HCI/design research the approach has been adapted to lab studies. It  involves observing people in a group’s behaviors (and verbalizations) in order to understand the group’s patterns of behavior  and meaning associated with them. We will cover this as a general approach, useful in HCI and design research. We will explore in more detail the qualitative methodology of ethnography. Ethnography is used to study a group to understand shared patterns of behaviors and meaning, typically constituted by a subculture or culture. We will come at ethnography from a design perspective, which builds on a mixture of sociological and anthropological traditions. We will cover concepts including: researcher participation, immersion, culture, social beliefs, and how to collect data on culture through what people say, do, and make/use. We will discuss realist ethnography and critical ethnography as two possible types of ethnography. We will discuss the challenges with conducting ethnography and ways to write up ethnographic records/reports.

In contrast, we also will explore observational studies in a laboratory setting where participants use a prototype or design as a part of tasks or situations. This will involve methods to capture data such as logged data, observational data, think-aloud data, etc. We will talk about the challenges in conducting lab-based observational studies and how to work around them.

Assignments

Talk about Assignment 3b: Methodological Traditions Assignment for Experience.  [Skim this in advance of class].

Extra Resources for thematic analysis

Braun & Clark Thematic Analysis in Psychology

Guest et al Introduction to Applied Thematic Analysis. Sage, Chapter 1.

In-class Activity

In small groups, go and observe the interactions and activities that people do in one of three areas: a coffee shop, walking to/from classrooms, or walking in a mall. Come back to the class and discuss the main points that you observed.

Due Today

Assignment 3a: Methodological Traditions Assignment for Effective in class.

Required Reading

Olson and Kellogg, Ways of Knowing in HCI, 2014, Springer: Chapter: Reading and Interpreting Ethnography (24 pages)  Available at: SFU Library Link

Re-read Creswell, J.W. Research Design: Qualitative, Quantitative, and Mixed Methods Approaches – Chapter 9. Qualitative methods (focus on ethnography/observation).

Optional:  Creswell, J.W. Qualitative Inquiry and Research Design: Choosing Among Five Approaches (SAGE Publication, 3rd edition, 2013): 

Chapter 4 – Five Qualitative Approaches to Inquiry

Chapter 5 – Five Different Qualitative Studies

Chapter 9 – Writing a Qualitative Study
(for the above chapters, just read sections about Ethnographic studies)

Howell, Signe, Two or Three Things I Love About Ethnography, Journal of Ethnographic Theory, 7(1), 15-20. (6 pages)

Evans, A., & Rooney, B. “Methods in Psychological Research” (Sage Publications, 2013): Chapter 11 – Data Collection Methods (read only sections on observations)

Week 7

Tues Oct 26 , 2021

09:30-12:20

Room: SUR 2750

Query Methodology: Surveys and Methods: Interviews and Questionnaires

We will explore the query methodology of surveys (a quantitative methodology) and data collection methods involving queries, such as interviews (verbal), and questionnaires (written), using both qualitative and quantitative data collection. We will discuss the wording of questions, the importance of using validated interviews/questionnaires as research instruments, types of questions, and an overview of analysis procedures. We will explore in more depth the role of interviews and varying types of interviews including open-ended, closed-ended, and semi-structured. We will talk about validity and bias in query methods..

Activity

In a small group of three people, create a list of interview questions about coming to grad school and past university experiences. Conduct the interview where one person acts as the interviewer, one as the interviewee, and one as the “critiquer." After the interview, the critiquer gives feedback on the interview based on the principles and ideas found in the readings. Students then rotate between groups and perform the interview again.

Due Today

Assignment 3b: Methodological Traditions Assignment for Experience in class.

Required Reading

Olson, J. and Kellogg, W. Ways of Knowing in HCI, Springer, 2014.

Chapter 10 – Survey Research in HCI (available online at SFU library) (38 pages). Note that the authors refer to “survey methods" and appear to mean query-based methodologies, and later interview and questionnaire methods of data collection and analysis. Available at: SFU Library Link

Weiss, R. Learning From Strangers: The Art and Method of Qualitative Interview Studies, The Free Press (1994):
Chapter 1: Introduction (15 pages)
Chapter 4: Interviewing (58 pages, but they are short pages)

Optional:

Fontana, A. and James F. (1994). Interviewing: The Art of Science. In Denzin, N. and Lincoln, Y. (Eds.) The Handbook of Qualitative Research (pp. 361-76). Thousand Oaks: Sage Publications. [Read up to first column on p. 368]

Evans, A., & Rooney, B. “Methods in Psychological Research” (Sage Publications, 2013)
Chapter 11 – Data Collection Methods (read only sections on surveys and questionnaires)

Week 8

Tues Nov 2, 2021

09:30-12:20

Room: SUR 2750

Creation: Art/Artifact/Design-Based Approaches

We will cover research practices that occur through the making of art or an artifact where the learning comes through the making process as well as the setup and self-usage of the technology. In many cases, this type of research does not have user research studies as a part of it. We will explore ideas around sketching, design portfolios, brainstorming, research through design, design scenarios, speculative design, and video prototyping. We will talk about artistic research vs. artistic practice, research into art, research through art, and research for art.

Activity

In a small group, think of objects that reflect notions of time to you. Describe ways of incorporating notions of time within them. Think about what “time” means. Sketch out design ideas. Come up with one design idea to share with the group.

Examples of Research Through Design:

William Odom,et al. 2018. Attending to slowness and temporality with Olly and Slow Game: A design inquiry into supporting longer-term relations with everyday computational objects. In Proceedings of the Conference on Human Factors in Computing Systems (CHI ’18). ACM, New York, NY, USA, Paper 77, 13 pages. DOI. [from assignment 2]

Alissa N. Antle, Leslie  Chesick, L. and ELgin-Skye McLaren. 2018. Opening up the design space of neurofeedback brain computer interfaces for children: Five strong concepts. ACM Transactions on Computer-Human Interaction (ToCHI), 24(6), 1-38. DOI

Hybrid Art/Design: 

Vote with Your Feet (CHI 2017 Art Program): (video), (PDF) [art track at CHI but research process feels like research through design — check out video for class]

Examples of Art-based Research
Fukushima Audio Census (CHI 2017 Art Program): (video), (PDF)

Never Alone: a Video Agents Based Generative Video-Sound Installation (CHI 2017 Art Program): (video), (PDF)

Ambient Video: Seasons [check out for class]

Assignments

Talk about Assignment 3c: Methodological Traditions Assignment for Creation. [skim this in advance of class]

Required Readings

Design focused: 

Olson, J. and Kellogg, W. Ways of Knowing in HCI, Springer, 2014.. Chapter by John Zimmerman and Jodi Forlizzi, Research Through Design p167-189.

Daniel Fallman. 2003. Design-oriented human-computer interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’03). ACM, New York, NY, USA, 225-232. DOI

Art focused: 

Frayling, C. (1993). Research in art and design. Royal Coll. of Art, London (United Kingdom).

Borgdorff, H. Chapter 3: The Production of Knowledge in Artistic Research, in Biggs, M., & Karlsson, H. (Eds.). (2010). The Routledge companion to research in the arts. Routledge.

Week 9

Tues Nov 9, 2021

09:30-12:20

Room: SUR 2750

Compare and Contrast: Qualitative Methods 

Grounded Theory

We will cover an overview of Grounded Theory looking at the use of literature and theory, participant recruitment and sampling, analysis procedures through open, axial, and selective coding, and writing up a Grounded Theory study. We will also talk about the different theoretical perspectives on Grounded Theory.

Case Studies

We will cover an overview of single and multiple case studies, within-site and multi-site, intrinsic case, instrumental case, and collective case. We will explore both holistic and embedded analysis, study write-up, and the challenges with case studies.

[We will do a research design activity on grounded theory and case studies]

Assignments: 

Talk about Assignment 4: In-Class Presentation.

We will discuss your final paper assignment — creation of a Research Proposal — and the importance of framing research topic, research problem and research questions in a way that is tightly connected, involves the correct type of question at a level of specificity that ensures rigor in your research. 

Due Today

Assignment 3c: Methodological Traditions assignment for Creation in class.

Required Reading

Creswell, “Qualitative Inquiry and Research Design: Choosing Among Five Approaches” (SAGE Publication, 3rd edition, 2013). [on reserve in library or borrow from Dr. Antle]
Chapter 4 – Five Qualitative Approaches to Inquiry
Chapter 5 – Five Different Qualitative Studies
Chapter 9 – Writing a Qualitative Study
Chapter 10 – Standards of Validation and Evaluation

In the above chapters, just read the sections on Grounded Theory and Case Studies

Week 10

Tues Nov 16 2021

09:30-12:20

Room: SUR 2750

Assignment 3C Videos and popcorn — TA Organizing w/ Grad Students

Historical, Biographical and Narrative Approaches

We will cover biographical accounts, life histories, and oral histories. We will detail ways to write-up these narrative studies focused on one or more individuals’ accounts and the challenges in conducting narrative  studies.

Phenomenology

We will cover the focal points of a phenomenological study, the philosophical underpinnings of the approach, epoche, hermeneutical phenomenology, transcendental phenomenology, and horizontalization as an analysis procedure. We will also explore the challenges with phenomenology and how to write-up phenomenological studies.

Activity

We will lead an activity on phenomenology where you will report your phenomenological experience after watching a video. In pairs, you will analyze each other’s text individually, then in groups of four, and finally you will discuss your analysis with the class.

Assignments

Any Q&A on Assignment 4: In-Class Presentation — prepare individual slides for next week.

Any Q&A on Assignment 5: Research Proposal — prepare topic and RQs for next week.

Required Reading

Creswell, “Qualitative Inquiry and Research Design: Choosing Among Five Approaches” (SAGE Publication, 3rd edition, 2013) [on reserve in library or borrow from Dr. Antle]:
Chapter 4 – Five Qualitative Approaches to Inquiry
Chapter 5 – Five Different Qualitative Studies
Chapter 9 – Writing a Qualitative Study
Chapter 10 – Standards of Validation and Evaluation

In the above chapters, just read the sections on Narrative Research and Phenomenology

Week 11

Tues Nov 23 2021

09:30-12:20

Room: SUR 2750

Activity: Workshop for Assignment 4 Presentations

We will do in-class activities as part of Assignment 4. You will break into small groups and do informal presentations of your presentation to the small group. You will use this time to discuss how to combine your presentations as part of the presentations due next week.

Activity: Workshop Assignment 5 Final Papers

We will use the first half of class to informally present and receive feedback on part of the research questions for your final paper. You should come to class with your research questions written and printed on paper. You will then split into three groups: Group 1 will break into pairs and read/comment on the research questions of their partners; Group 2 will do ‘speed dating’ and read/comment on research questions with partners; Group 3 will write one research question on the whiteboard and discuss it with the instructor and TA. After X minutes, each group will rotate

We will also address Q&A of Assignment 5: Research Proposal,  and talk about scope, structures and examples.

We will introduce the concept of using an alignment chart to connect research questions, constructs, data types, collection and analysis methods for your research design. 

Due Today

Submit your own presentation as part of Assignment 4: In-Class Presentation before class. 

Set up an appointment to do a group presentation dry-run w/ Alex and/or Ladan anytime this week! 

*UPDATE NOV 23, 2021: submit 1-page critique via email by Thursday Nov 25 @ 11:59pm PST

Required Reading

Week 12

Tues Nov 30, 2021

09:30-12:20

Room: SUR 2750

Presentations for Assignment 4

Teams present. Sign up order in class.

Q&A Assignment 5: Research Proposal as needed.

Due Today

Submit your group’s presentation as part of Assignment 4: In-Class Presentation.
*UPDATED NOV 23, 2021 submit your 1 page critique of your group members’ slides DUE Thursday, November 25 11:59pm PST via email

Required Reading

NA

Week 13

Tues Dec 7, 2021

09:30-12:20

Room: SUR 2750

Activity: Workshop for Final Paper

We will conduct in-class activities to critique and prepare Assignment 5: Research Proposal. You need to come to class with a full draft of your final paper. You will break into small groups and share your drafts with each other, read, and discuss.

Due Today

You must submit a draft of your paper into EasyChair here by 11:59 (midnight), December 7 2021*: https://easychair.org/conferences/?conf=siatgrm2021 

*revise paper based on feedback in class

Due Soon

Submit your peer review critique to EasyChair by 11:59pm (midnight), December 10 2021.

You may set up a time for instructor/TA to review your proposal twice during this period.

Email your final paper & revision report by 11:59pm (midnight), December 15 2021.

Required Reading

NA