IAT804 – Coursework

Timeline
Coursework
Resources
Announcements

IAT 804 – Foundations of Research Design for Human-Centred Design of Interactive Technologies, is a graduate course taught in the School of Interactive Art and Technology at Simon Fraser University in Surrey, BC, Canada. This course provides an introduction to different epistemological worldviews, research approaches and methodological traditions of inquiry that are used to conduct research within SIAT. Students are introduced to a range of ways of knowing and inquiring in human-centred design, development and analysis of interactive technologies including scientific, social science, humanities, design and art-based approaches.

Fall 2021 Leader course delivery and development: Dr. Alex Kitson [akitson@sfu.ca]

Instructor & mentor: Dr. Alissa Antle [aantle@sfu.ca]

Teaching Assistant: Ladan Fathi [ladan_fathi@sfu.ca]

Assignments

Assignment 1: Research Ethics (5%)

The Government of Canada has created an online tutorial to teach people about research ethics as documented in the Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans (TCPS 2). You need to complete the online tutorial found here and print out the certificate of completion. The tutorial will likely take you 2 to 3 hours as it carefully goes through a number of modules which are followed by questions that you need to answer.

Once you have completed the tutorial, you need to write a document that is up to one page in length (single spaced). In the document, you need to do the following:

  1. Describe the goal of TCPS 2 in your own words.
  2. Provide a definition of ethics as it pertains to research involving human participants.
  3. Describe three ethical concerns that you as a  student researcher at SIAT may need to think about and deal with as part of your graduate research. For each concern, provide at least one way to overcome it.

 

Submission:

Bring a paper copy to class.

Assignment 2: Short Paper and In-Class Group Discussion (15%)

The goal of this short paper assignment is to practice identifying and analyzing methodological approaches, research designs, and data collection and analysis methods for an existing piece of research.

Choose a paper from one of the three lenses of research:

1. Effective – experimental

2. Experience – observational

  • Rui Pan, Henry Lo, and Carman Neustaedter. 2017. Collaboration, awareness, and communication in real-life escape rooms. In Proceedings of the 2017 Conference on Designing Interactive Systems (DIS '17). ACM, New York, NY, USA, 1353-1364.  DOI: https://doi.org/10.1145/3064663.3064767

3. Creation – design or art based approaches

  •  William Odom, Ron Wakkary, Ishac Bertran, Matthew Harkness, Garnet Hertz, Jeroen Hol, Henry Lin, Bram Naus, Perry Tan, and Pepijn Verburg. 2018. Attending to slowness and temporality with Olly and Slow Game: A design inquiry into supporting longer-term relations with everyday computational objects. In Proceedings of the Conference on Human Factors in Computing Systems (CHI '18). ACM, New York, NY, USA, Paper 77, 13 pages. DOI: https://doi.org/10.1145/3173574.3173651

Read the paper, discuss it with your peers during class activities, and then write a report of maximum 1600 words that reflects on and analyzes the paper. You should format your paper according to the publication format usually used in the journals or conferences that you and your supervisor would typically submit to. You must also use the citation/reference style used as a part of these submissions. For example, I usually submit to the ACM CHI Conference, so I would the ACM CHI Proceedings format and citation style found here.

Your report should answer all of these questions:

  1. What was the worldview and strategy of inquiry (quant/qual; explore/confirm)?
  2. What was the methodology?
  3. What were the methods used to collect data?
  4. How (inductive/deductive) and in when position (in/out/both/none) was theory used/created? Explain.
  5. How was validity assessed (by authors)?
  6. What, if any, ethical concerns existed with the work?  (Note: you aren't necessarily looking for ethical problems with the published work. You are looking for ethical concerns that the researchers would have needed to address.)
  7. Why do you think the authors chose their approaches, including worldviews, methodology, methods? *
  8. Do the types of the knowledge claims fit with the approach? Explain why you believe so. *
  9. Can you think of other approaches to conduct the research? Why might they be valid? *

Justify your choices! We will grade you on each of the above plus one's ability to write clearly and effectively.

 * You should spend your time equally on all of the above questions except for these questions (7-9), which you should spend roughly twice as much time answering.

Submission:

Bring a paper copy to class of your report.

Assignments over the word/page limit will receive a penalty of 10% or only be graded up to the word/page limit, at the discretion of the instructor.

Assignment 3: Methodological Traditions (15%)

You will work in groups. You will complete a series of three assignments on methodological traditions where you are given the background of a research project, including the context, research problems or questions, method, and data analysis procedures. Your goal will be to perform some data collection and/or analysis and then draw conclusions and interpretations from this process. You will write-up your outcomes as part of a short paper.

The three methodological traditions you will explore are:

  1. Effective: experimental approaches
  2. Experience: observational approaches
  3. Creation: creation approaches

Details on each assignment are found in the links above. All have separate due dates listed in the course calendar.

Grading

We will take your best 2 out of 3 grades from Assignments 3a, b, c.

Assignment 4: Workshop Critique / In-Class Presentation (15%)

The goal of this assignment is to practice analyzing a research problem/topic and creating a research design from different methodological traditions including a compare/contrast reflection. You will generate and present a research study at a high level for one of three assigned research topics, problems, or phenomena by looking at and presenting a worldview, research design, methodology, ethical concerns, validity, and use of theory.

Research Topics:

1. Interaction in 3D Spaces: Large interactive wall displays are creating new paradigms for individuals and groups to interact with large amounts of data. However, the challenge is that traditional methods for interacting with large wall displays such as a mouse and keyboard no longer match the design context. One interaction approach is the use of touch: single or multi-finger interaction on a touch-sensitive display. Explore how you can study fatigue effects from using touch interaction on large wall displays.

2. Technology for Preserving Intangible Cultural Heritage: Cultural groups typically want to preserve their intangible cultural heritage, which is non-object-based cultural knowledge passed on between generations. Museums have historically performed this role and now we are seeing an increasing role for technology to play in such practices. Explore the topic of how Indigenous peoples think about and have historically kept records of their intangible cultural heritage, along with how they feel about present day technology that might help preserve their intangible cultural heritage.

3. Design for People Suffering from Chronic Pain: A growing number of people suffer from chronic pain, pain that persists over months or years. People with chronic pain may have a difficult time completing their normal day-to-day activities and they can also become socially isolated as a result. Explore how researchers can create technology interventions for people suffering from chronic pain.

Tasks:

1. Select one of the above research topics. You should choose the topic that you feel you would learn the most from studying and doing the assignment on.

2. Individually, create a presentation with slides of less than 10 minutes (max 20 slides) that presents a research study methodology to study the selected research topic. Your presentation should show the selected worldview, research questions, methodology, methods, ethical concerns, threats to validity/reliability and how you will address them (3 main points), and use of theory (you don't need to find examples of theory, just describe how you would use theory in a general sense). Provide brief rationale for why you have selected each of these.

You can and should include images or video within your presentation. Your slides should also include speaker notes of what you will say for each slide.

3.In class, we will split into groups (3-4 students per group) where you will present to your group members and critique the presentations. You will hand in your critiques of your peers work. You will then work together to produce the best presentation for your group.

4. In class the following week, you will present your final group presentation of less than 10 minutes. Presentations will be done as a group. You will also hand in your individual critiques from prior week.

 

Submission:

Each student should email their slide deck files to akitson@sfu.ca, aantle@sfu.ca and TA prior to our presentation workshop class.

The one page critique should be brought to class, on paper, the following week in class.

As a group, you will submit the final presentation as a file over email to akitson@sfu.ca, aantle@sfu.ca and TA before your in-class final presentation.

Your group presentation happens in class. Assignments over the time length limit will receive a penalty of 10% or only be graded up to the length limit, at the discretion of the instructor.

Grading:

Individual Slides (5%)

  • depth of content – did you show effort in trying to answer all points in your slides? I'm not necessarily looking for all right answers. The “right" answers will come from working together as a group to come up with your final presentation. Right now, I'm looking for effort.
  • critique – did you provide relevant and detailed feedback for all of your team members' presentation slides?

We will grade you mostly on the depth of content of your individual slides. The critique is more a check if you did this or not and with some effort.

Team Slides and Presentation (10%)

  • worldview
  • methodology
  • research questions
  • methods
  • ethical concerns
  • validity
  • use of theory
  • teamwork – did you work to ensure everyone played a part in the presentation?
  • slide quality – are the slides clean, simple, meaningful and visually compelling? Did you include visuals to demonstrate your points?
  • speaking quality – is the speaking lively and clear? (this will be individually based)

We will grade you equally on each of the above criteria. Make sure to justify all of your choices.

Resources for critiquing presentations
SFU Tips/Resources: https://www.lib.sfu.ca/about/branches-depts/slc/learning/presentations
 

Assignment 5: Research Proposal (50%)

You will write a paper that describes plans to conduct a research project. This will be a project proposal and similar to a Chapter 1 in many Master’s and PhD Theses. It is also similar to the first half of many conference papers (e.g., Introduction, Related Work, Study Method sections). To do so, read at least five papers in your research area. Frame a single new research question with sub-questions in the context of existing work. Create a plan for a research design in a single methodological tradition that is appropriate to answer your research questions.

Write a research proposal of a maximum 3200 words (this should be about 10 pages, but worry about word count) in SFU’s thesis format. You can find the thesis format here. When you submit your assignment, please remove the front matter from pages ii to xii since it is not needed.

Include the following sections:

1. Introduction (~1 page)

  • Describe your research topic (PACT) and research problem. Motivate your work (why does solving problem matter and to whom?) and provide a summary of the background literature that suggests the need and research directions for your work.
  • Describe an overview of the basic research design. What worldview will you take? What research approach (quantitative, qualitative, mixed, creation)? Will the research be exploratory or confirmatory? Deductive or inductive?

2. Related Work (~2-3 pages)

  • Provide background details on your research topic by describing the closest known research projects and their outcomes; as you do this use the deficit model to explain how your work addresses a research gap . If applicable, outline any theory needed for your work and explain how you will use it.
  • You should cite at least five papers on your topic area as part of the related work section.

3. Research Questions (~1-2 pages)

  • List out your research questions and explain why each of them is important, given the prior work.
  • Explain how answering the research questions push the state of knowledge in your field forward.

4. Research Design and Methodology (~2 pages) 

  • Explain the details of your research design and specifics of your (single) methodological approach. Why did you choose this approach?
  • Provide methodological details, including but not limited to …
  • Where will your study take place? Why? 
  • If there are participants, who will participate in your study? How will  you recruit? What is your selection criteria? Number of participants?
  • If you have tasks or activities or events, what will participants do? With what technology?
  • What is the duration of study and/or of each session if relevant? How many sessions?
  • If you have a prototype or system — what are the key features?

5. Methods: Data Collection, Analysis and Interpretation  (~1-2 pages) 

  • Describe and justify out all the variables, constructs or factors of importance in your study as applicable. Try to limit yourself to a few variables/constructs.  If you manipulate or control variables, describe and justify this. If you use theory as a lens for data collection, describe ad justify this.
  • Describe the data collection methods you will use as a part of your research and if applicable provide an operational definition for each variable. Justify each data collection method and provide details about data types.
  • How will you analyze your data for each method? Describe in detail the specifics of analysis and procedure.
  • How will you interpret your data to address your research questions? Discuss the subjective or objective nature of your methods. 

6. Limitations, Critical Reflection and Ethical Issues  (~1-2 page) 

  • Provide an analysis of the strengths and weaknesses of your approach including discussion of validity and reliability.
  • Describe other approaches that you considered and their strengths and weaknesses.
  • Describe the ethical concerns that you will need to address as a part of your research.

7. Conclusion (~1 page)

  • Describe the contributions of your work. Will you contribute the design of a technology? The results of a technology study? The study of people or culture in some form? Will you provide a theory from your work?
  • Describe the types and strength of knowledge claims that your work will be able to create based on the methodology chosen.

Note that the structure above assumes that the research is oriented towards a ‘research question/problem’. If your research is not oriented in this way, you will need to modify these sections to match the worldview and paradigm that you have selected.

Examples:
See Chapter 1s of these theses:

Note that many of the above examples are from Carman’s students, so they fit a particular style. You can adjust the style of your own writing to fit what is typical for your supervisor or discipline.

 

Peer Review and Revision 8%
You will write a draft of your paper and it will be distributed to the class for peer review. Each student must review one paper and provide written critique of less than one page (single spaced). Each student must reflect on the peer review suggestions and then refine the paper.

There are some example paper reviews on this page here as well as CHI’s guideline for reviewing. They are focused on reviewing conference papers, but I think the general principles still apply. When you do your peer review of a classmate’s proposal / paper, you’ll want to assess each section and provide a critique including what could be improved and what is already well done. Look for issues of clarity, proper articulation of research questions, appropriate methods and methodology, etc.

To submit your paper for review:

We will be using a conference management system called EasyChair. You will submit your paper to EasyChair and perform your review in it. Reviews will not be blind so the authors will not know who the review is written by.

  1. Go here and create an account: https://easychair.org/conferences/?conf=siatgrm2021
  2. Submit your paper through the system. It needs to be a PDF.
  3. When you get a review request in email, please accept it.
  4. Submit your review through EasyChair. Be respectful when you write your review. You need to be critical and helpful, but word your review in such a way that you would be comfortable saying the same thing to the person face-to-face.
To submit revision report in response to the review:
Attached a one page document along with your final paper that describes how you addressed the reviewer’s comments. For some examples, see here.
 

Final Submission
The final revised paper is worth 42% for a total of 50% for this activity.

Your final submission should contain:

  1. 1-pg revision report (PDF file)
  2. your revised version of the paper (source file, e.g., docx, latex)
  3. the final version of your paper (PDF file)

Submit the paper over email to aantle@sfu.ca and akitson@sfu.ca. The submission should contain the above three files only.

 

Grading

Peer Review Process (8%)

  • One page peer review – is it detailed? did the reviewer critique based on the concepts in class?
  • Changes based on peer review – did the changes address the concerns of the reviewer in detail?

We will grade you equally for the review and the revision report.

Final Version of Paper (42%) – for each of the following sections, you will be graded based on how well you addressed the above criteria

  • Introduction
  • Related Work *
  • Research Questions *
  • Methodological Approach **
  • Methods  **
  • Limitations, Critical Reflection, Ethics
  • Conclusion *

Other marks: writing quality, format, proper use of citations. 

* You should spend more time on these sections and

** even more time on these sections

Assignments over the word/page limit will receive a penalty of 10% or only be graded up to the word/page limit, at the discretion of the instructor.

RESOURCE ALERT! Drop into the Open Writing Lab (sorry, Burnaby and Vancouver campus only right now) for help with your writing. You can also schedule a writing consultation (available at all three campuses!) for a 30-60 min discussion around time management and goal setting for writing, stages of the writing process, and planning for different writing assignments.

Contact

  • Email: aantle@sfu.ca
  • Phone: 778.782.7438
  • Fax: 778.782.9422
  • Office Location: Podium 2-2832
  • Lab Location: Podium 3-3930

Address