EDUC 8804: Using Artificial Intelligence in Academic Research

Fall 2026 · Tuesdays 8:15–10:45 · MBE 170

Instructor

Derek Briggs

Office: MBE 400D | Tel: (303) 709-5381 | E-mail: derek.briggs@colorado.edu

Office Hours: Mondays 12:30–1:30 (book a time)

Zoom: https://cuboulder.zoom.us/my/derekbriggs

Course Overview

Purpose

The purpose of this course is to explore and critically evaluate the affordances and limitations of using generative Artificial Intelligence (AI) as a collaborative partner in the conduct, communication and dissemination of academic research.

Perspective on AI

Whether we like it or not, the fact is that we now live in a world with generative AI. It will soon become (if it isn’t already) a ubiquitous tool for academic research. The ethos of this course is motivated by the four principles for working with AI introduced by Ethan Mollick in his 2024 book Co-Intelligence.

Principle 1: Always invite AI to the table.

The only way to become knowledgeable about the nuances, limitations and abilities of AI tools is to try them out and learn from experience. This is especially important because what you think you know about the limitations of AI at one point in time might not apply a few months later (see principle 4).

Principle 2: Be the human in the loop.

AI has the potential to extend your capabilities as a researcher, but it can’t replace you. In particular, AI tends to not be very good at being meta-cognitive. Without the right oversight AI chatbots can hallucinate and lie. Being the human in the loop means taking responsibility and being on the alert for the ways that errors can be induced by the use of AI.

Principle 3: Treat AI like a person (give it context and tell it what kind of person it is).

This does not mean to anthropomorphize AI, but to appreciate that if it was a person with whom you were interacting, you’d always need to make sure that the person had sufficient context to be able to help you. Some of this falls under the label of what is known as “prompt engineering” or “context engineering.” But it also depends on what the AI knows about you and what it knows about how you want it to respond.

Principle 4: Assume this is the worst AI you will ever use.

When ChatGPT first came out in 2022, I would try it out with test questions from courses I taught about statistics and psychometrics. I was amazed that it could generally produce B- answers. Two years later those answers are as good or better than what I had written in my solutions. AI systems are no longer just LLMs—they can break down requests into chunks and use tools (Python code, the internet) to augment pre-trained knowledge. What AI can do is constantly evolving, so as researchers we need to stay on our toes.

Mollick conceives of AI as offering a form of co-intelligence. In other words, as not replacing human intelligence, but extending it such that we will all want to learn how to function as “cyborgs.” The hope is that at worst, AI will help us to become more efficient. At best, it could lead to innovations that would not have otherwise been possible. A core question I wish to have students explore empirically in this course is how and to what extent we should expect the process of research to change when we invite an AI to the table as a collaborator.

Locating Junctures for AI in Academic Research

Most academic research has some common structural features such as

  1. Identifying a problem that needs to be solved, or a debate that needs to be entered. Making a case for why research on the topic is necessary, and generating research questions.

  2. Providing relevant summary and synthesis of prior research related to the topic. (This can take the form of either a review of the literature, a “conceptual framework” or both.)

  3. Methods for collecting data relevant to the research questions.

    1. Direct observation
    2. Interviews
    3. Surveys
    4. Tests/Assessments

  4. Methods for analyzing data relevant to the research questions.

    1. Thematic coding
    2. Measurement
    3. Statistical Modeling and Inference

  5. Interpreting results

    1. Connecting results back to the literature
    2. Addressing threats to validity
    3. Recognizing limitations
    4. Suggesting next steps

  6. Disseminating Results for a Broader Audience

Of course, these features are seldom purely linear, and there are many exceptions to the rule. Of interest in this course is to explore not only how AI can be used in support of these different steps, but also how the results compare in a context where we already have a solid basis for ground truth in the form of previously published studies.

Course Goals

The goals of the course are to

  • equip students with the conceptual understanding and practical skills to use generative AI as a collaborative tool across the entire research lifecycle, from idea generation to dissemination;

  • foster a critical and empirical mindset for evaluating AI-generated content, recognizing its strengths, limitations, and ethical implications;

  • explore how AI may change the nature of academic research, for better or worse.

Learning Objectives

Upon successful completion of this course, students will be able to

LO1 (Conceptual): Explain the fundamental concepts, terminology, and capabilities of modern AI systems at a conceptual level (e.g., LLMs, transformers, training, fine-tuning, context windows).

LO2 (Application): Apply AI tools knowledgably to assist in discrete research tasks, including developing research questions, conducting literature reviews, operationalizing data collection methods, and analyzing both qualitative and quantitative data.

LO3 (Evaluation): Design and execute experiments to critically evaluate the performance of different AI models and prompting strategies against a known “ground truth” in their own domain of expertise.

LO4 (Synthesis): Integrate AI tools into a coherent research workflow, documenting the process and reflecting on how AI’s involvement shaped the research findings.

LO5 (Ethics): Identify, discuss, and critique the major ethical and philosophical challenges posed by using AI in research, including algorithmic bias, the alignment problem, environmental impact, and disclosure of use.

Weekly Structure

Part 1: Foundations & Core Concepts (Weeks 1–5)

Week 1: Introductions and Overview

Week 2: What Can AI Do?

Week 3: Inside the Black Box (Part 1)

Week 4: Inside the Black Box (Part 2)

Week 5: Big Picture Issues & Introducing the Class Project

Part 2: AI in the Research Lifecycle (Weeks 6–12)

Week 6: Using AI to Develop Research Questions

Week 7: Using AI for Literature Review

Week 8: Using AI to Design Survey Instruments, Assessments, & Protocols

  • Topic: AI-assisted design of surveys and assessments, interview protocols, and observation guides.

  • Class Project: Results from using AI to create survey questions, interview protocols

  • Required Reading

  • Optional Resources

Week 9: Using AI to Collect and Clean Data

Week 10: Using AI for Qualitative Data Analysis

  • Topic: AI-assisted thematic coding and qualitative analysis.

  • Class Project: Results from using AI for thematic coding in qualitative ground truth study

  • Required Reading

    • Yang, Y., Ma, L. Artificial intelligence in qualitative analysis: a practical guide and reflections based on results from using GPT to analyze interview data in a substance use program. Qual Quant 59, 2511–2534 (2025). https://doi.org/10.1007/s11135-025-02066-1
    • AlGhamdi, R. (2026). From Code Variability to Theme Convergence: AI–Human Alignment in Thematic Analysis With Claude Code. International Journal of Qualitative Methods, 25.

Week 11: Using AI for Quantitative Data Analysis

  • Topic: AI as a co-pilot for statistical modeling, measurement, and psychometrics. (Example: Using AI to write, debug, and interpret R or Python code).

  • Class Project: Results from using AI to replicate the analysis from the quantitative ground-truth studies.

  • Required Reading and Resources

Week 12: Communicating Results and Acknowledging AI Use

Part 3: Synthesis & Future (Weeks 13 & 15)

Week 13: Philosophical and Ethical Issues in the Use of AI

Week 14: Fall Break (No Class)

Week 15: Lessons Learned

  • Topic: Final “Lessons Learned” discussion

  • Student Project presentations

Class Activities and Student Assessment

Collaborative Class Project

I plan to come into the class with a small number of published studies that would serve as ground truth for exploring the use of AI in the research process. A constraint would be access to raw source data. For qualitative studies, this probably means de-identified observations or interview transcripts. For quantitative studies, this could mean de-identified data sets in row by column format.

As a class we could experiment with use of AI to retroactively replicate each stage of the research process with the common studies that have been selected. As an example, I am just about to publish a study that required a lot of understanding about educational curricula, standards and assessment in K-8 mathematics in the United States (https://osf.io/gcpt3). It also involved an empirical analysis of publicly available NAEP test results from about 2000 to 2019. My co-authors and I did not use AI at all to do this research and write up the results. Could we use AI to replicate the quantitative results? That seems easy to answer. If we had used AI for each stage of the research process, would we have come to similar findings? That’s harder to ascertain.

I will also work with the class to see if we can generate other ideas for candidate ground truth studies that would appeal to the interests of students enrolled in the course.

Small-Scale Experiments

At the outset of the course, I will have each student identify one or more domains in which they consider themselves “above average” in their expertise based on their prior knowledge and experience. A way to think about this: If someone asked you a question in this domain and you had access to the internet, would you be able to respond accurately 90% of the time?

Throughout the course students will be required to produce memos and 10-minute presentations on some form of “A/B” experiment they conducted using AI to answer a question (or series of questions) within an area of their own expertise. Examples:

  • Compare ChatGPT, Gemini, Claude
  • Compare different prompting strategies
  • Compare sub-models within a major model (e.g., “instant” vs “thinking”)

Requirement: Before asking the question, the person prompting must attempt to provide what they would regard as the “ground truth” answer.

Independent Project

There are at least the following options (I am open to working with you on others).

Option 1: Pick your own pre-AI or purely human-led study as ground truth, evaluate to what extent you would (or would not) replicate core aspects of the study in a collaboration with AI. For this option to be most worthwhile, you would really want to have access to whatever data was used as a basis for conclusions and claims made in the study.

Option 2: Replicate the approach that I took (described here) in which I started with the core idea of a research paper in place, but the execution of the idea was still in pieces. From this starting point, collaborate with an AI agent to write a full draft, submit it to an adversarial peer review, and then revise at least once.

Option 3: Collaborate with AI to produce a 10–15-page research grant proposal that would follow the requirements for a funding agency that regularly releases RFPs. Under this option you would need to have a real RFP issued in the past, and the criteria for evaluating grant proposals as key context.

Irrespective of the option, although I am interested in (and expect you to submit) what you actually produce in your AI collaboration, I am more interested in your reflection and evaluation of the way your collaboration proceeded. I’ll ask you to evaluate your AI research collaboration with respect to Anthropic’s “4D” framework of delegation, description, discernment and diligence as described in the AI Fluency: Framework & Foundations short course found at academy.claude.com.

Grading

  1. 2 Small-scale Experiment Reports and Short Oral Presentations (30%)
  2. Quiz on “Inside the Black Box of LLMs” Week 3 (10%)
  3. Active Participation in Class and Short Homework Assignments (20%)
  4. Independent Project (40%)

Course Website: Canvas

I will make all presentations, handouts, data sets, and other materials available to you on Canvas. This is where updated due dates, activities, announcements will be posted. Please make sure to turn on notifications for Canvas so that you receive course-related announcements during the semester. All assignments should be submitted via Canvas.

Use of AI Models in this Course

We will be primarily using ChatGPT (OpenAI) and Claude (Anthropic) in this course with some instances of Gemini (Google) as well. Anyone with a CU email account should have access to the “Plus” version of ChatGPT and the “Pro” version of Gemini (these otherwise cost $20 a month). I would personally recommend paying for the “Pro” version of Claude ($20 a month) as well, but this is not required. The only requirement is that you have access to either the Pro version of Claude or the Plus version of ChatGPT. Note that the CU enterprise versions of ChatGPT and Gemini have as a default setting that no information shared with the model is publicly disclosed or used to further train the model. If you use a privately purchased version of Claude, you can also disable the feature that allows Anthropic to train or improve their models using your prompts and data.

Missing Classes

Sometimes life and/or ill health intrudes, and you will be forced to miss a class. When this happens, please let me know as far in advance as possible. If you miss class, it is your responsibility to take the initiative to get back up to speed. Please don’t assume that I will seek you out electronically or in person to provide you with materials or announcements you may have missed. If you miss two or more classes for unexcused reasons this will lower the “active participation” component contributing to your final grade.

University Policies

Classroom Behavior

Students and faculty are responsible for maintaining an appropriate learning environment in all instructional settings, whether in person, remote, or online. Failure to adhere to such behavioral standards may be subject to discipline. Professional courtesy and sensitivity are especially important with respect to individuals and topics dealing with race, color, national origin, sex, pregnancy, age, disability, creed, religion, sexual orientation, gender identity, gender expression, veteran status, political affiliation, or political philosophy.

Additional classroom behavior information:

Accommodation for Disabilities, Temporary Medical Conditions, and Medical Isolation

If you qualify for accommodations because of a disability, please submit your accommodation letter from Disability Services to your faculty member in a timely manner so that your needs can be addressed. Disability Services determines accommodations based on documented disabilities in the academic environment. Information on requesting accommodations is located on the Disability Services website. Contact Disability Services at 303-492-8671 or DSinfo@colorado.edu for further assistance. If you have a temporary medical condition, see Temporary Medical Conditions on the Disability Services website.

If you have a temporary illness, injury or required medical isolation for which you require adjustment please alert me by email at derek.briggs@colorado.edu.

Student Names and Pronouns

CU Boulder recognizes that students’ legal information does not always align with how they identify. If you wish to have a name other than your legal name appear on your instructors’ class rosters and in Canvas, or if you wish to choose pronouns to appear on your instructors’ class rosters and in Canvas, visit the Registrar’s website for instructions on how to change your personal information in university systems.

Honor Code

All students enrolled in a University of Colorado Boulder course are responsible for knowing and adhering to the Honor Code. Violations of the Honor Code may include but are not limited to: plagiarism (including use of paper writing services or technology [such as essay bots]), cheating, fabrication, lying, bribery, threat, unauthorized access to academic materials, clicker fraud, submitting the same or similar work in more than one course without permission from all course instructors involved, and aiding academic dishonesty. Understanding the course’s syllabus is a vital part of adhering to the Honor Code.

All incidents of academic misconduct will be reported to Student Conduct & Conflict Resolution: StudentConduct@colorado.edu. Students found responsible for violating the Honor Code will be assigned resolution outcomes from Student Conduct & Conflict Resolution and will be subject to academic sanctions from the faculty member. Visit Honor Code for more information on the academic integrity policy.

Accommodation for Religious Obligations

Instructional faculty members must make every reasonable effort to accommodate all students who have conflicts with scheduled exams, assignment deadlines or required attendance due to a religious observance. Whenever possible, students must notify the instructional faculty member at least two weeks in advance of the expected exam, assignment deadline, or attendance conflict to request an accommodation for religious observance. If the start date of the course is less than two weeks before the date of requested accommodation, students must notify the instructional faculty member on the start date of the course. See the Student Academic Accommodations for Religious Observances Policy for more information.

Mental Health and Wellness

The University of Colorado Boulder is committed to supporting students’ mental health and overall wellbeing. If personal, academic, or emotional challenges are affecting your wellbeing or success, Counseling and Psychiatric Services (CAPS) is here to help. CAPS offers counseling, referrals, psychiatric care, crisis support, and much more. Visit CAPS in the C4C, or call (303) 492-2277, 24/7.

Footnotes

  1. From Claude (Opus 5): Two caveats worth knowing before you rely on it for lit reviews: research sessions burn through usage limits considerably faster than normal chats, and citation quality is uneven—it retrieves well from the open web but has no privileged access to paywalled journal content, so for anything requiring JSTOR/EBSCO/Web of Science coverage it will systematically under-retrieve and may substitute secondary sources.↩︎