A Quick Guide to Building Case Study Exam Questions Using ChatGPT

May 28

10.25.2024 | Written by Jessi Watson

The Role of Case Studies in Clinical and Process-Based Training

Case studies are a powerful tool in training and education, particularly for clinical and process-based fields. Scenario-based questions allow learners to apply theoretical knowledge to real-world situations, fostering engagement and the critical thinking and problem-solving skills necessary for professional success. Traditionally, case studies have been created manually by educators and trainers, requiring a significant investment of time. While effective, this method can limit the variety of scenarios available and may not be easily adaptable to different learning needs. With the right human influence, AI tools like ChatGPT can streamline this process, making case study development an efficient alternative.

Why Use Case Studies?

For Certification and Competency Assessments

Many professional certification and competency-based assessments incorporate case study scenarios. These exams are designed to test not only theoretical knowledge but also practical decision-making skills. Whether for technical training, compliance testing, or process-based learning, case study questions challenge learners to analyze situations and determine the most effective course of action.

For Engagement

Case studies create an interactive learning experience that is more engaging than traditional lecture content. They invite learners to immerse themselves in dynamic, real-world scenarios that help them connect concepts to practice. This active learning approach promotes deeper understanding and retention.

To Develop Critical Thinking and Problem-Solving Skills

Critical thinking is essential in any field that requires analysis, decision-making, and troubleshooting. Case-based learning encourages learners to evaluate complex situations, consider multiple perspectives, and make informed decisions. Research shows that this method leads to improved problem-solving skills and a deeper understanding of subject matter.

To Assess Preparedness

Case studies can be used to evaluate whether learners are ready for real-world responsibilities. They help identify gaps in knowledge, reinforce key concepts, and ensure that individuals can apply what they've learned in practical scenarios. By simulating workplace challenges, case studies prepare learners to handle real-life situations with confidence.

Using ChatGPT to Enhance Case Study Development

AI programs like ChatGPT use machine learning algorithms to generate responses using natural language processing. When paired with a subject-matter expert, AI can significantly enhance case study development by increasing efficiency, variety, and adaptability. Creating case studies manually can be time-consuming, especially for organizations that require frequent updates or large volumes of training materials. AI can generate multiple scenarios quickly, allowing for a broader range of cases and reducing the burden on trainers.

AI Bias is Human Bias: AI Use as a Partnership

AI should be seen as a partner in the case study creation process rather than a replacement for human expertise. While AI can generate content quickly, it needs a moral chaperone. It is well-known that AI can amplify gender and racial sterotypes based on the bias that exists in the content it is trained on (Nicoletti & Bass, 2023). Made by humans, biased like humans.

Without intentional use (i.e. planning, collection of feedback, reflection) AI will not create acceptable, unbiased material. Educators should consider incorporating bias checks along the way and carefully reviewing and adjusting the content produced. Like much content-building, this is front-loaded work that will pay off with frequent use.

Step-by-Step Guide to Using ChatGPT for Case Study Development

There are several published prompt-writing resources that are available to users. A quick search online will populate videos on YouTube that can guide you through the process. Kiyak (2023) provides a useful template for generating case-based questions in medical education.

The following guide considers customization of prompts for alignment of the output with instructional objectives and for different learning levels, feedback and revision strategies, and methods for addressing ethical concerns.

1. Define Learning Objectives

Before you start, define learning objectives, consult accrediting body resources, and consider cognitive level and integrated processes. Determine the key skills, knowledge, and concepts you want your learners to gain from the case study. This is useful information for both you and the AI, and will help ensure that the case study questions will be useful as a direct assessment of learning outcomes.

2. Craft a Specific Prompt

Providing ChatGPT with a well-structured prompt improves the quality of its output. Be specific. Include your learning objectives and course outcomes, and items like:

Number of questions you want to follow the scenario
Question format (multiple choice, select all that apply, short answer, etc.)
The intended audience (nursing student, construction safety training program students, etc.)
Learning level/difficulty
Ethical guidance.

Example 1:

Write a case study exam question in a single narrative paragraph for a [surgical technology student] about [a malignant hyperthermia (MH) crisis in the operating room].

Learning objective: [Assess whether the student understands that the patient is experiencing a malignant hyperthermia crisis and the surgical technologist’s role during the crisis].

Course outcome: [Provide a safe, efficient environment for the patient and workplace staff].

Related questions: [Six multiple choice questions with a range of difficulty levels].

Include: Patient details like [signs of disease, relevant clinical history].

Ethics: [Objectively describe individuals. Avoid terms that may perpetuate stereotypes. Avoid using pronouns and instead use a title or description for the operating room team and patient].

Example 2:

Write a case study exam question for a [process-based training program] about [a workplace safety incident where an employee fails to follow proper lockout/tagout procedures, resulting in a near-miss].

Learning objective: [The questions should assess knowledge of safety protocols, error identification, and corrective actions]

Course outcome: [Provide a safe, efficient workplace environment].

Related questions: [Six multiple choice questions with a mixture of difficulty levels].

Include: Details like [what led to the employee’s break in procedure].

Ethics: [Objectively describe individuals. Avoid terms that may perpetuate stereotypes. Avoid using pronouns and instead use a title or description.]

3. Generate Initial Case Study for Review and Editing

Use ChatGPT to generate the first draft of the case study. Carefully review the generated case and questions for accuracy, clinical relevance, and alignment with learning objectives. Consider making another request or editing your prompt.

For example:

Add a social issue/disagreement to the scenario and assess if the learner understands the appropriate response.
Make two of the questions related to the Dantrolene reconstitution process.
Make the scenario longer and include more specific descriptions of the surroundings of a work zone.

4. Edit Your Prompt and Repeat

The more you use ChatGPT, the more data it accumulates to use to anticipate your desired outcomes. What took five or six iterations/prompt revisions at first, now takes only one or two after about six months of consistent use. The AI will come to “understand” the context of your input based on previously posed prompts. You will need to be especially careful at first in editing and correcting any factual inaccuracies.

Refine the AI-generated case study to correct any errors and ensure it aligns with learning/training objectives. Consider adding questions that require critical thinking, decision-making, and have real-world implications.

5. Check for Bias and Legal and Ethical Concerns and Revise

AI models like ChatGPT can inadvertently reflect biases present in the data they were trained on. This might lead to stereotypical or biased representations of patient demographics, which can perpetuate harmful assumptions. There are various scorecards and evaluation tools available to educators for evaluating bias in school curriculums (Comprehensive Center Network, 2023).

For ease of use with generative AI, the author has created a simplified amalgamation of these tools to input into ChatGPT as an additional step following prompt revision. Educators are encouraged to review the content for bias using their own processes as well.

Input for Bias Check

Evaluate this output against these criteria to check for bias:

Factors: age, appearance, family structure, marital status, language, disability, race, ethnicity, nationality, sexual orientation, and gender identity.

Are any individuals in the generated scenario portrayed as less capable than others based on the listed factors?
Are individuals objectively described?
Is there mention of any of the listed factors that is not necessary in order to achieve the desired learning outcomes?

6. Pilot the Case Study and Interpret the Results

Before implementing AI-generated case studies in training programs or classroom exams, test them with a small group of learners or colleagues. Gather feedback and make necessary adjustments to improve clarity and effectiveness. Consider piloting them as discussion questions with a small group.

Assessing question difficulty, clarity, content validity, and reliability before using the questions in a scored exam is a critical step. Pilot the case study questions in a formal setting with a group of your learners. Inform the group that there will be pilot questions on a given exam and that they will not count toward their grade. Collect and analyze data using your exam administration platform or in your preferred way. Review exam results to identify patterns in student answers, use item analysis to evaluate how well the question differentiates between high and low-performing students, and analyze the distribution of scores to assess the overall difficulty level of the case study questions. Do the questions have validity and reliability? Can they be consistently trusted to represent what each student knows?

Finally, engage in a n open discussion with the learners who took the pilot questions in order to uncover their perception of the question’s meaning. Do the students recognize and understand the value of the questions? Was there anything confusing in the formatting of the question?

7. Finalize and Implement

Once refined, integrate the case study into your training materials. Provide clear instructions for learners and trainers on how to use the scenario effectively.

8. Evaluate Effectiveness

After using the case study in formal assessment, collect feedback and analyze performance data. Use this information to improve future case studies and optimize the learning experience.

Conclusion

By incorporating AI tools like ChatGPT into the case study development process, trainers and educators can save time, enhance engagement, and create more effective learning experiences. With careful oversight and refinement, AI-generated case studies can be a valuable resource for improving critical thinking and real-world preparedness in clinical and process-based training programs.

Bakkum, M. J., Hartjes, M. G., Piët, J. D., Donker, E. M., Likic, R., Sanz, E., de Ponti, F., Verdonk, P., Richir, M. C., van Agtmael, M. A., & Tichelaar, J. (2024). Using artificial intelligence to create diverse and inclusive medical case vignettes for education. British journal of clinical pharmacology, 90(3), 640–648. https://doi.org/10.1111/bcp.15977
Bauwens, E. E., & Gerhard, G. G. (1987). The use of the watson-glaser critical thinking appraisal to predict success in a baccalaureate nursing program. Journal of Nursing Education, 26(7), 278–281. https://doi.org/10.3928/0148-4834-19870901-06
Biggs, J. B., Collis, K. F., & Edward, A. J. (1981). Evaluating the quality of learning: The solo taxonomy. Academic Press.
Comprehensive Center Network. (2023). Tools and Guidance for Evaluating Bias in Instructional Materials: A Region 8 Comprehensive Center Report. CCN. chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://www.michigan.gov/-/media/Project/Websites/mde/Academic-Standards/Tools_Guidance_Eval_Bias_Instructional_Mats.pdf?rev=1cc5205c9d8a4911b1cb5d7905882727
Hosseinzadeh H, Ratan ZA, Shnaigat M, Edwards J, Verma I, Niknami M, Dadich A. Effectiveness of case scenario-based teaching to transition international Master of Public Health students specialising in health promotion from memorization to critical thinking. Health Promot J Austr. 2022 Oct;33 Suppl 1(Suppl 1):39-49. doi: 10.1002/hpja.631. Epub 2022 Jun 28. PMID: 35714044; PMCID: PMC9796551.
Herreid, C. F. (Ed.). (2007). Start With a Story: The Case Study Method of Teaching College Science. National Science Teachers Association.
Kıyak, Y. S. (2023). A ChatGPT prompt for writing case-based multiple-choice questions. Revista Española de Educación Médica, 4(3).https://doi.org/10.6018/edumed.587451
McLean SF. Case-Based Learning and its Application in Medical and Health-Care Fields: A Review of Worldwide Literature. J Med Educ Curric Dev. 2016 Apr 27;3:JMECD.S20377. doi: 10.4137/JMECD.S20377. PMID: 29349306; PMCID: PMC5736264.
Next Generation NCLEX®. (Winter 2022). Next Generation NCLEX: Test Design. National Council of State Boards of Nursing (NCSBN). https://www.ncsbn.org/public-files/NGN_Winter22_English_Final.pdf
Sauder M, Tritsch T, Rajput V, et al. (January 09, 2024) Exploring Generative Artificial Intelligence-Assisted Medical Education: Assessing Case-Based Learning for Medical Students. Cureus 16(1): e51961. doi:10.7759/cureus.51961
Singh, J., Samborowski, L., & Mentzer, K. (2023, November 6-9). A Human Collaboration with ChatGPT: Developing Case Studies with Generative AI. [Paper presentation]. In 2023 Proceedings of ISCAP Conference (Vol. 9). Albuquerque, NM, United States; Nichols College. https://iscap.us/proceedings/2023/pdf/6039.pdf
The Certifying Board of the American Association of Medical Assistants. (2021). Content Outline For the CMA (AAMA)® Certification Exam. AAMA. https://www.aama-ntl.org/cma-aama-exam/study/content-outline
The Federation of State Boards of Physical Therapy (FSBPT). (n.d.). The Federation of State Boards of Physical Therapy General Information. fsbpt.org. https://www.fsbpt.org/Free-Resources/NPTE-Candidate-Handbook/General-Information
Thistlethwaite, J. E., Davies, D., Ekeocha, S., Kidd, J. M., MacDougall, C., Matthews, P., Purkis, J., & Clay,
D. (2012). The effectiveness of case-based learning in health professional education. A Beme systematic review: Beme Guide no. 23. Medical Teacher, 34(6). https://doi.org/10.3109/0142159x.2012.680939
Toogood, C. (2023). Supporting students to engage with case studies: a model of engagement principles. Educational Review, 1–15. https://doi.org/10.1080/00131911.2023.2281227
NBSTSA. (2024). Comprehensive Guide to Professional Certification Exam Development. National Board of Surgical Technology and Surgical Assisting

Jessi Watson