Multilingual Template-based Automatic Item Generation for Medical Education Supported by Generative Artificial Intelligence Models ChatGPT and Claude

Objective: This study has two main aims. (1) To generate multiple-choice questions (MCQs) using template-based automatic item generation (AIG) in Polish and to evaluate the appropriateness of these MCQs in terms of assessing clinical reasoning skills in medical education; (2) to present a method for using artificial intelligence (AI) to generate new item models based on existing models for template-based AIG in medical education. Methods: This was a methodological study. For the first aim, we followed Gierl’s three- -step template-based AIG method to generate MCQ items in Polish. The quality of the generated MCQs were evaluated by two experts using a structured form. For the second aim, we proposed a four-step process for using a parent template in English to transform it into new templates. We implemented this method in ChatGPT and Claude by using two medical MCQ item models. Results: Both experts found the automatically generated Polish questions clear, clinically sound, and suitable for assessing clinical reasoning. Regarding the template transformation, our findings showed that ChatGPT and Claude are able to transform item models into new models. Conclusions: We demonstrated the successful implementation of template-based AIG in Polish for generating case-based MCQs to assess clinical reasoning skills in medical education. We also presented an AI-based method to transform item models for enhancing diversity in template-based AIG. Future research should integrate AI-generated models into AIG, evaluate their exam performance, and explore their use in various fields.