AI Tools in Education: Adoption and Implications of ChatGPT and GPT-4 Among German Students

By Dr. Sampath LonkaReviewed by Susha Cheriyedath, M.Sc.Nov 12 2023

In a recent publication in the journal Humanities and Social Sciences Communications, researchers explored the implications of the usage of artificial intelligence (AI) tools such as Chat Generative Pretrained Transformers (ChatGPT) and GPT-4 in teaching and assessments.

Background

AI emerges as a transformative influence, reshaping markets, industries, and business models. The spotlight intensified in November 2022 with the release of OpenAI's ChatGPT, a language-processing AI model. Rapidly gaining a global user base, it progressed to ChatGPT-4, exhibiting enhanced capabilities to process both text and image inputs, achieving human-level performance in professional and academic benchmarks.

These AI tools find multifaceted applications in business interactions, internal service organizations, and employee recruitment. Scientifically, they facilitate tasks like text analysis, translation, and co-authorship in research papers. In education, they support scientific reflection, text optimization, proofreading, and acting as tutors for exam preparation. However, these opportunities coexist with risks, from security concerns to misinformation. OpenAI acknowledges challenges, including ChatGPT generating plausible-sounding but incorrect answers.

In response, some entities impose temporary bans or restrictions on AI tool usage. Despite existing empirical studies, a comprehensive Germany-wide study on student use in studying and teaching is lacking. The current study aims to fill this gap by analyzing AI-based system utilization in studies, encompassing an introductory overview, theoretical and empirical state-of-the-art, methodological approach, study results, and a reflective conclusion.

Evolution and implications of AI

The term AI lacks a universal definition, evolving with technological progress and the intricate nature of intelligence. Its multi-disciplinary nature allows for diverse perspectives. Originating in the 1950s, recent advancements in computer systems, algorithms, and data storage have propelled AI's development, enabling machines to emulate human cognitive abilities.

AI encompasses methods enabling systems to interpret data and learn, performing tasks like visual perception, language processing, and strategic planning independently. AI-based tools prove valuable in informational work, utilizing artificial neural networks for efficient language conversion. The impact of such tools is substantial across society, business, and science, affecting a significant portion of the workforce.

Despite their potential, caution is warranted due to characteristics such as misinformation generation, acknowledged by OpenAI for ChatGPT. GPT-4 addresses some limitations but is not entirely foolproof, necessitating careful consideration in high-stakes contexts. Uniform use in teaching and learning poses risks, given the potential for false, misleading, or socially unacceptable outcomes. The provided table outlines the essential properties of AI-based tools.

Methodical approach

To achieve the study's objective, a quantitative survey was conducted via an online questionnaire, covering the general use and intensity of AI-based tools for studying. The survey incorporated a choice-based conjoint experiment (CBC) in which participants made eight fictitious purchase decisions. Followed by explicit questions assessing the most crucial characteristics in the students' evaluation, identified through a preliminary study with 36 students. A pre-test preceded the survey, distributed as a self-selection sample to students from various German universities through university contacts. The study reached out to 395 of the 423 universities in Germany, targeting 3,849 program coordinators.

Acknowledging limitations, the study employed a non-probabilistic sample for exploratory purposes, with potential biases introduced by the study's title, German-only questionnaire, and focus on AI tools, specifically ChatGPT. Potential biases also exist in self-assigned fields of study and the influence of social desirability on responses. The study recognizes that the university itself may induce AI tool use as a methodological-didactic instrument in teaching and learning.

Results and analysis

The surveyed population includes all individuals enrolled in German higher education institutions during the winter semester of 2022-2023. The distribution reveals varying percentages across states and fields of study. The total number of responses is 8,802. The gender distribution deviates, with 60.3 percent females.

Analysis shows that 63.4 percent of students have used AI-based tools, with different usage intensity across fields and academic levels. Bavaria, North Rhine-Westphalia, and Hesse host the most respondents. Notably, 49 percent use ChatGPT or GPT-4, and the main applications include clarifying understanding, research, and translations. The study's central focus is on AI tool usage among students, revealing nuanced patterns across disciplines and academic levels.

Despite acknowledging certain limitations, the authors demonstrate objectivity, reliability, and validity. Objectivity is maintained through standardized, independent online surveys. The study's reliability is supported by its ability to reproduce results with the same instrument and unchanged object, even though the sample distribution deviates from the population. Content validity is ensured by operationalizing language-based AI tools, yet construct validity remains inconclusive due to a lack of national-level studies on AI tool usage. The explorative nature of the study aligns with the nascent state of AI tool research.

Conclusion

In summary, researchers highlight the widespread adoption of AI-based tools among German students across all disciplines, with nearly two-thirds utilizing these tools. Engineering and natural sciences exhibit the highest usage. The prevalence could stem from program requirements, technological affinity, or, as gender-specific differences suggest, a higher proportion of male students in these fields. ChatGPT, or GPT-4, is prominently used by almost half of the surveyed students. Further investigation is warranted to explore varied usage patterns across disciplines and differing perceptions of AI system characteristics. A future focus on inferential statistical analyses will provide deeper insights into AI tool utilization in education.