Lynne N. Kennette(1) & Phoebe S. Lin(2)
1. Durham College
2. Framingham State University

Since its introduction in November 2022, ChatGPT (https://chat.openai.com/chat) has caused a lot of chatter, especially in educational circles. ChatGPT is a software application that uses artificial intelligence (AI) to simulate human speech and/or writing. Some see it as a cause to re-think assessments or a risk to academic integrity. Others welcome it as a new teaching tool. Regardless of your view, its presence is a good opportunity to re-think our assessments and to examine whether this new technology might be a threat to the skills we expect students to be able to demonstrate during their studies. As such, we provide some food for thought about how faculty might re-consider their assessments in the context of this new tool.

Pedagogy

Although an initial reaction to ChatGPT might be one of concern, it’s important to take a step back and really focus on the pedagogy. Starting with our course learning outcomes may help to re-focus and/or overcome our concerns. For example, do you have a specific learning outcome tied to writing or critical thinking that you need to ensure students are demonstrating (rather than ChatGPT)? If so, maybe there are other ways to focus on those skills in a way that departs from the standard writing assignment. Or, if the ability to detect AI-written text in students’ assignments is relevant for your pedagogical goals, there are several AI detectors which assess the extent to which text is likely to be written by a human rather than a machine. One example of these detectors is GPT-Zero (https://gptzero.me/), which provides general scores for the overall product as well as highlights which sentences are more likely to be written by AI (this sentence-by-sentence analysis is similar to what you see in plagiarism detection software like TurnItIn). So, if writing is a crucial part of your course, then perhaps detecting it will be important. If not, then perhaps writing doesn’t need to be so prominent in your assessments, rendering ChatGPT much less useful for students and consequently less concerning for faculty. Below, we provide some examples below of how instructors might modify or personalize assessments in ways that make it more challenging for AI to produce useful text for students to use. Then, we provide some ideas for how ChatGPT can be used to support student learning, rather than trying to fight against its use.

Assessment Modifications

            If you’re concerned about your current assessments, you can modify them. One way to circumvent this type of AI is to ask students to write about something that it doesn’t know about. The data used by ChatGPT is a couple of years old (though they will certainly be updating the corpus of text regularly), so something very recent from the news, or specifically related to students’ direct/campus experience and/or which might not have been written about, or a newly published empirical study will make students have to do at least the majority of the work themselves. Additionally, you could assign something very specific that you have done in class as the basis of a writing assignment. For example, an in-class experiment where you classically conditioned students to salivate to the word Pavlov using sour candy is something that AI likely doesn’t know about. Similarly, asking students to summarize the class’s specific talking points during a debate or group discussion would fit the criteria. Another example of preventing the use of ChatGPT would be to implement a writing assignment that utilizes the self-reference effect that is specific to the institution, for example by requiring students to point to specific student services available at the college and/or buildings/offices on campus. By having students write from their own perspective, this also highlights the self-reference effect, making their recall of the course material more likely. For instance, instructors may assign a writing prompt of “Our university has recently adopted an anti-racist mission statement to better support student learning. Which specific components of this mission statement do you think will have the greatest impact on the campus culture (e.g., which element(s) are you or other students most likely to be able to act upon)? Describe specific ways in which you think this mission statement will impact your experiences on our campus.” In addition to making the course material more personally relevant, another benefit of this is to allow students the opportunity for their authentic human voices to be heard, providing greater potential for them to grow as critical thinkers. Finally, requiring students to interpret or otherwise write about data collected in class and/or the outcome of a study that was (or not) replicated in class may be another way to circumvent the bot by requiring things it doesn’t know anything about. 

Another way to limit the usefulness of ChatGPT is to require that specific references be included in the assignment (or the basis of the assignment), which will also make it easier for faculty to detect inaccuracies written by AI. Currently, the AI is not very good at including specific sources as either in-text citations or references, or citing their sources in general (though with a bit of work and the right prompts, it can be done, though still not very effectively). In actuality, sometimes ChatGPT even invents references and sources that don’t exist at all. There are other AI tools such as Perplexity  (https://www.perplexity.ai/) that do a much better job with referencing sources; however, students will still have some work to do because sources are websites and may not be scholarly.

Reflections and personal accounts of feelings are more difficult for GPT to do, as are integrating life experiences with specific course concepts. With this type of prompt, ChatGPT will generally begin its response by saying something along the lines of it’s a robot so it can’t really reflect, but then will provide some kind of reflection text. So students can still use this, but it might not be as easy for them to obtain immediately useful text from ChatGPT.

Producing written work (or at least a draft or outline) in class, the old-fashioned way (pen and paper) is one way to limit the number of up-front contributions that ChatGPT can make. Encouraging students to collaborate in groups and produce a mind map or other visual representation of what they will be writing will also deter students from using AI to write for them. Further, implementing group work has been shown to increase the quality of communication in the classroom, establish boundaries for expectations of the amount of work each individual should contribute, and establish respectful social norms that each group member has a valuable contribution to make in the learning process (Aronson, 1978). These activities, if they serve as the basis of a written assignment, cannot be input into ChatGPT (at least not at the current time) because it is not text and therefore students wouldn’t be able to easily use the bot. It would be less work for them to write the assignment themselves than to convert the mind map into text and then somehow feed it to ChatGPT.

Another example of moving away from traditionally written work would be to include more oral work from students, whether live or pre-recorded, in front of the entire class or only the instructor, as a group or individually. There are many possible permutations such as presenting the content that would traditionally be in a paper, or a less traditional format such as an interview (e.g., with a fake researcher) to learn about the topic in question. Added benefits of this teaching technique might include strengthening oral communication skills and building rapport between the students and the instructor.

Using ChatGPT

As the saying goes, “If you can’t beat them, join them.” It is likely that students will be able to (at least somewhat) circumvent anything you try to do to limit their use of ChatGPT, so why not include this AI tool in their assignment? One way that ChatGPT can be leveraged is to help students provide higher quality writing as it can provide students with feedback on their writing. So perhaps you can assess students on how they addressed the feedback provided and their explanation for how they improved their writing. You might require them to submit their original draft, the feedback they received from ChatGPT, and their improved version, along with a reflection/explanation about how they used the feedback and what they found the most challenging to address and also the most helpful feedback they received from ChatGPT. 

Since GPT is often inaccurate (especially with references and in-text citations), students could also be tasked with asking it to write something on a particular topic and then tracking down scholarly sources to support the claims made (and if those claims cannot be supported, then re-work the text to reflect that). This will provide students the chance to practice using the library and tracking down sources as well as the mechanics of proper citations and a chance to work on their information literacy skills more broadly. This could be especially valuable in a research methods course to emphasize the rigorous process of publishing scientific research in addition to highlighting the merits of integrity in both writing and research practices.

 ChatGPT can also “grade” assignments, so students could ask it to write an assignment and then use the rubric to score it as a starting point. Students would then improve the writing in order to make it a better and more accurate version of what was produced. Again, asking students to reflect on the process and provide specific examples of where ChatGPT was the least accurate according to the rubric (for example) makes use of the tool and forces students to practice their own writing as well. This exercise could help develop stronger reading comprehension skills in addition to writing skills, as this assignment will help them differentiate between weak and strong writing as well as weak and strong arguments. Similarly, students can use AI to summarize research articles that they could then use in a paper where they synthesize or otherwise integrate the information in some way that is appropriate for the course. Alternatively, students could compare the generated summaries to their own and verify its accuracy, identify errors, and reflect on any differences in focus (e.g., perhaps ChatGPT focused its summary on the results of the study whereas students used a lot of their summary to describe the methods).

Transferable Skills

The other consideration, as we try to wrap our heads around the impact of this tool, should be about students’ eventual workplaces. Many workplaces are already using AI to assist in various tasks, whether overtly or covertly (Walia, 2023), and expectations in the workplace will likely adjust in terms of how long tasks should take their employees to complete in light of this new technology. As such, it would be a disservice for faculty not to give students a chance to use this tool and to become more efficient with it. Using AI is likely to become a new transferable skill (also known as a “soft skill”), which should be developed during their studies and then used in whatever workplace they end up. The transferable skill that current students need to develop in order to be competitive in the workforce may no longer be how to write from scratch, but rather, how to critically evaluat what ChatGPT (or a similar tool) is creating and be able to assess its accuracy, quality, and/or build on that text. Much like the invention of the hand-held calculator still required students to evaluate the answer given using their critical thinking skills, a similar skill is likely to be what needs to be developed to contend with ChatGPT. Further, with the increased efficiency of AI use, workers will have more time available in their daily work schedules to devote to more “human” tasks that involve original and creative thinking, such as problem solving or generating new ideas to implement for various projects.

One somewhat recent assessment trend and best practice has been to use authentic (as opposed to disposable) assessments (Jhangiani, 2017; Seraphin et al., 2019). That is, making students’ assessments something that has an audience and purpose beyond the instructor and classroom. For example, asking students in a writing/grammar course to proofread a local business’ website or tasking psychology students to create pamphlets on a particular issue to distribute through a student services office on campus (e.g., studying/memory tips). In this way, these assessments resemble their eventual workplace more and have a clear purpose. What will their eventual workplaces look like? That is the million-dollar question.

Conclusions

Whether friend or foe, its presence in the education landscape cannot be ignored. Moving away from written assignments or including more of a focus on work completed during class can be used by instructors to quickly modify their current assignments in light of the availability of ChatGPT. Although certain approaches to assessments might reduce a student’s ability to use ChatGPT to produce work for courses, perhaps the technology can be used in a way to encourage critical thinking or improve students’ writing skills. Using these types of tools in time-saving ways may be the expectation of workplaces in the not-so-distant future and students would be well-served to understand its functionality in our classrooms. 

Disclosure: Although we did not use ChatGPT in any way to write this article, we did ask it for feedback on our writing once we had finished and it thought our writing was organized, clear, and a well-written piece overall (Paraphrase from OpenAI's ChatGPT AI language model, personal communication, February 12, 2023).

References

Aronson, E. (1978). The jigsaw classroom. Sage.

Jhangiani, R. (2017, February 2). Ditching the ‘‘disposable assignment’’ in favor of open pedagogy. E-xcellence in Teaching Blog.http://teachpsych.org/E-xcellence-in-Teaching-Blog/4583103

Seraphin, S. B., Grizzell, J. A., Kerr-German, A., Perkins, M. A., Grzanka, P. R., & Hardin, E. E. (2019). A conceptual framework for non-disposable assignments: Inspiring implementation, innovation, and research. Psychology Learning & Teaching, 18(1), 84-97. https://doi.org/10.1177/1475725718811711

Walia, P. (2023, February 9).  Study finds more workers using ChatGPT without telling their bosses. Techspot. https://www.techspot.com/news/97553-study-finds-more-workers-using-chatgpt-without-telling.html