Using AI to Reflect on Student Reflections

A ChatGPT prompt for textual analysis

This is my first Substack, but the penultimate newsletter in a series reflecting on teaching AI & Digital storytelling. See full series here. Soon I will begin a new series on rhetoric and prompt engineering.

---

We usually don't have much time as students hand in their final work, which in my class is usually reflections on what they learned. 

I'm not going to lie ... my tendency is to mark these off as fast as I can, so I can get on with the summer. I try to give students some encouraging words and let them know one or two things they can do better. I'm not sure how many read them, honestly.

So I thought this week ... what if we can use AI to better understand student reflections without adding to my workload.

In other words, how can I use AI to help me reflect on my students reflections.

So I built a ChatGPT prompt that analyzes and compiles three things from a corpus of student reflections.

• Topics students discussed

• Supporting quotes

• Related course outcomes.

➡️ Click here to see the full prompt.

(Keep in mind that ChatGPT can currently only handle around 2000 words, so this only works with a small batch of papers.)

How the Prompt Works

It can be hard to get a birds-eye view of student learning, especially during the rushed part of the semester. Even for instructors, the amount of information that needs processed can be overwhelming ... so why not bring in AI as an assistant to help us see patterns.

ChatGPT was able to quickly compile all the data and present me with a comprehensive analysis of what topics my students discussed. 

You do have to keep an eye out for hallucinations (especially since I asked it to paraphrase quotes), but AI came up with some interesting observations to think about.

Screenshot of ChatGPT Table Output

Here are the three main trends found by ChatGPT. Though AI connected them to various course outcomes, I really think the most important was "developing confidence as writers." 

Creative Enhancement

This class taught me not only how to write stories, but how to utilize new AI tools to enhance my writing.

Many students reported feeling more confident in their writing. I think students are more afraid of the blank page than ever, and AI really helps them get past this anxiety.  Once they got over this anxiety, they enjoyed expanding their creativity and writing skills with AI tools like ChatGPT and Sudowrite. 

They could see their confidence as a writer growing as the semester progressed, even when not using AI. In fact, most of them used AI minimally (but strategically) by the end of the course. Building this kind of confidence was one of the main goals of the class! 

Flexibility in Writing

Writing is a fluid process, and being able to quickly change direction and pivot to a new idea is valuable.

Students appreciated the instant feedback and the ability to iterate quickly, which helped them develop their ideas for stories. 

Students talked about the importance of quickly changing directions when writing. One student mentioned that tools like Sudowrite helped them quickly generate new ideas when they hit a creative roadblock. They could easily pivot to a new idea when they felt stuck or uninspired.

I'm sure this added to their confidence as writers, but also helps them analyze their writing and the writing of others more closely. 

Limitations of AI

AI cannot create something entirely new, as it lacks the thought processes of a real human.

Finally, students discussed the drawbacks of artificial intelligence when it comes to creative writing, noting that it is difficult to use AI to come up with something new. 

AI can help writers generate content quickly and facilitate more efficient and effective writing processes, but it cannot replace originality or innovation and must be used in conjunction with human creativity. 

Of course, some may argue about that, but this at least gets student thinking about the cultural and ethical contexts of using AI for creative endeavors. Another course outcome for us!

After carefully reviewing my students' reflections on their AI-assisted writing process, I realized that most important outcome for them was building their confidence as writers.

Most of these students were first year students, just now adapting to the university ... or entering new modes of writing unfamiliar to them. Not only had they learned how to use AI tools to enhance their stories, but they had also gained a far richer level of insight into the creative writing process, feeling more equipped with the flexibility and confidence to make meaningful changes as they went along. This is something they take to any writing project, not just storytelling.

For most students, using AI did not harm their ability to explore writing, but rather helped them build confidence, so that they could experiment more. It is my opinion that this is where we truly learn about writing ... when we experiment and play.

Now that just happens to include a machine.