Is ChatGPT Living Up to the Hype?
For me, the answer is an emphatic “yes.” Here's how I used project based learning to teach myself AI prompting skills I could use in the classroom.
Key Takeways (by ChatGPT)
Educational Potential: Despite skepticism, the author finds generative AI tools like ChatGPT to be highly beneficial for educational purposes.
Project-Based Learning: The author used ChatGPT to co-write a literary analysis textbook, which improved their understanding of AI and facilitated valuable discussions with students.
Challenges in Adoption: Many users struggle with AI tools due to user-friendliness issues, ethical concerns, and unclear use cases, limiting broader adoption.
Practical Applications: AI was effectively used for outlining content, providing background knowledge, drafting model essays, generating discussion questions, and summarizing content, leading to improved student engagement and project quality.
Rethinking Assignments: The presence of AI tools necessitates designing more engaging and challenging assignments that cannot be easily completed by AI, enhancing the overall learning experience (Open AI, 2024).
After a week of pushback against the promise of generative AI including Goldman Sachs’ critical investment report entitled “Gen AI: Too Much Spend, Too Little Benefit?”, I’m just going to come out and say it: The most obvious use case for generative AI that many college writing teachers can think of continues to be the one we definitely never asked for: producing a bland, generic, college essay like the one Copilot wrote for me on the April 2024 solar eclipse.
If you’re still skeptical about the AI hype, you’re not alone. Goldman Sachs head of global equity research Jim Covello observed in the report that “AI technology is exceptionally expensive, and to justify those costs, the technology must be able to solve complex problems, which it isn’t designed to do.”
But even if you’re skeptical, your students are probably using these tools at this point. An October 2023 study reported that more than half of students surveyed were using generative AI, while 75% of faculty had not yet tried the tools.
If you haven’t tried generative AI yet, or if you interacted with a chatbot briefly but could not see how these tools can benefit you as a teacher, instead of hunting for the “perfect” training (or avoiding the subject altogether), I recommend taking it for a comprehensive test drive on a big project.
One of the first things I did with AI tools was to write a literary analysis textbook, Critical Worlds: A Targeted Approach to Literary Analysis. This type of large-scale project can help you to learn what works and what doesn’t when you prompt. Being transparent about your own AI use can also spark important conversations with students and colleagues about the “right” way to use these tools—or whether to use them at all.
Challenges to AI Use and Adoption
Before I share how I used AI to co-write a textbook last year, I want to explore the idea that AI is not living up to the hype based on my own experience after three semesters of teaching with and working with generative AI tools. Here are some challenges I’ve observed:
1. People still aren’t using AI. I hear this from my students all the time. Despite the massive surge in first-time users when Open AI released ChatGPT, for many people, the process of interacting with any of today’s LLMs is far from frictionless. People try the chat interface, can’t get what they want, or they get a nonsense hallucination like the infamous “eat rocks” or “put glue on pizza” responses, and they naturally conclude that this tool is not particularly useful for them. (Aside: I am personally curious to see how Apple’s rollout affects AI adoption more widely because I think Apple will provide a much more frictionless experience for its users. As a long time Apple user, that’s been my personal experience over time).
2. When people do use AI, they aren’t using it for the “right” reasons. What I mean here is that people aren’t using AI for tasks where it can actually be helpful. I don’t want to get into the ethics weeds in this essay, though these conversations are essential! We’re already seeing an entirely predictable upswing in more sophisticated scammers and phishing, for example. The Goldman Sachs report calls out what I have been worried about since last summer: The tremendous use of energy resources is putting a world that is already warming too fast at even greater risk.
3. The products still don’t have clear use cases for many people. The AI companies released a product to users in the hopes that the users themselves would identify the use cases. I’m not among the folks who think this was necessarily a bad plan per se, but people seem to be struggling to find the value in AI tools beyond parlor tricks (remember when Siri told us jokes on our iPhones back in 2011 and we thought it was so great for about five minutes?). And even when people identify positive use cases, there’s such a firehose of information about AI that it can be hard to know where to start.
Project Based Learning: Writing a Literary Analysis Textbook
With all these concerns, why do I continue to think that AI is potentially transformative for teaching? Like most people, I initially struggled to get AI to produce useful results. But unlike most people, I didn’t quit. Instead, I decided to give myself a project-based learning experience to discover AI use cases. I was already in the process of writing an open education resource for my English 211 Introduction to Literary Analysis class. The project had some urgency because the textbook I was using was about to go out of print, and at the time, there were no real options to replace the text.
I’ve been active in open education since 2020, and I’ve found that curating, remixing, or creating OER has positively impacted both my own pedagogy and my students’ engagement with my courses. The English 211 course was the only course I teach that I had not yet converted to OER, though the culminating assignment is a student-created OER critical analysis anthology, Beginnings and Endings (check it out! Students love this project, and the assignment scales to many other disciplines).
I was on a tight timeline to complete the textbook, and because I was a department chair at the time, I struggled to find the time to write and research. So I turned to ChatGPT 3.5. ChatGPT 4.0 had already been released, but I deliberately nerfed myself on this project because I wanted to use the same freely available tool my more ambitious and tech-savvy students might use in my courses. I also decided to be as transparent as possible about my writing process, citing and acknowledging my AI use and including links to the chats I used to co-write the book.
My favorite part of this book is the chat history appendix. You can see just how awful I was at prompting by reading the early conversations, but I quickly got better. I quickly discovered that giving the AI a role was important, and that conversation context windows were helpful.
I also learned that ChatGPT is not great at providing scholarly sources. I don’t even like the more capable Copilot or Perplexity for this task. Even though they provide “real” sources, I have discovered that selecting and curating my own resources is something I enjoy and want to do myself).
The earlier model I used, ChatGPT 3.5, is particularly prone to hallucinations (though 4o also makes plenty of mistakes), as this model essay and my annotations demonstrate.
One thing that is clear and that this textbook reinforces to students: Knowing facts and how to verify them with high-quality sources definitely matters in the age of generative AI.
I haven’t done a formal study of the textbook because I was too busy actually producing it to write up an IRB proposal. In the first semester I taught the new Critical Worlds text, I was still very much building the airplane in flight. But anecdotally (supported by course evaluation feedback), in Spring 2024, students loved the new textbook and the AI approach. I had better engagement and higher-quality final projects than I had previously seen in this course (which students often describe as “the hardest class I’ve had in college.” Hey! It’s an intro to critical theory. What did you expect? Something easy like statistics?).
Just like I used AI in my own work, I also allowed students to use generative AI in their papers as long as they acknowledged and cited. I like seeing how students are trying to incorporate generative AI to support their writing. Here’s an example of how one student used AI in their Creative Commons licensed essay, “Your Body, Your Rules.”
“For this essay, I used Chat GPT as a resource to give me a summary of the feminist and queer theory analysis lens, “Feminist queer theory is a critical analysis lens that combines feminist theory and queer theory to examine how gender and sexuality intersect and shape social power dynamics. This approach challenges the dominant cultural norms that promote heteronormativity, gender binary, and patriarchy, which result in marginalizing individuals who do not conform to these norms.” With this, it helped me better understand the material so I could write better essays. This information was accessed on May 6th, 2023.”
How I Used Generative AI to Co-Write the Textbook
As I mentioned previously, I used generative AI to explore how these tools could support my students’ writing process. Here are some some use cases I discovered:
1. Outlining content and making sure I was covering the important information. This is one use case where generative AI excels, in my opinion (see key takeaways above).
2. For background knowledge or basic definitions. Even ChatGPT 3.5 was pretty good at this, though I verified facts and provided sources myself. ChatGPT 4.0 is much, much better at this than 3.5 (I finally caved and used 4.0 for the Ecocriticism chapter).
3. Drafting model essays for me to annotate and critique. Since I suspected students would at least try to have the AI write their essays (I would have! It’s a cool new tool!), I preempted them and had ChatGPT 3.5 write an essay for each critical lens we study in the book. Then I annotated the essays with my own feedback, knowledge, and experience. And this was SUPER fun. Remember how I said the college essay is dead? My students like using the tool to write their analysis for them, then critique the analysis. It turns this difficult learning into a fun game. I plan to update the textbook essays with ChatGPT 4.0 or Microsoft Copilot essays so students can also compare how these tools are evolving and improving.
4. To generate several discussion questions about short literary texts. I don’t mind writing analysis questions myself, but it’s so much easier to start with several options and choose the ones I want.
5. To help me summarize content and create checklists for students. I’ve started doing this with my blog posts (and all my writing, really). It’s a helpful check to make sure I am saying what I think I am saying.
To propose book cover ideas, one of which I adopted (though I created the cover myself in Adobe Photoshop).
I have incorporated all these types of work (including the use of image generation) into my students’ writing process workflow (along with generative AI formative assessment feedback, which I’ll discuss in a future post).
Conclusion
If you’re worried about students using generative AI to write their essays, maybe it’s time to rethink some assignments. I personally think it’s a good thing that these tools shook up our traditional assessment methods. We needed it. If a predictive text algorithm can write a middle of the road college essay for our students, maybe we don’t need plagiarism checkers or even watermarking. Maybe we need a better way to teach and assess their progress. Or as I tell my students, if the writing assignment I give you is so boring that you just want to outsource it to ChatGPT, maybe I need to give you a more interesting assignment.
P.S. Since this blog has been an accidental travel log from the start, here’s a picture from the deck of the house at Bear Lake (aka “The Caribbean of the Rockies”) where I wrote this post. Happy prompting, and don’t forget to touch grass!
References
Coffey, L. (2023, October 31). Students outrunning faculty in AI use. Inside Higher Ed. https://www.insidehighered.com/news/tech-innovation/artificial-intelligence/2023/10/31/most-students-outrunning-faculty-ai-use
Goldman Sachs Global Macro Research (2024, June 25). Gen AI: Too much spend, too little benefit? Top of Mind [Newsletter]. https://www.goldmansachs.com/intelligence/pages/gs-research/gen-ai-too-much-spend-too-little-benefit/report.pdf
Long, L., ed. (2024). Beginnings and endings: A critical edition. CWI Pressbooks. https://cwi.pressbooks.pub/beginnings-and-endings-a-critical-edition/
Long. L. (2023). Critical Worlds: A targeted introduction to literary analysis. CWI Pressbooks. https://cwi.pressbooks.pub/lit-crit/
Microsoft Copilot (2024). The celestial dance: Understanding solar eclipses. https://docs.google.com/document/d/11Y_2-WEyIRAMInl7Np9zRHdKb774jp_VywLzJybo-rY/edit?usp=drive_link