Does the College Essay (or Writing Instructor) Matter Anymore?
Authentic Assessment in the Age of Artificial Intelligence
I spent several days in late March hiking in central and southern Utah, surrounded by petroglyphs created by the Fremont people more than 1000 years ago. The enduring figures of bighorn sheep, hunters in horned headdresses, and abstract geometric figures, some of which might have been calendars, spoke to something human in me. On other hikes, I saw more modern inscriptions carved into red rock (subject, as signs everywhere reminded visitors, to a $300 fine) declaring Kyle’s love for Jan or that Peter was here. These inscriptions—ancient and modern—say “I was here. I existed.”
Will anything we make today be around 1000 years from now? How will generative artificial intelligence affect one of the primary purposes of writing: to share what we think?
Technology, Productivity, and Creativity
I’m very secure in my own writing abilities—with or without generative artificial intelligence. I’ve written an award-winning book, The Price of Silence (that was scraped without my consent as part of the Books 3 data set).
I have also co-written an open literary analysis textbook with Chat GPT 3.5. I deliberately “nerfed” myself with an inferior model to imagine and explore the possibilities of the tool using the same interface my students would most likely use. I wanted to see what the free version could—and couldn’t—do.
What it could do astounded me. There’s no question that this new technology has increased my productivity. I’m a fast writer, but ChatGPT can knock out a draft that would take me 30 minutes on my own in a mere 30 seconds.
Technology has always enabled increased productivity, and new technological innovations have not killed off human creativity yet. While I’m inclined to worry at least a little about Matthew Kirschenbaum’s predicted textpocalypse, I’m also really enjoying my personal writing buddy. While I was working on a chapter about New Historicism, for example, I really couldn’t remember what Michel Foucault meant by “autochthonous transformation” in historical knowledge, so I asked ChatGPT 4.0. By engaging with an always ready, always informed conversation partner, I understood the concepts much better than I did when I was diligently taking notes during a professor’s lecture in graduate school.
And generative AI tools help with creative writing too. Microsoft Copilot makes a decent developmental story editor, for example, as I learned when, inspired by Sheila Heti’s AI-assisted story in the New Yorker, I engaged with the tool to write a flash fiction story about a woman who falls in love with her AI-powered digital twin in a dating app.
Now, though college professors continue to identify plagiarism and cheating as their main AI concern, we inevitably have a waterfall of scholarly articles written by (and probably assessed by) ChatGPT. This is where we have to ask ourselves what we have been incentivizing, and how those incentive structures are playing out in our current landscape. We commoditized education by connecting it to careers. Market-savvy students rationally care more about the piece of paper they receive after four years than the process of improving their minds. Learning to read Plato in ancient Greek does not translate into gainful employment.
And faculty are also fighting against incentives that reward productivity over deep inquiry. I work at a teaching college by choice, but I’m well aware of the “publish or perish” mentality that has led to an astonishing (and generative AI-assisted) increase in scholars’ productivity over the past year.
We’ve traded building something that will last 1000 years for the comforts of a commodity-driven culture, where knowledge is not a valued commodity. This could be a problem. But what does it mean for college writing instructors like me?
The New (Old) Challenge: Authentic Assessment
As a writing teacher and a scholar, I’m more concerned about creating new assignments that leverage these new technologies than I am about catching ChatGPT “cheaters.” I do not have either the time or the temperament to be the AI police. So I’ve turned instead to my work with open education resources and authentic assessments.
Authentic assessments encourage students to participate in “real world” learning scenarios, applying their knowledge in creative and meaningful ways. In his 1990 article, “The Case for Authentic Assessment,” Grant Wiggins observed that “Assessment is authentic when we directly examine student performance on worthy intellectual tasks” (p. 1).
I like this definition because it’s flexible. “Worthy intellectual tasks” can be defined in many ways. And flexibility is really what this moment calls for in higher education.
Does the proliferation of generative AI tools mean that the “traditional” college essay is dead, as Stephen Marche famously argued in December 2022? Maybe, and—hear me out—maybe that’s not a bad thing. If generative artificial intelligence can provide the scaffolds for academic writing, maybe students will be capable of so much more than the perfunctory five-paragraph essay. If I am being honest, these are super boring to grade anyway.
When we think about authentic writing assessments in the age of generative artificial intelligence, it all starts with prompting. Prompting involves a new and somewhat opaque rhetorical situation for most of my students. When I informally surveyed my community college students at the beginning of Spring 2024 about how (or if) they were using generative AI tools, most were not. A few had tried the tools but found them to be useless. “I couldn’t get it to do anything,” was a common complaint. But a few superusers were already leveraging the tools in their writing. These students had learned how to craft detailed, specific, role-task prompts that provided results they could work with.
These super users had also learned about the tools’ limitations. One assignment I give students early in the term, inspired by the tech YouTuber Marques Brownlee, is to have students test the tool with something they know really well, such as an anime or video game. This experience inevitably produces skepticism at the AI’s accuracy and “knowledge.” The thing about AI text is it “sounds” good. But experts will find it lacking, and they will know how to fill in the gaps.
For me, considering what authentic assessment means in light of generative AI has meant rethinking my assignments. I’ve given a “Writing in My Field” analytical essay assignment in English 102 for the past 8 years. In Fall 2023, I added a generative AI prompting component. And this spring, I’ve changed the assignment entirely to a multimedia project, where students are encouraged to use all the tools at their disposal, including AI, to explore what writing and communication looks like in their future careers.
One of the challenges of authentic assessment, as Wiggins noted in 1990, is that it just takes more time to design and assess this type of work. In a future post, I will share my experiences working with My Essay Feedback, a ChatGPT 4-powered formative feedback tool that has made my grading so much easier and vastly improved my students’ work. I’ll also talk about privacy concerns and student work.
So What? Who Cares?
This brings me back to my original question: what will we do today that will last 1000 years? When we incentivize productivity and quantity over quality what do we lose? At Fremont Indian State Park, I walked among exhibits of ancient tools for weaving, hunting, scraping. I saw stunning examples of hand-crafted grayware pottery, built to last. What will my students and I make that will be recognized 1000 years from now?
Maybe nothing. Any student of history knows how much depends on luck. But I think that generative AI tools, when used to augment our intelligence and to serve authentic assessments, will usher in a new era of human creativity and flourishing.
Now more than ever, we need to make students care about their learning and help them understand how to use writing to share what they think. AI can’t know the awe I experienced firsthand when I gazed at signs of an ancient culture.
But LLMs can be a powerful means to improve communication and create digital equity—if we have equitable access. (Aside: this is one of the reasons I love Snapchat’s AI and have encouraged my students to use it in the classroom—they all have access to it).
What does it mean to be an artisan in the age of artificial intelligence? As a teacher, I’m exploring these new tools with the goal of inspiring students to make their mark on history.
Resource: Multimedia Project: Writing and Communication in My Field
References
Brownlee, M. (2023, February 17). The biggest problem with AI! [Video] YouTube.
Cengage Group (2023, August 28). Apprehension of generative AI in higher education overstated, Cengage survey finds. https://www.cengagegroup.com/news/perspectives/2023/higher-ed-gen-ai-faculty-research-findings/
Heti, S. (2023, November 13). According to Alice. The New Yorker. https://www.newyorker.com/magazine/2023/11/20/according-to-alice-fiction-sheila-heti [Paywall]
Kirschenbaum, M. (2023, March 8). Prepare for the textpocalypse. Atlantic Monthly. https://www.theatlantic.com/technology/archive/2023/03/ai-chatgpt-writing-language-models/673318/ [Paywall]
Marche, S. (2022, December 6). The college essay is dead. The Atlantic Monthly. https://www.theatlantic.com/technology/archive/2022/12/chatgpt-ai-writing-college-student-essays/672371/ [Paywall]
Wiggins, G. (1990). The case for authentic assessment. Practical assessment, research, and evaluation, 2(1). https://openpublishing.library.umass.edu/pare/article/1292/galley/1243/view/