The Choices We Make in Chatbot-Assisted Writing

It's not just ... "Ask and you will receive."

Image Generated by Dall-E.

I've said it before, and I'll say it again. Don't treat AI chat tools like vending machines.

AI models are not God ... "Ask and ye shall receive" isn't the best way to leverage Chatbot-assisted writing.

I often caution against treating AI tools like vending machines—insert a prompt, and out pops a perfectly crafted piece of writing. This is a far cry from the reality and, frankly, undermines the very essence of what it means to write with AI.

The belief that students or our tech writing managers can simply command an AI to produce complete, quality work is a myth that ignites fears of rampant plagiarism and the taking of our writing jobs. Yet, as we've witnessed—and as history with any technological shift will tell us—it's never that simple.

At its core, writing is a series of choices, a careful process of selection and arrangement … a workflow that demands our awareness and engagement. This truth crystallized in the minds of my students and myself during a workshop where we delved into the potential of AI to transform their notes into polished documentation, which is one of the best use cases for chatbot-assisted writing.

If there is low-value, “mundane” writing that you do over and over again, creating a structured prompt or an AI process can free you up to focus on projects that matter to you more.

To give you some context, I asked students to document in a memo report their process for creating a customized chatbot. This is one way to get students writing, even when your project isn’t a traditional writing assignment.

It was the end of the semester, so I thought, let’s take a whole class and explore how AI can help generate this documentation using our notes.

This task probably seemed straightforward to students, but certainly was not. The project demanded as much writer-awareness as any traditional writing task.

The true art of writing with AI is not in the asking, but in the interaction that follows.

Making AI workflows visible … Just do it in public.

To give a little context, students have been writing online all semester, developing an audience for their writing by "niching" down on specific personas, for example students interested in muscle-building, shy students needing to make friends, etc. Their job then was to create a chatbot for that specific audience as their final project.

This is not a straightforward process. It requires students to engage deeply with the nuances of their chosen persona—understanding the needs, desires, and language of their specific audience. They had to put themselves in the shoes of their readers to craft a chatbot that would truly give value rather than a gimmicky trick.

Through this exercise, they practiced not just writing, but also the critical thinking that underpins effective communication. They had to analyze user contexts and explore the ethical implications of data use. This final project provided new opportunities to collaborate, iterate, and, most importantly, to blend analytical with the creative mindsets.

But the process isn’t done until they document their process and what they learned. Honestly, if they use AI to document this process, that’s fine with me … as long as it meets the specs and clearly communicates to readers their process.

So I provided students with a base structured prompt to make sure we were all starting on the same page … including ChatGPT.

[ROLE] You are an research assistant expert at creating user reports for new technologies like AI chatbots. You are responsible for clearly documenting the creative process behind chatbots. No one else can do this, so it is very important.

[STYLE] Write in a professional tone and style that clearly communicates the process to stakeholders without embellishment. Be thorough but concise. Use block paragraph formatting. Use section headings to help guide your audience through the information. This also helps them find and reference information during meetings and such. Section headings should be bolded, so they stand out. Do not use colons or underlining. If you use outside sources, they should be easily accessible in a reference list at the end of the memo.

[FORMAT] Usability reports should be written in memo report format, using the following sections:

Heading: Include 'To:', 'From:', 'Date:', and 'Subject:' lines.

Introduction: Briefly state the purpose, objective, and target audience of your chatbot.

Chatbot Design and Development: Summarize the process of choosing the task, implementing structured prompting, and technical details.

Testing and Refinement: Outline your testing approach, feedback incorporation, and any adjustments made.

User Experience: Describe the chatbot's interface and user interaction.

Ethical Considerations and Data Privacy: Mention how you addressed ethical use and data privacy.

Conclusion and Learning Outcomes: Conclude with a brief overview of achievements, challenges, and key learnings.

[TASK] I will provide you notes for each section and you will draft this memo report that documents how I developed my chatbot. Let me know if you need any more information. This should be 300-500 words. If you need more words, give a specific detail or reasoning.

### 

<notes> Add notes here. </notes>

I actually had a lot of ideas for making this prompt better, but I started here, so that we could adapt the prompt together as a class.

From there, it was all about iteration—refining the prompt collaboratively as a class to better suit our needs and objectives. This was a hands-on lesson in adaptability and precision, two skills that are invaluable in the field of content development and writing.

As we worked together to improve the AI ouptut, though, I noticed us taking three different kinds of choices, which helped elucidate for me the process of chatbot-assisted writing.

As I tried to explain this to students, it struck me … why don’t I try to visualize it in a flowchart.

Any kind of writing process is a system of choices.

Let me take you through how we implicitly worked through this system of writing choices.

Chatbot-Assisted Decision Flowchart (made in Whimsical)

I’m a firm believer in using AI in public. It’s not something to keep secret or be ashamed of. So as much as possible, I prefer to show rather than tell. It's one thing to discuss theory, but to see it in action as I make real-time choices to adapt chatbot documentation—that’s where I hope students can learn more nuanced approaches to AI writing.

Once we determined that this was a task that might be enhanced through chatbot-assisted writing, we deployed our structured prompt. Of course, the AI-generated text wasn't anywhere close to hitting the mark. Because documentation reports are often low-value, repeatable writing tasks, learning to adapt and re-run the prompt can improve the reusability of that structured prompt.

Writers can reuse it for other documentation tasks … or I can reuse it in future classes.

Initially, the output from ChatGPT was concise but too brief, offering only single paragraphs under each heading. Recognizing the need for depth, we asked ChatGPT to expand each section with an additional paragraph, specifically requesting that it include examples drawn from the students' notes. When it became evident that the notes themselves were insufficient for this task, it prompted the students to enrich their notes with more detailed examples, reinforcing the need for comprehensive initial input when working with AI.

The limitations of the free version of ChatGPT became apparent when it struggled to incorporate specific examples effectively. To overcome this, the class engaged in dialogue-based editing, interacting directly with the AI to refine each section. By requesting that the AI incorporate particular instances from the expanded notes, the results improved significantly.

This step-by-step engagement illustrates the potential of conversational AI to tailor content when provided with precise directives. We asked clarifying questions, refined our requests, and the AI responded in kind. This was a give-and-take, teaching the students the importance of clarity and interaction in achieving the desired outcome.

Despite the advancements made with AI assistance, the class observed that the reports lacked fluid transitions between sections and a clear takeaway in the introduction. To address this, students would need to manually edit these areas, crafting transitions that provided a natural flow and revising the introduction to ensure a concise summary of the report's goals and findings.

Using the AI's output as a foundation, this next step would add personal touch—rephrasing for style, injecting nuance, and ensuring the final product reflected human insight. It was a powerful demonstration of the AI's role as a collaborator, not a replacement, in the writing process.

This iterative process—starting with an AI-generated draft, enhancing it through targeted prompts and dialogue, and applying manual refinements—embodies the collaborative synergy between human writers and AI tools.

Through these exercises, students witnessed the tangible impact of their choices. They learned that while AI can be a powerful ally, it is their guidance and their decisions that truly shape the documentation.

This was not just a lesson in writing; it was a lesson in agency, in the power of human intentionality in the age of machines.

Actually … it didn’t got that well.

The final documentation reports that I received were pretty terrible, if I’m being honest. This work didn’t necessarily translate into the students’ work. The process of utilizing chatbot-assisted writing is more intricate than commonly perceived.

Writers must make nuanced choices throughout the writing process, requiring careful consideration and decision-making to achieve desirable outcomes. There are many opportunities for failure.

Here are my thoughts on why it didn’t work:

Limitations of Free Version of ChatGPT: Using the free version of ChatGPT proved to be challenging, falling short results I usually get with GPT4. The limited capabilities of the free version hindered its ability to generate high-quality content, highlighting the need for premium options for more advanced tasks.

Settling for Mediocrity: Students often settle for subpar work, especially towards the end of the semester. Their definition of "good enough" isn’t really good enough. Encouraging students to go beyond mere adequacy and strive for excellence is crucial to achieving quality outcomes with AI.

Formatting Challenges: Both ChatGPT and students struggled with formatting and organizing memo reports. Despite prior instruction, students still encountered difficulties in this aspect. Mastering new genres like memo reports requires practice, even when using AI.

Importance of Detailed Notes: Lack of detailed note-taking posed a challenge in the AI-assisted writing process. The student who included a five-page draft had a much better result. This showcased the significance of thorough groundwork in ensuring successful collaboration with AI tools.

That said, I think it was a great learning experience, especially when incorporated into what studenst already learned about AI-assisted writing using tailored chatbots and writing environments like Lex AI.

Lesson 1: The digital divide is real. Equal access to AI tools isn't just important, it's necessary for fairness in tech advancement.

Access to AI tools should not be limited to a privileged few; instead, it is crucial to ensure equal access for everyone. This is not only important for fairness in technological advancement but also for fostering inclusivity and providing opportunities for all individuals to harness the power of AI.

Lesson 2: AI won't serve you solutions on a silver platter. For complex tasks, it's an assistant, not a genie.

AI serves as a supportive tool that works in collaboration with human writers, especially in more complex tasks. It is important to approach AI integration with realistic expectations, understanding that it requires human guidance, decision-making, and critical thinking to achieve desired outcomes.

Lesson 3: Leave the structuring to humans. Our thought patterns are unique, and AI is still catching up in mimicking this aspect of our cognition.

While AI tools can assist in generating content, the expertise of human writers in structuring and organizing information is invaluable. Human writers possess the ability to craft coherent narratives, provide logical flow, and adapt the structure to suit the specific needs of the audience. Harnessing the synergy between human creativity and AI assistance leads to more effective and engaging writing.

Lesson 4: Your written content feeds AI. The richer your input, the richer the AI output. So keep writing, and write some more! ✍️

As writers, the more we invest in producing rich and detailed written content, the more meaningful and insightful the AI-generated output becomes. By continuously writing and honing our craft, we improve the training data that feeds AI, thereby enhancing its performance. This creative partnership between the writer and AI can facilitate more nuanced, effective, and efficient writing workflows, leading to refined final products.

As the semester draws to a close, it's clear that this technology's role in education and writing is not just an add-on—it's a window into a future where AI tools stand to be as common in the writer's toolkit as the thesaurus and style guide. Yet, our foray into using ChatGPT to draft reports was far from a seamless victory. It was a hands-on experiment with all the messiness and unpredictability that real learning entails.

So, let's bring this conversation out into the open. Let's not hide our use of AI, but rather, let's showcase it—warts and all—for it's through transparency that we can truly understand and harness its potential. Whether you're a student, a teacher, a technical writer, or a content creator, your experiences, challenges, and victories with AI are part of a larger narrative that we're all writing together.

I’d love to hear more stories from you! Where has AI smoothed the path, and where has it thrown up roadblocks? What unexpected lessons have you learned by integrating AI into your writing process? And how do you envision the future of this powerful collaboration?