Why Do All "How to Write Good Prompts" Guides Seem Similar?

If social media algorithms incorporated generative AI, you'd frequently encounter various "how to write good prompts" content. Every time I see such guides, I notice they all seem quite similar. They commonly suggest writing your requirements in detail using bullet points, structuring with sections like Goal, Tone, Format, and including examples. Whether it's COSTAR, CARE, or other frameworks, their suggestions don't seem fundamentally different.

If You Know What You Want to Do, Writing a Prompt is Simple. But Do You Really Know?

When learning new technology, we often think, "Now that I've learned the basics, I can start using it in practice." However, when actually applying it to our work, we realize that basic usage is just fundamental knowledge. The same applies to generative AI prompting. It's true that most prompt writing guides contain similar content. This is because prompt writing techniques focus on "how to make generative AI better understand given requirements." What we really need to focus on is what these "given requirements" are.

To answer "What do I want to do with AI?" leads to the question "What is possible with current AI capabilities?" If we add the condition that it must be truly helpful for our actual work, it might be more challenging than expected. Even if it's possible, we need to consider "How do we reach that desired outcome? Is it possible with just one prompt?" If you know the answers up to this point, if you know what you need to do, then writing a prompt is just a formality. You can even ask AI to write the prompt for you. If you tell AI what you want and ask it to write a prompt, it will find prompt writing guides online and create a good prompt for you. Therefore, the time-consuming part isn't writing the prompt. The process of 'designing' what to do with the prompt takes much longer and is more difficult.

This is because even thinking about what's possible with prompts is crucial. For example, I initially tried to use AI mainly for 'analysis/summarization of materials.' It wasn't until I saw someone else use it for 'review/evaluation' that I realized such use cases were possible. To put it more elegantly, we can apply the principle that recall is more difficult than recognition. Once you know about a case, it's easy to apply it to your situation, but coming up with new use cases you've never tried is harder than you might think.

That's why I think "the process of applying generative AI to work" is similar to someone who can't draw buying an Apple Pencil and Procreate app. (This is my story 😅) The same tools in the hands of a professional illustrator produce amazing landscapes, but when I hold them, I don't even know what to draw, let alone how to draw it. Beginners don't know which color combinations are beautiful or how to customize brushes to create certain types of lines. Moreover, whether you want to create watercolor paintings or character illustrations, the tool usage methods would be vastly different.

Being able to answer "What do I want to do with AI?" means knowing "specifically and accurately" what AI can do. Using an Apple Pencil to open Procreate and draw something is just a matter of doing it. However, what you want to draw in the first place, and whether it's something that can be drawn with Procreate, is a much more important, difficult, and complex question.

The process of reducing the gap between what AI can actually do (B) and what we think AI can do (A) is what I believe improves our ability to utilize generative AI.

Let's Map Out the Landscape of Generative AI Prompting

I currently work as a Product Manager in B2B SaaS and use generative AI in various ways in my work. However, I didn't want to use generative AI merely as an advanced search engine. For 'work purposes,' I thought it should at least replace the actual work I was doing. After exclaiming "Wow!" with genuine excitement while using Cursor, I enthusiastically tried various things. Some worked, but many didn't. On the other hand, approaching it from a 'replacement' perspective was also a limitation in thinking. There were cases where it couldn't 'replace' but 'accomplished what I couldn't do.' After these trials and errors, I felt I needed to go back to basics and properly fill in the big picture through study.

Components of Generative AI Prompting

Components of Generative AI Prompting

The 'generative AI prompting' discussed in this article focuses on applying natural language text generation AI to actual work. I believe generative AI prompting consists of five main components, as shown in the diagram above.

[Prompt Objective]

What task you want AI to perform and what output you want to obtain as a result. If the purpose is to replace your work, then the work you were doing directly becomes the definition of the output you want to secure with AI.

[Prompt Workflow]

Simply put, this refers to how many times AI needs to be executed to reach your goal and how to link the results of each execution. You could ask AI to analyze given materials with a single prompt, or you could split the materials according to certain criteria, analyze each part, and then ask it to summarize the combined results. Therefore, defining the input for each prompt is crucial in the prompt workflow.

[Prompt Structure]

This refers to how you organize requirements for AI using sections, like the COSTAR framework. Some prompt frameworks present 'sections,' while others present 'principles,' so I used the simpler term 'structure.'

[Prompt Style]

This refers to prompt writing 'principles' such as writing clearly and in detail, using writing separators (hashtags, quotes, etc.), and always providing examples. We don't write prompts emotionally like novels. Compared to structure, this is a recommended format applied to the entire prompt, so I used the term 'style.'

[Prompt Evaluation]

If you're dissatisfied with the results you're getting from AI, is this due to AI's limitations? Or are you giving up when the prompting could be improved? To improve, you need to evaluate, and to evaluate, you need evaluation criteria. When evaluating people, we can only apply qualitative criteria, making objective judgment difficult. The process of evaluating AI also tends to fall into the pattern of looking at and evaluating each result with human eyes. The more the work requires correct answers, the more important it becomes to find ways to evaluate more consistently and efficiently.

To Be Continued

There's so much to discuss about each of these five items. [Prompt Objective] is closely related to [Prompt Workflow]. The process of designing [Prompt Workflow] becomes the process of increasing the achievement rate of [Prompt Objective]. [Prompt Structure] and [Style] should be approached as understanding principles rather than just techniques. To write natural language from a generative AI perspective, we need to understand how generative AI thinks. When looking at publicly available materials about [Prompt Structure] and [Style], I found they were very helpful in understanding these operational principles of generative AI. Finally, the [Prompt Evaluation] stage is the most difficult to find established methodologies to reference. Due to the probabilistic nature of generative AI, it's difficult to obtain consistent results, making it challenging to apply quantitative evaluation criteria. Here, we also need to consider [Prompt Objective]. There are objectives where it's okay if the operation doesn't execute (failure) or produces incorrect answers (errors), and there are objectives where we can't tolerate even a 10% chance of failure/error.

In the next article, we'll look more closely at [Prompt Objective]. It's easy to find various examples of generative AI applications. When we gather these together, we can see what human capabilities generative AI can perform and to what level. As mentioned earlier, generative AI is not yet a magic wand. Just as we discuss expected capabilities when a new employee joins, let's set expectations for AI.