Let’s start off by establishing a simple fact: there is no ultimate prompt and there is no ultimate prompt structure, just like there is no ultimate way to talk to humans. Ironically, I see personification of LLMs as a very common (and somewhat problematic) occurrence, but it seems to be thrown out the window the moment prompting strategies are being discussed, even though this might be one of the few cases where thinking of LLMs as human-like can be decently useful, if done correctly.

Of course there are examples of objectively good prompts and objectively bad prompts, but just like with good and bad communication, it’s incredibly context-dependent and can often only be judged in hindsight (usually by asking “Did I *actually *achieve what I was trying to achieve?”).

The philosophy

So we will treat good prompting like good communication, which can tell us a lot about how to get good at it:

Step 1: Say the thing you want to say. Step 2: Observe. Step 3: Learn from what you’ve seen.

And here is the nice part about LLMs: As users, we have full control over their memories! It doesn’t matter if our first attempt at communicating something fails, we can just erase that from the LLM’s memory and start from scratch. Another benefit we get from this is that we don’t have to be sure what exactly we want and can just find out on the fly by observing results until we get one we like (this is something you should *not *be doing with humans unless you’re in something like a brainstorming setting).

The practice

So how can we apply this?

1. Start off by telling the LLM what you want it to do, in the clearest way you can.
2. Let the LLM generate its answer. Is it what you wanted? Great, we’re done! Good job! It’s not what you wanted? Let’s observe and refine.
3. Look at the LLM’s answer in relation to your prompt. What exactly did it get wrong? Did it misinterpret a part of your prompt? Did it lack some information? Is the answer alright, but just not to your taste?
4. Edit your prompt to provide additional clarity, instructions (some step-by-step guidance can go a long way!) and information. Yes, edit. Try to avoid continuing the conversation to fix errors, as there is a solid body of research (1,2,3,4) suggesting that LLMs are steered to make more mistakes after an initial error.
5. Repeat steps 3 and 4 as necessary.

I know this might not sound as sexy as “Here’s the ultimate prompt that will fix all your problems!”, but this approach will help you develop prompting as a skill. You will get better over time and your intuition will allow you to get more prompts right on the first try.

Since all of this was pretty abstract, here are a couple of more concrete tips:

• Be careful with role assignments (“You are an industry expert in XYZ”), this could lead to an increased chance of hallucinations since the LLM might simply start cosplaying (imagine you said that to another human, replace the word “hallucination” with “bullshit” and you will see where I’m coming from with this). However, role-assignments can be useful for setting the tone/style of a reply.
• When switching topics or jumping to a new task, unless you need the context of the current chat, open a new one. LLMs are steered by the total context given to them and you don’t want unrelated information to interfere with what you’re trying to do.
• LLMs are probabilistic, so regenerating an answer with the same inputs will, in most cases, lead to different outputs (this is especially useful in that “I don’t actually know what I’m looking for”-case, so just let the LLM give you options!)
• Keep in mind that you’ll have (possibly hidden) system prompts in place on most chat platforms, so there might be additional information and instructions that are fed into the LLM and are not written by you.
• Every LLM is different in both the way it interprets your inputs and the style of its outputs. This is where observation comes into play again!
• Structure is useful. Use line breaks and even markdown where appropriate.
• Write properly. Fix typos in your own prompt and make sure it wouldn’t cause someone teaching the respective language to break down in tears.