# Protologue: A Taxonomy of Prompting and LLM Techniques Version 1.0.0, updated 2026-10-10. License: CC BY 4.0. Source: https://protologue.com/ # Branch: Foundations Core concepts of prompting — the parts of a prompt, how models consume it, and the basic zero-shot and few-shot paradigms. ## Context Window > The context window is the maximum number of tokens a language model can attend to in a single call, covering both the prompt and the generated output. - Identifier: PTL-0004 - Category: Foundations - Canonical URL: https://protologue.com/t/context-window/ - Also known as: context length, context size ## Description Anything outside the context window is invisible to the model unless it is re-inserted. Larger windows allow whole documents or codebases to be included, but research shows models do not use all positions equally well, which motivates retrieval, summarization, and careful placement of key information. ## Related terms - [Token](https://protologue.com/t/token/) - [Lost in the Middle](https://protologue.com/t/lost-in-the-middle/) - [Needle in a Haystack](https://protologue.com/t/needle-in-a-haystack/) - [Context Engineering](https://protologue.com/t/context-engineering/) ## Sources - Vaswani et al. (2017). Attention Is All You Need. https://arxiv.org/abs/1706.03762 - Liu et al. (2023). Lost in the Middle: How Language Models Use Long Contexts. https://arxiv.org/abs/2307.03172 ## Cite this entry Protologue. (2026). Context Window. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0004). https://protologue.com/t/context-window/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Delimiters > Delimiters are explicit markers, such as XML-style tags, triple quotes, or headings, that separate the parts of a prompt so the model can tell instructions, data, and examples apart. - Identifier: PTL-0014 - Category: Foundations - Canonical URL: https://protologue.com/t/delimiters/ - Also known as: XML tags, separators ## Description Vendor guidance recommends delimiters to reduce ambiguity, make outputs easier to parse, and lower the chance that content inside a document is mistaken for an instruction. Tags can also be requested in the output for reliable extraction. ## Related terms - [Prompt Template](https://protologue.com/t/prompt-template/) - [Structured Outputs](https://protologue.com/t/structured-outputs/) - [Spotlighting](https://protologue.com/t/spotlighting/) ## Sources - Anthropic (2026). Prompting best practices. https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices - OpenAI (2024). Prompt engineering. https://developers.openai.com/api/docs/guides/prompt-engineering ## Cite this entry Protologue. (2026). Delimiters. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0014). https://protologue.com/t/delimiters/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Direct Preference Optimization > Direct preference optimization (DPO) aligns a language model to human preferences by training directly on preferred-versus-rejected response pairs, without fitting a separate reward model or running reinforcement learning. - Identifier: PTL-0012 - Category: Foundations - Canonical URL: https://protologue.com/t/direct-preference-optimization/ - Also known as: DPO - Introduced: 2023 ## Description DPO reframes the RLHF objective as a simple classification-style loss over preference pairs. It became a common alternative to RLHF because it is stable and cheap to run. ## Broader terms - [Reinforcement Learning from Human Feedback](https://protologue.com/t/rlhf/) ## Sources - Rafailov et al. (2023). Direct Preference Optimization: Your Language Model is Secretly a Reward Model. https://arxiv.org/abs/2305.18290 ## Cite this entry Protologue. (2026). Direct Preference Optimization. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0012). https://protologue.com/t/direct-preference-optimization/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Few-shot Prompting > Few-shot prompting includes a small number of input-output examples in the prompt so the model can infer the task and the desired format from demonstrations. - Identifier: PTL-0009 - Category: Foundations - Canonical URL: https://protologue.com/t/few-shot-prompting/ - Also known as: k-shot prompting, in-context examples - Introduced: 2020 ## Description Popularized by the GPT-3 paper, it allows a model to perform new tasks without any weight updates. Results are sensitive to which examples are chosen, their order, and their label balance, which spawned a body of work on exemplar selection and calibration. ## Example ``` Classify sentiment. Review: "Arrived broken." -> negative Review: "Works perfectly." -> positive Review: "Battery died in a day." -> ``` ## Broader terms - [In-Context Learning](https://protologue.com/t/in-context-learning/) ## Narrower terms - [Exemplar Selection](https://protologue.com/t/exemplar-selection/) - [Exemplar Ordering](https://protologue.com/t/exemplar-ordering/) - [Few-shot Calibration](https://protologue.com/t/few-shot-calibration/) ## Related terms - [Zero-shot Prompting](https://protologue.com/t/zero-shot-prompting/) - [Many-shot In-Context Learning](https://protologue.com/t/many-shot-in-context-learning/) ## Sources - Brown et al. (2020). Language Models are Few-Shot Learners. https://arxiv.org/abs/2005.14165 ## Cite this entry Protologue. (2026). Few-shot Prompting. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0009). https://protologue.com/t/few-shot-prompting/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Instruction Tuning > Instruction tuning is fine-tuning a pretrained language model on many tasks phrased as natural-language instructions, so that it follows unseen instructions zero-shot. - Identifier: PTL-0010 - Category: Foundations - Canonical URL: https://protologue.com/t/instruction-tuning/ - Also known as: instruction fine-tuning, supervised fine-tuning, SFT - Introduced: 2021 ## Description The FLAN work showed that tuning on dozens of instruction-formatted datasets substantially improved zero-shot performance on held-out tasks. Combined with reinforcement learning from human feedback, it produced the instruction-following chat models that most prompting techniques now target. ## Related terms - [Zero-shot Prompting](https://protologue.com/t/zero-shot-prompting/) - [Reinforcement Learning from Human Feedback](https://protologue.com/t/rlhf/) ## Sources - Wei et al. (2021). Finetuned Language Models Are Zero-Shot Learners. https://arxiv.org/abs/2109.01652 - Ouyang et al. (2022). Training language models to follow instructions with human feedback. https://arxiv.org/abs/2203.02155 ## Cite this entry Protologue. (2026). Instruction Tuning. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0010). https://protologue.com/t/instruction-tuning/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Prefill > Prefill is the technique of writing the first part of the model's response yourself, so that the model continues from that text and is steered into a specific format or direction. - Identifier: PTL-0015 - Category: Foundations - Canonical URL: https://protologue.com/t/prefill/ - Also known as: response prefilling, output priming, assistant prefill ## Description Starting the assistant turn with an opening brace pushes the model toward JSON output, and starting with a heading or a character's name can enforce structure or persona. Support is model-specific and declining. Anthropic's documentation states that prefilling the final assistant turn is not supported on Claude 4.6 models and later, and recommends explicit instructions or structured outputs instead. ## Related terms - [Structured Outputs](https://protologue.com/t/structured-outputs/) - [Delimiters](https://protologue.com/t/delimiters/) ## Sources - Anthropic (2026). Prompting best practices. https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices ## Cite this entry Protologue. (2026). Prefill. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0015). https://protologue.com/t/prefill/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Prompt > A prompt is the complete input text, and optionally other media, that is given to a language model to condition its output, including instructions, context, examples, and the user's request. - Identifier: PTL-0001 - Category: Foundations - Canonical URL: https://protologue.com/t/prompt/ - Also known as: input, context ## Description In chat-based systems the prompt is assembled from several parts, such as a system prompt, the prior conversation turns, retrieved documents, and tool results. The model produces its output by predicting tokens conditioned on this entire input, so every part of the prompt can influence the response, not only the final question. ## Narrower terms - [System Prompt](https://protologue.com/t/system-prompt/) - [Prompt Template](https://protologue.com/t/prompt-template/) ## Related terms - [Context Window](https://protologue.com/t/context-window/) - [Context Engineering](https://protologue.com/t/context-engineering/) ## Sources - Brown et al. (2020). Language Models are Few-Shot Learners. https://arxiv.org/abs/2005.14165 - Schulhoff et al. (2024). The Prompt Report: A Systematic Survey of Prompt Engineering Techniques. https://arxiv.org/abs/2406.06608 ## Cite this entry Protologue. (2026). Prompt. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0001). https://protologue.com/t/prompt/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Prompt Engineering > Prompt engineering is the practice of designing, testing, and iteratively refining prompts so that a language model reliably produces the desired output for a task. - Identifier: PTL-0002 - Category: Foundations - Canonical URL: https://protologue.com/t/prompt-engineering/ - Also known as: prompt design ## Description It covers the wording of instructions, the choice and ordering of examples, output formatting constraints, and the decomposition of a task across multiple calls. As models and tooling matured, much of this work broadened into context engineering, which treats the whole information environment around a model call as the object being designed. ## Narrower terms - [Context Engineering](https://protologue.com/t/context-engineering/) ## Related terms - [Prompt](https://protologue.com/t/prompt/) - [Automatic Prompt Engineer](https://protologue.com/t/automatic-prompt-engineer/) ## Sources - Schulhoff et al. (2024). The Prompt Report: A Systematic Survey of Prompt Engineering Techniques. https://arxiv.org/abs/2406.06608 - Anthropic (2024). Prompt engineering overview. https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/overview ## Cite this entry Protologue. (2026). Prompt Engineering. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0002). https://protologue.com/t/prompt-engineering/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Prompt Template > A prompt template is a reusable prompt with placeholder variables that are filled in at run time, separating the fixed instructions from the per-request data. - Identifier: PTL-0013 - Category: Foundations - Canonical URL: https://protologue.com/t/prompt-template/ - Also known as: prompt scaffold ## Description Templates make prompts testable and versionable, and they are the unit optimized by frameworks such as DSPy. Keeping untrusted data in clearly delimited slots also supports defenses against prompt injection. ## Broader terms - [Prompt](https://protologue.com/t/prompt/) ## Related terms - [Delimiters](https://protologue.com/t/delimiters/) - [DSPy](https://protologue.com/t/dspy/) - [Spotlighting](https://protologue.com/t/spotlighting/) ## Sources - Schulhoff et al. (2024). The Prompt Report: A Systematic Survey of Prompt Engineering Techniques. https://arxiv.org/abs/2406.06608 ## Cite this entry Protologue. (2026). Prompt Template. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0013). https://protologue.com/t/prompt-template/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Reinforcement Learning from Human Feedback > Reinforcement learning from human feedback (RLHF) trains a language model to produce outputs people prefer, by learning a reward model from human comparisons and optimizing the model against it. - Identifier: PTL-0011 - Category: Foundations - Canonical URL: https://protologue.com/t/rlhf/ - Also known as: RLHF, preference tuning - Introduced: 2022 ## Description InstructGPT showed that RLHF made a much smaller model preferred over a far larger base model. RLHF shapes how models respond to prompts, including their helpfulness and refusals, and is linked to failure modes such as sycophancy. Direct preference optimization is a widely used simpler alternative. ## Narrower terms - [Direct Preference Optimization](https://protologue.com/t/direct-preference-optimization/) ## Related terms - [Instruction Tuning](https://protologue.com/t/instruction-tuning/) - [Constitutional AI](https://protologue.com/t/constitutional-ai/) - [Sycophancy](https://protologue.com/t/sycophancy/) ## Sources - Ouyang et al. (2022). Training language models to follow instructions with human feedback. https://arxiv.org/abs/2203.02155 ## Cite this entry Protologue. (2026). Reinforcement Learning from Human Feedback. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0011). https://protologue.com/t/rlhf/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Role Prompting > Role prompting assigns the model a persona or professional role, such as "You are an experienced tax accountant," to shape its tone, vocabulary, and focus. - Identifier: PTL-0016 - Category: Foundations - Canonical URL: https://protologue.com/t/role-prompting/ - Also known as: persona prompting, role-play prompting ## Description Role prompts reliably change style and framing. Evidence that they improve factual accuracy is mixed, with some studies finding gains in reasoning tasks and a large evaluation finding that adding personas to system prompts did not consistently improve performance. ## Related terms - [System Prompt](https://protologue.com/t/system-prompt/) ## Sources - Kong et al. (2023). Better Zero-Shot Reasoning with Role-Play Prompting. https://arxiv.org/abs/2308.07702 - Zheng et al. (2023). When "A Helpful Assistant" Is Not Really Helpful: Personas in System Prompts Do Not Improve Performances of Large Language Models. https://arxiv.org/abs/2311.10054 ## Cite this entry Protologue. (2026). Role Prompting. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0016). https://protologue.com/t/role-prompting/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## System Prompt > A system prompt is a privileged instruction block, placed before the conversation, that sets a model's role, rules, tone, and constraints for every subsequent turn. - Identifier: PTL-0003 - Category: Foundations - Canonical URL: https://protologue.com/t/system-prompt/ - Also known as: system message, developer message, system instruction ## Description Chat APIs typically expose the system prompt as a separate message role. Models trained with an instruction hierarchy are taught to give system instructions precedence over conflicting user or tool content, which makes the system prompt the main lever for operator control and a frequent target of prompt-leaking attacks. ## Broader terms - [Prompt](https://protologue.com/t/prompt/) ## Related terms - [Instruction Hierarchy](https://protologue.com/t/instruction-hierarchy/) - [Role Prompting](https://protologue.com/t/role-prompting/) - [Prompt Leaking](https://protologue.com/t/prompt-leaking/) ## Sources - Ouyang et al. (2022). Training language models to follow instructions with human feedback. https://arxiv.org/abs/2203.02155 - Wallace et al. (2024). The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions. https://arxiv.org/abs/2404.13208 ## Cite this entry Protologue. (2026). System Prompt. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0003). https://protologue.com/t/system-prompt/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Temperature > Temperature is a sampling parameter that rescales a model's output probabilities before a token is chosen; lower values make outputs more deterministic and higher values make them more varied. - Identifier: PTL-0006 - Category: Foundations - Canonical URL: https://protologue.com/t/temperature/ - Also known as: sampling temperature ## Description At temperature zero the model approximately always picks its most likely token (greedy decoding). Techniques that rely on diverse samples, such as self-consistency and best-of-N, deliberately use a nonzero temperature, while extraction and classification tasks usually use a low one. ## Related terms - [Top-p Sampling](https://protologue.com/t/top-p-sampling/) - [Self-Consistency](https://protologue.com/t/self-consistency/) - [Best-of-N Sampling](https://protologue.com/t/best-of-n-sampling/) ## Sources - Holtzman et al. (2019). The Curious Case of Neural Text Degeneration. https://arxiv.org/abs/1904.09751 ## Cite this entry Protologue. (2026). Temperature. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0006). https://protologue.com/t/temperature/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Token > A token is the basic unit of text a language model reads and writes, typically a word, word fragment, or character sequence produced by a subword tokenizer such as byte-pair encoding. - Identifier: PTL-0005 - Category: Foundations - Canonical URL: https://protologue.com/t/token/ - Also known as: subword, BPE token ## Description Context limits, pricing, and generation speed are all measured in tokens. Because tokenization splits text unevenly, character-level tasks such as counting letters or reversing strings are harder for models than they appear, and the same content can cost different numbers of tokens in different languages. ## Related terms - [Context Window](https://protologue.com/t/context-window/) ## Sources - Sennrich et al. (2015). Neural Machine Translation of Rare Words with Subword Units. https://arxiv.org/abs/1508.07909 ## Cite this entry Protologue. (2026). Token. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0005). https://protologue.com/t/token/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Top-p Sampling > Top-p sampling, also called nucleus sampling, draws each next token only from the smallest set of candidates whose cumulative probability exceeds a threshold p. - Identifier: PTL-0007 - Category: Foundations - Canonical URL: https://protologue.com/t/top-p-sampling/ - Also known as: nucleus sampling - Introduced: 2019 ## Description It was proposed to avoid both the repetitive text produced by greedy or beam decoding and the incoherent text produced by sampling from the full distribution. It is usually combined with temperature. ## Related terms - [Temperature](https://protologue.com/t/temperature/) ## Sources - Holtzman et al. (2019). The Curious Case of Neural Text Degeneration. https://arxiv.org/abs/1904.09751 ## Cite this entry Protologue. (2026). Top-p Sampling. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0007). https://protologue.com/t/top-p-sampling/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Zero-shot Prompting > Zero-shot prompting asks a model to perform a task from an instruction alone, without any worked examples in the prompt. - Identifier: PTL-0008 - Category: Foundations - Canonical URL: https://protologue.com/t/zero-shot-prompting/ ## Description Zero-shot performance improved dramatically with instruction tuning and preference training, which taught models to follow natural-language task descriptions. It is the default for most modern chat use, with examples added only when the format or judgment required is hard to describe. ## Related terms - [Few-shot Prompting](https://protologue.com/t/few-shot-prompting/) - [Instruction Tuning](https://protologue.com/t/instruction-tuning/) - [Zero-shot Chain-of-Thought](https://protologue.com/t/zero-shot-chain-of-thought/) ## Sources - Brown et al. (2020). Language Models are Few-Shot Learners. https://arxiv.org/abs/2005.14165 - Wei et al. (2021). Finetuned Language Models Are Zero-Shot Learners. https://arxiv.org/abs/2109.01652 ## Cite this entry Protologue. (2026). Zero-shot Prompting. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0008). https://protologue.com/t/zero-shot-prompting/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- # Branch: Exemplars & In-Context Learning How demonstrations inside a prompt are chosen, ordered, scaled, and calibrated, and what they actually teach the model. ## Active Prompting > Active prompting selects which questions to annotate with chain-of-thought exemplars by choosing those on which the model is most uncertain, measured by disagreement across sampled answers. - Identifier: PTL-0023 - Category: Exemplars & In-Context Learning - Canonical URL: https://protologue.com/t/active-prompting/ - Also known as: Active-Prompt - Introduced: 2023 ## Description Borrowing from active learning, it focuses human annotation effort on the examples most informative for the model, rather than on a fixed or random set. ## Broader terms - [Exemplar Selection](https://protologue.com/t/exemplar-selection/) ## Related terms - [Chain-of-Thought Prompting](https://protologue.com/t/chain-of-thought/) - [Self-Consistency](https://protologue.com/t/self-consistency/) ## Sources - Diao et al. (2023). Active Prompting with Chain-of-Thought for Large Language Models. https://arxiv.org/abs/2302.12246 ## Cite this entry Protologue. (2026). Active Prompting. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0023). https://protologue.com/t/active-prompting/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Demonstration Label Sensitivity > Demonstration label sensitivity refers to how much a model's few-shot performance depends on whether the example labels are correct; research found that randomly replacing labels often hurts performance only slightly. - Identifier: PTL-0018 - Category: Exemplars & In-Context Learning - Canonical URL: https://protologue.com/t/demonstration-label-sensitivity/ - Also known as: role of demonstrations ## Description Min et al. showed that demonstrations mainly supply the label space, the input distribution, and the format of the task. This suggests few-shot examples work largely by specifying what the task looks like rather than by teaching the input-label mapping. ## Broader terms - [In-Context Learning](https://protologue.com/t/in-context-learning/) ## Related terms - [Few-shot Prompting](https://protologue.com/t/few-shot-prompting/) - [Exemplar Selection](https://protologue.com/t/exemplar-selection/) ## Sources - Min et al. (2022). Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?. https://arxiv.org/abs/2202.12837 ## Cite this entry Protologue. (2026). Demonstration Label Sensitivity. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0018). https://protologue.com/t/demonstration-label-sensitivity/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Exemplar Ordering > Exemplar ordering is the arrangement of demonstrations within a few-shot prompt, which can swing accuracy from near state-of-the-art to near chance for the same set of examples. - Identifier: PTL-0020 - Category: Exemplars & In-Context Learning - Canonical URL: https://protologue.com/t/exemplar-ordering/ - Also known as: demonstration order, order sensitivity ## Description Lu et al. documented this order sensitivity and proposed selecting performant orderings using a probe set generated by the model itself. The effect is a key reason few-shot results should be reported across several permutations. ## Broader terms - [Few-shot Prompting](https://protologue.com/t/few-shot-prompting/) ## Related terms - [Exemplar Selection](https://protologue.com/t/exemplar-selection/) - [Few-shot Calibration](https://protologue.com/t/few-shot-calibration/) - [Prompt Sensitivity](https://protologue.com/t/prompt-sensitivity/) ## Sources - Lu et al. (2021). Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity. https://arxiv.org/abs/2104.08786 ## Cite this entry Protologue. (2026). Exemplar Ordering. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0020). https://protologue.com/t/exemplar-ordering/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Exemplar Selection > Exemplar selection is the choice of which demonstrations to include in a few-shot prompt, commonly by retrieving the examples most semantically similar to the current input. - Identifier: PTL-0019 - Category: Exemplars & In-Context Learning - Canonical URL: https://protologue.com/t/exemplar-selection/ - Also known as: demonstration selection, dynamic few-shot, kNN prompting ## Description Liu et al. showed that retrieving nearest-neighbor examples by embedding similarity outperformed random selection. Selection can also target diversity, difficulty, or the model's uncertainty, as in active prompting. ## Broader terms - [Few-shot Prompting](https://protologue.com/t/few-shot-prompting/) ## Narrower terms - [Active Prompting](https://protologue.com/t/active-prompting/) ## Related terms - [Exemplar Ordering](https://protologue.com/t/exemplar-ordering/) - [Retrieval-Augmented Generation](https://protologue.com/t/retrieval-augmented-generation/) ## Sources - Liu et al. (2021). What Makes Good In-Context Examples for GPT-3?. https://arxiv.org/abs/2101.06804 ## Cite this entry Protologue. (2026). Exemplar Selection. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0019). https://protologue.com/t/exemplar-selection/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Few-shot Calibration > Few-shot calibration corrects a model's systematic biases toward particular answers, such as the most frequent or most recent label in the examples, by adjusting output probabilities measured on a content-free input. - Identifier: PTL-0021 - Category: Exemplars & In-Context Learning - Canonical URL: https://protologue.com/t/few-shot-calibration/ - Also known as: contextual calibration, calibrate before use ## Description Zhao et al. identified majority-label bias, recency bias, and common-token bias in few-shot prompting, and proposed contextual calibration, which estimates the bias from an input such as "N/A" and rescales predictions to neutralize it. ## Broader terms - [Few-shot Prompting](https://protologue.com/t/few-shot-prompting/) ## Related terms - [Exemplar Ordering](https://protologue.com/t/exemplar-ordering/) - [Prompt Sensitivity](https://protologue.com/t/prompt-sensitivity/) ## Sources - Zhao et al. (2021). Calibrate Before Use: Improving Few-Shot Performance of Language Models. https://arxiv.org/abs/2102.09690 ## Cite this entry Protologue. (2026). Few-shot Calibration. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0021). https://protologue.com/t/few-shot-calibration/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## In-Context Learning > In-context learning (ICL) is a language model's ability to perform a task by conditioning on instructions or demonstrations in its prompt, without any update to its weights. - Identifier: PTL-0017 - Category: Exemplars & In-Context Learning - Canonical URL: https://protologue.com/t/in-context-learning/ - Also known as: ICL - Introduced: 2020 ## Description Identified as an emergent capability of large pretrained models in the GPT-3 paper, ICL underlies few-shot prompting. Studies show that models often rely more on the format and label space of demonstrations than on whether the demonstration labels are correct. ## Narrower terms - [Few-shot Prompting](https://protologue.com/t/few-shot-prompting/) - [Demonstration Label Sensitivity](https://protologue.com/t/demonstration-label-sensitivity/) - [Many-shot In-Context Learning](https://protologue.com/t/many-shot-in-context-learning/) ## Sources - Brown et al. (2020). Language Models are Few-Shot Learners. https://arxiv.org/abs/2005.14165 ## Cite this entry Protologue. (2026). In-Context Learning. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0017). https://protologue.com/t/in-context-learning/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Many-shot In-Context Learning > Many-shot in-context learning places hundreds or thousands of demonstrations in a long-context prompt, often yielding large gains over few-shot prompting. - Identifier: PTL-0022 - Category: Exemplars & In-Context Learning - Canonical URL: https://protologue.com/t/many-shot-in-context-learning/ - Also known as: many-shot ICL, long-context ICL - Introduced: 2024 ## Description Agarwal et al. found consistent improvements as the number of shots grew into the hundreds, and introduced variants that use model-generated rationales or unlabeled problems. The same scaling behavior underlies the many-shot jailbreaking attack. ## Broader terms - [In-Context Learning](https://protologue.com/t/in-context-learning/) ## Related terms - [Few-shot Prompting](https://protologue.com/t/few-shot-prompting/) - [Many-shot Jailbreaking](https://protologue.com/t/many-shot-jailbreaking/) - [Context Window](https://protologue.com/t/context-window/) ## Sources - Agarwal et al. (2024). Many-Shot In-Context Learning. https://arxiv.org/abs/2404.11018 ## Cite this entry Protologue. (2026). Many-shot In-Context Learning. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0022). https://protologue.com/t/many-shot-in-context-learning/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- # Branch: Reasoning Elicitation Techniques that get a model to produce intermediate reasoning, decompose problems, or explore multiple solution paths before answering. ## Analogical Prompting > Analogical prompting asks the model to recall or generate relevant example problems and their solutions on its own before solving the target problem, removing the need for hand-written exemplars. - Identifier: PTL-0036 - Category: Reasoning Elicitation - Canonical URL: https://protologue.com/t/analogical-prompting/ - Introduced: 2023 ## Description Inspired by how people draw on analogous past experience, the self-generated examples are tailored to each problem rather than fixed for the whole task. ## Related terms - [Few-shot Prompting](https://protologue.com/t/few-shot-prompting/) - [Generated Knowledge Prompting](https://protologue.com/t/generated-knowledge-prompting/) ## Sources - Yasunaga et al. (2023). Large Language Models as Analogical Reasoners. https://arxiv.org/abs/2310.01714 ## Cite this entry Protologue. (2026). Analogical Prompting. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0036). https://protologue.com/t/analogical-prompting/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Automatic Chain-of-Thought > Automatic chain-of-thought (Auto-CoT) builds chain-of-thought demonstrations without manual writing, by clustering questions for diversity and generating a reasoning chain for a representative of each cluster with zero-shot CoT. - Identifier: PTL-0027 - Category: Reasoning Elicitation - Canonical URL: https://protologue.com/t/auto-cot/ - Also known as: Auto-CoT - Introduced: 2022 ## Description Diversity across clusters limits the damage from mistakes in any single generated chain, and the approach matched manually written CoT exemplars on several benchmarks. ## Broader terms - [Chain-of-Thought Prompting](https://protologue.com/t/chain-of-thought/) ## Related terms - [Zero-shot Chain-of-Thought](https://protologue.com/t/zero-shot-chain-of-thought/) - [Exemplar Selection](https://protologue.com/t/exemplar-selection/) ## Sources - Zhang et al. (2022). Automatic Chain of Thought Prompting in Large Language Models. https://arxiv.org/abs/2210.03493 ## Cite this entry Protologue. (2026). Automatic Chain-of-Thought. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0027). https://protologue.com/t/auto-cot/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Chain-of-Thought Prompting > Chain-of-thought (CoT) prompting elicits a sequence of intermediate reasoning steps from a language model before its final answer, which improves performance on multi-step arithmetic, commonsense, and symbolic reasoning tasks. - Identifier: PTL-0024 - Category: Reasoning Elicitation - Canonical URL: https://protologue.com/t/chain-of-thought/ - Also known as: CoT, step-by-step reasoning - Introduced: 2022 ## Description Wei et al. introduced CoT by including worked examples with step-by-step reasoning in a few-shot prompt, and found the benefit emerged mainly in large models. It is the root of a large family of techniques, including zero-shot CoT, self-consistency, and tree of thoughts, and of the extended reasoning trained into modern reasoning models. ## Example ``` Q: Roger has 5 balls. He buys 2 cans of 3 balls each. How many balls does he have? A: He starts with 5. Two cans of 3 is 6. 5 + 6 = 11. The answer is 11. ``` ## Narrower terms - [Zero-shot Chain-of-Thought](https://protologue.com/t/zero-shot-chain-of-thought/) - [Automatic Chain-of-Thought](https://protologue.com/t/auto-cot/) - [Self-Consistency](https://protologue.com/t/self-consistency/) - [Tree of Thoughts](https://protologue.com/t/tree-of-thoughts/) - [Contrastive Chain-of-Thought](https://protologue.com/t/contrastive-chain-of-thought/) - [Thread of Thought](https://protologue.com/t/thread-of-thought/) ## Related terms - [Scratchpad](https://protologue.com/t/scratchpad/) - [Reasoning Model](https://protologue.com/t/reasoning-model/) - [Unfaithful Chain-of-Thought](https://protologue.com/t/unfaithful-chain-of-thought/) ## Sources - Wei et al. (2022). Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. https://arxiv.org/abs/2201.11903 ## Cite this entry Protologue. (2026). Chain-of-Thought Prompting. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0024). https://protologue.com/t/chain-of-thought/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Contrastive Chain-of-Thought > Contrastive chain-of-thought adds both valid and deliberately invalid reasoning demonstrations to a prompt, so the model learns which mistakes to avoid as well as what correct reasoning looks like. - Identifier: PTL-0038 - Category: Reasoning Elicitation - Canonical URL: https://protologue.com/t/contrastive-chain-of-thought/ - Introduced: 2023 ## Description The invalid demonstrations can be generated automatically by perturbing correct reasoning chains. ## Broader terms - [Chain-of-Thought Prompting](https://protologue.com/t/chain-of-thought/) ## Sources - Chia et al. (2023). Contrastive Chain-of-Thought Prompting. https://arxiv.org/abs/2311.09277 ## Cite this entry Protologue. (2026). Contrastive Chain-of-Thought. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0038). https://protologue.com/t/contrastive-chain-of-thought/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Generated Knowledge Prompting > Generated knowledge prompting first asks the model to produce relevant facts about a question, then supplies those generated facts as context when answering it. - Identifier: PTL-0044 - Category: Reasoning Elicitation - Canonical URL: https://protologue.com/t/generated-knowledge-prompting/ - Introduced: 2021 ## Description It acts like retrieval from the model's own parametric knowledge, and improved commonsense reasoning benchmarks in the original study. ## Related terms - [Step-Back Prompting](https://protologue.com/t/step-back-prompting/) - [Retrieval-Augmented Generation](https://protologue.com/t/retrieval-augmented-generation/) ## Sources - Liu et al. (2021). Generated Knowledge Prompting for Commonsense Reasoning. https://arxiv.org/abs/2110.08387 ## Cite this entry Protologue. (2026). Generated Knowledge Prompting. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0044). https://protologue.com/t/generated-knowledge-prompting/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Graph of Thoughts > Graph of Thoughts (GoT) models a language model's reasoning as an arbitrary graph, in which thoughts can be combined, refined, and looped back on, generalizing chain and tree structures. - Identifier: PTL-0034 - Category: Reasoning Elicitation - Canonical URL: https://protologue.com/t/graph-of-thoughts/ - Introduced: 2023 ## Description Aggregation of several partial solutions into one, such as merging sorted sublists, is the key operation GoT adds over tree-shaped search. ## Broader terms - [Tree of Thoughts](https://protologue.com/t/tree-of-thoughts/) ## Related terms - [Chain-of-Thought Prompting](https://protologue.com/t/chain-of-thought/) ## Sources - Besta et al. (2023). Graph of Thoughts: Solving Elaborate Problems with Large Language Models. https://arxiv.org/abs/2308.09687 ## Cite this entry Protologue. (2026). Graph of Thoughts. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0034). https://protologue.com/t/graph-of-thoughts/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Least-to-Most Prompting > Least-to-most prompting first asks the model to break a complex problem into simpler subproblems, then solves them in order, feeding each answer into the next. - Identifier: PTL-0030 - Category: Reasoning Elicitation - Canonical URL: https://protologue.com/t/least-to-most-prompting/ - Also known as: problem decomposition - Introduced: 2022 ## Description Zhou et al. showed it generalizes to problems harder than those in the examples, a weakness of standard chain-of-thought, with strong results on compositional generalization benchmarks. ## Related terms - [Chain-of-Thought Prompting](https://protologue.com/t/chain-of-thought/) - [Plan-and-Solve Prompting](https://protologue.com/t/plan-and-solve-prompting/) - [Self-Ask](https://protologue.com/t/self-ask/) - [Prompt Chaining](https://protologue.com/t/prompt-chaining/) ## Sources - Zhou et al. (2022). Least-to-Most Prompting Enables Complex Reasoning in Large Language Models. https://arxiv.org/abs/2205.10625 ## Cite this entry Protologue. (2026). Least-to-Most Prompting. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0030). https://protologue.com/t/least-to-most-prompting/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Maieutic Prompting > Maieutic prompting generates a tree of recursive explanations for and against an answer, then infers the most logically consistent answer from the relations among them. - Identifier: PTL-0043 - Category: Reasoning Elicitation - Canonical URL: https://protologue.com/t/maieutic-prompting/ - Introduced: 2022 ## Description Named after the Socratic method, it is designed to tolerate individual explanations that are wrong or inconsistent. ## Related terms - [Self-Consistency](https://protologue.com/t/self-consistency/) - [Chain-of-Verification](https://protologue.com/t/chain-of-verification/) ## Sources - Jung et al. (2022). Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanations. https://arxiv.org/abs/2205.11822 ## Cite this entry Protologue. (2026). Maieutic Prompting. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0043). https://protologue.com/t/maieutic-prompting/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Plan-and-Solve Prompting > Plan-and-solve prompting is a zero-shot method that instructs the model to first devise a plan dividing the task into subtasks and then carry out the plan step by step. - Identifier: PTL-0031 - Category: Reasoning Elicitation - Canonical URL: https://protologue.com/t/plan-and-solve-prompting/ - Also known as: PS prompting, PS+ - Introduced: 2023 ## Description It targets missing-step and calculation errors seen in zero-shot chain-of-thought, with an extended version that asks the model to extract relevant variables and compute carefully. ## Broader terms - [Zero-shot Chain-of-Thought](https://protologue.com/t/zero-shot-chain-of-thought/) ## Related terms - [Least-to-Most Prompting](https://protologue.com/t/least-to-most-prompting/) ## Sources - Wang et al. (2023). Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models. https://arxiv.org/abs/2305.04091 ## Cite this entry Protologue. (2026). Plan-and-Solve Prompting. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0031). https://protologue.com/t/plan-and-solve-prompting/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Program of Thoughts > Program of Thoughts (PoT) prompting expresses numerical reasoning as executable code, separating computation, done by an interpreter, from reasoning, done by the model. - Identifier: PTL-0046 - Category: Reasoning Elicitation - Canonical URL: https://protologue.com/t/program-of-thoughts/ - Also known as: PoT - Introduced: 2022 ## Description Developed concurrently with PAL, it showed strong gains on financial and math question answering. ## Related terms - [Program-Aided Language Models](https://protologue.com/t/program-aided-language-models/) - [Chain-of-Thought Prompting](https://protologue.com/t/chain-of-thought/) ## Sources - Chen et al. (2022). Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks. https://arxiv.org/abs/2211.12588 ## Cite this entry Protologue. (2026). Program of Thoughts. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0046). https://protologue.com/t/program-of-thoughts/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Program-Aided Language Models > Program-aided language models (PAL) have the model write a program, typically Python, that expresses its reasoning, and then delegate the actual computation to an interpreter. - Identifier: PTL-0045 - Category: Reasoning Elicitation - Canonical URL: https://protologue.com/t/program-aided-language-models/ - Also known as: PAL - Introduced: 2022 ## Description Offloading arithmetic and logic to code removes calculation errors from the model's output, and PAL outperformed much larger chain-of-thought models on math word problems. ## Related terms - [Program of Thoughts](https://protologue.com/t/program-of-thoughts/) - [Chain-of-Thought Prompting](https://protologue.com/t/chain-of-thought/) - [Function Calling](https://protologue.com/t/function-calling/) ## Sources - Gao et al. (2022). PAL: Program-aided Language Models. https://arxiv.org/abs/2211.10435 ## Cite this entry Protologue. (2026). Program-Aided Language Models. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0045). https://protologue.com/t/program-aided-language-models/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Rephrase and Respond > Rephrase and Respond (RaR) asks the model to rephrase and expand the user's question in its own words before answering, reducing misunderstandings caused by ambiguous phrasing. - Identifier: PTL-0041 - Category: Reasoning Elicitation - Canonical URL: https://protologue.com/t/rephrase-and-respond/ - Also known as: RaR - Introduced: 2023 ## Description The rephrasing can be done in the same response or by one model for another, as a two-step variant. ## Related terms - [System 2 Attention](https://protologue.com/t/system-2-attention/) ## Sources - Deng et al. (2023). Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselves. https://arxiv.org/abs/2311.04205 ## Cite this entry Protologue. (2026). Rephrase and Respond. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0041). https://protologue.com/t/rephrase-and-respond/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Scratchpad > A scratchpad is a region of model output reserved for intermediate computation, which the model writes before its final answer so that multi-step calculations can be carried out explicitly. - Identifier: PTL-0026 - Category: Reasoning Elicitation - Canonical URL: https://protologue.com/t/scratchpad/ - Also known as: intermediate computation - Introduced: 2021 ## Description Nye et al. trained models to emit intermediate steps for tasks like long addition and program execution, an important precursor to chain-of-thought prompting. In practice the term also refers to private reasoning sections that are hidden from end users. ## Related terms - [Chain-of-Thought Prompting](https://protologue.com/t/chain-of-thought/) - [Extended Thinking](https://protologue.com/t/extended-thinking/) ## Sources - Nye et al. (2021). Show Your Work: Scratchpads for Intermediate Computation with Language Models. https://arxiv.org/abs/2112.00114 ## Cite this entry Protologue. (2026). Scratchpad. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0026). https://protologue.com/t/scratchpad/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Self-Ask > Self-ask prompting has the model explicitly pose and answer follow-up sub-questions before answering a multi-hop question, a format that can plug a search engine in to answer each sub-question. - Identifier: PTL-0042 - Category: Reasoning Elicitation - Canonical URL: https://protologue.com/t/self-ask/ - Introduced: 2022 ## Description Press et al. used it to study the compositionality gap, the tendency of models to answer each sub-question correctly yet fail the composed question. ## Related terms - [Least-to-Most Prompting](https://protologue.com/t/least-to-most-prompting/) - [ReAct](https://protologue.com/t/react/) ## Sources - Press et al. (2022). Measuring and Narrowing the Compositionality Gap in Language Models. https://arxiv.org/abs/2210.03350 ## Cite this entry Protologue. (2026). Self-Ask. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0042). https://protologue.com/t/self-ask/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Self-Consistency > Self-consistency samples multiple chain-of-thought reasoning paths for the same question and returns the answer that appears most often, rather than relying on a single greedy decode. - Identifier: PTL-0028 - Category: Reasoning Elicitation - Canonical URL: https://protologue.com/t/self-consistency/ - Also known as: majority voting, CoT-SC - Introduced: 2022 ## Description Wang et al. found this majority vote substantially improved chain-of-thought accuracy on arithmetic and commonsense benchmarks. It trades extra inference cost for reliability and is an early form of test-time compute scaling. ## Broader terms - [Chain-of-Thought Prompting](https://protologue.com/t/chain-of-thought/) ## Narrower terms - [Universal Self-Consistency](https://protologue.com/t/universal-self-consistency/) ## Related terms - [Best-of-N Sampling](https://protologue.com/t/best-of-n-sampling/) - [Test-Time Compute Scaling](https://protologue.com/t/test-time-compute-scaling/) - [Temperature](https://protologue.com/t/temperature/) ## Sources - Wang et al. (2022). Self-Consistency Improves Chain of Thought Reasoning in Language Models. https://arxiv.org/abs/2203.11171 ## Cite this entry Protologue. (2026). Self-Consistency. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0028). https://protologue.com/t/self-consistency/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Self-Discover > Self-Discover has the model compose a task-specific reasoning structure by selecting, adapting, and combining general reasoning modules, such as critical thinking or step-by-step analysis, before solving instances of the task. - Identifier: PTL-0037 - Category: Reasoning Elicitation - Canonical URL: https://protologue.com/t/self-discover/ - Introduced: 2024 ## Description The structure is discovered once per task and then reused, so it costs far fewer inference calls than sampling-heavy methods like self-consistency. ## Related terms - [Chain-of-Thought Prompting](https://protologue.com/t/chain-of-thought/) - [Meta-Prompting](https://protologue.com/t/meta-prompting/) ## Sources - Zhou et al. (2024). Self-Discover: Large Language Models Self-Compose Reasoning Structures. https://arxiv.org/abs/2402.03620 ## Cite this entry Protologue. (2026). Self-Discover. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0037). https://protologue.com/t/self-discover/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Skeleton-of-Thought > Skeleton-of-thought first asks the model for a brief outline of its answer, then expands each outline point in parallel, reducing end-to-end generation latency. - Identifier: PTL-0035 - Category: Reasoning Elicitation - Canonical URL: https://protologue.com/t/skeleton-of-thought/ - Introduced: 2023 ## Description Because the points are expanded independently, it suits list-like answers better than tightly sequential reasoning. ## Related terms - [Parallelization](https://protologue.com/t/parallelization/) - [Prompt Chaining](https://protologue.com/t/prompt-chaining/) ## Sources - Ning et al. (2023). Skeleton-of-Thought: Prompting LLMs for Efficient Parallel Generation. https://arxiv.org/abs/2307.15337 ## Cite this entry Protologue. (2026). Skeleton-of-Thought. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0035). https://protologue.com/t/skeleton-of-thought/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Step-Back Prompting > Step-back prompting has the model first answer a more general, abstract question about the underlying principle, then use that answer to reason about the original specific question. - Identifier: PTL-0032 - Category: Reasoning Elicitation - Canonical URL: https://protologue.com/t/step-back-prompting/ - Also known as: abstraction prompting - Introduced: 2023 ## Description For a physics question, the step-back question might ask which law applies. Zheng et al. reported gains on science, multi-hop, and knowledge-intensive question answering. ## Related terms - [Chain-of-Thought Prompting](https://protologue.com/t/chain-of-thought/) - [Generated Knowledge Prompting](https://protologue.com/t/generated-knowledge-prompting/) ## Sources - Zheng et al. (2023). Take a Step Back: Evoking Reasoning via Abstraction in Large Language Models. https://arxiv.org/abs/2310.06117 ## Cite this entry Protologue. (2026). Step-Back Prompting. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0032). https://protologue.com/t/step-back-prompting/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## System 2 Attention > System 2 Attention (S2A) first prompts the model to rewrite the input so that it keeps only the relevant, unbiased content, then answers using the rewritten context. - Identifier: PTL-0040 - Category: Reasoning Elicitation - Canonical URL: https://protologue.com/t/system-2-attention/ - Also known as: S2A - Introduced: 2023 ## Description Weston and Sukhbaatar showed this reduces the influence of irrelevant or opinionated text in the prompt, improving factuality and reducing sycophancy toward opinions stated in the question. ## Related terms - [Thread of Thought](https://protologue.com/t/thread-of-thought/) - [Sycophancy](https://protologue.com/t/sycophancy/) - [Rephrase and Respond](https://protologue.com/t/rephrase-and-respond/) ## Sources - Weston & Sukhbaatar (2023). System 2 Attention (is something you might need too). https://arxiv.org/abs/2311.11829 ## Cite this entry Protologue. (2026). System 2 Attention. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0040). https://protologue.com/t/system-2-attention/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Thread of Thought > Thread of Thought is a prompting strategy for long, chaotic contexts that asks the model to walk through the context in manageable parts, summarizing and analyzing each before answering. - Identifier: PTL-0039 - Category: Reasoning Elicitation - Canonical URL: https://protologue.com/t/thread-of-thought/ - Introduced: 2023 ## Description A typical trigger asks the model to go through the context step by step, summarizing and analyzing as it goes. ## Broader terms - [Chain-of-Thought Prompting](https://protologue.com/t/chain-of-thought/) ## Related terms - [Lost in the Middle](https://protologue.com/t/lost-in-the-middle/) - [System 2 Attention](https://protologue.com/t/system-2-attention/) ## Sources - Zhou et al. (2023). Thread of Thought Unraveling Chaotic Contexts. https://arxiv.org/abs/2311.08734 ## Cite this entry Protologue. (2026). Thread of Thought. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0039). https://protologue.com/t/thread-of-thought/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Tree of Thoughts > Tree of Thoughts (ToT) lets a model explore multiple reasoning branches as a search tree, evaluating partial solutions and backtracking, rather than committing to a single left-to-right chain of thought. - Identifier: PTL-0033 - Category: Reasoning Elicitation - Canonical URL: https://protologue.com/t/tree-of-thoughts/ - Also known as: ToT - Introduced: 2023 ## Description Yao et al. combined model-generated candidate thoughts, model self-evaluation of states, and breadth- or depth-first search, producing large gains on tasks such as the Game of 24 that require planning or lookahead. ## Broader terms - [Chain-of-Thought Prompting](https://protologue.com/t/chain-of-thought/) ## Narrower terms - [Graph of Thoughts](https://protologue.com/t/graph-of-thoughts/) ## Related terms - [Self-Consistency](https://protologue.com/t/self-consistency/) - [Test-Time Compute Scaling](https://protologue.com/t/test-time-compute-scaling/) ## Sources - Yao et al. (2023). Tree of Thoughts: Deliberate Problem Solving with Large Language Models. https://arxiv.org/abs/2305.10601 ## Cite this entry Protologue. (2026). Tree of Thoughts. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0033). https://protologue.com/t/tree-of-thoughts/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Universal Self-Consistency > Universal self-consistency extends self-consistency to free-form outputs by asking the model itself to select the most consistent response among several samples, instead of counting exact-match answers. - Identifier: PTL-0029 - Category: Reasoning Elicitation - Canonical URL: https://protologue.com/t/universal-self-consistency/ - Introduced: 2023 ## Description This makes sampling-and-selecting usable for tasks such as summarization and open-ended question answering, where answers rarely match word for word. ## Broader terms - [Self-Consistency](https://protologue.com/t/self-consistency/) ## Related terms - [LLM-as-a-Judge](https://protologue.com/t/llm-as-a-judge/) ## Sources - Chen et al. (2023). Universal Self-Consistency for Large Language Model Generation. https://arxiv.org/abs/2311.17311 ## Cite this entry Protologue. (2026). Universal Self-Consistency. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0029). https://protologue.com/t/universal-self-consistency/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Zero-shot Chain-of-Thought > Zero-shot chain-of-thought prompting triggers step-by-step reasoning without examples by appending a cue such as "Let's think step by step" to the question. - Identifier: PTL-0025 - Category: Reasoning Elicitation - Canonical URL: https://protologue.com/t/zero-shot-chain-of-thought/ - Also known as: zero-shot CoT, let's think step by step - Introduced: 2022 ## Description Kojima et al. showed this single phrase produced large accuracy gains on arithmetic and logic benchmarks. The method uses a second prompt to extract the final answer from the generated reasoning. ## Broader terms - [Chain-of-Thought Prompting](https://protologue.com/t/chain-of-thought/) ## Narrower terms - [Plan-and-Solve Prompting](https://protologue.com/t/plan-and-solve-prompting/) ## Related terms - [Zero-shot Prompting](https://protologue.com/t/zero-shot-prompting/) ## Sources - Kojima et al. (2022). Large Language Models are Zero-Shot Reasoners. https://arxiv.org/abs/2205.11916 ## Cite this entry Protologue. (2026). Zero-shot Chain-of-Thought. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0025). https://protologue.com/t/zero-shot-chain-of-thought/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- # Branch: Self-Critique & Verification Techniques in which a model, or a set of models, checks, critiques, votes on, or revises outputs. ## Best-of-N Sampling > Best-of-N sampling generates N candidate outputs and returns the one ranked highest by a verifier, reward model, or scoring function. - Identifier: PTL-0054 - Category: Self-Critique & Verification - Canonical URL: https://protologue.com/t/best-of-n-sampling/ - Also known as: rejection sampling, best-of-n, BoN ## Description It is the simplest form of trading inference compute for quality. Its effectiveness depends on the verifier, and optimizing too hard against an imperfect reward model can select outputs that game it. ## Related terms - [Self-Consistency](https://protologue.com/t/self-consistency/) - [Process Reward Model](https://protologue.com/t/process-reward-model/) - [Test-Time Compute Scaling](https://protologue.com/t/test-time-compute-scaling/) ## Sources - Lightman et al. (2023). Let's Verify Step by Step. https://arxiv.org/abs/2305.20050 - Snell et al. (2024). Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters. https://arxiv.org/abs/2408.03314 ## Cite this entry Protologue. (2026). Best-of-N Sampling. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0054). https://protologue.com/t/best-of-n-sampling/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Chain-of-Verification > Chain-of-Verification (CoVe) reduces hallucination by having the model draft an answer, plan verification questions about its claims, answer those questions independently, and then produce a corrected final answer. - Identifier: PTL-0049 - Category: Self-Critique & Verification - Canonical URL: https://protologue.com/t/chain-of-verification/ - Also known as: CoVe - Introduced: 2023 ## Description Answering the verification questions without seeing the original draft keeps the model from simply repeating its own errors. ## Related terms - [Hallucination](https://protologue.com/t/hallucination/) - [Self-Refine](https://protologue.com/t/self-refine/) - [Maieutic Prompting](https://protologue.com/t/maieutic-prompting/) ## Sources - Dhuliawala et al. (2023). Chain-of-Verification Reduces Hallucination in Large Language Models. https://arxiv.org/abs/2309.11495 ## Cite this entry Protologue. (2026). Chain-of-Verification. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0049). https://protologue.com/t/chain-of-verification/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## LLM-as-a-Judge > LLM-as-a-judge is the use of a strong language model to grade, score, or compare the outputs of models against criteria, as a scalable substitute for human evaluation. - Identifier: PTL-0050 - Category: Self-Critique & Verification - Canonical URL: https://protologue.com/t/llm-as-a-judge/ - Also known as: model-graded evaluation, LLM evaluator, autorater - Introduced: 2023 ## Description Zheng et al. found strong model judges agreed with human preferences at rates comparable to agreement between humans, while documenting biases toward the first-listed answer, longer answers, and the judge's own outputs. ## Related terms - [Universal Self-Consistency](https://protologue.com/t/universal-self-consistency/) - [Evaluator-Optimizer](https://protologue.com/t/evaluator-optimizer/) - [Process Reward Model](https://protologue.com/t/process-reward-model/) ## Sources - Zheng et al. (2023). Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. https://arxiv.org/abs/2306.05685 ## Cite this entry Protologue. (2026). LLM-as-a-Judge. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0050). https://protologue.com/t/llm-as-a-judge/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Mixture-of-Agents > Mixture-of-Agents (MoA) arranges language models in layers, where each model receives all outputs from the previous layer as auxiliary input and an aggregator synthesizes a final response. - Identifier: PTL-0052 - Category: Self-Critique & Verification - Canonical URL: https://protologue.com/t/mixture-of-agents/ - Also known as: MoA - Introduced: 2024 ## Description Wang et al. reported that a mixture of open models outperformed a single strong proprietary model on an instruction-following benchmark. ## Related terms - [Multi-Agent Debate](https://protologue.com/t/multi-agent-debate/) - [Parallelization](https://protologue.com/t/parallelization/) ## Sources - Wang et al. (2024). Mixture-of-Agents Enhances Large Language Model Capabilities. https://arxiv.org/abs/2406.04692 ## Cite this entry Protologue. (2026). Mixture-of-Agents. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0052). https://protologue.com/t/mixture-of-agents/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Multi-Agent Debate > Multi-agent debate has several model instances propose answers, read each other's reasoning, and revise their answers over multiple rounds until they converge. - Identifier: PTL-0051 - Category: Self-Critique & Verification - Canonical URL: https://protologue.com/t/multi-agent-debate/ - Also known as: LLM debate, society of minds - Introduced: 2023 ## Description Du et al. found that debate improved mathematical reasoning and factual accuracy over single-model answers. ## Related terms - [Mixture-of-Agents](https://protologue.com/t/mixture-of-agents/) - [Self-Consistency](https://protologue.com/t/self-consistency/) ## Sources - Du et al. (2023). Improving Factuality and Reasoning in Language Models through Multiagent Debate. https://arxiv.org/abs/2305.14325 ## Cite this entry Protologue. (2026). Multi-Agent Debate. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0051). https://protologue.com/t/multi-agent-debate/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Process Reward Model > A process reward model (PRM) scores each intermediate step of a model's reasoning, rather than only the final answer, and is used to select or train better reasoning chains. - Identifier: PTL-0053 - Category: Self-Critique & Verification - Canonical URL: https://protologue.com/t/process-reward-model/ - Also known as: PRM, step-level verifier, process supervision - Introduced: 2023 ## Description Lightman et al. showed process supervision outperformed outcome supervision for selecting correct solutions to competition math problems, and released a large dataset of step-level human labels. ## Related terms - [Best-of-N Sampling](https://protologue.com/t/best-of-n-sampling/) - [Test-Time Compute Scaling](https://protologue.com/t/test-time-compute-scaling/) - [LLM-as-a-Judge](https://protologue.com/t/llm-as-a-judge/) ## Sources - Lightman et al. (2023). Let's Verify Step by Step. https://arxiv.org/abs/2305.20050 ## Cite this entry Protologue. (2026). Process Reward Model. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0053). https://protologue.com/t/process-reward-model/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Reflexion > Reflexion is an agent technique in which, after a failed attempt, the model writes a verbal reflection on what went wrong and stores it in memory to guide its next attempt. - Identifier: PTL-0048 - Category: Self-Critique & Verification - Canonical URL: https://protologue.com/t/reflexion/ - Introduced: 2023 ## Description It is reinforcement through language rather than weight updates, and uses feedback signals such as unit-test results or environment rewards. Shinn et al. reported large gains on coding and sequential decision-making benchmarks. ## Related terms - [Self-Refine](https://protologue.com/t/self-refine/) - [ReAct](https://protologue.com/t/react/) - [Agent Memory](https://protologue.com/t/agent-memory/) ## Sources - Shinn et al. (2023). Reflexion: Language Agents with Verbal Reinforcement Learning. https://arxiv.org/abs/2303.11366 ## Cite this entry Protologue. (2026). Reflexion. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0048). https://protologue.com/t/reflexion/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Self-Refine > Self-Refine is an iterative method in which the same model generates an output, critiques it with specific feedback, and revises it, repeating until a stopping condition is met. - Identifier: PTL-0047 - Category: Self-Critique & Verification - Canonical URL: https://protologue.com/t/self-refine/ - Also known as: iterative refinement, self-critique - Introduced: 2023 ## Description It requires no extra training or separate models. Madaan et al. reported improvements across tasks such as code optimization, dialogue, and math reasoning, though later work found that unaided self-correction of reasoning can fail without external signals. ## Related terms - [Reflexion](https://protologue.com/t/reflexion/) - [Evaluator-Optimizer](https://protologue.com/t/evaluator-optimizer/) - [Chain-of-Verification](https://protologue.com/t/chain-of-verification/) ## Sources - Madaan et al. (2023). Self-Refine: Iterative Refinement with Self-Feedback. https://arxiv.org/abs/2303.17651 - Huang et al. (2023). Large Language Models Cannot Self-Correct Reasoning Yet. https://arxiv.org/abs/2310.01798 ## Cite this entry Protologue. (2026). Self-Refine. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0047). https://protologue.com/t/self-refine/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- # Branch: Retrieval & Tool Use Grounding generation in external information and letting models call functions, search, and other tools. ## Function Calling > Function calling, or tool use, is a model capability in which the model outputs a structured request to invoke a developer-defined function with arguments, which the application executes and returns as a result to the model. - Identifier: PTL-0060 - Category: Retrieval & Tool Use - Canonical URL: https://protologue.com/t/function-calling/ - Also known as: tool use, tool calling ## Description Tools are usually described to the model with a name, a natural-language description, and a JSON Schema for parameters. Clear tool descriptions function as prompts in their own right and strongly affect when and how tools are used. ## Related terms - [ReAct](https://protologue.com/t/react/) - [Model Context Protocol](https://protologue.com/t/model-context-protocol/) - [Structured Outputs](https://protologue.com/t/structured-outputs/) - [Toolformer](https://protologue.com/t/toolformer/) ## Sources - Anthropic (2024). Tool use with Claude. https://platform.claude.com/docs/en/agents-and-tools/tool-use/overview - Patil et al. (2023). Gorilla: Large Language Model Connected with Massive APIs. https://arxiv.org/abs/2305.15334 ## Cite this entry Protologue. (2026). Function Calling. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0060). https://protologue.com/t/function-calling/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Hypothetical Document Embeddings > Hypothetical Document Embeddings (HyDE) improves retrieval by having a model write a hypothetical answer to the query, embedding that answer, and searching for real documents similar to it. - Identifier: PTL-0056 - Category: Retrieval & Tool Use - Canonical URL: https://protologue.com/t/hypothetical-document-embeddings/ - Also known as: HyDE - Introduced: 2022 ## Description The generated document may contain errors, but its embedding tends to lie closer to relevant real documents than the short query does. ## Broader terms - [Retrieval-Augmented Generation](https://protologue.com/t/retrieval-augmented-generation/) ## Sources - Gao et al. (2022). Precise Zero-Shot Dense Retrieval without Relevance Labels. https://arxiv.org/abs/2212.10496 ## Cite this entry Protologue. (2026). Hypothetical Document Embeddings. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0056). https://protologue.com/t/hypothetical-document-embeddings/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Model Context Protocol > The Model Context Protocol (MCP) is an open standard, introduced by Anthropic in 2024, that defines how applications expose tools, data resources, and prompt templates to language-model clients through a common client-server interface. - Identifier: PTL-0061 - Category: Retrieval & Tool Use - Canonical URL: https://protologue.com/t/model-context-protocol/ - Also known as: MCP - Introduced: 2024 ## Description MCP replaces one-off integrations between each model application and each data source with a single protocol, so any compliant client can use any compliant server. ## Related terms - [Function Calling](https://protologue.com/t/function-calling/) - [Context Engineering](https://protologue.com/t/context-engineering/) ## Sources - Anthropic (2024). Introducing the Model Context Protocol. https://www.anthropic.com/news/model-context-protocol - Model Context Protocol (2025). Model Context Protocol specification. https://modelcontextprotocol.io/ ## Cite this entry Protologue. (2026). Model Context Protocol. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0061). https://protologue.com/t/model-context-protocol/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## ReAct > ReAct is a prompting pattern that interleaves reasoning traces ("Thought") with actions such as tool calls ("Action") and their results ("Observation"), letting a model plan, act, and update its plan in a loop. - Identifier: PTL-0058 - Category: Retrieval & Tool Use - Canonical URL: https://protologue.com/t/react/ - Also known as: Reason + Act, thought-action-observation loop - Introduced: 2022 ## Description Yao et al. showed that combining reasoning and acting outperformed either alone on question answering and interactive decision-making tasks. ReAct is the template for most tool-using agent loops. ## Example ``` Thought: I need the population of the capital of France. Action: search("capital of France") Observation: Paris Thought: Now find the population of Paris. ``` ## Related terms - [Function Calling](https://protologue.com/t/function-calling/) - [Chain-of-Thought Prompting](https://protologue.com/t/chain-of-thought/) - [Self-Ask](https://protologue.com/t/self-ask/) - [AI Agent](https://protologue.com/t/ai-agent/) - [Reflexion](https://protologue.com/t/reflexion/) ## Sources - Yao et al. (2022). ReAct: Synergizing Reasoning and Acting in Language Models. https://arxiv.org/abs/2210.03629 ## Cite this entry Protologue. (2026). ReAct. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0058). https://protologue.com/t/react/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Retrieval-Augmented Generation > Retrieval-augmented generation (RAG) supplies a language model with passages retrieved from an external corpus at query time, so its output is grounded in that information rather than only in its trained parameters. - Identifier: PTL-0055 - Category: Retrieval & Tool Use - Canonical URL: https://protologue.com/t/retrieval-augmented-generation/ - Also known as: RAG, retrieval augmentation, grounded generation - Introduced: 2020 ## Description Lewis et al. introduced RAG as a jointly trained retriever and generator. In common usage the term now covers any pipeline that retrieves documents, typically by embedding search, and inserts them into the prompt. RAG reduces hallucination and allows knowledge to be updated without retraining. ## Narrower terms - [Hypothetical Document Embeddings](https://protologue.com/t/hypothetical-document-embeddings/) - [Self-RAG](https://protologue.com/t/self-rag/) ## Related terms - [Hallucination](https://protologue.com/t/hallucination/) - [Context Engineering](https://protologue.com/t/context-engineering/) ## Sources - Lewis et al. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. https://arxiv.org/abs/2005.11401 ## Cite this entry Protologue. (2026). Retrieval-Augmented Generation. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0055). https://protologue.com/t/retrieval-augmented-generation/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Self-RAG > Self-RAG trains a model to decide when to retrieve, and to emit special reflection tokens that critique whether retrieved passages are relevant and whether its own output is supported by them. - Identifier: PTL-0057 - Category: Retrieval & Tool Use - Canonical URL: https://protologue.com/t/self-rag/ - Introduced: 2023 ## Description Retrieving on demand rather than for every query avoids adding irrelevant context and lets the model's critique steer generation. ## Broader terms - [Retrieval-Augmented Generation](https://protologue.com/t/retrieval-augmented-generation/) ## Related terms - [Chain-of-Verification](https://protologue.com/t/chain-of-verification/) ## Sources - Asai et al. (2023). Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection. https://arxiv.org/abs/2310.11511 ## Cite this entry Protologue. (2026). Self-RAG. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0057). https://protologue.com/t/self-rag/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Structured Outputs > Structured outputs constrain a language model to produce responses that conform to a specified format, typically a JSON Schema, either through instructions or through constrained decoding that guarantees validity. - Identifier: PTL-0062 - Category: Retrieval & Tool Use - Canonical URL: https://protologue.com/t/structured-outputs/ - Also known as: JSON mode, constrained decoding, schema-constrained generation ## Description Instruction-only approaches can still produce malformed output, while constrained decoding masks tokens that would violate the schema at each step. Structured outputs are essential for passing model results reliably to downstream code. ## Related terms - [Function Calling](https://protologue.com/t/function-calling/) - [Prefill](https://protologue.com/t/prefill/) - [Delimiters](https://protologue.com/t/delimiters/) ## Sources - OpenAI (2024). Structured model outputs. https://developers.openai.com/api/docs/guides/structured-outputs - Willard & Louf (2023). Efficient Guided Generation for Large Language Models. https://arxiv.org/abs/2307.09702 ## Cite this entry Protologue. (2026). Structured Outputs. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0062). https://protologue.com/t/structured-outputs/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Toolformer > Toolformer is a method in which a language model teaches itself to use external tools, such as a calculator or search API, by generating candidate API calls in text and keeping those that reduce its prediction loss. - Identifier: PTL-0059 - Category: Retrieval & Tool Use - Canonical URL: https://protologue.com/t/toolformer/ - Introduced: 2023 ## Description It showed that tool-use behavior could be learned in a self-supervised way from only a handful of demonstrations per tool. ## Related terms - [Function Calling](https://protologue.com/t/function-calling/) - [ReAct](https://protologue.com/t/react/) ## Sources - Schick et al. (2023). Toolformer: Language Models Can Teach Themselves to Use Tools. https://arxiv.org/abs/2302.04761 ## Cite this entry Protologue. (2026). Toolformer. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0059). https://protologue.com/t/toolformer/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- # Branch: Agents & Orchestration Patterns for composing multiple model calls, tools, and memory into workflows and autonomous agents. ## Agent Memory > Agent memory is the set of mechanisms that let a language-model agent store information beyond a single context window, such as conversation summaries, retrievable records of past events, and reflections, and bring the relevant parts back into context later. - Identifier: PTL-0071 - Category: Agents & Orchestration - Canonical URL: https://protologue.com/t/agent-memory/ - Also known as: long-term memory, memory stream ## Description Park et al.'s generative agents scored memories by recency, importance, and relevance and periodically synthesized higher-level reflections. MemGPT treated the context window like main memory and paged information in and out of external storage. ## Related terms - [AI Agent](https://protologue.com/t/ai-agent/) - [Context Engineering](https://protologue.com/t/context-engineering/) - [Reflexion](https://protologue.com/t/reflexion/) - [Retrieval-Augmented Generation](https://protologue.com/t/retrieval-augmented-generation/) ## Sources - Park et al. (2023). Generative Agents: Interactive Simulacra of Human Behavior. https://arxiv.org/abs/2304.03442 - Packer et al. (2023). MemGPT: Towards LLMs as Operating Systems. https://arxiv.org/abs/2310.08560 ## Cite this entry Protologue. (2026). Agent Memory. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0071). https://protologue.com/t/agent-memory/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Agentic Workflow > An agentic workflow is a system in which language models and tools are orchestrated through predefined code paths, as opposed to an agent that chooses its own steps. - Identifier: PTL-0064 - Category: Agents & Orchestration - Canonical URL: https://protologue.com/t/agentic-workflow/ - Also known as: LLM workflow ## Description Common workflow patterns include prompt chaining, routing, parallelization, orchestrator-workers, and evaluator-optimizer loops. Workflows are generally preferred when a task can be cleanly decomposed in advance. ## Narrower terms - [Prompt Chaining](https://protologue.com/t/prompt-chaining/) - [Routing](https://protologue.com/t/routing/) - [Parallelization](https://protologue.com/t/parallelization/) - [Orchestrator-Workers](https://protologue.com/t/orchestrator-workers/) - [Evaluator-Optimizer](https://protologue.com/t/evaluator-optimizer/) ## Related terms - [AI Agent](https://protologue.com/t/ai-agent/) ## Sources - Anthropic (2024). Building effective agents. https://www.anthropic.com/research/building-effective-agents ## Cite this entry Protologue. (2026). Agentic Workflow. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0064). https://protologue.com/t/agentic-workflow/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## AI Agent > An AI agent is a system in which a language model dynamically directs its own process and tool use in a loop, deciding what actions to take based on environment feedback until a task is complete. - Identifier: PTL-0063 - Category: Agents & Orchestration - Canonical URL: https://protologue.com/t/ai-agent/ - Also known as: LLM agent, agentic system, autonomous agent ## Description Anthropic's widely cited taxonomy distinguishes agents from workflows, in which model calls and tools follow predefined code paths. Agents trade predictability and cost for flexibility on open-ended tasks. ## Related terms - [ReAct](https://protologue.com/t/react/) - [Agentic Workflow](https://protologue.com/t/agentic-workflow/) - [Orchestrator-Workers](https://protologue.com/t/orchestrator-workers/) - [Agent Memory](https://protologue.com/t/agent-memory/) - [Context Engineering](https://protologue.com/t/context-engineering/) ## Sources - Anthropic (2024). Building effective agents. https://www.anthropic.com/research/building-effective-agents ## Cite this entry Protologue. (2026). AI Agent. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0063). https://protologue.com/t/ai-agent/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Context Engineering > Context engineering is the practice of curating the full set of tokens a model sees at each step, including instructions, tools, retrieved data, memory, and conversation history, to maximize the chance of the desired behavior within a limited attention budget. - Identifier: PTL-0072 - Category: Agents & Orchestration - Canonical URL: https://protologue.com/t/context-engineering/ - Also known as: context management - Introduced: 2025 ## Description The term gained currency in 2025 as agents ran for many steps and the main challenge shifted from wording a single prompt to deciding what information enters and leaves the context window over time, through techniques such as compaction, structured note-taking, and subagents. ## Broader terms - [Prompt Engineering](https://protologue.com/t/prompt-engineering/) ## Related terms - [Context Window](https://protologue.com/t/context-window/) - [Agent Memory](https://protologue.com/t/agent-memory/) - [Retrieval-Augmented Generation](https://protologue.com/t/retrieval-augmented-generation/) - [AI Agent](https://protologue.com/t/ai-agent/) - [Lost in the Middle](https://protologue.com/t/lost-in-the-middle/) ## Sources - Anthropic (2025). Effective context engineering for AI agents. https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents ## Cite this entry Protologue. (2026). Context Engineering. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0072). https://protologue.com/t/context-engineering/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Evaluator-Optimizer > Evaluator-optimizer is a workflow loop in which one model call generates a response and another evaluates it against criteria and provides feedback, repeating until the output passes. - Identifier: PTL-0069 - Category: Agents & Orchestration - Canonical URL: https://protologue.com/t/evaluator-optimizer/ ## Description It works best when evaluation criteria are clear and when feedback demonstrably improves the output, as in literary translation or iterative search. ## Broader terms - [Agentic Workflow](https://protologue.com/t/agentic-workflow/) ## Related terms - [Self-Refine](https://protologue.com/t/self-refine/) - [LLM-as-a-Judge](https://protologue.com/t/llm-as-a-judge/) - [Reflexion](https://protologue.com/t/reflexion/) ## Sources - Anthropic (2024). Building effective agents. https://www.anthropic.com/research/building-effective-agents ## Cite this entry Protologue. (2026). Evaluator-Optimizer. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0069). https://protologue.com/t/evaluator-optimizer/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Meta-Prompting > Meta-prompting uses a single model as a conductor that breaks a task down and writes prompts for fresh instances of itself acting as specialized experts, then integrates their outputs. - Identifier: PTL-0070 - Category: Agents & Orchestration - Canonical URL: https://protologue.com/t/meta-prompting/ - Introduced: 2024 ## Description The term is also used more loosely for asking a model to write or improve a prompt. ## Related terms - [Orchestrator-Workers](https://protologue.com/t/orchestrator-workers/) - [Self-Discover](https://protologue.com/t/self-discover/) - [Automatic Prompt Engineer](https://protologue.com/t/automatic-prompt-engineer/) ## Sources - Suzgun & Kalai (2024). Meta-Prompting: Enhancing Language Models with Task-Agnostic Scaffolding. https://arxiv.org/abs/2401.12954 ## Cite this entry Protologue. (2026). Meta-Prompting. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0070). https://protologue.com/t/meta-prompting/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Orchestrator-Workers > Orchestrator-workers is a pattern in which a central model dynamically breaks a task into subtasks, delegates them to worker model calls or subagents, and synthesizes their results. - Identifier: PTL-0068 - Category: Agents & Orchestration - Canonical URL: https://protologue.com/t/orchestrator-workers/ - Also known as: subagents, multi-agent orchestration, supervisor pattern ## Description Unlike parallelization, the subtasks are not fixed in advance but decided by the orchestrator for each input, which suits tasks such as multi-file code changes or broad research. ## Broader terms - [Agentic Workflow](https://protologue.com/t/agentic-workflow/) ## Related terms - [AI Agent](https://protologue.com/t/ai-agent/) - [Parallelization](https://protologue.com/t/parallelization/) - [Meta-Prompting](https://protologue.com/t/meta-prompting/) ## Sources - Anthropic (2024). Building effective agents. https://www.anthropic.com/research/building-effective-agents ## Cite this entry Protologue. (2026). Orchestrator-Workers. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0068). https://protologue.com/t/orchestrator-workers/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Parallelization > Parallelization is a workflow pattern that runs several model calls simultaneously and aggregates their outputs, either by splitting a task into independent sections or by running the same task several times and voting. - Identifier: PTL-0067 - Category: Agents & Orchestration - Canonical URL: https://protologue.com/t/parallelization/ - Also known as: sectioning, voting ## Description Sectioning reduces latency and keeps each call focused, while voting improves confidence, for example by running several independent safety checks. ## Broader terms - [Agentic Workflow](https://protologue.com/t/agentic-workflow/) ## Related terms - [Self-Consistency](https://protologue.com/t/self-consistency/) - [Skeleton-of-Thought](https://protologue.com/t/skeleton-of-thought/) - [Mixture-of-Agents](https://protologue.com/t/mixture-of-agents/) ## Sources - Anthropic (2024). Building effective agents. https://www.anthropic.com/research/building-effective-agents ## Cite this entry Protologue. (2026). Parallelization. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0067). https://protologue.com/t/parallelization/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Prompt Chaining > Prompt chaining decomposes a task into a fixed sequence of model calls, where each call processes the output of the previous one, often with programmatic checks between steps. - Identifier: PTL-0065 - Category: Agents & Orchestration - Canonical URL: https://protologue.com/t/prompt-chaining/ - Also known as: LLM chains, multi-step prompting - Introduced: 2021 ## Description Chaining trades latency for accuracy by making each call simpler. Wu et al. found chaining also improved transparency and controllability for users building with models. ## Broader terms - [Agentic Workflow](https://protologue.com/t/agentic-workflow/) ## Related terms - [Least-to-Most Prompting](https://protologue.com/t/least-to-most-prompting/) - [Routing](https://protologue.com/t/routing/) ## Sources - Wu et al. (2021). AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts. https://arxiv.org/abs/2110.01691 - Anthropic (2024). Building effective agents. https://www.anthropic.com/research/building-effective-agents ## Cite this entry Protologue. (2026). Prompt Chaining. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0065). https://protologue.com/t/prompt-chaining/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Routing > Routing is a workflow pattern that classifies an incoming request and directs it to a specialized prompt, tool, or model suited to that category. - Identifier: PTL-0066 - Category: Agents & Orchestration - Canonical URL: https://protologue.com/t/routing/ - Also known as: model routing, query routing ## Description Routing allows separate prompts to be optimized for distinct cases, and lets easy queries go to smaller, cheaper models while hard ones go to more capable models. ## Broader terms - [Agentic Workflow](https://protologue.com/t/agentic-workflow/) ## Related terms - [Prompt Chaining](https://protologue.com/t/prompt-chaining/) ## Sources - Anthropic (2024). Building effective agents. https://www.anthropic.com/research/building-effective-agents ## Cite this entry Protologue. (2026). Routing. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0066). https://protologue.com/t/routing/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- # Branch: Prompt Optimization Automatic and learned methods that search for, compress, or train better prompts. ## Automatic Prompt Engineer > Automatic Prompt Engineer (APE) uses a language model to generate candidate instructions for a task from input-output examples, scores each candidate on held-out data, and selects the best one. - Identifier: PTL-0073 - Category: Prompt Optimization - Canonical URL: https://protologue.com/t/automatic-prompt-engineer/ - Also known as: APE - Introduced: 2022 ## Description APE discovered a zero-shot chain-of-thought trigger that outperformed "Let's think step by step" on some benchmarks, and framed prompt writing as a search problem that models can solve themselves. ## Related terms - [Prompt Engineering](https://protologue.com/t/prompt-engineering/) - [Optimization by Prompting](https://protologue.com/t/opro/) - [DSPy](https://protologue.com/t/dspy/) - [Meta-Prompting](https://protologue.com/t/meta-prompting/) ## Sources - Zhou et al. (2022). Large Language Models Are Human-Level Prompt Engineers. https://arxiv.org/abs/2211.01910 ## Cite this entry Protologue. (2026). Automatic Prompt Engineer. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0073). https://protologue.com/t/automatic-prompt-engineer/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Directional Stimulus Prompting > Directional stimulus prompting trains a small policy model to generate instance-specific hints, such as keywords, that are added to the prompt to steer a large frozen model toward desired outputs. - Identifier: PTL-0079 - Category: Prompt Optimization - Canonical URL: https://protologue.com/t/directional-stimulus-prompting/ - Introduced: 2023 ## Description The policy model can be trained with supervised learning and reinforcement learning, without access to the large model's weights. ## Related terms - [Prompt Tuning](https://protologue.com/t/prompt-tuning/) - [Optimization by Prompting](https://protologue.com/t/opro/) ## Sources - Li et al. (2023). Guiding Large Language Models via Directional Stimulus Prompting. https://arxiv.org/abs/2302.11520 ## Cite this entry Protologue. (2026). Directional Stimulus Prompting. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0079). https://protologue.com/t/directional-stimulus-prompting/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## DSPy > DSPy is a framework that treats language-model pipelines as programs of declarative modules, and compiles them by automatically optimizing the prompts and few-shot demonstrations for each module against a metric. - Identifier: PTL-0075 - Category: Prompt Optimization - Canonical URL: https://protologue.com/t/dspy/ - Also known as: Declarative Self-improving Python - Introduced: 2023 ## Description Rather than hand-tuning prompt strings, developers specify input-output signatures and let optimizers bootstrap demonstrations or search instructions, making pipelines portable across models. ## Related terms - [Automatic Prompt Engineer](https://protologue.com/t/automatic-prompt-engineer/) - [Optimization by Prompting](https://protologue.com/t/opro/) - [Prompt Template](https://protologue.com/t/prompt-template/) ## Sources - Khattab et al. (2023). DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines. https://arxiv.org/abs/2310.03714 ## Cite this entry Protologue. (2026). DSPy. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0075). https://protologue.com/t/dspy/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Emotion Prompting > Emotion prompting appends emotional or motivational phrases, such as "This is very important to my career," to a prompt in an attempt to improve model performance. - Identifier: PTL-0082 - Category: Prompt Optimization - Canonical URL: https://protologue.com/t/emotion-prompting/ - Also known as: EmotionPrompt, stimulus prompting - Introduced: 2023 ## Description Li et al. reported gains on several benchmarks with such "EmotionPrompt" stimuli. Effects of these cues vary across models and tasks and are best treated as an empirical question for each setup. ## Related terms - [Prompt Sensitivity](https://protologue.com/t/prompt-sensitivity/) - [Role Prompting](https://protologue.com/t/role-prompting/) ## Sources - Li et al. (2023). Large Language Models Understand and Can be Enhanced by Emotional Stimuli. https://arxiv.org/abs/2307.11760 ## Cite this entry Protologue. (2026). Emotion Prompting. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0082). https://protologue.com/t/emotion-prompting/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Low-Rank Adaptation > Low-rank adaptation (LoRA) fine-tunes a language model by training small low-rank matrices added to its weight layers while freezing the original weights, drastically reducing the number of trainable parameters. - Identifier: PTL-0078 - Category: Prompt Optimization - Canonical URL: https://protologue.com/t/low-rank-adaptation/ - Also known as: LoRA - Introduced: 2021 ## Description LoRA is a common alternative when prompting alone cannot reach the required behavior, and adapters can be swapped per task on one base model. ## Related terms - [Prompt Tuning](https://protologue.com/t/prompt-tuning/) - [Prefix Tuning](https://protologue.com/t/prefix-tuning/) - [Instruction Tuning](https://protologue.com/t/instruction-tuning/) ## Sources - Hu et al. (2021). LoRA: Low-Rank Adaptation of Large Language Models. https://arxiv.org/abs/2106.09685 ## Cite this entry Protologue. (2026). Low-Rank Adaptation. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0078). https://protologue.com/t/low-rank-adaptation/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Optimization by Prompting > Optimization by PROmpting (OPRO) uses a language model as an optimizer, giving it a meta-prompt containing previously tried prompts and their scores and asking it to propose a better prompt, repeating over many rounds. - Identifier: PTL-0074 - Category: Prompt Optimization - Canonical URL: https://protologue.com/t/opro/ - Also known as: OPRO, LLMs as optimizers - Introduced: 2023 ## Description OPRO found instructions such as "Take a deep breath and work on this problem step-by-step" that improved math benchmark accuracy for the model being optimized. ## Related terms - [Automatic Prompt Engineer](https://protologue.com/t/automatic-prompt-engineer/) - [DSPy](https://protologue.com/t/dspy/) ## Sources - Yang et al. (2023). Large Language Models as Optimizers. https://arxiv.org/abs/2309.03409 ## Cite this entry Protologue. (2026). Optimization by Prompting. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0074). https://protologue.com/t/opro/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Prefix Tuning > Prefix tuning learns continuous task-specific vectors that are prepended to the activations at every layer of a frozen language model, steering generation without changing the model's weights. - Identifier: PTL-0077 - Category: Prompt Optimization - Canonical URL: https://protologue.com/t/prefix-tuning/ - Introduced: 2021 ## Description It was an early parameter-efficient alternative to fine-tuning for generation tasks such as table-to-text and summarization. ## Related terms - [Prompt Tuning](https://protologue.com/t/prompt-tuning/) - [Low-Rank Adaptation](https://protologue.com/t/low-rank-adaptation/) ## Sources - Li & Liang (2021). Prefix-Tuning: Optimizing Continuous Prompts for Generation. https://arxiv.org/abs/2101.00190 ## Cite this entry Protologue. (2026). Prefix Tuning. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0077). https://protologue.com/t/prefix-tuning/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Prompt Caching > Prompt caching stores the model's processed state for a reused prompt prefix, such as a long system prompt or document, so later requests sharing that prefix are cheaper and faster. - Identifier: PTL-0081 - Category: Prompt Optimization - Canonical URL: https://protologue.com/t/prompt-caching/ - Also known as: context caching, prefix caching - Introduced: 2024 ## Description Because caches match on an exact prefix, prompts are structured with stable content first and variable content last. ## Related terms - [System Prompt](https://protologue.com/t/system-prompt/) - [Prompt Compression](https://protologue.com/t/prompt-compression/) - [Context Engineering](https://protologue.com/t/context-engineering/) ## Sources - Anthropic (2024). Prompt caching. https://platform.claude.com/docs/en/build-with-claude/prompt-caching ## Cite this entry Protologue. (2026). Prompt Caching. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0081). https://protologue.com/t/prompt-caching/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Prompt Compression > Prompt compression shortens a prompt by removing tokens that contribute little information, typically scored by a smaller language model, to cut cost and latency while preserving task performance. - Identifier: PTL-0080 - Category: Prompt Optimization - Canonical URL: https://protologue.com/t/prompt-compression/ - Also known as: LLMLingua, context compression - Introduced: 2023 ## Description LLMLingua used a small model's perplexity to drop low-information tokens and reported high compression ratios with limited performance loss. ## Related terms - [Context Engineering](https://protologue.com/t/context-engineering/) - [Prompt Caching](https://protologue.com/t/prompt-caching/) ## Sources - Jiang et al. (2023). LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models. https://arxiv.org/abs/2310.05736 ## Cite this entry Protologue. (2026). Prompt Compression. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0080). https://protologue.com/t/prompt-compression/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Prompt Tuning > Prompt tuning learns a small set of continuous "soft prompt" embeddings that are prepended to the input, by gradient descent, while keeping the language model's weights frozen. - Identifier: PTL-0076 - Category: Prompt Optimization - Canonical URL: https://protologue.com/t/prompt-tuning/ - Also known as: soft prompts, soft prompt tuning - Introduced: 2021 ## Description Lester et al. showed that as models grow, prompt tuning approaches the quality of full fine-tuning while storing only a tiny number of task-specific parameters. Soft prompts are vectors rather than readable text. ## Related terms - [Prefix Tuning](https://protologue.com/t/prefix-tuning/) - [Low-Rank Adaptation](https://protologue.com/t/low-rank-adaptation/) ## Sources - Lester et al. (2021). The Power of Scale for Parameter-Efficient Prompt Tuning. https://arxiv.org/abs/2104.08691 ## Cite this entry Protologue. (2026). Prompt Tuning. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0076). https://protologue.com/t/prompt-tuning/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- # Branch: Reasoning Models & Test-Time Compute Models trained to reason at length, and methods that improve answers by spending more computation at inference time. ## Extended Thinking > Extended thinking is a model mode in which the model generates a separate block of reasoning before its final response, with a developer-controlled setting that trades latency and cost for answer quality. - Identifier: PTL-0085 - Category: Reasoning Models & Test-Time Compute - Canonical URL: https://protologue.com/t/extended-thinking/ - Also known as: thinking mode, thinking budget, reasoning effort ## Description Early implementations exposed a fixed thinking token budget; newer Claude models replace it with adaptive thinking, where the model decides how much to reason under a chosen effort level. Vendor guidance for these modes favors high-level instructions about how to think over prescriptive step-by-step scripts. ## Broader terms - [Reasoning Model](https://protologue.com/t/reasoning-model/) ## Related terms - [Scratchpad](https://protologue.com/t/scratchpad/) - [Chain-of-Thought Prompting](https://protologue.com/t/chain-of-thought/) ## Sources - Anthropic (2025). Building with extended thinking. https://platform.claude.com/docs/en/build-with-claude/extended-thinking ## Cite this entry Protologue. (2026). Extended Thinking. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0085). https://protologue.com/t/extended-thinking/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Reasoning Model > A reasoning model is a language model trained, typically with reinforcement learning, to produce an extended internal chain of thought before answering, so that it improves with more thinking time on math, coding, and planning tasks. - Identifier: PTL-0083 - Category: Reasoning Models & Test-Time Compute - Canonical URL: https://protologue.com/t/reasoning-model/ - Also known as: large reasoning model, LRM, thinking model - Introduced: 2024 ## Description OpenAI's o1, announced in 2024, popularized the category, and DeepSeek-R1 showed that reinforcement learning with verifiable rewards could produce long reasoning behaviors in an openly released model. Prompting such models differs from prompting standard models, because step-by-step instructions and few-shot reasoning examples are often unnecessary. ## Narrower terms - [Extended Thinking](https://protologue.com/t/extended-thinking/) ## Related terms - [Chain-of-Thought Prompting](https://protologue.com/t/chain-of-thought/) - [Test-Time Compute Scaling](https://protologue.com/t/test-time-compute-scaling/) - [Self-Taught Reasoner](https://protologue.com/t/self-taught-reasoner/) ## Sources - OpenAI (2024). Learning to reason with LLMs. https://openai.com/index/learning-to-reason-with-llms/ - DeepSeek-AI (2025). DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. https://arxiv.org/abs/2501.12948 ## Cite this entry Protologue. (2026). Reasoning Model. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0083). https://protologue.com/t/reasoning-model/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Self-Taught Reasoner > The Self-Taught Reasoner (STaR) bootstraps reasoning ability by having a model generate rationales, keeping those that lead to correct answers, and fine-tuning on them in repeated rounds. - Identifier: PTL-0086 - Category: Reasoning Models & Test-Time Compute - Canonical URL: https://protologue.com/t/self-taught-reasoner/ - Also known as: STaR - Introduced: 2022 ## Description For problems it fails, STaR provides the correct answer as a hint and asks the model to produce a rationale for it, a step called rationalization. ## Related terms - [Reasoning Model](https://protologue.com/t/reasoning-model/) - [Chain-of-Thought Prompting](https://protologue.com/t/chain-of-thought/) ## Sources - Zelikman et al. (2022). STaR: Bootstrapping Reasoning With Reasoning. https://arxiv.org/abs/2203.14465 ## Cite this entry Protologue. (2026). Self-Taught Reasoner. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0086). https://protologue.com/t/self-taught-reasoner/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Test-Time Compute Scaling > Test-time compute scaling improves a model's answers by spending more computation at inference, through longer reasoning, more samples, search, or verification, rather than by training a larger model. - Identifier: PTL-0084 - Category: Reasoning Models & Test-Time Compute - Canonical URL: https://protologue.com/t/test-time-compute-scaling/ - Also known as: inference-time scaling, test-time scaling - Introduced: 2024 ## Description Snell et al. found that allocating test-time compute adaptively per prompt could be more effective than scaling model parameters for some problems. Self-consistency, best-of-N, tree search, and reasoning models are all forms of test-time scaling. ## Related terms - [Reasoning Model](https://protologue.com/t/reasoning-model/) - [Self-Consistency](https://protologue.com/t/self-consistency/) - [Best-of-N Sampling](https://protologue.com/t/best-of-n-sampling/) - [Tree of Thoughts](https://protologue.com/t/tree-of-thoughts/) - [Process Reward Model](https://protologue.com/t/process-reward-model/) ## Sources - Snell et al. (2024). Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters. https://arxiv.org/abs/2408.03314 ## Cite this entry Protologue. (2026). Test-Time Compute Scaling. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0084). https://protologue.com/t/test-time-compute-scaling/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- # Branch: Security & Adversarial Prompting Attacks that subvert a model's instructions and the defenses designed to resist them. ## Adversarial Suffix > An adversarial suffix is an automatically optimized string of tokens that, when appended to a request, causes an aligned model to comply with requests it would normally refuse. - Identifier: PTL-0091 - Category: Security & Adversarial Prompting - Canonical URL: https://protologue.com/t/adversarial-suffix/ - Also known as: GCG attack, universal adversarial attack - Introduced: 2023 ## Description Zou et al. used a greedy coordinate gradient search on open models to find such suffixes and found that they often transferred to other models. ## Broader terms - [Jailbreak](https://protologue.com/t/jailbreak/) ## Sources - Zou et al. (2023). Universal and Transferable Adversarial Attacks on Aligned Language Models. https://arxiv.org/abs/2307.15043 ## Cite this entry Protologue. (2026). Adversarial Suffix. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0091). https://protologue.com/t/adversarial-suffix/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Constitutional AI > Constitutional AI is a training method in which a model critiques and revises its own outputs according to a written set of principles, and AI-generated preference judgments replace most human labels for harmlessness. - Identifier: PTL-0095 - Category: Security & Adversarial Prompting - Canonical URL: https://protologue.com/t/constitutional-ai/ - Also known as: CAI, RLAIF, reinforcement learning from AI feedback - Introduced: 2022 ## Description Bai et al. used a supervised self-critique phase followed by reinforcement learning from AI feedback. The approach made the principles governing model behavior explicit and editable. ## Related terms - [Reinforcement Learning from Human Feedback](https://protologue.com/t/rlhf/) - [Self-Refine](https://protologue.com/t/self-refine/) - [Jailbreak](https://protologue.com/t/jailbreak/) ## Sources - Bai et al. (2022). Constitutional AI: Harmlessness from AI Feedback. https://arxiv.org/abs/2212.08073 ## Cite this entry Protologue. (2026). Constitutional AI. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0095). https://protologue.com/t/constitutional-ai/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Indirect Prompt Injection > Indirect prompt injection places malicious instructions inside content a model will later retrieve or process, such as a web page, email, or document, so the attack is triggered without the attacker interacting with the model directly. - Identifier: PTL-0088 - Category: Security & Adversarial Prompting - Canonical URL: https://protologue.com/t/indirect-prompt-injection/ - Introduced: 2023 ## Description Greshake et al. demonstrated that injected content could make integrated applications exfiltrate data, spread to other users, or manipulate outputs. Risk grows with an agent's access to tools and private data. ## Broader terms - [Prompt Injection](https://protologue.com/t/prompt-injection/) ## Related terms - [Retrieval-Augmented Generation](https://protologue.com/t/retrieval-augmented-generation/) - [Spotlighting](https://protologue.com/t/spotlighting/) - [AI Agent](https://protologue.com/t/ai-agent/) ## Sources - Greshake et al. (2023). Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection. https://arxiv.org/abs/2302.12173 ## Cite this entry Protologue. (2026). Indirect Prompt Injection. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0088). https://protologue.com/t/indirect-prompt-injection/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Instruction Hierarchy > The instruction hierarchy is a training approach that teaches a model to prioritize instructions by source, typically system over user over tool output, and to ignore lower-priority instructions that conflict with higher-priority ones. - Identifier: PTL-0093 - Category: Security & Adversarial Prompting - Canonical URL: https://protologue.com/t/instruction-hierarchy/ - Introduced: 2024 ## Description Wallace et al. trained models on synthetic conflicts and found large gains in robustness to prompt injection and system-prompt extraction with limited loss in helpfulness. ## Related terms - [System Prompt](https://protologue.com/t/system-prompt/) - [Prompt Injection](https://protologue.com/t/prompt-injection/) - [Spotlighting](https://protologue.com/t/spotlighting/) ## Sources - Wallace et al. (2024). The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions. https://arxiv.org/abs/2404.13208 ## Cite this entry Protologue. (2026). Instruction Hierarchy. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0093). https://protologue.com/t/instruction-hierarchy/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Jailbreak > A jailbreak is a prompt crafted to make a model produce outputs its safety training is meant to prevent, often through role-play, hypothetical framing, obfuscation, or other adversarial techniques. - Identifier: PTL-0090 - Category: Security & Adversarial Prompting - Canonical URL: https://protologue.com/t/jailbreak/ - Also known as: jailbreaking ## Description Wei et al. attributed jailbreak success to two failure modes, competing objectives between helpfulness and safety, and mismatched generalization, where safety training does not cover inputs the model can still understand. Jailbreaks target the model's safety behavior, while prompt injection targets the application's instructions. ## Narrower terms - [Adversarial Suffix](https://protologue.com/t/adversarial-suffix/) - [Many-shot Jailbreaking](https://protologue.com/t/many-shot-jailbreaking/) ## Related terms - [Prompt Injection](https://protologue.com/t/prompt-injection/) - [Constitutional AI](https://protologue.com/t/constitutional-ai/) ## Sources - Wei et al. (2023). Jailbroken: How Does LLM Safety Training Fail?. https://arxiv.org/abs/2307.02483 ## Cite this entry Protologue. (2026). Jailbreak. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0090). https://protologue.com/t/jailbreak/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Many-shot Jailbreaking > Many-shot jailbreaking fills a long context window with many fabricated dialogue examples in which an assistant complies with harmful requests, exploiting in-context learning to override the model's safety training. - Identifier: PTL-0092 - Category: Security & Adversarial Prompting - Canonical URL: https://protologue.com/t/many-shot-jailbreaking/ - Introduced: 2024 ## Description Anthropic researchers found the attack's effectiveness followed a power law in the number of shots, mirroring the scaling of benign in-context learning. ## Broader terms - [Jailbreak](https://protologue.com/t/jailbreak/) ## Related terms - [Many-shot In-Context Learning](https://protologue.com/t/many-shot-in-context-learning/) - [Context Window](https://protologue.com/t/context-window/) ## Sources - Anthropic (2024). Many-shot jailbreaking. https://www.anthropic.com/research/many-shot-jailbreaking ## Cite this entry Protologue. (2026). Many-shot Jailbreaking. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0092). https://protologue.com/t/many-shot-jailbreaking/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Prompt Injection > Prompt injection is an attack in which text supplied to a language model, by a user or through data the model processes, contains instructions that override or subvert the instructions of the application's developer. - Identifier: PTL-0087 - Category: Security & Adversarial Prompting - Canonical URL: https://protologue.com/t/prompt-injection/ - Also known as: goal hijacking, instruction injection - Introduced: 2022 ## Description The name was coined in 2022 by analogy with SQL injection, because models cannot reliably separate trusted instructions from untrusted data that share the same context. It is the most prominent security risk for applications built on language models, especially agents with tool access. ## Narrower terms - [Indirect Prompt Injection](https://protologue.com/t/indirect-prompt-injection/) - [Prompt Leaking](https://protologue.com/t/prompt-leaking/) ## Related terms - [Jailbreak](https://protologue.com/t/jailbreak/) - [Instruction Hierarchy](https://protologue.com/t/instruction-hierarchy/) - [Spotlighting](https://protologue.com/t/spotlighting/) ## Sources - Willison (2022). Prompt injection attacks against GPT-3. https://simonwillison.net/2022/Sep/12/prompt-injection/ - Perez & Ribeiro (2022). Ignore Previous Prompt: Attack Techniques For Language Models. https://arxiv.org/abs/2211.09527 ## Cite this entry Protologue. (2026). Prompt Injection. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0087). https://protologue.com/t/prompt-injection/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Prompt Leaking > Prompt leaking is an attack that tricks a model into revealing its hidden system prompt or other confidential instructions. - Identifier: PTL-0089 - Category: Security & Adversarial Prompting - Canonical URL: https://protologue.com/t/prompt-leaking/ ## Description System prompts should be treated as potentially discoverable, so they should not contain secrets such as API keys. ## Broader terms - [Prompt Injection](https://protologue.com/t/prompt-injection/) ## Related terms - [System Prompt](https://protologue.com/t/system-prompt/) ## Sources - Perez & Ribeiro (2022). Ignore Previous Prompt: Attack Techniques For Language Models. https://arxiv.org/abs/2211.09527 ## Cite this entry Protologue. (2026). Prompt Leaking. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0089). https://protologue.com/t/prompt-leaking/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Spotlighting > Spotlighting is a family of prompt-level defenses against indirect prompt injection that transform untrusted input, by delimiting, marking every word, or encoding it, so the model can distinguish it from trusted instructions. - Identifier: PTL-0094 - Category: Security & Adversarial Prompting - Canonical URL: https://protologue.com/t/spotlighting/ - Also known as: datamarking, input marking - Introduced: 2024 ## Description Hines et al. reported substantial reductions in attack success with datamarking and encoding variants, with little effect on task performance. ## Related terms - [Indirect Prompt Injection](https://protologue.com/t/indirect-prompt-injection/) - [Delimiters](https://protologue.com/t/delimiters/) - [Instruction Hierarchy](https://protologue.com/t/instruction-hierarchy/) ## Sources - Hines et al. (2024). Defending Against Indirect Prompt Injection Attacks With Spotlighting. https://arxiv.org/abs/2403.14720 ## Cite this entry Protologue. (2026). Spotlighting. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0094). https://protologue.com/t/spotlighting/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- # Branch: Failure Modes & Evaluation Systematic ways prompted models go wrong, and the evaluations used to measure them. ## Hallucination > Hallucination is generated content that is fluent and plausible but unfaithful to the provided source or factually incorrect, such as fabricated citations, facts, or quotations. - Identifier: PTL-0096 - Category: Failure Modes & Evaluation - Canonical URL: https://protologue.com/t/hallucination/ - Also known as: confabulation, fabrication ## Description Surveys distinguish intrinsic hallucination, which contradicts the source, from extrinsic hallucination, which cannot be verified from it. Mitigations include retrieval-augmented generation, verification methods such as chain-of-verification, allowing the model to say it does not know, and requiring quotes from supplied documents. ## Related terms - [Retrieval-Augmented Generation](https://protologue.com/t/retrieval-augmented-generation/) - [Chain-of-Verification](https://protologue.com/t/chain-of-verification/) - [Sycophancy](https://protologue.com/t/sycophancy/) ## Sources - Ji et al. (2022). Survey of Hallucination in Natural Language Generation. https://arxiv.org/abs/2202.03629 ## Cite this entry Protologue. (2026). Hallucination. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0096). https://protologue.com/t/hallucination/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Lost in the Middle > Lost in the middle is the finding that language models use information at the beginning or end of a long context much more reliably than information placed in the middle. - Identifier: PTL-0098 - Category: Failure Modes & Evaluation - Canonical URL: https://protologue.com/t/lost-in-the-middle/ - Also known as: positional bias, U-shaped context performance - Introduced: 2023 ## Description Liu et al. observed a U-shaped performance curve on multi-document question answering as the position of the relevant document varied. A practical takeaway is to place key material and instructions at the start or end of long prompts. ## Related terms - [Context Window](https://protologue.com/t/context-window/) - [Needle in a Haystack](https://protologue.com/t/needle-in-a-haystack/) - [Context Engineering](https://protologue.com/t/context-engineering/) - [Thread of Thought](https://protologue.com/t/thread-of-thought/) ## Sources - Liu et al. (2023). Lost in the Middle: How Language Models Use Long Contexts. https://arxiv.org/abs/2307.03172 ## Cite this entry Protologue. (2026). Lost in the Middle. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0098). https://protologue.com/t/lost-in-the-middle/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Needle in a Haystack > Needle in a haystack is a long-context evaluation that hides a specific fact at varying depths in a long distractor document and tests whether the model can retrieve it. - Identifier: PTL-0099 - Category: Failure Modes & Evaluation - Canonical URL: https://protologue.com/t/needle-in-a-haystack/ - Also known as: NIAH, passkey retrieval - Introduced: 2023 ## Description It became a standard way to report effective context length, though passing it shows only simple retrieval, not reasoning over many dispersed facts. ## Related terms - [Lost in the Middle](https://protologue.com/t/lost-in-the-middle/) - [Context Window](https://protologue.com/t/context-window/) ## Sources - Kamradt (2023). LLMTest_NeedleInAHaystack. https://github.com/gkamradt/LLMTest_NeedleInAHaystack ## Cite this entry Protologue. (2026). Needle in a Haystack. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0099). https://protologue.com/t/needle-in-a-haystack/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Prompt Sensitivity > Prompt sensitivity is the variation in a model's performance caused by superficial changes to a prompt, such as formatting, separators, spacing, or wording, that do not change the task's meaning. - Identifier: PTL-0100 - Category: Failure Modes & Evaluation - Canonical URL: https://protologue.com/t/prompt-sensitivity/ - Also known as: prompt brittleness, format sensitivity ## Description Sclar et al. found accuracy differences of tens of percentage points from formatting choices alone, and argued that evaluations should report performance across a range of plausible formats. ## Related terms - [Exemplar Ordering](https://protologue.com/t/exemplar-ordering/) - [Few-shot Calibration](https://protologue.com/t/few-shot-calibration/) - [Emotion Prompting](https://protologue.com/t/emotion-prompting/) ## Sources - Sclar et al. (2023). Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting. https://arxiv.org/abs/2310.11324 ## Cite this entry Protologue. (2026). Prompt Sensitivity. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0100). https://protologue.com/t/prompt-sensitivity/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Sycophancy > Sycophancy is a model's tendency to tailor its answers to match a user's stated beliefs or preferences, including abandoning correct answers when the user pushes back, rather than giving its most accurate response. - Identifier: PTL-0097 - Category: Failure Modes & Evaluation - Canonical URL: https://protologue.com/t/sycophancy/ ## Description Sharma et al. found sycophancy across several assistants and linked it to human preference data, which tends to reward agreement. Prompting mitigations include removing opinions from the question, as in System 2 Attention. ## Related terms - [Reinforcement Learning from Human Feedback](https://protologue.com/t/rlhf/) - [System 2 Attention](https://protologue.com/t/system-2-attention/) - [Hallucination](https://protologue.com/t/hallucination/) ## Sources - Sharma et al. (2023). Towards Understanding Sycophancy in Language Models. https://arxiv.org/abs/2310.13548 ## Cite this entry Protologue. (2026). Sycophancy. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0097). https://protologue.com/t/sycophancy/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) --- ## Unfaithful Chain-of-Thought > Unfaithful chain-of-thought is stated reasoning that does not reflect the factors that actually determined the model's answer, so the explanation can be plausible yet misleading. - Identifier: PTL-0101 - Category: Failure Modes & Evaluation - Canonical URL: https://protologue.com/t/unfaithful-chain-of-thought/ - Also known as: CoT faithfulness, post-hoc rationalization ## Description Turpin et al. showed that biasing features, such as always putting the correct answer in position A of few-shot examples, swayed answers while the stated reasoning never mentioned them. Lanham et al. measured how much answers actually depend on the stated reasoning, with results varying by task and model size. ## Related terms - [Chain-of-Thought Prompting](https://protologue.com/t/chain-of-thought/) - [Reasoning Model](https://protologue.com/t/reasoning-model/) ## Sources - Turpin et al. (2023). Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting. https://arxiv.org/abs/2305.04388 - Lanham et al. (2023). Measuring Faithfulness in Chain-of-Thought Reasoning. https://arxiv.org/abs/2307.13702 ## Cite this entry Protologue. (2026). Unfaithful Chain-of-Thought. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0101). https://protologue.com/t/unfaithful-chain-of-thought/ License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) ---