Have you ever found that a particular prompt works well, only for it to suddenly stop working?
Have you ever been stuck in a cycle of constantly patching your system prompt, trying to improve results, only to find that nothing works?
System prompt learning may be exactly what you need.
System prompt learning (SPL) is an emerging area of interest in the AI community and was widely popularized by Andrej Karpathy on X in May.
System prompt learning addresses the limitations of inflexible, brittle AI systems that rely on static system prompts or unwieldy fine-tuning setups. It offers another way to support continuous learning in AI systems.
Before diving in, let’s briefly review some prompting fundamentals.
When developing an agent or custom model, we must first design two key components:
A system prompt
A user prompt
System prompts establish the ground rules for how a model should behave. When written for custom AI solutions, they often start with something like this:
“You are an intelligent assistant. Your role is to perform <insert task here>.
You must not do (A), (B), or (C).”
By contrast, user prompts typically contain a user’s query and other relevant information, such as their time zone and preferences. A user prompt might look like this:
I’m in the capital city of Portugal. Can you suggest some things I can do tonight?
System prompt leaks have become common after new model releases from major AI labs, as users jailbreak chatbots to reveal their underlying instructions. A popular GitHub repository now compiles many of these system prompts in one place. They reveal the “secret sauce” AI labs have developed over time to encourage appropriate model behavior. For example, the recently leaked GPT-5 system prompt (exposed within ChatGPT) contains approximately 6,000 words, illustrating how much knowledge and guidance must be encoded to shape system behavior.
These comprehensive system prompts typically cover several key areas, such as:
Search instructions
Tool definitions
User preferences
Citation instructions
Quick patches for known problems
In practice, developers of custom AI systems iteratively patch system prompts manually as they test and refine their applications, primarily using evals to guide this improvement process.
Other ways to guide a model’s behavior include:
Prompt engineering, including retrieval-augmented generation (RAG), which controls the content provided to a model
Fine-tuning (directly changing the underlying weights of the model)
What if there were another way to influence model behavior? Imagine a system that dynamically learns and refines its own system prompt using previously generated thoughts, plans, and strategies. It could draw on both user feedback and LLM-as-a-judge evaluations to assess its outputs.
Consider a persistent business challenge you want to automate using an agentic system. Effective solutions require reasoning capabilities beyond basic workflow automation. In such cases, incorporating a plan-generation component into your AI system becomes essential. This lets the system work with multiple agents in different ways, depending on the task. Individual steps may include instructions for accessing other agents to complete subtasks or using tools.
Note: An agent tool is any external function, API, or resource an AI agent can call to go beyond text and take real actions.
You might choose to “seed” the model’s system prompt with a plan that follows the logical steps a human would take, although LLMs typically need more specific guidance on tool usage, output formatting, and related requirements. Sometimes, the optimal strategy may be unclear, or you may be addressing a problem that has not been reassessed because it was previously considered solved. This is where system prompt learning (SPL) comes in.
SPL iteratively improves a system prompt by incorporating previously generated strategies. As new problems arise, the system gradually accumulates knowledge and becomes more robust. Think of it as building a handbook for solving problems in your domain.
SPL gradually incorporates insights from user feedback into the system prompt. As your system matures, you may discover recurring issues that can be distilled into more generalized, higher-level principles.
Let’s look more closely at how the process works, step by step:
Begin with the user query, asking the system to perform a specific task.
If your system addresses only one problem, you might take a “greedy” approach by selecting the highest-scoring strategies from previous runs. Alternatively, you can encourage exploration by sampling from a distribution that favors highly rated strategies while occasionally including lower-rated ones. This is particularly useful when you are just beginning to collect strategies.
For systems designed to handle diverse problem sets, consider adding a classification layer or using embeddings and cosine similarity (the same techniques typically used in RAG) to identify relevant approaches. This helps you select strategies suited to the specific problem—for example, strategies tailored to coding tasks.
Note: Embeddings used with cosine similarity let us measure how closely two pieces of information relate, making it easier to match documents, queries, or ideas, even when their exact wording differs.
Example starting point for a simplified strategy store for tackling coding problems.
Note: The “seed strategies” shown here are illustrative. In real coding scenarios, we would refine them further. Niche business problems would require collecting additional insights over time.
Generation_id (reverse order) | Subject | Score | Strategy_text | Explanation |
|---|---|---|---|---|
4 | coding | 1 | Understand the problem, constraints, and edge cases. Design an algorithm with the right data structures. Validate the plan on examples and invariants. Implement clean, readable code. Refine with refactoring, optimization, and final formatting. Tool use: When using a tool, briefly explain why it was needed. | Incorporates and blends the strongest elements from the three strategies below. |
3 | coding | 1 | Understand the problem, constraints, and edge cases. Design an algorithm with the right data structures. Validate the plan on examples and invariants. Implement clean, readable code. Refine with refactoring, optimization, and final formatting. | A more well-rounded strategy, but it includes no guidance on tool use. |
2 | coding | -1 | Understand the problem, constraints, and edge cases. Design an algorithm with the right data. Implement clean, readable code. Tool use: when accessing tools produce a short summary as to why you used that tool. | A better strategy that mentions tool use but could still improve. |
1 | coding | -1 | Skim the problem. Solve the problem. Create minimal tests. Submit whatever runs. | Mentions tests but is a weak overall strategy. |
3. After sampling N, incorporate them into the system prompt. This grounds plan generation in prior expert feedback rather than leaving the model to create plans with minimal guidance. Encourage the model to “think outside the box” and add steps when necessary instead of simply copying sample strategies verbatim.
4. With your dynamically created system prompt, generate a new strategy to address the user’s request. This process should produce additional tasks that improve the final output. The goal is creativity: blend the strongest elements of prior strategies, consolidate overlapping steps, and add helpful new steps where needed.
Note: Remember that temperature is a parameter that can be adjusted to produce more varied, less deterministic outputs, which is useful when creativity is desired. With a nonzero temperature, each generated plan may differ.
5. After receiving the model’s output, evaluate it using either a human rater or LLM judge based on specific criteria that define a good solution to your problem. For the Portugal activities example mentioned earlier, evaluation criteria might include:
Conciseness (an answer limited to one sentence)
Relevance of the suggested activity
Location accuracy
6. Based on this evaluation, use another model to refine the strategy. An optional feedback loop can incorporate human input and support collaborative improvements. Store the refined strategy in your database with appropriate metadata to track versions and changes.
So why go through all of this trouble? You could review outputs manually and adjust the system prompt accordingly. However, powerful reasoning models can refine strategies using both output context and human feedback. Although humans can easily spot flaws in simple approaches, identifying them becomes challenging and tedious in complex systems that address broader problem sets.
LLMs often require detailed instructions and additional steps to gather contextual knowledge that humans bring naturally to a problem. The number of required tasks can grow quickly as a system expands to address broader problem sets. For example, humans tackling coding problems may understand the surrounding codebase intuitively, while an LLM may first need to “read” multiple files.
When it helps: Imagine you run a customer support team, and an AI agent handles ticket triage. Over time, SPL might uncover a categorization method your team had not considered, reducing escalation rates.
When it doesn’t: If compliance requirements or regulations already define your workflows, such as financial reporting, SPL may offer little value because creativity becomes a liability rather than an asset.
When it helps: In research-intensive roles (such as market intelligence or product strategy), you can collaborate with the AI by refining its plans, enriching its output, and incorporating these improvements for future use. Each interaction enhances the system’s effectiveness.
When it doesn’t: If your team mostly uses AI for straightforward workflows where human input is minimal (e.g., invoice processing), collaboration overhead may outweigh the benefit.
When it helps: Suppose you expand into a new region and the AI must suddenly handle local tax inquiries. SPL allows you to quickly encode new rules and heuristics as they emerge, preventing repeated errors.
When it doesn’t: If your environment is static, such as when converting meeting transcripts into standardized summaries, constant adaptation offers minimal benefits.
In theory, all of this sounds promising, but implementing SPL presents real challenges. We discuss some of the main ones below:
In the early phases of strategy generation, progress often stalls: new outputs fail to build on previous ones, and momentum slows. Two main issues usually drive this:
Solution: Encode all available business knowledge up front so the system has depth to draw on.
Solution: Design a nuanced rubric that scores multiple aspects of an answer, such as accuracy, clarity, and relevance, and adjust your sampling to reflect these signals.
If your system generates hundreds of strategies but receives little feedback to distinguish good from bad, sampling quickly becomes unwieldy. The answer is pruning.
When refining your strategy store, consider:
Lifespan: Retire strategies once they exceed a defined time frame or number of generations.
Score: Use your evaluation rubric to filter out consistently low-performing strategies. Combining this with lifespan ensures you’re only keeping approaches that prove their value over time.
LLM judging: Periodically assess strategies to identify those that no longer add unique insights, since their useful elements are likely already absorbed into newer versions.
Solution: Treat your strategy database as a living system: regularly prune it so only relevant, high-value knowledge remains.
System prompt learning is still in its infancy, but its potential is enormous. Businesses that rely solely on static prompts or endless fine-tuning will encounter familiar limitations: fragile systems, mounting costs, and wasted effort. SPL offers a way out of that cycle by building systems that improve over time and encode higher-level principles rather than isolated patches.
SPL is still emerging, but its trajectory is clear: systems that can learn from themselves will outpace those that can’t. Now is the time to experiment, start small, capture lessons, and build the foundation for AI systems that improve with every interaction.