White PaperJuly 24, 202617 min readBy Reza Kashi, Founder of Sophyx

The Prompt Is Not the Target: How AI Search Rewrites, Expands, and Fans Out Your Queries

Why optimizing for the visible prompt is the wrong goal in ChatGPT, Gemini, Claude, and Perplexity — and how to optimize for the rewritten queries that actually decide whether AI mentions your brand.

1. Executive summary

The strongest claim this paper defends is simple: in ChatGPT search and deep research, the literal prompt is only the entry point, not the final optimization target. OpenAI’s own documentation says ChatGPT may decide whether to search based on the request, consider the full chat context, rewrite a prompt into one or more targeted queries, issue additional, more specific queries, and even use memory or general location to improve those rewritten queries.^[1][2] In deep research, the system goes further: it creates a plan, researches across sources, and synthesizes a documented, citation-backed report.^[3][4]

That means the thing that actually drives retrieval is a broader internal task formulation — user intent, conversation history, memory, location, instruction hierarchy, and tool choice — not the string a human typed. “Optimize for the prompt you think the user typed” is therefore the wrong framing. The better framing: optimize for the reformulated information need the model derives from the prompt. Classical information retrieval reached the same conclusion decades ago — users express needs imperfectly, and retrieval works better when those needs are rewritten into cleaner, more search-effective forms.^[11][12] Modern conversational AI inherits that problem and now solves it automatically, at multiple layers.

For anyone working on AI visibility, answer engine optimization (AEO), or generative engine optimization (GEO), this reframing changes what “ranking” even means. The rest of this paper walks through the pipeline, the research behind it, a hands-on way to observe it, and the practical playbook that follows.

2. The pipeline from prompt to answer

A concise way to describe the pipeline:

This is not speculation; it is consistent with OpenAI’s public product documentation and with the research literature on retrieval-augmented generation. OpenAI says ChatGPT search can choose to search based on what the user asks, and that follow-up questions are interpreted in the context of the full chat.^[2] Its help center says ChatGPT will sometimes rewrite a user query into one or more targeted queries and may send additional, more specific searches to outside providers.^[1]

Working backwards from the answer — a useful “one step back” ladder:

• The final answer is a synthesis over selected sources, with inline citations and a sources panel; deep research adds an activity history showing how the research progressed.^[1][3]
• One level back lies retrieval: documents chosen because they matched the model’s search strategy.
• Another level back lie multiple rewritten queries — not the unedited user wording.^[7]
• At the very start sits query understanding: what the user most likely means, what missing context to fill in, whether to search at all, and whether to clarify, rewrite, decompose, or answer directly.^[8][11]

OpenAI’s Model Spec explains the very first stage: instructions carry different levels of authority, and the assistant should infer the user’s intent, stating assumptions and asking clarifying questions when intent is unclear.^[6] So even before retrieval begins, the system is already interpreting the prompt through a hierarchy of instructions and contextual cues. A practical implication follows: the visible prompt is not the full input. The effective input can include system and developer instructions, conversation history, memory, and location — OpenAI explicitly says memory can be used when ChatGPT rewrites a prompt into a search query, and IP-based location can affect local results.^[1]

3. Why AI systems rewrite and decompose queries

The research literature strongly supports this product behavior, for a simple reason: human prompts are often vague, ambiguous, incomplete, multi-part, or poorly aligned with the language of the documents that need to be retrieved. Query reformulation exists to bridge that mismatch. A recent survey of conversational search describes reformulation — expansion, rewriting, and decomposition — as a crucial initial step, especially because later conversational turns omit key context or refer back to earlier turns.^[11]

Classic search research made the same point long before LLMs. Broder’s taxonomy distinguished navigational, informational, and transactional queries, emphasizing that the “need behind the query” differs from the literal tokens typed into a search box.^[12] Jansen and colleagues operationalized that taxonomy at scale, showing that more than 80% of web queries in their dataset were informational and building an automatic classifier to infer intent from sparse queries.^[13] The relevance here: what the words say and what the user is trying to do are not the same object.

Rewriting as a first-class optimization step

Modern LLM retrieval research treats rewriting as a first-class stage. Query Rewriting for Retrieval-Augmented Large Language Models proposes a Rewrite-Retrieve-Read pipeline, explicitly inserting a rewriting stage before retrieval and showing it improves downstream performance — including with a real-time web search engine as the retriever.^[7] RQ-RAG goes further: ambiguous or complex questions often need rewriting, decomposition, and disambiguation, with complex questions broken into simpler sub-queries before retrieval.^[8] A 2025 ACL paper shows that using an LLM to split a complex question into sub-queries broadens the candidate pool and improves multi-hop question answering.^[9]

There is also direct evidence that LLM-generated expansions beat older techniques: Google Research reports that prompted LLMs generate query expansions that outperform traditional pseudo-relevance-feedback methods, with chain-of-thought prompting especially useful because it produces related terms and stepwise decompositions that assist retrieval.^[10] And rewriting is not a research-only trick: OpenAI’s Retrieval API ships automatic query rewriting as a built-in product feature.^[5]

All of this matters because it means the system is not trying to honor the exact prompt string. It is trying to satisfy the underlying information need with the highest retrieval quality it can achieve. In retrieval terms, the real objective is retrievability after reformulation.

4. What this means for brand visibility and recommendations

OpenAI’s own examples make the point clearly. If a biotech researcher asks, “what’s the latest on the development of drugs that target CCR8 for cancer?”, ChatGPT might first query a search partner with “CCR8 immunotherapy drug development 2025”, then — after inspecting results — issue another query such as “CHS-114 conference 2025”.^[1] The optimization target is no longer the original sentence; it is the model’s evolving interpretation of what would best retrieve evidence.

A second example is even more relevant to recommendations: asked “what are some restaurants near me that I’d like”, ChatGPT may rewrite that into “good vegan restaurants San Francisco” when memory indicates the user is vegan and located in San Francisco.^[1] The “real” search object becomes a combination of prompt, profile, and inferred preference — not the user’s exact wording. This is exactly the mechanism behind competitive brand displacement: if a competitor’s content matches the rewritten query better than yours, the AI recommends them, regardless of what the user literally typed.

So if the system normalizes, enriches, and splits the query, you should optimize for the questions the model is likely to search, not merely the words a human is likely to type. In practice, content should be easy to retrieve under stand-alone query forms, synonym variants, disambiguated entity names, year-qualified queries, and subtopic-specific questions.^[10][11]

There is also a technical inclusion layer beneath wording. OpenAI says ranking in ChatGPT search depends on factors designed to surface reliable, relevant information, that there is no guaranteed top placement, and that sites must allow OAI-SearchBot to appear in ChatGPT search answers; sites opted out will not be shown even though they may still appear as navigational links.^[1][14] The optimization stack is not just semantic — it is infrastructural and credibility-based. (Check your crawl rules with the free robots.txt checker and publish an llms.txt file so AI crawlers can find your canonical pages.)

In AI search, the real battleground is not the user’s raw prompt but the model’s rewritten, enriched, and decomposed representation of intent, which then governs retrieval, ranking, and synthesis.

5. How to inspect fan-out queries manually

There is an important distinction between documented behavior and undocumented observability. The documented part is official: ChatGPT search may rewrite prompts into one or more targeted queries, issue additional specific searches, and use memory and location.^[1] The undocumented part is the browser-level inspection method below. Public reverse-engineering guides from Semrush and Sandbox Web report that, in some ChatGPT search sessions, the browser’s network response contains a field such as search_model_queries holding the rewritten search terms.^[15][16]

Step 1 — Trigger a search and copy the conversation ID

In your own ChatGPT session, run a prompt that clearly triggers web search (the answer shows sources/citations). Then copy the conversation ID from the browser URL — the long identifier after /c/.

Step 2 — Open DevTools and go to the Network tab

Open your browser’s developer tools (F12, or right-click → Inspect) and select the Network tab.

Step 3 — Reload and filter by the conversation ID

Reload the conversation page so the network panel captures the requests, then paste the conversation ID into the network filter box and look for fetch/XHR requests.

Step 4 — Open the JSON response

Click the matching request and open its Response (or Preview) tab to see the conversation’s JSON payload.

Step 5 — Search for search_model_queries

Press Ctrl+F (Cmd+F on Mac) inside the response and search for search_model_queries (or queries). The array you find contains the reformulated, fanned-out queries the system used instead of your literal prompt.^[15][16]

Compare what you typed with what it searched. The delta between the two is the optimization gap this paper is about — and it is exactly what Sophyx’s prompt optimization systematizes across hundreds of prompt variants.

6. Optimizing for the rewritten query: the playbook

If retrieval happens against reformulated queries, your content and infrastructure need to win under reformulation. Five moves follow directly from the evidence above:

1. Write answer-shaped content for stand-alone query forms. Cover the disambiguated, year-qualified, subtopic-specific questions a model would generate — not just your head keyword. Structure pages so a single section fully answers a single reformulated question. This is the core of AEO and LLM SEO.
2. Make entities unambiguous with structured data. Selection logic favors sources whose entities are easy to resolve. Publish entity-rich JSON-LD (Organization, Service, FAQ, Article) — the Sophyx JSON-LD builder and the free schema markup generator produce copy-ready markup.
3. Keep the infrastructural layer open. Allow OAI-SearchBot and peer AI crawlers in robots.txt,^[14] publish llms.txt, and keep canonical URLs clean. Blocked crawlers mean zero visibility no matter how good the content is.
4. Earn mentions where AI already looks. Citation tracking shows which third-party sources engines rely on for your category; being named there raises your selection odds under any phrasing. Find those gaps with the AI citation tracker.
5. Test at the prompt level, continuously. Because rewriting depends on context, memory, and location, a single manual check proves nothing. Run a tracked prompt set across engines and watch mentions, sources, and sentiment move over time with an AI mention tracker.

7. Measuring whether it works

The measurement loop mirrors the pipeline. Track whether engines mention you (ChatGPT, Perplexity, Gemini, Claude, and Google AI Overviews), which sources they cite when they do, which competitors they name when they don’t, and how your AI share of voice trends per engine and per topic. Add real-time alerts so a lost mention or a sentiment shift triggers action the day it happens, not the quarter after.

This is the loop Sophyx runs end to end — Analyze → Prioritize → Implement — from the free baseline audit to prioritized fixes to continuous tracking. If you want to see your starting point, the free AI visibility checker takes a few minutes.

Frequently asked questions

References

1. OpenAI Help Center — “ChatGPT Search” — States that ChatGPT can rewrite a prompt into one or more targeted queries, send additional more specific searches, use memory during rewriting, and use general location to improve results.
2. OpenAI — “Introducing ChatGPT search” — ChatGPT may choose to search based on the request and interprets follow-up questions in the context of the full conversation.
3. OpenAI Help Center — “Deep research in ChatGPT” — Describes the multi-step research workflow: proposed plan, source selection, synthesis into a documented, citation-backed report.
4. OpenAI — “Introducing deep research” — Deep research plans, browses, and synthesizes across sources with an activity trail and cited output.
5. OpenAI — Retrieval API documentation — Documents built-in automatic query rewriting (rewrite_query) — evidence that rewriting is a first-class product capability, not a fringe behavior.
6. OpenAI — Model Spec — Instruction hierarchy and intent inference: the assistant interprets what the user means before deciding how to act.
7. Ma et al. — “Query Rewriting for Retrieval-Augmented Large Language Models” (2023) — The Rewrite-Retrieve-Read pipeline: inserting a rewriting stage before retrieval improves downstream performance.
8. Chan et al. — “RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generation” (2024) — Ambiguous or complex questions need rewriting, decomposition, and disambiguation — not retrieval from the raw query.
9. Ammann et al. — “Question Decomposition for Retrieval-Augmented Generation” (ACL 2025) — Splitting a complex question into sub-queries broadens the candidate pool and improves multi-hop question answering.
10. Jagerman et al. (Google Research) — “Query Expansion by Prompting Large Language Models” (2023) — LLM-generated query expansions outperform classical pseudo-relevance-feedback expansion methods.
11. Ye et al. — “A Survey of Conversational Search” (2024) — Query reformulation — expansion, rewriting, decomposition — is a crucial first step in conversational search systems.
12. Broder — “A Taxonomy of Web Search” (SIGIR Forum, 2002) — The need behind the query differs from the literal tokens typed — navigational, informational, transactional intent.
13. Jansen et al. — “Determining the informational, navigational, and transactional intent of Web queries” (2008) — Over 80% of web queries were informational; intent can be inferred automatically from sparse queries.
14. OpenAI — Bots & crawlers documentation (OAI-SearchBot) — Sites that block OAI-SearchBot will not be shown in ChatGPT search answers — the infrastructural inclusion layer.
15. Semrush — “ChatGPT Searches Google Shopping to Create its Recommendations” — Documents the DevTools technique of finding search_model_queries in the network payload of a ChatGPT conversation.
16. Sandbox Web — “We Built a Free Chrome Extension to See Exactly What ChatGPT Searches” — Independent walkthrough of inspecting fan-out queries in the browser network panel.

About the author

Reza Kashi is the founder of Sophyx, the AI visibility platform that shows brands how AI engines perceive them — and exactly what to fix so ChatGPT, Gemini, Claude, and Perplexity mention and recommend them.