Can You Rank in ChatGPT and other LLMs?

Last updated on September 20th, 2026 at 01:29 pm

“Can we influence ChatGPT?” sounds like a practical question, but for a marketing team it is still far too broad. Influence what, exactly? A single answer? Whether a product page can be retrieved? How often a brand is cited across a defined set of prompts? Or what a custom assistant can access and use?

These are different problems, controlled by different parts of the system. And they require different interventions, different tests and, ultimately, different definitions of success.

The older version of this article leaned heavily on one distinction: a language model is not a search engine. That was a useful way to explain why ChatGPT could not simply be treated like another SERP. But it is no longer enough. Today, the products we interact with can combine an LLM with search, retrieval, external context, tools, memory and custom configurations. Before asking how to influence “ChatGPT”, we first need to identify which part of that system we are actually trying to affect.

Start with three questions: which component are we trying to affect, who controls it, and where should the effect become visible? A publisher, a user and the operator of a custom assistant do not control the same levers.

Research context: adapted from AI-Optimized Marketing: Understanding and Influencing LLMs, edition 1.5.0-LEGACY.

This article maps those control boundaries: what can actually be changed, by whom, and at which layer. The companion article on how sources are extracted and cited follows a different part of the same problem, showing what can happen between publishing a page and seeing information from that page appear in an answer.

This distinction is not merely technical. It changes how we should measure the result. “We updated the page”, “the source was retrieved”, “the brand appeared in the answer” and “the visit converted” describe four separate events. Treating one as proof of another creates a measurement problem before we even start discussing optimization.

Consider a simple example. A company updates its documentation to support a third export format, but an assistant continues to report only two. There is no single explanation for that failure. The answer may have been generated without retrieving the updated page. The retrieval layer may have surfaced a stale source. A third-party page may still contain the old information. Or the correct passage may have been retrieved but not preserved in the final answer.

Each case requires a different response. Simply saying that “ChatGPT has not updated” tells us almost nothing about where the problem actually occurred.

The Dinghy Studio podcast episode below is retained as historical material associated with this research. It is not an updated narration of the corrected text, so statements in the recording should be read against the distinctions introduced in this revised article.

1. Model Parameters, Adaptation and Publishing Are Different Controls

The first boundary is also the most important: publishing new information is not the same operation as changing a model.

Pre-training changes a model’s numerical parameters through optimization over training examples. During ordinary inference, the deployed model instead generates an output using its configured parameters together with whatever context the system makes available. Supplying a new product detail can therefore change an answer without changing the model’s parameters at all.

This is why using the word “learning” for both processes creates confusion. A model can use information that was supplied through a prompt, retrieved from a source or exposed through another context layer without having permanently incorporated that information into its parameters.

Those learned parameters can support generalization, factual associations and, in some cases, memorized sequences. But they are not a conventional searchable collection of pages waiting to be edited when a publisher changes a website. The neural-network explanation goes deeper into the computational mechanism; for our purposes here, the important distinction is simpler: editing a source and updating model parameters are two different operations.

For the full technical progression, begin with the LLM technical overview. Here, we only need enough of that architecture to understand who can modify each layer.

This also explains why saying that a model is “frozen during inference” needs some care. It does not mean that the model can never be adapted. Pre-training can be followed by supervised fine-tuning, preference optimization and other authorized interventions. What matters is who controls that process and whether it changes the model itself, the context around it, or simply a source available to the wider system.

The distinction becomes clearer when we look at model adaptation itself. LoRA, introduced by Hu and colleagues in 2021, is a useful example because it shows why “frozen” should not be confused with “permanently immutable”. The original weight matrix can remain fixed while trainable low-rank additions modify the effective computation. In simplified form, y = W x becomes y = (W + ΔW) x.

In other words, freezing W does not prevent an authorized adaptation mechanism from changing how the model behaves. But adaptation is only one way to change an AI system’s output. Many of the mechanisms marketers encounter operate outside the model parameters entirely:

• Retrieval-augmented generation: an external collection supplies relevant information at runtime. The original RAG paper explicitly separates this external information from the model’s parametric knowledge.
• External memory or knowledge storage: information can be retained outside the model and retrieved later according to the application’s permissions and persistence rules. There is no requirement that such persistence end when one conversation does.
• Tools or application data: a permitted call can return a current record, calculation or external result and make it available to the workflow. The model can use that information without incorporating it into its underlying weights.

These mechanisms are not alternatives that exclude one another. They can be layered together. An adapted customer-support model may still use retrieval for specifications that change every week. A stored user preference may persist across conversations and later become outdated. An application can change its citation policy without retraining the underlying model.

This is why naming the mechanism matters. Once we know where information is stored or introduced, we can ask the right operational questions: who can update it, how long it persists, who can access it, and what needs to happen when it becomes wrong.

OpenAI’s 2024 memory announcement, for example, explicitly described memory that could persist across conversations. That historical record is enough to reject the old assumption that every memory or retrieval effect necessarily disappears when a chat ends. It does not mean that every product stores the same information, for the same duration, under the same access rules.

Record the event you can actually inspect: “document uploaded”, “record updated”, “memory retained”, “adapter trained”. Saying that “the AI learned our brand” hides the mechanism, how long the change can persist, and who can actually observe its effects.

2. Crawling and Indexing Are Not Weight Updates

Once we separate model parameters from external context, a second source of confusion becomes easier to resolve: a page can become newly available to an AI product without the model itself being retrained.

Between publishing a page and seeing information from that page in an answer, the source may pass through several stages: collection, parsing, indexing, retrieval, context assembly and generation. Each stage does a different job, and failure at one stage should not be confused with failure at another.

This is also where the older claim that crawling, indexing or freshness could not matter to LLM-based products became too broad. They may matter considerably to the systems built around the model, even though none of them is equivalent to updating its weights.

• Crawling means that a system requested content from a source. A successful request does not prove that the content was retained, indexed, used for training or selected for an answer.
• Indexing makes some representation of that content searchable within a particular retrieval system. It does not provide a universal interface for changing foundation-model parameters.
• Freshness describes more than the date of the latest fetch. We need to distinguish when the page was obtained, when the page itself changed and when the underlying claim or product specification became valid. A newly fetched page can still contain obsolete information.
• Retrieval selects candidate information for a particular task. A page may be perfectly accessible and correctly indexed yet still fail to enter the selected context — or enter it and be misinterpreted later.

For marketers, the practical consequence is important. A sitemap, URL-inspection tool or indexing mechanism can absolutely matter when the AI product depends on the search infrastructure that uses it. What it cannot honestly be described as is a way to “submit a fact directly to the model’s brain”.

When information appears stale or incorrect, diagnose the actual path instead:

• Published source: is the corrected information visible and accessible at the intended canonical URL?
• Stored representation: does the relevant search index, document store or knowledge collection contain the correct version?
• Selected context: did the target workflow retrieve the relevant passage, and did that passage preserve the relationships needed to interpret it correctly?
• Final answer: did the generative system actually use the retrieved information correctly, and did the product attribute the source when attribution was expected?

This sequence gives us a much more useful debugging model than asking whether “the AI knows” the updated information. Each step produces different evidence and can fail independently.

OpenAI’s crawler documentation distinguishes search-related access, training-related crawling and user-triggered fetches. A generic rule such as “allow AI bots” collapses access paths that serve different purposes. A crawler request is not a citation, and a decision to block one form of access does not automatically describe every other route through which a product may obtain information.

3. Generation, Retrieval and Ranking Can Work Together

The next distinction is subtler. Saying that a language model is a generator does not mean that ranking disappears from an AI system.

A language model generates a probability distribution over possible continuations. A retrieval system searches for candidate information. A ranking or reranking component orders those candidates according to some objective. A modern product can combine all three functions in the same workflow — and in some architectures, a language model can itself participate in the ranking step.

Sun and colleagues’ 2023 research on LLM reranking is a concrete example: it evaluates language models as rerankers for information-retrieval tasks. This makes the blanket statement “LLMs cannot rank documents” technically indefensible.

The more useful distinction is different: ranking a set of candidate documents for a task is not the same thing as maintaining and serving a general-purpose public web index. Once those two ideas are separated, there is no contradiction between generation and ranking appearing inside the same product.

Two operations that sound similar but happen at very different levels make this easier to see:

• Token selection: the model assigns scores to possible continuations, after which a decoding procedure determines which output token is produced. This is a ranking over possible continuations, not a ranking of publishers or web pages.
• Information selection: a retriever or reranker scores and orders documents, passages or other candidate sources for a specific task. That ordering can determine what information reaches the generator, but it is still distinct from the answer the model ultimately produces.

This distinction matters when comparing AI products. A model name alone tells us very little about the complete information path. The same broader product family may support a conversation based primarily on model parameters and user context in one case, and a source-grounded workflow involving search, retrieval and reranking in another.

The table below should therefore be read as a map of information and control layers, not as a catalogue of which model happens to be the default in each product at a particular moment.

Information route / intervention	Object changed or supplied	Control boundary	What the evidence can establish
Ordinary prompt or supplied document	Context available to the interaction.	The user supplies material within the product’s access and context constraints.	A tested answer uses the supplied fact correctly; this does not establish a global weight update.
Public search / retrieval	Candidate information obtained through an index, fetch or retrieval collection.	The publisher controls its source, not the entire outside retrieval or generation stack.	The version retrieved and the claim produced; retrieval does not guarantee citation.
Persistent external memory	A stored record made available in later permitted contexts.	Persistence, audience, retrieval and deletion depend on application controls.	The record was stored or retrieved for a defined audience, not learned universally by the base model.
Controlled knowledge collection / tool	Versioned documents or records returned under authorization.	The operator controls the collection and permitted access; stale sources and field errors remain possible.	The intended evidence reached the workflow and was interpreted correctly.
Authorized training, adapter or model editing	Trainable parameters or effective model computation.	Requires the relevant implementation or service access; it is not caused merely by publishing a page.	Targeted behavior changes under evaluation, including side-effect checks; not guaranteed change in outside models.
Application citation or validation rule	Source requirements, formatting or acceptance checks around generation.	Available to an operator of the controlled application, not automatically to an external publisher.	The configured workflow follows the rule in the tested cases; rules do not guarantee universal compliance.

What Happens Between a Search Result and an Answer?

Once retrieval enters the picture, being available to the system is only the beginning. A source may still have to survive several selection steps before any of its information reaches the final answer.

Consider an invented pipeline. Imagine that 100 candidate passages exist. A first-stage retriever returns ten, a reranker prioritizes four, the available context budget admits three, and the final answer cites two. The numbers are deliberately illustrative: they explain the stages involved, not a recovered ChatGPT trace or a claim about any provider’s fixed limits.

1. Retrieve and prioritize: the system constructs or interprets the request, applies any relevant filters, and ranks candidate information to decide what remains available.
2. Assemble context: the application decides which passages or representations fit into the context supplied to the model. Important relationships can already be lost here: a qualification may be separated from the claim it limits, or a table value from the heading that explains it.
3. Generate and attribute: the model composes the answer from the available context, while the surrounding application may attach citations, run additional checks or trigger further processing before presenting the result.

This is why a relevant page can disappear even after it has been successfully crawled and indexed. The appropriate intervention depends on where the information was lost.

• A citation instruction cannot recover a document that never entered the candidate set.
• A better retriever cannot, by itself, guarantee that the generator preserves an important restriction contained in the retrieved passage.
• A model adaptation cannot automatically repair an obsolete product record, a broken parser or a transformation that separates a numerical value from its unit.

From a measurement perspective, this has an equally important consequence: a missing citation does not tell us which stage failed. We need to preserve enough information about the test to reconstruct the path.

Repeat a defined test and preserve its conditions: prompt, date, product surface, source access, answer and citations. One missing mention does not identify a failure mechanism, and one successful citation does not establish repeatable visibility for every user or future run.

Evaluate Correctness Rather Than Arguing from “Understanding”

The companion article on knowledge and factuality explains why fluent language should never be mistaken for guaranteed accuracy. But the opposite shortcut is not useful either: saying that “the model only predicts tokens” does not tell us whether a configured system can successfully perform the task in front of us.

A system may compare two sources correctly and fail on a third. It may calculate a result perfectly when a tool is available and make an arithmetic mistake without it. It may retrieve the right documentation and still overlook a qualification. These are observable failures, and they give us something concrete to test.

For a product, brand or technical question, ask what the system actually preserved:

• Entity: did the answer identify the correct company, product and edition rather than merge similar entities?
• Conditions: did it preserve the source’s prerequisites, limitations and applicable period?
• Evidence: did the cited passage actually support the statement, rather than merely mention the same topic?
• Transformation: did calculations, comparisons or conversions preserve the original values, relationships and units?
• Uncertainty: when the required evidence was absent, did the system identify the gap or invent a detail to complete the answer?

This also prevents us from dismissing tool-assisted performance for the wrong reason. If a configured system successfully answers because it can search, calculate, read a file or query a database, that is still a capability of the system the user interacted with. It should simply be documented with its tools and operating conditions.

The reverse is equally important. The presence of a tool does not prove that the system invoked it, received the right result or interpreted that result correctly. Capability needs to be observed, not inferred from the feature list.

4. Information Enters Through More Than Training and Public Search

There is another consequence of looking at the whole system rather than only at the foundation model: public web search is not the only route through which information can reach an answer.

A brand may appear because the model has learned associations involving it during training. But it may also appear because the system retrieved a third-party description, because a user uploaded a document, because the application stored a project fact, or because an authorized tool returned a current business record.

These routes matter because they change what “visibility” means. Public organic ranking may be essential in a search-grounded workflow and completely irrelevant to a private document comparison. Likewise, seeing a brand mentioned in an answer does not prove that the system visited or retrieved the brand’s own website.

For marketing and information maintenance, most controllable cases can be organized into three practical categories:

• Public sources: maintain accurate, accessible owned pages and relevant third-party descriptions. Then test the search or retrieval path actually used by the target product rather than assuming that every system shares one universal index.
• Directly supplied context: make sure uploaded documents, pasted specifications or other supplied materials contain the necessary facts in a form that remains understandable inside the available context. A private comparison does not require the source document to rank on the public web.
• Controlled stored information: maintain versions, permissions, ownership and correction processes for knowledge collections, memory and connected records. Persistence is useful only if the stored information can also be corrected when circumstances change.

This is where traditional SEO needs a more precise role. SEO remains highly relevant whenever search infrastructure mediates access to information. A generated answer does not make crawlability, indexing, ranking or source quality irrelevant.

But SEO is not the control surface for every AI interaction either. It cannot determine what a user uploads, what an enterprise knowledge base contains or what an authorized application returns through a private connection. The useful question is therefore not whether “SEO works for AI”, but where search participates in the information path we are trying to influence.

This is the practical scope of Organic Search Engineering in this research: connect accurate information, technical access and observation. Search visibility, source use, attribution and commercial response may influence one another, but they are not interchangeable measures of success.

What a Marketing Team Can Reasonably Promise

Once these boundaries are explicit, the marketing brief becomes much easier to write. A team cannot responsibly promise that it will “make ChatGPT know the brand” or secure a permanent position across generated answers. It can take responsibility for the information surfaces it controls, test the paths it depends on, and measure the outcomes it can actually observe.

• Maintain reliable sources with explicit entities, versions, applicable conditions and clear responsibility for keeping them current.
• Test the intended information path and distinguish failures of access, retrieval, extraction, generation and attribution rather than reporting all of them as a generic “AI visibility” problem.
• Use authorized controls where they exist, including controlled context or adaptation, without presenting a private intervention as a global change to another provider’s foundation model.
• Report observed behavior together with its conditions: product surface, date, prompt, retrieval state and relevant configuration. One answer should not be converted into a universal ranking or total market-visibility score.
• Separate technical changes from business effects. Correcting a source, being retrieved, receiving a citation, generating a visit and producing a conversion are different events and require different evidence.

This is a narrower promise than “ranking in ChatGPT”, but a much more useful one. It gives the team something that can be implemented, diagnosed and measured — and prevents a technical observation from being inflated into a commercial result it does not prove.

The full AI-Optimized Marketing research, edition 1.5.0-LEGACY, expands this map with the underlying model mechanics, source analysis and measurement framework. Use this article as the entry point: first define the object of the intervention and the information path involved; only then decide what should be optimized, tested or reported.

FAQs