AI preferences exist to give those who create and distribute content online a way to express how they would like to see that content used.

This post is a detailed technical update on the present state of the work, focusing on the recent developments. I make an attempt to explore some of the issues that arise from those recent discussions.

Background

This is for those who weren’t following along, others can skip to the substance.

Websites are created for many reasons, but many sites have the same motivations: to reach an audience of humans. This might be to inform, entertain, or to provide services. In many cases, gaining the attention of an audience is a complementary goal, with that attention being used to show advertisements. Those advertisements can be a significant source of revenue for site operators, supporting their ability to continue to provide other services.

Automated systems, or bots, have long been a part of the web. Web crawlers in particular are those bots that seek to explore the entirety of the web. This is done for many reasons, including archival and research, but a particularly important use of crawling is in the development of web search engines.

Web search engines operate an important class of crawler. These crawlers are part of providing a valuable service to sites, as the search engines they support drive traffic – and attention – to those sites. In exchange, search engines receive their own highly valuable form of attention. Search advertising is a highly lucrative business.

Enter Artificial Intelligence

AI has disrupted this equilibrium significantly for two reasons.

AI products have displaced some of the functions of search. If the purpose of a given search is to answer a question, a chatbot or AI-generated answer is far more convenient than search. The effect of this has been a reduction in attention for certain classes of sites.

The second reason is that the sites that are the source of the knowledge that AI uses need to be crawled to gain that information. This has caused a large increase in the volume of queries from AI, both to train models and to provide grounding in the use of those models. The significant increase in costs for site operators in answering requests does not result in the valuable human attention that might have been the reason for deploying a website in the first place.

In other words, AI can provide a substitute for the work of websites, depriving them of support, while also adding operational costs.

This is far from being a wholly bad outcome. There are good reasons for this change: AI can provide a vastly superior experience. Accurate and specific answers to questions are just one of the potential improvements. High quality sites will continue to engage audiences in meaningful ways. And many sites exist for reasons other than to attract attention.

Nonetheless, change is disruptive and there are risks that need to be managed. AI is potentially subject to the biases of its creators, which can distort messages, intentionally or accidentally. AI hallucination can mean that people or organizations can be misrepresented, propagating misinformation in ways that can be hard to trace. And the potential for AI to act as a substitute for the work of human writers and artists is a serious one.

Existing Tools

Presently, there are two tools that site operators can deploy to influence how AI interacts with the content on their sites: access control and robots.txt.

Access control can outright block requests from crawlers. Well-behaved crawlers make requests from IP address ranges that their operators publish and use distinctive identifiers in HTTP User-Agent headers. Requests that are identified in either way can be blocked. Even a malicious crawler would struggle to use IP addresses for long without being identified and blocked.

Most web crawler operators also respect the robots.txt file. Site operators can publish a robots.txt file that describes which parts of their site are off-limits. This is little more than a polite request to crawlers, rather than an access control mechanism. A site deploying robots.txt depends somewhat more on trusting the AI crawler to act responsibly.

Most crawlers do respect these requests in robots.txt. The risk to the reputation of an AI company that did not respect robots.txt seems sufficient to ensure compliance. After all, other AI companies would not look favorably on a competitor who gave sites cause to deploy access control instead, because they might be next.

However, these are both crude tools. By convention, crawlers use their crawler name^[1] to identify the purpose of their crawling. If a company crawls for several reasons, they need to crawl multiple times, each time with a different crawler name. Sites can then use robots.txt to target the specific crawlers they approve of. Effective use of robots.txt then requires that sites are able to identify the many different crawlers that exist, understand the purpose of each, before they can set the scope that each can crawl.

New crawlers, which are added all the time, only get default instructions until the site operator learns about them. This can overly constrain use for novel purposes by restricting more than necessary. Alternatively, setting more permissive defaults could endorse uses that the site might not intend.

How AI Preferences Might Help

The AI preferences work aims to address this by giving site operators a means to directly express preferences about how the content they serve might be used.

AI preferences are a collection of statements about different categories of use that are associated with a given piece of content (or asset). Sites can express a positive preference that indicates permission to use the content for that purpose, they can express a negative preference that requests that the usage not be applied to that content, or make no information about their preferences known.

Sites associate AI preferences with content using content metadata, HTTP headers, or their robots.txt file.

Standard Terms

A set of standard definitions for content use ensures that site operators do not need a comprehensive understanding of what every web crawler is used for. Preferences can be expressed in common terms that crawlers – both existing and newly introduced – can understand.

Companies that crawl the web for multiple reasons can unify their crawling. Content can be acquired once and different applications can limit their use to the content that has compatible preferences.

The cost of this is that terms will draw from a limited vocabulary of terms. A limited vocabulary is an asset in terms of making the terms understandable to those who will express their preferences. Applications are also better able to write software that respects preferences.

Importantly, common definitions make it more likely that different entities can agree on what each term means, even as new applications are developed.

Around a common vocabulary, the goal is to allow for preferences to be expressed in several ways. That way, the preferences mean the same thing, no matter how they are expressed. That ensures that content metadata can use forms of expression that fit the metadata idioms of a format, but ensure that the meaning is consistent with the core vocabulary.

Just Preferences

A key aspect of the design of AI preferences is their discretionary nature. Preferences are not an access control system. They have no means to force crawlers into compliance. Just like robots.txt, AI preferences rely on AI crawlers choosing to respect their requests.

In choosing a design that includes preferences the work recognizes that when sites express preferences they cannot perfectly anticipate the conditions where content might be used. Sometimes there are overriding interests that will mean that the right choice is to ignore the preference that was expressed. In developing the standard, a diverse set of reasons that might be cause to ignore preferences were discussed, including accommodating accessibility needs, public interest, research, identifying illegal or abusive content, archival, and more.

In choosing to use a design that expresses preferences, rather than a stricter prohibition, sites do rely more on trust than is even necessary with robots.txt. With robots.txt, a crawler might be caught out when it requests content after it was asked not to. In comparison, AI preferences say nothing about whether content is fetched or not, only about how it might be used, once obtained.

Site operators therefore have no obvious way of checking that their preferences are respected. For AI applications, models can leak their inputs, but this is something that AI developers seek to avoid, not to avoid getting caught out this way, but more to ensure that their models are capable and flexible. That means that using AI preferences is also an expression of trust in those who seek to use content. Sites are trusting AI companies to respect their preferences and to exercise good judgment in determining whether to override those preferences.

Finally, choosing preferences also acknowledges that there is no widely agreed legal protection involved. Different jurisdictions are working through the implications of AI on their copyright laws, but we cannot predict the outcome of those processes. It’s possible that some laws will have something to say about how expressions of preference need to be treated, but nothing is settled.

What Site Operators (Seem To) Want

Though we have no formally agreed requirements, there has been a lot of discussion about these. A recap is probably in order.

Sites are not all the same in their goals with respect to preferences. However, several key themes have emerged through the process of developing the standard.

Preferences About All Uses

A blanket approval or disapproval preference is contested, but a number of parties have indicated that it would be a useful, albeit insufficient, requirement.

The most common criticism of this is that is “too broad” or an imprecise instrument. That’s a poor argument: we cannot tell someone that they cannot hold this preference. Moreover, this is a preference that is trivial to define.^[2]

But there is a difficult question to answer at the heart of this objection: saying “no” unconditionally precludes unexpected and unforeseen uses. That’s a concern that cuts two ways.

AI was an unanticipated use for content before it became wildly successful. Developing foundational AI models would have been harder to achieve if a system of preferences existed that constrained the use of content for AI.

If you regard AI as a good thing, the potential for a lot of content to be withheld from models as a result of a broad preference could be bad. While there is a lot of content that would remain available, including public domain content, having some content excluded might reduce the quality of models.

Others might view the use of content by AI companies as extractive, taking content without compensation to those who created it. More so because it might be seen to enable the creation of work that competes in the market for the content that was taken. Had there been a way to express preferences about this unexpected use, perhaps that could be a starting point from which to negotiate for fair compensation.

For those who might outright oppose AI, it would be unrealistic to expect that preferences could have prevented or significantly delayed the creation of AI. There is simply too much content available for training.

The question of whether to include the option of stating a preference about all conceivable uses remains contested. Perhaps we need to accept that technical standards are not well-suited to addressing this conflict and leave the resolution to legislative or regulatory bodies. That might argue for including a broad category of use, leaving it to the law to determine how preferences about that category might take effect.

Model Training

The ability of AI models to reproduce their inputs, in whole or part, is a major concern for sites, artists, writers, and others.

One concern is direct reproduction of content by models. This could directly violate copyright, which is another reason that AI companies seek to avoid having this happen.

There are also aspects of works that are not protected by copyright, such as style. Seeking to protect style from reproduction might motivate the withholding of content from training.

Finally, there is a view that AI companies profit from the models they create using content that was published to the web for human consumption. Some actors might withhold content in the hopes that those performing model training might be willing to pay for its use.

To that end, a common request is to have a way to request that work not be included in model training.

Appearing in Search Only

The most common request is that sites be able to express a preference not to be used for model training or to have their content otherwise processed by automated systems, with the exception of search engines. Having content be discoverable through search applications is valued by many site operators.

In expressing a preference for only search, site operators seem to have two major concerns:

1. The substitutive effect of things like AI generated overviews and content, whether that be part of search applications, chatbots, or other modalities. For sites that depend on attention, their concern is that generated answers to questions cause a reduction in people visiting their site to seek answers. For artists, they might seek to avoid the generation of content that acts as a substitute for their own creative efforts.
2. The reputation risk that comes from misrepresentation, either due to misinformation in model training sets affecting outputs or due to the propensity of AI systems to hallucinate.

For the first, the question regarding any substitutive effect is a partly legal question. Of course, the scope of copyright law is definitely not a question for technical standards to resolve. The effort to define preferences for AI usage cannot answer that question; at best, a standard can only provide regulators with more options.

The real question for standards is whether this is a coherent preference to express. In answering that question, the challenge is to construct a definition that is clear, comprehensible, and implementable.

Making something implementable turns out to be especially difficult, because search engines use AI in multiple ways. While the details are intentionally obscure^[3], the operation of a search product involves both training and use of models. If content is excluded from any aspect of processing – training, inference, or otherwise – search providers could not guarantee that the content would be appropriately ranked.

In discussion, it was also pointed out that there is no meaningful distinction between what people think of as “traditional” search and chatbots. These applications exist on a continuum and we can expect points of difference to continue to be less distinct as providers experiment with new features.

That complication rules out simple definitions that focus on either the training of a model (that is, the process of producing model weights) or the use of trained models (or what is called inference) as part of a system.

Foundation Model Production Preference

This change is a relatively simple change. The idea is to collapse the two existing model training categories into one. That one would only address the creation of a foundation model.

Like the other categories being proposed, the goal is to focus on outputs. The output in this case is a foundation model.

Aside from a shift in emphasis, this change addresses some definitional challenges that were identified with the existing AI training category. It has become clear that there is no crisp delineation between what might be regarded as simple statistical techniques – such as logistic regression or ordinary least squares – and AI. Even questions of scale are not helpful when you consider that “classic” statistical models can still be enormous, such as the meteorological models used to predict weather.

In comparison, there are many well-established definitions for foundation models that all broadly agree. Producing a category of use where the output is a foundation model – or fine-tuned foundation model – seems like an approach that could get support.

There’s an open question about whether various fine-tuning techniques are included in the definition. One interpretation says that the output would have to be a foundation model, because that is what has the general purpose capabilities. A fine-tuned model might be specialized to a single purpose.

The proposed definition includes fine tuning, mostly because a fine-tuned model is likely to inherit most of the capabilities of the tuned foundation model. Also, this expansion to the definition closes a loophole, where applying a small amount of fine tuning could be used to avoid an obligation to respect a negative preference.

This still leaves several questions unresolved, such as the question about whether techniques like low rate adaptation (LoRA), which don’t alter the parameters of a foundation model, would – or should – fit this definition.

Addressing the Search Preference

If the goal is to enable more granular expressions of preference, such that different entities can either express a preference to either:

• exclude all uses except search; or
• exclude model training but allow search.

In doing so, we need to account for the fact that search applications use AI.

This effort might consider other statements of preference, but these are the two primary use cases to address.

The Original Search Definition

Rather than approach the problem from a procedural perspective, the first attempt at a definition for “search” looked at outcomes:

Using one or more assets in a search application that directs users to the location from which the assets were retrieved.

Search applications can be complex and may serve multiple purposes. Only those parts of applications that direct users to the location of an asset are included in this category of use. This includes the use of titles or excerpts from assets that are used to help users select between multiple candidate options.

Preferences for the Search category apply to those parts of applications that provide search capabilities, regardless of what other preferences are stated.

Parts of applications that do not direct users to the location of assets, such as summaries, are not covered by this category of use.

The strength of this definition is that it says nothing about how the identified outcomes are achieved. However, there are several shortcomings:

• What it means to “direct users” to a location is unclear. The fairly obvious question is whether providing a link suffices or, if not, what would be sufficient to meet this condition.
• There are also questions about the practicality of dividing search applications in the imagined manner. To some extent, the content presented in search applications could be linked together in non-obvious ways. For instance, content in AI overviews might be generated based on the content of pages that are linked from other parts of the page.
• This category was defined to be a subset of an “AI Use” category, which excluded model training. What we heard from search providers is that this does not reflect common practice in search applications, where model training is an integral part of the application.

This might not be a complete enumeration of the reservations. These specific concerns might be overcome with more work. In practice, it became clear from discussions that an alternative approach was more appealing to a number of participants.

AI Output

That approach is based on a proposal from Microsoft that suggested a focus on “display-based” preferences. This proposal suggested that generative AI training remain in the vocabulary, but the bulk of the proposal was focused on system outputs.

This proposal identified a number of controls that existing search engines respect with respect to how content is handled in the presentation of search results. These are largely just existing preference signals specific to search, but the proposal introduced some new ideas.

These ideas aren’t intended to be limited to search applications; search is just where the concepts originate. The goal is to apply the concepts to any AI application; or, if we just view AI as a new way of building software, any system that performs computation.

The key idea behind this preference is to avoid constraining use that is not observable outside of the system. These preferences do not care how the system is implemented; they only relate to what the system produces in its output.

If the preferences do not say anything about the internal processing that systems perform, search applications could perform ranking on content that won’t be displayed. AI applications would be able to process content internally as long as the content is not used in producing outputs.

What that means precisely is an important question that is discussed below, but the general shape is approximately: if the preference is to allow this new category of use, then the system can use the content in its output; if the preference is to disallow this usage, the content isn’t used in producing outputs.

Necessary Training

A key component of this approach is that it explicitly allows model training. This recognizes that the training of models is an integral part of the operation of this class of application. In effect, the preference would not distinguish between training a model, using a trained model, or even non-AI uses of content.

Exact Text Match or “Search”

The concept of “exact text match” is the main concession to “traditional” search in the proposal. When that option is chosen, either excerpts of content are presented in outputs or the content is not presented at all.

Microsoft’s proposal for “exact text match” included an extra stipulation: in addition to the usage being limited to excerpts from the content, the output needed to include a link back to the content.

The combination of verbatim excerpts and links is intended to reproduce the concept of “traditional” search, while allowing all of the internal processing that search applications

Without this condition, a preference to allow AI output would result in content being used by any AI system to produce outputs. This would include everything from chatbots to search. This extra condition ensures that output does not reinterpret content, but presents it verbatim.

Challenges to Resolve with AI Output

As with any change, the positives that motivate the change come with a number of new problems to resolve.

My goal here is to identify the issues that are central to disagreements. That is, the problems we need to address in order to make progress. No doubt there are many other issues that I haven’t identified.

“Search” Category Naming

In the current proposed vocabulary, the “exact text match” category has been tentatively labeled “search”. This is because it is intended to address this very narrow concept of “traditional” search where the output of the system is limited to links and context.

Having labels that are widely understood is a very important goal. However, that also requires that the label is a good match to the thing it applies to.

This choice of label has received some criticism, which might be based on this misrepresenting the value that modern search products provide. In practice, those services we think of as providing search have evolved to provide so much more than this basic means of discovering content.

Resolving this means dealing with this tension.

Category Nesting and Spelling

The “exact text match” category is presently defined as a subcategory of a more general AI output category. This makes sense in that the behavior described is a strict subset.

This differs subtly from some of the nesting in other parts of the vocabulary. As a result, this nesting might not be the best technical fit.

Consider the relationship between an overarching category and a foundation model training category. It is clear that there are uses within the broader category that are not foundation model training, such that it makes sense to express preferences for in any of the four possible combinations.

For AI output, allowing the broader category leaves no part of the subcategory excluded. It almost makes no sense to indicate a preference to allow AI output while disallowing the narrower category. The result is almost nonsensical: would it be forbidden to link to content? or, would this require reinterpretation rather than including snippets?

This is the only nonsensical combination of preferences that might arise from this arrangement. This could be managed by saying that the exclusion has no effect.

We might instead seek a different way of spelling these preferences. For instance, the first design discussed for this included placing conditions on a preference to allow AI output. Two conditions could be attached: “link” and “quote”, each representing a distinct restriction on use. This approach is moderately more complex in its design^[4], but the option remains.

Questions of spelling are normally relatively easy to resolve. The challenge here is in determining whether disagreements on the superficial matter mask real disagreements in principle.

Isolating Outputs

The core advantage of the proposal is that it doesn’t matter how the internals of the system are implemented, but it is still necessary to know that it is possible to respect preferences in practice.

If the preference only applies to output, an important question arises: how separable is any internal processing from the production of output? That is, if a preference requests linking and excerpts, how does a system that uses content for internal processing ensure that the content does not affect outputs, except as requested?

For a search application that seeks to produce a list of links, it might be possible to keep the process of ranking items separate from any presentation of those items. It seems reasonable to assume that content with preferences to disallow AI output could be presented using non-AI methods.

But how would a chat bot ensure similar isolation? Even if separate models are used for internal processing and output, what mechanism would ensure that the output stage is isolated from the internal stage? The internal processing stage needs to communicate its conclusions and instructions to the output stage, which risks carrying information about the content it processes.

The same applies whether the internal model is trained on the affected content, or whether that content is only provided as a reference at inference time.

An alternative interpretation is that this internal processing and training is only permitted if either category – AI output or search – is permitted. This is a much narrower interpretation, but it avoids questions about isolation.

The answer to this question was not clear in Microsoft’s proposal. It is perhaps the most important question to resolve with respect to this proposed path.

Model Separation

One potential consequence of allowing model training as part of an AI output category is that models will be trained (or fine-tuned) on content that might otherwise have an associated preference to disallow the production of models.

The consequence is that the models produced cannot be made available for other purposes if the content they use does not permit the production of models. This seems like it is manageable for applications that are deployed today, where the internals of their operation is often closed. However, the impact on more open systems is unclear.

For instance, if multiple actors cooperate to deliver a service, do these output-based preferences apply at the boundary of each system? or do they only apply to the overall system? how might preferences need to be propagated in either case?

The Role of Human Users

This question of system boundaries is most relevant when asking about whether the definition of “output” is in relation to what is presented to a human user. We need to determine whether preferences can only relate to the output of a given system, rather than require human interaction. In one model, the responsibility for respecting a preference ends at the boundary of the system that the preference-respecting entity controls. To include humans in a definition is more challenging.

Having definitions that seek to apply only at the point that something is presented to a human is appealing. However, that depends on having some expectation of humanity, where identifying clients as human is increasingly a contested problem. In addition to bots, agentic browsing will fundamentally change how sites are interacted with. This is all directly on behalf of users, but with the possibility that humans are not responsible for inputs and do not see outputs.

Is it sufficient to present outputs on an expectation that the recipients are human, even when you know that is not assured? Similarly, is it reasonable to allow sites to assume that their inputs come from human users?

No Easy Path to Success

Given the discussion thus far, it is clear that there is a lot more discussion needed before we can declare success. What encourages me is that there seems to be active engagement on the central problem: finding ways to address the expressed requirements on preferences in a way that can be reliably implemented in AI systems.

That is made harder by our insistence on reaching consensus. Discussions thus far have shown that there are gaps in understanding, communication, and trust to be crossed. From here, we try to reach an understanding on the principles, then build on that to develop a solution.

Success could provide people seeking to balance competing interests with more options. Though there’s no guarantee that this works out, it is worth the effort.

Thanks to Paul Keller and Mark Nottingham for their feedback on drafts of this post.