Low EntropyStandard Considerations2023-11-21T00:00:00Zhttps://lowentropy.net/Martin Thomsonmt@lowentropy.netNext Level Version Negotiation2020-12-11T00:00:00Zhttps://lowentropy.net/posts/vn/<p>The <a href="https://www.iab.org/activities/programs/evolvability-deployability-maintainability-edm-program/">IAB EDM Program</a><sup class="footnote-ref"><a href="https://lowentropy.net/posts/vn/#fn1" id="fnref1">[1]</a></sup> met this morning. While the overall goal of the meeting, we ended up talking a lot <a href="https://intarchboard.github.io/use-it-or-lose-it/draft-iab-use-it-or-lose-it.html">a document</a> I wrote a while back and how to design version negotiation in protocols.</p>
<p>This post provides a bit of background and shares some of what we learned today after what was quite a productive discussion.</p>
<h2 id="protocol-ossification">Protocol Ossification</h2>
<p>The subject of protocol ossification has been something of a live discussion in the past several years. The community has come to the realization that it is effectively impossible to extend many Internet protocols without causing a distressing number of problems with existing deployments. It seems like no protocol is unaffected<sup class="footnote-ref"><a href="https://lowentropy.net/posts/vn/#fn2" id="fnref2">[2]</a></sup>. IP, TCP, TLS, and HTTP all have various issues that prevent extensions from working correctly.</p>
<p>A number of approaches have been tried. <a href="https://tools.ietf.org/html/rfc7540">HTTP/2</a>, which was developed early in this process, was deployed only for HTTPS. Even though a cleartext variant was defined, many implementations explicitly decided not to implement that, partly motivated by these concerns. <a href="https://quicwg.org/base-drafts/draft-ietf-quic-transport.html">QUIC</a> doubles down on this by encrypting as much as possible.</p>
<p><a href="https://tools.ietf.org/html/rfc8446">TLS 1.3</a>, which was delayed by about <em>a year</em> by related problems, doesn’t have that option so it ultimately used trickery to avoid notice by problematic middleboxes: TLS 1.3 looks a lot like TLS 1.2 unless you are paying close attention.</p>
<p>One experiment that turned out to quite successful in revealing ossification in TLS was <a href="https://tools.ietf.org/html/rfc8701">GREASE</a>. David Benjamin and Adam Langley, who maintain the TLS stack used by Google<sup class="footnote-ref"><a href="https://lowentropy.net/posts/vn/#fn3" id="fnref3">[3]</a></sup> found that inserting random values into different extension points had something of a cleansing effect on the TLS ecosystem. Several TLS implementations were found to be intolerant of new extensions.</p>
<p>One observation out of the experiments with TLS was that protocol elements that routinely saw new values, like cipher suites, were less prone to failing when previously unknown values were encountered. Those that hadn’t seen new values as often, like server name types or signature schemes, were more likely to show problems. This caused Adam Langley to <a href="https://www.imperialviolet.org/2016/05/16/agility.html">advise</a> that protocols “have one joint and keep it well oiled.”</p>
<p><a href="https://intarchboard.github.io/use-it-or-lose-it/draft-iab-use-it-or-lose-it.html">draft-iab-use-it-or-lose-it</a> explores the problem space a little more thoroughly. The draft looks at a bunch of different protocols and finds that in general the observations hold. The central thesis is that for an extension point to be usable, it needs to be actively used.</p>
<!--
## Grease
David and Adam observed that the cause of this is that many extension points in protocols don't receive new usage for extended periods. They then suggested that apparent, but meaningless, use of extension codepoints by widely deployed implementations would ensure that new values were routinely encountered by other implementations.
As failure to handle new values is a bug, and one that is relatively easy to fix, the goal here is to ensure that implementations - especially new ones - encountered conditions that would trigger failures if they had that bug.
The community is still unsure that greasing is a generally applicable technique. There is also the suggestion that the bugs found in TLS were a one-off. Maybe new implementations of new protocols won't have the implementation flaws of the past^[This view is tempered somewhat by experience with HTTP/2 which has some pretty serious issues.].
And then there are limitations inherent to the design. We really only know how to use greasing where implementations are required to ignore unknown values. Not all protocol extension points use that sort of extension model.
Greasing is being included in QUIC and HTTP/3, so the hope is that we'll continue to learn more.
-->
<h2 id="version-negotiation">Version Negotiation</h2>
<p>The subject of the discussion today was version negotiation. Of all the extension points available in protocols, the one that often sees the <em>least</em> use is version negotation. A version negotiation mechanism has to exist in the first version of a protocol, but it is never really tested until the second version is deployed.</p>
<p>No matter how carefully the scheme is designed<sup class="footnote-ref"><a href="https://lowentropy.net/posts/vn/#fn4" id="fnref4">[4]</a></sup>, the experience with TLS shows that even a well-designed scheme can fail.</p>
<p>The insight for today, thanks largely to Tommy Pauly, was that the observation about extension points could be harnessed to make version negotiation work. Tommy observed that some protocols don’t design in-protocol version negotiation schemes, but instead rely on the protocol at the next layer down. And these protocols have been more successful at avoid some of the pitfalls inherent to version negotiation.</p>
<p>At the next layer down the stack, the codepoints for the higher-layer protocol are just extension codepoints. They aren’t exceptional for the lower layer and they probably get more use. Therefore, these extension points are less likely to end up being ossified when the time comes to rely on them.</p>
<h3 id="supporting-examples">Supporting Examples</h3>
<p>Tommy offered <a href="https://github.com/intarchboard/edm/issues/8#issue-759871255">a few examples</a> and we discussed several others.</p>
<p><a href="https://tools.ietf.org/html/rfc8200">IPv6</a> was originally intended to use the IP <a href="https://en.wikipedia.org/wiki/EtherType">EtherType</a> (0x0800) in 802.1, with routers looking at the IP version number to determine how to handle packets. That didn’t work out<sup class="footnote-ref"><a href="https://lowentropy.net/posts/vn/#fn5" id="fnref5">[5]</a></sup>. What did work was assigning IPv6 its own EtherType (0x86dd). This supports the idea that a function that was already in use for other reasons<sup class="footnote-ref"><a href="https://lowentropy.net/posts/vn/#fn6" id="fnref6">[6]</a></sup> was better able to support the upgrade than the in-protocol mechanisms that were originally designed for that purpose.</p>
<p>HTTP/2 was floated as another potential example of this effect. Though the original reason for adding <a href="https://tools.ietf.org/html/rfc7301">ALPN</a> was performance - we wanted to ensure that we wouldn’t have to do another round trip after the TLS handshake to do <a href="https://tools.ietf.org/html/rfc2817">Upgrade</a> exchange - the effect is that negotiation of HTTP relied on a mechanism that was well-tested and proven at the TLS layer<sup class="footnote-ref"><a href="https://lowentropy.net/posts/vn/#fn7" id="fnref7">[7]</a></sup>.</p>
<p>We observed that ALPN doesn’t work for the HTTP/2 to <a href="https://quicwg.org/base-drafts/draft-ietf-quic-http.html">HTTP/3</a> upgrade as these protocols don’t share a protocol. Here, we observed that we would likely end up relying on <a href="https://datatracker.ietf.org/doc/html/draft-ietf-dnsop-svcb-https-02">SVCB</a> and the HTTPS DNS record.</p>
<p>Carsten Bormann also pointed at <a href="https://tools.ietf.org/html/rfc8428">SenML</a>, which deliberately provides no inherent version negotiation. I suggest that this is an excellent example of relying on lower-layer negotiation, in this case the content negotiation functions provided by underlying protocols like <a href="https://tools.ietf.org/html/rfc7252">CoAP</a> or HTTP.</p>
<p>It didn’t come up at the time, but one of my favourite examples comes from the people building web services at Mozilla. They do not include version numbers in URLs or hostnames for their APIs and they don’t put version numbers in request or response formats. The reasoning being that, should they need to roll a new version that is incompatible with the current one, they can always deploy to a new domain name. I always appreciated the pragmatism of that approach, though I still see lots of <code>/v1/</code> in public HTTP API documentation.</p>
<p>These all seem to provide good support for the basic idea.</p>
<h3 id="counterexamples">Counterexamples</h3>
<p>Any rule like this isn’t worth anything without counterexamples. Understanding counterexamples helps us understand what conditions are necessary for the theory to hold.</p>
<p><a href="https://tools.ietf.org/html/rfc3584">SNMP</a>, which was already mentioned in the draft as having successfully managed a version transition using an in-band mechanism, was a particularly interesting case study. Several observations were made, suggesting several inter-connected reasons for success. It was observed that there was no especially strong reason to prefer SNMPv3 over SNMPv2 (or SNMPv2c), a factor which resulted in both SNMP versions coexisting for years.</p>
<blockquote>
<p>There was an interesting sidebar at this point. It was observed that SNMP doesn’t have any strong need to avoid version downgrade attacks in the way that a protocol like TLS might. Other protocols might not tolerate such phlegmatic coexistence.</p>
</blockquote>
<p>While SNMP clients do include probing code to determine what protocols were supported. However, as network management systems include provisioning information for devices, it is usually the case that protocol support for managed devices is stored alongside other configuration. Thus we concluded that SNMP - to the extent that it even needs version upgrades - was closest to the “shove it in the DNS” approach used for the upgrade to HTTP/3.</p>
<h2 id="in-practice">In Practice</h2>
<p>The lesson here is that planning for the next version doesn’t mean designing a version negotiation mechanism. It’s possible that a perfectly good mechanism already exists. If it does, it’s almost certainly better than anything you might cook up.</p>
<p>This is particularly gratifying to me as I had already begun following the practice of SenML with other work. For instance, <a href="https://tools.ietf.org/html/rfc8188">RFC 8188</a> provides no in-band negotiation of version or even <a href="https://tools.ietf.org/html/rfc7696">cryptographic agility</a>. Instead, it relies on the existing content-coding negotiation mechanisms as a means of enabling its own eventual replacement. This was somewhat controversial at the time, especially the cryptographic agility part, but in retrospect it seems to be a good choice.</p>
<p>It’s also good to have a strong basis for rejecting profligate addition of extension points in protocols<sup class="footnote-ref"><a href="https://lowentropy.net/posts/vn/#fn8" id="fnref8">[8]</a></sup>, but now it seems like we have firm reasons to avoid designing version negotiation mechanisms into every protocol.</p>
<p>Maybe version negotiation can now be put better into context. Version negotiation might only belong in protocols at the lowest levels of the stack<sup class="footnote-ref"><a href="https://lowentropy.net/posts/vn/#fn9" id="fnref9">[9]</a></sup>. For most protocols, which probably need to run over TLS for other reasons, ALPN and maybe SVCB can stand in for version negotiation, with the bonus that these are specifically designed to avoid adding latency. HTTP APIs can move to a different URL.</p>
<p>As this seems solid, I now have the task of writing a brief summary of this conclusion for the next revision of the “use it or lose it” draft. That might take some time as there are a few <a href="https://github.com/intarchboard/use-it-or-lose-it/issues">open issues</a> that need some attention.</p>
<hr class="footnotes-sep" />
<section class="footnotes">
<ol class="footnotes-list">
<li id="fn1" class="footnote-item"><p>Not electronic dance music sadly, it’s about Evolvability, Deployability, & Maintainability of Internet protocols <a href="https://lowentropy.net/posts/vn/#fnref1" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn2" class="footnote-item"><p>UDP maybe. UDP is simple enough that it doesn’t have <s>features</s>/bugs. Not to say that it is squeaky clean, it has plenty of baggage, with checksum issues, a reputation for being used for DoS, and issues with flow termination in NATs. <a href="https://lowentropy.net/posts/vn/#fnref2" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn3" class="footnote-item"><p><a href="https://boringssl.googlesource.com/boringssl/">BoringSSL</a>, which is now used by a few others, including Cloudflare and Apple. <a href="https://lowentropy.net/posts/vn/#fnref3" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn4" class="footnote-item"><p><a href="https://tools.ietf.org/html/rfc6709#section-4.1">Section 4.1 of RFC 6709</a> contains some great advice on how to design a version negotiation scheme, so that you can learn from experience. Though pay attention to the disclaimer in the last paragraph. <a href="https://lowentropy.net/posts/vn/#fnref4" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn5" class="footnote-item"><p>No one on the call was paying sufficient attention at the time, so we don’t know precisely why. We intend to find out, of course. <a href="https://lowentropy.net/posts/vn/#fnref5" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn6" class="footnote-item"><p>At the time, there was still reasonable cause to think that IP wouldn’t be the only network layer protocol, so other values were being used routinely. <a href="https://lowentropy.net/posts/vn/#fnref6" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn7" class="footnote-item"><p>You might rightly observe here that ALPN was brand new for HTTP/2, so the mechanism itself wasn’t exactly proven. This is true, but there are mitigating factors. The negotiation method is exactly the same as many other TLS extensions. And we tested the mechanism thoroughly during HTTP/2 deployment as each new revision from the -04 draft onwards was deployed widely with a different ALPN string. By the time HTTP/2 shipped, ALPN was definitely solid. <a href="https://lowentropy.net/posts/vn/#fnref7" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn8" class="footnote-item"><p>There is probably enough material for a long post on why this is not a problem in JSON, but I’ll just assert for now - without support - that there really is only one viable extension point in any JSON usage. <a href="https://lowentropy.net/posts/vn/#fnref8" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn9" class="footnote-item"><p>It doesn’t seem like TLS or QUIC can avoid having version negotiation. <a href="https://lowentropy.net/posts/vn/#fnref9" class="footnote-backref">↩︎</a></p>
</li>
</ol>
</section>
Thoughts on TAG Design Reviews2023-11-21T00:00:00Zhttps://lowentropy.net/posts/tag2023/<p>Before I start on my thoughts, if you work for a W3C member organization, please
head to <a href="https://www.w3.org/2023/10/tag-nominations">the 2023 TAG Election
page</a>. Voting is open until
2023-12-14.</p>
<p>If you are considering how you might like to rank me when voting, read on. I
can’t promise that this post will provide much additional context, but it might.</p>
<hr />
<p>The W3C <acronym title="Technical Architecture Group">TAG</acronym> is a bit of
a strange institution. The TAG occupies a position of some privilege due to its
standing within the W3C and the long-standing participation and sponsorship of
Sir Tim Berners-Lee.</p>
<p>The TAG also has a history marked by notable documents produced under its
letterhead. The TAG, through its <a href="https://tag.w3.org/findings/">findings</a>, has
been responsible for recognizing and analyzing certain key trends in the
evolution of the Web, providing some key pieces of architectural guidance. The
TAG also publishes documents with general guidance for people seeking to improve
the Web, like <a href="https://w3ctag.github.io/design-principles/">design principles</a>
and a <a href="https://www.w3.org/TR/security-privacy-questionnaire/">security and privacy
questionnaire</a>.</p>
<p>On a day-to-day basis, however, the TAG provides hands-on guidance to people
looking to add new capabilities to the Web, primarily through <a href="https://github.com/w3ctag/design-reviews">design
reviews</a>. Records of early reviews
trace back to 2013 in the TAG repository, but the practice has deeper roots.</p>
<p>The modern review record starts with a meager 5 reviews in the latter half of
2013. More recently, the TAG closed a total of 85 design reviews in 2022<sup class="footnote-ref"><a href="https://lowentropy.net/posts/tag2023/#fn1" id="fnref1">[1]</a></sup>. Already, in 2023, there have been 106 design review
requests opened.</p>
<p>The function of the TAG as a body primarily focused on reviewing new Web APIs is
one that took a while to settle. A key driver of this increase in volume has
clearly been the inclusion of TAG review as a formal precondition for shipping
Web-visible changes in the <a href="https://www.chromium.org/blink/launching-features/">Chromium
project</a>. Chromium
consequently drives a lot of this review load with 73 of the 106 new requests
that arrived in 2023 clearly marked as originating from “Google”, “Chromium”, or
“Microsoft” as a primary driver or funder of the work<sup class="footnote-ref"><a href="https://lowentropy.net/posts/tag2023/#fn2" id="fnref2">[2]</a></sup>.
That is nearly 70% of the total review load attributed to Chromium. This is in
addition to those design reviews that were initiated on behalf of a W3C group in
which Chromium contributors were instrumental in the work.</p>
<p>Obviously, at a rate of more than 2 reviews a week, that’s a fairly major outlay
in terms of time for the TAG. Proposals vary in size, but some of them are
quite substantial. A good review requires reading lengthy explainers and
specifications, filling gaps in understanding by talking to different people,
considering alternative options, and building an understanding of the broader
context. A proper review for a more substantial proposal can take weeks or even
months to research, discuss, and write up.</p>
<p>The TAG is <a href="https://github.com/w3c/AB-memberonly/issues/171">expanding in size</a>
this year. An increase to 12 members (8 elected, 4 appointed) does give the TAG
more capacity, albeit with added coordination costs reducing efficiency. This
is predicated on the idea is that reviews are the most important function of the
TAG. That being the case, then adding more capacity seems like a reasonable
reaction.</p>
<hr />
<p>That an action is superficially reasonable is not the standard to apply when
making such a decision. As with a design review, an examination of the
alternatives is generally illuminating. Once those alternatives are understood,
we might again conclude that the proposal on the table is the best possible
path, but we do so with a more complete understanding of what opportunities are
lost or foreclosed as a result. The <a href="https://www.w3.org/2023/05/12-ab-minutes.html#t09">AB
minutes</a> of the decision do
not reflect that process, but then they are only responding to a request from
the TAG.</p>
<p>There are several other equally reasonable ways of dealing with increased
workload. If reviews are taking too long, it might be possible to find ways to
make reviewing easier or faster. Perhaps the TAG has exhausted their options in
that area already. Maybe they have looked at selectively rejecting more design
review requests. Maybe they have considered finding ways to offload review work
onto other bodies, like <a href="https://www.w3.org/Privacy/IG/">PING</a>.</p>
<p>From my limited perspective, it is not clear that these avenues of investigation
have been fully explored. For instance, I have good experience with effective
directorate system that the <acronym title="Internet Engineering Steering
Group">IESG</acronym> uses to greatly alleviate their workload, but I see no
evidence of an effort to delegate in a similar fashion.</p>
<p>TAG members each volunteer time from their ordinarily busy day jobs, so any
excess load spent on reviewing is time that is not available for higher
functions. In addition to review load, the TAG has a role in <a href="https://www.w3.org/2023/Process-20231103/#council">W3C
Councils</a> and other critical
procedural functions in the W3C process. Those tasks are generally not easily
delegated or dealt with by a subset of TAG members.</p>
<p>I am supportive of efforts to better use the TAG in for key procedural
functions, like the W3C Council. Those functions make the TAG more important in
a good way. The W3C needs people in the role who have good judgment and the
experience to inform that judgment.</p>
<p>Along with that, it is important to reserve some space for the TAG to provide
technical leadership for the W3C and the Web community as a whole. After time
spent on the procedural functions demanded by the process, design reviews have
the potential to completely drain any time TAG members have to dedicate to the
role, leaving no spare capacity. Ideally, there needs to be some remaining
space for the careful and thoughtful work that leadership demands.</p>
<p>Effective technical leadership depends somewhat on the TAG being exposed to how
the platform is evolving. Reviews are a great way to gain some of that
exposure, but that does not mean that the TAG needs to review every single
proposal.</p>
<p>I don’t have a specific plan yet. If appointed, it will take some time to
understand what the role is and what options are available. I consider myself
quite capable of performing that sort of review and I expect it would be easy to
settle into that function. But I have no intent of letting design reviews
dominate my time; the TAG – and the Web – deserves better.</p>
<hr class="footnotes-sep" />
<section class="footnotes">
<ol class="footnotes-list">
<li id="fn1" class="footnote-item"><p>A note
on the numbers here: The TAG has a template that they use for design reviews and
I have only selected reviews that include the string “requesting a TAG review”,
as present in that template. There were other issues closed in this period,
some of which are probably also pre-template design reviews, but I haven’t
carefully reviewed those. <a href="https://lowentropy.net/posts/tag2023/#fnref1" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn2" class="footnote-item"><p>For posterity, this is
the search I used: <code>opened_since(2023-01-01) not(opened_since(2024-01-01)) body("requesting a TAG review") body("(?:driving the (?:design|specification)|funded by):\\s+\\[?(?:Microsoft|Google|Chromium)"))</code>,
using <a href="https://github.com/MikeBishop/archive-repo/tree/main/post-processing/viewer">a tool I
built</a>
in combination with the excellent <a href="https://github.com/MikeBishop/archive-repo">GitHub issue archival
tool</a> that Mike Bishop wrote. <a href="https://lowentropy.net/posts/tag2023/#fnref2" class="footnote-backref">↩︎</a></p>
</li>
</ol>
</section>
Standardizing Principles2021-01-05T00:00:00Zhttps://lowentropy.net/posts/standard-principles/<p>There is a perennial question in standards development about the value of the different artefacts that the process kicks out.</p>
<p>One subject that remains current is the relative value of specifications against things like compliance testing frameworks. Reasonable people tend to place different weight on tests, with a wide range of attitudes. In the past, more people were willing to reject attempts to invest in any shared test or compliance infrastructure.</p>
<p>In recent years however, it has become very clear that a common test infrastructure is critical to developing a high quality standard. Developing tests in conjunction with the standardization effort has improved the quality of specifications and implementations a great deal.</p>
<p>Recently, I encountered an example where a standards group deliberately chose not to document behaviour, relying exclusively on the common test framework. Understanding what is lost when this</p>
<h2 id="background">Background</h2>
<p>My experience with compliance testing in standards development is patchy. It might help to describe how these have worked out.</p>
<h3 id="standardize-first">Standardize First</h3>
<p>Some of the early projects I was involved in relied on testing being entirely privately driven. This can lead to each team relying almost exclusively on tests they develop internally. Occasional pairwise interoperability testing occurs, but it is ad hoc and unreliable.</p>
<p>This loose arrangement does tend to result in specifications being published sooner. The cost is in less scrutiny, especially when it comes to details, so the quality of the output is not as good as it could be.</p>
<p>This doesn’t mean that there is no compliance testing, but it requires effort. That effort can pay off, as I have seen with <a href="https://crossbar.io/autobahn/">WebSockets</a>, <a href="https://en.wikipedia.org/wiki/FIPS_140">FIPS-140</a>, <a href="https://cache-tests.fyi/">HTTP Caching</a>, and others.</p>
<h3 id="implement-in-parallel">Implement in Parallel</h3>
<p>My experience with <a href="https://tools.ietf.org/html/rfc7540">HTTP/2</a> was not a whole lot different to those early projects. The major improvement there was the level of active engagement from implementers in developing the specification.</p>
<p>This process did not involve active development of a compliance testing framework, but there were regular interoperability tests. I still remember Jeff Pinner deploying <a href="https://tools.ietf.org/html/draft-ietf-httpbis-http2-04">draft -04</a> to production on <a href="https://twitter.com/">twitter.com</a> during a meeting. Not everyone was so fearless, but live deployment was something we saw routinely in the 13 subsequent drafts it took finalize the work.</p>
<p>Good feedback from implementations was key to the success of HTTP/2, which now drives <a href="https://mzl.la/35bToXH">well over half of the HTTP requests in Firefox</a>.</p>
<p>The excellent <a href="https://github.com/summerwind/h2spec">h2spec</a> came out a little after the release of the specification. It has since become a valuable compliance testing tool for implementers.</p>
<h3 id="test-in-parallel">Test in Parallel</h3>
<p><a href="https://tools.ietf.org/html/rfc8446">TLS 1.3</a> followed a similar trajectory to HTTP/2, with a few interesting twists. Part of the testing that occurred during the development of the protocol was formal verification. For example, <a href="https://tls13tamarin.github.io/TLS13Tamarin/docs/tls13tamarin.pdf">a Tamarin model of TLS 1.3</a> was developed alongside the protocol, which both informed the design and provided validation of the design. Some implementations automated compliance testing based on <a href="https://boringssl.googlesource.com/boringssl/+/refs/heads/master/ssl/test/runner/">a tool developed for BoringSSL</a>, which turned out to be very useful.</p>
<p>With <a href="https://quicwg.org/base-drafts/draft-ietf-quic-transport.html">QUIC</a>, Marten Seeman and Jana Iyengar developed <a href="https://github.com/marten-seemann/quic-interop-runner/">a framework</a> that automates testing between QUIC implementations. This runs regularly and produces <a href="https://interop.seemann.io/">a detailed report</a> showing how each implementation stands up under a range of conditions, some of them quite adversarial. This has had a significant positive effect on the quality of both implementations and specifications<sup class="footnote-ref"><a href="https://lowentropy.net/posts/standard-principles/#fn1" id="fnref1">[1]</a></sup>.</p>
<p>Overall, I can see no way of going back to anything less. In all cases, tests have been so valuable that there is no way I would go back to a world without them.</p>
<h3 id="test-first">Test First</h3>
<p>Of course, no mention of testing would be complete with remiss here not to mention the excellent <a href="https://web-platform-tests.org/">Web Platform Tests</a>, which are now critical parts of the process adopted by the <a href="https://whatwg.org/working-mode#changes">WHATWG</a> and <a href="https://www.w3.org/2019/05/webapps-charter.html">some W3C groups</a>. Web Platform Tests are considered a prerequisite for normative specification changes under these processes.</p>
<p>Akin to <a href="https://en.wikipedia.org/wiki/Test-driven_development">test driven development</a>, this ensures that new features and changes are not just testable, but tested, before anything is documented. In practice the work continues in parallel, with tight feedback between development, specification, and testing. Shorter feedback cycles means that work can be completed faster and with higher quality.</p>
<h2 id="the-role-of-specifications">The Role of Specifications</h2>
<p>An obvious question that might be asked when it comes to this process, particularly where there are firm requirements for tests, is what value the specification provides. Given sufficiently thorough testing, it should be possible to construct an interoperable implementation based solely on those tests.</p>
<p>To go further, when <a href="https://w3c.github.io/ServiceWorker/#cache-storage-match">specifications consist of mostly code-like constructs</a> and real implementations are open source anyway, the value of a specification seems greatly diminished. As empirical observation of how things actually work is of more value to how they work in theory, it is reasonable to ask what value the specification provides.</p>
<p>As my own recent experience with the <a href="https://en.wikipedia.org/wiki/CUBIC_TCP">Cubic congestion control algorithm</a> taught me, what is implemented and deployed is what matters. <a href="https://tools.ietf.org/html/rfc8312">The RFC that purports to document Cubic</a> is not really implementable and barely resembles <a href="https://github.com/torvalds/linux/blob/fcadab740480e0e0e9fa9bd272acd409884d431a/net/ipv4/tcp_cubic.c">what real implementations do</a>.</p>
<p>So if testing is a central part of the development of new standards and people rely increasingly on tests or observing the behaviour of other implementations, it is reasonable to question what value specifications provide.</p>
<h3 id="a-specification-can-teach">A Specification Can Teach</h3>
<p>Specification documents often come with a bunch of normative language. Some of the most critical text defines what it means to be conformant, describing what is permitted and what is forbidden in precise terms.</p>
<p>Strictly normative text is certainly at risk from displacement from good testing. But there is often a bunch of non-normative filler in specifications. Though text might be purely informative, it is often of significant value to people who are attempting to understand the specification in detail:</p>
<ul>
<li>
<p>Informative text can motivate the existence of the specification.</p>
</li>
<li>
<p>Filler can provide insights into why things are.</p>
</li>
<li>
<p>Notes can point to outcomes results that might not be obvious.</p>
</li>
</ul>
<p>For specifications that are developed using <a href="https://open-stand.org/about-us/principles/">an open process</a>, much of this information is not hidden, but it can be difficult to find<sup class="footnote-ref"><a href="https://lowentropy.net/posts/standard-principles/#fn2" id="fnref2">[2]</a></sup>. Presenting timely, relevant information to readers is useful in putting things into context.</p>
<h3 id="a-specification-can-capture-other-forms-of-agreement">A Specification Can Capture Other Forms Of Agreement</h3>
<p>One of the hardest lessons out of recent standards work has been the realization that many decisions are made with only superficial justification. Developing standards based on shared principles is much harder than agreeing on what happens in certain conditions, or which bit goes where.</p>
<p>Though it might be harder, reaching agreement<sup class="footnote-ref"><a href="https://lowentropy.net/posts/standard-principles/#fn3" id="fnref3">[3]</a></sup> on principles is far more enduring and valuable. A specification can document that agreement.</p>
<p>Reaching agreement or consensus on a principle can be hard for a variety of reasons:</p>
<ul>
<li>
<p>Dealing with abstractions can be challenging because people can develop different abstract models based on their own perspective and biases. Subtle differences can mean a lot of talking past each other.</p>
</li>
<li>
<p>Abstractions can also become too far removed from reality to be useful. This might serve you well when filing a patent application, but ultimately we depend on principles being applicable to the current work<sup class="footnote-ref"><a href="https://lowentropy.net/posts/standard-principles/#fn4" id="fnref4">[4]</a></sup>.</p>
</li>
<li>
<p>Agreement on principles can be difficult because it forces people to fully address differences of opinion.</p>
</li>
</ul>
<p>Without first addressing disagreements in principle, it is possible that concrete decisions could be consistent with different perspectives. This might not have any immediate effect, but could produce inconsistencies. Some inconsistency can result in real problems, especially if it becomes necessary to rely more extensively on a principle that was in contention<sup class="footnote-ref"><a href="https://lowentropy.net/posts/standard-principles/#fn5" id="fnref5">[5]</a></sup>.</p>
<p>However hard agreement might be to achieve, a principle that is agreed can inform multiple decisions. Documenting a principle that has achieved agreement can therefore more efficient over time. Documentation can also help avoid application of inconsistent or conflicting principles over time.</p>
<p>Documenting principles does not have a direct normative effect. But a specification offers an opportunity to document more than just conformance requirements, it can capture other types of agreement.</p>
<h3 id="conformance-test-suites-can-overreach">Conformance Test Suites Can Overreach</h3>
<p>A problem that can occur with conformance testing is that the tests can disagree with specifications. If implementations depend more on the test than the specification, this can make the conformance test the true source of the definition of what it means to interoperate.</p>
<p>This is not inherently bad. It can be that the tests capture something that is inherently better, because it reflects what people need, because it is easier to implement, or just because that is what interoperates.</p>
<p>Of course, disagreement between two sources that claim authority does implementations a disservice. A new implementation now has to know which is “correct”. Ensuring that deployments, tests, and specifications align is critical to ensuring the viability of new implementations.</p>
<p>The true risk with relying on tests is the process by which conformance tests are maintained. Specification development processes are burdened with rules that govern how agreement is reached. Those rules exist for good reason.</p>
<p>Change control processes for conformance testing projects might not provide adequate protection for anti-trust or intellectual property. They also might lack opportunities for affected stakeholders to engage. This doesn’t have to be the case, but the governance structures underpinning most conformance suites is usually less robust than that of standards<sup class="footnote-ref"><a href="https://lowentropy.net/posts/standard-principles/#fn6" id="fnref6">[6]</a></sup>.</p>
<h2 id="conclusions">Conclusions</h2>
<p>The exact nature of how specifications are used to guiding the development of interoperable standards is something of a fluid situation. Here I’ve laid out a case for the value of specifications: for the non-normative language they provide, for their ability to capture agreement on more than just normative functions, and for the governance structures that they use. There are probably other reasons too, and likely counter-arguments, both of which I would be delighted to hear about.</p>
<hr class="footnotes-sep" />
<section class="footnotes">
<ol class="footnotes-list">
<li id="fn1" class="footnote-item"><p>I should also point at <a href="https://quic-tracker.info.ucl.ac.be/grid">QUIC Tracker</a> and <a href="http://d.hatena.ne.jp/kazu-yamamoto/">Kazu Yamamoto</a> has also started work on an <a href="https://github.com/kazu-yamamoto/h3spec">h3spec</a>, both of which have made significant contributions too. <a href="https://lowentropy.net/posts/standard-principles/#fnref1" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn2" class="footnote-item"><p>For example, the development of even a relatively small specification like QUIC involved more than 4000 issues and pull requests, more than 8000 email messages, not to mention all the chat messages that are not in public archives. <a href="https://lowentropy.net/posts/standard-principles/#fnref2" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn3" class="footnote-item"><p>…or consensus if that is how you spell it. <a href="https://lowentropy.net/posts/standard-principles/#fnref3" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn4" class="footnote-item"><p>This is perhaps a criticism that might be levelled at <a href="https://w3ctag.github.io/design-principles/#priority-of-constituencies">the priority of constituencies</a> or text like that in <a href="https://tools.ietf.org/html/rfc8890">RFC 8890</a>. However, these might be more correctly viewed as meta-principles, or ideals that guide the development of more specific and actionable principles. <a href="https://lowentropy.net/posts/standard-principles/#fnref4" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn5" class="footnote-item"><p>An example of this might be DNS, where the need for agreement on principles was neglected. As such, the global community has no documented principles that might guide decisions on issues such as having a single global namespace or whether network operators are entitled to be involved in name resolution. Now that encrypted DNS is being rolled out, reflective of a principle that values individual privacy, it is become obvious that people differing views but no shared principles have been coexisting. <a href="https://lowentropy.net/posts/standard-principles/#fnref5" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn6" class="footnote-item"><p>Not that these too lack opportunities for improvement, but they are the best we have. <a href="https://lowentropy.net/posts/standard-principles/#fnref6" class="footnote-backref">↩︎</a></p>
</li>
</ol>
</section>
Oblivious DoH2020-12-09T00:00:00Zhttps://lowentropy.net/posts/odoh/<p>Today we heard <a href="https://blog.cloudflare.com/oblivious-dns/">an announcement</a> that Cloudflare, Apple, and Fastly are collaborating on a new technology for improving privacy of DNS queries using a technology they call Oblivious DoH (ODoH).</p>
<p>This is an exciting development. This posting examines the technology in more detail and looks at some of the challenges this will need to overcome before it can be deployed more widely.</p>
<h2 id="how-odoh-provides-privacy-for-dns-queries">How ODoH Provides Privacy for DNS Queries</h2>
<p><a href="https://tools.ietf.org/html/draft-pauly-dprive-oblivious-doh-03">Oblivious DoH</a> is a simple <a href="https://en.wikipedia.org/wiki/Mix_network">mixnet</a> protocol for making DNS queries. It uses a proxy server to provide added privacy for query streams.</p>
<p>This looks something like:</p>
<pre><code class="language-graphviz">digraph ODoH {
graph [overlap=true, splines=line, nodesep=1.0, ordering=out];
node [shape=rectangle, fontname=" "];
edge [arrowhead=none];
{ rank=same; Client->Proxy; Proxy->Resolver; }
}
</code></pre>
<p>A common criticism of <a href="https://tools.ietf.org/html/rfc8484">DNS over HTTPS</a> (DoH) is that it provides DoH resolvers with lots of privacy-sensitive information<sup class="footnote-ref"><a href="https://lowentropy.net/posts/odoh/#fn1" id="fnref1">[1]</a></sup>. Currently all DNS resolvers, including DoH resolvers, see the contents of queries and can link that to who is making those queries. DoH includes connection reuse, so resolvers can link requests from the same client using the connection.</p>
<p>In Oblivious DoH, a proxy aggregates queries from multiple clients so that the resolver is unable to link queries to individual clients. ODoH protects the IP address of the client, but it also prevents the resolver from linking queries from the same client together. Unlike an ordinary HTTP proxy, which handle TLS connections to servers<sup class="footnote-ref"><a href="https://lowentropy.net/posts/odoh/#fn2" id="fnref2">[2]</a></sup>, ODoH proxies handle queries that are individually encrypted.</p>
<p>ODoH prevents resolvers from assembling profiles on clients by collecting the queries they make, because resolvers see queries from a large number of clients all mixed together.</p>
<p>An ODoH proxy learns almost nothing from this process as ODoH uses <a href="https://tools.ietf.org/html/draft-irtf-cfrg-hpke-06">HPKE</a> to encrypt the both query and answer with keys chosen by the client and resolver.</p>
<p>The privacy benefits of ODoH can only be undone if both the proxy and resolver cooperate. ODoH therefore recommends that the two services be run independently, with the operator of each making a commitment to respecting privacy.</p>
<h2 id="costs">Costs</h2>
<p>The privacy advantages provided by the ODoH design come at a higher cost than DoH, where a client just queries the resolver directly:</p>
<ul>
<li>The proxy adds a little latency as it needs to forward queries and responses.</li>
<li>HPKE encryption adds up to about 100 bytes to each query.</li>
<li>The client and resolver need to spend a little CPU time to add and remove the encryption.</li>
</ul>
<p>Cloudflare’s tests show that the overall effect of ODoH on performance is quite modest. These early tests even suggest some improvement for the slowest queries. If those performance gains can be kept as they scale up their deployment, that would be strong justification for deployment.</p>
<h2 id="why-this-design">Why This Design</h2>
<p>A similar outcome might be achieved using a proxy that supports HTTP CONNECT. However, in order to avoid the resolver from learning which queries come from the same client, each query would have to use a new connection.</p>
<p>That gets pretty expensive. While you might be able to tricks to drive down latency like sending the TLS handshake with the HTTP CONNECT, it means that every request uses a separate TCP connection and a round trip to establish the connection<sup class="footnote-ref"><a href="https://lowentropy.net/posts/odoh/#fn3" id="fnref3">[3]</a></sup>.</p>
<p>It is also possible to use something like <a href="https://www.torproject.org/">Tor</a>, which provides superior privacy protection. Tor is a lot more expensive.</p>
<p>Using HPKE and a multiplexed protocol like <a href="https://tools.ietf.org/html/rfc7540">HTTP/2</a> or <a href="https://quicwg.org/base-drafts/draft-ietf-quic-http.html">HTTP/3</a> avoids per-query connection setup costs. However, the most important thing is that it involves only minimal additional latency to get the privacy benefits<sup class="footnote-ref"><a href="https://lowentropy.net/posts/odoh/#fn4" id="fnref4">[4]</a></sup>.</p>
<h2 id="key-management-in-dns">Key Management in DNS</h2>
<p>The proposal puts HPKE keys for the resolver in the DNS<sup class="footnote-ref"><a href="https://lowentropy.net/posts/odoh/#fn5" id="fnref5">[5]</a></sup>. The idea is that clients can talk to the resolver directly to get these, then use that information to protect its queries. As the keys are DNS records, they can be retrieved from any DNS resolver, which is a potential advantage.</p>
<p>This also means that this ODoH design depends on DNSSEC. Many clients rely on their resolver to perform DNSSEC validation, which doesn’t help here. So this makes it difficult to deploy something like this incrementally in clients.</p>
<p>A better option might be to offer the HPKE public key information in response to a direct HTTP request to the resolver. That would ensure that the key could be authenticated by the client using HTTPS and the Web PKI.</p>
<h2 id="trustworthiness-of-proxies">Trustworthiness of Proxies</h2>
<p>Both client and resolver will want to authenticate the proxy and only allow a trustworthy proxy. The protocol design means that the need for trust in the proxy is limited, but it isn’t zero.</p>
<p>Clients need to trust that the proxy is hiding their IP address. A bad proxy could attach the client IP address to every query they forward. Clients will want some way of knowing that the proxy won’t do this<sup class="footnote-ref"><a href="https://lowentropy.net/posts/odoh/#fn6" id="fnref6">[6]</a></sup>.</p>
<p>Resolvers will likely want to limit the number of proxies that they will accept requests from, because the aggregated queries from a proxy of any reasonable size will look a lot like a denial of service attack. Mixing all the queries together denies resolvers the ability to do per-client rate limiting, which is a valuable denial of service protection measure. Resolvers will need to apply much more generous rate limits for these proxies and trust that the proxies will take reasonable steps to ensure that individual clients are not able to generate abusive numbers of queries.</p>
<p>This means that proxies will need to be acceptable to both client and resolver. Early deployments will be able to rely on contracts and similar arrangements to guarantee this. However, if use of ODoH is to scale out to support large numbers of providers of both proxies and resolvers, it could be necessary to build systems for managing these relationships.</p>
<h2 id="proxying-for-other-applications">Proxying For Other Applications</h2>
<p>One obvious thing with this design is that it isn’t unique to DNS queries. In fact, there are a large number of request-response exchanges that would benefit from the same privacy benefits that ODoH provides. For example, Google this week announced <a href="https://blog.chromium.org/2020/12/continuing-our-journey-to-bring-instant.html">a trial</a> of a similar technology for preloading content.</p>
<p>A generic design that enabled protection for HTTP queries of any sort would be ideal. My hope is that we can design that protocol.</p>
<p>Once you look to designing a more generic solution, there are a few extra things that might improve the design. Automatic discovery of HTTP endpoints that allow oblivious proxying is one potential enhancement. Servers could advertise both keys and the proxies they support so that clients can choose to use those proxies to mask their address. This might involve automated proxy selection or discovery and even systems for encoding agreements. There are lots of possibilities here.</p>
<h2 id="centralization">Centralization</h2>
<p>One criticism regarding DoH deployments is that they encourage consolidation of DNS resolver services. For ODoH - at least in the short term - options for ODoH resolvers will be limited, which could push usage toward a small number of server operators in exchange for the privacy gains ODoH provides.</p>
<p>During initial roll-out, the number of proxy operators will be limited. Also, using a larger proxy means that your queries are mixed in with more queries from other people, providing marginally better privacy. That might provide some impetus to consolidate.</p>
<p>Deploying automated discovery systems for acceptable proxies might help mitigate the worst centralization effects, but it seems likely that this will not be a feature of early deployments.</p>
<p>In the end, it would be a mistake to cry “centralization” in response to early trial deployments of a technology, which are naturally limited in scope. Furthermore, it’s hard to know what the long term impact on the ecosystem will be. We might never be able to separate the effect of existing trends toward consolidation from the effect of new technology.</p>
<h2 id="conclusion">Conclusion</h2>
<p>I like the model adopted here. The use of a proxy neatly addresses one of the biggest concerns with the rollout of DoH: the privacy risk of having a large provider being able to gather information about streams of queries that can be linked to your IP address.</p>
<p>ODoH breaks streams of queries into discrete transactions that are hard to assemble into activity profiles. At the same time, ODoH makes it hard to attribute queries to individuals as it hides the origin of queries.</p>
<p>My sense is that the benefits very much outweigh the performance costs, the protocol complexity, and the operational risks. ODoH is a pretty big privacy win for name resolution. The state of name resolution is pretty poor, with much of it still unprotected from snooping, interception, and poisoning. The deployment of DoH went some way to address that, but came with some drawbacks. Oblivious DoH takes the next logical step.</p>
<hr class="footnotes-sep" />
<section class="footnotes">
<ol class="footnotes-list">
<li id="fn1" class="footnote-item"><p>This is something all current DNS resolvers get, but the complaint is about the scale at which this information is gathered. Some people are unhappy that network operators are unable to access this information, but I regard that as a feature. <a href="https://lowentropy.net/posts/odoh/#fnref1" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn2" class="footnote-item"><p>OK, proxies do handle individual, unencrypted HTTP requests, but that capability is hardly ever used any more now that 90% of the web is HTTPS. <a href="https://lowentropy.net/posts/odoh/#fnref2" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn3" class="footnote-item"><p>Using 0-RTT doesn’t work here without some fiddly changes to TLS because the session ticket used for TLS allows the server to link connections together, which isn’t what we need. <a href="https://lowentropy.net/posts/odoh/#fnref3" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn4" class="footnote-item"><p>This also makes ODoH far more susceptible to <a href="https://en.wikipedia.org/wiki/Traffic_analysis">traffic analysis</a>, but it relies on volume and the relative similarity of DNS queries to help manage that risk. <a href="https://lowentropy.net/posts/odoh/#fnref4" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn5" class="footnote-item"><p>The recursion here means that the designers of ODoH probably deserve a prize of some sort. <a href="https://lowentropy.net/posts/odoh/#fnref5" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn6" class="footnote-item"><p>The <a href="https://github.com/bslassey/ip-blindness">willful IP blindness proposal</a> goes into more detail on what might be required for this. <a href="https://lowentropy.net/posts/odoh/#fnref6" class="footnote-backref">↩︎</a></p>
</li>
</ol>
</section>
RFCs in HTML2020-12-18T00:00:00Zhttps://lowentropy.net/posts/line-length/<p>I spend a shocking amount of my time staring at IETF documents, both Internet-Drafts and RFCs. I have spend quite a bit of time looking at GitHub README files and W3C specifications.</p>
<p>For reading prose, the format I routinely find to be the most accessible is the text versions. This is definitely not based on the quality of the writing, all of these formats produce unreadable documents. What I refer to here is not the substance, but the form. That is, how the text is laid out on my screen<sup class="footnote-ref"><a href="https://lowentropy.net/posts/line-length/#fn1" id="fnref1">[1]</a></sup>.</p>
<p>There is clearly a degree of familiarization and bias involved in this. A little while ago, I worked out that there is just one thing that elevates that clunky text format above the others: line length.</p>
<h2 id="relearning-old-lessons">Relearning Old Lessons</h2>
<p>This is hardly a new insight. A brief web search will return <a href="https://practicaltypography.com/line-length.html">numerous</a> <a href="https://smad.jmu.edu/shen/webtype/linelength.html">articles</a> on the <a href="https://www.fonts.com/content/learning/fontology/level-2/text-typography/length-column-width">subject</a><sup class="footnote-ref"><a href="https://lowentropy.net/posts/line-length/#fn2" id="fnref2">[2]</a></sup>. All of them say the same thing: shorter lines are more readable.</p>
<p>I was unable to find a single print newspaper that didn’t take this advice to heart, if not to extremes<sup class="footnote-ref"><a href="https://lowentropy.net/posts/line-length/#fn3" id="fnref3">[3]</a></sup>. Some magazines have ignored this, but those too turned out to be ill suited to reading prose and more geared toward looking at the pictures.</p>
<p>Recommendations from most sources put a hard stop somewhere around 80 characters. Some go a little lower or higher, but the general advice is pretty consistent. Of course, variable-width fonts make this imprecise, but it tends to average out.</p>
<h2 id="why-text-is-so-good">Why Text Is So Good</h2>
<p>I suppose that it is no accident that this corresponds to the width of the screen on a <a href="https://en.wikipedia.org/wiki/VT52">DEC 52</a>. The text format of old RFCs<sup class="footnote-ref"><a href="https://lowentropy.net/posts/line-length/#fn4" id="fnref4">[4]</a></sup> might have been to fit on these small screens, or it might have been to make printing easier, but the net effect is that you can get just <a href="https://tools.ietf.org/html/rfc7994#section-4.3">72 characters on a line</a>. The standard tools spend three of those on a left margin for text, so that means just 69 fixed-width characters per line.</p>
<p>That turns out to be very readable.</p>
<h2 id="why-html-is-so-bad">Why HTML Is So Bad</h2>
<p>The “official” HTML renderings of RFCs on <a href="https://www.rfc-editor.org/">rfc-editor.org</a> is a little wider than this. If I measure using whole alphabets<sup class="footnote-ref"><a href="https://lowentropy.net/posts/line-length/#fn5" id="fnref5">[5]</a></sup>, this results in a width of 98 characters. That’s more than the maximum in any recommendation I found.</p>
<p>Performing a similar test on the W3C specification style<sup class="footnote-ref"><a href="https://lowentropy.net/posts/line-length/#fn6" id="fnref6">[6]</a></sup> used for W3C publications, I got 102 characters. The WHATWG <a href="https://fetch.spec.whatwg.org/">Fetch Standard</a> had room for a massive 163 characters!</p>
<p>All of these wrap earlier than this on a smaller screen, but these are relatively small font sizes, so many screens will be wide enough to reach these values. Many people have a screen that has the 1300 horizontal pixels<sup class="footnote-ref"><a href="https://lowentropy.net/posts/line-length/#fn7" id="fnref7">[7]</a></sup> needed to get to 100 characters in a W3C specification. The official IETF HTML crams its 98 characters into just 724 pixels.</p>
<p>High text density comes from the font size and line height being quite small in official renderings of IETF documents. This compounds the problem as it makes tracking from one line to the next when reading more difficult. I consider the 14px/22.4px of the official IETF rendering to be positively tiny. I use a 9px (monospace) font in terminals, but I wouldn’t inflict that choice on others. That W3C and WHATWG settled on 16px/24px is far more humane, though with the selected font I still find this a little on the small side.</p>
<p>What is interesting here is that the text rendering on <a href="https://tools.ietf.org/html/">tools.ietf.org</a> uses a value of <code>13.33px</code>. This seems smaller, but - at least subjectively - it is no harder to read than the <code>16px</code> W3C/WHATWG specifications. Also, the default font configuration in Firefox is <code>16px</code> for most fonts and <code>13px</code> for monospace, suggesting that smaller font sizes are better tolerated for monospace fonts. That’s especially convenient here as it happens.</p>
<h2 id="making-html-readable">Making HTML Readable</h2>
<p>The fix is pretty simple, make the <code>max-width</code> small enough that lines don’t run so long. I set a value of <code>600px</code>. Combine this with a font size of <code>16px</code> and the result is a line length of 72<sup class="footnote-ref"><a href="https://lowentropy.net/posts/line-length/#fn8" id="fnref8">[8]</a></sup>.</p>
<p>The <a href="https://quicwg.org/base-drafts/draft-ietf-quic-transport.html">editor’s copy of the QUIC spec</a> is a fairly thorough example of this.</p>
<h3 id="fonts">Fonts</h3>
<p>I chose to change the font to something that is a little wider at the same time. Using <a href="https://docs.microsoft.com/en-us/typography/font-list/arial">Arial</a> - the default sans-serif font on Windows and the font chosen by the W3C and WHATWG - adds 4-5 characters to line length and is noticeably smaller on screen. <a href="https://docs.microsoft.com/en-us/typography/font-list/times-new-roman">Times New Roman</a> - the default serif font - adds 9-10 characters and is smaller again.</p>
<p><a href="https://fonts.google.com/specimen/Lora">Lora</a>, which has a light serif, was my choice for text. I know little enough about fonts that this was ultimately subjective. <a href="https://fonts.google.com/specimen/Noto+Sans">Noto Sans</a>, the font used in IETF official renderings, is comparable here, but I find it a little boring.</p>
<p>Some people don’t like the visual noise of a serif font for reading on a screen. Modern displays with high pixel density are less vulnerable to that and this is a light font with enough serif noise to add a little flair without adversely affecting readability. Lora is very readable at <code>16px</code>, where many other serif fonts require a larger size to be similarly clear.</p>
<h3 id="headings">Headings</h3>
<p>Fitting headings on a single line given the shorter line length turned out to be fiddly. I didn’t want headings to wrap, or to use too small a font. And IETF people have a deep and abiding love for <a href="https://quicwg.org/base-drafts/draft-ietf-quic-invariants.html#section-1">very long headings</a>. For this, a condensed font was ideal.</p>
<p>A semi-condensed font might have been ideal, but there are fewer of those and it was a little hard to find one that didn’t look too jarring next to the main text<sup class="footnote-ref"><a href="https://lowentropy.net/posts/line-length/#fn9" id="fnref9">[9]</a></sup>. Again Google Fonts was a great resource and <a href="https://fonts.google.com/specimen/Cabin+Condensed">Cabin Condensed</a> is OK.</p>
<h3 id="ascii-art">ASCII Art</h3>
<p>In setting this size, it is then necessary to consider the effect on diagrams. IETF documents are still stuck in the dark ages when it comes to diagrams and ASCII Art still dominates there. As the text format accepts 72 column text, so too must the figures in the HTML output.</p>
<p>This turns out to be a bit of a compromise. Styling of figures to include an offset from text, a border, and background shading eats up horizontal space. In the end, I managed to reduce the text size to <code>13.5px</code> and set <code>letter-spacing: -0.2px</code> to slightly compress the text further and fit 72 columns in<sup class="footnote-ref"><a href="https://lowentropy.net/posts/line-length/#fn10" id="fnref10">[10]</a></sup>.</p>
<h3 id="minimizing-distractions">Minimizing Distractions</h3>
<p>The styles used here are based on those from an earlier version of the official renderings. Once the major pieces were in place, the details need to be aligned to fit. After fixing major items like margins and line heights to match font and size choices, a bunch of work is needed to make documents look consistent. The first task was removing a bunch of design elements that I found distracting.</p>
<p>The HTML rendering includes a <a href="https://en.wikipedia.org/wiki/Pilcrow">pilcrow</a> at the end of each paragraph. This enables linking to specific paragraphs, which is a great feature.</p>
<p>The official styling only renders the pilcrow when the paragraph is hovered<sup class="footnote-ref"><a href="https://lowentropy.net/posts/line-length/#fn11" id="fnref11">[11]</a></sup>, but it renders very strongly when shown and so can be distracting. That needed softening.</p>
<p>The default blue (<code>#00f</code>) for links is strongly saturated, which is too assertive. Reducing the saturation makes links blend into text better.</p>
<p>Changing background colours on hover for titles is a nice way of indicating the presence of links, but that too was very strong. Making that lighter made moving the mouse less of a light show.</p>
<h3 id="cleanup">Cleanup</h3>
<p>Then there was a bunch of maintenance and tidying:</p>
<ul>
<li>Negative margins on headings, presumably to tweak the position of headings when following internal links to section headings, went<sup class="footnote-ref"><a href="https://lowentropy.net/posts/line-length/#fn12" id="fnref12">[12]</a></sup>.</li>
<li>Rules that were overwritten later in the file were consolidated.</li>
<li>The table of contents was moved closer to content.</li>
<li>Horizontal lines were given the flick.</li>
<li>Table and figure captions were tightened up.</li>
<li>Authors addresses were put into multiple columns.</li>
<li>The References section got a big cleanup too.</li>
<li>I use CSS variables (<code>var(--foo)</code>), which is a great feature.</li>
</ul>
<p>Finally, a bunch of work was put into making this look decent on a small screen.</p>
<h2 id="conclusion">Conclusion</h2>
<p>What I’ve learned from this is a newfound respect for the work designers do. My amateur fumbling here has helped me appreciate just how many detail work goes into making something like this look good.</p>
<p>Immense thanks are owed to Anitra Nottingham, who graciously provided feedback on earlier versions of this work. Those versions were obviously much worse. I also owe thanks to Mark Nottingham, James Gruessing, Adam Roach, Jeffrey Yasskin and those I’ve forgotten who each took the time to provide feedback and expertise.</p>
<p>None of this is truly professional. I’m still finding things that I don’t like. I’m still not happy with various pieces of spacing, for instance.</p>
<p>Even learning this much design is more of a curse than I’d like. I might not ace <a href="https://cantunsee.space/">cantunsee</a>, but I know enough to notice things like alignment issues and bad kerning<sup class="footnote-ref"><a href="https://lowentropy.net/posts/line-length/#fn13" id="fnref13">[13]</a></sup> now. I’m not sure that that has enriched my life all that much.</p>
<p>But the main thing remains: I can read these documents now. Cutting the line length was what did that. I now prefer HTML if it uses this stylesheet<sup class="footnote-ref"><a href="https://lowentropy.net/posts/line-length/#fn14" id="fnref14">[14]</a></sup>. The rest was just gravy.</p>
<p>The stylesheet can be found <a href="https://github.com/martinthomson/i-d-template/blob/main/v3.css">here</a>. Contributions are welcome. Anyone using <a href="https://github.com/martinthomson/i-d-template">my GitHub template</a> for generating Internet-Drafts already benefits from this work.</p>
<hr class="footnotes-sep" />
<section class="footnotes">
<ol class="footnotes-list">
<li id="fn1" class="footnote-item"><p>Reading from paper is not something I can countenance; the cost of in paper on my specification reading alone would be devastating and I like tree too much to do that to them. <a href="https://lowentropy.net/posts/line-length/#fnref1" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn2" class="footnote-item"><p>And those are just links from my browsing history <a href="https://lowentropy.net/posts/line-length/#fnref2" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn3" class="footnote-item"><p>So many hyphens… <a href="https://lowentropy.net/posts/line-length/#fnref3" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn4" class="footnote-item"><p>Officially, they are <a href="https://tools.ietf.org/html/rfc7991">all XML now</a> and only rendered to text or HTML. <a href="https://lowentropy.net/posts/line-length/#fnref4" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn5" class="footnote-item"><p>The way to do this is to find a paragraph and open it in browser developer tools. Add a style rule of <code>overflow: hidden</code> then modify the content to be “abcdef…” and repeat until the text cuts off. This follows the advice in <a href="https://practicaltypography.com/line-length.html">Butterick’s Practical Typography</a>. <a href="https://lowentropy.net/posts/line-length/#fnref5" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn6" class="footnote-item"><p>I tested the <a href="https://w3c.github.io/push-api/">Push API</a>, which uses <a href="https://github.com/w3c/respec">ReSpec</a>, but specifications using <a href="https://tabatkins.github.io/bikeshed/">Bikeshed</a> produced exactly the same result. <a href="https://lowentropy.net/posts/line-length/#fnref6" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn7" class="footnote-item"><p>Using the browser measure for pixel, which doesn’t correspond to dots on screen for devices with high pixel density. <a href="https://lowentropy.net/posts/line-length/#fnref7" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn8" class="footnote-item"><p>I wasn’t going for this deliberately, but that is how it worked out. <a href="https://lowentropy.net/posts/line-length/#fnref8" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn9" class="footnote-item"><p>In particular, I have this thing about the shape of ‘e’ and ‘a’. They can’t be dramatically different. <a href="https://lowentropy.net/posts/line-length/#fnref9" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn10" class="footnote-item"><p>The need for packing this tightly came when I discovered that pilcrows for figures were possible, but the official rendering put them on a blank line. That broke the document flow badly and I wanted space for those on the line as well. See <a href="https://quicwg.org/base-drafts/draft-ietf-quic-recovery.html#section-b.5-3">this example</a> for how that turned out. <a href="https://lowentropy.net/posts/line-length/#fnref10" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn11" class="footnote-item"><p>Is this an accessibility problem? I don’t know. <a href="https://lowentropy.net/posts/line-length/#fnref11" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn12" class="footnote-item"><p>I’ve learned that with CSS, like many other things, can lend itself easily to making small hacks. The net effect of introducing a hack is invariably that you have to add a whole bunch more corrective hacks in a death spiral. Avoid hacks. <a href="https://lowentropy.net/posts/line-length/#fnref12" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn13" class="footnote-item"><p>My 9 year old son finds <a href="https://www.brookes.net.au/">signs for this real estate company</a>, which seem deliberately bad, amusing. It’s clearly infectious. <a href="https://lowentropy.net/posts/line-length/#fnref13" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn14" class="footnote-item"><p>Mark Nottingham has <a href="https://mnot.github.io/I-D/http-grease/">a different stylesheet</a> that is also acceptable. He also uses a very nice font. <a href="https://lowentropy.net/posts/line-length/#fnref14" class="footnote-backref">↩︎</a></p>
</li>
</ol>
</section>
Fraud, Abuse, Fingerprinting, Privacy, and Openness2023-08-23T00:00:00Zhttps://lowentropy.net/posts/fraud/<p>Fraud and abuse online are pretty serious problems. How sites manage fraud is
something of a mystery to most people. Indeed, as this post will show, that’s
deliberate.</p>
<p>This post provides an outline of how fraud management operates. It looks at the
basic techniques that are used and the challenges involved. In doing so, it
explores the tension between fraud management and privacy.</p>
<p>Hopefully this post helps you understand why fingerprinting is bad for privacy;
why you should nevertheless be happy that your bank is fingerprinting you; and,
why efforts to replace fingerprinting are unlikely to change anything.</p>
<p>Fraud and abuse are a consequence of the way the Web works. Recognizing that
these are a part of the cost of a Web that values privacy, openness, and equity
is hard, but I can’t see a better option.</p>
<h2 id="what-sorts-of-fraud-and-abuse%3F">What sorts of fraud and abuse?</h2>
<p>This post concentrates on the conduct of fraud or abuse using online services.
Web-based services mostly, but mobile apps and similar services have similar
concerns.</p>
<p>The sorts of fraud and abuse of most interest are those that operate at scale.
One-off theft needs different treatment. Click fraud in advertising is a good
example. Click fraud is where a site seeks to convince advertisers that ads
have been shown to people in order to get more money. Click fraud is a constant
companion to the advertising industry, and one that is unlikely to ever go away.
Managing click fraud is an important part of participating in advertising, and
something that affects everyone that uses online services.</p>
<p>Outside of advertising, fraud management techniques<sup class="footnote-ref"><a href="https://lowentropy.net/posts/fraud/#fn1" id="fnref1">[1]</a></sup> are also used to manage the risk of fake accounts that
are created for fraud or abuse purposes. Online stores and banks also use fraud
management as part of an overall strategy for managing the risk of payment fraud
or theft.</p>
<p>This is a very high-level overview, so most of this document applies equally to
lots of different fraud and abuse scenarios. Obviously, each situation will be
different, but I’m glossing over the details.</p>
<aside>
<p>I find the parallels between fraud management and denial of service mitigation
interesting. I’ve called out some of those similarities and differences
throughout.</p>
</aside>
<h2 id="understanding-online-fraud-and-abuse">Understanding online fraud and abuse</h2>
<p>Let’s say that you have a site that makes some information or service available.
This site will attract clients, which we can split into two basic groups:
clients that the site wants to serve, and clients that the site does not want to
serve.</p>
<aside>
<p>Why the site does not want to serve the clients from the latter group does not
matter that much, but there are some common themes we tend to see.
Distinguishing between humans and bots is a very common goal.
<a href="https://en.wikipedia.org/wiki/CAPTCHA">CAPTCHAs</a> are supposed to be able to
distinguish this. Of course, CAPTCHAs have always had very poor accessibility
properties and increasingly, computers are better at solving CATCHAs than
humans<sup class="footnote-ref"><a href="https://lowentropy.net/posts/fraud/#fn2" id="fnref2">[2]</a></sup>.</p>
<p>That doesn’t stop sites from wanting to be able to pick out a bot. For
advertising cases, sites will want to serve humans – after all, bots are
unlikely to change their purchasing habits as a result of “viewing” an ad.
Similarly, sites that provide goods that are limited in quantity – such as theatre
tickets or limited run goods like sneakers<sup class="footnote-ref"><a href="https://lowentropy.net/posts/fraud/#fn3" id="fnref3">[3]</a></sup> – might prefer to ensure that their
inventory is only sold to people.</p>
</aside>
<p>The attacker in this model seeks to access the service for some reason. In
order to do so, the attacker attempts to convince sites that they are a real
client.</p>
<p>For click fraud, a site might seek to convince its advertising partners that ads
were shown to real people. The goal is to convince the advertiser to pay the
fraudlent site more money. Sophisticated click fraud can also involve faking
clicks or <a href="https://support.google.com/google-ads/answer/6365?hl=en">ad
conversions</a> in an
effort to falsely convince the advertiser that the ads on the fraudulent site
are more useful as they are responsible for sales.</p>
<p>An adversary rarely gains much by performing a single instance of fraud. They
will often seek to automate fraud, accessing the service as many times as
possible. Fraud at scale can be very damaging, but it also means that it is
easier to detect.</p>
<p>Automation allows fraud to be conducted at scale, but it also creates telltales:
signals that allow an attack to be recognized.</p>
<h3 id="detection">Detection</h3>
<p>Detection is the first stage for anyone looking to defeat fraud or abuse. To do
that, site operators will look for anomalies of any sort. Maybe the attack will
appear as an increase in incoming requests or a repetitive pattern of accesses.</p>
<p>Repetition might be a key to detecting fraud. An attacker might try to have
their attacks blend in with real humans that are also accessing the system. An
attacker’s ability to mimic human behaviour is usually limited, as they often
hope to execute many fraudulent transactions. Attackers have to balance the
risk that they are detected against the desire to complete multiple actions
before they are detected.</p>
<p>Detecting fraud and abuse relies on a range of techniques. Anti-fraud people
generally keep details of their methods secret, but we know that they use both
automated and manual techniques.</p>
<ul>
<li>
<p>Automated systems generally use machine learning that is trained on the
details of past attacks. This scales really well and allows for repeat
attacks to be detected quickly and efficiently.</p>
</li>
<li>
<p>Human experts can be better at recognizing new forms of attack. Attacks that
are detected by automated systems can be confirmed by humans before deploying
interventions.</p>
</li>
</ul>
<p>Of course, attackers are also constantly trying to adapt their techniques to
evade detection. Detecting an attack can take time.</p>
<h3 id="identification%2Fclassification">Identification/classification</h3>
<p>It is not enough to know that fraud is occurring. Once recognized, the pattern
of fraudulent behaviour needs to be classified, so that future attacks can be
recognized.</p>
<p>As noted, most fraud is automated in some way. Even if humans are involved, to
operate at any significant scale, even humans will be operating to a script.
Whether executed by machines or humans, the script will be designed to evade
existing defenses. This means that attacks need to be carefully scripted, which
can produce patterns. If a pattern can be found, attempts at fraud can be
distinguished from genuine attempts from people to visit the site.</p>
<p>Patterns in abuse manifest in one of two ways:</p>
<ol>
<li>
<p><em>Common software</em>. If attackers only use a specific piece of hardware or
software, then any common characteristics might be revealed by
fingerprinting. Even if the attacker varies some characteristics (like the
<code>User-Agent</code> header or similar obvious things), other characteristics might
stay the same, which can be used to recognize the attack. This is why
browser fingerprinting is a valuable tool for managing fraud.</p>
</li>
<li>
<p><em>Common practices</em>. Software or scripted interaction can produce fixed
patterns of behaviour that can be used to recognize an attempted attack.
Clues might exist in the timing of actions or the consistency of interaction
patterns. For instance, automated fraud might not exhibit the sorts of
variance in mouse movements that a diverse set of people could.</p>
</li>
</ol>
<p>The script that is followed by an attacker might try to vary some of these
things. However, unless the attack script is able to simulate the sorts of
diversity that real people do – which is unlikely – any resulting common
patterns can be used to identify likely attempts at fraud.</p>
<p>Once a pattern is established, future attempts can be recognized. Also, if
enough information has been recorded from past interactions, previously
undetected fraud might now be identifiable.</p>
<p>Learned patterns can sometimes be used on multiple sites. If an attack is
detected and thwarted on one site, similar attacks on other sites might be
easier to identify. Fraud and abuse detection services that operate across many
sites can therefore be very effective at detecting and mitigating attacks on
multiple sites.</p>
<h4 id="fingerprinting-and-privacy">Fingerprinting and privacy</h4>
<p>Browser makers generally regard browser fingerprinting as an attack on user
privacy. The <a href="https://amiunique.org/">fingerprint of a browser</a> is consistent
across sites in ways that are hard to control. Browsers can have unique or
nearly-unique fingerprints, which means that people can be effectively
identified and <a href="https://www.w3.org/2001/tag/doc/unsanctioned-tracking/">tracked</a>
using the fingerprint of their browser, against their wishes or expectations.</p>
<p>Fingerprinting used this way undermines controls that browsers use to maintain
<a href="https://digitalcommons.law.uw.edu/wlr/vol79/iss1/10/">contextual integrity</a>.
Circumventing these controls is unfortunately widespread. Services exist that
offer “cookie-less tracking” capabilities, which can including linking
cross-site activity using browser fingerprinting or “primary identifiers”<sup class="footnote-ref"><a href="https://lowentropy.net/posts/fraud/#fn4" id="fnref4">[4]</a></sup>.</p>
<p>Fingerprinting options in browsers continue to evolve in two directions:</p>
<ul>
<li>
<p>New browser features, especially those with personalization or hardware
interactions, can expand the ways in which browsers might become more
identifiable through fingerprinting.</p>
</li>
<li>
<p>Browser privacy engineers are constantly reducing the ways in which browsers
can be fingerprinted.</p>
</li>
</ul>
<p>Though these efforts often pull in different directions, the general trend is
toward reduced effectiveness of fingerprinting. Browsers are gradually becoming
more homogenous in their observable behaviour despite the introduction of new
capabilities. New features that might be used for fingerprinting tend not to be
accessible without active user intervention, making them far less reliable as a
means of identification. Existing rich sources of fingerprinting information –
like plugin or font enumeration – will eventually be far more limited.</p>
<aside>
<p>I’m deliberately ignoring the use of IP addresses for client identification. IP
addresses are still a very effective tool for managing fraud and abuse, just as
they are crucial tools for managing denial of service risk. IP provide some
amount of information that can increase the effectiveness of fingerprinting,
sometimes dramatically. Like other fingerprinting options, IP addresses are
something that might become less useful for fingerprint as time goes by. We
hope.</p>
</aside>
<p>Reductions in the effectiveness of fingerprinting are unlikely to ever result in
every browser looking identical. More homogenous browser fingerprints makes the
set of people who share a fingerprint larger. In turn, this only reduces the
odds that a site can successfully reidentify someone using a fingerprint.</p>
<p>Reduced effectiveness of fingerprinting might limit the ability of sites in
distinguishing between real and abusive activity. This places stronger reliance
on other signals, like behavioural cues. It might also mean that additional
checks are needed to discriminate between suspicious and wanted activity, though
this comes with its own hazards.</p>
<p>Even when fingerprinting is less useful, fingerprints can still help in managing
fraud. Though many users might share the same fingerprint, additional scrutiny
can be reserved for those browsers that share a fingerprint with the attacker.</p>
<h3 id="mitigation-strategies">Mitigation strategies</h3>
<p>Once a particular instance of fraud is detected and a pattern has been
established, it becomes possible to mitigate the effects of the attack. This
can involve some difficult choices.</p>
<p>With the difficulty in detecting fraud, sites often tolerate extensive fraud
before they are able to start implementing mitigation. Classification takes
time and can be error prone. Furthermore, sites don’t want to annoy their
customers by falsely accusing them of fraud.</p>
<h4 id="stringing-attackers-along">Stringing attackers along</h4>
<p>Tolerance of apparent abuse can have other positive effects. A change in how a
site reacts to attempted abuse might tip an attacker off that their method is no
longer viable. To that end, a site might allow abuse to continue, without any
obvious reaction<sup class="footnote-ref"><a href="https://lowentropy.net/posts/fraud/#fn5" id="fnref5">[5]</a></sup>.</p>
<p>A site that reacts to fraud in obvious ways will also reveal when fraud has
escaped detection. This can be worse, as it allows an attacker to learn when
their attack was successful. Tolerating fraud attempts deprives the attacker of
immediate feedback.</p>
<aside>
<p>This is where fraud mitigation differs from something like denial of service
attacks. In a denial of service attack, the attacker exhausts resources. Here,
attacks are often swift, as attackers attempt to surprise defenders or catch
them off guard. Detection, identification, and mitigation needs to occur
rapidly to blunt this sort of attack.</p>
<p>Denial of service mitigation generally involves finding the cheapest possible
way to block attacks. Leading an attacker to believe that they have not been
detected does little good when the resource that should be protected is actively
being expended.</p>
</aside>
<p>Delaying the obvious effects of mitigation allows abuse detection to remain
effective for longer. Similarly, providing feedback about abuse in the aggregate
might prevent an attacker from learning when specific tactics were successful.
Attackers that receive less feedback or late feedback cannot adapt as quickly
and so are able to evade detection for a smaller proportion of the overall time.</p>
<h4 id="addressing-past-abuse">Addressing past abuse</h4>
<p>A delayed response depends on being able to somehow negate or mitigate the
effect of fraud from the past. This is also helpful where instances of fraud or
abuse previously escaped detection.</p>
<p>For something like click fraud, the effect of fraud is often payment, which is
not immediate. The cost of fraud can be effectively managed if it can be
detected before payment comes due. The advertiser can refuse to pay for
fraudulent ad placements and disqualify any conversions that are attributed to
them. The same applies to credit card fraud, where settlement of payments can
be delayed to allow time for fraudulent patterns to be detected.</p>
<p>It is not always possible to retroactively mitigate fraud or delay its effect.
Sites can instead require additional checks or delays. These might not deprive
an attacker of feedback on whether their evasive methods were successful, but
changes in response could thwart or slow attacks.</p>
<h3 id="security-by-obscurity">Security by obscurity</h3>
<p>As someone who works in other areas of security, this overall approach to
managing fraud seems very … brittle.</p>
<p><a href="https://en.wikipedia.org/wiki/Kerckhoffs%27s_principle">Kerckhoffs’s principle</a>
– which guides the design of most security systems – says that you design
systems that depend only on protecting the key and not keeping the details of
how a system is built secret. A system design that is public knowledge can be
analysed and improved upon by many. Keeping the details of the system secret,
known as security by obscurity, is considered bad form and usually considered
indicative of a weak system design.</p>
<p>Here, security assurances rely very much on security by obscurity. Detecting
fraud depends on spotting patterns, then building ways of recognizing those
patterns. An attacker that can avoid detection might be able to conduct fraud
with impunity. That is, the system of defense relies on techniques so fragile
that knowledge of their details would render them ineffectual.</p>
<h2 id="is-there-hope-for-new-tools%3F">Is there hope for new tools?</h2>
<p>There are some technologies that offer some hope of helping manage fraud and
abuse risk. However, my expectation is that these will only support existing
methods.</p>
<p>Any improvements these might provide is unlikely to result in changes in
behaviour. Anything that helps attackers avoid detection will be exploited to
the maximum extent possible; anything that helps defenders detect fraud or abuse
will just be used to supplement existing information sources.</p>
<h3 id="privacy-pass">Privacy Pass</h3>
<p><a href="https://datatracker.ietf.org/doc/html/draft-ietf-privacypass-architecture">Privacy
Pass</a>,
offers a way for sites to exchange information about the trustworthiness of
their visitors. If one site decides that someone is trustworthy, it can give
the browser an anonymous token. Other sites can be told that someone is
trustworthy by passing them this token.</p>
<p>Ostensibly, Privacy Pass tokens cannot carry information, only the presence (or
absence) of a token carries any information. A browser might be told that the
token means “trustworthy”, but it could mean anything<sup class="footnote-ref"><a href="https://lowentropy.net/posts/fraud/#fn6" id="fnref6">[6]</a></sup>. That
means that the token issuer needs to be trusted.</p>
<p>How a site determines whether to provide a token also has consequences. Take
Apple’s <a href="https://developer.apple.com/news/?id=huqjyh7k">Private Access Tokens</a>,
which are supposed to mean that the browser is trustworthy, but they really
carry a cryptographically-backed assertion that the holder has an Apple device.
For sites looking to find a lucrative advertising audience, this provides a
strong indicator that a visitor is rich enough to be able to afford Apple
hardware. That is bankable information.</p>
<p>This is an example of how the method used to decide whether to provide a token
can leak. In order to protect this information, a decent proportion of tokens
need to use alternative methods.</p>
<p>We also need to ensure that sites do not become overly reliant on tokens.
Otherwise, people who are unable to produce a token could find themselves unable
to access services. People routinely fail to convince computers of their status
as a human for many reasons<sup class="footnote-ref"><a href="https://lowentropy.net/posts/fraud/#fn7" id="fnref7">[7]</a></sup>. Clients might be able to
withhold some proportion of tokens so that sites might learn not to become
dependent on them.</p>
<p>If these shortcomings are addressed somehow, it is possible that Privacy Pass
could help sites detect or identify fraud or abuse. However, implementing the
safeguards necessary to protect privacy and equitable access is not easy. It
might not even be worth it.</p>
<h3 id="questionable-options">Questionable options</h3>
<p>Google have proposed an extension to Privacy Pass that carries <a href="https://eprint.iacr.org/2020/072">secret
information</a>. The goal here is to allow sites
to rely on an assessment of trust that is made by another site, but not reveal
the decision to the client. All clients would be expected to retrieve a token
and proffer one in order to access the service. Suspicious clients would be
given a token that secretly identifies them as such.</p>
<p>This would avoid revealing to clients that they have been identified as
potentially fraudulent, but it comes with two problems:</p>
<ol>
<li>
<p>Any determination would only be based on information available to the site
that provides the token. The marking would less reliable as a result and
based only on the client identity or browser fingerprint<sup class="footnote-ref"><a href="https://lowentropy.net/posts/fraud/#fn8" id="fnref8">[8]</a></sup>. Consequently, any such marking would not be directly usable and it
need to be combined with other indicators, like how the client behaves.</p>
</li>
<li>
<p>Clients that might be secretly classified as dishonest have far less
incentive to carry a token that might label them as such.</p>
</li>
</ol>
<p>The secret bit also carries information, which – again – could mean anything.
Anything like this would need safeguards against privacy abuse by token
providers.</p>
<p>Google have also proposed <a href="https://github.com/RupertBenWiser/Web-Environment-Integrity/blob/main/explainer.md">Web Environment
Integrity</a>,
which seeks to suppress diversity of client software. Eric Rescorla has a good
explanation of <a href="https://educatedguesswork.org/posts/wei/">how this sort of approach is
problematic</a>. Without proper
safeguards, the same concerns apply to Apple’s Private Access Tokens.</p>
<p>The key insight for me is that all of these technologies risk placing
restrictions on how people access the Web. Some more than others. But openness
is worth protecting, even if it does make some things harder. Fraud and abuse
management are in some ways a product of that openness, but so is user
empowerment, equity of access, and privacy.</p>
<h2 id="summary">Summary</h2>
<p>It seems unlikely that anything is going to change. Those who want to commit
fraud will continue to try to evade detection and those who are trying to stop
them will try increasingly invasive methods, including fingerprinting.</p>
<p>Fraud and abuse are something that many sites contend with. There are no easy or
assured methods for managing fraud or abuse risk. Defenders look for patterns,
both in client characteristics and their behaviour. Fingerprinting browsers
this way can have poor privacy consquences. Concealing how attacks are
classified is the only way to ensure that attackers do not adapt their methods
to avoid protections. New methods for classification might help, but they
create new challenges that will need to be managed.</p>
<p>Fraud is here to stay. Fingerprinting too. I wish that I had a better story to
tell, but this is one of the prices we pay for an open Web.</p>
<hr class="footnotes-sep" />
<section class="footnotes">
<ol class="footnotes-list">
<li id="fn1" class="footnote-item"><p>I’m not comfortable using
the more widely used “anti-fraud” term here. It sounds too definite, as if to
imply that fraud can be prevented perfectly. Fraud and abuse can be managed,
but not so absolutely. <a href="https://lowentropy.net/posts/fraud/#fnref1" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn2" class="footnote-item"><p>This story has been widely misreported, see
(<a href="https://www.schneier.com/blog/archives/2023/08/bots-are-better-than-humans-at-solving-captchas.html">Schneier</a>,
<a href="https://www.theregister.com/2023/08/15/so_much_for_captcha_then/">The
Register</a>, and
<a href="https://hardware.slashdot.org/story/23/08/10/0439241/bots-are-better-than-humans-at-cracking-are-you-a-robot-captcha-tests-study-finds">Slashdot</a>). These
articles cite a recent <a href="https://arxiv.org/abs/2307.12108">study</a> from UC Irvine,
which cites <a href="https://arxiv.org/abs/2307.12108">a study from 2014</a> that applies
to a largely defunct CAPTCHA method. CAPTCHA fans might hold out
<a href="https://arxiv.org/abs/2209.06293">some</a>
<a href="https://www.nature.com/articles/d41586-023-02361-7">hope</a>, though maybe the
rest of us would be happy to never see another inane test. <a href="https://lowentropy.net/posts/fraud/#fnref2" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn3" class="footnote-item"><p>There is a whole industry around the
scalping of limited run sneakers, to the point that there are specialist cloud
services that <a href="https://sneakerserver.com/">boast extra low latency access</a> to
the sites for major sneaker vendors. <a href="https://lowentropy.net/posts/fraud/#fnref3" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn4" class="footnote-item"><p>Think
email addresses or phone numbers. These sites like to pretend that these
practices are privacy respecting, but collecting primary identifiers often
involves deceptive practices. For example, making access to a service
conditional on providing a phone number. <a href="https://lowentropy.net/posts/fraud/#fnref4" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn5" class="footnote-item"><p>It is widely believed that, during the second World War, that
the British chose not to act on intelligence gained from their breaking of
Enigma codes. No doubt the Admiralty did exercise discretion in how it used the
information it gained, but <a href="https://drenigma.org/2021/09/21/who-spilt-the-beans-how-the-enigma-secret-was-revealed/">the famous case of the bombing of Coventry in
November 1940 was not one of these
instances</a>. <a href="https://lowentropy.net/posts/fraud/#fnref5" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn6" class="footnote-item"><p>It could be bad if tokens
had something to say about the colour of a person’s skin or their gender
identity. There are more bad uses than good ones for these tokens. <a href="https://lowentropy.net/posts/fraud/#fnref6" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn7" class="footnote-item"><p>Finally, a good reason to cite <a href="https://arxiv.org/abs/2307.12108">the study mentioned
previously</a>. <a href="https://lowentropy.net/posts/fraud/#fnref7" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn8" class="footnote-item"><p>A fingerprint could
be re-evaluated on the other site without using a token, so that isn’t much
help. <a href="https://lowentropy.net/posts/fraud/#fnref8" class="footnote-backref">↩︎</a></p>
</li>
</ol>
</section>
Entropy and Privacy Analysis2022-05-27T00:00:00Zhttps://lowentropy.net/posts/entropy-privacy/<p>Aggregation is a powerful tool when it comes to providing privacy for users. But
analysis that relies on aggregate statistics for privacy loss hides some of the
worst effects of designs.</p>
<h2 id="background">Background</h2>
<p>A lot of my time recently has been spent looking at various proposals for
improving online advertising. A lot of this work is centred on the <a href="https://patcg.github.io/">Private
Advertising Technology Community Group</a> in the W3C
where the goal is to find designs that improve advertising while maintaining
strong technical protections for privacy.</p>
<p>Part of deciding whether a design does in fact provide strong privacy
protections requires understanding firstly what that means. That is a large
topic on which the conversation is continuing. In this post, my goal is to look
at some aspects of how we might critically evaluate the privacy characteristics
of proposals.</p>
<h2 id="limitations-of-differential-privacy">Limitations of Differential Privacy</h2>
<p>A number of designs have been proposed in this space with supporting analysis
that is based on differential privacy. Providing differential privacy involves
adding noise to measurements using a tunable parameter (usually called
$\varepsilon$) that hides individual contributions under a random distribution.</p>
<p>I’m a big fan of differential privacy, but while differential privacy provides a
good basis for understanding the impact of a proposal, it is recognized that
there is a need to continuously release information in order to maintain basic
utility in a long-running system.</p>
<p>Continuous release of data potentially leads to the protections offered by
differential privacy noise being less effective over time. It is prudent
therefore to understand the operation of the system without the protection
afforded by noise. This is particularly relevant where the noise uses a large
$\varepsilon$ value or is applied to unaggregated outputs, where it can be
easier to cancel the effect of noise by looking at multiple output values.</p>
<p>Information exposure is often expressed using information theoretic statistics
like entropy. This note explores how entropy — or any single statistic
— is a poor basis for privacy analysis and suggests options for more
rigorous analysis.</p>
<h2 id="information-theory-and-privacy">Information Theory and Privacy</h2>
<p>Some analysis of Web privacy features often looks at the number of bits of
information that a system releases to an adversary. Analyses of this type use
the distribution of probabilities of all events as a way of estimating the
amount of information that might be provided by a specific event.</p>
<p>In information theory, each event provides information or surprisal, defined by
a relationship with the probability of the event:</p>
<p>$$I(x)=-\log_2(P(x))$$</p>
<p>The reason we might use information is that if a feature releases too much
information, then people might be individually identified. They might no longer
be anonymous. Their activities might be linked to them specifically. The
information can be used to form a profile based on their actions or further
joined to their identity or identities.</p>
<p>Generally, we consider it a problem when information enables identification of
individuals. We might express concern if:</p>
<p>$$2^{I(x)} \ge \text{size of population}$$</p>
<p>Because surprisal is about specific events, it can be a little unwieldy.
Surprisal is not useful for reaching a holistic understanding of the system. A
statistic that summarizes all potential outcomes is more useful in gaining
insight into how the system operates as a whole. A common statistic used in
this context is entropy, which provides a mean or expected surprisal across a
sampled population:</p>
<p>$$H(X)=\sum_{x\in X}P(x)I(x)=-\sum_{x\in X}P(x)\log_2(P(x))=-\frac{1}{N}\sum_{i=1}^N\log_2(P(x_i))$$</p>
<p>Entropy has a number of applications. For instance, it can be used to determine
an optimal encoding of the information from many events, using entropy coding
(such as Huffman or Arithmetic coding).</p>
<h2 id="using-entropy-in-privacy-analysis">Using Entropy in Privacy Analysis</h2>
<p>The use of specific statistics in privacy analysis is useful to the extent that
they provide an understanding of the overall shape of the system. However,
simple statistics tend to lose information about exceptional circumstances.</p>
<p>Entropy has real trouble with rare events. Low probability events have high
surprisal, but as entropy scales their contribution by their probability, they
contribute less to the total entropy than higher probability events.</p>
<p>In general, revealing more information is undesirable from a privacy
perspective. Toward that end, it might seem obvious that minimizing entropy is
desirable. However, this can be shown to be counterproductive for individual
privacy, even if a single statistic is improved.</p>
<div class="callout">
<p>An example might help prime intuition. A cohort of 100 people is arbitrarily
allocated into two groups. If people are evenly distributed into groups of 50,
revealing the group that a person has been allocated provides just a single bit
of information, that is, surprisal is 1 bit. The total entropy of the system is
1 bit.</p>
<p>An asymmetric allocation can produce a different result. If 99 people are
allocated to one group and a single person to the other, revealing that someone
is in the first group provides almost no information at 0.0145 bits. On the
contrary, revealing the allocation for the lone person in the second group —
which uniquely identifies that person — produces a much larger surprisal of
6.64 bits. Though this is clearly a privacy problem for that person, their
privacy loss is not reflected in the total entropy of the system, which at
0.0808 bits is close to zero.</p>
<p>Entropy tells us that the average information revealed for all users is very
small. That conclusion about the aggregate is reflected in the entropy
statistic, but it hides the disproportionately large impact on the single user
who loses the most.</p>
</div>
<p>The more asymmetric the information contributed by individuals, the lower the
entropy of the overall system.</p>
<p>Limiting analysis to simple statistics, and entropy in particular, can hide
privacy problems. Somewhat counterintuitively, the adverse consequences of a
design are felt more by a minority of participants for systems with lower
entropy.</p>
<p>This is not a revelatory insight. It is well known that <a href="https://janhove.github.io/teaching/2016/11/21/what-correlations-look-like">a single metric is
often a poor means of understanding
data</a>.</p>
<p>Entropy can provides a misleading intuitive understanding of privacy as it
relates to the experience of individual users.</p>
<h2 id="recommendations">Recommendations</h2>
<p>Information entropy remains useful as a means of understanding the overall
utility of the information that a system provides. Understanding key statistics
as part of a design is valuable. However, for entropy measures in particular,
this is only useful from a perspective that seeks to reduce overall utility;
entropy provides almost no information about the experience of individuals.</p>
<h3 id="understand-the-surprisal-distribution">Understand the Surprisal Distribution</h3>
<p>Examining only the mean surprisal offers very little insight into a
system. Statistical analysis rarely considers a mean value in isolation. Most
statistical treatment takes the shape of the underlying distribution into
account.</p>
<p>For privacy analysis, understanding the distribution of surprisal values is
useful. Even just looking at percentiles might offer greater insight into the
nature of the privacy loss for those who are most adversely affected.</p>
<p>Shortcomings of entropy are shared by related statistics, like <a href="https://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence">Kullback–Leibler
divergence</a>
or mutual information, which estimate information gain relative to a known
distribution. Considering percentiles and other statistics can improve
understanding.</p>
<p>Knowing the distribution of surprisal admits the combination of privacy loss
metrics. As privacy is affected by multiple concurrent efforts to change the
way people use the Web, the interaction of features can be hard to understand.
Richer expressions of the effect of changes might allow for joint analysis to be
performed. Though it requires assumptions about the extent to which different
surprisal distributions might be correlated, analyses of surprisal that assume
either complete independence or perfect correlation could provide insights into
the potential extent of privacy loss from combining features.</p>
<p>For example, it might be useful to consider the interaction of a proposal with
extant browser fingerprinting. The users who reveal the most information using
the proposal might not be the same users who reveal the most fingerprinting
information. Analysis could show that there are no problems or it might help
guide further research that would provide solutions.</p>
<p>More relevant to privacy might be understanding the proportion of individuals
that are potentially identifiable using a system. A common privacy goal is to
maintain a minimum size of anonymity set. It might be possible to apply
knowledge of a surprisal distribution to estimating the size of a population
where the anonymity set becomes too small for some users. This information
might then guide the creation of safeguards.</p>
<h3 id="consider-the-worst-case">Consider the Worst Case</h3>
<p>A worst-case analysis is worth considering from the perspective of understanding
how the system treats the privacy of all those who might be affected. That is,
consider the implications for users on the tail of any distribution. Small user
populations will effectively guarantee that any result is drawn from the tail of
a larger distribution.</p>
<p>Concentrating on cases where information might be attributed to individuals
might miss privacy problems that might arise from people being identifiable in
small groups. Understand how likely smaller groups might be affected.</p>
<p>The potential for targeting of individuals or small groups might justify
disqualification of — or at least adjustments to — a proposal. The Web is
for everyone, not just most people.</p>
Bundling for the Web2021-02-26T00:00:00Zhttps://lowentropy.net/posts/bundles/<p>The idea of bundling is deceptively simple. Take a bunch of stuff and glom them
into a single package. So why is it so difficult to teach the web how to
bundle?</p>
<aside>
<p>First a note: this is my personal opinion and an incomplete one at that. Though
I work for Mozilla, this post is part of my process of grappling with a complex
problem. Mozilla has made <a href="https://mozilla.github.io/standards-positions/#bundled-exchanges">a
statement</a>
regarding Google’s bundled exchanges, which still applies. That position might
be revised and I’ll have some say in that if it does, but any process will
involve a discussion with a group of my colleagues who each add their own
perspectives and experience.</p>
</aside>
<h2 id="the-web-already-does-bundling">The Web already does bundling</h2>
<p>A bundled resource is a resource the composes multiple pieces of content.
Bundles can consist of content only of a single type or mixed types.</p>
<p>Take something like JavaScript<sup class="footnote-ref"><a href="https://lowentropy.net/posts/bundles/#fn1" id="fnref1">[1]</a></sup>. A very large proportion of the JavaScript content on the web is
bundled today. If you haven’t bundled, minified, and compressed your
JavaScript, you have left easy performance wins unrealized.</p>
<p>HTML is a bundling format in its own right, with inline JavaScript and CSS.
Bundling other content is also possible with <code>data:</code> URIs, even if this has
some drawbacks.</p>
<p>Then there are <a href="https://developer.mozilla.org/en-US/docs/Glossary/CSS_preprocessor">CSS
preprocessors</a>,
which provide bundling options, <a href="https://developers.google.com/web/fundamentals/design-and-ux/responsive/images#use_image_sprites">image
spriting</a>,
and myriad other hacks.</p>
<p>And that leaves aside the whole mess of zipfiles, tarballs, and self-extracting
executables that are used for a variety of Web-adjacent purposes. Those matter
too, but they are generally not Web-visible.</p>
<h2 id="why-we-might-want-bundles">Why we might want bundles</h2>
<p>What is immediately clear from this brief review of available Web bundling
options is that they are all terrible in varying degrees. The reasons are
varied and a close examination of the reasons for this is probably not
worthwhile.</p>
<p>It might be best just to view this as the legacy of a system that evolved in
piecemeal fashion; an evolutionary artifact along a dimension that nature did
not regard as critical to success.</p>
<p>I’m more interested in the balance of different pressures, both for and against
bundling. There are good reasons in support of bundling, and quite a few
reasons to be cautious, but it looks like the time has come to consider
bundling seriously.</p>
<p>I doubt that introducing native support for bundling technology will
fundamentally change the way Web content is delivered. I see it more as an
opportunity to expand the toolkit to allow for more use cases and more flexible
deployment options.</p>
<p>In researching this, I was reminded of work that Jonas Sicking did to identify
<a href="https://gist.github.com/sicking/6dcd3b771612b2f6f1bb">use cases</a>. There are
lots of reasons and requirements that are worth looking at. Some of the
reasoning is dated, but there is a lot of relevant material, even five years
on.</p>
<h3 id="efficiency">Efficiency</h3>
<p>One set of touted advantages for bundling relate to performance and efficiency.
Today, we have a better understanding of the ways in which performance is
affected by resource composition, so this has been narrowed down to two primary
features: compression efficiency and reduced overheads.</p>
<p>I also want to address another reason that is often cited: providing content
that a client will need, but doesn’t yet know about.</p>
<h4 id="shared-compression">Shared compression</h4>
<p>Compression efficiency can be dramatically improved if similar resources are
bundled together. This is because the larger shared context results in more
repetition and gives a compressor more opportunities to find and exploit
similarities.</p>
<p>Bundling is not the only way to achieve this. Alternative methods of attaining
compression gains have been explored, such as
<a href="https://docs.google.com/document/d/1REMkwjXY5yFOkJwtJPjCMwZ4Shx3D9vfdAytV_KQCUo/edit?usp=sharing">SDCH</a>
and <a href="https://datatracker.ietf.org/doc/html/draft-vkrasnov-h2-compression-dictionaries-03">cross-stream compression contexts for
HTTP/2</a>.
Prototypes of the latter showed immense improvements in compression efficiency
and corresponding performance gains.</p>
<p>General solutions like these have not been successful in find ways to manage
<a href="https://datatracker.ietf.org/doc/html/draft-handte-httpbis-dict-sec-00">operational security
concerns</a>.
The hope with bundles is that a bundling can occur as a build process. As the
build occurs before deploying content to servers, no sensitive or user-specific
data will be involved. This is somewhat at odds with some of the dynamic
features involved, but that sort of separation could be an effective strategy
for managing this security risk.</p>
<h4 id="reduced-overheads">Reduced overheads</h4>
<p>Bundling could also reduce overheads. While HTTP/2 and HTTP/3 reduce the cost
of making requests, those costs still compound when multiple resources are
involved. The claim here is that internal handling of individual requests in
browsers has inefficiencies that are hard to eliminate without some form of
bundling.</p>
<p>I find it curious that protocol-level inefficiencies are not blamed here, but
rather inter-process communication between internal browser processes. Not
having examined this closely<sup class="footnote-ref"><a href="https://lowentropy.net/posts/bundles/#fn2" id="fnref2">[2]</a></sup>, I can’t really speak to these
claims.</p>
<p>What I do know is that performance in this space is subtle. When we were
building HTTP/2, we found that performance was highly sensitive to the number
of requests that could be made by clients in the first few round trips of a
connection. The way that networking protocols work means that there is very
limited space for sending anything early in a connection<sup class="footnote-ref"><a href="https://lowentropy.net/posts/bundles/#fn3" id="fnref3">[3]</a></sup>. The main motivation for <a href="https://datatracker.ietf.org/doc/html/rfc7541">HTTP
header compression</a> was that it
allowed significantly more requests to be made early in a connection. By
reducing request counts, bundling might do the same.</p>
<h4 id="eliminating-round-trips">Eliminating round trips</h4>
<p>One of the other potential benefits of bundling is in eliminating additional
round trips. For content that is requested, a bundle might provide resources
that a client does not know that it needs yet. Without bundling, a resource
that references another resource adds an additional round trip as the first
resource needs to be fetched before the second one is even known to the client.</p>
<p>Again, experience with HTTP/2 suggests that performance gains from sending
extra resources are not easy to obtain. This is exactly what HTTP/2 server push
promised to provide. However, as we have learned with server push, the wins
here are not easy to realize. A number of attempts to improve performance with
server push often resulted in mixed results and sometimes large regressions in
performance<sup class="footnote-ref"><a href="https://lowentropy.net/posts/bundles/#fn4" id="fnref4">[4]</a></sup>.
The problem is that servers are unable to accurately predict when to push
content and so they push data that is not needed. To date, no studies have
shown that there are reliable strategies that servers can use to reliably
improve performance with server push.</p>
<h4 id="realizing-performance-improvements">Realizing performance improvements</h4>
<p>For bundles to realize performance gains from eliminating round trips, the
compression gains would need to be enough counteract any potential waste. This
is more challenging if bundles are built statically.</p>
<p>I personally remain lukewarm on using bundling as a performance tool.
Shortcomings in protocols – or implementations – seem like they could be
addressed at that level.</p>
<h3 id="ergonomics">Ergonomics</h3>
<p>The use of bundlers is an established practice in Web development. Being able
to outsource some of the responsibility for managing the complexities of
content delivery is no doubt part of the appeal.</p>
<p>Being able to compose complex content into a single package should not be
underestimated.</p>
<p>Bundling of content into a single file is a property common to many systems.
Providing a single item to manage with a single identity simplifies
interactions. This is how most people expect content of all kinds to be
delivered, whether it is
<a href="https://en.wikipedia.org/wiki/Self-extracting_archive">applications</a>,
<a href="https://en.wikipedia.org/wiki/Comparison_of_e-book_formats">books</a>,
<a href="https://en.wikipedia.org/wiki/Library_(computing)">libraries</a>, or any other
sort of digital artifact. The Web here is something of an abberation in that it
resists the idea that parts of it can be roped off into a discrete unit with a
finite size and name.</p>
<p>Though this usage pattern might be partly attributed to path dependence, the
usability benefits of individual files cannot be so readily dismissed. Being
able to manage bundles as a single unit where necessary, but identify the
component pieces is likely to be a fairly large gain for developers.</p>
<p>For me, this reason might be enough to justify using bundles, even over some of
their drawbacks.</p>
<h2 id="why-we-might-not-want-bundles">Why we might not want bundles</h2>
<p>The act of bundling subsumes the identity of each piece of bundled content with
the identity of the bundle that is formed. This produces a number of effects,
some of them desirable (as discussed), some of them less so.</p>
<p>As far as effects go, whether they are valuable or harmful might depend on
context and perspective. Some of these effects might simply be managed as
trade-offs, with site or server developers being able to choose how content is
composed in order to balance various factors like total bytes transferred or
latency.</p>
<p>If bundling only represented trade-offs that affected the operation of servers,
then we might be able to resolve whether the feature is worth pursuing on the
grounds of simple cost-benefit. Where things get more interesting is where
choices might involve depriving others of their own choices. Balancing the
needs of clients and servers is occasionally necessary. Determining the effect
of server choices on clients – and the people they might act for – is
therefore an important part of any analysis we might perform.</p>
<h3 id="cache-efficiency-and-bundle-composition">Cache efficiency and bundle composition</h3>
<p>Content construction and serving infrastructure generally operates with
imperfect knowledge of the state of caches. Not knowing what a client might
need can make it hard to know what content to serve at any given point in time.</p>
<p>Optimizing the composition of the bundles used on a site for clients with a
variety of cache states can be particularly challenging if caches operate at
the granularity of resources. Clients that have no prior state might benefit
from maximal bundling, which allows better realization of the aforementioned
efficiency gains.</p>
<p>On the other hand, clients that have previously received an older version of
the same content might only need to receive updates for those things that have
changed. Similarly, clients that have previously received content for other
pages that includes some of the same content. In both cases, receiving copies
of content that was already transferred might negate any efficiency gains.</p>
<p>This is a problem that JavaScript bundlers have to deal with today. As an
optimization problem it is made difficult by the combination of poor
information about client state with the complexity of code dependency graphs
and the potential for clients to follow different paths through sites.</p>
<p>For example, consider the code that is used on an article page on a
hypothetical news site and the code used on the home page of the same site.
Some of that code will be common, if we make the assumption that site
developers use common tools. Bundlers might deal with this by making three
bundles: one of common code, plus one each of article and home page code. For a
very simple site like this, that allows all the code to be delivered in just
two bundles on either type of page, plus an extra bundle when navigating from
an article to the home page or vice versa.</p>
<p>As the number of different types of page increases, splitting code into
multiple bundles breaks down. The number of bundle permutations can increase
much faster than the number of discrete uses. In the extreme, the number of
bundles could end up being factorial on the number of types of page, limited
only by the number of resources that might be bundled. Of course, well before
that point is reached, the complexity cost of bundling likely exceeds any
benefits it might provide.</p>
<p>To deal with this, bundlers have a bunch of heuristics that balance the costs
of providing too much data in a bundle for a particular purpose, against the
costs of potentially providing bundled data that is already present. Some sites
take this a little further and use service workers to enhance browser caching
logic<sup class="footnote-ref"><a href="https://lowentropy.net/posts/bundles/#fn5" id="fnref5">[5]</a></sup>.</p>
<p>It is at this point that you might recognize an opportunity. If clients
understood the structure of bundles, then maybe they could do something to
avoid fetching redundant data. Maybe providing a way to selectively request
pieces of bundles could reduce the cost of fetching bundles when parts of the
bundle are already present. That would allow the bundlers to skew their
heuristics more toward putting stuff in bundles. It might even be possible to
tune first-time queries this way.</p>
<p>The thing is, we’ve already tried that.</p>
<h4 id="a-standard-for-inefficient-caching">A standard for inefficient caching</h4>
<p>There is a long history in HTTP of failed innovation when it comes to
standardizing improvements for cache efficiency. Though cache invalidation is
recognized as one of the <a href="https://www.karlton.org/2017/12/naming-things-hard/">hard
problems</a> in computer
science, there are quite a few examples of successful deployments of
proprietary solutions in server and CDN infrastructure.</p>
<p>A few caching innovations have made it into HTTP over time, such as the recent
<a href="https://datatracker.ietf.org/doc//html/rfc8246">immutable Cache-Control
directive</a>. That particular
solution is quite relevant in this context due to the way that it supports
content-based URI construction, but it is still narrower in applicability than
a good solution in this space might need.</p>
<p>If we view bundling as a process that happens as part of site construction,
bundles might be treated as opaque blobs by servers. Servers that aren’t aware
of bundle structure are likely to end up sending more bits than the client
needs. To avoid this, servers and clients both need to be aware of the contents
of bundles.</p>
<h4 id="cache-digests">Cache digests</h4>
<p>Once both the client and server are aware of individual resources within
bundles, this problem starts to look very much like server push.</p>
<p>Previous attempts to solve the problem of knowing what to push aimed to improve
the information available to servers. <a href="https://tools.ietf.org/html/draft-ietf-httpbis-cache-digest-05">Cache
digests</a> is the
most notable attempt here. It got several revisions into the IETF working group
process. It still failed.</p>
<p>If the goal of failing is to learn, then this too was a failure largely for the
most ignomonious of reasons: no deployment. Claims from clients that cache
digests are too expensive to implement seem reasonable, but not entirely
satisfactory in light of the change to use <a href="https://en.wikipedia.org/wiki/Cuckoo_filter">Cuckoo
filters</a> in later versions. More
so with recent storage partitioning work.</p>
<p>The point of this little digression is to highlight the inherent difficulties
in trying to fix this problem by layering in enhancements to the caching model.
More so when that requires replicating the infrastructure we have for
individual resources at the level of bundled content.</p>
<p>My view is that it would be unwise to attempt to tackle a problem like this as
part of trying to introduce a new feature. If the success of bundling depends
on finding a solution to this problem, then I would be surprised, but it might
suggest that the marginal benefit of bundling – at least for performance – is
not sufficient to justify the effort<sup class="footnote-ref"><a href="https://lowentropy.net/posts/bundles/#fn6" id="fnref6">[6]</a></sup>.</p>
<h3 id="prioritization-is-harder">Prioritization is harder</h3>
<p>Mark Nottingham reminded me that even if servers and clients are modified so
that they are aware of individual resources, there are still limitations.
Bundles might contain resources with different priorities. It might be
impossible to avoid performance regressions.</p>
<p>It is certainly possible to invent a new system for ensuring that bundles are
properly prioritized, but that requires good knowledge of relative priority at
the time that bundles are constructed.</p>
<p>Putting important stuff first is likely a good strategy, but that has drawbacks
too. Servers need to know where to apply priority changes when serving bundles
or the low-priority pieces will be served at the same priority as high-priority
pieces. The relative priority of resources will need to be known at bundling
time. Bundling content that might change in priority in response to <a href="https://datatracker.ietf.org/doc/html/draft-ietf-httpbis-priority">client
signals</a>
might result in priority inversions and performance regressions.</p>
<p>Just like with caching, addressing prioritization shortcomings could require
replicating a lot of the machinery we have for prioritizing individual
resources within bundles.</p>
<h3 id="erasing-resource-identity">Erasing resource identity</h3>
<p>An <a href="https://github.com/WICG/webpackage/issues/551">issue</a> that was first<sup class="footnote-ref"><a href="https://lowentropy.net/posts/bundles/#fn7" id="fnref7">[7]</a></sup> raised by Brave is that the use of bundles creates
opportunities for sites to obfuscate the identity of resources. The thesis
being that bundling could <a href="https://brave.com/webbundles-harmful-to-content-blocking-security-tools-and-the-open-web/">confound content blocking
techniques</a>
as it would make rewriting of identifiers easier.</p>
<p>For those who rely on the identity of resources to understand the semantics and
intent of the identified resource, there are some ways in which bundling might
affect their decision-making. The primary concern is that references between
resources in the same bundle are fundamentally more malleable than other
references. As the reference and reference target are in the same place, it is
trivial – at least in theory – to change the identifier.</p>
<p>Brave and several others are therefore concerned that bundling will make it
easier to prevent URI-based classification of resources. In the extreme,
identifiers could be rewritten for every request, negating any attempt to use
those identifiers for classification.</p>
<p>One of the most interesting properties of the Web is the way that it insinuates
a browser – and user agency – into the process. The way that happens is that
the the Web<sup class="footnote-ref"><a href="https://lowentropy.net/posts/bundles/#fn8" id="fnref8">[8]</a></sup> is structurally biased toward functioning better when sites expose
semantic information to browsers. This property, sometimes called semantic
transparency, is what allows browsers to be opinionated about content rather
than acting as a dumb pipe<sup class="footnote-ref"><a href="https://lowentropy.net/posts/bundles/#fn9" id="fnref9">[9]</a></sup>.</p>
<h4 id="yes%2C-it%E2%80%99s-about-ad-blockers">Yes, it’s about ad blockers</h4>
<p>Just so that this is clear, this is mostly about blocking advertising.</p>
<p>While more advanced ad blocking techniques also draw on contextual clues about
resources, those methods are more costly. Most ad blocking decisions are made
based on the URI of resources. Using the resource identity allows the ad
blocker to prevent the load, which not only means that the ad is not displayed,
but the resources needed to retrieve it are not spent<sup class="footnote-ref"><a href="https://lowentropy.net/posts/bundles/#fn10" id="fnref10">[10]</a></sup>.</p>
<p>While <a href="https://www.statista.com/statistics/804008/ad-blocking-reach-usage-us/">many people might choose to block
ads</a>,
sites don’t like being denied the revenue that advertising provides. Some sites
use techniques that are designed to show advertising to users of ad blockers,
so it is not unreasonable to expect tools to be used to prevent classification.</p>
<p>It is important to note that this is not a situation that requires an absolute
certainty. The sorry state of Web privacy means that we have a lot of places
where various forces are in tension or transition. The point of Brave’s
complaint here is not that bundling outright prevents the sort of
classification they seek, but that it changes the balance of system dynamics by
giving sites another tool that they might employ to avoid classification.</p>
<p>Of course, when it is a question of degree, we need to discuss and agree how
much the introduction of such a tool affects the existing system. That’s where
this gets hard.</p>
<h4 id="coordination-artifacts">Coordination artifacts</h4>
<p>As much as these concerns are serious, I tend to think that Jeffrey Yasskin’s
<a href="https://medium.com/@jyasskin/why-do-url-based-ad-blockers-work-3a13b08a1167">analysis of the
problem</a>
is broadly correct. That analysis essentially concludes that the reason we have
URIs is to facilitate coordination between different entities. As long as there
is a need to coordinate between the different entities that provide the
resources that might be composed into a web page, that coordination will expose
information that can be used for classification.</p>
<p>That is, to the extent to which bundles enable obfuscation of identifiers, that
obfuscation needs to be coordinated. Any coordination that would enable
obfuscation with bundling is equally effective and easy to apply without
bundling.</p>
<h4 id="single-page-coordination">Single-page coordination</h4>
<p>Take a single Web page. Pretend for a moment that the web page exists in a
vacuum, with no relationship to other pages at all. You could take all the
resources that comprise that page and form them into a single bundle. As all
resources are in the one place, it would be trivial to rewrite the references
between those resources. Or, the identity of resources could be erased entirely
by inlining everything. If every request for that page produced a bundle with a
different set of resource identifiers, it would be impossible to infer anything
about the contents of resources based on their identity alone.</p>
<p>Unitary bundles for evey page is an extreme that is almost certainly
impractical. If sites were delivered this way, there would be no caching, which
means no reuse of common components. Using the Web would be terribly slow.</p>
<p>Providing strong incentive to deploy pages as discrete bundles – something
Google Search has done to <a href="https://developers.google.com/search/docs/guides/about-amp#about-signed-exchange">enable preloading search results for cooperating
sites</a>
– could effectively force sites to bundle in this way. Erasing or obfuscating
internal links in these bundles does seem natural at this point, if only to try
to reclaim some of the lost performance, but that assumes an unnatural pressure
toward bundling<sup class="footnote-ref"><a href="https://lowentropy.net/posts/bundles/#fn11" id="fnref11">[11]</a></sup>.</p>
<p>Absent perverse incentives, sites are often built from components developed by
multiple groups, even if that is just different teams working at the same
company. To the extent that teams operate independently, they need to agree on
how they interface. The closer the teams work together, and the more tightly
they are able to coordinate, the more flexible those interfaces can be.</p>
<p>There are several natural interface points on the Web. Of these the URI remains
a key interface point<sup class="footnote-ref"><a href="https://lowentropy.net/posts/bundles/#fn12" id="fnref12">[12]</a></sup>. A simple string
that provides a handle for a whole bundle<sup class="footnote-ref"><a href="https://lowentropy.net/posts/bundles/#fn13" id="fnref13">[13]</a></sup> of
collected concepts is a powerful abstraction.</p>
<h4 id="cross-site-coordination">Cross-site coordination</h4>
<p>Interfaces between components therefore often use URIs, especially once
cross-origin content is involved. For widely-used components that enable
communication between sites, URIs are almost always involved. If you want to
use <a href="https://reactjs.org/">React</a>, the primary interface is a URI:</p>
<pre><code class="language-html"><script src="https://unpkg.com/react@17/umd/react.production.min.js" crossorigin></script>
<script src="https://unpkg.com/react-dom@17/umd/react-dom.production.min.js" crossorigin></script>
</code></pre>
<p>If you want add <a href="https://developers.google.com/analytics/devguides/collection/gtagjs">Google
analytics</a>,
there is a bit of JavaScript<sup class="footnote-ref"><a href="https://lowentropy.net/posts/bundles/#fn14" id="fnref14">[14]</a></sup> as well, but the URI is still key:</p>
<pre><code class="language-html"><script async src="https://www.googletagmanager.com/gtag/js?id=$XXX"></script>
<script>
window.dataLayer = window.dataLayer || [];
function gtag(){dataLayer.push(arguments);}
gtag('js', new Date());
gtag('config', '$XXX');
</script>
</code></pre>
<p>The same applies to <a href="https://support.google.com/adsense/answer/9274634">advertising</a>.</p>
<p>The scale of coordination required to change these URIs is such that changes
cannot be effected on a per-request basis, they need months, if not years<sup class="footnote-ref"><a href="https://lowentropy.net/posts/bundles/#fn15" id="fnref15">[15]</a></sup>.</p>
<p>Even for resources on the same site, a version of the same coordination problem
exists. Content that might be used by multiple pages will be requested at
different times. At a minimum, changing the identity of resources would mean
forgoing any reuse of cached resources. Caching provides such a large
performance advantage that I can’t imagine sites giving that up.</p>
<p>Even if caching were not incentive enough, I suggest that the benefits of
reference stability are enough to ensure that identifiers don’t change
arbitrarily.</p>
<h4 id="loose-coupling">Loose coupling</h4>
<p>As long as loose coupling is a feature of Web development, the way that
resources are identified will remain a key part of how the interfaces between
components is managed. Those identifiers will therefore tend to be stable. That
stability will allow the semantics of those resources to be learned.</p>
<p>Bundles do not change these dynamics in any meaningful way, except to the
extent that they might enable better atomicity. That is, it becomes easier to
coordinate changes to references and content if the content is distributed in a
single indivisible unit. That’s not nothing, but – as the case of selective
fetches and cache optimization highlights – content from bundles need to be
reused in a different context, so the application of indivisible units is
severely limited.</p>
<p>Of course, there are ways of enabling coordination that might allow for
constructing identifiers that are less semantically meaningful. To draw on the
earlier point about the Web already having bundling options, advertising code
could be inlined with other JavaScript or in HTML, rather than having it load
directly from the advertiser<sup class="footnote-ref"><a href="https://lowentropy.net/posts/bundles/#fn16" id="fnref16">[16]</a></sup>. In the extreme,
servers could rewrite all content and encrypt all URIs with a per-user key. None
of this depends on the deployment of new Web bundling technology, but it does
require close coordination.</p>
<h3 id="all-or-nothing-bundles">All or nothing bundles</h3>
<p>Even if it were possible to identify unwanted content, opponents of bundling
point out that placing that content in the same bundle as critical resources
makes it difficult to avoid loading the unwanted content. Some of the
performance gains from content blockers are the result of not fetching
content<sup class="footnote-ref"><a href="https://lowentropy.net/posts/bundles/#fn17" id="fnref17">[17]</a></sup>. Bundling unwanted content might eliminate the cost and
performance benefits of content blocking.</p>
<p>This is another important criticism that ties in with the early concerns
regarding bundle composition and reuse. And, similar to previous problems, the
concern is not that this sort of bundling is enabled as a result of native,
generic bundling capabilities, but more that it becomes more readily accessible
as a result.</p>
<p>This problem, more so than the caching one, might motivate designs for
selective acquisition of bundled content.</p>
<p>Existing techniques for selective content fetching, like <a href="https://httpwg.org/http-core/draft-ietf-httpbis-semantics-latest.html#range.requests">HTTP range
requests</a>,
don’t reliably work here as compression can render byte ranges useless. That
leads to inventing new systems for selective acquistion of bundles. Selective
removal of content from compressed bundles does seem to be possible <a href="https://dev.to/riknelix/fast-and-efficient-recompression-using-previous-compression-artifacts-47g5">at some
levels</a>,
but this leads to a complex system and the effects on other protocol
participants is non-trivial.</p>
<p>At some level, clients might want to say “just send me all the code, without
the advertising”, but that might not work so well. Asking for bundle manifests
so that content might be selectively fetched adds an additional round trip.
Moving bundle manifests out of the bundles and into content<sup class="footnote-ref"><a href="https://lowentropy.net/posts/bundles/#fn18" id="fnref18">[18]</a></sup> gives clients the information
they need to be selective about which resources they want, but it requires
moving information about the composition of resources into the content that
references it. That too requires coordination.</p>
<p>For caches, this can add an extra burden. Using the Vary HTTP header field
would be necessary to ensure that caches would not break when content from
bundles is fetched selectively<sup class="footnote-ref"><a href="https://lowentropy.net/posts/bundles/#fn19" id="fnref19">[19]</a></sup>. But it takes full awareness of these requests and how they are
applied for a cache to not be exposed to a combinatorial explosion of different
bundles as a result. Without updating caches to understand selectors, the
effect is that caches end up bearing the load for the myriad permutations of
bundles that might be needed.</p>
<h3 id="supplanting-resource-identity">Supplanting resource identity</h3>
<p>A final concern is the ability – at least in active proposals – for
bundled content to be identified with URIs from the same origin as the bundle
itself. For example, a bundle at <code>https://example.com/foo/bundle</code> might contain
content that is identified as <code>https://example.com/foo/script.js</code>. This is a
<a href="https://github.com/w3ctag/packaging-on-the-web/issues/10">long-standing
concern</a> that applies
to many previous attempts at bundling or packaging.</p>
<p>This ability is constrained, but the intent is to have content in a bundle act
as a valid substitute for other resources. The reason being that you need a
fallback for those cases where bundles aren’t optimal or aren’t available. This
has implications for anyone deploying a server, who now need to ensure that
bundles aren’t hosted adjacent to content that might not want interference from
the bundle.</p>
<p>At this point, I will note that replacing the content of other resources is
also the point of <a href="https://datatracker.ietf.org/doc/html/draft-yasskin-http-origin-signed-responses">signed
exchanges</a>.
The difference is that in signed exchanges, the replacement extends to other
origins. The constraints on what can be replaced and how are important details,
but the goal is the same: signed exchanges allow a bundle to speak for other
resources.</p>
<p>As already noted, this sort of thing is already possible with <a href="https://w3c.github.io/ServiceWorker/">service
workers</a>. Service workers take what it
means to subvert the identity of resources to the next level. A request that is
handled by a service worker can be turned into any other request or even
multiple requests. Service workers are limited though. A site can opt to
perform whatever substitutions it likes, but it can only do that for its own
requests. Bundles propose something that might be enabled for any server, even
inadvertently.</p>
<p><a href="https://github.com/WICG/resource-bundles/blob/main/subresource-loading.md#optionality-and-url-integrity">One
proposal</a>
says that all supplanted resources must be identical to the resources they
supplant. The theory there is that clients could fetch the resource from within
a bundle or directly and expect the same result. It goes on to suggest that a
mismatch between these fetches might be cause for a client to stop using the
bundle. However, it is perfectly normal in HTTP for the same resource to return
different content when fetched multiple times, even when the fetch is made by
the same client or at the same time. So it is hard to imagine how a client
would treat inconsistency as anything other than normal. If bundling provides
advantages, giving up on using bundles for that reason could make bundles
completely unreliable.</p>
<p>One good reason for enabling equivalence of bundled and unbundled resources is
to provide a graceful fallback in the case that bundling is not supported by a
client. Attempting to ensure that the internal identifiers in bundles are
“real” and that the fallback does not change behaviour is not going to work.</p>
<h4 id="indirection-for-identifiers">Indirection for identifiers</h4>
<p>Addressing the problem of one resource speaking unilaterally for another
resource requires a little creativity. Here the solution is hinted at with both
service workers and JavaScript <a href="https://wicg.github.io/import-maps/#note-on-import-specifiers">import
maps</a>. Both
allow the entity making a reference to rewrite that reference before the
browser acts on it.</p>
<p>Import maps are especially instructive here as it makes it clear that the
mapping from the import specifier to a URI is not the <a href="https://datatracker.ietf.org/doc/html/rfc3986#section-5">URI resolution function
in RFC 3986</a> or <a href="https://url.spec.whatwg.org/#url-parsing">the
URL parsing algorithm in Fetch</a>;
import specifiers are explicitly not URIs, relative or otherwise.</p>
<p>This as an opportunity to add indirection, either the limited form provided in
import maps where one string is mapped to another, or the Turing-complete
version that service workers enable.</p>
<p>That is, we allow those places that reference resources to provide the browser
with a set of rules that change howidentifiers they use are translated into
URIs. This is something that HTML has had forever, with the
<a href="https://html.spec.whatwg.org/#the-base-element"><code><base></code></a> element. This is
also the fundamental concept behind the <a href="https://discourse.wicg.io/t/proposal-fetch-maps/4259">fetch maps
proposal</a>, which looks
like this<sup class="footnote-ref"><a href="https://lowentropy.net/posts/bundles/#fn20" id="fnref20">[20]</a></sup>:</p>
<pre><code class="language-html"><script type="fetchmap">
{
"urls": {
"/styles.css": "/styles.a74fs3.css",
"/bg.png": "/bg.8e3ac4.png"
}
}
</script>
<link rel="stylesheet" href="/styles.css">
</code></pre>
<p>In this example, when the browser is asked to fetch <code>/styles.css</code>, it knows to
fetch <code>/styles.a74fs3.css</code> instead.</p>
<p>The beauty of this approach is that the change only exists where the reference
is made. The canonical identity of the resource is the same for everyone (it is
always <code>https://example.com/styles.a74fs3.css</code>), only the way that reference is
expressed changes.</p>
<p>In other words, the common property between these designs – service workers,
<code><base></code>, import maps, or fetch maps – is that the indirection only occurs at
the explicit request of the thing that makes the reference. A site deliberately
chooses to use this facility, and if it does, it controls the substitution of
resource identities. There is no lateral replacement of content as all of the
logic occurs at the point the reference is made.</p>
<h4 id="making-resource-maps-work">Making resource maps work</h4>
<p>Of course, fitting this indirections into an existing system requires a few
awkward adaptations. But it seems like this particular design could be quite
workable.</p>
<p>Anne van Kesteren pointed out that the <code>import:</code> scheme in <a href="https://github.com/WICG/import-maps#import-urls">import
maps</a> exists because many of
the places where identifiers appear are concretely URIs. APIs assume that they
can be manipulated as URIs and violating that expectation would break things
that rely on that. If we are going to enable this sort of indirection, then we
need to ensure that URIs stay URIs. That doesn’t mean that URIs need to be
HTTP, just that they are still URIs.</p>
<p>You might choose to construct identifiers with a new URI scheme in order to
satisfy this requirement<sup class="footnote-ref"><a href="https://lowentropy.net/posts/bundles/#fn21" id="fnref21">[21]</a></sup>:</p>
<pre><code class="language-html"><a href="scheme-for-mappings:hats">buy hats here</a>
</code></pre>
<p>Of course, in the fetch map example given, those identifiers look like and can
act like URIs. They can be fetched directly, without translation, if there is
no map. That’s probably a useful feature to retain as it means that you can
find local files when the reference is found in a local file during
development. Using a new scheme won’t have that advantage. A new scheme might
be an option, but it doesn’t seem to be a necessary feature of the design.</p>
<p>I can also credit Anne with the idea that we model this indirection as a
redirect, something like an HTTP 303 (See Other). The Web is already able to
manage redirection for all sorts of resources, so that would not naturally
disrupt things too much.</p>
<p>That is not to say that this is easy, as these redirects will need to conform
to established standards for the Web, with respect to the origin model and
integration with things like <a href="https://w3c.github.io/webappsec-csp/">Content Security
Policy</a>. It will need to be decided how
resource maps affect cross-origin content. And many other details will need to
be thought about carefully. But again, the design seems at least plausible.</p>
<p>Of note here is that resource maps can be polyfilled with service workers. That
suggests we might just have sites build this logic into service workers. That
could work, and it might be the basis for initial experiments. A static format
is likely superior as it makes the information more readily available.</p>
<h4 id="alternatives-and-bundle-uris">Alternatives and bundle URIs</h4>
<p>Providing indirection is just one piece of enabling use of bundled content.
Seamless integration needs two additional pieces.</p>
<p>The first is an agreed method of identifying the contents of bundles. The <a href="https://datatracker.ietf.org/group/wpack/about/">IETF
WPACK working group</a> have had
several discussions about this. These discussions were inconclusive, in part
because it was difficult to manage conflicting requirements. However, a design
grounded in a map-like construct might loosen some of the constraints that
disqualified some of the past options that were considered.</p>
<p>In particular, the idea that a bundle might itself have an implicit resource
map was not considered. That could enable the use of simple identifiers for
references between resources in the same bundle without forcing links in
bundled content to be rewritten. And any ugly URI scheme syntax for bundles
might then be abstracted away elegantly.</p>
<p>The second major piece to getting this working is a map that provides multiple
alternatives. In previous proposals, mappings were strictly one-to-one. A
one-to-many map could offer browsers a choice of resources that the referencing
entity considers to be equivalent<sup class="footnote-ref"><a href="https://lowentropy.net/posts/bundles/#fn22" id="fnref22">[22]</a></sup>. The browser is then able to select the option that it prefers. If
an alternative references a bundle the browser already has, that would be good
cause to use that option.</p>
<p>Presenting multiple options also allows browsers to experiment with different
policies with respect to fetching content when bundles are offered. If bundled
content tends to perform better on initial visits, then browsers might request
bundles then. If bundled content tends to perform poorly when there is some
valid, cached content available already, then the browser might request
individual resources in that case.</p>
<p>A resource map might be used to enable deployment of new bundling formats, or
even new retrieval methods<sup class="footnote-ref"><a href="https://lowentropy.net/posts/bundles/#fn23" id="fnref23">[23]</a></sup>.</p>
<h4 id="selective-acquisition">Selective acquisition</h4>
<p>One advantage of providing an identifier map like this is that it provides a
browser with some insight into what bundles contain before fetching them<sup class="footnote-ref"><a href="https://lowentropy.net/posts/bundles/#fn24" id="fnref24">[24]</a></sup>. Thus, a browser
might be able to make a decision about whether a bundle is worth fetching. If
most of the content is stuff that the browser does not want, then it might
choose to fetch individual resources instead.</p>
<p>Having a reference map might thereby reduce the pressure to design mechanisms
for partial bundle fetching and caching. Adding some additional metadata, like
hints about resource size, might further allow for better tuning of this logic.</p>
<p>Reference maps could even provide content classification tools more information
about resources that they can use. Even in a simple one-to-one mapping, like
with an import map, there are two identifiers that might be used to classify
content. Even if one of these is nonsense, the other could be useable.</p>
<p>While this requires a bit more sophistication on the part of classifiers, it
also provides opportunities for better classification. With alternative
sources, even if the identifier for one source does not reveal any useful
information, an alternative might.</p>
<p>Now that I’m fully into speculating about possibilities, this opens some
interesting options. The care that was taken to ensure that pages don’t break
when Google Analytics is blocked could be managed differently. Remember that
script:</p>
<pre><code class="language-js">window.dataLayer = window.dataLayer || [];
function gtag(){dataLayer.push(arguments);}
gtag('js', new Date());
gtag('config', '$XXX');
</code></pre>
<p>As you can see, the primary interface is always defined and the
<code>window.dataLayer</code> object is replaced with a dumb array if the script didn’t
load. With multiple alternatives, the fallback logic here could be encoded in
the map as a <code>data:</code> URI instead:</p>
<pre><code class="language-html"><element-for-mappings type="text/media-type-for-mappings+json">
{ "scheme-for-mappings:ga": [
"https://www.googletagmanager.com/gtag/js?id=$XXX",
"data:text/javascript;charset=utf-8;base64,d2luZG93LmRhdGFMYXllcj1bXTtmdW5jdGlvbiBndGFnKCl7ZGF0YUxheWVyLnB1c2goYXJndW1lbnRzKTt9Z3RhZygnanMnLG5ldyBEYXRlKCkpO2d0YWcoJ2NvbmZpZycsJyRYWFgnKTs="
]}</element-for-mappings>
<script async src="scheme-for-mappings:ga"></script>
</code></pre>
<p>In this case, a content blocker that decides to block the HTTPS fetch could
allow the <code>data:</code> URI and thereby preserve compatibility. Nothing really
changed, except that the fallback script is async too. Of course, this is an
unlikely outcome as this is not even remotely backward-compatible, but it does
give some hints about some of the possibilities.</p>
<h2 id="next-steps">Next steps</h2>
<p>So that was many more words than I expected to write. The size and complexity
of this problem continues to be impressive. No doubt this conversation will
continue for some time before we reach some sort of conclusion.</p>
<p>For me, the realization that it is possible to provide finer control over how
outgoing references are managed was a big deal. We don’t have to accept a
design that allows one resource speaking for others, we just have to allow for
control over how references are made. That’s a fairly substantial improvement
over most existing proposals and the basis upon which something good might be
built.</p>
<p>I still have serious reservations about the caching and performance trade-offs
involved with bundling. Attempting to solve this problem with selective
fetching of bundle contents seems like far too much complexity. Not only does
it require addressing the known-hard problem of cache invalidation, it also
requires that we find solutions to problems that have defied solutions on
numerous occasions in the past.</p>
<p>That said, I’ve concluded that giving servers the choice in how content is
assembled does not result in bad outcomes for others. Unless we include signed
exchanges<sup class="footnote-ref"><a href="https://lowentropy.net/posts/bundles/#fn25" id="fnref25">[25]</a></sup>,
we are not talking about negative externalities.</p>
<p>If we accept that selective fetching is a difficult problem, supporting bundles
might not be all-powerful from the outset. It might only give servers and
developers more options. What we learn from trying that out might give us the
information that allows us to find good solutions later. Resource maps mean
that we can always fall back to fetching resources individually. Resources maps
could even be the foundation upon which we build new experiments with
alternative resource fetching models.</p>
<p>All that said, the usability advantages provided by bundles seem to be
sufficient justification for enabling their support. That applies even if there
is uncertainty about performance. That applies even if we don’t initially solve
those performance problems. One enormous problem at a time, please.</p>
<hr class="footnotes-sep" />
<section class="footnotes">
<ol class="footnotes-list">
<li id="fn1" class="footnote-item"><p>Have I ever mentioned that I loathe CamelCase
names? Thanks 1990s. <a href="https://lowentropy.net/posts/bundles/#fnref1" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn2" class="footnote-item"><p>Yoav Weiss makes this claim based on his
experience with Chromium. I respect his experience here, but don’t know what
was done to reach this conclusion. I can see there being a lot more
investigation and discussion about this point. <a href="https://lowentropy.net/posts/bundles/#fnref2" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn3" class="footnote-item"><p>This is due to the
way congestion control algorithms operate. These start out slow in case the
network is constrained, but gradually speed up. <a href="https://lowentropy.net/posts/bundles/#fnref3" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn4" class="footnote-item"><p>Eric Rescorla suggested a possible reason that server push
regresses performance: pushing only really helps if the transmission channel
from server to client has spare capacity. Because HTTP/2 clients can make lots
of requests cheaply, it’s entirely possible that the channel is – or will soon
be – already full. If pushed resources are less important than resources the
client has already requested, even if the client eventually needs those pushed
resources, the capacity spent on pushing will delay more important responses. <a href="https://lowentropy.net/posts/bundles/#fnref4" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn5" class="footnote-item"><p>Tantek Çelik pointed out that you can use a service worker to load old
content at the same time as checking asynchronously for updates. That’s even
better. The fact is, service workers can do just about anything discussed here.
That you need to write and maintain a service worker might be enough to
discourage all but the bravest of us though. <a href="https://lowentropy.net/posts/bundles/#fnref5" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn6" class="footnote-item"><p>You might reasonably suggest that this
sort of thinking tends toward suboptimal local minima. That is a fair
criticism, but my rejoinder there might be that conditioning success on a
design that reduces to a previously unsolved problem is not really a good
strategy either. Besides, accepting suboptimal local minima is part of how we
make forward progress without endless second-guessing. <a href="https://lowentropy.net/posts/bundles/#fnref6" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn7" class="footnote-item"><p>I
seem to recall this being raised before Pete Snyder opened this issue, perhaps
at the <a href="https://datatracker.ietf.org/doc/html/rfc8752">ESCAPE workshop</a>, but I
can’t put a name to it. <a href="https://lowentropy.net/posts/bundles/#fnref7" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn8" class="footnote-item"><p>In particular, the split between style (CSS) and semantics
(HTML). <a href="https://lowentropy.net/posts/bundles/#fnref8" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn9" class="footnote-item"><p>At this point, a footnote seems necessary. Yes, a
browser is an
<a href="https://martinthomson.github.io/tmi/draft-thomson-tmi.html">intermediary</a>. All
<a href="https://datatracker.ietf.org/doc/html/draft-hildebrand-middlebox-erosion-01">previous
complaints</a>
apply. It would be dishonest to deny the possibility that a browser might abuse
its position of privilege. But that is the topic for a much longer posting. <a href="https://lowentropy.net/posts/bundles/#fnref9" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn10" class="footnote-item"><p>This more than makes up
for the overheads of the ad blocker in most cases, with page loads being
considerably faster on ad-heavy pages. <a href="https://lowentropy.net/posts/bundles/#fnref10" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn11" class="footnote-item"><p>If it isn’t clear, I’m firmly of the opinion that Google’s
<a href="https://developers.google.com/amp/cache/overview">AMP Cache</a> is not just a bad
idea, but an abuse of Google’s market dominance. It also happens to be a gross
waste of resources in a lot of cases, as Google pushes content that can be
either already present or content for links that won’t ever be followed. Of
course, if they guess right and you follow a link, navigation is fast.
Whoosh. <a href="https://lowentropy.net/posts/bundles/#fnref11" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn12" class="footnote-item"><p>With increasing amounts of scripts, interfaces might
also be expressed at the JavaScript module or function level. <a href="https://lowentropy.net/posts/bundles/#fnref12" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn13" class="footnote-item"><p>Yep. Pun totally intended. <a href="https://lowentropy.net/posts/bundles/#fnref13" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn14" class="footnote-item"><p>Worth noting here is the care Google takes to
structure the script to avoid breaking pages when their JavaScript load is
blocked by an ad blocker. <a href="https://lowentropy.net/posts/bundles/#fnref14" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn15" class="footnote-item"><p>I
wonder how many people are still fetching
<code><a href="https://ssl.google-analytics.com/ga.js">ga.js</a></code> from Google. <a href="https://lowentropy.net/posts/bundles/#fnref15" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn16" class="footnote-item"><p>This isn’t a great example, because while it
prevents the code from being identified, it’s probably not a very good
solution. For starters, the advertiser no longer sees requests that come
directly from browsers, which it might use to track people. <a href="https://lowentropy.net/posts/bundles/#fnref16" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn17" class="footnote-item"><p>Note that, at least for ad blocking, the biggest gains come from not
<em>executing</em> unwanted content, as executing ad content almost always leads to a
chain of additional fetches. Saving the CPU time is the third major component
to savings. <a href="https://lowentropy.net/posts/bundles/#fnref17" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn18" class="footnote-item"><p>Yes, that
effectively means bundling them with content. <a href="https://lowentropy.net/posts/bundles/#fnref18" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn19" class="footnote-item"><p>Curiously, the
<a href="https://datatracker.ietf.org/doc/html/draft-ietf-httpbis-variants">Variants</a>
design is might not be a good fit here as it provides enumeration of
alternatives, which is tricky for the same reason that caching in ignorance of
bundling is. <a href="https://lowentropy.net/posts/bundles/#fnref19" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn20" class="footnote-item"><p>There is lots to quibble about in the exact spelling in this
example, but I just copied from the proposal directly. <a href="https://lowentropy.net/posts/bundles/#fnref20" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn21" class="footnote-item"><p>It’s tempting here to suggest <code>urn:</code>, but that might
cause some heads to explode. <a href="https://lowentropy.net/posts/bundles/#fnref21" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn22" class="footnote-item"><p>The thought occurs that this is something
that could be exploited to allow for safe patching of dependencies when
combined with semantic versioning. For instance, I will accept any version
<code>X.Y.?</code> of this file greater than <code>X.Y.Z</code>. We can leave that idea for another
day though. <a href="https://lowentropy.net/posts/bundles/#fnref22" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn23" class="footnote-item"><p>Using <a href="https://ipfs.io/">IPFS</a> seems far more
plausible if you allow it as one option of many with the option for graceful
fallback. <a href="https://lowentropy.net/posts/bundles/#fnref23" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn24" class="footnote-item"><p>To
what extent providing information ahead of time can be used to improve
performance is something that I have often wondered about; it seems like it has
some interesting trade-offs that might be worth studying. <a href="https://lowentropy.net/posts/bundles/#fnref24" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn25" class="footnote-item"><p>We’ve already established that <a href="https://mozilla.github.io/standards-positions/#http-origin-signed-responses">signed exchanges are not good for
the
Web</a>. <a href="https://lowentropy.net/posts/bundles/#fnref25" class="footnote-backref">↩︎</a></p>
</li>
</ol>
</section>