Prefetch

Draft Community Group Report,

This version:
https://wicg.github.io/nav-speculation/prefetch.html
Issue Tracking:
GitHub
Inline In Spec
Editors:
(Google)
(Google)

Abstract

A specification for navigational prefetching

Status of this document

This specification was published by the Web Platform Incubator Community Group. It is not a W3C Standard nor is it on the W3C Standards Track. Please note that under the W3C Community Contributor License Agreement (CLA) there is a limited opt-out and other conditions apply. Learn more about W3C Community and Business Groups.

1. Concepts

In light of storage partitioning, this specification defines prefetch for navigations which would occur within the same partition (for example, top-level navigations within the same site) and for navigations which would occur in a separate partition (for example, top-level navigations to a different site).

Conflicting credentials exist for response response given navigable navigable and network partition key sourcePartitionKey if the following steps return true:
  1. Let hypotheticalEnvironment be the result of creating a reserved client given navigable, response’s URL, and null.

  2. Let hypotheticalPartitionKey be the result of determining the network partition key given hypotheticalEnvironment.

  3. If hypotheticalPartitionKey is equal to sourcePartitionKey or there are no credentials associated with URL and hypotheticalPartitionKey, then return false.

  4. Return true.


An exchange record is a struct with the following items:

These records can be used to defer checks that would ordinarily happen during a navigate fetch, and to check for modified credentials.

A redirect chain is a list of exchange records.

To update a redirect chain redirectChain given a request request and response response:
  1. Assert: redirectChain is not empty.

  2. Assert: redirectChain’s last item’s request is the same as request and its response is null.

  3. Set redirectChain’s last item’s request to a clone of request.

    The cloning ensures that any modifications made to request during further redirects do not affect the stored request, which later needs to be consulted at activation time (during create navigation params from a prefetch record) to get a full picture of the original redirect chain. The only currently known case where a request field is mutated and is also consulted during activation is the referrer field, but for specification robustness we clone the entire request.

    Since prefetches are only ever performed for `GET` requests, the request body is always null, so cloning is relatively straightforward.

  4. Set redirectChain’s last item’s response to response.


Each Document has prefetch records, which is a list of prefetch records.

A prefetch record is a struct with the following items:

This tracks prefetches from when they are started to when they are ultimately used or discarded. Consequently some of these fields are immutable, some pertain to the ongoing activity (like fetch controller), and some (like expiry time) are populated when the prefetch completes.

A request which would have ordinarily sent credentials but could not due to cross-partition prefetch causes a prefetch to be abandoned.

A prefetch record’s response is the response of the last element of its redirect chain, or null if that list is empty.

The user agent may cancel and discard records from the prefetch records even if they are not expired, e.g., due to resource constraints. Since completed records with expiry times in the past will never be matching prefetch records, they can be removed with no observable consequences.

A prefetch record prefetchRecord matches a URL given a URL url if the following algorithm returns true:
  1. If prefetchRecord’s URL is equal to url, return true.

  2. If prefetchRecord’s response is not null:

    1. Let searchVariance be the result of obtaining a URL search variance given prefetchRecord’s redirect chain[0]'s response.

      It is important that we check the 0th response, not the last response (i.e. not prefetchRecord’s response), because the prefetch record’s URL is that of the first request/response pair, and so it is only that first response’s `No-Vary-Search` header that should impact matching.

    2. If prefetchRecord’s URL and url are equivalent modulo search variance given searchVariance, return true.

  3. Otherwise, return false.

A prefetch record prefetchRecord is expected to match a URL given a URL url if the following algorithm returns true:
  1. If prefetchRecord matches a URL given url, return true.

  2. If prefetchRecord’s response is null:

    1. Let searchVariance be prefetchRecord’s No-Vary-Search hint.

    2. If prefetchRecord’s URL and url are equivalent modulo search variance given searchVariance, return true.

  3. Otherwise, return false.

To cancel and discard a prefetch record prefetchRecord given a Document document, perform the following steps.
  1. Assert: prefetchRecord is in document’s prefetch records.

  2. Assert: prefetchRecord’s state is not "canceled".

  3. Set prefetchRecord’s state to "canceled".

  4. Abort prefetchRecord’s fetch controller. This will cause any ongoing fetch to be canceled and yield a network error.

  5. If prefetchRecord’s prerendering traversable is a traversable navigable, then destroy it.

  6. Remove prefetchRecord from document’s prefetch records.

  7. Trigger a prefetch status updated event given document’s node navigable, prefetchRecord, and "failure" status.

This means that even a completed prefetch or prerender will not be activated. However, the process of prefetching or prerendering might have modified the HTTP cache, making subsequent navigations faster anyway.

To complete a prefetch record prefetchRecord given Document document, perform the following steps.
  1. Assert: document is fully active.

  2. Let currentTime be the current high resolution time for the relevant global object of document.

  3. Let expiryTime be currentTime + 300000 (i.e., five minutes).

  4. Remove all elements of document’s prefetch records which have the same URL as prefetchRecord and whose state equals "completed".

  5. Set prefetchRecord’s state to "completed" and expiry time to expiryTime.

  6. Trigger a prefetch status updated event given document’s node navigable, prefetchRecord, and "ready" status.

To find a matching complete prefetch record given a top-level traversable navigable, source snapshot params sourceSnapshotParams, and URL url:
  1. Let exactRecord be null.

  2. Let inexactRecord be null.

  3. For each record of sourceSnapshotParams’s prefetch records:

    1. If record’s state is not "completed", then continue.

    2. If record’s URL is equal to url:

      1. Set exactRecord to record.

      2. Break.

    3. If inexactRecord is null and record matches a URL given url:

      1. Set inexactRecord to record.

  4. Let recordToUse be exactRecord if exactRecord is not null, otherwise inexactRecord.

  5. If recordToUse is not null:

    1. Let currentTime be the current high resolution time for navigable’s active document.

    2. If recordToUse’s expiry time is less than currentTime:

      1. Trigger a prefetch status updated event given navigable, recordToUse, and "failure" status.

      2. Return null.

    3. For each exchangeRecord of recordToUse’s redirect chain:

      1. If conflicting credentials exist for exchangeRecord’s response given navigable and recordToUse’s source partition key:

        1. Trigger a prefetch status updated event given navigable, recordToUse, and "failure" status.

        2. Return null.

      This handles the case where there were no cross-partition credentials initially, but there are now. User agents could use a slightly coarser algorithm, such as monitoring whether the cookies for the URL have been modified at all, or storing a hash, timestamp or revision number.
    4. Return recordToUse.

  6. Return null.

It’s not obvious, but this doesn’t actually require that the prefetch have received the complete body, just the response headers. In particular, a navigation to a prefetched response might nonetheless not load instantaneously.

It might be possible to use cache response headers to determine when a response can be used multiple times, but given the short lifetime of the prefetch buffer it’s unclear whether this is worthwhile.

To wait for a matching prefetch record given a top-level traversable navigable, source snapshot params sourceSnapshotParams, and URL url:
  1. Assert: this is running in parallel.

  2. Let timeout be null.

  3. Optionally, set timeout to an implementation-defined duration representing the maximum time the implementation is willing to wait for an ongoing prefetch to respond, before it gives up and restarts the navigation.

    Choosing a good timeout is difficult, and might depend on the exact initiator of the navigation or the prefetch (e.g., its speculation rule eagerness, or whether it will be used for a prerender). In general, a timeout is best avoided, as it is rare that restarting the navigation as a non-prefetch will give a better result than awaiting the ongoing prefetch. But some implementations have found that in some situations, a timeout on the order of seconds can give better results.

  4. Let startTime be the unsafe shared current time.

  5. Run these steps, but abort when timeout is not null and the unsafe shared current timestartTime is greater than timeout:

    1. While true:

      1. Let completeRecord be the result of finding a matching complete prefetch record given navigable, sourceSnapshotParams, and url.

      2. If completeRecord is not null, return completeRecord.

      3. Let potentialRecords be an empty list.

      4. For each record of sourceSnapshotParams’s prefetch records:

        1. If all of the following are true:

          then append record to potentialRecords.

        Each iteration of the loop recomputes potentialRecords, in a way so that a subsequent iteration’s potentialRecords will be a subset of the previous iteration’s potentialRecords. This follows from how sourceSnapshotParams’s prefetch records is a static snapshot, and none of these three conditions can flip from false to true.

        An equivalent strategy would be to build a single initial instance of potentialRecords, and remove items from it as they no longer meet the criteria.

      5. If potentialRecords is empty, return null.

      6. Wait until the state of any element of sourceSnapshotParams’s prefetch records changes.

  6. Return null.

Because sourceSnapshotParams’s prefetch records are a snapshot, prefetches that start after a navigation cannot be activated as part of that navigation.

A prefetch record prefetchRecord is still being speculated, given a list of speculative load candidates candidates, if the following steps return true:
  1. For each candidate of candidates:

    1. If prefetchRecord does not match a URL given candidate’s URL, then continue.

    2. If candidate is a prefetch candidate and prefetchRecord’s anonymization policy does not equal candidate’s anonymization policy, then continue.

    3. If candidate is a prerender candidate and prefetchRecord’s prerendering traversable is null, then continue.

    4. Return true.

  2. Return false.

A Document document has a matching prefetch record given a prefetch record recordUnderConsideration and an optional boolean checkPrerender (default false) if the following algorithm returns true:
  1. For each record of document’s prefetch records:

    1. If record’s state is "canceled", then continue.

    2. If checkPrerender is true:

      1. If record’s prerendering traversable is null, then continue.

      2. If record’s prerendering target navigable name hint is not equal to recordUnderConsideration’s prerendering target navigable name hint, then the user agent may continue.

        User agents which create separate prerendering traversables depending on the prerendering target navigable name hint will continue here, since they consider such records to be distinct. Whereas, user agents which are able to activate an existing prerendering traversable regardless of the prerendering target navigable name hint will consider such records equivalent.

    3. Let recordNVS be null.

    4. If record’s response is not null, then set recordNVS to the result of obtaining a URL search variance given record’s response.

    5. Otherwise, set recordNVS to record’s No-Vary-Search hint.

    6. If recordUnderConsideration’s No-Vary-Search hint is not equal to recordNVS, then continue.

    7. If recordUnderConsideration’s URL and record’s URL are equivalent modulo search variance given recordUnderConsideration’s No-Vary-Search hint, then return true.

  2. Return false.

See the discussion `No-Vary-Search` comparison in the HTML Standard.

To create navigation params from a prefetch record given a top-level traversable navigable, a document state documentState, a navigation id navigationId, a NavigationTimingType navTimingType, a request request, a prefetch record record, a target snapshot params targetSnapshotParams, and a source snapshot params sourceSnapshotParams, perform the following steps.
  1. Let responseOrigin be null.

  2. Let responseCOOP be null.

  3. Let coopEnforcementResult be the result of creating a cross-origin opener policy enforcement result for navigation given navigable’s active document and documentState’s initiator origin.

  4. Let finalSandboxFlags be an empty sandboxing flag set.

  5. Let responsePolicyContainer be null.

  6. Let urlList be an empty list.

  7. Let response be record’s response.

  8. For each exchangeRecord in record’s redirect chain:

    1. Let redirectChainRequest be exchangeRecord’s request.

    2. Let redirectChainResponse be exchangeRecord’s response.

    3. Append redirectChainRequest’s URL to urlList.

    4. Set responsePolicyContainer to the result of creating a policy container from a fetch response given redirectChainResponse and redirectChainRequest’s reserved client.

    5. Set finalSandboxFlags to the union of targetSnapshotParams’s sandboxing flags and responsePolicyContainer’s CSP list’s CSP-derived sandboxing flags.

    6. Set responseOrigin to the result of determining the origin given redirectChainResponse’s URL, finalSandboxFlags, documentState’s initiator origin, and null.

    7. Set responseCOOP to the result of obtaining a cross-origin opener policy given redirectChainResponse and redirectChainRequest’s reserved client.

    8. Set coopEnforcementResult to the result of enforcing a response’s cross-origin opener policy given navigable’s active browsing context, redirectChainResponse’s URL, responseOrigin, responseCOOP, coopEnforcementResult, and redirectChainRequest’s referrer.

    9. If finalSandboxFlags is not empty and responseCOOP’s value is "unsafe-none", then set response to an appropriate network error and break.

  9. If request’s URL is not equal to urlList[0], then insert request’s URL into urlList after the 0th item.

    In this case, we are navigating to request’s URL, but fulfilling it with a prefetch that came from a response whose URL is urlList[0], due to `No-Vary-Search`. We treat this as if there was a redirect from the 0th response to URL. If, after this insertion, urlList’s size is 2, then the resulting Document will use the navigated-to URL. Otherwise, if the size is greater, then this will have no effect.

    Consider the case where we prefetch /page?param=1, and the server responds with No-Vary-Search: params=("param") and a 200 response code.

    Later, if we navigate to /page?param=2, the prefetch will be used. This step will cause urlList to become « /page?param=1, /page?param=2 », so the resulting Document will have /page?param=2 as its URL.