Goals
The goal is to unify fetching across the web platform and provide consistent handling of everything that involves, including:
- URL schemes
- Redirects
- Cross-origin semantics
- CSP [CSP]
- Fetch Metadata [FETCH-METADATA]
- Service workers [SW]
- Mixed Content [MIX]
- Upgrade Insecure Requests [UPGRADE-INSECURE-REQUESTS]
- `
Referer
` [REFERRER]
To do so it also supersedes the HTTP `Origin
` header semantics
originally defined in The Web Origin Concept. [ORIGIN]
1. Preface
At a high level, fetching a resource is a fairly simple operation. A request goes in, a response comes out. The details of that operation are however quite involved and used to not be written down carefully and differ from one API to the next.
Numerous APIs provide the ability to fetch a resource, e.g. HTML’s img
and
script
element, CSS' cursor
and list-style-image
,
the navigator.sendBeacon()
and self.importScripts()
JavaScript
APIs. The Fetch Standard provides a unified architecture for these features so they are
all consistent when it comes to various aspects of fetching, such as redirects and the
CORS protocol.
The Fetch Standard also defines the fetch()
JavaScript API, which
exposes most of the networking functionality at a fairly low level of abstraction.
2. Infrastructure
This specification depends on the Infra Standard. [INFRA]
This specification uses terminology from ABNF, Encoding, HTML, HTTP, MIME Sniffing, Streams, URL, Web IDL, and WebSockets. [ABNF] [ENCODING] [HTML] [HTTP] [MIMESNIFF] [STREAMS] [URL] [WEBIDL] [WEBSOCKETS]
ABNF means ABNF as augmented by HTTP (in particular the addition of #
)
and RFC 7405. [RFC7405]
Credentials are HTTP cookies, TLS client certificates, and authentication entries (for HTTP authentication). [COOKIES] [TLS] [HTTP]
A fetch params is a struct used as a bookkeeping detail by the fetch algorithm. It has the following items:
- request
- A request.
- process request body chunk length
(default null)
- process request end-of-body (default null)
- process early hints response (default null)
- process response (default null)
- process response end-of-body (default null)
- process response consume body (default null)
- process request end-of-body (default null)
- Null or an algorithm.
- task destination (default null)
- Null, a global object, or a parallel queue.
- cross-origin isolated capability (default false)
- A boolean.
- controller (default a new fetch controller)
- A fetch controller.
- timing info
- A fetch timing info.
- preloaded response candidate (default null)
- Null, "
pending
", or a response.
A fetch controller is a struct used to enable callers of fetch to perform certain operations on it after it has started. It has the following items:
- state (default "
ongoing
") - "
ongoing
", "terminated
", or "aborted
" - full timing info (default null)
- Null or a fetch timing info.
- report timing steps (default null)
- Null or an algorithm accepting a global object.
- serialized abort reason (default null)
- Null or a Record (result of StructuredSerialize).
- next manual redirect steps (default null)
- Null or an algorithm accepting nothing.
To report timing for a fetch controller controller given a global object global:
-
Assert: controller’s report timing steps is non-null.
-
Call controller’s report timing steps with global.
To process the next manual redirect for a fetch controller controller:
-
Assert: controller’s next manual redirect steps is non-null.
-
Call controller’s next manual redirect steps.
To extract full timing info given a fetch controller controller:
-
Assert: controller’s full timing info is non-null.
-
Return controller’s full timing info.
To abort a fetch controller controller with an optional error:
-
Set controller’s state to "
aborted
". -
Let fallbackError be an "
AbortError
"DOMException
. -
Set error to fallbackError if it is not given.
-
Let serializedError be StructuredSerialize(error). If that threw an exception, catch it, and let serializedError be StructuredSerialize(fallbackError).
-
Set controller’s serialized abort reason to serializedError.
To deserialize a serialized abort reason, given null or a Record abortReason and a realm realm:
-
Let fallbackError be an "
AbortError
"DOMException
. -
Let deserializedError be fallbackError.
-
If abortReason is non-null, then set deserializedError to StructuredDeserialize(abortReason, realm). If that threw an exception or returned undefined, then set deserializedError to fallbackError.
-
Return deserializedError.
To terminate a fetch controller
controller, set controller’s state to
"terminated
".
A fetch params fetchParams is aborted if
its controller’s state is
"aborted
".
A fetch params fetchParams is canceled if
its controller’s state is
"aborted
" or "terminated
".
A fetch timing info is a struct used to maintain timing information needed by Resource Timing and Navigation Timing. It has the following items: [RESOURCE-TIMING] [NAVIGATION-TIMING]
- start time (default 0)
- redirect start time (default 0)
- redirect end time (default 0)
- post-redirect start time (default 0)
- final service worker start time (default 0)
- final network-request start time (default 0)
- first interim network-response start time (default 0)
- final network-response start time (default 0)
- end time (default 0)
- redirect start time (default 0)
- A
DOMHighResTimeStamp
. - final connection timing info (default null)
- Null or a connection timing info.
- server-timing headers (default « »)
- A list of strings.
- render-blocking (default false)
- A boolean.
A response body info is a struct used to maintain information needed by Resource Timing and Navigation Timing. It has the following items: [RESOURCE-TIMING] [NAVIGATION-TIMING]
- encoded size
(default 0)
- decoded size (default 0)
- A number.
- content type (default the empty string)
- An ASCII string.
- content encoding (default the empty string)
- An ASCII string.
To create an opaque timing info, given a fetch timing info timingInfo, return a new fetch timing info whose start time and post-redirect start time are timingInfo’s start time.
To queue a fetch task, given an algorithm algorithm, a global object or a parallel queue taskDestination, run these steps:
-
If taskDestination is a parallel queue, then enqueue algorithm to taskDestination.
-
Otherwise, queue a global task on the networking task source with taskDestination and algorithm.
To serialize an integer, represent it as a string of the shortest possible decimal number.
This will be replaced by a more descriptive algorithm in Infra. See infra/201.
2.1. URL
A local scheme is "about
", "blob
", or
"data
".
A URL is local if its scheme is a local scheme.
This definition is also used by Referrer Policy. [REFERRER]
An HTTP(S) scheme is "http
" or
"https
".
A fetch scheme is "about
", "blob
",
"data
", "file
", or an HTTP(S) scheme.
HTTP(S) scheme and fetch scheme are also used by HTML. [HTML]
2.2. HTTP
While fetching encompasses more than just HTTP, it
borrows a number of concepts from HTTP and applies these to resources obtained via other
means (e.g., data
URLs).
An HTTP tab or space is U+0009 TAB or U+0020 SPACE.
HTTP whitespace is U+000A LF, U+000D CR, or an HTTP tab or space.
HTTP whitespace is only useful for specific constructs that are reused outside the context of HTTP headers (e.g., MIME types). For HTTP header values, using HTTP tab or space is preferred, and outside that context ASCII whitespace is preferred. Unlike ASCII whitespace this excludes U+000C FF.
An HTTP newline byte is 0x0A (LF) or 0x0D (CR).
An HTTP tab or space byte is 0x09 (HT) or 0x20 (SP).
An HTTP whitespace byte is an HTTP newline byte or HTTP tab or space byte.
To collect an HTTP quoted string from a string input, given a position variable position and an optional boolean extract-value (default false):
-
Let positionStart be position.
-
Let value be the empty string.
-
Assert: the code point at position within input is U+0022 (").
-
Advance position by 1.
-
While true:
-
Append the result of collecting a sequence of code points that are not U+0022 (") or U+005C (\) from input, given position, to value.
-
If position is past the end of input, then break.
-
Let quoteOrBackslash be the code point at position within input.
-
Advance position by 1.
-
If quoteOrBackslash is U+005C (\), then:
-
If position is past the end of input, then append U+005C (\) to value and break.
-
Append the code point at position within input to value.
-
Advance position by 1.
-
-
Otherwise:
-
-
If extract-value is true, then return value.
-
Return the code points from positionStart to position, inclusive, within input.
Input | Output | Output with extract-value set to true | Final position variable value |
---|---|---|---|
""\ "
| ""\ "
| "\ "
| 2 |
""Hello" World "
| ""Hello" "
| "Hello "
| 7 |
""Hello \\ World\"" "
| ""Hello \\ World\"" "
| "Hello \ World" "
| 18 |
The position variable always starts at 0 in these examples.
2.2.1. Methods
A method is a byte sequence that matches the method token production.
A CORS-safelisted method is a
method that is `GET
`,
`HEAD
`, or `POST
`.
A forbidden method is a method that is a
byte-case-insensitive match for `CONNECT
`,
`TRACE
`, or `TRACK
`.
[HTTPVERBSEC1], [HTTPVERBSEC2], [HTTPVERBSEC3]
To normalize a
method, if it is a byte-case-insensitive
match for `DELETE
`, `GET
`,
`HEAD
`, `OPTIONS
`, `POST
`, or
`PUT
`, byte-uppercase it.
Normalization is done for backwards compatibility and consistency across APIs as methods are actually "case-sensitive".
Using `patch
` is highly likely to result in a
`405 Method Not Allowed
`. `PATCH
` is much more likely to
succeed.
There are no restrictions on methods. `CHICKEN
` is perfectly
acceptable (and not a misspelling of `CHECKIN
`). Other than those that are
normalized there are no casing restrictions either.
`Egg
` or `eGg
` would be fine, though uppercase is encouraged for
consistency.
2.2.2. Headers
HTTP generally refers to a header as a "field" or "header field". The web platform uses the more colloquial term "header". [HTTP]
A header list is a list of zero or more headers. It is initially « ».
A header list is essentially a specialized multimap: an ordered list of
key-value pairs with potentially duplicate keys. Since headers other than `Set-Cookie
`
are always combined when exposed to client-side JavaScript, implementations could choose a more
efficient representation, as long as they also support an associated data structure for
`Set-Cookie
` headers.
To get a structured field value given a header name name and a string type from a header list list, run these steps. They return null or a structured field value.
-
Assert: type is one of "
dictionary
", "list
", or "item
". -
Let value be the result of getting name from list.
-
If value is null, then return null.
-
Let result be the result of parsing structured fields with input_string set to value and header_type set to type.
-
If parsing failed, then return null.
-
Return result.
Get a structured field value intentionally does not distinguish between a header not being present and its value failing to parse as a structured field value. This ensures uniform processing across the web platform.
To set a structured field value given a tuple (header name name, structured field value structuredValue), in a header list list:
-
Let serializedValue be the result of executing the serializing structured fields algorithm on structuredValue.
-
Set (name, serializedValue) in list.
Structured field values are defined as objects which HTTP can (eventually) serialize in interesting and efficient ways. For the moment, Fetch only supports header values as byte sequences, which means that these objects can be set in header lists only via serialization, and they can be obtained from header lists only by parsing. In the future the fact that they are objects might be preserved end-to-end. [RFC9651]
A header list list contains a header name name if list contains a header whose name is a byte-case-insensitive match for name.
To get a header name name from a header list list, run these steps. They return null or a header value.
-
If list does not contain name, then return null.
-
Return the values of all headers in list whose name is a byte-case-insensitive match for name, separated from each other by 0x2C 0x20, in order.
To get, decode, and split a header name name from header list list, run these steps. They return null or a list of strings.
-
Let value be the result of getting name from list.
-
If value is null, then return null.
-
Return the result of getting, decoding, and splitting value.
This is how get, decode, and split functions in practice with
`A
` as the name argument:
Headers (as on the network) | Output |
---|---|
| « "nosniff ", "" »
|
| |
| « "" » |
| null |
| « "text/html;", x/x " »
|
| |
| « "x/x;test="hi" ", "y/y " »
|
| |
| « "x / x ", "", "", "1 " »
|
| |
| « ""1,2" ", "3 " »
|
|
To get, decode, and split a header value value, run these steps. They return a list of strings.
-
Let input be the result of isomorphic decoding value.
-
Let position be a position variable for input, initially pointing at the start of input.
-
Let temporaryValue be the empty string.
-
While true:
-
Append the result of collecting a sequence of code points that are not U+0022 (") or U+002C (,) from input, given position, to temporaryValue.
The result might be the empty string.
-
If position is not past the end of input and the code point at position within input is U+0022 ("):
-
Append the result of collecting an HTTP quoted string from input, given position, to temporaryValue.
- If position is not past the end of input, then continue.
-
-
Remove all HTTP tab or space from the start and end of temporaryValue.
-
Append temporaryValue to values.
-
Set temporaryValue to the empty string.
-
If position is past the end of input, then return values.
-
Assert: the code point at position within input is U+002C (,).
-
Advance position by 1.
-
Except for blessed call sites, the algorithm directly above is not to be invoked directly. Use get, decode, and split instead.
To append a header (name, value) to a header list list:
To delete a header name name from a header list list, remove all headers whose name is a byte-case-insensitive match for name from list.
To set a header (name, value) in a header list list:
To combine a header (name, value) in a header list list:
-
If list contains name, then set the value of the first such header to its value, followed by 0x2C 0x20, followed by value.
-
Otherwise, append (name, value) to list.
Combine is used by XMLHttpRequest
and the
WebSocket protocol handshake.
To convert header names to a sorted-lowercase set, given a list of names headerNames, run these steps. They return an ordered set of header names.
-
Let headerNamesSet be a new ordered set.
-
For each name of headerNames, append the result of byte-lowercasing name to headerNamesSet.
-
Return the result of sorting headerNamesSet in ascending order with byte less than.
To sort and combine a header list list, run these steps. They return a header list.
-
Let headers be a header list.
-
Let names be the result of convert header names to a sorted-lowercase set with all the names of the headers in list.
-
For each name of names:
-
If name is `
set-cookie
`, then: -
Otherwise:
-
-
Return headers.
A header is a tuple that consists of a name (a header name) and value (a header value).
A header name is a byte sequence that matches the field-name token production.
A header value is a byte sequence that matches the following conditions:
-
Has no leading or trailing HTTP tab or space bytes.
-
Contains no 0x00 (NUL) or HTTP newline bytes.
The definition of header value is not defined in terms of the field-value token production as it is not compatible with deployed content.
To normalize a byte sequence potentialValue, remove any leading and trailing HTTP whitespace bytes from potentialValue.
To determine whether a header (name, value) is a CORS-safelisted request-header, run these steps:
-
If value’s length is greater than 128, then return false.
-
Byte-lowercase name and switch on the result:
- `
accept
` -
If value contains a CORS-unsafe request-header byte, then return false.
- `
accept-language
`- `
content-language
` - `
-
If value contains a byte that is not in the range 0x30 (0) to 0x39 (9), inclusive, is not in the range 0x41 (A) to 0x5A (Z), inclusive, is not in the range 0x61 (a) to 0x7A (z), inclusive, and is not 0x20 (SP), 0x2A (*), 0x2C (,), 0x2D (-), 0x2E (.), 0x3B (;), or 0x3D (=), then return false.
- `
content-type
` -
-
If value contains a CORS-unsafe request-header byte, then return false.
-
Let mimeType be the result of parsing the result of isomorphic decoding value.
-
If mimeType is failure, then return false.
-
If mimeType’s essence is not "
application/x-www-form-urlencoded
", "multipart/form-data
", or "text/plain
", then return false.
This intentionally does not use extract a MIME type as that algorithm is rather forgiving and servers are not expected to implement it.
If extract a MIME type were used the following request would not result in a CORS preflight and a naïve parser on the server might treat the request body as JSON:
fetch
( "https://victim.example/naïve-endpoint" , { method: "POST" , headers: [ [ "Content-Type" , "application/json" ], [ "Content-Type" , "text/plain" ] ], credentials: "include" , body: JSON. stringify( exerciseForTheReader) }); -
- `
range
` -
-
Let rangeValue be the result of parsing a single range header value given value and false.
-
If rangeValue is failure, then return false.
-
If rangeValue[0] is null, then return false.
As web browsers have historically not emitted ranges such as `
bytes=-500
` this algorithm does not safelist them.
-
- Otherwise
-
Return false.
- `
-
Return true.
There are limited exceptions to the `Content-Type
` header safelist, as
documented in CORS protocol exceptions.
A CORS-unsafe request-header byte is a byte byte for which one of the following is true:
-
byte is less than 0x20 and is not 0x09 HT
-
byte is 0x22 ("), 0x28 (left parenthesis), 0x29 (right parenthesis), 0x3A (:), 0x3C (<), 0x3E (>), 0x3F (?), 0x40 (@), 0x5B ([), 0x5C (\), 0x5D (]), 0x7B ({), 0x7D (}), or 0x7F DEL.
The CORS-unsafe request-header names, given a header list headers, are determined as follows:
-
Let unsafeNames be a new list.
-
Let potentiallyUnsafeNames be a new list.
-
Let safelistValueSize be 0.
-
For each header of headers:
-
If safelistValueSize is greater than 1024, then for each name of potentiallyUnsafeNames, append name to unsafeNames.
-
Return the result of convert header names to a sorted-lowercase set with unsafeNames.
A CORS non-wildcard request-header name is a header name that is a
byte-case-insensitive match for `Authorization
`.
A privileged no-CORS request-header name is a header name that is a byte-case-insensitive match for one of
- `
Range
`.
These are headers that can be set by privileged APIs, and will be preserved if their associated request object is copied, but will be removed if the request is modified by unprivileged APIs.
`Range
` headers are commonly used by downloads
and media fetches.
A helper is provided to add a range header to a particular request.
A CORS-safelisted response-header name, given a list of header names list, is a header name that is a byte-case-insensitive match for one of
- `
Cache-Control
` - `
Content-Language
` - `
Content-Length
` - `
Content-Type
` - `
Expires
` - `
Last-Modified
` - `
Pragma
` - Any item in list that is not a forbidden response-header name.
A no-CORS-safelisted request-header name is a header name that is a byte-case-insensitive match for one of
- `
Accept
` - `
Accept-Language
` - `
Content-Language
` - `
Content-Type
`
To determine whether a header (name, value) is a no-CORS-safelisted request-header, run these steps:
-
If name is not a no-CORS-safelisted request-header name, then return false.
-
Return whether (name, value) is a CORS-safelisted request-header.
A header (name, value) is forbidden request-header if these steps return true:
-
If name is a byte-case-insensitive match for one of:
- `
Accept-Charset
` - `
Accept-Encoding
` - `
Access-Control-Request-Headers
` - `
Access-Control-Request-Method
` - `
Connection
` - `
Content-Length
` - `
Cookie
` - `
Cookie2
` - `
Date
` - `
DNT
` - `
Expect
` - `
Host
` - `
Keep-Alive
` - `
Origin
` - `
Referer
` - `
Set-Cookie
` - `
TE
` - `
Trailer
` - `
Transfer-Encoding
` - `
Upgrade
` - `
Via
`
then return true.
- `
-
If name when byte-lowercased starts with `
proxy-
` or `sec-
`, then return true. -
If name is a byte-case-insensitive match for one of:
- `
X-HTTP-Method
` - `
X-HTTP-Method-Override
` - `
X-Method-Override
`
then:
-
Let parsedValues be the result of getting, decoding, and splitting value.
-
For each method of parsedValues: if the isomorphic encoding of method is a forbidden method, then return true.
- `
-
Return false.
These are forbidden so the user agent remains in full control over them.
Header names starting with `Sec-
` are reserved to allow new
headers to be minted that are safe from APIs using fetch that allow
control over headers by developers, such as XMLHttpRequest
. [XHR]
The `Set-Cookie
` header is semantically a response header, so it is not useful on
requests. Because `Set-Cookie
` headers cannot be combined, they require more complex
handling in the Headers
object. It is forbidden here to avoid leaking this complexity into
requests.
A forbidden response-header name is a header name that is a byte-case-insensitive match for one of:
- `
Set-Cookie
` - `
Set-Cookie2
`
A request-body-header name is a header name that is a byte-case-insensitive match for one of:
- `
Content-Encoding
` - `
Content-Language
` - `
Content-Location
` - `
Content-Type
`
To extract header values given a header header, run these steps:
To extract header list values given a header name name and a header list list, run these steps:
-
If list does not contain name, then return null.
-
If the ABNF for name allows a single header and list contains more than one, then return failure.
If different error handling is needed, extract the desired header first.
-
Let values be an empty