URL

Goals

The URL standard takes the following approach towards making URLs fully interoperable:

• Align RFC 3986 and RFC 3987 with contemporary implementations and obsolete the RFCs in the process. (E.g., spaces, other "illegal" code points, query encoding, equality, canonicalization, are all concepts not entirely shared, or defined.) URL parsing needs to become as solid as HTML parsing. [RFC3986] [RFC3987]

• Standardize on the term URL. URI and IRI are just confusing. In practice a single algorithm is used for both so keeping them distinct is not helping anyone. URL also easily wins the search result popularity contest.

• Supplanting Origin of a URI [sic]. [RFC6454]

• Define URL’s existing JavaScript API in full detail and add enhancements to make it easier to work with. Add a new URL object as well for URL manipulation without usage of HTML elements. (Useful for JavaScript worker environments.)

• Ensure the combination of parser, serializer, and API guarantee idempotence. For example, a non-failure result of a parse-then-serialize operation will not change with any further parse-then-serialize operations applied to it. Similarly, manipulating a non-failure result through the API will not change from applying any number of serialize-then-parse operations to it.

As the editors learn more about the subject matter the goals might increase in scope somewhat.

1. Infrastructure

This specification depends on Infra. [INFRA]

Some terms used in this specification are defined in the following standards and specifications:

• Encoding [ENCODING]
• File API [FILEAPI]
• HTML [HTML]
• Unicode IDNA Compatibility Processing [UTS46]
• Web IDL [WEBIDL]

To serialize an integer, represent it as the shortest possible decimal number.

1.1. Writing

A validation error indicates a mismatch between input and valid input. User agents, especially conformance checkers, are encouraged to report them somewhere.

const url = new URL("https:foo.html", "https://example.org/");

const url = new URL("💩");

const url = new URL("💩", "mailto:[email protected]");

const url = new URL("/c:/path/to/file", "file:///c:/");

1.2. Parsers

The EOF code point is a conceptual code point that signifies the end of a string or code point stream.

A pointer for a string input is an integer that points to a code point within input. Initially it points to the start of input. If it is −1 it points nowhere. If it is greater than or equal to input’s code point length, it points to the EOF code point.

When a pointer is used, c references the code point the pointer points to as long as it does not point nowhere. When the pointer points to nowhere c cannot be used.

When a pointer is used, remaining references the code point substring from the pointer + 1 to the end of the string, as long as c is not the EOF code point. When c is the EOF code point remaining cannot be used.

If "mailto:username@example" is a string being processed and a pointer points to @, c is U+0040 (@) and remaining is "example".

If the empty string is being processed and a pointer points to the start and is then decreased by 1, using c or remaining would be an error.

1.3. Percent-encoded bytes

A percent-encoded byte is U+0025 (%), followed by two ASCII hex digits.

It is generally a good idea for sequences of percent-encoded bytes to be such that, when percent-decoded and then passed to UTF-8 decode without BOM or fail, they do not end up as failure. How important this is depends on where the percent-encoded bytes are used. E.g., for the host parser not following this advice is fatal, whereas for URL rendering the percent-encoded bytes would not be rendered percent-decoded.

The C0 control percent-encode set are the C0 controls and all code points greater than U+007E (~).

The fragment percent-encode set is the C0 control percent-encode set and U+0020 SPACE, U+0022 ("), U+003C (<), U+003E (>), and U+0060 (`).

The query percent-encode set is the C0 control percent-encode set and U+0020 SPACE, U+0022 ("), U+0023 (#), U+003C (<), and U+003E (>).

The query percent-encode set cannot be defined in terms of the fragment percent-encode set due to the omission of U+0060 (`).

The special-query percent-encode set is the query percent-encode set and U+0027 (').

The path percent-encode set is the query percent-encode set and U+003F (?), U+005E (^), U+0060 (`), U+007B ({), and U+007D (}).

The userinfo percent-encode set is the path percent-encode set and U+002F (/), U+003A (:), U+003B (;), U+003D (=), U+0040 (@), U+005B ([) to U+005D (]), inclusive, and U+007C (|).

The component percent-encode set is the userinfo percent-encode set and U+0024 ($) to U+0026 (&), inclusive, U+002B (+), and U+002C (,).

This is used by HTML for registerProtocolHandler(), and could also be used by other standards to percent-encode data that can then be embedded in a URL’s path, query, or fragment; or in an opaque host. Using it with UTF-8 percent-encode gives identical results to JavaScript’s encodeURIComponent() [sic]. [HTML] [ECMA-262]

The application/x-www-form-urlencoded percent-encode set is the component percent-encode set and U+0021 (!), U+0027 (') to U+0029 RIGHT PARENTHESIS, inclusive, and U+007E (~).

The application/x-www-form-urlencoded percent-encode set contains all code points, except the ASCII alphanumeric, U+002A (*), U+002D (-), U+002E (.), and U+005F (_).

2. Security considerations

The security of a URL is a function of its environment. Care is to be taken when rendering, interpreting, and passing URLs around.

When rendering and allocating new URLs "spoofing" needs to be considered. An attack whereby one host or URL can be confused for another. For instance, consider how 1/l/I, m/rn/rri, 0/O, and а/a can all appear eerily similar. Or worse, consider how U+202A LEFT-TO-RIGHT EMBEDDING and similar code points are invisible. [UTR36]

When passing a URL from party A to B, both need to carefully consider what is happening. A might end up leaking data it does not want to leak. B might receive input it did not expect and take an action that harms the user. In particular, B should never trust A, as at some point URLs from A can come from untrusted sources.

3. Hosts (domains and IP addresses)

At a high level, a host, valid host string, host parser, and host serializer relate as follows:

• The host parser takes an arbitrary scalar value string and returns either failure or a host.

• A host can be seen as the in-memory representation.

• A valid host string defines what input would not trigger a validation error or failure when given to the host parser. I.e., input that would be considered conforming or valid.

• The host serializer takes a host and returns an ASCII string. (If that string is then parsed, the result will equal the host that was serialized.)

3.1. Host representation

A host is a domain, an IP address, an opaque host, or an empty host. Typically a host serves as a network address, but it is sometimes used as opaque identifier in URLs where a network address is not necessary.

A typical URL whose host is an opaque host is git://github.com/whatwg/url.git.

The RFCs referenced in the paragraphs below are for informative purposes only. They have no influence on host writing, parsing, and serialization. Unless stated otherwise in the sections that follow.

A domain is a non-empty ASCII string that identifies a realm within a network. [RFC1034]

The domain labels of a domain domain are the result of strictly splitting domain on U+002E (.).

The example.com and example.com. domains are not equivalent and typically treated as distinct.

An IP address is an IPv4 address or an IPv6 address.

An IPv4 address is a 32-bit unsigned integer that identifies a network address. [RFC791]

An IPv6 address is a 128-bit unsigned integer that identifies a network address. This integer is composed of a list of 8 16-bit unsigned integers, also known as an IPv6 address’s pieces. [RFC4291]

Support for <zone_id> is intentionally omitted.

An opaque host is a non-empty ASCII string that can be used for further processing.

An empty host is the empty string.

3.2. Host miscellaneous

A forbidden host code point is U+0000 NULL, U+0009 TAB, U+000A LF, U+000D CR, U+0020 SPACE, U+0023 (#), U+002F (/), U+003A (:), U+003C (<), U+003E (>), U+003F (?), U+0040 (@), U+005B ([), U+005C (\), U+005D (]), U+005E (^), or U+007C (|).

A forbidden domain code point is a forbidden host code point, a C0 control, U+0025 (%), or U+007F DELETE.

Specifications should prefer the origin concept for security decisions. The notion of "public suffix" and "registrable domain" cannot be relied-upon to provide a hard security boundary, as the public suffix list will diverge from client to client. Specifications which ignore this advice are encouraged to carefully consider whether URLs' schemes ought to be incorporated into any decisions made, i.e. whether to use the same site or schemelessly same site concepts.

3.3. IDNA