Several of our users have asked us to include data relative to their account in the HTTP headers of requests we send them, or even responses they get from our API. What is the general convention to add custom HTTP headers, in terms of naming, format... etc.
Also, feel free to post any smart usage of these that you stumbled upon on the web; We're trying to implement this using what's best out there as a target :)
The normal approach is was to start their name with "X-". E.g. X-Forwarded-For, X-Requested-With. This is also mentioned in section 5 of RFC 2047.
Update: On June 2011, the first IETF draft was posted to deprecate the use of the "X-" prefix for non-standard headers. The reason is that when non-standard headers prefixed with "X-" become standard, removing the "X-" prefix breaks backwards compatibility, forcing application protocols to support both names (E.g, x-gzip & gzip are now equivalent). So, the recommendation is to just name themsensibly without the "X-" prefix.
Update 2: On June 2012, the deprecation of "X-" prefix has become official as RFC 6648.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The format for HTTP headers is defined in the HTTP specification. I'm going to talk about HTTP 1.1, for which the specification is RFC 2616. In section 4.2, 'Message Headers', the general structure of a header is defined:
message-header = field-name ":" [ field-value ] field-name = token field-value = *( field-content | LWS ) field-content = <the OCTETs making up the field-value and consisting of either *TEXT or combinations of token, separators, and quoted-string>
This definition rests on two main pillars, token and TEXT. Both are defined in section 2.2, 'Basic Rules'. Token is:
token = 1*<any CHAR except CTLs or separators>
In turn resting on CHAR, CTL and separators:
CHAR = <any US-ASCII character (octets 0 - 127)> CTL = <any US-ASCII control character (octets 0 - 31) and DEL (127)> separators = "(" | ")" | "<" | ">" | "@" | "," | ";" | ":" | "\" | <"> | "/" | "[" | "]" | "?" | "=" | "{" | "}" | SP | HT
TEXT = <any OCTET except CTLs, but including LWS>
Where LWS is linear white space, whose definition i won't reproduce, and OCTET is:
OCTET = <any 8-bit sequence of data>
There is a note accompanying the definition:
The TEXT rule is only used for descriptive field contents and values that are not intended to be interpreted by the message parser. Words of *TEXT MAY contain characters from character sets other than ISO- 8859-1 [22] only when encoded according to the rules of RFC 2047 [14].
So, two conclusions. Firstly, it's clear that the header name must be composed from a subset of ASCII characters - alphanumerics, some punctuation, not a lot else. Secondly, there is nothing in the definition of a header value that restricts it to ASCII or excludes 8-bit characters: it's explicitly composed of octets, with only control characters barred (note that CR and LF are considered controls). Furthermore, the comment on the TEXT production implies that the octets are to be interpreted as being in ISO-8859-1, and that there is an encoding mechanism (which is horrible, incidentally) for representing characters outside that encoding.
So, to respond to @BalusC in particular, it's quite clear that according to the specification, header values are in ISO-8859-1. I've sent high-8859-1 characters (specifically, some accented vowels as used in French) in a header out of Tomcat, and had them interpreted correctly by Firefox, so to some extent, this works in practice as well as in theory (although this was a Location header, which contains a URL, and these characters are not legal in URLs, so this was actually illegal, but under a different rule!).
That said, i wouldn't rely on ISO-8859-1 working across all servers, proxies, and clients, so i would stick to ASCII as a matter of defensive programming.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The accepted answer here is, with its updates, a good answer to the question of naming non-standardheaders [which might someday be expected to become standard headers]. But I see that as a nearly-unrelated tangent to the original question, which relates to custom headers that are application-specific("data relevant to their account") and thus have nothing to do with expected eventual support absent the X prefix. Unlike with the "x-gzip/gzip" example given, here there is no expectation that "X-ClientDataFoo" might ever be supported by any proxy or vendor without the "X".
The former case is similar to vendor prefixing of CSS properties, where future-proofing and thinking about vendor support and official standards is appropriate. The latter is more akin to choosing URL query parameter names. Nobody should care what they are. But namespacing the custom ones is a perfectly valid thing to do, IMHO.
There's nothing special or magical about the "X-" prefix, but it helps in cases like this to make it clear that it is a custom header. Might as well go a step further and do e.g. "X-ACME-ClientDataFoo" (if your widget company is "ACME").