ABNF Update

[mnot@…]

Update BNF to RFC4234.

[julian.reschke@…]

RFC4234 will be obsoleted by RFC5234 very soon.

[julian.reschke@…]

From [135]:

Consistent indentation for all ABNF rules (addresses #36)

[julian.reschke@…]

From [136]:

Add missing whitespace in BNF (addresses #36)

[julian.reschke@…]

From [154]:

Use names of RFC4234 (RFC5234) core rules (HTAB, DQUOTE); addresses #36.

[julian.reschke@…]

From [155]:

fix BNF rule for chunk-data; addresses #36.

[julian.reschke@…]

From [185]:

Get rid of duplicate BNF rule names ("host" -> "uri-host", "trailer" -> "trailer-part"), avoid underscore character in rule names ("http_URL" -> "http-URL", "abs_path" -> "path-absolute"), add rules for terms imported from URI spec ("absoluteURI", "authority", "path-absolute", "port", "query", "relativeURI", "host) -- these will have to be updated when switching over to RFC3986; addresses #36.

[julian.reschke@…]

From [186]:

Synchronize core rules with RFC5234 (except the CHAR vs NUL issue), get rid of prose rules that span multiple lines, get rid of unused rules LOALPHA and UPALPHA; addresses #36.

[julian.reschke@…]

From [187]:

Fix typo in ALPHA rule (we're still using the RFC2616's syntax); addresses #36.

[julian.reschke@…]

From [188]:

Exclude NUL from CHAR (see <​http://www.w3.org/mid/52304CE2-0ADE-499A-9A5A-35284BD428BA@gbiv.com>); addresses #36.

[julian.reschke@…]

From [190]:

Move Section "Product Tokens" from P2 into P1 (back below "Protocol Parameters") ("token" is used in the definition of the Upgrade header); addresses #36.

[julian.reschke@…]

From [191]:

Explicitly import BNF rules for "challenge" and "credentials" from RFC2617; addresses #36.

[julian.reschke@…]

From [205]:

Add sections that for now normatively reference the ABNF syntax and basic rules from Part1, also start work on referencing ABNF rules adopted from other parts (ongoing); relates to #36.

[julian.reschke@…]

From [206]:

Work on referencing ABNF rules adopted from other parts (done for P2 and P3); relates to #36.

[julian.reschke@…]

From [207]:

Work on referencing ABNF rules adopted from other parts (finished for all parts, also grouped by part); relates to #36.

[julian.reschke@…]

From [210]:

Move definition of "delta-seconds" into Part1; relates to #36.

[fielding@…]

I would prefer that delta-seconds remain in P6, where it is most used, and the single use in P2 be replaced with 1*DIGIT. Alternatively, move it to P2 and refer to it there in P6, or simply define it more than once.

IMO, P1 should not define ABNF that it does not use.

[julian.reschke@…]

From [212]:

Remove delta-seconds def from P1, just redefine it in P2 (revisits change [210]); relates to #36.

[julian.reschke@…]

From [214]:

Rewrite "token" rule, not using prose; relates to #36.

[julian.reschke@…]

From [215]:

Rewrite "TEXT" rule, not using prose; relates to #36.

[julian.reschke@…]

From [229]:

add linking between ABNF rules (does not affect TXT version), relates to #36.

[julian.reschke@…]

TODO: use proper notation for case-sensitive char sequences, such as in HTTP-Version and method.

[julian.reschke@…]

From [260]:

Do not use string literals in ABNF when matching is case-sensitive (related to #36)

[julian.reschke@…]

From [261]:

upper-case hex value for consistency (related to #36)

[julian.reschke@…]

From [273]:

Do not use string literals in ABNF when matching is case-sensitive (related to #36)

[julian.reschke@…]

From [309]:

s/HEX/HEXDIG/, also add section targets for some RFC2396 references related to percent escaping (related to #36)

[julian.reschke@…]

From [334]:

use "/" instead of "|" for alternatives (related to #36)

[julian.reschke@…]

From [335]:

reference RFC5234 instead of RFC822 for ABNF, but continue to extend it with implied LWS and #rule (related to #36)

[julian.reschke@…]

Plan for LWS:

• take out LWS

• put int OWS:

OWS = *WSP [obs-folding] obs-folding = CRLF 1*WSP

• explain when obs-folding is allowed (consumer vs producer)
• point out that this disallows continuations for producers, and only allows simple cont. for consumers (all-whitespace lines considered security risk)
• add appendix explaining for RFC2616 productions went

[julian.reschke@…]

From [349]:

only reference RFC 6234's core rules (related to #36)

[julian.reschke@…]

From [350]:

resync with trunk wrt core rules (related to #36)

[julian.reschke@…]

From [351]:

introduce BWS, OWS, RWS, but do not use them yet (related to #36)

[julian.reschke@…]

From [352]:

s/chunk-extension/chunk-ext/ (related to #36)

[julian.reschke@…]

From [353]:

start adding BWS/OWS/RWS (related to #36)

[julian.reschke@…]

From [354]:

Rewrite header ABNFs to spell out whitespace rules, factor out value format definitions. (related to #36)

[julian.reschke@…]

From [356]:

Rewrite header ABNFs to spell out whitespace rules, factor out value format definitions. (related to #36)

[julian.reschke@…]

From [357]:

Rewrite header ABNFs to spell out whitespace rules, factor out value format definitions. (related to #36)

[julian.reschke@…]

From [360]:

simplify ABNF production by using WSP (related to #36)

[julian.reschke@…]

From [362]:

Rewrite header ABNFs to spell out whitespace rules, factor out value format definitions. (related to #36)

[julian.reschke@…]

From [363]:

Rewrite header ABNFs to spell out whitespace rules, factor out value format definitions. (related to #36)

[julian.reschke@…]

From [364]:

Rewrite header ABNFs to spell out whitespace rules, factor out value format definitions. (related to #36)

[julian.reschke@…]

From [365]:

Rewrite header ABNFs to spell out whitespace rules, factor out value format definitions. (related to #36)

[julian.reschke@…]

From [366]:

Rewrite header ABNFs to spell out whitespace rules, factor out value format definitions. (related to #36)

[julian.reschke@…]

From [367]:

fix typo (related to #36)

[julian.reschke@…]

From [369]:

Explain rationale for OWS/BWS/RWS, do not use LWS production anymore (related to #36)

[julian.reschke@…]

From [370]:

Rewrite header ABNFs to spell out whitespace rules, factor out value format definitions. (related to #36)

[julian.reschke@…]

From [371]:

Cleanup. (related to #36)

[julian.reschke@…]

From [372]:

Mention implied LWS change in the Changes section (related to #36)

[julian.reschke@…]

From [375]:

simplify ABNF production (related to #36)

[julian.reschke@…]

From [376]:

fix width problems in ABNF (related to #36)

[julian.reschke@…]

From [377]:

remove TODO statements (related to #36)

[julian.reschke@…]

From [378]:

add note about future ABNF appendix containing flattened list rules (related to #36)

[ylafon@…]

Regarding list production: #foo => *(OWS "," [OWS foo]) 1#foo => *(OWS ",") OWS foo *(OWS "," [OWS foo])

or better

#foo => OWS *( "," OWS [foo OWS]) 1#foo => OWS *( "," OWS) foo OWS *( "," OWS [foo OWS])

[ylafon@…]

If we need to keep the empty list elements (for consumers):

#foo => [ ( "," / foo) *(OWS "," [OWS foo]) ]

1#foo => *( "," OWS) foo *( OWS "," [OWS foo])

n#foo => *( "," OWS) foo (n-1)*( OWS "," OWS foo ) *(OWS "," [OWS foo])

Without the empty list elements (for producers):

#foo => [1#foo]

1#foo => foo *(OWS "," OWS foo)

n#foo => foo (n-1)*(OWS "," foo) *(OWS "," OWS foo)

[julian.reschke@…]

From [400]:

rewrite definition of list rule, deprecate empty list elements (related to #36)

[julian.reschke@…]

Left to do:

• rewrite general ABNF so that it refers to the header *value* ABNFs instead to the header ABNFs, then get rid of the header ABNFs.

• automatically generate appendix sections with expanded list productions.

[julian.reschke@…]

From [417]:

add core.abnf, add make targets for reformatted ABNF (related to #36)

[julian.reschke@…]

From [418]:

add sorting to the production of *.parsed-abnf (related to #36)

[julian.reschke@…]

From [420]:

add code and Makefile rules to generate ABNF appendix sections (related to #36)

[julian.reschke@…]

From [421]:

add collected abnf appendices (related to #36)

[julian.reschke@…]

From [424]:

reorganize ABNF introductions to match Part1 (related to #36)

[julian.reschke@…]

From [425]:

reference RFC5234 core rules directly, fix line end bug in appendix generation (related to #36)

[julian.reschke@…]

From [427]:

ABNF appendix: group by first letter, add internal links to definitions (related to #36)

[julian.reschke@…]

From [428]:

ABNF appendix: group by first letter, add internal links to definitions -- fix group detection (related to #36)

[julian.reschke@…]

From [429]:

insert SP around ABNF brackets (related to #36)

[julian.reschke@…]

From [431]:

add forward ref in ABNF comment for obs-fold (related to #36)

[julian.reschke@…]

From [434]:

adjust whitespace in ABNF; make name of header value production for "Range" consistent with other headers (related to #36 and #85)

[julian.reschke@…]

From [444]:

reorganize header introductions consistent (related to #36, see also [364])

[julian.reschke@…]

Fixed in [543]:

Resolve #36: In P2..P7, mention list rule defined in P1; also point to Appendix containing collected ABNF with list rule being expanded (closes #36)

[julian.reschke@…]

With [543], all planned activities with respct to the ABNF syntax change are done. Please open new tickets in case new ABNF related issues come up.

[julian.reschke@…]

From [738]:

Syntax notation: mention "obs-" prefix convention. List rule: clarify intro, add note about how empty list elements contribute to the total number of elements, add examples. Related to #36.

[fielding@…]

From [1425]:

Fix incorrect definition of line folding (obs-fold) from [351]. Remove optional WSP from chunked encoding grammar added in [353]. Do not use WSP anywhere (it is misleading because it dosn't match the real definition of whitespace).

Related to #36

[julian.reschke@…]

From [1462]:

Fix examples for list syntax not to include *leading* whitespace (that's not part of the list production) (see #36)

[naruse@…]

Collectted ABNF in HTTP/1.1, part 5: Range Requests and Partial Responses draft-ietf-httpbis-p5-range-19 has typo.

WITH LIST RULE EXTENSION:

     byte-range-set  = 1#( byte-range-spec / suffix-byte-range-spec )

WRONG:

   byte-range-set = ( *( "," OWS ) byte-range-spec ) / (
    suffix-byte-range-spec *( OWS "," [ ( OWS byte-range-spec ) /
    suffix-byte-range-spec ] ) )

CORRECT:

   byte-range-set = *( "," OWS ) ( byte-range-spec / suffix-byte-range-spec )
     *( OWS "," [ OWS ( byte-range-spec / suffix-byte-range-spec ) ] )

[julian.reschke@…]

Indeed. It seems that there's a bug in the conversion code that we are using. See #358. Thanks for the report.