Changeset 2157 for draft-ietf-httpbis/latest/p5-range.html

Timestamp:

24/01/13 12:02:20 (9 years ago)

Author:

fielding@…

Message:

Address range flooding security issue (#175 and #311) by direct
requirements and recommendations.

Actually require Content-Range and Content-Type (when appropriate) inside
multipart/byteranges body parts instead of assuming that the reader will
read between the lines of the MIME registration template.

Simplify description of required headers in 206 responses.

File:

• draft-ietf-httpbis/latest/p5-range.html (modified) (16 diffs)

draft-ietf-httpbis/latest/p5-range.html

@@ -449 +449 @@
   } 
   @bottom-center {
-       content: "Expires July 27, 2013"; 
+       content: "Expires July 28, 2013"; 
   } 
   @bottom-right {
@@ -492 +492 @@
       <meta name="dct.creator" content="Reschke, J. F.">
       <meta name="dct.identifier" content="urn:ietf:id:draft-ietf-httpbis-p5-range-latest">
-      <meta name="dct.issued" scheme="ISO8601" content="2013-01-23">
+      <meta name="dct.issued" scheme="ISO8601" content="2013-01-24">
       <meta name="dct.replaces" content="urn:ietf:rfc:2616">
       <meta name="dct.abstract" content="The Hypertext Transfer Protocol (HTTP) is an application-level protocol for distributed, collaborative, hypertext information systems. This document defines range requests and the rules for constructing and combining responses to those requests.">
@@ -518 +518 @@
             </tr>
             <tr>
-               <td class="left">Expires: July 27, 2013</td>
+               <td class="left">Expires: July 28, 2013</td>
                <td class="right">J. Reschke, Editor</td>
             </tr>
@@ -527 +527 @@
             <tr>
                <td class="left"></td>
-               <td class="right">January 23, 2013</td>
+               <td class="right">January 24, 2013</td>
             </tr>
          </tbody>
@@ -552 +552 @@
          in progress”.
       </p>
-      <p>This Internet-Draft will expire on July 27, 2013.</p>
+      <p>This Internet-Draft will expire on July 28, 2013.</p>
       <h1><a id="rfc.copyrightnotice" href="#rfc.copyrightnotice">Copyright Notice</a></h1>
       <p>Copyright © 2013 IETF Trust and the persons identified as the document authors. All rights reserved.</p>
@@ -604 +604 @@
          </li>
          <li><a href="#rfc.section.6">6.</a>&nbsp;&nbsp;&nbsp;<a href="#security.considerations">Security Considerations</a><ul>
-               <li><a href="#rfc.section.6.1">6.1</a>&nbsp;&nbsp;&nbsp;<a href="#overlapping.ranges">Overlapping Ranges</a></li>
+               <li><a href="#rfc.section.6.1">6.1</a>&nbsp;&nbsp;&nbsp;<a href="#overlapping.ranges">Denial of Service Attacks using Range</a></li>
             </ul>
          </li>
@@ -745 +745 @@
          when forwarding the request.
       </p>
-      <p id="rfc.section.3.1.p.5">The Range header field is evaluated after evaluating the preconditions of <a href="#Part4" id="rfc.xref.Part4.1"><cite title="Hypertext Transfer Protocol (HTTP/1.1): Conditional Requests">[Part4]</cite></a> and only if the result of their evaluation is leading toward a <a href="p2-semantics.html#status.200" class="smpl">200 (OK)</a> response. In other words, Range is ignored when a conditional GET would result in a <a href="p4-conditional.html#status.304" class="smpl">304 (Not Modified)</a> response.
-      </p>
-      <p id="rfc.section.3.1.p.6">The If-Range header field (<a href="#header.if-range" id="rfc.xref.header.if-range.1" title="If-Range">Section&nbsp;3.2</a>) can be used as a precondition to applying the Range header field.
-      </p>
-      <p id="rfc.section.3.1.p.7">If all of the preconditions are true, the server supports the Range header field for the target resource, and the specified
+      <p id="rfc.section.3.1.p.5">A server that supports range requests ought to ignore or reject a <a href="#header.range" class="smpl">Range</a> header field that consists of more than two overlapping ranges, or a set of many small ranges that are not listed in ascending
+         order, since both are indications of either a broken client or a deliberate denial of service attack (<a href="#overlapping.ranges" title="Denial of Service Attacks using Range">Section&nbsp;6.1</a>). A client <em class="bcp14">SHOULD NOT</em> request multiple ranges that are inherently less efficient to process and transfer than a single range that encompasses the
+         same data.
+      </p>
+      <p id="rfc.section.3.1.p.6">A client that is requesting multiple ranges <em class="bcp14">SHOULD</em> list those ranges in ascending order (the order in which they would typically be received in a complete representation) unless
+         there is a specific need to request a later part earlier. For example, a user agent processing a large representation with
+         an internal catalog of parts might need to request later parts first, particularly if the representation consists of pages
+         stored in reverse order and the user agent wishes to transfer one page at a time.
+      </p>
+      <p id="rfc.section.3.1.p.7">The Range header field is evaluated after evaluating the preconditions of <a href="#Part4" id="rfc.xref.Part4.1"><cite title="Hypertext Transfer Protocol (HTTP/1.1): Conditional Requests">[Part4]</cite></a> and only if the result of their evaluation is leading toward a <a href="p2-semantics.html#status.200" class="smpl">200 (OK)</a> response. In other words, Range is ignored when a conditional GET would result in a <a href="p4-conditional.html#status.304" class="smpl">304 (Not Modified)</a> response.
+      </p>
+      <p id="rfc.section.3.1.p.8">The If-Range header field (<a href="#header.if-range" id="rfc.xref.header.if-range.1" title="If-Range">Section&nbsp;3.2</a>) can be used as a precondition to applying the Range header field.
+      </p>
+      <p id="rfc.section.3.1.p.9">If all of the preconditions are true, the server supports the Range header field for the target resource, and the specified
          range(s) are valid and satisfiable (as defined in <a href="#byte.ranges" title="Byte Ranges">Section&nbsp;2.1</a>), the server <em class="bcp14">SHOULD</em> send a <a href="#status.206" class="smpl">206 (Partial Content)</a> response with a payload containing one or more partial representations that correspond to the satisfiable ranges requested,
          as defined in <a href="#range.response" title="Responses to a Range Request">Section&nbsp;4</a>.
       </p>
-      <p id="rfc.section.3.1.p.8">If all of the preconditions are true, the server supports the Range header field for the target resource, and the specified
+      <p id="rfc.section.3.1.p.10">If all of the preconditions are true, the server supports the Range header field for the target resource, and the specified
          range(s) are invalid or unsatisfiable, the server <em class="bcp14">SHOULD</em> send a <a href="#status.416" class="smpl">416 (Range Not Satisfiable)</a> response.
       </p>
@@ -793 +802 @@
 Content-Type: image/gif
 
-... 26012 bytes of image data ...
-</pre><p id="rfc.section.4.1.p.4">If multiple parts are being transferred, the server generating the 206 response <em class="bcp14">MUST</em> generate a "multipart/byteranges" payload, as defined in <a href="#internet.media.type.multipart.byteranges" title="Internet Media Type multipart/byteranges">Appendix&nbsp;A</a>, and send a <a href="p2-semantics.html#header.content-type" class="smpl">Content-Type</a> header field containing the "multipart/byteranges" media type and its required boundary parameter. For example:
+... 26012 bytes of partial image data ...
+</pre><p id="rfc.section.4.1.p.4">If multiple parts are being transferred, the server generating the 206 response <em class="bcp14">MUST</em> generate a "multipart/byteranges" payload, as defined in <a href="#internet.media.type.multipart.byteranges" title="Internet Media Type multipart/byteranges">Appendix&nbsp;A</a>, and send a <a href="p2-semantics.html#header.content-type" class="smpl">Content-Type</a> header field containing the multipart/byteranges media type and its required boundary parameter. Within the header area of
+         each body part in the multipart payload, the server <em class="bcp14">MUST</em> generate a <a href="#header.content-range" class="smpl">Content-Range</a> header field corresponding to the range being enclosed in that body part. If the selected representation would have had a <a href="p2-semantics.html#header.content-type" class="smpl">Content-Type</a> header field in a <a href="p2-semantics.html#status.200" class="smpl">200 (OK)</a> response, the server <em class="bcp14">SHOULD</em> generate that same <a href="p2-semantics.html#header.content-type" class="smpl">Content-Type</a> field in the header area of each body part. For example:
       </p>
       <div id="rfc.figure.u.18"></div><pre class="text">HTTP/1.1 206 Partial Content
@@ -813 +823 @@
 ...the second range
 --THIS_STRING_SEPARATES--
-</pre><p id="rfc.section.4.1.p.6">A server <em class="bcp14">MAY</em> combine requested ranges when those ranges are overlapping (see <a href="#overlapping.ranges" title="Overlapping Ranges">Section&nbsp;6.1</a>).
-      </p>
-      <p id="rfc.section.4.1.p.7">A response to a request for a single range <em class="bcp14">MUST NOT</em> be sent using the multipart/byteranges media type. A response to a request for multiple ranges, whose result is a single range, <em class="bcp14">MAY</em> be sent as a multipart/byteranges media type with one part. A client that cannot decode a multipart/byteranges message <em class="bcp14">MUST NOT</em> ask for multiple ranges in a single request.
-      </p>
-      <p id="rfc.section.4.1.p.8">When a client asks for multiple ranges in one request, the server <em class="bcp14">SHOULD</em> send them in the order that they appeared in the request.
-      </p>
-      <p id="rfc.section.4.1.p.9">When a 206 response is generated, the sender <em class="bcp14">MUST</em> generate the following header fields: 
-      </p>
-      <ul>
-         <li>Either a <a href="#header.content-range" class="smpl">Content-Range</a> header field (<a href="#header.content-range" id="rfc.xref.header.content-range.2" title="Content-Range">Section&nbsp;4.2</a>) indicating the single range included with this response, or a multipart/byteranges <a href="p2-semantics.html#header.content-type" class="smpl">Content-Type</a> indicating that multiple ranges are enclosed as multipart body-parts.
-         </li>
-         <li> <a href="p2-semantics.html#header.date" class="smpl">Date</a> 
-         </li>
-         <li> <a href="p6-cache.html#header.cache-control" class="smpl">Cache-Control</a>, <a href="p4-conditional.html#header.etag" class="smpl">ETag</a>, <a href="p6-cache.html#header.expires" class="smpl">Expires</a>, <a href="p2-semantics.html#header.content-location" class="smpl">Content-Location</a> and/or <a href="p2-semantics.html#header.vary" class="smpl">Vary</a>, if the header field would have been sent in a <a href="p2-semantics.html#status.200" class="smpl">200 (OK)</a> response to the same request
-         </li>
-      </ul>
-      <p id="rfc.section.4.1.p.10">If a 206 is generated in response to a request with an <a href="#header.if-range" class="smpl">If-Range</a> header field, the sender <em class="bcp14">SHOULD NOT</em> generate other representation header fields beyond those described above. Otherwise, the sender <em class="bcp14">MUST</em> generate all of the same representation header fields that would have been sent in a <a href="p2-semantics.html#status.200" class="smpl">200 (OK)</a> response to the same request.
+</pre><p id="rfc.section.4.1.p.6">When multiple ranges are requested, a server <em class="bcp14">MAY</em> coalesce any of the ranges that overlap or that are separated by a gap that is smaller than the overhead of sending multiple
+         parts, regardless of the order in which the corresponding byte-range-spec appeared in the received <a href="#header.range" class="smpl">Range</a> header field. Since the typical overhead between parts of a multipart/byteranges payload is around 80 bytes, depending on
+         the selected representation's media type and the chosen boundary parameter length, it can be less efficient to transfer many
+         small disjoint parts than it is to transfer the entire selected representation.
+      </p>
+      <p id="rfc.section.4.1.p.7">A server <em class="bcp14">MUST NOT</em> generate a multipart response to a request for a single range, since a client that does not request multiple parts might not
+         support multipart responses. However, a server <em class="bcp14">MAY</em> generate a multipart/byteranges payload with only a single body part if multiple ranges were requested and only one range
+         was found to be satisfiable or only one range remained after coalescing. A client that cannot process a multipart/byteranges
+         response <em class="bcp14">MUST NOT</em> ask for multiple ranges in a single request.
+      </p>
+      <p id="rfc.section.4.1.p.8">When a multipart response payload is generated, the server <em class="bcp14">SHOULD</em> send the parts in the same order that the corresponding byte-range-spec appeared in the received <a href="#header.range" class="smpl">Range</a> header field, excluding those ranges that were deemed unsatisfiable or that were coalesced into other ranges. A client that
+         receives a multipart response <em class="bcp14">MUST</em> inspect the <a href="#header.content-range" class="smpl">Content-Range</a> header field present in each body part in order to determine which range is contained in that body part; a client cannot rely
+         on receiving the same ranges that it requested, nor the same order that it requested.
+      </p>
+      <p id="rfc.section.4.1.p.9">When a 206 response is generated, the server <em class="bcp14">MUST</em> generate the following header fields, in addition to those described above, if the field would have been sent in a <a href="p2-semantics.html#status.200" class="smpl">200 (OK)</a> response to the same request: <a href="p2-semantics.html#header.date" class="smpl">Date</a>, <a href="p6-cache.html#header.cache-control" class="smpl">Cache-Control</a>, <a href="p4-conditional.html#header.etag" class="smpl">ETag</a>, <a href="p6-cache.html#header.expires" class="smpl">Expires</a>, <a href="p2-semantics.html#header.content-location" class="smpl">Content-Location</a>, and <a href="p2-semantics.html#header.vary" class="smpl">Vary</a>.
+      </p>
+      <p id="rfc.section.4.1.p.10">If a 206 is generated in response to a request with an <a href="#header.if-range" class="smpl">If-Range</a> header field, the sender <em class="bcp14">SHOULD NOT</em> generate other representation header fields beyond those described above because the client is understood to already have
+         a prior response containing those header fields. Otherwise, the sender <em class="bcp14">MUST</em> generate all of the representation header fields that would have been sent in a <a href="p2-semantics.html#status.200" class="smpl">200 (OK)</a> response to the same request.
       </p>
       <p id="rfc.section.4.1.p.11">Caches <em class="bcp14">MAY</em> use a heuristic (see <a href="p6-cache.html#heuristic.freshness" title="Calculating Heuristic Freshness">Section 4.1.2</a> of <a href="#Part6" id="rfc.xref.Part6.1"><cite title="Hypertext Transfer Protocol (HTTP/1.1): Caching">[Part6]</cite></a>) to determine freshness for 206 responses.
@@ -860 +871 @@
       <p id="rfc.section.4.2.p.5">A Content-Range field value with a <a href="#header.content-range" class="smpl">byte-range-resp</a> that has a <a href="#rule.ranges-specifier" class="smpl">last-byte-pos</a> value less than its <a href="#rule.ranges-specifier" class="smpl">first-byte-pos</a> value, or a <a href="#header.content-range" class="smpl">complete-length</a> value less than or equal to its <a href="#rule.ranges-specifier" class="smpl">last-byte-pos</a> value, is invalid. The recipient of an invalid <a href="#header.content-range" class="smpl">Content-Range</a>  <em class="bcp14">MUST NOT</em> attempt to recombine the received content with a stored representation.
       </p>
-      <p id="rfc.section.4.2.p.6">A server generating a <a href="#status.206" class="smpl">206 (Partial Content)</a> response to a byte range request <em class="bcp14">MUST</em> send, in each body-part of a multipart response or in the header block of a single part response, a Content-Range header field
+      <p id="rfc.section.4.2.p.6">A server generating a <a href="#status.206" class="smpl">206 (Partial Content)</a> response to a byte range request <em class="bcp14">MUST</em> send, in each body part of a multipart response or in the header block of a single part response, a Content-Range header field
          containing a <a href="#header.content-range" class="smpl">byte-range-resp</a> value that reflects the corresponding range being sent. The following example would apply when the complete length of the
          selected representation is known by the sender to be 1234 bytes:
@@ -915 +926 @@
          length of the selected representation.)
       </p>
-      <p id="rfc.section.4.4.p.2">When this status code is sent in response to a byte-range request, the sender <em class="bcp14">SHOULD</em> generate a <a href="#header.content-range" class="smpl">Content-Range</a> header field specifying the current length of the selected representation (see <a href="#header.content-range" id="rfc.xref.header.content-range.3" title="Content-Range">Section&nbsp;4.2</a>).
+      <p id="rfc.section.4.4.p.2">When this status code is sent in response to a byte-range request, the sender <em class="bcp14">SHOULD</em> generate a <a href="#header.content-range" class="smpl">Content-Range</a> header field specifying the current length of the selected representation (see <a href="#header.content-range" id="rfc.xref.header.content-range.2" title="Content-Range">Section&nbsp;4.2</a>).
       </p>
       <div id="rfc.figure.u.27"></div> 
@@ -1024 +1035 @@
                   <td class="left">http</td>
                   <td class="left">standard</td>
-                  <td class="left"> <a href="#header.content-range" id="rfc.xref.header.content-range.4" title="Content-Range">Section&nbsp;4.2</a> 
+                  <td class="left"> <a href="#header.content-range" id="rfc.xref.header.content-range.3" title="Content-Range">Section&nbsp;4.2</a> 
                   </td>
                </tr>
@@ -1049 +1060 @@
          range request mechanisms. More general security considerations are addressed in HTTP messaging <a href="#Part1" id="rfc.xref.Part1.3"><cite title="Hypertext Transfer Protocol (HTTP/1.1): Message Syntax and Routing">[Part1]</cite></a> and semantics <a href="#Part2" id="rfc.xref.Part2.2"><cite title="Hypertext Transfer Protocol (HTTP/1.1): Semantics and Content">[Part2]</cite></a>.
       </p>
-      <h2 id="rfc.section.6.1"><a href="#rfc.section.6.1">6.1</a>&nbsp;<a id="overlapping.ranges" href="#overlapping.ranges">Overlapping Ranges</a></h2>
-      <p id="rfc.section.6.1.p.1">Range requests containing overlapping ranges can lead to a situation where the server is sending far more data than the size
-         of the complete resource representation.
+      <h2 id="rfc.section.6.1"><a href="#rfc.section.6.1">6.1</a>&nbsp;<a id="overlapping.ranges" href="#overlapping.ranges">Denial of Service Attacks using Range</a></h2>
+      <p id="rfc.section.6.1.p.1">Unconstrained multiple range requests are susceptible to denial of service attacks because the effort required to request
+         many overlapping ranges of the same data is tiny compared to the time, memory, and bandwidth consumed by attempting to serve
+         the requested data in many parts. Servers ought to ignore, coalesce, or reject egregious range requests, such as requests
+         for more than two overlapping ranges or for many small ranges in a single set, particularly when the ranges are requested
+         out of order for no apparent reason. Multipart range requests are not designed to support random access.
       </p>
       <h1 id="rfc.section.7"><a href="#rfc.section.7">7.</a>&nbsp;<a id="acks" href="#acks">Acknowledgments</a></h1>
@@ -1136 +1150 @@
       <div id="rfc.iref.m.2"></div>
       <h1 id="rfc.section.A" class="np"><a href="#rfc.section.A">A.</a>&nbsp;<a id="internet.media.type.multipart.byteranges" href="#internet.media.type.multipart.byteranges">Internet Media Type multipart/byteranges</a></h1>
-      <p id="rfc.section.A.p.1">When a <a href="#status.206" class="smpl">206 (Partial Content)</a> response message includes the content of multiple ranges, they are transmitted as body-parts in a multipart message body (<a href="#RFC2046" id="rfc.xref.RFC2046.1"><cite title="Multipurpose Internet Mail Extensions (MIME) Part Two: Media Types">[RFC2046]</cite></a>, <a href="http://tools.ietf.org/html/rfc2046#section-5.1">Section 5.1</a>) with the media type of "multipart/byteranges". The following definition is to be registered with IANA <a href="#BCP13" id="rfc.xref.BCP13.1"><cite title="Media Type Specifications and Registration Procedures">[BCP13]</cite></a>.
-      </p>
-      <p id="rfc.section.A.p.2">The multipart/byteranges media type includes one or more body-parts, each with its own <a href="p2-semantics.html#header.content-type" class="smpl">Content-Type</a> and <a href="#header.content-range" class="smpl">Content-Range</a> fields. The required boundary parameter specifies the boundary string used to separate each body-part.
+      <p id="rfc.section.A.p.1">When a <a href="#status.206" class="smpl">206 (Partial Content)</a> response message includes the content of multiple ranges, they are transmitted as body parts in a multipart message body (<a href="#RFC2046" id="rfc.xref.RFC2046.1"><cite title="Multipurpose Internet Mail Extensions (MIME) Part Two: Media Types">[RFC2046]</cite></a>, <a href="http://tools.ietf.org/html/rfc2046#section-5.1">Section 5.1</a>) with the media type of "multipart/byteranges". The following definition is to be registered with IANA <a href="#BCP13" id="rfc.xref.BCP13.1"><cite title="Media Type Specifications and Registration Procedures">[BCP13]</cite></a>.
+      </p>
+      <p id="rfc.section.A.p.2">The multipart/byteranges media type includes one or more body parts, each with its own <a href="p2-semantics.html#header.content-type" class="smpl">Content-Type</a> and <a href="#header.content-range" class="smpl">Content-Range</a> fields. The required boundary parameter specifies the boundary string used to separate each body part.
       </p>
       <p id="rfc.section.A.p.3"> </p>
@@ -1212 +1226 @@
       <p id="rfc.section.B.p.1">A weak validator cannot be used in a <a href="#status.206" class="smpl">206</a> response. (<a href="#status.206" id="rfc.xref.status.206.2" title="206 Partial Content">Section&nbsp;4.1</a>)
       </p>
-      <p id="rfc.section.B.p.2">The Content-Range header field only has meaning when the status code explicitly defines its use. (<a href="#header.content-range" id="rfc.xref.header.content-range.5" title="Content-Range">Section&nbsp;4.2</a>)
+      <p id="rfc.section.B.p.2">The Content-Range header field only has meaning when the status code explicitly defines its use. (<a href="#header.content-range" id="rfc.xref.header.content-range.4" title="Content-Range">Section&nbsp;4.2</a>)
       </p>
       <p id="rfc.section.B.p.3">Servers are given more leeway in how they respond to a range request, in order to mitigate abuse by malicious (or just greedy)
@@ -1331 +1345 @@
             </li>
             <li><a id="rfc.index.C" href="#rfc.index.C"><b>C</b></a><ul>
-                  <li>Content-Range header field&nbsp;&nbsp;<a href="#rfc.xref.header.content-range.1">2</a>, <a href="#rfc.xref.header.content-range.2">4.1</a>, <a href="#rfc.iref.c.1"><b>4.2</b></a>, <a href="#rfc.xref.header.content-range.3">4.4</a>, <a href="#rfc.xref.header.content-range.4">5.3</a>, <a href="#rfc.xref.header.content-range.5">B</a></li>
+                  <li>Content-Range header field&nbsp;&nbsp;<a href="#rfc.xref.header.content-range.1">2</a>, <a href="#rfc.iref.c.1"><b>4.2</b></a>, <a href="#rfc.xref.header.content-range.2">4.4</a>, <a href="#rfc.xref.header.content-range.3">5.3</a>, <a href="#rfc.xref.header.content-range.4">B</a></li>
                </ul>
             </li>