Opened 7 years ago

Last modified 7 years ago

#30 assigned defect

Benoit's comments

Reported by: acooper@… Owned by: bob.briscoe@…
Priority: minor Milestone:
Component: concepts-uses Version:
Severity: - Keywords:
Cc:

Description

Initially, I put a DISCUSS-DISCUSS because I was missing some information to correctly review the draft (This was advised by Ron Bonica). Discussing with Wes Eddy off line, maybe I should have put "Defer". Anyway, sorry for the confusion. Now that I had a 1:1 with Eddy, many points are clarified, and here is my feedback... which I can't insert back in the tracker.

Regards, Benoit (On-the-job-experience new A.D.)

  • From the charter:

The purpose of the CONEX working group is to develop a mechanism
by which senders inform the network about the congestion encountered
by previous packets on the same flow.

Question about how to do define the notion of flow inside the IP layer?
I see an IPv6 destination option in http://tools.ietf.org/html/draft-ietf-conex-destopt-02, but I don't see any encoding for a flow in there?

  • Section 1, Introduction

As described in Figure 1, congestion control currently relies on the transport receiver detecting these 'Congestion Signals' and informing the transport sender in 'Congestion Feedback Signals.'

Do you always assume that, because you receive a congestion signals for a specific TCP session, the congestion is for this entire session? I see more and more cases for which a router does a Deep Packet Inspection, and treat applications differently, as opposed to the 5 tuple. Is this a use case for CONEX? Without a DPI engine on the sender, I don't see how this could work...

  • What is happening in case of ECMP? One router is congested, and not the other one. Is the CONEX enabled sender clever enough to mark only the affected flow (based on the 5 tuple)?
  • Following on the two previous points, I would like to see the issues/limitations documented.

Issue 1: one round trip late
Issue 2: the congested device MUST support CONEX. The Sender MUST support CONEX
Issue 3: DPI engine must be supported on the sender, if we want to differentiate per application
Issue 4: ECMP (if this is one)
Issue 5: not UDP
etc..
From there, the use cases are limited. I see one though: managed services, bought by an enterprise customer, out of a single service provider

  • Section 2.3

"rest-of-path
congestion" (also known as "downstream congestion") is the level of
congestion that a traffic flow is expected to experience between the
measurement point and its final destination.

What is a flow? The draft doesn't tell if the congestion is at layer 3, layer 4 or layer 7.

In contrast,
the ConEx? signals inserted into IP headers as shown in Figure 1
indicate the congestion along a whole path from source to
destination.

source and destination what? IP addresses, or IP address/protocol/ports, or maybe IP address/protocol/ports/session id.
That's a key question in my mind.

Same remark in section 2.4, what is user's traffic

Congestion: In general, congestion occurs when any user's traffic
suffers loss, ECN marking, or increased delay as a result of one
or more network resources becoming overloaded.

  • Section 2.4 definitions

You define congestion here, while section 2.1 already defines congestion

The definition used for the purposes of ConEx? is expressed as the
probability of packet loss (or the probability of packet marking if
ECN is in use). This definition focuses on how congestion is
measured, rather than describing congestion as a condition or state.

  • Section 3.1

By
deploying a congestion policer at the BRAS location, the network
operator can measure the congestion-volume created by users within
the access nodes and police misbehaving users before their traffic
affects others on the access network.

My issue (again) with this use case, is if the BRAS sees a high congestion-volume ratio, it will police all traffic from that users.
However, that users might at the same time be part of a webex call (voice + video), doing a backup, and downloading with P2P.
And the flow/session/application (to be clarified) you want to slowdown are the P2P, then then backup, and not the webex.

  • 6. Security.

Am I able to send a message (from my own IP address) with an IPv6 destination option containing the IP address of my neighbor, so that his packets are slowed down, and I can benefit from the full cable infrastructure? Seems like an important question...

Change History (3)

comment:1 Changed 7 years ago by acooper@…

Bob has the token to draft a response to Benoit. We can hash out if there are changes that need to be made to the document after that.

comment:2 Changed 7 years ago by acooper@…

  • Owner changed from draft-ietf-conex-concepts-uses@… to bob.briscoe@…
  • Status changed from new to assigned

comment:3 Changed 7 years ago by rbriscoe@…

[BB]: I've put the following at the end of the section explaining the concept of rest-of-path congestion:
" If traffic is not ECN-capable, upstream congestion is not always

possible to measure. [I-D.ietf-conex-abstract-mech] has further
discussion of the constraints around the network's ability to measure
upstream and rest-of-path congestion in these circumstances. Given
these constraints, there are still some significant intial deployment
arrangements that need ConEx? but work without ECN (see Section 5).

"

Then in Section 5, I've added:
" Three supporting documents give concrete examples of feasible initial

deployment scenarios, respectively in a broadband access network
scenario [I-D.briscoe-conex-initial-deploy], a mobile communications
network scenario [I-D.ietf-conex-mobile], and a multi-tenant data
centre scenario [I-D.briscoe-conex-data-centre]. The first two of
these scenarios work well without ECN support, but others are more
difficult without ECN. For instance, the data centre scenario works
best with ECN, but this is reasonable given some data centres already
enable ECN.

"

  • From the charter:

The purpose of the CONEX working group is to develop a mechanism

by which senders inform the network about the congestion encountered

by previous packets on the same flow.

Question about how to do define the notion of flow inside the IP layer?
I see an IPv6 destination option in http://tools.ietf.org/html/draft-ietf-conex-destopt-02, but I don't see any encoding for a flow in there?

[BB]: We didn't want to go too much mechanism detail, but I think you're right that we're missing the relation between

  • the sender adding ConEx? signals to flows and
  • the network measuring congestion-volume in aggregates.

I've added the following to the rest-of-path congestion section:

" A network monitor can count the volume of all passing ConEx? marked

packets and subtract the volume of all ECN marked packets regardless
of the flows within the aggregate. The resulting measure of
downstream congestion-volume reflects the total 'harm' the aggregate
contributes to other traffic beyond that point, on its way to its
various destinations.

"

  • Section 1, Introduction

As described in Figure 1, congestion control currently relies on the transport receiver detecting these 'Congestion Signals' and informing the transport sender in 'Congestion Feedback Signals.'

Do you always assume that, because you receive a congestion signals for a specific TCP session, the congestion is for this entire session?

[BB]: Not at all. I've guessed that the example with Alice's constant congestion levels might have led you to think we assumed congestion is constant within a flow. I think the following subtle changes should fix that:

" Imagine Alice sends 1GB of a file while the loss-probability is a constant


0.2%. Her contribution to congestion -- her congestion-volume -- is
1GB x 0.2% = 2MB. If she then sends another 3GB of the file while


the loss-probability is 0.1%, this adds 3MB to her congestion-volume.
Her total contribution to congestion is then 2MB+3MB = 5MB.
[...]
(A queue with varying percentage loss does these multiplications


and additions inherently.)

"

I see more and more cases for which a router does a Deep Packet Inspection, and treat applications differently, as opposed to the 5 tuple. Is this a use case for CONEX?

[BB]: As the doc says, ConEx? is designed to reveal at the IP layer the congestion that traffic causes as a result of how it actually behaved, not just based on what it ought to behave like from what app it seems to look like. Then operators have a good metric to do app-agnostic traffic management without needing to go deeper. Then, if there are policy concerns or encryption, we still have a solution.

Having said that, ConEx? could also be combined with DPI, to give richer info - this point is made in the supporting I-D "Mobile Communication Congestion Exposure Scenario".

I hope you agree that we already have sufficient text about this.

Without a DPI engine on the sender, I don't see how this could work...

[BB]: I don't understand what this sentence is getting at. An app does deep packet inspection by definition (L7).

  • What is happening in case of ECMP? One router is congested, and not the other one. Is the CONEX enabled sender clever enough to mark only the affected flow (based on the 5 tuple)?

[BB]: I know there are concerns about ECMP where congestion is measured edge-edge in an aggregate that includes ECMultiplePaths with different congestion on each. But, by design, ConEx? signals are created by the transport sender in the end host (as in Fig 1). The feedback it sees from the transport receiver is solely about the congestion that its packets actually experience over the path they actually traverse, including if ECMP is involved. That's how e2e transport protocols already work.

Then, when the network measures the aggregate of ConEx? markings in an aggregate of many flows (as clarified earlier), it gets the correct aggregate measure, because the markings are created separately and accurately by all the different end-points that only see their own 5-tuple flows.

If its' OK, I'd rather not introduce ECMP as a potential problem then explain why it's a non-problem. I think that will distract from the main message.

  • Following on the two previous points, I would like to see the issues/limitations documented.

Issue 1: one round trip late

[BB]: Actually there's a powerful aspect of the ConEx? protocol called credit marking that we decided not to go into in this introductory document. It's described in <...conex-abstract-mech>, and the protocol encoding <...conex-destop> supports it. It ensures senders have to post a conservative amount of 'credit' at the start of each flow so that the sum of ConEx? signals represents expected congestion, not just congestion an RTT ago.

This introductory doc is about what ConEx? can be used for (the "why" not the "how"): primarily traffic management. Traffic management works over timescales far greater than round trips, so we decided we didn't need to go into credit and RTT timescales in this doc.

The main purpose of credit marking is to simplify audit, which is the main subject of conex-abstract-mech, and only mentioned in passing in this introductory doc, to avoid distracting from the main message.

Issue 2: the congested device MUST support CONEX.

[BB]: There's no such thing as ConEx? support in a forwarding device. As S.5 on Deployment says:
"no changes to network forwarding elements are needed"
The congested device just does drop like it always has.

The Sender MUST support CONEX

[BB]: The ONLY thing that MUST support ConEx? is the sender (not all senders, just one at a time). A good property for incremental deployment is that only one thing has to support the new protocol. If zero things support the new protocol, that's a requirement for non-deployment, not incremental deployment!

But the developer of sending s/w needs to expect some networks to do something with ConEx? markings to have a reason to implement. The para in "5. Deployment Arrangements" explains:

" The above two steps seem to represent a stand-off where neither step

is useful until the other has made the first move: i) some sending
hosts must be modifed to give information to the network and ii) a
network must deploy policy devices to monitor this information and
act on it. Nonetheless, the developer of a scavenger transport
protocol like LEDBAT does stand to benefit from deploying ConEx?. In
this case the developer makes the first move, expecting it will
prompt at least some networks to move in response, using the ConEx?
information to reward users of the scavenger transport protocol.

"

(BTW, the BitTorrent? CTO and some developers were instrumental in instigating the ConEx? wg - they already have a record of a similar deployment model - introducing uTP (LEDBAT) unilaterally in the hope that some operators would whitelist it).

Issue 3: DPI engine must be supported on the sender, if we want to differentiate per application

[BB]: Applications on the sender can differentiate by application. It is expected that developments like this would happen (as described in the 2nd para of "Additional Benefits"), but this isn't a pre-requisite for the initial traffic management benefits.

Issue 4: ECMP (if this is one)

Non-issue (see above)

Issue 5: not UDP

[BB]: Already <...conex-abstract-mech> says under "5. Support for Incremental Deployment":
" The implementations of some of the transport protocols on a host

might not support ConEx? (e.g. the implementation of DNS over UDP
might not support ConEx?, while perhaps RTP over UDP and TCP will).
Any non-upgraded transports and non-upgraded hosts will simply
continue to send regular Not-ConEx? packets as always.

"

To avoid misleading readers of the present document, I'll add this sentence under "5. Deployment Arrangements":
" Deployment can start with one implementation of TCP and proceed to other TCP

implementations and other transports (e.g. LEDBAT, or RTP over UDP) as the need
arises.

"

etc..
From there, the use cases are limited. I see one though: managed services, bought by an enterprise customer, out of a single service provider

[BB]: Yup.

We're none of us under any illusions about deployment - ConEx? has always been described as ambitious but worth a shot as a potential alternative to a lot of the complexity around isolating and managing traffic.

However, having seen that we had already covered each of your concerns (and many more), I hope you're less pessimistic now.

  • Section 2.3

"rest-of-path
congestion" (also known as "downstream congestion") is the level of
congestion that a traffic flow is expected to experience between the
measurement point and its final destination.

What is a flow? The draft doesn't tell if the congestion is at layer 3, layer 4 or layer

7.

[BB]: A drop, whether due to congestion at L1, 2 or 3, drops headers at all layers (incl L4 where it is detected).
Explicit congestion marking has to be propagated up the layers explicitly (as in RFC5129 about MPLS->MPLS & MPLS->IP).

Already in nearly all IP networks, congestion at any of layers 1 & 2 is collected up at L3 and detected by the receiver at L4 to be fed back to the sender at L4. ConEx? doesn't change this, it just uses it.

There are some schemes to detect congestion over single logical links or over PWs, and if they can mitigate congestion before the e2e transport sees it, then fine. However, ultimately, feedback to the source of the load is always going to be needed and ConEx? signalling was deliberately pitched at L3 to sit alongside the existing congestion signals that L3 collects along the path.

Also, because ConEx? markings are mostly monitored by policy devices, it is best to sit at the internetwork layer, which is more likely to be visible at trust boundaries.

Having said all that, I propose not to address this question in this doc - this merely answers your question.

In contrast,
the ConEx? signals inserted into IP headers as shown in Figure 1
indicate the congestion along a whole path from source to
destination.

source and destination what? IP addresses, or IP address/protocol/ports, or maybe IP address/protocol/ports/session id.
That's a key question in my mind.

[BB]: I've modified to "...a whole path from transport source to transport destination."

Same remark in section 2.4, what is user's traffic

[BB]: I can't see the problem here - hopefully the previous edit fixes this.

Congestion: In general, congestion occurs when any user's traffic
suffers loss, ECN marking, or increased delay as a result of one
or more network resources becoming overloaded.

  • Section 2.4 definitions

You define congestion here, while section 2.1 already defines congestion

The definition used for the purposes of ConEx? is expressed as the
probability of packet loss (or the probability of packet marking if
ECN is in use). This definition focuses on how congestion is
measured, rather than describing congestion as a condition or state.

[BB]: I've modified the introductory sentence to this section to explain why everything comes up twice:
" ConEx? relies on a precise definition of congestion and a number of

newer concepts that are introduced then formally defined in this section.


"

  • Section 3.1

By
deploying a congestion policer at the BRAS location, the network
operator can measure the congestion-volume created by users within
the access nodes and police misbehaving users before their traffic
affects others on the access network.

My issue (again) with this use case, is if the BRAS sees a high congestion-volume ratio, it will police all traffic from that users.However, that users might at the same time be part of a webex call (voice + video), doing a backup, and downloading with P2P.
And the flow/session/application (to be clarified) you want to slowdown are the P2P, then then backup, and not the webex.

[BB]: Well, ConEx? doesn't dictate just one design of policer, I've designed a per-flow policer that uses ConEx?, but the v simple bulk design seems to work - the following reasoning may help explain why.

Overall, the response of the most responsive (TCP) apps compensates for the less responsive ones. This works because apps that don't work well if the rate is over-sensitive to congestion, are programmed not to be over-sensitive to congestion (unsurprisingly!).

I've modified the text a little to cover this:
OLD:

Those users would be likely to
react in the typical way to drops, backing off (assuming use of
standard TCP), and thereby lowering their congestion-volumes back
within the quota limits.

NEW:

Those users' apps would be likely
to react in the typical way to drops, backing off (assuming at least
some use TCP), and thereby lowering the users' congestion-volumes
back within the quota limits. If none of a user's apps responded,
the policer would continue to increase focused drops and effectively
enforce its own congestion control.

Certainly, bulk policing may not lead to optimal results (e.g. all TCPs reduce when only one needs to). But policing isn't meant to be business as usual. My attitude is: "When you've been thrown in jail for a hit-and-run, you can't complain that there's no mint on your pillow."

  • 6. Security.

Am I able to send a message (from my own IP address) with an IPv6 destination option containing the IP address of my neighbor, so that his packets are slowed down, and I can benefit from the full cable infrastructure? Seems like an important question...

[BB]: Yes, this is one of the most important attacks against the protocol. I'm not so concerned about this attack being launched by a man-in-the-middle (because a MITM can do far more damage anyway - e.g. just drop everything).

However, that still leaves the slightly less worrying case of an attacker who guesses which 5-tuples to attack. The analysis of this attack is in my PhD dissertation, which is all about the integrity of what is now called ConEx? (see <http://www.bobbriscoe.net/projects/refb/refb_dis.pdf> S.7.5.3). The attack isn't as easy as TCP connection hijacking, because it is only effective if it keeps sending a train of attack packets to sustain the attack, so it needs to know when it has hit on the right 5-tuple, but it doesn't see any feedback to know when it has hit on a 5-tuple that works.

Anyway, there are a couple of alternative defences proposed in my dissertation, and we will only see if we need them once ConEx? is out there.

I don't propose to say anything about this attack in this doc, because the security considerations are all deferred to <...conex-abstract-mech>.

Note: See TracTickets for help on using tickets.