wiki:coin

Version 26 (modified by marie@…, 13 months ago) (diff)

--

Computing in the Network (COIN)

Introduction

Integrating computing and networking has been widely investigated and applied at several network layers in the past. Most notably, “Active Networking” research in the 1990s explored approaches for allowing packets and datagrams flowing through a network to modify the behavior of the network itself. This could be done at several layers, e.g., enabling/modifying transport protocol behavior, configuring or programming link layer functionality upon connection establishment etc.

Hence are experiencing a convergence between the concepts of networking and computing, triggered not only by the softwarization of networking functions (SDN, NFV) but also by the evolution of the network architecture. The move to edge distributed computing/networking is also encouraging the development of local networking and computing facilities to support low delay and low loss services that are emerging from AR/VR, autonomous vehicles and intelligent/smart cities.

Recent research in network data plane programmability has enabled new ways for relaxing the boundaries between strictly network layer and application layer programmability. For example, programming abstractions such as P4 and more powerful programmable DC switch platforms enable the implementation of different support functions for application layers entities, supporting applications such as DNN (Deep Neural Network) training, frontend KV (Key-Value) caching for skewed and dynamic workloads, and high-performance consensus protocols such as Paxos.

In addition, there are scalable stream processing frameworks such as Apache Spark or Apache Flink that apply programmed functions on data flowing in a distributed system. These platforms are typically concerned with guaranteeing certain semantics and providing high reliability and performance by orchestrating the set of functions accordingly. Such as distributed processing platforms are overlays(from a network layer perspective) but also have to deal with flow/congestion control etc. on their respective layer.

In parallel, there are approaches for connecting so-called network functions (such as NFV -- Network Functions Virtualization) and derived/related computing/networking models such as CDN and Edge Computing -- which are mostly concerned with setting up and maintaining overlays and virtual networks between application logic in virtual machines etc. Consequently the idea of a “programmable” network is central to the evolution of the Internet.

Motivation

Network programmability provides new opportunities to enhance performance and availability as well as to develop new types of networked applications and systems. Looking at different instantiations of integrating computing and networking, the following questions arise:

1) Are there common principles, abstractions and mechanisms that can be applied across this range of different types of computing/networking approaches?

2) What are best practices and relevant considerations for computing/networking systems, in particular with respect to previous discussions regarding active networking and end-to-end-arguments?

3) Many of the computing/networking systems developed for and by the networking community have been designed from a networking perspective (assuming running applications that need to be connected by connections, overlays, service chains). Is there potential to include design patterns from the distributed computing and applications community (e.g., stream processing) that would allow for a more holistic design, i.e., considering both networking and computing for optimizing the layout of processing functions and distribution of data?

Research Challenges

The research questions that the COIN group wants to address include but are not limited to:

(1) Even within the traditional and "end-to-end argument", will distributed computing in the network provide enough motivation and benefits to justify the introduction of non-forwarding functions into the network?

(2) Will forwarding function be eventually be integrated into the computing paradigm for example ML for route determination?

(3) To which level must the abstraction of the programmable data plane be for a network with non-forwarding functions? With new functionalities, the network an infrastructure, need to be decoupled from some applications so that it can be kept stable with permanently evolving applications.

(4) What will the impact of these in-network functions on end-to-end transport protocols and security? Will transport start being hop-by-hop?

(5) With the network as a database what will be the impact on the privacy of users' data and identities.

(6) What are the economic, social and environmental incentives for the network to add new computing capabilities and resources in an open ecosystem.

COIN Objectives

The main objectives of the COIN RG include:

1) Understanding the use cases and different types of network programmability and their different characteristics (for example, DC switch programmability vs. distributed/edge computing).

2) Investigating architectural questions such as system architecture and protocol designs for in-network computing, for example interactions of data and control planes as well as overall system and protocol security.

3) Understanding relationship to and impact on existing Internet protocols (transport, traffic steering) and frameworks (security, management).

4) Developing common terminology, concepts and potentially system elements such as data plane protocols and management concepts.

5) Providing guidance for potential future IETF work on distributed and in-network computing.

Draft Charter

The COIN research group wants to explore the research on how to make use of new programmable data planes and distributed computing to introduce non-forwarding functions into networks and functional federation to improve network and application performance and user experience.

In order to achieve this goal the methodology will include specific future-looking use cases with their outcomes, the trade-offs between the benefits from the new functionality in network and the extra cost to network devices, related research on edge computing applications that benefit from programmability and research on applications that could be moved into the network to provide added functionality. Use cases will include the collaboration between centralized and controlled environments like DCN, and the widely distributed networks characterized by edge/pervasive computing. While it is not mandatory it is hoped that later in the RG's work the combination of both approaches in a common architecture may lead to common protocols. With it's focus on research in the overall field of computing in the network the RG will not address a specific architecture, system or protocol but will assume that for example specific solutions in COIN may have the equivalent of a transport layer to consider both network and compute resources jointly.

The use cases and related research may lead to new architecture and layering design with comparison to traditional architectures in terms of complexity, performance and cost and create incentivesthe research into new abstraction of the data plane and the development of some potential new protocols. Finally the impacts of COIN on transport protocols, security and privacy in different environments, and the incentives for both the network providers to provide the capabilities and the application developers to use them will also be investigated.

Scope

(1) Use case analysis/targeted research: DCN, edge networks, IoT networks etc. and the potential benefits to these networks from in-network non-forwarding functions like compute, cache, manage etc.

(2) Research on solutions to use current and coming programmable network devices to implement non-forwarding functions and demonstrate the relationship between performance and benefit/gains.

(3) Research on novel architectures, new data-plane abstraction and new protocol designs that make full use of the constrained compute and cache capabilities in programmable network devices and how to expand them,

(4) Research on novel architectures, new data-plane abstraction and new network protocol designs that can help to efficiently use the decentralized computing resources, inside the network devices in the DC, the core and the edge, or even in the end-user devices.

(5) Research on new transport protocol and new privacy and security mechanisms enabled by in-network non-forwarding functions.

(6) Research on incentive mechanisms to encourage both the network providers to provide the capabilities and the application developers to use them.

Outcomes

COIN wants to build a forum to explore and discuss how the network architectures and protocols will adapt to the introduction of distributed systems and decentralized computing resources. Hence the following outcomes are proposed:

(1) An informative RFC on COIN in Datacenters

(2)An informative RFC on COIN at the Edge

(3) An informative RFC on COIN in Networked Applications

<other specific contributions to be added with the help of the community - to come>

Initial Meeting: IETF 103 Bangkok (10am-12pm room Boromphimarn 3)

Meeting Chairs Jeffrey He, Huawei and Marie-José Montpetit, consultant

Draft Agenda

Welcome and agenda

Review of the agenda (Chairs - 5 minutes)

Presentation of the motivation and draft charter (Jeffrey He, Huawei - 10 minutes)

Computing in the Network

In-Network Data-Center Computing - draft-he-coin-datacenter-00 (Jeffrey He - 10 minutes)

Compute-First Networking: Rethinking Network Programmability (Dirk Kutscher - 10-15 minutes)

Topic TBD (Marco Canini - 10-15 minutes)

Design Challenges for Combining Compute and Networking (Dave Oran - 10-15 minutes)

Topic TBD (Diego Lopez - 5 minutes)

Applications Enabled by COIN

Ubiquitous Witness/reverse CDN (Eve Schooler - 5-10 minutes)

Machine Learning (Rachel Chen - 10 minutes)

In Network Computing Enablers for Extended Reality - draft-montpetit-coin-XR-01 (MJM - 10 minutes)

Edge Computing in Future IoT Applications and Architectures (Ravi Ravindran - 5 minutes)

Next steps (Chairs - 5 minutes)

Documents

In-Network Data-Center Computing - draft-he-draft-he-coin-datacenter-00 available at https://datatracker.ietf.org/doc/draft-he-coin-datacenter/ In Network Computing Enablers for Extended Reality - draft-montpetit-coin-XR-01 available at https://datatracker.ietf.org/doc/draft-montpetit-coin-xr/ (errata on v00).

References

Agrawal, Divy. " Data on the Edge: Leveraging the Network Edge for Internet Applications.", https://www.cse.ust.hk/pg/seminars/S18/agrawal.html

Bastug, Ejder, Bennis, Mehdi, Médard, Muriel and Debbah Mérouane, "Towards Interconnected Virtual Reality:Opportunities, Challenges and Enablers.", March 2017 /https://arxiv.org/pdf/1611.05356.pdf

Chang, Michael Alan, Panda, Aurojit , Bottini, Domenic, Jian, Lisa, Kumar, Pranay and Shenker, Scott, “Network Support for DNN Training”, SysML, Feb 2018, Palo Alto, California https://www.sysml.cc/doc/182.pdf

Dang Huynh Tu, Sciascia, Daniele, Canini, Marco, Pedone Fernando and Soulé, Robert, NetPaxos?: Consensus at Network Speed SOSR15 https://mcanini.github.io/papers/netpaxos.sosr15.pdf

Dang, Huynh, Canini, Marco, Pedone, Fernando and Soulé, Robert, "Paxos Made Switch-y", Sigcomm CCR 2016, /https://www.sigcomm.org/sites/default/files/ccr/papers/2016/April/0000000-0000002.pdf

Fan, Bin, Lim, Hyeontaek, David G. Andersen, David G. and Kaminsky, Michael, “Small Cache, Big Effect: Provable Load Balancing for Randomly Partitioned Cluster Services.” 2011.ACM SOCC www.istc-cc.cmu.edu/publications/papers/2011/loadbal-socc2011.pdf

Forster, Nate, <to come>

Graham, Richard L, et al. (16 authors) “Scalable Hierarchical Aggregation Protocol (SHArP): A Hardware Architecture for Efficient Data Reduction.” In COM-HPC, 2016. https://ieeexplore.ieee.org/document/7830486

Hadoop Distributed File System, http://hadoop.apache.org/

Jin, Xin, Li, Xiaozhou, Zhang, Haoyu, Robert Soulé, Robert, Lee, Jeongkeun, Foster, Nate, Kim, Changhoon and Stoica, Ion, “NetCache?: Balancing Key-Value Stores with Fast In-Network Caching", SOSP2017, https://www.cs.jhu.edu/~xinjin/files/SOSP17_NetCache.pdf

Li, Jialin, Michal, Ellis and Ports, Dan R.K. “Eris: Coordination-Free Consistent Transactions Using In-Network Concurrency Control”, (University of Washington) SOSP 2017 https://syslab.cs.washington.edu/papers/eris-sosp17.pdf

Li , Xiaozhou, Sethi, Raghav, Kaminsky, Michael, Andersen, David G and Freedman, Michael J." Be fast, cheap and in control with SwitchKV", NSDI'2016, https://dl.acm.org/citation.cfm?id=2930614

NetCompute? 2018: In-Network Computing Workshop, Sigcomm2018, https://conferences.sigcomm.org/sigcomm/2018/workshop-netcompute.html

P4, p4.org

Ports, Dan R.K., Li, Jialin, Liu, Vincent, Sharma, ViNaveen? Kr. and Krishnamurthy, Arvind, "Designing Distributed Systems Using Approximate Synchrony in Data Center Networks”, (University of Washington) NSDI 2015. https://www.usenix.org/node/188949

Rexford, Jennifer, Sigcomm 2018 Keynote Address, https://youtu.be/t_5__v6CNYE?t=4652

Sapio, Amadeo, Abdelaziz Ibrahim, Aldilaijan, Abdulla, Canini, Mario and Kalnis Panos, " In net computing is a dumb idea whose time has come ", Hotnets 2017 https://dl.acm.org/citation.cfm?id=3152461

Soh, Lemuel, Burke, Jeff and Zhang, Lixia. "Supporting Augmented Reality (AR): Looking Beyond Performance" ACM SIGCOMM 2018 Workshop on Virtual Reality and Augmented Reality (VR/AR Network 2018).

Soulé, Robert, Netcompute Workshop Keynote Address, Sigcomm 2018, http://conferences.sigcomm.org/sigcomm/2018/files/slides/netcompute/2018-08-20-sigcomm.pdf

Srikanteswara, Srikathyayani, Foerster, Jeff and Schooler, Eve, "ICN-WEN Information Centric-Networking in Wireless Edge Networks", Presentation at ICNRG@IETF-98, March 2017 https://www.ietf.org/proceedings/98/slides/slides-98-icnrg-information-centric-networking-in-wireless-edge-networks-eve-schooler-00.pdf

Subedi, Tara Nath, Nguyen, Kim Khoa and Chériet, Mohamed, “OpenFlow?-based in-network Layer-2 adaptive multipath aggregation in data centers”, Computer Communications, Volume 61, May 2015, https://www.sciencedirect.com/science/article/pii/S0140366414003715

Zsolt, István, Sidler, David, Alonso, Gustavo and Vukolić, Marko, “Consensus in a Box", NSDI 2016, https://dl.acm.org/citation.cfm?id=2930639

Attachments (16)