I.

SOFTWARE DEFINED NETWORKS (SDN)

➔ OVERVIEW

● SDN is an architecture designed to make a network more flexible and easier to

manage.

● SDN can create and control a virtual network – or control a traditional

hardware – via software.

● SDN is the separation of the control functions from the forwarding

functions, which enables greater automation and programmability in the

network

● Uses software-based controllers or APIs to communicate with underlying

hardware infrastructure and direct traffic on a network.

● Centralizes management in discrete networking devices

● The goal of SDN is to improve network control by enabling enterprises and

service providers to respond quickly to changing business requirements.

● The role of software defined networks in cloud computing lets users respond

quickly to changes.

➔ ARCHITECTURE

● SDN architecture comprises three layers:

1. Application: the applications and services running on the network

2. Control: the SDN controller or “brains” of the network

3. Infrastructure: Switches and routers, & the supporting physical hardware

● The distinguishable layers are connected through northbound and southbound

APIs.

● Applications can use a northbound interface to talk to the controller.

● The controller and switches can use southbound interfaces to communicate.

● Directly Programmable

● Agile

● Centrally Managed

● Programmatically Configured

● Open Standards-based And Vendor-neutral

➔ THE CHALLENGES OF SDN ARE :

● Security risks of centralized management: Centralized management is a

single point of attack and if it goes down, the whole network is affected.

● SDN controller bottleneck: When there is only a single instance of an SDN

controller, it can become a bottleneck for a network with a large amount of

traffic, routers, and switches.

● No universally-accepted standard for northbound APIs

➔ OVERVIEW

● As stated, SDN facilitates a separation of the control plane from the forwarding

plane.

● But, establishing forwarding techniques on the control plane is mostly done

manually.

● To address this issue, this paper presents NetworkAI, an intelligent

architecture for self-learning control strategies in SDN networks.

● NetworkAI employs deep reinforcement learning

● Network monitoring technology - The in-band network telemetry (INT) (to

dynamically generate control policies and produces a near optimal decision)

➔ PROPOSED ARCHITECTURE

● NetworkAI - Applying ML to solve network control problems.

● NetworkAI - To intelligently operate networks, it uses the following

1. Software-defined networking,

2. Network monitoring (e.g. traffic identification, INT), and

3. Reinforcement learning.

● Close-Loop Control of Network - Done using

1. Network state upload link and

2. Decision Download link

● Builds a centralised intelligent agent to learn policies from the network.

relying on manual processes or network assumptions

● In a large-scale dynamic network, NetworkAI can intelligently control and

optimise a network.

➔ NETWORK ARCHITECTURE

Fig- NetworkAI Architecture

1. Forwarding plane - Forwarding, processing, and monitoring data packets

● The network hardware, consisting of line-rate programmable forwarding

technology, does not include any control techniques.

● The SDN controller issues control rules using southbound protocols like

OpenFlow or P4.

● When a packet enters the node, it is forwarded and processed

accordingly.

● Some nodes have monitoring programmes.

network state information to the AI plane.

2. Control plane - Connect the AI plane and the forwarding plane

● Resources (such as connection bandwidth, network adaptor, and CPU

capacity) are pooled to the AI plane via this plane's abstractions.

● The SDN controller manages the network and interacts with the AI plane

via northbound interfaces.

● This logically centralised layer alleviates the geographically scattered

network control challenge.

3. AI plane - Generates policies

● Uses SDN and monitor techniques to obtain a global view and control of

the entire network

● The AI agent uses the forwarding plane's network state to create a policy,

which it then uses to make judgments and optimise.

➔ NETWORK CONTROL LOOP

● Network state upload link and the decision download link is used to

accomplish a close-loop control of the network.

1. ACTION ISSUE

Fig : Action Issuing process

protocols according to the decisions made at the AI plane

● The network node at the forwarding plane operates based on the updated

rules imposed by the SDN controller.

● In this manner, the global controllability of the entire network is realized.

2. NETWORK STATE UPLOAD

● SDN network architecture allows controllers to deliver action decisions to

underlying networks for comprehensive network control.

● Making near-optimal decisions requires a thorough real-time view of the

entire network.

● The most important data to acquire is network state information and traffic

information.

● So the upload link is built to acquire detailed network and traffic data.

A. NETWORK INFORMATION

● Network information includes network physical topology, hop delay,

and queue occupancy (information below the L2 layer).

● This paper uses INT for fine-grained network monitoring.

● Monitoring traffic (SNMP, synthetic probes) is sent separately from

data traffic (probe traffic).

● These cause excessive probe traffic and extra processing to the

control plane in large-scale dynamic networking, reducing real-time

control performance.

● In-band network telemetry allows the data plane to gather and report

network state without requiring control plane involvement or

computation.
 ● The aim behind INT is to insert the network status into the header of

a data packet to ensure packet-level monitoring.

● The telemetry data is simple to add. Thus, end-to-end monitoring

data can be retrieved straight from the forwarding node via Kafka or

IPFIX to the AI plane's big data platform.

Fig : Collection Process of Network Infromation

● A source node embeds instructions in the packets

● Each network element inserts the requested network state in the packet as it

traverses the network.

● When the packet is sent to the INT Sink, the load data is sent to the user and the

telemetry data is sent to the network analytical plane.

B. TRAFFIC INFORMATION

● Includes service-level information (e.g., QoS/QoE), anomaly traffic detection

information(e.g., elephant flow), etc.

● In order to better manage and control networking, the identification of network

traffic plays a significant role

● Several monitor processes embedded in some nodes to transfer the raw traffic

data(e.g., flow granularity data, relevant traffic feature, and Deep Packet

Inspection (DPI) information) to the traffic information via data mining methods,

such as traffic classification and traffic anomaly detection.

● Then, the traffic information will be upload to the network analytical plane to

assist AI plane in decision making.

3. POLICY GENERATION

● RL method used to solve the large-scale network control problem

Fig : RL Process
 ● In our architecture, we apply DRL to generate network policy.

● Combining RL with DL leads to a general artificial intelligence solution for solving

complex network control problems.

● Introducing DRL for network decision making presents two main advantages:

1. The DRL algorithm is a black-box approach

2. DRL agent does not need to converge again when the network state

changes.

➔ USE CASE

● To demonstrate that it is possible to model the behavior of a network with the

proposed NetworkAI architecture.

● A simple example in the context of QoS routing, where the NetworkAI was

used to make intelligent decisions to select the best routing path aiming at

satisfying the QoS requirements.

Fig: The Experiment Example

● QoS routing mainly involves path selection that meets the QoS requirements

arising from different service flows.

● It is a mechanism of routing based on QoS requests of a data flow and the

available network resources.

QoS-Routing problem.

● The RL approach can calculate a near-heuristic solution in one step, which can

benefit real-time control of large-scale networks.

● And the closed-loop architecture in NetworkAI provides an interaction framework

which achieves applying DRL in a geo distributed network.

● In the experiment, we applied the Deep Q-learning algorithm in a central agent,

aimed to select the best routing path to minimize the network delay

● They simulated a network with 36 nodes and 50 full-duplex links, with uniform

link capacities and different transmission delay.

● In this experiment, they have employed a neural network that has five fully

connected hidden layers with a sigmoid activation function as well as a trained

DQN on the gym platform and Keras.

● In addition, they devised 10 traffic intensity (TI) levels, ranging from 1 to 10,

which represent the volumes of the network traffic at different times.

● The DRL agent was trained for 200K steps for each TI.

➔ EXPERIMENTAL RESULTS & ANALYSIS

● DRL performance increases with training steps and the DRL agent converges

when the training step is more than 200K.

● The benchmark algorithm is the shortest path algorithm

● With the increase of network load, congestion occurs on the shortest path of the

network, and the network agent will have to choose non-congested links for

transmission.

● Thus, in this situation, the DRL performance performs much better than the

benchmark.
➔ CHALLENGES

● Due to the black-box feature of RL approach, for different network

decision tasks and optimization goals, only the action space and reward

have to be designed.
 ● Near-optimal solution can be calculated in single step once the DRL

trained. Compared with the traditional heuristic algorithms, this feature

presents a huge advantage for real time network applications

➔ FUTURE ENHANCEMENT

● NetworkAI architecture is aimed at a complex, highly dynamic multi

application network environment

● So, it is difficult to obtain convincing experiments through the network

simulator.

● Therefore, in the immediate future, they plan to build a largescale real

NetworkAI testbed to expand the experiments.

➔ CONCLUSION

● Concept of NetworkAI- a novel paradigm.

● Combines SDN, INT, and DRL to automatically control and optimize a

network.