As the internet traffic along with the processing power in data centers are exponentially growing, the need for the design of energy efficient with highly elastic networking infrastructure to support the different applications and cloud services that can be hosted in data centers have become a hot research area. A key departure from the norm is that conventional routers and switches in conventional data centers are replaced with high performance Passive Optical Networks (PONs) to take over the role of routing and traffic forwarding through efficient resource provisioning algorithms. In this paper, the different aspects of PONs in the design of energy efficient, high capacity, and highly elastic networking infrastructures to support the applications and services hosted by modern data centers are considered. In this work, a mathematical optimization model for energy-efficient and delay-minimized scheduling in AWGR based PON data center for PON cell fabric configuration will be presented. The performance of the proposed architecture in terms of efficient scheduling against average delay and power consumption for different traffic loads and patterns will be evaluated. Different scenarios of traffic; random and unbalanced with hotspots are examined to evaluate the average delay and power consumption with and without sleep mode. Results have shown that with sleep mode enabled, power savings for two evaluated objective functions have shown similar results when examining different traffic patterns. The power savings range between 8% and 55% during low and high load activities, respectively. However, minimization of delay model has shown improvement in reducing total average delay reaching up to 42% if compared with the model with objective of minimization of power consumption.
In recent years, efforts for the re-design of data center architectures have been devoted to tackle two major issues appeared in conventional data center designs; power consumption in the first place which has main impact on the global warming, and high cost of the electricity bill resulted from high number of electronic devices used in the fabric networking interconnection [
In the last decade, the emergence of PONs in the design of data centers fabric interconnections have been introduced in several publications. In [
Another partial PON implementation was presented in [
In our previous work, we investigated the feasibility of PONs deployment in future data centers by re-designing the current paradigm of PONs (used traditionally in residential access networks) to furnish scalable, low cost, energy-effi- cient, and high capacity interconnections infrastructure to accommodate the different traffic patterns in data centers. We have proposed, compared, and patented five novel designs relaying mostly on PONs to handle intra and inter rack communications [
In this paper, further investigation is carried on the AWGR PON architecture design presented in [
The remainder of the paper is organized as follows. In Section 2, a re-visit with a brief description of the AWGR based PON data center architecture shall be given for completeness and through presentation of the work in this paper. In Section 3, the idea of PON cell fabric configuration through energy-efficiency and delay-minimized scheduling is described. In Section 4, a presentation of the MILP model is given. In Section 5, discussion of the results is presented. Section VI concludes the paper.
In this section, the modeled architecture for the PON data center is revisited and briefly described. The PON architecture is designed employing mainly passive optical devices to manage connectivity for traffic routing among servers located in the same rack or in different racks. Intra-rack communication is the term used for traffic exchanged among servers within the same rack, while inter-rack communication is the server to server traffic forwarded through different racks.
The architecture is comprised of the three main parts. The first main segment of the network is the rack and it is defined as a PON group. In a PON group a number of servers say 16 - 32 are connected together via one of the three technologies depicted in
The use of active electronic or optical switches is completely avoided for the intra-rack communication. All servers are equipped with tunable ONU like
transceiver with a PCI-e interface that connects directly to the servers. The second part of the network is called the PON Cell and shown in
The PON cell fabric interconnection is designed to be flexible to accommodate any number of PON groups through mathematical model that solves for topology physical interconnection and the wavelength routing and assignment within the cell [
A network can be designed to accommodate multiple OLT switches and can be scaled up to increase the number of servers that can be hosted within the PON data center by adding more switches. Each OLT switch consists of a chassis that can host 16 Access Module cards (AM) where each AM has the capacity to connect 16 ports, each of which provides a transmission rate up to 10 Gb/s [
The architecture is designed in fashion that is similar to the cellular network. The rationale behind this topology design is to have wavelengths that can be reused for other PON groups connected to different OLT ports connecting different PON cells. The cellular based architecture improves scalability to allow such architectures to host millions of servers without having limitation on num- ber of wavelengths as these wavelengths are reused in all different cells connec- ted to different ports either in the same or different OLTs.
In high performance routers and switches, schedulers are responsible of configuring the backplane crossbar to match a conflict-free pair of inputs and outputs with unmatched demands [
For illustration purposes,
Adoption of the same concept in the proposed PON design is taken into consideration; a local scheduler at the OLT is responsible for configuring the PON cell fabric for intra-cell traffic scheduling. Upon receiving requests from demanding servers, the scheduler assigns resources and plans the scheduling of accessing the channels by means of maximum matching for efficient utilization of resources and minimization of delay as possible. For every pair of demanding servers, time slots are assigned and a pair of wavelengths that transceivers need to tune to for transmission/reception are assigned. Scheduler in the proposed architecture is responsible for scheduling transmission for all requesting servers; some demanding servers will take part in the first TDM frame time, and other take part in subsequent frame times. In a given frame time a conflict free pairing between sources and destinations is selected by the scheduler so each destination is paired with at most one source and each source is matched with one destination. Configuration of the PON cell fabric is carried by assigning tuning wave-
lengths for transceivers of demanding servers. A scheduler performs a maximum size bipartite matching to maximize flows and connections made in each frame time for achieving efficient resource utilization and maximum throughput.
All intra cell communications are coordinated locally via the OLT. Servers with demands send a request control message containing destination address and resources requirements to the OLT. If OLT grants the request, it replies with control messages to the source and the destination ONUs connecting the pair of servers with information about the assignment of time slots with resources in the designated wavelength which both servers’ ONUs need to tune to. At initial state, all ONUs transmitters and receivers are tuned to designated wavelength connecting them with OLT switch. Based on gate message received from the OLT, ONUs tunes their transmitters and receivers to assigned pair of wavelengths. ONUs needs to retune to other wavelengths in subsequent frame times for communication with different servers located at different PON groups. The switching time of converting wavelengths at tunable transceivers is in nanoseconds scale and has been demonstrated by experiments in Bell Labs [
Next, a mathematical model for configuring and optimizing the PON cell fabric interconnection in the AWGR based PON cell for efficient scheduling examining minimization of delay and power consumption shall be presented.
In this section, a MILP model is presented with two objective functions, minimization of delay and minimization of power consumption. The objective function with the minimization of the total delay aims to minimize total delay through minimization of total TDM frame times needed to serve all queued demand requests received by the scheduler. The model shall maximize the connection made in each frame time to minimize number of frame times and the number of transceivers’ retuning needed and hence minimize the total delay. Minimization of the number of frames will not only minimize total delay but also will result in maximization of throughput through efficient utilization of uplink and downlink capacities.
The second objective function with the minimization of total power consumption aims to minimize the power consumption by minimizing the total number of active ONUs needed to be switched on in the designated frame times while serving all demands.
The parameters and variables used in the model are as follows:
Parameter:
S
Set of ONUs
P G
Set of PON Groups (Racks).
T
Set of TDM time frames
P { P G }
Set of ONUs in each PON group PG.
C u
Wavelength uplink capacity
C d
Wavelength down capacity
α i j
Demand matrix between every source and destination servers ( i , j ) .
P C _ O N U
Power consumption of an ONU
Objective:
Minimize Delay:
∑ t ∈ T t ≠ 1 ( ( t − 2 ) ∑ i ∈ S ∑ j ∈ S Z i j t ) (1)
The mathematical expression in equation (1) gives the model objective which is to minimize the total delay by minimizing the waiting time for requests to be served. The reduction of required number of frame times improves the through- put by the maximization size of granted requests in each frame time.
Minimize Power Consumption:
∑ t ∈ T t ≠ 1 ∑ i ∈ S ∑ j ∈ S i = j ϕ i j t P C _ O N U (3)
The mathematical Equation (2) gives the model objective which is to minimize the power consumption by minimizing the total number of ONUs needed to be switched on in the designated frame times while serving all demands.
Subject to:
∑ j ∈ S i ≠ j X i j t ≤ 1 ∀ i ∈ S , ∀ t ∈ T j and t ≠ 1 (3)
Constraint (3) ensures that any requesting server with a queued demand should be matched with at most one destination in frame time t .
∑ i ∈ S i ≠ j X i j t ≤ 1 ∀ j ∈ S , ∀ t ∈ T j and t ≠ 1 (4)
Constraint (4) ensures that each destination server should be matched to at most one source server in frame time t .
Λ i j t ≥ ψ i j t (5)
Λ i j t ≤ M ψ i j t ∀ i , j ∈ S , ∀ t ∈ T j and t ≠ 1 (6)
Constraints (5) and (6) are binary equivalent to indicate that a request exist between ( i , j ) servers in frame time t .
Z i j t ≤ X i j t (7)
Z i j t ≤ ψ i j ( t − 1 ) (8)
Z i j t ≥ ψ i j ( t − 1 ) + X i j t − 1 ∀ i , j ∈ S , ∀ t ∈ T i ≠ j and t ≠ 1 (9)
Constraints (7), (8) and (9) are binary equivalent to indicate a request from server i (source) to server j (destination) is granted for transmission in frame time t .
Λ i j t = α i j ∀ i , j ∈ S and t = 1 (10)
Equation (10) defines the demand map requests received by the scheduler at t=1 is equal to the traffic demand.
Δ i j t = Z i j t α i j ∀ i , j ∈ S , ∀ t ∈ T i ≠ j and t ≠ 1 (11)
Equation (11) presents the size of accepted demand between ( i , j ) servers scheduled for transmission in frame time t .
Λ i j t = Λ i j ( t − 1 ) − Δ i j t ∀ i , j ∈ S , ∀ t ∈ T and t ≠ 1 i ≠ j (12)
Equation (12) present the not served traffic to be scheduled for transmission in subsequent frame times following frame time t .
∑ i ∈ P [ k ] ∑ j ∈ P [ q ] i ≠ j Δ j i t ≤ C d (13)
∑ i ∈ P [ k ] ∑ j ∈ P [ q ] i ≠ j Δ i j t ≤ C u ∀ k , q ∈ P G , ∀ t ∈ T and t ≠ 1 (14)
Constraint (13) and (14) ensures that uplink and downlink wavelengths’ capacities (10 Gb/s) for each PON group are not exceeded while assigning resources to demanding servers in the designated frame times.
S i t = ∑ j ∈ S i ≠ j Z i j t ∀ i ∈ S , ∀ t ∈ T and t ≠ 1 (15)
S j t = ∑ i ∈ S i ≠ j Z i j t ∀ j ∈ S , ∀ t ∈ T and t ≠ 1 (16)
Equations (15) and (16) allocate the set of granted source and destination ONUs for transmission in frame time respectively.
ϕ i j t ≥ S i t (17)
ϕ i j t ≥ S j t (18)
ϕ i j t ≤ S i t + S j t ∀ i , j ∈ S , ∀ t ∈ T i = j and t ≠ 1 (19)
Constraints (17-19) are binary equivalent to indicate one if either of i or j , or both i and j are granted for transmission in frame time t , otherwise equal zero.
ξ i j t = ∑ i ∈ S ∑ j ∈ S i = j ϕ i j t ∀ t ∈ T and t ≠ 1 (20)
Equation (20) calculates the total number of ONUs in TDM frame time t scheduled for transmission.
∑ t ∈ T t ≠ 1 ∑ i ∈ S ∑ j ∈ S i ≠ j Δ i j t = ∑ i ∈ S ∑ j ∈ S i ≠ j α i j (21)
Constraint (21) is the demand satisfaction constraint to ensure granted demand is equal to the total demand.
In this section we evaluate the performance of the described architecture depicted in
The model with the objective to minimize total delay aims for minimizing total number of frame times needed and hence minimize the overall average waiting time to service all queued demands. Minimization of frame times maximizes the number of granted requests in each frame time by finding the largest possible conflict-free matching between source and destination servers. Maximizing the matching can efficiently reduce number of iterations/frame times to successfully grant all demands and hence reduce the waiting time for queued requests. While minimizing total delay, the model ensures that uplink and downlink bandwidth assignment from and to each ONU in the designated PON group is not exceeding the capacities of the assigned wavelengths. Hence, while attempting to achieve best maximum match for a frame time, the output of the matching graph is mastered and constrained by the size of flows to be received and transmitted by each ONU.
The modeled PON cell is depicted in
AverageDelay = ∑ t ∈ T t ≠ 1 ( ( t − 2 ) ∑ i ∈ S ∑ j ∈ S Z i j t ) TotNumberofReq (22)
Power consumption is governed by the total number of ONUs needed in all TDM frame times and is calculated using Equation (2). For comparison purposes, two objective functions are evaluated; one with minimization of delay and the second with minimization of power consumption. Minimization of delay approach target the minimization of average waiting time per request to be scheduled and serviced and can provide the optimum number of TDM frame times needed. For the model with minimization of power consumption, number of TDM frame times is obtained from the minimization of delay objective and will be given as an input parameter for the minimization of power consumption model.
Power consumption of ONUs (PC_ONUs) is evaluated with respect to traffic load and activity. For further energy savings, ONUs can be switched off if not scheduled by OLT for transmitting or receiving in the designated frame time. The power consumptions of the ONU during the on and off times are 2.5W and 0 W [
load increases for both cases. Introduction of hot spots causes an increase in the total power consumption for each load applied. For low loads such as the case with 75 requests (30% load), if we compare random traffic with the traffic with 4 hot spots, power consumption increases by 26.6%. This is mainly due to the selection of the granted servers in each frame time as with random scenario more options for granting are available, unlike the case with hotspots where more frame times are needed and less number of demanding servers competing for a grant in each frame time.
Evaluation results for power consumption against the two objective functions presented in
Random selection of demanding pairs of servers shows an increase of request average waiting time as the load increases. Results shows average delay ranges from 2 to 7 frame times for 30% (75 requests) to 100% loads (240 requests) respectively. Introducing hotspots result in unbalanced load assignments; where some nodes are targeted heavily. Hotspots increases average cell delay and degrade the throughput especially at lower loads as it introduces more frame times for conflict free pair matching of demanding servers.
Considerable savings reaching up to 42% for average delay are achieved with the minimization of delay objective compared to the minimization of power consumption model. This is justified by the efficient scheduling through minimization of average waiting time per request achieved by the minimized-delay objective.
The less number of transmission frames needed, the lower is the total time required to serve queued demands. Therefore; the objective with the minimization of total delay targets the minimization of overall frames needed and hence minimize the overall queuing delay of the network. While the objective with power consumption minimization targets the reduction of the total number of switched on devices when serving queued demands regardless of the overall time needed to serve the demands.
In this paper, PONs is considered in the design of the fast-switched and delay minimized fabric interconnection for future data centers networks. A mathematical optimization model using MILP was developed and presented to minimize the overall network delay through efficient scheduling and resource assignments. Different loads with random unbalanced traffic representing the nature of traffic in data centers have been evaluated. Results have shown that power savings with sleep mode enabled for the different examined traffic patterns range between 8% and 55% during low and high load activities respectively. Model with delay minimized scheduling has shown reduction in average delay reaching 42% with a trade-off of negligible maximum increase in power reaching 6% if compared with the model with minimized power consumption. The work presented in this article has been limited to mathematical modeling. However, the design of real time heuristics for hardware implementation and also to validate and verify the results obtained from the MILP mathematical model is part of the planned future work.
Hammadi, A. (2017) Mathematical Optimization Modelling for Fast-Switched and Delay Minimized Scheduling for Intra-Cell Communication in an AWGR-Based PON Data Center. Int. J. Communications, Network and System Sciences, 10, 13-29. https://doi.org/10.4236/ijcns.2017.102002