Journal of Transportation Technologies
Vol.03 No.01(2013), Article ID:27310,7 pages

Ethernet Implementation of Fault Tolerant Train Network for Entertainment and Mixed Control Traffic

Tarek K. Refaat, Mai Ibrahim, Ramez M. Daoud, Hassanein H. Amer

Electronics Engineering Department, American University in Cairo , New Cairo , Egypt


Received October 9, 2012; revised November 8, 2012; accepted November 22, 2012


This paper studies the integration of the control system and entertainment on board of train wagons. Both the control and entertainment loads are implemented on top of Gigabit Ethernet, each with a dedicated controller/server. The con- trol load has mixed sampling periods. It is proven that this system can tolerate the failure of one controller in one wagon. In a two wagon scenario, fault tolerance at the controller level is studied, and simulation results show that the system can tolerate the failure of 3 controllers. The system is successful in meeting the packet end-to-end delay with zero packet loss in all OPNET simulated scenarios. The maximum permissible entertainment load is determined for the fault tolerant scenarios.


Ethernet; Networked Control Systems; Fault-Tolerance; Railways; Intelligent Transportation Systems; WiFi

1. Introduction

Networked Control Systems (NCS) is a rapidly expand- ing field, with many applications, from industrial auto- mation to intelligent transportation systems [1-6]. The origins of NCS can be traced back to CAN, PROFIBUS and PROFINET; however Ethernet has spread rapidly in the past few decades, introducing non-deterministic pro- tocols to the world of NCS [7-9]. Ethernet is a non-de- terministic protocol, with several sources of randomness due to the use of Carrier Sense Multiple Access with Collision Detection (CSMA/CD) [10]. Such probabilistic nature is undesirable in real-time NCS implementations until certain modifications were made in order to ac- commodate real-time applications. Packet formats were modified in order to give certain messages priority over others [11,12]. Other modifications can be found imple- mented by Rockwell Automation, the ODVA, EtherNet/ IP, CIP, TT Ethernet and FTT Ethernet. Some of these solutions are in course of standardization [13-19]. One famous form of NCS exists in terrestrial transportation systems and studies on the subject are numerous and ex- tensive [1,2,20,21]. Recent studies have researched a specific implementation that utilizes the Ethernet proto- col (IEEE 802.3) in trains [22-28].

Entertainment is now common in most forms of transportation ranging from terrestrial transportation sys- tems to sea and air travel. The existence of an entertain- ment load alongside the control network on the same infrastructure may introduce an increased level of traffic congestion. Such a concept was tested in previous studies and it was shown that the control network performed within required deadlines [22,23].

Another growing trend in NCS is the incorporation of Fault Tolerance. Reference [3] studied the use of Fast and Gigabit Ethernet in advanced networked control sys- tems. It was proven that the use of redundant control nodes minimizes down-time. Reference [29] investigated the effect of failures on the productivity of fault-tolerant networked control systems under varying loads. Refer- ence [30] studied the availability of the Pyramid archi- tecture in the context of Networked Control Systems. Reference [31] also studied the Mean Time to Failure (MTTF) of a fault-tolerant two-machine production line in the context of Networked Control Systems (NCS). More details about fault-tolerance in NCS can be found in [32].

This paper presents a study of a train control network utilizing the Ethernet protocol without modifications. It includes all previously mentioned issues namely the use of unmodified protocol communicating both real-time (control) and non-real-time applications (entertainment). It also incorporates fault-tolerance aspects. The study consists of several OPNET Network Modeler [33] simu- lations which model two train wagons for control and entertainment loads. The main contribution of this paper is that the control load proposed is more realistic, con- sisting of a mixture of different sampling periods for sensors and actuators (SA) [27]. The entertainment load is comprised of video streaming and bounded WiFi ac- cess to the Internet. The proposed design also incorpo- rates a fault-tolerance study at the controller level. It will be shown that the simulated model is successful in both the fault-free and fault-tolerant scenarios.

The rest of this paper is organized as follows. Section 2 gives a recap of previous work on train control net- works. Section 3 will introduce the newly proposed model. Section 4 presents the simulated scenarios and shows the results. Section 5 concludes this paper.

2. Previous Work

There have been several research studies modeling train networks such as LonWorks and Train Control Networks (TCN) [25-28,34]. Recent studies have modeled train wagons using Switched Ethernet [22-24,35]. The re- search revolved around the feasibility of a single network carrying both control and entertainment, fault-tolerance on the controller level and sensor level.

Reference [22] tested the performance of a single Ethernet infrastructure supporting both a control and an entertainment system in train wagons. Using OPNET, several scenarios with different settings were simulated. SAs were set up to use a unified sampling period across the model. Depending on the scenario, some were simu- lated using a sampling period of 1ms and others using 16ms, each simulated independently. The results of the study guaranteed that the packet end-to-end delay is al- ways within constraints of the maximum allowable delay.

Reference [23] successfully introduced fault-tolerance at the controller level to the system proposed by [22]. A follow-up study introduced fault-tolerance at the sensor level using Triple Modular Redundancy (TMR), a con- cept described in [36,37], achieving successful results [35].

3. Proposed Model

The proposed model utilizes a 1 Gigabit Ethernet infra- structure without modifications based on the IEEE 802.3 standard for the whole network and follows the regula- tions described in the IEC 61375 [27]. Several sampling periods were mentioned in the standard, concerning the SAs, however the most common values were 1ms (the smallest sampling period) and 16 ms. References [22,23] studied these values independently. However, the cur- rently proposed model incorporates both sampling peri- ods in a single control network, representing the different possible applications of SAs. A typical number of SAs on a single train wagon is around 250 with sampling pe- riods 1ms (minority) and 16 ms (majority) [27]. These will be broken down into 3 groups. Group 1 (G1) con- sists of 30 sensors and 30 actuators (1:1 ratio) operating with the most demanding sampling period of 1ms. Group 2 (G2) consists of 100 sensors and 50 actuators (2:1 ratio) operating at 16ms sampling period. Finally, Group 3 (G3) also operating at 16ms sampling period, consists of 30 sensors and 10 actuators (3:1 ratio). The distribution is illustrated in Figure 1. This design minimizes the num- ber of switches, using standard 128 port switches, readily available in the market. Also note that the locations of the SAs have been chosen to increase the distance be- tween the switches and the controller, maximizing the trip distance to simulate a worst case scenario. The main switch (labeled MS1) utilizes a forwarding rate of 6.6 Mbps, which is much lower than rates available in mod- els such as the Cisco Catalyst 3560 Gigabit Ethernet switches [38].

The entertainment load in the train model can be de- scribed in terms of the number of streams, the quality of the video screens as well as the number of WiFi nodes and the number of applications per node. The worst case scenario, in a large wagon of 60 seats [25], supports 60 different and simultaneously playing DVD quality video streams (one per seat) [39], as well as one WiFi user per seat. Each WiFi user runs several simultaneous applica- tions: Database access, email sending/receiving, web browsing and file transfer. This gives a total of 60 Video streams and 60 WiFi users, each running 4 simultaneous applications requiring access to the network. These WiFi applications are simulated using the generic built-in heavy load applications of OPNET. This WiFi load re- quires a maximum bandwidth of 6Mbps and therefore only 6Mbps are allocated for the WiFi traffic. The WiFi access is provided via the Access Point (AP) in the mid- dle of the wagon, maximizing coverage area. Lastly, the model has 2 dedicated controllers per wagon. The exis- tence of two controllers allows the incorporation of fault- tolerance at the controller level as explained next. Wagon 1 has K1 and E1, where K1 is a controller, handling the control load only, and E1 is the entertainment server. The same is true for Wagon 2.

Each controller can readily take over in the case of failure of the other. This is achieved by having sensors send the sampled data at every sampling instant to all available controllers. Should any given controller have to take over for a failed controller, it must have the most updated sampled data in order to achieve zero packet loss. All simulations are modeled using OPNET.

4. Simulated Scenarios and Results

The proposed model is used to form 2 main tests, each broken down into 2 scenarios.

4.1. Simulated Scenarios

The first set of simulations is performed for a single wagon model with 60 video streams and 60 WiFi users. The first scenario simulates the fault-free case, where

Figure 1. Two-wagon overview.

both controllers are fully functional; K1 handling all con- trol data and E1 handling the entertainment load. In the second scenario (fault-tolerant), one of the two control- lers is off, simulating a failure. Both control and enter- tainment loads are handled by the remaining controller. Then an attempt is made to maximize the number of video streams and WiFi users without jeopardizing the control load.

The second set of simulations takes the model a step further, concatenating two identical wagons and hard- wiring the main switch of each wagon to the other. Modifications are made such that all sensors will send their data to all 4 controllers rather than merely the two on their corresponding wagons. Other communication techniques can be found in [40]. The total number of video streams and WiFi users is now double. Again, the first scenario simulates the fault-free case, where all 4 controllers are fully functional. The next scenario (fault- tolerant) models an extreme case, where 3 of the 4 con- trollers have failed, and now only 1 controller has to carry the control and entertainment load of both wagons. The goal again is to maximize the number of video streams and WiFi users, without the control load suffering.

4.2. Simulated Results

In order to gauge the performance of the system, end- to-end delay and packet loss must be monitored. In all simulations, zero packet loss (no packets dropped or de- layed) is observed and total end-to-end delays across all SAs are within their respective constraints [41]. A 95% confidence analysis is applied to all results. Figures 2-5 show examples of results from different scenarios. In all figures, the x-axis represents the simulation time in min- utes and seconds, while the y-axis shows the delay in seconds. The figures show delay measured at an actuator from one of the three groups (as indicated in the caption: e.g., G1). These delays represent the time taken for a packet to travel from the K to the A, and in cases like Figures 3 and 4 the packet delays oscillate between sev- eral values, depending on the level of network congestion faced.

With all video streams at DVD quality and all WiFi users running the full load of applications described in Section 3, the maximum total end-to-end delay (with a 95% confidence) for each group of SAs is shown in Ta- ble 1. The maximum number of streams and WiFi users to be supported in each simulated scenario are shown in Table 2. These results guarantee that all SAs operating with a sampling period of 1ms have an end-to-end delay of less than 1ms, and those operating at 16 ms have an end-to-end delay of less than 16 ms (with 95% confi- dence). It is important to note that the overhead incurred on the network by the WiFi nodes is negligible in com- parison to the control and the video streaming loads. As this load is extremely small, no matter what state the network is in, the WiFi network would be unaffected (as can be seen in Table 2). This can also be attributed to the fact that the bandwidth allotted to the WiFi users is bounded/restricted to 6 Mbps.

5. Conclusion

Ethernet implementation for Networked Control Systems (NCS) is a growing field. Train networks utilizing Ethernet have been previously studied, carrying both an entertainment load and a control load on a single network. This paper studied a fault-tolerant train NCS utilizing Switched Ethernet without modifications, with an enter- tainment load of video streams and WiFi access. The model was successfully simulated, showing correct packet transmission and reception, with zero losses in

Figure 2. Two wagon fault free scenario (G1).

Figure 3. Two wagons fault tolerant scenario (G1).

Figure 4. Two wagons fault tolerant scenario (G2).

Figure 5. Two wagons fault free scenario (G3).

Table 1. Total end-to-end delay (ms) per group.

Table 2. Entertainment load per scenario.

control data. Both the single wagon and the two-wagon model were simulated, in fault-free and faulty scenarios showing zero packet loss and delays within the con- straints of the sampling periods of the control network. However, when a load increases on a single controller due to a failure, the maximum number of supported streams, is reduced. This does not reduce the number of passengers with a capability of watching a video; it merely limits the number of possible simultaneously playing videos at any given time.


  1. 1. N. Navet, Y. Song, F. Simonot-Lion and C. Wilwert, “Trends in Automotive Communication Systems,” Proceedings of the IEEE, Vol. 93, No. 6, 2005, pp. 12041223. doi:10.1109/JPROC.2005.849725

  2. 2. R. M. Daoud and H. H. Amer, “Fast Ethernet Implementation of Cost Effective Vehicle On-Board Networks,” In: S. Pennacchio, Eds., Emerging Technologies, Robotics and Control Systems, Ass. Internationalsar, Bologna, 2007, pp. 81-85.

  3. 3. R. M. Daoud, H. M. Elsayed and H. H. Amer, “Gigabit Ethernet for Redundant Networked Control Systems,” Proceedings of the IEEE International Conference on Industrial Technology ICIT, Vol. 2, Hammamet, 8-10 December 2004, pp. 869-873.

  4. 4. J. D. Decotignie, “Ethernet-Based Real-Time and Industrial Communications,” Proceedings of the IEEE, Vol. 93, No. 6, 2005, pp. 1102-1117. doi:10.1109/JPROC.2005.849721

  5. 5. F.-L. Lian, J. R. Moyne and D. M. Tilbury, “Performance Evaluation of Control Networks: Ethernet, ControlNet, and DeviceNet,” IEEE Control Systems, Vol. 21, No. 1, 2001, pp. 66-83.

  6. 6. M. Tabbara, A. Rantzer and D. Ne?i?, “On Controller & Capacity Allocation Co-Design for Networked Control Systems,” Systems & Control Letters, Vol. 58, No. 9, 2009, pp. 672-676. doi:10.1016/j.sysconle.2009.06.002

  7. 7. Bosch, “CAN in Passenger and Cargo Trains. CAN in Automation,” 2011.

  8. 8. PI, “PROFIBUS & PROFINET,” 2011.

  9. 9. T. Skeie, S. Johannessen and C. Brunner, “Ethernet in Substation Automation,” IEEE Control Systems, Vol. 22, No. 3, 2002, pp. 43-51. doi:10.1109/MCS.2002.1003998

  10. 10. IEEE 802.3 Standard.

  11. 11. S. H. Lee and K. H. Cho, “Congestion Control of High-Speed Gigabit-Ethernet Networks for Industrial Applications,” Proceedings of the IEEE International Symposium on Industrial Electronics ISIE, Vol. 1, Pusan, 12-16 June 2001, pp. 260-265.

  12. 12. J. S. Meditch and C. T. A. Lea, “Stability and Optimization of the CSMA and CSMA/CD Channels,” IEEE Transactions on Communications, Vol. 31, No. 6, 1983, pp. 763-774. doi:10.1109/TCOM.1983.1095881

  13. 13. ODVA, “EtherNet/IP Adaptation on CIP,” CIP Common.

  14. 14. Allen-Bradley, “EtherNet/IP Performance and Application Guide,” Rockwell Automation Application Solution, 2003.

  15. 15. IEC 61784-1.

  16. 16. IEC 61784-2.

  17. 17. J. Ferreira, P. Pedreiras, L. Almeida and J. Fonseca, “Achieving Fault-Tolerance in FTT-CAN,” Proceedings of the 4th IEEE International Workshop on Factory Communication Systems WFCS, Vasteras, August 2002, pp. 125-132. doi:10.1109/WFCS.2002.1159709

  18. 18. P. Pedreiras, L. Almeida and P. Gai, “The FTT-Ethernet Protocol: Merging Flexibility, Timeliness and Efficiency,” Proceedings of the IEEE Euromicro Conference on RealTime Systems ECRTS, Vienna, 19-21 June 2002, pp. 134142.

  19. 19. K. Steinhammer and A. Ademaj, “Hardware Implementation of the Time-Triggered Ethernet Controller,” Embedded System Design: Topics, Techniques and Trends, Vol. 231, Springer, Boston, 2007, pp. 325-338.

  20. 20. Vector CANtech Inc., “Overview of Current Automotive Protocols,” 2003.

  21. 21. B. P. Upender, “Analyzing the Real-Time Characteristics of Class C Communications in CAN through Discrete Event Simulations,” SAE Technical Paper 940133, United Technologies Research Center, East Hartford, 1994.

  22. 22. M. Aziz, B. Raouf, N. Riad, R. M. Daoud and H. M. Elsayed, “The Use of Ethernet for Single On-board Train Network,” Proceedings of the IEEE International Conference on Networking, Sensing and Control ICNSC, Sanya, 6-8 April 2008, pp. 1430-1434.

  23. 23. M. Hassan, S. Gamal, S. Louis, G. F. Za and H. H. Amer, “Fault Tolerant Ethernet Network Model for Control and Entertainment in Railway Transportation Systems,” Proceedings of the Canadian Conference on Electrical and Computer Engineering CCECE, Niagara Falls, 4-7 May 2008, pp. 000771-000774.

  24. 24. M. Hassan, R. M. Daoud and H. H. Amer, “Two-Wagon Fault-Tolerant Ethernet Networked Control System,” Proceedings of the Applied Computing Conference, Istanbul, 2008, pp. 346-351.

  25. 25. H. Kirrmann and P. A. Zuber, “The IEC/IEEE Train Communication Network,” IEEE Micro, Vol. 21, No. 2, 2001, pp. 81-92.

  26. 26. T. Sullivan, “The IEEE 1473-L Communications Protocol: Experience in Rail Transit,” Transportation Systems Design Inc., Oakland, 2002.

  27. 27. International Electrotechnical Committee, “IEC 61375,” Train Communication Network, Geneva, 1999.

  28. 28. Siemens, “Siemens AG Transportation Systems Trains,” In: Trains Reference List, 2006, pp. 41-46.

  29. 29. H. H. Amer, M. S. Moustafa and R. M. Daoud, “Optimum Machine Performance in Fault-Tolerant Networked Control Systems,” Proceedings of the IEEE EUROCON Conference, Belgrade, 21-24 November 2005, pp. 346349.

  30. 30. H. H. Amer, M. S. Moustafa and R. M. Daoud, “Availability of Pyramid Industrial Networks,” Proceedings of the Canadian Conference on Electrical and Computer Engineering CCECE, Ottawa, May 2006, pp. 1862-1865.

  31. 31. H. H. Amer and R. M. Daoud, “Parameter Determination for the Markov Modeling of Two-Machine Production Lines,” Proceedings of the International IEEE Conference on Industrial Informatics INDIN, Singapore, 16-18 August 2006, pp. 1178-1182. doi:10.1109/INDIN.2006.275805

  32. 32. M. Blanke, M. Kinnaert, J. Lunze and M. Staroswiecki, “Diagnosis and Fault-Tolerant Control,” 2nd Edition, Springer-Verlag Berlin Heidelberg, Berlin, 2006.

  33. 33. OPNET Network Modeler, 2011.

  34. 34. H. Kitabayashi, K. Ishid, K. Bekki and M. Nagasu, “New Train Control and Information Services Utilizing Broadband Networks,” 2004.

  35. 35. M. Hassan, G. F. Zaki, R. M. Daoud, H. M. ElSayed and H. H. Amer, “Reliable Train Network Using Triple Modular Redundancy at the Sensors Level,” In: S. Pennacchio, Eds., Emerging Technologies, Robotics and Control Systems, Ass. Internationalsar, Bologna, 2008, pp. 39-44.

  36. 36. D. P. Siewiorek and R. S. Swarz, “Reliable Computer Systems—Design and Evaluation,” A. K. Peters, Natick, 1998.

  37. 37. K. S. Trivedi, “Probability and Statistics with Reliability, Queuing, and Computer Science Applications,” Wiley, New York, 2002.

  38. 38. Cisco Systems, “Cisco Catalyst 3560 Series Switch,” 2011.

  39. 39., Inc., “V.O.D Streaming Speed,” 2011.

  40. 40. G. Marsal, “Evaluation of Time Performances of Ethernet-Based Automation Systems by Simulation of HighLevel Petri Nets,” Ph.D. Thesis, école Normale Supérieure de Cachan, Cachan, 2006.

  41. 41. R. M. Daoud, “Wireless and Wired Ethernet for Intelligent Transportation Systems,” D.Sc. Dissertation, LAMIH-SP, University of Valenciennes and HainautCambresis, Valenciennes, 2008.