A reinforcement learning-based routing for real-time multimedia traffic transmission over software-defined networking

Journal article


Al Jameel, M., Kanakis, T., Turner, S., Al-Sherbaz, A. and Bhaya, W. 2022. A reinforcement learning-based routing for real-time multimedia traffic transmission over software-defined networking. Electronics. 11 (15), p. e2441. https://doi.org/10.3390/electronics11152441
AuthorsAl Jameel, M., Kanakis, T., Turner, S., Al-Sherbaz, A. and Bhaya, W.
AbstractRecently, video streaming services consumption has grown massively and is foreseen to increase even more in the future. The tremendous traffic usage has negatively impacted the network’s quality of service due to network congestion and end-to-end customers’ satisfaction represented by the quality of experience, especially during evening peak hours. This paper introduces an intelligent multimedia framework that aims to optimise the network’s quality of service and users’ quality of experience by taking into account the integration of Software-Defined Networking and Reinforcement Learning, which enables exploring, learning, and exploiting potential paths for video streaming flows. Moreover, an objective study was conducted to assess video streaming for various realistic network environments and under low and high traffic loads to obtain two quality of experience metrics; video multimethod assessment fusion and structural similarity index measure. The experimental results validate the effectiveness of the proposed solution strategy, which demonstrated better viewing quality by achieving better customers’ quality of experience, higher throughput and lower data loss compared with the currently existing solutions.
KeywordsVideo streaming services; QoE; QoS; SDN; Reinforcement learning
Year2022
JournalElectronics
Journal citation11 (15), p. e2441
PublisherMDPI
ISSN2079-9292
Digital Object Identifier (DOI)https://doi.org/10.3390/electronics11152441
Official URLhttps://www.mdpi.com/2079-9292/11/15/2441
Publication dates
Online05 Aug 2022
Publication process dates
Accepted26 Jul 2022
Deposited08 Aug 2022
Publisher's version
License
File Access Level
Open
Output statusPublished
References

Sandvine. The Global Internet Phenomena Report COVID-19 Spotlight; Sandvine: Waterloo, ON, Canada, 2020.

Trestian, R.; Comsa, I.S.; Tuysuz, M.F. Seamless multimedia delivery within a heterogeneous wireless networks environment: Are we there yet? IEEE Commun. Surv. Tutor. 2018, 20, 945-977.
https://doi.org/10.1109/COMST.2018.2789722

Doumanoglou, A.; Zioulis, N.; Griffin, D.; Serrano, J.; Phan, T.K.; Jiménez, D.; Zarpalas, D.; Alvarez, F.; Rio, M.; Daras, P. A system architecture for live immersive 3D-media transcoding over 5G networks. In Proceedings of the 2018 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB), Valencia, Spain, 6-8 June 2018; pp. 11-15.
https://doi.org/10.1109/BMSB.2018.8436942

Jawad, N.; Salih, M.; Ali, K.; Meunier, B.; Zhang, Y.; Zhang, X.; Zetik, R.; Zarakovitis, C.; Koumaras, H.; Kourtis, M.A.; et al. Smart television services using NFV/SDN network management. IEEE Trans. Broadcast. 2019, 65, 404-413.
https://doi.org/10.1109/TBC.2019.2898159

Barakabitze, A.A. QoE-Centric Control and Management of Multimedia Services in Software Defined and Virtualized Networks. Ph.D. Thesis, University of Plymouth, Plymouth, UK, 2020.

Martin, A.; Egaña, J.; Flórez, J.; Montalban, J.; Olaizola, I.G.; Quartulli, M.; Viola, R.; Zorrilla, M. Network resource allocation system for QoE-aware delivery of media services in 5G networks. IEEE Trans. Broadcast. 2018, 64, 561-574.
https://doi.org/10.1109/TBC.2018.2828608

Comşa, I.S.; Muntean, G.M.; Trestian, R. An innovative machine-learning-based scheduling solution for improving live UHD video streaming quality in highly dynamic network environments. IEEE Trans. Broadcast. 2020, 67, 212-224.
https://doi.org/10.1109/TBC.2020.2983298

Huang, X.; Yuan, T.; Qiao, G.; Ren, Y. Deep reinforcement learning for multimedia traffic control in software defined networking. IEEE Netw. 2018, 32, 35-41.
https://doi.org/10.1109/MNET.2018.1800097

Grigoriou, E. Quality of Experience Monitoring and Management Strategies for Future Smart Networks. 2020. Available online: https://iris.unica.it/handle/11584/284401 (accessed on 16 February 2022).

Ullah, Z.; Al-Turjman, F.; Mostarda, L.; Gagliardi, R. Applications of artificial intelligence and machine learning in smart cities. Comput. Commun. 2020, 154, 313-323.
https://doi.org/10.1016/j.comcom.2020.02.069

Lekharu, A.; Moulii, K.; Sur, A.; Sarkar, A. Deep learning based prediction model for adaptive video streaming. In Proceedings of the 2020 International Conference on COMmunication Systems & NETworkS (COMSNETS), Bangalore, India, 7-11 January 2020; pp. 152-159.
https://doi.org/10.1109/COMSNETS48256.2020.9027383

Anand, D.; Togou, M.A.; Muntean, G.M. A Machine Learning Solution for Automatic Network Selection to Enhance Quality of Service for Video Delivery. In Proceedings of the 2021 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB), Chengdu, China, 4-6 August 2021; pp. 1-5.
https://doi.org/10.1109/BMSB53066.2021.9547176

Kattadige, C.; Raman, A.; Thilakarathna, K.; Lutu, A.; Perino, D. 360NorVic: 360-degree video classification from mobile encrypted video traffic. In Proceedings of the 31st ACM Workshop on Network and Operating Systems Support for Digital Audio and Video, Istanbul, Turkey, 28 September-1 October 2021; pp. 58-65.
https://doi.org/10.1145/3458306.3460998

Anerousis, N.; Chemouil, P.; Lazar, A.A.; Mihai, N.; Weinstein, S.B. The Origin and Evolution of Open Programmable Networks and SDN. IEEE Commun. Surv. Tutor. 2021, 23, 1956-1971.
https://doi.org/10.1109/COMST.2021.3060582

Egilmez, H.E.; Civanlar, S.; Tekalp, A.M. An optimization framework for QoS-enabled adaptive video streaming over OpenFlow networks. IEEE Trans. Multimed. 2012, 15, 710-715.
https://doi.org/10.1109/TMM.2012.2232645

Juttner, A.; Szviatovski, B.; Mécs, I.; Rajkó, Z. Lagrange relaxation based method for the QoS routing problem. In Proceedings of the Conference on Computer Communications-Twentieth Annual Joint Conference of the IEEE Computer and Communications Society (Cat. No. 01CH37213), Anchorage, AK, USA, 22-26 April 2001; Volume 2, pp. 859-868.

Yu, T.F.; Wang, K.; Hsu, Y.H. Adaptive routing for video streaming with QoS support over SDN networks. In Proceedings of the 2015 International Conference on Information Networking (ICOIN), Siem Reap, Cambodia, 12-14 January 2015; pp. 318-323.

Ongaro, F.; Cerqueira, E.; Foschini, L.; Corradi, A.; Gerla, M. Enhancing the quality level support for real-time multimedia applications in software-defined networks. In Proceedings of the 2015 International Conference on Computing, Networking and Communications (ICNC), Garden Grove, CA, USA, 16-19 February 2015; pp. 505-509.
https://doi.org/10.1109/ICCNC.2015.7069395

Rego, A.; Sendra, S.; Jimenez, J.M.; Lloret, J. OSPF routing protocol performance in Software Defined Networks. In Proceedings of the 2017 Fourth International Conference on Software Defined Systems (SDS), Valencia, Spain, 8-11 May 2017; pp. 131-136. [Google Scholar]
https://doi.org/10.1109/SDS.2017.7939153

Rego, A.; Sendra, S.; Jimenez, J.M.; Lloret, J. Dynamic metric OSPF-based routing protocol for software defined networks. Clust. Comput. 2019, 22, 705-720.
https://doi.org/10.1007/s10586-018-2875-7

Elbasheer, M.O.; Aldegheishem, A.; Lloret, J.; Alrajeh, N. A QoS-Based routing algorithm over software defined networks. J. Netw. Comput. Appl. 2021, 194, 103215.
https://doi.org/10.1016/j.jnca.2021.103215

Uzakgider, T.; Cetinkaya, C.; Sayit, M. Learning-based approach for layered adaptive video streaming over SDN. Comput. Netw. 2015, 92, 357-368. [Google Scholar] [CrossRef]
https://doi.org/10.1016/j.comnet.2015.09.027

Sendra, S.; Rego, A.; Lloret, J.; Jimenez, J.M.; Romero, O. Including artificial intelligence in a routing protocol using software defined networks. In Proceedings of the 2017 IEEE International Conference on Communications Workshops (ICC Workshops), Paris, France, 21-23 May 2017; pp. 670-674.
https://doi.org/10.1109/ICCW.2017.7962735

Al-Jawad, A.; Shah, P.; Gemikonakli, O.; Trestian, R. LearnQoS: A learning approach for optimizing QoS over multimedia-based SDNs. In Proceedings of the 2018 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB), Valencia, Spain, 6-8 June 2018; pp. 1-6.
https://doi.org/10.1109/BMSB.2018.8436781

Hossain, M.B.; Wei, J. Reinforcement learning-driven QoS-aware intelligent routing for software-defined networks. In Proceedings of the 2019 IEEE global conference on signal and information processing (GlobalSIP), Ottawa, ON, Canada, 11-14 November 2019; pp. 1-5.
https://doi.org/10.1109/GlobalSIP45357.2019.8969320

Godfrey, D.; Kim, B.S.; Miao, H.; Shah, B.; Hayat, B.; Khan, I.; Sung, T.E.; Kim, K.I. Q-learning based routing protocol for congestion avoidance. Comput. Mater. Contin. 2021, 68, 3671.
https://doi.org/10.32604/cmc.2021.017475

Al-Jawad, A.; Comşa, I.S.; Shah, P.; Gemikonakli, O.; Trestian, R. An innovative reinforcement learning-based framework for quality of service provisioning over multimedia-based sdn environments. IEEE Trans. Broadcast. 2021, 67, 851-867.
https://doi.org/10.1109/TBC.2021.3099728

Al-Jawad, A.; Comşa, I.-S.; Shah, P.; Gemikonakli, O.; Trestian, R. REDO: A reinforcement learning-based dynamic routing algorithm selection method for SDN. In Proceedings of the 2021 IEEE Conference on Network Function Virtualization and Software Defined Networks (NFV-SDN), Online, 9-11 November 2021; pp. 54-59.
https://doi.org/10.1109/NFV-SDN53031.2021.9665140

Guo, Y.; Wang, W.; Zhang, H.; Guo, W.; Wang, Z.; Tian, Y.; Yin, X.; Wu, J. Traffic Engineering in hybrid Software Defined Network via Reinforcement Learning. J. Netw. Comput. Appl. 2021, 189, 103116.
https://doi.org/10.1016/j.jnca.2021.103116

Liu, W.x.; Cai, J.; Chen, Q.C.; Wang, Y. DRL-R: Deep reinforcement learning approach for intelligent routing in software-defined data-center networks. J. Netw. Comput. Appl. 2021, 177, 102865.
https://doi.org/10.1016/j.jnca.2020.102865

Gueant, V. iPerf-The Ultimate Speed Test Tool for TCP, UDP and SCTPTEST the Limits of Your Network + Internet Neutrality Test. Available online: https://iperf.fr/ (accessed on 10 December 2021).

Asadollahi, S.; Goswami, B.; Sameer, M. Ryu controller's scalability experiment on software defined networks. In Proceedings of the 2018 IEEE international conference on current trends in advanced computing (ICCTAC), Bangalore, India, 1-2 February 2018; pp. 1-5.
https://doi.org/10.1109/ICCTAC.2018.8370397

Vega, M.T.; Perra, C.; Liotta, A. Resilience of video streaming services to network impairments. IEEE Trans. Broadcast. 2018, 64, 220-234. [Google Scholar] [CrossRef]
https://doi.org/10.1109/TBC.2017.2781125

Kim, H.J.; Yun, D.G.; Kim, H.S.; Cho, K.S.; Choi, S.G. QoE assessment model for video streaming service using QoS parameters in wired-wireless network. In Proceedings of the 2012 14th International Conference on Advanced Communication Technology (ICACT), Pyeongchang, Korea, 19-22 February 2012; pp. 459-464.

Chen, Y.; Wu, K.; Zhang, Q. From QoS to QoE: A tutorial on video quality assessment. IEEE Commun. Surv. Tutor. 2014, 17, 1126-1165.
https://doi.org/10.1109/COMST.2014.2363139

Oginni, O.; Bull, P.; Wang, Y. Constraint-aware software-defined network for routing real-time multimedia. ACM SIGBED Rev. 2018, 15, 37-42.
https://doi.org/10.1145/3267419.3267425

Benmir, A.; Korichi, A.; Bourouis, A.; Alreshoodi, M.; Al-Jobouri, L. GeoQoE-Vanet: QoE-aware geographic routing protocol for video streaming over vehicular ad-hoc networks. Computers 2020, 9, 45.
https://doi.org/10.3390/computers9020045

Mammeri, Z. Reinforcement learning based routing in networks: Review and classification of approaches. IEEE Access 2019, 7, 55916-55950.
https://doi.org/10.1109/ACCESS.2019.2913776

Sutton, R.S.; Barto, A.G. Reinforcement Learning: An Introduction; MIT Press: Cambridge, MA, USA, 2018. [

Juluri, P.; Tamarapalli, V.; Medhi, D. Measurement of quality of experience of video-on-demand services: A survey. IEEE Commun. Surv. Tutor. 2015, 18, 401-418.
https://doi.org/10.1109/COMST.2015.2401424

Al Shalabi, L.; Shaaban, Z. Normalization as a preprocessing engine for data mining and the approach of preference matrix. In Proceedings of the 2006 International Conference on Dependability of Computer Systems, Szklarska Poręba, Poland, 25-27 May 2006; pp. 207-214.
https://doi.org/10.1109/DEPCOS-RELCOMEX.2006.38

de Oliveira, R.L.S.; Schweitzer, C.M.; Shinoda, A.A.; Prete, L.R. Using Mininet for emulation and prototyping Software-Defined Networks. In Proceedings of the 2014 IEEE Colombian Conference on Communications and Computing (COLCOM), Bogota, Colombia, 4-6 June 2014; pp. 1-6.
https://doi.org/10.1109/ColComCon.2014.6860404

Henni, D.E.; Ghomari, A.; Hadjadj-Aoul, Y. A consistent QoS routing strategy for video streaming services in SDN networks. Int. J. Commun. Syst. 2020, 33, e4177.
https://doi.org/10.1002/dac.4177

Kirstein, P.T. European International Academic Networking: A 20 Year Perspective. In Proceedings of the TERENA Networking Conference, Rhodes, Greece, 7-10 June 2004.

Liu, Y. Current situation and prospect of CERNET. In China's e-Science Blue Book 2020; Springer: Berlin/Heidelberg, Germany, 2021; pp. 327-334.
https://doi.org/10.1007/978-981-15-8342-1_19

Lahoulou, A.; Larabi, M.C.; Beghdadi, A.; Viennet, E.; Bouridane, A. Knowledge-based taxonomic scheme for full-reference objective image quality measurement models. J. Imaging Sci. Technol. 2016, 60, 60406-1.
https://doi.org/10.2352/J.ImagingSci.Technol.2016.60.6.060406

Li, Z.; Bampis, C.; Novak, J.; Aaron, A.; Swanson, K.; Moorthy, A.; Cock, J. Netflix Technology Blog-VMAF: The Journey Continues. 2018. Available online: http://mcl.usc.edu/wp-content/uploads/2018/10/2018-10-25-Netflix-Wor... (accessed on 25 March 2022).

Sara, U.; Akter, M.; Uddin, M.S. Image quality assessment through FSIM, SSIM, MSE and PSNR-A comparative study. J. Comput. Commun. 2019, 7, 8-18.
https://doi.org/10.4236/jcc.2019.73002

Big Buck Bunny. Available online: https://peach.blender.org/ (accessed on 18 January 2022).

Additional information

Publications router: Date 2022-07-26 of type 'accepted_date' included in notification.
Publications router: Date 2022-08-05 of type 'publication_date' with format 'electronic' included in notification
Publications router: Date 2022-07-26 of type 'accepted' included in notification
Publications router: Date 2022-08-05 of type 'pub-electronic' included in notification

Publications router: License for this article: https://creativecommons.org/licenses/by/4.0/ included in notification

Licensehttps://creativecommons.org/licenses/by/4.0/
Permalink -

https://repository.canterbury.ac.uk/item/91z49/a-reinforcement-learning-based-routing-for-real-time-multimedia-traffic-transmission-over-software-defined-networking

  • 97
    total views
  • 5
    total downloads
  • 0
    views this month
  • 0
    downloads this month

Export as

Related outputs

An intelligent routing approach for multimedia traffic transmission over SDN
Turner, S., Al Jameel, M., Kanakis, T., Al-Sherbaz, A. and Bhaya, W. 2023. An intelligent routing approach for multimedia traffic transmission over SDN.
Why should everybody learn Artificial Intelligence?
Turner, S. and Souag, A. 2022. Why should everybody learn Artificial Intelligence? ETD blog, Canterbury Christ church University
Driver drowsiness detection using Gray Wolf Optimizer based on voice recognition
Sasim, S. S., Hassan, A. K. A. and Turner, S. 2022. Driver drowsiness detection using Gray Wolf Optimizer based on voice recognition. Aro - The Scientific Journal of Koya University. 10 (2), pp. 142-151. https://doi.org/10.14500/aro.11000
Practical ways to analyse Twitter data (quantitative and qualitative)
Turner, S. and Kelly, O. 2022. Practical ways to analyse Twitter data (quantitative and qualitative).
#LTHEchat 243: Self exclusion – through digital inequalities
Turner, S., Ward, G. and Elliott, C. 2022. #LTHEchat 243: Self exclusion – through digital inequalities. LTHEchat.
Driver drowsiness detection using Gray Wolf Optimizer based on face and eye tracking
Jasim, S., Abdul Hassan, AK and Turner, S. 2022. Driver drowsiness detection using Gray Wolf Optimizer based on face and eye tracking. Aro - The Scientific Journal of Koya University. 10 (1), pp. 49-56. https://doi.org/10.14500/aro.10928
Deep learning approach for real-time video streaming traffic classification
Jameel, Mohammed Al, Turner, Scott, Kanakis, Triantafyllos, Al-Sherbaz, Ali and Bhaya, Wesam S. 2022. Deep learning approach for real-time video streaming traffic classification. in: 2022 International Conference on Computer Science and Software Engineering (CSASE) IEEE.
#SocMedHE more than a conference
Turner, S. 2021. #SocMedHE more than a conference.
Referencing within code in software engineering education
Turner, S. and Hill, G 2021. Referencing within code in software engineering education. National Repository of Teaching and Learning. https://doi.org/10.25416/NTR.14907891.v1
Free augmented reality
Turner, S. 2021. Free augmented reality. Edge Hill University. https://doi.org/10.25416/NTR.13622918.v1
Why everyone should learn a bit about Machine Learning
Turner, S. 2020. Why everyone should learn a bit about Machine Learning.