A reinforcement learning-based routing for real-time multimedia traffic transmission over software-defined networking

Journal article

Al Jameel, M., Kanakis, T., Turner, S., Al-Sherbaz, A. and Bhaya, W. 2022. A reinforcement learning-based routing for real-time multimedia traffic transmission over software-defined networking. Electronics. 11 (15), p. e2441. https://doi.org/10.3390/electronics11152441

Publication dates
Authors	Al Jameel, M., Kanakis, T., Turner, S., Al-Sherbaz, A. and Bhaya, W.
Abstract	Recently, video streaming services consumption has grown massively and is foreseen to increase even more in the future. The tremendous traffic usage has negatively impacted the network’s quality of service due to network congestion and end-to-end customers’ satisfaction represented by the quality of experience, especially during evening peak hours. This paper introduces an intelligent multimedia framework that aims to optimise the network’s quality of service and users’ quality of experience by taking into account the integration of Software-Defined Networking and Reinforcement Learning, which enables exploring, learning, and exploiting potential paths for video streaming flows. Moreover, an objective study was conducted to assess video streaming for various realistic network environments and under low and high traffic loads to obtain two quality of experience metrics; video multimethod assessment fusion and structural similarity index measure. The experimental results validate the effectiveness of the proposed solution strategy, which demonstrated better viewing quality by achieving better customers’ quality of experience, higher throughput and lower data loss compared with the currently existing solutions.
Keywords	Video streaming services; QoE; QoS; SDN; Reinforcement learning
Year	2022
Journal	Electronics
Journal citation	11 (15), p. e2441
Publisher	MDPI
ISSN	2079-9292
Digital Object Identifier (DOI)	https://doi.org/10.3390/electronics11152441
Official URL	https://www.mdpi.com/2079-9292/11/15/2441
Online	05 Aug 2022
Publication process dates
Accepted	26 Jul 2022
Deposited	08 Aug 2022
Publisher's version	A Reinforcement Learning-Based Routing for Real-Time Multimedia Traffic Transmission over Software-Defined Networking _ Enhanced Reader.pdf License CC BY 4.0 File Access Level Open
Output status	Published
References	Sandvine. The Global Internet Phenomena Report COVID-19 Spotlight; Sandvine: Waterloo, ON, Canada, 2020. Trestian, R.; Comsa, I.S.; Tuysuz, M.F. Seamless multimedia delivery within a heterogeneous wireless networks environment: Are we there yet? IEEE Commun. Surv. Tutor. 2018, 20, 945-977. https://doi.org/10.1109/COMST.2018.2789722 Doumanoglou, A.; Zioulis, N.; Griffin, D.; Serrano, J.; Phan, T.K.; Jiménez, D.; Zarpalas, D.; Alvarez, F.; Rio, M.; Daras, P. A system architecture for live immersive 3D-media transcoding over 5G networks. In Proceedings of the 2018 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB), Valencia, Spain, 6-8 June 2018; pp. 11-15. https://doi.org/10.1109/BMSB.2018.8436942 Jawad, N.; Salih, M.; Ali, K.; Meunier, B.; Zhang, Y.; Zhang, X.; Zetik, R.; Zarakovitis, C.; Koumaras, H.; Kourtis, M.A.; et al. Smart television services using NFV/SDN network management. IEEE Trans. Broadcast. 2019, 65, 404-413. https://doi.org/10.1109/TBC.2019.2898159 Barakabitze, A.A. QoE-Centric Control and Management of Multimedia Services in Software Defined and Virtualized Networks. Ph.D. Thesis, University of Plymouth, Plymouth, UK, 2020. Martin, A.; Egaña, J.; Flórez, J.; Montalban, J.; Olaizola, I.G.; Quartulli, M.; Viola, R.; Zorrilla, M. Network resource allocation system for QoE-aware delivery of media services in 5G networks. IEEE Trans. Broadcast. 2018, 64, 561-574. https://doi.org/10.1109/TBC.2018.2828608 Comşa, I.S.; Muntean, G.M.; Trestian, R. An innovative machine-learning-based scheduling solution for improving live UHD video streaming quality in highly dynamic network environments. IEEE Trans. Broadcast. 2020, 67, 212-224. https://doi.org/10.1109/TBC.2020.2983298 Huang, X.; Yuan, T.; Qiao, G.; Ren, Y. Deep reinforcement learning for multimedia traffic control in software defined networking. IEEE Netw. 2018, 32, 35-41. https://doi.org/10.1109/MNET.2018.1800097 Grigoriou, E. Quality of Experience Monitoring and Management Strategies for Future Smart Networks. 2020. Available online: https://iris.unica.it/handle/11584/284401 (accessed on 16 February 2022). Ullah, Z.; Al-Turjman, F.; Mostarda, L.; Gagliardi, R. Applications of artificial intelligence and machine learning in smart cities. Comput. Commun. 2020, 154, 313-323. https://doi.org/10.1016/j.comcom.2020.02.069 Lekharu, A.; Moulii, K.; Sur, A.; Sarkar, A. Deep learning based prediction model for adaptive video streaming. In Proceedings of the 2020 International Conference on COMmunication Systems & NETworkS (COMSNETS), Bangalore, India, 7-11 January 2020; pp. 152-159. https://doi.org/10.1109/COMSNETS48256.2020.9027383 Anand, D.; Togou, M.A.; Muntean, G.M. A Machine Learning Solution for Automatic Network Selection to Enhance Quality of Service for Video Delivery. In Proceedings of the 2021 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB), Chengdu, China, 4-6 August 2021; pp. 1-5. https://doi.org/10.1109/BMSB53066.2021.9547176 Kattadige, C.; Raman, A.; Thilakarathna, K.; Lutu, A.; Perino, D. 360NorVic: 360-degree video classification from mobile encrypted video traffic. In Proceedings of the 31st ACM Workshop on Network and Operating Systems Support for Digital Audio and Video, Istanbul, Turkey, 28 September-1 October 2021; pp. 58-65. https://doi.org/10.1145/3458306.3460998 Anerousis, N.; Chemouil, P.; Lazar, A.A.; Mihai, N.; Weinstein, S.B. The Origin and Evolution of Open Programmable Networks and SDN. IEEE Commun. Surv. Tutor. 2021, 23, 1956-1971. https://doi.org/10.1109/COMST.2021.3060582 Egilmez, H.E.; Civanlar, S.; Tekalp, A.M. An optimization framework for QoS-enabled adaptive video streaming over OpenFlow networks. IEEE Trans. Multimed. 2012, 15, 710-715. https://doi.org/10.1109/TMM.2012.2232645 Juttner, A.; Szviatovski, B.; Mécs, I.; Rajkó, Z. Lagrange relaxation based method for the QoS routing problem. In Proceedings of the Conference on Computer Communications-Twentieth Annual Joint Conference of the IEEE Computer and Communications Society (Cat. No. 01CH37213), Anchorage, AK, USA, 22-26 April 2001; Volume 2, pp. 859-868. Yu, T.F.; Wang, K.; Hsu, Y.H. Adaptive routing for video streaming with QoS support over SDN networks. In Proceedings of the 2015 International Conference on Information Networking (ICOIN), Siem Reap, Cambodia, 12-14 January 2015; pp. 318-323. Ongaro, F.; Cerqueira, E.; Foschini, L.; Corradi, A.; Gerla, M. Enhancing the quality level support for real-time multimedia applications in software-defined networks. In Proceedings of the 2015 International Conference on Computing, Networking and Communications (ICNC), Garden Grove, CA, USA, 16-19 February 2015; pp. 505-509. https://doi.org/10.1109/ICCNC.2015.7069395 Rego, A.; Sendra, S.; Jimenez, J.M.; Lloret, J. OSPF routing protocol performance in Software Defined Networks. In Proceedings of the 2017 Fourth International Conference on Software Defined Systems (SDS), Valencia, Spain, 8-11 May 2017; pp. 131-136. [Google Scholar] https://doi.org/10.1109/SDS.2017.7939153 Rego, A.; Sendra, S.; Jimenez, J.M.; Lloret, J. Dynamic metric OSPF-based routing protocol for software defined networks. Clust. Comput. 2019, 22, 705-720. https://doi.org/10.1007/s10586-018-2875-7 Elbasheer, M.O.; Aldegheishem, A.; Lloret, J.; Alrajeh, N. A QoS-Based routing algorithm over software defined networks. J. Netw. Comput. Appl. 2021, 194, 103215. https://doi.org/10.1016/j.jnca.2021.103215 Uzakgider, T.; Cetinkaya, C.; Sayit, M. Learning-based approach for layered adaptive video streaming over SDN. Comput. Netw. 2015, 92, 357-368. [Google Scholar] [CrossRef] https://doi.org/10.1016/j.comnet.2015.09.027 Sendra, S.; Rego, A.; Lloret, J.; Jimenez, J.M.; Romero, O. Including artificial intelligence in a routing protocol using software defined networks. In Proceedings of the 2017 IEEE International Conference on Communications Workshops (ICC Workshops), Paris, France, 21-23 May 2017; pp. 670-674. https://doi.org/10.1109/ICCW.2017.7962735 Al-Jawad, A.; Shah, P.; Gemikonakli, O.; Trestian, R. LearnQoS: A learning approach for optimizing QoS over multimedia-based SDNs. In Proceedings of the 2018 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB), Valencia, Spain, 6-8 June 2018; pp. 1-6. https://doi.org/10.1109/BMSB.2018.8436781 Hossain, M.B.; Wei, J. Reinforcement learning-driven QoS-aware intelligent routing for software-defined networks. In Proceedings of the 2019 IEEE global conference on signal and information processing (GlobalSIP), Ottawa, ON, Canada, 11-14 November 2019; pp. 1-5. https://doi.org/10.1109/GlobalSIP45357.2019.8969320 Godfrey, D.; Kim, B.S.; Miao, H.; Shah, B.; Hayat, B.; Khan, I.; Sung, T.E.; Kim, K.I. Q-learning based routing protocol for congestion avoidance. Comput. Mater. Contin. 2021, 68, 3671. https://doi.org/10.32604/cmc.2021.017475 Al-Jawad, A.; Comşa, I.S.; Shah, P.; Gemikonakli, O.; Trestian, R. An innovative reinforcement learning-based framework for quality of service provisioning over multimedia-based sdn environments. IEEE Trans. Broadcast. 2021, 67, 851-867. https://doi.org/10.1109/TBC.2021.3099728 Al-Jawad, A.; Comşa, I.-S.; Shah, P.; Gemikonakli, O.; Trestian, R. REDO: A reinforcement learning-based dynamic routing algorithm selection method for SDN. In Proceedings of the 2021 IEEE Conference on Network Function Virtualization and Software Defined Networks (NFV-SDN), Online, 9-11 November 2021; pp. 54-59. https://doi.org/10.1109/NFV-SDN53031.2021.9665140 Guo, Y.; Wang, W.; Zhang, H.; Guo, W.; Wang, Z.; Tian, Y.; Yin, X.; Wu, J. Traffic Engineering in hybrid Software Defined Network via Reinforcement Learning. J. Netw. Comput. Appl. 2021, 189, 103116. https://doi.org/10.1016/j.jnca.2021.103116 Liu, W.x.; Cai, J.; Chen, Q.C.; Wang, Y. DRL-R: Deep reinforcement learning approach for intelligent routing in software-defined data-center networks. J. Netw. Comput. Appl. 2021, 177, 102865. https://doi.org/10.1016/j.jnca.2020.102865 Gueant, V. iPerf-The Ultimate Speed Test Tool for TCP, UDP and SCTPTEST the Limits of Your Network + Internet Neutrality Test. Available online: https://iperf.fr/ (accessed on 10 December 2021). Asadollahi, S.; Goswami, B.; Sameer, M. Ryu controller's scalability experiment on software defined networks. In Proceedings of the 2018 IEEE international conference on current trends in advanced computing (ICCTAC), Bangalore, India, 1-2 February 2018; pp. 1-5. https://doi.org/10.1109/ICCTAC.2018.8370397 Vega, M.T.; Perra, C.; Liotta, A. Resilience of video streaming services to network impairments. IEEE Trans. Broadcast. 2018, 64, 220-234. [Google Scholar] [CrossRef] https://doi.org/10.1109/TBC.2017.2781125 Kim, H.J.; Yun, D.G.; Kim, H.S.; Cho, K.S.; Choi, S.G. QoE assessment model for video streaming service using QoS parameters in wired-wireless network. In Proceedings of the 2012 14th International Conference on Advanced Communication Technology (ICACT), Pyeongchang, Korea, 19-22 February 2012; pp. 459-464. Chen, Y.; Wu, K.; Zhang, Q. From QoS to QoE: A tutorial on video quality assessment. IEEE Commun. Surv. Tutor. 2014, 17, 1126-1165. https://doi.org/10.1109/COMST.2014.2363139 Oginni, O.; Bull, P.; Wang, Y. Constraint-aware software-defined network for routing real-time multimedia. ACM SIGBED Rev. 2018, 15, 37-42. https://doi.org/10.1145/3267419.3267425 Benmir, A.; Korichi, A.; Bourouis, A.; Alreshoodi, M.; Al-Jobouri, L. GeoQoE-Vanet: QoE-aware geographic routing protocol for video streaming over vehicular ad-hoc networks. Computers 2020, 9, 45. https://doi.org/10.3390/computers9020045 Mammeri, Z. Reinforcement learning based routing in networks: Review and classification of approaches. IEEE Access 2019, 7, 55916-55950. https://doi.org/10.1109/ACCESS.2019.2913776 Sutton, R.S.; Barto, A.G. Reinforcement Learning: An Introduction; MIT Press: Cambridge, MA, USA, 2018. [ Juluri, P.; Tamarapalli, V.; Medhi, D. Measurement of quality of experience of video-on-demand services: A survey. IEEE Commun. Surv. Tutor. 2015, 18, 401-418. https://doi.org/10.1109/COMST.2015.2401424 Al Shalabi, L.; Shaaban, Z. Normalization as a preprocessing engine for data mining and the approach of preference matrix. In Proceedings of the 2006 International Conference on Dependability of Computer Systems, Szklarska Poręba, Poland, 25-27 May 2006; pp. 207-214. https://doi.org/10.1109/DEPCOS-RELCOMEX.2006.38 de Oliveira, R.L.S.; Schweitzer, C.M.; Shinoda, A.A.; Prete, L.R. Using Mininet for emulation and prototyping Software-Defined Networks. In Proceedings of the 2014 IEEE Colombian Conference on Communications and Computing (COLCOM), Bogota, Colombia, 4-6 June 2014; pp. 1-6. https://doi.org/10.1109/ColComCon.2014.6860404 Henni, D.E.; Ghomari, A.; Hadjadj-Aoul, Y. A consistent QoS routing strategy for video streaming services in SDN networks. Int. J. Commun. Syst. 2020, 33, e4177. https://doi.org/10.1002/dac.4177 Kirstein, P.T. European International Academic Networking: A 20 Year Perspective. In Proceedings of the TERENA Networking Conference, Rhodes, Greece, 7-10 June 2004. Liu, Y. Current situation and prospect of CERNET. In China's e-Science Blue Book 2020; Springer: Berlin/Heidelberg, Germany, 2021; pp. 327-334. https://doi.org/10.1007/978-981-15-8342-1_19 Lahoulou, A.; Larabi, M.C.; Beghdadi, A.; Viennet, E.; Bouridane, A. Knowledge-based taxonomic scheme for full-reference objective image quality measurement models. J. Imaging Sci. Technol. 2016, 60, 60406-1. https://doi.org/10.2352/J.ImagingSci.Technol.2016.60.6.060406 Li, Z.; Bampis, C.; Novak, J.; Aaron, A.; Swanson, K.; Moorthy, A.; Cock, J. Netflix Technology Blog-VMAF: The Journey Continues. 2018. Available online: http://mcl.usc.edu/wp-content/uploads/2018/10/2018-10-25-Netflix-Wor... (accessed on 25 March 2022). Sara, U.; Akter, M.; Uddin, M.S. Image quality assessment through FSIM, SSIM, MSE and PSNR-A comparative study. J. Comput. Commun. 2019, 7, 8-18. https://doi.org/10.4236/jcc.2019.73002 Big Buck Bunny. Available online: https://peach.blender.org/ (accessed on 18 January 2022).
Additional information	Publications router: Date 2022-07-26 of type 'accepted_date' included in notification. Publications router: Date 2022-08-05 of type 'publication_date' with format 'electronic' included in notification Publications router: Date 2022-07-26 of type 'accepted' included in notification Publications router: Date 2022-08-05 of type 'pub-electronic' included in notification
	Publications router: License for this article: https://creativecommons.org/licenses/by/4.0/ included in notification
License	https://creativecommons.org/licenses/by/4.0/

Permalink -

https://repository.canterbury.ac.uk/item/91z49/a-reinforcement-learning-based-routing-for-real-time-multimedia-traffic-transmission-over-software-defined-networking

Download files

Publisher's version

	A Reinforcement Learning-Based Routing for Real-Time Multimedia Traffic Transmission over Software-Defined Networking _ Enhanced Reader.pdf
License: CC BY 4.0
File access level: Open

184
total views
26
total downloads
0
views this month
0
downloads this month

Export as

Related outputs

The role of use cases when adopting augmented reality into higher education pedagogy

Ward, G., Turner, S., Pitt, C., Qi, M., Richmond-Fuller, A. and Jackson, T. 2024. The role of use cases when adopting augmented reality into higher education pedagogy.

The National Teaching Repository and social media

Turner, S., Faulkner, S and Withnell, N 2023. The National Teaching Repository and social media. https://doi.org/10.25416/NTR.24942471.v1

Trustworthy Insights: A Novel Multi-Tier Explainable framework for ambient assisted living

Kasirajan, M., Azhar, H. and Turner, S. 2023. Trustworthy Insights: A Novel Multi-Tier Explainable framework for ambient assisted living . https://doi.org/10.1109/TrustCom60117.2023.00357

The National Teaching Repository − Sharing effective interventions: Learning from each other so that we can continue to enhance and improve what we do

Turner, S., Beckingham, S, Bullingham, L, Hartley, P, Cuthbert, K, Irving-Bell, D, Wooff, D, Tasler, N, Stinson, L and Withnell, N 2023. The National Teaching Repository − Sharing effective interventions: Learning from each other so that we can continue to enhance and improve what we do. Educational Developments. 24 (2), pp. 5-7.

An intelligent routing approach for multimedia traffic transmission over SDN

Jameel, Mohammed Al, Kanakis, Triantafyllos, Turner, Scott, Al-Sherbaz, Ali, Bhaya, Wesam S. and Al-khafajiy, Mohammed 2023. An intelligent routing approach for multimedia traffic transmission over SDN. in: IEEE.

Why should everybody learn Artificial Intelligence?

Turner, S. and Souag, A. 2022. Why should everybody learn Artificial Intelligence? ETD blog, Canterbury Christ church University

Optimizing artificial neural networks using LevyChaotic mapping on Wolf Pack optimization algorithm for detect driving sleepiness

Turner, S., Jassin, S.S. and Hassan, A.K.A 2022. Optimizing artificial neural networks using LevyChaotic mapping on Wolf Pack optimization algorithm for detect driving sleepiness. Iraqi Journal of Computers, Communications, Control & Systems Engineering (IJCCCE). 22 (3), pp. 128-136. https://doi.org/10.33103/uot.ijccce.22.3.12

Driver drowsiness detection using Gray Wolf Optimizer based on voice recognition

Sasim, S. S., Hassan, A. K. A. and Turner, S. 2022. Driver drowsiness detection using Gray Wolf Optimizer based on voice recognition. Aro - The Scientific Journal of Koya University. 10 (2), pp. 142-151. https://doi.org/10.14500/aro.11000

Practical ways to analyse Twitter data (quantitative and qualitative)

Turner, S. and Kelly, O. 2022. Practical ways to analyse Twitter data (quantitative and qualitative).

#LTHEchat 243: Self exclusion – through digital inequalities

Turner, S., Ward, G. and Elliott, C. 2022. #LTHEchat 243: Self exclusion – through digital inequalities. LTHEchat.

Driver drowsiness detection using Gray Wolf Optimizer based on face and eye tracking

Jasim, S., Abdul Hassan, AK and Turner, S. 2022. Driver drowsiness detection using Gray Wolf Optimizer based on face and eye tracking. Aro - The Scientific Journal of Koya University. 10 (1), pp. 49-56. https://doi.org/10.14500/aro.10928

Deep learning approach for real-time video streaming traffic classification

Jameel, Mohammed Al, Turner, Scott, Kanakis, Triantafyllos, Al-Sherbaz, Ali and Bhaya, Wesam S. 2022. Deep learning approach for real-time video streaming traffic classification. in: 2022 International Conference on Computer Science and Software Engineering (CSASE) IEEE.