A resource aware MapReduce based parallel SVM for large scale image classification
Journal article
Guo, W., Khalid, N., Liu, Y., Li, M., Qi, M., Guo, W., Khalid, N., Liu, Y., Li, M. and Qi, M. 2015. A resource aware MapReduce based parallel SVM for large scale image classification. Neural Processing Letters. 44 (1), pp. 161-184.
| Authors | Guo, W., Khalid, N., Liu, Y., Li, M., Qi, M., Guo, W., Khalid, N., Liu, Y., Li, M. and Qi, M. |
|---|---|
| Abstract | Machine learning techniques have facilitated image retrieval by automatically classifying and annotating images with keywords. Among them support vector machines (SVMs) are used extensively due to their generalization properties. However, SVM training is notably a computationally intensive process especially when the training dataset is large. This paper presents RASMO, a resource aware MapReduce based parallel SVM algorithm for large scale image classifications which partitions the training data set into smaller subsets and optimizes SVM training in parallel using a cluster of computers. A genetic algorithm based load balancing scheme is designed to optimize the performance of RASMO in heterogeneous computing environments. RASMO is evaluated in both experimental and simulation environments. The results show that the parallel SVM algorithm reduces the training time significantly compared with the sequential SMO algorithm while maintaining a high level of accuracy in classifications |
| Keywords | Parallel SVM; MapReduce; image classification and annotation; load balancing |
| Year | 2015 |
| Journal | Neural Processing Letters |
| Journal citation | 44 (1), pp. 161-184 |
| Publisher | Kluwer |
| ISSN | 1370-4621 |
| Official URL | https://dl.acm.org/citation.cfm?id=J580&picked=prox |
| Publication dates | |
| Online | 18 Sep 2015 |
| Publication process dates | |
| Deposited | 03 Apr 2018 |
| Accepted author manuscript | |
| Output status | Published |
Permalink -
https://repository.canterbury.ac.uk/item/88q11/a-resource-aware-mapreduce-based-parallel-svm-for-large-scale-image-classification
Download files
Accepted author manuscript
135
total views29
total downloads4
views this month0
downloads this month
Export as
Related outputs
Shaping the future of metaverse research
Enis Karaarslan, Ömer Aydın, Senem Yazıcı Yılma, Hasan Tınmaz, Süleyman Eken, Nida Gökçe Narin, Umit Cali, Kürşad Özkaynar, Önder Gürcan, Mina Fanea-ıvanovici, Osman Gazi Güçlütürk, Qi, Mandy, Huseyin Kusetogullari, Moharram Challenger and Nebojsa Bacanin 2025. Shaping the future of metaverse research. Journal of Metaverse. 5 (2), pp. 73-91. https://doi.org/10.57019/jmv.1664054Default clearing and ex-ante contagion in financial systems with a two-layer network structure
Qi, M., Ding, Y., Chun, Y., Liu, W. and Liu, J. 2025. Default clearing and ex-ante contagion in financial systems with a two-layer network structure. Communications in Nonlinear Science and Numerical Simulation. 142 (1), p. 108515. https://doi.org/10.1016/j.cnsns.2024.108515Two-stage trading mechanism in enabling the design and optimization of flexible resources interaction in smart grid
Qi, M., Chen, Y., Li, H., Liu, Y., Wu, S. and Li, X. 2024. Two-stage trading mechanism in enabling the design and optimization of flexible resources interaction in smart grid. Computing and Informatics. 43 (6), p. 1483–1515. https://doi.org/10.31577/cai_2024_6_1483Corporate fraud detection based on improved BP neural network
Qi, M., Liu, W., Liu, M,, Yan, C. and Zhang, L. 2024. Corporate fraud detection based on improved BP neural network. Computing and Informatics. 43 (3), pp. 611–632-611–632. https://doi.org/10.31577/cai_2024_3_611Normalizing flow based uncertainty estimation for deep regression analysis
Qi, M., Zhang, B., Sui, W, Li, M. and Huang, Z. 2024. Normalizing flow based uncertainty estimation for deep regression analysis. Neurocomputing. 585 (6), p. 127645. https://doi.org/10.1016/j.neucom.2024.127645Focusing on the golden skills of effective communication and collaboration to further enhance graduate employability attributes
O'Leary, S., McGee, M., Qi, M., Millns, S. and Eyden, A. 2024. Focusing on the golden skills of effective communication and collaboration to further enhance graduate employability attributes. Graduate College Working Papers.Feed-forward ANN with Random Forest Technique for Identifying malicious Internet of Things network intrusion
Obarafor, Victor, Qi, Mandi and Zhang, Leishi 2024. Feed-forward ANN with Random Forest Technique for Identifying malicious Internet of Things network intrusion. in: 2024 2nd International Conference on Cyber Resilience (ICCR) IEEE.
The role of use cases when adopting augmented reality into higher education pedagogy
Ward, G., Turner, S., Pitt, C., Qi, M., Richmond-Fuller, A. and Jackson, T. 2024. The role of use cases when adopting augmented reality into higher education pedagogy.A review of privacy-preserving federated learning, deep learning, and machine learning IIoT and IoTs solutions
Obarafor, Victor, Qi, Man and Zhang, L. 2023. A review of privacy-preserving federated learning, deep learning, and machine learning IIoT and IoTs solutions. in: 2023 8th IEEE International Conference on Signal and Image Processing (ICSIP) Wuxi, China IEEE. pp. 1074-1078Pre-processing of social media remarks for forensics
Gao, Xuhao and Qi, Man 2023. Pre-processing of social media remarks for forensics. IEEE. https://doi.org/10.1109/icnc-fskd59587.2023.10280980Alignment-based conformance checking of hierarchical process models
Wang, L, Han, X., Qi, M., Wang, K. and Lu, P. 2023. Alignment-based conformance checking of hierarchical process models. Computing and Informatics. 43 (1), pp. 149-180. https://doi.org/10.31577/cai_2024_1_149Research on UBI auto insurance pricing model based on parameter adaptive SAPSO optimal fuzzy controller
Wang, X., Liu, W., Ou, Z., Yan, C., Qi, M. and Jia, W. 2022. Research on UBI auto insurance pricing model based on parameter adaptive SAPSO optimal fuzzy controller. Computing and Informatics. 41 (4), pp. 1078-1113. https://doi.org/10.31577/cai_2022_4_1078Novel architecture for human re-identification with a two-stream neural network and attention ,echanism
Rahi, B. and Qi, M. 2022. Novel architecture for human re-identification with a two-stream neural network and attention ,echanism. Computing and Informatics. 41 (4), pp. 905-930. https://doi.org/10.31577/cai_2022_4_905A novel data analytic model for mining user insurance demands from microblogs
Chun Yan, Lu Liu, Wei Liu and Man Qi 2022. A novel data analytic model for mining user insurance demands from microblogs. Computing and Informatics. 41 (3). https://doi.org/10.31577/cai_2022_3_689Deep learning based real-time facial mask detection and crowd monitoring
Chan-Yun Yang, Hooman Samani, Nana Ji, Chunxu Li, Ding-Bang Chen and Man Qi 2022. Deep learning based real-time facial mask detection and crowd monitoring. Computing and Informatics. 40 (6), pp. 1263-11294. https://doi.org/10.31577/cai_2021_6_1263Repairing process models with non-free-choice constructs based on token replay
Bai, E., Qi, M., Luan, W., Li, P. and Du, Y. 2022. Repairing process models with non-free-choice constructs based on token replay. Computing and Informatics. 41 (4), pp. 1054-1077. https://doi.org/10.31577/cai_2022_4_1054Transformers only look once with nonlinear combination for real-time object detection
Xia, R., Li, G., Huang, Z., Pang, Y. and Qi, M. 2022. Transformers only look once with nonlinear combination for real-time object detection. Neural Computing and Applications. https://doi.org/10.1007/s00521-022-07333-yDeviation detection in clinical pathways based on business alignment
Tian, Y., Li, X., Qi, Man, Han, D. and Du, Yuyue 2022. Deviation detection in clinical pathways based on business alignment. Scientific Programming. 2022, pp. 1-13. https://doi.org/10.1155/2022/6993449Security vulnerabilities of popular smart home appliances
Qi, M., Induruwa, A. and Hussain, F. 2021. Security vulnerabilities of popular smart home appliances. in: Proceeding of The Twentieth International Conference on Networks April 18, 2021 to April 22, 2021 - Porto, PortugalImproved adaptive genetic algorithm for the vehicle insurance fraud identification model based on a BP neural network
Yan, C., Li, M., Liu, W. and Qi, M. 2020. Improved adaptive genetic algorithm for the vehicle insurance fraud identification model based on a BP neural network. Theoretical Computer Science. 817, pp. 12-23. https://doi.org/10.1016/j.tcs.2019.06.025Hybrid Intrusion Detection System for Smart Home Applications
Hussain, Fida, Induruwa, Abhaya and Qi, Man 2020. Hybrid Intrusion Detection System for Smart Home Applications. in: Mahmood, Z. (ed.) Developing and Monitoring Smart Environments for Intelligent Cities IGI Global. pp. 300-322Payments per claim model of outstanding claims reserve based on fuzzy linear regression
Yan, C., Liu, Q., Liu, J., Liu, W., Li, M. and Qi, M. 2019. Payments per claim model of outstanding claims reserve based on fuzzy linear regression. International Journal of Fuzzy Systems. 21, pp. 1950-1960. https://doi.org/10.1007/s40815-019-00617-xFuzzy interacting multiple model H∞ particle filter algorithm based on current statistical model
Wang, Q., Chen, X., Zhang, L., Li, J., Zhao, C. and Qi, M. 2019. Fuzzy interacting multiple model H∞ particle filter algorithm based on current statistical model. International Journal of Fuzzy Systems. 21, pp. 1894-1905. https://doi.org/10.1007/s40815-019-00678-yTemporal sparse feature auto-combination deep network for video action recognition
Wang, Q., Gong, D., Qi, M., Shen, Y. and Lei, Y. 2018. Temporal sparse feature auto-combination deep network for video action recognition. Concurrency and Computation: Practice and Experience. https://doi.org/10.1002/cpe.4487Soundness analytics of composed logical workflow nets
Liu, W., Wang, L., Feng, X., Qi, M., Yan, C. and Li, M. 2017. Soundness analytics of composed logical workflow nets. International Journal of Parallel Programming. https://doi.org/10.1007/s10766-017-0536-8A sliding window-based dynamic load balancing for heterogeneous Hadoop clusters
Liu, Y., Jing, W., Liu, Y., Lv, L., Qi, M. and Xiang, Y. 2016. A sliding window-based dynamic load balancing for heterogeneous Hadoop clusters. Concurrency and Computation: Practice and Experience. 29 (3). https://doi.org/10.1002/cpe.3763Gaussian-Gamma collaborative filtering: a hierarchical Bayesian model for recommender systems
Luo, C., Zhang, B., Xiang, Y. and Qi, M. 2017. Gaussian-Gamma collaborative filtering: a hierarchical Bayesian model for recommender systems. Journal of Computer and System Sciences. https://doi.org/10.1016/j.jcss.2017.03.007Facilitating visual surveillance with motion detections
Qi, M. 2017. Facilitating visual surveillance with motion detections. Concurrency and Computation: Practice and Experience. 29 (3). https://doi.org/10.1002/cpe.3770Data security of android applications
Obiri-Yeboah, J. and Qi, M. 2016. Data security of android applications. in: 2016 12th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery : ICNC-FSKD 2016 : 13-15 August, Changsha, China IEEE Xplore.AL-DDCNN : a distributed crossing semantic gap learning for person re-identification
Cheng, K., Zhan, Y. and Qi, M. 2017. AL-DDCNN : a distributed crossing semantic gap learning for person re-identification. Concurrency and Computation: Practice and Experience. 29 (3). https://doi.org/10.1002/cpe.3766