Elastomer-based visuotactile sensor for normality of robotic manufacturing systems

Journal article


Hassanin, H., Zaid, I., Halwani, M., Ayyad, A., Imam, A., Almaskari, F. and Zweiri, Y. 2022. Elastomer-based visuotactile sensor for normality of robotic manufacturing systems. Polymers. 14 (23), p. 5097. https://doi.org/10.3390/polym14235097
AuthorsHassanin, H., Zaid, I., Halwani, M., Ayyad, A., Imam, A., Almaskari, F. and Zweiri, Y.
Abstract

Modern aircrafts require the assembly of thousands of components with high accuracy and reliability. The normality of drilled holes is a critical geometrical tolerance that is required to be achieved in order to realize an efficient assembly process. Failure to achieve the required tolerance leads to structures prone to fatigue problems and assembly errors. Elastomer-based tactile sensors have been used to support robots in acquiring useful physical interaction information with the environments. However, current tactile sensors have not yet been developed to support robotic machining in achieving the tight tolerances of aerospace structures. In this paper, a novel elastomer-based tactile sensor was developed for cobot machining. Three commercial silicon-based elastomer materials were characterised using mechanical testing in order to select a material with the best deformability. A Finite element model was developed to simulate the deformation of the tactile sensor upon interacting with surfaces with different normalities. Additive manufacturing was employed to fabricate the tactile sensor mould, which was chemically etched to improve the surface quality. The tactile sensor was obtained by directly casting and curing the optimum elastomer material onto the additively manufactured mould. A machine learning approach was used to train the simulated and experimental data obtained from the sensor. The capability of the developed vision tactile sensor was evaluated using real-world experiments with various inclination angles, and achieved a mean perpendicularity tolerance of 0.34°. The developed sensor opens a new perspective on low-cost precision cobot machining.

KeywordsElastomer; Tactile sensor; Robotic; Manufacturing; Hole quality; Drilling; Deburring
Year2022
JournalPolymers
Journal citation14 (23), p. 5097
PublisherMDPI
ISSN2073-4360
Digital Object Identifier (DOI)https://doi.org/10.3390/polym14235097
Official URLhttps://www.mdpi.com/2073-4360/14/23/5097
Publication dates
Print24 Nov 2022
Publication process dates
Accepted20 Nov 2022
Deposited30 Nov 2022
Publisher's version
File Access Level
Open
Output statusPublished
References

Phanden, R.K.; Sharma, P.; Dubey, A. A review on simulation in digital twin for aerospace, manufacturing and robotics. Mater. Today Proc. 2021, 38, 174–178. [Google Scholar] [CrossRef]
Onstein, I.F.; Semeniuta, O.; Bjerkeng, M. Deburring using robot manipulators: A review. In Proceedings of the 2020 3rd International Symposium on Small-scale Intelligent Manufacturing Systems (SIMS), Gjøvik, Norway, 10–12 June 2020; pp. 1–7. [Google Scholar]
Alvares, A.J.; Rodriguez, E.; Jaimes, C.I.R.; Toquica, J.S.; Ferreira, J.C. STEP-NC Architectures for Industrial Robotic Machining: Review, Implementation and Validation. IEEE Access 2020, 8, 152592–152610. [Google Scholar] [CrossRef]
Luo, H.; Zhang, D.; Luo, M. Tool Wear and Remaining Useful Life Estimation of Difficult-to-machine Aerospace Alloys: A Review. China Mech. Eng. 2021, 32, 2647. [Google Scholar]
Sun, D.; Lemoine, P.; Keys, D.; Doyle, P.; Malinov, S.; Zhao, Q.; Qin, X.; Jin, Y. Hole-making processes and their impacts on the microstructure and fatigue response of aircraft alloys. Int. J. Adv. Manuf. Technol. 2018, 94, 1719–1726. [Google Scholar] [CrossRef]
Akula, S.; Nayak, S.N.; Bolar, G.; Managuli, V. Comparison of conventional drilling and helical milling for hole making in Ti6Al4V titanium alloy under sustainable dry condition. Manuf. Rev. 2021, 8, 12. [Google Scholar] [CrossRef]
Yuan, C.G.; Pramanik, A.; Basak, A.; Prakash, C.; Shankar, S. Drilling of titanium alloy (Ti6Al4V)—A review. Mach. Sci. Technol. 2021, 25, 637–702. [Google Scholar] [CrossRef]
Abdelhafeez, A.; Soo, S.; Aspinwall, D.; Dowson, A.; Arnold, D. Burr formation and hole quality when drilling titanium and aluminium alloys. Procedia Cirp 2015, 37, 230–235. [Google Scholar] [CrossRef]
Aamir, M.; Giasin, K.; Tolouei-Rad, M.; Vafadar, A. A review: Drilling performance and hole quality of aluminium alloys for aerospace applications. J. Mater. Res. Technol. 2020, 9, 12484–12500. [Google Scholar] [CrossRef]
da Silva Santos, K.R.; de Carvalho, G.M.; Tricarico, R.T.; Ferreira, L.F.L.R.; Villani, E.; Sutério, R. Evaluation of perpendicularity methods for a robotic end effector from aircraft industry. In Proceedings of the 2018 13th IEEE International Conference on Industry Applications (INDUSCON), Sao Paulo, Brazil, 12–14 November 2018; pp. 1373–1380. [Google Scholar]
Yu, L.; Zhang, Y.; Bi, Q.; Wang, Y. Research on surface normal measurement and adjustment in aircraft assembly. Precis. Eng. 2017, 50, 482–493. [Google Scholar] [CrossRef]
Zhang, Y.; Bi, Q.; Yu, L.; Wang, Y. Online adaptive measurement and adjustment for flexible part during high precision drilling process. Int. J. Adv. Manuf. Technol. 2017, 89, 3579–3599. [Google Scholar] [CrossRef]
Hassanin, H.; Abena, A.; Elsayed, M.A.; Essa, K. 4D printing of NiTi auxetic structure with improved ballistic performance. Micromachines 2020, 11, 745. [Google Scholar] [CrossRef] [PubMed]
Brambilla, C.R.; Okafor-Muo, O.L.; Hassanin, H.; ElShaer, A. 3DP printing of oral solid formulations: A systematic review. Pharmaceutics 2021, 13, 358. [Google Scholar] [CrossRef]
Khosravani, M.R.; Reinicke, T. On the environmental impacts of 3D printing technology. Appl. Mater. Today 2020, 20, 100689. [Google Scholar] [CrossRef]
Shah, U.H.; Muthusamy, R.; Gan, D.; Zweiri, Y.; Seneviratne, L. On the Design and Development of Vision-based Tactile Sensors. J. Intell. Robot. Syst. 2021, 102, 82. [Google Scholar] [CrossRef]
Lambeta, M.; Chou, P.W.; Tian, S.; Yang, B.; Maloon, B.; Most, V.R.; Stroud, D.; Santos, R.; Byagowi, A.; Kammerer, G.; et al. DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulation. IEEE Robot. Autom. Lett. 2020, 5, 3838–3845. [Google Scholar] [CrossRef]
hyun Choi, S.; Tahara, K. Dexterous object manipulation by a multi-fingered robotic hand with visual-tactile fingertip sensors. Robomech. J. 2020, 7. [Google Scholar] [CrossRef]
Kumagai, K.; Shimonomura, K. Event-based Tactile Image Sensor for Detecting Spatio-Temporal Fast Phenomena in Contacts. In Proceedings of the 2019 IEEE World Haptics Conference, WHC 2019, Tokyo, Japan, 9–12 July 2019; Institute of Electrical and Electronics Engineers Inc.: Interlaken, Switzerland, 2019; pp. 343–348. [Google Scholar] [CrossRef]
Muthusamy, R.; Huang, X.; Zweiri, Y.; Seneviratne, L.; Gan, D.; Muthusamy, R. Neuromorphic Event-Based Slip Detection and Suppression in Robotic Grasping and Manipulation. IEEE Access 2020, 8, 153364–153384. [Google Scholar] [CrossRef]
Rigi, A.; Baghaei Naeini, F.; Makris, D.; Zweiri, Y. A Novel Event-Based Incipient Slip Detection Using Dynamic Active-Pixel Vision Sensor (DAVIS). Sensors 2018, 18, 333. [Google Scholar] [CrossRef]
Sun, H.; Kuchenbecker, K.J.; Martius, G. A soft thumb-sized vision-based sensor with accurate all-round force perception. Nat. Mach. Intell. 2022, 4, 135–145. [Google Scholar] [CrossRef]
Baghaei Naeini, F.; Alali, A.M.; Al-Husari, R.; Rigi, A.; Al-Sharman, M.K.; Makris, D.; Zweiri, Y. A Novel Dynamic-Vision-Based Approach for Tactile Sensing Applications. IEEE Trans. Instrum. Meas. 2020, 69. [Google Scholar] [CrossRef]
Naeini, F.B.; Makris, D.; Gan, D.; Zweiri, Y. Dynamic-vision-based force measurements using convolutional recurrent neural networks. Sensors 2020, 20, 4469. [Google Scholar] [CrossRef] [PubMed]
Lepora, N.F.; Lloyd, J. Optimal Deep Learning for Robot Touch: Training Accurate Pose Models of 3D Surfaces and Edges. IEEE Robot. Autom. Mag. 2020, 27, 66–77. [Google Scholar] [CrossRef]
Ward-Cherrier, B.; Pestell, N.; Lepora, N.F. NeuroTac: A Neuromorphic Optical Tactile Sensor applied to Texture Recognition. In Proceedings of the 2020 IEEE International Conference on Robotics and Automation, Paris, France, 31 May–31 August 2020; pp. 2654–2660. [Google Scholar] [CrossRef]
Fang, B.; Sun, F.; Yang, C.; Xue, H.; Chen, W.; Zhang, C.; Guo, D.; Liu, H. A dual-modal vision-based tactile sensor for robotic hand grasping. In Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, QLD, Australia, 21–25 May 2018; pp. 4740–4745. [Google Scholar]
Hu, C.B.; Zhang, C.; Xie, X.; He, J.Y.; Xie, W.A. Real-time Marker Recognition Using Vision-based Tactile Sensor. In Proceedings of the 2018 14th IEEE International Conference on Solid-State and Integrated Circuit Technology (ICSICT), Qingdao, China, 1 October–3 November 2018; pp. 1–3. [Google Scholar]
Kamiyama, K.; Vlack, K.; Mizota, T.; Kajimoto, H.; Kawakami, K.; Tachi, S. Vision-based sensor for real-time measuring of surface traction fields. IEEE Comput. Graph. Appl. 2005, 25, 68–75. [Google Scholar] [CrossRef]
Yang, Y.; Wang, X.; Zhou, Z.; Zeng, J.; Liu, H. An Enhanced FingerVision for Contact Spatial Surface Sensing. IEEE Sens. J. 2021, 21, 16492–16502. [Google Scholar] [CrossRef]
Sui, R.; Zhang, L.; Li, T.; Jiang, Y. Incipient Slip Detection Method With Vision-Based Tactile Sensor Based on Distribution Force and Deformation. IEEE Sens. J. 2021, 21, 25973–25985. [Google Scholar] [CrossRef]
Gomes, D.F.; Paoletti, P.; Luo, S. Generation of gelsight tactile images for sim2real learning. IEEE Robot. Autom. Lett. 2021, 6, 4177–4184. [Google Scholar] [CrossRef]
Sferrazza, C.; D’Andrea, R. Sim-to-real for high-resolution optical tactile sensing: From images to 3D contact force distributions. arXiv 2020, arXiv:2012.11295. [Google Scholar]
Wang, S.; Lambeta, M.; Chou, P.W.; Calandra, R. TACTO: A Fast, Flexible, and Open-Source Simulator for High-Resolution Vision-Based Tactile Sensors. IEEE Robot. Autom. Lett. 2022, 7, 3930–3937. [Google Scholar] [CrossRef]
Hassanin, H.; Jiang, K. Net shape manufacturing of ceramic micro parts with tailored graded layers. J. Micromech. Microeng. 2013, 24, 015018. [Google Scholar] [CrossRef]
Hassanin, H.; Jiang, K. Fabrication and characterization of stabilised zirconia micro parts via slip casting and soft moulding. Scr. Mater. 2013, 69, 433–436. [Google Scholar] [CrossRef]
Hassanin, H.; Jiang, K. Multiple replication of thick PDMS micropatterns using surfactants as release agents. Microelectron. Eng. 2011, 88, 3275–3277. [Google Scholar] [CrossRef]
Zhu, Y.; Chen, X.; Chu, K.; Wang, X.; Hu, Z.; Su, H. Carbon Black/PDMS Based Flexible Capacitive Tactile Sensor for Multi-Directional Force Sensing. Sensors 2022, 22, 628. [Google Scholar] [CrossRef] [PubMed]
Wang, L.; Peng, H.; Wang, X.; Chen, X.; Yang, C.; Yang, B.; Liu, J. PDMS/MWCNT-based tactile sensor array with coplanar electrodes for crosstalk suppression. Microsyst. Nanoeng. 2016, 2, 16065. [Google Scholar] [CrossRef] [PubMed]
Sun, X.; Sun, J.; Li, T.; Zheng, S.; Wang, C.; Tan, W.; Zhang, J.; Liu, C.; Ma, T.; Qi, Z.; et al. Flexible tactile electronic skin sensor with 3D force detection based on porous CNTs/PDMS nanocomposites. Nano-Micro Lett. 2019, 11, 57. [Google Scholar] [CrossRef]
Sferrazza, C.; D’Andrea, R. Design, Motivation and Evaluation of a Full-Resolution Optical Tactile Sensor. Sensors 2019, 19, 928. [Google Scholar] [CrossRef] [PubMed]
Li, M.; Zhang, L.; Li, T.; Jiang, Y. Continuous Marker Patterns for Representing Contact Information in Vision-Based Tactile Sensor: Principle, Algorithm, and Verification. IEEE Trans. Instrum. Meas. 2022, 71, 1–12. [Google Scholar] [CrossRef]
Ahmed, A.M.; Duran, O.; Zweiri, Y.; Smith, M. Hybrid spectral unmixing: Using artificial neural networks for linear/non-linear switching. Remote Sens. 2017, 9, 775. [Google Scholar] [CrossRef]
Ioffe, S.; Szegedy, C. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. In Proceedings of the 32nd International Conference on International Conference on Machine Learning, Lille, France, 6–11 July 2015; pp. 448–456. [Google Scholar]
Andrew, L.; Maas, A.Y.; Hannun, A.Y.N. Rectifier Nonlinearities Improve Neural Network Acoustic Models. In Proceedings of the 30th International Conference on Machine Learning, Atlanta, GA, USA, 16–21 June 2013. [Google Scholar]
Srivastava, N.; Hinton, G.; Krizhevsky, A.; Sutskever, I.; Salakhutdinov, R. Dropout: A Simple Way to Prevent Neural Networks from Overfitting. J. Mach. Learn. Res. 2014, 15, 1929–1958. [Google Scholar]
Kingma, D.P.; Ba, J. Adam: A method for stochastic optimization. arXiv 2014, arXiv:1412.6980. [Google Scholar]
Ge, G.; Wang, Q.; Zhang, Y.Z.; Alshareef, H.N.; Dong, X. 3D printing of hydrogels for stretchable ionotronic devices. Adv. Funct. Mater. 2021, 31, 2107437. [Google Scholar] [CrossRef]

Permalink -

https://repository.canterbury.ac.uk/item/93345/elastomer-based-visuotactile-sensor-for-normality-of-robotic-manufacturing-systems

Download files


Publisher's version
polymers-14-05097.pdf
File access level: Open

  • 16
    total views
  • 3
    total downloads
  • 1
    views this month
  • 1
    downloads this month

Export as

Related outputs

Techno-economic feasibility of retired electric-vehicle batteries repurpose/reuse in second-life applications: A systematic review
Hassanin, H., Al-Alawi, M. and Cugley, J. 2022. Techno-economic feasibility of retired electric-vehicle batteries repurpose/reuse in second-life applications: A systematic review. Energy and Climate Change. 3 (100086). https://doi.org/10.1016/j.egycc.2022.100086
Planning, operation, and design of market-based virtual power plant considering uncertainty
Hassanin, H., Ullah, Z., Arshad, Cugley, J. and Al-Alawi, M. 2022. Planning, operation, and design of market-based virtual power plant considering uncertainty. Energies. 19 (15), p. 7290. https://doi.org/10.3390/en15197290
The epistemic insight digest: Issue : Autumn 2022
Gordon, A., Shalet, D., Simpson, S., Hassanin, H., Lawson, F., Lawson, M., Litchfield, A., Thomas, C., Canetta, E., Manley, K. and Choong, C. Shalet, D. (ed.) 2022. The epistemic insight digest: Issue : Autumn 2022. Canterbury Canterbury Christ Church University.
Modeling, optimization, and analysis of a virtual power plant demand response mechanism for the internal electricity market considering the uncertainty of renewable energy sources
Ullah, Z., Arshad and Hassanin, H. 2022. Modeling, optimization, and analysis of a virtual power plant demand response mechanism for the internal electricity market considering the uncertainty of renewable energy sources. Energies. 15 (14), p. 5296. https://doi.org/doi.org/10.3390/en15145296
Interdisciplinary engineering education - essential for the 21st century
Gordon, A., Simpson, S. and Hassanin, H. 2022. Interdisciplinary engineering education - essential for the 21st century.
Fabrication and optimisation of Ti-6Al-4V lattice-structured total shoulder implants using laser additive manufacturing
Hassanin, H., Bittredge, O., El-Sayed, M., Eldessouky, H., A. Alsaleh, N., Alrasheedi, N., Essa, K. and Ahmadein, M. 2022. Fabrication and optimisation of Ti-6Al-4V lattice-structured total shoulder implants using laser additive manufacturing. Materials. 15 (9), p. 3095. https://doi.org/10.3390/ma15093095
Multipoint forming using hole-type rubber punch
Hassanin, H., Tolipov, A., El-Sayed, M., Eldessouky, H., A. Alsaleh, N., Alfozan, A., Essa, K. and Ahmadein, M. 2022. Multipoint forming using hole-type rubber punch. Metals. 12 (3), p. 491. https://doi.org/10.3390/met12030491
Influence of bifilm defects generated during mould filling on the tensile properties of Al–Si–Mg cast alloys
El-Sayed, M., Essa, K. and Hassanin, H. 2022. Influence of bifilm defects generated during mould filling on the tensile properties of Al–Si–Mg cast alloys. Metals. https://doi.org/10.3390/met12010160
Effect of runner thickness and hydrogen content on the mechanical properties of A356 alloy castings
El-Sayed, M., Essa, K. and Hassanin, H. 2021. Effect of runner thickness and hydrogen content on the mechanical properties of A356 alloy castings . International Journal of Metalcasting. https://doi.org/10.1007/s40962-021-00753-x
Parts design and process optimization
Hassanin, Hany, Bidare, Prveen, Zweiri, Yahya and Essa, Khamis 2021. Parts design and process optimization. in: Salunkhe, S., Hussein, H. and Davim, J. (ed.) Applications of Artificial Intelligence in Additive Manufacturing USA IGI Global. pp. 25-49
Multi stages toolpath optimisation of single point incremental forming process
Hassanin, H., Yan, Z, El-Sayed, M., Eldessouky, H., Djuansjah, J., Alsaleh, N., Essa, K. and Ahmadein, M. 2021. Multi stages toolpath optimisation of single point incremental forming process. Materials. 14 (22), p. 6794. https://doi.org/10.3390/ma14226794
Micro-additive manufacturing technologies of three-dimensional MEMS
Hassanin, H., Sheikholeslami, G., Pooya, S. and Ishaq, R. 2021. Micro-additive manufacturing technologies of three-dimensional MEMS . Advanced Engineering Materials. https://doi.org/10.1002/adem.202100422
Machine learning applied to the design and inspection of reinforced concrete bridges: Resilient methods and emerging applications
Fan , W., Chen, Y., Li, J., Sun, Y., Feng, F., Hassanin, H. and Sareh, P. 2021. Machine learning applied to the design and inspection of reinforced concrete bridges: Resilient methods and emerging applications. Structures. 33, pp. 3954-3963. https://doi.org/10.1016/j.istruc.2021.06.110
Porosity, cracks, and mechanical properties of additively manufactured tooling alloys: A review
Bidare, P., Jiménez, A., Hassanin, H. and Essa, K. 2021. Porosity, cracks, and mechanical properties of additively manufactured tooling alloys: A review. Advances in Manufacturing. https://doi.org/10.1007/s40436-021-00365-y
Laser powder bed fusion of Ti-6Al-2Sn-4Zr-6Mo alloy and properties prediction using deep learning approaches
Hassanin, H., Zweiri, Y., Finet, L., Essa, K., Qiu, C. and Attallah, M. 2021. Laser powder bed fusion of Ti-6Al-2Sn-4Zr-6Mo alloy and properties prediction using deep learning approaches. Materials. 14 (8), p. 2056. https://doi.org/10.3390/ma14082056
3DP printing of oral solid formulations: a systematic review
Brambilla, C., Okafor-Muo, O., Hassanin, H. and ElShaer, A. 2021. 3DP printing of oral solid formulations: a systematic review. Pharmaceutics. 13 (3), p. 358. https://doi.org/10.3390/pharmaceutics13030358
Powder-based laser hybrid additive manufacturing of metals: A review
Hassanin, H. 2021. Powder-based laser hybrid additive manufacturing of metals: A review. The International Journal of Advanced Manufacturing Technology.
Micro-fabrication of ceramics: additive manufacturing and conventional technologies
Hassanin, H., Essa, K., Elshaer, A., Imbaby, M. and El-Sayed, T. E. 2021. Micro-fabrication of ceramics: additive manufacturing and conventional technologies. Journal of Advanced Ceramics. 10, pp. 1-27. https://doi.org/10.1007/s40145-020-0422-5
4D Printing of origami structures for minimally invasive surgeries using functional scaffold
Langford, T, Mohammed, A., Essa, K., Elshaer, A. and Hassanin, H. 2020. 4D Printing of origami structures for minimally invasive surgeries using functional scaffold. Applied Sciences. 11 (1), p. 332. https://doi.org/10.3390/app11010332
Reconfigurable multipoint forming using waffle-type elastic cushion and variable loading profile
Hassanin, H., Mohammed, M., Abdel-Wahab, A. and Essa, K 2020. Reconfigurable multipoint forming using waffle-type elastic cushion and variable loading profile. Materials.
3D printing of solid oral dosage forms: numerous challenges with unique opportunities
Hassanin, H. 2020. 3D printing of solid oral dosage forms: numerous challenges with unique opportunities. Journal of Pharmaceutical Sciences. https://doi.org/10.1016/j.xphs.2020.08.029
Design optimisation of additively manufactured titanium lattice structures for biomedical implants
El-Sayed, M.A., Essa, K., Ghazy, M. and Hassanin, H. 2020. Design optimisation of additively manufactured titanium lattice structures for biomedical implants. The International Journal of Advanced Manufacturing Technology. https://doi.org/10.1007/s00170-020-05982-8
4D Printing of NiTi auxetic structure with improved ballistic performance
Hassanin, H., Abena, A., Elsayed, M.A. and Essa, K. 2020. 4D Printing of NiTi auxetic structure with improved ballistic performance. Micromachines. 11 (8), p. 745. https://doi.org/doi.org/10.3390/mi11080745