Elastomer-based visuotactile sensor for normality of robotic manufacturing systems

Modern aircrafts require the assembly of thousands of components with high accuracy and reliability. The normality of drilled holes is a critical geometrical tolerance that is required to be achieved in order to realize an efficient assembly process. Failure to achieve the required tolerance leads to structures prone to fatigue problems and assembly errors. Elastomer-based tactile sensors have been used to support robots in acquiring useful physical interaction information with the environments. However, current tactile sensors have not yet been developed to support robotic machining in achieving the tight tolerances of aerospace structures. In this paper, a novel elastomer-based tactile sensor was developed for cobot machining. Three commercial silicon-based elastomer materials were characterised using mechanical testing in order to select a material with the best deformability. A Finite element model was developed to simulate the deformation of the tactile sensor upon interacting with surfaces with different normalities. Additive manufacturing was employed to fabricate the tactile sensor mould, which was chemically etched to improve the surface quality. The tactile sensor was obtained by directly casting and curing the optimum elastomer material onto the additively manufactured mould. A machine learning approach was used to train the simulated and experimental data obtained from the sensor. The capability of the developed vision tactile sensor was evaluated using real-world experiments with various inclination angles, and achieved a mean perpendicularity tolerance of 0.34°. The developed sensor opens a new perspective on low-cost precision cobot machining.

Phanden, R.K.; Sharma, P.; Dubey, A. A review on simulation in digital twin for aerospace, manufacturing and robotics. Mater. Today Proc. 2021, 38, 174–178. [Google Scholar] [CrossRef]
Onstein, I.F.; Semeniuta, O.; Bjerkeng, M. Deburring using robot manipulators: A review. In Proceedings of the 2020 3rd International Symposium on Small-scale Intelligent Manufacturing Systems (SIMS), Gjøvik, Norway, 10–12 June 2020; pp. 1–7. [Google Scholar]
Alvares, A.J.; Rodriguez, E.; Jaimes, C.I.R.; Toquica, J.S.; Ferreira, J.C. STEP-NC Architectures for Industrial Robotic Machining: Review, Implementation and Validation. IEEE Access 2020, 8, 152592–152610. [Google Scholar] [CrossRef]
Luo, H.; Zhang, D.; Luo, M. Tool Wear and Remaining Useful Life Estimation of Difficult-to-machine Aerospace Alloys: A Review. China Mech. Eng. 2021, 32, 2647. [Google Scholar]
Sun, D.; Lemoine, P.; Keys, D.; Doyle, P.; Malinov, S.; Zhao, Q.; Qin, X.; Jin, Y. Hole-making processes and their impacts on the microstructure and fatigue response of aircraft alloys. Int. J. Adv. Manuf. Technol. 2018, 94, 1719–1726. [Google Scholar] [CrossRef]
Akula, S.; Nayak, S.N.; Bolar, G.; Managuli, V. Comparison of conventional drilling and helical milling for hole making in Ti6Al4V titanium alloy under sustainable dry condition. Manuf. Rev. 2021, 8, 12. [Google Scholar] [CrossRef]
Yuan, C.G.; Pramanik, A.; Basak, A.; Prakash, C.; Shankar, S. Drilling of titanium alloy (Ti6Al4V)—A review. Mach. Sci. Technol. 2021, 25, 637–702. [Google Scholar] [CrossRef]
Abdelhafeez, A.; Soo, S.; Aspinwall, D.; Dowson, A.; Arnold, D. Burr formation and hole quality when drilling titanium and aluminium alloys. Procedia Cirp 2015, 37, 230–235. [Google Scholar] [CrossRef]
Aamir, M.; Giasin, K.; Tolouei-Rad, M.; Vafadar, A. A review: Drilling performance and hole quality of aluminium alloys for aerospace applications. J. Mater. Res. Technol. 2020, 9, 12484–12500. [Google Scholar] [CrossRef]
da Silva Santos, K.R.; de Carvalho, G.M.; Tricarico, R.T.; Ferreira, L.F.L.R.; Villani, E.; Sutério, R. Evaluation of perpendicularity methods for a robotic end effector from aircraft industry. In Proceedings of the 2018 13th IEEE International Conference on Industry Applications (INDUSCON), Sao Paulo, Brazil, 12–14 November 2018; pp. 1373–1380. [Google Scholar]
Yu, L.; Zhang, Y.; Bi, Q.; Wang, Y. Research on surface normal measurement and adjustment in aircraft assembly. Precis. Eng. 2017, 50, 482–493. [Google Scholar] [CrossRef]
Zhang, Y.; Bi, Q.; Yu, L.; Wang, Y. Online adaptive measurement and adjustment for flexible part during high precision drilling process. Int. J. Adv. Manuf. Technol. 2017, 89, 3579–3599. [Google Scholar] [CrossRef]
Hassanin, H.; Abena, A.; Elsayed, M.A.; Essa, K. 4D printing of NiTi auxetic structure with improved ballistic performance. Micromachines 2020, 11, 745. [Google Scholar] [CrossRef] [PubMed]
Brambilla, C.R.; Okafor-Muo, O.L.; Hassanin, H.; ElShaer, A. 3DP printing of oral solid formulations: A systematic review. Pharmaceutics 2021, 13, 358. [Google Scholar] [CrossRef]
Khosravani, M.R.; Reinicke, T. On the environmental impacts of 3D printing technology. Appl. Mater. Today 2020, 20, 100689. [Google Scholar] [CrossRef]
Shah, U.H.; Muthusamy, R.; Gan, D.; Zweiri, Y.; Seneviratne, L. On the Design and Development of Vision-based Tactile Sensors. J. Intell. Robot. Syst. 2021, 102, 82. [Google Scholar] [CrossRef]
Lambeta, M.; Chou, P.W.; Tian, S.; Yang, B.; Maloon, B.; Most, V.R.; Stroud, D.; Santos, R.; Byagowi, A.; Kammerer, G.; et al. DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulation. IEEE Robot. Autom. Lett. 2020, 5, 3838–3845. [Google Scholar] [CrossRef]
hyun Choi, S.; Tahara, K. Dexterous object manipulation by a multi-fingered robotic hand with visual-tactile fingertip sensors. Robomech. J. 2020, 7. [Google Scholar] [CrossRef]
Kumagai, K.; Shimonomura, K. Event-based Tactile Image Sensor for Detecting Spatio-Temporal Fast Phenomena in Contacts. In Proceedings of the 2019 IEEE World Haptics Conference, WHC 2019, Tokyo, Japan, 9–12 July 2019; Institute of Electrical and Electronics Engineers Inc.: Interlaken, Switzerland, 2019; pp. 343–348. [Google Scholar] [CrossRef]
Muthusamy, R.; Huang, X.; Zweiri, Y.; Seneviratne, L.; Gan, D.; Muthusamy, R. Neuromorphic Event-Based Slip Detection and Suppression in Robotic Grasping and Manipulation. IEEE Access 2020, 8, 153364–153384. [Google Scholar] [CrossRef]
Rigi, A.; Baghaei Naeini, F.; Makris, D.; Zweiri, Y. A Novel Event-Based Incipient Slip Detection Using Dynamic Active-Pixel Vision Sensor (DAVIS). Sensors 2018, 18, 333. [Google Scholar] [CrossRef]
Sun, H.; Kuchenbecker, K.J.; Martius, G. A soft thumb-sized vision-based sensor with accurate all-round force perception. Nat. Mach. Intell. 2022, 4, 135–145. [Google Scholar] [CrossRef]
Baghaei Naeini, F.; Alali, A.M.; Al-Husari, R.; Rigi, A.; Al-Sharman, M.K.; Makris, D.; Zweiri, Y. A Novel Dynamic-Vision-Based Approach for Tactile Sensing Applications. IEEE Trans. Instrum. Meas. 2020, 69. [Google Scholar] [CrossRef]
Naeini, F.B.; Makris, D.; Gan, D.; Zweiri, Y. Dynamic-vision-based force measurements using convolutional recurrent neural networks. Sensors 2020, 20, 4469. [Google Scholar] [CrossRef] [PubMed]
Lepora, N.F.; Lloyd, J. Optimal Deep Learning for Robot Touch: Training Accurate Pose Models of 3D Surfaces and Edges. IEEE Robot. Autom. Mag. 2020, 27, 66–77. [Google Scholar] [CrossRef]
Ward-Cherrier, B.; Pestell, N.; Lepora, N.F. NeuroTac: A Neuromorphic Optical Tactile Sensor applied to Texture Recognition. In Proceedings of the 2020 IEEE International Conference on Robotics and Automation, Paris, France, 31 May–31 August 2020; pp. 2654–2660. [Google Scholar] [CrossRef]
Fang, B.; Sun, F.; Yang, C.; Xue, H.; Chen, W.; Zhang, C.; Guo, D.; Liu, H. A dual-modal vision-based tactile sensor for robotic hand grasping. In Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, QLD, Australia, 21–25 May 2018; pp. 4740–4745. [Google Scholar]
Hu, C.B.; Zhang, C.; Xie, X.; He, J.Y.; Xie, W.A. Real-time Marker Recognition Using Vision-based Tactile Sensor. In Proceedings of the 2018 14th IEEE International Conference on Solid-State and Integrated Circuit Technology (ICSICT), Qingdao, China, 1 October–3 November 2018; pp. 1–3. [Google Scholar]
Kamiyama, K.; Vlack, K.; Mizota, T.; Kajimoto, H.; Kawakami, K.; Tachi, S. Vision-based sensor for real-time measuring of surface traction fields. IEEE Comput. Graph. Appl. 2005, 25, 68–75. [Google Scholar] [CrossRef]
Yang, Y.; Wang, X.; Zhou, Z.; Zeng, J.; Liu, H. An Enhanced FingerVision for Contact Spatial Surface Sensing. IEEE Sens. J. 2021, 21, 16492–16502. [Google Scholar] [CrossRef]
Sui, R.; Zhang, L.; Li, T.; Jiang, Y. Incipient Slip Detection Method With Vision-Based Tactile Sensor Based on Distribution Force and Deformation. IEEE Sens. J. 2021, 21, 25973–25985. [Google Scholar] [CrossRef]
Gomes, D.F.; Paoletti, P.; Luo, S. Generation of gelsight tactile images for sim2real learning. IEEE Robot. Autom. Lett. 2021, 6, 4177–4184. [Google Scholar] [CrossRef]
Sferrazza, C.; D’Andrea, R. Sim-to-real for high-resolution optical tactile sensing: From images to 3D contact force distributions. arXiv 2020, arXiv:2012.11295. [Google Scholar]
Wang, S.; Lambeta, M.; Chou, P.W.; Calandra, R. TACTO: A Fast, Flexible, and Open-Source Simulator for High-Resolution Vision-Based Tactile Sensors. IEEE Robot. Autom. Lett. 2022, 7, 3930–3937. [Google Scholar] [CrossRef]
Hassanin, H.; Jiang, K. Net shape manufacturing of ceramic micro parts with tailored graded layers. J. Micromech. Microeng. 2013, 24, 015018. [Google Scholar] [CrossRef]
Hassanin, H.; Jiang, K. Fabrication and characterization of stabilised zirconia micro parts via slip casting and soft moulding. Scr. Mater. 2013, 69, 433–436. [Google Scholar] [CrossRef]
Hassanin, H.; Jiang, K. Multiple replication of thick PDMS micropatterns using surfactants as release agents. Microelectron. Eng. 2011, 88, 3275–3277. [Google Scholar] [CrossRef]
Zhu, Y.; Chen, X.; Chu, K.; Wang, X.; Hu, Z.; Su, H. Carbon Black/PDMS Based Flexible Capacitive Tactile Sensor for Multi-Directional Force Sensing. Sensors 2022, 22, 628. [Google Scholar] [CrossRef] [PubMed]
Wang, L.; Peng, H.; Wang, X.; Chen, X.; Yang, C.; Yang, B.; Liu, J. PDMS/MWCNT-based tactile sensor array with coplanar electrodes for crosstalk suppression. Microsyst. Nanoeng. 2016, 2, 16065. [Google Scholar] [CrossRef] [PubMed]
Sun, X.; Sun, J.; Li, T.; Zheng, S.; Wang, C.; Tan, W.; Zhang, J.; Liu, C.; Ma, T.; Qi, Z.; et al. Flexible tactile electronic skin sensor with 3D force detection based on porous CNTs/PDMS nanocomposites. Nano-Micro Lett. 2019, 11, 57. [Google Scholar] [CrossRef]
Sferrazza, C.; D’Andrea, R. Design, Motivation and Evaluation of a Full-Resolution Optical Tactile Sensor. Sensors 2019, 19, 928. [Google Scholar] [CrossRef] [PubMed]
Li, M.; Zhang, L.; Li, T.; Jiang, Y. Continuous Marker Patterns for Representing Contact Information in Vision-Based Tactile Sensor: Principle, Algorithm, and Verification. IEEE Trans. Instrum. Meas. 2022, 71, 1–12. [Google Scholar] [CrossRef]
Ahmed, A.M.; Duran, O.; Zweiri, Y.; Smith, M. Hybrid spectral unmixing: Using artificial neural networks for linear/non-linear switching. Remote Sens. 2017, 9, 775. [Google Scholar] [CrossRef]
Ioffe, S.; Szegedy, C. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. In Proceedings of the 32nd International Conference on International Conference on Machine Learning, Lille, France, 6–11 July 2015; pp. 448–456. [Google Scholar]
Andrew, L.; Maas, A.Y.; Hannun, A.Y.N. Rectifier Nonlinearities Improve Neural Network Acoustic Models. In Proceedings of the 30th International Conference on Machine Learning, Atlanta, GA, USA, 16–21 June 2013. [Google Scholar]
Srivastava, N.; Hinton, G.; Krizhevsky, A.; Sutskever, I.; Salakhutdinov, R. Dropout: A Simple Way to Prevent Neural Networks from Overfitting. J. Mach. Learn. Res. 2014, 15, 1929–1958. [Google Scholar]
Kingma, D.P.; Ba, J. Adam: A method for stochastic optimization. arXiv 2014, arXiv:1412.6980. [Google Scholar]
Ge, G.; Wang, Q.; Zhang, Y.Z.; Alshareef, H.N.; Dong, X. 3D printing of hydrogels for stretchable ionotronic devices. Adv. Funct. Mater. 2021, 31, 2107437. [Google Scholar] [CrossRef]