A novel triplet loss architecture with visual explanation for detecting the unwanted rotation of bolts in safety-critical environments

Bolton, Tom; Bass, Julian; Gaber, Tarek; Mansouri, Taha; Adam, Peter; Ghavimi, Hossein

Abstract

In the commonly used method of bolting to secure parts of equipment and structure, the bolts must be tightened to an adequate preload force. Failure to do so could affect the integrity of the structure, as well as the efficient running of the site and, crucially, employees’ safety. In this project, we consider the use of artificial intelligence (AI) techniques to analyse maintenance videos and identify the unwanted loosening of bolts over time in order that they might be used as additional tools in a continuous maintenance plan. We found that accuracy levels of up to 97% could be achieved in identifying bolt rotation with our proposed machine learning-based triplet loss architecture. The use of gradient-weighted class activation mapping (Grad-CAM) visualisations to identify areas of the image where change had occurred enabled us to test how robust our model was to noise in the data. This explanation may assist users in safety-critical environments guiding them to the problem, and helping mitigate the black-box nature of machine learning algorithms. Whilst the accuracy of the models varies depending on the rotational angle of the bolt, we clearly demonstrate that triplet loss is a good basis for performing change detection in industrial settings. Furthermore, Grad-CAM has shown to be a useful technique to help a user understand the decisions made by the network and allow them to see where unwanted rotation has occurred.

• Journal Article Type: Article
• Acceptance Date: May 5, 2025
• Online Publication Date: Jun 3, 2025
• Publication Date: 2025-09
• Deposit Date: Jun 3, 2025
• Publicly Available Date: Jun 6, 2025
• Journal: Engineering Applications of Artificial Intelligence
• Print ISSN: 0952-1976
• Publisher: Elsevier
• Peer Reviewed: Peer Reviewed
• Volume: 156
• Pages: 111097
• DOI: https://doi.org/10.1016/j.engappai.2025.111097