Railway safety depends on regular track inspections to detect damage to infrastructure components, such as railway sleepers. Traditionally, inspectors walk along the tracks to assess damage, but this method is both time-consuming and dangerous, especially with high-speed trains in operation. With growing railway traffic and increasing safety regulations, there was a need for an automated, AI-based system capable of detecting cracks, chipping, and other sleeper defects in a short time.
Goldschmidt, a leader in railway technology, aimed to develop an inspection system for track monitoring trains that could assess railway sleepers’ condition efficiently. The system needed to capture high-resolution images of sleepers and analyze them to detect and classify defects. Accuracy was a critical requirement, as false positives could lead to unnecessary maintenance costs, while false negatives could result in safety risks. The challenge was to build a highly precise, AI-driven inspection system that could reliably identify sleeper damage under various conditions.

To meet these requirements, we developed a machine learning-based system that automates railway sleeper damage detection. The solution is built around a deep learning model trained on thousands of labeled images, enabling it to recognize defects with high accuracy. A linear video camera mounted on an inspection train captures images of railway sleepers, which are then processed by the AI system.
The image analysis process begins with preprocessing steps that enhance sleeper features and reduce noise, ensuring optimal conditions for defect detection. A convolutional neural network extracts critical features from the images, allowing for precise identification of cracks, chipping, and other types of wear. Boosted trees models further classify and evaluate the detected defects, determining their severity and categorizing them based on predefined criteria. The system is optimized to minimize false positives while maintaining high enough detection rates for true defects.
A key component used during building of the solution is the Explainable AI (XAI) approach, which was incorporated to provide insights into how the AI model makes decisions. This allows ML engineers to validate and refine the system based on validation results. To further improve detection accuracy, a software tool was developed to browse and analyze images, enabling ML-engineers to review both correct and incorrect classifications and fine-tune the model accordingly.

The AI-based inspection system successfully automated the detection of railway sleeper damage, significantly reducing the need for manual inspections on tracks. The system was well received by the client and has been fully implemented by the end user. Its high accuracy ensures that railway maintenance teams receive reliable information about infrastructure conditions, improving both safety and operational efficiency.
Goldschmidt showcased the solution at the Machine Learning Week conference in Berlin, where industry professionals had the opportunity to explore the technology. The system’s ability to identify defects with precision has positioned it as a groundbreaking innovation in railway inspection, demonstrating the power of AI in automating critical safety tasks.
By integrating AI-driven image analysis into railway maintenance, this project has potential to revolutionize railway sleeper inspections, making them safer, faster, and more efficient. The system minimizes human risk while providing highly accurate defect detection, demonstrating the transformative potential of machine learning in infrastructure monitoring.

The project began with preparing a large dataset of images, ensuring that the AI model had access to diverse examples of sleepers with and without defects. The neural network was designed to extract relevant features, supporting accurate classification of damage. The team conducted an extensive XAI analysis to interpret the model’s decision-making process, ensuring that its classifications were based on meaningful visual patterns rather than irrelevant artifacts.
Several boosted trees models were developed to classify different types of sleeper damage, and they were optimized through hyperparameter tuning and custom multidimensional cut-off adjustments. The system was continuously improved through an iterative process that involved reviewing misclassified images and refining the model’s ability to differentiate between damage types. Validation was performed using diverse datasets to confirm the model’s robustness across different environmental conditions. Finally, the system was prepared for seamless integration into Goldschmidt’s railway inspection workflow.

The core of the solution was a convolutional neural network designed to extract features from railway sleeper images. Feature extraction techniques were applied to enhance relevant details while reducing noise. Boosted trees models were used to classify defects, and a multi-level optimization process fine-tuned the model’s thresholds, structure, and hyperparameters. The XAI approach provided transparency into the model’s decisions, while validation methods, including manual visual inspection of misclassified images, ensured reliability.