Feeding Your Digital Twin: the Challenge for Railroads

Technology Section Feature

Nick Smith
Co-founder & CEO, Cordel

Digital Twins are a hot topic in all significant railroads today. The technology offers the promise of improved design, more efficient construction programs, safer operations and more.

Specifically for the maintenance of rail and transit networks, the idea of assembling large lakes of data about the network and then using computation and analytics to build up a detailed digital replica of every aspect of the network has enormous business appeal.

Digital twin technology has also been used to complement the implementation of Positive Train Control (PTC) deployments, and also to offer simulated validation of the network behaviours necessary to optimize Precision Railroading operations.

Outside of rail, everything from jet engines to cities and urban planners have used this modelling technique to understand the current state, model future developments’ effects, and predict the likely impact of changes over time.

Mostly this works well, but not always.

Evil Twins

As some cities have learned, digital twins can also have a less positive side – an “evil twin”.

Much like a dead actor who is brought back to life through AI and special effects to “act” in one more movie or finish a scene, a railroad’s digital twin can also be used to simulate the network’s behaviour in various future scenarios that haven’t yet happened in reality. In both cases, the authenticity of the output is directly related to the quality and comprehensiveness of the data input. If the data is not right or not sufficient, the twin will be a bit weird.

In a movie, that might just look cheesy, but on a railroad, it can be catastrophic.

Data Sources

Any survey of vendors and proponents of Digital Twin technology will uncover many references to the Internet of Things (IoT).

It seems like a match made in heaven: lots of IoT-enabled devices, cameras, LiDAR and sensors all over the rail network streaming endless rich data into your data lake for the digital twin to devour. And with 5G networks, this stream can become a fire hose of data. It’s all wonderful.

But IoT is a communication vector, not a data type. Other communications vectors exist, too, including wired transmission, periodic uploads, and others. However, what’s more critical than the communication is the nature of the data itself. Is it timely and reliable? Is it redundant (that is, are there multiple sources or angles or dimensions available of the same targets)? And is it sufficient to make trusted inferences and deductions about the context of the data?

Additionally, is the data that sits in the digital twin trustworthy, i.e. can we be sure the quality of the information is trusted to be accurate and devoid of errors.

For railroads, getting this right can be the difference between building a “good” twin that can add efficiency and avoid disasters and an “evil” twin with the potential to cause disasters.

The Crossing Problem

Let’s look at a specific example: a railroad crossing.

Typical approaches to data monitoring of a crossing include signal sensors to check the function of automatic signals. For some busy crossings, CCTV cameras and stationary LiDAR systems are also used to detect congestion even in limited visibility.

This combination of visual data and activity data can give an excellent overall understanding of the crossing, its environs and the signals’ operational condition.

But now consider these important maintenance-of-way questions:

• Has there been any change in the relative surface levels on the crossing over time?
• Is the signage sufficiently visible at night time?
• Are there any periodic obstructions caused by vegetation growth or movements?
• Is the rail track unimpeded by debris?
• Are there any recent variations in the road-track alignment?

The signal sensors won’t help you with these questions. In principle, the CCTV vision might help, but there are some severe limitations.

The most significant problem with CCTV, and all photos and video, is that they are imprecise. Firstly, CCTV typically does not detect objects well at night as it uses Near Infrared. Some machine interpretation is possible, but reliably interpreting them to a level of detail necessary for rail involves human labour. Given the volume of data involved, this is just infeasible.

So what’s left? What can a railroad do to feed digestible data about that rail crossing to its digital twin?

LiDAR is not enough, but it can help.

One possible alternative to photos and video is LiDAR, both stationary and mobile!

Photos and video are imprecise because they are 2D data of a 3D world. This means that:

• Perspective must be estimated
• Objects can blend in with the background
• Distortion is not always obvious
• Low light can obscure edges.

By comparison, LiDAR data is 3D data. LiDAR collects millions of laser returns. Every return forms a “point” in the cloud, and the exact distance is known for every point in the point cloud. The result is rich data output that has perfect perspective, no distortions, is not affected by blending-in in low light, and has much greater precision.

Thinking back to the rail crossing example, this means that LiDAR data can be used to accurately determine the exact relative elevation of every tiny portion of pavement, the night-time reflectivity of the crossing signage, any unexpected variations in rail or roadway positioning, and the identity of any debris or blocking objects.

Figure 1. A LiDAR image of a rail crossing. Every single “point” on the image contains mathematical values on the distance and position of the return and other related attributes, avoiding the imprecision of video alone and enabling direct image interpretation by AI.

This is a significant advance compared to that CCTV, but by itself, LiDAR still doesn’t solve the problem of interpretation. The data is now richer and better, but your digital twin may still not know how to tell the difference between a stray leaf and a missing bolt on a crossing barrier.

This is where an AI engine that has been trained on railroad data comes in.

AI for Rail LiDAR

When trained well, AI can do amazing things, and especially in the area of data interpretation. There are smartphone apps in narrow areas of specialization to identify a breed of dog from a photo, for example. But the railroad problem space, or even just the specific example of a rail crossing, is a multi-faceted wide-domain AI problem of an entirely different level of complexity.

For wide-domain problems like this, iterative machine learning is used based on many different use cases and scenarios and vast volumes of data. Collecting and learning to identify all the other elements, situations and unexpected influences in a massive data set is how AI for Rail LiDAR works. LiDAR scans of hundreds of thousands of rail miles for many different networks and operators in all environment types are the minimum building block for training reliable AI for Rail LiDAR.

As for how a digital twin benefits from AI, what is can be achieved is the amplification of effort of already trained engineers and inspectors; highly trained engineers use the ability to quickly and efficiently make judgment calls with all of the available information streams presented to them. For example, they might look at a problem, then listen to a passing train. They can then use their experience to compare that situation to something they have previously experienced unconsciously.

Figure 2. An image overlay for a human viewer automatically generated by Cordel AI, which shows a simplified view of the automatic element interpretation of LiDAR data, made possible by prior training on hundreds of thousands of miles of rail data sets.

And, of course, this process does not end. Every new scan and data collection is another learning iteration for the AI, providing a virtuous circle of continuous learning improvement.

After all this, suddenly you have not just comprehensive data of that rail crossing but interpreted data with positively identified elements that can provide usable business insights. This is precisely the input that digital twins need.

Feeding your Digital Twin

Digital Twin technology offers enormous benefits to railroads, especially for maintenance-of-way, but even the best digital twin is only as good as the digestible data it is fed.

Many railroads are now using vehicle-mounted rail-specific LiDAR, either on in-service trains or on inspection vehicles, to collect run-over-run LiDAR data of their networks. Coupling this with an industry-grade AI for Rail can deliver the richest and most accurate data source to a digital twin, enabling better insights and better predictions, all contributing to more efficient and safer railroads.

About the Author

;