By Sharon Lo, Managing Editor, DTJ & NDTAGram

Photo courtesy BNSF Railway

In May 2014 it was announced that BNSF would be one of three companies to partner with the Federal Aviation Administration (FAA) on the Pathfinder Program, a public-private partnership designed to help the FAA determine if and how to safely expand unmanned aircraft operations in the United States. BNSF was tasked with exploring the challenges of using long range drones to inspect their rail infrastructure beyond visual line-of-sight in isolated areas.

Todd Graetz has 15 years of experience with advanced technology including Unmanned Aerial Vehicles (UAV)/Unmanned Aircraft Systems (UAS), wireless and fiber optic systems, telecom operations, and transportation communications. He is a member of the FAA’s Drone Advisory Committee and its UAS Identification and Tracking Aviation Rulemaking Committee (ARC). He is also an active private pilot and UAS operator.

DTJ: Thanks for taking the time to interview today. To start off, can you tell us about BNSF’s role in the Pathfinder Program?

Mr. Graetz: BNSF’s role in the Pathfinder Program, which started in mid-2015, has been focused solely on how a commercial entity could operate an unmanned aircraft—also known as a drone—over long distances, well beyond the line of sight of the operator.

We were given that charge because out of a great number of organizations that may have an interest in that research, we bring some key elements to the table. First and foremost is our safety focus. The FAA knows we are going to be a safe operator and a safe partner. Second is our infrastructure and right of way, which creates a known, predictable flight path for the efforts.

We kicked off the program in 2015 and we’ve had multiple milestones since then. In 2015, we developed the safety case and created the structure so that we could prove to both our research partner at the FAA and the industry as a whole that this can be done in North America, in the continental United States, it can be done in lightly populated areas, and it can be done in a safe manner. And we did so working with another organization, Insitu, a subsidiary of Boeing. We worked with their ScanEagle system and our resources, and we deployed the first long-range beyond line of sight civil operation ever in the lower 48 states.

The next step after proving we could fly was to start talking about rail specific systems. That includes rail specific aircraft, sensors, software and systems—and that’s where the program evolved in 2016 and into 2017. We started creating and/or co-developing a number of different technologies and systems that allowed us to perform the operation in a way and means that was conducive to railway operations. Some of the things we needed were an aircraft that didn’t require a runway or some kind of launcher, and sensors that have the precision and the capability to see small items on the right of way whether that be small breaks in the rail or changes in tie condition or ballast. We needed highly sensitive sensors and then we needed a way to analyze that data.

Our role as the Beyond Visual Line of Sight Pathfinder with the FAA was focused on the flight technology, procedures and rules, etc., and how can this be done in a safe manner—that’s been the guiding focus of the partnership since day one and it remains as our partnership with the FAA continues. So we explored how to bring this to fruition and then more importantly, how do our efforts benefit the other railroads, how do our efforts benefit other linear asset operators such as pipelines and powerlines—and that’s our role.

DTJ: Over course of the program, how has the technology evolved?

Mr. Graetz: Back in 2015, if we could have just gone and used the standard military technology, we would have been a lot further down the road. But, unfortunately none of that met our needs of precision. We’re not at high altitude and we’re not looking around for people moving on the ground, we’re at low altitude and we’re looking at something that’s very small on the right of way so that required a lot of innovation.

DTJ: And how does a drone inspection compare to inspections performed by humans?

Mr. Graetz: We have a very high-quality maintenance program at BNSF. In many cases, we exceed the requirements given to us by the Federal Railroad Administration (FRA), so that puts a challenge on anything new whether it’s drones, automated geocars, or things we put on locomotives—we have to be better than human or more importantly, we have to provide the human with information that they otherwise couldn’t easily get for themselves. So that takes you to a level where you have to be better, in many cases, than either the human eye or better than human recognition.

The whole technology has been focused on how to supplement what we are doing. We want to see how to give our humans tools to make their jobs easier, safer, and that much more effective for railway safety. That drove a lot of the technology requirements, so when you think about a drone inspection duty comparing to what a human is going to do in most cases we are doing something that is more difficult for the human to do. In these conditions it is certainly possible for a human to get there, but why wouldn’t you send the robot and do it a little bit easier then provide the human with information?

We’ve been given some challenges and a lot of what we’ve done is supplemental which means we take the information our inspectors are seeing now and we’re just trying to identify, detect, or measure something from the sky. Someday that will give us the ability for the rail inspector to only have to get onto the track, be on the side of a bridge, under something, in a tunnel, or something like that when various sensor systems have indicated that the track occupancy is warranted or that hanging over the side of this bridge makes sense versus doing it in scale for all the right reasons.

DTJ: Do you have areas where inspections are done solely by drone and/or do you see there being a day when all rail inspections can be performed by drones?

Mr. Graetz: So because everything we’re doing today is supplemental, it’s an additive and has not replaced anything we are doing with humans or other inspections.

What we are developing right now is expanding the reach of the daily flight operations of our various drones. And we’re expanding that reach so that we can go out there and collect more data in a larger scale. We’re doing that in scale right now so that at some point down the road there might be a day where our human track and bridge inspectors trust the information they’re getting enough that they say ‘well I have to check this bridge today, but I’m going to send the drone first and then I’ll go on there only when necessary.’ That’s still a ways off, but that’s what we’re trying to get to—this becoming part of the tools that our workforce leverages just like they might leverage a special hammer or some special measuring tool—drones can be just another tool in our toolbox.

DTJ: In preparation for talking with you today I read a bit about how difficult and sometimes treacherous track inspections can be so that sounds like a great goal.

Mr. Graetz: Yeah, it can be. You know, the way we look at it is that drones are just part of a series of technologies. At our company, we’re looking at all kinds of things that can be done because our railroad is a safe, but unforgiving environment. So we want to add to our tool box, we want to make some track occupancy optional, and we want to empower our workforce to be better. Why not do that with things like drones?

DTJ: What sort of data do you collect during inspections?

Mr. Graetz: When you look at the data we collect there’s the obvious stuff—getting high resolution video, lots of photos of different things, but the real important thing about whatever we are collecting, whether it be infrared data, LIDAR data which is data you generate from sending multiple laser beams at something, or whatever it is, the core of it is that it has to be intelligent data. That means it has to be data that can be easily related to some other data points that we generate, it has to be easily mined, or it can be turned into something.

One example of that would be if you take thousands of photographs of our bridge and it can be turned into a 3-D model of the bridge. Or, another instance of this would be analyzing the video stream coming from something on a locomotive of what a track looked like when the locomotive was going by. You can then check if that date and time relates to something that the drone got in a still image and use the data so it can be easily related, mined, researched, and/or used to create additional inspection capabilities.

The data is just the element, what’s important is how smart, how useful, or how relatable is that data to other things in our inspection system. For example, today a team of ours is meeting at the Southwest Division Headquarters in Belen, New Mexico. One thing that they’re going to be talking about is that they can get on the phone with us and say ‘I need to know what this switch, this crossing, or this piece of rail looked like on these days.’ We just have to get into our little system and say ‘here are the three images.’ Or it can be engineering saying ‘look the last time you guys were at that bridge—tell me what this segment looked like at this point, send me the video, send me the images.’ So we’re starting now to educate our workforce on what is available.

DTJ: I would think this helps a lot with things like natural disasters where you are looking at changes over a specific timeframe.

Mr. Graetz: Yes, that’s right, change detection is a big piece, along with mining the data, storing the data, and forecasting. There’s a group in our organization within Operations Research that is leveraging a series of data sets to ultimately build a predictive model. This is meant to look at things like whether we have to replace every tie in every subdivision every X number of years or if these ties we need to replace because they have the most prevalence to do XY and Z. That’s what we’re trying to get to with all of these different sensors. The nice thing about a drone is it’s not on the track and it doesn’t get in the way of anything so it can take pictures, it can get video, it can scan with lasers. That’s really what we’re trying to do—that’s the core of it.

DTJ: I’d love to hear a bit more on the technical aspects of using drones. I know you need night vision, but what other features are required on the drones BNSF uses?

Mr. Graetz: We’ve had to add quite a bit of technical capability to the existing systems. As I mentioned earlier, we jointly developed a number of our most critical systems. You mentioned night vision, and we do have to be able to fly at night which really means we have to have to be able to see with enough resolution at night the same as we see during the day. We do that with your typical infrared cameras, different thermal imagery, and we have started putting on these extremely bright LED lighting arrays—even onto our smallest 15 minute flight time line of sight drones—to be able to look at railyards and bridges at night. So we’ve really had to perfect how we can fly at night in support of our efforts.

More important than how do you fly at night, though is how to be visible to other aircraft or to people on the ground. That’s where a lot of our Pathfinder research has delivered benefits because we’ve created a series of systems and sensors that you can put on an aircraft, that allows that aircraft to now become visible to other cooperative aircraft, visible to air traffic control, and that is also being worked on going into 2018 as we start our expansion.

In fact, just the other day there was a larger Cessna twin engine airplane flying between Albuquerque and Clovis, New Mexico. It decided to get some fuel about 1,000 feet from our right of way at an airport. As that pilot got on the radio and announced his intentions, our pilots heard him and said ‘hey, just an advisory, this is N410BN, we’re at this heading, this altitude.’ The pilot thought for a second and you could hear him pause on the radio before saying ‘what are you doing so low?’ And our pilots explained that they were an unmanned aircraft patrolling the right of way of the BNSF railroad. But, this guy was talking to our pilots and he has no idea that they’re hundreds of miles away even though the drone is right near him.

So those are some of the technologies that we had to develop—night flying, aircraft visibility, air traffic control communications, amongst many other things. These all make it possible to put a drone into the air whether it’s in a flooding situation, there’s other aircraft, weather events or service interruptions, or it’s just day-to-day inspections of the infrastructure.

DTJ: We’ve touched a bit on how the data collected is analyzed, but can you tell us more about that?

Mr. Graetz: Yeah, we’re really proud of this part and it continues to evolve almost monthly. When I step back and I look at presentations we gave two years ago versus what we’re talking about today, it’s amazing how fast this moves. Early on we knew the flying object thing that has to happen and there’s going to be some difficulty there, we’re going to have to work with the FAA, and we’ve got to find the right sensors to do all this other stuff, but the real crux of the matter is that you’re going to generate a mountain of data and the data has to be analyzed.

As I said earlier, we’re taking smart images that can be related, but it has to be analyzed and it has to be analyzed quickly. So we had to develop machine vision systems that understood what a tie should look like and what it shouldn’t look like, what a rail should look like and what it shouldn’t look like, and from the most rudimentary basis we take literally thousands images in a single flight. Those images are then processed by a fairly sizable array of graphical processing systems and supercomputing technology that spits out reports that say okay, these are the areas of interest in the subdivision (subdivisions are about 200 miles of track). The reports say here are the areas of interest, this is a picture of the area of interest, this is where it’s located and things like that.

I talked earlier about the models of bridges and so forth, but the net benefit when you talk about what we’re doing in long-range flight and how we are adding subdivisions is about processing. In an average day we were generating just under a terabyte of data, and we’re about to add to that now to end up being about 3 terabytes a today—that’s a lot of data. And, we don’t give ourselves more than about 30 minutes to crank through that.

So we really worked hard at refining the technology to analyze data and we’re not stopping there. Now we have a number of real-time products because it doesn’t matter if you find the broken rail if the trains have already gone over it by the time you send the data out. So we have to do that day and night in real-time as well.

DTJ: Besides normal day-to-day inspections, in what ways is BNSF utilizing drones?

Mr. Graetz: We starting to use it for a lot of environmental remediation efforts. We also utilize it to protect some parts of our infrastructure from malicious activity.

We have leveraged them obviously in a service interruption standpoint too. One prime example of that is getting eyes in the air in Houston after Hurricane Harvey. This was also where a lot of our safety capability, aircraft systems, pilots, and procedures came into play because as you can imagine once the weather—and we were flying in some of that weather—but, once the weather died down everybody wanted something in the air, whether it was a news helicopter, the National Guard, or a drone so we were working closely with the FAA and getting our little pockets of air space protected so that we can operate.

Occasionally will have a small service interruption, some kind of a small derailment or something like that. Drones are great to be able to get in there at the situation long before you put somebody down in the area to assess, get some volume metrics of what spilled, or where it is, and thankfully we don’t have to use those a ton.

And then the other thing which has been nice is a lot of our customers have significant facilities alongside ours and sometimes it helps everyone in planning for something like a new service delivery, a new product, or expansion of a facility. We do a lot of aerial work to give them that bird’s-eye view of how the whole system will work together.

The other thing I’ll mention that I think is really interesting is a really simple thing you can do with a drone that has a very meaningful impact on rail safety. We have a fair number of locomotives that we keep in storage for surge capacity or some kind of emergency need. They have to be inspected, as they can’t just sit there for three years and we hope that when we bring them out they’re working. The mechanical department has to inspect them and I don’t know if you’ve ever seen locomotives all tied up in a rail yard, but there can be 20 in a line and 140 locomotives in a small area. To get on top of a locomotive you have to have fall protection, you have to have it set out somewhere, there’s a lot of things to do. But one day, the mechanical department came to us and said ‘look we’ve got 300 locomotives sitting at Kansas City tied up and ready to go, can you go inspect the tops of them and just tell us if there’s caps on them?’ So in two hours we did what would’ve taken them weeks and entailed climbing all over and all the other things that goes with it. It’s the little things like that go a long ways in providing that return on safety.

One other thing we are really excited about is that we do a lot of yard measurement. We are required to know about certain assets—their locations, has that asset moved, is it too close to something, is it not applicable to a situation, or is it not compliant. Again, when you take a large yard that might be several miles north to south by several miles wide you can send up one of these simple drones and a few hours later we can hand the yardmasters and transportation people an analysis of what’s in the area. And little simple things like that go a long way in making the safety case and a little bit of an efficiency case, and we’ve just begun to roll that out so I’m pretty excited about what we will be able to do in our yards and some of our other larger facilities in 2018.

DTJ: It sounds like this is really going to benefit work in your rail yards. Are there any additional ways you see drones being used by rail in the future?

Mr. Graetz: We’re still fairly young in the use of this technology so we know there will be new ways we can find use of the technology, and in the future, we will begin to explore all of them.

DTJ: What you all have done with drones is significant for the rail industry. How does that information get shared with other railways?

Mr. Graetz: We have as part of our Pathfinder relationship certainly provided a lot of information that the FAA shares with other linear asset companies, but specific to rail we’ve actually been pretty collaborative with some of our competitor rails. While we might be aggressively looking for business in the marketing world, at the end of the day we have to be collaborative when it comes to moving freight.

And we’ve been very collaborative on the technology side of the house. Obviously, we don’t share a lot of our proprietary things that we’ve developed, but there’s a very active dialogue right now with a number of the other Class I railroads and we are sharing lessons learned, benefits, and they’re doing the same things. So from a sharing standpoint, we’ve been very collaborative with others that would have an interest in using this specific system. And what we’re doing is obviously not for everyone because not everyone has the kind of right of way and assets we do, but for those who have a similar use case we’ve shared quite a bit of our findings.

DTJ: What advice would you give to organizations looking to utilize drones in their operations?

Mr. Graetz: I would say the best way to start is like we did—start with the safety case for why one would begin using drones. The reason I say that is that if you start with the safety case, you also start with the mentality that you would not utilize drones to increase safety on one side of your operation all while increasing risk on the other hand.

I think too many times people forget when they look at a civil-commercial type UAS operation, and there’s many growing especially in the line of sight world, to consider the risks. They don’t consider the safety case and they don’t manage the risk out of the operation whether it’s not using the right pilots, maybe not using the best aircraft, or the right system. So I think if you start with the safety case and you build your commercial capabilities around it you will end up being much better off.

DTJ: Is there anything we haven’t covered that you would like to talk about?

Mr. Graetz: One thing that may be of interest to DTJ readers is that as part of our Pathfinder arrangement, we’ve developed a number of flight agreements with various military organizations and have had some very positive interactions with the US Air Force in particular. We’ve actually even created a letter of agreement on one Air Force Base to operate in, around and through it.

So what we’re helping to do is that as BNSF, and organizations like it, interact with organizations like the Air Force or Army that have airspace over our areas of operation, if we can develop ways of safely interacting with them then two things happen: First, others might have the same need and those who operate in proximity will be able to do so in a manner that does not detract from the defense mission of the airspace user. More importantly, think about the rise in drones, how many are registered and how many are operating. The US Military counts on military operating areas, restricted airspace, and other volumes of air need to train and accomplish their mission, and they feel like that’s somewhat at risk and that they are kind of under attack from their constituents. So what we’re trying to do in working with the DOD—at the Pentagon level, as well as at the base level—is to create and maintain those positive interactions. And this first letter of agreement that will be operated in and around Cannon Air Force Base is a start to what ultimately could help ensure that the military is able to continue to have control over the airspace that they need, all the while not standing in the way of transportation safety and related safety initiatives.