In response to a recent question about NextGen benefits, I decided to devote this blog entry to that topic. The question came from Bud Bensel, who said he had developed the initial SOIA procedures and PRM operations. I presume this must have been for SFO or STL. Bensel felt that NextGen benefits were elusive. He wanted to know where I felt “definitive, quantifiable and verifiable improvements in capacity and flight operational efficiencies” would be realized in NextGen.

 

If you’ve been following my NextGen blog, you know how I believe benefits are going to be crucial to the continued support of NextGen and, therefore, its success. We have a situation today where the entire aviation community has rallied behind NextGen. It’s a rare kind of situation and more fragile than one might suspect. RTCA Task Force 5 expressed the industry’s view of what NextGen priorities ought to be: real, meaningful benefits in the near term. In the current federal budget/deficit climate, a program like NextGen has got to demonstrate strong, near-unanimous support to withstand budget cutbacks, which in your gut you know are coming. Public opinion probably counts the most on the Hill. Public opinion is heavily influenced by the media, which includes many more channels of information flow than just a few short years ago. Media channels quickly detect tensions and disagreements in the aviation community, and they tend to gain a disproportionate share of the attention. If NextGen funding is to be supported by the Congress and Administration over the long haul, the one essential ingredient to success will be the early and continuous delivery of benefits. In addition, the word about NextGen’s importance and progress must get out to the public in a form that can be understood easily. In the current situation, NextGen benefits are not just a huge challenge, they are an imperative.

When it comes to NAS benefits, there are many considerations that need to be taken into account. Let me start by covering a few major points about benefits and create an opening and an invitation for further dialogue on this important subject.

What’s operationally different?
Many people view benefits calculation as an economic endeavor. While benefits are usually expressed in monetary terms at the end of the analysis, the question at the core of benefits analysis is: what’s operationally different? If we make an investment to improve the system, what will be operationally different after we implement the change? Will we be able to move more aircraft per hour? Will we reduce fuel consumption and carbon emissions? How much time will be saved? How much more reliable will the schedule be? Will we be able to increase the utilization of expensive equipment and workers? How do these aspects of the operation look before and after we make the change? It’s only after we define what’s operationally different that we can translate the value of those differences into economic terms. The economic translation is almost trivial. Figuring out what is operationally different is the hard part.

Defining the sequence of improvements: staking claim to benefits
When you have a big, visible problem, there are naturally going to be several sometimes competing ideas to address the same problem. Whenever this happens, you run into the risk of each idea laying claim to fixing the problem. When several potential solutions do that for the same problem, each effectively is laying claim to the same set of benefits. This is a no-no in benefits analysis; some refer to this as double-counting. Conceptually, the economists and “bugeteers” want us to invest first in the solution that produces the biggest bang for the buck. Allow that investment to lay claim to the improvements and benefits associated with mitigating the problem. Once this is done, then consider how much more room there is for further improvement and allow other solutions to compete for the left-over benefits. If any of the solutions is capable of producing more benefits than it costs to implement, then it can be considered for implementation. But, in most cases the bar is set pretty high, since most of the benefits should have already been claimed by the preceding improvement. In practice, all of this doesn’t work as neatly as one might hope. It’s usually a food fight. The biggest bang for the buck alternative might take a lot longer to implement or might require more cash than people are able to commit. Sometimes the beneficiaries of the improvements are not the ones who have to make the investment, so there’s not enough motivation to spend the money. Each alternative has an associated level of risk that may vary widely from competing alternatives. Alternatives don’t always pursue the exact same set of benefits, so an apples-to-apples comparison is sometimes difficult. Then, there are hidden agendas at work as well, and some individuals are not immune to manufacturing misinformation to advance their agenda.

Considering baskets of investments and benefits
When you work with improving a nation’s air transportation system, many of the required investments are huge. There is a tendency to want to subject each and every incremental investment to economic analysis and justification. The problem is that some of the required investments are enablers of downstream improvements. This is often the case with infrastructure investments. If you just look at the infrastructure investment by itself, it may be difficult to economically justify. It’s a bit analogous to trying to do a cost-benefit analysis on a transmission for a car. In this case, it’s clearer that doing a cost-benefit analysis on a transmission doesn’t make much sense. It makes more sense to do it on the entire car, which vitally depends on the transmission and if employed properly is capable of delivering substantial benefits. Doing a cost-benefit analysis on data link communications is analogous to doing a cost-benefit analysis on a transmission.

Everything is connected — well, almost everything
In the past, when various projects were launched to improve the arrival and approach phase of flight, not much attention was paid to what was happening to departures, or surface operations. In fact, there were times in the past when there were individual projects addressing arrivals, departures and surface operations, each individually striving to improve its own operation without regard for the others. The optimization algorithms for arrivals, departures and surface operations were fighting with each other. This is an obvious case where the three processes should be worked on simultaneously and in a coordinated way to produce an optimization of the whole, across all three processes. But, you shouldn’t stop there. There are overhead streams of traffic, inbound flights that are rerouted and delayed due to weather or congested sectors en route, and flights that are flying to destinations that are projected to be over-saturated when the flights are scheduled to arrive, as well as a myriad of similar interdependencies. The point is, you can focus on a specific problem, but unless you consider what is going on in the bigger picture, a local improvement might not produce the envisioned benefits as a result of some other connected problem in the system. And, just because you fix an isolated problem, there’s no guarantee that the fix will work when it has to function as part of a larger system. There is a body of work known as Theory of Constraints, which directs focus on the ultimate objective of a system (or organization) and bottlenecks that prevent the system from attaining its objectives. Focus is placed on removing the most obvious bottleneck, recognizing that as soon as one bottleneck is eliminated, another one is likely to emerge elsewhere in the system. Each constraint is relieved until the system begins to function as intended. Often, it’s difficult to predict where the next constraint will emerge, since systems are more complex than we make them out to be. It’s another way of looking at the system we operate in. Our work in Air Traffic Flow Management over the years has taught us a lot about the connectedness of our system. It’s caused us to develop an integrated, end-to-end suite of tools, which we call Metron Harmony. Check it out.

Nothing is static, especially stakeholder behaviors
In operational and benefit analyses, as in most studies, we make assumptions. Many studies have a section devoted to assumptions. Often assumptions are made to make the problem tractable. Assumptions can simplify the problem or the analysis, so that it’s possible to complete the analysis. For this reason, sometimes we refer to these as “simplifying assumptions.”  As an example, studies will say, let’s assume that growth will continue to occur at 3 percent per year as it has for each of the last 5 years on average, or let’s assume that the price of fuel remains the same in terms of 1995 constant dollars. If you don’t make these kinds of assumptions, it’s a lot harder to complete the analyses. The problem is that these often are bad assumptions, bad in the sense that they are unlikely to hold. So, the results of the analysis are not likely to be reliable. A simple case from my own experience occurred while I was with the FAA. Around 1997, the agency was faced with mounting delays at DFW. The airlines were pleading with the FAA to develop and deploy new technologies to increase the capacity and thereby relieve delays. Without capacity improvements, delays would mushroom and gridlock was a real possibility. We did the analysis and made some simplifying assumptions. We showed that if we invested in the new technology, throughput capacity would be increased significantly and delays would be substantially reduced for years to come. The benefits significantly exceeded the cost. So, the FAA made the investment. The technology was developed and installed. When it finally went into operational use, as expected, throughput capacity was increased. Success! Then, the unanticipated happened. American added a lot of new flights to the schedule. So, delays didn’t decrease as predicted. We later learned that American was concerned that if they didn’t fill that capacity, a competitor would. They felt it was vital to protect their dominant position at DFW, so they were compelled to add the flights. This is a simple example. There are many and far more subtle behavior changes that occur whenever improvements are introduced. Fortunately, today we have tools that can help us model and predict primary, secondary and tertiary responses to changes in complex systems. To each action, there are reactions and subsequent reactions rippling through the system until a new equilibrium is established.

POET (T is last)
When I was still a young person at the FAA, I can’t recall exactly where or from whom, I learned about POET – political, operational, economic, and technical. That’s the order of importance of considerations that need to be accounted for when bringing about change. Political factors cover a lot of things. We’ve already talked about the need for continued support from the Congress and the Administration, particularly in a difficult fiscal climate. Public opinion is a driver, along with a myriad of political considerations. But, political considerations also include stakeholders, their interests and their cultures. These stakeholders include the airlines, general aviation, airports, labor unions, travellers, manufacturers and service providers, government organizations, and others. Many of the changes envisioned under NextGen will entail changes to the concept of operations. People in various parts of the systems will be asked to take on new or modified tasks, and to perhaps stop doing things that had been done for many years. More functions will be automated, in many cases to keep the human workload at an acceptable level. When people, computers, procedures and new concepts are integrated, the operation must effectively achieve what was envisioned and achieve it safely. In the realm of economics, the FAA requires a positive return on investments, so the benefits must exceed the cost of changes. But, in those instances where complementary investments must be made by the users or other stakeholders, it can be dicey, irrespective of what the benefit-cost study says. The required investment by each user must stand up to benefit-cost scrutiny from the perspective of that user, assuming that their cash flow or debt load even allow them to consider the investment. Lastly, the technical work must be done – the technology must work. In many cases, this is the easiest part of the problem. That’s the reason T is last.

FAA’s response to RTCA TF5
In late January 2010, the FAA issued its response to the RTCA Task Force 5 recommendations of September 2009. The FAA’s response is a thorough, point-by-point enumeration of commitments it’s making to address the RTCA recommendations. The responses thoughtfully address the recommendations and show some creativity by introducing some fresh approaches to old problems. As an example, the FAA’s response includes a discussion about the change in handling controller operational errors (OE). You have to go back about five years to recall that OEs seemed to be growing at a disconcerting rate (this was probably more visible internally than externally to the FAA). As a result, the FAA “declared war” on OEs. They were going to be hunted down and exterminated. While obvious in hindsight, at the time no one seemed to care about the potential impact of the “war on OEs” on efficiency. If you’re a controller and there’s no reward for increasing throughput, but a harsh penalty for any infraction of the separation standard, including the most common but least worrisome 2.9 nm in-trail separation, what are you going to do? You spread out the traffic, so you never have an OE. But, what does that do to traffic? It drops the acceptance rate and reduces capacity during busy periods. This FAA response takes credit for changes it instituted last summer. The FAA now reports OEs by removing specific employee identification and has implemented a systematic, non-punitive method of identifying, evaluating and correcting potential safety hazards. The process includes the controller union as an integral part of the process. There are technologies that are being introduced as well to simplify the task of merging, spacing and separating aircraft. But, the non-technical change in handling OEs is likely to have the most far-reaching effects. It’s an example of the kind of change that can make a huge difference in the way the system functions. Change can happen quickly and inexpensively, yet produce tremendously beneficial results.

This blog is already several times longer than a respectable blog entry, so let’s end it here for now. We’ll pick it up again and discuss where in NextGen we might expect to see verifiable, measureable benefits. Without question, there are many opportunities to generate clear, substantial benefits in the near term. But, as you can see, benefits can be elusive, since there are numerous ways of falling off the tightrope if you don’t know the ropes.