“Many of the city’s gas, steam, sewer, and water lines are not only aging, but are made of materials not in use today, and prone to leaks and breaks. Much of the city’s infrastructure is not mapped, making it hard to pinpoint issues to make efficient repairs or improvements.” [1]
Preface
The above quote is a description of a city’s crumbling foundation. A crumbling foundation that leads to continuous road, lane, block, and sidewalk closures to address buried infrastructure repairs and replacements. From a resident’s perspective, these road closures disrupt their daily movement. From the city’s perspective these disruptions cumulatively impact a city’s social and economic engines that affect viability. Quantifying these disruptions is a key step towards having a Smart, Strong and Just City.
The intent of this blog is to provide a proactive, open source, draft plan of how big data can move towards big information that quantifies a city’s Disruption Occurrence Index (DOI). Part 2 of 2 continues the DIKW journey by discussing how the DOI can become a planning tool for imparting wisdom as city develops a master plan for implementing utilidors.
Disruption /dis-ruhp-shuhn/ - a condition where the normal continuance is destroyed.
Hazard /haz-erd/ - something causing unavoidable danger, peril, risk, or difficulty.
Introduction
Even though buried distribution infrastructure under our public right of way (PROW) is intended to be out-of-sight-out-of-mind, there are daily reminders that it really is not – out of mind. Our buried infrastructure finds a way to remind us that it exists. In the United States we have 720 water main breaks daily, and in New York City, as an example, the streets are cut open hundreds of times per day to facilitate buried infrastructure repairs, replacements, and maintenance. [2]
The term “indirect” costs are most often used with natural disasters.[3] Unlike many natural hazards that provide a warning, e.g., time until a storm approaches, the chronic hazard of aging buried infrastructure manifests as either Planned Closures of our streets to facilitate proactive maintenance, or as Unplanned Closures that reactively address failed infrastructure. In both cases, a disruption to the city occurs. These daily disruptions need to be evaluated by magnitude, impact, and risk velocity.[4]
Which disruptions rise to a level of being a hazard? What KPIs and spatial analytics are required to better quantify the cumulative economic, environmental, and social impacts to the city caused by these disruptions?
True Costs of Infrastructure
There has been considerable academic research to date. While this is not intended to be an exhaustive literature review on the topic, it is worthwhile to summarize some key previous works. The terms used to find the True, or Total, cost of infrastructure in the research are similar, but note that the two causes of failed infrastructure fits well into the groupings of Unplanned and Planned Closures.
Unplanned or Emergency Road Closure
Beginning with Ray Sterling’s paper that maintained two categories of direct and indirect costs but introduced the term “Societal”: “The total societal costs of construction, maintenance, repair or upgrading of utilities include the indirect as well as the direct costs of such work.” [5] That is, Total Societal Costs = Direct Costs + Indirect Costs.
This additional term – Societal – opened a new way of characterizing costs when the economic impacts of failed infrastructure could be attributed to the private/municipal utilities that owned the assets but were sheltered from the societal costs of failed private infrastructure. This led to the term “Social Costs” becoming a separate category: “…when property damage occurs at a later date and is not attributable to the contractor this does become a social cost”. [6] Defining “social costs” led to a complex set of three categories of social costs with over 100 indicators. Adding to the research, Hunt, et al, added additional categories of Environmental and Sustainable Costs. [6]
Planned Closure
Planned road (road, lane, sidewalk) closures are intended for proactive maintenance of buried infrastructure. This allows the city, or its excavation permit holders, time to put up road signs and some warning of the disruption. Generally, a known begin and expiration date are given on the permit for excavation.
Accidental Strikes
Whether working in Planned or Unplanned closures, even with 811 "one-call" system in place, occasionally other buried utilities are accidentally cut during due to lack of precise location data. Accidental strikes are frequent and expensive and also lead to damaged property. Studies provided quantitative ratios comparing Social Costs to Direct Costs: 29:1 [7]. Makana, et al also proposed that: Total Costs = direct costs (paid directly by the utility owner) + indirect costs (those borne by third parties in the contractual agreement) + social costs (those borne by other parties not engaged in the contractual agreement).
Whether our buried infrastructure fails (Unplanned Closure), are accidently damaged, or a planned closure for proactive maintenance, the result is a disruption. What needs to be modeled is how we quantify the impact to a city caused by these disruptions.
Current Efforts of the Town+Gown: NYC Utilidor Working Group
For the past five years, I’ve volunteered as a member of the Town+Gown: NYC (T+G) “Utilidor Working Group”. T+G is a city-wide university-community partnership program, resident at the New York City Department of Design and Construction (NYC DDC) that brings academics and practitioners together to create actionable knowledge in the built environment. The concept of a Disruption Occurrence Index (DOI) has evolved over the past few years as T+G and its Utilidor Working Group has been working with graduate student programs:
Columbia University School of International and Public Affairs (SIPA)
T+G Utilidor Working Group
NYU Tandon School of Engineering MOT Capstone. LAMP 1, August 2021
NYU Tandon School of Engineering MOT Capstone. LAMP 2
NYU Tandon School of Engineering MOT Capstone. LAMP 3, May 2023.
The next steps, and intent of this blog, is to provide a roadmap for further T+G capstones, and/or individual studies, to build sequentially and answer the questions of big data, scoring, and the impact of duration.
Creating the DOI
The goal of this section is to communicate, in English sentences, not SQL or python codes, a roadmap for moving from the theoretical to a Minimum Viable Product (MVP) that provides a robust foundation for developing the Disruption Occurrence Index (DOI). This begins with an authoritative, publicly available, and consistent data source, and a methodology that will allow research to move from the basic to complex analyses. The complexity of sorting and scoring each disruption based on cause, characteristics, scale, and duration needs to be approached one sequential step (Phase) at a time.
The authoritative data must provide the basics: location, field ID, scale, reasons for the excavation permit, and the beginning date (end date discussed later). From this foundation, the next phases can be determined.
This blog proposes the following four phases to build upon each other. Each phase will provide lessons learned, revisions, and results that will be the starting point for the subsequent phase. Revisions to each phase will depend on the nature of publicly available data in a given locality.
Phase 1: DOI MVP Foundation
Just as a smart city should build upon a solid foundation of buried infrastructure, the DOI MVP must be based on a solid foundation of data. For a long-term solution, the DOI is most closely related to the city agency responsible for issuing and tracking the excavation permitting of planned/unplanned road closures (road, single lane, shoulder, or sidewalk) to facilitate the repair, replacement, or maintenance of buried infrastructure. The basic starting point of a disruption is knowing its spatial location, providing a field identification number (to track), and the beginning date.
Additions – Static Criteria
From this foundation of location, ID, and date, additional static (unchanging) criteria can be added. These are data that will not change during the life of the disruption. As shown in Table 1, data fields A through G provide criteria that will enhance the description of the disruption, e.g., a 21” diameter water main failure in one community can be compared to a 21” diameter water main in another community.
Data fields A and B are binary, yes/no. Data field C should be static for Unplanned and Planned Closures. If the spatial scale (C) does increase after the initial permit is created, the cause will likely be from a new disruption, caused by an accidental strike, and therefore a new disruption ID. The temporal scale (D) is unique in that the beginning date is static, but the planned end vs reality end date (aka, expiration date) must be dealt with separately. Data fields E, F, and G provide a description as to the scale of disruption, e.g., a busy road with large, pressurized pipes being disrupted, or a residential feeder road with smaller pipes. These data may/may not exist depending on locality.
Data Field | Description | Weighting Considerations |
A: Planned Closure | Proactive for repair/maintenance | Low or no weighting |
B: Unplanned Closure | Reactive for repair/replacement | Higher impact to accidents |
C: Spatial Scale | Lane, sidewalk, road, block, blocks | Weights increase with scale |
D: Temporal Scale | Beginning date | See Duration discussion below |
E: Traffic Impact | Type of road, ADT | Weights by road type, ADT quintile |
F: Type of Infrastructure | Pressurized pipes, pipes, cables | Weights by pressurized, gravity, energy |
G: Capacity of Infrastructure | Diameter or kV | Weights by diameter or kV quintile |
Table 1: Basic Static Datasets for a Disruption
Fields should be provided to allow weighting for each data field, either 1 or 0 for data fields A and B, or up to five categories for C, E, F, and G (i.e., C1-5, E1-5, F1-5, and G1-5). If the data are available. For example, weighting pressurized water mains by size classifications: 2” – 8” small, >18” medium, >36” large, >48” very large, <48” critical can be done using these subclassifications. The duration of the disruption is discussed separately, below.
Whether the classifications are scored 1 through 5, or 1 through 100 will be determined in Phase 2. The goal of Phase 1 is to provide the ability to sum these scores for each disruption and allow the flexibility of statistical analyses and additional weighting for duration in Phase 2.
Summing and Unique Circumstances
While basic summing of the limited list of criteria in Table 1 seem simplistic, a foundation will be built for a big data environment that will allow for additional modeling that better defines the range of unique circumstances of each disruption. For example, consider the range of how DOIs can be characterized from this initial limited set of criteria: A, C2, E5, F1. By characterizing each disruption, analyses in Phase 2 (and further in Phase 4) can identify those combinations that led to significant disruptions, i.e., hazards.
At this point, a sum can be generated for each disruption using data fields A, B, C, E, F, and G.
Compounding Multipliers – Duration and Active Assets
As previously mentioned, the duration is likely the most consequential criterion for determining whether a disruption becomes a hazard (i.e., large disruption). A local business, or daily commuter, can likely whether a half-day interruption to their schedule. But when does the disruption become a serious hazard? A longer duration increases the impact of all criteria – economic, social, environmental.
The anticipated issue is that an exact ending date to the disruption will not be on the initial permit. Rather the DOI MVP must accept regular updates to determine the actual duration of the disruption. The key for each update is to determine if a disruption has expired.
If resolved, the “expired” permit will have an end date to provide actual duration.
If un-resolved, the “renewed” provides duration at time of update but kept current until final end date “expired” is provided.
The duration of each disruption should be a multiplier applied to the sum of each disruption. The multiplier is determined by the duration in Phase 2. A second multiplier field should be added in Phase 1. This could be used for pressurized assets (i.e., “active” assets) vs gravity (i.e., “passive” assets), or for another reason yet to be determined in Phase 2.
At this point, the DOI MVP can sum a disruption and provide at least two multipliers to that sum.
Phase 2: DOI MVP Weighting, Multipliers, and Statistical Analyses
Why an index? An index is a statistical measure or composite value that provides a simple, standardized way to represent complex information. This allows decision-makers to understand and gauge the importance of a single value. This step cannot be underestimated for creating a tool. As discussed in Phase 3, the DOI can be color-coded to represent themes, e.g., equity, economics, environment. By making outcomes understandable, the DOI can better address a community perspective of local social and economic disruption and cities can better address the true implications of infrastructure failures. [8]
Score and Index: First Update
Phase 2 will determine whether an arithmetic mean or a geometric mean should be used to calculate DOI and analyze the sensitivity of scoring with multipliers to better sort DOI into quintiles.
For each update of the data, a disruption will have either an end date (expired) or will have a longer disruption (renewed) than the previous dataset. By allowing both a score (that currently has no limitations on range) and an index (scale 0-1), a history can be maintained. For example, historic scores can be maintained for future analyses to identify the largest disruptions over a city’s history. While the index provides current ‘hot spots.’
The Multipliers
The two multiplier fields risk skewing both total score and the index. This is a critical part of Phase 2 scope of work. The result of Phase 2 is to provide guidance as to which disruptions are worthy of further study as hazards in Phase 3. By providing guidance as the range and circumstances of adding multipliers, disruptions can be classified as hazards and moved for further evaluation in Phase 3.
Frequency
The spatial location of all disruptions might be evaluated separately. That is, the sheer number, big or small, in specific locations, might cumulatively represent a hazard. This can be considered further in Phase 3, or Phase 4.
Phase 3: DOI MVP Supplemental Data, Themes, and Chronic Hazards
Which disruptions are chronic hazards? This could be based on the annual highest score, the longest duration, or highest DOI. This should be part of the recommendations from Phase 2. In Phase 3, the focus is on developing a methodology for adding depth for characterizing those disruptions that have been deemed as chronic hazards to a community/city. The outcomes of Phase 3 would provide a classification of the chronic hazards by themes: Environmental (noise, PM2.5 emissions, …etc.), Economic (local businesses closed, long-term economic recovery, …etc.), Safety (change in flow of traffic, auto/ped/bike accidents, property damage, insurance claims, …etc.), and Equity (defining the community where chronic hazards occur, impact to access to health centers, schools, and grocery stores, …etc.).
Supplemental Data and Additional Theme Scoring
Adding supplemental data from different sources (e.g., census, or traffic) that is updated at different frequencies (ranging from real-time to decadal), adds complexities that must be considered in this phase. But the story-telling potential is what politicians, impacted communities, those with fiscal responsibility – decision-makers – need to have to promote change. The methodology may lead to new classifications (e.g., tourism impacts) and may lead to multiple theme classifications (e.g., EnvSaftey). The conclusion of Phase 3 should also consider whether to further add weights (multipliers) based on classification.
Phase 4: DOI MVP AI and Refining Modeled to Reality
This phase is left blank for now. Not that it is not important, but AI should be explored when you understand exactly why you need it. The lessons learned from each Phase will culminate in the iterative and frequent decisions required to produce a meaningful DOI that will help cities determine where utilidors would best be placed (to be addressed in Part 2).
Conclusion
As mentioned in the preface, the DOI will be a tool for quantifying, or characterizing, how buried infrastructure disrupts a city. Doing something about these disruptions is the basis for part 2/2.
I used the term “roadmap” rather than blueprint for the four phases as a blueprint is typically a final drawing with specifications. Whereas ‘roadmap’ implies there is an intended destination, but between start and finish there will be detours and dead ends. The lessons learned all lend to moving an MVP to a Version 1.0. Followed by V2.0, V3.0…etc.
Footnotes
[1] NYC Gov, retrieved from: One New York: The Plan for a Strong and Just City pg. 34/354
[2] Bloomberg News, retrieved from: https://www.bloomberg.com/news/features/2017-08-10/nobody-knows-what-lies-beneath-new-york-city?src=longreads
[3] “indirect costs” are also by the NIST Community Resilience Guide used when the infrastructure failure is caused by natural hazards
[4] Mark Reiner, PhD, "The Risk Velocity of Crumbling Infrastructure," WISRD, 2022.
[5] Sterling, Ray (1994). Indirect Costs of Utility Placement and Repair Beneath Streets, MdDoT, Report Number MN/RC-94/20, August
[6] Hunt, D., Nash, D., & Rogers, C. (2014). Sustainable utility placement via Multi-Utility Tunnels. Tunneling And Underground Space Technology, 39, 15-26. doi: 10.1016/j.tust.2012.02.001
[7] 2020 Makana, Lewis; Metje, Nicole; Jefferson, Ian; Sackey, Margaret, Rogers, Chris Cost estimation of utility strikes: towards proactive management of street works. https://doi.org/10.1680/jinam.17.00033
[8] Reiner, Mark. “The Repair vs Replace Debate – from a City’s Perspective.” WISRD, October 22, 2020.
Mark. Great piece on the Disruption Occurrence Index or DOI. The issue of creating a standard or common method of recording these is crucial, and I applaud that. Imagine if this could be as commonplace as Google's "General Transit Feed Specification" which unified how every transit agency communicates their routes, time, and service so that everyone "speaks the same language"?