Systemic Risk

A 'Top-Down' Risk Quantification Approach

An alternative method to Monte Carlo simulation consists of a hybrid of the top-down Parametric method of forecasting project time and cost execution phase systemic risk contingencies based on a very large pool of previous project time and cost forecasts at funding and their outcomes, feeding into an MCS-based Expected Value modelling of all known treated risk events from the project risk register. This methodology, developed by John K Hollmann of ValidEst (represented in Australia and New Zealand by RIMPL), is available through RIMPL. It does not require the use of a critical path schedule and is straightforward to use. It is also useful for assessing Management Reserve.

This Hybrid methodology is referred to as Parametric (P) plus Expected Value (EV), or “P+EV”. It enables realistic assessment of both time and cost project execution contingencies. While it is a relatively straightforward methodology to use, it only forecasts overall contingencies and does not enable schedule risk to be optimised.

RIMPL has developed an effective way of combining the Parametric assessment of project delivery systemic risk with IRA methodology, producing the benefits of both methods. We abbreviate this as P+IRA methodology.

Systemic Risk and its Quantification

John Hollmann in his book Project Risk Quantification (PRQ) and in alignment with AACE International Recommended Practice, prefers to divide risks at the highest level into

      • Systemic or
      • Project specific
      • (Hollmann adds, specifically for cost, escalation and exchange rate).

Hollmann asserts that Systemic risk is usually not identified and appropriately quantified because project owners usually do not address the risk that the project delivery system (strategy, process, systems, organisation, etc.) may be sub-optimal.

Hollmann and AACE define systemic risks as characteristics of system attributes. Quoting from PRQ (page 15), examples of system attributes are “the internal project system, its maturity, company culture, complexity, bias”, while examples of the project’s interaction with external systems are “regional culture, political and regulatory systems”. Hollmann notes that others call such risks “background or strategic risks. Others call them issues. Research has shown that systemic risks are the greatest source of uncertainty on projects.” It is worth noting that John Hollmann worked for IPA for seven years and that this experience underlies the reference to research.

Later in PRQ (page 48), after identifying the close inverse relationship between scope definition and cost and schedule uncertainty, Hollmann notes that research has identified “other key systemic risks including:

      • Process and Project Complexity
      • Team Development, and
      • Project control practices.

In addition, weak systems can compound the impact of risk events, which requires a holistic treatment of all risk types.”

Hollmann states (Chapters 11 & 18 of PRQ) that systemic risk is quantified using parametric models based on historical data on project performance, correlated to known drivers of project outcomes.

John Hollmann has developed input fields in his parametric spreadsheet model in which such inputs can be assigned rating points which drive the level of uncertainty incorporated in the parametric outputs. The rating schemes are designed to help assure objectivity. These are calibrated as part of the process of applying the tool set to the client’s projects and organisation.

Quantifying Project Specific Risk and combining them with Systemic Risk using Hybrid P+EV methodology

Hollmann explains that the Expected Value/Project-Specific Contingency Estimating Tool is primarily used to determine the proper levels of contingency to be added to Class 4 or 3 (Feasibility) estimates and schedules. The methodology uses both the parametric/systemic tool and the expected value/project-specific tool. These tools are integrated (the outputs of the parametric tool are incorporated directly in the expected value tool).

The project specific tool is an Excel-based line-item estimating template spreadsheet that provides a Risk Input table (a mini-register), a burn-rate calculation table (used in determining cost impact of schedule delays), a cost and schedule impact estimating worksheet, probability distribution tables and curves (outputs), and a risk ranking table. After identifying and inputting the risk drivers, their probabilities, and their cost and schedule impact ranges, the tool will calculate the expected value of the cost and schedule impact of each risk and for the overall project. By also running a Monte-Carlo Simulation (MCS) using the Palisade @Risk Excel add-on program) the tool also provides distributions of cost and schedule outcomes. The tool also includes a correlation matrix table to address risk dependencies (a fundamental step in MCS).

Only major threat project-specific risks are included in the EV assessment. These are usually the risks in the red areas of the Probability-Impact risk matrix, or in the amber areas if there are no treated risks in the red areas. Treated risks below these areas are unlikely to be severe enough to have been excluded from the projects included in the normalised project databases on which the parametric modelling has been based, so they are deemed to be covered by the Parametric modelling.

RIMPL has experience in performing P+EV assessments of cost & schedule contingencies on smaller projects and projects at earlier stages of development than full Feasibility Study / FEED. Experience running SRA in parallel confirms the validity of P+EV assessment of overall project schedule risk.

Combining Parametric and Integrated Cost & Schedule Risk Analysis in a new Hybrid Approach - P+IRA

RIMPL has combined the Parametric methodology with the IRA methodology by replacing the EV step described above with mapping of major project specific risks into the CPM-based IRA schedule model. This automatically takes care of the effect of schedule impacts at the task level.

The P+IRA methodology also avoids “double-counting” of Systemic Risk by subtracting Inherent Risk due to the application of risk factors or ranging of schedule durations and of cost line items from Systemic Risk, using MCS.

The net systemic risk probability distributions for schedule impact and for cost impact are applied as risk factors to the IRA model.

The process can be visualised by referring to the accompanying Venn Diagram which shows the relationship between Systemic and Project Specific Risk events versus Inherent and Contingent risk.

P+IRA methodology determines the net uncertainty in the annular space between Project Delivery Risk and Schedule & Estimate Ranging. The diagram also shows that a portion of the risks in a typical project risk register are systemic risks which should be excluded from consideration to avoid duplication of systemic risk.

The detailed methodology is explained in a paper presented at the 2019 AACE International Conference & Expo, in New Orleans (“Combining Parametric and CPM-based Integrated Cost-Schedule Risk Analysis.”).

Combining Parametric modelling of systemic risk with CPM-based Integrated Cost & Schedule Risk Analysis enables past project performance to be incorporated in contingency assessments while retaining the ability to optimise schedule risk and thus also time-dependent cost risk.

Use of the methodology requires the following inputs:

  • Project Base Schedule & Basis of Schedule document
  • Project Base Estimate & Basis of Estimate document
  • Project Risk Register