Papers & Articles

For the benefit of users of Safran Risk (SR), the powerful CPM Risk Factors-based Monte Carlo Simulation (MCS) modelling tool, this 16-page article describes the search for the best methods of quantifying cost and schedule risk and contingencies for projects.

After a brief history of the use of MCS for project contingencies, the article describes best-practice using SR to quantify cost and schedule contingencies through CPM-based SRA and ICSRA. 

Then follows a discussion of the problem that contingencies produced by the above "bottom up" methods generally rely on the experience of the Subject Matter Experts (SMEs) providing the inputs for analysis.  The rest of the paper addresses how to correct this "fatal flaw".

The need to include past project performance is explained and the development and use of Parametric modelling of Systemic Risk explored.  Reference is made to John K Hollmann's important book "Project Risk Quantification" which details the hybrid contingency assessment methodology "Parametric + Expected Value", best suited to early stages of development of projects and smaller scale projects, when no schedule is available.

The incorporation of Parametric modelling into CPM-based ICSRA & SRA to produce the hybrid methodology P+CPM-based ICSRA (P+IRA) is described.  The recently published AACE RP 117R-21 defining the P+IRA methodology is referenced and the methodology outlined.  

Results of IRAs and SRAs are compared with P+IRAs and P+SRAs and Lessons Learned described.  The use of P+IRA is strongly recommended for large and complex projects, especially just prior to Execution Gate Reviews.

Worked examples with files of the IRA and P+IRA methodologies will be made available to those who register with Safran. 

Parametric modelling (“P”) of systemic risk plus expected value (EV) modelling of project specific risks to quantify project contingency is described in detail by John K Hollmann in his book “Project Risk Quantification” which highlights the value of empiricism to forecast project cost and schedule outcomes, consistent with AACE RP 40R-08. Hollmann recommends the combined P+EV methodology as reliable, easy to perform and not requiring the use of Critical Path Method (CPM). However, the assessment of schedule risk is not straightforward without use of a CPM schedule.

CPM-based Integrated Cost-Schedule Risk Analysis (ICSRA), described in RP 57R-09, represents the most common approach recognised by AACE used to quantify contingencies. Even more common in practice are CPM-based non-integrated methods using schedule risk analysis (SRA) feeding into separate cost risk analysis (CRA).

All of these CPM-based variants can only refer to past project performance through expert opinions and they are criticised for failing to forecast adequate cost contingency. But when practised carefully using good quality schedules, ICSRA is a good predictor of schedule contingency and enables schedule risk and thus time-dependent cost risk optimisation.

Combining parametric modelling of systemic risk with CPM-based ICSRA is considered invalid because the parametric forecasting covers all project risk except major project specific risk events. This paper describes a valid method of combining P+ICSRA, to optimise schedule risk and forecast realistic cost contingency. It describes experience implementing this methodology.

Globally, urban transit projects average around 50% cost overruns, with rail generally having worse time and cost outcomes than road. This paper examines reasons for this, with lessons learned on characteristics of rail that contribute to the problem, identifying some issues that exacerbate normal inaccuracies when forecasting duration and final cost of rail projects,. Even when probabilistic methods are used, these often have inherent faults in the way they are applied. This is compared with recent experience with industries in which inherent risks/uncertainties are often greater than typical rail projects but better outcomes are being achieved by using improved probabilistic methodologies that better capture the significant sources of risk and uncertainty, and, importantly, provide tools to better manage them. The paper concludes by examining a methodology that extends use of the base information in an improved risk-moderated project delivery time and cost forecasting system to then significantly improve modelling the full life-cycle of PPP rail projects to test the overall resilience of the investment proposition over a wide range of scenarios.

After several decades of development of project quantitative risk analysis for time and cost forecasting of project outcomes, achievement of consistently accurate results has remained elusive. Realistic time forecasts of projects have been found to be forecast well by sound Monte Carlo Simulation (MCS) based schedule risk analysis processes integrating schedule impact risk events. But realistic cost forecasting has proven more problematic. This paper demonstrates the capacity of an integrated cost and schedule risk analysis (IRA) method using cost-loaded critical path method (CPM) schedules and MCS to utilise parametric methods to represent and model systemic risks and to include decision-making during iterations to represent realistic treatment of project risks. But CPM based IRA can be extended to incorporate modelling of operation of the assets created by the project to produce probabilistic cashflows based on Capex and Opex uncertainties and risk events as well as revenue uncertainties and risk events, escalation and exchange rate uncertainties and risk events. In this way integrated modelling and scenario analyses can embrace the Total Cost Management framework envisaged by AACE® to support project final investment decisions.

Synopsis

Independent Project Analysis, Inc. (IPA) reported in 2009 that 74% of completed Large and technically Complex Projects they had assessed in Australia had been failures, mostly due to pre-execution causes.  

In 2012, PwC reported that only 2.5% of Australian companies delivered their projects with their planned scope, within time and cost targets and with planned benefits.  Of the remainder, nearly 90% of the failures were due to managerial causes.

Other exacerbating factors are discussed.

Despite these effects, such was the strength of rising demand from China in particular that rising commodity and energy prices rescued many resource projects, until about mid-2012.

Performances of mining & LNG projects and their futures are discussed, with examples. 

The paper goes on to describe an extension of the use of Monte Carlo Method Integrated Cost & Schedule Risk Analysis (IRA), proven on a number of complex and mega projects, to propose an approach that covers all the described uncertainties. 

Evidence based analysis by Independent Project Analysis, Inc. (IPA) in 2009 and PwC in 2012 showed that the dominant cause of major project failures in Australia has been poor management, especially of the development but also the execution.

One significant way to improve the success rate of Australian projects in the future is to model how likely the projects are to succeed using Monte Carlo Method simulation on carefully developed time/cost models.

This paper describes the proven use of Monte Carlo Method Risk Analysis and especially Integrated Cost & Schedule Risk Analysis (IRA) on a number of complex and mega projects with aggregate values exceeding $50 billion.

It starts by examining the development of the use of MCM simulation on projects and reasons why MCM simulation is viewed with scepticism by some.

The importance of the schedule to the rigor of the analysis is examined and some examples of success using SRA given, demonstrating the importance of interaction with the project team, whose input is essential for successful outcomes, often after several rounds of analysis and modification of inputs.

The paper then sets out the features of the IRA methodology and its application to major resource projects. It compares IRA with the traditional approach to risk analysis of projects, using separate Schedule Risk Analysis (SRA) on a summary schedule and passing a schedule contingency cost allowance to a subsequent Cost Risk Analysis (CRA). It carefully demonstrates the inherent shortcomings of the traditional separate SRA & CRA approach compared with IRA, based on sound technical reasons.

The paper concludes with a description of the competing demands of IRA for increasing model size with larger and more complex projects to produce model accuracy, versus the challenges of handling larger schedule and cost models and how the use of Quantitative Exclusion Analysis (QEA) provides the project team with real opportunities for optimising project risk to produce materially better project outcomes.

Total Cost Management (TCM) is a systematic approach to managing cost throughout the life cycle of any enterprise, program, facility, project, product, or service. This paper focuses on projects created to make a return on assets created, from conception through to the end of their economic life.

The paper sets out reasons why too many projects still fail to achieve their projected profitability targets or to be completed within their budgeted time and cost. It explains why deterministic planning and estimating lead to optimistic project schedules and budgets. Also, why separating planning, estimating, risk analysis and project economics analysis prevents a full and holistic assessment of the riskiness and viability of projects and what drives them.

It explains how the use of Integrated Cost & Schedule Risk Analysis (IRA) can replace deterministic schedules (with single value task durations) by schedules with probability distributions for each uncertain duration task and incorporate probabilistic logic where appropriate. The probabilistic schedule can be overlaid with the project estimate, with costs (split into time-dependent and time-independent) spanning appropriate groups of tasks and each of the uncertain costs replaced by a probability distribution. In addition, the significant time and cost impact risks from the project risk register (also expressed in probability distributions) can be appropriately mapped into the cost-loaded schedule. The whole model can then be subject to Monte Carlo method simulation and the probabilistic time and cost forecasts for each milestone and cost component of the project analysed and reported.

From the above process, realistic estimates of cost and time contingency can be assessed and drivers of project time and cost uncertainty can be measured by difference and ranked together as drivers of project cost.

The above approach has been used on studies and projects ranging from a few million dollars to many billions. IRA is compared with conventional separate Serial Schedule and Cost Risk Analyses (Serial SRA&CRA) and IRA’s advantages highlighted.

By extending the modelling to include asset operation and Revenue (IRRA), their associated uncertainties and risk events can be incorporated to generate probabilistic NPVs and IRRs. The balance between profit, break-even and loss can be explored and the drivers ranked for risk optimisation in fulfilment of TCM goals.

Deterministic methods of planning and estimating projects tend to be inherently optimistic, for reasons set out in this paper.

After setting out the challenges of realistic planning of oil & gas exploration in PNG, the paper explores the reasons for inherent optimism in project planning. Use of probabilistic planning and estimating based on an Integrated Cost & Schedule Risk Analysis (IRA) version of the Monte Carlo Method (MCM) is shown to enable the user to take account of these reasons to produce more realistic forecasts of time and cost outcomes. IRA is contrasted with conventional separate cost and schedule risk analyses to highlight the benefits of the integrated approach.

The application of the IRA approach to produce realistic planning of exploration for oil and gas in Papua New Guinea (PNG) is then described.

While PNG represents extremes of time and cost uncertainty that demonstrate the value of probabilistic planning and estimating, the principles and approach are just as applicable to all kinds of projects where time and cost uncertainty are less extreme but still significant.

Key points

• This presentation describes RIMPL’s experiences in developing and implementing a methodology for combining Parametric and CPM-based ICSRA modelling (P+IRA) to produce realistic assessments of time and cost contingencies.  

• In addition, it reports on RIMPL’s experience using the John K Hollmann Parametric + Expected Value (P+EV) methodology to forecast contingencies for smaller projects and earlier project development phase assessments, compared with parallel SRAs in a couple of instances.

• This presentation describes RIMPL’s experiences in developing and implementing a methodology for combining Parametric and CPM-based ICSRA modelling (P+IRA) to produce realistic assessments of time and cost contingencies.  In addition, it reports on RIMPL’s experience using the John K Hollmann Parametric + Expected Value (P+EV) methodology to forecast contingencies for smaller projects and earlier project development phase assessments, compared with parallel SRAs in a couple of instances.

These hybrid methodologies are highly recommended and the reasons for recommending them are explained.

This presentation gives examples of megaprojects across sectors that overran their planned budget and schedule and discusses what makes megaprojects harder to deliver and forecast.

It gives a brief history of attempts to forecast project outcomes accurately and examines the pros and cons of CPM-based Monte Carlo Simulation (MCS) methods.

It introduces Systemic Risk and the use of parametric modelling to assess it, based on past project performance.  The presentation concludes with a new hybrid methodology combining Parametric and CPM-based MCS methodology and reports on its use on projects to date.

In August 2016, AACE International invited submissions for a special track of presentations and a panel discussion on the theme “Project Cost and Schedule Risk Quantification: Alternative Methods”. The accepted papers were to be presented at the 61st Annual Meeting of AACE International in Orlando Florida in June 2017.  The purpose of the track was to present in a single venue, representations of the leading project cost and schedule risk quantification methods in use by various parties and industries in different settings, referencing RP 40R-08 to show how each method addresses the 40R-08 attributes, including an appraisal of the strengths and weaknesses of the subject method. A closing panel was to use a compiled table of the various methods and their strengths and weaknesses as a focus for discussion. 

This presentation reports on the papers presented and the panel discussion.  It aims to provide a practitioner’s guide to the merits of the various methods in use and the industries and sectors in which they have been used, with particular emphasis on when in the project life-cycle and under what conditions the different methods have their strongest advantages. 

Chapter 4 of "Handbook of Research on Leveraging Risk and Uncertainties for Effective Project Management", Raydugin, Y., Editor,  pp 76-108, C.H. Cropley, Published by IGI Global, November 2016.

Time and cost outcomes of large and complex projects are forecast poorly across all sectors. Over recent years, Monte Carlo (MC) simulation has increasingly been adopted to forecast project time and cost outcomes more realistically. It is recognised that the simultaneous analysis of time and cost impacts makes sense as a modelling objective, due to the well-known relationship of time and money in projects.

But most MC practitioners advocate the use of Schedule Risk Analysis (SRA) feeding into Cost Risk Analysis (CRA) because they believe it is too hard to perform Integrated Cost & Schedule Risk Analysis (IRA) realistically. This chapter elaborates an IRA methodology that produces realistic forecasts without relying on questionable assumptions and enables identification and ranking of all sources of cost uncertainty for risk optimisation as part of the process. It also describes an extension of IRA methodology to include assessment of the assets produced by the project as well as the project itself, thus enabling the analysis of business risks as well as project risks. 

Key points

• Major resource project performance in Australia continues to be poor.

• Improved risk analysis before projects are committed, to expose the full range of risks due to operational and business risks as well as project risks, is clearly needed.

• A quantitative risk analysis methodology integrating project development and delivery with operations and revenue streams has been developed that provides a comprehensive understanding of the balance between profit and loss when all sources of uncertainty and risk are incorporated.