We Again Consider the Problem of Estimating the Parameters of a Model
When you need to estimate the cost of a project or parts of a project, you almost inevitably come across the technique of parametric estimating. This is a quantitative approach to determine the expected cost based on historic or market data. It is also a method that is used in the 'estimate cost' process in PMI's Project Management Body of Knowledge (see PMBOK®, 6th ed., ch. 7.2).
In this article, we are introducing the technique of parametric estimation. We will also provide guidance to and an example of the practical use of this method.
What Is Parametric Estimating?
Parametric estimating is a statistics-based technique to calculate the expected amount of financial resources or time that is required to perform and complete a project, an activity or a portion of a project. It is an established method in several project management frameworks such as the Project Management Institute's PMI Project Management Body of Knowledge (PMBOK) where it is listed under the tools and techniques of the 'estimate cost' and 'estimate activity duration' processes.
The determination of an estimate is based on a statistical (or assumed) correlation between a parameter and a cost or time value. This observed correlation is then scaled to the size of the current project (source: PMI Practice Standard for Project Estimating, 2nd edition, ch. 4.2.2). For instance, in highway construction, the cost and time to build 1 mile in a previous project could be the basis for calculating the resources and schedule of the current construction project. However, this requires that there is statistical evidence of the correlation and if the characteristics of both projects are comparable).
To calculate the cost or duration per parameter, a set of historical data is required. This could be obtained from previous projects (companies in construction, consulting, IT and other industries sometimes store such data centrally) publicly available market data or agencies that provide statistics for benchmarking.
While parametric estimation is a common technique to estimate costs in different levels of granularity, the form of its implementation varies greatly.
Some projects build complex statistical models and perform a comprehensive regression analysis for various parameters. They might also develop algorithms and assign a significant number of resources for deploying and (back)testing such models. This is an approach applicable to large projects or so-called 'mega projects' where even small shortcomings in the accuracy of estimates could cause a material impact.
The PMI Practice Standard for Project Estimating provides detailed guidance to project cost estimation. PMI members can access it through the PMI website.
Smaller projects, on the other end of the range, can use parametric estimation by developing functions or simply applying the 'rule of three' if there is evidence or a reasonable assumption that observed parameters and values correlate. This may also involve some expert judgment whether assumed regressions are reasonable and applicable to the project or activity.
According to PMI's Practice Standard, there are 2 types of results:
- Deterministic and
- Probabilistic estimates.
Deterministic Estimates
The deterministic result type of the parametric estimation is a single number for the amount of cost or time needed, calculated based on parametric scaling. It is sometimes manually adjusted to account for differences between the current and historic projects (e.g. different levels of experience of the teams) or to add a contingency reserve.
Probabilistic Estimates
This result type is not producing a single estimate but a range of estimates based on the probability of different cost and duration amounts. This is often presented in the form of a probability density curve as shown in the below chart.
A method to convert this function into a more practical range of estimates is the identification of three points on that curve:
- The most likely estimate which is normally the cost or time value with the highest single probability,
- The pessimistic, and
- The optimistic estimate.
The optimistic and pessimistic cost and duration estimates can be determined by defining a target probability (e.g. 90%, 95% or 99%, subject to the quality of the underlying data and the type of the value distribution) and/or a multiplier to their standard deviations. Depending on the form of the probability density curve, these 3 points can then be transformed into a so-called final estimate, a similar approach as for the triangular or the PERT beta distribution.
How to Perform Parametric Estimating?
This section describes the steps needed to perform a parametric estimation. As mentioned before, the extent and complexity of the estimating process and the deployed tools should be tailored to the needs of a project. In the below steps, we have added a note where we would expect differences between small and complex projects.
Determine the Parts of Your Project for Which You Can (Potentially) Use Parametric Estimating
As a first step, a project manager needs to identify which portions of the work. The selection criteria are mainly
- Required level of accuracy, i.e. for a rough estimate, you might be able to estimate the whole project at once (e.g. building cost per square foot) but for definitive estimates, you will need to go into a more granular level.
- Correlation of parameters and values, i.e. you can only estimate work or resources using this technique if you know or assume that there is a correlation between a parameter and the duration and/or cost (subject to testing).
- Availability of data for parametric estimation (see next step).
The work breakdown structure (WBS) can be a good starting point to select the scope of parametric estimation.
Research Historic and Market Data on the Cost and/or Time Requirements of Similar Projects
If you have identified areas for which parametric estimation could be applicable, you need to gather the relevant data. Potential data sources are internal cost/time/resource databases that are fed with observed values from previous projects (often available in companies that are working on certain types of projects regularly), publicly available data such as public statistics or industry benchmarks.
Identify the Parameters that You Wish to test for Correlation with the Cost or Time Values
Once you have created a set of data, you need to select the parameters that could potentially correlate with the cost or time requirements. These correlations will be subject to further statistical analysis if you are using a model.
In smaller projects, you would probably apply expert judgment or common sense to decide which parameters would be reasonable. If this suffices the needs of your project, you can skip the next two steps and move on to the calculation section.
Determine the Parameter(s) that Drive Cost or Durations (e.g. Through a Regression Analysis and Further Statistical Analysis, if Needed)
Test the set of parameters identified in the previous step for correlations and/or regressions. This will usually involve the use of statistical software such as R or other free or commercial solutions. The use of artificial intelligence (machine learning) can also be considered, e.g. to identify patterns in complex datasets. At the end of the analysis, select those parameters that are appropriate for your estimation model.
[For Complex Estimates / Projects] Develop a Model and Perform (Back)testing if Possible
Develop a model to predict the cost and duration amounts of your project based on the set of parameters that have been identified in the previous step. Make sure you back-test the results against historical data.
Note that this step requires statistical expertise and data analysis experience. In fact, these models can be quite complex, in particular for large projects. So, be aware of the cost, time and resources that are needed to develop such type of model. Balance this against the potential benefits and the requirements of the project and its stakeholders with respect to the estimation.
Compute the Parametric Estimate(s)
If you have built a model, you will calculate a probabilistic or a deterministic estimate by feeding the current project's parameters into the model.
If you have used expert judgment rather than a model to identify the relevant parameters, you will need to calculate the amount of cost or time per parameter unit first.
You can then develop and apply a cost or duration function that considers these parameters as independent variables. If you fill in the parameter values of your current project, the result will be the cost or duration estimate (deterministic) for this project.
In its simplest form, the parametric estimation comprises of only one parameter and a linear relationship between the parameters and the amount of cost or time. In this case, you can use the 'rule of three' calculation and multiply the cost or duration per parameter unit with the value of the parameter in your current project. The formula is:
E_parametric = a_old / p_old x p_curr,
where:
E_parametric = parametric estimate,
a_old = historic amount of cost or time,
p_old = historic value of the parameter,
p_curr = value of that parameter in your current project.
You will find a few examples in the respective section below. These examples of parametric estimating are also based on a 'rule of three' approach.
What Are the Advantages and Disadvantages of Parametric Estimating?
Pros
- The parametric estimation technique can be very accurate when it comes to estimating cost and time.
- It is therefore easier to get stakeholders' support and approval of budgets determined this way.
- Once the model is established, it can be reused for other similar project and the quality of data becomes better with every additional project.
- Manual adjustments to the calculated results to account for differences between historic and the current project can help address weaknesses of a model or underlying data, e.g. if qualitative and environmental factors are not fully fed into the model.
Cons
- Parametric estimating can be time-consuming and costly. Obtaining the historic data and building a model requires some efforts and resources.
- The required availability of historic data and the expected scalability are further constraints for the use of this technique.
- It can often only be used for some parts of a project while others need to be estimated with different techniques.
- Relying on the data may not be appropriate if certain factors differ between the current and previous projects. Aspects such as the experience of the personnel, the progress on the learning curve, environmental factors and other criteria may not be fully reflected in a model. Thus, the reliability of calculated estimates may be affected.
- The quality of the historic data may also be an area of concern in some cases. The saying 'garbage in, garbage out' applies to parametric estimating in the same way it is true for any other use of data.
- Parametric estimating has the inherent risk of providing a false sense of accuracy if models are inaccurate or data from other projects prove not to apply to the current project.
Examples
This section comprises of 2 simple examples that will help you understand the principles of parametric estimating. However, keep in mind that the models and the statistical analysis are usually more complex in practice.
Example 1: Determining Construction Cost Using a Parametric Estimate
A project team in a construction company is asked to estimate the construction cost for a new office building. The company has completed several similar projects over the last couple of years. It uses an in-house database to granularly track the activity durations and costs of previous projects.
For an initial estimate, a rough order of magnitude, the company intends to use parametric estimation with the building cost per square foot as the relevant input parameter for the parametric estimation. The estimate shall then be calculated with the rule of three.
For similar types of buildings, the average construction cost amounted to $200 per square foot in the past (= cost per parameter unit).
The new building is supposed to have a total area of 3,000 square feet (= parameter value in the new project).
The calculation of the order of magnitude of the construction cost, using a parametric estimate (deterministic) determined with the rule of three, is as follows:
Estimated construction cost = $200 x 3,000 sq ft = $6,000,000.
In practice, there are obviously a lot more factors to consider and the model would be much more complex, obviously. This simple calculation may however even suffice for a rough order of magnitude in the initiation stage of a project.
Example 2: Estimating Implementation Cost of an IT System
A software vendor is asked to estimate the implementation cost of its solution. The implementation consists of 4 parts – installation, customizing, the establishment of interfaces to other systems and testing (data migration is not in the scope of this project).
While the cost of the installation is fixed, the vendor is using different parameters to determine the cost and time estimates of the other parts. These are based on historic data and have been included in the following sample estimation sheet.
| Part | Parameter | Historic avg. Cost per Parameter Unit | Historic avg. Time per Parameter Unit | Parameter Value in Current Project | Estimated Cost | Estimated Duration |
| Installation | Fix | $25,000 | 10 days | Fix | $25,000 | 10 days |
| Customizing | Number of different product lines the client produces | $12,000 | 5 days | 15 product lines | $180,000 | 75 days |
| Establishment of Interfaces | Number of Interfaces with other systems | $20,000 | 5 days | 5 system interfaces | $100,000 | 25 days |
| Testing | Cost of Customizing + Cost of interfaces | $300 (per $1,000 spent on parameter) | 0.0089 days per $1,000 spent on parameter | Sum of Customizing and Interface cost = $280,000 | $84,000 | 25 days |
| SUM | $389,000 | 135 days |
You have probably noted that the vendor applied different parameters for customizing and establishment of interfaces. For testing, the estimate is cross-referencing to the estimation results of the other two areas.
Conclusion
Parametric estimating can be a highly accurate approach for cost, resource requirements and duration if sufficient historical data is available and if a proven correlation exists between the parameters and the estimated values.
In practice, parametric estimation is deployed in the form of complex statistical models as well as in the straightforward form of performing 'rule of three' calculations (as shown in the examples above). Thereby, the complexity of the estimation depends on the need for accuracy, the availability and quality of historic data and the resources available for estimating.
If you are work on estimating costs, durations or resource requirements in your project, make sure that you also read our article on activity duration estimates as well as our guide to project cost estimation where we compare parametric estimation with other estimating techniques.
willoughbyhadd2001.blogspot.com
Source: https://project-management.info/parametric-estimating/
0 Response to "We Again Consider the Problem of Estimating the Parameters of a Model"
Post a Comment