Software Testing Tools Directory – Testertools.com

ISBSG Project Reality Check tool.

• What best describes the platform of your planned project?
• What best describes the primary development environment to be used?
• What best describes the development type?
• Which phases are included in your estimate?
• What is the expected size of the functionality to be delivered (in IFPUG function points)?

ISBSG is a ‘best practice’ project repository

The ISBSG Repository probably represents the best performing 25% of industry projects. As such it’s a great way to benchmark your projects against ‘best practice’. When reality checking your estimates, take this into account, especially if your expectations are rated as optimistic. Some projects might also appear to be under-performing when, compared to common practice, they may be considered ‘average’.

Sample Size (the number of projects like yours found in the database)

Sample size is important because it affects the confidence you can place in the results of statistical analysis. The larger the sample the better. With less than 10 projects the results should be treated as an indication only. A sample of 30 or more is accepted as sufficient for many forms of statistical analysis – at that level you can have about 80% confidence in the results.

Importance of assuming the correct Application Size

A primary reason that project estimates are unrealistic and that projects are deemed to have failed is that the project size was not properly understood and did not represent what was finally delivered by the project. Within the ISBSG Repository the majority of projects have been sized using a Functional Size Measure developed by IFPUG (International Function Point Users Group).

The Reality Check has been developed with an assumption that the project size has been expressed in IFPUG function points.
The accuracy of the Project Size measured early in the project will vary depending on what is known of the project’s final functionality, the method used to size (eg. detailed measure or rough estimate) and the capability of the person who measured the functional size.

Deciding how ‘Realistic’ your estimate is

If, for example, 35% of matching projects in the ISBSG Repository were delivered within your expected budget or time, that means that when positioned against the sample set of completed projects, only 35% of the matching projects completed within your estimate, (ie 65% could not be delivered within your estimate). You really want a success rate of more than 50% to have a realistic chance of meeting your estimate. The less mature your organization or more demanding your project then the higher the percentage needs to be for your estimate to be considered to be realistic. The table on Page 2 of the report shows where your project sits within the comparative sample.

Comparison Criteria and Project Risk Factors

The comparison criteria required for the tool are those requested from all organizations contributing to the ISBSG Repository. This enables the tool to search for the sub-set similar to your project for comparison purposes.
However, other environmental circumstances will have a considerable influence on the project estimate and need to be taken into account when analyzing the result.

In the development environment, risk factors such as team experience, use of an advanced toolset for the first time, strength of domain knowledge and the like will affect your ability to perform at ‘best practice’ levels.

In the project environment, risk factors such as an immoveable completion date, customer stakeholder involvement, multiple delivery sites and the like will influence the effort and duration estimates.

Accordingly, such project risk factors should be documented as having been considered in the estimate. Project Boards should take these risk factors into consideration when ‘checking the reality’ of the estimate provided against comparative ‘best practice’ projects in the ISBSG database.