Overview
Project requests are evaluated in a manner which aims to mitigate some of the differences between each request so they can be compared more accurately on a common scale. Within the University of Alaska (UA) Enterprise Service Management (ESM) Project & Portfolio (PPM) system a project request is evaluated on Goals, Risks, and Scorecard Criteria whose combination makes up the total score.
In this article:
Goals
The ESM PPM system has a list of UA organizational Goals that a project would achieve.
The score contributed to the composite score is a function of the number of goals supported by the project, which produces a number between 0 - 1. This represents the un-weighted goal score.
Goals Score = ( Number of Goals Supported ) / ( Total Number of Goals )
For example, if there are 4 total goals from which to select and a project aligns with 3 of the 4, the un-weighted goals score will be 0.75: 3 / 4 = 0.75.
Organizational Risks
Organizational Risks help evaluate the risk of the request and project against preset UA organizational risk categories to help compare organizational risk levels between projects and requests.
For each organizational risk one of the following choice values must be selected.
- Not Applicable - has a risk score of 1
- Low - has a risk score of 0.67
- Medium - has a risk score of 0.33
- High - has a risk score of 0
The total un-weighted Risk Score is calculated as follows:
Risk Score = ( Sum of all risk scores / Total number of risks )
For example, if there are 5 risks in total and 3 are N/A (each equal to 1 point), 1 risk is High (=0), and 1 risk is Medium (=.33) the un-weighted risk score would be 0.67: (1 + 1 + 1 + 0 + 0.33)/5 = 0.67.
A lower risk score means the project is more risky. Risk scores will fall between 0 and 1.
Scorecard
Project request scorecards are created supporting the different types of projects managed by the PPM system. Each scorecard define a set of criteria that is used to generate a calculated score for the project request. The criteria are presented as a series of questions, in which individuals are asked to rate the relative value of each criterion for the project request begin assessed.
The score for an individual scorecard criterion is a function of the value of the criterion, which is dependent upon the choice selected, multipled by the percent of value multiiplier.
Criterion Score = Criterion Value * Percent of Value multipler
For example, assume a criterion with a percent of value of 100%, and five (5) choices with the following values:
- Not Applicable = 0
- Choice A = 3
- Choice B = 5
- Choice C = 8
- Choice D = 10
If N/A is selected then the value contributed to the scorecard is (0 * 100%) = 0
If A is selected then the value contributed to the scorecard is (3 * 100%) = 3
If B is selected then the value contributed to the scorecard is (5 * 100%) = 5
If C is selected then the value contributed to the scorecard is (6 * 100%) = 6
If D is selected then the value contributed to the scorecard is (10 * 100%) = 10
Important
Scorecard criteria that are not listed on the scorecard will be included in the calculations and assigned a value of 0 per criterion.
The total score for the scorecard is the sum of the scores of the criteria.
Scorecard Score = Sum of all Scorecard Criterion
For example, if there are 5 criterion in total and 3 are N/A (each equal to 0 point), 1 criterion is High (=0), and 1 risk is Medium (=.33) the un-weighted risk score would be 0.67: (0 + 0 + 0 + 0 + 0.33)/5 = 0.67.
Composite Score
The Composite Score represents the total possible weighted score at the time it was scored.
The equation is as follows:
Composite Score = (Goals Score x Goals Weight) + (Risks Score x Risks Weight) + Scorecard Score
For example,
If the Goals percentage of value is 3000% within ESM, the weighted Goals score becomes 0.75 * 3000% = 22.5 for 22.5 points out of a possible 30 points.
If the Risks percent of value is 3500% within ESM, the weighted Risks score becomes 0.67 * 3500% = 23.45 for 23.45 points out of a possible 35 points.
If the Scorecard score is 51.05 points out of a possible 10 points.
The composite score calculation would be (0.75 * 3000%) + (0.67 * 3500%) + 5.10 = 51.05 for 51.05 points out of a possible 75 total points.
Composite Score %
The Composite Score % represents the relative position of the request in relation to the total possible score at the time it was scored. The equation is as follows:
Composite Score % = Composite Score / Maximum Possible Composite Score
For example, if there is a total possible score of 100 points and the project has a score of 50 then the composite score % will be 50%. If at some point in the future the scoring is changed so that the max score is 120, the composite score % helps to clarify that when the project request was originally scored it achieved 50% of the possible points, even though with today's model (120 total possible) it would be 42%. It allows for changes to the scoring model, yet still evaluates the relative score of projects scored under a different model.
Using Scores Wisely
We recommend scorecards be used as an input or factor in decision-making, but project request scores should not be the sole determining factor. For example, if project request A receives a score of 87 and project request B receives a score of 84, this does not necessarily mean that project request A should be approved over project request B.
The score should not be the only deciding factor for several reasons:
- The scorecard criteria may be missing questions or be weighted incorrectly.
- There may be qualitative factors to consider.
- Making decisions based on the score creates a large incentive for people to optimize for a high score, rather than for good project outcomes.
Scores can and should be used to band or group project requests. For example, project requests receiving a score in the lower quartile usually should not be considered.
Need additional help or have issues
For support, requests may be submitted anytime using the appropriate Enterprise Service Management form. Requests generate a Ticket which will be worked in order received and urgency by IT Employees with the knowledge and permissions to assist with the request.
For immediate assistance please review the Contact IT page for the appropriate support group.