Back to the Scottish Parliament Public Audit Committee Report
Archive Home

Business Bulletin 1999-2011

Minutes of Proceedings 1999-2011

Journal of Parliamentary Proceedings Sessions 1 & 2

Committees Sessions 1, 2 & 3

Annual reports

SP Paper 216 PAU/S3/09/R3

3rd Report, 2009 (Session 3)

Major Capital Projects

CONTENTS

Remit and membership
Report
Introduction
Background
Improving early project cost and time estimating
Project management & governance
Post project evaluation
Overall programme management
Annexe A: Extracts from the minutes of the Committee
Annexe B: Oral Evidence and associated written evidence

Remit and membership

Remit:

1. The remit of the Public Audit Committee is to consider and report on—

(a) any accounts laid before the Parliament;

(b) any report laid before or made to the Parliament by the Auditor General for Scotland; and

(c) any other document laid before the Parliament, or referred to it by the Parliamentary Bureau or by the Auditor General or Scotland, concerning financial control, accounting and auditing in relation to public expenditure.

(Standing Orders of the Scottish Parliament, Rule 6.7)

Membership:
Willie Coffey
Cathie Craigie
George Foulkes
Murdo Fraser (Deputy Convener)
Hugh Henry (Convener)
Stuart Mcmillan
Nicol Stephen
Andrew Welsh

Committee Clerking Team:

Clerk To The Committee
Tracey Reilly

Senior Assistant Clerk
Joanna Hardy

Assistant Clerk
Jason Nairn

Committee Assistant
Stuart McLean

Major Capital Projects

The Committee1 reports to the Parliament as follows—

Introduction

1. This report sets out the Committee’s findings in relation to the report Review of major capital projects in Scotland2 which was published by Audit Scotland on behalf of the Auditor General for Scotland on 24 June 2008.

Background

2. Between 2002 and 2007 the Scottish Government and its agencies, non departmental public bodies (NDPBs) and the NHS completed 43 publicly funded major capital projects valued at £811 million.3 The Scottish Government’s 2005 infrastructure investment plan identified £7.8 billion of capital investment projects for the period 2004/05 to 2007/08 and the 2008 infrastructure investment plan identifies another £10.5 billion to be invested between 2008/09 and 2010/11.

3. At the time the Auditor General for Scotland’s report was published there were 104 major capital projects in progress, valued at £4.7 billion. These projects were mostly in the transport and health sectors.

4. The Auditor General for Scotland’s report pulled together a summary of central government and NHS projects’ performance against cost, time and quality targets. It reported progress against targets for all major capital projects completed between 2002 and 2007 and a sample of projects currently in progress. It also provided a high-level assessment of project management and governance for a sample of 20 projects (five completed projects and 15 projects in progress) and outlined lessons to be learned for future projects.

Committee scrutiny
5. The Committee received a briefing on the report from the Auditor General for Scotland at its meeting on 25 June 2008. Following this briefing, the Committee agreed to take evidence from the Principal Accountable Officer.

6. At its meeting on 8 October 2008 the Committee took oral evidence from Sir John Elvidge, Permanent Secretary (and Principal Accountable Officer for the Scottish Government) and Alyson Stafford, Director of Finance, Scottish Government. Following this evidence session, the Committee corresponded with the Principal Accountable Officer on a number of issues that had been raised in evidence. The Committee considered this correspondence at its meeting on 26 November 2008 and agreed to produce a draft report on the matter.

7. The Committee’s scrutiny focussed on four main areas:

  • Improving early project cost and time estimating;
  • Project management & governance;
  • Post project evaluation; and
  • Overall programme management.

These areas are explored in more detail below.

Improving early project cost and time estimating

Auditor General for Scotland’s report
8. The Auditor General for Scotland’s report indicated that early cost and time estimates for major projects needed to improve. Of the projects considered in the report around two-fifths were completed within the initial cost estimates and one third completed on time.4

9. The report found that errors in early cost and time estimates for major projects had occurred because project managers made insufficient allowances for unforeseen problems and risks that increased costs and time. There were also issues in accounting for construction cost inflation. Some projects reviewed either did not include a specific inflation estimate or assumed that construction inflation would be the same as general inflation.5

10. The report found that there was also scope to improve other aspects of estimating cost. For example, the choice of construction method, fixtures, fittings and services may significantly affect the costs and require careful assessment during project appraisal. Explicit consideration should also be given to the effect of design choices on operational and maintenance costs after construction.6

11. The report recommended that the Scottish Government should ensure robust procurement strategies and cost estimates have been developed prior to awarding funding to projects.7

Evidence
12. The Committee asked the Principal Accountable Officer what was being done to improve early estimates of time and cost. Sir John Elvidge noted that performance in relation to estimates of costs and time was much better after contracts have been awarded. He argued that it was right that the first focus had been on controlling performance against contract costs and timing estimates at the time of entering into a contract. However, he also recognised that the area in which performance was in greatest need of improvement was the pre-contract stage.8

13. Alyson Stafford explained that the costs of a construction project could be expressed in terms of capital cost or whole life costs. Capital cost estimates are based on established systematic methods accredited by the Royal Institute of Chartered Surveyors. Expertise is also available through the construction procurement manual and the work of a team in the procurement directorate that specialises in providing advice. Alyson Stafford noted that the construction procurement manual clearly provides for the use of a base estimate and a risk allowance when estimating the capital costs of projects. This enables project teams to assess the value for money of a project in terms of the range of costs, and the relative probability of low, medium and high costs being incurred.9

14. The Treasury Green Book has also, since 2003, required an assessment of optimism bias to be built in to allow for risk and inflationary factors. However, Alyson Stafford noted that some of the projects examined in the report had started before the guidelines were introduced.10

15. Alyson Stafford explained that the estimating processes for whole life costs were much more diverse and that a standardised method had only been published this year. She stated that now that this methodology was available, all new projects taken forward will bring in whole life costings as part of their estimates. As this methodology is only applied at the planning stage it appears that it will not be applied to projects already underway.11 However, where whole life costings have been developed they will be regularly reviewed until project completion to ensure that forecasts remain in line with the original cost estimates for the purposes of budgeting and demonstrating value for money.12

16. Sir John Elvidge explained that the Scottish Government was aiming to extend and systematise the gathering of both estimated and outturn costs with the aim of improving estimates. The Scottish Government’s Infrastructure Investment Group (IIG) will also develop its work on reviewing estimated project costings to assess what improvements can be made and what further action it would be appropriate to take across the Scottish Government.13

17. Alyson Stafford noted that improving time estimates would be more difficult to address effectively. However, she drew attention to the provisions of the construction procurement manual which give advice on the factors to be taken into account when considering a project’s duration. She noted the importance of ensuring that an assessment is carried out rigorously at the early stages of a project’s development. This will be achieved through the work of the IIG.14

18. The IIG brings together a range of senior officials, at director level, from across the Scottish Government and its agencies. The Group takes a strategic view of infrastructure issues across the Scottish Government including major capital projects and the forward investment programme. The Principal Accountable Officer stated that the IIG’s work has helped to increase awareness in directorates across the Scottish Government of policy issues which have an impact on infrastructure investment and encouraged early planning to promote opportunities to co-locate investments. The Group also oversaw the production of the Scottish Government’s 2008 infrastructure investment plan.15

19. The Committee welcomes the Scottish Government’s commitment to make project cost and time reporting more systematic. It notes that the IIG will review estimated and outturn costs to determine what further action needs to be taken to improve early costings. The Committee further notes that it is hoped that the IIG work will improve the accuracy of time estimates for projects. The Committee wishes to be kept informed of progress on this and requests that the Scottish Government update the Committee at the end of 2009 on the IIG work and the actions taken.

Project management & governance

Auditor General for Scotland’s report
20. The Auditor General for Scotland’s report found that project management and governance arrangements for individual projects were broadly effective but that the quality of project appraisals could be improved.16

21. The report noted that a range of approaches to project management exist. Bodies with large investment programmes like the Scottish Prison Service and Transport Scotland have established project teams with significant practical experience and knowledge. Bodies with more limited experience of major projects employed individuals and project management companies to bring the skills needed to help deliver their projects.17

22. Gateway reviews can also be used to provide an independent stock-take at key points in a project, with an opportunity to identify and correct any deficiencies. However, these reviews are not always done. Since 2005, 44 gateway and similar reviews for 20 major construction projects have been completed.18

Evidence
23. The Committee asked how the Scottish Government ensured that sufficient skills exist in smaller bodies to manage such projects. Sir John Elvidge explained that many smaller organisations did not have such expertise and that they were not seeking to build expertise in every organisation, because such expertise could be redundant some of the time. Instead, the Scottish Government’s approach was to strengthen the central expertise that smaller organisations could draw on and the connections between that central expertise and smaller organisations. In addition, smaller bodies must use appropriate external help at the right stage.19

24. Since 2005, it has also been mandatory for all high risk or mission critical projects worth more than £5 million to go through a “gateway” review, which is a 5 stage process involving regular reviews at critical stages of a project’s development. These stages are business justification, delivery strategy, investment decision, readiness for service and gateway 5, which is the operations review and benefits realisation which is conducted post delivery. A mixture of peer and external review is involved in the gateway process and the Scottish Government has a pool of more than 200 gateway reviewers from a broad range of backgrounds. Gateway review teams are structured in a way which allows them to use independent expertise, as they are led by full time reviewers from outside the Scottish Government.20 Teams have two or three additional members who are usually senior public sector managers or functional specialists. Further external expertise can be recruited if necessary.21

25. A total of 21 capital projects currently underway are the subject of gateway reviews, with an estimated capital value of around £8 billion.22 The Scottish Government is currently reviewing the criteria used to determine which projects are subject to gateway reviews to ensure that the review resources are targeted effectively on the most complex and high risk projects.23

26. Sir John Elvidge noted that the Scottish Government was in a position to provide the Committee with strong assurances about the best practice guidance that exists for the planning and delivery of projects.24 However, there are issues regarding the enforcement of such guidance across the public sector. He explained that within central government, the line management chain provides a straightforward way of ensuring that the guidance is followed. The gateway process is used to reinforce that management chain with results being reported to the project team, relevant line managers and the accountable officer for that budget line. If a review flagged up that guidance was not being followed then senior managers could move in and deal with the situation.25

27. For projects outside the Scottish Government, Sir John Elvidge noted that wider issues arose about the relationship between central government and NDPBs, such as the meaning of the arm’s length principle.26 Alyson Stafford explained that the IIG comprises lead directors across all portfolio areas, each of whom will have a relationship with a number of arm’s length bodies. The IIG aims to ensure that directors are more closely linked to the work of these bodies, to consider the budget management issues arising from the management of the capital programme across government and to examine the standards and disciplines used by arm’s length bodies.27

28. Alyson Stafford went on to explain that each director has been given specific directions in relation to their accountability. Each director must, in providing assurances to their own accountable officer and the principal accountable officer, address how they interact with those bodies and ensure that the funds being disbursed are in line with relevant policies, processes and systems. Specific skills development courses also exist for senior staff to ensure that the early stages of a project are well supported and competencies in these areas have been defined. She explained that “As well as being clear about where the responsibility sits and how people should carry out that role, we are ensuring that they have the tools to do that”.28

29. The Committee also asked what information the Principal Accountable Officer received on the progress of major projects and any significant risks to delivery. Sir John Elvidge explained that he would not routinely be made aware of developing problems in a project, except in the area in which he is a budget accountable officer. He noted that although he spoke to all accountable officers regularly, the warning of a red gateway assessment (which means that corrective action is required) would not automatically be passed on. He also noted that any alerting would often take place in conversation rather than being on paper.29

30. He stated that the purpose of having designated accountable officers was so that they could take responsibility for these issues and tell him only in the event of a particularly acute problem. He went on to note that “there is no institutional complacency … there is no realistic prospect of an accountable officer concealing from me a problem that has not been addressed”.30 He later wrote to the Committee on this point stating that “It is the responsibility of Directors General to take the necessary action to manage the delay or cost risk effectively and to alert Ministers, and me, where the significance of the risk would make it appropriate to do so.”31

31. He noted that the work of the IIG and the regular reports prepared by the Director of Finance would pick up any major capital project where the budget or management was in difficulty.32 Alyson Stafford also emphasised the role of the gateway process, saying that it was there as a review mechanism to keep projects on track.33

32. The Committee asked Sir John Elvidge how often he had been informed of significant risks to projects over the past two years and what form this advice had taken. He responded that he had focused his review of records and personal recollections on those projects with a cost of above £20 million and which had been subject to the gateway process in the last two years. 10 capital projects fell into this category. He stated that “From a review of files and my own recollections, I believe that 4 of these projects have been brought to my attention since 2006, either in the form of written submissions or in the course of regular meetings. It would be misleading to suggest that each project was presented to me as being at significant risk, but my recollection is that each was discussed in advance of key decisions being taken about subsequent phases of each project.”34

33. The Committee welcomes the use of the gateway process to review high risk or mission critical projects worth more than £5 million. A high proportion by value of projects currently benefit from this process. But the proportion by number is low – only around one in five current major projects have been reviewed. The Committee therefore also welcomes the Scottish Government’s current review of the criteria for deciding when a gateway review is needed.

34. It is not entirely clear what processes the Scottish Government uses to monitor and challenge the four-fifths of projects that currently do not benefit from gateway reviews. The Committee recommends that the Scottish Government adjust the criteria for gateway reviews to ensure that, where it is warranted, projects first assessed as lower risk which start to exhibit problems are promoted to full gateway procedure.

35. The Committee requests that the Scottish Government update the Committee at the end of 2009 on the results of their review of the gateway criteria, including how far the proportion of projects receiving reviews may increase.

36. The Committee notes how the Permanent Secretary and Directors General ensure accountability if any project gets into difficulty. However, the Committee believes that reporting in this area, especially in relation to arm’s length bodies, is often inadequate and piecemeal in nature. The Committee does not consider it sufficient for this reporting to take place by way of conversations between the Permanent Secretary and Accountable Officers. To reinforce accountability the Committee recommends that the Scottish Government consider adopting formal reporting and audit trails – based on systematic time and cost reporting – to ensure that Accountable Officers receive timely information about any project which may get into difficulty and can take whatever corrective action is necessary.

Post project evaluation

Auditor General for Scotland’s report
37. The Auditor General for Scotland’s report noted that business cases should define the outputs and outcomes from a project. This helps ensure value for money, provides a base to manage any changes during implementation and will help to realise intended benefits. However, the report found that business cases were not available for some projects and others had not been kept up to date.35

38. The report also found that, while the Scottish Public Finance Manual requires project teams to complete post-project evaluations, this did not happen for 21 of the 43 completed projects.36

39. The report recommended that the Scottish Government should collect information on all projects and get explanations for cost, time and quality changes, and lessons learned. It also recommended that this information should be reported publicly. In addition, the report recommended that all public bodies should carry out post project evaluations within a reasonable timescale to determine whether projects have delivered the benefits intended.37

Evidence
40. The Committee asked what systems the Scottish Government had in place to monitor that it did in fact receive the promised benefits from the projects that it funds. Sir John Elvidge noted that there was a difference between the post project evaluation, which evaluates how well the process was planned and managed, and post occupancy evaluation which was about whether the project delivered the promised benefits.38

41. He noted that the Auditor General for Scotland’s report had flagged up that the need to improve was strongest at the post project evaluation stage. He drew a distinction between having the guidance and enforcing the use of guidance. Current guidance provides that post project evaluation should be mandatory.39

42. He stated that “the framework is clear and the guidance makes it clear that those processes should be followed. However, the evidence tells us that compliance is not nearly as good as it should be.”40

43. In this respect, Alyson Stafford again drew attention to the work of the IIG and the use of the gateway process. She noted that the fifth gateway review looked at the achievement of business case objectives; the benefits against forecast benefits and other benefits; the extent to which there is continued alignment with the business strategy; the effectiveness of the actual improved business operations; the ways in which to maximise benefits and minimise whole life costs; risk assessment going forward; and business and user satisfaction. She noted that post occupancy reviews would continue to take place over the operational life of a facility.41 These evaluations will continue to feed back into the process in order to inform future projects. In written evidence following the evidence session Sir John Elvidge told the Committee that he had asked that the IIG take action to ensure that evaluations are produced, that the key lessons from them are reviewed, and that information is shared systematically among project teams to support continuous improvement.42

44. The Committee believes that post project evaluation is an essential element of good project practice. The Committee regards the current lack of compliance with the requirements of the Scottish Public Finance Manual as unacceptable. Accordingly, the Committee welcomes the assurance from the Scottish Government that it has tasked the IIG with improving the compliance rate. The Committee believes that there needs to be significant improvement in this area and requests that the Scottish Government update it on progress made at the end of 2009.

Overall programme management

Auditor General for Scotland’s report
45. The Scottish Executive set up the overarching IIG in 2006. However, the Auditor General for Scotland’s report noted that there is currently no reporting mechanism that brings together current progress on all major projects across the government. The Scottish Executive also developed a database of major projects, but there are no plans to report publicly on the progress of individual projects.43

46. The report also found that the Scottish Government does not have standardised systems for reporting and appraising the current status of projects and how they are performing against cost, time and quality targets.44

47. The report noted the Scottish Government proposal to develop the Scottish Futures Trust initiative (SFT) which includes the creation of the Scottish Futures Trust Ltd Delivery and Development. This is intended to introduce greater efficiency in major investment through greater scrutiny, oversight and coordination across the public sector.45

48. The Auditor General for Scotland’s report recommended that the Scottish Government should strengthen strategic direction and investment planning through stronger, government wide coordination and providing more constructive challenge of projects, to help deliver them better.46

Evidence
49. The Committee asked the Principal Accountable Officer what plans there were for the Scottish Government to publicly report on projects. Sir John Elvidge noted that the development of the infrastructure investment plans was a starting point which provided an overall picture of investment activity. He noted that publication of further information raised issues regarding the quality of the data for individual projects and the amount of information that could be placed in the public domain.47

50. He stated that the IIG was in the process of developing the database on major capital projects and that this will chart such information. However, although this information will be available internally, commercial confidentiality issues must be considered to ensure that the Scottish Government can continue to get the best value for money. Alyson Stafford undertook to consider further whether information could be published in a similar manner to the way outcomes are published on the “Scotland performs” website.48 In written evidence, Sir John Elvidge told the Committee that he had asked the IIG to review what more can be done to provide regular updates on progress with capital projects and undertook to revert to the Committee as soon as the IIG brings forward firm proposals.49

51. The Committee also asked about the information provided to Scottish Ministers as a result of the gateway process and, in more general terms, on the management of the overall capital projects budget. Sir John Elvidge noted that this information was not monitored centrally as it adapted to the preferences of Ministers.50 Ministers do not routinely receive a summary of progress on all capital projects because the Scottish Government have not identified a need for routine reporting of this kind. However, the Principal Accountable Officer told the Committee he would keep the possible requirement for such reporting under review. 51

52. The IIG also receives a breakdown on the capital budget for programmes of activity, with specific attention on programmes within each area of £50 million or more.52 Alyson Stafford noted that discussion occurs with directors to ensure that the figures match with delivery and to ensure that no gaps occur between perception and delivery. A brief commentary on each of the programmes is then provided to the strategic board.53

53. The Committee also asked whether systems existed to ensure that priority was given to projects within the capital investment programme that would provide direct benefits. Alyson Stafford noted that investment in individual capital projects was linked to budgetary decisions about what money is allocated across portfolios. These decisions are made by Ministers. She went on to say that it is up to Cabinet to decide on the mix and the prioritisation across and within portfolios.54

54. Prioritisation is achieved though the Scottish Government’s structure of objectives with the single overarching priority of economic growth and a series of targets and strategic objectives that relate back to that priority. Projects are judged against their contribution to those objectives.55

55. The Committee also asked whether proposals received robust independent challenge before being accepted into the capital investment programme. Sir John Elvidge described the challenge process internally through mechanisms such as the finance directorate and the work of the IIG as being increasingly robust. However, he noted that there was no routine external challenge.56

56. Sir John Elvidge noted that there was scope to do more to strengthen the strategic direction and investment planning through co-ordination and increased challenge. However, he drew attention to the question of proportionality and the need to demonstrate that the information required was proportionate to the value received from it. In his view, that judgment must hinge on an assessment of how many projects go wrong that would have been improved by requiring increased or more regular information flow.57

57. He also noted that one of the objectives of the Scottish Futures Trust will be to increase the efficiency of procurement by aggregating procurement activity and that it will also move on to consider a number of wider issues as it develops as an organisation.58

58. The Committee welcomes the commitment by the Principal Accountable Officer to examine whether, and in what form, regular updates on the progress of major capital projects can be published and his promise to revert to the Committee on this. The Committee requests that the Scottish Government update it on the progress on this issue as soon as possible or in any event before the end of June 2009.

59. The Committee recognises that the ultimate decision on the priority allocated to major projects is for Ministers. However, the Committee remains unclear as to the objective information provided by officials to assist Ministers in their decisions. The Committee therefore requests that Scottish Government provide the Committee with details of the types of information used to inform project priority in various sectors.

60. The Committee notes that the development of the Scottish Futures Trust aims to increase the efficiency of the procurement process and assist the spread of best practice in this area, and requests that the Scottish Government report back to the Committee on progress in this area in 12 months time.

Annexe a: EXTRACTS FROM THE MINUTES

11th Meeting, 2008 (Session 3) Wednesday 25 June 2008

Major capital projects: The Committee received a briefing from the Auditor General for Scotland on his report entitled “How Government works: review of major capital projects in Scotland”. (AGS/2008/6).

Consideration of approach: The Committee considered its approach to the Auditor General for Scotland’s report entitled “How Government Works: Review of major capital projects in Scotland”. (AGS/2008/6). The Committee agreed to invite the Accountable Officer to give oral evidence to the Committee.

14th Meeting, 2008 (Session 3) Wednesday 8 October 2008

Section 23 report: The Committee took evidence on the Auditor General for Scotland's report entitled "A review of major capital projects" from—

Sir John Elvidge, Permanent Secretary, and Alyson Stafford, Director of Finance, Scottish Government.

Consideration of evidence (in private): The Committee considered the evidence received at agenda item 3. The Committee noted the witnesses' undertaking to provide more details on some of the issues raised during evidence.

17th Meeting, 2008 (Session 3) Wednesday 26 November 2008

Section 23 report: The Committee considered a response from the Accountable Officer on the Auditor General for Scotland's report entitled "A review of major capital projects". The Committee agreed to consider a draft report at a future meeting.

3rd Meeting, 2009 (Session 3) Wednesday 4 February 2009

Section 23 report (in private): The Committee considered a draft report on the Auditor General for Scotland's report entitled "A review of major capital projects". The Committee agreed to publish the report, subject to revisions which will be circulated for agreement by correspondence.

Annexe b: oral evidence and associated written evidence

AUDIT COMMITTEE OR 25 JUNE 2008

AUDIT COMMITTEE OR 8 OCTOBER 2008

LETTER TO SIR JOHN ELVIDGE, PERMANENT SECRETARY, THE SCOTTISH GOVERNMENT, FROM THE CONVENER OF THE AUDIT COMMITTEE, 13 NOVEMBER 2008

I am writing to thank you for giving oral evidence to the Audit Committee at its meeting on 8 October 2008. The Committee was grateful for your attendance and your willingness to accommodate the additional evidence in relation to NHS Western Isles.

I note that you offered to revert to the Committee on a number of points in relation to the AGS report on major capital projects and these are summarised below for convenience. In addition, several questions that were not able to be asked due to pressure of time are also included below, and the Committee would be grateful for a written response to these queries.

Cost Estimating

  • What will the Scottish Government do to ensure decision-makers get more realistic early costings, for example, has consideration been given to requiring a range rather than a point estimate?
  • What evidence are you collecting to assure you of continuing improvement in the development of more realistic early costings?
  • What is Scottish Government doing to ensure that whole life costings are applied to projects that are already underway?

Gateway Reviews

  • Can you provide the committee with more information on the project management background and wider skill sets of the people involved in conducting gateway reviews?
  • How many and what value of projects currently underway have been examined as part of a gateway review?
  • What are you doing to ensure that the coverage of gateway reviews is adequate across the entire investment programme?

Project Management & Evaluation

  • From what you told us, the Infrastructure Investment Group has a major role in project monitoring and planning. Can you give us additional information on its composition, remit, programme and main achievements?
  • How many projects were brought specifically to your attention as at significant risk in the last two years?
  • When were these projects brought to your attention and what form did this advice take?
  • Specifically, can you tell us when and how you were first advised about the problems on Stirling-Alloa-Kincardine, and any action you took?
  • How does the Scottish Government gather and use evidence from post project evaluations to understand better project performance and to share experience and best practice?

Provision of Information on projects

  • What work is being done to allow the Scottish Government and other public bodies to report publicly on the progress of projects? When do you expect decisions on this issue to be made and are there any key milestones you have identified in this process?
  • What information is routinely provided to Scottish Ministers on the progress of projects and what form does this information take?

Prioritising Projects

  • How does Scottish Government test and show that it is achieving a good balance across the investment programme?
  • What criteria does Scottish Government use to decide the priority for projects?

I would be grateful for a response by close of Friday 31 October. Please copy your response to Tracey Reilly, Clerk to the Audit Committee, Room T3.60, The Scottish Parliament, Holyrood, Edinburgh, EH99 1SP email:

audit.committee@scottish.parliament.uk. Should you require any further information please do not hesitate to contact the Clerk.

LETTER FROM SIR JOHN ELVIDGE, PERMANENT SECRETARY, THE SCOTTISH GOVERNMENT, TO THE CONVENER OF THE AUDIT COMMITTEE, 13 NOVEMBER 2008

Thank you for your letter of 10 October. I am happy to provide responses to the questions you raise on the AGS report on major capital projects. I apologise for the delayed response – some of the Committee’s questions required a broad search of records. The AGS report concentrated on conventionally-procured projects and my answers are similarly focused.

For ease of reference I have repeated the Committee’s questions below in bold, followed by my responses.

What will Scottish Government do to ensure decision-makers get more realistic early costings, for example, has consideration been given to requiring a range rather than a point estimate?

Mandatory guidance provided in the Scottish Government’s Construction Procurement Manual emphasises the need for clarity about the scope of capital projects, and for adequate cost estimates (both capital costs and whole-life costs) to be provided including appropriate allowance for risk and for optimism bias. As Alyson Stafford explained at the hearing on 8 October, the guidance requires project teams to develop cost estimates consisting of a base estimate and a risk allowance, both of which must be based on a systematic analysis of factors bearing on the project. This enables project teams to assess the value for money of a project in terms of the range of costs, and the relative probability of low, medium and high costs being incurred. The additional, recently-developed standardised approach to life cycle costing published by the British Standards Institution will assist project teams across the Scottish Government to further improve the quality and consistency of their overall whole-life costing estimates.

What evidence are you collecting to assure you of continuing improvement in the development of more realistic early costings?

Business areas examine and compare estimated and outturn costs with the aim of improving cost estimating and we aim to extend and systematise the gathering of such information more widely to all capital projects. As the Audit Scotland report pointed out, the accuracy of project capital cost estimates has been improving, but there is always room for further improvement. Alyson Stafford and I will ensure that the Infrastructure Investment Group (IIG) develops its work on reviewing estimated project costings, including early stage costings, to assess how further improvements can be made in preparing these and what further action it would be appropriate to take across the Scottish Government.

What is Scottish Government doing to ensure that whole life costings are applied to projects that are already underway?

Capital project teams across Scottish Government are required by the Construction Procurement Manual to carry out whole-life costing on projects at the planning stage, to enable them to carry out a systematic examination and comparison of costs of alternative approaches. The Manual describes aspects of undertaking whole-life costings. Once a project is under way, meaning that a decision has been made to invest in a particular solution to meet a business need, the role of whole-life costings in informing choices about project options is complete. However whole-life costings should be regularly reviewed until project completion to ensure that forecasts remain in line with the original cost estimates for the purposes of budgeting and demonstrating value for money.

Can you provide the committee with more information on the project management background and wider skill sets of the people involved in conducting gateway reviews?

All reviews are led by experienced Review Team Leaders who are typically full time reviewers, drawn from outside the Scottish Government using framework contracts. Review Team Leaders have considerable experience of project and programme delivery in the public and private sectors, and bring extensive knowledge and skills in areas such as risk management, benefits management, managing business change, construction, transport infrastructure and contract management.

Teams have 2 or 3 additional members, who are usually senior public sector managers and / or functional specialists who are released temporarily from their jobs to contribute to specific reviews. When review teams are being selected, the skills and knowledge of the team members will as far as possible reflect the key issues faced by the project under review. External review team members can be appointed where a particular skill set is not available internally.

The Scottish Government’s pool of approximately 200 review team members have a wide range of skills accumulated in a variety of middle and senior management roles, primarily in the public sector. The majority of reviewers have practical experience in project or programme delivery; policy delivery; financial management; risk management; managing business change; IT; construction; or procurement.

How many and what value of projects currently underway have been examined as part of a gateway review?

A total of 21 capital projects currently under way are the subject of gateway reviews, with an estimated capital value of around £8 billion.

What are you doing to ensure that the coverage of gateway reviews is adequate across the entire investment programme?

All Senior Responsible Owners of capital projects are required by current guidance to follow the Scottish Public Finance Manual, which states that all capital projects with a total budget exceeding £5 million must be assessed in terms of their level of risk. Where projects are assessed as high risk or mission critical, a formal gateway review must be undertaken at each key decision stage. The Scottish Government is currently reviewing the criteria used to determine which projects are subject to gateway reviews, to help ensure that review resources are targeted effectively on the most complex and high risk projects.

From what you told us, the Infrastructure Investment Group has a major role in project monitoring and planning. Can you give us additional information on its composition, remit, programme and main achievements?

The IIG brings together a range of senior officials at Director level from across the Scottish Government and its agencies. The Group takes a strategic view of infrastructure issues across the Scottish Government including major capital projects and the forward investment programme. Attached are: a current membership list; the Group’s remit, and the agendas for the last 2 meetings. IIG meets approximately 4 times a year.

The Group’s work has helped to increase awareness in Directorates across the Scottish Government of policy issues which have an impact on infrastructure investment such as the introduction of the new National Planning Framework and environmental sustainability issues. It has also encouraged early planning to promote opportunities to co-locate investments. The Group provides a high level forum for discussing and agreeing how these issues should be taken forward.

The Group has commissioned and introduced a major capital projects database, facilitated better expenditure control which helped achieve a very small capital underspend in 2007-08 (of £2m on a budget of £3.5 billion), and oversaw the production of the Scottish Government’s March 2008 Infrastructure Investment Plan.

We are clear that IIG must build on its existing achievements and develop further momentum in providing assurance on our performance across the range of capital projects undertaken by the Scottish Government.

How many projects were brought specifically to your attention as at significant risk in the last two years? When were these projects brought to your attention and what form did this advice take?

As I explained to the Committee in my oral evidence, I expect Directors General, as accountable officers, to ensure that they receive systematic reports on capital projects in their areas so that they are alerted to significant risks in terms of delay and overspending. It is the responsibility of Directors General to take the necessary action to manage the delay or cost risk effectively and to alert Ministers, and me, where the significance of the risk would make it appropriate to do so.

I would not normally expect Directors General to draw my attention to small projects and have therefore focused my review of records and personal recollections on projects with a cost of above £20 million and which have been subject to the Gateway process in the last two years. 10 capital projects fall into this category. From a review of files and my own recollections, I believe that 4 of these projects have been brought to my attention since 2006, either in the form of written submissions or in the course of regular meetings. It would be misleading to suggest that each project was presented to me as being at significant risk, but my recollection is that each was discussed in advance of key decisions being taken about subsequent phases of each project.

Specifically, can you tell us when and how you were first advised about the problems on Stirling-Alloa-Kincardine, and any action you took?

In the case of transport projects, because of the role of Transport Scotland as an Agency of the Scottish Government, there is a layer of senior level governance that goes above and beyond that for other projects. The Chief Executive of Transport Scotland is the responsible Accountable Officer, reporting to the Director General Economy. I believe I was first advised of potential cost overruns on the Stirling-Alloa-Kincardine rail project in the Autumn of 2007, in conversation with the Director General Economy. I satisfied myself that the relevant Director General, Andrew Goudie, and the Chief Executive of Transport Scotland were taking the necessary action to manage these risks, in concert with the Project Board.

How does the Scottish Government gather and use evidence from post project evaluations to understand better project performance and to share experience and best practice?

As Alyson Stafford and I indicated to the Committee on 8 October, we are committed to ensuring that the guidance on post-project evaluations and post-occupancy evaluations set out in the Scottish Public Finance Manual and the Construction Procurement Manual is followed. We indicated that through the gateway review process, there is a mechanism for making an independent check on whether these larger and higher risk projects have produced post-project evaluations and post-occupancy evaluations. I have asked that the IIG take action to ensure that evaluations are produced, that the key lessons from them are reviewed, and that information is shared systematically among project teams to support continuous improvement.

What work is being done to allow the Scottish Government and other public bodies to report publicly on the progress of projects? When do you expect decisions on this issue to be made and are there any key milestones you have identified in this process?

As Alyson Stafford and I explained to the Committee, information is already made available to the public and the Parliament about infrastructure investment projects. The Committee wanted to know what further information could be made publicly available about progress on projects. The Committee accepted that some project-related information would be commercially confidential, but members pointed out that Scotland Performs provided a potential model for presenting performance information in an accessible form. We undertook to look further into this and respond. I have asked that the IIG review what more can be done to provide regular updates on progress with capital projects, and I will revert to the Committee as soon as IIG brings forward firm proposals. I will aim to provide a progress report in the new year.

What information is routinely provided to Scottish Ministers on the progress of projects and what form does this information take?

Individual Directorates and agencies ensure that Portfolio Ministers are kept abreast of progress with major projects in their portfolios. In addition, the Cabinet Secretary for Finance and Sustainable Growth has an over-arching interest in infrastructure investment issues and is kept informed about progress on critical projects. However Ministers do not routinely receive a summary of progress on all capital projects because we have not identified a need for routine reporting of this kind. We will keep the possible requirement for such reporting under review.

How does Scottish Government test and show that it is achieving a good balance across the investment programme?

The Government’s Economic Strategy and its supporting national outcomes provide the strategic direction and national performance framework within which the capital investment programme is developed and to which it contributes. Ministers collectively make decisions about aggregate levels of resources, including capital resources, to be applied to individual Portfolios. Within these aggregate spending decisions, Portfolio Ministers decide on relative priorities and therefore take views on the balance of capital spending required. The Infrastructure Investment Plan first published in 2005 and revised and re-published in March this year provides readily accessible information on the full range of capital projects proposed by the Scottish Government. In addition, the Government retains flexibility to respond to changing circumstances and needs, to ensure that the capital investment programme continues to provide balance across its priorities.

What criteria does Scottish Government use to decide the priority for projects?

As explained above, the key criteria used by Ministers in deciding on relative priorities for capital investment projects are derived from the Economic Strategy and national outcomes. In addition, there are operational criteria specific to individual Portfolios such as (for example) traffic forecasts and risk and safety assessments; building condition surveys; and NHS Boards’ infrastructure strategies.

I hope that the Committee finds these observations helpful. I or Alyson Stafford would be very happy to provide the Committee with further information if necessary.

Attachments: Infrastructure Investment Group: current membership list, remit, and agendas for the last 2 meetings

INFRASTRUCTURE INVESTMENT GROUP

Current Membership List – September 2008

Chair: Scottish Government Director of Finance Alyson Stafford
Director, Transport John Ewing
Director, Housing and Regeneration Mike Foulis
Director, Environmental Quality John Mason
Director, Schools Colin MacLean
Director, Lifelong Learning Andrew Scott
Director, Health Finance John Matheson
Head, Health Private Finance & Capital Unit Mike Baxter
Chief Planner Jim Mackinnon
Chief Executive, Transport Scotland Malcolm Reed
Director, Finance and Corporate Services, Transport Scotland Guy Houston
Director of Funding, Scottish Funding Council Riona Bell
Director of Finance, Scottish Prisons Service Willie Pretswell
Head of Efficient Government Unit Craig Russell
Director, Scottish Procurement Directorate Nick Bowd
Scottish Futures Trust rep To be Confirmed
Chief Quantity Surveyor/Gateway Director Alastair Wyllie
Office of Chief Economic Adviser Gary Gillespie
Deputy Director of Finance Alistair Brown
Finance Directorate - Infrastructure Investment Team (Secretary) Alan Lister

INFRASTRUCTURE INVESTMENT GROUP – REMIT

1) To consider strategic issues relating to infrastructure investment and, where appropriate, make recommendations for change to the Strategic Board and the Cabinet Secretary for Finance and Sustainable Growth;

2) to oversee the refinement and implementation of the IIG proposals to address the action points relating to planning and delivery identified in the Executive’s Infrastructure Investment Plan (IIP);

3) to ensure that projects are completed, and deliver on the ground results which are consistent with the original policy objectives;

4) to ensure that annual variances from planned expenditure, whether underspends or overspends, are kept to a minimum through the effective monitoring and use of investment funds;

5) to make use of the Group’s wide-ranging portfolio responsibilities to maximise investment synergies; and

6) to review the need for enhancing the corporate machinery for dealing with infrastructure investment after SR07.

INFRASTRUCTURE INVESTMENT GROUP MEETING 27 FEBRUARY 2008
Agenda

1. “Sustainability Assessment”
Presentation by Professor Jan Bebbington, Vice-chair (Scotland), Sustainable Development Commission

2. Cross- compliance and climate change
Presentation by Cameron Maxwell

3. Note of last meeting (Paper 1)

4. Matters arising not covered elsewhere

National Planning Framework

Scottish Futures Trust

IFRS impact on capital programmes

Strategic issues
5. Infrastructure Investment Plan

Capital expenditure
6. Spending 2007-08

7. AOB

INFRASTRUCTURE INVESTMENT GROUP MEETING 6 MAY 2008
Agenda

1. Note of last meeting (Paper 1)

2. Matters arising not covered elsewhere – updates on

National Planning Framework

Scottish Futures Trust

IFRS impact on capital programmes

Strategic issues
3. IIP, longer term vision for investment plans and progress monitoring
(Paper 2)

4. Public Procurement Reform Agenda (Paper 3)

5. Finance forward look

6. Audit Scotland Report on Major Capital Projects

Capital expenditure
7. Outturn spending 2007-08

8. Spending 2008-09

9. AOB


Footnotes:

1 On 10 December 2008 the Parliament agreed to a change of name and remit for the former Audit Committee which is now called the Public Audit Committee.

2 Audit Scotland (2008) Review of major capital projects in Scotland, available at http://www.audit-scotland.gov.uk/docs/central/2008/nr_080624_major_capital_projects.pdf (accessed 13 December 2008)

3 A major capital project is defined as having a capital cost of £5 million or more.

4 Audit Scotland (2008) Review of major capital projects in Scotland pp 12-14

5 Audit Scotland (2008) Review of major capital projects in Scotland p 29

6 Audit Scotland (2008) Review of major capital projects in Scotland p 30

7 Audit Scotland (2008) Review of major capital projects in Scotland p 4

8 Scottish Parliament Audit Committee, Official Report 8 October 2008, col 648

9 Scottish Parliament Audit Committee, Official Report 8 October 2008, col 648-9 and Scottish Government Letter to the Convener of the Audit Committee, 13 November 2008

10 Scottish Parliament Audit Committee, Official Report 8 October 2008, col 649

11 Scottish Parliament Audit Committee, Official Report 8 October 2008, col 649

12 Scottish Government Letter to the Convener of the Audit Committee, 13 November 2008

13 Scottish Government Letter to the Convener of the Audit Committee, 13 November 2008

14 Scottish Parliament Audit Committee, Official Report 8 October 2008, col 649

15 Scottish Government Letter to the Convener of the Audit Committee, 13 November 2008

16 Audit Scotland (2008) Review of major capital projects in Scotland p 27

17 Audit Scotland (2008) Review of major capital projects in Scotland p 32-33

18 Audit Scotland (2008) Review of major capital projects in Scotland p 33

19 Scottish Parliament Audit Committee, Official Report 8 October 2008, col 653

20 Scottish Parliament Audit Committee, Official Report 8 October 2008, col 646 and 650

21 Scottish Government Letter to the Convener of the Audit Committee, 13 November 2008

22 Scottish Government Letter to the Convener of the Audit Committee, 13 November 2008

23 Scottish Government Letter to the Convener of the Audit Committee, 13 November 2008

24 Scottish Parliament Audit Committee, Official Report 8 October 2008, col 654

25 Scottish Parliament Audit Committee, Official Report 8 October 2008, col 654

26 Scottish Parliament Audit Committee, Official Report 8 October 2008, col 655

27 Scottish Parliament Audit Committee, Official Report 8 October 2008, col 655

28 Scottish Parliament Audit Committee, Official Report 8 October 2008, col 656

29 Scottish Parliament Audit Committee, Official Report 8 October 2008, col 657

30 Scottish Parliament Audit Committee, Official Report 8 October 2008, col 658

31 Scottish Government Letter to the Convener of the Audit Committee, 13 November 2008

32 Scottish Parliament Audit Committee, Official Report 8 October 2008, col 658

33 Scottish Parliament Audit Committee, Official Report 8 October 2008, col 659

34 Scottish Government Letter to the Convener of the Audit Committee, 13 November 2008

35 Audit Scotland (2008) Review of major capital projects in Scotland p 16 and 28

36 Audit Scotland (2008) Review of major capital projects in Scotland p 16

37 Audit Scotland (2008) Review of major capital projects in Scotland p 17

38 Scottish Parliament Audit Committee, Official Report 8 October 2008, col 661

39 Scottish Parliament Audit Committee, Official Report 8 October 2008, col 661

40 Scottish Parliament Audit Committee, Official Report 8 October 2008, col 661

41 Scottish Parliament Audit Committee, Official Report 8 October 2008, col 661-662

42 Scottish Government Letter to the Convener of the Audit Committee, 13 November 2008

43 Audit Scotland (2008) Review of major capital projects in Scotland p 6

44 Audit Scotland (2008) Review of major capital projects in Scotland pp 32-33

45 Audit Scotland (2008) Review of major capital projects in Scotland p 34

46 Audit Scotland (2008) Review of major capital projects in Scotland p 4

47 Scottish Parliament Audit Committee, Official Report 8 October 2008, col 662

48 Scottish Parliament Audit Committee, Official Report 8 October 2008, col 664 and Scottish Government Letter to the Convener of the Audit Committee, 13 November 2008

49 Scottish Government Letter to the Convener of the Audit Committee, 13 November 2008

50 Scottish Parliament Audit Committee, Official Report 8 October 2008, col 665

51 Scottish Government Letter to the Convener of the Audit Committee, 13 November 2008

52 Scottish Parliament Audit Committee, Official Report 8 October 2008, col 665

53 Scottish Parliament Audit Committee, Official Report 8 October 2008, col 666

54 Scottish Parliament Audit Committee, Official Report 8 October 2008, col 666

55 Scottish Parliament Audit Committee, Official Report 8 October 2008, col 669

56 Scottish Parliament Audit Committee, Official Report 8 October 2008, col 666

57 Scottish Parliament Audit Committee, Official Report 8 October 2008, col 668

58 Scottish Parliament Audit Committee, Official Report 8 October 2008, col 670