ATC130916: Report of the Standing Committee on Appropriations on its study tour to the United States of America and Canada from 29 September to 10 October 2012, Dated 31 July 2013

Report of the Standing Committee on Appropriations on its study tour to the United States of America and Canada from 29 September to 10 October 2012, Dated 31 July 2013

The Standing Committee on Appropriations having undertaken an international study tour to the United States of America and Canada from 29 September to 10 October 2012, reports as follows:

1. Introduction

The Standing Committee on Appropriations (the Committee) was established in terms of the Money Bills Amendment Procedure and Related Matters Act, No 09 of 2009 (the Act). In terms of section 4(3) of the Act, each House must establish a Committee on Appropriations whose powers and functions include considering and reporting on the following matters:

· Spending issues;

· Amendments to the Division of Revenue, the Appropriation Bill, Supplementary Appropriation Bill and the Adjusted Appropriation Bill;

· Recommendations of the Financial and Fiscal Commission (FFC);

· Reports on actual expenditure published by the National Treasury (section 32 reports); and

· Any other related matters.

In addition to the above mandate, the Committee has been given an extended mandate by the National Assembly Rules Committee on 1 November 2011, in terms of Rule 199 (b) to assume the legislative and oversight function over the Department in the Presidency for Performance Monitoring and Evaluation, including the National Youth Development Agency.

The Department of Performance Monitoring and Evaluation was established in 2009, in terms of section 85(3) of the Constitution of the Republic of South Africa. In the Fourth Parliament, Government adopted an Outcome Based Approach wherein each department is expected to achieve a particular outcome at the end of the five year period.

To enforce the achievement of these outcomes the President signed performance agreements with individual Ministers linked to a particular outcome. Furthermore, Government departments signed delivery agreements with agencies and other stakeholders which are involved in the achievement of particular outcomes. Therefore, the establishment of the Department was aimed to bring more focus on performance monitoring and evaluation of service delivery.

1.1. Delegation of the trip

The delegation from Parliament was as follows:

1. Mr EM Sogoni (Chairperson, African National Congress)

2. Mr GT Snell (African National Congress)

3. Mr JP Gelderblom (African National Congress)

4. Ms NNP Mkhulisi (African National Congress)

5. Ms LE Yengeni (African National Congress)

6. Mr L Ramatlakane (Congress of the People)

7. Mr M Swart (Democratic Alliance)

8. Mr D Arends (Committee Secretary)

9. Mr PA Dlomo (Committee Researcher)

The delegation from the Department was as follows:

1. Deputy Minister, Mr O Bapela

2. Deputy Director-General, Dr I Goldman

3. Deputy Director-General, Ms N Gasa

4. Director of Evaluation, Ms C Jacobs

5. Personal Assistant, Ms C Mangwane

6. Information and Content Director for the Deputy Minister, Ms N Mngomezulu

1.2. Terms of Reference

The Committee has been invited by the Department in the Presidency for Performance Monitoring and Evaluation (the Department) to join it on a study tour to the United States of America and Canada from 29 September to 10 October 2012. The team was led by the Deputy Minister in the Presidency for Performance Monitoring and Evaluation as well as Administration, Mr O Bapela , and the coordination was undertaken by the Deputy Director-General in the Department responsible for Evaluation and Research, Dr I Goldman.

Since the establishment of the Department more lessons and capacity building mechanisms were required. The United States of America (USA) and Canada have managed to develop and establish sound and more solid monitoring and evaluation systems. The two countries have a long history in monitoring and evaluating the performance of their governments. Therefore, the aim of the study tour was to enable both the Committee and the Department to learn best practices from these counterparts. The focus area for the Department in the study tour was on the executive aspects while the Committee was focused more on the legislative aspects.

The envisaged outcome of the tour was the Committee obtaining a clear understanding of its role and responsibilities with regard to the Department and defining a strategic relationship with the Department. Based on the Committee’s mandate which includes in-year monitoring of performance and budget expenditure, the Committee would also be able to understand how it could utilise the information produced by the Department more effectively. This would enhance the efficiency of the Committee in its oversight function both to the Department as well as other Government departments.

The Committee would also be able to learn how it could execute its responsibilities in terms of its original mandate derived from the Act without compromising its co-mandate. The lessons would also assist both the Committee and the Department to improve its coordination and facilitation of Government outcome monitoring and performance evaluation process and the better utilisation of ‘think-tank’ institutions such as Public Service Commission, Human Science Research Council, Auditor-General of South Africa, Civil Societies and other interested stakeholders. Therefore, the report would provide detailed facts, lessons learnt, findings and recommendations by the Committee on the study tour.

1.3. Purpose and approach

The purpose of the study tour was for the Committee to gain an in-depth understanding of the issues underlying successful monitoring and evaluation (M&E) systems, and in particular the appropriate role of Parliament/Congress in effective performance M&E. Some of the learning questions included:

· How does the overall Performance, Monitoring & Evaluation (PM&E) system operate, including the relationship between planning, budgeting and M&E – what are the successes and failures and why?

· What roles do different organizations play, how are these coordinated, how have the M&E institutional arrangements evolved, why and what are the lessons?

· What roles are Parliament/Congress playing in performance M&E? What are the lessons from this experience?

· How is the information from performance M&E fed back into decision-making, planning, budgeting, programming?

· Success factors, main obstacles and lessons learned in the path towards institutionalization of PM&E, including the role of a legal basis for performance M&E, roles of the executive, Parliament/Congress, and other independent agencies (eg Auditor-General) in ensuring a successful system.

The delegation met with the following institutions in the two countries during the study tour:

United States of America:

· National Academy for Public Administration (Robert Shea, Joseph Mitchell, and Amit Magdieli )

· Congressional Budget Office (Doug Elrendorf , Deborah Kilroe , Theresa Gullo , and Susan Willie)

· House Committee on Oversight and Government Reform (Majority Staff: Lawrence Brady, Robert Borden, and Adam Fromm)

· Urban Institute (Katharine Mark)

· World Bank Independent Evaluation Group ( Nidhi Khattri , Ximena Fernandez Ordonez, Ari Wessal , Anya Reva , Arturo Rea, and Maria Padrina )

· World Bank Office of the Executive Director for South Africa ( Renosi Mokate , and Vuyelwa Vumendlini-Schalk )

· Government Accountability Office (Gene Todaro , Nancy Kingsbury, and Katherine Siggerud )

Canada:

· Standing Committee on Government Operations and Estimates (Mike Wallace, Marc-Olivier Gerard, and Tinalise Legresley )

· Federal Treasury Board Secretariat, including Centre for Excellence in Evaluation (David Enns , Anne Routhier , and Ms A J Preece , Brian Moo Sang, and Arvind Srivastava )

· Canadian Evaluation Society, including the International Programme for Development Evaluation Training (Julia Thompson, Simon Roy, Eric Champagne, Pierre Martineau, and Martha McGuire)

· House of Commons Public Accounts Committee (Bryan Hayes, and Andrew Saxton)

· Ontario Legislative Assembly (Trevor Day, Anne Stokes, and John Inca Anderson)

· Ontario Ministry of Finance and Treasury Board (Lee Coplan , Megan Borner , and Keiko Kuji-Shikantani )

Components

The main elements of the study tour included:

(1) An initial meeting in Washington with World Bank/US Congress

· CLEAR/World Bank - an overview of PM&E systems and some context to how SA’s system fits into the wider approaches to PM&E.

· An overview of the PM&E system in Canada and a look into some of the issues (Independent Evaluation Group).

· Meeting with US Congress, Congress Budget Office and Government Accountability Office to understand the roles of these institutions in PM&E.

(2) Visit to Canada

· Meeting with federal government’s Treasury Board Secretariat around the overall approach to performance monitoring and evaluation, including its Centre for Excellence in Evaluation.

· Meeting with Federal Parliamentary Committees to understand the roles they play in performance M&E, accountability processes and what agencies report to them in this regard. This included a meeting with the relevant committee to understand the role they play.

· Meeting with the Ontario State Government on the province’s approach to performance monitoring and evaluation and the linkages between national and state levels.

2. United States of America’s role in performance monitoring and evaluation

2.1. Structure of the State

The United States of America (US) covers 9.1 million square kilometres which is 8 times the size of South Africa (SA – 1.2 million square kilometres), has a population of 311 million, and a federal system with 50 states. Each state has its own written constitution, government and code of laws and around 15 per cent of each state’s of revenue is generated locally. The President appoints the heads of the federal agencies but Congress approves the budget. The heads of the 15 departments, chosen by the President and approved with the "advice and consent" of the U.S. Senate, form a council of advisers generally known as the President's "Cabinet". The President appoints the ministers (Secretary) but Congress has to approve the budget. Congress has an oversight role to prevent waste and fraud, protect civil liberties and individual rights, ensure executive compliance with the law, gather information for making laws and educating the public, and evaluate executive performance. It applies to cabinet departments, executive agencies, regulatory commissions and the Presidency. Ministers in the US are not Members of Parliament.

2.2. Parliamentary system

2.2.1. Roles of relevant committees - House Committee on Oversight and Government Reform

The delegation visited the House Committee on Oversight and Government Reform which has jurisdiction over all governmental activities. The Authorising Committee and the Appropriations Committee also look at how money is spent. Given the fact that the majority in Congress is not the same as the party of the President of the US, they do a lot of oversight. They also focus on policies that the Executive is trying to impose through Executive orders.

The Committee on Oversight and Government Reform is responsible for oversight and reform of all work of government and tends to focus on process, to ensure that laws are properly applied. They don’t have legislative jurisdiction. There are 23 Republicans and 17 Democrats in the Committee which has 7 subcommittees, i.e :

(i) Federal workforce, US Postal Service and Labour Policy;

(ii) Government organisation, efficiency and financial management;

(iii) Health care, District of Columbia, Census and the national archives;

(iv) National security, homeland defence, and foreign operations;

(v) Regulatory affairs, stimulus oversight and government spending;

(vi) Troubled Asset Recovery Programme (TARP), financial services and bailouts of public and private programs; and

(vii) Technology, information policy, Inter-governmental reform and procurement reform

Their focus areas are waste and fraud, regulation evaluations and policy. Waste and fraud usually made up 5 to 10 per cent of any budget. The Committee also had close links with the Inspectors-General in the various agencies and the Government Accountability Office (GAO). With regard to oversight, the Committee place particular interest in the High Risk Series which is produced by GAO. The role of the Government Accountability Office will be explained in the section 2.2.2 hereunder.

The management of the Committee is responsible for communications, policy, clerks, oversight, and investigations. They have a staff complement of 78 for the majority party, and 39 for the minority, which is the standard rule for staffing and financial assets. The tools for oversight of the Committee include the following:

· Subpoena power – for documents and testimony which can be difficult to enforce since cooperation by the executive can be minimal.

· Building relationships – this is key as they can’t prosecute – so they try to name and shame. The use of media also helps with cooperation by agencies.

· Investigations – these come about in a number of ways:

- Whistle blowers - Preliminary reviews are done to see if legitimate. They do not have direct ability to protect whistle blowers but there is a Whistle Blower Act in place to that effect.

- Inspectors-General in agencies have a critical role and look for fraud, abuse, mismanagement.

- Requests for GAO to do detailed investigations.

- Chairperson has discretion on what type of investigations to undertake.

- Other Committee Members may also request investigations.

· Hearings – over 100 hearings conducted in 2011.

· Use internet as a driver to put all agency expenditure in the public eye.

· Communication on the Committee’s role – using media, internet website, and social media such as YouTube page.

One investigation “Fast and Furious” had 4 fulltime staff and 3 most of the time, began early last year and took 1.5 years to complete. The Investigations Unit is closer to a prosecutorial role using subpoenas and depositions, whereas the oversight is more explanatory. Most investigations are generated by their staff, but oversight would often rely on outside research. There are also a number of private sector organisations that do investigations. A significant project for GAO will often take a year so they only use them on some investigations. Furthermore, the Congressional Research Service is used where quicker turnaround is needed on investigations.

The main challenge for the Committee is that it has no powers to enforce, and in the end the President controls the Cabinet Secretaries, as it is not a parliamentary system. They do however value the fact that the Congressional Budget Office (CBO) and GAO are de-politicised institutions. When legislation is considered, the Committee will often request GAO to report progress on a programme. For instance the federal government spends $80 billion on Information Technology (IT), but many of these systems do not function well. An effective IT system is important for accountability and transparency.

2.2.2. Institutions Supporting Congress and Evaluations

2.2.2.1. Government Accountability Office (GAO)

The Government Accountability Office (GAO) is an arm of Congress that was created in 1921 and is responsible for auditing the federal government, with 3500 staff. Only 10 to 15 per cent of their work is financial audit, with most of their work being on performance evaluations/audits. It is headed by a Controller-General who is appointed for 15 years, providing considerable stability. They work with 4 main committees and report to the Appropriations Committees in both houses. The House Oversight Committee and its equivalent in the Senate provide oversight of GAO’s operations.

In terms of evaluation, GAO has a legal mandate to look at all programmes over a 3 year period but has a limited mandate around overlapping and duplicated functions. GAO does around 150 surveys a year; they produce 1000 reports per year, and provide around 250 testimonies to federal committees. They are organised by sector and subject area and the staff work in multidisciplinary teams. They receive about 900 requests a year and have a protocol about how to receive these and prioritise them, as follows:

· Studies in relation to legislation for instance the requirement to do evaluations after 3 years;

· Requests from Committee Chairperson or high ranking Member;

· Request from any member of a committee; and

· The hotline where any member of public can make requests but there is no requirement to respond.

Some committee investigations are very political to support the Chairperson of a Committees’ agenda and they would not call on GAO. This is due to the fact that GAO is a depoliticised institution.

GAO also has its own authority to initiate work. They draw up a strategic plan of issues to follow up on and interact with Congress. They make around 150 presentations on evaluations and audits. A request is provided in a letter, which in many cases they have been involved in crafting. Prior to the 1970s only 20 to 30% were requested by Congress, the rest were self-initiated.

GAO have no power to enforce the recommendations it makes to agencies, therefore a lot of thought and effort would be put into crafting recommendations that are practical and affordable and foster relationships with federal agencies. As a result around 80% of recommendations are implemented.

Every two years, GAO provides Congress with an update on its High-Risk Program, which highlights major problems that are at high risk for waste, fraud, abuse mismanagement or in need of broad reform. There are 30 areas on GAO's High-Risk list. These are highlighted on its website with explanatory notes. They participate personally in meetings with the heads of agencies around these items.

2.2.2.2. The Congressional Budget Office

The Congressional Budget Office (CBO) was created in 1974 and prior to this Congress had no capacity to analyse the budget. CBO is non-partisan and prepares options but not recommendations on what is good or not. It has 235 fulltime staff, with a Director appointed by both Houses. All staff is employed based on professional background and not political orientation. While CBO looks at potential budget implications, once a programme is up and running, GAO would audit or look at how it is performing.

All products are vetted by at least 3 people, which helps to ensure documents are objective. One of the Appropriation Bills funds Congress and a line item funds the CBO, but does not go through committees. This gives it some independence. The main activities of CBO are as follows:

· To assist Congress as they develop the budget. They develop a baseline based on new laws and the way the economy is expected to change. They also do an analysis of the President’s budget proposal. Based on this they produce the budget resolution;

· Helping the Congress stay within its budget plan – compares cost estimates of legislative options;

· Providing information for enforcement of budget rules and statutory requirements – statutory pay-go tables; and

· Helping the Congress when they are designing/considering legislative proposals.

It is to be noted that CBO does not make policy recommendations, write legislation, implement programmes, or audit expenditure. However, staff needs to be able to understand legislation and also be very good communicators, writing clearly and documenting assumptions. Cost estimates can be very quick and take a day, sometimes much longer. When they do a cost estimate they only look at the budgetary effects, not cost-benefit analysis.

A key feature of the CBO is its non-partisan and transparent nature. The estimates have the names of the analysts, and are loaded on the CBO website. A Member or Chairperson of a Committee can make proposals for CBO to do some analyses which would be kept confidential as long as the Chairperson did not make it public, where after it would be published by the CBO. They speak frequently with Office for Management and Budget (OMB) and budget offices of agencies to get their viewpoint and exchange information

2.2.2.3. Congressional Research Service (CRS)

The Congressional Research Service (CRS) works exclusively for Congress, providing policy and legal analyses to Committees and Members of both the House and Senate. CRS provides analyses that are authoritative, confidential, objective and nonpartisan, with its service being available 24 hours a day, 7 days a week. The CRS tends to provide a quick turnaround of analyses which is treated as confidential unless the member requests that it is made public. CRS services come in many forms which are as follows:

· reports on major policy issues;

· tailored confidential memoranda, briefings and consultations;

· seminars and workshops;

· expert congressional testimony; and

· responses to individual inquiries

CRS employs about 450 policy analysts, attorneys and information professionals in a variety of disciplines working in one of five research divisions. The breadth and depth of this expertise enables CRS to mobilize quickly, working together in flexible groups to provide integrated analyses of complex issues facing the Congress. Its work incorporates program and legislative expertise, quantitative methodologies, and legal and economic analysis. The Knowledge Services Group provides research support services to the policy experts in each of the five divisions. The research divisions are as follows:

· American Law ;

· Domestic Social Policy ;

· Foreign Affairs, Defence and Trade ;

· Government and Finance ; and

· Resources, Science and Industry .

2.2.3. Institutions Supporting the Executive

The Office for Management and Budget (OMB) is in the President’s Office. It oversees the budget for individual agencies and programmes, including both capital and recurrent budgets, and is similar to a Finance Ministry, while the Treasury deals with big fiscal policy questions, and manages federal finances. OMB has a staff complement of 200 analysts in a overall group a overall staff compliment of 450, also covering information issues and regulatory review. Analysts usually have a public policy or social science background, and analytical skills. Staff are organised by Cabinet department with 6-10 analysts who oversee a particular department and help the President to develop the budget. They negotiate backwards and forwards over the budget and then send a budget request to Congress. Then OMB also advises the President on how to deal with Congress over the budget. It has a deputy director for management, who is the Chief Performance Officer of government. OMB also has a legislative review section which looks at all legislation.

The Treasury Department is the executive agency responsible for promoting economic prosperity and ensuring the financial security of the US. The basic functions of the Treasury Department include the following:

· Managing Federal finances, Government accounts and the public debt, issuing currency and coinage;

· Collecting taxes, duties and monies paid to and due to the country and paying all bills of the U.S.;

· Supervising national banks and thrift institutions;

· Advising on domestic and international financial, monetary, economic, trade and tax policy;

· Enforcing Federal finance and tax laws; and

· Investigating and prosecuting tax evaders, counterfeiters, and forgers.

Each federal agency has an Inspector-General to perform internal audit created in an Inspector General Act in 1978. This creates independent and objective units with the following responsibilities:

· to conduct and supervise audits and investigations relating to programs and operations of several departments;

· to provide leadership and coordination and recommend policies for activities designed (a) to promote economy, efficiency, and effectiveness in the administration of, and (b) to prevent and detect fraud and abuse in, such programs and operations; and

· to provide the means for keeping the head of the establishment and the Congress fully and currently informed about problems and deficiencies relating to the administration of such programs and operations and the necessity for and progress of corrective action.

The Inspectors-General are appointed by the President, and around half of these have an evaluation function. In some cases evaluation is in the strategic planning office and in some cases in the budget office. Performance measurement is usually in finance. There are about 70-80 Inspectors-General across the federal government. They send reports to the Agency head but send copies to Congress. They have a government-wide organisation they belong to, and there is some common planning and training.

2.2.4. Evolution of the legal basis for PM&E

Prior to 1993 the US had little law in relation to programme planning and budget, and performance assessment. The Government Performance and Results Act of 1993 (GPRA) required strategic plans and annual performance reports, as well as indicators. However this became a compliance exercise and these plans were not closely tied to programmes. After the passing of the GPRA, departments did not do anything in terms of evaluations for 5 years.

In July of 2002, the OMB announced the development of a tool for formally evaluating the effectiveness of federal programs, called the Program Assessment Rating Tool (PART). At the end of 2010 Congress passed the Government Performance and Results Modernization Act (GPRMA). The Act is very specific about a range of issues on performance management and aims to: encourage greater use of performance information by encouraging agencies to demonstrate leadership commitment; align individual, programmes and agency goals; improve the usefulness of performance information; build analytical capacity to analyse and use performance information; and communicate performance information frequently and effectively. In terms of the results of non-compliance, the GPRMA states that if the agency doesn’t meet the goals in year 1 they have to indicate how to address this, in year 2 they have to do a report, in year 3 there is an automatic 10 per cent budget cut across the board. This could however have an impact on services delivered but the threat of a budget cut is regarded as helpful.

Agencies report on how they have achieved outcomes, how far the data is valid, and the agency has to attest to the validity of the data. However, this does not mean that they could show attribution of impacts, and so the emphasis has shifted to independent evaluation. Recently there is much more emphasis on transparency which requires better IT systems. Congress has a developing interest in this regard.

In 2012 the Data Act was passed, which requires recipient reporting, eg contractors, to bring in transparency. The Act establishes a new Federal Accountability and Spending Transparency Commission. The Data Act requires reporting on all tiers of sub-awards and includes a reporting threshold of $100,000. This Act will cut Federal agency spending on conferences by 20% from their 2010 levels, cap spending on individual conferences at $500,000 and limit the number of attendees to international meetings to 50.

2.2.5. Planning

With regard to planning, a major weakness is that there is no requirement for a government-wide plan. Instead, the President’s budget was viewed as the plan. The law required that stakeholders and Congress be involved, but this has been ignored. However the GPR Modernisation Act is specific that government has to define priority goals within one year of a new administration.

Programmes are not necessarily well defined, as to get programmes approved many constituents and goals have to be satisfied, and so programmes are often not very specific. In terms of monitoring, much data is collected but not used effectively, eg the Education Department collects data on over a thousand indicators, much of which is not used.

2.2.6. Budget process

The President proposes a budget but Congress actually makes the decision on how much is spent, and can override the President’s proposal. The President can veto this but Congress can override such a veto. If the President and Congress fail to reach a consensus, then a continuing resolution can be adopted to hold spending at the same rate as the previous year, which can happen for 6 months. In practice this means that discretionary payments would shut down.

Agencies submit their budgets as well as performance targets to OMB in September, and between September and February they debate the final numbers. In February OMB submits the President’s budget request to Congress and sets the terms of the Budget debate. The proposals also set out what total spending and revenues will be should the plan be followed as is. Congress takes that information and a baseline is done by CBO and through a budget resolution an overall budget is given per committee, which may differ markedly from the President’s proposal. Then the 13 committees allocate funds to all the accounts under their jurisdiction through annual Appropriation Bills. The Budget Committees then pass the top line numbers that the Appropriations Committee then allocates to each Agency. So the federal budget is not one document but is a combination of budget resolutions and Appropriation Bills.

In 1995 a law was passed called the Unfunded Mandates Reform Act (UMRA) to avoid imposing unfunded federal mandates on state, local, and tribal governments (SLTG), or the private sector. Most of UMRA's provisions apply to a Federal mandate that may result in the expenditure of funds by SLTGs, in the aggregate, or by the private sector for $100 million or more in any one year. If a rule meets these conditions, the agency must:

· Prepare a written statement that includes:

- the legal authority for the rule,

- a cost-benefit assessment,

- a description of the macro-economic effects, and

- a summary of SLTG concerns and how they were addressed.

· Consider a reasonable number of regulatory alternatives and select the least costly, least burdensome or most cost-effective option that achieves the objectives of the rule, or explain why the agency did not make such a choice.

The federal budget of the US was around $3.6 trillion in 2012, of which 60% was mandatory based on underlying laws, and therefore does not have to be authorised each year. 40% of the budget was discretionary and had to be appropriated each year. 19% is defence discretionary, 19% is non-defence discretionary (agriculture, natural resource, transport etc) and 6% is interest payments. The deficit for 2012 was reported to be $1.1 trillion.

2.2.7. Monitoring and Evaluation

In terms of monitoring, more focus is placed on expenditure figures in the performance measurement of programmes. The GPRA requires strategic plans and annual performance reports, as well as indicators. GAO did a survey in 1997 and 2007 on the use of information for decision-making but didn’t find much change. This led to some of the changes in the GPRMA. The GPRMA underlines the reporting and also asks for cross-cutting goals at the beginning of each new administration.

In terms of incentives and consequences, there is a lot of freedom around evaluation, and if departments undertake evaluations, then funding requests are looked on more favourably. If they have not met performance measures, what works is congressional oversight since people do not want to be publicly interrogated on why they did not meet targets. This is used by Congress to incentivise departments to perform. OMB is tuned to agency performance and when they review the budget they do use evaluation information. The budget is however not directly linked to the evaluation information due to the long term (2 year) budget preparation cycle. In respect of coordination, it is to be noted that the system is very decentralised, so there is a lot of variability in how M&E is done and how the information is used.

3. The Role of the World Bank in Performance Monitoring and Evaluations

The World Bank has different components relevant to PM&E which will be described in the following sections.

3.1. Independent Evaluation Group (IEG)

The Independent Evaluation Group (IEG) is charged with evaluating the activities of the World Bank. The Director-General of IEG reports directly to the World Bank Group's Board of Directors. The goals of evaluation are to provide an objective assessment of the results of the World Bank’s work and to identify and disseminate lessons learned from experience. IEG also has a capacity building role, including producing knowledge products. They house the secretariat for the CLEAR Initiative, a multi-donor initiative supporting M&E, and in SA this is based at the University of the Witwatersrand. The Operations Policy and Country Services (OPCS) group does the tracking of projects, and sends the information to IEG for use in evaluations.

3.2. Poverty Reduction and Economic Management Group

The Poverty Reduction and Economic Management Group in the World Bank also has a section focusing on M&E. DPME has worked with this group in the past.

3.3. Development Impact Evaluation Initiative

The Development Impact Evaluation Initiative (DIME) was created in 2005 with the objective of generating knowledge on selected policies. In 2004, the World Bank only had 28 active impact evaluations. By 2008, that number had grown seven-fold. Through the Africa Impact Evaluation Initiative (AIM), the Africa Region's impact evaluation portfolio grew 40 times over so today it represents 43 percent of active impact evaluations. DIME has three objectives, which are as follows:

· To improve the quality of operations through iterative learning. DIME works to integrate the Bank's operational and analytical functions to incorporate evidence, empirically test the effectiveness of policy alternatives, scale up best-performing policies, and improve the effectiveness of programs during implementation.

· To strengthen client institutions for evidence-based policy making. DIME builds in-country capacity to understand and use impact evaluation to inform policy and operational decisions. Modalities include training, networking, and learning-by-doing via joint Bank-government evaluations.

· To generate knowledge on critical development questions. DIME seeks to secure the validity of learning for the country in which it operates by working with government programs at scale and in the prevailing institutional environment. By working programmatically across many countries and institutional settings, DIME also works to extract broader lessons of global interest.

4. Committee Observations in respect of the US:

· The United States of America has managed to build and consolidate strong institutions to support Congress to evaluate and monitor government programmes. However, it can be stated that even though they have such strong institutions (i.e. the number of Committee support staff, Congressional Budget Office, Government Accountability Office, Congressional Research Services, and Inspectors-General in Agencies) they have no power to enforce agencies to implement their recommendations.

· Congressional Committees make use of the media as an incentive to highlight any poor performance of agencies, therefore encouraging agencies to ensure that their performance is of an acceptable standard.

· There is a coordinated approach to evaluate the performance of programmes in the US, i.e. GAO, CBO, Congressional Research Service, and Inspectors-General have an integrated approach and they share performance information.

· The GAO produces a High Risk Series on a bi-annual basis which highlights all the major problems that are a high risk for waste, fraud, mismanagement or in need of broad reform. This is used by the Committees for improved oversight of agencies.

· The budget proposals as introduced by the US President fundamentally can be amended or rejected by Congress.

· The World Bank plays an important role in supporting M&E implementation system across democracies. An offer was made by the World Bank to build the capacity of the Parliament of the Republic of South Africa on PM&E and how best Government systems could be co-ordinated.

· The US Parliamentary Committees can request that performance audits be done whereas in SA, AGSA usually initiates performance audits. The GAO focuses more on performance audits whilst AGSA focuses more on financial audits.

5. Canada’s role in performance monitoring and evaluation

5.1. Background to the country

Canada is a large country of 9 million square kilometres which is similar in size to the United States of America (US), but with a relatively small population of 35 million people. It has a decentralised government with services being delivered across the country. The large and multilayered government system poses unique challenges for monitoring and reporting, which are as follows:

· Highly decentralised federation – 10 provinces, 3 territories;

· Hundreds of federal-provincial agreements and federal programming; and

· 3700 municipal governments, which are in effect subdivisions of the province, and for whom laws were exclusively made by provinces.

There is only one Constitution under which the federal government is allocated some powers and the rest are by default with provinces. For example education, health care, welfare, local works are provincial responsibilities; immigration, agriculture and pensions are joint responsibilities between the federal and provincial; while security, taxation, criminal law, and first nations, are federal responsibilities.

The governing party at the time of the study tour was the Conservative Party, with the official opposition being the New Democratic Party. The Cabinet comprised of f ederal ministers chosen by the Prime Minister usually from among the members of his own party in Parliament.

Federal government comprised of over 148 government organisations, departments and agencies and crown corporations. At the time of the study tour, over 300 000 core federal public servants were employed in over 1600 points of service across Canada. There were over 2000 federal programs.

The highest court is the Supreme Court of Canada, where legislation that is introduced by the Federal Government can be challenged. At times, federal government would be given one year to rectify any legislation that has been challenged successfully. The Canadian Government also seeks opinions from the Supreme Court of Canada on legislation which it intended to introduce.

The provinces are recognized under the Canadian Constitution (and its principle of federalism ) as being constitutionally autonomous. In other words, they are granted constitutional powers that cannot be altered unilaterally by other levels of government, such as the federal government. With this in mind, the provinces are significantly different from territories, the other type of regional governments in Canada. Canada has three territories: the Northwest Territories, the Yukon, and Nunavut. Whereas the provinces are constitutionally autonomous, the territories are constitutionally subordinate to the federal government. Hence the federal government has the power to create territories, as well as to decide what powers and jurisdictions those territories would enjoy.

While the Constitution forbids federal and provincial governments from passing laws in areas that were under the other’s exclusive jurisdiction, it did not prohibit them from spending money in those areas. The federal government has actively used its spending power to influence provincial policies and programs, particularly in the areas of health care and social services. In the case of health care, for example, the federal government would transfer billions of dollars annually to the provinces in order to support their public health systems. Moreover, the federal government would place important conditions on the provinces in order to receive this money, which was stipulated in the Canada Health Act. If a province failed to meet the stipulated conditions, then federal government would withhold portions of its federal transfers. In this way, the federal government could indirectly influence provincial policies in areas that fell outside federal jurisdiction. With regard to provincial revenue, approximately 83 percent resulted from provincial/territorial governments’ own sources of revenue, while 17 percent was in the form of transfers from other government subsectors and the federal government.

Local government was completely dependent on provinces, so effectively there were two levels of government, and the structures differ from province to province. Local governments received a significant portion of revenue from local taxes. Therefore, a federal department would find it extremely difficult to keep itself abreast of activities at a local governmental level.

5.2. Canada’s Parliamentary system

5.2.1. Roles of relevant committees to the Standing Committee on Appropriations

The Prime Minister chooses the Cabinet from the Members of Parliament. There are two Houses, i.e. House of Commons and the Senate.

There were around 27 committees which were mostly based on sectors, i.e. health sector. There were a total of 308 members of Parliament at the time of the study tour. The Committees consisted of 12 members, currently 1 Liberal Party, 4 New Democratic Party and the rest Conservative, and this matched the composition of the House of Commons (the House). The legislation was viewed as the top priority activity of committees and their core role was to review legislation after the first reading in the House. Debates would normally occur during the second reading in the House. The third reading was usually seen as the approval of legislation in principle since there was often a reluctance to change the proposed legislation after this point.

There are three main House committees in Canada that are relevant to the Standing Committee of Appropriations in South Africa. The Committees are as follows:

· The Standing Committee on Government Operations and Estimates which looks at government operations (current spending);

· The Public Accounts Committee looks at audit reports; and

· The Finance Committee consults widely in the country and makes suggestions for what should be in the budget. This is discussed in more detail under the budget section.

The above Committees will be discussed in more detail hereunder.

· Standing Committee on Government Operations and Estimates

The core of the committee’s work is government operations (current spending). The Committee’s mandate includes primarily the study of:

· the effectiveness of government operations;

· expenditure plans of central departments and agencies, commissions, selected Crown corporations and organizations;

· new and emerging information and communications technologies (ICTs) in the government; and

· statutory programs, tax expenditures, loan guarantees, contingency funds and private foundations deriving the majority of their funding from the Government of Canada.

This Committee is responsible for the budgets of the Treasury Board, Public Works and other departments. However the Treasury Board reports directly to Cabinet. They have no relationship with Public Accounts Committee, which deals with audit. This committee meets twice a week for 2 hours, and does a lot of investigations.

Of interest is the fact that Committees in the Canadian Parliament rarely make changes to legislation, as this would be deemed as admitting that government has made a mistake. In some cases legislation will be referred before the House for the second reading, when it is easier to change it. Nevertheless the committees still provide oversight as they can highlight what is good, or not in legislation. Of further importance is the fact that if committees do not scrutinise the budgets of Ministries, then those budgets would be deemed as accepted. The budget is technically viewed as a policy document which does not indicate how allocations would be spent but focuses on the areas where it will be spent. The Main Estimates which is normally a very comprehensive document will include all the relevant information on how funding would be spent. Deputy Ministers are viewed as the Accounting Officers (Directors-General) and are often invited to brief the Committees. The Ministers would only brief Committees when they were presenting the budgets for their respective Departments.

· Committee on Public Accounts

The core mandate of the Standing Committee on Public Accounts is to review and report on the following:

· the Public Accounts of Canada;

· all reports of the Auditor-General of Canada; and

· the Office of the Auditor-General’s Reports on Plans and Priorities and Annual Performance Reports.

The Committee was chaired by a member of the opposition, New Democratic Party and the Chairperson only voted on a matter in the event of a tie. The Auditor-General of Canada (AG) presents two reports annually (in spring and fall) to the Public Accounts Committee. The tabling of reports by the Auditor-General was normally met with widespread media coverage. The AG reports would be interrogated by the Committee and witnesses would be called to substantiate the findings contained in the said reports.

The Committee looks at the expenditure trends of government through the AG report. The AG is an independent body which reports to Parliament of Canada. There is no Standing Committee on Auditor-General which ensures that AG executes its duties in line with the mandate. In terms of ensuring AG’s independency, Peer Reviews are requested by it through the AG from other countries.

5.2.2. Institutions supporting Parliament

5.2.2.1. Parliamentary Budget Office

Canada has a Budget Office which serves Parliament and provides independent analyses on the state of the nation's finances, the government's estimates and trends in the Canadian economy. The Budget Office may upon request from a committee or parliamentarian, provide estimates on the financial cost of any proposal for matters over which Parliament had jurisdiction. It has a staff compliment of 14 and a budget of C$2.8 million (about R22 million) per annum. The Parliamentary Budget Office (PBO) also acts as a witness during the proceedings of Committee hearings with the government departments. The PBO provides support to capacitate Members of Parliament in order to engage with the budget more effectively.

5.2.2.2. Office of the Auditor General (OAG)

The Office of the Auditor-General (OAG) audits federal government operations and provides Parliament with independent information, advice, and assurance regarding the federal government’s stewardship of public funds. While the OAG may comment on policy implementation in an audit, it does not comment on the merits of the policy itself. They also conduct legislative audits, performance audits of federal departments and agencies, annual financial audits of the government’s financial statements, and special examinations and annual financial audits of Crown corporations, and the governments of the Territories. Since 1995, the Office had a specific environmental and sustainable development mandate, established through amendments to the Auditor-General Act. The audit findings which include good practices, areas requiring attention, and recommendations for improvement are reported to Parliament. The Auditor-General typically submits three reports on performance audits to the House of Commons every year: an annual report in the fall; a spring report; and a status report, which follows up on progress made by the government in responding to recommendations contained in previous performance audits.

5.3. Evolution of the PM&E system in Canada

Table 1 below shows the evolution of the PM&E system in Canada. Unlike the US, Canada’s performance system is not based on a legal system for compulsion. The expenditure management system was designed 15 years ago to fight budget deficits and control new spending. In the late 1990s budget surpluses led to new spending without an anchor to discipline spending. There were ad-hoc exercises to control spending with no real focus on performance, while in the 2000s Australia, the United Kingdom, New Zealand and the US were moving towards focusing on outcomes. The budget of 2006 and 2007 committed the government to a new ongoing approach to managing overall spending, and the Expenditure Management System Renewal was approved in June 2007. The government wants to deliver high quality services and high value programmes at a reasonable cost.

Table 1 : Chronology of development of M&E in Canada

Dates	Initiative	Covers
Late 70s	First programme evaluation policy	Implemented government-wide
Mid 90s	Program review	Dealing with a large deficit through expenditure cuts
	Planning reporting and accountability structures	Unsuccessful try at linking resources to results
	Departmental reporting to Parliament	Moving to results-based plans/reports on performance
2000	Canada’s Performance	Linking programmes to social performance
	Results for Canadians	Results based management as a stated priority of government
	Transfer Payment Policy	Performance frameworks/evaluations reviewed by TBS
	Modern Comptrollership	Investment in management practices and controls
Today’s agenda	Management, Resources and Results Structures	Regaining detailed programme level knowledge
	Improved reporting to Parliament	Moving to whole of government planning and reporting
	Renewed policy on evaluation	Quality, capacity, credibility and expansion of coverage
	Management Accountability Framework	Assessing management performance across government
	Expenditure management system renewal	Management excellence and fiscal credibility
	Federal Accountability Act 2006	Cyclical evaluation of all transfer payment programmes

A common approach to collection, management and reporting of performance information has been developed in the Policy on Management, Resources and Results Structures. This Policy provides detailed information on all programmes in a consistent way, links resources and results, and has been implemented across government. The Management, Resources and Results Structures (MRRS) system has information on departmental programmes, costs, how programmes align with government’s priorities and intended outcomes, expected results, targets.

5.4. Canada’s Planning System

There is a Whole of Government Framework for government’s action. The purpose of this framework is to map the financial and non-financial contributions of federal organizations receiving appropriations by aligning their program activities to a set of high level outcome areas defined for the government as a whole.

The Policy on Management, Resources and Results Structures (MRRS) provides the Programme Alignment Architecture (PAA) – with Strategic outcomes/programs/sub-programs/sub-sub programmes and a common government-wide approach to the collection, management and reporting of financial and non-financial performance information.

Departmental annual plans (Reports on Plans and Priorities - RPP) are based on the PAA and there are targets for the different levels of the architecture, while the expenditure is at two levels. Departments are obliged to be clear on the priorities, linking to the government priorities, and show how the PAA lines up with those priorities. Departments develop a Performance Strategy Framework linked to the PAA which links planned financial allocations, expected results performance indicators, targets and actual financial and non-financial results for each budget programme. This structure is used for reporting to Parliament, and MPs find this helpful in fulfilling their oversight role.

Every programme is a result of a Cabinet decision which then gets to TBS (Treasury Board Secretariat) for approval. Parliamentary scrutiny is on the Reports on Plans and Priorities and is essentially to approve the appropriations.

5.5. Budget

The main budget document, which is tabled in spring, is technically a policy document rather than a financial document, the full financial information is provided in the Estimates. The main spending estimates are called the Main Estimates, which go through different versions. The estimates are produced in autumn and reflect the budget. The Report on Plans and Priorities document for each Ministry is produced in March usually a few weeks after the main estimates and has the budget broken down in more detail. There is no standardisation on the Plans and Priorities. In autumn there is a document on what has been accomplished by departments, however, most MPs don not pay much attention to these reports.

The important dates in respect of the budget are as follows:

1 April	23 June	10 December	26 March
Beginning of Financial year	· Tabling of supplementary estimates (A) · Introduction of Main Estimates	· Tabling of Public Accounts · Departmental performance reports · Tabling of Supplementary Estimates (B)	· Tabling of Supplementary Estimates (C) · Budget presentation · Tabling of Main Estimates · Departmental Reports on Plans and Priorities

The Standing Committee on Finance is authorized to consider and make reports upon proposals regarding the budgetary policy of the government.  The Finance Committee consults widely in the country and makes suggestions for what needs to be in the budget. Reports may be made no later than the tenth sitting day before the last normal sitting day in December, as set forth in Standing Order 28(2). The review of budgetary policy is called the Committee’s pre-budget consultations.

It should be noted that at provincial level (e.g. Ontario) there is a mechanism for monitoring expenditure on an ongoing basis, unlike nationally.

5.6. Roles of key players at national level in the Planning, Budgeting and PM&E system

5.6.1. Privy Council Office (PCO)

The Privy Council Office (PCO) is the Prime Minister’s department and acts as advisor to the Prime Minister, secretary to Cabinet, and provides leadership for the public service. The roles of the PCO covers the following:

· Advisor to the Prime Minister - policy advice and information to support the Prime Minister and Cabinet. This includes non-partisan advice and information from across the Public Service, consultation and collaboration with international and domestic sources inside and outside government (including the provinces and territories) and information on the priorities of Canadians.

· Secretary to the Cabinet - this includes:

- Management of the Cabinet's decision-making system, scheduling and support services for meetings of Cabinet and Cabinet committees;

- Coordination of departmental policy proposals to Cabinet (with supporting policy analysis);

- Advancing the Government's agenda across federal departments and agencies and with external stakeholders;

- Advice on government structure and organization;

- Preparation of Orders-in-Council and other statutory instruments to give effect to Government decisions;

- Administrative services to the Prime Minister's Office, PCO Ministers and, in some cases, to Commissions of Inquiry.

· Public Service Leadership - management of the appointments process for senior positions in federal departments, Crown corporations and agencies, setting policy on human resources issues and public service renewal, drafting and submitting an annual report on the state of the Public Service.

5.6.2. Treasury Board Secretariat (TBS)

The Treasury Board Secretary (TBS) is tasked with providing advice and support to Treasury Board Ministers in their role of ensuring value-for-money as well as providing oversight of the financial management functions in departments and agencies. The Secretariat makes recommendations and provides advice to the Treasury Board on policies, directives, regulations, and program expenditure proposals with respect to the management of the government's resources. Its responsibilities for the general management of the government affect initiatives, issues, and activities that cut across all policy sectors managed by federal departments and organizational entities (as reported in the Main Estimates ). The Secretariat is also responsible for the financial management function of government. The Secretariat supports the Treasury Board in its role as the general manager and employer of the public service.

The role of TBS includes improving the evidence based analyses on programmes, the expenditure management system, and evaluation, since as far back as the 1970s. Their expenditure management system includes the following:

· Expenditure management;

· Employer of the public service;

· Evaluation and results-based management; and

· Managing spending reviews.

TBS supports the Treasury Board in effectively allocating spending in a manner that ensures operational efficiency and effectiveness by:

· Establishing and monitoring adherence to Treasury Board management policies (financial and non-financial);

· Supporting Treasury Board approval of detailed operational plans and recommendation of resources operations for new programs;

· Supporting Treasury Board determination of resource needs/investment opportunities for existing programs.

The Expenditure Management System is supported by 3 pillars:

· Existing spending managed to transparent results/outcomes with clear measures for performance, and demonstrated value for money;

· Up-front discipline on new spending proposal – clear measures of success in cabinet memos, link to programme architecture; and

· Strategic reviews assessing existing spending to ensure alignment with priorities, decision-making using objective information.

The Centre for Excellence in Evaluation (CEE) of the TBS sets the standards and provides outreach support around evaluations. TBS analyses over 70% of evaluations done and make assessments of the quality. They also assess the TBS submissions to see the use of evaluation results, and to see that it is being used objectively.

5.6.3. Federal departments

Each department prepares an annual performance plan called a Report on Plans and Policies (RPP), quarterly financial reports, and an annual Departmental Performance Report (DPR). Performance management is the responsibility of programme managers. Evaluation is also the responsibility of departments, who budget for evaluations, have evaluation units, and in most cases do the evaluations internally. They also have a Departmental Evaluation Plan - a five year rolling plan of evaluations, developed by the Head of Evaluation and advised by a Departmental Evaluation Committee which is submitted to TBS.

5.6.4. Finance Department (Finance Canada)

Finance Canada ensures aggregate fiscal discipline is maintained by establishing the fiscal framework and determining overall spending levels and budget. It develops policies and provides advice to the Government with the goal of creating a healthy economy. Its responsibilities include the following:

· Plans and prepares the federal government’s budget;

· Analyzes and designs tax policies;

· Develops regulations for Canada’s banks and other financial institutions;

· Administers the transfer of federal funds to the provinces and territories;

· Develops policies on international finance and helps design the country’s tariff policies; and

· Monitors economic and financial developments in Canada and provides policy advice on a wide range of economic issues.

5.7. The federal M&E system in Canada

5.7.1. The types of monitoring undertaken and the experiences

Monitoring is done by departments and federally there is only a requirement to report annually in the Departmental Performance Report (DPR). The DPR follows a very similar structure to the Report on Plans and Priorities (RPP), using the PAA as a basis. In the DPR the tables now reflect actual performance. It is high level annual reporting at output level where in many cases one wouldn’t see significant changes over a quarter. Quarterly performance monitoring is internal to the departments.

There are quarterly financial reports which have a lag of 3 months. There is a real time system, but there are old legacy systems that need to be integrated. However government often wants to see the reports before they are made public resulting in a challenge of ensuring transparency.

The Management Accountability Framework (MAF) covers management performance assessment of 97 departments. They give rewards if departments don’t keep changing programmes so that it is easier to see historical trends and can project forward, ideally 3 years historically and 3 years forward. This has been a specific request of Parliamentarians.

TBS found that linking performance information to reporting was helpful but linking it to decision-making in the budget process has been critical. They do not wait for perfection and so have to do outreach – working with departments who are often wary of how TBS will use the information, and whether it will be used against them.

In addition to departmental reporting, Canada also produces a whole of government report on performance called Canada’s Performance: The Year in Review. This aims to provide a snapshot of the government’s contribution to the prosperity and well-being of Canadians in the year in question.

5.7.2. Evaluation

Around 150 evaluations are undertaken per year, but this is decentralised with each department responsible for establishing its own evaluation function. The evaluation policy which was adopted 20 years ago stated that all programmes should be evaluated. This was subsequently changed to high risk programmes, especially those involving transfers. The 2009 Evaluation Policy changed this slightly to cover all areas of expenditure within 5 years. Evaluations can be of programmes (as defined in the PAA), groups of programmes, or a component of a programme. Sometimes they need to do evaluations at sub-sub programme level; sometimes they need to group them to see the linkages between them. If they are grouped there can be problems in that they do not get sufficient detail (granularity). This however requires a lot of resources and has significant cost implications and is more demanding for larger departments. It should be noted that 67 organisations only account for 2% of national spending hence the focus is on the larger departments. No additional resources have been added for this more comprehensive approach and this leads to superficial evaluation outcomes. Only 10 per cent of total programme expenditure is normally evaluated. In terms of budgeting for evaluation, there is no specified percentage of total programme expenditure that has to be allocated towards evaluations.

Programmes are renewed every 3-5 years and the timing of evaluations is often too late to feed in to decisions as evaluations typically take 1 year to produce. So evaluators are often being asked for interim reports at programme renewal stage, and are even changing the way findings are being presented. TBS will ask if departments have taken into account evaluation recommendations in the budget submissions.

CEE has 12 to 14 employees who do outreach, guidance and capacity development to ensure quality evaluation is being done. They are establishing standards for evaluation. The 2009 Policy on Evaluation indicates that departments must identify:

· Value for money;

· Clarify requirements and responsibilities for performance measurement;

· Establish required competencies of heads of evaluations;

· Enhanced standards to support quality improvement and use; and

· Strengthened governance and neutrality.

In terms of the implementation of evaluations, the following are 3 scenarios exist:

1. Internal evaluators who do evaluations for their organisation. This started in the 1960s and 70s. The challenge is adapting to highs and lows in demand, lack of specialist skills, and HR management can be difficult if teams are small. A challenge is independence but internal evaluators are given a degree of independence – a key to independence is that they have job security.

2. Internal evaluation managers who use consultants. The advantage of this is in terms of flexibility. The challenge is finding the right consultant and managing external resources and there is a danger of not developing internal capacity.

3. Hybrid team which has an internal evaluator working with the external evaluator. The advantage is flexibility and knowledge transfer to internal evaluators and this can be part of a training strategy. Challenges can be conflicts between internal and external evaluators if the respective roles and responsibilities are not clear upfront. It can also be costly because of an overlap of responsibilities. A coordination function may well be needed and this may require additional personnel. There can be conflicts, eg when the internal evaluator does not contribute as expected. Also the hierarchy can be unclear, and sometimes junior internal evaluators pull rank on experienced external evaluators.

In practice most evaluations are led by internal evaluators, but in some the internal is an evaluation manager with the work undertaken by contractors. Issues emerging include ethical issues, role conflicts, training challenges, and meeting the scale of evaluation demand with the limited resources available.

Participatory evaluation models are not traditionally widely used but there are times when representatives from the community are part of the Steering Committees. It is important for community groups to have access to the results of evaluation as government is transferring resources to non-profit organisations and community groups, as they are close to the people and cheaper. Community groups don’t have much capacity to evaluate and need training in that regard. The Urban Institute was of the view that surveys of citizens are essential as a source of data to performance information.

5.7.3. The incentives for ensuring that the lessons of M&E are implemented

Canada is not a litigious society and they do not use legislation in a similar way to the US as they do not need legislation for results-based management to promote compliance. Their view is that the need for legislation must be judged in each environment. If it is difficult to drive results-based management (RBM), it is not necessarily made easier with legislation. So peer pressure is the main incentive, not legislation. Some incentives that have emerged include:

· When departments saw that if they had good evidence this helped them in their spending reviews. The use of the same programme architecture strengthened the demand for evaluation;

· In the MAF points were given that if departments showed they have used evidence for decision-making; and

· There are informal consequences, for instance with spending reviews where departments don’t have adequate data to support programmes, delegated authorities could be removed.

5.7.4. Coordination of M&E

Federal and provincial government have separate M&E systems. It is impossible for federal government to know what is happening at provincial and municipal level. TBS is the clear leader of the federal system, although M&E is carried out by departments. There is some cross-level coordination for instance Ministries across provinces do meet around specific areas, typically around areas of transfers.

5.8. Role of provincial (State) parliaments and portfolio committees - example of Ontario

Members of Provincial Parliaments are referred to as MPPs. There are 107 seats in the Ontario House of Commons (the House), of which 1 is the speaker. The governing Liberal Party has 52 seats, second is the Progressive Conservatives with 36 seats, and the New Democratic Party has 18. Often the federal government is of one party, while the provincial governing party is different. Federal government can have national priorities which don’t necessarily align with the provincial priorities. Those responsible for federal-provincial relations have to find a means to align the priorities but there is often conflict. For instance around pipelines which cross several provinces in Canada. Federal government does not necessarily trump provincial government with regard to what the priorities should be.

Committees comprise of Members in the same proportion as in the House. These Committees concentrate more on legislation, rather than oversight. However by questioning they provide the opportunity for public and media to scrutinise what is happening in provincial government.

Parliament and its committees are supreme, but where the governing party is in a majority it is not necessarily given the same level of scrutiny. Currently there is a minority government in Ontario with 3 parties represented in the House. This has changed the dynamics considerably, as Committees are not following the governing party line, and government cannot control what happens, so Committees have much more oversight power. In addition to discussing provincial budgets and programmes, Committees of the provincial legislature do discuss federal transfers.

5.8.1. Provincial parliamentary Committees similar to the Standing Committee on Appropriations

There are 3 financial standing committees, i.e. Finance and Economic Affairs, Estimates, and Public Accounts.

The most important role for the Standing Committee on Finance and Economic Affairs is pre-budget consultation. People make presentations to the committee for funding. In many cases there is a majority opinion, and dissenting opinions from the minority parties. The theory is that government will adjust the budget based on this consultation. In practice the committee does not have enough time to alter the budget. In practice one of the members is from Finance, and so can feed in the findings and views into the budget process. The Minister of Finance also has its own pre-budget consultations around the province.

Once the estimates are presented in the House it is referred to the Standing Committee on Estimates, which goes through the requests from each Ministry, line by line. Typically 6-12 ministries are brought in for detailed hearings with the Minister and Ministry staff. When there was a majority government this was more of a technicality but with a minority government there is higher level of scrutiny. The preparation for committees is where the work happens and as they do not know which ministries will be picked, all have to prepare a budget booklet in preparation for possibly being called to committee.

Each party is allowed to pick 1 to 2 ministries to focus on for a total of 15 hours, followed by a second round of hearings. The government tends to pick smaller obscure ministries. Afterwards they vote per vote – normally it is always passed, with the point being the questioning. The Committee has to report by the 3 ^rd Thursday in November, and they meet 6 hours a week. Finally there is a concurrent vote in the House to say do they agree with the Committee. A Supply Act is then passed which gives authority to spend. By then often spending has already begun.

The Standing Committee on Public Accounts looks at how the money was spent after the fact and has to be chaired by an opposition party member. They can ask the Auditor-General (AG) to do a value for money audits however this requires a vote and was not popular with the majority party. In the past this was used to embarrass the government and would be voted down. Furthermore, past proposals around legislation could not be passed, so proposals were made to get media coverage.

The Federal Government indicated that legislation needed to be introduced at the provincial level for greater accountability and transparency and would be able to demand this through legislation and not tradition. Hence, there is no formalised performance monitoring and evaluation at a provincial level in Canada. There is also no Parliamentary Budget Office at the provincial level and the AG was given additional responsibility to give the formal state of the provincial economy, both before and after an election.

5.9. Committee Observations in respect of Canada:

· With regard to Canada, the Treasury Board Secretariat (TBS) seems to be the most powerful institution for PM&E since it prescribes the Performance Measurement Framework (PMF) and the Programme Alignment Architecture (PAA).

· Canada’s Parliamentary Budget Office is not sufficiently capacitated like their American counterpart; as a result more focus is given to the Auditor-General’s work.

· The Treasury Board in Canada performs the combined functions of which are separated in other democracies, such as Public Service and Administration, National Treasury and Performance Monitoring and Evaluation in South Africa.

· The Programme Alignment Architect (PAA) assists Members of Parliament to increase their understanding of Government programmes hence it improves the Committee oversight function. There is however no requirement in provinces for departments to develop PAAs since they have their own jurisdiction over the matter as per the provisions of the Canadian Constitution.

· There is a lack of integration between the Federal and Provincial PM&E Units in Canada as these institutions are not aligned. The provinces usually conduct their own M&E of programmes even in cases of transfers and subsidies from the Federal Departments.

· The programmes are evaluated on a five year basis in Canada to assess whether the desired impact or outcome was reached. There are monitoring systems in place during that five year period that are used to measure the outputs or performance of programmes in the short and medium term period (quarterly and annually).

· There is a lack of the alignment of programme planning and resources in Canadian Departments which makes it difficult to measure performance against the resources used.

· The Canadian Parliament utilises information published by the Auditor-General, unlike the US which uses Government Accountability Oversight (GAO).

· There is no Standing Committee on Auditor-General in Canada which ensures that AG executes its duties in line with the mandate.

· Federal government can have national priorities which do not necessarily align with the provincial priorities. Those responsible for federal-provincial relations have to find a means to align the priorities but there is often tension.

o The participatory evaluation models are not widely used but there are instances when representatives from the community are part of the Steering Committees. The surveys of citizens are however an essential source of data with regard to performance information.

6. Observations for the South African context

6.1. There is a need for alignment of non-financial data of performance with funding in South Africa.

6.2. There is a need for the development of a comprehensive integrated and regulated programme for PM&E across the three spheres of Government in South Africa.

6.3. Citizen based monitoring should be strengthened to enhance alignment of what departments report in terms of performance and what citizens are experiencing at a grass roots level.

6.4. There is a need to strengthen cooperation between government departments in respect of results-based planning. Mechanisms including legislation should be investigated to achieve this.

6.5. Surveys of citizens in the evaluation process are important data that needs to be included in performance information as this would clearly indicate whether there is value for money in programmes of Government. This in turn would be an indicator whether for Government to terminate or to continue with a specific programme.

6.6. There is a need for the DPME to reduce its turn-around time on the submission of performance reports to Parliament as these are critical in the latter’s oversight role.

Report to be considered.