Audit of the Canadian Space Agency Class Grant and Contribution Program to Support Research, Awareness and Learning in Space Science and Technology

Audit Report

Project No. 12/13 01-01

Prepared by the
Audit and Evaluation Directorate

May 2013

Table of Contents

1.0 Summary

1.1 Audit objective

The objective of the audit project was to determine whether a management framework is in place to ensure that the Canadian Space Agency (CSA) Class Grant and Contribution Program to Support Research, Awareness and Learning in Space Science and Technology (the Program) is managed in accordance with relevant legislation and policies and the approved terms and conditions of the Program, and whether there is proper accountability reporting of the Program.

1.2 Audit opinion

In my view, the management framework for the CSA Class Grant and Contribution Program to Support Research, Awareness and Learning in Space Science and Technology has certain deficiencies that require management's attention. However, accountability for the Program's operations was deemed appropriate.

1.3 Statement of assurance

It is my opinion, as Chief Audit Executive, that sufficient and appropriate audit procedures have been followed and evidence gathered to support the accuracy of the opinion contained in this report. The opinion is based on a comparison of the conditions, as they existed at the time of the audit, against pre-established audit criteria determined in consultation with the directorates concerned, and is only applicable to the particular entity being audited. The evidence was gathered in compliance with Treasury Board (TB) policy, directives and standards for internal audit. The procedures followed comply with the professional standards of the Institute of Internal Auditors. The evidence gathered is sufficient to convince senior management of the validity of the opinion derived from the internal audit.

1.4 Summary of recommendations

During our audit, it was determined that the CSA Grants and Contributions (G&C) Centre of Expertise had put in place a control framework and introduced best practices for the management of agreements. In general, we found that existing procedures had been followed to ensure compliance with Program terms and conditions and TB policies and guidelines regarding financial management and transfer payments. Furthermore, the Program's performance is subject to accountability reporting and results are measured.

Nonetheless, we found some deficiencies that require management's attention and for which we make the following recommendations:

  1. Apply existing controls systematically to ensure that files are documented in accordance with prescribed standards and procedures;
  2. Apply existing controls systematically to ensure that all relevant clauses and required project information are included in funding agreements, in accordance with the Directive on Transfer Payments;
  3. The internal directive concerning the auditing of recipients should be completed and an audit plan should be drafted and implemented.

The original version signed by the
Chief Audit Executive

Audit Team members

2.0 Audit Report

2.1 Background and risk

In September 2002, the Government of Canada approved the creation of the Class Grant and Contribution Program to Support Awareness, Research and Learning ("the 2002 Program"). The objective of the Program was to support the CSA's priority objectives in order to allow the CSA to work closely with Canadian granting agencies, universities and industries with a view to advancing space knowledge, developing and demonstrating new technologies and assisting in the training of the qualified personnel needed by universities and the high-tech sector in Canada. In 2004, a new component was added to the Program. Program changes were approved by the Treasury Board (TB) in 2005 and 2007. In addition, the Minister exercised his authority, as prescribed in the Policy on Transfer Payments, to extend the terms and conditions of the Program until the end of March 2010.

An internal audit of the 2002 Program was conducted in 2005. The findings of this audit revealed that the majority of the sectors kept administrative records to ensure a satisfactory audit trail. Nevertheless, program managers reviewed some aspects that had been identified as requiring improvement. The two main recommendations of the audit were to improve performance measurement as well as the method used to determine recipient eligibility (including ongoing eligibility in the case of multi-year agreements).

In October 2009, the TB approved the new terms and conditions of the Class Grant and Contribution Program to Support Research, Awareness and Learning in Space Science and Technology ("the Program").

The Program initially had two components: research, and awareness and learning. Currently, only the research component is operational. The main objectives of the research component are to:

Budget for the CSA Class Grant and Contribution Program
2012-2013
$M
2013-2014
$M
2014-2015
$M
Grants 6.1 6.4 6.5
Contributions 1.5 2.1 1.3
Total 7.6 8.5 7.8

*Source: 2013–2014 Report on Plans and Priorities

The Program is subject to its own terms of reference, as well as to the TB Policy on Transfer Payments and related directives, and the federal Financial Administration Act.

The CSA's Grants and Contributions (G&C) Centre of Expertise is responsible for establishing the control framework for the management of G&C agreements, and providing the tools and training required for its application. The mandate of the G&C Centre of Expertise, as approved by the CSA Executive Committee, is to offer expertise with respect to G&C, and to provide management, supervisory and performance monitoring services for all of the CSA's G&C programs. The G&C Centre of Expertise also supports the activities of the G&C Advisory Committee.

G&C governance within the CSA includes a G&C Advisory Committee and a G&C Committee. The mandate of the G&C Advisory Committee is to act as a liaison with respect to the review and analysis of all activities and documents associated with the management of grants and contributions prior to the formulation of recommendations to the G&C Committee. The mandate of the G&C Committee is to serve as a forum for CSA senior management and to provide strategic and organizational direction for the selection, design, implementation and establishment of policies and departmental frameworks for the CSA's transfer payment programs. The Executive Committee acts as the G&C Committee.

Agreements are managed by the managers responsible for them within the various branches. As part of its mandate, the G&C Centre of Expertise works closely with and provides support for the managers of CSA G&C projects. The Centre may also manage initiatives and agreements under an MOU with the responsible branches.

This audit project is part of the 2012–2015 Risk-Based Audit Plan (RBAP) approved by the Audit Committee. During the drafting of the RBAP, the entity was evaluated based on its risks and its relative importance. It obtained a score of 6 out of a maximum of 16, which corresponds to a medium level of risk. The principal risk factors identified were the development of a new program, the establishment of a centre of expertise, the review of the control framework, the development of a computerized management tool and the time elapsed since the previous audit.

These risks were discussed with the auditee's representatives who were present at the audit launch meeting.

2.2 Audit objective, scope and approach

Objective

The objective of the audit project was to determine whether a management framework is in place to ensure that the Program is managed in accordance with relevant legislation and policies and the approved terms and conditions of the Program, and whether there is proper accountability reporting of the Program.

Scope

The audit project focused on expenses incurred under the Program from the time of its approval by the TB in October 2009 up until March 31, 2012.

Approach

The audit involved various audit procedures, including interviews with staff, reviews of policies, directives and procedures, and document analyses. It should be noted that no recipients were interviewed.

The audit criteria were based on legislation, policies, standards and/or best practices, and were discussed with management.

2.3 Findings, recommendations and management's response

To ensure that the Program is managed in accordance with relevant legislation and policies and the approved terms and conditions of the Program, we expected to find that

The audit objective and criteria were discussed with management.

2.3.1 Follow-up on the OCG's recommendations

Audit objective: A management framework is in place to ensure that the Class Grant and Contribution Program is managed in accordance with relevant legislation and policies and the approved terms and conditions of the Program, and that there is proper accountability reporting of the Program.

Findings
Criterion 1

With reference to the OCG's horizontal audit of the G&C management framework set up following an internal self assessment, the management action plan is appropriate and being implemented satisfactorily.

Condition

Implementation of an action plan following the OCG audit

In our view, the action plan implemented by management following the OCG audit is appropriate, and the plan's implementation is satisfactory.

Following the OCG's horizontal audit of the G&C management framework, published in 2012, the CSA G&C Centre of Expertise conducted a self-assessment in order to evaluate the CSA's position regarding the OCG's recommendations. An action plan was subsequently drawn up in response to the applicable recommendations.

These recommendations were as follows:

The following were implemented by management over the past year:

The following action plan items have yet to be implemented:

The G&C Centre of Expertise has worked hard over the past few years to complete the development of the control framework required to manage G&C agreements. Thus, the implementation of the management action plan in response to the OCG audit completes the planned improvements to the control framework. The action plan is scheduled to be fully implemented by March 31, 2014.

Cause

N/A

Effect

N/A

Recommendation

N/A

Identified Responsibility
Organization

N/A

Function

N/A

Management's Response

N/A

Management Action Plan
Details of the action plan

N/A

Deadline: N/A

2.3.2 Compliance with the terms and conditions of the Program

Audit objective: A management framework is in place to ensure that the Class Grant and Contribution Program is managed in accordance with relevant legislation and policies and the approved terms and conditions of the Program, and that there is proper accountability reporting of the Program.

Findings
Criterion 2

Required procedures were followed to ensure that G&C agreements between the CSA and recipients comply with Program terms and conditions.

Condition

Compliance with Program terms and conditions

In our view, procedures for ensuring that G&C agreements between the CSA and recipients comply with Program terms and conditions were followed in the majority of cases. However, we identified some aspects that require management's attention and for which we have made a recommendation.

For the purposes of this criterion, a sample of 10 agreements totalling $6.7 million was selected and reviewed. All of the agreements included at least one disbursement made between September 1, 2009, and March 31, 2012. The sample had the following characteristics:

The files were selected based on the auditors' judgment. At least one file was selected for each CSA Program Alignment Architecture (PAA) sub-sub-program (SSP) that receives funding under the CSA Class G&C Program. An additional file was also included for the two SSPs with the largest budgets. For each selection, the amount of the agreement was taken into consideration.

In order to determine whether required procedures were applied, we looked at each file to determine if:

The following are the detailed findings of the file review, for each aspect:

1. Eligibility requirements

After reviewing the 10 agreements, we found that, in general, eligibility criteria with respect to recipients, costs and initiatives were applied in compliance with the terms and conditions of the Program. However, we noted the following deficiencies:

2. Determining the amount and duration of agreements

After reviewing the 10 agreements, we found that, for each agreement signed, the criteria for determining the amount and duration of the agreement had been applied in compliance with the terms and conditions of the Program. However, we noted the following deficiency:

3. Initiative evaluation

After reviewing the 10 agreements, we found in 9 cases out of 10 that the initiative evaluation criteria had been applied in accordance with the terms and conditions of the Program. However, we noted the following deficiency:

4. Requirements regarding funding applications

After reviewing the 10 agreements, we noted in 9 cases out of 10 that the funding application requirements had been applied in accordance with the terms and conditions of the Program, for each agreement signed. However, we noted the following deficiency:

5. Requirements regarding the production of reports on results obtained

After reviewing the 10 agreements, we noted in 8 of the 10 files that the requirements for the production of reports on the results obtained had been applied in accordance with the terms and conditions of the Program, for each agreement signed. However, we noted the following deficiencies:

6. Official language requirements

After reviewing the 10 agreements, we noted that official language requirements had been applied in accordance with the terms and conditions of the Program, for each agreement signed.

Cause

Compliance with Program terms and conditions

There are a number of possible causes for the identified deficiencies. The main one is most likely a lack of knowledge, experience and/or training on the part of the employees responsible for applying the established controls. The lack of knowledge can be partially explained by the fact that, since the Program's expansion in 2009, there are more managers responsible for delivering the Program than there were before. Some of the identified deficiencies may also result from a lack of meticulousness in assembling the documentation for files.

Effect

Compliance with Program terms and conditions

Incomplete files that do not adequately support the decision making process.

Recommendation
  1. Apply existing controls systematically to ensure that files contain documentation in accordance with prescribed standards and procedures.
Identified Responsibility
Organization
Function

Directors General

Management's Response

The three branches are in agreement with this recommendation.

Management Action Plan
Details of the action plan

The Centre of Expertise will continue to provide leadership with respect to the implementation of controls in order to provide managers with tools to ensure that the CSA's files consistently contain documentation in accordance with prescribed standards and procedures.

Deadline: Ongoing

Mandatory training sessions will be developed and given to key staff members, identified by the sectors, who are involved in the management of G&C. These sessions will focus on the use of tools and the compiling of documentation for files.

Deadline: Developed by March 31, 2014 / Submitted by June 30, 2014

A review of the Centre of Expertise's roles and responsibilities will be undertaken in order to determine the best way of applying the existing controls.

Deadline: March 31, 2014

2.3.3 Compliance with TB policies and directives on financial management and transfer payments

Audit objective: A management framework is in place to ensure that the Class Grant and Contribution Program is managed in accordance with relevant legislation and policies and the approved terms and conditions of the Program, and that there is proper accountability reporting of the Program.

Findings
Criterion 3

The Program is carried out in accordance with TB policies and directives on financial management and transfer payments.

Condition

Further to our review, we noted that, in the majority of cases, the Program was carried out in accordance with TB policies and directives on financial management and transfer payments. However, we observed some deficiencies that require management's attention and for which we have made recommendations.

To determine whether the Program is implemented in accordance with TB policies and directives on financial management and transfer payments, we used the same files as those selected for the preceding criterion, and looked at the following aspects:

In our review of the files, we found the following deficiencies requiring management's attention:

Cause

Compliance with TB policies and directives

There are a number of possible causes for the identified deficiencies. The main cause is most likely a lack of knowledge, experience and/or training on the part of the employees responsible for applying the established controls. The lack of knowledge can be partially explained by the fact that, since the Program's expansion in 2009, there are more managers responsible for delivering the Program than there were before. Some of the identified deficiencies may also result from a lack of meticulousness in assembling the documentation for files.

Audit of recipients

The implementation of a recipient audit plan poses a number of challenges for the organization in terms of expertise and cost. Hiring an external firm to conduct a financial audit of a recipient, as suggested in the Directive on Transfer Payments, could represent a significant cost for the organization.

Effect

Incomplete files that do not adequately support the decision making process.

Failure to comply with the requirements of the Directive on Transfer Payments.

Recommendation
  1. Recommendation No. 1 also applies here: Apply existing controls systematically to ensure that files are documented in accordance with prescribed standards and procedures.
  2. Apply existing controls systematically to ensure that all relevant clauses and required project information are included in funding agreements, in accordance with the Directive on Transfer Payments.
  3. The internal directive concerning the auditing of recipients should be completed and an audit plan should be drafted and implemented.
Identified responsibility
Organization
Function

Directors General

Management's Response

The three branches are in agreement with these recommendations.

Management Action Plan
Details of the action plan
  1. The action plan in response to Recommendation No. 1 also applies here.

    Deadline: March 31 and June 30, 2014

  2. The Centre of Expertise will continue to ensure that all relevant clauses are included in funding agreement templates, and that all required project information is entered in the agreements, in accordance with the Directive on Transfer Payments, prior to the signing of the agreement by the Director General.

    In fact, these controls have already been in place since February 2012, further to the request submitted to the Centre of Expertise by the directors general, and following the implementation of the Policy on Internal Control. The Centre of Expertise is currently reviewing the content of each draft agreement (over $25K) and is working with the CSA's Finance Directorate to ensure compliance with TB policies and directives on financial management and transfer payments, prior to the signing of the agreements by the Director General of the sector concerned.

    Deadline: Already completed

  3. The Centre of Expertise will draft the final version of the internal directive regarding recipient audits and have it approved by the G&C Committee.

    Deadline: March 31, 2014

    If necessary, the Centre of Expertise will draw up a recipient audit plan, to be implemented as required.

    In addition, the recipient risk management framework and associated mitigation measures may be adjusted accordingly.

    Deadline: June 30, 2014

2.3.4 Accountability and performance measurement

Audit objective: A management framework is in place to ensure that the Class Grant and Contribution Program is managed in accordance with relevant legislation and policies and the approved terms and conditions of the Program, and that there is proper accountability reporting of the Program.

Findings
Criterion 4

There is accountability reporting of the auditee's operations, and results are measured.

Condition

Accountability

We found that there was accountability reporting of all program activities. The accountability reported is done in two ways: individual accountability reporting of each PAA SSP that receives G&Cs, and overall accountability reporting of the G&C Program as the umbrella program.

Individual accountability reporting: At the beginning of the fiscal year, SSP managers identify expected results in their work plans, as part of the CSA integrated management cycle. Each SSP manager submits detailed operational plans describing the activities to be carried out during the coming year, along with the resources that will be allocated to these activities, the desired outputs and the performance targets. Follow-up is done at mid-year and at year-end. The results will later be reported in the Departmental Performance Reports (DPRs) of their respective branches.

Overall accountability reporting: The results of the G&C Program are compiled by the CSA's G&C Centre of Expertise and presented in the DPR in a separate appendix. The TB requires this separate presentation for all transfer payment programs involving more than $5 million. A breakdown of the results of the Class Program is also to be included in the work plans for the current fiscal year (2013–2014).

Performance measurement strategy

In accordance with the Directive on the Evaluation Function, a performance measurement strategy (PMS) has been in place since 2009. This strategy allows for the systematic monitoring and measurement of the Program's results, as well as for the production of useful and relevant reports for decision making and the gathering of credible and reliable information to effectively support the Program's evaluation. As stipulated in section 6.1.4 of the Directive on the Evaluation Function, which has been in force since April 1, 2009, "developing, implementing and monitoring ongoing performance measurement strategies for programs is the responsibility of program managers." However, the strategy was not fully implemented, as some indicators posed a challenge in terms of the data gathering methodology and were not adopted as planned in 2010. We noted that approximately 80% of the indicators were measured as originally planned. Over the past year, a lot of work has been done into updating the strategy and reviewing the problematic indicators. A new version of the strategy, including the new indicators, is currently in consultation. A new data gathering system was also set up in recent months to facilitate communication with recipients and improve data management. Performance measurement tools are now combined with project monitoring tools.

Overall, even though the performance measurement strategy is in the process of being improved, we find that the strategy is being implemented and that it allows for the measurement of a significant portion of the Program's expected results. The changes currently being made should help improve the strategy's effectiveness.

Cause

N/A

Effect

N/A

Recommendation

N/A

Identified responsibility
Organization

N/A

Function

N/A

Management's Response

N/A

Management Action Plan
Details of the action plan

N/A

Deadline: N/A

Date modified: