Audit of the Management Framework for the Class Grant and Contribution Program to Support Research, Awareness and Learning in Space Science and Technology

Criterion 1

Program activities are planned appropriately.

Condition

Conclusion on the criterion:

The audit showed that program operations are planned appropriately. Stakeholder needs are identified regularly, and the investment areas selected take the priorities identified into account. Investment plans are prepared, approved and reviewed regularly.

Investment planning

Identification of needs

The programmatic sectors regularly consult the industry, academic institutions and national and international partners to identify needs. Consultations are conducted through advisory committees, workshops and working groups. The programmatic sectors also participate in national and international conferences, and have regular contact with international partners. Environmental analyses are conducted to identify context, situate Canada in terms of trends in the space domain and establish the profile of Canadian resources. There are also several internal consultations among the sectors of the CSA.

Prioritization and approval

The objective of the program is to support knowledge growth and innovation and increase the awareness and participation of Canadians in space-related activities, prioritization criteria vary with the scientific discipline. Preparation work for investment plans differs from one programmatic sector to another depending on the activity. Some sectors coordinate their operations in collaboration with other space agencies, and roadmaps are prepared on the basis of international opportunities and planned missions.

Each sector presents its director general with the investments it considers to have the greatest advantages for the Canadian space industry, the Canadian academic environment, Canada as a whole and the CSA. The investments approved by the director general are then presented to the Integrated Investment Review Board (IIRB) for approval, followed by AO preparation. Unsolicited proposals are all presented to the IIRB for approval.

The program budget was allocated among the sectors a few years ago, and it is renewed from year to year with some variations. Optimal planning of available resources should include a periodic review in accordance with any changes to organizational priorities and expected results.

Financial planning

A review of the budgets and expenditure forecasts for the ongoing fiscal year is conducted monthly using the SAP financial system. Potential surpluses are identified during the year and redistributed to another G&C program at the CSA.

For 2017-2018, the workplan total went from $21.6 M at the beginning of the fiscal year to $19.2 M at the end, with a decrease of $2.4 M (11.0%). For 2018-2019, the workplan total went from $23.8 M at the beginning of the fiscal year to $26.4 M at the end, with an increase of $2.6 M (11%). The increase occurred due to additional funds being obtained for new initiatives approved over the course of the year. Without those additional funds, the potential surpluses identified over the course of the year and transferred to another program were approximately $2.5 M (10.5%), which is a similar proportion to the previous year.

Organization

N/A

Function

N/A

Action plan details

N/A

Deadline

N/A

Criterion 2

Program activities are monitored appropriately.

Condition

Conclusion on the criterion:

Our audit enabled us to conclude that program activities are monitored appropriately. Expenditures are authorized and made in accordance with applicable laws, regulations and program terms and conditions, and the transactions are accounted for appropriately. The preparation and publication of AOs, as well as the reception, analysis, evaluation and selection of proposals, follow structured and documented processes and procedures. Recipients are monitored on an ongoing basis.

However, areas for improvement were identified with respect to updating the documentation for procedures associated with G&C operations in accordance with the new CSA Investment Governance and Monitoring Framework (IGMF/CGSI) and establishing a methodology for auditing recipients.

Applicable laws, policies, regulations, guidelines and Program terms and conditions

Our audit enabled us to conclude that expenditures are approved and made in accordance with the Policy on Transfer Payments, the FAA and the program terms and conditions.

To reach this conclusion, we reviewed a sample of 10 agreements: 6 contribution agreements and 4 grant agreements. Our review showed that expenditure authority is exercised in accordance with delegated authority (FAA sect. 32), that expenditures are made in accordance with G&C agreements, that payments are supported by certifications to the effect that the terms of the agreements have been met, and that invoices are certified under the FAA sect. 34 by an authorized party. Furthermore, payments are certified under FAA sect. 33 by an authorized party, and expenditures are accounted for appropriately.

With respect to payments, we selected the last invoice submitted for each of the 10 agreements in our sample. We concluded that the amounts invoiced generally corresponded to the basis of payment in the agreement. However, in two cases, we noted that part of the payment had not been retained as indicated in the agreement (5% and 10%).

Announcement of opportunity and proposals

We found that the preparation and publication of AOs, as well as the reception, analysis, evaluation and selection of proposals, follow structured and documented processes and procedures. These processes and procedures allow for compliance with the program terms and conditions.

We examined all the activities conducted as part of an AO and concluded that the processes and procedures had been applied as planned. The prescribed checklists and templates were used. The preparation and posting work was supervised by the G&C COE, and the AO was approved by the IIRB. Proposals were selected in accordance with the criteria set out in the AO, the results were communicated to the applicants and the agreements were signed by the appropriate delegation level. The G&C COE provided support to the team responsible for the AO throughout the process.

Recipient monitoring

In the course of our audit, we found that agreements were monitored on an ongoing basis through progress reports, annual reports and final reports submitted by recipients.

The annual reports and final reports are documented in the CSA G&C management infrastructure (UNITAS). The information is entered by recipients via the web portal using specific templates, and then transferred electronically into the UNITAS system. The annual progress report compares the results to the project's general objectives, so the project's ongoing eligibility is periodically evaluated. The monitoring is conducted by the project authority with the assistance of the technical authority.

Other progress reports are submitted during the year for the purpose of monitoring agreements. These reports are submitted by recipients in hard copy and/or digital format, but they are not transferred into the UNITAS system. Entering these reports into the UNITAS system would allow for complete information to be obtained from a single source.

Recipient audits

The Directive on Transfer Payments requires that a time be identified to conduct recipient audits, and that an audit plan be developed and implemented based on risk analysis. To meet that requirement, the CSA has produced two documents to direct risk assessment. Currently, no recipient audits have been conducted at the CSA, and none are planned. Furthermore, no plans have been developed and no audit methodology is in place to support such a measure. A proper plan developed using a precise methodology would enable the CSA to meet the requirements of the directive.

Roles and responsibilities

The roles and responsibilities for Program governance have been defined. The main stakeholders are the various branches of the CSA, the G&C COE and the G&C steering committee formed in 2017.

Among other things, the sectors are responsible for planning, implementing and evaluating AO processes, and for issuing and managing program agreements.

The G&C COE is responsible for providing advice and support to the sectors throughout the AO and agreement issuance processes. The G&C COE is also responsible for developing, standardizing, implementing and maintaining G&C processes, management tools and infrastructures, and for monitoring and reporting on certain aspects of G&C. The G&C COE provides support to the G&C steering committee and to the IIRB in its approval decisions for G&C projects.

Chaired by the CSA Vice-President, the G&C steering committee has two main roles. it supervises G&C governance and management for the CSA, and it provides strategic advice on the use of funding.

Processes and procedures

Processes, procedures, guides and templates have been developed to support G&C activities, and are available through the G&C COE Livelink portal. However, we noted that some have not been updated since the changes were made to the CSA governance processes. Updates to the operational guidelines, processes and procedures would enable them to reflect current practices.

G&C management infrastructure (UNITAS)

The UNITAS system is a digital G&C management infrastructure. It was developed and is supported by the information technology (IT) sector of the CSA. UNITAS has been implemented gradually since 2009-2010. Over the years, since the system has been in use, there have been new requests for added G&C functionalities. The various functionalities of the UNITAS system have been grouped into modules. An initial development plan for the UNITAS system was established in 2010, and updates have been conducted over the years. Some elements of the plan have been fully implemented, and others only partially or not at all.

The development of the UNITAS system is ongoing. The work being done involves electronic functions related to the preparation and posting of AOs, reception of proposals, management of service standards and proactive disclosure of agreements. Investments were approximately $200,000 per year in 2017-2018 and 2018-2019. These amounts are primarily made up of time spent by IT and professional services employees. Every year, UNITAS development needs are sent to the IT sector by the G&C COE and are analyzed. Due to limited resources, development priorities are established together with the G&C COE. Future developments are planned or discussed, including the automation of agreements and amendments, the automation of the documentation required for governance, a control point module, the upgrading of the agreement management dashboard, and availability of historical data on funded organizations.

With respect to the governance of the UNITAS system developments, several control measures are applied, and the work is guided. Annual implementation plans are established with deadlines. An annual plan is made for the distribution of IT labour based on the activities set out, and needs for professional services are established. Futhermore, documents showing system development and progress are produced regularly. Meetings are held between IT and the G&C COE to ensure that the development work meets expectations. The G&C steering committee has been consulted annually for the last two years, and consultations to determine the applicability of the IGMF/CGSI are ongoing.

Organization

Finance Directorate - G&C COE

Function

Manager, Grant & Contribution Centre of Expertise

Action plan details

Recommendation #1:

1. The G&C COE will develop a recipient audit plan
2. The G&C COE will implement the steps required to carry out the plan (e.g. select files to review, audit recipients, draft a final report)

Deadline

1. June 30, 2020
2. March 31, 2021

Criterion 3

Program operations are subject to accountability.

Condition

Conclusion on the criterion:

The audit showed that the CSA is generally in compliance with the reporting obligations set out in the applicable laws, policies, regulations and guidelines issued by the CSA and central agencies.

However, we did identify some deficiencies in terms of proactive disclosure and establishment of service standards. We also identified an opportunity for improvement with respect to obtaining information on the results of the program following the issuance of G&C.

Reporting obligations

The main external reporting obligations for the program are as follows: proactive disclosure of G&C issuance; establishment and communication of service standards; and disclosure of results in terms of public accounts, plans and departmental results. We found that the external reporting obligations are generally met. However, we did identify the deficiencies below.

Proactive disclosure

For 1 agreement of the 13 reviewed, its proactive disclosure was not included on the CSA website. According to the sector responsible, there was an oversight in entering the information into the CSA's proactive disclosure system. Following the audit, this agreement was published with the proactive disclosures for the first quarter of 2019/2020 with a "late publication" note. The sector responsible also states that it intends to develop a procedure to validate proactive disclosures once they have been published.

According to the G&C COE, improvements are in development with respect to the proactive disclosure process. A new functionality in the UNITAS G&C infrastructure should allow for the process to be automated from the AO. The new process should eliminate the risk of oversights by the various sectors that, until now, have been responsible for entering the information manually into a separate system. As the appropriate actions are being taken, no recommendations have been made in this report.

Service standards

With respect to G&C management framework, the Policy on Transfer Payments requires that reasonable and practical service standards be established for services provided to Canadians. The results are reported under the Management Accountability Framework (MAF) and published on the CSA website. To meet that requirement, the CSA has established service standards for the reception of proposals, decision timelines following the reception of proposals, and payment processing timelines following claims. Service standards are announced publically when the AO is issued.

The service standard results for 2015-2016, 2016-2017 and 2017-2018 were examined. One of the standards was not met: the response time for rendering a decision on proposals received following an AO. The standard established by the CSA, which can vary between AOs, is 26 to 46 weeks. The target is that 80% of the proposals received meet that standard. For the years reviewed, as shown in the table below, the target was met only once, in 2016-2017. The reasons given are an unrealistic assessment of the timeline and the deadline for the approval of proposals selected.

Results of G&C initiatives

Short-term results

G&C agreements include the disclosure of results obtained following the implementation of the activities associated with the agreements. Recipients must demonstrate the work completed after funding has been obtained. Reports are required, results and data on indicators must be communicated to the CSA. The CSA compiles the results, allowing it to meet its aforementioned reporting obligations.

Long-term results

In the audit of the Space Technology Development Program (STDP/PDTS) in December 2018, an opportunity for improvement was identified with respect to obtaining information on results following the issuance of G&C. It involved identifying the more long-term impacts and benefits that the agreements had generated for recipients. We considered that more detailed statistics and reports could better support planning and decision-making for future investments. An action plan is being implemented; it involves a mechanism for collecting information using surveys and communicating results in report form. We believe that a similar approach could be implemented to evaluate the results of the program's initiatives.

Because a recommendation was already made as part of the STDP/PDTS audit, we suggest that the G&C COE follow the progress of the action plan implementation and determine how it could be beneficial to the program.

Organization

• Finance Directorate – G&C COE
• Programmatic sectors or G&C users

Function

• Manager, Grant & Contribution Centre of Expertise
• Directors and managers, Programmatic sectors or G&C users

Action plan details

1. The G&C COE will add a specific component regarding service standards to the IIRB submission template to facilitate decision-making for senior management.
2. The G&C COE will review the process for establishing service standards with the sectors.
3. The G&C COE will update the CSA guide on service standards.

Deadline

1. March 31, 2020
2. June 30, 2020
3. December 31, 2020