Evaluation of the Space Capacity Development Program of the Canadian Space Agency

Senior Management Commitment

In addition, the transition to the SCDP's integrated approach represents a major change in organizational culture. Several internal and external respondents interviewed as part of the evaluation, including experts in organizational management and program managers with an approach similar to that of the SCDP, emphasized the importance of senior management commitment and support as a condition for success in defining and implementing the program's vision, as well as the optimal mobilization of the teams involved. While all employees undeniably have a role to play in integrating the various program initiatives and operationalizing the changes, it is up to senior management to take the lead in conceptualizing and promoting the program. The leadership provided by senior management of the program, and of the whole of CSA, is seen by these interviewees as essential to ensure the transition to the SCDP and the establishment of program priorities, as well as to facilitate teamwork.

Published annually for CSA employees, the Space Science and Technology Branch's Business Plan is a good opportunity to promote the SCDP vision and team spirit among Branch employees. The various case studies developed as part of this evaluation also illustrated past and present opportunities where senior management demonstrated its commitment to promoting collaboration among the various initiatives of the SCDP, even before the program was formally created. Projects such as LEAD or the CubeSat Project benefited from the outset from the program's senior management's desire to see several initiatives contribute to the achievement of common results for the development of space capabilities. That said, these projects are specific examples of collaboration. In order to promote an integrated approach, it is essential that senior management's commitment be reflected in all program activities and that it increase opportunities to demonstrate effective leadership to the various stakeholders (program representatives and CSA staff, partners and external recipients).

Fundamentally, leadership is about transforming people in an organization and leading them to higher performance as a result. […] Leadership is more than just meeting targets; it's about articulating where you want to go and how you might have to change to get there. (M. Roberto. 2011. p. 7)

Communication Strategy

Based on their experience in public administration or managing programs with a similar approach to the SCDP, all respondents to the Comparative Study emphasized the importance of developing and implementing a clear communication strategy to ensure that all stakeholders – internal and external – understand the rationale, nature and extent of the program integration process. Several program representatives questioned future changes and how their job responsibilities and the initiative to which they are attached will be affected by the transition to an integrated approach and the renewal of the program. These questions are a reminder of the importance of communicating, on an ongoing basis, a clear vision of the program and its objectives, structures and processes. The experts consulted also reminded us that it is preferable to communicate changes sufficiently in advance so as not to take work teams and program recipients by surprise.

Moreover, the public administration experts consulted are of the opinion that the communication strategy should include consultation mechanisms – both internal and external – whereas in the context of organizational change, stakeholder buy-in is often the result of a participatory approach. It is also recommended that the pool of stakeholders consulted be broadened to ensure a diversity of discourse and proposals for change.

How to foster links between different initiatives without overloading people with information; how to share the right information with the right people? (Senior Management Representative)

As such, the management framework setting out the foundations of the SCDP (CSA, 2018b) proposed the creation of an advisory committee made up of representatives of all program stakeholders, namely representatives of associations, academia, the space industry and other government departments. Established as part of the program's continuous improvement process, the purpose of this committee would be to provide strategic advice, guidance and recommendations on SCDP-related activities, as well as to discuss and assess opportunities for collaboration among organizations responsible for innovation support programs (CSA, 2018b; 2018c). The creation of this committee had not yet been approved at the time of writing this evaluation report. That said, the vast majority of SCDP initiatives have their own external consultation mechanisms to improve their activities or review their investments. The functioning of the existing committees and consultation mechanisms should be clarified as to how they would juxtapose or merge with the new SCDP Advisory Committee, without risking multiplying the channels of communication with external representatives.

The communication and information transfer strategies and practices deployed by each of the SCDP initiatives are also potentially to be strengthened in the context of an integrated program. At the time of the interviews, several representatives of the program's initiatives said they would like to have more information on the other initiatives making up the program, their mandates and priorities, and the schedule of upcoming activities, with a view to identifying possible collaborations. In addition, two of the SCDP initiatives come under the Space Utilization Branch and the Communications and Public Affairs Directorate.

Opportunities for dialogue between these two initiatives and those under the Space Science and Technology Branch should be maximized. The Space Science and Technology Branch also recently commissioned a report on innovation ecosystems in the context of SCDP implementation (Arche Innovation, 2019). This report reiterates the importance of communication among program teams and proposes several strategies, including the publication of a newsletter for program employees, the use of virtual collaboration tools, and the organization of working sessions between teams or lunch and learns sessions, as examples. That said, some communication mechanisms are currently being implemented, including the publication of the Space Science and Technology Branch Business Plan and biannual meetings with Branch employees. The evaluation therefore suggests that program managers confirm whether the information needs identified during the interviews have been met or, if so, the nature, scope and depth of the desired information and the preferred communication mechanisms.

Visibility and Promotion

The visibility of the program was also the subject of much discussion during the interviews conducted for this evaluation. The SCDP was first presented to Canadian space sector stakeholders in May 2018 and again in June 2019 at conferences organized by the Canadian Aeronautics and Space Institute. According to some of its representatives, the CSA should maximize the visibility of the SCDP's flagship initiatives while its partners and recipients are used to dealing with STD, ESA, and STEDiA initiatives – particularly with its Space Science and Technology Flight and Ground Investigation component. Other respondents, on the other hand, questioned whether the program should promote an integrated platform by developing its own brand image. In all cases, several suggested multiplying the platforms for promoting the program and/or its initiatives in order to reach a broad audience (e.g., through the CSA Web site, social media, webinars, or the Innovation Canada portal).

Also, given the current context of the space industry – marked by a revival and the emergence of many small and medium-sized enterprises (SMEs) driven by the New Space movement – it is important to ensure that simple and popularized language is used to capture the attention of these new players in the space sector who are unfamiliar with government jargon.

Finally, with regard specifically to efforts to raise awareness among young people and students, it is interesting to note that several external respondents wanted the CSA to increase the visibility of a wide variety of professions in the space sector (including engineers, technicians, scientists, computer scientists and artificial intelligence specialists) to complement the promotion of astronauts. They also suggest addressing various themes of interest to young people, such as climate change, or maximizing the use of rovers as promotional tools for young people.

Implementation Plan for the Integrated Approach

As previously mentioned, each of the SCDP initiatives, or their respective activities, were already part of the CSA's service offering prior to the creation of the program, with several constituting sub-programs of the 1.3 Canada's Future Space Capabilities Program (CSA, 2016a). Nevertheless, the promotion of a more integrated approach is initiating a paradigm shift in the management of these activities. At the time of the interviews, several CSA interviewees highlighted uncertainties regarding the operationalization of the integrated approach and the changes to be expected for each of the initiatives. These uncertainties highlighted the need for a detailed "game plan" to guide the implementation of this new integrated program and its timeline. A SCDP Management Framework (CSA, 2018b) is currently being developed; however, there is little documentation of the interrelationships between the various initiatives, as well as the strategies and preferred actions for implementing the integrated SCDP approach and ensuring greater coordination and collaboration among the initiatives. Program officials should continue their efforts to clarify the program's new directions and plan the changes needed to achieve the program's vision and harmonize the practices and activities of the various initiatives that make up the program. For some program representatives, this downtime is crucial to properly plan for integration and a successful transition to the new program.

Program Activity and Investment Planning

The planning of program activities and investments was central to the interviews, both internally and externally. The Space Strategy for Canada, released in March 2019, was very well received by the stakeholders interviewed (ISDE, 2019b). Nevertheless, some respondents stressed the need for the CSA to establish a detailed, long-term implementation plan for the Space Strategy in order to be able to better guide the selection and implementation of SCDP activities and help program partners and recipients plan their own activities and investments.

For many respondents, the process of prioritizing and planning activities in the context of implementing the new SCDP will have to lead to decision‑making; in short, choices will have to be made. One of the public administration experts interviewed for this evaluation warned program officials of the risk of dispersion inherent in adopting an integrated approach in the context of a program that will have to intervene in a wide range of activities, as is the case with the SCDP. It is important to clearly define needs and areas of intervention – based on past investments and their impact, as well as program and CSA objectives and expected results – in a limited budget context. Many recognized the sensitive nature of this type of exercise, but also the need to ensure greater predictability of interventions and operational consistency within the SCDP and with the CSA's other program sectors.

In addition, the evaluation identified opportunities for improvement with respect to program activity planning. Program governance is responsible for setting priorities and approving expenditures for the various SCDP initiatives. That said, some respondents called for the strengthening of investment planning and coordination mechanisms among initiatives in order to maximize the harmonization of collective interventions within the program and the program's potential impact on its target clienteles. The transitional phase toward an integrated approach thus provides an opportunity to review and clarify the roles and responsibilities assigned to the various initiatives under the program, and even those of the CSA's other program sectors. The SCDP aims to offer recipients a comprehensive approach – from idea to space – by intervening along the innovation continuum. Each initiative must therefore ensure that it enters the scene and mobilizes its resources at the most opportune time for its recipients.

The case studies also highlighted the challenges associated with planning the program's activities. Projects have received funding from the SCDP (or the sub-programs of the former Program Alignment Architecture that now make up the SCDP) on various occasions in past years, in addition to using program support during demonstration or simulation exercises. That said, some of these projects are awaiting a CSA decision on their ability to integrate a demonstration orbital mission, after having been tested in a quasi-space environment under the STRATOS program. Some CSA officials have called for a tightening of the CSA's project and mission prioritization, planning and selection exercises in order to increase value for money. To that end, they propose, among other things, that a roadmap covering all SCDP activities be developed to better orchestrate the program's activities according to departmental priorities and complement the roadmaps of the CSA's other program sectors.

"How far do we want to go with integration?" asked one respondent. "How should we work together? People are used to working in a certain way and until they see benefits to change, that's the status quo," said another program representative.

Organizational Model

The coming into force of the Policy on Results (TBS, 2016) led the CSA to review its programmatic structure. With the exception of Space Science and Technology employees assigned in a matrix format to projects conducted by the Space Utilization and Space Exploration Branches, the former Program Alignment Architecture (CSA, 2016a) mirrored the organizational structure, while the administration of each program was the responsibility of a single Branch. The current – results-based – model provides an innovative conceptual lens. That said, the discrepancy between the SCDP's organizational structure and its programmatic structure raised questions with several CSA representatives. The grouping of seven initiatives – attached to two branches and one directorate – is the very foundation of the program's conceptualization, which is called upon to intervene at various stages of space capacity development. Nevertheless, employees, who are used to working on one initiative, find it difficult to see how they can maximize collaboration with their colleagues in their own directorate, but also with teams from two other branches, in a coordinated and concerted manner, without risking compromising value-for-money efforts.

The reflex of distinguishing who is responsible for an activity at the organizational level and to which program that activity contributes is sometimes difficult to have. This is new and we still don't have all the structures or mechanisms to look at programs through this lens. (Senior Management Representative)

Furthermore, this dissonance between the organizational and programmatic structure could have an impact on the sense of ownership of the program, anticipates a senior management representative. This complexity of the SCDP's organizational model leads some program representatives to suggest that it might be desirable to consider a different grouping of SCDP initiatives. In the course of this evaluation, some questioned the possibility of redesigning the program's organizational structure while retaining the seven initiatives. Others, however, raised the possibility of taking the logic of integration even further by implementing a model in which certain initiatives would be consolidated or merged. No consensus was reached on this issue following the key informant interviews.

SCDP Governance

The complexity of the program's governance structure was also questioned during the interviews. While some initiatives already have their own steering committee, the program is considering doing the same for all initiatives. This multiplication of steering committees would make it difficult to share information among the work teams, thus replicating the old program structure in isolation. Conversely, several noted the absence of a steering committee for the SCDP itself. The SCDP Management Framework (CSA, 2018b) currently being developed proposes the establishment of a committee to oversee the steering committees for the various initiatives, but this had not yet been implemented at the time of the evaluation. In addition, the identification of expected results common to all initiatives, through the Performance Information Profile, has certainly represented a significant step forward in results-based decision-making.

That said, the transition that has begun towards greater integration of the initiatives is expected to materialize even more strongly in the program's governance structure. The establishment of an integrated structure could have the effect of optimizing operational coherence, generating more synergies between initiatives, increasing information sharing, and strengthening the sense of ownership of the program.

Resource Allocation

In addition, the organization and allocation of human, material and financial resources present many challenges for the SCDP implementation. The current structure for allocating resources among the various initiatives that make up the SCDP is considered too rigid by some CSA representatives. They would like to see the ratio of program investments devoted to the various components of the innovation continuum – that is, activities that promote youth learning, science and technology and application development, demonstration activities, consulting services and access to laboratories, and commercialization support – be more flexible in terms of the program's priorities and strategic directions, as well as the status of ongoing projects and activities. This change in practice, and this increased flexibility in the way the program's budget envelope is administered, would better reflect the integrated approach proposed by the program while encouraging collaboration among the various initiatives and their concerted effort to develop space capabilities. In addition, a few respondents, including funding recipients, questioned the splitting of support, financial or in-kind, provided to certain technology development projects or activities. This project splitting – and the uncertainty it creates – increases the risk for companies and researchers of losing highly qualified personnel and a loss of efficiency in their technology development activities. As well, it becomes more difficult to track funding for technology development when it is fragmented over time and across program initiatives.

In addition, collaboration takes time, as several respondents noted. The workload of team members currently leaves them with little flexibility to develop collaborations across initiatives and, if necessary, with external partners, or at least to ensure that they bridge the different stages of a comprehensive "idea-to-space" approach to space capacity development. Yet, employee involvement is at the very heart of the transitional phase initiated by the program and must help to ensure the sustainability of organizational changes.

Concerns were also raised about the ability of some teams to complete their activities due to a lack of human resources. The risk lies in the low number of staff available by area of expertise in a timely manner to complete projects. This non-redundancy in human resources planning could weaken their ability to support projects. To date, the teams have managed to keep pace, but the risk remains. In addition, the case studies revealed certain issues with respect to the availability of equipment – particularly the rover fleet – thus highlighting the importance of clarifying the priority level of recipients using this equipment, in the context of close involvement of the initiatives in joint projects. Complementarity between SCDP initiatives, promoted as part of an integrated approach, could be limited because of the risks posed by potential equipment breakdowns, in addition to the workload of the teams responsible for this equipment, which are often called upon by several projects simultaneously.

Some challenges were also raised during the interviews – both internal and external – as to the practices, processes and terms and conditions for funding program activities. The delicate issue of the role of experts in monitoring and supporting funded projects proved to be at the heart of the program representatives' concerns. Assuming that by offering a concerted range of services along the innovation continuum, the SCDP should maximize its contribution to the development of space capabilities.

This integrated approach calls for the participation of the various initiatives in carrying out projects funded by the SCDP. However, this collaborative model – which is expected to grow in the context of an integrated program – raises complex issues, particularly with regard to the distribution of the roles and responsibilities of the various internal and external teams involved in carrying out projects in order to guarantee the independence of the various stakeholders. The recent Guide to Departmental Collaboration with Grant and Contribution Recipients (TBS, 2019) states that "if only one department funds and collaborates on a project, a clear separation is required between the unit that administers the funding agreement and the unit that collaborates on the project.

The Canadian CubeSat Project, which is the result of close collaboration between the STEDiA, Capability Demonstration and Provision of Expertise and Facilities initiatives (see box), has faced challenges of this nature. How, for example, can the involvement of CSA experts be maximized without them interfering in the development of technologies – that is, CubeSats – whose design is the responsibility of funding recipients? One of the strategies identified by respondents was to ensure that the information provided by CSA experts, at the request of a team, was shared with all 15 participating university teams. Also, this technical or scientific information is likely to guide the teams in the design of their CubeSat but cannot dictate the use of a method or process, in order to preserve the independence of funding recipients. This example illustrates how those responsible for SCDP initiatives have succeeded in adapting their practices while maximizing the support offered to funding recipients.

A similar issue was raised regarding the loan of equipment and the use of some of the CSA-owned laboratories and workshops by external clients. A few respondents called for clarification of the guidelines and selection criteria for equipment loans and the use of these laboratories and workshops, which would result in operational efficiency gains for employees responsible for analyzing applications.

Finally, multi-year approval of projects or recurring activities was also proposed by a few respondents, which would lead to efficiency gains. Also, external respondents reiterated, as a good practice, the importance of simplifying funding applications and increasing the predictability of funding offer notices. This last point was raised during consultations recently conducted by the Application Development Initiative, as well as in the evaluations of the Space Expertise and Proficiency Program (which included the STEDiA Initiative; CSA, 2017b) and the Class Grant and Contribution Program (CSA, 2017a).

Strategic Collaborations

Collaboration must become a reflex in order to decompartmentalize initiatives. The effort is twofold: at the outset of any project, those in charge should ask themselves, on the one hand, about the potential contribution of other initiatives (or partners) to the realization of the project and, on the other hand, about the extent to which the project can contribute to the achievement of the greatest number of expected results. The objective is to strategically identify possible synergies between the initiatives according to the nature of the activities or projects and their objectives. The integrated approach proposed by the program does not require the participation of each initiative in all of the program's activities; rather, it is a matter of identifying the initiatives concerned based on the targeted results and the anticipated benefits for the program's clients.

Therefore, collaborations should be planned and structured from the earliest stages of activities or projects in order to secure the resources needed to carry them out. In addition, the case studies have shown that, on some occasions, the leverage effect generated by collaboration can be realized through the injection of limited additional resources. The organization of the Youth Open House at the Stratospheric Balloon Launch Base in Timmins and the presence of participants of the CAN-SBX Challenge are convincing examples of how this project was enhanced to maximize the program's contribution to the development of space capabilities in Canada.

This leverage effect can also materialize through intangible practices at zero cost to the program, such as modifying the requirements of funding agreements (for example, the STD initiative encourages the space industry to collaborate with the academic sector when implementing Announcement of Opportunities by favouring projects that will include the participation of students and academics). The added value of the program's integration is likely to be embodied in the multiplication of this type of practice.

Some respondents also suggested continuing to experiment with the integrated approach through cross-cutting pilot projects. This suggestion is consistent with the proposal made by the authors of the Innovation Ecosystem Report commissioned by the CSA (Arche Innovation, 2019). In addition, several respondents cited the Canadian CubeSat Project as a model for the future of the program. As illustrated in the case study, this initiative combined the strengths of the STEDiA, Provision of Expertise and Facilities, and Capability Demonstration initiatives to enable post-secondary students to design and build their own miniature satellite and participate in a real space mission, while once tested and certified, CubeSats will be launched to the International Space Station, from where they will be launched into orbit in 2021-22 (CSA, 2018d).

The persons responsible for the initiatives look at them in such detail that they do not see the links between them. More exploration of common ground would be desirable. (CSA Representative)

Strengthening Partnerships

In addition, several respondents – both internal and external – stressed the need for the SCDP to work more closely with other government departments in order to maximize complementarity of services and, once again, to benefit from a leverage effect. Collaborations already exist with several departments, including the National Research Centre of Canada (NRC). That said, there is a need to optimize business practices by increasing opportunities for interdepartmental collaboration. This area for improvement had already been highlighted in previous evaluations, including the evaluation of the Space Expertise and Proficiency Program (CSA, 2017b). In particular, the Innovation for Defence Excellence and Security (IDEaS) program was mentioned several times during the interviews as a potential partner.

The importance of strengthening collaboration between the academic research community and the industrial sector was also emphasized a few times by internal and external respondents. Some funding recipients would like the CSA to further facilitate the development of partnerships between these two key sectors. Program officials also mentioned the possibility of better coordinating SCDP investment efforts in these two sectors, which are often called upon to participate jointly or subsequently in the various stages of preparation for a space mission.

We need to think about what unites the initiatives. Where are the intersections? (Senior Management Representative)

Alternatives to the SCDP Delivery Model

The exploration of new service delivery and project life-cycle management models was also proposed by some CSA representatives, with a view to improving accessibility to services by recipients and the SCDP's response to their needs. The challenge for the program lies, as mentioned by several respondents, in its ability to seize opportunities as they arise while maintaining the predictability of its service offering.

Two models were proposed by CSA representatives interviewed for this evaluation. Some raised the idea of consolidating access to services and funding offered by the SCDP for recipients through a "single window" approach. This would not involve consolidating or merging the initiatives, but rather putting in place a mechanism that would act solely as a conduit to channel applications to the initiatives concerned. This "single-window" approach would make it possible to have an overall view of the activities and projects seeking support from the program and to avoid potential duplication. That said, several respondents raised doubts about the feasibility of implementing this approach because of the risk of not redirecting the application to the appropriate interlocutors. Moreover, this approach would represent a major change for recipients, who are used to responding separately to notices of funding offers issued by the various initiatives.

The use of a project gating process in the orientation and management of the project life cycle was also suggested by a few respondents. Under this approach, calls for proposals would be administered separately by the various SCDP initiatives, as is currently the case for the STEDiA, STD and Application Development initiatives, for example.

However, recipients could, in a single application, already identify all the services offered by the SCDP to which they would like to have access during the life of the project (such as financing, demonstration, expertise, qualification and testing services, or equipment loans, in particular). Respondents who suggested adopting this approach saw a two-fold benefit. On the one hand, this approach would avoid the need for recipients to resubmit a request to access the various services offered by the SCDP in a timely manner. As a result, recipients would have access to a wide range of services throughout the duration of the project, thus facilitating their navigation on the innovation continuum.

In addition, the adoption of a project gating process would allow program officials to periodically assess the status of projects and grant authorization to proceed to the next gating point. This approach would have the advantage of limiting the fragmentation of support for high-potential projects, but would require careful planning, a concerted approach, and flexibility in implementation.

Furthermore, these alternatives to the program delivery model should consider the current environment of the spatial ecosystem, as some stakeholders tend to move away from the traditional model of government program support based primarily on the provision of funding and/or long-term support. Some of these new players are more interested in one-time support (such as expertise, testing, or qualification services) that will serve as a stepping-stone on the innovation continuum.

Results, Indicators and Targets

Under the former Program Alignment Architecture (CSA, 2016a), each sub-program or sub-sub-program had its own Performance Measurement Strategy that included a logic model and performance indicators. As a result, performance measurement for the initiatives making up the SCDP was broken down into seven Performance Measurement Strategies totalling more than 100 indicators. The consolidation of these strategies into a single Performance Information Profile (PIP; CSA, 2018a) and logic model, as well as the rationalization of performance indicators, is undeniably a significant step toward the implementation of an integrated approach within the SCDP.

Thus, in general, the exercise that led to the drafting of the program's PIP was welcomed by most program representatives. A majority expressed satisfaction with the logic model developed and felt that it adequately reflects the program's expected outcomes. However, some respondents questioned the choice of certain indicators and associated targets. An initial report on the program's performance, covering the period from April 1, 2018 to March 31, 2019, was produced by program officials (CSA, 2019c). Of the SCDP's 14 output and outcome indicators, a large majority (12/14) had a target that had already been met or was in a good position to be met. This report, which identified methodological issues, confirms the program's willingness to refine certain indicators and to review the PIP soon.

According to the PIP, the indicators measuring the increased competitiveness of the space sector are all related to the ESA initiative. However, some respondents – both internal and external – regretted the lack of information on the medium-term benefits of projects funded by the STD initiative (including commercial benefits, development of partnerships, and integration of technologies into CSA or foreign missions). The same comment could apply to the Application Development initiative, as the evaluation of the Earth Observation Business Line (CSA, 2017c) showed that of the 41 application development projects surveyed, 21 had been commercialized and 15 others were expected to be commercialized. Moreover, one senior management member wanted to see a better linkage between the performance measurement tools of the Application Development and STD initiatives.

In addition, the STEDiA, STD and Application Development initiatives use the CSA's Class Grant and Contribution Program to Support Research, Awareness and Learning in Space Science and Technology for the delivery of G&Cs. A better linkage with the Performance Measurement Strategy (CSA, 2013) for this program was suggested by several respondents, as well as with the indicators and data-gathering tools used by the other CSA program areas. Some changes have already been made by those responsible for performance measurement at the CSA, but greater coordination of performance measurement practices is still desirable.

One of the public administration experts consulted for the evaluation pointed out the challenge of attributing results in the context of a program with an integrated approach. It then becomes more difficult to identify the initiatives or factors that have led to change. The issue of double counting was raised by a program representative when concerted action between initiatives could make it more complex for the various work teams to compile results.

The development of the Performance Information Profile (PIP) and Logic Model provided an opportunity to reflect on what is being sought to be accomplished and which initiatives contribute to the expected outcomes. (Program Representative)

Measuring the Added Value of the Integrated Approach

The evaluation highlighted the performance measurement challenges associated with the integration of initiatives under one program. The current SCDP logic model and its indicators focus on workforce and space technology development and the competitiveness of the Canadian space sector. These key components of the program's performance measurement have been built primarily by amalgamating the expected results of each of the initiatives that make up the SCDP. However, little effort has been made to measure the added value of the integrated approach. Previous evaluations have demonstrated the relevance and ability of each of the initiatives to achieve its expected results and thus contribute to the development of space capabilities. However, how can the added value resulting from their consolidation be qualified and measured? This fundamental question is not currently represented in program performance measurement documentation. Depending on the nature and extent of the desired integration of the program's various initiatives, efforts should be made to measure the status of implementation of the integrated approach and its benefits for program recipients and partners. Access to this data would also make it possible to adequately support a future evaluation of the implementation, relevance, effectiveness and efficiency of the program.

The challenge of the program's integrated approach is to clarify the links between its various components. How does one feed into the other? How is each component part of a greater synergy? To really bring the model to life, the CSA must demonstrate the added value of an integrated approach. (Space Sector Association Representative)

Scope of Youth Activities

With regard specifically to the Youth Learning initiative, the interviews revealed a disparity in the discourse regarding the activities under this SCDP initiative versus the other promotional and awareness-raising activities for young people that are part of the corporate function of the Communications and Public Affairs Directorate. The case studies also highlighted STEM promotional activities organized by other SCDP initiatives, such as the open house at the stratospheric balloon launch base in Timmins and the demonstration activities organized by MCSS at CSA facilities. The SCDP should determine whether all concerted and coordinated youth communications efforts should be identified and measured under the SCDP and adjust the level of resources allocated to this initiative accordingly.