Evaluation of the Space Astronomy Missions and Planetary Missions Programs

3.3.1 Documentation review

Documents external and internal to the SAM & PM programs were reviewed for content to evaluate the programs' continued relevance. Performance data pertaining to the achievement of the programs' outputs and outcomes over the evaluation time frame were documented in the Report on Performance Measurement for Sub-sub Programs SAM & PM^{Footnote 8} and a variety of files. The external documentation included various strategy papers, expert panel reports, assessments and industry submissions, as well as Government of Canada statements on the importance of research and development in the science, technology and innovation space. The internal documentation included Reports on Plans and Priorities, Departmental Performance Reports, the Performance Measurement Strategy, performance data and media tracking files, financial data files, summary reports on invitations received from international partners, and approximately 30 mission-level project files. Most of the documentation was loaded, coded and analyzed in Atlas.ti.

3.3.2 Key informant interviews

In-person group and individual interviews were conducted with CSA management and staff in Saint-Hubert. Twenty-four individuals participated in the interviews, including three director generals and the Program Director. All four planned interviews were conducted with representatives of NRC/Herzberg Institute of Astrophysics. Telephone interviews were conducted with three international partners; NASA, ESA and JAXA. All of the interviews were transcribed, loaded, coded and analyzed in Atlas.ti.

3.3.3 E-surveys

The e-survey was sent to 41 contact email addresses, one of which bounced back; 19 completed responses were received out of the remaining 40, for a response rate of 48%. The survey data cannot be considered statistically reliable because of the high margin of error. Nevertheless, valuable qualitative data were collected that provide an insight into how the SAM & PM programs are viewed by funding recipients.

3.3.4 Case studies

Two case studies were completed, one for each of the two programs under evaluation. The focus of these case studies was Canada's contribution to the NASA JWST mission and the Mars Science Lab/Curiosity rover mission. Two in-person group interviews of CSA staff involved in these missions were conducted, involving a total of six individuals. Telephone interviews were also conducted with NASA's program director for JWST, the Canadian principal investigators and an industry representative. Mission-level documentation and the transcribed interview notes were loaded, coded and analyzed in Atlas.ti.

4.1.1 Alignment with federal government priorities

The importance of innovation as a contributing factor to future productivity and economic growth has long been accepted in Canada by government, academia and industry.^{Footnote 9},^{Footnote 10},^{Footnote 11} As follows, the 2011 review of federal support to research and development by a government-appointed, independent panel set out quite eloquently how innovation contributes to productivity.

In the context of productivity growth, the process of innovation diffusion and adaptation is most important, since most innovation that occurs in any given area or jurisdiction is through adaptation of significant innovations originating elsewhere. The adoption/ adaptation by an individual enterprise of a new or better way of doing something is therefore also recognized as a form of business innovation — indeed, the most common.^{Footnote 12}

The CSA has traditionally accounted for the largest percentage of research and development contracted to Canadian business, estimated by Statistics Canada at $167 million or 60% of the total in 2010–11.^{Footnote 13} The important role that the CSA has played in supporting Canadian business innovation was evidenced in the 2012 Aerospace Review. Its recommendations focused on policy and program improvement specific to the space sector, including the establishment of a Canadian Space Advisory Council and Space Program Management Board, recognition of the importance of space technologies and capacity to economic prosperity and growth, stabilized core funding, and additional funding for technology development programs.^{Footnote 14}

Made public in February 2014, Canada's Space Policy Framework was to serve as a guide for Canada's future space programming activities, including space exploration that would inspire young Canadians to pursue studies and careers in science and engineering. The Government committed to the following points:

• Ensuring that Canada is a sought-after partner in the international space exploration missions that serve Canada's national interests
• Continuing to invest in the development of Canadian contributions in the form of advanced systems and scientific instruments as part of major international endeavours
• Continuing Canada's Astronaut Program so as to have Canadians aboard current and future space laboratories and research facilities^{Footnote 15}

The SAM & PM Performance Measurement Strategy was developed and approved shortly after the Space Policy Framework was made public. It drew its guidance from the framework's first two commitments, with emphasis placed on working with international space agency partners and contributing to joint international space exploration missions. Contained within the strategy is the SAM & PM Logic Model (Appendix A), which sets out the programs' outcomes with two clear streams of logic aligned with the Government's commitments for science, technology and innovation, as outlined above. The first stream of logic is focused on enhancing Canada's space exploration profile through advancing knowledge about the universe and our solar system, developing highly qualified personnel (HQP), and providing scientific communities with access to scientific data. The second is focused on generating economic growth through the transfer of know-how and technology to terrestrial and space applications by developing the competence of the Canadian space exploration sector, and through positioning the Canadian space exploration sector for future space opportunities by maintaining a presence in space through space missions. Both streams of logic and their ultimate outcomes addressed different aspects of the Government's Science and Technology Strategy to optimize Canada's people, knowledge and entrepreneurial advantages.^{Footnote 16}

In December 2014, the Government of Canada released its renewed Science, Technology and Innovation Strategy, the objective of which was to "strengthen Canada's position as a global leader in scientific research and innovation"^{Footnote 17} and "continue to support and deepen research across a broad spectrum of disciplines that include both discovery- and application-driven research."^{Footnote 18} The strategy was based upon three pillars. Encouraging young Canadians to pursue careers in science, technology, engineering and mathematics, as well as attracting and retaining HQP, were priorities of the strategy's People pillar. Supporting world-leading research with the potential to generate long-term economic benefits was a priority of the strategy's Knowledge pillar. Encouraging partnerships between academia and industry to drive innovation and position Canadian businesses in the global marketplace was a priority of the Innovation pillar.^{Footnote 19} This strategy affirmed the Government's intention to strengthen Canada's capacity for science and technology innovation and advanced research and development in sectors such as space exploration, where academia and industry would need government support. The SAM & PM outcomes remain well aligned with these priorities to inspire Canadians and advance space science capacity in order to position the country as a respected, innovation-driven and space-faring nation.

The 2015 Ministerial Mandate Letters for the Minister of Innovation, Science and Economic Development and the Minister of Science stated that the government intended to "to partner closely with businesses and sectors to support their efforts to increase productivity and innovation"^{Footnote 20} and to ensure that "investments in scientific research, including an appropriate balance between fundamental research to support new discoveries and the commercialization of ideas, [lead] to good jobs and sustainable economic growth."^{Footnote 21} The SAM-PM programs have, generally, remained well aligned with government priorities. However, the implementation of these priorities specifically as they relate to the CSA's Space Exploration Program has caused some concern among internal interviewees given recent Government announcements. It should be noted that document review and interview data indicate the current Government's priorities have been reflected in recent announcements to fund human space flight and post-ISS space exploration among competing space priorities, while the SAM-PM programs have gone without a newly approved and funded mission since the beginning of the evaluation timeframe.

4.1.2 Alignment with the CSA's priorities

The evaluation's internal document review of the CSA's Reports on Plans and Priorities and Departmental Performance Reports covering the entire evaluation time frame reveals that the CSA's Strategic Outcome has remained unchanged as follows, "Canada's exploration of space, provision of space services, and development of its space capacity meet the nation's needs for scientific knowledge, innovation and information."^{Footnote 22},^{Footnote 23}

The SAM & PM programs facilitate the participation of the Canadian space exploration sector, including academia and industry, in space astronomy and planetary missions most often sponsored by international space agency partners. These opportunities build on the existing strengths of the Canadian space industry and scientific community to contribute valued Canadian technologies and scientific expertise to the design, development and operation of spacecraft, subsystems, and/or scientific instruments.^{Footnote 24} Canada has demonstrated a world-class level of know-how, expertise and innovation on many international space astronomy and planetary exploration missions, making it a sought-after partner by space agencies around the world. This acknowledged capacity of the Canadian space exploration sector has taken years to develop and has given Canadian astronomers and planetary scientists access to scientific instruments and data that have led to new discoveries and advances in knowledge about the solar system and the universe.^{Footnote 25}

The SAM & PM programs are well aligned with the CSA's Strategic Outcome and specific priorities identified in the 2016–17 Report on Plans and Priorities, such as conducting "fundamental research and new discoveries" and "positioning of the space sector for global opportunities."^{Footnote 26} The interview data confirm the strong alignment of these programs with the CSA's mandate related to "advancing the knowledge of space through science and technology."^{Footnote 27} The programs were acknowledged to be conducting scientific research, providing scientific data and facilitating the use of information to advance knowledge.

4.1.3 Alignment with federal roles and responsibilities

The activities and objectives of the SAM & PM programs are consistent with the essential functions assigned to the CSA by the Canadian Space Agency Act, notably to "plan, direct, manage and implement programs and projects relating to scientific or industrial space research and development and the application of space technology," to "promote the transfer and diffusion of space technology to and throughout Canadian industry," and to "encourage commercial exploitation of space capabilities, technology, facilities and systems."^{Footnote 28}

In the same vein, the Aerospace Review report highlighted that

The third category of space activity is space exploration and science, which focuses primarily on satisfying our thirst and need for fundamental knowledge. The inspiring feats of astronauts, missions to the moon and other planets, space labs, and deep-space telescopes expand our understanding of the universe and our place in it. They are wellsprings of national pride and prestige, and generate technological and economic spinoffs. Such activities are almost always government-funded and, given their scale and complexity, usually carried out through international cooperation.^{Footnote 29}

The Aerospace Review report also recommended that the government "develop mechanisms to support the efforts of companies to keep their workforces technologically adept and adaptable through continual up-skilling."^{Footnote 30} Numerous other reports over the years have also encouraged the federal government to play an important role in supporting basic and applied research, as well as related training of HQP.^{Footnote 31},^{Footnote 32},^{Footnote 33} The National Research Council Canada (NRC) and the Natural Sciences and Engineering Research Council (NSERC), for example, have mandates related to research and the development of HQP.^{Footnote 34} NRC has responsibility for ground-based astronomy, but the CSA is the only agency that provides a scientific community with access to space astronomy opportunities and data. NSERC funds fundamental and applied scientific research, and was identified by e-survey respondents as another important funder of space research, although not in coordination with SAM & PM missions. Only the CSA has a mandate to "advance the knowledge of space through science and technology" and does so in coordination with the NRC Canadian Astronomy Data Centre and with little duplication or overlap with other government departments.

The research, development and innovation that take place in the SAM & PM programs have been consistent with the CSA mandate as stated above. Providing space hardware (instruments, subsystems or whole spacecraft) requires specialized infrastructure, expertise and resources that no other government department or agency possesses. The interview data also suggest that the CSA is best positioned to develop a technologically adept workforce for the space sector, as well as HQP researchers to analyze the scientific data, in order to fulfill its scientific mandate and to contribute to Canada's competitive advantage.

Although outside the time frame of the evaluation, it is noteworthy that Budget 2017 affirmed that the federal government has an important role to play in advancing science, research and innovation in the pursuit of building a skilled and stronger middle class. Specifically for space exploration, the fact that the government decided to invest in a mission to Mars indicates that planetary exploration is a priority. Following through on this principle was an $80.9 million cash allocation to the CSA beginning in fiscal year 2017–18 "to demonstrate and utilize Canadian innovations in space" in the field of quantum technology and to support Canada's participation in NASA's next Mars Orbiter Mission.^{Footnote 35}

4.1.4 Continued need for the Programs

Canada has a long and proud history as a space-faring nation, which is a source of national pride, prestige and inspiration for young Canadians to pursue careers in science, technology, engineering and mathematics fields. It seems only fitting that an evaluation of the continued need for these programs should be first examined from the perspective of the scientific communities involved.

The long history of astronomy in Canada, detailed in the James Webb Space Telescope (JWST) case study (Appendix B), has seen the Canadian astronomy community grow from roughly a hundred members in 1971 to approximately 300 professional astronomers and another 300 graduate and postgraduate students in universities across Canada. The community is among the most influential in the world in terms of its contribution to scientific advances in the field of astronomy and astrophysics. On the other hand, the planetary exploration community is relatively new, as described in the Mars Science Lab (MSL) case study (Appendix B). The community has grown from a small number of researchers at the turn of the millennium who conducted fundamental research that didn't require access to space, to an estimated 26 faculty members holding key research positions in top Canadian universities who have been or are currently science team members (Co-Investigators) in planetary exploration missions.

Of the SAM & PM funding recipients from academia and industry surveyed by the evaluation, 90% rated the continued need for these programs as significant, the highest rating possible. Three categories of needs were identified by the respondents:

1. Advancing knowledge in astronomy and astrophysics through supporting the development of scientific instruments and technologies, enabling Canadian researchers to participate in international space missions, and producing and facilitating access to scientific data.
2. Attracting highly qualified personnel and students to space-related research and improving their training.
3. Supporting Canadian companies to innovate, develop and commercialize high-tech products and services.

The three main arguments put forward by e-survey and interview respondents in support of continued funding of the SAM & PM programs are summarized as follows.

1. There is a need to continue the exploration of space in order to respond to fundamental questions about planets, our solar system, and the shape and composition of the universe. Respondents also indicated a need to understand the characteristics of the planet Mars and to develop new technologies to facilitate its exploration and habitation. Canadian involvement in space exploration through its leading scientists would contribute significantly to advancing knowledge in these areas.
2. There are currently no funding alternatives to the SAM & PM programs to provide access to the space facilities, missions and scientific data that are essential for the conduct of world class scientific research and HQP training, without which the respective scientific communities in Canada would be diminished. It should also be noted that documented research and interview data indicate that at least 5 HQP have pursued opportunities in the United States and elsewhere, while others have also expressed their intention to do the same should there be a further decline of opportunities to conduct scientific research in their fields.
3. The SAM & PM programs provide Canadian companies with the opportunity to develop advanced and reliable technologies for exceptionally challenging space applications that demonstrate their capacity to innovate, thereby raising their profiles and reputations internationally and making them more competitive in the international space marketplace.

The irregular cadence of the SAM & PM programs funding has been very challenging for the space industry leaders such as British Columbia-based MacDonald, Dettwiler and Associates (MDA) and Ontario-based COM DEV International (COM DEV), who have recently found new business partners in the United States. MDA's Canadian and US operations are now controlled by a US corporation with its headquarters in San Francisco.^{Footnote 36} In November 2015, the US company Honeywell announced the purchase of COM DEV International's space hardware and systems line of business,^{Footnote 37} leaving the space exploration business line in Cambridge, Ontario.

Similarly, international space agency partners (e.g., NASA, ESA) have formed other partnerships in the absence of Government of Canada support as evidenced in the MSL case study. The low interview response rate from international partners was revealing in this respect, as was the inability of respondents to answer many of the interview questions due to the amount of time that had passed since the last joint mission as "these missions took place many years ago." Canadian scientists' participation in international space agency missions and subsequent access to the scientific data are generally contingent on the provision of a technology hardware contribution supplied by a Canadian space industry firm and paid for by the Government of Canada. The evaluation found that there is a strong continued need for the SAM-PM programs to maintain the world-class scientific research conducted by Canadian space astronomy and planetary exploration community stakeholders. With a committed plan of mission approvals in the future, stakeholders would continue to benefit from the SAM-PM programs.

4.2.1 Outputs

Enabling science investigations refers to the development of knowledge by the scientific community involved in space missions in astronomy or planetary sciences. Science priorities are identified through information gathered about international opportunities and through consultations with the Canadian scientific community. Proposals are solicited through announcements of opportunities and funded with grants to the university proponents. These grant-funded projects can be used to increase the science returns of archived mission data or to define the science objectives and measurement needs for identified mission opportunities. Requests for proposals with well-defined statements of work are issued on a competitive basis and funded through contracts. Typical requirements have included business case input for mission opportunities, with option analysis, costing and scheduling, and the design and development of scientific instruments or subsystem concepts as part of pre-project activities for approved missions. Science investigations are supported to perform work for the CSA related to specific requirements of upcoming or ongoing space astronomy and planetary missions.^{Footnote 38}

Targets of two (2) and six (6) science investigations enabled by Canadian universities were set for space astronomy and planetary missions respectively over the course of the evaluation time frame. In the case of space astronomy, three universities worked on the ASTRO-H CAMS subsystem, and one other worked on the Herschel SPIRE instrument for the first four years. A fifth university began to provide support in the last year on the ASTROSAT UVIT instrument. The number of universities providing science support to the MSL APXS instrument increased from two (2) to four (4) over the course of the five years. On the other hand, the number of universities providing support to the OSIRIS-REx OLA instrument declined from four (4) to two (2) over the last three years. Two science investigations were terminated in 2012 after the withdrawal of NASA and subsequently the CSA from the ESA ExoMARS mission. Both SAM & PM exceeded their targets for grant-funded science investigations enabled over the evaluation timeframe.^{Footnote 39}

A target of one (1) science investigation under development and supported through contracts was set for each of the space astronomy and planetary missions over the course of the evaluation time frame. The target for space astronomy was exceeded considerably given the number of missions initially under consideration (e.g., EUCLID, SPICA) and in development (e.g., JWST – NIRISS/FGS, NEOSSat, ASTROSAT – UVIT, ASTRO-H CAMS). Similarly, the target for planetary missions was only exceeded in the first two years when Mars 2020 and the ExoMARS missions were under consideration, while MSL APXS was under development. Development of the OSIRIS-REx OLA instrument was initiated in 2013–14 and has remained the only science investigation under development for this program since. The number of universities, research centres and private sector companies involved declined markedly from nine (9) to five (5) for space astronomy and seven (7) to one (1) for planetary missions over the time frame of the evaluation.^{Footnote 40}

Canada's participation in international partner space missions is contingent on making a contribution in the form of a science instrument, subsystem or related component. Open and formal invitations to participate in international missions generally specify the scientific or technology requirements sought after by the sponsoring space agency. More personalized invitations are sometimes extended to Canada when the Canadian space sector is known for specific technologies and capabilities. For example, as described in the JWST case study, NASA sought out Canada's contribution for a mission critical subsystem in the form of an infrared tunable camera that would allow the telescope to be aimed very precisely. Based on the interview data, COM DEV's technology and demonstrated capability was key to the invitation and contract award; it also facilitated the CSA's negotiation with NASA of a scientific instrument as part of its JWST contribution, which will generate data for the benefit of Canadian astronomers and astrophysicists.

The CSA led two space astronomy missions since the inception of Canadas's space program: MOST and NEOSSat. More often, CSA has contributed science instruments and/or subsystems to other space agencies' space astronomy or planetary missions. Figure 1 presents the past, present and future operational missions by life-cycle phase over the duration of the evaluation time frame, followed by a brief description of each mission, scientific instrument and subsystem based on the available internal documentation and web research.

Figure 1: SAM & PM Missions by life-cycle phase, 2011–2016

Source: Internal document and file review

4.2.2 Immediate outcomes

The 2015–16 Departmental Performance Report on sub-programs and sub-sub programs states that five (5) space astronomy and planetary missions provided data to the Canadian scientific community, surpassing the set target of four (4) missions. Based on additional internal document review, as summarized in Figure 1, and the accompanying mission descriptions, the following space astronomy missions and their scientific instruments were in operation and transmitting data during the evaluation time frame: Herschel (HIFI, SPIRE), Planck (HFI, LFI), MOST and NEOSSat. On the planetary missions side, the APXS instrument has been the most consistent and productive provider of scientific data with the Canadian contribution of the APXS to the MSL Curiosity rover. These missions and their scientific instruments produced a variety of data types, including infrared, optical, X-ray and metric data, for unique research study purposes. Data quality has been an issue for NEOSSat since its launch, and testing continues to determine how the satellite can produce useful scientific data.^{Footnote 41}

The movement towards open data has increasingly gained momentum in space science. For example, NASA requires principal investigators (PIs) to ensure that all transmitted data are calibrated and archived within six months of the download date. In principle, this has broadened public access to the scientific data and given non-mission-related scientists with the appropriate software platforms and tools the opportunity to conduct their own research.

While the closure of mission operations signals the end of the data collection phase, pre-and post-operations support for data requirements definition, instrumentation, data reduction and analysis are equally as important. The NASA program provides end-to-end funding for scientific investigations into instrument design, software development, data reduction and returned sample analysis. Based on the interview data, NASA's competitive tendering criteria require mission contribution proposals to include funding for these scientific activities. This evaluation noted that the Herschel and Planck missions ended in 2013, that CSA support for post-operations data reduction and analysis was extended to March 2016, and that the CSA provided grant funding for the development of data analysis software for the Planck HFI/LFI instruments. However, according to the interview data, the principle of end-to-end funding for science team participation in SAM & PM missions has been unevenly applied.

The evaluation's two case studies documented the important role played by the PIs responsible for the science instruments in the success of the MSL and JWST missions to date. Close collaboration was critical with space industry partners and CSA staff to ensure that the instrument design met the data requirement parameters as defined by the science team. Early involvement of the science team in defining the mission's science requirements was identified as a key success factor in the interview data, as was an early commitment to grant funding for data analysis by the PI/science team. One of the barriers to science team participation identified in the interview data was as follows:

• The space industry partners' contract budget line item "science team support" left ample room for interpretation as to its purpose. What is often unclear is whether the funds are intended to be used when the industry partner requires support, or to be used to support the science team in carrying out its responsibilities?

The availability of grant funding to support PI/science teams during the pre-and post-operations mission phases was considered preferable, enabling them to better guide instrumentation design, develop data analysis software/tools and support post-mission scientific research. At present, they are considered disadvantaged relative to their research colleagues, who are unencumbered by the responsibilities of operating the science instrument and meeting the open access requirements, which leave little time to contribute to the advancement of knowledge.

The CSA Annual Survey of the Canadian Space Sector has tracked the number of highly qualified personnel (HQP) by Canadian region relative to the national workforce since 1996; "HQP measurement consists of tracking the number of employed engineers, scientists and technicians in the Canadian space sector."^{Footnote 42} Disaggregated data at the level of the sub-sub programs were not found in the available internal documentation. Issues with the reliability of the reported performance measurement data for fiscal year 2015–2016 also affects the reliability of the entire HQP data set, given the year-over-year fluctuation in the number of faculty members that received SAM-PM funding and the number of students and postdoc fellows who worked on SAM-PM-funded projects. Nevertheless, on average, the performance targets of 40 faculty and 30 students were surpassed at 55 and 46 respectively for the space astronomy program. As for the planetary missions program, the targets of 60 faculty and 50 students fell short at 14 and 29 respectively.^{Footnote 43}

The evaluation's two case studies do shed some light on the fluctuation in the number of HQPs receiving CSA funding or engaged in funded projects. For the JWST mission, COM DEV had employed at least a dozen HQP throughout the project, while the universities involved approximately a dozen faculty members. During the peak periods of designing, building and testing the NIRISS/FGS flight hardware prior to delivery to NASA, the COM DEV contingent of HQPs was estimated to have peaked at 50 people, the majority of whom would otherwise not have been employed with the company. The University of Montréal will also employ additional postdoc fellows in the autumn of 2017 to analyze the NIRISS data, developing expertise that would not otherwise be possible.

The MSL APXS case study followed a similar pattern, with a small core group of HQP working for MDA for the duration of the project, along with 10–15 university faculty and students working with the Principal Investigator. The number of HQPs employed by MDA may not have peaked to the same extent as at COM DEV due to the established nature of the APXS technology; however, the University of Guelph will employ additional faculty and students to continue to analyze the wealth of data that it has collected.

The very significant fluctuations in HQP performance measurement data at this sub-sub program level may be more understandable given the observed swings in HQP employment reported by the space industry and university partners over the course of a project's life cycle.

As illustrated in Figure 1 above, at the beginning of the evaluation time frame Canada had three (3) space astronomy missions (Herschel, Planck, MOST) and one (1) planetary mission (MSL) were in operation. The three space astronomy missions ended operations in 2013. Canada's space astronomy missions experienced some setbacks with the NEOSSat data quality issues faced since its launch in 2013, and the unfortunate loss of JAXA's ASTRO-H mission with its Canadian payload in 2016. The Indian ASTROSAT mission, with the Canadian UVIT science instrument, is operational and has been sending data since its successful launch in September 2015; it is currently the only operational space astronomy mission that gives Canada a presence in space.

NASA's MSL mission, with its Canadian APXS science instrument, has been operational since 2012 and is the only planetary mission that currently gives Canada a presence in space. It is a high-profile and by all accounts successful mission, to which Canada has made a significant contribution as detailed in the case study. It should also be noted, that NASA's OSIRIS-REx mission with the Canadian OLA science instrument aboard was launched in 2016 and will reach the Bennu asteroid in 2018, while NASA's high-profile JWST mission with its Canadian payload (i.e., NIRISS/FGS) is scheduled to launch in 2018-19 and will become operational one year later. The evaluation found that the important contribution made to this prestigious scientific undertaking has enhanced Canada's reputation. In sum, Canada's role and contributions in space astronomy and planetary missions over the evaluation time frame have enhanced Canada's presence in space.

The interview and e-survey data indicate that the SAM & PM programs have positively affected the competencies of academia and space industry partners active in the space exploration sector. Based on their personal experiences as members of a mission science team, new and enhanced competencies were achieved in software development, instrumentation, image acquisition, and modulation and data management, among others. From an institutional perspective, participation in space missions facilitated the recruitment to science programs of postdoctoral fellows and high-performing students, whose learning accelerated as a result of working on mission-related projects. Furthermore, the possibility of participating in highly visible space missions and/or research has spurred both young scientists' interest in space exploration and institutional development, as evidenced by the constitution of research teams at the University of Guelph and the creation of the Institut de Recherche sur les Exoplanètes at the University of Montréal, respectively.

The space industry partners also acknowledged the enhanced competencies accrued through their involvement in space astronomy and planetary missions, specifically in terms of large-scale project management, technology innovation, collaboration with academia and international space sector relations experience. These competencies could be reinvested in future space missions and commercial applications. Of the e-survey respondents, 48% assessed the improvements to their competencies as equal to or higher than what was expected, 38% responded that this was to a considerable extent, and 11% to some extent. Key competency areas identified by both stakeholder groups included science instrument development, software development and techniques for data processing, business development, and proposal preparation.

It is noteworthy to point out that space industry partners, even though improvement in competencies is reported, found it challenging to maintain their revenue streams and retain HQP between mission contracts. The sporadic and irregular cadence of SAM & PM mission approvals, with none over the past five years, has been a cause of concern because capacities and competencies dwindle in the absence of new contract opportunities. The procurement rules and practices exercised by foreign space agencies, who also favour their national space industries and scientists when awarding contracts, makes the successful development of alternate space markets challenging.

4.2.3 Intermediate outcomes

The Canadian space exploration sector's involvement in space astronomy and planetary missions was relatively modest given the total number of Canadian universities and space sector companies that could have potentially been involved. Internal document review revealed that there were 14 Canadian universities participating in national and international space astronomy missions in 2011 and the same amount at the end of the evaluation time frame in 2016; for planetary missions, the number of Canadian universities involved rose from 9 to 12 during the same time frame. Four (4) Canadian space sector companies participated in space astronomy missions and two (2) participated in planetary missions over the course of the evaluation time frame.

Although not large, this active segment of the space exploration sector was sought out over the years for its technological expertise and scientific knowledge in astronomy and planetary sciences, such as the design of optical systems for cryogenic applications, star tracking algorithm expertise, science algorithm expertise, data management and processing, and robotics. A compilation of invitations to participate in international missions from NASA, ESA, JAXA, and the Russian and Indian space agencies attest to the high regard with which Canadian academia and industry partners are held in the international community. The interview data collected in the context of the JWST and MSL case studies confirm that the high visibility of these two NASA missions contributed considerably to Canada's international competitiveness by demonstrating the application of innovative technologies, engineering and scientific know-how. Successful collaboration between academia, industry and government in both cases was noted as exceptional and uncommon in other jurisdictions.

Past experience, demonstrated expertise and a successful track record of accomplishments have led to an increasing number of space exploration opportunities. For example, the interview data attributed the involvement of Canadian scientists in the ESA Planck mission as key to their participation in the ESA Euclid mission; the Canadian contribution to NASA's Far Ultraviolet Spectroscopic Explorer (FUSE) mission, with fine error sensors used to stabilize and point the telescope, was attributed with the invitation from NASA to contribute a Fine Guidance Sensor for the JWST; MDA's role in building the Mars Phoenix lander was attributed with a successful proposal for an ESA ExoMars rover contract; and, more recently, Canada's contribution to NASA's Phoenix lander and MSL Curiosity rover missions was attributed with a competitive and successful bid to participate in the OSIRIS-REx mission.

The e-survey response data confirm these findings, as 42% of the respondents met or exceeded their expectations to undertake follow-on space exploration work, while 37% met their expectations to some extent. Several examples were cited of direct attribution: having Canadian scientists involved in the ESA Herschel mission was attributed with an invitation to participate in the JAXA Space Infrared Telescope for Cosmology and Astrophysics (SPICA) mission scheduled to launch in 2027–28; similarly, the development of the Canadian ASTRO-H Metrology System (CAMS) for the JAXA ASTRO-H mission was attributed with a successful contract bid by the company to provide the same laser metrology system technology for ESA's Proba-3 dual satellite mission.

There were, however, some opportunities for the Canadian space exploration sector that were not seized for which Canada was well positioned; these include NASA's Mars 2020 and New Frontiers-4 missions, the circumstances of which are described in detail in the evaluation case studies. Other opportunities were not pursued as well, as evidenced by the lists of invitations the CSA received for space astronomy and planetary missions over the evaluation time frame.

The evaluation has already assessed in this report the extent to which competencies were enhanced for both industry and academic partners through the participation in space astronomy and planetary missions. The knowledge, expertise and confidence that were acquired with past mission experience, as well as the innovative technologies that were developed, could have been transferred to other space-related or terrestrial settings. The extent to which this occurred has not been rigorously studied, documented or reported. The notable exception was found in the 2013–14 Departmental Performance Report,^{Footnote 44} which cited seven examples of space exploration knowledge and technology reuse; Opal-360, LiDAR, Nuvu Cameras, Sitelle, Xiphos Technologies, NORCAT, and Key Mars Rover Elements.

The flux of large and small space astronomy and planetary mission projects involving Canadian industry and academia over the past decade has created a unique space exploration sector demanding a high degree of mobility among HQP. As previously mentioned, the attraction and retention of HQP by the space industry firms involved has been challenging; according to the interview data, the acquired know-how is often transferred to another setting in academia, government research labs or other recently successful space industry companies. The acquired know-how is never lost to the extent that HQP tend to remain within the Canadian space sector; however, that has not always been the case due to the lack of opportunities, as previously described.

The e-survey data are telling in this regard, as 31% of respondents either met or exceeded their expectations for know-how and technology transfer to other space exploration applications, and only 10% met or exceeded their expectations for transfer to terrestrial applications. The continued need for a robust space exploration program was a critical factor in the extent to which the acquired know-how is transferred.

The technologies developed for space as described in the JWST and MSL case studies were designed to operate in cryogenic and otherwise extreme environments for which there are few, if any, parallels on Earth. Highly innovative and proven to meet exacting performance standards, these technologies may only find a terrestrial application in the distant future as the innovation process is not predictable, linear or fast. The interview data provided some "spin-in" and "spin-out" examples, such as MDA's light detection and ranging (LiDAR) technology. At the very least, these companies can claim to have a technology innovation edge, having proven performance under the most extreme conditions.

A few technologies developed for space uses that were then "spun-out" for terrestrial applications were also identified in the interview and e-survey data; they included a terahertz microscope for breast cancer diagnosis, a spectrometer for nuclear fusion diagnostics, miniature digital cameras for biomedical usage, Mars rover platforms used in heavy duty vehicles, remote sensing instruments for extreme environments, and aerial observation technologies. Some of these examples overlap with those previously cited in the 2013–14 Departmental Performance Report and appear to be common points of reference to evidence the potential for economic benefits. What is more evident from the previous finding and supporting text, however, is the high potential for the reuse of know-how and innovative technologies in future space missions.

The JWST case study sets out a brief history of astronomy in Canada, the expansion of the Canadian astronomy community, the development of ground-based and space observatories, and the growth of its professional association, the Canadian Astronomical Society, which plays an influential role in setting the scientific priorities for the community. While somewhat dated and broad in scope, a 2010 bibliometric study of the performance and impact of Canadian research in astronomy and astrophysics drew the following conclusion based on the industry standard indicators of average of relative citations (ARC) and the average of relative impact factors (ARIF):

Canada stands out among world leaders in the field, mainly for the impact of its research on the scientific community (second place for ARC) and for the quality of its research (first place for ARIF).^{Footnote 45}

More recent bibliometric data collected by the CSA library indicate a continued growth in scientific production by SAM & PM funded researchers, as measured by the number of peer-reviewed scientific publications produced. The number of publications almost doubled in the case of space astronomy, from 82 to 149, while that of planetary sciences increased from 15 to 20 over the evaluation time frame.^{Footnote 46} The interview data also confirm the scientific production of the space astronomy scientists involved in the ESA Planck mission.

In exchange for Canada's support, Canadian researchers were able to obtain access to the data collected by the mission on the cosmic microwave background. An early data release took place in 2011, followed by a public release of data in 2013 and a further, more complete release in 2015. 28 research papers relating to the 2015 data were published in Astronomy & Astrophysics in October 2016.

Similarly, the interview data with regard to the ESA Herschel mission indicated that the HIFI science instrument produced a strong scientific output about detecting the presence of water in space, and resulted in 33 published papers, 25 of which were directly linked to the guaranteed time that Canada received. On the planetary mission side, the APXS data was also widely used by the science team, two thirds of whom used the data in over 60 peer-reviewed publications. Although outside the time frame of the evaluation, the Canadian NIRISS science instrument onboard the JWST also has a very high potential for important scientific discoveries as a result of the 420 hours of allocated observation time.

4.2.4 Ultimate outcomes

Sustained economic growth is generally understood to be the increase in the value of goods and services produced by an economy for an extended period of time without interruption. Based on the financial statements provided by the CSA's Finance Directorate on March 31, 2017, previously presented in this report (Table 1), the SAM & PM programs disbursed $111 million over the five-year evaluation time frame. The flow-through to the space exploration sector—based on the breakdown by disbursement category of grants and contributions, capital expenditures and contracts—amounted to 83.5% of the total, or $92.7 million. By comparison, the Canadian space sector generated total revenues of $3.4 billion in 2011^{Footnote 47} and $5.3 billion in 2015.^{Footnote 48}

A closer analysis of space revenues by subsector of activity permits a general assessment of the relative importance of the SAM & PM programs to economic growth. Revenues for the space exploration sector were estimated at $114 million in 2009; they decreased to $86 million by 2013,^{Footnote 49} before rebounding to $112 million in 2015.^{Footnote 50} Space exploration revenues declined relative to the space sector, from 3.35% to 2.11% of total revenues between 2011 and 2015. With combined total flow-through disbursements of $92.7 million, or $18.54 million annually, to the space exploration sector stakeholders, the SAM & PM programs represented approximately 16.5% of the space exploration subsector's revenues and less than half of a percentage point of the total revenues of the Canadian space sector as a whole. While the direct economic stimulus may not be significant, it may be worthwhile to examine the indirect economic benefits of these programs.

The evaluation examined the extent to which the SAM & PM projects led to follow-on funding by other organizations and/or contracts with other space agencies. The e-survey data indicated that there were few alternatives for space exploration research projects. Of the survey respondents, 63% identified these programs as a sole source of funding, while 37% reported other sources—notably research granting councils (e.g., NSERC) and researchers' own organizations. The SAM & PM funding contribution was estimated as more than 70% of the total project cost for 80% of the respondents. A singular and significant exception was identified in the JWST case study, where the University of Montréal has been endowed with $2 million over five years from the Trottier Family Foundation to establish a new Institute for Research on Exoplanets. It will fund students and postdoctoral fellowships to analyze the data collected by the NIRISS science instrument and publish the research findings.

Follow-on contracts with other space agencies are uncommon for individual scientists and the space industry stakeholders, although a number of cases have been identified in this report. The interview data identified the following challenges facing the Canadian space industry's attempts to export their space-related goods and services: procurement policies and practices that favour the use of national scientists and firms, export control laws and regulations (such as the US International Traffic in Arms Regulations), as well as strong international competition. A perceived strength of the Canadian space industry is to leverage innovations developed through participation in space missions and use them to establish niche areas of global technology leadership in optics, robotics, fuel cell and laser technologies.

The Canadian light detection and ranging (LiDAR) technology was one of the most frequently cited examples of a spin-in technology and space-use application cited in the interview data, and this evidence was supplemented by web-based grey literature. LiDAR uses light in the form of a pulsed laser to measure variable distances and was extensively used in Canada for terrestrial applications such as remote sensing. It was further developed for space applications by companies such as Toronto-based Optech and MacDonald, Dettwiler and Associates (MDA), who built the weather sensing system as Canada's contribution to NASA's Phoenix Mars lander. MDA then applied this expertise and technology to Canada's science instrument on NASA's OSIRIS-REx mission, launched in 2016. The OSIRIS-REx Laser Altimeter (OLA) assists in navigating the spacecraft towards the Bennu asteroid; it will scan the asteroid, create a 3D digital map and assist in landing the spacecraft. The Ottawa-based Neptec Design Group also has a long history of using laser technology for space applications in the US Space Shuttle program and the International Space Station. Its TriDAR (i.e., Triangulation and LiDAR Automated Rendezvous and Docking Sensor) technology, which is used to deliver cargo to the space station, has considerable potential in the evolving commercial space market. Further development of advanced space-based sensors and autonomous systems for space navigation led to a European export contract to develop and deliver navigation and localization cameras for the ESA ExoMars mission.

One of the most frequently cited examples of a spin-out technology and terrestrial application cited in the interview data involved the Lethbridge Alberta-based company Blue Sky Spectroscopy. A review of grey literature obtained by the evaluation revealed that the company "leveraged more than two fold through contracts from the European Space Agency, International partners and public-sector collaborators to develop a Terahertz microscope for breast cancer diagnosis." CSA funding to support the Canadian Principal Investigator on ESA's Herschel mission to design and develop the Spectral and Photometric Imaging Receiver (SPIRE) instrument contributed to the development of the technology and its medical application. The technology will allow pathologists to pre-screen biopsy samples to first determine if they have a cancerous signature before further diagnostic processing, thus reducing the time and cost associated with cancer detection.

Assessing the contribution to sustained economic growth of these or other technology innovations applied to terrestrial and space settings would require extensive research. It would also involve the collaboration of the space industries directly involved and sufficient evidence to attribute the economic benefits in terms of increased revenues and employment to SAM & PM program funding for design, build and launch contracts and/or space research grants and contributions. While such an endeavour is beyond the scope and resources available to this evaluation, it is a necessary undertaking to demonstrate outcome achievement. Nevertheless, the examples cited herein have considerable potential to contribute to economic growth—assuming that the companies involved remain financially viable and are not merged with or sold to foreign corporations and their assets moved outside of Canada.

Two instances of this latter scenario occurring are worth noting based on the case study data and previously reported herein. MDA's Canadian and US operations are now controlled by a US corporation.^{Footnote 51} In November 2015, the American multinational conglomerate Honeywell International Inc. announced the purchase of COM DEV International's space hardware and systems line of business^{Footnote 52} and merged it with the company's defense and space business line; it did not include in the purchase COM DEV's space exploration business line in Cambridge, Ontario. These events may impact sustained economic growth in a Canadian space exploration sector that has not grown over the past five years.

CSA staff and Canadian science teams and space industries have played important roles in NASA's high-profile JWST and MSL missions, while the APXS and NIRISS/FGS instruments and subsystems have fulfilled and will fulfil critical scientific functions, as described in the evaluation's two case studies. Canada's reputation with NASA as a trusted and reliable partner in space exploration has been enhanced because of these contributions. The Canadian space sector has long had a can-do reputation among international space agency partners for delivering high-value science instruments and subsystems with low budgets and on schedule. The interview data also confirm that Canadian scientists are highly regarded by their international peers and are sought after teammates. The overall positive sentiment expressed in the interview data was also reflected in the e-survey data, as 79% of the respondents assessed the impact of the SAM & PM programs on Canada's space exploration profile and visibility as equal or superior to their expectations, citing publications, press conferences, press releases, speaking engagements and feedback from international space agency personnel.

Canada's responsiveness to invitations to participate in space missions has not always been clear or timely. For example, for NASA's Mars 2020, the Government of Canada confirmed that it would not participate only near the end of the instrument selection process. This came as a disappointment to international partners and Canadian stakeholders who had submitted proposals at risk, without formal support from the CSA.The interview data that could be collected from this important stakeholder group were telling in this regard: "our cooperation with Canada in space science has been spotty at best in the recent decade," and "Canada once played an important role, so it would be regrettable if the capacity and capabilities developed in Canada were lost." The profile and visibility of the SAM & PM programs among Canadian and international partners is perceived to be at risk: "There is a strong wish, and hope, that Canada should be more involved."

With media coverage and interest, 267 national lay press articles were published about Canadian space astronomy missions or missions with Canadian participation, whereas 460 international lay press articles were published over the evaluation timeframe. Similarly, 245 national lay press articles were published regarding Canadian participation in planetary missions, whereas 376 international lay press articles were published.

Case study: Mars Science Lab – Alpha Particle X-Ray Spectrometer (APXS)

Case study: James Webb Space Telescope – Near Infrared Imager and Slitless Spectrograph (NIRISS) and Fine Guidance Sensor (FGS)