Evaluation of Human Space Missions and International Space Station Utilization

Relevance

Canada invested $2.2 billion from the start of the ISS to 2017 to participate in its construction and maintenance. This investment gives it the right to carry out scientific research on board the ISS using up to 2.3% of the available resources of the international portion (facilities, crew time and associated services, such as the transportation of research equipment and supplies). Human space missions and ISS Utilization are Canada's return on investment in the Mobile Servicing System.^{Footnote 2} Also, the document review demonstrates the interest that Canadians have in the importance of space exploration, an opinion that is shared both by the general public and the Canadian space sector. The five specific needs, identified in the program performance measurement strategy, were deemed relevant and are still in keeping with the new Canadian Space Strategy launched in March 2019:

1. to position Canada at the forefront of science and technology;
2. to recruit, train and secure flight opportunities for astronauts;
3. to identify, understand, reduce and eliminate the risks to the health of astronauts;
4. to design, build, launch and operate space hardware to conduct life sciences experiments and support the health of astronauts; and
5. to create new jobs in fields such as engineering, science, manufacturing and administration.

Since 2013, needs have not changed significantly, but the context has, and the program was adapted to remain relevant. In 2012, the program decided to focus on certain research priorities in the field of health and life sciences: fields with the potential to resolve health-related problems on Earth. In terms of astronauts, new selection criteria were used during the last recruitment campaign to adapt to the increased duration of missions from approximately two weeks to over six months. For example, considering that long space missions have become the norm, the CSA tightened the requirements related to the physical, psychological and social health of the candidates to support the profile of these missions. The CSA sought out individuals who were able to work well on a team for long periods in a confined environment.

In addition to the five needs identified by the program, the evaluation did not identify other needs not covered by the program, but some interview respondents said that public awareness of Human space missions and ISS Utilization was not clearly represented in the five needs, although this is a key element of the program.

The delivery model is deemed sufficient and the new Space Strategy allows to position the program for the future. Regarding human resources, interview respondents indicated that some positions were very specialized and unique, thus posing a risk of not achieving the results (loss of expertise in the event of an employee departure), but the small size of the team also allows to be agile.

Regarding governmental priorities, the activities undertaken through the program are in keeping with the Canadian strategy to use innovation as a lever for economic development. Space exploration missions require the development of state of the art technology, including applications that bring together space and Earth needs. The recruitment of new astronauts and the announcement in February 2019 of Canadian participation in the Lunar Gateway project demonstrate Canada's long-term commitment in this field.

Concerning the CSA's legitimacy to intervene through the program, it is clear that the CSA plays a unique role that could not be taken on by others, whether in the public, private or academic sector. There is no other program in Canada that is pursuing the same objectives or producing the same results. Through its enabling legislation, the CSA is a federal agency whose task is "to collaborate with space agencies, or organizations working in a related field, from other countries on the peaceful use and exploration of space."

Results

Overall, the outputs and results targeted by the program were achieved and surpassed. Canadian researchers and astronauts are recognized by the international community and garner the interest of the general public. Canada's allocation of time to use the ISS is 2.3%, which represents approximately 20 hours of crew time per six months. When partners cannot fully use all of their own allocation, the other partners may use the remaining time. As a result, the utilization target of 100% of the time per year was surpassed, except in 2015–2016, when 87% of the allocated time was used. According to an interview respondent, the CSA has practically doubled this allocation by collaborating with ISS partners for some research (e.g., using the same sample for different research). The international partners also acknowledged the excellence of Canadian astronauts by assigning them to critical duties and leadership roles during their mission. For example, Chris Hadfield was commander of the ISS for expeditions 34/35; Jeremy Hansen is currently responsible for the new class of astronauts at NASA; and before 2013, Dave Williams and Steve MacLean held important management positions at NASA. During the interviews, respondents, including ISS partners and external stakeholders, stated that the Canadian astronauts are highly qualified and able to perform well under pressure.

The lead researchers working on research projects that receive funding from the Agency employed highly qualified personnel (HQP) (58 per year on average) and students (13 per year on average) to help them with their work, thus contributing to the development of a highly specialized labour force in Canada and ultimately positioning Canada at the forefront of science and technology. Each year, on average, five Canadian university establishments (universities and teaching hospitals) participated in one or two research projects during the period evaluated, and, on average, six Canadian businesses participated in a research project.

Some projects led to use on Earth and re-use in space. For example, research studies funded by the CSA, which were begun in 1998, led to the development of the OSTEO instrument. Since then, this instrument has been used in a number of missions on the space shuttle, in particular with NASA in 2015 and the ESA in 2018. Also, Carré Technologies launched a commercial version of the bio-monitor called AstroSkin in the form of a line of intelligent clothing that can be used for research purposes in health, aerospace, security, defence and pharmaceuticals.

During the evaluation period, from 2013 to 2018, 47 peer-reviewed publications resulting from program funding were published. These publications were cited 254 times in other scientific articles, 5.4 times each on average, and 58% of them were an international collaboration with 10 countries, involving co-authors from 26 foreign teaching and research institutions and four foreign space agencies.

The results demonstrate a considerable collaboration both with Canadian and foreign universities and with the Canadian industry. This collaboration benefits Canada's profile abroad and allows for the advancement of knowledge about the ability of humans to live in space. Also, certain projects leading to re-use in space or use on Earth contribute to the Canadian economy and to other social benefits. The evaluation did not identify any unexpected negative or positive impacts.

Efficiency and economy

The evaluation focused on the qualitative aspects of efficiency given the absence of a comparable program and the varied nature of the program activities. The international cooperation required for projects as large and expensive as the ISS allows Canada to participate in scientific breakthroughs and the resulting technology. Also, Canada helped to advance the state of the art technology industry (in the fields of aerospace and robotics) by getting involved with the ISS. On a national level, the program collaborated with a number of partners, other government departments and agencies, the research industry and academia, leading to discoveries and innovative ways of treating illnesses, the creation of technology and discovery of new advances in life sciences and health.

It was mentioned during the interviews that the memoranda of understanding (MOUs) underlying the IGA agreement and the governance of multiple committees and working groups between partners are efficient and well established. The document review and interviews conducted as part of this evaluation nonetheless identified two factors that would contribute to improving the program's efficiency in the long term: increasing flight opportunities and having more collaborations with foreign partners. Canada is the smallest ISS partner,^{Footnote 3} which does not prevent it from making very large accomplishments. Also, Canada ranks well internationally with regard to the space field, whether for its contribution in science and robotics, its effectiveness, its efficiency or the quality of its collaboration. The human resources working on the program are deemed competent, but the small size of the group is also an issue for employee development and career opportunities. In interviews, some people said that the introduction of a new approval process in recent years had created additional pressure on human resources. However, interview respondents agreed that the process, which was more demanding than before in terms of preparation remains nonetheless more rigorous.

Impacts of the program on GBA+ groups

In March 2017, the CSA adopted its own directive for GBA+, which allows us to operationalize this requirement within the specific context of the Agency. The CSA directly integrated GBA+ considerations into its recruitment campaign for two new astronauts, and into studies on bed rest and isolation conducted in collaboration with the ESA. For example, during the latest recruitment campaign that led to the hiring of Jennifer Sidey-Gibbons and Joshua Kutryk, the following activities were undertaken: training on GBA+ taken by recruitment committee members before carrying out their mandate, a campaign that directly targeted women and visible minorities, and efforts made to engage northern communities and Indigenous peoples.

Conclusion

The evaluation demonstrated that the program is relevant, and that it achieves the targeted results while remaining efficient. It allows Canada to continue to play a key role in human space missions and also in research in health and life sciences in space.

The needs identified at the start of the program are still present and current. The ISS partnership model provides Canada with the opportunity to benefit from the use of a unique research laboratory in space and the resulting scientific and technological opportunities, in addition to producing significant benefits on Earth for the entire Canadian population. The immediate results of the program were not only achieved, but in the case of HQP and students, the established targets were greatly surpassed. The intermediate results were mostly surpassed and the final results demonstrate that the program achieves the expected results, especially with regard to the quality of the science and the promotion of Canada internationally. Lastly, the delivery model is considered effective and efficient despite the small size of the CSA. The personnel assigned to the program, including the astronauts, is deemed competent and have a good reputation nationally and internationally. Also, the report highlights the significant leverage effect of partnerships between the CSA and the other ISS member space agencies that are coordinating their research efforts.

For these reasons, the report does not contain any recommendations and no management action plan is required.

Partners and target populations

Partners of Human Space Missions and ISS Utilization include:

• The partner space agencies of the ISS: NASA, main partner that plays a leading role in the ISS, is responsible for all management of the ISS and the integration of activities and equipment in the international section of the ISS. It is also responsible for the transportation of Canadian research equipment and supplies. The space agencies of the other ISS partners are Russia represented by Roscosmos, Japan represented by the Japan Aerospace Exploration Agency (JAXA) and Europe represented by the ESA. Canada is represented by the CSA.
• University institutions: the scientific community (scientists, doctors and engineers at universities and research centres and hospitals in Canada) developing and/or using instruments in space laboratories or carrying out experiments in laboratories on Earth concerning the risks to human health associated with human space missions.
• Private businesses: the working groups led by the industry, associations representing the industry and businesses involved in the development, construction and operation of instruments and software.
• Other federal departments and agencies that are collaborating on the achievement of horizontal objectives and that are consulted at the various stages of projects, planning and implementation.

ISS partners collaborate widely in scientific experiments, in the development of instruments and the documentation of results, in scientific publications and other media.

The target populations^{Footnote 11} include:

• University institutions and research centres involved in the research and development of space-related science and technology;
• Private businesses and not-for-profit organizations: small, medium and large enterprises, including not-for-profit organizations involved in the development of science and technology related to space exploration;
• Canadian citizens: young people and the general public.

2.3.1 Financial resources

The data in Table 1 demonstrate that the majority of resources (87%) went to financial support for external entities (54%) (43% in contracts, 6% in memoranda of understanding and 5% in grants and contributions) and salaries (33%). The other expenses were for the ongoing management of the program through operating and maintenance costs.

2.3.2 Human resources

The CSA personnel assigned to the program includes project managers, specialists (including one head doctor and one flight doctor), scientists and administrative support officers. Other CSA specialists also provide support for specific projects (matrix support).

In terms of human resources, on average, 35 full-time equivalents (FTE) were assigned annually to the program. Two elements emerge with regard to human resources (see Table 2): a small number of resources is assigned to two sub-sub-programs (1.2.3.2. Operational Space Medicine and 1.2.3.3. Health and Life Sciences), on average 3-4 FTE, and an increasing gap should be noted for the sub-sub-program 1.2.3.1. Astronaut Training and Missions in the past two years (2016–2017 and 2017–2018).

Regarding the increasing gap for 2016–2017 and 2017–2018, the gap between the actual and expected FTEs in the space exploration program for these years is mainly attributable to the reassignment of resources to astronaut David Saint-Jacques's mission, to the astronaut recruitment campaign and to new positions resulting from additional funding to support the ISS's activities until 2024–2025.^{Footnote 12}

2.3.3 Management structure

The General Director of space exploration assumes overall responsibility for the activities carried out as part of the program. He acts in accordance with all of the applicable policies from the Treasury Board of Canada as well as the Project Management Policy and the Integrated Governance and Monitoring Framework Directive adopted by the CSA.

The Director General of the program chairs the Space Exploration Management Committee to support the ongoing coordination of the program's activities, projects, finances and human resources.

3.2.1 Comparative study and literature review

The program evaluation is based on a comparative study and review of the literature using CSA internal documents, public reports and national and international academic publications. These methods were intended to evaluate the collaboration between partners and the way in which the ISS was used to date in order to deepen knowledge related to space medicine and life sciences and investigate alternatives available to the ISS to carry out such research. Also, the comparative study and review of the literature included missions beyond low orbit and were intended to analyze the role that the CSA or the Canadian industry could play in such missions in the field of space medicine and life sciences. Lastly, the comparative study was also intended to learn more about the various ways in which astronauts from ISS member countries engage in public awareness activities.

3.2.2 Document review

The document review is varied and mainly concerns the relevance of the program. It includes various strategic documents, reports by expert groups, data, evaluations and statements by the Government of Canada. It also includes information provided by the CSA with regard to Human Space Missions and ISS Utilization, as well as documents identified or provided by other stakeholders, in particular as part of interviews conducted for the needs of this evaluation. The evaluation is also based on other evaluations carried out in recent years that cover topics common to Human Space Missions and ISS Utilization (e.g., Evaluation of the ISS Assembly and Maintenance Operations Program). Also, the document review covered activities carried out by other countries in the activities and scientific mission in which Canada participated. Lastly, a number of studies from external organizations were also reviewed, including the 2018 Evaluation of NASA, NASA's Management and Utilization of the International Space Station, a 2016 article from the OECD, New Trends in Space Innovation and another on the risks to human health from NASA in 2017, and Human Health and Performance due to in-Flight Medical Conditions.

3.2.3 Interviews with key informers

The interviews conducted with key informers helped to garner more in-depth knowledge of the activities of Human Space Missions and ISS Utilization, including the results obtained and challenges faced by key stakeholders. These interviews also corroborated, explained or clarified the findings of other sources of data. In total, 52 people from various groups of stakeholders were consulted as part of the individual and group interviews. The distribution of these interviews is outlined in Table 4.

The information gathered during these interviews is included throughout this report, as applicable.

3.2.4 Case studies

Four case studies were carried out based on a document review, analysis of internal and external data and interview data related to the selected cases. In order of priority, the following selection criteria were used to select the case studies:

• The representativeness of the diversity of program activities;
• The duration of the research theme;
• The amount of funding;
• The importance of international collaboration.

A brief description of each of the case studies can be found in section 2.2. Program logic model, activities, clients and partners in the box. Also, throughout the report, various boxes related to the case studies discuss the evaluation questions.

3.2.5 Internal data

The internal data of the sub-program Human Space Missions and Support and the sub-sub-program International Space Station Utilization were reviewed to consider ongoing relevance. The internal data included reports on performance measurement, financial data, administrative data, the Performance Measurement Strategy (PM Strategy), bibliometric data and reports on plans and priorities.

4.1.1 How did the program meet needs?

First, this section discusses the justification of the need at the beginning of the program. Second, it covers the evolution of needs, and third, the existence of other needs not covered by the program.

4.1.2 How are the program objectives in keeping with federal government priorities?

In order to encourage Canada's economic growth, the federal government is specifically focused on innovation and resulting economic opportunities. In 2017, the unveiling of the Innovation and Skills Plan reflects this strategy, which includes fundamental and applied research, the development of new products and services and their marketing and positioning on Canadian and foreign markets.

Program activities are in line with this approach. Canada's participation in space exploration activities paves the way for an array of some of the most innovative research, development and marketing activities with applications for both space and Earth. As was highlighted by stakeholders consulted as part of the evaluation, the program not only offers professional and economic opportunities to those who work to support space exploration missions, but it also helps position the whole of Canada as one of the nations that has worked on some of the most complex issues and challenges that exist.

The addition in 2017 of two new recruits to the CSA astronaut corps, and the unveiling of the new Space Strategy for Canada in March 2019, confirm a direct connection between the program and federal priorities. Through the program's activities, Canada will be able to maintain its participation in future ISS missions and the Lunar Gateway project.

4.1.3 What are the roles and responsibilities of the federal government in relation to the operation of the program?

The justification for public intervention in space exploration can be explained in part by the flaws of a market that introduces a number of obstacles to entry. These obstacles include the fact that the risk is too high to interest private investors, that the development of technology is long and that investments do not necessarily lead to commercial opportunities.

In many cases, the investment in innovative space activity may not necessarily yield goods, services or industrial processes which can be reserved for the exclusive use of the innovator, even if it is a private entity. This is particularly true for space missions that may include many diverse actors, all benefiting in some ways.^{Footnote 55}

The Agency plays a unique role in human space missions in Canada and utilization of the ISS, and there is no other program in Canada pursuing the same results. The Agency is also a federal organization whose function is to "collaborate with space agencies, or organizations working in related fields, from other countries on the peaceful exploitation and use of space" (incorporating act).

The Agency benefits from a number of partnerships to implement the program, as outlined in section 4.3.1. These partnerships help to share resources and funding in order to achieve the targeted results. According to a program lead, the alignment of the priorities and the impact of the program could be strengthened by collaborating more with other departments.

4.2.1 Data on performance

4.2.2 How were the expected outputs achieved?

From 2013 to 2018, the targeted outputs outlined in the performance measurement strategy were achieved or surpassed. Various scientific research and design of scientific instruments and space sub-systems were begun, under way or achieved from 2013 to 2018, surpassing the targets in the performance measurement strategy as outlined in table 6.

The list of research and scientific instruments is outlined in detail in Appendix A.

4.2.3 How were the expected immediate results achieved?

The immediate results were not only achieved, but, in the case of HQP and students, the established targets were largely surpassed.

The lead researchers working on research projects receiving funding from the Agency employed HQP and students to help with their work. On average, 71 people were assigned annually to these projects, contributing to the development of a highly qualified labour force in Canada and ultimately to the positioning of Canada at the forefront of science and technology.

4.2.4 How were the expected intermediate results achieved?

The intermediate results were surpassed for the most part, except for scientific publications, which were just slightly below the target. These results include the participation of universities and the industry, scientific publications, Earth re-uses or applications of instruments or vessels and sub-systems developed for space.

4.2.5 How were the expected final results achieved?

As explained above, little data on the final results could be produced during the evaluation. Most of the data were collected from interview respondents. These partial results are nonetheless positive and demonstrate that the program achieved the desired results, especially with regard to the quality of the science and the promotion of Canada internationally.

4.2.6 Did the implementation of the program have unexpected negative or positive impacts?

The evaluation did not identify any unexpected negative or positive impacts, and issues such as environmental impacts of the program (e.g. space debris) were not considered in the scope of the evaluation, and no such consideration came out of the interviews. Moreover, some indirect results were mentioned during the interviews, such as international collaborations, from which Canadian researchers were able to benefit, what with the reputation Canada acquired after having successfully conducted research as part of the program. As outlined in the previous section (4.2.5), some of these researchers won significant national and international awards. Also, businesses that developed scientific instruments for the program had subsequent contracts in Canada and abroad, such as with other space agencies, thanks to the expertise and reputation they acquired. For example, a company in the field fo technology confirms that their experience in space inspired their product, a configurable modular switch, and the reputation acquired thanks to their participation in projects in space had helped them to attract investors. Their product is now sold on the site of a large electronics retailer.

Lastly, according to an internal interview respondent, the credibility and influence acquired over the years by the expertise of people working on the program mean that Canada is well regarded internationally, for example at the Organization of the United Nations (UN) and in interactions with countries that are developing their space programs. Human space missions are the most visible aspect of the program and have a positive impact on Canada's profile, and the space program as a whole with regard to other countries.

4.3.1 Leveraging of partnerships

International partnerships. The international cooperation required for projects as significant and expensive as the ISS allows Canada to participate in the resulting scientific and technological breakthroughs. Thanks to the ISS, Canada had the opportunity to take part in the largest scientific and technological project that humans have carried out to date. Also, Canada advanced the high technology industry (aerospace and robotics) by being involved in the ISS. In general, it was demonstrated that the technology developed for space also had applications on Earth, in areas as extensive as weather forecasts, micro-electronics, resource cartography, solar energy and laser surgery, to name just a few.

The ISS partner space agencies tend to prioritize specific research areas. Figure 4 below allows us to visualize both the absolute number and the relative share of investigations by research discipline led by ISS partner agencies. Although the capacity of each agency to lead research is dictated and limited by its respective crew flight credits,^{Footnote 62} collaboration with other ISS partners allowed each agency to considerably multiply its impact (see table 10).^{Footnote 63}

Figure 4: Research areas for each ISS partner agency

In this regard, a number of internal respondents believe that it would be preferable to strengthen and increase existing international partnerships and create new partnerships in the area of health and life sciences research in space, as well as with other space agencies. According to a program manager, Canada's allocation of 2.3% is too small to be able to provide commercial opportunities, but we could consider a partnership program in health research with Canadian and American entities. This partnership could allow us to provide funding for Canadian researchers who would work with American researchers. This would be a beneficial arrangement for all parties and would open a private use component of the ISS. Also, a number of opportunities seem to be available to the CSA, whether with countries that already have an established space capacity or with other emerging countries. According to one of the internal respondents, international cooperation will be increasingly important with future missions that will also be more complex and expensive. Along those lines, another respondent indicated that it would be more sensible, from an economic perspective, if all space agencies played "on the same team."

National partnerships. Canada leveraged scientific research in a number of areas related to life sciences and health. Since discovering new ways of treating illnesses, creating technology and finding new advances in life sciences and health, Canada funds various programs, such as the Canada Research Chairs Program, the National Research Council of Canada and other grants, and makes contributions to industry leaders, university researchers and public research centres.^{Footnote 65} During the evaluation period, the program collaborated with a number of partners on a national scale, including government departments and agencies, the research industry and academia.

4.3.2 ISS model

Interview respondents said that the memoranda of understanding underlying the IGA, governance and the multiple committees and working groups between partners are efficient and well-established. Also, because sending samples to Earth for analysis is less efficient than developing analysis capabilities on board the ISS, the CSA is currently working on developing these abilities. Canada's contribution in this area is significant. The 2013–2018 period represents a pivotal step in the development of new Canadian technology that will allow us to support the health of astronauts during long missions beyond low orbit. Microflow^{Footnote 69} was used for the first time on board the ISS by Chris Hadfield in 2013 and the bio-monitor^{Footnote 70} was sent to the ISS in December 2018 so that David Saint-Jacques could use it first. It was also during this period that the bio-analyzer^{Footnote 71} technology was completed, which allowed it to be used for a first time in May 2019 by David Saint-Jacques. Lastly, in 2014, engineers and scientists from the National Research Council of Canada (NRCC) began to design the MicroPREP^{Footnote 72} technology, a "chip-based laboratory" that is already considered a significant asset for future exploration missions in deep space.

However, the document review and interviews carried out as part of this evaluation raised two factors that could potentially present an efficiency issue for the program in the long term.

1. Some nations do not participate in research projects. Although it is undeniable that the existing international partners help Canada increase the benefits of its research and investments (see section 4.3.1), almost all respondents would like the CSA to collaborate more with other space agencies. One of the internal respondents indicated that it would be more reasonable, from an economic standpoint, for all agencies to play on "the same team" and for the CSA to develop partnerships with emerging countries. According to interview respondents, the increase in partnerships would accentuate the leverage effect of partnerships and would therefore be beneficial for the efficiency of the program in the long term. However, Canada's collaboration with certain countries represents an issue with regard to the transfer of technology and intellectual property.
2. Contribution and flight opportunities. Historically, Canada has negotiated its flight opportunities by percentage of flight time, which is 2.3% owing to Canada's initial contribution to the ISS, rather than in terms of quantity of flights as other countries do (e.g. Italy). For example, Chris Hadfield's last mission in 2008 was negotiated for a duration of 142 days in orbit. Interview respondents indicated that this way of working could potentially become an issue for the CSA over time. Long missions are increasingly becoming the norm, and this means that other countries now benefit from more long flights to the station than Canada.

4.3.3 CSA model

4.3.4 Are there more cost-effective ways to implement the current program?

Participating in commercial flights is one of the alternatives suggested by interview respondents. It was said that commercial flights would increase utilization time but that they would result in high costs. However, other respondents said that this option would be difficult to carry out, since our utilization e time is calculated according to our contribution to the ISS. Canada has a utilization time of 2.3%, independent of the possibility of sending an astronaut to the station on a commercial flight. These considerations demonstrate that we still need to conduct analyses to determine the cost-effectiveness of the use of commercial flights, with the aim of increasing the program's utilization of the ISS.

Moreover, according to the plans outlined in the ISS program, new astronaut transportation vehicles that will be developed by private businesses SpaceX and Boeing will allow us to have a fourth astronaut assigned to the station. The addition of a fourth astronaut will add 35 utilization hours to the 35 existing hours, which will be shared among the partners, doubling the time to carry out experiments.