Artificial Intelligence and Global Security Initiative

The Shawn Brimley Next Generation National Security Leaders Fellowship

Joseph S. Nye, Jr. National Security Internship and Mentoring Program

Beyond ISIS: Whats Next for the U.S. in Syria and the Middle East?

The Future of U.S. Coercive Economic Measures

2019 National Security Conference: Sharpening Americas Edge

Clues to Chinese Strategic Thinking on Artificial Intelligence and National Security

In the second half of 2018, I traveled to China on four separate trips to attend major diplomatic, military, and private-sector conferences focusing on Artificial Intelligence (AI). During these trips, I participated in a series of meetings with high-ranking Chinese officials in Chinas Ministry of Foreign Affairs, leaders of Chinas military AI research organizations, government think tank experts, and corporate executives at Chinese AI companies. From these discussions as well as my ongoing work analyzing Chinas AI industry, policies, reports, and programs I have arrived at a number of key judgments about Chinese leaderships views, strategies, and prospects for AI as it applies to Chinas economy and national security. Of course, Chinas leadership in this area is a large population with diversity in its views, and any effort to generalize is inherently presumptuous and essentially guaranteed to oversimplify. However, the distance is large between prevailing views in American commentary on Chinas AI efforts and what I have come to believe are the facts. I hope by stating my takeaways directly, this report will advance the assessment of this issue and be of benefit to the wider U.S. policymaking community.

1. Chinas leadership including President Xi Jinping believes that being at the forefront in AI technology is critical to the future of global military and economic power competition.

In July 2017, Chinas State Council issued theNew Generation Artificial Intelligence Development Plan(AIDP).1This document along withMade in China 2025,2released in May 2015 form the core of Chinas AI strategy. Both documents, as well as the issue of AI more generally, have received significant and sustained attention from the highest levels of Chinas leadership, including Xi Jinping. Total Chinese national and local government spending on AI to implement these plans is not publicly disclosed, but it is clearly in the tens of billions of dollars. At least two3Chinese regional governments have each committed to investing 100 billion yuan (~$14.7 billion USD).4The opening paragraphs of the AIDP exemplify mainstream Chinese views regarding AI:

AI has become a new focus of international competition. AI is a strategic technology that will lead in the future; the worlds major developed countries are taking the development of AI as a major strategy to enhance national competitiveness and protect national security.5

The above quote also reflects how Chinas AI policy community6is paying close attention to the AI industries and policies of other countries, particularly the United States. Chinese government organizations routinely translate, disseminate, and analyze U.S. government and think tank reports about AI. In my conversations with Chinese officials and my reading of Chinese government AI reports, they demonstrated substantive and timely knowledge of AI developments in the United States and elsewhere. Chinese government AI reports frequently cite U.S. national security think tank publications.7The U.S. policymaking community ought to make it a priority to be equally effective at translating, analyzing, and disseminating Chinese publications on AI for the insights they provide into Chinese thinking.8

2. Chinas leadership including Xi Jinping believes that China should pursue global leadership in AI technology and reduce its vulnerable dependence on imports of international technology.

In October 2018, Xi Jinping led a Politburo study session on AI. Such sessions are reserved for the high-priority policy issues where leaders need the benefit of outside expertise. Xis publicly reported comments during and after the study session reiterated the main conclusions of both theAIDPandMade in China 2025, which were that China should achieve world-leading levels9in AI technology and reduce its vulnerable external [foreign] dependence for key technologies and advanced equipment.10

In his speech during the study session, Xi said that China must ensure that our country marches in the front ranks where it comes to theoretical research in this important area of AI, and occupies the high ground in critical and AI core technologies.11Xi further said that China must pay firm attention to the structure of our shortcomings, ensure that critical and core AI technologies are firmly grasped in our own hands. Xis speech demonstrates that Chinas leadership continues to subscribe to AIDPs andMade in China 2025s two major conclusions that China should pursue both world leadership and self-reliance in AI technology. The Chinese AI sectors dependence on foreign technology is discussed further in point nine.

Chinese Views on AIs International Security Implications

3. Recently, Chinese officials and government reports have begun to express concern in multiple diplomatic forums about arms race dynamics associated with AI and the need for international cooperation on new norms and potentially arms control.

In a keynote speech during Chinas largest international relations conference on July 15, 2018, Fu Ying,12the Vice-Chair of the Foreign Affairs Committee of the National Peoples Congress, said that Chinese technologists and policymakers agree regarding the threat of the new [AI] technology to mankind. She further stated that We believe that we should cooperate to preemptively prevent the threat of AI.

Madam Fus depiction of AI as posing a shared threat to international security was echoed by many other Chinese diplomats and PLA think tank scholars in my private meetings with them. For instance, one official told me he was concerned that AI will lower the threshold of military action, because states may be more willing to attack each other with AI military systems due to the lack of casualty risk. Chinese officials also expressed concern that increased used of AI systems would make misperceptions and unintentional conflict escalation more likely due to the lack of well-defined norms regarding the use of such systems. Additionally, Chinese officials displayed substantive knowledge of the cybersecurity risks associated with AI sytems, as well as their implications for Chinese and international security.

Madam Fu said that China was interested in playing a leading role in creating norms to mitigate these risks. At the World Peace Forum private roundtable on AI, one senior PLA think tank scholar privately expressed support for mechanisms that are similar to arms control for AI systems in cybersecurity and military robotics. However, he also said that AI-related arms control would be uniquely difficult since AI is low-cost and can be disseminated easily and cannot be monitored easily.

Notably, the recent Artificial Intelligence Security White Paper, published in September 2018 by the China Academy of Information and Communications Technology (CAICT), an influential Chinese government think tank, calls upon the Chinese government to avoid Artificial Intelligence arms races among countries.13The AIDP does not address arms races but does state that China will deepen international cooperation on AI laws and regulations, international rules and so on, and jointly cope with global challenges.

Such concerns extend to the Chinas private sector. Jack Ma, the chairman of Alibaba, said explicitly in a speech at the 2019 Davos World Economic Forum that he was concerned that global competition over AI could lead to war.14

4. Despite expressing concern on AI arms races, most of Chinas leadership sees increased military usage of AI as inevitable and is aggressively pursuing it. China already exports armed autonomous platforms and surveillance AI.

At the Beijing Xiangshan Forum on October 24, 2018, Major General Ding Xiangrong, Deputy Director of the General Office of Chinas Central Military Commission, gave a major speech in which he defined Chinas military goals to narrow the gap between the Chinese military and global advanced powers by taking advantage of the ongoing military revolution . . . centered on information technology and intelligent technology. Chinese military leaders increasingly refer to intelligent or intelligentized () military technology as their confident expectation for the future basis of warfare. Use of the term intelligentized is meant to signify a new stage of military technology beyond the current stage based on information technology.15Chinas AIDP strategy document states that China will Promote all kinds of AI technology to become quickly embedded in the field of national defense innovation.

The next day at the Xiangshan Forum, Zeng Yi, a senior executive at Chinas third largest defense company,16gave a speech in which he described his companys (and Chinas) expectations for the future implementation of AI weapons: In future battlegrounds, there will be no people fighting. Zeng predicted that by 2025 lethal autonomous weapons would be commonplace and said that his company believes ever-increasing military use of AI is inevitable [] We are sure about the direction and that this is the future.

Zengs comments are consistent with ongoing Chinese autonomous military vehicle development programs and Chinas current approach to exports of military unmanned systems. Chinas government already is exporting many of its most advanced military aerial drones to Middle Eastern countries such as Saudi Arabia and the UAE. Chinas government has stated that it also will export its next generation stealth drones when those are available.17Though many current generation drones are primarily remotely operated, Chinese officials generally expect drones and military robotics to feature ever more extensive AI and autonomous capabilities in the future. Chinese weapons manufacturers already are selling armed drones with significant amounts of combat autonomy. Ziyan, a Chinese military drone manufacturer, has sold its Blowfish A2 model to the UAE and in November 2019 reportedly was in negotiations with Saudi Arabia and Pakistan for Blowfish A2 sales.18Ziyans website states that the 38kg Blowfish A2 autonomously performs more complex combat missions, including fixed-point timing detection, fixed-range reconnaissance, and targeted precision strikes.19Depending on customer preferences, Ziyan offers to equip Blowfish A2 with either missiles or machine guns.

Beyond using AI for autonomous military robotics, China is also interested in AI capabilities for military command decisionmaking. Zeng Yi expressed some remarkable opinions on this subject, stating that today mechanized equipment is just like the hand of the human body. In future intelligent wars, AI systems will be just like the brain of the human body. Zeng also said that Intelligence supremacy will be the core of future warfare and that AI may completely change the current command structure, which is dominated by humans to one that is dominated by an AI cluster. Zeng did not elaborate on his claims, but they are consistent with broader thinking in Chinese military circles. Several months after AlphaGos momentous March 2016 victory over Lee Sedol, a publication by Chinas Central Military Commission Joint Operations Command Center argued that AlphaGos victory demonstrated the enormous potential of artificial intelligence in combat command, program deduction, and decisionmaking.20

China is currently making extensive use of AI in domestic surveillance applications. General Wang Ning of the Chinese Peoples Armed Police Force recently boasted about Chinas success in using AI in Xinjiang province:

In Xinjiang, we use big data AI to fight terrorists. We have intercepted 1200 terror organizations when still planning an attack. We use technology to identify and locate activities of terrorists, including the smart city system. We have a face recognition system, and for all terrorists there is a database.21

Xinjiang is home to millions of Chinas Uighur ethnic minority, which has been subject to extraordinary persecution aided by AI surveillance technology.22Chinas SenseTime corporation, a national champion in computer vision AI, is a major provider of surveillance technology to Chinas government, including for Xinjiang. SenseTimes security and surveillance products often are described using the smart city euphemism. However, SenseTime also has many non-security products, such as computer vision machine learning related to autonomous vehicles.

SenseTime is a major exporter of surveillance technology in government and commercial markets across Latin America, Africa, and Asia. Chinas government and leadership is enthusiastic about using AI for surveillance. One scholar at a Chinese think tank told me that he looks forward to a world in AI will make it impossible to commit a crime without being caught, a sentiment that echoes the marketing materials put out by Chinese AI surveillance companies.

Chinas behavior of aggressively developing, utilizing, and exporting increasingly autonomous robotic weapons and surveillance AI technology runs counter to Chinas stated goals of avoiding an AI arms race. However, this by itself does not necessarily mean that Chinese officials are being insincere in their expressions of concern about such arms races. Lamenting arms race dynamics while aggressively participating in them is a common story in the history of international relations. The strongest behavioral indication that China might be insincere comes from Chinas April 2018 United Nations position paper,23in which Chinas government supported a worldwide ban on lethal autonomous weapons but used such a bizarrely narrow definition of lethal autonomous weapons that such a ban would appear to be both unnecessary and useless. This rhetorical gambit allowed China to reap positive media attention for their support of global restrictions while avoiding hypocrisy over Chinese development of more advanced military AI and autonomy.24More broadly there seems to be less grassroots concern of the issue among Chinese AI researchers than their counterparts in the West, though evidence on this point is limited.25

5. Chinas Ministry of National Defense has established two major new research organizations focused on AI and unmanned systems under the National University of Defense Technology (NUDT).

The National Innovation Institute of Defense Technology (NIIDT, an NUDT subsidiary), has established and is rapidly growing two Beijing-based research organizations focusing on the military use of AI and related tech. These are the Unmanned Systems Research Center (USRC), led by Yan Ye, and the Artificial Intelligence Research Center (AIRC), led by Dai Huadong.26Each organization was created in early 2018, and each now has a research staff of over 100 (more than 200 total), which makes it one of the largest and fastest growing government AI research organizations in the world. However, there are larger private sector AI research organizations in both China and the United States. SenseTime, for example, has roughly 600 full-time research staff. DeepMind a Google subsidiary focused on AI research has around 700 total staff and annual expenditures of over $400 million.27Salaries of Chinese AI PhDs educated in China are generally much lower than salaries of Western AI PhDs, or Western-educated Chinese, which makes estimating the AIRCs budget based on staff difficult. AIRC staff are engaged in basic research into dual-use AI technology, including applying machine learning to robotics, swarm networking, wireless communications, and cybersecurity. The AIRC also likely does classified work for the Chinese Military and Intelligence Community.

6. Chinas government sees AI as a promising military leapfrog development opportunity, meaning that it offers military advantages over the US and will be easier to implement in China than the United States.

The term leapfrog development describes a technology for which laggard countries can skip a development stage, or one for which being behind on the current generation of technology actually offers an advantage in adopting the next generation. A commonly cited example is the rapid and widespread adoption of cellular phone technology in countries that had only minimal landline phone adoption. Kai-Fu Lee, one of the leading venture capitalists in Chinas AI sector, argues that the absence of many developed-economy capabilities, such as easy credit checks, have led to a flood of Chinese entrepreneurs making innovative use of AI capabilities to fill those gaps.28Plastic credit cards are nearly nonexistent in China, but mobile phone payments secured by facial recognition are ubiquitous.

Chinas emphasis on AI as a leapfrog technology enabler extends to national security applications. Chinas 2017 National AI Development Plan identifies AI as a historic opportunity for national security leapfrog technologies.29Chinese Defense executive Zeng Yi echoed that claim, saying that AI will bring about a leapfrog development in military technology and presents a critical opportunity for China.

If China is correct that AI presents a leapfrog opportunity, it would mean that China is better positioned to adopt military AI than the United States. In this theory, the United States current advantages in stealth aircraft, aircraft carriers, and precision munitions actually would be long-term disadvantages because the entrenched business and political interests that support military dominance today will hamper the United States in transitioning to an AI-enabled military technology paradigm in the future.30As one Chinese think tank scholar explained to me, China believes that the United States is likely to spend too much to maintain and upgrade mature systems and underinvest in disruptive new systems that make Americas existing sources of advantage vulnerable and obsolete. Chinas military also faces perverse incentives to protect legacy systems, but to a far lesser extent: Military spending tripled over the 20072017 period,31modernization is a top priority, and there is a general understanding that many of its current platforms and approaches are obsolete and must be replaced regardless.

Just one of many examples of Chinas AI leapfrog strategy is its prioritized investment32and technology espionage33for low-cost, long-range, autonomous, and unmanned submarines. China believes these systems will be a cheap and effective means of threatening U.S. aircraft carrier battlegroups and an alternative path to projecting Chinese power at range. In general, China sees military AI R&D as a cheaper and easier path to threatening Americas sources of military power than developing Chinese equivalents of American systems.

Chinese Views on the Strengths of Chinas AI Ecosystem

7. Chinas government and industry believe that they have largely closed the gap with the United States in both AI R&D and commercial AI products. China now sees AI as a race of two giants, between itself and the United States.

Chinas July 2017 national AI strategy set a 2020 goal for Chinas AI industrys competitiveness [to] have entered the first echelon internationally. In truth, Chinas leadership already assesses China as having achieved this objective as of mid-2018. At the World Peace Forum, Tsinghua Universitys Xue Lan delivered a briefing on Tsinghua Universitys major report on the state of the AI sector in China.34This study found that China has secured a leading position in the top [AI] echelon in both technology development and market applications and is in a race of two giants with the U.S. It also finds that China is:

Chinas assessment of being in the first echelon is correct, though there are important caveats that will be discussed more below. Not only is China advancing the state of the art in AI research, its companies are very successfully developing genuinely innovative and market-competitive products and services around AI applications. SenseTime, for example, is undisputedly one of the world leaders in computer vision AI and claims to have achieved annual revenue growth of 400 percent for three consecutive years. DJI offers another example. DJI leads the world in consumer drones with 74 percent market share.35DJI has innovatively incorporated machine learning technology into its most recent products.

In many cases the products and underlying technologies between commercial AI and military/security AI products are identical or nearly so. DJI recently was selected as the sole drone provider to the New York Police Department, which will use DJIs consumer model drones. Similarly, SenseTimes consumer facial recognition systems share infrastructure and technology with its security systems, used by both Chinese law enforcement and intelligence organizations.

8. Chinas strong current position in AI R&D and commercial applications has been enabled by access to international markets, technology, and research collaboration.

Chinas success has been enabled by its access to global technology research and markets. Many seemingly Chinese AI achievements are actually achievements of multinational research teams and companies, and such international collaboration has been critical to Chinas research progress.36According to the Tsinghua University study of Chinas AI ecosystem, More than half of Chinas AI papers were international joint publications, meaning that Chinese AI researchers the top tier of whom often received their degrees abroad were coauthoring with non-Chinese individuals. Even purely Chinese successes often build upon open source technologies developed most often by international groups.

Partly as a result of this, leading Chinese technology companies have significant and under-reported dependencies on the United States. For example, DJI, the Shenzhen-headquartered, world-leading drone manufacturer, is vertically integrated with nearly all design, manufacturing, and marketing done in-house. However, all of DJIs drone flight software development is performed at DJIs American office in Palo Alto, which predominantly employs U.S. citizens as staff. Additionally, nearly 35 percent of the bill of materials in each of DJIs products are from the United States, mostly reflecting semiconductor content.

Chinese View on the Weaknesses of Chinas AI Ecosystem

9. Despite Chinas strength in AI R&D and commercial applications, Chinas leadership perceives major weaknesses relative to the United States in top talent, technical standards, software platforms, and semiconductors.

Though most in Chinas leadership agree that China is one of two giants in AI, there is a similarly widespread understanding that China is not strong in all areas. Chinas January 2018 White Paper on Artificial Intelligence Standardization points out that the Chinas AI ecosystem lags in several key areas:

Although China has a good foundation in the field of AI, even as core technologies such as speech recognition, visual recognition, and Chinese-language information processing have achieved breakthroughs and possess huge market environments for applications, the overall level of development still lags behind that of developed countries.37

Similarly, the Tsinghua University China AI Development report finds:

Chinas strengths are mainly shown in AI applications and it is still weak on the front of core technologies of AI, such as hardware and algorithm development, Chinas AI development lacks top-tier talent and has a significant gap with developed countries, especially the U.S., in this regard.38

There are additional comparative weaknesses in Chinas AI ecosystem worth discussing, but I will focus on the four that most often came up in my meetings in China: top talent, technical standards, software platforms, and semiconductors.

The Tsinghua University China AI report did a remarkable study of the global AI talent distribution, concluding that by the end of 2017, the international AI talent pool comprised 204,575 individuals, with the United States having 28,536 such individuals and China in second place with 18,232. However, Chinas ranks eighth in the world in terms of Top AI talent, with only 977 individuals compared to the United States 5,518. Though acknowledging the disparity, venture capitalist Kai-Fu Lee argues that this is not a major barrier because the current age of implementation [AI application commercialization] appears well-suited to Chinas strengths in research: large quantities of highly skilled, though not necessarily best-of-best, AI researchers and practitioners.39Some researchers at leading Western AI research insitutions have told me they agree with this conclusion, noting that AI breakthroughs by leading institutions are quickly replicated by other institutions worldwide.

Lee is influential among Chinas technology industry, but not everyone agrees with his theory. Many that I spoke with said that Chinas shortage of top talent will be a handicap in the future development of Chinas AI sector, and Chinas government is taking aggressive action to improve the size and quality of Chinas AI talent pool.40In April 2018, Chinas Ministry of Education (MOE) launched itsAI Innovation Action Plan for Colleges and Universities. Among other elements, the plan:

Will create 50 world-class teaching materials for undergraduate and graduate studies related to AI applications for specific industries;

Will create 50 national-level high-quality online open courses;

Will establish 50 artificial intelligence faculties, research institutions, or interdisciplinary research centers.

In a separate initiative, the MOE also plans to launch a new five-year AI talent training program to train 500 more AI instructors and 5,000 more top students at top Chinese universities.42

The determination and common adoption of international technical standards is a key enabler of technology interoperability and market growth. Common adoption of Wi-Fi standard, for example, is what allowed such a wide diversity of modems, routers, mobile phones, and computers to all effectively connect to each other over Wi-Fi networks. Companies that create intellectual property related to such standards often reap significant rewards, especially when their patents, such as the design of a specific semiconductor chip, are declared essential to effective operation of any device using the standard.43For example, Qualcomms intellectual property was critical to development of the Code-Division Multiple Access (CDMA) cellular standard. It is essentially impossible for a device to access CDMA cellular networks unless the device uses Qualcomm semiconductor patents, hence why they are an example of so-called Standard Essential Patents (SEPs). Historically, Chinese companies and government organizations produced very few SEPs, but China has made rapid progress on this front. Huawei, ZTE, and the China Academy of Telecommunications Technology have produced hundreds of SEPs related to fifth generation (5G) cellular standards.44

AI technical standards are far less mature than those in cellular networking, but Chinas government strategy for pursuing leadership in AI technical standards is informed by its experience in the cellular networking. Chinas government and Chinese corporations want to ensure that their intellectual property and products are critical features of the future of AI. Because of Chinas experience with ZTE export restrictions, Chinese leadership perceives its success in technical standards as critical to both economic growth and national security.

Developers of AI systems rarely start from scratch. More often, they leverage pre-written programs developed by others and shared into code libraries. This allows developers to focus on the unique specifics of their application usage requirements, rather than solving generic problems faced by all AI developers. Some organizations have combined machine learning code libraries with other AI software development tools into mature machine learning software frameworks, many of which are open source. Popular machine learning frameworks include, but are not limited to, TensorFlow (Google), Spark (Apache), CNTK (Microsoft), and PyTorch (Facebook).

Notably, none of the most popular machine learning software frameworks have been developed in China. The importance of leadership in software frameworks is debated even among Americas leading technology companies. Companies that do prioritize framework development claim that it offers opportunities to attract top talent, influence technical standards, and guide the overall ecosystem toward increased usage of their products and services. The absence of Chinese AI companies among the major AI framework developers and open source AI software communities was identified as a noteworthy weakness of Chinas AI ecosystem in several of my conversations with executives in Chinas technology industry. Additionally, Chinas CAICT AI and Security White Paper lamented the fact that At present, the research and development of domestic artificial intelligence products and applications is mainly based on Google and Microsoft.45SenseTime has devoted extensive resources its own machine learning framework, Parrots, which is intended to be superior for computer vision AI applications. So far, the company appears to have had limite