The U.S. and China Must Explore Pacing the Frontier During September AI Dialogue

When officials from the United States and China meet later this month for a dialogue on artificial intelligence (AI), they need to have a serious conversation about what the world’s two leading AI powers should do about the risk of humans losing control of the most advanced frontier AI models. Given the lack of trust between the two nations and their fierce competition to dominate AI, this might seem a fool’s errand. But accelerating AI development and recent rogue AI incidents make the timing of these discussions critical, and there are signs that there might be an opening for such a dialogue. While no one should expect the September dialogue to produce an AI treaty, the United States and China should use the session to share their mutual concerns about “loss of control” of AI models and explicitly begin to discuss steps the two sides should take in order to moderate the speed—or “pace the frontier”—of AI development. It would be far more foolish not to test that opening in the face of this potentially dangerous technology’s breathtaking rate of progress.

Treasury Secretary Scott Bessent will lead the U.S. delegation, and the dialogue is intended as a follow-up to President Donald Trump and President Xi Jinping’s May 2026 summit and a precursor to President Xi’s visit to the United States in late September—though there has also been a recent contradictory report denying the AI discussion will occur. It is unknown how either Washington or Beijing will respond to a call to pace the frontier of automated AI development. On September 8, Secretary Bessent poured cold water on the idea of a “pause” at a public event stating, “We can’t pause. You can’t, because the Chinese won’t pause.” Yet, amid this increasingly uncertain AI environment, the Trump administration has an obligation to try to engage in a serious dialogue with its Chinese counterpart on the topic of pacing and to explore potential areas of cooperation and joint interest.

Should the Chinese government be open to this dialogue, the United States should propose working toward the following five deliverables: (1) establishing an “AI red phone” for urgent safety concerns; (2) defining “genuine security threat” from frontier AI that would impel steps to pace the frontier in the future; (3) announcing a working group and date to meet to share how each side is using testing and restrictions to pace the frontier; (4) jointly declaring that the latest research and advancements in AI alignment and model control are an existentially important public good that should be shared publicly; and (5) issuing a joint statement from presidents Trump and Xi on the topic following the September dialogue.

For AI control to keep up with AI advancements, what’s needed is not semiconductors, energy, or capital, but rather time—the commodity in shortest supply given the economic and geopolitical pressures that are pushing AI development forward. The U.S. and China can help create that needed time starting with their September meeting.

The meeting will occur in the shadow of alarmingly rapid advancements in frontier AI models, including with unprecedented offensive cyber abilities, as manifested by recent instances of internal AI models that escaped containment and hacked third parties, and the continuing improvement of AI model capabilities that could soon outstrip humans’ ability to understand and control them. That loss of control, either through misalignment or inability to control AI models, could result in significant risks to national security, economic security, and the safety and welfare of Americans, Chinese, and people around the world from unprecedented cyberattack capabilities that could cripple critical infrastructure; individuals with increased ability to create chemical, biological, radiological, and nuclear weapons; or the loss of control of governments’ own digital systems.

Concerns around the acceleration of AI recently led more than 1,300 employees from leading AI companies to sign an open letter later endorsed by the leading AI frontier labs OpenAI and Anthropic. The letter notes that “there is a real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems” and urges the U.S. government to back international efforts to “deliberately pace the frontier of automated AI development,” though authors of the letter left those potential pacing mechanisms undefined. That such a letter was signed by those at the forefront of building frontier AI models should seize the attention of policymakers in both countries. These risks grow as advanced AI models become increasingly capable of contributing to and automating the development of their successors, a process that a leading frontier AI lab has said is already underway. This could lead to recursive self-improvement (RSI)—the ability of an AI model to autonomously improve itself—in the near future. That in turn might accelerate AI development beyond effective human oversight and enable artificial general intelligence (AGI), in which AI models meet or exceed human capacities—what many consider the critical development goal of the AI race. But well before RSI or AGI there is also the more immediate concern that frontier AI models may outstrip humans’ capabilities to understand and control them—if they have not already done so.

There is significant recent evidence that these capabilities are nearing or progressing beyond the safeguards of their developers. In July, OpenAI and Anthropic both disclosed incidents of internal models hacking third-party sites. They paused training on their new models but largely restarted after they strengthened internal safeguards. Both companies also addressed pacing, with Anthropic noting, “To be clear about where we stand: we believe the world would benefit if the industry adopted a lawful, verifiable, effective mechanism for coordinated pacing as soon as possible” and OpenAI’s chief scientist writing that the best way forward to “Pacing RSI” includes “coordinating to slow down future development as needed to build confidence in these measures,” but neither company instituted any formal pause.

Although there have been some initial attempts to outline pacing possibilities following the letter, and there are significant previous efforts to outline potential methods to address these issues, key groups may not engage with the concept unless or until the American and/or the Chinese governments identify pacing the frontier as a priority or at least a possibility, making it critical to approach the issue and send a serious signal of engagement in the September AI meeting.

The upcoming U.S.-China dialogue takes place in the midst of the dual AI races: All of the various American and Chinese labs are competing fiercely among themselves in their respective home markets, and AI features as one of the principal arenas of strategic and economic competition between Washington and Beijing. President Trump has pursued an agenda of American AI that includes “national and economic security and dominance” and President Xi Jinping has made clear China’s ambitions for global AI leadership. Both countries are pouring tremendous resources into their AI companies domestically and their AI alliances globally. At the same time, Washington accuses Chinese AI labs of unfairly “distilling” or copying the more advanced models of their U.S. competitors while Beijing complains that U.S. export controls on advanced chips and semiconductor tools and equipment are an attempt to hold back Chinese development.

But mutual trust is not a prerequisite for serious talks about pacing the frontier. All that is needed is for both sides to comprehend the severity of the risks the world faces if we lose control of frontier AI. As Bill Gates wrote in August, “polarization within and between countries makes it harder than ever to get things done. Some cooperation between the U.S. and China will be required. We do not have the luxury of moving slowly.” The greater risk for the United States and China would be to not at least explore the possibility of pacing AI until it is too late.

Even more concretely, despite the Trump administration’s clear antiregulatory bias, it seems to have been mugged by reality, prompting it to create a secret, de facto licensing system for frontier AI models over the past few months. This rapid about-face may create an opening for diplomatic engagement with China on this topic, as it demonstrates U.S. willingness to act.

Similarly, for Beijing, there may be self-interest in engaging in discussions. The core interest of the Chinese Communist Party (CCP) is to maintain power and legitimacy, and it has gone to great lengths under President Xi to crack down on any internal political threats to its position, whether from Tibetans and Uyghurs to rights activists and demonstrators in Hong Kong. But a rogue AI that generated nationwide chaos could also threaten the party’s rule. The CCP likely perceives that the greatest risk of an uncontrollable AI model comes not from its own homegrown AI labs, which it believes it has a handle on, but from a U.S. frontier AI lab that rushes a new model out under intense competitive and investor pressure. If true, Beijing ought to be keenly interested in knowing how the United States intends to mitigate such risk. This would not be altruism—China remains the top exporter of surveillance equipment and other tools of repression—but a cold-blooded calculation in the service of regime preservation. It seems Xi Jinping is paying attention to exactly this concern. At his recent opening speech at the launch of the World AI Cooperation Organization—Beijing’s answer to the Trump administration’s international AI supply chain initiative Pax Silica—he outlined key points, two of them most relevant to the September meeting. The first, which dominated the headlines, focused on openness, widely interpreted to mean open-source or open-weight models that Chinese AI companies allow customers to download for free, which China has made a centerpiece of its global AI strategy in contrast with the closed-weight proprietary model approach of the leading U.S. labs.

But President Xi’s second point received less attention, “We should strengthen risk-awareness and ensure that AI is secure and controllable. … We should take seriously the various types of inherent and secondary risks that AI may trigger … and ensure that AI is always under human control.” He further expressed some openness to international cooperation on global AI governance. Some experts interpreted this as signaling possible restrictions on Chinese open-source or open-weight models should security issues arise. But it also highlighted an awareness of potential loss of control issues for AI broadly.

There is some reason to believe that top leaders on the two sides are increasingly seeing a reason to engage on the topic, no doubt aided by the shock of the recent OpenAI and Anthropic rogue agent incidents. On the U.S. side, in late August at the G20 finance meeting in North Carolina, Secretary Bessent previewed an agenda that reportedly included “artificial ⁠intelligence guardrails aimed at keeping powerful AI models from falling into the hands of non-state actors,” which could be interpreted as a way of obliquely starting to approach the issue. The same day, a China Central Television (CCTV) post (translation) laid out two principles for discussion with the United States on AI, starting with “a distinction between what constitutes a genuine security threat and what is merely technological competition,” a position that would have to be the starting point for any serious discussion. Later that week at the G20 innovation ministerial, the United States introduced the “Carolina Principles,” which called to “reserve new regulation for novel consideration” that could leave open space to address the novel concerns around the AI control issues. Even Secretary Bessent’s recent comments dismissing a pause were predicated on the idea China would not engage on international collaboration on global AI governance, which could optimistically mean Chinese cooperation could change that calculus and pessimistically that the U.S. calculus would not change.

At the same time that Office of Science and Technology Policy Director Michael Kratsios was introducing the aforementioned Carolina Principles at the G20 ministerial, a China Central Television (CCTV) post (translation from Geopolitechs), widely viewed as signaling Chinese conditions for the September talks, attacked U.S. frontier AI company Anthropic and laid out two principles for discussion with the United States on AI starting with “a distinction between what constitutes a genuine security threat and what is merely technological competition.”

There are numerous ways to interpret this; the Chinese have many different definitions of “loss of control” as Zilan Qian of the Oxford China Policy Lab has noted, and there will always be ways to refuse to come to agreement on a definition. But fundamentally, a narrow and mutually agreed-upon definition of “genuine security threats” is the first step to any AI security discussion of AI safety risks.

The second principle seems boisterous at first read:

Second, if the United States wants to talk about rules, it must first prove that the rules are equally effective against its own companies. Relevant figures on the American side are already discussing how to get China to agree to restrict “AI models with dangerous capabilities.” The question is this: with companies like Anthropic in existence, before the United States asks China to restrict model releases and disclose risks, should it not first investigate its own companies, make public the purposes, scope, and rules of their identification mechanisms, and subject them to third-party audits?

After the July release of Moonshot AI’s Kimi K3 open-weight model, White House officials publicly accused Chinese AI companies of adversarial distillation and in September released a cybersecurity warning about Chinese distillation of U.S. AI models. The Trump administration reportedly considered taking action against Chinese open-weight AI models. But facing significant pushback from domestic tech companies, the United States has not yet taken steps against Chinese open-source or open-weight models and exempted them from the nonpublic “voluntary framework” ordered by the AI executive order issued June 2.

That means in the United States, Chinese AI models are subject to far fewer restrictions than American models currently under the Trump administration’s secret de facto licensing framework that exempts Chinese open-source or open-weight models. In contrast, the United States has already restricted model releases for American AI but not for Chinese models, applying export controls to Anthropic’s release of Mythos and Fable and removing them from the market for weeks, and having OpenAI agree to a controlled release of GPT-5.6.

As for the CCTV post asking whether the United States “should it not first investigate its own companies,” American frontier AI labs are already under investigation as state attorneys general have opened investigations into OpenAI over the Hugging Face hack and some members of Congress are demanding hearings—though it is unclear whether the U.S. federal government will, or is, investigating OpenAI over the incident. Congress and many organizations, including CAP, have demanded the administration make public its “voluntary framework,” lawsuits have been filed to compel its release, and its continued secrecy is likely unsustainable. Furthermore, Illinois law SB0315, which goes into effect in January 2027, requires independent third-party audits for frontier AI models beginning in January 2028.

While not reading too much into a single post, it is interesting that the second principle articulated may seem significant at first glance but is mostly already achieved when closely examined.

Reuters has reported ahead of the meeting that “The U.S. wants to discuss cooperation on monitoring AI-directed cyberattacks, and has floated a proposal to ask U.S. and Chinese AI labs to ‘police themselves’ and share information to prevent AI-linked cyberattacks.” But in light of the severity of the recent rogue AI incidents and the accelerating pace of AI advancement, self-policing proposals are insufficient to the moment.

Instead, the principle floated by the Chinese, “distinction between what constitutes a genuine security threat and what is merely technological competition” should serve as the basis for discussion at the upcoming dialogue in September.

Helen Toner, executive director at Georgetown University’s Center for Security and Emerging Technology and former OpenAI board member, has noted, “a huge way to influence China is just to honestly show what we’re observing and what we’re doing about it on the US side. The Hugging Face attack is by far the most visceral evidence we have so far of what losing control of advanced AI could look like.” To illustrate this concern, the United States should start the discussion with a technical briefing about the OpenAI/Hugging Face incident and allow the Chinese to ask questions.

Then the U.S. side should outright ask whether China is open to developing mechanisms for pacing further AI development to address “genuine security threats” from AI. Both sides will have to avoid the temptation to devote all their time to U.S. allegations about Beijing’s distillation and China’s complaints about U.S. export controls. Those topics are certainly important from the competitive perspective, but this conversation ought to stay focused on the potentially existential threat of loss of control. But should both sides express interest in the discussion, here are five potential deliverables for the September AI event.

First, Washington and Beijing should establish an “AI red phone”—a communications channel to share evolving and urgent AI safety concerns in real time. Despite the mixed history of usage of red phones between the United States and China, their existence is still a potentially critical bridge as the technology continues to develop at breakneck speed.

Second, the United States and China should attempt at this meeting to define what constitutes a frontier AI “genuine security threat”—and not just commercial competition—that would require steps to pace the frontier in the future. This definition can and should purposefully start narrowly, could be expanded over time, and would allow for future steps.

Third, the two sides should arrange a regular working group meeting to develop discussions on pacing the frontier and announce the next date for later this year. This group should include relevant government officials, scientists, AI companies, and nongovernmental AI safety groups from both countries, as appropriate. This group would explore the meaning of the Chinese proposal for “joint testing and possible regulation” of “capabilities that could genuinely cause serious harm.” The United States would of course want to coordinate with like-minded partners and allies before seriously considering any such joint mechanisms with Beijing, but this conversation could help inform subsequent discussions with partners and allies.

Fourth, the two sides should jointly declare that the latest research and advancements in AI alignment and model control are best shared publicly and openly. Stronger AI alignment and control techniques are not actually a long-term competitive advantage for a country or a company. A country or company that develops the best technique but does not share that knowledge only exacerbates the risk of another country or company developing an unaligned or uncontrollable frontier AI model. Embracing selective research transparency may be a key future mechanism for joint testing and restriction.

Consequently, both countries should commit to publishing their latest alignment and AI control research.

Should only one side adhere to such a deal—say only the U.S. government and U.S. frontier labs published their advanced research on alignment and control and the Chinese did not— there might be safety practices that the United States would want to share publicly with the entire world, including China, regardless of whether China reciprocates as there may be less risk of misalignment or losing control of Chinese frontier AI models. One-sided adherence might even have benefits if that research were incorporated into Chinese open-source or open-weight models that were released to the public. To be clear, CAP is recommending this approach only with regard to AI alignment and model control. Given Beijing’s long track record of appropriating the intellectual property of foreign innovators to unfairly boost its own companies, the U.S. government and AI labs have every right to be vigilant about Chinese attempts to gain access to proprietary U.S. technology.

Fifth, while the AI September discussion is to be held at the ministerial level, Presidents Trump and Xi are also scheduled to meet later in the month. After their meeting, the two presidents should issue a joint statement affirming their concerns about risks from misalignment and loss of control and committing to dialogue to discuss potential efforts to pace the frontier to send an official message of concern to stakeholders around the world.

If this administration or the Chinese government will not try to seriously and substantively engage on this topic, then other countries, technology companies, and civil society should strive to keep dialogue alive in anticipation of a better negotiating environment in the future. They may be able to draw on critical lessons from the scientific community and the climate movement on keeping global issues alive in the face of indifferent or even hostile governments of major powers. Even if these concerns are not engaged with this September, at the rate of AI advancement and incidents, it may not be long until they top a future agenda.

Warnings on AI safety and potential loss of control are coming from many of the people closest to the development of frontier AI systems and both countries ignore those warnings at their own peril. The dual-race dynamics of the frontier AI race, domestically between leading U.S. frontier AI labs on the one hand and internationally between U.S. and Chinese firms on the other, make coming to an agreement on pacing the frontier of AI a difficult task. But it is not an impossible one.

Some critics have expressed concern that China would not be a good faith counterpart and that any agreed-upon cooperation would be impossible to monitor or enforce, would weaken America vis-a-vis its most serious rival, or would be a general affront to technological progress. But other AI experts, former government officials, and elected officials have recently expressed that the recent incidents of rogue agents have created an opportunity to press for collaboration with China.

The agenda for the September AI meeting between the U.S. and Chinese governments has yet to be set and it is unclear if pacing the frontier will be discussed. Yet there is little downside in the U.S. government engaging seriously now with the Chinese government on potential mechanisms to prevent the worst outcomes from these technologies, outcomes that would endanger global peace and security. The potential upside, that this revolutionary technology is advanced in a manner that reduces the possibility of the two most powerful nation-states coming into conflict, is worth exploring. The difficulties inherent in this bilateral relationship should not preclude discussion.

It would be unrealistic to imagine Washington and Beijing emerging from the September meeting with a negotiated AI bilateral agreement. But the two sides can start to understand whether they both consider the loss of control of frontier AI models a “genuine security threat” and discuss how to lay the groundwork for meaningful development of potential “pacing” mechanisms.

Neither government has to agree that frontier AI should be paused. But they can agree that, if human control begins to fail, the world’s two leading superpowers should have real options available to them.