BEIJING — The U.S. and China recently took a significant first step toward keeping U.S.-listed Chinese stocks like Alibaba from being forced off U.S. stock exchanges.

What needs to happen next is a smooth on-ground inspection in China by the U.S. with adequate support from Chinese authorities, analysts said.

"Many implementation details probably can only be figured out by the auditing firms and the [Ministry of Finance] — together with [the China Securities Regulatory Commission] — through real-case auditing trials under this unprecedented agreement," said Winston Ma, adjunct professor of law at New York University.

The U.S. Public Company Accounting Oversight Board said its inspectors are set to arrive in Hong Kong in mid-September, shortly after which "all audit work papers requested by the PCAOB must be made available to them."

Audit work papers differ from the actual information on companies gathered by accounting firms.

The work papers record the audit procedure, tests, gathered information and conclusions about the review, according to the PCAOB website. It is not clear what level of highly sensitive information, if any, would be included in the work papers.

The ability of the U.S. to inspect those work papers for Chinese companies listed in the U.S. has been a years-long dispute. U.S. political and legal developments in the last two years have sped up the threat that the Chinese companies might need to delist from U.S. stock exchanges.

A turning point came in late August when the PCAOB and China Securities Regulatory Commission signed a cooperation agreement that laid the regulatory basis for allowing U.S. inspections of audit firms within China's borders.

That's according to statements from both government entities, which also said China's Ministry of Finance signed the deal.

"I see this as a big 'progress,' meaning that both sides were willing to take steps to move this forward," said Stephanie Tang, head of private equity for Greater China and partner at Hogan Lovells.

"The subject or the audience of this PCAOB investigation would be the audit firms," she said, emphasizing she is not an accountant.

China's registered accounting firms are overseen by the the Ministry of Finance, making it the leader on the Chinese side of next steps, said Ming Liao, founding partner of Beijing-based Prospect Avenue Capital.

However, there's uncertainty around implementation of the agreement as it only established a framework, analysts said.

"Our accounting firms still don't know how to proceed," said Peter Tsui, president of the Hong Kong-based Association of Chinese Internal Auditors. That's according to a CNBC translation of his Mandarin-language remarks Thursday.

He said questions remain over what information the firms should share in order to remain compliant with Chinese regulation.

"Give [us] some guidelines," Tsui said.

Tsui said the inspections should go smoothly if it's just a matter of accountants on both sides, and there is no political interference on the U.S. side. He said the big four accounting firms — KPMG, PwC, Deloitte and EY — are members of the association.

China's Ministry of Finance has yet to release a public statement on the audit cooperation agreement. The ministry did not immediately respond to a CNBC request for comment.

One development Prospect Avenue Capital's Liao is watching is whether U.S. President Joe Biden and Chinese President Xi Jinping meet in-person this fall for the first time under the Biden administration. That could speed up a final agreement on the audit dispute, he said.

"In the end, resolving the audit work paper problem relies on political interaction between China and the U.S.," Liao said in Chinese, according to a CNBC translation. "With trust, this problem can very easily be resolved."

The PCAOB said it will make a determination in December on whether China was still obstructing access to audit information.

U.S. regulators will likely "start to know in October or November" what determination the PCAOB will make on whether U.S.-listed Chinese companies might be headed for delisting, Gary Gensler, chair of the U.S. Securities and Exchange Commission, told CNBC's David Faber in late August.

Alibaba and many other U.S.-listed Chinese companies have started in the last few years to issue shares in Hong Kong — partly seen as a way to hedge against a potential delisting from U.S. stock exchanges. Since Chinese ride-hailing company Didi's U.S. IPO in the summer of 2021, Beijing has also increased its scrutiny of Chinese companies wanting to list overseas.

The combined political uncertainty has slowed the flow of Chinese IPOs in the U.S., especially of larger companies.

Since July 1, 2021, 16 Chinese companies have listed in the U.S., excluding special-purpose acquisition companies, according to Renaissance Capital. Back in 2020, 30 China-based companies had listed in the U.S., the firm said then.

By value, the five largest U.S. institutional holdings of U.S.-listed Chinese stocks are: Alibaba, JD.com, Pinduoduo, NetEase and Baidu. That's according to Morgan Stanley research dated Aug. 26.