ChatGPT Misused for Fake News in China: An Arrested Man and Its Implications

ChatGPT is an open-source AI system developed by OpenAI that uses deep learning and natural language processing to create coherent and diverse texts on various topics. ChatGPT has many potential applications, such as writing stories, poems, essays, summaries, lyrics, code, and more. However, it can also be misused to spread misinformation, propaganda, or malicious content online.

On April 25, 2023, a fake news article claiming that nine people had been killed in a local train accident in Gansu province, China, was posted on Baijiahao, a blog-style platform run by Chinese search engine giant Baidu. The article was simultaneously posted by more than 20 accounts and received more than 15,000 clicks before it was detected by the cyber division of the county police bureau. The police traced the origin of the article to a company owned by a suspect surnamed Hong, who operated personal media platforms registered in Shenzhen, Guangdong province. Hong was detained on May 8 for "using artificial intelligence technology to concoct false and untrue information", according to a statement by the Gansu public security department.

Hong confessed that he used ChatGPT to generate the fake news article by inputting the elements of trending social stories in China from past years. He then bypassed Baijiahao's duplication check function to publish the article on multiple accounts he had acquired. He said he did this to attract attention and increase traffic to his platforms. Hong was charged with "picking quarrels and provoking trouble", a crime that normally carries a maximum sentence of five years in China. However, in cases that are deemed especially severe, offenders can be jailed for 10 years and given additional penalties.

This is the first time that the public has been made aware of an arrest by Chinese authorities for misuse of ChatGPT after Beijing's first provisions to regulate the use of "deepfake" technology officially took effect in January 2023. The provisions define deep synthesis as the use of technologies - including deep learning and augmented reality - to generate text, images, audio, and video and to create virtual scenarios. The provisions require internet information service providers to clearly mark any content that is generated or modified by deep synthesis technology and to obtain consent from the subjects involved. The provisions also prohibit the use of deep synthesis technology to endanger national security, public safety, social order, or infringe on the legitimate rights and interests of others.

China closely monitors its social media through firewalls, especially Sina Weibo, which has over 592 million users, to ensure no critical content against the ruling Communist Party of China (CPC) is circulated. The country has also cracked down on online rumors, fake news, and cybercrimes in recent years. The arrest of Hong for using ChatGPT to spread fake news is a warning to other potential offenders and a reminder to the public to be vigilant and discerning about the information they consume online.