Don’t ask a chatbot about the privacy and security implications of generative AI

This article was not written by a chatbot. If it were, would you be able to tell the difference? Would you still read it? How would you know if the information was accurate? You may already use an AI-powered chatbot like ChatGPT. There are many other chatbots on the market right now, including Bard, Bing Chat, and BlenderBot. Maybe you use them to help answer a question, write an email, brainstorm an idea, or come up with a clever joke. While this technology is fun to play around with, there are sector-specific applications and implications of this technology. For example, it could be used to summarize internal reports, draft contracts, evaluate student papers, and generate patient reports, among other possibilities. It has the potential to increase productivity, produce revenue, and could be worth billions of dollars. Despite the economic potential, technologies like ChatGPT could also be used to cause harm. ChatGPT and other chatbots have been used in dangerous ways—from spreading misinformation and hateful content, to providing source codes for malware, providing instructions on weapons creation, and outlining detailed information about illegal activities. Kristen Csenkey is a PhD at the Balsillie School of International Affairs and a fellow with the North American and Arctic Defense Security Network. Photograph courtesy of Kristen Csenkey There are clearly security and privacy implications, not only with ChatGPT, but also the use of this specific generative AI technology. In early April 2023, the Office of the Privacy Commissioner of Canada (OPC) launched an investigation into OpenAI, the U.S. company that owns and developed ChatGPT. Concerned with the “collection, use and disclosure of personal information without consent,” the OPC has joined the growing number of concerned regulatory, privacy, and security bodies around the world who have questions about the technology and concerns about its implications. To appreciate some of these concerns, let’s first understand the how the technology works. ChatGPT is a "Generative Pre-trained Transformer" (GPT). This generative AI system, or model, is trained on natural human language as text. It can complete certain tasks by transforming an input, or prompt, into a response as an output. This type of AI system is pre-trained on large datasets to generate an output that humans can understand and use, for example, in answering a question. Generative AI systems, large language models using natural language processing, and models using transformer architecture have been around for years. There are also digital image and audio-based models, like DALL-E or Midjourney. Although the latest version of ChatGPT was released in November 2022, this technology is not new, nor are the privacy and security concerns. These concerns with ChatGPT and other AI-powered chatbots are related to the use of large datasets to train and generate outputs. Information from these datasets comes from the internet and, as we know, the information on the internet is not always accurate or kind. Some of this information might also be confidential and personal. There is also a lack of transparency about how personal data is collected, processed, stored, and shared. Citing privacy concerns, Italy temporarily banned the technology. Yet, is a ban a practical approach to the problems associated with generative AI technologies? Two actions are needed to address concerns associated with this technology: first, regulation is needed to ensure data protection and privacy. Second, co-operation is needed manage associated security issues and ensure transparency. The OPC and regulators in Canada need to leave room for the positive potentials of the technology while steering clear of an outright ban. Canada should create an expert taskforce to understand and meet the challenges posed by the unregulated use of this technology. The United Kingdom and the European Union are both taking steps to ensure the safe development of generative AI technologies. Both are creating taskforces with data protection and regulation in mind. Co-operation is also needed with government, the private sector, academia, and civil society partners. This is especially important as generative AI technologies become more readily available for internal use as a service, with applications in the health-care, education, financial, security, and other sectors. Kristen Csenkey is a PhD (ABD) at the Balsillie School of International Affairs and a fellow with the North American and Arctic Defense Security Network. She researches cyber governance and emerging technologies. The Hill Times