Anthropomorphic AI and emergent vulnerabilities: An empirically informed legal evaluation of the protection of users of AI chatbots (AAI)

Chatbots and other large language models (LLMs) are increasingly displaying anthropomorphic qualities. Anthropomorphism, namely human-like qualities attached to non-human life or objects, can help us to make sense of the world, enabling social connection and emotional bonds. However, it can also bring risks and contribute towards serious harms. Notable examples include the tragic case of a Belgian man's suicide in 2023 and the death of an American teenager in 2024 following interactions with their AI chatbot ‘companions’. These examples do not merely represent isolated incidents, but also highlight an increasing concern over the effects of anthropomorphic AI, which may open a gateway to manipulation, increased vulnerability, and loss of control. This is increasingly a societal concern, as these chatbots are gaining rapid popularity around the world. Yet, despite these developments, we still know little about the risks of manipulation and exploitation of human vulnerabilities through AI chatbots, and there is lack of clarity about how existing legal frameworks protect users against these risks. Further, legal instruments protecting against manipulation and vulnerability exploitation are insufficiently informed by empirical research that looks at actual usage of these technologies. By conducting an empirical study with users of anthropomorphic AI chatbots, this interdisciplinary project will address these gaps with the aim of strengthening legal protections for individuals.