Website privacy policies outline how a website handles personal information. Many such policies are written vaguely, making it difficult for users to comprehend what data is collected and how it’s used, posing a threat to privacy and security. The empirical study in this paper aims to address this issue by applying transformer models to automatically detect the level of vagueness in privacy policy texts. We calculate the vagueness scores based on annotations obtained through arithmetic mean, geometric mean, and harmonic mean of the individual annotations and discretize the policy text into four distinct vagueness categories. Our detailed numerical study shows that transformer models perform well for the vagueness classification tasks and they can be effective in enhancing the clarity and transparency of privacy policies. Our experiments also highlight the importance of reliable annotations and show that the transformer model performance can be enhanced with more reliable annotations with lower standard deviation and entropy values.
Article ID: 2023L30
Month: June
Year: 2023
Address: Online
Venue: The 36th Canadian Conference on Artificial Intelligence
Publisher: Canadian Artificial Intelligence Association
URL: https://caiac.pubpub.org/pub/t7g1t33y