ChatGPT has demonstrated the ability to generate plausible human-like text and research is underway to evaluate and benchmark its current performance in various domains. The research we present here provides a preliminary benchmark on ChatGPT’s ability to emulate the style and information presented in financial statement note disclosures. Using text from Canada’s major banks (n = 5) over the period of 2019–2021, we query ChatGPT to generate two required note disclosures and compare its text against the note disclosures written by the banks in their corporate annual reports. We find that the similarity between ChatGPT’s text and the human-authored text is very low, but also find that ChatGPT’s text is significantly more readable for one of the two disclosures (p < 0.05).
Article ID: 2023L9
Publisher: Canadian Artificial Intelligence Association