The initial repository for research on evaluating GPT language models’ performance on the Uniform CPA Examination. This project served as the foundation for the more comprehensive “GPT as Knowledge Worker” research.

Project Status

This repository represents the early stages of the CPA exam evaluation research. The authors recommend referring to the gpt-as-knowledge-worker repository for the complete and current version of this research.

Research Context

Part of the broader investigation into AI capabilities for professional knowledge work, specifically focusing on:

Accounting and financial reasoning
Regulatory understanding
Ethical decision-making
Professional certification requirements

Publication

The research evolved into the paper:

GPT as Knowledge Worker: A Zero-Shot Evaluation of (AI)CPA Capabilities
Authors: Jillian Bommarito, Michael James Bommarito II, Daniel Martin Katz, Jessica Katz
Published: January 2023
Also available on arXiv

Key findings: GPT models achieved up to 57.6% accuracy on CPA exam questions with optimal prompting, significantly outperforming random guessing (25%) and showing strong non-entailment with 82.1% top-2 accuracy.