Chain-of-thought reasoning in language models

Abstract

This work looks at whether chain-of-thought prompting actually helps language models think through multi-step logic problems, or if it just makes their mistakes look more convincing. The key takeaway is that while walking through the reasoning step-by-step does improve accuracy, the models are still prone to subtle logic slips and the explanations they generate do not always reflect how they really got to the answer.