Zero-Shot Verification-guided Chain of Thoughts

(P1) Base: Base represents the baseline zero-shot prompting style for question answering that we use. Given a question, such as “What is 2+2?", the prompt to elicit an answer would look like: “User: Q: What is 2+2? \n Assistant: A:".

(P2) COT: COT (Chain Of Thoughts) represents the standard zero-shot COT prompt (Kojima et al.,2022). This can be expressed as “User: Q: What is 2+2? \n Assistant: Let’s think step by step.". Such a prompt is designed to prime the models to generate more explicit chain of reasoning steps before producing an answer.

(P3) PS+: PS+ represents the plan-and-solve zero shot PS+ prompt presented by Wang et al. (2023a) as an extension over plain COT. This can be expressed as: “User: Q: What is 2+2? \n Assistant: A: Let’s first understand the problem, extract relevant variables and their corresponding numerals, and devise a plan. Then, let’s carry out the plan, calculate intermediate variables (pay attention to correct numeral calculation and commonsense), solve the problem step by step, and show the answer."