Causal Claims in Economics

We introduce a novel approach by constructing a knowledge graph for each paper in our dataset. In these graphs, nodes represent economic concepts classified using JEL codes, and edges represent relationships from a source node to a sink node. This means that if a paper discusses how one economic concept relates to another, we capture this as a directional link between those concepts. Whether or not a claim is considered causal depends on the method used to substantiate it. Specifically, we identify an edge as a causal edge if the claim is evidenced using causal inference methods such as Difference-in-Differences (DiD), Instrumental Variables (IV), Randomized Controlled Trials (RCTs), Regression Discontinuity Designs (RDDs), Event Studies, or Synthetic Control

To systematically evaluate these knowledge graphs, we develop three broad categories of measures. First, we track the narrative complexity of a paper, including the breadth and depth of claims. Second, we examine novelty and contribution, capturing whether a paper’s relationships are genuinely new or whether it “fills gaps” previously underexplored in the literature, distinguishing between causal and non-causal contributions. Third, we consider conceptual importance and diversity, focusing on how centrally a paper’s concepts sit within the overall network of economic ideas and whether the paper balances multiple causes (sources) with multiple outcomes (sinks). Additionally, by distinguishing these measures based on claims in the non-causal subgraph from those in the causal subgraph, we parse out the difference between general narrative features and those supported by rigorous identification.