Computational structuralism: Toward a formal theory of meaning in the age of digital intelligence

The discovery that “next-token predictor” language models can fluently produce text has important but underappreciated theoretical implications. Most notably, their success demonstrates that fully relational models with no access to external referents or human actors are sufficient to generate contextually appropriate discourse. Building upon this insight, this article proposes computational structuralism, a perspective that synthesizes insights from deep learning, information theory, and French structuralism to interpret the success of large language models and provide a vocabulary for rigorous and formalized inquiry into systems of meaning. Computational structuralism conceptualizes the world as an information system rife with pattern and redundancy. Patterned observations can be compressed into latent structures that enable both interpretation and the generation of contextually appropriate responses. Large language models achieve this with web text, suggesting that discourses can be sufficiently modeled as systems of transformations over a set of relationally defined elements. Today’s AI models thus provide an empirical “proof of concept” for what structuralist theorists sought to achieve decades ago, a distillation of social life’s complexity into the latent forms that render its content meaningful.

In this article, I argue that recent advances in modeling and generating language can be interpreted through a synthesis of deep learning and information theory with classical theories of language and culture. This perspective, which I call computational structuralism, begins with the premise that the social world is highly patterned, that these patterns make learning and communication possible, and that machine learning algorithms can distill these social patterns into generative models when given enough observations. Moreover, by learning to generate discourse through access exclusively to words and their relations rather than their content, LLMs successfully operationalize a fully relational model of meaning. This perspective orients analysts toward the discovery of the organizing principles of social and cultural systems, and by privileging form over content, lays the groundwork for a programmatic and cumulative science of meaning and culture.

It is important to note that this framework privileges sufficiency in modeling over necessity. LLMs’ capacity to generate coherent texts does not imply that they draw upon the same operations as humans to encode and decode socio-linguistic patterns. But it demonstrates with formal and mathematical precision one way this can be done. While the human mind largely remains a black box, inaccessible to direct observation and resistant to formalization, LLMs are fully mathematized systems. The internal representations of machine learning models are difficult to interpret, but they operate through known computational processes that can be directly observed and analyzed. For the first time, fluent natural language has been achieved through a fully transparent sequence of operations. Analyzing this success promises not only to theoretically clarify some of these new technical advances, but may also lay a foundation for a theory of culture and communication grounded in what we have only recently learned is possible with language.

The argument proceeds as follows. I begin with a discussion of deep learning, highlighting the foundational concept of scaling. Specifically, I describe a core tenet of deep learning – that complex tasks, including language modeling, are best accomplished through inductive patterns discovery by supplying massive models with tremendous amounts of data and compute. Second, I recast this phenomenon in the terms of information theory. I clarify that “pattern” can be conceptualized as redundancy or mutual information between observations. In becoming a “next-token predictor,” LLMs also become text compressors, efficiently storing knowledge of the data distribution by learning general discursive patterns that can be flexibly reapplied across contexts. Third, I again recast this discussion in the terms of French structuralism. Structuralism began with the recognition that languages can be studied formally, that they comprise internal structures that can be understood independent of specific linguistic content. Later theorists expanded on this intuition to argue that this social and cultural life more broadly can be distilled into underlying principles that guide and constrain the generation of appropriate social action. Fourth, I tie these strands together, arguing that the capacity of LLMs to generate meaningful and contextually appropriate texts implies that (a) transformations over relational structure is sufficient for the generation of culturally and situationally specific discourse, and (b) such a structure can be inductively derived from discourse traces – phenomenal or embodied engagement with the world is not a necessary condition. Finally, building on recent studies of LLM internal representations, I describe how processes of interpretation can be formally and mathematically specified..

Structuralism was initially conceived as a theory of language. Linguistics had, until the twentieth century, been a historical and comparative enterprise, largely focusing on ancient languages and their evolution over time. This changed with the work of Saussure (1959 [1916]), who reoriented the field away from history and toward the structure and function of contemporary languages. Saussure understood language to be fundamentally relational; there are not a predetermined set of things in the world to which language assigns labels. Rather, it is language that carves up the world, and word's denotation is given scope by the presence of other words. Saussure evocatively describes this vision with a metaphor of a sheet of paper; on one side is all the stuff of the world, blending into each other by continuous gradations of difference. On the other side is the system of words, each delimiting the others. Poking through the paper reveals the mapping between language and the world. But critically, the "language side" can be understood as a system unto itself, with meanings defined relationally and principles of usage that determine how words can be arrayed in valid sequences. While language in the world is observed only through specific writings and utterances (parole), these statements are intelligible because they are generated according to the principles of a latent, collectively shared system (langue). For Saussure, the task of linguistics is to understand this latent system, both in its static, synchronic principles of organization and its dynamic, diachronic patterns of evolution.

Motivated by linguistics’ shift from the study of particular languages to the discovery of general linguistic forms, Lévi-Strauss (1963) sought to initiate a similar move in anthropology. The overarching thrust of Lévi-Strauss's theoretical work can be conceived as an effort to generalize structuralism to the full breadth of symbolic and communicative action. For Lévi-Strauss, meaning systems are not comprised solely of relations between phonemes, but between concrete entities like places and social groups. Moreover, because the various domains of social life are organized around relatively few sets of binary oppositions, objects from one domain can become standins to reason about another. Thus, the relations between groups can be understood through relations between totemic animals, or the relations between men and women through terms of geography (Lévi-Strauss, 1966). Two features of Lévi-Strauss’s contribution are of particular importance here. First is the extension of structuralist thinking beyond linguistics to culture more broadly. This theoretical move established that structuralist models are valuable not only for describing languages, but the underlying systematization of human thought and understanding. Second, is Lévi-Strauss’s insistence that meaning systems are structured by relatively few binary oppositions such as light/dark, male/female, life/death. These binary oppositions at once outline a clearer semantic organization of the cultural system, but also suggest a compression of observations into a structured set of features. The diversity and complexity of the world is rendered meaningful through the repeated application of a broadly generalizable system of classifications.

Although Lévi-Strauss may seem a marginal character in modern sociology, the core insights and concerns of his structuralist approach have persisted through the work of Bourdieu, who inherited and elaborated on many of structuralism’s core tenets. Lévi-Strauss's influence on Bourdieu is perhaps clearest in his descriptions of the classification schemes that demarcate social positions. Where Lévi-Strauss (1963), for example, describes a structuring contrast between the sun, the East, and the masculine on one hand, and the moon, the West, and the feminine on the other, Bourdieu (1984) highlights how the neat, modest, thin, and bourgeois stand in contrast against the messy, generous, thick, and working class. Bourdieu’s description of the system of classifications is often described as being less rigid than Lévi-Strauss’s, perhaps because Bourdieu’s representation of classification is more geometric where Lévi-Strauss’s is algebraic. For Levi Strauss, one myth can be mapped onto another through a series of discrete substitutions and transformations – instead of ascending upward, the hero descends downward; instead of gaining a wife, he loses a wife. This vision of culture forms a lattice of distinctions along with a set of operations defining transitions between states. By contrast, Bourdieu’s representation of social space is continuous. Actors position themselves along smooth gradients between the demarcated positions of social space. A high school teacher sits somewhere between a banker and an artist, “Rhapsody in Blue” sits between “Blue Danube” and “The Well-Tempered Clavier” (Bourdieu, 1984). Moreover, in this continuous space, many of the distinctions used to categorize and classify the world are themselves relationally defined. To be “avant-garde” is to be some combination of “young,” “intellectual,” “brash,” and “inaccessible.” Ultimately, while Bourdieu's social space retains more complexity than Lévi-Strauss's, it is still an extreme compression of the total diversity of observable social positions. The relation of the academic outsiders to the chaired professors is analogous to the relation between the starving artists and the gallery curators. Even if the actors occupy different "semi-autonomous fields," Bourdieu is quick to note the homologies between fields; with familiar dynamics of power and capital characterizing the art world, academia, journalism, the peasantry, and middle-class society (Bourdieu, 1984, 1988, 1989; 1990).

The structuralist vision is also central to Bourdieu's most famous theoretical contribution, his concept of habitus. While structuralism is certainly suggested in his famous description of the habitus as a "structuring structure and a structured structure," it is his description of the habitus as a set of dispositions that are "transposable" and "generalized" that most crucially maps onto the core features of structuralist theory. The banker who is modest in his dress is modest in his speech, is modest in his décor, and is modest in his politics (Bourdieu, 1984). By applying a classification schema that interprets the world around him according to a relatively low-dimensional system of distinctions, he can easily recognize what is "for him" or "not for him". This disposition toward modesty is generalized and transposed across contexts, greatly simplifying an otherwise unmanageable choice set. In applying this schema, and adopting the clothes, the speech, and the décor of a banker, he reproduces this symbolic order; and meanwhile, rebellious artists see the banker and recognize everything that is "for him" is "not for them." Through this iterative process of socio-cultural reproduction, individuals build cognitive models of social space, then through acting upon that model, re-structures social space in its image. The cognitive patterns of mental operation and material patterns of the external world thus become convergent and self-reinforcing, aligning the information structure of the world with mental structures of classification.

Scholars debate Bourdieu's precise relation to structuralism (Joas & Knöbl, 2011; Lizardo 2011; Sewell, 1992), arguing that his theory aimed to combat the deterministic and “rule-like” systems described by classical structuralist thinkers. But structuralism does not, at its core, require deterministic patterns of behavior. Indeed, Bourdieu's usage of structure is not unlike the Saussurian structure of language. A language does not predetermine what an individual says. Within a language, a vast array of valid statements are possible, but an exponentially greater set of statements are impossible. Within this tremendously broad yet starkly limited structure, actors play out familiar patterns, strategize, and improvise (Bourdieu, 1990). Structuralism also persists in contemporary sociology through a more empirical program launched by Harrison White and continued by his followers. White’s pioneering work in social network analysis emerged out of a program to mathematize the structuralist intuitions expressed by Lévi-Strauss and his contemporaries (White, 1963; see also Bearman, 1997). But the program advanced by White and his students was not merely mathematical or methodological; network analysis and related techniques served as an operationalization of a relational ontology born out of classical structuralist thought. Specifically, this work was motivated by the commitment to the co-constitution of actors and relations – that relations are defined by actors and actors are defined by relations (Breiger, 1974, 2000; Carley, 1991; Padgett & Ansell, 1993; White, 2008) – and the conviction that complex social forms can be distilled into relatively simple generative principles (Bearman et al., 2004; Martin, 2009).

While relatively few sociologists today identify as "structuralists," the core agenda of structuralism remains an animating influence in sociological research, especially among the quantitative and increasingly computational. “Relational sociology” revitalized some of structuralism’s core theoretical tenets (Emirbayer, 1997), and notably shifted the formal and relational thinking to questions of culture (Fuhse, 2009; Gibson, 2005; Mische, 2009; Mohr & Duquenne, 1997). For a new generation of researchers, methods like dimension reduction (Fuhse et al., 2020), belief network analysis (Boutyline and Vaisey 2017), relational/correlational class analysis (Boutyline, 2017; Goldberg, 2011), and word embeddings (Kozlowski et al., 2019) hold the same promise that Lévi-Strauss saw in careful ethnology – the potential to reveal the underlying form of social life that gives meaning to its content. Text has proven to be a particularly fruitful avenue for these investigations (Boutyline & Arseniev-Koehler, 2025; Grimmer et al. 2022). While text has long served as an essential form of data for the analysis of meaning, the mass digitization of texts and ongoing improvements in computational power have pushed researchers to go beyond manual coding to the development of purely formal and data-driven approaches (Hoffman et al., 2018; Lee & Martin, 2015). In a way, LLMs are an extension of this sociological program, but on a much larger scale – inductively deriving formal, relational models of meaning from the discursive patterns of text. But by massive scaling of data, parameters, and compute, LLMs have realized the potential of this paradigm far beyond the prior explorations of social scientists.

The concerns of the machine learning engineers and the cultural theorists hence come to an unlikely convergence. Both groups now ask: does a semiotic system like language have an underlying structure, and if so, how can we capture it? I argue that LLMs, more than any model before them, operationalize Saussure's concept of langue. Critically, langue is not the set of all valid statements; it is the system that can interpret and generate all valid statements. In some ways, however, LLMs are realizing the visions of Lévi-Strauss and Bourdieu more than that of Saussure. Being trained on millions of pages of actual natural language published on the web, LLMs do not learn some ideal and pure “language” abstracted from actual and practical use in social life. Rather, they learn language that is socially and culturally situated (Park et al., 2024). While these models certainly do learn to discriminate grammatically correct from incorrect statements, they also learn to tell socially and contextually correct from incorrect sentences.2 They learn which voices are likely to make which statements in response to which situations. And conditional on those likely (or unlikely) statements, they learn how the audience is likely (or unlikely) to respond. What LLMs learn, therefore, is not merely the structure of language, but the structure of culturally specific linguistic action and interaction (Kozlowski & Evans, 2025). This essay began with the question of why large language models work at all. After an initial overview spanning deep learning, information theory, and structuralist theory, we are positioned to provide a preliminary answer. Language, and the diverse linguistic cultures that it supports, are defined by a multitude of patterns, simple and complex, subtle and obvious, micro and macro. To model language is to compress it – to remove redundancies by replacing them with generative principles. This distilled compression of language can be achieved through prediction, as the same statistical dependencies that inform prediction are those that compose the compressed model. Because linguistic practices are patterned, prior words contain information about subsequent words, and it is thus possible to define a function that predicts subsequent words conditional on prior words. And because neural networks, given sufficient parameters, data, and compute, can act as universal function approximators, we can have confidence that the model’s general architecture can theoretically capture whatever “discourse functions” give structure to culturally situated language. Moreover, because LLMs are trained on examples of natural text, the statistical dependencies they learn do not merely constitute rules of grammar, but cultural patterns like how to talk like a Republican, how to write a poem, how to make a sarcastic joke, how to cheer up a friend who is having a bad day, and how to describe the experience of falling in love (Argyle et al., 2023; Jones & Bergen, 2025). In the first years of the 2020s, all this subtle and intangible stuff of social life was, for the first time, successfully mathematized and formalized (Achiam et al. 2023; Brown et al., 2020). Because this was achieved through deep learning, we still do not understand these formalisms, but we know that they are possible. And although it remains in its early stages, the growing field of mechanistic interpretability is making rapid progress in “reverse engineering” LLMs to elucidate their internal operations.

Perhaps most critically, LLMs have no contact with the external world described in their training texts (Bender et al., 2021). LLMs have no conception of what "water" is, except by how it relates to words like "wet," "ocean," "drink," and every other word. And it only knows what those words mean by how they relate to "water" and other arbitrary signifiers. An LLM, at least in its classical form, never sees, feels, or hears water; it only sees dependencies between words.3 In structuralist terminology, terms are defined through a system of differences in relation to other terms, and the patterning of these differences enables general classification. Returning to Saussure's classic metaphor, the LLM only ever sees one side of the paper – only words and never their referents. In a surprising vindication, LLMs demonstrate the possibility of learning the “synchronic structure” of practical conversation purely from examples of natural language, with absolutely no contact with the world. Historically, many critics have argued that humanlike behavior requires embodiment or sentience or some advanced form of neuro-computation that we have not yet discovered (Dreyfus, 1972; Penrose, 1989). While there are certainly some instances where LLMs’ lack of human experience is betrayed through its behaviors, such cases are shockingly rare (Jones & Bergen, 2025).

The recognition that LLMs are encoding the multitude of socio-linguistic structures that constitute discourse immediately suggests the tantalizing possibility that we may peek inside the model to view these structures ourselves. Fortunately, the field of “mechanistic interpretability,” which aims to reverse engineer the internal conceptual circuitry of deep neural networks, has made rapid progress over the past several years (Templeton et al., 2024). This program has already developed relatively reliable methods for extracting individual features such as sentiment (Tigges et al., 2023), honesty (Templeton et al. 2024), or political leaning (Kim et al., 2025), from deep models’ latent spaces. Current work is now orienting towards more advanced and subtle behaviors, like discovering how complex tasks like writing coherent rhymes or solving math problems can be decomposed into complex "circuits" out of assemblies of simpler individual features (Lindsey et al., 2025).

Although this stream of research is still in its infancy, it already promises to enable investigations into a variety of questions of linguistic and discursive structure: How are perspectives such as political ideologies or scientific schools of thought composed from simpler, constitutive features? Can we identify the features that best discriminate opposing ideologies or schools of thought? Does the model reconfigure the geometric relations between basic features like "good-bad,” “male–female,” and “strong–weak” when prompted to generate texts the styles of different discourses, social identities, or personas (Argyle et al., 2023; Park et al., 2024)? Interpretability techniques also present the unique methodological opportunity to casually intervene upon a discourse’s latent structures, then observe the effects of the intervention on model outputs. For instance, how are inter-partisan interactions modulated if certain features, such different forms of affect or emotion, are amplified or muted?