The Order of Criticism Revisited: A mixed-methods study of a century and a half of literary criticism in Sweden
In Lina Samuelsson’s 2013 study »The Order of Critique: Swedish Book Reviews 1906, 1956, 2006», a key feature was the study of the critical text s normative discourses: What constitutes a review? Which key concepts permeate the critical text? Despite several chosen limitations, the work gathered a significant quantity of transcribed material.
In the project The Order of Criticism Revisited: A mixed-methods study of a century and a half of literary criticism in Sweden we now intend to conduct a methodological and epistemological study of corresponding materials using digital tools. Samuelsson’s discourse analysis will be compared to a language technological analysis. By »corresponding material» we mean partly Samuelsson own transcriptions, but above all (the privilege of the big data) the ambition is to make as exhaustive analysis as possible of the critical material that is available for the chosen years. A new study, of the year 2016, will be implemented, and the project will also launch an interactive interface for visualizing – and operate – the symbiosis of text mining and discourse analysis.
The project will be able to compare, in detail, analog and digital analyzes of the same historical moments. The study will thus encourage several creative challenges for the technical language analysis, and highlight several methodological meta-perspectives at the intersection of traditional humanities, language technology analysis and epistemological aspects of digitization.
Final report
Purpose and development of the project
The aim of The Order of Criticism Revisited: A Mixed Methods Study of a Century and a Half of Literary Criticism in Sweden [Kritikens nya ordning: Mixade metoder i studiet av svensk litteraturkritik under ett och ett halvt sekel] (MXM19-1096:1) has been to “repeat and challenge” a study in literary scholarship by means of data-driven methods. The project has examined how “traditional” interpretative and digital analysis can strengthen one another in both practical and epistemological respects. The research has been based on a comparison of findings on the discourse of literary criticism in newspapers and periodicals in the years 1906, 1956, and 2006, drawing both on a study by project member Lina Samuelsson (Kritikens ordning, 2013) and on various forms of language technology analysis on a broader body of book-review material.
The project’s overarching objectives have been to: (1) reanalyse the approximately 700 book reviews that Samuelsson transcribed for her earlier study (2013) using language-technology methods in an interdisciplinary setting; (2) retrieve literary criticism from the National Library of Sweden’s (Kungliga biblioteket, KB) digitised newspaper collections for the years 1906, 1956, and 2006 (the same years as in Samuelsson’s study) in order to compare research results and train algorithms to automatically identify literary criticism in large-scale text collections; (3) reflect epistemologically on differences and possible synergies between data-driven and hermeneutic methods; and (4) study contemporary literary criticism from 2016 using digital tools developed within the project.
Brief description of the implementation
The project has been based on collaboration between the Gothenburg Research Infrastructure in Digital Humanities (GRIDH), Språkbanken Text (SB Text), the Department of Literature, History of Ideas and Religion (LIR) at the University of Gothenburg (GU), and Mälardalen University (MDU), resulting in joint project publications (Ingvarsson, Brodén, Samuelsson, Wåhlstrand Skärström & Zechner 2022; Brodén, Samuelsson, Zechner, Ingvarsson & Karimi 2025).
Project members at GRIDH, LIR, and MDU worked on interdisciplinary analytical and epistemological foci (project objectives 1 and 3) (Brodén, Ingvarsson, Samuelsson & Wåhlstrand Skärström 2024; Samuelsson, Brodén, Ingvarsson & Wåhlstrand 2024; Brodén, Samuelsson, Alfter & Malmstedt 2025; Brodén & Samuelsson 2026; Brodén, Samuelsson & Alfter 2026; Brodén, Samuelsson, Alfter & Karimi in press), as well as on more recent review material (project objective 4) (Samuelsson, Brodén, Ingvarsson & Karimi 2025). In addition to publications, this work (at GRIDH, LIR, and MDU) includes two interactive visualizations.
The SB Text project member developed, among other things, algorithms for categorizing book reviews, both in relation to other textual material and internally among different types of reviews (project objective 2), and for detecting semantic patterns within them (project objective 3), and also coordinated the associated annotation work carried out by paid student assistants at the KB Lab (project objective 2). In addition to a publication (Zechner 2024), this work has resulted in open-access datasets made available through Språkbanken Text.
Most significant results and conclusions
Through new analyses of already familiar book review material, the project has discussed epistemological aspects of synthesizing data-driven and hermeneutic methods. In particular, we have shown how the concept of “defamiliarization” can capture the potential of data visualizations and other data-driven analyses to make overlooked aspects of a well-known research corpus visible (Brodén, Ingvarsson, Samuelsson & Wåhlstrand Skärström 2024; Samuelsson, Brodén, Ingvarsson & Wåhlstrand 2024; Brodén, Samuelsson & Alfter 2026; Brodén, Samuelsson, Alfter & Karimi in press).
As part of this work, the project collected approximately 5,800 reviews from KB’s digitised newspaper collections, as well as from other digitisation projects focusing on cultural periodicals. Annotations of the KB material have been deposited at the KB Lab (see Dataset). This data collection has addressed an infrastructure-level challenge concerning the automated retrieval of specific article types in KB’s newspaper collections (Brodén, Samuelsson, Zechner, Ingvarsson & Karimi 2025). We have also examined how generative models can mitigate shortcomings in digitisation in data-driven analysis, while also discussing the epistemological uncertainty that this introduces with respect to the source material (Brodén, Samuelsson, Alfter & Malmstedt 2025; Brodén, Samuelsson & Alfter 2026).
The review corpus has been analysed using a range of language-technology methods, from simple word-frequency analyses to data visualisations, topic modelling, and word vectors based on the KB Lab’s large Swedish language models (KB-BERT) and generative models (including GPT). These analyses have clarified the strengths and weaknesses of different approaches in our discourse-analytical setting, while also demonstrating the importance of “mixing” data-driven and humanistic methods in order to produce more complex results (Brodén, Ingvarsson, Samuelsson & Wåhlstrand Skärström 2024; Samuelsson, Brodén, Ingvarsson & Wåhlstrand 2024; Samuelsson, Brodén, Ingvarsson & Karimi 2025; Brodén, Samuelsson & Alfter 2026; Brodén, Samuelsson, Alfter & Karimi in press).
The investigations have confirmed arguments in digital humanities about the importance not only of methodological robustness but also of context-sensitive analysis grounded in humanistic expertise when applying data-driven methods to historical material. Expertise in digital methods, literary criticism as a genre, and the relevant media-historical contexts has made it possible, among other things, to problematise what counts as a “book review,” as well as the assumption that human annotation necessarily provides a better benchmark than automated approaches. We emphasize that the automated retrieval of specific article types in digitised newspaper collections is supported by syntheses of data-intensive and context-sensitive methods, which together can deepen our understanding of the complexity of the task (Brodén, Samuelsson, Zechner, Ingvarsson & Karimi 2025).
New research questions
The breakthrough of large and generative language models during the project period has dramatically changed the possibilities for addressing our research questions. It would likely be possible to replicate several of the case studies with substantial results on the basis of such models.
Researchers from the project received seed funding from the Riksbankens Jubileumsfond (RJ) for three residential workshops on Literature’s Values (SEK 424,000), which later formed the basis for a larger – though as yet unsuccessful – application to RJ for a research programme.
Dissemination of results and collaboration
Research findings have been published continuously – open access – in international and Swedish journal articles, depending on focus (including Journal of Computational Literary Studies, Tidskrift för litteraturvetenskap (TFL), and Samlaren), as well as in conference papers at events oriented toward digital humanities (Digital Humanities in the Nordic and Baltic Countries 2022, 2024, and 2026; the Annual Conference of Computational Literary Studies 2024; and Huminfra 2025).
A central project publication is the edited volume Flows & Frictions: Mixed Methods for AI-Driven Research on Historical Media (Brodén and Samuelsson 2026), which is based on contributions to the symposium Flows & Frictions organized at the University of Gothenburg by the project, bringing together internationally and nationally leading scholars in the field (see Brodén 2026). The anthology has just come off the press and will be published digitally, open access, via GU's publication platform.
In addition, the project has been presented at conferences and symposia such as Synergies 2020 (Denmark), the National Subject Conference in Literary Studies 2024, Digital Epistemologies at Linnaeus University (2025), and Literature & AI (2025) (Norway). Project results have also been integrated into teaching in the master’s programme in Digital Humanities at the University of Gothenburg and in courses in Literary Studies at Mälardalen University.
With regard to public outreach (the so-called “third mission”), the project has received attention in national media, including Swedish Radio P1’s Godmorgon världen (2023-10-01), Aftonbladet (2019-06-30), and Biblioteksbladet (2023-06-16). The project has also been represented at Almedalen Week (2019, 2022) and Kritikbyrån in Helsinki (2026).
The aim of The Order of Criticism Revisited: A Mixed Methods Study of a Century and a Half of Literary Criticism in Sweden [Kritikens nya ordning: Mixade metoder i studiet av svensk litteraturkritik under ett och ett halvt sekel] (MXM19-1096:1) has been to “repeat and challenge” a study in literary scholarship by means of data-driven methods. The project has examined how “traditional” interpretative and digital analysis can strengthen one another in both practical and epistemological respects. The research has been based on a comparison of findings on the discourse of literary criticism in newspapers and periodicals in the years 1906, 1956, and 2006, drawing both on a study by project member Lina Samuelsson (Kritikens ordning, 2013) and on various forms of language technology analysis on a broader body of book-review material.
The project’s overarching objectives have been to: (1) reanalyse the approximately 700 book reviews that Samuelsson transcribed for her earlier study (2013) using language-technology methods in an interdisciplinary setting; (2) retrieve literary criticism from the National Library of Sweden’s (Kungliga biblioteket, KB) digitised newspaper collections for the years 1906, 1956, and 2006 (the same years as in Samuelsson’s study) in order to compare research results and train algorithms to automatically identify literary criticism in large-scale text collections; (3) reflect epistemologically on differences and possible synergies between data-driven and hermeneutic methods; and (4) study contemporary literary criticism from 2016 using digital tools developed within the project.
Brief description of the implementation
The project has been based on collaboration between the Gothenburg Research Infrastructure in Digital Humanities (GRIDH), Språkbanken Text (SB Text), the Department of Literature, History of Ideas and Religion (LIR) at the University of Gothenburg (GU), and Mälardalen University (MDU), resulting in joint project publications (Ingvarsson, Brodén, Samuelsson, Wåhlstrand Skärström & Zechner 2022; Brodén, Samuelsson, Zechner, Ingvarsson & Karimi 2025).
Project members at GRIDH, LIR, and MDU worked on interdisciplinary analytical and epistemological foci (project objectives 1 and 3) (Brodén, Ingvarsson, Samuelsson & Wåhlstrand Skärström 2024; Samuelsson, Brodén, Ingvarsson & Wåhlstrand 2024; Brodén, Samuelsson, Alfter & Malmstedt 2025; Brodén & Samuelsson 2026; Brodén, Samuelsson & Alfter 2026; Brodén, Samuelsson, Alfter & Karimi in press), as well as on more recent review material (project objective 4) (Samuelsson, Brodén, Ingvarsson & Karimi 2025). In addition to publications, this work (at GRIDH, LIR, and MDU) includes two interactive visualizations.
The SB Text project member developed, among other things, algorithms for categorizing book reviews, both in relation to other textual material and internally among different types of reviews (project objective 2), and for detecting semantic patterns within them (project objective 3), and also coordinated the associated annotation work carried out by paid student assistants at the KB Lab (project objective 2). In addition to a publication (Zechner 2024), this work has resulted in open-access datasets made available through Språkbanken Text.
Most significant results and conclusions
Through new analyses of already familiar book review material, the project has discussed epistemological aspects of synthesizing data-driven and hermeneutic methods. In particular, we have shown how the concept of “defamiliarization” can capture the potential of data visualizations and other data-driven analyses to make overlooked aspects of a well-known research corpus visible (Brodén, Ingvarsson, Samuelsson & Wåhlstrand Skärström 2024; Samuelsson, Brodén, Ingvarsson & Wåhlstrand 2024; Brodén, Samuelsson & Alfter 2026; Brodén, Samuelsson, Alfter & Karimi in press).
As part of this work, the project collected approximately 5,800 reviews from KB’s digitised newspaper collections, as well as from other digitisation projects focusing on cultural periodicals. Annotations of the KB material have been deposited at the KB Lab (see Dataset). This data collection has addressed an infrastructure-level challenge concerning the automated retrieval of specific article types in KB’s newspaper collections (Brodén, Samuelsson, Zechner, Ingvarsson & Karimi 2025). We have also examined how generative models can mitigate shortcomings in digitisation in data-driven analysis, while also discussing the epistemological uncertainty that this introduces with respect to the source material (Brodén, Samuelsson, Alfter & Malmstedt 2025; Brodén, Samuelsson & Alfter 2026).
The review corpus has been analysed using a range of language-technology methods, from simple word-frequency analyses to data visualisations, topic modelling, and word vectors based on the KB Lab’s large Swedish language models (KB-BERT) and generative models (including GPT). These analyses have clarified the strengths and weaknesses of different approaches in our discourse-analytical setting, while also demonstrating the importance of “mixing” data-driven and humanistic methods in order to produce more complex results (Brodén, Ingvarsson, Samuelsson & Wåhlstrand Skärström 2024; Samuelsson, Brodén, Ingvarsson & Wåhlstrand 2024; Samuelsson, Brodén, Ingvarsson & Karimi 2025; Brodén, Samuelsson & Alfter 2026; Brodén, Samuelsson, Alfter & Karimi in press).
The investigations have confirmed arguments in digital humanities about the importance not only of methodological robustness but also of context-sensitive analysis grounded in humanistic expertise when applying data-driven methods to historical material. Expertise in digital methods, literary criticism as a genre, and the relevant media-historical contexts has made it possible, among other things, to problematise what counts as a “book review,” as well as the assumption that human annotation necessarily provides a better benchmark than automated approaches. We emphasize that the automated retrieval of specific article types in digitised newspaper collections is supported by syntheses of data-intensive and context-sensitive methods, which together can deepen our understanding of the complexity of the task (Brodén, Samuelsson, Zechner, Ingvarsson & Karimi 2025).
New research questions
The breakthrough of large and generative language models during the project period has dramatically changed the possibilities for addressing our research questions. It would likely be possible to replicate several of the case studies with substantial results on the basis of such models.
Researchers from the project received seed funding from the Riksbankens Jubileumsfond (RJ) for three residential workshops on Literature’s Values (SEK 424,000), which later formed the basis for a larger – though as yet unsuccessful – application to RJ for a research programme.
Dissemination of results and collaboration
Research findings have been published continuously – open access – in international and Swedish journal articles, depending on focus (including Journal of Computational Literary Studies, Tidskrift för litteraturvetenskap (TFL), and Samlaren), as well as in conference papers at events oriented toward digital humanities (Digital Humanities in the Nordic and Baltic Countries 2022, 2024, and 2026; the Annual Conference of Computational Literary Studies 2024; and Huminfra 2025).
A central project publication is the edited volume Flows & Frictions: Mixed Methods for AI-Driven Research on Historical Media (Brodén and Samuelsson 2026), which is based on contributions to the symposium Flows & Frictions organized at the University of Gothenburg by the project, bringing together internationally and nationally leading scholars in the field (see Brodén 2026). The anthology has just come off the press and will be published digitally, open access, via GU's publication platform.
In addition, the project has been presented at conferences and symposia such as Synergies 2020 (Denmark), the National Subject Conference in Literary Studies 2024, Digital Epistemologies at Linnaeus University (2025), and Literature & AI (2025) (Norway). Project results have also been integrated into teaching in the master’s programme in Digital Humanities at the University of Gothenburg and in courses in Literary Studies at Mälardalen University.
With regard to public outreach (the so-called “third mission”), the project has received attention in national media, including Swedish Radio P1’s Godmorgon världen (2023-10-01), Aftonbladet (2019-06-30), and Biblioteksbladet (2023-06-16). The project has also been represented at Almedalen Week (2019, 2022) and Kritikbyrån in Helsinki (2026).