The New Wave of Review

Digital history is riding a "review wave." In the fall of 2015, the American Historical Association released its new "Guidelines for the Evaluation of Digital Scholarship in History". In February 2016, the association's flagship journal, The American Historical Review, published an exchange titled "Reviewing Digital History" that inaugurated its first venture into digital project reviews. In my own field, the Western Historical Quarterly began printing "Born-Digital Reviews" in the fall of 2015. The Journal of American History first started publishing website reviews in 2001, but in September 2013 changed this section to "Digital History Reviews" (the journal also publishes lengthier reviews of digital research projects in its "Metagraph" section). Moving forward, digital historians will increasingly find their work evaluated in some of the discipline's major print journals.

What's odd is the degree to which supposedly hidebound print journals are the ones propelling this recent wave of review. After all, it's not as if digital historians need print journals to review each other's work. Blogging, Twitter, and other online platforms have stood at the heart of the field for years. We often tout the speed and openness of these platforms compared to the molasses-slow publishing cycles or gated paywalls of print journals. And yet, with some rare exceptions, we don't use these platforms to engage in substantive or critical evaluation of the work of our peers. New digital history projects are released all the time. If you're like me, you stick mostly to virtual high-fives: you tweet a link to the project, offer congratulations and commendations, and maybe add it to a syllabus or workshop. Deeper engagement takes place mainly through informal conversations or behind the doors of classrooms - not exactly the sort of public, rigorous intellectual evaluation that drives a field forward. Our colleagues deserve better.

Digital history's reticence for critical online evaluation stands in contrast to, say, the lively exchange that unfolded in 2015 over Matt Jockers's Syuzhet package, a method that Jockers developed for identifying literary plot shapes using sentiment analysis. After Jockers first announced Syuzhet, Annie Swafford wrote a pointed critique of the method, and over the course of roughly one month the two literary scholars debated the validity of the method in a series of back-and-forth posts. Other digital humanities scholars weighed in from across the disciplinary spectrum. Whatever your thoughts on Syuzhet, the entire online exchange was a rigorous, substantive, and transparent evaluation of digital scholarship. So why do digital historians seem to prefer virtual high-fives to this kind of deeply evaluative online engagement?

There are a few reasons for the dearth of online reviews and critiques within the field of digital history. For one, there are real drawbacks to online platforms. The immediacy of writing a blog post affords less time for measured reflection or carefully crafted or revised responses than, say, a review in a print journal. Self-published posts also lack editorial oversight. A good journal editor can vet the qualifications of reviewers, help them improve and refine their critiques, and serve as a mediator between reviewers and the people they're reviewing. Without an editorial presence or a shared platform, online reviews run the risk of operating on unequal playing fields. One historian might be writing from a position of seniority or have a much larger or more vocal online readership than another. It's also a lot easier for someone like me to tout online exchanges as "lively" or "freewheeling" when I don't run the risk of getting denigrated or harassed because of my race or gender. Gatekeeping may be a dirty word, but openness isn't exactly a panacea.

There's also the broader challenge of subject specialization and expertise. Digital history's unifying thread is methodological, not thematic. As a historian of the nineteenth-century United States, just how deeply can I engage with, say, Vincent Brown's spatial history narrative of Jamaica's 1760-1761 slave revolt? I might be able to discuss its interactive design or the way it uses a spatial framework to circumvent textual silences in the archive. But am I really capable of evaluating Brown's interpretation of the revolt as a unified, strategic rebellion rather than a series of haphazard insurrections? Even more importantly, am I qualified to evaluate the significance of this claim in terms of how it changes our understanding of Caribbean history? Probably not. This is why it was so encouraging to see deep, thoughtful reviews of Slave Revolt in Jamaica in recent issues of Social Text and The American Historical Review. The reviews were written by Elizabeth Maddock Dillon, Claudio Saunt, and Natalie Zacek, all of whom combine subject expertise with considerable experience in digital humanities projects. Both Social Text and The American Historical Review also gave Vincent Brown the opportunity to respond to these reviews - exactly the type of substantive, scholarly exchange that seems to be in such short supply for digital history projects.

But, again: these exchanges took place in print journals. Consequently, there was a gap of more than two years between the project's release and the publication of reviews. This lag doesn't make the exchanges any less valuable, but it hews far more closely to the way the discipline reviews print monographs. In an alternate scenario, the scholarly exchanges between Vincent Brown and his reviewers might have unfolded in a series of online posts over the course of a few months, rather than a few years, after the project's release. Moving this back-and-forth out from behind the paywalls of Duke University Press and Oxford Journals could have allowed for other scholars to weigh in, much like what happened after the initial posts between Matt Jockers and Annie Swafford during the Great Syuzhet Debates of 2015.

Ultimately, though, I find the format of this new wave of digital history less interesting than its substance. There are a few different ways to evaluate digital history projects, which I would group loosely under pedagogy and public engagement, academic scholarship, and what historian Fred Gibbs terms "data and design criticism." Most digital history reviews fall under the first category of public engagement and pedagogy. The Journal of American History's "Digital History Reviews", for instance, frames its reviews follows: "The goal is to offer a gateway to the best works in digital history and to summarize their strengths and weaknesses with particular attention to their utility for teachers [emphasis added]." As I write in a forthcoming article for Debates in Digital Humanities 2016, this emphasis reflects the field's particular genealogy and its roots in public history initiatives. Both the reviewers and the projects themselves continue to position digital history in terms of public engagement rather than academic scholarship.

Some reviewers, of course, do try to evaluate digital history projects as works of academic scholarship, akin to a scholarly monograph. This second approach, conducted in large part by field specialists rather than "digital" historians, often compliment the public-facing dimension of a digital project before ultimately critiquing its shortcomings in terms of historiography and interpretation. In a review of Richard S. Dunn's website Two Plantations, Kirt von Daacke notes that the site's archival collections "represent the best of digital media." He ends the review, however, with a standard complaint: "Frustratingly, Two Plantations never indicates its target audience, only hints at interpretation, and ignores historical literature altogether. Its analysis section never really answers the questions it poses, nor does it situate Dunn's interpretation in the broader scholarship on slavery." Without explicit interpretive claims to grab onto, trying to evaluate the scholarly contributions of digital projects can feel like trying to scramble up a smooth wall.

The third approach to reviewing digital history focuses on a project's design, interface, methods, pipelines, and datasets. These kinds of "data and design criticism", to borrow Fred Gibbs's formulation, often make a passing appearance in digital history reviews, such as describing a website's layout or critiquing the usability of certain features. Few reviewers, however, put it at the center of their evaluations. One recent exception is Joshua Sternfeld's lengthy review of Digital Harlem in the American Historical Review. In it, Sternfeld offers a prolonged description of the site's digital infrastructure and features before launching a blistering critique of the project. He questions the representativeness of the project's archive, criticizes its method of data entry and sampling, and ultimately describes Digital Harlem as "subverting the provenance of the source data." For his part, the project's co-creator Stephen Robertson returns serve with an equally blistering counter to Sternfeld's review. Robertson argues that Sternfeld "misrepresents the design and content of the site" and "only fitfully engages with the spatial orientation of Digital Harlem." Whatever side of the exchange you come down on, the back-and-forth illustrates how questions of data and design can stand at the center of digital history reviews.

I find myself frustrated by all three kinds of digital history reviews. First, I appreciate the value of evaluating projects in terms of pedagogy and public engagement. But the preponderance of this first kind of review reinforces the (false) notion that digital history does not, in fact, add substantive new academic knowledge to the field. This notion feeds into the second kind of review, one that takes digital projects to task for shortcomings surrounding academic argument and interpretation. I'm actually sympathetic to this kind of review, but they often mistakenly evaluate digital projects in terms of what the reviewer wants them to be (a traditional academic monograph) rather than what they are (an online exhibit, research tool, pedagogical resource, etc.). Finally, I worry that the third strand of digital history review - "data and design criticism" - will further exacerbate what I see as the field's problematic privileging of method over argument. Data collection, interactivity, visualization and design - all these features should be part of the review process, but they need to be grounded in a frank evaluation of whether and how they lead to new knowledge or interpretations about the past.

Does a digital history project fundamentally change how we understand a particular topic? How does it fit within the existing literature about this subject? What are a project's methodological strengths or flaws specifically in relation to the project's historical contributions? As a field, we need to dig deeper into these kinds of questions when we evaluate each other's work. A call to burrow into the scholarly weeds of historiography and interpretive nuance puts me at odds with one of digital history's core tenants: cultivating a broad audience. The general public doesn't necessarily care how a particular scholarly brick fits within the grand edifice of historical knowledge. Neither, for that matter, do literary critics, media theorists, philosophers, or the rest of our colleagues in the broader digital humanities community. Hell, a lot of historians don't want to wade too deeply into debates and arguments outside their specialization. But that doesn't mean we shouldn't do it.

I'm calling for digital historians to seize and shape the current wave of review. Regardless of whether we do so in blogs or print journals, we need to more substantively evaluate the work of our peers. We need to evaluate and critique each other's work not just in terms of public engagement and pedagogy or data and design, but in terms of new historical knowledge, insight, and interpretations that these projects contribute to the field. In the next few days I'm going to follow my own advice and post a review of a digital project related to my particular sub-field of nineteenth-century U.S. history. Readers who aren't in this sub-field might find it tedious, but my hope is that it will spark similar evaluations of other digital history projects. Stay tuned...