A panel (“The Situated Seen & Heard: (De)Constructing Digital Material Contexts”) with Albert Rintrona and Tessa Brown at MSU’s Cultural Rhetorics Conference

My name is Jana Rosinski and I’m a second year doctoral student at Syracuse. This is an attempt at articulating my interests in post humanism, new media, material rhetorics, historiography, and digital humanities to create more materially minded practices of research and research methods as mattered through interface design.

This is a mattering of concern. What matters?

I recently had the opportunity, through a graduate seminar in game studies, to immerse myself in Oculus Rift, for those unfamiliar, it’s a virtual reality headset for 3D gaming.

For this experience, I wore the OR headset and a pair of headphones to be surrounded by visual and aural sensoria that demanded my engagement. Through the Rift I interacted with SightLine, a game described as “a surreal experience designed as a demoing tool. Crafted to throw users into the world of VR, and show off its full potential both in creating a variety of worlds that feel real and creating realities that behave unlike anything you might know.”

SightLine begins with a body, your body or at least a placeholder or placement, seated at a desk about which are books, papers, glasses, a picture frame, a plant—materials unremarkable. Within the picture frame a video of a man speaking begins to play. SightLine doesn’t need directional input from a controller or keypad because the direction in which you are looking with your head, your looking, is the mechanical input for action. The man explains that seeing is believing, but that you shouldn’t believe what you saw. He calls your attention back to the unremarkable materials of the desk to ask whether or not the plant that resides there is the same one from moments before. Everything out of your SightLine is subject to change.

My purpose today isn’t to talk about SightLine or 3D gaming technology, but in what it calls my attention to. What interests me about SightLine and the Oculus Rift is its direct engagement and even betrayal of sight as a mode of understanding. My looking was what oriented not only my spatial presence and affect in and on the materials of SightLine—that which I illuminated with my attention—but what obscured my noticing or knowledge of other materials that surrounded me, what my looking did and did not encounter.  I say encounter because the effects of my attention are indeterminate: was I causing affect in my presence and absence of materials? Or was I bearing witness to the inherent potential of materials? Everything out of my line of sight was subject to change, but for what purpose? And what of the materials that I was trying to see?

I liken these questions, this experience, to my interest in digital humanities methods and tools and the complex consideration of their effect on what can be seen, heard, felt.  In “Humanistic Theory and Digital Scholarship”, Johanna Drucker questions of humanities scholars what impact the humanities have had on the digital environment, and the possibility of digital platforms and interfaces that are created from humanistic methods instead of the borrowing of methods from outside of the discipline, which she describes as at odds with the cares and concerns of humanities work. She explains that humanities work has encountered digital tools, but what of humanities tools in digital contexts? A humanistic approach, she explains quote

means that the premises are rooted in the recognition of the interpretative nature of knowledge, that the display itself is conceived to embody qualitative expressions, and that the information is understood as graphically constituted.

The interface is not a lens through which we see—it is not transparent; what is there is as significant as what is not in showcasing and obscuring what we can and cannot encounter and how. What might be possible if we imagined interface as an access point to material—as methods that are audible, tactile, visible in accessing materials? In Lingua Fracta, Collin Brooke explains that the interface can show us that what we think of as products (books, articles, essays) are but specialized instances of an ongoing process at the level of the interface. Those products, or materials, are a temporal and situated binding of thoughts at a time. Quoting Paul Miller, a.k.a DJ Spooky, “At the end of the day, it’s all about reprocessing the world around you”. What would it mean to imagine interfaces as encounters, recontextualizations, and reprocessing of the worlds around us? To be able to articulate our sight line, our focus, and engage it with those of others looking at the same materials to process and reprocess materials differently? What follows is a practicing of reprocessing the  potential of materials at the level of the interface that make visible, alter, and construct reflexive interfaces to collections of digitized and digital materials that attend to the multitude of individuals, institutions, contexts, and ideologies that determine accessibility to materials. There are more projects that we might discuss that engage interface as method for exploring materials and more complex concerns than need be addressed about the rhetoric of the interface, but In this space I will focus on three  digital projects as interfaces for material potential—that is, in the potential to make matter material considerations in research through interface design.

HyperCities is a space for collaborative digital mapping of cultural, historical, and social dynamics. The project describes itself as “Born out of Web 2.0 social technologies, That represents a new digital media environment that links together cultures, languages, generations, and knowledge communities, mobilizing an array of technologies (from GPS-enabled cell phones to GIS mapping tools and geo-temporal databases) to pioneer a participatory, open-ended learning ecology grounded in real places and real times.” The impetus of the project is to answer the question: where are you from? Using Google Maps and Google Earth, a city becomes a digital research site for creating layered and interactive histories of place from the material input of users, connecting digital archives, maps, and stories with digitized renderings of the physical place with the goal of creating a “geo-temporal human web”.

While the concept of ecology is not unfamiliar to rhetoric, creating ecological texts that materialize the dynamics of thriving and potentially dying is less so. What is afforded by an interface constructed as participatory and open ended? How does this process or reprocess our concepts of knowledge production and circulation? Who or what become material record?

Florida State University houses a postcard archive, but what I’m looking at is the Kairos webtext on developing a digital archive for multimodal research from this archive by Michael Neal, Katherine Bridgman, and Stephen J. McElroy. They frame their project by asking “How researchers visiting this archive make meaning from within the infrastructure that they have constructed” and “How we continue to curate a digital space designed to enable growth over time at the same time that it hopes to reflect the voices and perspectives that are engaging the archive from both within and outside of the academy”. They explain that the digital and physical spaces of this archive emerge at the interstices of borders that reflect multiple and overlapping contexts in: university and community; modality and materiality; print and screen; time and geography; and positionality and hierarchy which they use as interface or access for their work. Their interest is in making visible the ways in which we always reconstruct the subjects of our research through the production of scholarship, which grows out of ongoing conversations on archival methodologies that focus on building collaborative relationships with archivists and strengthening research through constructing archives of artifacts that deserve scholarly attention. They posit that the intersections of modalities and materialities that weave through the meaning that researchers and browsers take from this postcard are acknowledged in the digital representation of the postcard in three primary ways: through digitization of the actual card, Dublin Core metadata standards, and Postcard Item Type metadata. The metadata makes matter the published words on the cards, the handwritten words on the cards, the visuals other than the front image, how the card is laid out, color, and technological and/or material features of the cards (such as nontraditional materials like wood or ways of reading like hold to the light cards).

Instead of privileging any one material aspect of the postcards, they worked to conceive of a site that is more conducive to studying the relationships between and among the modalities represented through the cards. The different metadata tags with each postcard are a way of flattening a Dewey Decimal style hierarchy for browsing nested categories of postcards in order to afford researchers of the archive to determine arrangement as the most salient point for research based on geographical location, topic of the card, or even temporal chronology when the cards are entered into the archive. Visitors who are not registered can only view content. Once the visitor becomes a registered user, they can begin adding to and editing the materials in the archive. As a “researcher,” a user can begin tagging images. As a “contributor,” users can create an exhibit as well as add and edit only the items that they have contributed to the archive. As an “Admin,” users can edit items already entered into the collection as well as the preset vocabularies that appear for archivists in pulldown menus. Finally, the “Super” user has permissions to do all of this as well as edit the general “settings” for the archive.

This project extends beyond researchers in the academy to consider materials digitized and categorized by interested persons; what can this distributed model of creating an archive do differently than one that is more tightly bound? While archives don’t come with predetermined ways of making meaning, what is the potential in constructing an archive of materials that emphasizes the spaces between materials as a material itself?

Darwin’s On the Origin of Species evolved over the course of several editions he wrote, edited, and updated during his lifetime. The first edition was approximately 150,000 words and the sixth is a much larger 190,000 words. In the changes are refinements and shifts — increasing emphasis, adding details, or even a change in the idea itself. Ben Fry, using the evolution of the six editions as guide, developed an interactive visualization of the text over time, His fascination guided by the ability to make visible the change over time in a person’s thinking. Hovering over the text, the viewer can see the removal, addition, and change in text.

What would become available in the digitization and juxtaposition of oft-cited texts? What if the evolution of ideas, of history, could materialize; how might we read texts and ourselves differently?

Composing digital texts and making use of digital humanities tools as methods for research don’t make research more mindful of material, but we can make research that does—that makes materials matter not just in our attention to them, but our attention through them, with them. In ”Humanistic Theory and Digital Scholarship”, Johanna Drucker explains that we can cast an interpretive gaze on instruments of digital technology from a humanistic perspective and we can build humanities content on their base; but rarely have we imagined computational protocols grounded in humanistic methods. She states

if we are to assert the cultural authority of the humanities in a world whose fundamental medium is digital that we demonstrate that the methods and theory of the humanities have a critical purchase on the design of platforms that embody humanistic value.

I am interested in developing interface design as matter of concern in research, not just a rhetorical attention to interfaces, but interface as material method, as intermediary, as reprocessing. I return to the Oculus Rift experience through SightLine, the simultaneous engaging and obscuring of my looking and its effect on engaging materials. What could become available to digital and digitized materials if interfaces were responsive to action, that is interacting with them? What if our interfaces mattered to us and us to them?