accidentals and substantives

vectors on a grecian urn

Web enthusiasts know that the world of two-dimensional computer graphics can be crudely subdivided into two main groups: bitmapped and vector images. Jpeg and gif fall into the former camp; swf (macromedia's shockwave flash) into the latter. The different file formats reflect different media ontologies: bitmapped images represent information about the location and value of each individual pixel, while vector images represent paths, lines, and shapes. Macromedia Flash notwithstanding, we've become largely accustomed to a bitmapped web ecology. This was not, however, a foregone conclusion, as the history of computing makes clear: Ivan Sutherland created the first interactive vector drawing application, Sketchpad, back in 1962. While the promise of vector graphics was largely eclipsed by bitmapped graphics in the decades to follow, that situation is poised to change, thanks in no small measure to the influence and standardization of the Flash format. But it's not Flash that has really caught my attention of late. It's Scalable Vector Graphics, or SVG.

My interest lies primarily with the semiotics of SVG, a relative newcomer to the computer graphics scene. Depending on how things play out in the next couple of years, SVG may become a real contender for the brass ring of vector graphics. What sets it apart from Flash's proprietary format is that it is an open standard. It is also, crucially, a text rather than binary format. Put differently, SVG is a linguistic and mathematical code--XML compliant, no less--for describing pictures. Unlike bitmapped images, SVGs are human readable and writable as well as machine readable and writable. I can, for example, code my image in my favorite text editor (Notepad), view it in my favorite browser with a plug-in (Mozilla), or open it in my favorite drawing program (Adobe Illustrator).  With the right software, I can toggle between text and image views, authoring the image in a text-editing environment one minute and drawing it in a visual-editing environment the next, hand-coding it and auto-coding it, shifting seamlessly through semiotic gears. 

Here are a couple of screenshots demonstrating SVG's native ekphrasis. The first is a circle I created in Sketsa, a vector drawing application.  The screenshot shows the circle rendered in the visual editor, but it was hand-coded in the text editor.

Here is the underlying code:

<?xml version="1.0" standalone="no"?>

<svg contentScriptType="text/ecmascript" width="400.0" xmlns:xlink="http://www.w3.org/1999/xlink"
zoomAndPan="magnify" contentStyleType="text/css" height="400.0" preserveAspectRatio="xMidYMid meet"
xmlns="http://www.w3.org/2000/svg" version="1.0">
<circle fill="blue" r="100" cx="200" cy="200" stroke="red" stroke-width="4"/>
<text x="79" y="50" style="font-size:19; fill:red; font-family:palatino; font-weight:bold">
computational ekphrasis
</text>
</svg>

Readers familiar with html, sgml, and xml will recognize the syntax and vocabulary of elements and attributes.

Most web browsers still won't render native SVG content without a third party plug-in tool, which is why I'm providing screenshots. Adobe's SVG viewer is a good stopgap measure for the time being.

(An aside about the text displayed above the image: the words "computational ekphrasis" render as character data, meaning the individual letters can be copied, searched, indexed, cut-and-pasted, etc.)

The second screenshot shows a digital photograph of a scaled-down replica of the Townley urn, which served as the inspiration for Keats's famous ode.  I first captured it as a jpeg and then exported it as an SVG. To compensate for SVG's inability to render the rich tonal values of photography, I've heavily embossed its "leaf-fringed legends," "Attic shapes," and "marbled men."

Next I plan to experiment more with SVG's text element and attributes, which could, I think, find application in humanities computing--specifically in text encoding. If my proposal is accepted, I'll be presenting some of these ideas at the Elective Affinities conference. (CJ and some of the other herders: this would be a great venue for you, too. The deadline for submissions was officially 1 Oct., but I wouldn't be surprised if they extended it; might be worth an inquiry. I would have advertised it here sooner, but didn't submit anything myself until the last minute).

And a plug for Matt's MLA 2002 paper, "Vector Futures," which looks primarily at Flash, but SVG gets a mention too (available as a pdf. Look under "selected writing" on his blog.) Highly recommended.

dirt graffiti

From NPR: "A British street artist known as Moose creates graffiti by cleaning dirt from sidewalks and tunnels."

There's something almost steganographic about the process--as though the visual language were already there, just concealed beneath the impasto of urban grime. Moose's reverse graffiti reminds me of Blake working his copper plate, "melting apparent surfaces away, and displaying the infinite which was hid" (Marriage of Heaven and Hell).

word and image

One of my earliest blog posts attempted to grapple with the semiotic distinctions between ascii and bitmapped words within the context of applied Humanities Computing. I eventually reworked some of the prose, appending it to the Blake Archive DTD documentation and incorporating it into my dissertation. Here's the slightly revised version:

Like many other first-generation humanities computing projects, the WBA delivers both high-quality digital facsimiles and meticulously prepared ascii transcriptions of its primary source materials. The images enable the study of visual attributes that aren't easily accommodated by the cramped codes of ascii text, while the transcriptions offer increased legibility (Blake's serried calligraphy can strain the eyes) and, perhaps more importantly, finely structured search queries. These two editorial mandates--one pictorial, the other textual--are realized in separate documents rather than unified in one. But the desire for unification runs deep. The project team's cyclical debates over the transcriptions nearly always revolve around issues of iconicity: how far should we go in trying to render the visual codes of Blake's poetry and prose? Can we capture his horizontal and vertical spacing, his mirror-writing, his multi-directional text? One might suppose that because the transcriptions are accompaniments to the images, the former might not feel the onus of the latter. But the material strain of the WBA is paramount, and so iconic codes occupy a place of honor in both the images and the transcriptions. In the WBA, as in contemporary editorial theory at large, the word is an image. Consequently we have strained the limits of browser technology to deliver granular representations of Blake's text via course sgml-encoded ascii. Phenomenologically we want an image, but ontologically we want a text. Or, to put it somewhat differently, we want a machine-readable text and a human-readable image to coalesce in one and the same document.

Until now, our editorial reach has exceeded our technological grasp. Our transcriptions have become pictures just to the extent that mid-90s software packages (Dynaweb) and browser technologies permitted. At the time of this writing, the project has reached a crossroads where some important decisions need to be made. As we transition into xml/xsl stylesheets, the opportunities for pictorial rendition of text increase by several orders of magnitude. I have seen stylesheet-rendered ascii that can float over other text, drop the ascenders and descenders of letters, and lay claim to the screen in a breathtaking array of spatial configurations. The extent to which the transcriptions converge on the images is an issue that now devolves back onto the editors.

Since then, I've wanted to put a tangible face on some of these ideas. Today I spent a couple of hours with html/css to generate something of a proof of concept:

The screenshot shows two word collages. Although visually very similar--nearly identical, in fact, when viewed under optimal conditons--computationally they couldn't be more different from each other: the upper collage is an html/css file, the lower a gif file--the same gif that greets you each time you load Accidentals and Substantives into your browser window. The different file formats translate to different media behaviours: I can open and read the ascii collage in a text editor (NoteTab) and view the gif collage in an image editor (Photoshop). The individual letterforms of the ascii collage are subject to automated search and retrieval, cut and paste, and other kinds of character string manipulation, while those of the gif collage are not; in Matt Kirschenbaum's words, the latter are "largely opaque to the computational eyes of the machine" ("The Word as Image in an Age of Digital Reproduction" in Eloquent Images MIT P 2003).

Perceptually, the ascii collage--with its overlapping letters, colorful palette, and mixed fonts--is an image; ontologically, a text. The experiment raises larger editorial issues: to what extent might a digital archive like the WBA pictorialize its transcriptions? Abandon a letterpress model of textuality in favor of a visual model that is realized not only through photomechanical reproduction (digital facsimiles) but also typographical reproduction (xml/xslt)? Leverage the same data optically as an image and computationally as a text? I find such questions endlessly fascinating.

I've posted the two collages here. Caveat lector: The html letters look like so many windblown leaves strewn across the screen unless you view them under optimal conditions, which means in IE 6.0 or higher on a Windows OS. You also need to have Lucida Handwriting and Bookman Old Style fonts installed on your machine. (The html looks god-awful in Mozilla and Netscape. Ditto in IE if you don't have the appropriate fonts installed.) If you do visit the page, I hope you'll suspend the optical illusion of two nearly identical collages by locating the interactive hotspots: use CTRL-F to search for individual letters or words in the ascii collage, or use your mouse to select and copy some of them onto your clipboard and paste into MS Word or your favorite text editor. Then try to repeat the experiment with the inert characters of the gif collage: no dice.

image reconstitution again

Just an extension, really, of my previous post.

More links: my student Matt's image description and Emy's "reverse ekphrasis" here. ("Reverse ekphrasis" is my student Lauren's neologism. I like it.)

I mentioned that I'd first experimented along these lines at the Eastman School of Music. I gave my students the William Blake Archive's description of plate 25 of Songs of Innocence and of Experience copy Z ("Infant Joy"). My student Chanjae Park drew the following (without having ever seen the original upon which the description is based):

(C) 2001. This image may not be reproduced without the express permission of its author. Click on the image for a larger reproduction.

I've always liked Chanjae's image for its immersiveness. The blossom opens into a vast landscape that from its own internal perspective loses any semblance of a flower: stamens, pistils, leaves, petals--all the recognizable trappings of flowerness disappear. Among other things, it points up the omission of proportional information in the image description.

Why, you might ask, does the Archive include such granular descriptions of visual objects alongside its high-res facsimiles of them? Redundancy, of course, is a core tenet of information theory, but I'm afraid our reasons are far more practical and mundane. It's a problem I've written about extensively elsewhere, but suffice it to say that it's exceedingly difficult at this stage of the technological game to search image databases via images themselves, i.e., via low-level image properties such as shape, color, texture, etc. While vast R&D money is currently being earmarked for the development of such tools (see Blobworld for one example among many), we still depend primarily on linguistic metadata to do the job for us. This isn't a localized problem, but a historical and semiotic one: language has enduringly played a pivotal role in the transmission of pictorial information.

writing for artists

I've a little breathing room to blog before the final onslaught of papers, projects, and grading. I also just sent Morris Eaves, my advisor, 130 pages of dissertation material, not a single word of which came easily (which is not to say it wasn't satisfying). In any case, blogging feels like a welcome change of pace.

I've felt like an American abroad this semester while adjuncting in the Art and Visual Technology program at George Mason University, an expat enjoying the new vistas too much to feel much nostalgia for the old. I like being able to duck into the gallery to view the multimedia installations or the sculpture studio to see the half-finished busts; I like the sounds, smells, and sights of the Fine Arts building: wet paint and pixels and fumes and solvents and plaster and found objects. There's a lot of kinetic (and synaesthetic) energy in these halls.

I'm sure I'll find my way back to a literature department soon enough, but in the meantime I'm making the most of my stay in AVT. Next semester, assuming enough students register, I'll be designing and teaching a Graduate Research Methodologies course (Art and Visual Technology 600), and if that doesn't run [update: Lynne Constantine tells me it probably will], I'll have the chance to work out the kinks in my Writing for Artists course. Now that I'm past the initial learning curve, I'm looking forward to tinkering with the syllabus, overhauling some aspects of it, all while keeping the broad outlines intact. Among other things, I want to devote more attention to blogging the next time around. We experimented with it this semester--with not one but two course blogs--but it didn't assume a central role, largely because I was developing content and lesson plans from scratch. Now that I've got the spadework out of the way, I can start to concentrate on fine-tuning things. You can find a copy of the course syllabus here.

I have a GREAT bunch of students. I'd say roughly 70 percent are majoring in the digital arts--graphics and animation--while the rest generally fall into one of three categories: painting, sculpture, and photography. I've tried to bring my own interarts scholarship to bear on the structure and curriculum. We had a module on the aesthetics of information transmission, for example. Instead of having them slog through G. Thomas Tanselle or Fredson Bowers or Norbert Wiener, I assigned them or introduced them to Phillip K. Dick's "The Preserving Machine," Sol Lewitt's Wall Drawings, Eduardo Kac's Genesis, and Alvin Lucier's "I am Sitting in a Room" (the last at the suggestion of Midnight Platypus and Heart of Darkness. See the comments to this post and this one).

The combination writing/studio arts assignment that grew out of that unit has absorbed a lot of my time and attention of late. In spirit, it resembles Surrealist chain games. Have a look here and follow the links for both the assignment itself and some of my students' work. I first experimented with ekphrasis and reverse ekphrasis when I taught at the Eastman School of Music a few years ago--they were successful assignments then, but I think they worked out even better this time around. (Part of it, I think, is that as artists, they really take to visual, concrete writing. I remember reading an article a few years back speculating that our language organ is, from an evolutionary perspective, a neurological rewiring of that portion of our brains originally consecrated to vision processing. Maybe visual artists, then, have an edge when it comes to imagistic or highly descriptive language?)

Here's to winding down the semester . . . Cheers everyone!

course description

Here is my syllabus introduction to AVT 395 / Writing for Artists:

Welcome to AVT 395-–a writing class for AVT students that approaches language from a studio arts perspective. We’ll move back and forth between the writing lab and atelier over the course of the semester, treating text as both a verbal and visual medium. We won’t just write words--we’ll cut and paste them; paint them; collage them; deform, blog, compute, and code them; even eat them. Writing across media and technologies in this way goes hand in glove with writing across genres. We’ll read and in many cases produce encyclopedic entries, analytic essays, image descriptions, SMS poems, artists’ and treated books, cut-ups and codework. We’ll look at how the medium shapes the message: why you might want to text message a haiku but not an epic poem; or blog a movie review, your favorite rap lyrics—even a serial novel--but not necessarily the Freemason’s handshake or a multivolume treatise. And we won’t neglect grammar, the formal system underlying our use of language. Grammar too often gets a bad rap these days, in part because it is perceived as a straitjacket for renegade prose. But grammar, as we shall see, is as much an analytical, creative, and generative tool as it is a prescriptive tool.

An important theme of the course is the relationship between word and image. Traditionally that relationship has been expressed as a paragone, or contest, between the arts. But the word-image dynamic can be collaborative as well as adversarial. Sometimes, for example, one art survives by piggybacking off another: there are lost paintings that have descended to us in verse; and perhaps—just perhaps—spoken words in the grooves of an antique clay pot (trying to recover these archeological sounds is the task of an emergent field known as paleoacoustics).

Writing for Artists wouldn’t be complete without an excursion into the book arts. The book as material object figures prominently in the syllabus and coursework. We’ll establish a baseline definition of book and then do our best to test its tensile strain through a series of related readings and assignments.

of the bibliophagi or book-eaters

bibliophage: biblio-. Prefix. Book: bibliophile.[From Greek biblion, book.];
-phage. Suffix. One that eats: macrophage. [From Greek -phagos, eating, from phagein, to eat.]

"Of the Bibliophagi or Book-Eaters" is the self-glossing title of chapter 9 of Holbrook Jackson's The Anatomy of Bibliomania (1930), a veritable baedeker to the symptoms and varieties of what is surely the best documented form of literary madness (bibliomania has spawned a vast and eccentric body of literature over the centuries). Appropriately, I picked up my copy a few years back from Blue Whale Books, a second-hand bookstore located in the downtown mall of Charlottesville, Virginia. C'ville is arguably the Bibliomaniac capital of the academic world, home to the Rare Book School, the U of Virginia, Studies in Bibliography, the Bibliographic Society of the U of Virginia, and some of the finest bibliographic minds of the last half century, not to mention the birthplace of the Hinman collator.

I'm particularly enchanted by the bibliophagi, whose regular ingestion of pulp and ink serves as a valuable reminder that there really are viable alternatives to the Atkins diet. Books are edible, Holbrook tells us, and "to serve their purpose must be eaten: you must eat the book, you must crush it, and cut it with your teeth and swallow it."

As every parent knows, books are one of the first solid foods a child acquires a taste for. On the authority of one Rosenbach, Holbrook speculates that the principle reason first editions of Alice in Wonderland are so scarce is that they've quite literally been eaten into extinction. The essayist Anne Fadiman provides corroborating evidence:

When my son was eight months old, he devoured literature. Presented with a book, he chewed it. A bit of Henry's DNA has been permanently incorporated into the warped pages of Goodnight Moon, and the missing corners of pages 3 and 8 suggest that a bit of Goodnight Moon has been permanently incorporated into Henry. ("A Glutton for Books," Civilization December 1996.)

Bibliophagia naturally lends itself to (ahem) sound-bites. According to Fadiman, Charles Lamb was partial to books containing traces of buttered muffins. More ecumenical in taste, Elizabeth Barrett Browning preferred to sample the full range of biblio fare, nibbling here and there "like some small nimble mouse between the ribs / Of a mastodon."

Bibliophagia appeals to the analytic bibliographer in me. When Matt and I stumbled across Eat This Book in a Washington DC toystore a year ago, I greedily snatched it up. Its edible pages ("wafer paper") can be made legible with the edible-ink pen (the "FooDoodler"). According to the FDA-mandated nutritional label on the inside cover, the wafer paper, which is made out of potato starch and turnip oil, contains only four calories per serving, while the FooDoodler comes in at an astounding zero calories. (Rice cake lovers, eat your heart out.)

Edible words are a frequent theme in the book arts. John Latham's notorious Still and Chew involved a complex production cycle of chewing, regurgitating, and recontextualizing: students were first invited to gnaw away at a library copy of Clement Greenberg's Art and Culture. The resultant masticated pulp (I'm reminded of an Asian delicacy that goes by the euphemistic name of "Swallow's Nest") was further broken down in an acid bath, then bottled, labeled ("Essence of Greenberg"), and finally returned to circulation in its altered state, much to the dismay of St. Martin's librarians, who, I gather, were at a loss as to how to catalog their prandial acquisition. (The story goes that Latham lost his job over the incident).

One of my favorite finds is Books2Eat, the international edible books festival. The website includes a photogallery guaranteed to make any red-blooded bibliophage salivate. (More grist for my Writing for Artists mill . . . )

So what about electronic books? Are they destined to be the offal of the book world? My prediction is that we'll see MIT launch an Edible Media program in the next decade. Hey, they already have tangible computing and wearable computing--can edible computing really be far behind?

unnaturally speaking

Next semester, assuming Virginia's state universities are still financially solvent, I'll be teaching Writing for Artists in the Interarts Department at George Mason University. The course has been so successful that Claire MacDonald, who designed it, is now in the process of staffing multiple sections. That's where I come in. AVT 395 was offered to me as a temp job, one attractive enough to lure me out of adjunct semi-retirement (I've been working on the diss. full time for the past year). It's a fabulous course, but don't take my word for it: check out Claire's syllabus for yourself.

As an initial assignment, Claire has her students do a Burroughs cut-up. I plan on following her lead, perhaps pairing the cut-up with an Oulipian exercise to contrast the two modes of literary creation: aleatory (Burroughs in the tradition of Dadaism and Surrealism) and algorithmic (the Oulipians, self-consciously renouncing surrealist games of chance). But I've also been toying with updating the cut-up for a new generation of technology-savvy students. How might I port the exercise into the twenty-first century? What are its possible extensions or variations?

Oddly enough, I'm tempted to take a page from Scott Rettberg's interview with Jason Nelson last month. Because it was transcribed with Dragon Naturally Speaking's notoriously buggy voice recognition software, the interview reads a lot like a cut-up. When I first saw it, I was reminded of a random text that the software once generated when I left my microphone on overnight with the Dragon window open. I still don't know how to account for it: was Dragon transcribing ambient noise, purging its memory banks, or channeling the ghost in the machine? Whatever the explanation, computational or occult, I've been unable to duplicate the results.

What is abundantly clear, though, after further experimentation, is that Dragon--at least the older versions of it--interprets everything within its auditory range as human speech: the crumpling of paper, the repeated thump of a shoe or hammer, the opening or closing of a book, the drumming of fingers on a table. In Dragon's Looking Glass world, inanimate objects speak in tongues; cups, saucers, hammers, and styrofoam are endowed with the gift of speech; and the babble of a brook is literal, not metaphorical. It's personification (and heteroglossia) taken to an extreme. It strikes me that this peculiar propensity for hearing the cacophony of voices in anything capable of generating friction has potential artistic application. I think of the sounds of a painting or etching or collage coming into being: fresh paint slapped with the flick of the brush onto taut canvas, acid eating away at a metal plate, paper torn and cut and pasted. Filtered through Dragon, these sonic waves become the choral voices of the atelier.

I can imagine a two-part exercise. Part I would require students to describe the acoustics of their work space in essay form. What does the dark room sound like? The sculptor's studio? the printer's workshop? The computer lab? Is it possible to impart an idea of one's artistry by channeling the experience through a single sensory modality? Part II would consist of Dragon's phoeneticized transcription of those sounds, that space: the glossolalia of artistic expression. The assignment would be a hybrid of sorts, part surrealist or Dadaist in its incorporation of chance procedures, part ekphrastic in its synaesthetic co-mingling of visual and verbal stimuli.

Would the experiment be more interesting if there were some predictability to the results, i.e., if, for instance, analog brush strokes were consistently rendered by Dragon as fricatives, digital processes (hammering or pounding or stippling) as stops? My own thinking is no, that the quest for purposeful agency misses the point, which is to situate the transcriptions within the history of aleatory art. The idea is to embrace rather than eschew chance and indeterminacy. My touchstones are Cozens, automatic writing, and Surrealism, not Pope, Rationalism, and Raymond Queneau. (For a superb thumbnail history of chance art, see the entry by that name in the Dictionary of the History of Ideas.)

That said, a trial run did reveal some hidden purpose. Coloring a plastic plate with swift, consistent marker strokes, I obtained the following text, notable for its repetitiveness:

With the bus to distance business assistance assistance with the only if the Pentagon and I think about that the think about that did it did that it with the submitted to think that the independent and that defendant at that it with the independent independent attempt to that defendant at the event of the equipment

*********************

I may or may not end up incorporating these nascent ideas in the classroom (how do I make Dragon available to fifteen or so students without violating IP law or asking them to shell out dollars they don't have?). But as a concept, the project has led me to some interesting finds. In the process of browsing for keyphrases like "voices of objects" or some such, I stumbled across an emergent field of research of which I was wholly unaware: archeoacoustics, also known as paleoacoustics. It's a bracing reminder of why I'm a textual critic, committed to the study of the transmission of information through time and space. The idea behind paleoacoustics--that sound waves are sometimes inadvertently recorded in artifacts--is in itself disarmingly simple; it's the playback that still proves elusive. Imagine a potter of antiquity at his wheel, singing, uttering oaths perhaps, or reciting Homeric verse. Those notes, those expletives or meters, it turns out, are durable media that may have survived in the grooves of the clay pot, like sound on vinyl. The question then becomes one of retreival.

The task of paleoacoustics is to restore the tongue of Keats' unravished bride of quietness; to make that cold, silent urn speak again.

Read about paleoacoustics here, here, and here. (And on a tangentially related note, don't miss Digital Needle: A Virtual Gramaphone).