Today I'd like to revisit a story I linked to last week in a morning coffee post and ask a few questions about it.
Comic books are a very visual art form, and so they translate to non-visual mediums about as well as portraiture. ShapeReader is a one-off experiment which tries to address that limitation by introducing a set of boards which are covered with textures that are used to define parts of the story.
The result is Shapereader, a system of tactile ideograms, or “tactigrams:” haptic equivalents for objects, actions, feelings, characters, and other features of any story. They’re raised shapes on wooden board, and have more in common with Chinese pictograms than with braille letters or the Roman alphabet, in that they’re textural depictions of what they represent.
Arctic Circle, a 57-page graphic novel, is the first narrative work designed with the Shapereader system. Here, anxiety is represented by a series of ridged zig-zags; arctic moss by clusters of leaf shapes; falling snow is translated into circular blobs. A walrus is a bunch of jagged concentric diamonds, and Sir Alfred Cook, one of the main characters, is a series of horizontal stripes. In designing tactigrams, Manouach aimed for simplicity and distinctiveness — the shapes had to be easy to memorize by touch.
Here's one of the 57 pages from Arctic Circle. Note: None of the textures are defined on the page; you have to check what I would call the glossary.
NY Magazine calls this a "new language", which it is. But given that no one understands what is written here without the glossary, and given that said glossary is so cumbersome that it will be too expensive to copy and distribute, this is not a language which will be widely adopted.
ShapeReader shares a problem common to languages like Japanese and Chinese which use ideograms. These languages have thousands of unique symbols which a reader has to learn before they become fluent. This makes this type of language very hard for an outsider to learn, and when combined with ShapeReader's replication and distribution issue it turns the project from a breakthrough into a dead-end.
And that's no big loss.
No, seriously, what with the latest advancements in image recognition, we are rapidly approaching the point where we will have apps which can take a digital comic and read the dialog while describing the action.
Several years back a startup released a Google Glass app which could use the Glass's camera to identify what the user was looking at. And of course there's Google Goggle, an app which can identify the text or other content in front of a smartphone's camera.
Are we at the point where this tech would work with a digital comic? Probably not, but that day is coming.
I give it five years, on the outside, before this kind of app is widely available. And that's a pessimistic view; we could see that app in as little as three years. (Heck, it's entirely possible that some PhD student has already demonstrated the basic idea and we just don't know about it yet.)
What did you think of it?