Translating Visual Ideas Into Language

Fall is here, and the Fathom office is back in full swing teaching its second semester of “Information Design: Exploration, Navigation, and Understanding.” In the spirit of the very process we teach, we took a chance this summer to do some self-reflection as instructors, iterate and refine our ideas, and update a few assignments, lectures, and activities to make the course even more engaging and informative for the students.

One of our most recent lectures involved a hands-on activity in sketching and iteration, with an emphasis on algorithmic thinking and its considerations. The instructions for the activity will probably sound familiar:

  • Students have three minutes to sketch a picture of the house they grew up in, then five minutes to write a set of instructions that another student can use to reproduce their drawing.
  • After trading instructions with another person in the class, they have three minutes to draw someone else’s house, using the instructions they received.
  • Then, they repeat the entire process. The time limits are the same, the prompt is the same; the only difference now is that the students know that they’re going to have to write instructions for the picture they sketch.

In a popular variation of this activity— commonly referred to as the “Peanut Butter and Jelly Exercise”— a group is asked to create a set of instructions for making a peanut butter and jelly sandwich, and then someone follows the instructions as literally as possible, often with hilarious results.

Photo Credit: https://mindstorms.dreschallenges.com/

Its relevance to computer science and programming is obvious, but it’s also a fun, easy, and memorable way to demonstrate effective communication, specificity in language, and the dangers of assumption, in fields ranging from construction management to philosophy. But, it stops just short of one additional important lesson, especially when teaching programming as a tool for design.

Here’s an example of the sketches from the first part of the activity we did in class:

A student first drew the image on the left (1a), then was asked to write a set of instructions (1b) to recreate that image. A different student used those instructions to produce the drawing on the right (1c). Unsurprisingly, because the student wasn’t expecting to deconstruct the first sketch into simple, reproducible steps, the left and right images look pretty different.

The second time around, the student sketched the same house (14a), but in a way that was better suited to being systematically described (14b) and reproduced (14c), and to great effect. The drawings on the left and right look much more similar this time. The crux of the lesson, though, comes when comparing the original drawings from the first and second parts of the activity.

The first drawing (1a) shows a home with a curved, corrugated roof, plants, and a courtyard with a pool. By contrast, the second drawing (14a) shows a much more technical view of the same house with no textures, few details, and only straight lines. It may be easier to write instructions for the second drawing, but that comes at the expense of the subjective qualities that make it unique and interesting.

Here’s another example.

Between the first (7a) and second (21a) round of drawings, the house loses its bricks, the driveway and bushes out front, and other details that make it personal to the artist.

This may be the most striking example— the first (8a) and second (22a) round of drawings look like they were done by two different people.

It comes as no surprise that in order to leverage the power of computation in design, we must find a way to describe visual representations of our ideas with a vocabulary that is both complete and unambiguous. The main point of this sketching activity is that people will commonly, without even realizing it, sacrifice many of the more subtle, individual, and appealing aspects of their visual ideas in order to match an existing vocabulary. Instead, we should be doing the opposite. Developing a personal language that is still complete and unambiguous, but better suited for describing our creative, individual ideas, is a big step towards using programming as a tool to push design further, rather than constrain it.

Related posts
Lights, Cameras, Action!
MIT 4.s50: Information Design

Founded in 2010 by Ben Fry, Fathom Information Design works with clients to understand complex data through interactive tools and software for mobile devices, the web, and large format installations. Out of its studio in Boston, Fathom partners with Fortune 500s and non-profit organizations across sectors, including health care, education, financial services, media, technology, and consumer products.

How can we help? hello@fathom.info.