Andrew Bell's Barbarian Blog

Augmenting Esquire

andrewfb@barbariangroup.com(Andrew Bell) — Mon, 16 Nov 2009 11:02:38 -0500

We just did something we’re pretty pumped about. The December issue of Esquire is the first Augmented Reality issue of a major magazine, and we got to bring it to life. It’s something we started talking to Esquire’s editors about in early spring, and we’ve been quietly making it since this summer, so it’s pretty exciting to finally get to share it with the world.

How does it work? Two steps. First, go buy the latest issue of Esquire. Back? OK now go grab the app we wrote at http://esquire.com/ar (it works on both Mac OS X and Windows, but you do need a webcam).

Now hold the magazine up to your computer while the software is running and you’ll see all kinds of awesomeness based on the page you’ve got up. Hold up the cover and Robert Downey Jr. will leap off the page and start shouting at you. In the fashion story you’ll see Jeremy Renner dressed for all kinds of weather, and you can change the season by rotating the page. You can listen to a track from jazz musician Robert Glasper while you read the article about him, or see actress Gillian Jacobs tell a joke instead of just reading it. And if you ask her politely after midnight, she’ll even tell you a dirty joke.

If you haven’t seen it yet, Augmented Reality is the technology that makes all this possible, and it’s pretty sweet. It is showing up in a lot of different forms, but the general definition is pretty basic – just overlaying information or imagery onto the “real world” – usually images of the real world through a webcam or cell phone camera. In our case, we are using special markers printed on certain pages in the magazine, and our software can detect those markers and figure out in 3D where you’re holding the page and at what angle. Then we can overlay images in exactly the same spot you’re holding the magazine (like we do on the cover, for example) or create “mouseless” user interfaces, saying by letting you change the weather in the fashion spread as you turn the page different directions.

This is all the result of a three-way collaboration between the editorial staff of Esquire, the uber-talented designers at Psyop, and us. It was awesome getting to see first-hand all the exceptional talent that gets poured into an issue of the magazine, and collaborating with the brilliant design minds of Psyop was something we’ve wanted to do for a long time.

The whole process started when Esquire approached TBG looking for an idea that would make their December issue, also known as their Best and Brightest issue, be something really unique. We suggested Augmented Reality as a piece of technology that would fit the bill perfectly. We started to collaborate on what that might look like, considering the possibilities for an AR-ified version of magazine staples like the Funny Joke from a Beautiful Woman as well as articles specific to the issue.

Right around the same time experiments with Flash-based AR started to emerge on the internet. We knew we wanted to do something that would take AR to the next level in terms of visual sophistication, and to do that we’d need to use OpenGL to do real 3D graphics. Enter Cinder, our framework for creative coding in C++. You’ll be hearing more about Cinder in the future, but it was the key piece of technology that helped us pull all this off.

After we nailed down with Esquire the stories we’d do AR for, and the technology we wanted to use to do it, we began the collaboration with our friends at Psyop. In addition to providing their own insights on look and feel, they handled directing the video shoots and then of course the animation for several of the pieces. When it comes to visual creativity they are as good as it gets, and their talents really made this thing look next-level.

So far the press has been pretty excited as well. The Wall Street Journal previewed it a couple of weeks ago, and the editor of Esquire, David Granger was invited to talk about it on the Today Show last Tuesday.

Special thanks goes to our partners at Psyop for their beautiful design work, and to Esquire for having the balls to experiment.

Reading this description really doesn’t do it justice – you should put down the internet and go pick up the latest issue of Esquire to see it for yourself.

Formal Friday: Elevated

andrewfb@barbariangroup.com(Andrew Bell) — Fri, 10 Apr 2009 17:52:27 -0400

Kevin Chan is on a well-deserved break this week, so I’ve taken over Formal Friday posting duties.

In 1857 a building across the street from our office (presently a Staples) was equipped with the world’s first passenger elevator. Here’s us looking formal in ours.

Running out of Images

andrewfb@barbariangroup.com(Andrew Bell) — Mon, 09 Feb 2009 18:45:17 -0500

I was thinking recently, one day we might run out of new images. Let’s take the current standard for high quality images, 1080p hi def video. It’s surprising to realize that that frame contains a finite number of possible images. I thought it would be interesting to figure out just how many, so I wrote a little Python expression to do the math. The total number of pixels is 1920 horizontally x 1080 vertically = 2,073,600 pixels. There are 256 possible intensities of red, green and blue for each pixel, so that’s 256³ = 16,777,216 possible colors. To figure out how many possible images there are, we need to raise the second number to the power of the first, so 16,777,216^2,073,600 = 1.5 * 10^14,981,180 possible images. That’s a pretty big number – it’s almost fifteen million digits long. Printing it in 10 point Monaco would take over 2,700 pages of paper. Scientists estimate that there are 10⁸⁰ atoms in the observable universe – a tiny number in comparison.

However big it may be, the fact that the number is finite is a surprising thing to realize. It means that every possible image has a unique ID number. So instead of asking me, “did you see that picture of MIA performing pregnant at the Grammys”, you might ask, “did you see image number 1,394,239,...,572?” Obviously that is totally impractical and it would make you a huge nerd, but it’s interesting that you could.

We think of visual artists as generally creating things that are original, and if they’re doing their job, they are. But I think it’s also surprising to realize that in another sense, they are just exploring a fixed set of possibilities. “After all that hard work and all those revisions, we decided on image number 884,297,...,493.”

Another weird thing is that the answer to nearly any question is contained in these numbers. Who killed JFK? There is a number which is a picture of the answer. What would Michael Phelps hitting a bong wearing an orange floral sari on the moon on July 19, 2033 at 3:19pm look like? What does a McDonald’s look like flooded with water? The answer to that question in full color and high definition is contained in image number whatever. Strange but true – the things which are, the things which are not and the things which will be – they all have a number.

Most of these possible images are pretty similar to what you get if you crank up the Add Noise filter in Photoshop, and aren’t really that interesting to look at. Part of my job at The Barbarian Group is to help figure out technical ways to create images. Put another way, my job is finding efficient ways to skip over the boring numbers and arrive at the interesting ones.

Update:
Thanks to an anonymous commenter who pointed out that I misplaced a comma. Fixed.

Also, with respect to the successor of an image, let’s assume a straightforward encoding of the image that lines up the pixels row by row, and stores them as interleaved RGB channel ordering. This would make the majority of images’ lower right pixel imperceptibly bluer than its predecessor, but otherwise the two images would be identical.