Now the Gimmicks

Now that I've told you about my story, here is how I want to tell my story.

I want to make a digital comic book incorporating motion and sound. I want to take advantage of every method to enhance immersion and turn my story into a personal experience.

It will still be sequential art. You will turn pages and recieve the story frame by frame. However, I will encourage the audience to follow their eyes with their mouse cursor because they will be able to trigger events on the page and visuals and audio will evolve as they progress through the story in ways that are slightly beyond the traditional frame-by-frame.

So before I get into specifics, let me explain the reasoning behind this. I don't know where this idea first came from. It just popped into my head when I was talking to a friend about capstones -- "I could try combining animation and comics" -- and I casually mentioned it to him. He told me it was a great idea that I really needed to hang onto, which kind of surprised me, so I thought about it a whole bunch more.

I started thinking about the strengths of comics and film, and found that they are wonderfully opposed. In my opinion, comics have their strength in pacing and dialogue. The reader can dwell on any frame of artwork or any line of text as long as they feel like, and absorb it as much as they need to before moving on. This allows them to fully appreciate a single piece of art which may introduce a setting or character or they can ponder the meaning of a single line of dialogue that would be gone in seconds if recited on film. I have also always personally found that dialogue is more powerful when read off a page and recited in your head, rather than heard from an actor. I think comics find their weakness in action. So many comics are so action oriented, but all we ever get to see is heroes and villains frozen in dynamic poses.

This is where film thrives. We get to see the motion and hear the sounds associated. Motion and sound can really make us feel the woosh of a flying object or the jar of an impact. When something's moving fast, we can really see how fast it moves. This makes a huge difference in the experience of a story.

Unfortunately, film does not allow us to experience a story at our own pace. Stories that deserve several hours are crammed into 2 hours. When an actor delivers a line, we don't have time to think about it. Something else is immediately happening. When we discover a new person or place, we don't get to just stop and stare and absorb the visual. We get a quick pan or zoom and then we're ushered along to something else. Plus, actors are only as good as they actually are, where a character on a page is as convincing as we can imagine them.

I attended Scott McCloud's presentation a while ago, and he gave me another angle. He criticized one existing attempt at a comic incorporating motion/sound. He called it a "self-presenting" comic. The problem with it was, it kept grabbing control away from the audience and then giving it back. It couldn't figure out if it was presenting itself or if the audience was exploring it by their own will.

So I want to take the best of both worlds. I have thought of several do's and don'ts and alot of possibilities and their implications for a story.

First, and most obvious, I will make use of minimal, looping background music. Music can set a mood/atmosphere, tell a story by itself, and generally increase immersion. Music is especially powerful when it's taken away from a story. When you have music, and then suddenly you don't have music, you know something important is happening. I want to take full advantage of this powerful tool of storytelling. Of course, I will give the audience as much control as possible, even over music. As the audience moves their mouse from point A to point B on the page, the music might evolve as the scene progresses. Layers can be added to or taken away from a musical loop as cued by the mouse cursor, it can be sped up/slowed down, or it can even be moved from speaker to speaker for spatial effects. There can even be buttons on the page relating to your typical stereo controls.

Next, I'm going to make use of alot of animated GIFs. This is where most of the motion in the comic will come into play. I'm especially imagining alot of looping GIFs. Imagine a picture looking up at a superhero standing at the top of a building with his cape flapping in the wind, or grass swaying in a field. I understand that this may be distracting and draw your eye to a frame you haven't reached yet, so most likely these looping animations will be triggered by mouse-over and halt when the mouse is removed. This is where following your eyes with the mouse cursor becomes very important.

Even more so with the next example. Imagine being able to trigger events on a page with motions or clicks of your mouse. Imagine that the final frame on a page depicts the protagonist's hand on a doorknob. Instead of turning the page to find out what's behind it, you click on the doorknob and see the door open to reveal what's behind. More immersion. More personal experience for the audience.

I have noticed that I always have a tendency to glance at the bottom of a page before I have actually gotten there. Or I just can't resist reading the very last sentence of a book before I'm even halfway through. I HATE having the ability to do that. It's so difficult to restrain my own eyes from that split second glance which can ruin a surprise. An animated GIF triggered by mouse-over can completely solve this problem.

You're reading a page in a comic book. It's a quiet scene in a restaurant, with two characters engaged in casual conversation. Looks pretty calm all the way down the page. You're following your eyes with the mouse cursor as you read and appreciate the artwork. Looping gifs show your common cafe activity with waitresses walking around and people entering and exiting. You move on to another frame about halfway down the page. Another looping gif much in the same style as those before it. A second or two into this frame, a character walking by in the background pulls out a gun and shoots somebody. Suddenly, everything beyond this point on the page transforms to reveal what's really happening. It catches you completely by surprise. The sound of the gunshot even makes you jump. Totally possible.

I wouldn't call this interactive. The audience has no control over the content of the story (though this could be done, it's not what I want to do). Instead, the audience has control over their experience of the story.

Also imagine how fun it would be to be presented with a 3d sculpture of an important object or character in the story, and be able to interactively rotate it on the digital comic page. Look at one of those superhero comics where they profile a character. They'll sometimes just give you a picture of the guy on a blank white background with a bunch of information about them. Same thing, except you get to move him around and look at him from any angle, zoom in/out, etc.

Finally, here's an idea that I cant claim. A friend offered this to me when I was talking to him about my idea after the Scott McCloud thing. I call it the zFrame. What if you had a static frame on a page, the same as any other. Except you can trigger this frame to become a free-floating camera in a 3d scene. You're reading down the page and you come to this frame. You click on it, and suddenly you can move around inside of it and explore it in full 3d. Imagine the details you could hide in such a scene for an audience to discover? I'm not dead set on any practical storytelling use for this device yet, but I'm sure it's there. I just haven't thought of it yet.

How am I going to make all of this happen? Well first I need help with writing the story, and I have very limited experience in sound production. This is also going to involve alot of scripting and a couple of plug-in applications which will require programming. I need help, and I need it from creative people -- people who can think sideways!

Otherwise, I will be creating all of my images and most of my animation with ZBrush. I would not be attempting this program at all without certain features that just became available in ZBrush 3. First, it's incredibly easy to create high-detail 3d images and render them quickly. Second, ZBrush 3 introduced something called transpose. Transpose allows me to fully pose and deform any 3d character without any rigging or preparation. Third, ZBrush 3 even introduced layers in 3d. You know how layers work in photoshop? It's the same thing as applied to 3d sculpture. This means I can make a base model of any character and quickly modify him and pose him for any scene and create high-quality 3d imagery frame by frame with very little effort.

Also, I discovered an application called Neobook, which describes itself as a tool for creating multimedia artwork, tools, and applications. With Neobook, I can arrange images, videos, and sounds on a digital page along with scripting and plug-ins written in several programming languages. It's a match made in heaven. It even exports your finished work as a stand-alone .exe file, so anybody who wants to read my digital comic will need nothing to do so. They just download and run the .exe file.

So there's my idea. I've put alot of thought into it, but I think I can push it even farther. However, I don't think I can do it without help. I need alot of input and discussion if I'm going to make this idea into everything it can be.

I also want to ask something of everybody who read this post. I am really trusting everybody in this class by openly discussing all of these ideas. I'm aiming to do some things that I think nobody has ever seen before. Please be careful about taking these ideas outside of the class, if you do so at all. People like me get taken advantage of all the time by others with a little ambition and alot more resources.

I am really looking forward to comments!