A manifesto for a new visual and procedural language

Work in progress. By Geert Nellen & Team 8. Made during Hackastory, 02/06/2015.

Introduction
What started as an extension for computer games, is intriguing more and more filmmakers all over the world. The promise of VR for cinema is grand: A new and better way of experiencing story. Interactivity and non-linearity are new techniques to cinema that create the opportunity to further immersing the user in a story, and increasing the story’s emotional impact.

But cinema in virtual reality is a completely new approach to filmmaking. Traditional filmmaking will change a lot if filmmakers want to make cinema for virtual reality. Writing treatments and script writing will change drastically when in service of a Virtual Reality experience. The role of director will change more to one of a designer.

Virtual reality is not for every film. Some films will be improved by being experienced in a VR device, others won’t. We think traditional film on a movie screen will co-exist with movies experienced on a traditional movie or television screen. Specific tools and skills, new to filmmaking, are needed for filmmaking in VR, but every director can make the shift if they want to.

We believe there is a lot of potential. Potential for much greater immersion in the space the protagonist is in, making the user believe to be physically present in a non-physical world. A potential of being able to experience the journey of the protagonist, improving emotional impact of a story. A potential for the user to be able to change perspective and put itself in the shoes of another individual, possibly diminishing several differences between people.

How exactly should we revise cinema and filmmaking for Virtual Reality?

Virtual Reality for Cinema
There are multiple ways of film making for Virtual Reality. Like traditional film, there are live action films, computer generated (CG) films, animation films or a combination of live action and CG / animation. All of these are still possible in VR, but some will have advantages over others. Mostly, it is easier to incorporate interactive and non-linear elements in CG films. Every film is different, so over time, technical and creative solutions to incorporate interactivity in live action films might be developed as well.

This manifesto is a way we propose filmmaking could change, hopefully giving inspiration and guidance to create well designed cinematic experiences for VR. A lot of inspiration comes from game design, level design for games, stage design, and interaction design. We believe all film makers have the means to make a film in VR.

The Function of Traditional Visual Language and a New Language
With more than a century of traditional filmmaking history, the ‘visual language’ in film has evolved to a common language. All people grow up with this language by watching film and television, creating a visual literacy amongst the public. Rules can differ amongst different cultures, but the overall rules are similar and can be used by filmmakers to help tell the story, transfer a premise, or evoke emotions.

The visual language in film has rules for framing (creating compositions to evoke emotions or shift the focus of the viewer to a point of interest), cutting (in time and space) and editing (sequencing shots & scenes). We all understand when a character exits a room through a door, and in the next shot we see a new room with a character entering through a door, it’s the same door, and just the next room. We all feel that a character has power over another character when shot from a below.

Not all of these elements transfer well to VR, some cannot be transferred at all. That’s why we propose a new language for cinema in VR.

The User and its Role
The user can be treated as a persona in the narrative. The user has to be acknowledged by other characters, or have a clear physical presence in the space of the story.

It could be interesting for users to be able to roleplay, which could increase immersion and adds a new incentive for users to engage with a film. It is unclear at this point if this is necessary for a good VR story experience, but we would love to see experiments

The Visual and Procedural Language for VR

The rest of the manifesto is work in progress. The manifesto will incorporate dozens of rules and guidelines about shifting the focus point of the user, using audio cues, game mechanics, subtle interactions, interfacing, composition, transitions, pacing and improving immersion in general.