Abstract Lines _edited.jpg

VERTICAL REALITY

// A project by Jay-Cee for Cross-TIC //

Research Blog - June 2022

Artist
Name: Jay-Cee Klees

Animated Virtual Reality Artist

Broedplaats coordinator
Name: Kees De Groot
Company: PLANETART

https://www.planetart.nl/

Research Blog - June 2022

Artist
Name: Jay-Cee Klees

Animated Virtual Reality Artist

Broedplaats coordinator
Name: Kees De Groot
Company: PLANETART

https://www.planetart.nl/

Concept

The initial concept I came up with was a vertical experience, in which the user is suspended in the air. Using their arms to move up or down. Additional features would include a fan that blows wind based on the speed of the user for immersion and a screen for guests to have an idea of what the user can see.

Artboard 1.png
Artboard 1 copy.png

Brainstorm

Showing this concept around the breeding ground WARP made for some good brainstorm ideas;

  • Making the interaction more like a game where you shoot enemies

  • Progress unlocks layers to ascend higher

  • Thematic it could include multiverse jumps, makes it possible to change scenery every layer 

  • Visually add particles similarly to hyperjumps

  • Have the user wear a sub pack (wearable sub woofer)

  • Switch between different worlds / art styles

  • Ant farm world

  • Music changes depending on altitude

  • Have interactive world where you can activate different elements (start from minimal to more filled with content)
    Have the music also react to each addition

  • Pharmacy theme (room is themed after pharmacy)
    You take medication and the VR world gets crazier
    The VR headset is disguised as an 'eye test'
    Cool LED pharmacy animations

Introduction

Tekengebied 10.png

This blog is written by Jay-Cee, an artist that specialises himself in VR art, animation and motion graphics.
https://www.diejaycee.com/
Virtual reality brings a unique way of experiencing and creating digital art to the table. On this website, said the topic would be explored for PLANETART to inspire artists about the possibilities of creating virtual art in verticle space for crosstic.

ezgif.com-gif-maker (1).gif

PLANETART (founded in 1995) is an artists collective that organises projects, events and festivals in the field of art and technology, mass media, popular culture, DIY, music, experiment and activism.

Since 2004, PLANETART organises the annual GOGBOT festival in the centre of Enschede, and since 2014 TEC Art in Rotterdam. PLANETART is co-initiator of the development of Enschede's station square into an urban playground, from WARP Technopolis to SPACEBAR.

PLANETART (founded in 1995) is an artists collective that organises projects, events and festivals in the field of art and technology, mass media, popular culture, DIY, music, experiment and activism.

Since 2004, PLANETART organises the annual GOGBOT festival in the centre of Enschede, and since 2014 TEC Art in Rotterdam. PLANETART is co-initiator of the development of Enschede's station square into an urban playground, from WARP Technopolis to SPACEBAR.

PLANETART (founded in 1995) is an artists collective that organises projects, events and festivals in the field of art and technology, mass media, popular culture, DIY, music, experiment and activism.

Since 2004, PLANETART organises the annual GOGBOT festival in the centre of Enschede, and since 2014 TEC Art in Rotterdam. PLANETART is co-initiator of the development of Enschede's station square into an urban playground, from WARP Technopolis to SPACEBAR.

crosstic.png

In Cross-TIC, artists and culture makers who live and work in Twente are selected for various cross-media development processes. They can participate in research, productions, intensive development processes and collective activities.

Cross-TIC will be performed from 2021 to 2024 by nine core partners from Twente (Library Almelo & Theater Hof 88, Kunsten op Straat, Oyfo, PLANETART, Schouwburg Hengelo, Theater Sonnevanck, Stichting Oldenzaalse Musea, Tetem and Concordia (secretary)) and the cultural partners and training courses in Twente. The project is supported by the Province of Overijssel and the 14 municipalities that together form the Twente Culture Region.

Research Plan

The project

With Project Vertical Reality I want to experiment with interaction between Virtual reality, music and the feeling of taking off.
A multi-layer experience that is experienced vertically. This can be exhibited in museums, festivals, etc.

I made a VR experience for GOGBOT 2021 where you can walk around freely and discover things, SYNTHCITY. Now I would like to explore another direction in VR, up. As an animator, world builder and storyteller I want to take the user into an experience where 'ascend' is the main theme.

Maybe that's taking an elevator, floating up from the depths of the sea, or being attached to a huge balloon, maybe all of them. One possible way to do this is, for example, a harness that the user can put on, along with a VR headset. The user is lifted off the ground. Now suspended in the air, the user can move their arms to determine the speed at which the user ascends and virtually moves through the experience.

My expertise lies in creating and animating digital images and worlds in VR.
In addition, this project can use people who have experience with game engines and interaction, set design for spectators, (what does the playing field look like from the outside?) and a music producer for music as a lead.

The project

With Project Vertical Reality I want to experiment with interaction between Virtual reality, music and the feeling of taking off.
A multi-layer experience that is experienced vertically. This can be exhibited in museums, festivals, etc.

I made a VR experience for GOGBOT 2021 where you can walk around freely and discover things, SYNTHCITY. Now I would like to explore another direction in VR, up. As an animator, world builder and storyteller I want to take the user into an experience where 'ascend' is the main theme.

Maybe that's taking an elevator, floating up from the depths of the sea, or being attached to a huge balloon, maybe all of them. One possible way to do this is, for example, a harness that the user can put on, along with a VR headset. The user is lifted off the ground. Now suspended in the air, the user can move their arms to determine the speed at which the user ascends and virtually moves through the experience.

My expertise lies in creating and animating digital images and worlds in VR.
In addition, this project can use people who have experience with game engines and interaction, set design for spectators, (what does the playing field look like from the outside?) and a music producer for music as a lead.

The project

With Project Vertical Reality I want to experiment with interaction between Virtual reality, music and the feeling of taking off.
A multi-layer experience that is experienced vertically. This can be exhibited in museums, festivals, etc.

I made a VR experience for GOGBOT 2021 where you can walk around freely and discover things, SYNTHCITY. Now I would like to explore another direction in VR, up. As an animator, world builder and storyteller I want to take the user into an experience where 'ascend' is the main theme.

Maybe that's taking an elevator, floating up from the depths of the sea, or being attached to a huge balloon, maybe all of them. One possible way to do this is, for example, a harness that the user can put on, along with a VR headset. The user is lifted off the ground. Now suspended in the air, the user can move their arms to determine the speed at which the user ascends and virtually moves through the experience.

My expertise lies in creating and animating digital images and worlds in VR.
In addition, this project can use people who have experience with game engines and interaction, set design for spectators, (what does the playing field look like from the outside?) and a music producer for music as a lead.

Ideally, for this reseach, I would like to create a demo-video that would get the concept and idea across of the project, this would include everything as a starting point for a potential  and a website where my findings are written down (this website).

Approach 

Approach 

Approach 

This research program will consist of defining 6 categories. Starting off with defining the concept, and exploring different features by brainstorming. After that, I will explore the look and feel and art style of the project by finding inspiration online and creating a mood board. Next up, I'll dive into the technologies and make a prototype that can be used with VR using game enines and other sofware.

Concept

Art Style

Technologies

Story Mapping

Conclusion

 
 
 

Art Style

To establish an art style, one of several professional methods to save and organise creative ideas during a project is using an inspiration wall. An inspiration wall is a way to save and organize creative ideas during a project, or even permanently, in order to have access to them very quickly and let ideas ‘simmer’ for a while. I used Pinterest to gather inspiration and assembled 3 inspiration boards, all with their own theme.

https://nl.pinterest.com/AppleflapJC/vertical-reality-visual-research/

Screenshot 2022-08-23 at 10.34.45.png

The first style focuses on black and white MC Escher-type structures. Since the boundaries between up and down are being blurred in these illustrations, this could work very well for a vertical-oriented experience.

f333f26ed708de86aa26986ea5544350.jpeg
8b2368bc3a3917b02e178ec201be5b29.jpeg
Screenshot 2022-08-23 at 10.34.56.png
428bd6c8851675c5193dd513b6d3a2d4.jpeg
10b8c6de06d4531297ef7de52954ee84.jpeg

The second style focuses on underwater themes and the deep sea. This style would allow for some ginormous creatures to explore. The deep sea is also a fantastic inspirational source and a great setting for all kinds of colourful glowing creatures.

Screenshot 2022-08-23 at 10.35.06.png

With the third board, I found art that resembles tall architecture, either inside or outside these themes would lend for interesting visuals in which we can showcase whole communities or lifestyles within the different apartments or potential rooms.

Almost like a doll house, this style would lend itself to close inspection to see what people inside the living areas are doing.

46d696bd9038dcd015e87aa9f6a002a6.jpeg
0e07e77404ba1c0514616129539d1574.jpeg
 

Conclusion

To conclude this section, I have made a separate artwork / VR sketch inspired by different images from the pinterest boards. I took aspects of every idea and made an original work. The work is themed after a giant dragon that emerged out of a lab, divided into three sections. I have used the shape language of the architecture and swirly patterns of the black and white boards to create the first section, which is the inside of the lab.

I used the concept of a giant animal in the second part, in where the dragon escapes from the building. A fog could be added to this scene to make it feel even more massive.

The last scene is the inside of the dragon, as if you would be swallowed, you could look at a world with swimming bacteria, cells and microbes. Videos from Kurzgesagt about the immune system have always inspired me.

capture00129.png
capture00131.png
capture00130.png
capture00128.png
capture00127.png

Technologies

There are different ways to develop a VR experience. Previously I have created a VR experience for GOGBOT using Quill and Meta's build in software Quill theatre, this would allow for a fully VR world you can explore by walking around with animation and sound. For this project, it would be interesting to look at video game engines, these are programs that run video games. They are a great choice for experiences since they allow for a lot of interaction. This chapter will include a few hardware and software options.

Hardware

Meta Quest 2

Meta Quest 2 (formerly known as the Oculus Quest 2)

This VR headset is the device I own myself and my main tool in creating virtual reality artwork.

'Meta Quest 2 (initially sold as Oculus Quest 2) is a virtual reality (VR) headset developed by Meta Platforms (formerly Facebook, Inc.). (...)

As with its predecessor, the Oculus Quest, the Quest 2 can run as either a standalone headset with an internal, Android-based operating system, or with Oculus Rift-compatible VR software running on a desktop computer. It is a refresh of the original Oculus Quest with a similar design, but with a lighter weight, updated internal specifications, a display with a higher refresh rate and per-eye resolution, and updated Oculus Touch controllers with improved battery life.'

Source: Wikipedia

piopng9oupf2zsept68z.png

The Meta Quest 2 can run VR experiences without being connected to a computer, but with a mobile chipset, it can't run power-intensive games/experiences.  However, this headset allows for a connection to a PC, wired or wirelessly, which means that the PC can handle more of that power. Another thing to keep in mind with this headset is that due to the battery and internals of a standalone headset, it might be heavier than a PC-only headset, which can make extended sessions uncomfortable.

It also allows for 'hand-tracking' in which the inhouse cameras recognize the shape of your hands, and making them appear in VR.

Meta-Quest-Pro-220.png

Meta Quest Pro

'The Meta Quest Pro is a mixed reality (MR) headset developed by Reality Labs, a division of Meta Platforms (formerly Facebook, Inc.). (...) It is a high-end headset designed for mixed reality and virtual reality applications, targeting business and enthusiast users.'

Source: Wikipedia

The newest device from Meta is similar to the Meta Quest 2 in a lot of ways. However, this one is even more powerful and has the ability to merge Virtual Reality with real life, an Augmented reality/ mixed reality approach. This could allow for the user to see bystanders within the VR space or have the visual elements be introduced into the real world. A similar device to this would be the Microsoft HoloLens.

Software

Unity

'Unity is a cross-platform game engine developed by Unity Technologies. (...) The engine has since been gradually extended to support a variety of desktopmobileconsole and virtual reality platforms. It is particularly popular for iOS and Android mobile game development, is considered easy to use for beginner developers, and is popular for indie game development.'

Source: Wikipedia

'Unity has a huge community of both amateur and professional developers. Its detailed documentation is easy to understand. This makes it far more accessible than other game development engines, giving you the chance to learn as you build your games.'

Source: https://www.makeuseof.com/

62e132087fe3599fdd46ecb5.png
fa29ccfe-ce2a-4087-bf60-001b38d6cf7f_image_1.png
ue-logo-stacked-unreal-engine-w-677x545-fac11de0943f.png
202108251200036000.jpg

Unreal Engine

'Unreal Engine (UE) is a 3D computer graphics game engine developed by Epic Games, (...). Initially developed for PC first-person shooters, it has since been used in a variety of genres of games and has seen adoption by other industries, most notably the film and television industry. Written in C++, the Unreal Engine features a high degree of portability, supporting a wide range of desktopmobileconsole, and virtual reality platforms.'

Source: Wikipedia

'High-fidelity graphics has always been one of Unreal’s main selling points. While it may be harder to optimize than engines like Unity, you can achieve truly stunning results in Unreal if you have a machine with enough power. This is perfect for immersive experiences and VR games.'

Source: https://www.makeuseof.com/

 

Saxion XR-lab

IMG_2834.HEIC

A jet-pack controller as a possible way to interact with the experience.

40270-12509170.jpg

Saxion XR-lab team of developers working on their latest project.

Next to desk research, I met up with an XR developer at Saxion XR-lab.

Saxion XR lab is hosted by Saxion university, a non-profit lab in where students and partners can work on projects with state-of-the-art technology within Extended Reality, game technology, storytelling and computer graphics. During my time as a student in Saxion I also worked on a few projects for the XR-lab.

I met up with Bram Scholten. His expertise as a developer in XR-projects helped me with the approach of such a project as this. 

A couple of ideas came from this session:
 

  • Unity and Unreal are mostly used for XR projects

  • Unity works well with simple art-styles

  • Unreal has easy coding

  • Vertical VR movement should be better and less nausciating

  • Using a parachute harnass to suspend the user might be a good idea

  • If the user doesnt move, a PC experience might be ideal and allows for more ways to imput movements

  • Jet pack chair could also be a way of interaction

  • Creating physical buttons or custom controllers are possible with a PC experience

 

Story / Mapping

Story / Mapping

Story / Mapping

Focussing on the virtual world and its contents, deciding on a general story and theme for this research project, I had a session with playwright Michiel Bijmans at Theater Sonnevanck, Enschede. We used the mapping technique to touch and remark on different aspects and possibilities with the concept of the experience and the story within and focus upon relevant topics for this research.

Focussing on the virtual world and its contents, deciding on a general story and theme for this research project, I had a session with playwright Michiel Bijmans at Theater Sonnevanck, Enschede. We used the mapping technique to touch and remark on different aspects and possibilities with the concept of the experience and the story within and focus upon relevant topics for this research.

IMG_2786.HEIC

Jay-Cee and Michiel use the wall behind them to illustrate the mapping process

Concept in relation to story

For this research, we shortly discussed three important aspects of the concept.

  1. Which physical experience do I want to evoke?

  2. What is the target audience?

  3. What is the digital world, and which story structure does it have?

We took the third point as the focus for the session, but since everything is connected, the other points were acknowledged during the session.

Story Mapping

As a starting point, we took the short story ideas I already had earlier on in this blog. When you ascend within a skyscraper, at the top of the skyscraper you find a dragon's egg, which emerges through certain events. Next up you travel alongside the dragon upwards, that's towering over the city. Finally, you explore the insides of the dragon after it swallows you. We talked about artstyle possibilities and target audiences, in which I, for this project, would lean into a more kid friendly artstyle that also could be enjoyed by young adults. As an inspiration source, take a look at Cartoon Network's Adventure Time, a cartoon series which looks cute on a surface level, but tackles some complex themes and can catch you off guard.

Point 0

Point 0

Point 0

Treating a story as a linear experience with different story points in it, we took a moment to think about the very beginning of the story, Point 0, what does the user experience before putting on the headset? What kind of introduction does the user get? What will the user see? We landed on a screen in which viewers could see what the user was going through, but outside of that minimal information is given. There are three levels and short stories to experience that loop around, point 0 let the user choose which story they can start off with after being strapped within the installation. We also thought about immersion, in that when the user is mounted into the installation, the amount of immersion would be changed based on the level that the user chooses. For instance, the headphones would isolate the environment once the user gets into the belly of the dragon.

_url=https___california-times-brightspot.s3.amazonaws.com_f8_0e_4f194b863d72c3e5b46c80fb6e

A promotional image for the series finale of “Adventure Time” on Cartoon Network.

(Cartoon Network)

zelda.large.jpg

Breath of the Wild is an open-world videogame by Nintendo in which the player has the freedom to explore as much or as little as they want. After the tutorial level, they have the option to go straight for the final boss.

Main character

Main character

Who is the person that experiences the story? We mapped the possibilities and landed on not creating a specific protagonist, in this experience I want the user to feel like they are the protagonist. The choices you make as the user makes you experience the consequences. You are in control of anything good or bad that can happen.
This begs the question, how is the user going to find out how to control the experience and move vertically? There should be hints within the world (a scientist doing the motion to move up) that should give the user an incentive to move through the world. If the user doesn't pick up on this, they won't experience the full thing, and thats okay. This sandbox-like freedom, is something I want within my experience and is what I like within open-world games such as Breath of the Wild.

'Linear' Sandbox Story

Taking the approach from Breath of the Wild, I want the user to have the freedom to progress within the story or not. I recognize that not every human is capable to experience VR for the same amount of time. This way everyone who is able to can enjoy the experience at their pace. And on top of that, if someone missed a cool event within the experience, and hear their friends talk about it, they might want to try that again out of curiosity.

The experience itself will revolve around three worlds, and they last one will loop back to the first. Those three events will be a linear experience (meaning there won't be multiple endings and also there wont be any story branches), but whatever the user decides to do in between makes it for more a sandbox approach. This is why I wont go for a story in where the user need to do something, like saving the world or something else, they only will do go further if they want or are curious.

story.png