VERTICAL REALITY
// A project by Jay-Cee for Cross-TIC //
Research Blog - June 2022
Artist
Name: Jay-Cee Klees
Animated Virtual Reality Artist
Broedplaats coordinator
Name: Kees De Groot
Company: PLANETART
Cross-TIC
Research Blog - 2022
Artist
Name: Jay-Cee Klees
Animated Virtual Reality Artist
Broedplaats coordinator
Name: Kees De Groot
Company: PLANETART
Cross-TIC
Concept
The initial concept I came up with was a vertical experience, in which the user is suspended in the air. Using their arms to move up or down. Additional features would include a fan that blows wind based on the speed of the user for immersion and a screen for guests to have an idea of what the user can see.
Brainstorm
Showing this concept around the breeding ground WARP made for some good brainstorm ideas;
-
Making the interaction more like a game where you shoot enemies
-
Progress unlocks layers to ascend higher
-
Thematic it could include multiverse jumps, makes it possible to change scenery every layer
-
Visually add particles similarly to hyperjumps
-
Have the user wear a sub pack (wearable sub woofer)
-
Switch between different worlds / art styles
-
Ant farm world
-
Music changes depending on altitude
-
Have interactive world where you can activate different elements (start from minimal to more filled with content)
Have the music also react to each addition -
Pharmacy theme (room is themed after pharmacy)
You take medication and the VR world gets crazier
The VR headset is disguised as an 'eye test'
Cool LED pharmacy animations
Introduction
This blog is written by Jay-Cee, an artist that specialises himself in VR art, animation and motion graphics.
https://www.diejaycee.com/
Virtual reality brings a unique way of experiencing and creating digital art to the table. On this website, said the topic would be explored for PLANETART to inspire artists about the possibilities of creating virtual art in verticle space for crosstic.
PLANETART (founded in 1995) is an artists collective that organises projects, events and festivals in the field of art and technology, mass media, popular culture, DIY, music, experiment and activism.
Since 2004, PLANETART organises the annual GOGBOT festival in the centre of Enschede, and since 2014 TEC Art in Rotterdam. PLANETART is co-initiator of the development of Enschede's station square into an urban playground, from WARP Technopolis to SPACEBAR.
PLANETART (founded in 1995) is an artists collective that organises projects, events and festivals in the field of art and technology, mass media, popular culture, DIY, music, experiment and activism.
Since 2004, PLANETART organises the annual GOGBOT festival in the centre of Enschede, and since 2014 TEC Art in Rotterdam. PLANETART is co-initiator of the development of Enschede's station square into an urban playground, from WARP Technopolis to SPACEBAR.
PLANETART (founded in 1995) is an artists collective that organises projects, events and festivals in the field of art and technology, mass media, popular culture, DIY, music, experiment and activism.
Since 2004, PLANETART organises the annual GOGBOT festival in the centre of Enschede, and since 2014 TEC Art in Rotterdam. PLANETART is co-initiator of the development of Enschede's station square into an urban playground, from WARP Technopolis to SPACEBAR.
In Cross-TIC, artists and culture makers who live and work in Twente are selected for various cross-media development processes. They can participate in research, productions, intensive development processes and collective activities.
Cross-TIC will be performed from 2021 to 2024 by nine core partners from Twente (Library Almelo & Theater Hof 88, Kunsten op Straat, Oyfo, PLANETART, Schouwburg Hengelo, Theater Sonnevanck, Stichting Oldenzaalse Musea, Tetem and Concordia (secretary)) and the cultural partners and training courses in Twente. The project is supported by the Province of Overijssel and the 14 municipalities that together form the Twente Culture Region.
Research Plan
The project
With Project Vertical Reality I want to experiment with interaction between Virtual reality, music and the feeling of taking off.
A multi-layer experience that is experienced vertically. This can be exhibited in museums, festivals, etc.
I made a VR experience for GOGBOT 2021 where you can walk around freely and discover things, SYNTHCITY. Now I would like to explore another direction in VR, up. As an animator, world builder and storyteller I want to take the user into an experience where 'ascend' is the main theme.
Maybe that's taking an elevator, floating up from the depths of the sea, or being attached to a huge balloon, maybe all of them. One possible way to do this is, for example, a harness that the user can put on, along with a VR headset. The user is lifted off the ground. Now suspended in the air, the user can move their arms to determine the speed at which the user ascends and virtually moves through the experience.
My expertise lies in creating and animating digital images and worlds in VR.
In addition, this project can use people who have experience with game engines and interaction, set design for spectators, (what does the playing field look like from the outside?) and a music producer for music as a lead.
The project
With Project Vertical Reality I want to experiment with interaction between Virtual reality, music and the feeling of taking off.
A multi-layer experience that is experienced vertically. This can be exhibited in museums, festivals, etc.
I made a VR experience for GOGBOT 2021 where you can walk around freely and discover things, SYNTHCITY. Now I would like to explore another direction in VR, up. As an animator, world builder and storyteller I want to take the user into an experience where 'ascend' is the main theme.
Maybe that's taking an elevator, floating up from the depths of the sea, or being attached to a huge balloon, maybe all of them. One possible way to do this is, for example, a harness that the user can put on, along with a VR headset. The user is lifted off the ground. Now suspended in the air, the user can move their arms to determine the speed at which the user ascends and virtually moves through the experience.
My expertise lies in creating and animating digital images and worlds in VR.
In addition, this project can use people who have experience with game engines and interaction, set design for spectators, (what does the playing field look like from the outside?) and a music producer for music as a lead.
The project
With Project Vertical Reality I want to experiment with interaction between Virtual reality, music and the feeling of taking off.
A multi-layer experience that is experienced vertically. This can be exhibited in museums, festivals, etc.
I made a VR experience for GOGBOT 2021 where you can walk around freely and discover things, SYNTHCITY. Now I would like to explore another direction in VR, up. As an animator, world builder and storyteller I want to take the user into an experience where 'ascend' is the main theme.
Maybe that's taking an elevator, floating up from the depths of the sea, or being attached to a huge balloon, maybe all of them. One possible way to do this is, for example, a harness that the user can put on, along with a VR headset. The user is lifted off the ground. Now suspended in the air, the user can move their arms to determine the speed at which the user ascends and virtually moves through the experience.
My expertise lies in creating and animating digital images and worlds in VR.
In addition, this project can use people who have experience with game engines and interaction, set design for spectators, (what does the playing field look like from the outside?) and a music producer for music as a lead.
Ideally, for this reseach, I would like to create a demo-video that would get the concept and idea across of the project, this would include everything as a starting point for a potential and a website where my findings are written down (this website).
Approach
Approach
Approach
This research program will consist of defining 6 categories. Starting off with defining the concept, and exploring different features by brainstorming. After that, I will explore the look and feel and art style of the project by finding inspiration online and creating a mood board. Next up, I'll dive into the technologies and make a prototype that can be used with VR using game enines and other sofware.
Concept
Art Style
Technologies
Story Mapping
Conclusion
Art Style
To establish an art style, one of several professional methods to save and organise creative ideas during a project is using an inspiration wall. An inspiration wall is a way to save and organize creative ideas during a project, or even permanently, in order to have access to them very quickly and let ideas ‘simmer’ for a while. I used Pinterest to gather inspiration and assembled 3 inspiration boards, all with their own theme.
https://nl.pinterest.com/AppleflapJC/vertical-reality-visual-research/
The first style focuses on black and white MC Escher-type structures. Since the boundaries between up and down are being blurred in these illustrations, this could work very well for a vertical-oriented experience.
The second style focuses on underwater themes and the deep sea. This style would allow for some ginormous creatures to explore. The deep sea is also a fantastic inspirational source and a great setting for all kinds of colourful glowing creatures.
With the third board, I found art that resembles tall architecture, either inside or outside these themes would lend for interesting visuals in which we can showcase whole communities or lifestyles within the different apartments or potential rooms.
Almost like a doll house, this style would lend itself to close inspection to see what people inside the living areas are doing.
Conclusion
To conclude this section, I have made a separate artwork / VR sketch inspired by different images from the pinterest boards. I took aspects of every idea and made an original work. The work is themed after a giant dragon that emerged out of a lab, divided into three sections. I have used the shape language of the architecture and swirly patterns of the black and white boards to create the first section, which is the inside of the lab.
I used the concept of a giant animal in the second part, in where the dragon escapes from the building. A fog could be added to this scene to make it feel even more massive.
The last scene is the inside of the dragon, as if you would be swallowed, you could look at a world with swimming bacteria, cells and microbes. Videos from Kurzgesagt about the immune system have always inspired me.
Technologies
There are different ways to develop a VR experience. Previously I have created a VR experience for GOGBOT using Quill and Meta's build in software Quill theatre, this would allow for a fully VR world you can explore by walking around with animation and sound. For this project, it would be interesting to look at video game engines, these are programs that run video games. They are a great choice for experiences since they allow for a lot of interaction. This chapter will include a few hardware and software options.
Hardware
Meta Quest 2
Meta Quest 2 (formerly known as the Oculus Quest 2)
This VR headset is the device I own myself and my main tool in creating virtual reality artwork.
'Meta Quest 2 (initially sold as Oculus Quest 2) is a virtual reality (VR) headset developed by Meta Platforms (formerly Facebook, Inc.). (...)
As with its predecessor, the Oculus Quest, the Quest 2 can run as either a standalone headset with an internal, Android-based operating system, or with Oculus Rift-compatible VR software running on a desktop computer. It is a refresh of the original Oculus Quest with a similar design, but with a lighter weight, updated internal specifications, a display with a higher refresh rate and per-eye resolution, and updated Oculus Touch controllers with improved battery life.'
Source: Wikipedia
The Meta Quest 2 can run VR experiences without being connected to a computer, but with a mobile chipset, it can't run power-intensive games/experiences. However, this headset allows for a connection to a PC, wired or wirelessly, which means that the PC can handle more of that power. Another thing to keep in mind with this headset is that due to the battery and internals of a standalone headset, it might be heavier than a PC-only headset, which can make extended sessions uncomfortable.
It also allows for 'hand-tracking' in which the inhouse cameras recognize the shape of your hands, and making them appear in VR.
Meta Quest Pro
'The Meta Quest Pro is a mixed reality (MR) headset developed by Reality Labs, a division of Meta Platforms (formerly Facebook, Inc.). (...) It is a high-end headset designed for mixed reality and virtual reality applications, targeting business and enthusiast users.'
Source: Wikipedia
The newest device from Meta is similar to the Meta Quest 2 in a lot of ways. However, this one is even more powerful and has the ability to merge Virtual Reality with real life, an Augmented reality/ mixed reality approach. This could allow for the user to see bystanders within the VR space or have the visual elements be introduced into the real world. A similar device to this would be the Microsoft HoloLens.
Software
Unity
'Unity is a cross-platform game engine developed by Unity Technologies. (...) The engine has since been gradually extended to support a variety of desktop, mobile, console and virtual reality platforms. It is particularly popular for iOS and Android mobile game development, is considered easy to use for beginner developers, and is popular for indie game development.'
Source: Wikipedia
'Unity has a huge community of both amateur and professional developers. Its detailed documentation is easy to understand. This makes it far more accessible than other game development engines, giving you the chance to learn as you build your games.'
Source: https://www.makeuseof.com/
Unreal Engine
'Unreal Engine (UE) is a 3D computer graphics game engine developed by Epic Games, (...). Initially developed for PC first-person shooters, it has since been used in a variety of genres of games and has seen adoption by other industries, most notably the film and television industry. Written in C++, the Unreal Engine features a high degree of portability, supporting a wide range of desktop, mobile, console, and virtual reality platforms.'
Source: Wikipedia
'High-fidelity graphics has always been one of Unreal’s main selling points. While it may be harder to optimize than engines like Unity, you can achieve truly stunning results in Unreal if you have a machine with enough power. This is perfect for immersive experiences and VR games.'
Source: https://www.makeuseof.com/
Saxion XR-lab
A jet-pack controller as a possible way to interact with the experience.
Saxion XR-lab team of developers working on their latest project.
Next to desk research, I met up with an XR developer at Saxion XR-lab.
Saxion XR lab is hosted by Saxion university, a non-profit lab in where students and partners can work on projects with state-of-the-art technology within Extended Reality, game technology, storytelling and computer graphics. During my time as a student in Saxion I also worked on a few projects for the XR-lab.
I met up with Bram Scholten. His expertise as a developer in XR-projects helped me with the approach of such a project as this.
A couple of ideas came from this session:
-
Unity and Unreal are mostly used for XR projects
-
Unity works well with simple art-styles
-
Unreal has easy coding
-
Vertical VR movement should be better and less nausciating
-
Using a parachute harnass to suspend the user might be a good idea
-
If the user doesnt move, a PC experience might be ideal and allows for more ways to imput movements
-
Jet pack chair could also be a way of interaction
-
Creating physical buttons or custom controllers are possible with a PC experience
Borculo project
One of the conclusions that came from this meeting, was that little physical additions could really enhance the overall experience. For instance, the XR-lab worked on a project to represent a typhoon that went through the town of Borculo. They used fans and a shaking floor to enhance the experience.
https://www.gld.nl/nieuws/7360340/zo-komt-de-borculose-cycloon-van-1925-weer-tot-leven
Maak verwoestende cycloon Borculo na bijna 100 jaar levensecht mee: ‘Je plaatst jezelf in een andere tijd en op een andere plek’ - DeStentor
Project Scope
According to Bram, the realization of a project like Vertical Reality, with basic interaction and three worlds to travel through, would need a clear baseline to kick-start it, and it would roughly take half a year to complete a functional prototype. On top of that, you would need a team with two engineers; one focuses on the contraption where the user would hang, and the other focuses on the technicalities within the virtual game engine. And of course, an artist would create the content. This would be me.
So for this research, no prototype would be made, but I would focus on creating a small video/presentation of the concept that was researched.
Story / Mapping
Focussing on the virtual world and its contents, deciding on a general story and theme for this research project, I had a session with playwright Michiel Bijmans at Theater Sonnevanck, Enschede. We used the mapping technique to touch and remark on different aspects and possibilities with the concept of the experience and the story within and focus upon relevant topics for this research.
Focussing on the virtual world and its contents, deciding on a general story and theme for this research project, I had a session with playwright Michiel Bijmans at Theater Sonnevanck, Enschede. We used the mapping technique to touch and remark on different aspects and possibilities with the concept of the experience and the story within and focus upon relevant topics for this research.
Jay-Cee and Michiel use the wall behind them to illustrate the mapping process
Concept in relation to story
For this research, we shortly discussed three important aspects of the concept.
-
Which physical experience do I want to evoke?
-
What is the target audience?
-
What is the digital world, and which story structure does it have?
We took the third point as the focus for the session, but since everything is connected, the other points were acknowledged during the session.
Story Mapping
As a starting point, we took the short story ideas I already had earlier on in this blog. When you ascend within a skyscraper, at the top of the skyscraper you find a dragon's egg, which emerges through certain events. Next up you travel alongside the dragon upwards, that's towering over the city. Finally, you explore the insides of the dragon after it swallows you. We talked about artstyle possibilities and target audiences, in which I, for this project, would lean into a more kid friendly artstyle that also could be enjoyed by young adults. As an inspiration source, take a look at Cartoon Network's Adventure Time, a cartoon series which looks cute on a surface level, but tackles some complex themes and can catch you off guard.
Point 0
Point 0
Point 0
Treating a story as a linear experience with different story points in it, we took a moment to think about the very beginning of the story, Point 0, what does the user experience before putting on the headset? What kind of introduction does the user get? What will the user see? We landed on a screen in which viewers could see what the user was going through, but outside of that minimal information is given. There are three levels and short stories to experience that loop around, point 0 let the user choose which story they can start off with after being strapped within the installation. We also thought about immersion, in that when the user is mounted into the installation, the amount of immersion would be changed based on the level that the user chooses. For instance, the headphones would isolate the environment once the user gets into the belly of the dragon.
A promotional image for the series finale of “Adventure Time” on Cartoon Network.
(Cartoon Network)
Breath of the Wild is an open-world videogame by Nintendo in which the player has the freedom to explore as much or as little as they want. After the tutorial level, they have the option to go straight for the final boss.
Main character
Main character
Who is the person that experiences the story? We mapped the possibilities and landed on not creating a specific protagonist, in this experience I want the user to feel like they are the protagonist. The choices you make as the user makes you experience the consequences. You are in control of anything good or bad that can happen.
This begs the question, how is the user going to find out how to control the experience and move vertically? There should be hints within the world (a scientist doing the motion to move up) that should give the user an incentive to move through the world. If the user doesn't pick up on this, they won't experience the full thing, and thats okay. This sandbox-like freedom, is something I want within my experience and is what I like within open-world games such as Breath of the Wild.
'Linear' Sandbox Story
Taking the approach from Breath of the Wild, I want the user to have the freedom to progress within the story or not. I recognize that not every human is capable to experience VR for the same amount of time. This way everyone who is able to can enjoy the experience at their pace. And on top of that, if someone missed a cool event within the experience, and hear their friends talk about it, they might want to try that again out of curiosity.
The experience itself will revolve around three worlds, and they last one will loop back to the first. Those three events will be a linear experience (meaning there won't be multiple endings and also there wont be any story branches), but whatever the user decides to do in between makes it for more a sandbox approach. This is why I wont go for a story in where the user need to do something, like saving the world or something else, they only will do go further if they want or are curious.
Keiken exhibition at Tetem
For this research, I visited an art exhibition at Tetem in Enschede. The exhibition is called Morphogenic Angels and Pink Moonrock Cries. It's an interactive exhibition with three distinct artworks, all presented in a themed room that really immerses you in the experience. During my time I had some interesting findings and insights which I consider for the development of this project.
Morphogenic Angels and
Pink Moonrock Cries
'In the exhibition ‘Morphogenic Angels and Pink Moonrock Cries’, Keiken collective simulates new structures of existence and explores how society’s imposed worlds of experience governs the way we feel, think and perceive. (...) Keiken is an artist collective, co-founded by Tanya Cruz, Hana Omori and Isabel Ramos. Keiken means “experience” in Japanese; the lived experience being at the core of their practice. They are collaboratively building and imagining a metaverse to simulate new structures and ways of existing and to test-drive possible futures' -Tetem
The exhibition consisted of a video that was projected on a screen, a lounging chair with a tactile belly experience and a full-body VR experience.
Jay-Cee trying out the VR experience at Tetem
Exhibition room
One particular thing about this exhibition that I noticed immediately, was the attention to detail within the interior. The floor was covered in these shiny stones, later I found out it was fish tank gravel. And I was told that most people pick it up or touch it, because of the unique sensation of walking on this. It didn't feel quite like sand or gravel. It almost has a glass-like quality to it.
Next to the flooring, the furniture seemed to be picked by color and shape as well. All of the chairs were in white or pink colors and they all had round or flowing shapes to them. Meanwhile, the room was filled with pink light.
With Michiel, I shortly talked about Point 0, this experience showed me how much you can already get immersed even before putting on the VR goggles.
VR-experience
The VR experience had you go through quite a set-up process. Trackers were attached to my feet, hands, and stomach. This allowed me to have full control of my virtual avatar. Once I put on the headset, I could see a mesmerizing world with a mirror on one side, in which I could see my virtual avatar, mirroring my every move.
This experience also has a lot of verticality in it. If I looked up and pressed a certain button on a controller, I would move one level up, like an elevator. Every level has more things to see and when you get to the top level, a statue was revealed that stretches across all levels.
The focus of this experience was mainly on the consciousness and the connection between the user and their digital avatar. Walking around and waving at the mirror was the most interesting thing about it.
The experience was a tethered VR experience, connected to a PC. Even though it should have more power the entire experience was running at a low frame per second, which made it pretty difficult to get through without getting nauseous. This shows that it's very important that the software is optimized enough and the PC is powerful enough to run it smoothly.