SYNTHCITY

// A project by Jay-Cee at PLANETART //

Bachelor Thesis - June 2021

Student
Name: Jay-Cee Samar
Student number: 448837
Studying Creative Media & Game Technologies (CMGT) at Hogeschool Saxion University of Applied Sciences

Company Supervisor
Name: Kees De Groot
Company: PLANETART

Graduation Coach
Name: Kasper Kamperman
Institution: Hogeschool Saxion University of Applied Sciences

Introduction

This blog is written by Jay-Cee, a student at Hogeschool Saxion University of Applied Sciences, studying Creative Media & Game Technologies (CMGT) and an intern at PLANETART for this current project.
Jay-Cee is an artist that specialises himself in animation and motion graphics.
https://www.diejaycee.com/
Virtual reality brings a unique way of experiencing and creating digital art to the table. On this website, said the topic would be explored for PLANETART to inspire artists about the possibilities of creating virtual art and summarise the design of a Virtual Reality product and any relevant information.

Assignment

The project

The project consists of conducted research on creating and adding animated art to a physical space using Virtual Reality.

The final product is expected to be a functional exhibition that is visually interesting and experienced by physically walking around in a space and being able to look around and discover animated art pieces within virtual reality.

The Virtual Reality exhibition will be used to liven up a physical room at Warp Technopolis to attract visitors to experience virtual art. The product can also be shown off at festivals such as GOGBOT to attract more visitors.

The company

PLANETART (founded in 1995) is an artists collective that organises projects, events and festivals in the field of art and technology, mass media, popular culture, DIY, music, experiment and activism.

Since 2004, PLANETART organises the annual GOGBOT festival in the centre of Enschede, and since 2014 TEC Art in Rotterdam. PLANETART is co-initiator of the development of Enschede's station square into an urban playground, from WARP Technopolis to SPACEBAR. [1]

Objectives

The project

The client is looking for an art installation that can attract more visitors to WARP Technopolis and festivals such as GOGBOT. Said installation is also used to inspire artist about the possibilities of virtual art.

Products and services

This art installation will be created in the form of a virtual exhibition.
The stated exhibition should be experienced by walking around a physical space and should include visually appealing animated visuals. Additionally, the product should be experienced by using a VR headset, preferably standalone or otherwise connected to a PC.
Next to the exhibition itself, this project will include concept art and animated 3D models. Sound design will be briefly researched for this project as well.
To attract more visitors, a potential promotional campaign for this experience should be looked into.

Product goal

This product aims to attract visitors to WARP Technopolis and events organised by PLANETART, such as GOGBOT festival. Next to that, it should inspire artists about virtual art and the possibilities of creating virtual art.

Project boundaries

The realisation of the project is limited by the time constraints given by Saxion, as a creative project could have only a limited amount of iterations and test phases. Furthermore, due to a pandemic happening, there needs to be looked into the measures and the safety of a virtual experience where visitors need to use a shared headset during an exhibition.
Saxion's XR-Lab provides the necessary tools to develop the stated project, such as powerful PC's and VR sets. In addition, PLANETART provides space for project development and the eventual showcase, and connections to artists who have experience in exhibitions for consulting.
Next to this, virtual reality artists and developers possibly need to be contacted for professional consulting and feedback.

Stakeholders

PLANETART

PLANETART is one of the stakeholders in this assignment. The finished product will be displayed and showcased at WARP Technopolis and GOGBOT. They expect a creative and inspiring product that fits with the image they portray as a company and the theme of the GOGBOT festival.

Jay-Cee Samar

Jay-Cee, the writer and executor of this project, also has personal goals for this project. With this project, they want to grow as an artist and improve their skill in animation and VR painting within the Quill VR software. In addition, with this project, they want to profile themselves as a freelance Virtual reality artist and animator by creating and animating all assets for this project using Virtual Reality and Quill VR.

Saxion

Saxion, the institution the student is graduating at, requires documentation of research and development of the product and gives a limited timeframe to work with. The project also needs to fulfil the twelve design competencies of Saxion to be sufficient. Saxion can also provide students with state-of-the-art virtual and augmented reality devices, computers and such through their XR-Lab.

Approach & Timeline

Double diamond process model

Double Diamond is a process model created by Design Council, a British organisation, in 2005. The model provides a graphic representation of a design process. [...]
The model presents four main stages across two adjacent diamonds.
Each of the four stages is characterised by either convergent or divergent thinking. These stages are:

Discover –identify, research and understand the initial problem.
Define – limit and define an apparent problem to be solved.
Develop – focus on and develop a solution.
Deliver – test and evaluate, ready the concept for production and launch. [2]

Double diamond process model [2]

Scope

Since this will be a one-person project under a time limit, the end product must be limited by some amount. The scope of this project has to be determined to be a single user one-room experience. The main focus of this project is going to be on the visual side of the experience. There will be one VR setup used for the entirety of this project to avoid differences in user experience that can occur across multiple hardware.
Due to the measures a pandemic brings with them, this product can only be tested by limited people to assure safety, which means that most testing must be done using employees. The current pandemic brings with it that the full potential of this product can't be fully realised. As of now, it's advised not to bring visitors to a building. This is why it needs to be considered that this experience should be simple to set up for later use when covid measures soften.

Planning & Approach

The planning for this project is based on planning similar to the process of creating an animation. Divided amongst six parts are different methods that are used in the creative industry. To assure the quality of research, methods from the CMD-method pack are applied. Since this project is an interactive experience, it requires testing. Because of this, the testing and iterating phase has been added to the planning. Furthermore, the double diamond process model has been implemented within this planning. Finally, the planning has been visualised as a timeline.
Each phase of the double diamond, discover, define, develop and deliver, has a place within this timeline.

Introduction

Objectives

Stakeholders

Approach & Timeline

The following chapter is about the specifications of the product, the hardware and software used.

Problem definition

Problem Analysis

By looking at the current goals for the project, the two problems given are as follows: More people need to visit WARP Technopolis and GOGBOT to experience and get inspired by virtual art. A virtual reality exhibition would be a step in the right direction to solve this problem if successful. The problem behind the problem could be caused because the WARP Technopolis has unused rooms meant to host art installations. Apart from festivals, these exhibition rooms are primarily unused. The given assignment could help give WARP Technopolis more content within the building, bringing in more visitors.

If successful, the product could help the client bring in more visitors. PLANETART is an artist collective; connected artists often visit the building. Local artists can try the product, and if it delivers sufficiently, this could have an inspiring influence on the artists about virtual artworks.

The main question

The main question of this assignment is as follows:

"How to develop a VR experience to liven up a room at WARP Technopolis in the theme of the GOGBOT festival?"

Sub-questions
Sub-questions are meant to help answer the main question:

"How can the GOGBOT theme be translated to a VR experience?"
"How to create an appealing style within Quill?"
"How to connect a Quill created VR experience to a physical room?"

Planned features of the product are:

– Visually appealing virtual space
– Sound effects
– Animated assets
– Room-scale VR
– Untethered VR experience
– Simple and foolproof experience
– Easy exhibition set up in a room

Potential additions to this project are:

– Set up manual
– Moodboard
– Concept Art
– Marketing campaign (Social media posts, branding and poster)
– Showcase website
– Product presentation

The clients' problem could originate from the empty spaces and the lack of incentive for people to visit WARP Technopolis. Apart from its decoration, an exhibition in the building could strengthen the focus on creative technology.

Problem definition

360 scan

Hardware

A VR headset is needed to provide virtual reality to the user. A virtual reality headset provides a stereoscopic display, stereo sound, motion tracking sensors and motion controllers.
For this current project, the headset needs to comply with particular conditions:

Easy and safe to use
Allows for the user to walk around in a physical space using the headset (Roomscale VR)
Load custom VR experiences

Numerous headsets were available to use. Saxion provides several headsets and hardware to lend and experiment with from their XR-Lab. In addition to that, I purchased the Oculus Quest 2 when it came out, which makes this headset convenient to use since it was available to use at all times.

Oculus Quest 2

The Oculus Quest 2 is a wireless virtual reality headset that can load room-scale VR games without a cable. This is because of an integrated processor and battery. In addition, the Oculus tracking technology allows the user to have six degrees of freedom. Meaning the user can walk around and rotate in real space while the headset tracks the movement and adjust the virtual world to the position and orientation of the user. This is crucial for a virtual exhibition.
The wireless nature of the device makes it possible to create an easy to use and safe experience for the user since there is no wire for the user to get tangled in potentially or trip over, breaking the emersion.
A potential downside to this headset is the battery life of 2-3 hours. This would mean the headset needs to be charged in between uses, or the user would need to carry a battery pack if the battery of the headset happens to run low.
The resolution of the screen within the headset is 1832×1920 per eye [3]. This high resolution increases image quality and immersion due to eliminating the 'screen-door effect, a mesh-like appearance that occurs due to gaps between pixels that can cause eyestrain [4].
The Oculus Quest 2 supports a Qualcomm® Snapdragon™ XR2 chip that can handle VR games to be loaded and played on the device itself. This would mean the experience could be loaded on the headset by itself without connecting it to a desktop. Due to the fact this chip being a mobile chip, it would mean the experience needs to be optimised since these chips aren't as powerful as a high-end desktop. It is also possible to play desktop VR games and experiences using the Quest 2 by plugging in a link cable or even wirelessly using a local network [5].

Oculus Quest 2 [3]

Guardian System

The Oculus Quest 2 also provides a Guardian System.
The Guardian boundary is designed to display in-application wall and floor markers when users get near the play-area borders they defined. When the user gets too close to the edge of a boundary, a translucent mesh grid is displayed as a layer that is superimposed over the game or experience [6].

This guardian boundary should help the user walk around the physical space safely, preventing them from walking into a wall.
The boundary can be set up by pointing the controller towards the floor and drawing the shape of the border on the floor. The headset will remember the boundary shape when it recognises the room.

Setting up a boundary system is demonstrated in a published YouTube video [20].

Setting up Guardian System [20]

Hardware

Quill

Quill is the VR illustration and animation tool built to empower artists and creators, whether to create final art or as a production tool for concept creation aid.

Quill allows users to paint and animate in virtual reality on an infinitely scalable canvas - with rich colors and intuitive tools. Quill is designed to be expressive, precise and to let the artist's "hand" come through clearly - whether that's a watercolor style, pencil style, oil painting style or other.

Uploading and distributing your Quill creations to a VR audience has been made easier with the media management tool, Oculus Media Studio [7].

Quill is made to be intuitive to artists. It's expressive, efficient and comfortable to use over a long period of time.

Quill supports various animation approaches such as frame-by-frame, keyframe, anim brush and puppeteering techniques. This set of powerful tools and workflows allow artists to control the look and feel of the animation without requiring any traditional CG technical knowledge, such as rigging or curve manipulation.

WAV, MP3 files and Ambisionic sounds can be imported. These sounds can be used as stereo or spatial audio sources. Quill supports sphere, cone and frustum-based spacial audio emitters [8].

Personal goals

Mastering this software is something I strive for. I have previously used Quill, but I want to use this software within my professional arsenal as a freelancer, and I want to be known for my VR art. Creating all art and animation for this project in Quill allows me to gain experience in this field.

For these reasons, Quill has been chosen to use for creating and developing this experience.
Similar experiences can be loaded up on a Quest 2 using Quill Theater. An Oculus Media Studio profile is needed to publish a Quill project to Quill Theater.

Oculus Media Studio

Oculus Media Studio is a new media management tool for immersive creators to upload, publish, and analyze VR-first content. From this single hub, creators can easily distribute their stories directly to VR, where VR content is best enjoyed, while also enabling out-of-headset content discovery.

Visual fidelity is given a high priority in this pipeline. Accepted formats are high quality 4k+ 360 videos, 180 videos, and Quill creations that look good in VR [9].

Quill Theater

Regularly updated, the Quill Theater lets you visit high quality immersive animations from the comfort of Oculus TV, bringing you the best in immersive entertainment. These immersive animations are fully 6DOF, some including multiple viewpoints to let your move positionally within the art [10].

Vibe Aliens

An animation called Vibe Aliens has been created and optimised to work with Quill Theater to test if Quill Theater was applicable for hosting the VR experience.

Quill Theater requires spawn areas.

Spawn areas are used to determine the starting point from the user measured from the floor.

A YouTube tutorial has been followed to use spawn areas properly. [21]

Because of this, the animation has been made a static experience, where the user can watch dancing aliens on a floating rock in space.
The number of polygons has to be reduced to save resources on the Oculus Quest, which can be done with the optimisation feature within Quill.
After publishing the experience to Quill Theater, it's sharable with an oculus media link and set public after approval, which takes a few days.

Vibe Aliens:
https://www.oculus.com/experiences/media/187905889343641/465574341522018/

Quill Theater [10]

Vibe Aliens on Oculus Media Studio

Quill

The following chapter is about coming up with content for within the product, the message the art should convey and determining the look and feel of the product.

Brainstorming

The goal of this brainstorm is to come up with ideas for the content within the product.

PLANETART would like to display the VR experience at WARP-Technopolis and the GOGBOT 2021 festival.
Both these projects are heavily themed after neo-future, retro-future and technology, which inspired me to create a similar themed experience to strengthen the consistency of the building and the festival.
For GOGBOT 2021, a brainstorm has been held about the theme, 'INFOCALYPSE NOW – Recalibrate Reality'. Kees de Groot has sent an email containing information about this theme and brainstorm, including several keywords and descriptions:

In 2021, GOGBOT will research the idea of the truth/post-truth from within our digitalised society and look at the use of technology in document manipulation, propaganda, populism and conspiracy theories. Within this central topic, GOGBOT looks into issues such as what these technologies can mean for our privacy, the rise of alternative media, debates about gender and a new definition of reality.

Under the theme INFOCALYPSE NOW, Recalibrate reality, the program of GOGBOT 2021 dives into the transformation of our information ecosystem in a time of synthetic media, deep fakes, populism, physical distancing, trust crisis, the rise of conspiracy theories etc.

Wondering the impact of the democratisation of AI technologies on our society and its communication networks, GOGBOT invites artists, creators, and thinkers to present their latest work to inspire us and show us how to recalibrate our senses...

Adjacent words:

Reality
Human
Unreal
Future
Loading
Synthetic
Paradise
Deepfake
SFX
Collision
GAN
Alternative Facts

Based on this document, I started brainstorming about the contents of the VR experience. Abstract concepts needed to be implemented and twisted to make the product more appealing.

Sub-question:
"How can the GOGBOT theme be translated to a VR experience?"

Method
The CMD method 'Storytelling' was applied to realise this. With storytelling, you can make abstract concepts concrete and strengthen the empathy for the user in your team [11].

Already a big part of the inspiration came from the neo-future theming going on at WARP.
The fundamental nature of this experience already complies with a lot of the theming of the GOGBOT festival itself.
The experience will be something that's only visual digitally, using a VR headset to see and experience a new reality within a physical room that is empty. Creating the collision between the digital world and the physical, where a visitor in the same room is not wearing the goggles, gets an entirely different experience than the visitor.

More ideas for the cybercity:

Freeways are computer circuits
Skyscrapers are servers
Vending machines are arcade machines
Social bubbles apartments

Concept

Brainstorming

Inspiration wall

Goal

The next goal is to establish an art direction and look and feel for the product and the promotional campaign.

One of several professional methods to save and organise creative ideas during a project is using an inspiration wall. An inspiration wall is a way to save and organize creative ideas during a project, or even permanently, in order to have access to them very quickly and let ideas ‘simmer’ for a while [12].

Commonly used service to collect inspiration is Pinterest. This service allows you to create a pinboard to collect images and algorithmically suggest similar ones.

A couple of ideas were generated using this method.

1. Character design direction
Colourful characters created from simple shapes to fill the world. The simplicity of the designs would translate well inside a busy city.

2. Landscape direction
To illustrate the digital look and feel of the neo-city, the landscape this city is located in could be shown off in the style of retro wave artworks where the mountains are visualised using lines and grids.

3. Colours
This city is going to be a neo future style city. A few vibrant colour palettes that fit this aesthetic were found on art from 'Kurzgesagt'.

4. Marketing
Retrowave design fits quite well with the neo-city, WARP Technopolis, GOGBOT and the 'Infocalypse' theme. To apply this design into marketing, such as through poster design or social media posts, would only make sense and add consistency.

Inspiration wall

Style test

Goal

A small scene was created in Quill to test out if the look and feel from the inspiration wall would translate to VR,
A screenshot from the Pinterest board was taken and imported into Quill. This way, the colours could be colour picked from the image to apply within the scene.

The results left me pretty positive, and I believe this would work well in the VR experience. This simplistic style would allow me to animate and create environments entirely. They were also shared with stakeholder Kees de Groot, who was also delighted with the results.

World style tests in Quill

Character style tests in Quill

Style test

Mood board

Sub-question:

"How to create an appealing style within Quill?"

Goal

The next goal is to establish an art direction further and look and feel for the product and in a more uniform and pitchable way.

Mood boards are an essential part of many aesthetic design projects, in particular for communication across stakeholders. This way, the client can be involved early in the process.
To create a mood board, you need to make a physical or digital collage of images, typography and a colour scheme to describe 'fluffy' stuff such as the ‘mood’, ‘feel’, or other core design concepts you want your product to have. The images are often gathered from magazines or visual discovery tools like Pinterest [13].

A mood board was constructed using images from the inspiration wall within the software Adobe Photoshop. This mood board has been pitched to stakeholder Kees de Groot.

Mood board iterations

Final mood board created using images from the inspiration wall.

Mood board

Sketching

Goal

The product is mainly visual. The goal is to make the ideas that came out of the brainstorm concrete and visual to communicate to stakeholders.

Sketching is a standard method that is used to explore and communicate ideas [14].

Ideas that came from the brainstorm were put on paper. Then, more ideas were generated by sketching these initial ideas. Eventually, this leads to comprehensive concept art about the different items that the product is supposed to touch.
These were sent to client Kees de Groot, who was happy with the results. Communicating these ideas through sketches and concept art helped the client get involved early on within the process.

A concept art piece was created based on colours inspired by the mood board and inspiration wall to make the experience cohesive in colour.

The sketches and the concept art are created using Clip Studio Paint, a digital art software used for illustration.

Sketching

Asset list

Goal

For managing this project, there needs to be a way to prioritise certain elements and assets. Otherwise, the developer could sink too much time into parts of the project that are not as important as others.

MoSCoW is a prioritisation method used to decide which requirements to complete first, which must come later and which to exclude.
Unlike a numbering system for setting priorities, the words mean something and make it easier to discuss what's important.

MoSCoW stands for must, should, could and would:

M - Must have this requirement to meet the business needs
S - Should have this requirement if possible, but project success does not rely on it
C - Could have this requirement if it does not affect anything else on the project
W - Would like to have this requirement later, but delivery won't be this time [15]

The following asset list has been created using this method. The primary purpose of this list is to motivate the developer to work on the project in a organised matter, making sure the prioritised items get worked on first.

The most important things are at the top. They have been chosen based upon what I thought makes a basic city, such as a ground, sky, streets, sound and buildings. Next to those, I chose the concepts that appeal the most to me and are directly influenced by the main theme of GOGBOT written in the e-mail from Kees.

Asset list

The following chapter is about the creation of the product. It goes into development using Quill, sound design, optimisation and prototyping.

Production

Creating assets in Quill

Quill is used for creating the assets for the VR experience. Quill is a VR illustration and animation tool. Quill was chosen because I wanted to master this tool and gain more experience using this. Quill includes a feature where the project could be exported to Quill Theater, a software that is used to experience Quill creations within 6 degrees of freedom.

Several assets being created in Quill.

The coloured concept art was imported into Quill to make sure the art was cohesive across the experience. Then, the eyedropper tool within Quill was used to pick colours directly from the concept art. This way, the colours kept consistent and colourful across the different assets.

Animation
Quill has animation features that allow the artist to animate their assets and work intuitively. Using keyframes, the artist can make clean interpolated movements. This technique was helpful for animating robots, giving the robot a robotic motion. Quill allows for a workflow similar to 2D traditional animation, where the artist can paint frame-by-frame with onion skin. The frame-by-frame technique was used to add smear frames to certain animated assets, to convey sudden movements such as Tunnelbear entering its tunnel or Surfshark doing a kickflip.

A smear frame

Surfshark animated frame by frame

Robotic movement created with keyframe animations

Next to those techniques, Quill allows the artist to puppeteer using the grab tool, enabling the artist to grab parts of the model and move them while the timeline plays. These movements are then recorded instantly, which streamlines the animation process significantly.

Optimisation
For the creation of any asset, optimisation needs to be kept in mind. If not, this will result in more work later in production.
Every Quill scene can only have a certain amount of draw calls and polygons. Draw calls can be taken up by the use of different VR brushes and layers. To optimise the asset and reduce the number of draw calls it takes up, the artists need to create the asset with as few brushes as possible and make sure it uses the least amount of layers.
The Building asset below has only been created using two different brushes, the cylinder brush and the cube brush.
All of the animated assets have been baked and combined into one layer.
Quill also includes a tool where the number of polygons can be reduced. This will make certain curves and bends shapes look polygonal and minimise detail in colour gradients. The artist needs to be selective in what amount and where this optimisation tool needs to be used. The number of draw calls and polygons can be seen at all times using the performance tab.

A YouTube tutorial that goes further deep into optimisation techniques has been followed. [22]

Optimisation tool to reduce polygons

Cube and cylinder brush only takes up 2 draw calls

The entire building after optimising also just takes up 2 draw calls

This slideshow features objects, characters and buildings that have been created and animated using Quill. They are based on their respective sketches, but other design decisions have been made during the creation process.

Every asset has been created within a separate Quill file, which is later used by importing them into one master file.