Web Games?
Over a year ago we were approached by the team at Sunset Division who wanted to push what is possible on the social immersive web with us. We decided to start building PuttClub WebXR mini golf as an experiment to see what we could do in the Meta Quest Browser. The VR game was developed to showcase the capabilities of Ethereal Engine, and serves as a key driving force in developing what we believe to be the most powerful web based immersive engine available today.
Ethereal Engine is a free and open source multiplayer XR-first web platform engine built on WebXR, able to host highly scalable and graphically beautiful worlds, real time A/V media sessions with sophisticated administration, permission & deployment control systems. Focusing on a firm foundation of optimized ECS data structures, networking and creator workflows, it rivals existing 3D engine options with its ease of use and extensive scope.
Our goal is to open up the web to be part of a spatially aware and user owned global network. PuttClub exists to showcase how the web can perform to the same degree as native apps, while escaping closed app store ecosystems and naturally targeting many devices cross-platform from a single codebase. Originally, PuttClub was designed to target the Meta Quest 2, yet has been successfully played on the Pico 4 and Steam VR without any changes needed, with mobile and desktop controls in the works.
Why Mini Golf?
We are avid fans of Golf in VR games, and see social VR golf as a fantastic example of why social focused XR gaming sets itself apart from traditional mobile, console, and desktop gaming. We wanted to scaffold the engine around meeting such requirements of immersion and presence, emulating prior games while adding our own unique spin and aesthetic flavor, highlighting custom avatars with full body IK.
The real trick would be doing all of this while still keeping high frame rates in VR.
Implementation
We have been striving for a feeling in this concept of a wholly new game property and branding that’s fun, strange, and reflects our feeling of presence. After trying many style concepts, we settled on recreating a 50s era golf course. The design feel is intended to be reminiscent of the community golf course and scenes from Caddyshack as a backdrop for Avatars of all kinds. We designed a menu UI with skeuomorphic scorecards, with a serif font on faded yellow paper and tiny pencils.
This easy styling shows the easy flexibility and highlights the power of XR UI
Below: Alternate UI style with a more modern design we workshopped in development of the game.
We love VRChat and dream of a web with lifelike avatars. When we started building PuttClub we started with simple avatars. As the project progressed, avatar advancements in Ethereal Engine got to a point that we decided to switch the game to full body and add the ability to import your own avatar
We created Sand Island as a 3D level early in game development and it was built around doing a lot with a little. The island is an experiment in modeling a large space with enough graphics and compute resources to spare for many avatars, balls, sounds and music. We plan to build much more graphics intensive experiences at this scale using what we have learned.
Tech Highlights
.png)
ECS
Ethereal Engine leverages bitECS for its ECS engine data state and Rapier.js for its physics simulation, which we have found to be the most performant and elegant options available.
The ECS augments bitECS with a stateful object map, allowing reactive component and query bindings, much like common modern front end tools. The engine implements React for this, which is leveraged under the hood for our ECS state reactors. This is a powerful tool that simplifies many of the tasks previously handled by the verbose combinations of state changes and enter & exit queries. This approach is relatively slow for constantly changing data, such as entity transforms, so in these cases bitECS is used for code hot paths, rivaling the performance of native engines.
Ethereal Engine has an agent centric networking configuration, meaning all users are the owners and authority of their respective physics objects (in the case of PuttClub, their avatar, golf club and golf ball). This enables isomorphic networking across both client/server and p2p configurations, though at the moment only client/server is implemented. The benefit here is client/server architectures support far more realtime throughput than p2p configurations due to the nature of network complexity scaling. Although more importantly for XR applications, it guarantees zero network latency for a user’s apparent simulation, as the data does not have to go through the host and back in order for authoritative state to be observed.
The game has been through many iterations and seen many physics engines. We began with cannon.js, a staple of web physics. Cannon.js is dated and too slow for the needs of lightweight XR devices, so we opted to refactoring around NVidia’s PhysX 4.3, which gave the performance boost we needed, especially when multithreaded, but being a community port to WASM, there were many drawbacks and issues that caused memory crashes and limitations. Eventually we opted for Rapier.js, the new kid on the block, written entirely in Rust and packages to a more lightweight WASM for the web. This saw further performance benefits, allowing more mesh colliders in the world, more physics timesteps to reduce tunneling effects, but most important, a fully typed physics engine, with a vibrant community.
Game Progression & State
The game state has gone through a few iterations itself, beginning as a naive schema of state and “checker” functions to progress state. We quickly moved to a more robust action/receptor pattern, as is common in front end development known as the FLUX pattern. Our fancy in-house action and state management library Hyperflux allows actions to be natively networked via a ‘topic’ abstraction, meaning actions can be dispatched to certain places, such as just the local app instance, or to various network topologies (currently just the world instance and media instances). These actions are aggregated and processed inside the ECS as queues, much the same as entity queries, meaning the developer has complete control of the ordering of responding to actions.
The game’s state itself involves a few concepts: currentPlayerId, currentHole, and the players array.The current state of the game is governed by the curentPlayerId and currentHole, and progression of the game is governed by the player’s state. Each player has stroke and scores properties. stroke stores the amount of time the user has hit the ball on the current hole, and every time a hole is completed, that number is appended to the scores array, and is reset to 0. This scores array obviously becomes the running score of the player as the game progresses.
The last piece to the core golf game loop is progressing player turns. This is calculated by finding the player with the fewest completed holes, such that a user that leaves and returns, who is now a few holes behind, is directed to play shot after shot until they are caught up. This elegant algorithm is all that is needed to play through a full 18 holes! There are also some extraneous state keeping track of the current ‘game’, inProgress specifies whether a user is currently in a multiplayer game or in the lobby, if the latter, then only the local user is in the player list, thus can hit the ball ad infinitum.
Ball Physics
PuttClub’s physics setup uses a dynamic sphere body for the golf balls, a cuboid position based kinematic body for the club, and a kinematic body for the avatar, controlled via Rapier’s character controller API. We ran into a lot of issues with tunneling, which is to be expected with a small object moving at a high velocity. We first attempted to address this with Continuous Collision Detection, though when using CCD in conjunction with the kinematic body, rapier seemed to have odd artifacts in the simulation. So instead we ramped up our fixed timestep to 90 and, increased our substeps to 3, meaning we were simulating 270 physics steps per second, which seems to fix tunneling in almost all cases. In the next update to the game, we will implement an algorithm to ensure the maximum speed of an object does not exceed its diameter per physics step.
We’ve covered game turn progression, but there is a bit of trickery around how we handle the various active physics states. The ball is a fixed body until it is your turn to hit it, in which case it becomes dynamic. We tried to ensure that the ball stays locked to the ground until it is hit, but proved more difficult than we originally thought, so it still may roll slightly due to slight gradients in the geometry. The lesson we learned here is to be extremely discerning with the geometry - only steep slopes or perfectly flat areas to reduce unwanted rolling.
Once the ball is hit, we use its velocity to detect when it is coming close to a stop. Once it passes below a threshold, we start a timer that runs for a few seconds. If this timer successfully makes it to 3 seconds without jumping above the threshold, the ball is considered stopped, and the turn is progressed. There is also a timer for the ball going out of bounds.
Due to the way the geometry was originally created, we had to find some workarounds to ensure out of bounds checks are correct. We detect if the ball comes in contact with out of bounds colliders, or if it’s a significant distance from the course. If it is, the turn is progressed with the ball resetting to its position at the start of that turn. In the future, we will restructure the geometry to properly separate the in-bounds green with the rest of the geometry (currently course edges and some rocks are considered in-bounds) which will simplify this logic.
Dynamic Colliders Optimization
Performance was hard to tie down, with 18 separate colliders, all quite complex with trimesh colliders, we had to ensure the physics simulation stayed below about 1ms per frame. We tried a few methods to achieve this, firstly dynamically loading the colliders of the active hole and whichever ones the player is nearest, but being an asynchronous process, we ran into race conditions. The solution we landed on is to remove the collision groups of inactive holes, which allowed rapier to ignore these in the expensive trimesh intersection tests, and got us well within our budget. Non-course colliders still use async dynamic loading, as we can guarantee the user will only teleport to a green collider, and use a generous distance based calculation to load them in.
Transform System Optimizations
To get this game to play at 90fps on the Quest 2, a lot more optimizations than just physics were necessary. Our biggest improvement was in optimizing the transform system. Our previous naive attempt relied on using three.js' internal matrix calculations, which are recomputed every frame, for every object. The first priority here is disabling this behavior in three.js and creating a “dirty transforms” map in the engine, which specifies a list of entities to update each frame. This is done in conjunction with lazy hierarchy sorting, to ensure children entities are operating with the up-to-date parent transform data. This allowed us to opt into the expensive matrix calculations, rather than having effectively no control. Our next priority was taking advantage of bitECS’ performance capabilities. We rewrote the entire transform and physics systems to do calculations as efficiently as possible, re-ordering functions utilizing L1 CPU caching via bitECS’ Structure-of-Arrays (SoA) implementation - meaning there are as few fetches between the CPU and RAM as possible. With these optimizations, we in some cases saw a 10x performance boost, from 9ms down to less than 1ms on the Quest 2.
Avatar Priority Queue Optimizations
Another big hot spot in the codebase is a feature we’re very proud of - full body avatars with IK. Calculating the 50 odd bones, along with controller and head IK was VERY heavy, 1ms or so for just a few avatars. We can do better. Our solution is to splay these updates over multiple frames, which in the future will evolve into a self profiling adaptive system, but for now, it’s a naive ‘budget’ based priority accumulator. We found that for the Quest 2, 1 avatar update per frame across all avatars was enough to balance aesthetics with performance. Essentially, all avatar mixer, animation graph, IK logic and skeleton matrix updates are done for each avatar that meets a few heuristics. If an avatar is outside the camera render frustum and more than a few units away from the camera, the entity has its priority reset to 0.
For all other avatars, priority accumulates relative to its distance to the camera, meaning avatars closer will update more often. After all accumulation is done, the priority entities are then normalized by the budget, such that each frame deterministically there will be a list populated with entities to update, if there are enough in the query. This allows performance to scale in this system effectively infinitely for all reasonable experiences, as the number of calculations never exceeds the budget.
Renderer Optimizations
That’s enough optimizing for the CPU, we still need to make sure the GPU isn’t being hammered! Taking inspiration from many lightweight yet visually pleasing games, we decided to go with vertex coloring via lightmap baking for the scene geometry. We experimented with baking lightmaps in blender and even implementing it in Ethereal Engine’s studio (which will be revisited in the future). We found Unity’s Bakery to be hands down the best option.
The process is as follows:
- Use Bakery Lightmapper in Unity to generate lightmaps
- Export GLTFs using fork of glTFast
- Bake diffuse, emissive, lightmap into vertex colors in the Ethereal Studio
- Delete all attributes except position and color from the mesh
- Save baked meshes as GLTF
- Use Ethereal Studio’s Model Transform Tool, a built-in asset optimizer with gltf-transform and Draco
Inside a running instance of the engine, these geometries are batched together using a modified version of three.js’ Batched Mesh class (currently sitting in a PR in three.js). This collapses their draw calls down to 1, as they are rendered as one.
Our next step is to add a grass texture to the baked compressed mesh's diffuse to make the terrain more defined.
XRUI (XR UI)
Ethereal Engine uses the ethereal.js package (soon to be integrated directly into the engine) to generate UIs that work in both immersive and desktop/mobile contexts, called XRUI. Using HTML5 & JSX, (DOM elements and CSS under the hood) to render textures to meshes, developers can write UI with the same front end frameworks that are widespread through the web development world. Ethereal Engine and PuttClub use React and TSX for reactive UI design. There are a few abstractions for the XRUI system, that adaptively add these UI panels to the camera, the user’s hand, fixed points in the world, anything at all. They are all wrapped in three.js’ object hierarchy, as well as accessible through the ECS, meaning they can be animated, manipulated, and have any other transformation you could think of applied. Since they exist as 3D meshes, we can also use shaders and postprocessing on them, all from DOM elements!
Room Codes
The main UI panels PuttClub users encounter are for creating or joining rooms. Rooms are private world instances that have randomly generated alphanumeric identifiers associated (PuttClub uses 6 digit numbers), allowing more than enough realtime games to be spun up as needed. These instances are dedicated world and media servers, one of each for each game, enabling real time distributed physics world simulation, and high quality audio / video / screen share capabilities in XR.
Project API & Developer Workflow
The PuttClub project exists as its own GitHub repository, although it does not use npm or another package manager to use Ethereal Engine. Instead, Ethereal Engine contains a projects subpackage, which contains a directory in which any number of user projects can exist. This workflow is similar to other game engines, such as Unity and Unreal, in which a file system represents a user’s data separate from the engine. Being a web engine, these projects are both available on a user’s local file system, and are then served either locally or uploaded to a cloud service (such as Amazon S3). Due to being installed on deployments via GitHub repositories, projects are version controlled via git and come with all the advantages of git and GitHub you can expect.
Once a project has been installed to an Ethereal Engine deployment, the engine must then be rebuilt. Luckily, we have a service for this called the builder, which is a custom docker process that handles various activities necessary to build all the client and various server docker images. These are then deployed to the Kubernetes cluster, and an update can be served in real time. Future work in this area includes per-project building, which will radically improve the efficiency of on-site upgrades for both assets and code.
We are also working on the Ethereal Engine Control Center, which allows users to connect to both local deployments and remote deployments of the engine. From here, users can configure deployments and access their administration controls & kubernetes tools. We are working towards being able to use Kubernetes as the default developer environment, enabling spinning up multiple world servers simultaneously and more closely simulating a production environment. For now, developers use npm commands to manage their local environment, running vite for the client, and ts-node for the servers. There are a multitude of servers, including the API server, file server, world server, media server, and task server.
What's next for PuttClub.io?
We have had several major updates since the start of our Public Beta in October, and more to come.
Ready to hit the links?#VR #MiniGolf right from your browser.
Learn more at https://t.co/81sZPjIScN
Join Beta testing now!
Join our discord: https://t.co/iLkwvCdnYH#WebXR #MetaQuest2 #MetaQuestPro #MetaQuest @Meta @MetaQuestVR @MetaQuestGaming @Oculus_Dev pic.twitter.com/4O5bRVwJGu
— Putt Club MiniGolf - Social #WebXR (@putt_club) October 29, 2022
We have many ideas and prototypes in the works building on our little golf game.
Look out for….
- Advanced Desktop VR Graphics
- Multiplayer AR mode
- More Courses and Upgrades
- Secret CRAZY NEW Mode
JOIN OUR COMMUNITY
We are off to an amazing start making the web more equitable, safe, and decentralized, but we can’t do it alone. Help make this dream a reality by joining the effort!
