Duolatera

Context:

We are going for mid-poly, NPR, cartoonish cel-shading effect.

Decision:

- Compared to normal asset pipeline, when texturing, instead of creating texture maps, we ask our artists to only create RGB Channel Masks for different areas.
Each channel marks up the areas with the same material properties (in most cases, base color/albedo).
If more than 3 areas are needed, we create a second RGB Channel Mask, and more if needed.

- After the masks are imported to the engine, we then assign the desired color for each RGB channel.
And in the cel shader, it goes through all channel used and applies the color we choose.

Rationale:

There are two reasons behind this decision:
1. Using RGB Channel masks, we can create many variants of one asset with no time.
This is especially useful for 2-player game, where both players need the same asset with slight difference (I.E. Their laser tools).

2. To create an unique cel-shading effect, we want to directly control the shaded color on our assets.
Using the masks, we can directly assign the final rendered color in Engine.
a) It makes it much easier for cel shader to find the right area and apply the desired color.
b) It also saves so much time for artists to iterat colors of assets - instead of changing the colors in texture map and reimport it, now they only need to select a new color value.

Albedo to RGB Channel Mask Conversion GUI

After establishing the pipeline, we found out the artists were struggling with creating the RGB Channel Masks, because it's not intuitive and time consuming.
To speed up the process and make everyone's life easier, I created this conversion tool GUI.
Using the color clustering function in OpenCV, users can easily get the converted masks with a drag and drop and click of a button.

Its functionalities in detail:
- Quanitize (cluster) the source image, generate a clustered image with the assigned number of colors.
- Iterate through each quanitized color, and overwrite them with order as R(1,0,0), G(0,1,0), or G(0,0,1).
- Output one mask for every 3 colors iterated.

Quick Asset Check Blender Add-on

Due to our unique asset texture pipeline, it's necessary to check the masks for every assets.
However, it is tedious to manually import .fbx file, organize the scene, add all related masks every time for a new asset.
So, this Blender add-on, and does the following:

- Disable other collections of objects and create a new collection.
- Import the .fbx file.
- Examine the imported object, find all "Mesh" type (ignoring skeletons and empty objects).
- For each "Mesh" type object, apply a material to it.
- Find and import all related texture files by name.
- Piece textures together and connect to output.

Unreal PCG to Static Mesh Converter

Context And Decision:

Using Unreal PCG drastically saves our time building the level, but since it's an experienmental feature, we've found glitches especially in the multiplayer context.
So to avoid any unforeseeable bugs, we decided to convert all PCG into static meshes.
To save time, I made this selecting/replacing tool using Editor Utilites and Python.

Functions:

- Select all PCG actors in current level.
- Clean up any empty static mesh actors in level.
- Locate / create the file path for the merged static mesh asset.
- After manually merging selected PCG actors using Unreal's MergeActorTool, get reference to the merged static mesh.
- Replace all PCG actors with merged static mesh, preserving original transform.

PCG Decoration Stripe

To save the time for our level designer to quickly decorate the level, I designed and implemented this PCG stripe.
User only need to edit the control points of spline, and the stripe will be generated with on click.

In addition, I implemented a auto snapping tool for spline control points. When getting close to surface, it automatically snaps to nearby surfaces with configured margin.
Currently, auto snapping supports 1 face, 2 faces with inside and outside corner, and 3 faces.

To increase the immersion, we go for full body player avatar. But Meta Quest can only provide us with 3 tracking points: the head and two hands.
In order to implement full body movement, I designed an algorithm to predict the player's movement using Unreal's IK animation system.

For legs:
- At the start of the game, player's height is measured.
- Using the head's height, it computes the hip's height by assuming the player is standing straight.
- Cast a line downwards from the hip and get ground location.
- Based on the hip height and ground height, interpolate the calf and thigh by assuming it's a squating move.

For arms:
- Assume the hand is in resting position and the player doesn't spin their wrist.
- Set elbow's joint target location using the hand's orientation.
- Clamp the elbow's joint target location so it doesn't bend towards the chest.

Unreal Online Subsystem

I implemented and maintained Unreal Online Subsystem into our game with the listen server model.
As our development went further, I kept optimizing the Online Subsystem’s reliability and efficiency. For example, I optimized syncing players’ physical movement to each other bypassing the server's validation.
I also kept expanding more gameplay functions like grabbing, triggers and laser tools. Making sure that all states are synced is a challenge, i.e. if the host picks up a cube before the client joins the session, the client still needs to see that the cube right now is in the host's hand.
We had some trouble when spawning and initializing all game objects in the level, so I spent some time examing the Unreal Engine source code on initialization phase, and overrode some default functions to suit our game.

Some useful notes:

- For syncing to be reliable and quick, use replicated variables instead of Remote Procedure Calls (RPC). Because replicated variables are well optimized synced in Unreal Engine, take less traffic, and is synced by an internal interval, and can also work with net cull distance. Most importantly, it keeps the progress if the client accidently logs off.

- Even we're using listen-server model, all authority are still kept in the host, and the client can't do anything without validated by the server. However, there's a workaround by using net role of autonomous proxy, giving some freedom to the client to send data on their own (like their own movement).