One Bizarre Game Dev

SO, yes, it sounds like an appealing idea to do expressions where you create a single mesh for the eyes, and an image with all the eye variants, and you simply move where the eyes map to to change the eyes on the model. THAT'S A GREAT IDEA, but I don't feel like that's how these vtuber softwares are designed to operate. Is it possible? Yes?, But not without headache and limitations.

The alternative (and how we ended up doing it) is to create a mesh for the eyes, map it to the "default" eyes, then duplicate that mesh any number of times and map them to each variant. Those eyes now hide just barely inside the face. Now you can create a blendshape to smoove that mesh in front of the default eyes. DONE, and just create a new blendshape for each expression. So all the faces are stacked on top of each other and hiding away inside their face

Lets ignore how to make Madame pose for a second, and simply imagine that we can. To be able to make it look like they're sitting on things, there's a greenscreen box that they sit on. This box actually starts very tiny and hidden inside Madame's pelvis. You then blendshape it bigger and move it to position. SIMPLE. It's done exactly the same way as the mouth and eyes. (GIF not included at this location, see GIF above)

We can stop pretending that we already have a Madame pose and talk about how to do it. It's actually an animation, not a blendshape! To make it, you can head over to the animation tab and create a new action in the action editor. For each new pose, create a new action. Even if you don't have an animation and you just want it static, this is probably how you'd do it

So this was the initial file I was handed by Ame. An empty object filled with clearly separated objects. Job 1 was to get in a super basic skeleton. Job 2 was get in the bones for the extra little doodads that get saved in their own group. Pro tip, I initially made the bones for the hair sticking outwards, but the spring bones seem to work better if the bones point in the direction the thing dangles (seems obvious in hindsight to make the bone for the thing that dangles down, point down)

To add physics to stuff like the hair, you need an extra bone there that's connected to the head bone. These bones will then have springy-ness, drag, and gravity. I'll mention how to add physics later, that's in the Unity step. The extra bones I added beyond the most basic human skeleton were:

Hair bone

3 Bones in each pigtail

Head wing bone

1 for each shoulder frill

1 for each bat bow (1 of these bones is also connected to the hip wings, because I'm sneaky like that)

2 Skirt bows

Easy. Some funny quirks are that she only has 1 finger bone for all fingers. She's also got a bone for the jaw and each eye, despite not using them

Now for the weight painting

This becomes an interesting case of: there are so few vertices, so I don't have much to do, BUT I have so few vertices so each one influences the final result MUCH more. Fun, anyway.

On each object, I added an armature modifier to the mesh. This gives the bones the ability to deform the mesh. Next, I made vertex groups with the exact same name as the bone I want to control it, this automatically links that bone to that set of vertices. I'm not really good enough to have an opinion on what is the best rule of thumb for picking weight percentages. I simply guess, and try it out in pose mode

I had an object that I imported, I auto-generated some collision, things worked all hunkydory. But then I re positioned the mesh in Blender, re-imported it into Unreal, and now it sits in the ground??

Figured out that Unreal generated the collision once, but doesn't do it upon re-import. Which I guess is OK, I just didn't know about it

3D education with JC had a great video on this

If you head into your view properties, you can turn on collision, and see what's going wrong. The collision box is way out of wack (And the auto-generated collision is too complex for my taste on a simple book). We can head into collision in the mesh window thing, delete the existing, create a simple box. DONE

We are going to need 5 things

- An Action that could happen upon pressing a button (IA_Interact)

- Tell Unreal to run that action when a key is pressed (IMC_Default)

- A means to say, yes this object can be interacted with (BP_InteractInterface)

- A means to tell if the player CAN interact with the thing (BP_FirstPersonCharacter)

- What the object does when interacted with (BP_BaseItem)

That's a lot, let me explain. The IA_Interact is the event that will sit in wait eagerly to be triggered. That's kinda of it. Create it and forget it.

IMC_Default is responsible to actually trigger that event. Setup the key that has to be pressed to trigger the action. DONE. We can move on

Now we need something to attach to an object that gives it the ability to be interacted with. We can say: We are giving an object an interface that can handle interactions. That is what BP_InteractInterface is. The only thing this handler will do is make a slot to save what interacted with it (the interactor), and if the interactor is currently in the "being interacted with" state (the did interact?)

Now we get to the meat. BP_FirstPersonCharacter. When IA_Interact gets triggered we want to check what the player is looking at, see if it's interactable, and if it is, save what that object was and put the player in a "Is interacting" state

To do this: When the IA_Interact gets triggered, draw a line in the direction you are looking. If that line hits an object, check if the object has an interface that can handle interactions (I.E. check if it has the BP_InteractionInterface attached to it). If it does, remember what that object was. We are now going to use the BP_InteractInterface to set who interacted with it, and set their "Is in interaction" (Did Interact?). After 1 second, it will reset their "Is in interaction" state.

Now the potatoes. The BP_BaseItem is the thing you'd make instances of and attach the mesh you want, giving it all the interaction functions in the process.

We can save a copy of the player, so we don't need to recalculate it everytime.

Now every tick of the object's existence it checks if the player is in the "is interacting" state and if the object you interacted with was this specific item. Now that we know the player IS interacting, and the object they want to interact with is THIS one, we can do whatever we want from there... DONE! (Oh yea, we also need to go into the class settings and attach the interface in "implemented interfaces")

I'm following a set of tutorials for this, because I really haven't touched blueprints until now. All four are by Code Things on YouTube. Not done it yet, but so far so good. It's a tad bit of a rabbit hole, but it's worth it

First: https://www.youtube.com/watch?v=LTZhNb0eKa8 Second: https://www.youtube.com/watch?v=u-JxS8yy6ao Third: https://www.youtube.com/watch?v=SKO4DDjnP70 Forth: https://www.youtube.com/watch?v=TDlnOg22YNQ

So far I've got it where as you approach the object the interaction icon pops up. The setup was fairly quick

Create a blueprint, drop in your mesh (the item to interact with), and put as many collision radii you want. I have 2. An "in view", and an "in reach" radius. But I wouldn't fault you for having just an "in reach" radius. To those collisions, attach your icons (Unreal calls them billboards)

To set up the events we need a "begin overlap", if the actor that enters the overlap is the player character, set the visibility of the "InReachIcon" to visible. DONE. Do the same for the "end overlap" event. Here I needed another "set visibility" at the end of that to turn off/on the visibility of the "InViewIcon" so they aren't both visible at the same time. Bellow that are the "begin/end overlap" for the "InViewIcon" aswell

The construction script is the thing that runs when the object gets put into the world. Here we can make the blueprint more versatile promoting the radii of the collisions to a variables. I also went ahead and promoted the height of the icon to a variable so we can adjust it for each item we drop into the world

I completely reworked the arm bones. I realized the eyes and pupils were slightly misaligned, so I spruced that up. I got the springy bones working for the ears, neck floof, and butt tail. I fixed up the weights on the outline. Sadly I didn't set up the expressions for the VRChat avatar, but they do work on the Vtuber and that's a whole other adventure. There they are, in all their glory. (They are quite short)

Oh yea, for the leg stripes all I did was cut ribbons into the outline mesh, then reversed the normals so they're facing outwards, instead of inwards like the rest of the outline.

For reference, I used the 2022 avatar creator in the VRChat Creator Companion. Unfortunately, I couldn't use the same blend file that I used to create the Vtuber model, but whatever. I wasn't that hard to start from scratch in Unity.

So first and foremost, the obvious thing is the arms. The first pic is literally me standing still, and the arms are different. I'm 99% sure this is due to my pose mode in blender not being reset before exporting.

Pic 2 you can see the transparent material for the eyes and pupil are not exactly transparent. I think that's a Unity thing? Like I need to adjust the shader or something?

Next next, it looks like the weight paint on the model is not the exact same as the outline?

It's back to the drawing board and I think I have a better setup for the bones. VRChat was really bitching about FOUND having shoulders, so I have to fit those in. FOUND's new X6 model will have shoulders, and a bicep where you'd expect, but the forearm will be a small-er bone in the elbow, then the hand bone will be the forearm looking part. So the next model will have an arm bone overhaul

Getting this to work this way was quick once you've figured out creating a VRM. Basically just have an image of all the expressions I want mapped to the mesh. In Blender I defined a 2nd material for the face area and applied the new map like you would any other material.

When importing your FBX into Unity, change the materials location to external, and fix the materials "Albedo" so it actually displays right. Once you export and re-import your VRM to continue working on it, head over to your Blendshapes, select the emotion you want, (here's the secret) head to "Material List" and pick which material and its new offset values. Mine has 4 rows total, so each expression is an offset of -0.25 down from the top. Use "_MainTex_ST" and it'll work. Not sure what that means yet, but it works. Side note: I tried to swap to URP for my rendering pipeline, but then this step doesn't seem to work with Vnyan. Probably something I'm missing to get it to work, but as far as I know rn, it's best to stick to the default Unity rendering pipeline and only if you know what you're doing should you switch pipelines.

It took a long fucking time to get a version of the model I was happy with. I restarted from scratch like 5 separate times. I started with sub-dividing the planes by hand, then I discovered I can create curves and skin them. I immediately hated doing that, went back to sub-dividing and moving the vertices by hand.

By the time I had Found V3, the ears were much bigger, and the leg were cylinders. But there was an unforgivably major issue... The geometry on this model was a bigger nightmare than anything Alien Colonial Marines ever had, and that's not the standard I want to live up too.

(Side note: Ignore that fact that all the files are called "Lost" instead of "Found". I miss-named the first file, then by the time I realized, I already had 20 different files and objects with the wrong name)

Probably the biggest shot to the head was making the neck floof. There was moments where I'd rather be pulling teeth. This is the part that went through the most renditions and even led me to restart completely because I though the problem wasn't just the floof shape, but also neck and body shape. In the end, changing the neck shape helped a shit ton.

At some point I started playing with an outline. I started with playing with the shader and used the geometry nodes to made the sharp bits pink. Then I moved on to using a copy of the whole mesh that's slightly bigger and reversing the normals, so the mesh is facing inwards, then I enabled backface culling. The massive downside to this is that it doubles the polygon count, but looks lovely.

To finish off this post, here's what I got so far. The design was redone to have smaller ears, less polygons and more volume to all portions. The entire mesh was redone to be more flexible. The yellow feathers on the ears can move with their own bones connected to the main feathers. All floofy spots can wiggle and bob. They have only 3 fingers, a thumb, an index, and a middle finger. I added a little chicken butt. The arm bones are a little devious, because you're supposed to have a bicep, forearm, and hand bone.... Found only has a bicep and hand bone.... So I added a very tiny bicep inside the chest that has no weight, and made Found's bicep their "forearm"

I'm still working on the face with expressions. I'm currently having a time trying to get Vnyan to swap expressions, and once it can do that, I'll get the pupil to look around with the eye tracking. But everything else works and bends in a way I'm happy with.

This is assuming that your Blender file has many unrelated meshes that have their own armatures with multiple animations on each armature.

My first mistake, ALWAYS have a different name for all your bones across all your meshes. I've adopted using <ObjectName>Root for the root, and <ObjectName>Bone.001...<ObjectName>Bone.999 for all other bones.

Next up, make a bone group for each independent armature. I use <ObjectName>Bones for the primary collection.

All this hubbub was to make sure there's no bone conflicts when bringing it into UE5.

Now lets make more than a single animation. I found you can make your dope sheet like normal, and when you're done, head on over to the action editor, and "stash" your animation. Then you can start a new animation for the same armature by duplicating the current one (click the 2 papers icon), or click "new" (I guess, I never tried "new"). This should make a new NLA track.

Now we've got the bones organized, the animations are organized, lets export.

My export settings for skeletal meshes is above. The main changes here are the "Apply transform". This doesn't seem to do anything if you don't need it, but it does if you do. So there's no harm in having it turned on. Without it, you'll get an error that UE5 can't create the physics asset.

Supposedly making material instances doesn't just make your life easier, it's also much easier on the game. Probably because under the hood, all these materials now share code or something.

Making one is easy, just make a master material like any other material, and any nodes that you want to vary, right click and select "Convert to parameter". Master material done, now lets make an instance. In the content drawer, right click on the master material, and select "Create material instance", done done.

Started opening the spring skeletal mesh and adding a socket to the bone I wanted things to attach to

Then I start a new actor blueprint and dropped in the spring skeletal mesh, anvil, and claw. I set the animation of the spring to the spring animation just like in the regular editor. I then parented the anvil and claw both to the spring, and set both their "Parent socket"s to the newly created socket. An important thing to do here is now to fix your location, rotation, and scale. If they're set to relative, they follow the socket, if they're absolute, then they don't. I.E. It's probably a good idea to set at the very least your scale to absolute so it doesn't become ridiculously big or small.