#motion tracking

So, in the 5 years since that post, I’ve learned a lot of useless information, and some of that is how to track footage in 3D, using Blender.

Since I never had any luck getting the battle scene geometry out, and I really wanted to know some more intimate details like the camera’s field of view and positioning, I thought I’d try some reverse-engineering. As usual with my grand ideas, the line between genius and lunacy is fuzzy.

I started to track some normal old emulator footage, but sub-pixel fuzz was making accuracy impossible. You need 8 perfectly tracked points for blender to calculate the camera angle, and I barely managed 4 wobbly ones.

I soon lost patience and decided to look into uprezzing DesMuMe. I did some research and found an old, Japanese, windows-only 2014 build that someone had hacked to increase the resolution for models. Perfect!

I tried to record some more footage in OBS... but it was compressed to hell, and the big blobs of colour that endeared me to this generation were not getting me any sort of precision to track anyway. Another bust.

I realised my best chance at accurate tracking was going to be to make it render in wireframe. This is a pretty easy switch for OpenGL - you can just add a single line of code. Maybe I could get the source code and recompile it?

After a lot of searching, I did find the source code - but it was relying on libraries compiled in 2012, and would not compile at all. I also got very upset with Windows and Visual Studio, and had to steam off for a few days before trying anything else lol.

Of course... when I came back, I discovered the Mac version of DesMuMe has a setting called “GPU Scaling Factor”...

I felt very stupid.

DesMuMe is actually very easy to download and compile on Mac. After a bit of fiddling, I got my 4x scale precision wireframes! Some tracking in blender, and bam! Accurate camera positions, angles, and FOV.

Then I spent three days getting very upset, until a passing stranger explained to me the difference between left-handed and right-handed coordinates, and the final piece of the puzzle fell into place.

At some point in the future, as part of the engine’s Help docs, I’ll post the .blend file publicly so that anyone can model custom battle backgrounds and plinths.