#debugging

Isn't it beautiful? The lovely formatted tables of register and stack contents, the trace of function addresses and parameters, the error message ... it's the most beautiful kernel panic I have ever seen.

Why on earth would I be so excited to see a computer crash? What could possibly be beautiful about a kernel panic?

This kernel panic is well-earned. I fought hard to get it.

This kernel panic came from a current NetBSD kernel, freshly compiled and running on Wrap030, my 68030 homebrew computer. It is the result of hours upon hours of work reading through existing code, scattered documentation and notes, writing and rewriting, and endless compiling.

And it's just the start.

As I've said before, a goal of this project has always been to build something capable of running some kind of Unix-like operating system. Now that I finally have all the necessary pieces of hardware, plus a good bootloader in ROM, it's time to give it a shot. I'm not that great with this type of programming, but I have been getting better. I might just be able to brute force my way through hacking together something functional.

It is hard.

There is some documentation available. The man(9) pages are useful, and NetBSD has a great guide to setting up the build environment for cross-compiling the kernel. There are some published papers on what some people went through to port NetBSD to this system or that. But there's nothing that really explains what all these source code files are, and which parts really need to be modified to run on a different system.

I had a few false starts, but ultimately found an existing 68k architecture, cesfic, which was a bare minimum configuration that could serve well as a foundation for my purposes. I copied the cesfic source directory, changed all instances of the name to wrap030, made sure it still compiled, then set about removing everything that I didn't need. It still compiled, so now it's was time to add in what I did need.

... how ... do I ... ?

This is where things get overwhelming very quickly. There is documentation on the core functions required for a new driver, there's documentation on the autoconf system that attaches drivers to devices in the tree, and there's plenty of drivers already to reference. But where to start?

I started by trying to add the com driver for the 16550 UARTs I'm using. It doesn't compile because I'm missing dependencies. The missing functions are missing because of a breaking change to bus.h at some point; the com driver expects the new format but the cesfic port still uses the old. So I needed to pull in the missing functions from another m68k arch. Which then required more missing functions and headers to be pulled in. Eventually it compiled without error again, but that doesn't mean it will actually run. I still needed to add support for my new programmable timer, customize the startup process, update hardware addresses, make sure it was targeting 68030 instead of 68040 ...

So many parts and pieces that need to be updated. Each one requiring searching for the original function or variable declaration to confirm expected types or implementation, then searching for existing usages to figure out what it needs ... which then requires searching for more functions and variable types.

But I got something that at least appeared to have all the right parts and compiled without error. It was time to throw it on a disk, load it up, and see what happened.

Nothing happened, of course. It crashed immediately.

I have no debugging workflow I can rely on here, and at this stage there isn't even a kernel console yet. All I could do was add little print macros to the locore startup code and see where it failed. Guess, test, and revise.

I spent a week debugging the MMU initialization. If the MMU isn't properly configured, everything comes to an abrupt halt. Ultimately, I replaced the cesfic machine-specific initialization code and pmap bootstrapping code with functions from yet another m68k arch. And spent another day debugging before realizing I had missed a section that had comments suggesting it wasn't for the 68030 CPU, but turned out to be critical for operation of kernel memory allocation.

Until this point, I was able to rely on the low-level exception handling built into my bootloader if my code caused a CPU exception. But with the MMU working, that code was no longer mapped.

So then came another few hours learning how to create a minimal early console driver. An early console is used by the kernel prior to the real console getting initialized. In this case, I'm using the MC6850 on my mainboard for the early console, since that's what my bootloader uses. And finally the kernel was able to speak for itself.

It printed its own panic.

The first thing the kernel does is initialize the console. Which requires that com driver and all the machine-specific code I had to write. The kernel is failing at its step #1.

OH DAMMIT DAMMIT DAMMIT

Tonight I got distracted by a fly buzzing around my office. I guessed the root cause: flies are attracted to light, and my office was the only room with lights on. I tested my hypothesis by turning off the overhead light in my office and turning on the one in the hallway. In support of my hypothesis, the fly flew into the hallway. I then toggled the lights and closed the door.