Joshua Enfield

There was a comment made the other day about a particular piece of functionality that had been implemented. This piece of functionality functionally worked. However one person perceived as not done the "right way". 

I think the idea of a "right way" in software is a paradoxical unicorn. There are a lot of ways to solve a problem. Some have more features, some are faster, and some work better than others. However, at the end of the data any number of solutions are perfectly acceptable for solving a problem.

Full Disclosure: the publisher provided me with a free copy of the book.

Redmine is a fantastic free open source project management system. It's been a while since I've had the pleasure of using it, but I started with it several years ago while working for the university I was attending at the time.

I had researched several tools and Trac was one I had liked quite a bit previously over the other project management options. The only con at the time was that Trac only supported only single projects without monkeying around. This sucked. Luckily I found Redmine that provided everything I loved about Trac (and more) and also offered multi-project and hierarchical project management.

I thought it would be cool to have a chat client in the system for the team so I stumbled through building a chat plugin over the course of a month. It wasn't exceptionally hard, but it was a journey. I've loved the system ever since, but unfortunately haven't had much time in it do to using different systems professionally. Although I did get the favorable experience of setting it up again to play with some code from the book. It really is a fantastic product, and I think it's much better than most paid offerings. I can't recommend it enough.

Redmine has rich support for plugins and rails is really an amazing framework for extension. Having a book to accompany the process is a nice complement to the journey of developing a plugin. I was somewhat excited to see a resource like this appear. What follows is a review of the book itself.

Redmine Plugin Extension and Development ASIN: B00J4LO3Q8 http://www.amazon.com/Redmine-Plugin-Extension-Development-Bevilacqua-ebook/dp/B00J4LO3Q8

http://www.packtpub.com/redmine-plugin-extension-and-development/book

Quick Summary

This book highlights the authors perspective on building a plugin from his experiences building the Knowlegebase plugin for Redmine. It covers he basics as well as some of the interesting aspects you'll likely need like hooking into Redmine itself and working with permissions along with several other aspects you may or may not need (e.g. file attachments, integrating with the activity stream, modifying views, etc.). The end of the book wraps up with a quick overview of writing tests, although it's more overview than something as comprehensive as the other parts of the book.

Who this book works well for

This book is well written for most beginning audiences. It doesn't have extra fluff and get's straight to the point. It's concise almost like a cookbook, but still reads well if you like casual perusal.  Basically anyone wanting to build a Redmine plugin will probably get something out of it.

What This Book Isn't

This is not a book to take a beginner to Redmine and Ruby into developing Redmine plugins without some effort. You won't get by very easily if you don't have a basic knowledge of Ruby and preferably of Rails. You will likely have to pull information from other sources or stumble around yourself a bit. That said it's something manageable as the concepts and ideas underlying everything are pretty easy to figure out.

By the end of this post you will be able to figure out where IL Code is in memory (memory address and hex opcodes), have a basic understanding of WinDBG, and understand some mechanics of the CLR, JIT, and App Domains.

My goal is to write a dynamic code injector for .NET. In this aim I decided to start with learning WinDBG, and using it to explore the inner machinery of an app domain. I've completed my first exercise of finding a method using the debugger and printing out the ILCode (and thus discovering all of the addresses and metadata along the way. ) This article is a guide through that experience so that you may have it for yourself.

Why WinDBG?

If you have ever stumbled into WinDBG before, it looks rather ugly and cryptic. Something someone from the days of WindowsNT or 95 might have used. However, even with my brief flirtation with the software I have come to already appreciate it's power. I was surprised to learn in my rersearch that many people at Microsoft actually favor WinDBG over Visual Studio for much of their debugging. I believe this comes from the power the debugger has.

As off-putting as it might be at first, there is also something soothing about using a console to debug and explore. WinDBG was the easiest path I found to be able to explore things at a level beyond Visual Studios default debugger, and most of the JIT/.NET guides seemed to use it 

1 - Getting WinDBG and a simple Sample Application

Getting WinDBG is pretty much just a google search. I used the Windows SDK download.

I started with a simple console application. Out of the box of course I had a Main Method. I added two more methods [Foo() and Bar()] with a call only to Foo. Foo prints a pretty a "hello world" like message. My goal was to use injection to eventually dynamically swap the JIT calls to Foo with calls to Bar, however for this exercise I just wanted to explore. Below you can find my simple application I was operating on with WinDBG:

namespace FooBar { class Program { static void Main(string[] args) { var program = new Program(); Console.ReadKey(); program.Foo(); Console.ReadKey(); } public int Foo() { int y = 15; Console.WriteLine("Foo Called!"); return y; } public int Bar() { int x = 27; Console.WriteLine("Bar Called!"); return x; } } }

My goal was to be able find Foo() using WinDBG.

2 - SOS and WinDBG

 WinDBG has an extension called SOS. WinDBG by itself is a great debugger for unmanaged code. You can do all of the usual debug stuff. It get's a lot more challenging when you want to debug managed code. In order to help you out with managed code, the Son of Strike extension is used to fill the gap providing manged code debugging help that lets you view IL Code, see if things have been JITed or not, set managed breakpoints, and more. 

As an interesting aside on the naming of SOS and COR for that matter which pops up in the names of many of the DLLS such as mscorwks.dll; Chris Schmich on stackoverflow provided some insight:

Jason Zander's blog post explains it perfectly:

The original name of the CLR team (chosen by team founder and former Microsoft Distinguished Engineer Mike Toutonghi) was "Lighting". Larry Sullivan's dev team created an ntsd extension dll to help facilitate the bootstrapping of v1.0. We called it strike.dll (get it? "Lightning Strike"? yeah, I know, ba'dump bum). PSS really needed this in order to give us information back to the team when it was time to debug nasty stress failures, which are almost always done with the Windows debugger stack. But we didn't want to hand out our full strike.dll, because it contained some "dangerous" commands that if you really didn't have our source code could cause you confusion and pain (even to other Microsoft teams). So I pushed the team to create "Son of Strike" (Simon from our dev takes credit/blame for this), and we shipped it with the product starting with Everett (aka V1.1).

Also, I had heard of the CLR being referred to as "COM+ 2.0" before, but apparently it's had a few names in its time (from here):

The CLR runtime lives in a DLL called MSCOREE.DLL, which stands for Microsoft Common Object Runtime Execution Engine. "Common Object Runtime," or COR, is one of the many names this technology has had during its lifetime. Others include Next Generation Windows Services (NGWS), the Universal Runtime (URT), Lightning, COM+, and COM+ 2.0

Loading SOS

Loading SOS is done by a command:

.loadby sos clr

This actually failed for me the first time I tried it. After trying a bunch of things I just about gave up assuming WinDBG was just a pain in the ass to work with, then as a last resort I decided to verify the process was x64. I quickly discovered the problem. I was running WinDBG x64 , but running a x32 application. I assumed I was running x64 on my application since I have a x64 laptop, however console applications are 32 bit. After switching to WinDBG 32 bit, the SOS extension loaded without issue.

Attaching, Setting Symbol Paths, and Source File Paths

So after I had this all setup I realized I didn't have any symbols loaded. Symbols are found in program database files (.PDB). These files can contain different levels of information that can allow more granular debugging. The PDB file for my program was generated with the program FooBar.exe. as FooBar.pdb in the same output directory. In order for symbols to be found for my program and all of the Microsoft ones too I had to set them up.

First I went to the File menu in WinDBG and set the Source File Path to the bin directory of my created application. I'll need to investigate later exactly what this setting is for. It may be what actually finds the PDB file for my application

Second and probably more importantly I set my Symbol Path to the following:

SRV*C:\Users\Josh\Google Drive\Playground\WinDbgMem\FooBar\ConsoleApplication1\bin\Debug*http://msdl.microsoft.com/download/symbols

Some people recommended defining a general symbol path to store all symbols locally. I found with the above setting the Microsoft Symbols for the .NET Framework DLLs I was using were getting loaded and cached into the bin directory of my application. Notice the Microsoft symbol server line at the end will ensure we can pull any available symbols we don't have from the Microsoft symbol servers.

At this point I am basically ready to attach. I goto the File menu and click "Attach to Process". I find FooBar.exe in the list and attach. At this point the debugger attaches itself and gives me a console.

I type the SOS load command:

.loadby sos clr

and get back a blank line indicating it probably succeeded. The .chain call allows me to see loaded extensions to WinDBG. I typed it out:

.chain

And get back:

Notice that I typed the command wrong the first time and got back an error. Then I typed it correctly (.loadby instead .load) and it loaded. The .chain command now shows SOS in the extensions list. We can now use SOS.

Finally setup is complete and we can start looking at some cool stuff.

Modules

Modules are basically the generic term to refer to the unmanged and managed versions of what library or executable projects are. The first thing to start with when looking at a program in WinDBG is to view the modules. For this we use the list modules command:

lm

Notice that the symbols are only loaded for a couple of these. Being the eager bastard I am I'd like to see symbols loaded for all of these. So I am going to go ahead and load the assemblies and force the symbols to be loaded by using the reload command:

.reload /f

Now we check the loaded modules using lm again:

Notice that now deferred has been replaced with pdb symbols, and that even the FooBar.exe has symbols!

Starting Down the Rabbit Hole

Let's start at the top and look at the Domains. When the CLR loads a domain three domains are actually initialized. Two of these are automatic and are not known even by the host. The other is the domain your application actually runs in. The first two are created as part of bootstrapping by mscoree.dll and mscorwks.dll, or if you have multiple processors the latter may be mscorsvr.dll [1].

The system domain sets up the shared domain and the default application domain and loads mscorlib.dll into the shared domain. Remember from the aside earlier that "cor" in these library names refers to the Common Object Runtime, a synonym for the Common Language Runtime (CLR).  The system domain also handles string interning and setting up/tearing down app domains [1].

The shared domain is where common code is loaded.  User applications can be loaded in this domain if they are loaded as domain neutral. ASP.NET apparently is supposed to do this by default, but I have yet to see it in practice. However, there are some nuances around "binding closures" though that may be preventing applications I've worked on from taking advantage of it.

The default domain is where my application will run!

Extensions to WinDBG follow the convention of using the exclamation mark to denote an extension command. The first one we will use is:

!dumpdomain

Notice that we see the three domains we were just talking about, and we even have some sexy looking memory addresses. The domain I am interested in here is "Domain 1" since it holds my application. I can use that memory address next to my "FooBar.exe" module to get some more information about it using the !dumpmodule command (-mt wil provide some more information on the types):

!dumpmodule -mt 00ed2ed4

So now I know where the method table for FooBar.Program is! Let's take a look at it using !dumpmt -md (md of course adds more detail):

Notice we have some information here about if the method was precompiled (PreJIT), already has been compiled as part of Just In time compilation (JIT), or has yet to be compiled and still contains a JIT stub for compilation (NONE). This is pretty cool!

The second column is the address of the Method Descriptor. We can now find the IL for the method using the !dumpil command and passing the Method Descriptor address: !dumpil 00ad379c

That looks the IL code for the main function! So this is pretty neat, but it would be even cooler if we could see the code for a function directly in memory. Let's grab Foo's IL though since we that integer in there, a string, and a call.

As an aside the nop are often used to have an address for breakpoints when they are set on brackets in the source code.

Notice the ilAddr line. Let's open a Memory window (View Menu > Memory) and then in the address field let's type that iladdr number:

Now if you look at the IL Code and then reference it against the opcodes for the CIL instructions you can start to see where the IL is in memory. Below are small color divets indicated some of the first mappings. I stopped at the string because I was tired and unsure how to interpet it. I'm guessing the string "Foo Called!" is stored at a memory location and that what follows what I have so far is a memory address.

So with that we have successfully found the IL code in memory. Hopefully next time we will start to able to tweak and manipulate some of these values.

Using Wikipedias CIL Page against the above opcodes:

00 - nop - no operation, often used as placeholder for breakpoints on brackets in IL/C#/.NET.

1F - ldc.i4.s <int8 (num)> - push int32 number onto stack as short

0F - in this context is just the number "15"

0A - stloc.0 - pops value from stack to local var 0

72 - ldstr - loads a string

Probably address to string

... and so on

For the past year on the side in my free time I have been working on building what will eventually be a small Villa at my parents house. This project is an exploration of carpentry and all that goes into building a house. A few pictures follow:

Into this project I plan on doing something interesting. I thought it would be interesting to be able to control the house from my phone. I was originally going to use a Rasberry PI board to accomplish this, but due to availability issues I ended up using a Arduino Uno R3 instead:

The Arduino will be the "computer" driving the house. It will talk to the outside world, and decide what circuits should be turned on and off. Unfortunately I can't just jam 10 amps of 110AC into this bored without potentially starting a  myriad of forest fires. In order to control the circuits I need a way to control larger AC voltages with smaller DC voltages.

Relays are a great way to accomplish this. At first I was looking at mechanical relays, but these had two issues. One is that they are noisy. I don't want to hear clicking noises every time a circuit turns off and on. The other problem is the ones I looked at were on at low voltage. This is undesirable if the system fails in some way as it would mean lights and other services would be stuck on.

To address this I ended up going with an 8-channel solid state relay. These are quiet and the particular model I looked at was off at low voltage. You can see it working below:

I have a temperature/humidity sensor on the way. This will provide me with data to be able to figure out whether heating or cooling circuits should be activated. I also have an Ethernet shield on the way which will allow me to plug this board into my home network.

Communicating with my Phone

The idea is that I can forward and route a public port on my router directly to the UNO. I can then develop a phone application to communicate on this public port. With this system I should be able to turn lights on and off, and adjust temperatures all from my phone, while sitting on the couch.

Wall Mounted Color Touch Screen

I also plan on eventually using a color touch screen that will sit in the wall, and allow users to set temperature schedules and turn circuits on and off. (http://imall.iteadstudio.com/im120417020.html) The TFT touch screen I am looking at has a physical mounting issue, but this is easily alleviated with pin header extenders (http://www.amazon.com/gp/product/B004G56J8W).

I was hoping to spend more time working close to the hardware on firmware, but the ADK is too convenient. I'm still trying to figure out a good reason to do some kernel work.

Ethernet Shield: http://www.amazon.com/gp/product/B006UT97FE

Temprature Sensor: http://www.amazon.com/gp/product/B0066YD3GM

Tomato Potato,

I was having a discussion about a week ago with some other students, and a question came up in class. "Why don't we just use a static class?"

The professor replied: Static class?

I elaborated on the original question: In C# we are able to use a static class to create a non-instantiable class with only static methods. Java has static methods. Why not static classes like C#?

The professor, a little puzzled, replied: A standard idiom is just to have a private constructor and only have public static methods.

He then moved on, paying little more attention to the question, sufficiently satisfied that the discussion itself was addressed.

Why doesn't Java have static classes?

Well to answer this question we need to have a bit more enlightenment about what a static class is in C#.

The .NET family of languages compile down to a byte-code coined IL Code. It is like an assembly language with object oriented support, that eventually get's compiled and cached on the fly into native assembly. There are numerous benefits to doing things this way, but that's outside the scope of this article.

So when we write C#, the C# compiler isn't converting our code to native assembly, it is converting it to IL Code.

Lets look at what a static class looks like in IL Code

Notice that ILCode the static class gets compiled into 

There's a couple things to notice. First is that there is no static modifier. The second is that there is now an abstract keyword and a sealed keyword.

An abstract modifier means that we can't instantiate the class. A sealed modifier means we can't inherit from the class. So a static class is, glossing over a couple small details, a non-instantiable class that cannot be inherited from.

It's a hop and a skip over a small but dangerous creek, to realize that in Java we could probably get away with the same functionality by using an abstract modifier with a final modifer or what may be the more standard idiom of making a private constructor on a normal class, with a final modifier in the class declaration.

static class could be loosely translated into Java as abstract final class

Your self appointed ruler of the world,

Why Build a Boot-Loader?

I've been needing something outside of what I do as a job to invest my more eccentric energies. So I have started moonlighting on the side of operating systems and embedded systems. My first self-proposed project was to learn what a boot-loader was, how it worked, and how to build one. After encountering GPT partitions in my research I was also curious where something like GPT fits in. GPT is something you might have run into as a requirement when you try to format a hard-drive more than 2 TB in size, as MBR only supports up to 2 TB of space.. Having achieved the goals and answered the questions listed above I am now writing about it

How Easy is it to Build (and Test) A Boot-Loader?

It was actually pretty easy. There are many guides out there that made it quite enjoyable. I am even more excited to say that I tested the boot-loader with a very low level, but easy approach that doesn't seem to have been done before, that is from the light research I did when I was trying to achieve it. This method basically consists of writing the boot-loader directly into a VirtualBox HardDisk by hand.

Boot-loaders themselves are actually quite simple to build. As long as you follow a few rules you have a valid bootloader.

Must 512 Bytes in Size

Must have a magic number signature at the end of the 512 Bytes  0xAA55h or 0x55AAh depending on weather your system is little endian or big endian.

Must be compiled as 16 bit code if you are using an assembler.

The processor actually starts in 16 bit mode and is in 16 bit mode when your boot-loader code is called. Somewhere in your code you actually have to switch it to 32 bit in order to take advantage of 32 bit functionally. The terminology for these two states is real mode and protected mode.

When compiling with GAS (the Gnu assembler) I had problems. I couldn't get my generated binary/HEX to have the correct ending signature of 0xAA55h. I switched over to NASM and everything seemed to work well.

How Do I Test the Boot-Loader?

I have a net-book, and I don't have any spare USB drives. This means I couldn't write my boot-loader to an actual piece of hardware. I also didn't want to break my computer. These constraints meant I needed to use a virtual machine, and either create a floppy disk image or write the boot-loader directly to disk.

I really liked the idea of writing my boot-loader directly onto the VDI by hand. It seemed like this might help solidify how things work and where things are actually on the disk in my mind. It's still abstracted when you use a tool to write a boot-loader to the disk, I mean who knows what all that tool is doing? When you do it by hand though, you know exactly what is on the physical media, and what needs to happen to make a boot-loader work.

There aren't any guides that seem to tackle this problem in this way so I had to do a little bit of investigation and discovery. 

Looking at the VDI specs I found that it has a 512 Byte header. I figured the disk image is supposed to start right after that. I found, however, when writing boot-loader into this address space that it didn't work.

I tried a couple more times failing, and then decided to just launch a LiveCD iso image I downloaded and run cfdisk on the partition.

Cfdisk is the graphical version of fdisk, and basically lets you easily create partitions. It also lets you mark them as bootable. Marking them as bootable adds that AA 55 marker I was talking about earlier. So I just searched for this marker in the VDI file (after running cfdisk above), and low and behold I found a basic MBR with some partition information. I simply went to the offset 512 bytes prior to the marker, turned on insert mode, and pasted my assembled boot-loader from one window to the other. To my astonishment it worked!

For my environment I used the Linux distrobution JolliOS, which is ubuntu based.

Tools used:

NASM

Bless Hex Editor

VirtualBox

I also read a tool called bochs, a PC-Emulator, works well for testing boot-loaders. I toyed with it a little, but after only a little trouble decided to pursue the virtualized approach instead.

Next Steps?

My next objective is to get the processor into 32-bit mode and load in my own kernel.

Weight ROI towards Quality

In business there seems to be a tendency to migrate torwards asking the question "How can we deliver a decent product more quickly with the same resources?" Of course with the assumption that more product means more value, and more ROI.

I think the question should instead be "How can we deliver a quality product without the extra fluff  and in the most productive way possible?" We need to look more at the ROI of quality. We have to be careful with this though. These are dangerous waters.

This goal implied by this question doesn't mean spending more time on over-engineering, excessive design, or slowing development. This goal implies taking time to develop existing code well, paying down technical debt, and loosening the expectations in length of development; but what does this really say? Basically  I am making the point that management should be desiring a Quality product and ensuring that plenty of time is allowed for the product to be of certain level of quality. Management should not be squeezing deadlines or making teams feel pressured to deliver faster, they should be encouraging them to develop a product thoroughly. In return developers need to be more efficient and avoid wasting excessive time on quality that provides little value (Pre-optimization, over-design, beautifying code, etc.). 

Avoid Burning Your Creative Process On Improvements With Small Returns

Often matured developers tend to want to spend significant amounts of time on optimizations, generalizations, and design under the warrant of craftsmanship, pride, and aspirations of creating a perfect product. These aspirations are noble, but when these aspirations are engaged prior to a demonstrated need they are short-sighted and lacking in wisdom. Creating a perfect product is impossible. You can't build perfection on the first go around no matter how much you decouple, open-close, or compose. We need to stop trying to achieve perfection as developers. There is a cost to perfection, it is high, and most importantly perfection is short-lived in a world of changing needs. 

Remember Why You Do What You Do

We are in the business of providing value. This value is manifested in two forms. One form is of course the stakeholder or what is sometimes called the client. Producing value for the client means producing something they can use to achieve their goals. Our code also provides value both to and via future developers. This future developer may very well be you six months down the road. Well designed extensible software is wonderful. It makes everyone's lives easier. The value here though is an indirect return, and often diminishes as requirements shift. You can make the code as shiny and pretty as you want, but at a certain point you are providing very little value at the cost of massive amounts of time.  There is a diminishing return to your coding brethren.  The direct value you provide is in providing a functioning easy to use product for the stakeholder; always remember this value is most of your value. I believe you should get to that goal as quickly as possible while sticking to your coding morals. Design but don't linger on design, you need to ship, and design can always be re factored. Optimize code if convenient, but don't go out of your way. Avoid investing yourself into things that do not have reasonable answers.

Our Reinvented Wheels Are Often Squares

We should also avoid rewriting software. If the first developer did a bad job with a solution, how likely is it that we can really do better; especially considering the existing software may have years of fixes and usage represented in it's code base? We need to improve code to a better product. It's easier to write code than to read and understand code. It's the easy way out to write new solutions, and often we build a worse solution. Do the work to reap the insights from an existing solution both via it's shortcomings and its time-tested working parts.

Time and Focus are Limited Resources

We have a tendency to think there is always more time. You learn the more you mature the lesson that both focus and time are limited resources, and as you build more complex products both of these become more precious. Avoid falling into the trap of always thinking you can do something more in your extra time, or that you can just "buckle down" and get more done. Be reasonable with yourself, and stop deluding yourself into thinking there is always a "later".

May the Source be with You,

Hi you. I'm going to give you a task. I want you to write an application to allow users to submit feedback about The Company....

Oh and while your at it I would like you to make sure users are logged-in....

Also users should only be allowed to submit one feedback comment....

Oh and they should only be able to submit feedback on Tuesdays after we provide them with free updates, and send every fifth comment a gift basket.

Oh and make sure it's bug-free. 100% maintainable (whatever that means), and that it can make me coffee.

One last thing....you need to have it on my desk tomorrow or your fired.

Requirements are refined over time. Time in the end is what matters. Nearly every decision has time as a motivating factor. It's a very sad truth, but a sad truth nonetheless. So where does optimization fit into this?

As software developers we are tasked with wearing many hats, and solving some incredibly complex problems. Ideally we would make a perfect product. That's not possible. We have limited time, and we need to make maintainable ,flexible, adequate products.  The problem with optimization when done wrong is it costs time. It doesn't only cost time when optimizing. It frequently costs time in future maintenance cost. 

Optimization often destroys clear consistent code, and makes it less consistent. Optimization by nature usually requires solving a very specific problem with very specific logic.The problem with very specific logic for very specific problems is it is different from generic consistent logic.

Now before I lose you, Optimization is necessary. Your product needs to perform well. The problem is when you spend 40 hours blindly optimizing your product turning that beautiful generic CRUD code into a lean performance beast for a net gain of 1% performance when 99% of your time is spent in travel across the Atlantic to your servers in another country.

In order to optimize you need to profile the code. You need to know where performance costs are. Blindly optimizing and destroying consistency and maintainability in the name of performance kills kittens every day, and leaves the real performance monsters in the closet.

Unless it's really a problem and you know the performance profile, save the kittens and don't compromise design and consistency for performance.

With a Single Tear,

I am a fan of Jon Skeet. If you don't know who Jon Skeet is and write C# you probably haven't immersed yourself in the community much. Jon is not on the C# language team, but he is prolific on answering community questions and understanding every corner of the language in ways the normal mortals like us cannot approach. He has the highest reputation on StackOverflow. In addition, Jon is a phenomenal communicator in regard to concepts, and he should be especially considering he has answered almost 20,000 questions on C# asked by green programmers like myself. 

So when Jon writes articles I usually read them. I've read two of his books. So when I found out he was doing a C# Mastery Video series I couldn't resist watching even the basics.

Let me jump gears for a moment now that I have convinced you of the pure awesome that is Jon Skeet ....ahem the Chuck Norris of languages...let me give you a little more context on optional parameters before we get into the meat of this post.

What are optional parameters and how do they work?

Optional parameters allows you to specify default values for method parameters. Optional parameters are implemented by embedding the default parameter and value as a metadata attribute on the method in the compiled code. Callers of methods using optional parameters must know to use this metadata if it exists to make use of "optional parameters". I put optional parameters in quotes because optional parameters are sort of "faked" in that the caller looks at the default values and substitutes those into the call to the real method. This means the method being called no idea about the optional parameters. It just receives the optionals as normal arguments. The caller is what looks up the information and makes the substitutions. 

So what happens when you use optional parameters is the compiler if it supports optional parameters looks at the method you are calling and sees if it has optional parameters and then sticks those in for you when you don't supply them.

A big catch to be aware of with optional parameters is what we call a versioning problem. This means you need to be careful when working with different assemblies to take into account the above information when you have client assemblies using assemblies with optional parameters.

Overloads vs. Optionals?

Now you can implement optional parameters or alternatively use overloads. Why would you want to use one or the other?

Overloads [in contrast to Optionals]

Cons

More code to write [and sometimes this is alot of code - just 4 optionals covers 24 overloads]

Documentation must also be duplicated [if documented properly].

Doesn't comply with DRY as much as optionals

May be more code to maintain and less flexible due to needing to change every caller overload when a base method is altered.

Pros

Adding an overload never breaks the client assemblies using the old calls when deploying assemblies seperately.

As an extension of the above you don't need to worry about versioning issues.

Sometimes seen as more object oriented.

Can be cleaner and lead to smaller methods breaking functions into extensions of functionality and the base functionality.

Optional parameters basically have the inverse of this with one more interesting twist. When combined with named arguments they become quite powerful.

I never thought of named parameters going hand in hand with optional parameters prior to seeing Jon show them in this way in these mastery videos I was watching on TekPub. Then I realized hey this is pretty cool.

See the tricky thing with optional parameters previously is that if you wanted to supply an argument in the middle of a bunch of optional parameters you actually had to supply all of the arguments up to that optional parameter. Named parameters allow you to supply any arguments you want to supply in any position. This is simply a fantastic result of these two features being used together.

Optional parameters come with gotchas in some cases. Overloads do not.

Why Optional Parameters can be Better

Personally I don't really see a problem with using optional providing clients outside of your control are not making direct use of the code base. Optional Parameters are so much cleaner than 3 or 4+ overloads. It's really annoying when you open up a class and see 16 overloads, and even more confusing when code is split up all over the place. Optional parameters would break this down into one method. In addition the construct makes for something much more readable and immediately understandable than overloads. More than anything optional parameters remove extra unnecessary bloat and extra code to maintain.

It is worth emphasizing the named parameters alone are a great feature and don't require optional parameters to be used with them.

It would be an interesting exercise to write an extension to the compile process that transforms optional into overloads (thus making optional parameters simply an alternative cleaner form of defining overloads). I am actually a little curious why the actual process doesn't work this way.

Many moons ago I created a devious little function....For some time I loved it....it was so....elegant....then after a staunch "come to Stallman" talk with a Senior Developer I came back to the light and shunned this little function. Lets call this little function FindOrCreate. 

It was Genius. If something wasn't found return a new thing. After all in most cases if we can't find something we probably want to make a new something right?

Well it worked. But after this talk....well it didn't look so great anymore.....I realized it wasn't a good idea because....well I don't really remember why...

So at that point for some reason I can't remember I started seperating Find and Create.

Then something interesting happened. I saw another developer do it. After some discussion I decided a change mental model was in order.

There is a polar opposite to FindOrCreate and it's UpdateOrInsert. Ironically I settled and fell in love with UpdateOrInsert (we referred to it as Save). It was wonderful to just Save something and know it got taken care of. I didn't need to worry about whether to Insert or Update. Yet I was against Find or Create. Why? Because callers of the function are querying for an item and here we are creating a side effect creating a new item. This is a basic violation of CQS or CQRS if you prefer. Command (Side Effects) and Querys (Get Stuff) should be separated, because it makes reasoning about the function easier. 

What I never considered is using FindOrCreate and UpdateOrInsert at the same time. I still need to play around with this idea, but I see now reason to change my mental model of persistence to simplify down to "Get me the representation of this Item" and "Persist the Representation of this item." Combined with DDD I find this concept  quite interesting, and I notice this is how some ORMs seem to function.

I am noticing CQS' importance in the controllers. We are returning partial views back from some controller actions that update items in order to make only one call to the server. This is combination of Command and Query.

I am finding it annoying with the two combined in the controllers because sometimes you want to get different return values (Views)  but execute the same action in different parts of the application. 

However, I need to do some more thinking on perspectives on this.

I hate Regions in almost every situation they are used. Every time I look at them my eyes begin bleeding profusely, and then I need to clean my desk because there is blood all over the place. Regions are almost always unnecessary. Frankly they get in my way when it comes to daily coding. Things that get in my way frustrate me. There is a few cases where I think Regions are appropriate, but I will get to that at the end. Most of the time they seem to be used to try to organize everything.

The first thing I see when I open a file with Regions to the extreme is a nice beautiful collapsed set of code grouped together.

It looks neat and tidy. There is just one thing....it HIDES THE DAMN CODE. When I open a class it's not to look at something pretty. It is to get in and do some work. Regions immediately get in the way.

"But Josh", you say with a smug smirk and then continue, "It groups the methods together and it allows to see just the sets of methods you want to work on."

No. Just no. Grouping and sorting methods can be done without regions, and regions get in the way more than they help. Nothing is more annoying than when upon cursory scan of the code my eyes get caught up on these ridiculous C++ Preproccessor Macro looking lines of code in my beautiful C#.  Regions when expanded are extra noise.

Without Regions it looks so much better:

Now with all this said I am a huge fan of an Extension called Regionorate. If you develop in Visual Studio I highly recommend using it. Surprisingly it offers an option to automatically sort and organize all methods without regions (in addition to supporting regions.) I find it to organize the code in a fantastic way that improves the accessibility and the visual cues of the organization of the code are perfect. It produces all the benefits of regions without using regions. 

Now with all of this said, I do find Regions are occasionally a nice goto for organization, but only for rare cases. One example I ran into recently is when doing some unit tests for some complex date logic. For these unit tests it was nice to be able to set up scenarios with specifc dates. I had about 7 or 8 dates I needed "fake" as the current date to test against some nasty logic that ran against Start and End Dates. In order to clean up the tests I created date variables with names like "Jan15", "Feb1", "Jan14", "Jan20", etc. that had a DateTime object created with the proper day,month, and year for each one. This allowed me to refer to dates using these nice little handles. I used these dates in all my tests so I pulled them from the methods into the class such that they were common class fields. The only thing that sucked is that you had to browse a couple screens worth of code to see anything interesting. This logic was so dead simple and similar that very little value is benefited from seeing it. Perfect Candidate for Regions. In this case the Region cleaned up the code nicely. Mid to large size sets of the code the programmer usually does not care about I find to be the best candidates for Regions, because in this case they provide value by cleaning up alot of "noise" for the cost of adding a little noise when they are expanded.

It is my zealous opinion that Regions should not be used as a rule, but as an exception.

If you think you know MVC and View Logic you can probably skip down to the actual controversy. I have believed this to be an easy sell to put View Logic in Views, however in practice it has been quite difficult.

What Exactly is View Logic?

I have seen a tendency for many developers to put view logic inside controllers. What do I mean by view logic? Lets take an example:

Requirements

If I am a moderator I should see a button letting me remove an inappropriate post.

I should not see this button if I am some random internet hobo.

See that "Remove From Internet Button"?

Well if I am a casual internet fellow I should not see that button. Whether that button shows or not is logic. Furthermore it is view logic.

Where MVC Fits in?

MVC is an acronym for Model View Controller. Isn't that fun. For this post we don't really care even in the slightest of the slightest what the Model is, but if you really want to know it is where all logic that isn't user interaction related goes.  

The Views have the direct responsibility of displaying information to the user. This includes all the fancy whiz-bang we as developers like to use like "textboxes" and "buttons".  

So how do we display a button for only certain power-hungry users like "Remove Inappropriate" post above?

One solution is painting directly on the screens of only these elite few. This has the overreaching effect of providing the button on every site they visit. Surely we don't want to give them that kind of power!

The clever solution decades of computer science has provided for MVC architectures is choosing to Render the markup only if a condition is met. "If a condition is met" - that sounds like an if statement! So this is how we miraculously turn buttons on for some users and not others. We simply check what kind of user they are and then render the markup for the button if they meet the condition(s) for it to show.

Now I will quickly mention here what the controller does. The controller's job is to take requests in get needed information from the model, and then provide it to the view. The controller simply retrieves and provides information based what the user requests. This information is seen through the perspective of the view.

The Controversy

What actually determines if I can see a "Remove Inappropriate Post Button"? Almost certainly this is determined by my current privileges (permissions) on the site I am on. My current privileges are provided by the model. 

So the controller gets the privileges of the user and at this point we can go one of two directions:

Check the rights in the controller and then send a boolean to the view telling the view to show or not show a particular button. This has the benefit of making the views thin since the view simply checks a boolean to see if it should show the button or not.

Simply pass the rights to the view and put the logic in the view. This has the benefit of thinning down the controllers at the expense of adding a small amount more logic to the views.

Frankly I view this logic as View Logic and therefore it belongs in the views. Adding new functionality is made much easier since the likelihood of adding new buttons and doo-dads to show or hide will use the same type of logic is high.

New logic like showing or hiding a button if put in the controllers means modifications in the Views, View Models, and any Controllers using the view...

...in contrast to modification of the view alone.

There is a fundamental trade off and it comes down to where the complexity is. I argue and believe the complexity increase is marginal when this logic is in the views and significant when put in the controllers.

As a consequence in general, maintainability suffers when view logic is put in the controllers.

What about Helpers?

What about them? We can use helper methods to encapsulate more complex view logic in controllers, but we can also do the same in Views.

What about unit testing?

UI concerns are not often unit tested anyway, but helper methods are always an option even in the views.

Is it really worth moving the logic out of the controllers?

Three  design principles come to mind: 

Separation of Concerns, Single Responsibility Principle (SRP), and Don't Repeat Yourself (DRY). All three of these are less satisfied by having view logic in the controllers. It is true now the Views have a little more responsibility, but that responsibility already exists as a concern since at the very least you need to check Boolean "flags" anyway.

I'll end this post by saying I think it's plain ugly to have view logic in the controller. It makes them fatter and seeing "IsShowX" and "IsXVisible" all over the controllers screams dead-kittens to me.