Avoiding the billion dollar mistake

System.NullReferenceException - Object reference not set to an instance of an object.

This is by far the most common error message seen by C♯ developers the world over, and unfortunately, often seen by end-users too. The legendary null reference has caused countless headaches and dogged the lives of hapless developers for years.

I call it my billion-dollar mistake. It was the invention of the null reference in 1965... This has led to innumerable errors, vulnerabilities, and system crashes, which have probably caused a billion dollars of pain and damage in the last forty years.

Tony Hoare

In this post I will present a few strategies for dealing with this age old hazard.

Defensive programming

More specifically preconditions - also devised by Tony Hoare. Adding predconditions to guard against null references on method arguments is always a smart move. This ranges from the most simple means of if-then-throw:

if (argument == null)
    throw new ArgumentNullException("argument");

... to slightly more elaborate mechanisms such as CodeContracts, an implementation of DesignByContract for the .NET platform:

Contract.Requires(argument != null);

Regardless of the mechanism used the goal is the same, to replace runtime NullReferenceExceptions - which are extremely hard to debug - with a slightly more useful exception such as an ArgumentNullException, which at least gives some indication of what went wrong & why. An ArgumentNullException is always more preferable than a NullReferenceException.

The tester-doer pattern

This is a pattern which most C♯ developers will be familiar with, though perhaps didn't realise it had a name. A canonical example of this pattern in .NET is the TryParse methods on the numeric primitive types. The crux of the tester-doer pattern is that a method returns a boolean value indicating whether the operation succeeded or failed, rather than relying on returning a null reference to indicate faliure. Take this simple contrived example:

class WidgetRepository {
    public Widget FindById(int widgetId) {
        return _db.Widgets
            .Where(x => x.Id == widgetId)
            .SingleOrDefault();
    }
}

This method, by virtue of the call to SingleOrDefault, returns a null reference if no Widget exists with the specified id. It is embarassingly easy for consuming code to overlook this possibility and assume that a valid Widget object has been returned, inevitibly leading to a NullReferenceException.

var widget = _widgetRepository.FindById(widgetId);
widget.DoSomething(); // ERROR: NullReferenceException!

Using the tester-doer pattern we could re-write the method like so:

class WidgetRepository {
    public bool TryFindById(int widgetId, out Widget widget) {
        widget = _db.Widgets
            .Where(x => x.Id == widgetId);
            .SingleOrDefault();
        return widget != null;
    }
}

By introducing the boolean return value we are making the possibility of failure much more explicit, the name of the method alone is much more indicative that the output Widget is not guaranteed. Developers calling this function are much more likely to confront the very real possibility that the output of the function may be null.

Widget widget;
if (_widgetRepository.TryFindById(widgetId, out widget) {
    return View(new WidgetViewModel(widget));
}
else {
    return new HttpNotFoundResult();
}

The null-object pattern

The null-object approach, which is a specialization of the special-case pattern, involves substituting a null reference with a real object which defines the null behaviour.

class NullWidget : Widget {
    public string Name {
        get { return "Not Found"; }
    }
}
class WidgetRepository {
    public Widget FindById(int widgetId) {
        return _db.Widgets
            .Where(x => x.Id == widgetId)
            .SingleOrDefault() ?? new NullWidget();
    }
}

This approach can be problematic for all but the simplest scenarios. It tends to create vertical slices of an application which must be aware of the null object, making it very difficult to retrofit into an existing code base. This is because it's extremely difficult for a null object to obey the Liskov substition principle. Overall I'd only recommend considering this pattern for POCO objects which have no behaviour, such as a DTO or view model.

A collection of either zero or one elements

This is my favourite approach to take when working in a team of developers who may not wish to take the plunge into monads, or when future maintainability is a primary concern (hint: that's nearly always). Conceptually, a collection which contains either zero or one elements perfectly encapsules the "something or nothing" dichotomy. Once you get used to the idea this becomes an extremely powerful concept. Using this method our repository might look like this:

class WidgetRepository {
    public IEnumerable<Widget> FindById(int widgetId) {
        var widgets = _db.Widgets
            .Where(x => x.Id == widgetId);
        if (widgets.Count() == 1) {
            yield return widgets.Single();
        }
        yield break;
    }
}

Like the tester-doer pattern we are prompting the programmer calling our method to confront the possibility that the Widget they're looking for wasn't found, without the slightly jarring syntax of an out parameter. Calling code is forced to extract the object from the return value, if there is one:

var widgets = _widgetRepository.FindById(widgetId);
if (!widgets.Any()) {
    return new HttpNotFoundResult ();
}
return View(new WidgetViewModel(widgets.Single()));

As the return value is IEnumerable we also get the benefit of being able to call LINQ methods...

return _widgetRepository.FindById(widgetId)
    .Select(widget => new WidgetViewModel(widget))
    .Select(viewmodel => new ViewResult(viewModel))
    .SingleOrDefault(new HttpNotFoundResult());
    /// Assuming an overload of SingleOrDefault which accepts a default value...

The upside of being able to call LINQ mehods is also a downside. It is still possible and perhaps even a little too tempting for calling code to simply call SingleOrDefault on the return value of our method, landing us right back at square one.

The maybe monad

The maybe monad, referred to in many languages as Option, could be considered a succint expression of all of the previous three approaches. I won't try to explain monads in any detail here as it's an extremely rich subject and there are far better resources online. It suffices to understand that a monad is simply the union of a value and an item of meta-data about the value. This is a gross oversimplification.

In this case the meta-data is a boolean indicating whether or not the value is null. Nullable types in C♯ came close to introducing this concept into .NET but their scope and usage was artifically limited to value types. A simple implementation of an option type in C♯ might look like this:

public abstract class Option<T> {
    private readonly T _value;
    private readonly bool _hasValue;
    public Option<T>(T value) {
        _hasValue = value != null;
        _value = value;
    }
}

class WidgetRepository {
    public Option<Widget> FindById(int widgetId) {
        return new Option<T>(
            _db.Widgets.Where(x => x.Id == widgetId)
               .SingleOrDefault();
        );
    }
}

At this point we could take our cue from the System.Nullable type and introduce HasValue and Value properties, in which case our calling code might look like this:

var widget = _widgetRepository.FindById(widgetId);
if (widget.HasValue) {
    return View(new WidgetViewModel(widget.Value));
}
else {
    return new new HttpNotFoundResult();
}

If this seems like a lot of convoluted code for little gain, you're absolutely right. Saying if (widget.HasValue) is really no better than saying if (widget != null). And it's still far too easy to call the Value property when we don't actually have a value, resulting inevitibly in a NullReferenceException. To fix these issues we need our Option type not to expose the value at all, and instead of exposing a HasValue property we need to internalize the branch. Both of these can be achieved by introducing a map method which exposes two callbacks, one for each possibility of "something" or "nothing":

public U Map<U>(Func<T, U> some, Func<U> none) {
    if (this._hasValue) {
        return some(this._value);
    }
    else {
        return none();
    }
}

The some callback is only executed when the object has a value, otherwise the none callback is executed. Calling code now looks like this:

return _widgetRepository.FindById(widgetId)
    .Map<ActionResult>(
        some: (widget) => View(new WidgetViewModel(widget)),
        none: () => new HttpNotFoundResult()
    );

It may not be obvious at first but we have achieved two important things here. The first great achievement is that it is no longer possible to access the value when it is null - meaning that a NullReferenceException is no longer possible, at all, ever!. The second great achievemnent is that we have forced the programmer calling our function to confront the scenario where 'widget' is missing. Since not supplying a callback to the the none parameter would result in a compile time error, not a runtime error.

Both of those properties combined help to massively improve the robustness of the code.

The difficulty is that a "maybe monad" is a very hard sell to a team of developers who are not familiar with them. The term itself is quite scary and involves a lot of concepts which are not idiomatic in the world of C♯. In many cases it's easier to introduce the "collection of either zero or one elements" which in reality shares a lot of the same properties as the maybe monad, but doesn't have the benefit of being impossible to misuse.

Maybe monad - sample implementation in C♯

public abstract class Option<T> : IEnumerable<T> {
    public abstract IEnumerator<T> GetEnumerator();
    
    System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator() {
        return GetEnumerator();
    }
    
    public sealed class Some : Option<T> {
        public readonly T Value;
        public Some(T value) {
            Value = value;
        }
    
        public override IEnumerator<T> GetEnumerator() {
            yield return Value;
            yield break;
        }
    }
    
    public sealed class None : Option<T> {
    public override IEnumerator<T> GetEnumerator() {
            yield break;
        }
    }
    
    public void Match(Action<T> some, Action none) {
        if (this is Option<T>.Some) {
            some((this as Option<T>.Some).Value);
        }
        else {
            none();
        }
    }
    
    public U Map<U>(Func<T, U> some, Func<U> none) {
        if (this is Option<T>.Some) {
            return some((this as Option<T>.Some).Value);
        }
        else {
            return none();
        }
    }
    
    public Option<U> Bind<U>(Func<T, U> callback) {
        if (this is Option<T>.Some) {
            return Option.Maybe<U>(callback((this as Option<T>.Some).Value));
        }
        
        return Option.None<U>();
    }
}

public static class Option {
    public static Option<T> Some<T>(T value) {
        return new Option<T>.Some(value);
    }
    
    public static Option<T> None<T>() {
        return new Option<T>.None();
    }
    
    public static Option<T> Maybe<T>(T value) {
        if (value == null || DBNull.Value.Equals(value))
            return new Option<T>.None();
            
        return new Option<T>.Some(value);
    }
}