Why writing docs and tests makes our code better

I’ve written a lot of Python code in my career; I’d guess over 10,000 lines per year for each of the last eight years. Twiggy is the first project on which I’ve taken the time to do everything right. In the past, that’s mostly been due to the demands of bosses or business (startups are not great for taking one’s time), but I’ve had my share of projects that I just haven’t followed through on.

So I’m taking this opportunity to reflect on what makes good code into great code. I’m not interested in what distinguishes mediocre code from good – that’s often a matter of experience, education or aptitude. I believe that anyone who can write good code can write great code, and want to explore why our projects so often fail to live up to their full potential. I can only hope these observations prove useful or at least thought-provoking.

We already know

For code, documentation and tests separate the good from the great. No big surprise there; we already know we should be writing docs and tests. What’s been less explored is the effect that writing docs and tests has on the code itself. Along the way, I hope to offer some insight into what good docs and tests look like, and why we don’t write them more often.

Code that can be documented and tested is more likely to be good; the act of writing those docs and tests tends to transform it into great code. We don’t do this more often though, because it’s time consuming and difficult.

Tests

A metric

Learning to write tests changed the way I structure my code; it’s cleaner, units are more independent and gosh, everything’s just more… testable. Even code that never gets tested is better because it’s written in such a way that it could be tested. The various parts are well contained, better defined, more extensible and replaceable… basically, all those adjectives we want code to be, but don’t have a way to measure. So let me propose Fein’s First Metric:

Well structured code is code that can be tested.

Testing sucks

I’m a test-early guy, not test-first. During the initial braindump stage of a project, things are flying around too quickly for the tests to keep up: methods getting refactored, classes split in half, idioms invented and rearranged. I don’t want to break the stream of ideas by maintaining a test suite that’s going to be 80% out of date every two days. That’s how I do things – if you like to write your tests first, good (and you’re probably working on mathematical or otherwise very functional code).

In the early stages, dynamic language programmers tend to “test” as we go, using the interactive shell and small ad-hoc driver scripts. These are what I call “smell tests” – does the code feel right, and do what it’s supposed to do? (Not to be confused with smoke tests, which is when you execute your code, and if it doesn’t crash, you run around yelling “Fire!” and then ship it).

That means by the time we settle down to writing our test suite, we’re almost always testing code that we know already works. That sucks. It’s boring, it takes a long time, and it sucks. For Twiggy, I have almost twice as many lines of test code as lines of code code, almost all of them written around the same time. Far and away the least fun part of developing.

Coverage: escaping the suck

Coverage is a rope out of the testing suck-hole. Using the reports, it can turn testing into a little game of inching your percentage up. Make sure to test a single unit of code at a time; otherwise, it’ll give higher numbers than you deserve, as modules import and use each other.

Still not much fun, but at least it’s a way of measuring progress. Reward yourself with an ice cream, or a YouTube video or Facebook check every 10% or module. Take the time to really cover every branch and line, including those trivial ones we know will work. Attending to those details is hard when we’ve been slogging through tests for days, but it takes little time and energy by comparison. Truly full coverage gives us the confidence and ability to quickly make changes to the code later on.

Documentation

We know documentation is important, even for programs in easy to read interpreted languages. What’s less appreciated is that writing documentation makes our code better too.

Writing good documentation is hard. It’s incredibly time consuming – on Twiggy, I probably spent half as much time on the docs as on the code (the tests, perhaps less than a quarter). The raw restructured texts are a third more bytes and twice as many lines as the code itself.

Sphinx is awesome here – it supports all the different kinds of documentation we need to write, can integrate docstrings from source, validate code examples, includes in-browser keyword search, etc.. It also produces docs that look good. Easy cross referencing of documentation objects via a simple syntax that “just works” lets us create docs that are more than an over-glorified print manual. Use these heavily; your users will thank you. Heck, you will thank you in a few months when you refer back to your own docs instead of browsing the source.

I still find Restructured Text a little awkward at times; but it’s hard to imagine how a general-purpose markup language could be simpler without dropping features (at which point, just use Markdown).

I print my docs as I work on them, often. I think I killed half a tree while writing Twiggy’s documentation. Taking a pen to paper helps me focus on one section at a time, while improving the overall flow. There’s something about seeing your documentation in all its fully-formatted glory that causes the changes you need to make to pop out at you.

API

API docs mirror the structure of the code – a fairly straightforward explanation of methods, classes, argument types, etc.. They’re often just the docstrings:

def frobnicate(x): “”” :arg int x: how hard to frob “”“

Docs like this don’t really tell readers much, and aren’t adding to our understanding. They’re still necessary as a reference and the minor details are important, but at best, they save us from having to re-read the code every time we want to use it. Most projects, especially non-public ones, stop here.

API docs are the documentation that programmers write for ourselves. They’re the absolute minimum necessary for another person to use your code, but they don’t give a reader anything to grab on to if they aren’t intimately familiar with the project to begin with.

Reference

Reference documentation is higher-level. It describes how to use the features of an application or library to accomplish particular tasks. Use cases, basically. Reference docs are well suited for readers who are familiar with the problem you’re working on, but not your specific solution. Most documentation for open source projects consists of reference docs.

Occasionally, writing these docs will lead to ideas for new features, or point out problems with existing ones.

Narrative

Great documentation tells a story. Narrative docs explain why to a hypothetical reader who not only has never seen our code before, but has never seen anything like our code. Unfortunately, programmers are generally bad at narrative – that’s why we write code instead of fiction. Writing these docs requires getting outside your head and thinking like a total newbie. That’s tough when we’ve just spent weeks or months working with the code – we’ve got no perspective.

Taking the time to write these docs reveals ways that the code could be cleaner, simpler, easier and more intuitive. You’ll change your code so you can tell a better story about it. Unlike API or reference docs, there’s no existing structure to organize around. So I often begin there – what are the important points I want to cover? A phrase or short example is often enough to start – the details get slowly filled in as I come back and iterate.

Giving talks helps here, and not only for the feedback from a live audience (blank stares vs. nods). A presentation forces you to explain your project concisely and clearly to audience that’s there for the pizza and beer. The shorter the talk the better – as a speaker, you’re not going to learn anything by taking an hour. Thirty minutes max, and I’m a huge fan of the five minute lightning talk. But that’s a subject for another post.

Great takes time

If you’re reading this, you can probably already write good code (hey, I know my audience). In my opinion, that means you can write great code. Doing so takes time – and not necessarily where we expect. By writing tests and narrative documentation, we gradually discover how our code can be improved. That process is often harder and takes longer than we would wish; but the great code that results is the reward.