We can answer on question what is VPS? and what is cheap dedicated servers?


Why doctest.js is better than Python’s doctest

I’ve been trying, not too successfully I’m afraid, to get more people to use doctest.js. There’s probably a few reasons people don’t. They are all wrong! Doctest.js is the best!

One issue in particular is that people (especially people in my Python-biased circles) are perhaps thrown off by Python’s doctest. I think Python’s doctest is pretty nice, I enjoy testing with it, but there’s no question that it has a lot of problems. I’ve even thought about trying to fix doctest, and even made a repository, but I only really got as far as creating a list of issues I’d like to fix. But, like so many before me, I never actually made those fixes. Doctest has, in its life, only really had a single period of improvement (in the time leading to Python 2.4). That’s not a recipe for success.

Of course doctest.js takes inspiration from Python’s doctest, but I wrote it as a real test environment, not for a minimal use case. In the process I fixed a bunch of issues with doctest, and in places Javascript has also provided helpful usability.

Some issues:

Doctest.js output is predictable

The classic pitfall of Python’s doctest is printing a dictionary:

>>> print {"one": 1, "two": 2}
{'two': 2, 'one': 1}

The print order of a dictionary is arbitrary, based on a hash algorithm that can change, or mix things up as items are added or removed. And to make it worse, the output usually stable, such that you can write tests that unexpectibly fragile. But there’s no reason why dict.__repr__ must use an arbitrary order. Personally I take it as a bit of unfortunate laziness.

If doctest had used pprint for all of its printing it would have helped some. But not enough, because this kind of code is fairly common:

def __repr__(self):
    return '<ThisClass attr=%r>' % self.attr

and that %r invokes a repr() that cannot be overridden.

In doctest.js I always try to make output predictable. One reason this is fairly easy is that there’s nothing like repr() in Javascript, so doctest.js has its own implementation. It’s like I started with pprint and no other notion existed.

Good matching

In addition to unpredictable output, there’s also just hard-to-match output. Output might contain blank lines, for instance, and Python’s doctest requires a very ugly <BLANKLINE> token to handle that. Whitespace might not be normalized. Maybe there’s boring output. Maybe there’s just a volatile item like a timestamp.

Doctest.js includes, by default, ellipsis: ... matches any length of text. But it also includes another wildcard, ?, which matches just one number or word. This avoids cases when the use of ... swallows up too much when you just wanted to get a single word.

Also doctest.js doesn’t use ... for other purposes. In Python’s doctest ...` is used for continuation lines, meaning you can’t just ignore output, like:

>>> print who_knows_what_this_returns()

Or even worse, you can’t ignore the beginning of an item:

>>> print some_request
X-Some-Header: foo

The way I prefer to use doctest.js it doesn’t have any continuation line symbol (but if there is one, it’s >).

Also doctest.js normalizes whitespace, normalizes " and ', and just generally tries to be reasonable.

Doctest.js tests are plain Javascript

Not many editors know how to syntax highlight and check doctests, with their >>> in front of each line and so forth. And the whole thing is tweaky, you need to use a continuation (...) on some lines, and start statements with >>>. It’s an awkward way to compose.

Doctest.js started out with the same notion, though with different symbols ($ and >). But recently with the rise of a number of excellent parsers (I used Esprima) I’ve moved my own tests to another pattern:

// => expected output

This is already a fairly common way to write examples. Like how you may have read pre-Python pseudocode and thought: that looks like Python!: doctest.js looks like example pseudocode.

Doctest.js tests are self-describing

Python’s doctest has some options, some important options that effect the semantics of the test, that you can only turn on in the runner. The most important option is ELLIPSIS. Either your test was written to use ELLIPSIS or it wasn’t – that a test can’t self-describe its requirements means that test running is fragile.

I made the hackiest package ever to get around this in Python, but it’s hacky and lame.

Exception handling isn’t special

Python’s doctest treats exceptions differently from other output. So if you print something before the exception, it is thrown away, never to be seen. And you can’t use some of the same matching techniques.

Doctest.js just prints out exceptions, and it’s matched like anything else.

This particular case is one of several places where it feels like Python’s doctest is just being obstinate. Doing it the right way isn’t harder. Python’s doctest makes debugging exception cases really hard.

Doctest.js has a concept of "abort"

I’m actually pretty okay with Python doctest’s notion that you just run all the tests, even when one fails. Getting too many failures is a bit of a nuisance, but it’s not that bad. But there’s no way to just give up, and there needs to be. If you are relying on something to be importable, or some service to be available, there’s no point in going on with the tests.

Doctest.js lets you call Abort() and further tests are cancelled.

Distinguishing between debugging output and deliberate output

Maybe it’s my own fault for being a programming troglodite, but I use a lot of print for debugging. This becomes a real problem with Python’s doctest, as it tracks all that printing and it causes tests to fail.

Javascript has something specifically for printing debugging output: console.log(). Doctest.js doesn’t mess with that, it adds a new function print(). Only stuff that is printed (not logged) is treated as expected output. It’s like console.log() goes to stderr and print() goes to stdout.

Doctest.js also forces the developer to print everything they care about. For better or worse Javascript has many more expressions than Python (including assignments), so looking at the result of an expression isn’t a good clue for whether you care about the result of an expression. I’m not sure this is better, but it’s part of the difference.

Doctest.js also groups your printed statements according to the example you are in (an example being a block of code and an expected output). This is much more helpful than watching a giant stream of output go to the console (the browser console or terminal).

Doctest.js handles async code

This admittedly isn’t that big a deal for Python, but for Javascript it is a real problem. Not a problem for doctest.js in particular, but a problem for any Javascript test framework. You want to test return values, but lots of functions don’t "return", instead they call some callback or create some kind of promise object, and you have to test for side effects.

Doctest.js I think has a really great answer for this, which is not so much to say that Python’s doctest is so much worse, but in the context of Javascript doctest.js has something really useful and unique. If callback-driven async code had ever been very popular in Python then this sort of feature would be nice there too.

The browser is a great environment

A lot of where doctest.js is much better than Python’s doctest is simply that it has a much more powerful environment for displaying results. It can highlight failed or passing tests. When there’s a wildcard in expected output, it can show the actual output without adding any particular extra distraction. It can group console messages with the tests they go with. It can show both a simple failure message, and a detailed line-by-line comparison. All these details make it easy to identify what went wrong and fix it. The browser gives a rich and navigable interface.

I’d like to get doctest.js working well on Node.js (right now it works, but is not appealing), but I just can’t bring myself to give up the browser. I have to figure out a good hybrid.

Python’s doctest lacks a champion

This is ultimately the reason Python’s doctest has all these problems: no one cares about it, no one feels responsible for it, and no one feels empowered to make improvements to it. And to make things worse there is a cadre of people that will respond to suggestions with their own criticisms that doctest should never be used beyond its original niche, that it’s constraints are features.

Doctest is still great

I’m ragging on Python’s doctest only because I love it. I wish it was better, and I made doctest.js in a way I wish Python’s doctest was made. Doctest, and more generally example/expectation oriented code, is a great way to explain things, to make tests readable, to make test-driven development feasible, to create an environment that errs on the side of over-testing instead of under-testing, and to make failures and resolutions symmetric. It’s still vastly superior to BDD, avoiding all BDD’s aping of readability while still embracing the sense of test-as-narrative.

But, more to the point: use doctest.js, read the tutorial, or try it in the browser. I swear, it’s really nice to use.


Comments (19)


The Browser Desktop, developer tools

I find myself working in a Windows environment due to some temporary problems with my Linux installation. In terms of user experience Windows is not terrible. But more notable, things mostly just feel the same. My computing experience is not very dependent on the operating system… almost. Most of what I do is in a web browser — except programming itself. Probably a lot of you have the same experience: web browser, text editor, and terminal are pretty much all I need. I occasionally play with other tools, but none of them stick. Of course underlying the terminal and text editor UI is a whole host of important software — interpreters, version control tools, checkouts of all my projects, etc. So really there’s two things keeping us from a browser-only world: a few bits of UI, and a whole bunch of tools. Can we bridge this? I’m thinking (more speculatively than as an actual plan): could I stay on Windows without ever having to "use" Windows?

Browsers are clearly capable of implementing a capable UI for a terminal or editor; not a trivial endeavor, but not impossible. We need a way of handling the tools. The obvious answer in that case is a virtual machine. The virtual machine would certainly be using Linux, as there’s clear consensus that if you remove the UI and hardware considerations and just consider tools then Linux is by far the best choice — who uses Mac servers? And Windows is barely worth mentioning. I worked in a Linux VM for a while but found it really unsatisfying — but that was using the Linux UI through a VMWare interface.

So instead imagine: you start up a headless VM (remembering the tools are not about UI, so there’s no reason to have a graphical user interface on the VM), you point your browser at this VM, and you use a browser-based developer environment that mediates all the tools (the lightest kind of mediation is just simulating a terminal and using existing console-based interfaces). Look at your existing setup and just imagine a browser window in place of each not-browser-window app you are using.

I’m intrigued then by the idea of adding more to these interfaces, incrementally. Like HTML in the console, or applications lightly wrapping individual tools. IDEs never stick for me, maybe in part because I can’t commit, and also there’s collaboration issues with these tools (I’m never in a team where we would be able to agree on a single environment). But incremental decentralized improvements seem genuinely workable — improvement more in the style of the web, the browser providing the central metaphor.

I call this a Browser Desktop because it’s a fairly incremental change at this point and other terms (Web OS, Cloud OS) are always presented with unnecessarily hyperbole. What "operating system" you are using in this imagined system is a somewhat uninteresting semantic question; the OS hasn’t disappeared, it’s just boring. "The Cloud" is fine, but too easy to overthink, and there are many technical reasons to use a hybrid of local and remote pieces. "Internet Operating System" is more a framing concept than a thing-that-can-be-built. Chromium OS is essentially the same idea… I’m not really sure how they categorize themselves.

What would be painful right now? Good Javascript terminals exist. Bespin is hard at work on an editor worthy of being used by programmers. The browser needs to be an extremely solid platform. Google Chrome has done a lot in this direction, and Firefox is moving the same direction with the Electrolysis project. It’s okay to punt for now on all the "consumer" issues like music and media handling… and anyway, other people are hard at work on those things. Web sockets will help with some kinds of services that ideally will connect directly to a port; it’s not the same as a raw socket, but I feel like there’s potential for small intermediaries (e.g., imagine a Javascript app that connects to a locally-hosted server-side app that proxies to ssh). Also AddOns can be used when necessary (e.g., ChatZilla <https://addons.mozilla.org/en-US/firefox/addon/16>).

I’d like much better management of all these "apps" aka pages aka windows or tabs — things like split screens and workspaces. Generally I think using such a system heavily will create all sorts of interesting UI tensions. Which might be annoying for the user, but if it’s a constructive annoyance…

On the whole… this seems doable. It’s navel gazing in a sense — programmers thinking about programming — but one good thing about navel gazing is that programmers have traditionally been quite good at navel gazing, and while some results aren’t generally applicable (e.g., VM management) the exercise will certainly create many generally applicable side products. It would encourage interesting itch-scratching. There’s lots of other "web OS" efforts out there, but I’ve never really understood them… they copy desktop metaphors, or have weird filesystem metaphors, or create an unnecessarily cohesive experience. The web is not cohesive, and I’m pretty okay with that; I don’t expect my experiences in this context to be any more cohesive than my tasks are cohesive. In fact it’s exactly the lack of cohesiveness that interests me in this exercise — the browser mostly gives me the level of cohesiveness I want, and I’m open to experimentation on the rest. And maybe the biggest interest for me is that I am entirely convinced that traditional GUI applications are a dead end; they rise and fall (mobile apps being a current rise) but I can’t seriously imagine long term (10 year) viability for any current or upcoming GUI system. I’m certain the browser is going to be along for the long haul. Doing this would let us Live The Future ;)


Comments (11)


Joining Mozilla

As of last week, I am now an employee of Mozilla! Thanks to everyone who helped me out during my job search.

I’ll be working both with the Mozilla Web Development (webdev) team, and Mozilla Labs.

The first thing I’ll be working on is deployment. In part because I’ve been thinking about deployment lately, in part because streamlining deployment is just generally enabling of other work (and a personal itch to be scratched), and because I think there is the possibility to fit this work into Mozilla’s general mission, specifically Empowering people to do new and unanticipated things on the web. I think the way I’m approaching deployment has real potential to combine the discipline and benefits of good development practices with an accessible process that is more democratic and less professionalized. This is some of what PHP has provided over the years (and I think it’s been a genuinely positive influence on the web as a result); I’d like to see the same kind of easy entry using other platforms. I’m hoping Silver Lining will fit both Mozilla’s application deployment needs, as well as serving a general purpose.

Once I finish deployment and can move on (oh fuck what am I getting myself into) I’ll also be working with the web development group who has adopted Python for many of their new projects (e.g., Zamboni, a rewrite of the addons.mozilla.org site), and with Mozilla Labs on Weave or some of their other projects.

In addition my own Python open source work is in line with Mozilla’s mission and I will be able to continue spending time on those projects, as well as entirely new projects.

I’m pretty excited about this — it feels like there’s a really good match with Mozilla and what I’m good at, and what I care about, and how I care about it.


Comments (32)