Jeff Atwood makes a particularly salient point in taking about the lack of direction for Markdown. The specification for Markdown seems to languished for the last five years while the number of implementations have sky-rocketed. I have also written an implementation for Nu. My implementation was based on [PHPMarkdown][phpm]. At the time, I had realized that Markdown was somewhat abandoned and that PHPMarkdown was undergoing rather frequent updates and revisions. It also came with a test suite. In the end, I had decided for myself that Markdown proper was no longer the reference version of the specification, but rather that PHPMarkdown had usurped the original implementation. As more implementations of Markdown are written (available in nearly every popular language and using all manner of parsing algorithms), we must ask if we need one to be the basis of comparison or if the spec is already pretty much complete.

[phpm]: http://michelf.com/projects/php-markdown/

A few words from other people

I think Michel Fortin, the creator and sustainer of PHPMarkdown, provides insight into the nature of the argument. He explains the purpose of PHPMarkdown and the reason for PHPMarkdown Extras. Basically, PHPMarkdown is to be comparable to Markdown proper whereas Extras is a staging grounds for new ideas. He also provides explanations for Atwood’s three assumed bugs in Markdown. Given as how I view PHPMarkdown as the reference standard (and, to some degree, Extras as the potential future for the spec), Fortin’s response is greatly appreciated.

Daniel Jalkut also responds on the obligation of ope source software developers. Basically, the argument is that if you give something away for free, you don’t have any additional obligation to the users. In open source, this is especially true since the users can simply take up the code and make the changes they need. While this argument may work with consumer goods and possibly with open source (although let’s face it: open source projects die a slow, painful death most of the time), I don’t think the model applies directly here. I’ll explain more later.

Finally, it’s worth noting that Github has their own flavor of Markdown that has been explicitly endorsed by Gruber. This certainly supports the potential of open source software in such a way that boosts Jalkut’s claims. However, I don’t think that it quite undercuts Atwood’s argument.

A specification, not code

Markdown is not simply a Perl script. It may have been at one time: an application written in Perl for the formatting of text. However, it is now a specification. There’s not one single Markdown implementation. There are dozens. And they all attempt to produce the same general effect. This is the part that complicates the issue.

Let’s consider these two basic arguments from Atwood:

It’s almost at the point where John Gruber, the very person who brought us Markdown, is the biggest obstacle preventing Markdown from moving forward and maturing.

Right now we have the worst of both worlds. Lack of leadership from the top, and a bunch of fragmented, poorly coordinated community efforts to advance Markdown, none of which are officially canon.

While the first argument, that Gruber is the biggest obstable to Markdown maturing, is the debate at hand, I don’t think that the second argument can be discounted in any real sense. It’s a significant problem that can’t be argued away. Look at how many Markdown implementations there are for Ruby. Sweet Jesus, it’s like everyone who just found out about Rails has to re-write Markdown.

And then we get into the issues of Github-flavored Markdown and PHPMarkdown Extras. Sure, making a few minor changes specific to a website is good and all (and officially endorsed by Gruber), but what does it mean for the specification? There’s the point about underscores in words and how that changes formatting. Fortin provides good insight into why it’s still done in PHPMarkdown but also says that he’s changed the rules for PHPMarkdown Extras. If this idea gets enough grounding, can we change the specification?

For the record, underscores should indicate underlining, not emphasis. The idea that only links should be underlined is bullocks and ignores standard conventions of written language. But I didn’t write Markdown’s specification.

In the end, the power of Markdown is that it’s a well-written standard. People like the syntax and, most importantly, like that it’s portable. It’s available on nearly every popular scripting language and is easily implemented in web-based applications (thanks to PHPMarkdown and the wealth of Python and Ruby implementations). Markdown is a learn once, use often skill that becomes more useful as it becomes more ubiquitous. And it’s not just about the web. I have scripts that turn Markdown documents into PDF and RTF files as needed (although to HTML first, but in theory Markdown as a specification can be output-agnostic).

The specification hasn’t changed terribly significantly since it’s inception although Gruber’s own website now has various new elements such as footenotes. Footenotes are a part of PHPMarkdown Extras but developers have a choice. Do they follow the first-to-market styling of Extras or do they create their own syntax? Do they keep having two spaces indicate a line break or do they break at every carriage return inside a paragraph? Do they have to follow the underscoring rules or can they change it up?

Gruber’s version of Markdown suits him. And Github’s suits them. And eventually we’ll have multiple versions of Markdown that suit different things and there still won’t be a modern canonical version. As more people add their own ideas to their implementations, Markdown as a specification will become diluted. As the implementations diverge, Markdown users face the complexity of learning the differences in implementations or limiting themselves to only the core specification features. Being limited or confused would not serve users and Markdown would become a less appealing choice. Eventually, someone like me may wish to break spec altogether and make underscores actually mean underlining.

Moving forward or calling it finished?

Atwood seems to think that Markdown needs to be advanced. I can’t help but wonder if that’s really so necessary. The way I see it, Markdown covers the vast majority of my formatting needs. I don’t need footnotes or pull quotes or a lot of other things. To a certain extent, Markdown is complete. If it never changed, I’d continue using it until I found something better.

If it’s finished, though, we need a reference. It’s not necessarily on Gruber, but rather on the open source community. We need to create a reference compiler for Markdown that generates accurate, bug-free code that conforms to the specification exactly. We also need to create some tool for comparing outputted markup (at least in HTML) so that implementations could compare to the reference.

With a reference compiler, the core language of the Markdown specification can be preserved. Additional flavors of Markdown can diverge when appropriate. Large changes (like those in Extras) can even be new forks with different names to indicate that they aren’t Markdown-proper, but rather an extension that offers several new changes.

Or we can appoint someone (conceivably Gruber) to be benevolent dictator for life over Markdown. No additional code needs to be added to Markdown.pl (the Perl application) as it’s merely one implementation out of dozens (and one that may be obsolete or deprecated). Instead, the BDfL merely needs to provide opinions and approve proposals to extending Markdown, the specification. The community can provide the work in extending the reference compiler and updating their own implementations.

This should go without saying, but I find that it often needs to be said anyway: whenever something is updated, the market fragments. Not everyone will jump to the latest version of their favored implementation of Markdown just because the spec changed. Updates can fracture usage just as much as anything else, especially if site-specific changes are implemented on top. Having a stable specification may be a predominant feature, not a significant problem.