the academic world — another place where reusability
should triumph — has been equally and sadly myopic.

the business segment has likewise been thoughtless…

even, believe it or not, the book-scanning operations
like the internet archive haven’t been smart enough to
see the advantages of flexible, powerful light markup.
(amazingly, peter brantley, the director now, often puts
out their position papers in a very sloppy form of .pdf.)

so this failure to adopt an intelligent archival format has
been very broad, reaching across a variety of segments.

-bowerbird

but .pdf sucks for a worse reason. .pdf sucks because
– even though it’s easy to put content _into_ a .pdf –
it’s very difficult to scrape it back out in a systematic way.
that’s why .pdf is called “the roach motel format”, because
content can go in, but it cannot come out. and that sucks.
because it’s important that government data can come out.

.xml sucks too. and forget what i said above, because
.xml might even suck worse than .pdf. that’s because
it’s difficult to apply the .xml markup in the first place,
and then it’s also often difficult to scrape it back out…
and even if you _can_ scrape the content back out again,
you have to repeat the difficult step of reapplying .xml
if you want to make the content useful in its next round.

what’s needed is a simple straight-out plain-text format
which can serve as a “master format” that can generate
.html (for the web) _and_ .pdf (for those who prefer it).
(with extra credit if a straight copy from either of the
output formats gives you the same plain-text “master”,
such that you could do infinite “round-tripping” of it.)

it’s not hard to invent a light-markup format to do this…

i’ve done it myself — something called “z.m.l.”, short for
“zen markup language” — and it works astonishingly well.
(you don’t just get “serviceable” output in .html and .pdf,
you get _powerful_ documents that have lots of features.)

or — ironic, being that you’re riffing off daring fireball –
you could use gruber’s “markdown”, a fairly similar beast.

the funny thing is that — since the format is “plain-text”
in nature — it’s dirt-simple for people to learn and use it,
and surprisingly easy to code apps like authoring-tools…
and, for conversion routines, you don’t have .xslt difficulty.

-bowerbird

p.s. plus end-users can still keep using their old tools,
because all that old software can produce plain-text files.

XML is not a universal *format*. XML is a universal *language* that can be used to define data formats – but you need to know how to interpret it. Average humans are generally very bad at interpreting raw XML. Programs that use XML generally only know how to interpret the formats they define, and maybe a few other publicly-defined formats; they can’t interpret the language well enough to understand every random format. Heck, Filemaker Pro deals with data in a far broader way than Word or Pages, and imports/exports XML – but try sending it a random XML file without an XSLT stylesheet to translate it, and see how far you get!

This falls prey to the same fallacy that many open source advocates have – as long as the code is open (document is in XML) everything is somehow OK. It’s only OK for those who have the skill and knowledge to work with the raw data, and that is a tiny tiny TINY fraction of the population. Documents in “XML” are utterly useless to 99% of the public; to be actually useful, they need to be in a defined XML format that most people can read. ODF sort of counts, in that anyone skilled enough to download, install and use Open Office can read it; but a fair number of people simply aren’t. (And I, and probably a fair number of other people, view forcing someone to download and use the entire OO suite just to read government documents to be an unfair imposition; it’s a lot more to download and a lot more work to set up and use than even Acrobat, let alone the simple PDF readers like Preview.)

As for the fantasy that if you mandate it, they will come… that worked real well for ODF, didn’t it? Oh, wait. Microsoft leapt to the call – no, they put on a crash program to develop their own incompatible XML format in OOXML, which last I heard still contains big binary lumps that make it just as hard for other programs to interpret as the old binary Office formats were. *And* started a massive lobbying/astroturfing campaign to both reverse the ODF decision and establish their own format instead, turning it into a nasty political fight. “[If] you mandate clean HTML or a particular XML version Word will be very quickly updated to produce that format”? Uh-huh. Riiiiiiight.

I’m more of an Adobe-loather than an Adobe-lover these days, but I have to admit PDF is the best tool for distributing human-readable documents. I’ve spent my time down in the trenches too; in my case, it was managing the library of Material Safety Data Sheets for all the products our company distributed. I was there when we still got mostly paper copies from the manufacturers, and we had to scan them to get them in digital format; I’ve dealt with manufacturers that sent them in ASCII text or Word documents; I’ve dealt with them in PDF. As a tool for storing them and forwarding them on to customers, who would either print them out or read them on the computer, PDF was by far the best tool, bar none.