Notice: this is a draft for a forthcoming article. Comments are invited but please do not copy or distribute it without permission.

The XML Imbroglio

Does what Microsoft is doing with XML spell the end for open source office applications in general and in particular?

Gary Edwards a Web application design consultant and's representative on the OASIS Open Office XML Format Technical Committee, seems to think so. Certainly he has repeatedly expressed his concern that next generation Microsoft Office Suites will force an even greater degree of Microsoft lock-in than current releases do.

Edwards is undoubtedly right to be worried. Microsoft's use of XML is sufficiently Microsoft centric that the price of use will almost certainly include platform consistency -meaning that all users must be at the same release level for both the Microsoft operating system and Microsoft Office. Since Microsoft Office documents operate a bit like internet worms, spreading Microsoft Office wherever they land simply because there's no other reasonable way for the recipient to access them, the concern is that early adopters will force their business partners to follow suit and thus eventually lock out open source products like

Of course the open source community could follow Microsoft's lead to add XML functionality to its products, thus maintaining inter-operability, but there are some difficult issues that may make this technically impractical and legally impossible.

The impracticallity arises because it's actually much harder to maintain bad software than good - and what Microsoft is promising to do with XML is bad software. Microsoft can throw a few thousand people at keeping the resulting house of cards more or less standing; open source cannot. The legal issue is more difficult as Microsoft has become increasingly willing to use litigation to prevent open source programmers from matching its technologies -and matching some of the stuff they are promising will almost inevitably require using the same algorithms and thus expose the programmer to the threat of legal action.

Doomsayers therefore conclude that the sky is falling and the end is near for open source.

I don't think either that perception or the arguments behind it are right; on the contrary, Office 1X and Microsoft's vision for XML spell opportunity, not Armageddon, for groups like

To start with, Microsoft is not using XML as the rest of us understand it. Get trapped into thinking that XML is XML and you're halfway to a "follow blindly" decision; understand the differences and you'll quickly realize that there are better options.

Standard XML defines a class of document type definitions (DTDs) for use in document markup. As such an XML DTD such as SAML (Security Assertion Markup Language) is fundamentally passive; acting merely as an information container whose known characteristics make its contents easily accessible to a rendering engine.

Unfortunately, this isn't how Microsoft sees XML; in fact, their vision is about as related to standard XML as Active-X was to X-Windows.

Microsoft sees XML as a web programming language; not as a framework for markup. A Microsoft XML document processor can actively trigger applications which replace content in the current document with the output of programs running on the same or other machines. That change makes it possible to see XML as a web programming language - something which will, when implemented, make it possible to write documents that contain truly active, self-updating, components. For example companies could send out financial statements as Office 1X documents that continually update themselves from the production database behind the source company's financials.

This, of course, is the promise the Unix community responded to with the development of RPCs and distributed file systems in the early eighties. RPCs turned out to depend on mutual trust and the devolution of internet access to the masses thus required these ideas to morph into more secure forms, eventually giving rise to the ideas behind specifications for things like CORBA object brokers at the internet level and javabeans, enterprise or otherwise, at the local level.

Even within the Unix community, these secured forms of RPC have been much less widely accepted than expected, in part because the PC got in the way, in part because this stuff isn't easy, but mainly because most companies turned out not to want to do this most of the time.

Recognize that Microsoft's vision for XML isn't XML but yet another Microsoft attempt at a web based RPC framework, and you'll probably agree that it's likely to eventually suffer the fate of its predecessors. In the short term the security issues are going to prove insurmountable; in the medium term most customers will find they really don't want to do this; and, in the longer term Microsoft will be forced to adopt the ideas that the Unix community evolved to effectively manage and use RPCs.

Meanwhile, however, it can do a lot of damage to the open source movement if enough customers are befuddled into thinking both that it will work and that lots of other people will want to use it. After all it is marketing success, not technical success, that counts for Microsoft.

The pre-marketing push appears to be focusing on document security and related applications. Thus customers are promised the ability to distribute documents the recipient can't copy or forward to third parties; documents that authenticate the user before allowing themselves to be displayed; and documents that keep their own contents current through live links to remote databases.

There are a lot of users in business and government to whom this sounds like hot stuff because it promises to combine the security of paper documents with the flexibility of network distributed electronic documents. In reality, however, some of these things cannot be done and there are already far better open source ways of achieving the others.

It is not possible, for example, to make a document which can be read but not copied. You can make copying harder, but you can't make it impossible as long as the person doing the reading is free to leave the building and talk to others.

It appears that the Microsoft way to make it harder to copy or distribute documents will be based on embedding active XML controls in the documents. The application reading the document will then first read the authorization component, perform appropriate on-line checks, download the decryption key for the text itself, and then decrypt and process the remaining XML to produce the document.

For this to work the user needs to have the right tools and the on-line verification has to proceed smoothly - meaning that all of the services required have to be on-line at the time the document is opened. The document could not be opened, for example, if the remote server were off-line, had its name or address changed, or had its software substantively changed. Document life would therefore be limited to that of the shortest lived library or external resource called from within it.

There could be advantages to putting out documents knowing that inevitable hardware or software change at the source server will leave them unreadable in the hands of remote users but there are also risks. For example, the next SQL-slammer could render internal documents unusable for the duration of the external network emergency.

The underlying reality here is that the idea of putting the authentication client in the document and then using RPCs to authorize them isn't very smart. In contrast the open source solution, to regenerate documents at source on an as-authorized basis, is both elegant and effective.

In my Nichievo project, for example, I use Cocoon to do both things. Documents are assembled on request at the web server, marked up for formatting to meet user needs, given unique internal identifiers, formatted, encrypted, and dispatched to authorized requesters identified via a key exchange. This approach takes some minor risks with document integrity but has strong controls on document identification that should make it impossible to pass off a forgery as the real thing. Someone with the right skills and access could steal documents - but not exploit them against the company except through publicity and the resulting minor embarrassment.

It is hard to believe, however, that the people in charge at Microsoft don't know this. Look at Microsoft's attempt to extend XML into a web programming language from a technical perspective and it looks naive at best and suicidal at worst - not a combination of characteristics usually associated with the people who run Microsoft. So why do they think this is a great idea?

I've heard a number of opinions, some of which start out from the premise that Microsoft is fundamentally dishonest. For example I've been told that Microsoft is doing this because:

  1. The open source movement depends on people motivated by technical excellence - and they won't be willing to follow Microsoft down this path. Thus giving Microsoft back their desktop monopoly.

  2. So far it's just a puppet show - they issue press releases and unquestioning media acceptance spreads fear, uncertainty, and doubt in the open source community while Microsoft doesn't actually need to do anything.

  3. They have no idea what they're getting into. Remember this is a company that's never invented anything of substance, it's run by brilliant marketers who hire some of the best technical people coming out of the schools just to keep them away from companies like Sun - and then give them enough money to keep them hanging in while their potential as technical contributors is ground down by the bureaucracy. Their history is riddled with ideas like this, superficially attractive but technically stupid things that can be hyped and then dropped; why do you think it takes fifty million lines of code to deliver the Windows interface?

  4. Microsoft XML isn't part of any deep strategy, it's just another quick fix, like DLLs, thats gotten so far out of hand they have to call it a strategy.

Those kinds of answers have their attractions, but I don't think either malice or incompetence need be invoked when a better fitting answer is at hand. I think some people at Microsoft see an opportunity here to address what they see as a critical problem in licensing and copy protection.

Microsoft has a number of serious structural problems - for example, its continual failure to expand outside its core markets; the open source assault on its former government monopolies outside the U.S; and, the fundamental incompatibilities between its home entertainment and enterprise markets - but the most important daily management problem is revenue and expectations management.

Each time peripheral investment failures and low core market growth force a price increase to meet revenue needs, that price increase has two unhappy side effects:

  1. it increases the rewards for theft; and,

  2. it lowers psychological barriers to theft.

Asking people to donate $20 if they like your shareware works in part because people don't want to rip-off the little guy and in part because the amount is, for most people, below the point at which greed overwhelms conscience.

Justifying a few shady copies of Office/XP on Windows/XP is a lot easier. Not only can Microsoft be portrayed as the giant corporate bad guy but a simple "don't ask" policy can make taking home a PC with unaccounted for software installed under your organization's general license seem like a perfectly reasonable employment benefit.

To you this may not seem like a big deal and in reality the U.S. numbers are probably quite small, but it doesn't look that way to Microsoft. To them, even failure to upgrade on their schedule represents a form of revenue theft to be fought tooth and nail.

One of those teeth consists of making common cause with the recording industry but, beyond that, I think they see passport and palladium as among the first nails and Microsoft XML as the hammer that drives them home.

The big issue in licensing is ensuring that every working copy of the software can be matched to a revenue receipt. Extend the file format used to store Office documents to include mandatory active links that verify licensing and every document transfered from one PC to another becomes a license enforcement tool for Microsoft.

That's the central attraction. Many people buy their Microsoft Windows Operating Systems just so they can run Microsoft Office. Take that franchise away and the company stands revealed as a conglomerator of poor investments; strengthen it, and the company can report further revenue increases.

So that's the vision: Office applications that embed source and destination license verification into the document loading process.

Implement that vision via a hypothetical MLVML (Microsoft License Verification Markup language) embedded as one of the default DTDs available to the XML parser and you get a miraculous bit of synergy.

< ?mlvml version="1.0"?>
< Workbook xmlns="urn:schemas-microsoft-com:office:spreadsheet"
< registration description="E2KXPSP2" progid="XP802AC54C" version="1.09"
< /registration>
< DocumentProperties xmlns="urn:schemas-microsoft-com:office:office">
< LocationOfComponents HRef="file:///H:\2KApps\MSOffice%20XP'/>

With this, Microsoft gets an unbeatable framework for Microsoft license enforcement -and gets customers to pay for it by selling the controls needed to make it work as a fine grained method for controlling document distribution.

With it, user organizations can issue multiple levels of document reading and reproduction rights to their users and enforce them through a pair of closely related XML/DTDs -one of them on a Microsoft document verification server and one under the user organization's own control. Users whose access is limited to the same Windows technology used to impose the limits can't escape from them. Provided only that the organization has perfect platform consistency, this approach delivers the otherwise mythical non forwardable, non-printable, read-once document.

The opportunities for synergy here are dazzling. The sales force can honestly trumpet the technology as a security first for Microsoft and probably make money selling Microsoft document verification servers into "Enterprise 6.0" organizations -while using the documents people work with and exchange to force upgrades and license concurrency down to the last PC in the network.

If Microsoft genuinely had no competition in office software the security failures, complexities, and costs associated with trying to make this scheme work wouldn't matter. The bottom line for people who have no choice, is that they have no choice.

In reality Microsoft does have competition, and problems "getting there" therefore have significance.

If you were advising Microsoft on this what would you suggest? I think that the obvious thing for them to do is exactly what they are doing - use a little bit of more or less standard XML in some products while trying to leverage market perception of their inevitable dominance in Office applications to force the open source community into following them down this road. If the people follow Microsoft's lead they'll consign themselves to a poor second place, and if they don't Microsoft can just reshuffle the deck and pretend the whole thing never happened.

On the other hand, if you were advising the open source movement, what would you suggest? In my case, I'd suggest that trying to be a cheaper substitute for Microsoft Office is a losing strategy. In the long run people don't care about cheap; what they want are better options built around better ideas. That's why Linux has won the battle for hearts and minds in the Intel server arena - not because it's cheaper, but because it's better.

The necessary pieces are in place. The Liberty Alliance has defined a framework for document authentication, user identification, and access authorization that can be used with SAML now. The Unix smart display architecture offers organizations that need it highly secure document routing and control. Product sets like Cocoon point the way to secure, on the fly, document generation and delivery without imposing proprietory restrictions on the user's hardware or software.

One thing that's missing is an easily deployable format gateway. In firewall mode these gateways would translate all incoming Microsoft Office documents to formats on the way in and optionally reverse that transformation on the way out, automatically handling any needed cryptographic functions and third party server calls in both directions.

In web server mode -like the TOM document conversion server at CMU- these would accept documents in one format and forward or return them in the alternative format -except that Microsoft Office formats would not be forwarded to addresses inside the organization.

Format gateways would let organizations with thousands of users participate in Microsoft document exchanges while using open source products internally and limiting their Microsoft exposure to a couple of racks used to offload conversion processing at the gateway.

Put these pieces together and what open source offers is a better option, thus making the real bottom line on Microsoft's use of XML in Office 1X a simple one: follow and lose, or continue to diverge and win by offering a smarter alternative that also happens to be cheaper.