Today, the XML Activity, with several Working Group charters, was approved. This is a major milestone for W3C, not because of the activities of these groups themselves, but for W3C’s process of developing standards.

For the first time, all of W3C’s active Working Groups now operate in public.

When W3C began, it operated largely as a Member-only consortium. Member companies paid substantial dues, and convened behind closed doors with each other and a handful of Invited Experts,  in Member-only Working Groups (WGs). WG mailing lists, teleconferences (telcons), and face-to-face (f2f) meetings were all Member-only, as were editor’s drafts of specifications, and even the list of which organizations and people were participating. Periodic drafts of works in progress (Working Drafts, or WDs) were published on W3C’s public Technical Reports (TR) directory, and feedback was processed on public mailing lists. But the public conversations and member-only conversations didn’t mix, and specific decisions were not transparent. W3C was not a truly open standards body.

When I joined the W3C team in July 2007, one of my personal goals was to open up the organization. I joined the Team to help with SVG; 2 years prior to that, I had actually joined W3C as an Affiliate Member via my small startup, Vectoreal, to move a struggling SVG specification along. I spent the entire revenue of one of my consulting contracts to do so, in the hope that I could make it up in the long run if SVG took off and increased my business; I became a W3C Member because I didn’t feel like the SVG WG was responsive to the then-active SVG community, and I wanted to represent the needs of the average developer. In joining the W3C Team, I took a salary that was about half of my previous years’ earnings. For me, it was important that W3C should not be a “pay-to-play” organization, that Web developers –and not just paying W3C members or hand-picked Invited Experts– should have a strong voice.

This was one of the few things I had in common with the contentious WHATWG folks (among others): a core belief that standards should be developed in the public. When I took over as staff contact of the WebAPI WG, I (along with the WG chairs, Art Barstow and Chaals McCathieNevile) set about merging it with the Web Application Formats (WAF) WG, to make a single WebApps WG that would operate in the public, following on the heels of the newly rechartered HTML WG’s grand experiment as a public WG; unlike the HTML WG, however, a person didn’t have to join the WebApps WG to read and post on the mailing list, lowering the bar to participation and decreasing W3C’s overhead in processing Invited Expert applications. This proved to be the model that the SVG, CSS, and later Working Groups followed.

We were off to a good start, but then we stalled out. Many Working Groups didn’t want to become public, and conversations about making WGs public by default were shut down by Members who’d paid to have a seat, by Invited Experts who liked their personal privilege, and by W3C staffers who had a variety of concerns.

But slowly, with external and internal pressure, including encouragement by some W3C members who put a premium on openness, Working Group charters (renewed every couple of years) more and more commonly designated their group as public. Within a few years, this became the norm, even without pressure. A few Member-only holdouts persisted: the Multimodal Interaction (MMI) WG; the Web Accessibility Initiative (WAI) WGs; and the XML WGs.

</xml> (Closed XML)

In January of 2000, in response to complaints that an update to the public version of an XML specification had taken too long, Tim Bray wrote on an external mailing list, XML-Dev, “[…] But it’s a symptom of the W3C’s #1 problem, lack of resources.” This is a tune we’re all still familiar with, sadly.

Lee Anne Phillips replied to this, on 16 January 2000:

With all respect, I think the lack of resources are the fault of the W3C membership policies, which seem designed to strongly discourage individuals and small organizations and businesses from participating in the process. US$5000 for an Affiliate Membership is beyond the reach of most of us and of many small businesses since that’s in addition to the value of the time spent on the process itself.

Whether this policy is because the big players want negotiations to go on in secret (and secrecy is inherent in the W3C structure so it can’t be an accident) or because W3C just can’t be bothered with the “little people” is a matter of speculation.

What’s certainly true is that there is a vast pool of talent available, many of whom are passionately interested in the development of XML and XML-related standards and might well have more time to spend than the human resources on sometimes grudging lend-lease from major corporations. Witness this and other lists which represent a collective effort of major proportions and a tremendous pool of knowledge and skills.

While we all appreciate the enormous efforts of the organizational participants in the W3C process, who’ve done yeoman service trying to juggle activities which might directly advance their careers at their organizational home with the community responsibilities of the standards process, there just might be a better and more open way.

The Internet standards process started in the RFC methodology, which, though sometimes awkward, chaotic, and slow, allowed rapid innovation and standardization when warranted and was fully public, ensuring participation by the *real* stakeholders in the process, the community served, rather than being dominated by the vendors who want to sell products to them.

In one way or another, we’re the ones who pay for all this work. Surely a way could be found to ensure that we know what the heck is going on. Even better, we could help in the initial stages rather than waiting in front of the curtain until a team of magicians come out and present us with whatever they think we want and are then either cheered or booed off the stage.

Others defended W3C, citing the Invited Expert policy, but Lee Anne (and many other people with similar thoughts) was essentially correct on two major points: it was an exclusionary policy; and it limited the pool of possible contributors that could help a resource-constrained organization.

It took 16 years, but I can now say, truthfully, that W3C is an open standards organization, both in its specifications and in its process.

Yeah, but it is really open?

There are aspects of W3C that are still not open, and are never likely to be, for pragmatic business and collaboration reasons.

Michael Champion from Microsoft said in this same XML thread:

The W3C’s secrecy policies are there because it is a treaty organization of competitors, not a friendly group of collaborators.

This was 16 years ago; I don’t know if he’d say the same thing today, but it’s only a little less true… W3C is a forum for coopetition, and discussions are usually friendly, but it is business. As a W3C staffer, I’m privy to future implementation plans, concerns about patents and Intellectual Property, and other in camera, off-camera discussions that are important for standards makers to know, but which companies won’t or can’t talk about in public. If they couldn’t talk about it in Member-only space, it wouldn’t be talked about at all, or would be couched in lies, deception, and misdirection. This frankness is invaluable, and it should be and is respected by participants. This is part of the bond of trust that makes standards work.

There are other parts of W3C’s standards-making process that are still not completely open for participation, due to logistical issues:

• telcons: Trying to make decisions, or even have coherent conversations, on a 1-hour telcon with dozens of people, some of whom may not be known to the WG or may not be familiar with the discussion style of the WG, would be a nightmare. Telcons are limited to WG participants, including Invited Experts, and any topic experts they invite on a one-off basis. The logs (meeting minutes) are published publicly, however.
  
• f2f meetings: The same constraints for telcons apply to f2f meetings, with the extra factors of the costs for the host (meeting room, food, network, and so on), facility access (NDAs, etc.), planning (who’s available and when and where), and travel costs (which are prohibitive for some Invited Experts, and would be for many members of the public). As with telcons, the meeting minutes are scribed and published publicly. In addition, around these f2f meetings there are sometimes public meet-ups for the locals to see presentations by the WG, to meet them, make suggestions or ask questions, and otherwise socialize with the WG.
• informal brainstorming: This might happen in a hallway or a cafe, or on an unrelated mailing list or issue tracker; it might be a collaboration between colleagues or competitors, or it might be an individual tinkering with their own thoughts until they feel they have something worth sharing. It’s hard to make this fully open.
• decision-making: One of the most significant benefits of W3C membership is the right to help make final technical decisions in a Working Group. Anyone can make suggestions and requests, but ultimately, it is the participants of the WG (and often, more specifically, the editors of the spec) who make the decisions, through a process of consensus. This is not only logistical (somebody has to decide among different options) but usually pragmatic as well, since the members of the WG often are the implementers who will have to code and maintain the feature, and who are well-informed about the tradeoffs of factors like usability, performance, difficulty of implementation, and so on. But even if the WG has the final decision power, there is still an appeal process: anyone, whether a W3C member or an average developer, can lodge a Formal Objection if a technical feature is flawed, and the technical merits will be reviewed and decided by W3C’s Director.
• write access: Who can edit the specifications? Who has control over merging pull requests? There are IP issues with contributions, and also a coherent editorial and technical tone that needs to be adhered to. Currently, this is limited to WG participants, and it’s likely to stay that way.

But the vast majority of the standardization process is now public, including almost all technical discussions, meeting minutes, decision processes, and specification drafts, as well as the lists of people and organizations involved.

W3C has greatly benefited from this openness; we get invaluable feedback and discussion from our mailing lists, and WGs tend to take such feedback very seriously. We value this public input so much that W3C has also expanded its offerings into a project called W3C Community Groups that are free and open to everyone to participate, where anyone can propose a topic for technical discussion, and if others are interested, they can form a group to develop that idea further, to write use cases and requirements and even a technical specification; if the idea gets traction, W3C may even pick it up for the Recommendation-track formal standardization process.

Some people have taken a more extreme view, that W3C should lower its membership dues by getting rid of the technical staff. I’m all for finding ways to lower our membership dues, to be more inclusive, but getting rid of W3c technical staff would mean decreasing oversight and openness, not the reverse; most of the W3C staff are dedicated to making sure that we serve society, as well as our members, and a lot of critical technical work wouldn’t happen without the W3C technical staff. Our members are doing W3C and society a great service by sponsoring our work, even while they benefit technologically and financially themselves.

●。 (Full Circle, Full Stop)

The schism between the XML community and W3C was one of several such schisms in W3C, perhaps the first major schism. The fight over whether W3C should adopt a Royalty-Free Patent Policy on its specifications was another (we did, and W3C’s Royalty-Free patent policy is now one of its crown jewels); the battle of control over HTML with WHATWG was yet another; the ongoing debate about whether the W3C’s specification document license should allow forking and require attribution is still another (W3C has recently released an optional forkable, GPL-compatible, attribution license for software and documents that may be used on a per-Working Group level). All of these were about openness, in one way or another. Openness is about transparency, and accountability, and ability to participate, and freedom of use, but also about control, and who controls what; this will always be a matter of heated debate.

The particular flavor of openness that was being debated in the XML community in 2000 was about a process open to participation, and transparent oversight into the discussions and decision-making at all formal stages of creating a standard. XML is widely used, and largely stable, so interest in further XML standards development has waned over the years; participation has dwindled, and with the final versions of these specifications (XML Core, XSLT, XQuery, XPath, XProc, EXI) being published, this may well be their final charter renewal. Now, with the XML Working Groups rechartered finally and finally as public, we have a nice bookend to a process of open standards.

---

 

Closing Words

As usual, the reality is not quite as tidy as the story. After I first wrote this article, I went back to fact-check myself, and found some dusty corners that need cleaning out. For full disclosure, I’m compelled to mention the exceptions. We have some non-technical coordination groups, such as the Patents and Standards Interest Group and Advisory Board that are not public. Some of our Web Accessibility Initiative (WAI) WGs are (ironically) not now public (specifically, the Protocols and Formats WG and WAI Coordination Group), but are currently being rechartered and all the WAI groups will be public later this year when the rechartering is settled. The Member-only Voice Browser WG is winding down, and is scheduled to close at the end of this month. The most notable exception is the Math WG, one of our oldest WGs (chartered in 1997); through a loophole in the charter process, the Math WG has not been rechartered since 2006; instead, it’s been extended by W3C management, on the grounds that they are only doing errata on existing MathML specs, rather than new technical publications. Still, this is an unwarranted exception, and my colleague Mikeâ„¢ Smith and I are now advocating to resolve this as soon as possible, so no technical W3C Working Group, even one doing only specification maintenance, operates outside the public.

Once we’ve made sure every technical W3C Working Group is chartered to operate in the public, we all need to ensure that W3C doesn’t slip back into allowing Member-only technical Working Groups. This could take the form of a W3C policy change (petition, anyone?), or failing that, a mindfulness by W3C Advisory Committee Representatives that when we last tried the Member-confidential model, it led to worse specifications, slower progress, more time consuming feedback processes, and a community schism that nearly tore W3C apart. Let’s prevent that mistake from happening again. Eternal vigilance, and all that…