There has been a heated argument recently on the W3C Canvas API mailing list between accessibility advocates and browser vendors over a pretty tricky topic: should the Canvas API have graphical “objects” to make it more accessible, or should authors use SVG for that? I think it’s a false dichotomy, and I offer a proposal to suggests a way to improve the accessibility potential of the Canvas 2D API by defining how SVG and the Canvas 2D API can be used together.

This brings together some ideas I’ve had for a while, but with some new aspects. This is still a rough overview, but the technical details don’t seem too daunting to me.

Retained-Mode vs Immediate-Mode Graphics

First, some background…

SVG is a retained-mode graphics format, meaning that each shape persists as a separate object, with distinct characteristics such as location, size (bounding box), and precise outline (stroke). These characteristics can be acted upon within the object structure (DOM) as discrete entities (or groups of entities) for manipulation by script or by user-interface actions (e.g. mouse clicks or keyboard tabbing); one of these interactions may be hit-testing.

The Canvas 2D API is an immediate-mode API, meaning that it allows the programmer to create visual shapes as individual pixels that are unstructured; in other words, if a red circle and a red square are drawn which overlap one another, there is no way to distinguish these from one another, or from the background canvas element (though a programmer could maintain a separate structure in script).

With retained-mode graphics, each object which is added to the DOM increases the memory “weight” of the page or application, potentially slowing down the browser or otherwise decreasing performance; by contrast, immediate-mode graphics, which are not represented as objects in the DOM, do not decrease performance, which is an advantage over retained-mode graphics.

Some accessibility advocates have noted that hit-testing and zooming are critical accessibility features of graphics; this requires the ability to access the location, size, and other characteristics of individual graphics. Unfortunately, adding this retained-mode feature to an immediate-mode API effectively removes the primary advantage of using an immediate-mode graphics system.

However, by defining how SVG and the Canvas 2D API can be used together, a model may be created which allows authors to easily use both together, each for their appropriate strengths. This approach involves two aspects: defining the behavior, and a potential shared API syntax.

Defining Behavior

Currently, the Canvas 2D API is only defined to work with the <canvas> element defined in HTML5, and it does not allow SVG to be drawn or imported into the Canvas. However, it could be extended to work inside SVG or to allow SVG to be used inside Canvas.

Canvas in SVG

At its simplest, the Canvas 2D API could simply be defined to allow authors to draw on the SVG raster <image> element, just as it does in the HTML <canvas> element, but there does not seem to be a reason to limit it to this element. The Canvas API could be defined to draw to any SVG element, rendering or structural, as an overlay. It would be merely a visual effect, and would participate in the rendering structure, including “z-index”, only as its “target element” does.

SVG in Canvas

SVG shapes could be allowed to be imported into the <canvas> drawing context, such that they render there but are also inserted into the DOM or shadow DOM. Currently, as I understand it, this is not allowed for security restrictions involving content from potentially different sources, but for content that does not have external references (for example, a <use> element to another site, this should not be a security risk.

SVG in the Shadow Tree

The accessibility features of the Canvas 2D API include a “shadow tree” and focus ring for those elements, such as text, which are not otherwise accessible to users with limited vision. Each such “graphic piece” which is created and painted to the screen is also represented in this underlying shadow DOM (IIRC).

SVG elements used in conjunction with Canvas could also be placed in this shadow tree, so that navigability is preserved. In order to save memory usage, a concept like the <use> element in SVG could alternative be used to “point to” an SVG in the main DOM from its appropriate place in the shadow tree.

SVG elements could either be used as visual shapes themselves, or could be invisible “hot-spots” for specific areas of the screen.

Common API

I have suggested before that we create a simple graphics API that will allow either 2D Canvas (immediate-mode) or SVG (retained-mode) images with the same set of methods and parameters, with the only difference being which “mode” the author selects to have as the instantiated form. If the author draws a circle to the canvas context, it would simply be drawn to the screen with no structure; if the author draws that same circle to the SVG context, it creates a <circle> element with the appropriate attribute values and style and inserts it into the DOM.

There would be certain differences in some of the renderings, because of the different rendering and compositing models of SVG and Canvas, and there would be certain specialized functionality and features each would have, but the majority of the visual appearance would be the same.

This would allow authors to learn only one API for 2D graphics for the Open Web Platform. With this ease of learning and use, the author could decide on a case-by-case basis, even within the same application, which mode works best. It would provide incentive to authors to use each to its best effect.

I call this approach COG: Common Open Graphics.

Example

An example of the effective use of SVG and Canvas together is a hypothetical game in which the user pilots a spaceship amidst an asteroid field.

The background of the game includes animated nebulae, passing comets, rotating planets and moons, and other visual effects which are not interactive, and which do not affect the play of the game, but which add visual appeal. The foreground of the game (the “playable area”) includes asteroids, enemy ships, and energy beams that the player must dodge or shoot at, which increase in size the closer they get, and which benefit from hit-testing and keyboard focusability. The middle-ground of the game contains many smaller asteroids and ships that are not yet interactive, and so don’t need hit-testing or focusability.

Adding all of these graphics shapes as retained-mode objects in the DOM would unnecessarily increase the memory footprint of the game, degrading performance and playability. However, since only a small number of these elements (those in the playable area) need the retained-mode interactivity, the majority of them could be drawn with immediate-mode characteristics, then when a certain “distance threshold” is reached, changed to interactive SVG elements, perhaps using the same shared API. This preserves the accessibility of the game, while also causing minimal performance degradation, and possibly even helping performance, since the hit-testing doesn’t need to be calculated in script. From an authoring point of view, using the same shared API means that both types of shape, retained-mode and immediate-mode, are equally easy to generate and maintain.

To enhance the visual appearance of the SVG asteroids, the finer control of the Canvas API could even draw an overlay on the SVG stroke and fill areas, much as filters do, to show craters and other surface details in a more fine-grained manner then can be done efficiently in SVG. Since this would be drawn specifically on the SVG element, the element itself would still provide the geometry and position characteristics.

Caveats

This proposal has two caveats:

1. Canvas can only be extended to be as accessible as SVG is, in this model. Even with object retention, there are still many enhancements needed to make SVG truly widely accessible. However, as plans for SVG accessibility improvements are made, these would be largely importable directly into the Canvas API, rather than working in parallel on two unrelated but overlapping graphics models.
2. Simply because the Canvas 2D API may be made more accessible, does not mean that all Canvas API applications or content will be automatically accessible, any more than all HTML or SVG content is accessible. Authors will still need to use best practices and take specific steps to make their applications accessible. This proposal only provides the means and the incentive to do so.

Feedback

I’m interested in feedback on this, either in the comments section of this blog, or on the W3C Canvas mailing list, public-canvas-api@w3.org.

Update

Someone pointed out offlist that this post emphasizes the accessibility aspect of this proposal, because that was the context of the most recent discussion, but this is a win for anyone who wants to do graphics in the browser, for any reason, not just accessibility. Like any good accessibility feature, my proposal has general benefits, so it is more likely to be used in more content. In the example scenario above, the developer gets the hit testing for free, regardless of whether or not Accessibility Technology is involved. Other general advantages include ease of learning and use of graphics by developers (as mentioned above), sharing of features between modes (like using Canvas.toDataURL() to save an SVG as a PNG), pixel operations (like painting or detecting a color at a point) in SVG, metadata (like microdata, RDFa, or microformats) in select parts of a Canvas app, and so on. I’m sure that clever people will come up with a whole boatload of cool things to do with a combined retained-mode/immediate mode API that I couldn’t imagine here.

Last week, I published the first draft (and subsequent updates) of the Web Interface specification, which defines touch events. This is the first spec from the new W3C Web Events Working Group.

Peter-Paul Koch (PPK) gave it a positive initial review. Apparently, others thought it was news-worthy as well, because there were a few nice write-ups in various tech sites. Naturally, cnet’s Shank scooped it first (he has his ear to the ground), and it was fairly quickly picked up by macgasm, electronista, and Wired webmonkey.

I thought I’d go into a few of the high-level technical details and future plans for those who are interested.

Touch interfaces are nothing novel, nor is using them with Web content, but there are enough differences from mouse interaction that specifying them should lead to easier development and better user experiences on the Open Web platform (or in HTML5 webapps, if you prefer that buzzword).

The most important thing to note for multi-touch events (using more than one finger, say) is that you want the different touch-points to be considered together, not as completely unrelated events. The WebKit implementation, used in the iPhone and Droid smartphones, as well as many, many others, used a good model for this, with each event exposing lists of currently-active individual touch-points; with wide deployment of WebKit already, it was an obvious starting point. PPK’s blog post was off on one minor detail here: while I did base the spec on WebKit’s behavior, I didn’t base it directly off of Apple’s documentation; I wrote it from scratch, for better or worse, and extended it for broader applicability.

One of these extensions was to take PPK’s suggestion to add ‘touchenter’ and ‘touchleave’ events, the equivalent of the ‘mouseenter’ and ‘mouseleave’ events that we finally specified in DOM3 Events, which work much more intuitively than ‘mouseover’ and ‘mouseout’.

I also extended the WebKit model to add ‘radiusX’ and ‘radiusY’ attributes, to indicate the general size and shape of the thing that touched the surface… say, a fingertip or a stylus (a fancy way of saying “pen-like stick”), described as an ellipse. I’ve been soliciting use cases and requirements for a couple of years, and this is meant to cover a similar request across different devices.

That leads me to mention the other use case for this specification: pen-tablets. Almost a decade ago, I was writing a browser-based SVG editor called SVG-Whiz (long since decomposed into digital fertilizer), and I wanted to be able to use my Wacom tablet with it. It worked okay as a mouse replacement, but I had to choose the stroke-width manually… I couldn’t use the tablet’s built-in pressure sensitivity.

We’re about to change that. Another change to the WebKit model is the addition of a ‘force’ attribute to indicate the level of pressure used on the touch or tablet interface. This will be useful for SVG or Canvas API drawing apps, games, and myriad other purposes. For example, most touch screens don’t have the capability to detect hovering (some capacitive screens being the exception)… this is a challenge for accessibility, but perhaps the pressure sensitivity—expressed in the ‘force’ attribute—could be applied to that end. We may specify a new related CSS mode, or simply define when existing ones apply, such as triggering the CSS ‘:hover’ pseudoclass past a certain pressure threshold (device-specific and user configurable, of course).

(On the subject of new CSS properties, Nokia has made the suggestion of haptic feedback properties for touchscreens, based on their implementation. This could be a very interesting addition to touch interfaces, even if it’s not available on all devices. That work would be handled by the CSS Working Group.)

In addition to pressure-sensitivity, pen input has other characteristics that distinguish it from touch, including the angle of the pen, so we may add that as well. Pen-inputs, either through tablets or stand-alone digital pens, often include particular mode-shifting buttons, so we have added attributes to detect when the alt, shift, and control keys (or buttons) have been pressed.

So, to cover a wide range of devices, we will be defining some capabilities beyond basic touchscreens: multi-touch, touch area, force, perhaps pen-angle, and even haptic feedback. Not every feature will be available for every device, but they will be useful for those that do, but we want the spec to be future-proof and device-independent.

This leads me to another touchy area: gestures. One of the common requests I’ve gotten is to add events for specific gestures, like pinch, swipe, twist, flick, etc. There are two problems with trying to standardize this right now:

1. Each device and each platform has its own gestures, sometimes several to do the same thing, and there is still room for innovation there;
2. It’s a patent minefield.

Instead, we intend to specify higher-level “intentional” events, like zooming in or out, panning, navigating, editing, etc., and define a way to map between the physical actions (touching the screen and moving your fingers) and the intention event (zooming in); it also means we can map between input on other devices, like keyboard and mouse, and the intent behind those actions. An additional aspect of this is another proposal by Nokia, the transform action events; these match the transforms in SVG and CSS, such as translateX and translateY, scale, and rotate, and add the ability to detect the rate of the actions. Having all these high-level events will ultimately make it much easier to author content and applications that work on multiple devices.

And though gestures are different on each device, we do plan to create an informative document detailing some common ones, perhaps something like the excellent Touch Gesture Reference Guide by Luke Wroblewski, Craig Villamor, and Dan Willis. That might eventually lead to some industry consensus about future efforts to define gesture events.

Last but not least, one of the additions I made to the spec after the first draft was to add my co-editors, Sangwhan Moon from Opera and Matt Brubeck from Mozilla, both of whom have already made excellent suggestions we’ve integrated into the spec, and who have volunteered to help me write the spec. It’s a positive sign that we’ve got active participation from so many different browser-engine developers, since interoperability here is the most important thing.

For me, this specification has been about a decade in the making… it’s one of the things I wanted to tackle when I first came to W3C, even before I worked there. So, I’m very excited to finally be doing it, and to help make touchscreens and pen-tablets part of the Open Web platform. With some hard work and a little luck, we could get this spec completed and implemented in browsers by early next year.

So, if you are interested in helping us make sure the Touch Interface spec meets your needs as a user, developer, or implementer… get in touch!

I’m working with a professional designer, Michael, on some graphics for my upcoming MIX presentation.  I’ve worked with designers before on various projects, some SVG, some traditional Web design, but this is my first time working on an SVG project with someone who never used SVG before, and it’s been an interesting experience, which I thought I’d jot down for anyone interested.

Both of us have been busy with other projects, and since we are in different places (me in Chapel Hill NC, him in Atlanta GA), we have only gotten to touch base a few times.  Michael has delivered some first drafts of the graphics, which look lovely, and I decided to dig into the underlying code, confident that I could trim down the file size and thus help browser performance.

My personal graphical editor of choice is Inkscape, which is a fine tool for all its warts (though it’s a little painful on Mac, where it is hosted as an X-11 application); typically, though, I create SVG either programmatically with a script or manually with a text editor (yes, I know that’s crazy… but it can be done).  Michael, being a professional designer, uses Adobe Illustrator, and I am keen to have him use the tools he is most comfortable with.  Since I want SVG to be used by mainstream designers, I want to understand how their tools work and what their workflow is like.

So, in order to make sure that we can roundtrip the files as seamlessly as possible, while still leveraging the features of SVG, I set about today to establish a workflow with Illustrator, TextMate (my go-to text editor, with a custom SVG code template bundle), Firefox, and its native debugger extension Firebug (with its handy “Inspect Element” context-menu selection).  My goal: to make reusable icons out of some of the graphical assets in the larger image, to be referenced by SVG’s <use> element.

The <use> Element

Some earlier graphical languages, especially special-use ones such as CAD languages for engineering, architecture, or mapping, included a native symbol library.  These would normally include a variety of arrowheads, iconic symbols, line styles (dotted, dashed, etc.), and other graphical primitives.  The problem with this idea is that such symbol libraries were by necessity limited in the number of symbols, and usually only had one or two styles for each basic symbol. Not very extensible.  SVG took a different approach.

SVG has the ability to “clone” an image, to reposition or resize it, and to change some of its styling.  This works on a shape or collection of shapes, whether the image is defined in the same document, or in an external SVG file.  The mechanism to do this is the <use> element, and it looks like this:

<use xlink:href="#myImageId"
     x="100" y="300"
     transform="scale(1.5)"
     fill="cornflowerblue"/>

Translated into English, this code says,

Find the image with the id of “myImageId”, and copy it.  Move it 100 pixels to the right, and 300 pixels down.  Increase its size by half again.  Oh, and if it doesn’t already have a color, make it blue. Thanks.

(That final “/” is the part that says “thanks”, a quaint courtesy missing from HTML5.  SVG was specified in a gentler time.)

Thus, designers can provide their own symbols, in the style appropriate to the task at hand.  There are several good SVG clipart libraries that allow SVG authors to share their symbols and simple reusable images.

You can see that if you draw an image of a balloon, with several paths (the bubble, the knot, the string, and maybe a highlighting accent) and some special graphical effects (a gradient, and perhaps a filter effect) all amounting to a dozen lines of code, and you want to reuse this balloon, you don’t want to redraw it each time; it would be a waste of effort, and result in a much larger file.  For print, this may not be as much of an issue, but for a Web-based format, it takes longer to download and uses more memory in the browser, resulting in poorer performance.  So, instead of rebuilding a score of balloons, or copying and changing the original image description, you can simply make one basic model, sans color, and reference that multiple times, with a different color and size.  It’s easier to do, and each extra balloon only increases the size of the file by a small amount, much smaller than adding dozens of duplicates.

To really take advantage of this feature takes a different way of thinking about the design.  A clever designer can even create several image components and recombine them into composites with even more variation.  Of course, it’s not always appropriate for every project, but my current project has quite a few duplicate images.

This is a feature supported by every browser with native SVG (though not every one yet allows you to reference images in external documents).  So, it’s a safe and natural feature for an SVG authoring tool to utilize in its SVG output.

Best Practice: Make liberal use of <use>.

<use> It and Lose It

Illustrator doesn’t support <use>.

Or rather, it does recognize it, and render it… but it promptly converts any instances of the <use> element to simple verbose copies of the referenced image.  I could not find any way to create an instance of <use>.

Also, when creating gradients, Illustrator seems to get overly specific about the position and dimensions (width, height) of the gradient, needlessly yoking them to a specific element at a particular location and of a particular size.  SVG allows you to define a gradient once and reuse it among several shapes, decreasing the number of gradients; Illustrator should make better use of this feature, and so should Inkscape.

This is not a tool created with the Web in mind.

Because of this quirk, my current file size is about 6 or more times what it needs to be.  Let’s see how we can work around that.

Symbolic Gestures

I spent quite a bit of time –hours, truth to tell, because I’m an Illustrator newbie– isolating individual images (trees, benches, etc.) that were used through the graphic as a whole.

When I make “symbols” (for lack of a better word), I like to find their positioning anchor –that is, the point at which they are most usefully positioned– and set that to the {0,0} point, so there are no unpleasant surprises when placing the symbol.  For most symbols, this is the centerpoint of the symbol; if I have an icon (a badge, a logo), I will position it smack dab at the point in my graphic that it should be applied, relative to other images.  For other symbols, it is one of the extremities, where it will meet up with another shape… for a tree this would be the bottom of the trunk where it sprouts from the ground, for an arrowhead it would be the base where it it attaches to a line.  I try to pick the most intuitive anchor for each symbol.

When you drag a shape to a new location, as I did for creating anchors, Illustrator converts the path or attribute values to their new coordinates, rather than simply applying a transformation.  This is something that Inkscape gets wrong (at least, last time I checked).  Nested transforms are harder for developers to work with than simple coordinates.  So, kudos to Illustrator on this one.

Each symbol is best enclosed within a group… in SVG, this is a <g> element (SVG also has a <symbol> element, but in practice, I don’t think it’s worth much).  Grouping allows the designer to collect all the constituent parts of the image together for greater portability.

When a designer works, if they are anything like me, they tweak a bit here and there, sketching out the rough shape, working on another part of the big picture, then focus on adding details.  A problem arises for a developer who has to subsequently dig into the resulting code… elements are added to the document in the order they are created, so a bunch of shapes that appear to be one symbol might be scattered throughout the document, and might even be in different layers.

Best Practice: Group early, group often.  Create graphical objects, and think about graphical / document structure.  Your developer co-workers will thank you.

Being the non-designer developer I am, my first instinct was to group the elements in a text editor, and as a clumsy first attempt to find the different elements, I used Firebug to inspect each element to find its id or unique data I could use in a text search in the text editor, but the different parts were too scattered for this to be effective; pretty quickly, I turned to Illustrator myself to do the grouping visually.

The process for converting these graphics into symbols was made much clumsier by the selection mechanism of Illustrator.  When things are grouped in Illustrator, it logically selects all the bits of that grouped image.  The problem is that it also does this with layers, as well as groups; you can “drill down” from a layer into an individual group, but when you are first trying to select a set of images that aren’t already grouped, this can be twiddly and frustrating.  Illustrator also defines far too many nested groups, decreasing their usefulness and creating markup cruft that makes later programming harder. Using the selection marquee (the draggy box), you can more easily select the constituent parts of the image, but be careful of any background elements.

Best Practice: When creating symbols, select the elements you want as part of that symbol, either with the selection marquee or with shift+click.  Right-click on the selection, and choose “Group” to group the elements, which should also group any gradients, filters, clipping paths, etc. that the shapes use; in the Layers dialog, you will see the graphics collected under a new group labeled “<group>”; double-click this, and give it a meaningful name.  (Note: you might need to right-click again and choose “Isolate Selected Group” in order to the collect the gradients and other graphical effects into the group; these are not shown in the Layers dialog, so I wasn’t sure at what point they are included in the group.)

“Let Me Tell You What I Think You Mean”

As I isolated each image, I deleted extraneous shapes that weren’t part of any symbol.  I made each symbol into its own file, just to keep them tidy.  Each symbol looked great by itself, but when I finally recomposed all the symbols together, the colors were all wrong!  A little digging revealed the problem… Illustrator changes the ids of the “paint servers” –referenceable graphical effects like gradients and filters– as you delete other shapes.  So, when I recombined the symbols into a single file, some of the gradient ids overlapped and defaulted to the first instance… which screwed up the colors.  This is a portability and maintenance nightmare.

Illustrator tried to be a little too clever here.  Like the way it unwinds <use> references and discards the <use> element, it doesn’t play well with a designer-developer roundtripping workflow.  I ran into many such cases where it discarded changes I had edited by hand.  I tried to resize the document dimensions in my text editor, but when I loaded it in Illustrator, made some changes, and saved it again, it scaled down my graphics (which I’d just explicitly scaled up) and changed the viewport to a much smaller set of dimensions.  Hey, Illustrator, if I wanted it that size, that’s the size I would have made it!  Finally, I gave up, and set the dimensions I wanted in a new file I created in Illustrator itself, and copy-pasted from one Illustrator window into the new document, which seems to have taken.  I don’t like programs which introduce inconsistency into a workflow.

What are you smirking about, Inkscape?  You pull the same kind of nonsense!

Copy-Paste Error

After all this tweaking, I ended up with more or less what I want to start with: a document with the dimensions I need, with a set of reusable symbols at the right scale.

But when I tested it out by selecting a symbol on the canvas, copying it, and pasting it in its new location, I was disappointed to find that Illustrator copied the parts of the symbol, but not the grouping… it just added the individual parts of the graphic directly in the target layer.

FAIL!

I messed around a bit more with the interface, and through luck, I stumbled on a different way of selecting the symbol.  By toggling a selection button for the group in the Layers dialog, and copying that (Ctrl/Cmd+C only… Illustrator doesn’t provide a right-click menu on items in the Layers dialog list), I could paste the group.

Best Practice: Copy and paste symbols from the Layers dialog to preserve their logical structure.

This is a far cry from having a <use> element, but I plan to apply a post-processing step to the final document, replacing duplicates with <use> references.  Luckily, Illustrator didn’t give the pasted copy a completely random id, but simply appended an incremented number to the original symbol id, so this should be fairly easy.

My next step is to turn the new document over to Michael, my long-suffering designer, and see if any of this makes sense to him, or if he will come to his senses and throttle me for imposing all this developer madness on him.

To Be Continued…

I recognize that my own inexperience with Illustrator is probably behind some of my missteps (the full extent of which I have shielded from you, dear reader).  On the other hand, a fresh pair of eyes can often spot flaws that could be eliminated where an experienced hand would simply repeat the same mistakes.

This project is far from over, and as I refine my workflow, I’ll report back what I discover.  Some of what I learn may inform future versions of SVG.  Other parts should be changes to the authoring tools.  Hopefully some of it will help help designers and developers work around issues in creating SVG with the tools they use today.

I attended Google I/O a couple weeks ago, and had a great time.  They really put the “hype” in HyperText Markup Language 5, identifying things like the Geolocation API and some of the WebApps Working Group deliverables as part of “HTML5″.  Not quite accurate, but it was a branding exercise, not a technical one.  Notably absent from the things they rolled into HTML5 during the keynote, though, was SVG… in fact, they seemed to go out of their way to avoid it.

But it’s in there.

Behind the curtains, I talked to several Googlers at the conference committed to SVG.  Brad Neuberg, a whipsmart Google engineer and all-around swell guy, is making a Javascript shim that takes SVG (referenced or inline) and renders it consistently across browsers, including Internet Explorer, using the Flash plugin.  Patrick Chanezon and Ignacio Blanco, also of Google, spoke about SVG and Canvas at JavaOne the week after.

It wasn’t a certainty that SVG integration would make it into HTML5 (rather than HTML6, etc.).  I’d been asking for it to be included for a while, but the the editor, Ian Hickson, was resistant to the idea.  At one point, he even suggested supporting some other vector graphics language, even a stripped-down custom format (which was likely a testing of the waters… in a similar vein, rather than simply integrating custom namespaces or RDFa or ITS, he reinvented his own syntax for each instead).  But ultimately, even he knew that wouldn’t fly for vector graphics.  SVG is already too well supported in browsers, in authoring tools, and as a general format for anything else to make sense.

Because it’s all about the platform.  And SVG is part of the platform.

Already, inline SVG (that is, SVG mixed into a single document with XHTML) works in all the browsers that support both XHTML and SVG (that is, all but Internet Explorer), and even worked in plain-text HTML in Internet Explorer with the Adobe plug-in almost a decade ago.  Steven Pemberton points out that that was the true innovation of XHTML, the ability to plug in new syntaxes and new functionality without having to change the core of the language or revisit parsing rules.  It has been argued that XHTML’s extensibility is superior to HTML5′s parsing rules that only work with a set vocabulary.

But the browser vendors were determined that text/html, being more common and robust than XHTML, had a market advantage, and so to fit in with that, SVG needed to work not just with XHTML, but also with text/html.  There were compromises along the way with SVG… probably chief among these are parsing issues.  Unlike XHTML, SVG was designed from the ground up for an XML syntax.  HTML uses a much more complicated parsing that defines error recovery behavior for things like missing quotes around the attribute values, case insensitivity, and non-well-formed markup, and it doesn’t support namespaces properly.  SVG in text/html has to (some say “gets to”) follow some of these rules in order to fit in.  The SVG WG was a bit reluctant to adopt these changes, out of concern that SVG might fly into tag soup, making it harder for existing authoring tools and XML-based SVG viewers, as well as generic XML processing tools.

But ultimately, we had to agree that SVG needs to conform to the platform.

So, we’ve taken a step backwards in some ways: right now, there’s pretty much no non-conforming SVG content in the wild, and the Adobe SVG viewer was fast, and had features still not found in some browsers.  But we’ve now got better error recovery, and SVG in almost all major browsers, which is more robust than a single-vendor plug-in, and is ultimately better for authors and users.  The way to use it with HTML is now better defined:

• as with HTML, for SVG in text/html, you don’t have to use quotes on your attribute values (if the value doesn’t contain spaces, etc.)
• authors don’t need to declare namespaces (in fact, this was the case with the Adobe SVG Viewer already, so not really much of a loss)
• mistakes in casing for element and attribute names will be corrected
• well-formedness errors will be recovered as best as possible, rather than halting rendering
• custom namespaces, such as round-tripping metadata inserted by authoring tools, will not be supported
• most importantly, normal valid, well-formed SVG with no custom namespaces will Just Work, whether as a standalone document or referenced as an external file, or inline in text/html or XHTML.

In fact, though HTML5 is not yet done, SVG inline in text/html already works in special experimental builds of Firefox and Opera, and should work out of the box soon.  Forward-looking authors can already use SVG in modern browsers, using XHTML, and that content will soon work when served as text/html as well. The final barrier to common use of SVG is being solved by Brad’s “SVG Web” shim, allowing SVG content to work in IE.  This will not be a perfect solution; it would be better if IE supported SVG natively without making developers jump through hoops, but that won’t happen for at least a couple years (if at all), and SVG Web at the very least will bridge the gap.  I’ve seen it run, and it’s pretty impressive already.

That SVG has had to bend in the wind gives the SVG language a unique opportunity to innovate.  If the syntax is already going to need an update in existing browsers and authoring tools, we can reexamine some old assumptions and past missteps, and extend SVG to be better integrated with HTML and CSS and existing scripting practices.  And even more importantly, we can examine use cases and make SVG better suited for what authors need to do.

Why is HTML5 a powerful enough platform that Google throws it’s weight behind it, despite the inaccurate and ungainly name?  Is it the error-correcting parsing?  No, that’s a good feature, even if the need for it is unfortunate, but it’s unclear that this is better than XHTML.  Is it the new features like the <video> and <audio> elements, which allow it to compete with the most common use of Flash?  No, that capability has been possible using XHTML and SMIL for years, and is even part of SVG Tiny 1.2 (N.B.: HTML’s script interfaces are simpler and better than SMIL’s, but its lack of synchronization or timing containers is a big step back).  Is it the new WebApp-centric user interface focus?  No, we would probably be better served by a language designed for making UIs (like XUL).

So, what is the true power of HTML5?  Coordination.  Getting all the browser vendors to standardize on a common set of features.  HTML5 is not a technical achievement, it’s a social movement.  It’s an instantiation of the Open Web in the form of several specifications across several groups, and implemented fairly uniformly across all major browsers and platforms.  This is largely in response to user demands to the browser vendors.  They are tired of the special-casing and workarounds that make their lives and jobs harder and less satisfying, that make users struggle… they want a consistent platform, one without script libraries or shims or plug-ins or arcane rules.  And the browsers realize this, and know that a rising tide raises all ships.

My personal advice to vendors who have excellent authoring tools, but who want to also control the runtime environment?  The Web has moved on from vendor lock-in and black-box plug-ins.  In a world where there is coordination between multiple vendors to deliver and support a central set of technologies, where everyone is unifying on a standard Open Web platform, from Apple to Google to Mozilla to Nokia to Palm to Opera to Vodafone.  Nobody wants the hassle of being the only one to maintain the platform, but they all want the ability to change it without going though the gatekeeper of a single vendor.

It’s all platform games, and if you want to play, you have to jump to the next platform.  Google knows this… they are hosting the next SVG Open, and are aggressively pushing the whole platform.  Looks like a fun year ahead!

A while back, Henri Sivonen stumbled upon a diagram on the W3C site of the technology stack, a curvy-block Venn-diagram overview of the different technologies from W3C and where they fit into the Big Picture.  It’s an attractive diagram, but it oversimplifies things, and shows a decidedly W3C bias toward the Web.  It’s clearly been used past its expiration date, and those who consume it might feel a bit queasy.

Henri strongly criticized this depiction.  He rightfully points out that HTML is not included in this vision (showing XHTML only, which looks a bit silly these days), but then complains that HTML5 and XHR are not included in the diagram.  But of course, HTML5 isn’t even in Last Call yet, much less a W3C Recommendation, so it doesn’t really belong in that particular diagram (oddly, Henri credits XMLHTTPRequest to WHATWG, rather than its originator, Microsoft).

To his credit, Henri put his money where his mouth is, and took the trouble to make a diagram of the Web stack the way he sees it, which presumably better reflects the “real Web”.  It omits many of the W3C technologies, and inserts some of the more common ones that aren’t from W3C (most notably Javascript).  It’s a good diagram, but oversimplifies the landscape dramatically.  He follows the W3C diagram in putting “Internet” at the base of the stack, but doesn’t correct it to include such ubiquitous technologies as email or chat (XMPP, IRC, etc.) even though those are often part of browser-based technologies (GMail, et al).  Of course, he deliberately omits intranets-connected devices, even though that’s part of the browser world, because the official doctrine of the WHATWG is that the intranets (including those that are partly open, such as at universities) are “not the Web”.  I will also quibble that he overlaps Accessibility only with HTML, not with SVG.  But most glaringly, he includes Ogg Vorbis and Ogg Theora, though they aren’t (yet!) really used on the Web, and omits the dominant technologies in that space, MP3 and Flash (and more specifically, H.264).  He covers himself here by saying that this is for a “contemporary browser”, with the insinuation that it doesn’t include plugins, though to users and authors that is a pale distinction.  He also neglects PDF (ISO 32000), which is all too prevalent on the Web, and which several browsers do render (if I recall correctly).  So, it’s not really a picture of the real-world Web stack, either.

The Frames remind us, in their song in “God Bless Mom”:

You’ll see how hard it can be
To keep your side of the deal,
And you’ll see how hard it can be
To keep one foot in the real.

So, his diagram is flawed.  So what?  Why am I picking on it?  I’m not, really… it’s a good diagram, and it serves a certain purpose.  I’m picking on that purpose itself.  Henri was quick to criticize the W3C diagram (on a page where nobody can comment, I note), not because it wasn’t accurate, but because it advanced his agenda to do so (just as the W3C was advancing its agenda by making the original diagram).

Data visualization, like statistics or slogans, has a way of territorializing the map, in a kind of graphical gerrymandering. I’m sure that Henri didn’t mean to make such glaring omissions, but I’m equally sure that the creator of the original W3C diagram didn’t have sinister motives either.  People get busy, and reuse what they have to hand that meets their needs, even when it’s sometimes not quite correct.

I really respect Henri, but what he fails to understand here, or at least to admit, is that different data visualizations are best suited for different audiences and different purposes.  He’s shown a clear bias in his diagram toward depicting the “Open Web Stack” (a bias I have to admit I share) and toward desktop browsers (which I find too narrow), with a Web developer audience.  That’s perfectly cromulent.  But his diagram is not at all suited toward showing the different work going on at W3C, and where it fits in the larger Internet, in an executive summary.  Both the offending W3C diagram and Henri’s own diagram are gross oversimplifications… which is the point of data visualizations.  The map is not the territory.  If I were to make a diagram that encompasses the Web tech landscape, it would include both W3C technologies and technologies from other sources, and code the origins with styling; it would clearly indicate which technologies are open (that is, not proprietary), which are under development and which are stable, and link each node to the definitive resource for that tech; it would not stack them up in a neat little box, but would show the interconnections via lines.  And it would serve a different purpose than either of these other diagrams.

Why was only XHTML included in that W3C diagram, and not HTML?  Wishful thinking.  Say it enough times, and it just might come true, and a picture is worth a thousand words.  We’re all dreaming the Web we want into reality, every day.  I’m tired of the false dichotomy that’s too often drawn between W3C and its members and participants.  How about we lay off the divisive rhetoric?

While answering a question about styling SVG with CSS in the Freenode #svg IRC channel (yes, people still use IRC), I threw together a simple example to illustrate.  I like to do this, since in keeps me in practice, and gives me a chance to check the state of current implementations in the fast-changing world of SVG browser support.

The question was how to use CSS stylesheets, both internal and external, with SVG in Inkscape.  While I know that Inkscape makes heavy use of CSS inline style declarations on elements (more than is to my personal taste, to be honest), I didn’t (and don’t) know if it has any UI features for adding internal stylesheet blocks, or links to external stylesheets.  So I made a little test…

svgstyle.svg

<?xml-stylesheet type="text/css" href="svgstyle.css" ?>
<svg xmlns="http://www.w3.org/2000/svg"
     xmlns:xlink="http://www.w3.org/1999/xlink"
     width="100%" height="100%" viewBox="40 0 145 70">

  <style type="text/css"><![CDATA[
     ellipse
     {
       fill: lime;
       stroke: green;
       stroke-width: 10px;
     }
  ]]></style>

  <rect id="inline" x="50" y="10" width="30" height="50"
        rx="5" ry="5"
        style="fill:lime; stroke:green; stroke-width:3px;"/>
  <ellipse id="internal" cx="103" cy="35" rx="15" ry="25"
        style="stroke-width:3px;"/>
  <circle id="external" cx="150" cy="35" r="25"
        style="stroke-width:3px;"/>
</svg>

svgstyle.css

circle
{
   fill: lime;
   stroke: green;
   stroke-width: 10px;
}

Which looks like this (at least, in Firefox, Opera, Safari, and presumably Chrome):

Both internal styles worked fine in Inkscape, but the external reference did not, showing just a black circle.  I’m not really surprised.  Note the clunky way that you have to link to external stylesheets in SVG:

<?xml-stylesheet type="text/css" href="svgstyle.css" ?>

That’s really XSLT-licious. I have to confess, I don’t really care for PIs (insert dated Magnum PI joke here), or XML prologs in general… they seem somehow clumsy and un-XMLy, like a throwback to SGML. And as far as I know, they can’t be created or changed dynamically via clientside script. Compare that to the relatively easy and straightforward X/HTML way of linking to external stylesheets:

<link href="svgstyle.css" rel="stylesheet" type="text/css"/>

In the SVG WG, we intend to allow authors to reference external stylesheets in a manner a little more modern and consistent with what authors are already used to using, in some upcoming spec. We could do that a couple of different ways. We could allow them to use an xlink:href attribute on the <style> element, in the same way we currently treat the <script> element (that is, if there’s a resolvable external link, we use that, otherwise we use the child content of the element), or we could add a <link> element like X/HTML, or both. I kinda like the idea of allowing either.

To that end, I made a couple of tests, just playing around, to see if any browsers accidentally supported either of those options, by merit of having some shared codebase with X/HTML in their implementations. Unsurprisingly, neither worked.

But then I had another idea… use the an <xhtml:link> element in the XHTML namespace…

<xhtml:link href="svgstyle.css" rel="stylesheet" type="text/css"/>

With this rather happy result, which works in at least Firefox, Opera, and Safari.
external-link-xhtml.svg:

I don’t know if this is by design, or if it just fell out of the model, but I have to say I’m pleased as punch that it works. If nothing else, it’s a great proof of concept for adding more ways of linking to CSS in the SVG spec. This one goes in my toolkit, and I’m going to try to ensure that it gets standardized, if it’s not already there in some corner of some other spec. Probably somebody already knows about this (hell, probably everybody does, and I’m late to the party), but for me it was a w00ty discovery.

Update: heycam speculated that you might also be able to use @import rules, and indeed you can:

<style type="text/css">@import url(svgstyle.css);</style>

Since before the iPhone was released, there was speculation that it would support SVG.  After all, WebKit supports SVG, and Safari is based on WebKit, and the iPhone uses Safari.  But alas, the build of Safari that went on the iPhone did not include SVG support… nor, more famously, did the iPhone support Flash or Silverlight. Rats!

Realistically, it doesn’t make a huge impact in the total deployment of SVG on mobile devices.  For all that they are cool, iPhones make up a pretty small margin of mobile devices.  Opera is probably deployed on more devices, and it’s supported SVG for a while.  SVG is used on the BlackBerry, as I understand.  The BitFlash and Ikivo SVG players are deployed on something like half a billion phones, both for content viewing and as the GUI of the device itself.  But… still, having it on the iPhone would be a bit of a coup, and would enable lots of neato Webapps.

And as of yesterday, when I updated the firmware on my iPod Touch, SVG is now supported natively!

It even support scripting! Mouse over the lil buttons below the SVG logo…

But not yet SMIL (declarative animation) it seems…

WebKit itself does support SMIL, so I’m guessing it’s only a matter of time before it works on the iPhamily as well.

Thanks to Luis Vielva for bringing this to the SVG community’s attention.

W00t!

Update: Some folks have asked me what version I’m running: it’s v2.1, which is apparently not available for the iPhone itself until tomorrow.

I’m here in Sydney to attend the SVG Working Group F2F. This marks the third time I’ve done this trip. When I first joined the SVG WG, I flew here for my first face-to-face meeting, and SVG 1.2 Tiny was in the can, no new features, only such changes as required by Last Call comments; this was a bit frustrating for someone who set out to represent authors, since I really felt some new functionality was needed. But it had already been too long between versions, and SVG 1.2 Tiny just needed to be published before new work could be started, so I took one for the team. I was assured that it would be published and done within months.

Well, the best-laid plans… I was here last January again, and the same situation applied. We’d received an avalanche of Last Call comments, from typos to tweaks to time-wasters to trip-wires. It amounted to a sort of Denial of Specification attack. Many of the comments were valid criticisms (though a few of those couldn’t be helped due to legacy or other dependencies), but an equal number were ideological. But all of them had to be dealt with. This had occupied the past year, and looked to occupy the foreseeable future, but we soldiered on. The biggest chunk of work was dealing with errata for SVG 1.1, and with the test suite and implementor feedback (which is the point of “CR”, the Candidate Recommendation stage of becoming a W3C Recommendation). We also split out some functionality into the WebAPI WG, so it could be more generally used by other Web technologies besides SVG. So last year, it looked like one step up and two steps back.

And now another year has passed. In this last year, I came to work for W3C directly, and the SVG WG has two new Chairs, one each from a desktop and a mobile implementor (Opera and BitFlash). SVG 1.1 is better specified, due to errata, and better implemented, due to good, interoperable native implementations in Opera, Safari/WebKit, and Firefox. SVG 1.2 Tiny is widely deployed on mobile phones (more widely than Flash Lite), due to excellent implementation by BitFlash and Ikivo, and has a good test suite that’s still growing (and will keep growing even after it’s published… can’t have too many tests). We’ve cleaned up the SVG 1.2 Tiny spec, and made progress on ancillary specifications. We’ve pared down what needs doing in order to move forward to the next stage on the Recommendation Track. We’ve learned from past mistakes, and we’re working more openly with implementors and the general public.

And we’re talking about new features. We spent a day this F2F planning several small modules for the next year or so that will add capability to SVG, and make it easier to author compelling content in this iPhone age. SVG was a bit ahead of its time (especially as regards uptake in desktop browsers), but with the delays of the past couple of years, competing technologies are pulling ahead. We’ve got an aggressive plan that includes bringing in more direct feedback from designers and developers, and targeted feature additions.

When I joined the Working Group three years ago, SVG was struggling and morale was low. But now we’re really excited by all the recent progress and the momentum behind SVG. This looks like a good year for SVG.

ACID3, that is. Most of you will have heard of the ACID tests put together by the Web Standards Project in order to promote interoperability among browsers. Microsoft recently made a hit in the blogosphere by announcing that the next version of their browser, IE8, passed the ACID2 test, showing their commitment to Web standards.

Ian Hickson, who wrote the second ACID test, is now working on ACID3. He recently started a contest to get contributions from the community on what features they want tested.

SVG has had a huge surge of popularity in the past few years; it’s now used on Wikipedia and Google Maps, and largely implemented in 3 of the 4 major browsers (it works in Opera, Safari, and Firefox). There are a few inconsistencies between implementations, so we thought ACID3 would be a great chance for a push for SVG interoperability. The SVG Working Group, most notably Erik Dahlstrom of Opera and Invited Expert Cameron McCormack, devised a few tests that we hope will be included in ACID3.

You can read the explanations for the tests, and see the tests themselves, in Erik’s email. Let us know what you think, and if you support the inclusion of SVG in ACID3 (and let Ian Hickson know, too). Maybe by the release of IE8, it will pass ACID3 –and any SVG tests– too.

[originally published on the W3C Questions and Answers blog]

Update (25-01-2008): Just so you know, our efforts paid off.  Hixie accepted our tests, and SVG will be in slots  70-74 (or 75) of the Acid3 test.  I think this will be a great win for interoperability, and nice acknowledgment that SVG is a first-class citizen of Web architecture.  Thanks again to everyone who contributed to the tests and to the conversation, and to Ian Hickson for putting the tests in (and for driving Acid3 in general).

Okay, first a disclaimer: none of this speculation comes from “insider information” I’ve gotten as an employee of W3C; but being surrounded by the swirl of standards, I suspect I may have picked up on a zeitgeist. I’m interested in seeing how well my predictions play out.

Prediction 1: Microsoft will merge its browser and office platforms

That’s what I would do if I were them. Right now they are in a bit of mindshare competition with online apps, like Google Docs; as Web apps transition to the desktop (like, the downloadable desktop GMail client), Microsoft will take their core competencies (the desktop Office suite) and make it more Web-friendly. They’ll take advantage of the fact that desktop apps are often more robust and speedier, and tie in the chief benefits of Web apps: distributed authoring, ease of deployment, and ubiquitous availability. To enable the latter, they also make a Office-lite Web app, sold as “software as a service”, for those who’ve bought the real product (though of course it won’t be as good as their desktop version); they might open this up in the razor-and-blade sales model.

They will merge two of their products, SharePoint Server and Exchange Server, into a sort of portal/wiki thing that ties into their new Office Browser. They will probably integrate chat (MSN, or whatever) into the whole thing, so you have some uber-communications suite: email, chat, collaborative distributed document creation, and all that enthralling office goodness.

This could be IE8 (or IE9), or a totally new product, but it will probably be a new version of MS Office. It will be their typical monolithic entry into the agile Web space. And of course, it will use OOXML as a native format.

Why I’m wrong: I think this would be smart, but the different divisions of Microsoft actually compete for resources. I wonder if they could play nicely together for long enough.

Update (25-01-2008): I’m actually going to Microsoft next week to talk about standards stuff. Maybe I will get an inside scoop.

Prediction 2: A cross-platform browser based on WebKit (but not Safari) will supplant Firefox

WebKit has a cleaner, faster (if less complete) engine than Mozilla, and it’s already been reused by Apple for Safari, Nokia for their mobiles, Adobe for Apollo (I think), and by Google for Android. But all those are controlled, as far as I can tell, by their name-brand deployers.

I think a new, more open repackaging will happen. It will be standards-based, and truly open-source, from the engine on up to the executable. It will implement Widgets (as being specified at W3C), so that people can make simple desktop applications based on common Web technologies. But the key enhancement will be a new, cleaner, lessons-learned, easier-to-author Extensions platform. Extensions, in my mind, are the most brilliant thing about Firefox, allowing slightly-above-average users to create new features for the browser, or to use features added by others; this gives great community buy-in. But it will be even easier to do in this new browser; this will probably necessitate a new XUL-like language, too, so people can add chrome, or it might just be in the form of script APIs. But what would be even smarter is to have the Extension editor, and Widget editor, built right into the browser, so making a new theme or feature or widget is as point-and-click as possible, with common tasks encapsulated (though for advanced enhancements, some scripting will be needed, in a smart editor with code completion).

Oh, and it will have SVG support.

Naturally.

Why I’m wrong: This is just my dream browser. Browsers are a hard market to break into, because there’s established competition, and you’re not making a product you can sell… you have to give it away. There’s already a lot of momentum around other efforts. But this is the browser I want. Too bad I’m so lazy.