There has been a heated argument recently on the W3C Canvas API mailing list between accessibility advocates and browser vendors over a pretty tricky topic: should the Canvas API have graphical “objects” to make it more accessible, or should authors use SVG for that? I think it’s a false dichotomy, and I offer a proposal to suggests a way to improve the accessibility potential of the Canvas 2D API by defining how SVG and the Canvas 2D API can be used together.

This brings together some ideas I’ve had for a while, but with some new aspects. This is still a rough overview, but the technical details don’t seem too daunting to me.

Retained-Mode vs Immediate-Mode Graphics

First, some background…

SVG is a retained-mode graphics format, meaning that each shape persists as a separate object, with distinct characteristics such as location, size (bounding box), and precise outline (stroke). These characteristics can be acted upon within the object structure (DOM) as discrete entities (or groups of entities) for manipulation by script or by user-interface actions (e.g. mouse clicks or keyboard tabbing); one of these interactions may be hit-testing.

The Canvas 2D API is an immediate-mode API, meaning that it allows the programmer to create visual shapes as individual pixels that are unstructured; in other words, if a red circle and a red square are drawn which overlap one another, there is no way to distinguish these from one another, or from the background canvas element (though a programmer could maintain a separate structure in script).

With retained-mode graphics, each object which is added to the DOM increases the memory “weight” of the page or application, potentially slowing down the browser or otherwise decreasing performance; by contrast, immediate-mode graphics, which are not represented as objects in the DOM, do not decrease performance, which is an advantage over retained-mode graphics.

Some accessibility advocates have noted that hit-testing and zooming are critical accessibility features of graphics; this requires the ability to access the location, size, and other characteristics of individual graphics. Unfortunately, adding this retained-mode feature to an immediate-mode API effectively removes the primary advantage of using an immediate-mode graphics system.

However, by defining how SVG and the Canvas 2D API can be used together, a model may be created which allows authors to easily use both together, each for their appropriate strengths. This approach involves two aspects: defining the behavior, and a potential shared API syntax.

Defining Behavior

Currently, the Canvas 2D API is only defined to work with the <canvas> element defined in HTML5, and it does not allow SVG to be drawn or imported into the Canvas. However, it could be extended to work inside SVG or to allow SVG to be used inside Canvas.

Canvas in SVG

At its simplest, the Canvas 2D API could simply be defined to allow authors to draw on the SVG raster <image> element, just as it does in the HTML <canvas> element, but there does not seem to be a reason to limit it to this element. The Canvas API could be defined to draw to any SVG element, rendering or structural, as an overlay. It would be merely a visual effect, and would participate in the rendering structure, including “z-index”, only as its “target element” does.

SVG in Canvas

SVG shapes could be allowed to be imported into the <canvas> drawing context, such that they render there but are also inserted into the DOM or shadow DOM. Currently, as I understand it, this is not allowed for security restrictions involving content from potentially different sources, but for content that does not have external references (for example, a <use> element to another site, this should not be a security risk.

SVG in the Shadow Tree

The accessibility features of the Canvas 2D API include a “shadow tree” and focus ring for those elements, such as text, which are not otherwise accessible to users with limited vision. Each such “graphic piece” which is created and painted to the screen is also represented in this underlying shadow DOM (IIRC).

SVG elements used in conjunction with Canvas could also be placed in this shadow tree, so that navigability is preserved. In order to save memory usage, a concept like the <use> element in SVG could alternative be used to “point to” an SVG in the main DOM from its appropriate place in the shadow tree.

SVG elements could either be used as visual shapes themselves, or could be invisible “hot-spots” for specific areas of the screen.

Common API

I have suggested before that we create a simple graphics API that will allow either 2D Canvas (immediate-mode) or SVG (retained-mode) images with the same set of methods and parameters, with the only difference being which “mode” the author selects to have as the instantiated form. If the author draws a circle to the canvas context, it would simply be drawn to the screen with no structure; if the author draws that same circle to the SVG context, it creates a <circle> element with the appropriate attribute values and style and inserts it into the DOM.

There would be certain differences in some of the renderings, because of the different rendering and compositing models of SVG and Canvas, and there would be certain specialized functionality and features each would have, but the majority of the visual appearance would be the same.

This would allow authors to learn only one API for 2D graphics for the Open Web Platform. With this ease of learning and use, the author could decide on a case-by-case basis, even within the same application, which mode works best. It would provide incentive to authors to use each to its best effect.

I call this approach COG: Common Open Graphics.

Example

An example of the effective use of SVG and Canvas together is a hypothetical game in which the user pilots a spaceship amidst an asteroid field.

The background of the game includes animated nebulae, passing comets, rotating planets and moons, and other visual effects which are not interactive, and which do not affect the play of the game, but which add visual appeal. The foreground of the game (the “playable area”) includes asteroids, enemy ships, and energy beams that the player must dodge or shoot at, which increase in size the closer they get, and which benefit from hit-testing and keyboard focusability. The middle-ground of the game contains many smaller asteroids and ships that are not yet interactive, and so don’t need hit-testing or focusability.

Adding all of these graphics shapes as retained-mode objects in the DOM would unnecessarily increase the memory footprint of the game, degrading performance and playability. However, since only a small number of these elements (those in the playable area) need the retained-mode interactivity, the majority of them could be drawn with immediate-mode characteristics, then when a certain “distance threshold” is reached, changed to interactive SVG elements, perhaps using the same shared API. This preserves the accessibility of the game, while also causing minimal performance degradation, and possibly even helping performance, since the hit-testing doesn’t need to be calculated in script. From an authoring point of view, using the same shared API means that both types of shape, retained-mode and immediate-mode, are equally easy to generate and maintain.

To enhance the visual appearance of the SVG asteroids, the finer control of the Canvas API could even draw an overlay on the SVG stroke and fill areas, much as filters do, to show craters and other surface details in a more fine-grained manner then can be done efficiently in SVG. Since this would be drawn specifically on the SVG element, the element itself would still provide the geometry and position characteristics.

Caveats

This proposal has two caveats:

1. Canvas can only be extended to be as accessible as SVG is, in this model. Even with object retention, there are still many enhancements needed to make SVG truly widely accessible. However, as plans for SVG accessibility improvements are made, these would be largely importable directly into the Canvas API, rather than working in parallel on two unrelated but overlapping graphics models.
2. Simply because the Canvas 2D API may be made more accessible, does not mean that all Canvas API applications or content will be automatically accessible, any more than all HTML or SVG content is accessible. Authors will still need to use best practices and take specific steps to make their applications accessible. This proposal only provides the means and the incentive to do so.

Feedback

I’m interested in feedback on this, either in the comments section of this blog, or on the W3C Canvas mailing list, public-canvas-api@w3.org.

Update

Someone pointed out offlist that this post emphasizes the accessibility aspect of this proposal, because that was the context of the most recent discussion, but this is a win for anyone who wants to do graphics in the browser, for any reason, not just accessibility. Like any good accessibility feature, my proposal has general benefits, so it is more likely to be used in more content. In the example scenario above, the developer gets the hit testing for free, regardless of whether or not Accessibility Technology is involved. Other general advantages include ease of learning and use of graphics by developers (as mentioned above), sharing of features between modes (like using Canvas.toDataURL() to save an SVG as a PNG), pixel operations (like painting or detecting a color at a point) in SVG, metadata (like microdata, RDFa, or microformats) in select parts of a Canvas app, and so on. I’m sure that clever people will come up with a whole boatload of cool things to do with a combined retained-mode/immediate mode API that I couldn’t imagine here.

Offered without comment.

Post about HTML5 logo and overblown “controversy” surrounding it:
1,078 hits on the first day.

Post about details on W3C touch events spec, including a roadmap and future plans:
273 hits on the first day.

Last week, I published the first draft (and subsequent updates) of the Web Interface specification, which defines touch events. This is the first spec from the new W3C Web Events Working Group.

Peter-Paul Koch (PPK) gave it a positive initial review. Apparently, others thought it was news-worthy as well, because there were a few nice write-ups in various tech sites. Naturally, cnet’s Shank scooped it first (he has his ear to the ground), and it was fairly quickly picked up by macgasm, electronista, and Wired webmonkey.

I thought I’d go into a few of the high-level technical details and future plans for those who are interested.

Touch interfaces are nothing novel, nor is using them with Web content, but there are enough differences from mouse interaction that specifying them should lead to easier development and better user experiences on the Open Web platform (or in HTML5 webapps, if you prefer that buzzword).

The most important thing to note for multi-touch events (using more than one finger, say) is that you want the different touch-points to be considered together, not as completely unrelated events. The WebKit implementation, used in the iPhone and Droid smartphones, as well as many, many others, used a good model for this, with each event exposing lists of currently-active individual touch-points; with wide deployment of WebKit already, it was an obvious starting point. PPK’s blog post was off on one minor detail here: while I did base the spec on WebKit’s behavior, I didn’t base it directly off of Apple’s documentation; I wrote it from scratch, for better or worse, and extended it for broader applicability.

One of these extensions was to take PPK’s suggestion to add ‘touchenter’ and ‘touchleave’ events, the equivalent of the ‘mouseenter’ and ‘mouseleave’ events that we finally specified in DOM3 Events, which work much more intuitively than ‘mouseover’ and ‘mouseout’.

I also extended the WebKit model to add ‘radiusX’ and ‘radiusY’ attributes, to indicate the general size and shape of the thing that touched the surface… say, a fingertip or a stylus (a fancy way of saying “pen-like stick”), described as an ellipse. I’ve been soliciting use cases and requirements for a couple of years, and this is meant to cover a similar request across different devices.

That leads me to mention the other use case for this specification: pen-tablets. Almost a decade ago, I was writing a browser-based SVG editor called SVG-Whiz (long since decomposed into digital fertilizer), and I wanted to be able to use my Wacom tablet with it. It worked okay as a mouse replacement, but I had to choose the stroke-width manually… I couldn’t use the tablet’s built-in pressure sensitivity.

We’re about to change that. Another change to the WebKit model is the addition of a ‘force’ attribute to indicate the level of pressure used on the touch or tablet interface. This will be useful for SVG or Canvas API drawing apps, games, and myriad other purposes. For example, most touch screens don’t have the capability to detect hovering (some capacitive screens being the exception)… this is a challenge for accessibility, but perhaps the pressure sensitivity—expressed in the ‘force’ attribute—could be applied to that end. We may specify a new related CSS mode, or simply define when existing ones apply, such as triggering the CSS ‘:hover’ pseudoclass past a certain pressure threshold (device-specific and user configurable, of course).

(On the subject of new CSS properties, Nokia has made the suggestion of haptic feedback properties for touchscreens, based on their implementation. This could be a very interesting addition to touch interfaces, even if it’s not available on all devices. That work would be handled by the CSS Working Group.)

In addition to pressure-sensitivity, pen input has other characteristics that distinguish it from touch, including the angle of the pen, so we may add that as well. Pen-inputs, either through tablets or stand-alone digital pens, often include particular mode-shifting buttons, so we have added attributes to detect when the alt, shift, and control keys (or buttons) have been pressed.

So, to cover a wide range of devices, we will be defining some capabilities beyond basic touchscreens: multi-touch, touch area, force, perhaps pen-angle, and even haptic feedback. Not every feature will be available for every device, but they will be useful for those that do, but we want the spec to be future-proof and device-independent.

This leads me to another touchy area: gestures. One of the common requests I’ve gotten is to add events for specific gestures, like pinch, swipe, twist, flick, etc. There are two problems with trying to standardize this right now:

1. Each device and each platform has its own gestures, sometimes several to do the same thing, and there is still room for innovation there;
2. It’s a patent minefield.

Instead, we intend to specify higher-level “intentional” events, like zooming in or out, panning, navigating, editing, etc., and define a way to map between the physical actions (touching the screen and moving your fingers) and the intention event (zooming in); it also means we can map between input on other devices, like keyboard and mouse, and the intent behind those actions. An additional aspect of this is another proposal by Nokia, the transform action events; these match the transforms in SVG and CSS, such as translateX and translateY, scale, and rotate, and add the ability to detect the rate of the actions. Having all these high-level events will ultimately make it much easier to author content and applications that work on multiple devices.

And though gestures are different on each device, we do plan to create an informative document detailing some common ones, perhaps something like the excellent Touch Gesture Reference Guide by Luke Wroblewski, Craig Villamor, and Dan Willis. That might eventually lead to some industry consensus about future efforts to define gesture events.

Last but not least, one of the additions I made to the spec after the first draft was to add my co-editors, Sangwhan Moon from Opera and Matt Brubeck from Mozilla, both of whom have already made excellent suggestions we’ve integrated into the spec, and who have volunteered to help me write the spec. It’s a positive sign that we’ve got active participation from so many different browser-engine developers, since interoperability here is the most important thing.

For me, this specification has been about a decade in the making… it’s one of the things I wanted to tackle when I first came to W3C, even before I worked there. So, I’m very excited to finally be doing it, and to help make touchscreens and pen-tablets part of the Open Web platform. With some hard work and a little luck, we could get this spec completed and implemented in browsers by early next year.

So, if you are interested in helping us make sure the Touch Interface spec meets your needs as a user, developer, or implementer… get in touch!

I was involved in rolling out the new HTML5 logo for W3C; I wasn’t the person in charge, but I helped out with some aspects of it.

The interesting thing to me was the reaction to the logo, aesthetics aside. (I like the logo personally, but I also like the retro-futurism of Soviet Realist art, like the statuary in Memento Park in Budapest, and comic books.) I loved the parodies and jokes about the logo, but I was both unsurprised and disappointed at the some of the negative dialog (and in some cases, monolog) about what the logo is meant to represent.

Just like the Gap and Starbucks logos, there were going to be some people who would be unhappy because that’s what makes them happy. There would be no pleasing them, and it’s senseless to try.

But there was another set of people that we really value, and we wanted to make sure that they were comfortable with the logo: the perfectionists. As you can imagine, it’s not the easiest group to please, but they are the ones who make standards work, by reviewing the specs to make sure there are as few errors as possible, by creating exacting tests to ensure interoperability between browsers, and by teaching web developers the precision of terminology and best practices that lead to high-quality web sites and webapps… usually on a volunteer basis.

These are our people, and we wanted to please them. But at the same time, a logo has to serve lots of different communities… including managers and clients who don’t know or care that there is an “HTML5 specification” (whatever that might be), and people throwing around the word “HTML5” like it’s birdseed at a geek wedding. Even people who know better use “HTML5” as shorthand for “Finally, browsers are doing all this cool stuff I want to play with! WHEEEE!”

So, how do you strike a balance between people who want us to say, “This logo is for the HTML5 specification only, and we will sue anyone who puts it on a site that uses HTML4.01+CSS3” and those who want us to give them the shiny and shut up? You make an “identity system” which includes sub-icons for specific technologies and general technology classes, and you say that the identity system, in toto, represents HTML5 and the peripherally-related technologies like CSS3, SVG, WOFF, Geolocation, etc.

And how do you screw it up? You oversimplify in the FAQ and use the word “logo” instead of “identity system”, and let people infer that the main logo, the one for HTML5, is meant to represent not only HTML5, but CSS3 as well. (Interestingly, some of the more vocal people who complained on Twitter don’t mind the geolocation API or SVG being conflated with HTML5 nearly so much as CSS3… that seems too fine a distinction to me.)

But we knew we would make some mistakes. We went out of our way to explain to people that we were not trying to shove it down their throats… that we wanted to have a community conversation about it.

Once we had something we thought was ready to share, we released it to the public, rather in a rush. We took to email, to Twitter, and to blog posts and comments, to collect feedback and clarify our message. Turns out 140 characters is not enough to convey much nuance. After a couple of days of seeing if the message was clear enough, giving people a chance to have a second look and see if the idea of a “logo system” stuck, we decided it wasn’t good enough. We updated the FAQ to split out the HTML5 logo from the sub-icons. That seemed clearer to people, though it’s really saying the same thing in a different way… but the way you say something matters.

And that’s my point, here: The way you say something matters.

W3C started the conversation about the logo by clearly stating, and reinforcing consistently, that we were interested in feedback and we were listening to what people were saying. Sometimes people didn’t believe us, but we were. (And I sincerely appreciate Jeremy Keith taking the trouble to explain what was frustrating him about the “logo / identity system”.)

But some people didn’t hear that we were listening, because they were too busy shouting. Shouting things like “Fire W3C Communications Person Who Led Messaging”… I’m not the person that’s targeted at, but I found it a rude and mean-spirited thing to say nevertheless. (Pet peeve: I can’t stand when people make a public rant in a post and don’t allow others to comment on it… it demonstrates a certain lack of accountability.) Tantek Çelik has been in Web standards for a long while, and knows exactly who the (overworked) Comm person is whose job he was calling for, and I think that’s irresponsible.

Tantek compounded this with a tweet that somehow manages to pretend that W3C wasn’t listening, but all the shouting forced us to change our FAQ, then goes on to say in his next blog post:

Yet with W3C’s first draft HTML5 FAQ, we almost gave up. Almost. But apparently sometimes angry rants can effect change. Sometimes passion works.

Within 24 hours of the last of our posts, W3C updated the FAQ, removed the errant entry, added more clarification

You know what also works? Having a calm conversation, reading the FAQ carefully and seeing where a mistaken impression (or even an outright mistake) might have been made, and making suggestions to the people who have invited you to share your thoughts.

Folks, you didn’t need to get angry. Yes, there was some ambiguity in the FAQ, and that was a mistake… one we were sorry to make and happy to correct, only a couple days after the logo was released. But the result of all the shouting was that, several days later, I’m still talking to people who think that W3C is sending this mixed message with this logo, because that’s what everyone was shouting about… and all the clarifications and updates now aren’t going to put that spilled milk back in the glass, since you sent out the cry to arms. A few simple words and a modicum of patience and trust would have served your cause—the precision we all value—much better.

W3C went out of our way to give you a voice. You’re long-time contributors to standards. You’re all insiders… use your inside voice.

With SVG being integrated more and more into HTML5, both included via <object> and <img> elements, and inline in the same document, some natural questions about SVG and CSS are receiving more focus. This includes box model questions like background and border, and pointer events.

I’m interested in comments from the community on what direction SVG should take.

It’s pretty clear that the intent of the SVG 1.1 spec was that the box-model CSS properties must not apply to any SVG elements. In addition to what the spec states on the subject, my conversations with people who participated in the SVG WG at the time leave me with no doubt on the matter.

The question is, is that the behavior we want? How hard or easy is it to change that in modern SVG+HTML+CSS implementations? What is most useful and intuitive for content authors?

In my opinion, we could specify that some aspects of the box model apply without screwing with SVG’s coordinate-based positioning system. Here are my initial thoughts on some specific properties:

Borders

A borders is different than a stroke. An SVG element, including any graphical (<rect>, <circle>, <path>, etc.) or container element (<svg>, <g>) can have both a stroke (which conforms to the geometry) and a border (which conforms to the element’s bounding box). A common use case is setting the graphics apart from other content, or doing mouseover highlights of an SVG element; for example, when I mouse over or focus a circle, it would be much nicer, and just as semantic, to simply have a CSS property draw a rectangle around that element as a highlight than for me to generate a rectangle based on the bbox and insert that into the DOM, keep a pointer to it, then remove it when I mouse out.

Paint servers and similar resources (gradients, filters, etc.) should not have any border. Note: <defs> is a non-rendering element, akin to <head>, so no border would be visible for that either, but if the content of a <defs> is <use>d, any border properties would apply in the same manner as for other properties.

The border should not have any effect on the layout or rendering; it would be strictly visual.

Margin

No effect on SVG content, but may affect HTML content around that SVG content. So, an <svg> element with 'margin: 20px;' has a 20-pixel gap around the SVG content, just as if it were <img style="margin: 20px;"/>

Alternately, maybe margin only has an effect for <svg> elements; that might be simpler to specify, implement, and understand for content authors.

Padding

Again, this has no effect on SVG content, but it does affect how the border is drawn. This would be a nice way to compensate for thick stroke widths.

Background

A background is different than a fill. Background is much the same as border; it is a rectangle (modulo the border-radius effects) around the shape, based on its geometric bounding box, taking into account padding.

By default, there should be no background color, including for the <svg> element. Right now, there is a WebKit bug in Chrome and Safari which renders a white, non-transparent background for SVG fragments or documents; this is a major pain for any content creators trying to use SVG, much like the old Internet Explorer bug on transparency in PNGs. I hope it receives some attention from the WebKit developers soon.

SVG Tiny 1.2 defines a 'viewport-fill' and 'viewport-fill-opacity'. I think those should be deprecated in favor of 'background' properties; I have no opinion on which should have precedence… maybe they should be seen as aliases of 'background', if that makes it simpler.

A background on SVG content raises the question of pointer events, which is also relevant to borders.

Pointer Events

Should pointer events on the border or background fire on the element itself? I think that the default should be “no”, just to be safe and consistent with older SVG content, but I could easily be persuaded to reverse that. Either way, there should be a new property or set of properties that can modify this, in a manner consistent with how CSS is defining 'pointer-events'.

This touches on the question, asked in a related thread, on whether the <svg> element, by default, should be a proximal event target for pointer events. My answer on that is a pretty firm “no”; if you want to have it as an event target, give it a background, or change the 'pointer-events' value.

Questions

A few high level questions:

• Is this a change from SVG 1.1 and SVG Tiny 1.2? Yes.
• Is this backwards compatible? Kind of. It introduces new behavior that is not consistent with past behavior, but doesn’t explicitly change very much.
• Does this break existing content? Very doubtful.
• Is this intuitive for content authors? My intuition says yes.
• Does this match existing implementations? Some, at least.
• Does this provide value for content authors? Yes.
• Does this make it harder to implement? I don’t know; I suspect it makes it easier for at least some implementations.
• Would defining this improve interoperability? Yes.

Conclusion

I propose we change SVG 2 to explicitly define this behavior. As the editor of the HTML5 spec, Hixie seems to imply that this is not for HTML5 to define, and I tend to agree with that, though maybe some informative mention in the HTML5 or CSS specs for how SVG treats margins and padding might be appropriate. We recently formally closed the Compound Document Formats Working Group, which was formed to look at exactly these sorts of questions, and published its specifications as Working Group Notes for posterity; we wanted to be clear about its status. The CDF WG succumbed to the politics around the reformation of the HTML WG, and probably to good effect overall; but in its absence, we still need to resolve these questions by cooperation between the various groups involved.

I don’t think this should be any different in standalone SVG files than in SVG files referenced or inlined in HTML. That would be very unintuitive, in my opinion.

If this is not controversial, I would be happy to put any or all of this in the SVG Integration spec, which is one of the foundations of SVG 2. If someone disagrees with this, we’ll have to talk about it before I start editing the spec. In either case, it would need to be fleshed out some… I’m probably glossing over some possible stumbling blocks in the box model, which I’d appreciate feedback on.

Please leave a comment if you like my suggestions, if you see a flaw, or have a better suggestion.

Update:

Just to show what I’m talking about, I created a sample. Here is a good example of a bad example; below are 3 instance of the same SVG with the same CSS background and border values, with a border width of 2px. The first 2 are external files referenced as objects, the third is inline in the HTML (you won’t see this one unless you are using preview releases of Firefox 4, WebKit, or IE9); the second image also has a padding and margin of 10px.

I tested this in several browsers, and in the referenced cases, the border width increases the image size, spawning scrollbars; I don’t think that’s a good outcome… it should clip the border, just as it does for any graphical overflow… borders should not contribute to the geometry of SVG images.

In the second case, where I added padding, the argument is less clear, but I still believe that for referenced content, the border and background should not displace the rendered graphics, and should be clipped. For inline content, it would make sense for the padding and margin to be visible in the rendered document, but it should still not change the geometry of the SVG.

We’d need to specify this in order for it to behave the same across browsers (and authoring tools). So, do people agree or disagree with my conlcusions?

I’ve been doing a lot of website stuff this week. Here’s some highlights:

W3C Working Group Sites

A few weeks ago, after long neglect, I finally updated the SVG WG page with a template that better matches the new(ish) W3C site redesign. Each working group has its own eclectic home page, which seems a bit silly to me. So, I took the template I made and converted the group pages of the WebApps WG, SVG Interest Group, and FX Task Force, and I’m talking with other W3C team contacts about converting theirs as well, though we’ll see how many switch over. From my perspective, having a consistent look&feel is important, and would make the W3C site more usable.

The new template I made is done in HTML5, with the <header>, <footer>, <article>, <section>, <aside>, and <nav> elements (and a reference to Remy Sharp’s HTML5 shiv), possibly even used correctly; the SVG WG page has a scripted live microblog feed from the @svgwg Twitter account, and an inline SVG easter egg (look for it in Firefox and WebKit nightlies).

Short Notice

I have been wanting to write or install a URL shortener on my own site for a while, so I didn’t have to rely on the brittle third-party services like TinyURL, which I feel don’t fit the spirit of the Web. They obscure the link, and act as a massive throttling point. I looked around a bit before for a custom URL shortener, but never found one that suited my fancy.

But on Saturday, after converting my URLs to have nicer, more meaningful, more RESTful names on the advice of the ever-helpful Helder Magalhães, I was in a mood to tackle the problem, and I found a great summary of some of the options by a fellow named Eric Severance. He described a better list of criteria than I’d considered, and I went with his top pick, Pretty Link by Blair Williams.

Despite the silly name, Pretty Link has almost all the features I wanted and then some. It’s easy to use, it integrates into my WordPress blog, it lets me pick a meaningful shortname, and I can shorten not only my own blog’s URLs (lots of WP plugins let you do that), but any random URL I want; it even tracks how many people have followed the links, which to me is mildly interesting, but for people more into their brand would be very handy. The only thing I’d like to see is a way to automatically publish a public page that lists all the shortened URLs and the title of each page, along with the expanded original URL; maybe I can hack that in, somehow (and maybe I just haven’t found it yet). Just for posterity, I compiled a list of all the links I had tweeted before, and dereferenced them, which you can find on my expanded URLs page.

Commenter Avatars

I’m playing a bit with my site theme. I’m happy with it overall, but I think the column width could be a bit wider; I’ve already bumped up the font size. I want to enable nested comment threads, because I think they are easier to read, but I’ll need to overhaul my theme quite a lot to do so, since it’s from an older version of WordPress.

In the meantime, I added commenter avatars. The default image is a Gravatar; as with TinyURL, I don’t like artificial centralization in avatars, and in this case, it seems particularly silly. So I cobbled together my own WordPress plugin to take an alternative approach, which I’ll blog about next time.

SVG paths have a pretty good set of shape commands, enough to let you draw any 2D shape you might want in an authoring tool: horizontal, vertical, and diagonal straight lines, elliptical arc segments, and cubic and quadratic Bézier curves. Béziers are great, giving you precise control over the position and curve interpolation of each point in a concise format, using control points (“handles”), and are easily chained together to create complex shapes.

But let’s say you have a series of points, and you want to draw a smooth line between them, e.g. for stock data, an audio wave, or a mathematical graphing equation. Béziers are not as handy there, because not all the points needed to define a Bézier spline are on the curve itself. Obviously, you can decompose any set of points into a set of Béziers, but that takes math, and who wants to do that? (Put your hand down, nerd. I’m talking to the normals.)

Sometimes, you just want to lay down a set of points, and let the browser draw a smooth curve (unlike polylines, where each segment is just a straight line between the points). Like this:

Mouse over the line above to see the equivalent cubic Bézier points. They are off the line. Off the line! Oh, Math, why do you do this to me?

One thing that’s always bothered me about SVG… there should be more simple ways of doing things, and this is one of things. Over the years, I’ve idly asked math-minded folks about how to achieve this, and the answers always soared right over my pointy little head. I probably didn’t ask the question the right way.

But a few months ago, I was chatting with Ben Fry, one of the creators of Processing (the graphics programming language), and he mentioned Catmull-Rom curves as a possible solution. I took note of that, and later read up on them; they seemed a pretty good fit for my use case. In my spare time I looked for any libraries out there to convert Catmull-Rom curves to Bézier splines; I found references that talked about the the conversion using inverted matrixes, but no simple code to be had. I’m lazy, and busy, and the problem wasn’t urgent, so I kept putting it off.

But in talking to folks about SVG throughout the UK, including designers, the thought kept running through my mind that SVG really does need a better authoring experience (and I don’t just mean authoring tools, though some nice animation tools would help). So, when I got back home from vacation, before I really got back into the thick of the dozen other things I should be doing, I put on some music and spent a few hours after work scouring the intartubes for some suitable code. And as luck would have it, Maxim Shemanarev’s Anti-Grain Geometry library contained a single simplified conversion matrix and a line of code that did the grunt work in C++, easily ported to Javascript; whip up a custom parser for the <path> syntax, feed it into the convertor function, and you’ve got yourself a working proof-of-concept.

Here is a scientific chart showing how awesome the SVG <path> element is, with values derived from the overall awesomeness by different path commands, rendered using an experimental Catmull-Rom path command:

As you can see, adding Catmull-Rom curves would increase the awesomeness of SVG paths by almost 50%. That is a lot more awesome!

The syntax is pretty intuitive… it’s just the command ‘R’ followed by a set of coordinate points:

  <path stroke="#BADA55"
        stroke-width="2"
        fill="none"
        d="M20,380
           R58,342
           100,342
           100,300
           140,250
           190,210
           220,197
           250,184
           280,155
           310,260
           404,20"/>

And the path syntax isn’t just used for the ‘d’ attribute of <path> elements. It’s also used for laying out text-paths, and the shapes of SVG Font glyphs. And it’s used in animation, both for motion paths and (in the case of Béziers) for timing functions in the ‘keysplines’ easing attribute that controls the rate of animation. So, making it easier to hit specific points for paths could have side benefits for all those uses as well.

I’m really interested in feedback from others on the usefulness of this idea. I’m not married to the idea of Catmull-Rom curves specifically (Fun fact: Ed Catmull is now the president of the Walt Disney and Pixar animation studios; I believe Rom became an astronaut, was captured by aliens and turned into a cyborg, and is now a Spaceknight), just in the idea of adding this type of point-on-the-path command. So, if you support the notion of adding this in SVG 2, or have a better idea along the same lines, send an email to the SVG Working Group’s public list, www-svg@w3.org, to let us know.

The javascript library I wrote to demonstrate this is, of course, open source, available under the MIT or GPL licenses. Feel free to play around with it, and improve it to make a better proposal. You’re welcome to use it for other purposes, but I intended it just as a quick-and-dirty prototype to solicit feedback about extending SVG, so the code isn’t that great.

Update 1:

Erik Dahlström, Chief Vectorizer at Opera and chair of the SVG WG, posted a visual reply, which WordPress filtered, but I’ve now restored this inline. Nothing to see here, move along.

Update 2:

Here is my Catmull-Rom code trying to achieve the nice curves of Spiro, as suggested by Dave Crossland in his comment:

Update 3:

Looking at Spiro curves a bit closer (just the image, not the code), it seems that there are actually different node types illustrated, not simply an undifferentiated set of coordinate points as in my interpretation of the Catmull-Rom algorithm. This means that, from an syntax standpoint, those segments would need additional parameters in addition to the coordinate points on the curve, which somewhat undercuts the simplicity.

Given that SVG already has a ‘Lineto’ command, I set out to see how closely I could match the letter “a” in Raph Levien’s example, using just my Catmull-Rom and Lineto segments, and with minimal effort:

Mouse over the raster image to hide the SVG path. There are definite artifacts, and I am running into some known limitations of my script implementation in how it interpolates the last coordinate segment, but with not much effort, I was able to get pretty close.

This is not to belittle Spiro at all… Spiro generates painfully elegant curves. It’s just to compare apples to apples.

My last post was slashdotted. Not servers-melting slashdotted, but unusual-volume-of-comments slashdotted. I posted it late on a Saturday night, so I guess they had no other news fit to print.

It was interesting, and a little bit exciting, to be linked from Slashdot. I have no great insights, but a few observations.

1. I thought something was odd when I started getting comments from outside the SVG community, which is the only place I can imagine I have any sort of fame or notoriety. So I followed a Twitter reference on SVG back to its source. I’m sure this is the most I’ve been referenced in Twitter, too… though most weren’t retweets of my own pointer to my blog. Interesting to see how Twitter and Slashdot interact, and how aggregators like Slashdot are the primary focus of links from Twitter, moreso than the original source. Some aggregators retweeted the story several times at intervals, I reckon to get more attention from those who blinked the first few times (srsly, smashingmag? Retweeting it 11 times? Okay).
2. I was surprised that “indexing SVG” was the topic that got the attention of the Slashdot community. I didn’t expect anyone to be particularly interested in my little study, outside the rarefied Web Standards community, which is who it was written for. My guess is that it may have seemed a bit controversial, because someone from the W3C looked with a critical eye at Google (a Giant Corporation, and one of our members). Everyone loves scandal! I’ve learned my lesson… now all my posts will attack some company. (If you are interested in some press-grabbing attention, my fee for insulting your company on this blog is very reasonable.)
3. In true Slashdot fashion, it’s pretty obvious that many of those who bothered to comment clearly didn’t read my blog post, or didn’t get my central point; it’s not surprising, since that wasn’t really the audience I expected (and I do ramble a bit). Some folks tried to turn it into an SVG vs. Flash debate, which entirely missed the point. That said, there were some thoughtful comments that I replied to on Slashdot and on my blog.
4. I spent much more time following up on my post than I expected. I’m glad it wasn’t a weekday, where it may have gotten even more attention. Still, I hope it did give some people food for thought.
5. Either I’m too prolix (guilty!), or Twitter has reduced the average attention span to… ooh, shiny! Adobe’s JD took a friendly dig at me in his comment, and maybe on Twitter, too. My post was a very long one, simply because I had no leisure to make it shorter. I prefer to thoroughly explore an idea and provide context and evidence when I bother writing at all. I suppose that is old fashioned. I’ll work on it.

Sorry, I have to go now… my fifteen minutes are up.

Recently, a browser implementer asked me for examples of SVG. He was having trouble finding good examples of SVG in use, particularly as parts of an HTML document. This question has come up again and again, actually, and it always vexes me. I’ve been active in the SVG community for close to a decade, and I’ve seen thousands of amazing SVG files (and many more of mediocre to average quality), but somehow they seem to have disappeared or bitrotted over the years. Some of those files only worked with the slightly-unstandard Adobe SVG Viewer, or didn’t quite work with Firefox’s incomplete support, I know, but surely not all of them. Where is all the great SVG content I remember, the games and GUIs and design and development? Where are all those files to be found?

I hear some browser implementers say that people just don’t use SVG. Intuitively, this feels false to me, based on my own experience. But could it be true?

The statistical insignificance of SVG is often cited by some people in the WHATWG, based on a large dataset of Web content indexed by Google. In the WHATWG, where HTML5 started, great stock is placed on statistics, particularly those conducted by the editor, Ian Hickson, a Google employee.

There’s no question about it: HTML is the king of the Web. I did some rough calculations, similar to claims I’ve heard before, by counting the number of returns for HTML files versus SVG files. A search for the filetype “.svg” yields around 18,165,500 hits on Google. (Note that this doesn’t count the false hits on the word “SVG” from St. Vincent and the Grenadines, Stan Van Gundy, the Sexy Valley Girls, or any of the numerous other bizarrities that the acronym stands for.) SVG content makes up just 0.106% of all Web content, by my rough estimation. Flash is almost 5 times as common as SVG. That’s pretty grim for SVG.

But wait, let’s put that into perspective. Flash is about 4.8 times more common than SVG. HTML is roughly 838 times more common than SVG. 838 times. Flash content comprises approximately 0.52% of all Web content, and HTML is roughly 189 times more common than Flash. So, Flash is clearly much more popular than SVG (even when you consider that some large percentage of Flash content is actually just encapsulated video content, these days). But that doesn’t mean that nobody’s using SVG. Nearly 20 million documents is pretty impressive, actually, especially given the fact that SVG has been hindered by a lack of native support in browsers for most of its existence (and more recently, even poor support by the Adobe plugin for IE), and a lack of common authoring tools for dynamic content (Inkscape is an excellent vector editor, but it doesn’t yet do animation or interactivity).

Eighteen million documents. That’s a lot of files. So, given that, why is it so hard to find SVG content?

Maybe because the most popular search engine in the world, Google, doesn’t index SVG.

Indexing SVG

A long time ago, back in 2002, I made a page discussing my experiments with text search and translation. The results were not very encouraging, but I reckoned it was just a matter of time. I optimistically wrote to Google to encourage them to enable text search and translation of SVG files.

8 years down the line, things don’t seem to have changed much on that front.

To be fair, many SVG files don’t contain any text at all, not even a <title> element, so indexing them might not yield much. But many other files do have at least a title, and SVG infographics and webapps usually have at least labels that might be meaningful as search terms. Often SVG files are even text-heavy.

It’s not that Google doesn’t take note of the files… obviously, you can search for the filetype, or in the worst case, the specific file URL, and normally get back positive results.  But Google doesn’t seem to search the contents of the SVG files and present them in the relevant result set.  To test this, I tried searching for a few files that I knew to have indexable text content.

As an example, I looked for some SVG files on my little (long out-of-date) SVG promotion site, SVG-Whiz.com. First, I searched for a file I knew to have a cogent block of text, my explanation of the distinctions between ‘display’, ‘visibility’, and ‘opacity’, called HideShow.svg:

This file has been hosted on my site since 2003, I’ve gotten several positive comments about it, and a direct search for that file URL turns up a few hits linking to it, so it’s seems like a reasonable candidate for indexing. But what are the results of my in-site Google search for the word ‘opacity’? Okay, that just turned up the explanation page linking to the SVG file in question. Fair enough, maybe Google doesn’t treat SVG as a “document” file, only as an image. So, how about an image search for the same term? Nada. So, maybe Google doesn’t consider SVG to be either a “document” nor an “image”… let’s search for the word ‘opacity’ in the site ‘svg-whiz.com’ with the filetype ‘svg’. As specific as that is, at the time of writing, I got not a single resulting hit.

Google can find the files… why doesn’t it do something with them?

Comparison of File Extension Frequency

So, what criteria does Google use to decide which file types it is going to index?

The Google FAQ on search filetypes lists 23 file extensions that it indexes, and says:

There are 13 main file types searched by Google in addition to standard web formatted documents in HTML. The most common formats are PDF, PostScript, Microsoft Office formats [...] Google is also scouring the Web for additional file types that are very rare. You may see them pop up in your results from time to time. [...] PDF formatted files are the most popular after HTML files. PostScript and Microsoft Word files are also fairly common. The other file types are relatively uncommon by comparison.

So, I took the liberty of conducting my own survey of the relative frequency of various filetypes, as collected by Google itself, by using the “filetype:extension” query term. I’m not totally convinced this is at all an accurate means to collect and analyze the data, but it’s what I had at hand.

I put together a table that compares the different file types that Google explicitly mentions. (I thought about representing the data as an SVG barchart, but I was afraid it wouldn’t be indexed… just kidding, the sheer volume of HTML files would make every other bar just a blip.)

I also threw in some other filetypes of interest, including some with functional similarity to SVG, such as Illustrator, PhotoShop, and Silverlight. I expected non-Web filetypes such as Illustrator’s “*.ai” to be disproportionately underrepresentated in the results compared to their actual usage, and that was indeed borne out; it’s hard to know what percentage of SVG files are intended for and presented on the Web (I’ve spoken to many Inkscape users who only use SVG for print or local hard-drive, which surprised me), but I would guess that it is far, far more heavily tilted toward Web usage… but I still thought it would be interesting to compare.

What did surprise me was how “*.svg” compared to such ubiquitous file extensions as “*.txt”, and those for Excel, PowerPoint, and the venerable PostScript. To be frank, the results make me question my methodology, or perhaps the accuracy of Google’s reporting.

Caveat: I originally compiled this information a few months ago, and when rechecking it for accuracy, I got a surprising result. Normally, the Google search for filetype:svg returned 18,100,000 hits, but late one night, it returned only 2,000,000 hits; now, my most recent check showed around 4,300,000 hits. Jumping around in the results for an explanation, I noticed that there is a lot of duplication of Wikipedia content, and since Wikipedia uses SVG, that might account for some discrepancy. One possibility is that the lower figure represents 2 million unique documents, which are duplicated in a lot of places; the same should be said of any HTML content, and probably to a lesser extent of Flash content. I don’t know if this is the right conclusion, but it would be an interesting data point. Even with the much more modest figure of 2 million documents, I still think that represents an impressive body of work, particularly in light of the fact that SVG documents are normally authored individually, not through forum or blog software, or exporting or reformatting of email and text content as HTML.

Conclusions

I don’t think this is some grand conspiracy by Google to suppress SVG. Simple neglect is much more plausible. They don’t seem to see the value in indexing SVG. But the end result is the same: SVG seems to be statistically underrepresented in terms of access through Google searches, and thus, it is harder to find SVG content.

Relying on the results of a search engine that doesn’t index SVG, to draw conclusions about how many people are using SVG is not statistically sound. This is a bit like conducting a phone survey of English speakers in China, and concluding that nobody speaks with a Southern US accent. I reckon y’all might could see the problem with that methodology if you lived here in North Carolina.

SVG is at least as plentiful on the Web, by Google’s own reckoning, as most other file types that Google does index. Search engines, Google included, should index SVG files. They should read the text inside the file, and if the file is referenced in an HTML page, they should associate those keywords with the SVG file, just as they do with raster images. SVG files should display in image searches, as well. Here’s a list of the kind of useful content that can be gleaned from most SVGs:

• file name: while pretty primative, many files give some clue as to their contents in their name
• text elements: there are several elements in SVG that contain text to be rendered to the screen, including <text>, <tspan>, <textPath>, and <textArea>, and the content should be indexed as if it were text in any other format
• embedded HTML: HTML (and other markup) can be embedded inline in SVG, and search engines should look for that and index it as they would standalone HTML content
• links: Google, and probably most other modern search engines, give weight to files that are linked from other files, and files referenced from SVG content should benefit in the same way; the @xlink:title and @rel can help define the relationship between the files
• descriptive elements: like HTML, SVG has a <title> element that doesn’t display, but adds to the information about its parent file or element, and SVG also has a <desc> element for a longer description
• metadata: SVG can contain RDF, RDFa, microformats, and ARIA markup, which search engines are starting to pick up on these days; these metadata can reveala lot about a file, from its license information to structured content (like calendars or dates or contact info) to intent (such as ARIA roles, which will soon be expanded to include things like different chart types)

And the SVG Working Group would be happy to work with any search engine developers to make improvements to SVG 2 to help make indexing SVG content easier or more fruitful.

I’m not trying to pick on Google here (though I do note that a Bing search for ‘svg opacity svg-whiz.com’ listed the SVG file as the first hit), I’m just noting a discrepancy and an opportunity for improving the experience that people have on the Web with regards to SVG. At the very least, SVG should be recognized by Google as a legitimate file type, rather than a formata non grata.

Update

Rob Russell delivered great news to us in his SVG Open keynote. As of August 31, 2010, Google now indexes SVG and delivers it in some search results. Kudos to Google for stepping up! I’m very pleased… solid results only 6 weeks later. (I guess I should thank Slashdot, too.)

I’m really excited about W3C’s new public Audio Incubator Group, just launched today, and open for collaborators, innovators, and instigators. Go take a look for yourself, and see if you can contribute.

To celebrate the occasion, here’s a simple example of an experimental audio inteface, in the world’s first (and worst) audio synthesizer in SVG (you’ll need a special Minefield build to use it). Just click on the keyboard… it’s pretty rough still, but it shows some of the potential:

(standalone SVG file)

For some background, read on after this break…

Background

Years ago, I was one of several SVG developers interested in doing for audio what SVG did for graphics, and we talked a bit about a format we called SynthML. The old Adobe SVG viewer included an audio element for MP3 playback, and I found it really useful for adding subtle sounds for button mouseovers, clicks, and other UI effects, so the idea of programmatic control of music on the Web had intrigued me for some time (anyone remember Thomas Dolby’s Beatnik?), for enhancing user interfaces, adding ambient background music, and providing full fledged music webapps. The SynthML effort fizzled out, and while there are some interesting music markup languages (most notably MusicXML, which is more like the MathML of music), there hasn’t been much traction beyond simple recording playback.

That’s why I was so psyched when I saw what Alistair MacDonald, David Humphrey, Corban Brook, and a few others were up to while attending a Processing.js community event at Bocoup. David hacked up a special build of Gecko to add script access to the raw audio stream of the HTML5 <audio> element, and Al, Corban, and friends were building cool demos on top of it.

I immediately knew I wanted to see this as a part of the native functionality of the Web platform, across browsers. In some ways, it’s the last major missing piece from the open Web… the ability to view source and hack sound, the last commonly digitizable sensorium. I also knew that I wanted the use cases and requirements to come from the larger community, the musicians and UI designers and developers who will ultimately be the ones using this functionality to sound out the depths of what’s possible on the Web (okay, I’m corny). So, I talked with Corban, David, and Al, and we agreed that a public discussion forum would be the best way to connect with this community, to get them talking and experimenting and building and sharing.

Chris Blizzard of Mozilla was there too, co-sponsoring the event, and he was extremely supportive of getting this work started at W3C. I wrote up a charter for a public incubator group, and started knocking on doors of the browser vendors, our other members, and the wider community who might be stakeholders, to drum up interest. Pretty quickly, I got support from folks at the University of Rio (PUC-Rio), BBC, and Google; currently, a W3C Incubator Group takes a minimum of three W3C members to kick it off, though once it’s started, anyone can join. With that nicety taken care of, W3C management approved the charter, and the new W3C Audio Incubator Group (or Audio XG) was launched today!

Joining the Audio XG

I invite anyone interested in audio, music, speech synthesis, games, or other related topics to get involved, join the Audio XG, and start shaping the future of Web audio on the mailing list, and chat on the #audio IRC channel (irc.w3.org, port 6665, channel #audio). Incubator participation is free and open to anyone who wants to help out.

About the SVG Synth

My SVG synthesizer is a true product of the Web. I know next to nothing about music… I can’t play an instrument, can’t read music, don’t know music theory… I love music, but I’m a consumer, not a creator. I’d love to change that, when I have time, but I’ll settle for now for enabling others.

So, when I decided to make something with the audio API, naturally my first thought was to do something I didn’t know how to do, for which I’m totally unqualified: create a synthesizer in SVG.

I started with the HTML example from the documentation for the experimental audio API on the Mozilla wiki, and converted it to SVG (with a few missteps along the way). As it turns out, the HTML5 audio interface is exposed in Gecko (the Firefox engine) even when there is no HTML content. This was good news for me. It made it even easier to programmatically create sound with it. I suspect that this is something that lies outside any current specification, but it should be codified somewhere.

Next, I looked for a piano keyboard in SVG; I found a few on Wikimedia Commons, but they were not quite what I wanted (and in typical Inkscape fashion, were a little bloated), so ultimately I just made my own, which was trivial to do.

Next, I had to understand which notes each key should sound. So I read a lesson on keyboard basics, which sufficed for my purpose.

Then I had to figure out what the frequency of each note is in Hertz (Hz), which is what the audio API understands. It was no surprise that someone had published just such a table corresponding notes to frequency, with the companion equations for deriving them.

To put the pieces together, I took the simplest possible approach: I handcoded the keyboard (I could have instantiated it with script, because it’s very deterministic, but I favor declarative markup that can be tweaked in an authoring tool); I assigned an id to each key based on the note (C4, Af0-Bb0, etc., where “f”==”flat” and “b”==”sharp”, and the number denotes the octave); I then created an associative array with the note-id as the index and the frequency as the value, so I could look it up with the key’s id; then it was just a matter of sending the frequency to the script from the HTML example, and adding a duration timer.

Finally, just as a decorative flourish, I found a partial musical staff with a treble clef and beamed eighth-note, in SVG, of course, and tweaked it to my liking.

My tools? A Web browser with the audio API enabled (libre source), a search engine, and a text editor.

Voilà! From knowing almost nothing about formal music (I’d never even heard the term “middle C” before), I created a simple synthesizer, and learned a bit about music along the way. That’s the power of the open Web. That’s why I’m so pumped up about this new focus on audio. This one goes to 11.