So, one of the cool things we aimed for with Buttercup was ensuring that we not only worked well with screen readers, but, that we worked well without screen readers too. If you think about scenarios like the public library, chances are they will not have a screen reader installed but Silverlight is a much better probability.

To achieve this we wanted to support the idea of a self voicing UI. We did this, but, it comes with quite a few caveats. This includes the fact that it only works in Internet Explorer and even then you’ve got to do some security acrobatics to enable the Active X calls that we make.

I’ve been kicking around a few ideas of how we might improve the self voicing experience. I’m big on doing this because it’s inherently re-usable functionality. if we can get cross browser self voicing working it’ll allow us to ‘Self Voice’ not only Silverlight applications but also plain old HTML apps as well.

So, some of the thoughts I’ve had:

• Pre recorded MP3s for UI elements. Basically we’ll create an MP3 to be played by the UI when you button hover or field focus.
  This is pretty easy to achieve, but, it has a few draw backs too. We will only be able to support static speech- if you run a search in Buttercup today we’ll actually speak the result count back to you. Using pre-recorded MP3s is also a fairly time consuming process- we’ll either need to use voice talent, or generate each MP3 using a synthesizer and package it up. Sure we could probably build a tool that used reflection to interrogate the UI Automation tags and then created the speech files. But, overall this still feels a bit fugly to me.
• Calling up to the server. We could make a web service call to the server passing up the speech we’d like generated. We could then use one of the variety of Text to Speech generators server side to create a speech MP3 to be returned to the client. We’d obviously do some caching on the client and probably push this cached data into isolated storage to persist across runs. This gets us around the dynamic generation limitations from above, but, gives us a server side dependency which is something we’ve thus far managed to avoid in Buttercup- we really want people to just be able to grab the Buttercup *.xap file and stick it on their site. Making server calls doesn’t really suit this model. I’m also not 100% on whether the latency will be tolerable- realistically we’re probably talking 3000ms at a minimum. I think if we did go this way it’s probably only be a stop gap and we might just run the speech gen service in say Windows Azure.
• Generating the Speech in Silverlight. This is kind of the nirvana. In Silverlight 2.0 (which is what Buttercup is currently built in) we just can’t do it. But, Silverlight 3 we have the ability to use the new MediaStreamSource and this might just let us do some sort of Audio generation. I think that we’d probably need to look at porting an existing open source TTS engine to C# managed code and then surfacing this into Silverlight. I’d love to get feedback from anyone who has had good or bad experiences with any of the open source TTS engines.

Now that 3rd point got me thinking about another commonly requested feature- variable speed playback. The Silverlight 3 Beta doesn’t support variable speed MP3 playback and it doesn’t look like it’s a feature that we’ll get in this release. It might be possible, though, to build a custom MP3 decoder to support this feature- maybe a port of MP3Sharp.

Anyway, that’s my braindump for today. I know that Andrew Tokeley will have some interesting samples to post after his Twilight Session tomorrow. I’d love to hear in the comments if people have other ideas for the above problems.

I thought I’d give an overview of some of the accessibility features we included in the first demo release of ButtercupReader.

Web Standards and Accessibility

One of the interesting aspects of this project is that, while we have built a highly accessible application, it does not comply with New Zealand Government Web Standards (nor, I suspect, with any other country’s guidelines).

In New Zealand, like other countries, we have a set of accessibility guidelines  we have to follow when building web sites for government departments and agencies. The goal of such guidelines is pretty clear - to make web applications accessible to as wide an audience as possible without disadvantaging those with disabilities.

Of course, no-one’s going to disagree with the intent of these standards – but, as a developer of web applications I am often frustrated by the user experience I’d like to provide to my users but can’t. The web is a restrictive medium at the best of times but not being able to rely on, for example, JavaScript (13.1) makes things even harder (AJAX anyone?). Of course, you can invest time and effort to ensure your site degrades when JavaScript is not enabled but this is often not an option on typically tight budgets and timelines.

Even building an application in HTML is itself a major limitation. We all know that HTML was never intended to be the foundation for delivering complex, interactive web applications. I remember a year or two back when  I thought the end of our obsession with the web was coming - smart clients were the promise of a new future and I could sense the renaissance of the rich client. But no, bloody Web 2.0 arrived on the scene and everyone went gaga for web again.

Microsoft are clearly looking to invest in Rich Internet Applications (RIA) (see .NET RIA Services) but with this comes the challenge of making this relevant in a world where accessibility is increasingly important.

So when the opportunity to work on an accessible Silverlight application came along I thought - great, let’s see what’s possible and where the future of accessible RIA might take us. It would be great if ButtercupReader prompted a rethink in the web standards community about how we can provide accessible applications over the web.

Supporting Screen Readers

One of the reasons that traditional web sites can be made accessible to screen readers is that they know how to interpret well formed HTML and can traverse a document and 'read' its contents. So what happens when you're building an application using XAML? Silverlight makes it possible to mark your XAML up with special attributes to expose accessibility information via a standard known as UIAutomation (UIA). Screen readers, and other accessibility tools, that know about UIA (or its predecessor Microsoft Active Accessibility (MSAA)) can interpret these user interface elements.

<Button    

    x:Name="openBookButton" 

    AutomationProperties.Name="Open Book"  

    AutomationProperties.HelpText="Open a Daisy book from your local computer"

    AutomationProperties.AcceleratorKey="O"

    ...>

</Button>

Using a tool like UISpy you can see the AutomationElement properties that have been exposed.

Mark Rideout has written a more detailed overview of Silverlight accessibility features in his article here, Creating Accessibility-aware Silverlight Content.

Other Accessibility Features

Some of the other accessibility features in ButtercupReader include,

• Shortcut Keys - all functionality is available through the keyboard. One gotcha here is that it is not possible to override the shortcut keys used by the host browser. So, for example, we couldn't trap Ctrl + F to initiate a search. Instead, we chose to implement most shortcuts as single (standard keyboard) keys.

• Self-Voicing - throughout the application, and when you set the focus to an element, you will hear speech to provide contextual information. For example, when a button receives the focus its name (AutomationElement.Name) will be spoken and when you press Numpad * additional help text (AutomationElement.HelpText) is spoken. Note that this feature relies on SAPI and currently only work in IE on Windows.
• Contrast Settings - lots of partially sighted users have a preferred contrast setting. For example, photo-sensitive users will likely choose yellow on black, whereas other users may depend on a high contrast, black on white setting.

• Zoomability - clearly, the ability to be able to zoom into text and images is important and Silverlight makes this possible without losing any quality or definition.

So you can see we've had fun on this project and I really hope it opens up some debate on the future of accessibility on the web.

One of the cool features of ButtercupReader is the ability to open DAISY books that you have published on your own web site. Once you’ve followed the few simple steps below people will be able to link to your books using a link that looks something like,

All you are doing is setting the ref query string parameter to the URL of the package file (*.opf) that describes your book. Easy.

Now, in order for this to work there are a couple of things that you need to do on the web server that is going to be hosting the DAISY book files.

Add MIME Types for DAISY File Extensions

In order for your web server to be able to serve up requests for DAISY files you will need to add some additional MIME types. Currently the only file extensions that ButtercupReader relies on are,

Allow ButtercupReader Access to Your Domain

By default Silverlight is not permitted to access resources (like your DAISY files) on another domain (like your web site). But this is easy to fix. All you need to do is to add a file, naming it clientaccesspolicy.xml, to the root of your web site. This file should look like,

For more information check out the official documentation.

So, give it a try and let me know how you get on.

Hi and welcome to the ButtercupReader blog.

If you nod your head at any of the following, then please click on the Subscribe link – would be great to have you on board.

• You’re interested in Web Accessibility beyond traditional HTML based web sites
• You’re interested in Rich Internet Applications (RIA) and accessibility
• You’re interested to learn about how Silverlight makes it possible to write accessible web applications
• You’re keen to learn more about ButtercupReader – how it works and how to use it
• You want to hear about updates to ButtercupReader

I’d really like to encourage you to comment on posts and send in feedback (the good, bad or indifferent). We’d love to hear what you think and where you’d like to see ButtercupReader go.

We also hope to discuss the wider implications of RIA in the context of existing Government and Industry accessibility guidelines.

Thanks for dropping by.

Andrew Tokeley