GW Micro Blog

Recently, a document titled, "The Facts About Mirror Drivers in Assistive Technology" was brought to my attention. This document was co-authored by Matt Campbell and Mike Calvo from Serotek. Unfortunately the document was written as an advertising technique interspersing fact with fiction, smoke and mirrors. This document has gained some attention because of the scare tactics employed. I plan on using my experience (over 25 years worth) in the assistive technology community and strong relationship with Microsoft over most of this time to clear up the fact from fiction.

Follow up:

There are so many things that caught my attention while reading this document it is hard to know where to begin. I hope to cover all the issues. However, the biggest thing that caught my attention was the omission of the technique actually used by System Access. Knowing a great deal about screen readers, I made a guess at what they must be doing. We verified this using a demonstration version of their software. After confirming my guess was correct, I now understand why no specifics were mentioned. They used terms such as “emerging alternative techniques” which of course sounds impressive. I want to first start by examining these techniques they are using and then comparing them with what serious AT products are using along with Microsoft’s full support.

System Access is not using an emerging technique. In fact this technique they are now using was used by Window-Eyes 1.0 under Windows 3.1x and even used by Window-Eyes 5.5 under Windows 9x/Me. The approach used is called API hooking. Basically, the application is hooking or patching Windows itself on the fly which means when their application is launched it hooks into the operating system. This is where you find, for example, the location where an application would send text to be displayed on the screen. You then insert a bit of your own code into that location which in turn causes the execution to temporarily jump to your own code first. This allows you to see the information before Windows gets a chance to fully process the information. Once you've had your look at the data, you simply call back to the original location you hooked into. To call this an emerging technique and imply this is the future is completely false and deceptive. In fact I remember reading a white paper written by Microsoft’s Greg Lowney back in the late 80’s discussing this exact technique. Greg was the sole accessibility person at Microsoft at that time. He, along with other Microsoft engineers, came up with this approach in order to make Windows 3.1X accessible to screen readers. As mentioned, we used this API hooking for all of Windows 3.1x, 95, 98, and Me. Window-Eyes never used a display driver for those operating systems, though the document implies all screen readers did.

Not only is API hooking not a new technology, it has many issues—which is why Window-Eyes is moving away from API hooking. Microsoft is also pushing AT products to stop using API hooking, mainly because of security. Let me outline some of the issues:

1. Microsoft has only put up with API hooking because there was nothing better. They obviously don’t want applications to hook into their operating system. Microsoft has, however, been working on long term solutions such as UI Automation (UIA). I’ll talk a bit more about UIA later. But it is interesting that Microsoft is pushing hard to stop API hooking, yet the Serotek document states API hooking is a state of the art, long term solution.
2. Obviously hooking into Windows code itself can make your system unstable. You are basically rewriting Windows code on the fly.
3. Oddly enough, API hooking is a technique that spyware and rootkits use to infect your system, so as anti-malware software gets smarter it will start to target your assistive technology.
4. API hooking won’t work on a system that’s properly secured against unauthorized writes to code segments, which makes API hooking impossible. Again, Microsoft's long-term goal is to disallow hooking because of stability and security issues.
5. API hooking depends on Microsoft not switching to new technologies that dynamically rewrite function prologues and epilogues to deter stack-smashing and other attacks. Security is top on Microsoft’s hit list.
6. API hooking makes no provisions for co-existence with other applications also dependent on API hooking. The document spent a great deal of time criticizing DCM on co-existence yet made no reference about co-existing with other API hooking applications. Launching and unlaunching API hooking applications in the wrong order can easily make your system unstable. This is not a long term solution.
7. API hooking could very easily disable HD video content as Microsoft could choose to interpret nonstandard access to code segments as hostile activity typical of that used by applications that are used today to hijack protected Windows Media content. Again, API hooking is a huge security hole.
8. API hooking can miss graphical information that is generated and presented entirely at the kernel level (this is a much lower level then API hooking allows). Serotek just lucked out that all of that content is accessible through MSAA and other more advanced technologies that wouldn’t be there if AT vendors hadn’t been working with Microsoft a decade ago to move away from antiquated technology like API hooking.

Companies who truly understand the AT landscape know that the window of opportunity for API hooking is closing, and those of us who are still using them are actively looking for ways to reduce our reliance on them for all the reasons listed above. Yes, I did say “us.” Most screen readers and screen magnifiers still implement API hooking even under Vista, including Window-Eyes. However, we are becoming less and less dependent on API hooking with each release as we continue to work with Microsoft on long term solutions.

Now, obviously, API hooking is encouraging given that you’ve seen the benefits. For example, the ability to hook on the fly (meaning when your product is launched) does make for impressive demos. Some people may see my 8 bullet points as using the same scare tactic employed by the original document, so let’s look at Windows Vista. It was stated that entrenched assistive technology products (got to love that phrase) require administration rights to install their mirror driver, which is very true. But they imply that you don’t need administrative rights when API hooking is used. That is not true. What is interesting about Windows Vista, is if you need support for the User Account Control (UAC) dialog, or any secure desktop for that matter, you must either be registered with the new Ease of Access option or install a service to basically do what Ease of Access does. However, in order to be registered with Ease of Access or install a service, you must have admin rights. This is a perfect example of the misleading statements in the original document. It also shows that Microsoft is serious about security and isn’t allowing an application to do anything they want without administration rights.

So now that we know what technique Serotek is using, and the limitation it imposes, and the fact that Microsoft is strongly encouraging AT products to stop API hooking, lets look at what Window-Eyes is using.

Window-Eyes uses a host of techniques to get access to your Windows applications. There isn’t just one technique that works. All the techniques are used together to get the job done. This is why Window-Eyes works the best out of the box with most Windows applications. The following are just a few techniques that are commonly used:

1. Off Screen Model (OSM) – This was well described in the original document. Techniques used to populate the OSM are API hooking and display drivers (whether using DCM or mirror). Window-Eyes uses DCM for Windows 2000, 2003, and XP and uses mirror drivers for Windows Vista. API hooking is used as sparingly as possible as we continue to wean off a dying technology.
2. Win32 controls – Window-Eyes has rich support for talking directly to standard Windows controls such as buttons, listview, and treeviews. Applications which use standard Windows controls or subclass standard controls will work extremely well.
3. Microsoft Active Accessibility (MSAA) – This is a standard application programming interface (API) that was designed by many assistive technology companies along with Microsoft back in the '90s. This was a good solution at the time but had limited expandability making it difficult for today’s applications to make themselves accessible exclusively with MSAA.
4. IA2 – This is an open source extension to MSAA. IBM spearheaded the development of this approach. It is very new and has lots of potential. Window-Eyes 6.0 ships with IA2 support. This support will grow as more developers implement this technology.
5. UI Access (UIA) – This is Microsoft's replacement for MSAA. Where IA2 is an extension to MSAA, UIA is basically a new approach that is extensible. Although it is a new approach, Microsoft has made it relatively easy for application developers to switch from MSAA to UIA. Currently Window-Eyes does not support UIA, and to my knowledge no screen reader does. This will certainly change as this powerful approach gets used in newer applications.
6. Document Object Model (DOM) – This is basically an interface into an application itself. It typically offers a very rich interface giving very detailed information regarding the data. For example the Window-Eyes Office support uses the Word, Excel, PowerPoint, and Outlook DOM. It also uses the IE DOM. Working with an application’s DOM, as mentioned, allows access to very detailed information. But not all applications have a DOM, or they may not make it available outside their own application. But Window-Eyes obviously can’t support every unique application's DOM. So only major applications like Office and IE were considered.
7. Hack – Because Window-Eyes is expected to work with all applications and all operating systems, if the above techniques fail for one reason or another then we must come up with some other approach. Whether we hack or not, we always work with the manufacturer to fix the accessibility hole correctly so such hacks aren’t necessary in the future. We go with the hack approach kicking and screaming.

Hopefully that gives you an idea of the major techniques currently employed in Window-Eyes and most serious screen readers.

I need to spend some time discussing mirror drivers, and discussing many of the scare tactics discussed about mirror drivers. Before doing this, I would like to talk about the history of Vista development. For the first time, Microsoft has kept the AT community well informed regarding the major changes in Vista. GW Micro, along with other AT companies, traveled to Microsoft several times. We worked with Microsoft to make sure that legacy applications as well as new technology applications using the new benefits of Windows Vista remained supported. It is only because of our active involvement that Vista is accessible today. Having Window-Eyes ship on the day Vista shipped is proof that we have been working hard with Microsoft. One of the turning points with Windows Vista accessibility support was back in January 2006. Microsoft offered a two week porting lab for all major AT companies. I don’t know why, but Serotek did not attend this porting lab. In fact, GW Micro was the only screen reader vendor that took this porting lab seriously. We sent two of our top programmers along with myself to Redmond for the entire two weeks. This commitment was unmatched by any other screen reader developer. This commitment paid off as by the end of the two weeks we had Window-Eyes running giving basic Vista support using the Microsoft documented mirror driver approach. This commitment continued and allowed us to have the first full featured screen reader support on the day Vista shipped. This initial release didn’t require any Vista features to be disabled. This was history in the making.

It is true that mirror drivers have been around for a long time. Serotek’s continual use of “the Windows 2000 display driver model” terminology is calculated to make a psychological point. Even Microsoft calls it the XP display driver model. The choice of Windows 2000 is puzzling. Mirror drivers did exist in Windows 2000; however, this technology has evolved. In fact, under Windows 2000 Microsoft encouraged AT products to use the mirror driver approach instead of the pre-DCM video driver hooking. However, the mirror drivers didn’t have the necessary information required so they were not used by the AT community. Only with the release of Vista did Microsoft resolve the holes in mirror drivers making them usable for AT purposes. Why would Microsoft put time and money into developing mirror driver technology if they didn’t feel is was the solution for Windows Vista?

Don’t get me wrong; I also don’t believe that mirror drivers are the long term solution. Just as I don’t believe MSAA is the long term solution. As operating systems and application developers continue to innovate, new accessibility techniques are needed to keep up. Of course what Serotek didn’t mention is that API hooking is already outdated and strongly discouraged by Microsoft. The computer industry is too volatile to expect to say we have a technology that will last forever. But let me go point by point through some of what I consider misinformation regarding mirror drivers.

• The document stated “…it is highly unusual for display drivers to be given the information that assistive technology products need. For example, it is very rare, and even considered strange, that a display driver would receive the text to be rendered to the display as text. However, this happened to be the case with Windows.” - My response: There's an actual advantage to providing textual information, because high-end display adapters can do hardware TrueType rendering. It’s not a historical accident, it’s by design. (Though actually, most text output these days shows up as glyph indices, not as text.) In fact, those DDI calls still happen at some level, even with Aero on, for non-WinFX apps. They just go to a software driver that Microsoft didn’t want us to chain to for reasons of security and stability. For WinFX apps, there’s UIA.
• The document stated “Luckily for the entrenched assistive technology vendors, mirror drivers are still based on the Windows 2000 display driver model, so they still receive the level of information that assistive technology products need.” – My response: Microsoft actually improved mirror driver support in Vista to the point where it’s actually used by AT software, because they themselves realized that there are still lots of products that need that level of information. And not just accessible technology, either: Microsoft’s own remote desktop uses that “old” display driver model when dealing with GDI-based applications, which is to say most applications, because it’s more efficient than trying to squirt a bunch of bitmaps through a wire.
• The document stated “Direct3D is apparently available, but it might be limited while a mirror driver is active. After all, the Desktop Window Manager, which uses Direct3D, is disabled while a mirror driver is active; perhaps DWM is being disabled because the mirror driver introduces some limitations in Direct 3D.” – My response: This is pure speculation, and probably not true. DWM is disabled because the classical uses for mirror drivers don’t work efficiently with pure bitmaps, as was more or less already said. This is just to introduce an element of fear without any factual information to back it up.
• The document stated “Clearly, Microsoft’s primary reason for retaining the Windows 2000 display driver model in Windows Vista was to maintain compatibility with existing hardware. However, as hardware advances, Microsoft will probably want to eliminate the Windows 2000 display driver model entirely in a future version of Windows.” - My response: No, it was to maintain compatibility with those thousands of GDI-based applications people use every day. Even under the Vista driver model, there’s still a stub driver that implements DDI, because those GDI apps are going to be around a lot longer than your old video card. Microsoft clearly has no problem with expecting you to buy new hardware, and most of their OS sales are for new systems anyway.
• The document stated “While no currently known applications require DWM to be enabled, it’s conceivable that future applications wishing to take full advantage of Windows Vista’s much-touted new features, will require DWM. Furthermore, it’s likely that Microsoft will focus future user interface developments on DWM and pay much less attention to the classic, pre-DWM windowing system.” My reply: Again, a scare tactic based solely on speculation. But if turning off DWM is supposed to be so detrimental to proper operation of Vista, how do they explain the fact that some versions of Vista itself disable DWM all the time?
• The document stated “The entrenched assistive technology vendors make the seemingly reasonable assertion that disabling Windows Aero does not cause any loss in functionality but only changes how the desktop looks, which is irrelevant to blind users anyway." They go on to state two reasons this is false. First because sighted people just aren’t going to comprehend the screen if aero isn’t enabled and secondly the lack of DWM seriously degrades the functionality of the system. My response: Yet another scare tactic. Aero is just a theme that you can choose to use or not use. Claiming that sighted users won't comprehend Vista without Aero is like saying that because you prefer to use high contrast black, it would be impossible for a sighted person to recognize and deal with your system. Speaking as a sighted person this is simply false. I think we need to give sighted people a little more credit here. I would like to discuss this with more low vision users but my first impression is that the glass effect would be more of a detriment to low vision users in the first place. Aero is a theme, which means it is a choice. If Microsoft were so dependent on Aero then it wouldn’t have been an option, it would require all video cards to support it, and RDP and terminal services would support it.

Now, I’m not saying everything stated in the document is false. As was stated, there currently are no applications that directly support DWM, but, when there are, the mirror driver will not help those applications. But what wasn’t stated is API hooking as used by Serotek also won’t work with those applications. We are all going to have to use the new accessibility interfaces that were built in to WinFX through the cooperation of Microsoft and mainstream AT vendors.

I apologize for the technical nature of this reply. I tried to keep it as readable as possible yet expose the scare tactics and misleading information used to promote one product over another. Window-Eyes is not entrenched to any technology. As stated, Window-Eyes 1.0 started with API hooking as Serotek is using today, but we learned with experience and a close relationship with Microsoft that this is not a good long term solution. That’s why, with our first version of Window-Eyes for Windows 2000, we switched to using a display driver. We then convinced Microsoft that each AT vendor writing their own video driver was not a good experience for the end user. With the aid of a few AT vendors, and Microsoft, we came up with the DCM specifications, which Microsoft contracted GW Micro to write. We then switched to using DCM. With the release of Vista we switched to using the better mirror driver approach. If we were entrenched with one way, why have we adjusted to the drastically different approaches over time? I have no doubt we’ll move on to other techniques leaving mirror drivers behind. But I can guarantee that approach won’t be the out-dated API hooking method Serotek is so proud of. We are a company that puts our sights on the future and have the rock solid base required to make core development changes as needed, which we proved with the release of Vista. We will continue to work with Microsoft to ensure Vista remains accessible as well as all future operating systems.