by Matt Campbell and Mike Calvo
Assistive technology products for the blind often need to obtain detailed information about the contents of the display. These products have historically obtained such information by chaining their own code to the display driver. Today, the new display driver model introduced in Windows Vista does not support such driver chaining. Thus, to retain as much existing code as possible, the entrenched assistive technology products now install a mirror driver instead. However, Windows must disable significant functionality to make the mirror driver work. The results of disabling that functionality have the potential to adversely affect even users who are totally blind. To support mirror drivers as used by assistive technology, Windows must also continue to support the antiquated Windows 2000 display driver model. Meanwhile, there are newer assistive technology products that use alternative techniques to obtain essentially the same information about the contents of the display, without any crippling side effects, even under Windows Vista. These alternative techniques have considerable advantages over the entrenched approach now, and the advantages will become more significant as Windows continues to evolve. This paper explains the history that has led to the use of mirror drivers, the problems introduced by mirror drivers, and the advantages of the alternative techniques.
The entrenched assistive technology products now use mirror drivers because that approach is the most natural progression for them; it allows them to retain as much existing code as possible. It is therefore important to understand the history that has led to the use of mirror drivers.
The primary problem that Windows assistive technology developers face is that neither Windows nor the majority of applications were designed to be used by the blind. Assistive technology developers must therefore retrofit accessibility onto Windows and a wide variety of applications. This was especially true in the formative years of the Windows assistive technology industry. Before the introduction of Microsoft Active Accessibility and the rich document object models of Internet Explorer and Microsoft Office, assistive technology products received very little help from applications or Windows itself. Even today, assistive technology developers must spend considerable resources retrofitting accessibility onto applications which do not support it by design.
To make applications or operating system components accessible in the absence of an explicit accessibility infrastructure or a sufficiently rich object model, assistive technology products must obtain detailed information about the contents of the display itself. At its simplest, this information includes all of the text on the display as well as some form of identifier for each image or icon. This information is often much more detailed; it has historically included the position, foreground color, background color, and font attributes of each piece of text on the display, and even the exact position of each character. All of this information about the contents of the display is collectively known as an off-screen model.
Because neither Windows nor most applications provide this information in a form that can be readily queried or presented through speech output or Braille, assistive technology products must use unusual techniques to gather this information. Specifically, these products intercept basic graphics operations at some point between the application and the display driver, before the contents of the display are reduced to pixels. (Once the information is reduced to pixels, optical character recognition and other artificial intelligence are required to process it.) Luckily for assistive technology developers, the primitive graphics commands sent to Windows display drivers have historically contained the information that assistive technology products need. Thus, assistive technology products have historically obtained the information they need for their off-screen models by inserting their own code between Windows and the display driver.
It is important to note that assistive technology products which follow the approach just described are depending on a stroke of luck in the history of Windows. In the broader context of modern operating systems, it is highly unusual for display drivers to be given the information that assistive technology products need. For example, it is very rare, and even considered strange, that a display driver would receive the text to be rendered to the display as text. However, this happened to be the case with Windows. While this coincidence initially served assistive technology developers well, it will soon become apparent that they have depended on it too long.
While assistive technology products for the blind existed under Windows 3.1, they matured under Windows 95, 98, and Millennium Edition, collectively known as 9x. This line of operating systems provided a means for assistive technology products to install their own display driver hooks at run time, for the purpose of intercepting primitive graphics commands. Windows 9x had no meaningful security constraints; all applications, including assistive technology products, had unfettered access to the system. No system restart was needed to install or remove display driver hooks, so this technique was suitable even for the installers of these assistive technology products. Furthermore, no significant functionality needed to be disabled; at worst, these products had to make slight changes to the appearance of the display. Thus, run-time display driver hooking was a reasonable solution under Windows 9x.
Windows NT, 2000, XP, and Server 2003, collectively known as NT, differed radically from Windows 9x. This line of operating systems did not provide a run-time display driver hooking mechanism analogous to the one provided by Windows 9x. Thus, to minimize the amount of code that needed to be rewritten for NT, the assistive technology developers devised a technique called display driver chaining. When an assistive technology product which uses this technique is installed, it installs a virtual display driver and registers that virtual display driver as the system's primary display driver. After a system restart, this virtual display driver receives all graphics commands from Windows, sends the appropriate information to the assistive technology product if it is running, and passes those commands on to the next driver in the chain, which may be either the actual display driver or another assistive technology product's virtual display driver. When the assistive technology product is uninstalled, it attempts to remove its virtual display driver without breaking the display driver configuration.
This technique was fraught with problems from its conception. The most infamous problem was that in this technique's first implementations, multiple assistive technology products had to be uninstalled in precisely the reverse of the order in which they were installed, or the display driver configuration would be broken. To solve this problem, the assistive technology developers of the time jointly devised a system called the Driver Chaining Manager (DCM) which Microsoft ratified. Even with DCM, it is not unusual for the display driver chain to be broken, rendering an assistive technology product unusable or even severely crippling the system's display functionality. Furthermore, because administrative privileges are required to set up display driver chaining, an assistive technology product which uses this technique cannot be run from portable media on a locked-down system, unless the system administrator has already installed the required virtual display driver for that product. In short, display driver chaining was far from an optimal solution.
Windows Vista introduced an overhauled infrastructure for display drivers. Unlike display drivers designed for earlier Windows NT systems, display drivers designed for Windows Vista receive primitive graphics commands that are useless to assistive technology products. For example, when a display driver designed for Windows Vista is being used, text is not sent to that display driver as text but as pixels. From Microsoft's vantage point, this overhaul simplifies display driver development and has the potential to make Windows much more robust. However, to assistive technology developers, it means that display driver chaining is useless if it were even possible under the Windows Vista display driver model.
However, Windows Vista does support a type of virtual display driver called a mirror driver. Luckily for the entrenched assistive technology vendors, mirror drivers are still based on the Windows 2000 display driver model, so they still receive the level of information that assistive technology products need. A mirror driver is installed along with the assistive technology product and runs alongside the actual display driver. Windows sends graphics commands to all active mirror drivers as well as the actual display driver, so a mirror driver is not responsible for passing commands on to the actual display driver or to any other mirror driver that may be active. A mirror driver is only active while it is needed, that is, while the assistive technology product is running. Because Windows treats mirror drivers as separate from the actual display driver, mirror drivers can be managed more easily and more reliably than a driver chain. In this sense, the use of mirror drivers is an improvement over display driver chaining.
The fundamental problem with mirror drivers under Windows Vista is that while a mirror driver is active, both the Windows Vista display driver model and the Windows 2000 display driver model must operate concurrently. This requirement imposes a few significant limitations on the display subsystem.
Perhaps the most visible new feature of Windows Vista is Windows Aero, a new look and feel that sports transparency in window title bars and borders, fade effects when windows open and close, 3D task switching, window thumbnails, and more. At the heart of Windows Aero is the Desktop Window Manager (DWM), a new system program which radically changes the way that the desktop is rendered. Prior to DWM, applications usually rendered their window contents directly to the screen. With DWM, applications first render their windows into off-screen areas in video memory. DWM then combines all of these off-screen areas into a single image on screen in a process called compositing, many times per second. In addition to enabling the eye candy mentioned above, the use of off-screen rendering and compositing generally makes the visual experience smoother than before.
However, the Desktop Window Manager depends heavily on the new features of the Windows Vista display driver model. It cannot operate efficiently under the Windows 2000 display driver model, which is still used by mirror drivers. Even if it could, it's possible that the information required by assistive technology products would be lost by the time DWM renders the desktop to the screen. Therefore, while a mirror driver is active, Windows disables DWM and Windows Aero.
The entrenched assistive technology vendors make the seemingly reasonable assertion that disabling Windows Aero does not cause any loss in functionality but only changes how the desktop looks, which is irrelevant to blind users anyway. However, there are two problems with disabling Windows Aero that these vendors overlook. First, even though blind users are not directly affected by the appearance of the desktop, the sighted people that they work with will notice the change. In particular, if a blind person is providing technical support (perhaps remotely) for a non-technical sighted user, it would be undesirable for the sighted user's first reaction to be, "What did you do to my screen? It doesn't look the same!" Non-technical sighted users tend to judge by appearance without understanding what is happening.
The second, potentially more serious problem with disabling the Desktop Window Manager is that it has the potential to seriously degrade the functionality of the system. DWM provides an application programming interface (API) which application developers can use to take advantage of DWM's advanced features. While no currently known applications require DWM to be enabled, it's conceivable that future applications, wishing to take full advantage of Windows Vista's much-touted new features, will require DWM. Furthermore, it's likely that Microsoft will focus future user interface developments on DWM and pay much less attention to the classic, pre-DWM windowing system. It's not hard to imagine that in a future version of Windows, disabling DWM will seriously cripple Windows itself. Microsoft may even want to remove the classic windowing system in future versions of Windows; in this case, disabling DWM would not be an option. While disabling DWM is not currently known to significantly degrade functionality, this is a considerable risk for all assistive technology products that use a mirror driver.
According to the Windows DirectX diagnostics tool (DXDiag), DirectDraw acceleration and AGP texture acceleration are not available while a mirror driver is active. Direct3D is apparently available, but it might be limited while a mirror driver is active. After all, the Desktop Window Manager, which uses Direct3D, is disabled while a mirror driver is active; perhaps DWM is being disabled because the mirror driver introduces some limitation in Direct3D. Note that Direct3D underlies the new Windows Presentation Foundation (WPF), one of the most touted new features of Windows Vista for application developers. This means that the use of a mirror driver has the potential to reduce system performance, especially as advanced graphics technologies make their way into mainstream productivity applications. However, this is speculation; more research is required to determine the actual consequences of this reduced acceleration, especially the effects of a mirror driver, if any, on Direct3D and WPF.
It is possible that the presence of a mirror driver will interfere with Windows Vista's content protection system. If this is the case, then playback of high-definition video, such as HD-DVD movies or HDTV, would be either disabled or significantly degraded while a mirror driver is active. Again, this is speculation; further research is needed to determine whether it is a real problem.
As stated above, the deepest problem with mirror drivers under Windows Vista is that they require both the Windows Vista display driver model and the Windows 2000 display driver model to operate concurrently. During the development of Windows Vista, this requirement and the need to be compatible with existing hardware limited the degree to which Microsoft could create a new, clean infrastructure for display drivers. Microsoft has stated that display drivers are a substantial stability problem, and that one major goal of the Windows Vista display driver model was to improve stability through a clean break from legacy infrastructure. Clearly, Microsoft's primary reason for retaining the Windows 2000 display driver model in Windows Vista was to maintain compatibility with existing hardware. However, as hardware advances, Microsoft will probably want to eliminate the Windows 2000 display driver model entirely in a future version of Windows. In this case, two scenarios are possible as long as assistive technology products use mirror drivers; either Microsoft will maintain some form of the Windows 2000 display driver model for the sake of assistive technology, thus compromising system robustness at least while the mirror driver is active, or assistive technology developers who have depended on mirror drivers will need to switch to alternative techniques. Either way, if Microsoft attempts to move further from the Windows 2000 display driver model, someone will lose as long as assistive technology products depend on mirror drivers.
Fortunately for the future of Windows assistive technology, there are viable alternatives to intercepting display driver commands, which is the aim of both display driver chaining and using a mirror driver. Newer assistive technology products, such as Serotek's System Access, are using these alternative techniques to gather detailed information about the contents of the display and build an off-screen model. These alternative techniques have considerable advantages over the entrenched approach now, and the advantages will become more significant as Windows continues to evolve.
Both display driver chaining and mirror drivers require administrative privileges to set up. In contrast, the alternative techniques are effective even under the most restrictive guest accounts on public computers. This makes these alternative techniques suitable for assistive technology products which run directly from portable media without prior installation. As blind people become more mobile, this feature will become increasingly important, because it means that blind people can take their accessibility with them.
These alternative techniques do not disable or hinder any system functionality. They work well with the Desktop Window Manager and therefore Windows Aero. They also have no effect on advanced graphics acceleration, including Direct3D and the Windows Presentation Foundation. This means that assistive technology products which use these alternative techniques will not cripple cutting-edge applications or render them useless.
The effectiveness of both display driver chaining and mirror drivers in assistive technology hinges on the assumption that Windows will send graphics commands to the display driver in a form that is useful to the assistive technology. As more advanced graphics technologies make their way into mainstream applications beyond gaming and multimedia, this assumption is proving unreliable. In other words, even when using a mirror driver, it is dangerous for assistive technology developers to keep depending on this stroke of luck in the history of Windows. For example, the new Windows Calendar application in Windows Vista renders some important text in such a way that the entrenched assistive technology products cannot intercept that text, making it impossible for them to make Windows Calendar accessible. In contrast, the alternative techniques can be extended to support emerging graphics technologies without requiring a massive rewrite. System Access already supports Windows Calendar, and it will also be adapted to support other cutting-edge applications as needed.
By their nature, these alternative techniques have greater immunity to changes in Windows than those used by the entrenched assistive technology products. For example, the System Access off-screen model required little modification to work well under Windows Vista, even with Windows Aero enabled. If Microsoft decides to completely eliminate the Windows 2000 display driver model in a future version of Windows, these alternative techniques will continue to work well with little or no modification. Any change to Windows that rendered these alternative techniques useless would also have a profound effect on current mainstream applications. Thus, assistive technology products which use these alternative techniques are prepared for the future.
In the context of assistive technology under Windows Vista and beyond, the use of mirror drivers is clearly not a long-term solution. The entrenched assistive technology vendors have depended too long on a stroke of luck in the history of Windows. During the development of Windows Vista, Microsoft has attempted to break with the past and radically change many aspects of Windows for the better, including the display driver model; this trend will surely continue in future versions of Windows. It is short-sighted to assume that the use of mirror drivers will never lead to serious degradation of system functionality. Furthermore, it is naive to assume that Microsoft will indefinitely allow innovation and progress to be stifled for the sake of currently entrenched assistive technology products. Nevertheless, these assistive technology vendors are stubbornly resistant to change, so they have chosen the quick fix of using a mirror driver instead of pursuing alternative techniques that will endure as Windows evolves. Fortunately for blind users, such alternative techniques do exist, and newer assistive technology is using them to ensure that access to Windows for the blind has a bright future.