By Michael Cherry [bio]
Posted: Oct. 13, 2008

• Illustration: Singularity Components
• Illustration: MinWin Components

The following is the full text of an article published by Directions on Microsoft, an independent research firm focused exclusively on Microsoft strategy & technology. More samples of our content, as well as a list of upcoming articles and reports are also available.

Groups within Microsoft are researching new OS technology, such as Singularity and Midori, designed to overcome the limitations of the current generation of software and to exploit changes in hardware, networking, and how people use computers. Although some of this OS research will find its way into future versions of Windows, a completely new Microsoft OS to replace Windows is likely a long way off. Meantime, Microsoft will continue to make incremental improvements to Windows by exploring smaller builds such as MinWin and Server Core.

New OS Research

While the Windows team works to release service packs, critical security updates, out-of-band features, and new versions of the Windows client and server OSs, other groups in Microsoft continue to explore the longer-term future of the OS. Microsoft Research is working on Singularity, a new OS concept emphasizing high reliability, and another group is attempting to use the Singularity base to build a fully functional OS, code-named Midori.

Singularity

According to Microsoft Research, Singularity is a research project designed to answer the question: "What would a software platform look like if it were designed for dependability instead of current goals, such as performance?"

Included in this research is an OS prototype developed to study how technologies such as programming languages can work with the OS to specify and verify an executing program's behavior.

Singularity is built from three key components: a kernel, software-isolated processes, and channels. (For an illustration depicting the relationship between the three parts, see "Singularity Components".)

The kernel resembles most OS kernels, managing memory, processes, channel operations, scheduling, and input/output. The difference is that only 5% of the kernel is written in assembler or C++. The remaining kernel code is written in a safe language (Sing#, an extension of C# designed for writing communication primitives and system programming); translated into Microsoft Intermediate Language (MSIL), which is the processor-independent instruction set for the Microsoft Common Language Runtime (CLR); and then compiled by the Microsoft Research Bartok compiler. The use of safe programming languages like Sing# makes the code more reliable by preventing problems such as buffer overruns, thereby eliminating an area of vulnerability typically exploited by worms and viruses and allowing for static analysis with verification tools to remove program errors early in the process so that the code is reliable from the earliest builds.

Software Isolated Processes (SIPs) encapsulate and isolate the data and executing code to lessen the likelihood that a failure in one process, such as a device driver, could bring down the entire OS. Every device driver, system process, application, and extension executes in a SIP and each SIP has its own run-time system and garbage collector (for freeing up unused memory, objects, and processes). SIPs are closed object spaces rather than memory address spaces: two SIPs cannot dynamically load or generate code or simultaneously access the same object. Instead, communication between SIPs, which can transfer exclusive ownership of data among themselves, occurs on channels. This isolated system architecture contributes to reliability by stopping any errors at well-defined boundaries, again making the overall system more dependable and reliable.

Channels are the primary method of communication between Singularity's SIPs (communication between SIPs and the kernel is through an Application Binary Interface). Channels are bidirectional, and communication across a channel is described, managed, and enforced by a contract that specifies valid message formats. Channels and the associated contracts allow static analysis to ensure that the processes communicate correctly, and therefore provide improved reliability.

Although Microsoft Research will continue to refine its work in both the Singularity OS and languages for writing applications for dependable systems (a Singularity SDK is available), their research will focus on proving that the concepts behind the technology are sound and repeatable, but they will not create anything that resembles a complete OS product. Instead, the technology will be transferred to other Microsoft teams, who could use it in supportable, shipping products.

Midori

Software Development Times, in a story based on documents from the Microsoft Technical Strategy team, reports that "Midori is an offshoot of the Singularity OS, the tools and libraries of which are completely managed code. Midori is designed to run directly on native hardware (x86, x64 and ARM), be hosted on the Windows Hyper-V hypervisor, or even be hosted by a Windows process."

Microsoft confirms that Midori is an incubation product under the leadership of Eric Rudder, senior vice-president of technical strategy, but the company is not commenting further. In this information vacuum rumors flourish, making it hard to understand what Midori is, and what it is not.

Incremental Improvements

Designing, developing, and testing a new OS can take years. Even as in the case of Singularity, where there is a bootable kernel and low-level functionality, many critical decisions remain, such as which processors the OS will run on, which APIs will be supported, how to handle backwards compatibility, and which features will be exposed to users from a command line or graphical interface. Therefore, Microsoft has to work in parallel to design new OSs for the future, while continuing to make incremental improvements to the existing Windows OS as shown by Windows MinWin and Server Core efforts.

MinWin

MinWin (or Minimum Windows) is the smallest set of Windows OS components capable of running and providing the fundamental OS services that are common to building any version of Windows, including derivative products such as Windows Storage Server. Although Microsoft has not publicly disclosed exactly what's in MinWin, simple networking (for example) would likely be included because all versions of Windows need connectivity, but full graphic support might not be included because the graphic and user interface requirements vary significantly by version. Currently, it is not a product or a version of Windows, but merely a package of a small set of system-level components that Microsoft's internal design teams use to ensure the core OS's stability and reliability, and to enhance the efficiency of the Windows development process. (For an illustration, see "MinWin Components".)

MinWin is the logical extension of work Microsoft began during Windows Vista development to explore fully the dependencies between the components of the Windows OS. MinWin primarily helps Microsoft developers build Windows; users are unlikely ever to interact directly with MinWin, such as installing MinWin on hardware separately from installing Windows.

Specifically, MinWin has several benefits for Microsoft developers:

A stable code base. By keeping MinWin to a small number of key components, and then aggressively restricting any changes to those components, the Windows core team can continually test and harden the code so that any version of Windows builds from a reliable and stable base.

Development efficiency. MinWin could help Microsoft better manage the dependencies created by developing a client and server OS simultaneously. In the past, key components such as the kernel have passed between the client and server teams with the release of each version of Windows. Without a single set of core components such as MinWin, one of the teams always owned the latest version of the critical files, potentially blocking the other team from a stable foundation needed for development. With MinWin, teams can get the then-current or most stable build of the core components from the core team whenever they are needed.

MinWin also helps Microsoft counter criticism that Windows is bloated; at its heart, the Windows OS is still relatively streamlined. Although Microsoft will not confirm which components are in MinWin, a video shows a Microsoft engineer running a test version of MinWin consisting of approximately 100 files, or about 25MB of files that run in less than 40MB of memory. This contrasts with Windows Vista, which typically requires a 40GB hard drive and 1GB of system memory.

Server Core

Because MinWin is often described as containing just the basics or core components of the OS, it is sometimes confused with Server Core, a new installation mode of Windows Server 2008 that installs a reduced number of OS components to lessen vulnerability to malware attack, and to reduce maintenance by having fewer components that might need patching.

The components in Server Core include all of the Windows OS components that run in kernel mode, as well as other user-mode OS components and services needed to run the supported roles. For example, a Core server running the Distributed Host Configuration Protocol (DHCP) role would include components such as the networking stack to allow clients to connect to the server to get an IP address and other network configuration information.

Although Server Core includes components that are key to supporting a graphical interface, it does not include the Explorer shell, meaning administration of Server Core is performed at a command prompt or by remote administration tools. Another component that is not currently part of Server Core is the .NET Framework, which limits Server Core's ability to run dynamic Web content written in ASP.NET or support Microsoft's new PowerShell administration scripting language.

Server Core illustrates the tradeoffs involved in trying to reduce the size of an OS. Server Core must include enough components to run the designated roles, but at times adding one more component, such as the .NET Framework, triggers the need to install so many associated components that soon the entire OS is being installed.

Server Core 2008 (and Windows Server 2008) was built on MinWin. Regardless of the server's intended role, all of the OS components which are part of MinWin would be needed by a server installation. Although not a separate base in Windows Server 2008, the MinWin packaging of components is included such that it could be easily separated out in the future.

A Major New Microsoft OS Is Years Away

The allure of any new OS is that it will eliminate any computing limitations in effect when it is being designed. For example, the allure of Windows NT was to overcome the limitations of Windows built on MS-DOS and to take advantage of coming trends such as faster processors and networks.

However, a new OS takes a long time to develop. A minimal build of Windows NT first ran in 1990, but it was not until 1993 that Microsoft shipped the first commercial version, Windows NT 3.1, and only with Windows NT 4.0 in 1996—six years after initial implementation—did adoption take off, as customers found that the hardware and applications needed for day-to-day work were finally available. Certainly, changes in tools and development processes, as well as virtualization, might contribute to faster development times, but the longest part of the development schedule comes after the kernel boots and runs, when decisions about what APIs to support and how to expose OS features force developers to face hard choices that create design and testing problems.

Consequently, customers should be cautious about deferring deployment decisions for Windows Vista, Windows Server 2008, or even Windows 7 in anticipation of the imminent release of a significant new OS from Microsoft. Customers should also pay little attention to MinWin—in the end, it will have little impact on how they use Windows or its functionality. However, customers should install Windows Server 2008 Server Core whenever it makes sense to do so; for example, when the customer wants to host virtual machines on Microsoft's Hyper-V technology, on an OS that is maximally secure and needs minimal patching.

Resources

A MSDN video describing MinWin and Server Core, presented by Mark Russinovich, a Microsoft Technical Fellow on the Windows Core team, is available by searching for "Russinovich" and "MinWin" at channel9.msdn.com/Search/Default.aspx.

A video demonstrating MinWin, presented by Eric Traut, a Microsoft Distinguished Engineer, is available at www.istartedsomething.com/20071019/eric-talk-demo-windows-7-minwin.

A video discussing differences between MinWin and Server Core is available at edge.technet.com/Search/Default.aspx?Term=andrew%20mason.

Server core is described in "Server Core May Impact Infrastructure" on page 5 of the May 2007 Update.

For in-depth treatment of the internals of the Windows OS, see "Microsoft Windows Internals, 4^th Edition," by Mark Russinovich and David Solomon.

Technical information on Server Core is available at msdn.microsoft.com/en-us/library/ms723891(VS.85).aspx.

Information on Singularity, including some videos on the OS architecture, is available at research.microsoft.com/os/Singularity.

The Singularity Research Development Kit (RDK) 1.1 is now available for academic noncommercial use. It can be downloaded from CodePlex, www.codeplex.com/singularity.