My original plan with Sanos was not to build an operating system. It was developed as part of an experiment on investigating the feasibility of running Java server applications without a traditional operating system only using a simple kernel. This experiment was done back in 2001, so some of the descriptions below might not be completely accurate for newer versions, but I think the main conclusions from the experiment are still valid.
I started out by looking at what was actually going on under the hood when running Tomcat in a Sun HotSpot Java VM under Windows. My suspicion was that this setup didn't actually use much of the Windows operating system. In order to investigate this I made a simple experiment based on two simple observations. First, the Tomcat server is written in Java and can only "see" the operating system through the Java VM and the Java SDK. Secondly, the Java VM runs inside a single Win32 process, so I concentrated on the interactions between the Java VM/SDK and the Win32 API inside this process. It is important to notice that the experiment was running Tomcat, which is a web server that mainly does some network and file I/O, and not a complex GUI application like Eclipse, which would have made the experiment much more difficult. The rationale for this was that many of the systems I was working on at the time were web applications similar to the Tomcat setup. Implementing a state-of-the-art desktop operating system is quite another ball game.
The Win32 process hosting the Java VM essentially provides three abstractions to the Java VM:
I initially focused on the Operating System API to see what services were required by the Java VM/SDK in this setup. By starting up Tomcat and inspecting the running process I could examine which DLLs that were loaded in the process:
This showed me that six DLLs from the JVM were used (jvm.dll, java.dll, net.dll, zip.dll, verify.dll, and hpi.dll). The java.exe program is just a simple launcher for the JVM. These six Java DLLs were depending on six Windows DLLs for all the Win32 API calls (kernel32.dll, user32.dll, advapi32.dll, wsock32.dll, winmm.dll, and msvcrt.dll). The msvcrt.dll is the Visual C runtime library and only depends on the other Win32 DLLs. By inspecting the six Java DLLs I could see that 194 Windows API calls were used:
KERNEL32 CloseHandle CreateEventA CreateFileA CreatePipe CreateProcessA CreateSemaphoreA DebugBreak DeleteFileA DisableThreadLibraryCalls DuplicateHandle EnterCriticalSection FindClose FindFirstFileA FindNextFileA FlushFileBuffers FormatMessageA FreeLibrary GetCurrentDirectoryA GetCurrentProcess GetCurrentThread GetCurrentThreadId GetEnvironmentVariableA GetExitCodeProcess GetFileAttributesA GetLastError GetLogicalDrives GetModuleFileNameA GetNumberOfConsoleInputEvents GetProcAddress GetStdHandle GetSystemDirectoryA GetSystemInfo GetSystemTime GetSystemTimeAsFileTime GetTempPathA GetThreadContext GetThreadLocale GetThreadPriority |
GetThreadTimes GetTimeZoneInformation GetVersionExA GetWindowsDirectoryA InitializeCriticalSection InterlockedDecrement InterlockedIncrement IsDBCSLeadByte LeaveCriticalSection LoadLibraryA PeekConsoleInputA PeekNamedPipe QueryPerformanceCounter QueryPerformanceFrequency ReleaseSemaphore RemoveDirectoryA ResetEvent ResumeThread SetConsoleCtrlHandler SetEndOfFile SetEvent SetFileAttributesA SetFilePointer SetFileTime SetHandleInformation SetThreadContext SetThreadPriority Sleep SuspendThread SystemTimeToFileTime TerminateProcess TlsAlloc TlsGetValue TlsSetValue VirtualAlloc VirtualFree VirtualQuery WaitForMultipleObjects WaitForSingleObject WideCharToMultiByte |
USER32 MessageBoxA ADVAPI32 GetUserNameA RegCloseKey RegEnumKeyExA RegOpenKeyExA RegQueryInfoKeyA RegQueryValueExA WSOCK32 __WSAFDIsSet accept bind closesocket connect gethostbyaddr gethostbyname gethostname getprotobyname getsockname getsockopt htonl htons ioctlsocket listen ntohl ntohs recv recvfrom select send sendto setsockopt shutdown socket WSACleanup WSAGetLastError WSAStartup |
MSVCRT new delete __dllonexit __mb_cur_max _access _adjust_fdiv _assert _beginthreadex _CIfmod _close _control87 _endthreadex _errno _except_handler3 _finite _fstati64 _ftol _fullpath _get_osfhandle _getdcwd _getdrive _initterm _iob _isctype _isnan _lseeki64 _mkdir _onexit _open _open_osfhandle _pctype _purecall _read _setjmp3 _setmode _stat _stati64 _strdup _vsnprintf _write abort |
atof calloc exit fclose fflush fgets fopen fprintf fputc free getc getenv isalnum isspace longjmp malloc memmove printf putchar qsort raise realloc rename signal sprintf sscanf strchr strerror strncmp strncpy strrchr strstr strtol toupper vfprintf vsprintf WINMM timeEndPeriod timeBeginPeriod timeGetTime |
This simple exercise showed me that the only parts of Windows that the Java web server application could see were the services exposed through these 194 API calls. All the other Windows services were not visible to the application. You could argue that some of these were used for management and administration, but these web server applications often had their own management tools for user administration, package deployment, load balancing, logging, etc., so while Windows has facilities for all these, they are usually not used by this type of web application servers. A closer inspection of these 194 API calls shows that they fall into the following categories:
In order to test my hypothesis that these were the only services required for running the Java web application I built an "OS emulator" for running the Tomcat server and the Java VM. I have found this pre-sanos version that still contains the emulator in the emul directory. The emulator is a Win32 application that loads and execute the Java VM. The emulator has stub implementations for the six Win32 DLLs. These have implementations for simple API functions (like strcpy) or calls though the emulators API, which is implemented in os.dll. The functions in os.dll are implemented using a syscall thunking layer to the OS emulator (osexec), which then calls the equivalent Win32 function. In this setup the Java VM would never call any Win32 functions directly. The Win32 wrapper DLLs only depend on os.dll. This way I could trace all calls from the Java VM. In order to have full control of which Win32 API function that were used, the OS emulator did not use the standard C runtime library, but only depended on the Windows version of kernel32.dll.
While I could now load and execute the Java VM using the OS emulator, I was still relying on Windows for a number of OS services. I then gradually started to implement more and more functions in the emulator that did not rely on the corresponding functions in Windows, but instead relied on more primitive functions. For example, I could implemented printf in terms of my own implementation of sprintf and the simpler WriteFile call. Critical sections could be implemented using atomic instructions and Win32 events. I used Doug Lea's malloc to implement the memory management functions in msvcrt.dll, so the only Win32 memory management function I depended on was VirtualAlloc and VirtualFree.
I then implemented a DLL/EXE module loader which only used VirtualAlloc and VirtualFree. It could bind and relocate modules itself, so now Windows would no longer have any of the Java VM or wrapper Win32 DLLs in its module database. I then went ahead and replaced all the Win32 file system calls with my own file system implementation, which later became the DFS file system in Sanos. The files used in the OS emulator were now stored in disk image files and I only used Win32 file I/O in a block-style fashion like you would do with a low-level hard disk interface.
The final version of the OS emulator only depended on kernel32.dll and wsock32.dll in Windows and used 70 API calls:
GetSystemTimeAsFileTime CreateEventA TlsGetValue TlsAlloc WaitForSingleObject ResetEvent SetEvent Sleep ReadFile SetFilePointer WriteFile GetFileSize GetStdHandle HeapAlloc HeapFree ExitProcess WriteConsoleA VirtualAlloc VirtualFree CloseHandle DuplicateHandle GetCurrentProcess |
CreateThread SuspendThread ResumeThread ExitThread GetCurrentThread GetCurrentThreadId GetThreadContext GetThreadPriority SetThreadPriority WaitForMultipleObjects TlsFree CreateSemaphoreA ReleaseSemaphore GetTickCount TlsSetValue SystemTimeToFileTime GetSystemTime SetConsoleCtrlHandler CreateFileA GetProcessHeap GetCommandLineA |
socket connect listen bind accept closesocket shutdown select recv send recvfrom sendto |
gethostname gethostbyaddr gethostbyname getprotobyname htons htonl ntohs ntohl getsockopt setsockopt getsockname ioctlsocket __WSAFDIsSet WSAGetLastError |
At this point the experiment was essentially done and I had showed that you could run a Java web server application on an operating system implementing only these 70 API calls, where many of them were only used in a very simple manner. I thought that was pretty cool. Even though the experiment was just running a Tomcat server, there were a lot of stuff going on in that process. The Java VM produced JIT code for executing the Java application and did automatic memory management with garbage collection. Tomcat was a general Servlet container that could host any .war-based web application and it was all still running with very minimal support from the OS. While the HotSpot JIT compiler is fairly sophisticated it doesn't require much in terms of OS support. The same goes for the Java runtime system. It mostly translates into some thread and virtual memory management calls, but these are pretty simple. For example, while the memory manager in Java is a complex piece of machinery, from an OS perspective this just translates into a few calls to allocate and commit virtual memory blocks.
At this point I realized something astonishingly simple that in hindsight is so obvious that I should probably have been able to figure it out to begin with: applications are executed by the CPU, not the operating system. Even when it comes to memory management and thread execution, this is to a large extent true. While the OS will schedule and preempt threads, it is not executing them, they are still executed by the CPU, and even though there is more to scheduling than thread switching, switching between threads is not exactly rocket science (see setjmp). The same goes for virtual memory. This is essentially implemented by the CPU using page tables. The OS is only responsible for maintaining the page tables when switching process context and when virtual memory is allocated and committed. Again, these are fairly simple operations that involve tracking memory page allocations and mapping virtual addresses to physical memory by updating the page tables. Also, when it comes to I/O, the operating system's role is more coordinating than performing I/O, e.g. reading a sectors from a hard disk. Reading sectors from an IDE disk essentially consists of telling the ATA controller which LBAs to read, set up a PRD table to tell where the data should placed in memory, and ask the ATA controller to start reading. The OS will then get an interrupt when this has been done (or failed). Again, it is to a large extent the computer (i.e. CPU, RAM, PCI bus, ATA controller, etc.) that performs this operation. Sending and receiving network packets using a NIC is done along the same line.
After having done the OS emulator experiment I started thinking about writing a simple kernel that could support these few services. Remember that although the OS emulator implemented many services which are traditionally implemented by the operating system, it still ran inside a normal Win32 process under Windows. It was easy to get a boot loader and a simple kernel with support for a text console and keyboard input up and running and Sanos booted for the first time on September 9, 2001. I then added basic support for kernel memory management and thread scheduling. Next step was adding IDE hard disk support and porting the DFS file system and the module loader from the OS emulator to run in the kernel and on January 2, 2002, I had a kernel that could execute user mode code in ring 3. One major piece I was missing was a TCP/IP stack, something that is not trivial to implement. Luckily, Adam Dunkels had just released lwIP, which I ported to Sanos and added a BSD socket interface. On April 24, 2002, I had the first version of Sanos that could run a Java VM with a Tomcat web server (see here for the early history of Sanos).
While all this sounds pretty simple it took 9 months of coding to go from the emulator to having an self-contained operating system that could do the same. The final version of the OS emulator had 11,800 lines of code while the first version of Sanos that could run Tomcat had 46,400 lines of code. While that is a lot of coding just to do an experiment, it is still nothing compared to traditional operating systems which typically have millions of lines of code.
So, what has happened to Sanos in the decade since the initial experiment? I have since lost interest in Java server applications, and Sanos is now much more targeted at C programs. I have not updated the code to support newer versions of Java, but the Sanos API has been extended to cover a much larger part of the POSIX API. Today, Sanos is pretty POSIX compatible. For example, when I tried to compile NASM under Sanos, it didn't require any modifications to make it run. I have also added a SDK to Sanos, so it now has a C compiler as well as other development tools running under Sanos. Today Sanos is self-contained in the sense that is can build itself. After the initial bootstrap built under Windows it no longer relies on Windows for building.
Last, I need to address a controversial question: Was it a good idea implementing my own operating system? Short answer: Probably not! While I managed to prove my initial hypothesis that complex operating systems are not needed to run Java server applications, I could probably have done this without having to implement my own operating system. From a practical point of view, I could just have made a bare bone Linux installation, which is what I would recommend to most people who want to try this, and this is what we are doing where I work now. On the other hand, it was a lot of fun making Sanos, and I learned a lot doing it, also many things that are useful even if your job is not implementing operating systems. If you are thinking about implementing your own operating system I can warmly recommend OSDev.org, and especially the "Beginner Mistakes" section. I also recommend reading Hadrien Grasland's article on osnews.org, "Hobby OS-deving", before starting your own OS project.