;;; how-it-works-2008-04-19.txt How SBCL-os works. Alastair Bridgewater, 2008 April 19. ;;; Introduction. SBCL-os is an experimental standalone x86 port of SBCL. As of the 18th, it is capable of demand-paging dynamic space from a hard disk. It is not yet complete enough to run arbitrary lisp code. ;;; Startup. Bootstrap is performed by a 32-bit protected-mode Forth system that I originally wrote years ago. This system can boot from a 1.44meg floppy disk or as an el-torrito cd image. The main functions the Forth system serves in terms of SBCL-os is to reprogram the interrupt controllers, set up the interval timer, load the non-demand-paged parts of the core file, and set up the initial memory map page tables. The lisp-specific bootstrap code is nine Forth source blocks at 1k each. This code loads the core file header and the page data for read-only and static spaces, sets up the page tables, enables paged operation in the CPU, sets up the GDT (global descriptor table), and calls the initial function specified in the core file header. At this point, the core file is assumed to reside on the first hard disk, as a contiguous run of sectors starting from the first sector on the disk. This obviously precludes using a normal MBR, as the core header resides in the same place on the disk. The core file header has been simplified drastically compared to the original. It now contains just a (boxed) pointer to the initial function (SB!OS-KERNEL:BOOT-KERNEL), and descriptions of the three core memory spaces (read-only, static, and dynamic). BOOT-KERNEL runs in ring 0. It sets up the GDT descriptor and address-space allocators, sets up space for the IDT (interrupt descriptor table) and tells the CPU about it, and sets up a minimal TSS (task state segment). It then creates an initial ring 3 task running the function RING3-STARTUP and starts it, leaving ring 0. RING3-STARTUP initializes and sets up, in order, the text-mode display driver, the ring0 interrupt handlers (a set of stubs that reflect interrupts down to a ring3 task), the default ring3 interrupt handlers (a common function which outputs a register dump and halts the system), the idle task, the "syscall selector" and the timer and page-fault interrupt handlers. It then displays a startup string, registers the currently running task as the "manager thread" for interrupt handling, creates a new ring 3 task running TEST-RING3-FUNCTION, and calls MANAGER-THREAD-TOPLEVEL to dispatch interrupts as they occur. The first thing that MANAGER-THREAD-TOPLEVEL does is SWITCH-TASK-AND-WAIT to the new ring 3 task, which effectively enables interrupts, and the critical part of the system is running. ;;; Design. The three critical parts to SBCL-os are the task/thread structures, the interrupt handling scheme, and the changes to support non-paged code. ;;; Design / Tasks. Each lisp thread structure (defined in src/compiler/generic/objdef) is allocated #x400 bytes into a memory page. The kernel task structure is deemed to start at the same place in memory, but expands down, as it doubles as the ring 0 stack for the task. Each entry point to ring 0 goes through a common assembly-routine which saves the machine state on the stack and calls SB!OS-KERNEL::COMMON-INTERRUPT-HANDLER. ;;; Design / Interrupt Handling. COMMON-INTERRUPT-HANDLER deals with two cases. If the interrupt reason given is an interrupt or exception it switches to the manager thread, passing the address of the interrupted or faulting task. Otherwise, the interrupt reason is a "syscall", and is handled entirely in ring 0. The two syscalls currently defined are SYSTEM-PANIC-STOP, which locks up the computer by HLTing the CPU with interrupts disabled (although it could conceivably restart the Forth system, allowing for some measure of post-mortem analysis), and SWITCH-TASK-AND-WAIT, which sets a specified task as the current task and resumes execution. MANAGER-THREAD-TOPLEVEL is essentially a loop calling SWITCH-TASK-AND-WAIT, obtaining the interrupt reason (interrupt or exception number) for the interrupted task and calling the handler registered for that interrupt or exception. As a practical upshot of this design, having an exception occur within the manager thread is bad, and should probably be checked for as a fatal condition in COMMON-INTERRUPT-HANDLER. ;;; Design / Non-paged Functions. The mechanism for causing certain functions to be loaded in non-paged (static) space is a kludge. It comes in three parts. First, a macro, DEFUN-NONPAGED, which is a simple wrapper around DEFUN which adds a (COLD-NONPAGED) form before the DEFUN. Second, a hook in the compiler similar to the handling of COLD-FSET which dumps a new FOP (Fasl OPeration), FOP-COLD-NONPAGED. Third, modifications to Genesis to split the *dynamic* gspace into *dynamic* and *real-dynamic*, with FOP-COLD-NONPAGED setting *dynamic* to *static*, causing all dynamic-space allocation to occur in static-space instead, and FOP-FSET setting *dynamic* back to *real-dynamic*. This method appears to be somewhat hit-or-miss as to what actually appears in static space. Debug info appears to be in dynamic space. Symbol names appear to be in dynamic space unless the symbols are first referenced from a function in static space. Fdefinitions appear to be in static space, but the obvious possibility is that they might appear in dynamic space if they are referenced before they are defined and their first reference is from dynamic space. EOF