Operator-system/docs/overview.md

Overview

This document explains the high-level control flow of the operating system – from firmware entry through to the interactive shell – and introduces the major subsystems that the rest of the documentation explores in depth.

• Boot loader: a UEFI application implemented in main.c that reads kernel.elf, maps its segments, prepares the BootInfo interface, and jumps into the kernel.
• Kernel entry: kmain in kernel.c, which initialises the IDT, memory, and tasking subsystems, then spawns the Starling Terminal task.
• Subsystems: memory management (memory.c), cooperative multitasking (task.c), interrupt/exception handling (idt.c), and the command/terminal layer (commands.c + kernel.c).

The remaining sections walk through this path step by step and show how these modules interact.

---

Firmware and boot loader (main.c)

The system starts execution inside the UEFI firmware, which invokes the PE32+ entry point of the loader. GNU-EFI arranges for this to be the efi_main function in main.c:

EFI_STATUS
EFIAPI
efi_main(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
{
    EFI_STATUS Status;
    VOID *KernelImage = NULL;
    UINTN KernelSize = 0;
    UINT64 KernelEntry = 0;
    BootInfo Boot;
    KernelEntryFn EntryFn = NULL;

    /* Initialise the GNU-EFI library */
    InitializeLib(ImageHandle, SystemTable);

    Print(L"Loading kernel...\n\r");

    Status = read_file_to_buffer(ImageHandle, L"\\kernel.elf", &KernelImage, &KernelSize);
    ...
    Status = load_elf_kernel(KernelImage, KernelSize, &KernelEntry);
    ...
    /* Populate the BootInfo struct with generic UEFI-backed services */
    Boot.print          = Print;
    Boot.clear_screen   = loader_clear_screen;
    ...
    Boot.free_pages     = loader_free_pages;
    Boot.firmware_vendor = SystemTable->FirmwareVendor;
    ...

    /* Jump to the kernel – this should not return */
    EntryFn = (KernelEntryFn)(UINTN)KernelEntry;
    EntryFn(&Boot);

    Print(L"Kernel returned. Halting.\n\r");
    return EFI_SUCCESS;
}

Key steps performed by efi_main:

• Load kernel image: read_file_to_buffer opens \\kernel.elf from the EFI System Partition and reads it into a pool buffer.
• Parse ELF64: load_elf_kernel verifies ELF headers and iterates PT_LOAD segments, mapping each to its requested virtual/physical address via UEFI AllocatePages, then zero-fills .bss:

static EFI_STATUS load_elf_kernel(VOID *Image, UINTN Size, UINT64 *EntryOut)
{
    Elf64_Ehdr *Ehdr = (Elf64_Ehdr *)Image;
    ...
    for (Index = 0; Index < Ehdr->e_phnum; Index++) {
        Elf64_Phdr *Segment = (Elf64_Phdr *)((UINT8 *)Phdr + (Index * Ehdr->e_phentsize));
        ...
        Address = (EFI_PHYSICAL_ADDRESS)SegmentStart;
        if (EFI_ERROR(uefi_call_wrapper(BS->AllocatePages, 4, AllocateAddress,
                                        EfiLoaderData, SegmentPages, &Address))) {
            return EFI_OUT_OF_RESOURCES;
        }

        CopyMem((VOID *)(UINTN)Segment->p_vaddr,
                (UINT8 *)Image + Segment->p_offset,
                (UINTN)Segment->p_filesz);
        if (Segment->p_memsz > Segment->p_filesz) {
            SetMem((VOID *)(UINTN)(Segment->p_vaddr + Segment->p_filesz),
                   (UINTN)(Segment->p_memsz - Segment->p_filesz), 0);
        }
    }

    *EntryOut = Ehdr->e_entry;
    return EFI_SUCCESS;
}

• Prepare the kernel ABI: BootInfo is a compact struct of function pointers and metadata shared between loader and kernel:

typedef struct {
    /* Console I/O */
    KernelPrintFn    print;
    ConsoleClearFn   clear_screen;
    ConsoleSetAttrFn set_attribute;
    KeyReadFn        read_key;
    KeyReadFn        try_read_key;

    /* System control */
    void (*shutdown)(void);

    /* Physical memory */
    KSTATUS (*alloc_pages)(UINTN pages, UINT64 *addr);
    KSTATUS (*free_pages)(UINT64 addr, UINTN pages);

    /* Firmware metadata (for informational commands only). */
    const CHAR16 *firmware_vendor;
    UINT32        firmware_major;
    UINT32        firmware_minor;
} BootInfo;

UEFI-specific calls (console I/O, page allocation, shutdown) are wrapped in small adapter functions (loader_clear_screen, loader_alloc_pages, etc.) and stored in this struct. The kernel never calls firmware entry points directly; instead it depends only on BootInfo.

Finally, the loader:

• Casts the ELF entry point to KernelEntryFn.
• Invokes EntryFn(&Boot), transferring control to the kernel.

---

Kernel entry and subsystem initialisation (kernel.c)

The C-level kernel entry point is kmain in kernel.c. It receives a single BootInfo * argument from the loader:

void kmain(BootInfo *Boot)
{
    KSTATUS Status;
    Task *terminal_task = NULL;
    StarlingContext *ctx = NULL;

    if (Boot == NULL) {
        return;
    }

    if (Boot->clear_screen != NULL) {
        Status = Boot->clear_screen();
        ...
    }

    if (Boot->set_attribute != NULL) {
        Status = Boot->set_attribute(TEXT_ATTR(COLOR_LIGHTGREEN, COLOR_BLACK));
        ...
    }

    /* ---- Subsystem initialisation ---- */
    idt_init(Boot);
    memory_init(Boot);
    task_init(Boot);

    /* ---- Welcome banner ---- */
    SAFE_PRINT(Boot, L"  Welcome to Simple 64-bit Operating System!\n\r");
    ...
    SAFE_PRINT(Boot, L"Type 'help' for a list of commands.\n\r\n\r");

    /* ---- Spawn Starling Terminal as its own task ---- */
    ctx = (StarlingContext *)kmalloc(sizeof(StarlingContext));
    ...
    terminal_task = task_create(L"starling-term", starling_terminal_task, ctx);
    if (terminal_task == NULL) {
        ...
        starling_terminal_task(Boot);
        return;
    }

    SAFE_PRINT(Boot, L"[core] Started Starling Terminal (PID %d).\n\r", terminal_task->pid);

    /* Core thread becomes an idle loop, yielding to the terminal and others. */
    while (TRUE) {
        task_yield();
    }
}

kmain performs three major duties:

1. Console setup – clears the screen and sets a green-on-black colour scheme using firmware-backed services from BootInfo.
2. Subsystem initialisation – calls:
   • idt_init(Boot) to install the kernel's Interrupt Descriptor Table and exception handlers.
   • memory_init(Boot) to bring up the physical allocator, paging helpers, and heap.
   • task_init(Boot) to bootstrap the cooperative scheduler and register the current thread as task 0.
3. User interface – prints a banner and spawns the Starling Terminal as a separate task via task_create, then turns the core thread into an idle loop that continuously task_yields to allow other tasks to run.

At this point, the system has:

• A working IDT for CPU exceptions and IRQs.
• A memory stack providing page allocation, virtual mappings, and heap.
• A cooperative scheduler with at least two tasks: the core thread (task 0) and the terminal.

---

Starling Terminal and command dispatch

Interactive user input is handled by the Starling Terminal task in kernel.c. It runs a read–eval–print loop that delegates command execution to commands.c:

static void starling_terminal_task(void *arg)
{
    StarlingContext *ctx = (StarlingContext *)arg;
    BootInfo *Boot = NULL;
    KeyEvent Key;
    ...

    if (ctx == NULL || ctx->Boot == NULL) {
        return;
    }

    Boot = ctx->Boot;
    depth = ctx->depth;

    SAFE_PRINT(Boot, L"\n\r[Starling Terminal depth %d] ready.\n\r\n\r", depth);
    SAFE_PRINT(Boot, L"starling> ");

    while (TRUE) {
        /* Try non-blocking read first; yield to other tasks while idle */
        if (Boot->try_read_key != NULL) {
            Status = Boot->try_read_key(&Key);
            if (Status != 0) {
                task_yield();
                continue;
            }
        } else if (Boot->read_key != NULL) {
            Status = Boot->read_key(&Key);
        } else {
            SAFE_PRINT(Boot, L"Console input unavailable.\n\r");
            break;
        }
        ...
        if (Key.unicode_char == L'\r' || Key.unicode_char == L'\n') {
            /* Enter pressed: execute the buffered command */
            line[len] = L'\0';
            SAFE_PRINT(Boot, L"\n\r");
            trim_spaces_inplace(line);
            ...
            } else {
                Task *cmd_task = execute_command(Boot, line);

                /* If a command task was spawned, wait for it to finish. */
                if (cmd_task != NULL) {
                    task_wait(cmd_task);
                }

                /* Reset for next command */
                len = 0;
                SAFE_PRINT(Boot, L"starling> ");
            }
        }
        ...
    }

    /* Free our context on exit (allocated by the spawner). */
    kfree(ctx);
}

Key points:

• Non-blocking idle: when try_read_key returns no key, the terminal calls task_yield() so other tasks can run while the user is idle.
• Line editing: handles printable ASCII and backspace to maintain a simple line buffer (line[128]).
• Command execution: on Enter, the line is trimmed and passed to execute_command(Boot, line) in commands.c. If that function spawns a dedicated command task, the terminal waits for it via task_wait.
• Nested terminals: entering starling recursively spawns another Starling Terminal task with increased depth, demonstrating multi-level shells.

The command registry and dispatch path are documented in detail in commands-and-terminal.md.

---

Subsystem overview

The kernel is organised into focused subsystems, each in its own translation unit:

• Type layer (kernel_types.h): defines fixed-width and utility types such as UINT8, UINT64, UINTN, and CHAR16, deliberately avoiding firmware headers so that core kernel code remains decoupled from UEFI.
• Boot ABI (boot_info.h): defines BootInfo, KeyEvent, and function pointer types (KernelPrintFn, ConsoleClearFn, etc.) forming the sole contract between loader and kernel.
• Memory management (memory.c + memory.h):
  • PMM – bitmap-based page-frame allocator over a 16 MB pool obtained from the loader.
  • Paging – helpers to walk and extend the 4-level x86-64 page tables, map/unmap pages, and translate virtual to physical addresses.
  • Heap – first-fit free-list allocator with block splitting and coalescing, backed by PMM pages.
• Tasks and scheduler (task.c + task.h):
  • Static process control block (PCB) pool.
  • Cooperative round-robin scheduler.
  • Stack management and context switch support (via an external context_switch assembly routine).
• Interrupts and exceptions (idt.c + idt.h):
  • IDT mirroring of firmware entries.
  • Replacement of CPU exception vectors 0–31 with kernel stubs.
  • Central isr_handler that prints diagnostics and halts on unrecoverable faults.
• Commands and shell (commands.c + commands.h):
  • Command registry and help/man system.
  • System control commands (shutdown, about, mem, ps).
  • Test commands (memtest, tasktest, spawn) that exercise memory and scheduler subsystems in isolation.

Each of these subsystems is covered in a dedicated document:

• memory-and-allocation.md – PMM, paging, and heap internals.
• tasks-and-scheduler.md – task lifecycle, stacks, context switching, and scheduling.
• interrupts-and-exceptions.md – IDT construction, ISRs, and fault handling.
• commands-and-terminal.md – command pipeline from user input to handler execution.