Refine command execution and enhance documentation. Improved the CommandTaskContext structure for better task management and updated README.md with clearer instructions for 'memtest' and 'tasktest' commands.
This commit is contained in:
297
docs/overview.md
Normal file
297
docs/overview.md
Normal file
@@ -0,0 +1,297 @@
|
||||
## Overview
|
||||
|
||||
This document explains the high-level control flow of the operating system – from firmware entry through to the interactive shell – and introduces the major subsystems that the rest of the documentation explores in depth.
|
||||
|
||||
- **Boot loader**: a UEFI application implemented in `main.c` that reads `kernel.elf`, maps its segments, prepares the `BootInfo` interface, and jumps into the kernel.
|
||||
- **Kernel entry**: `kmain` in `kernel.c`, which initialises the IDT, memory, and tasking subsystems, then spawns the Starling Terminal task.
|
||||
- **Subsystems**: memory management (`memory.c`), cooperative multitasking (`task.c`), interrupt/exception handling (`idt.c`), and the command/terminal layer (`commands.c` + `kernel.c`).
|
||||
|
||||
The remaining sections walk through this path step by step and show how these modules interact.
|
||||
|
||||
---
|
||||
|
||||
## Firmware and boot loader (`main.c`)
|
||||
|
||||
The system starts execution inside the UEFI firmware, which invokes the PE32+ entry point of the loader. GNU-EFI arranges for this to be the `efi_main` function in `main.c`:
|
||||
|
||||
```298:345:/home/lochlan/Documents/Coding/c/os/main.c
|
||||
EFI_STATUS
|
||||
EFIAPI
|
||||
efi_main(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
|
||||
{
|
||||
EFI_STATUS Status;
|
||||
VOID *KernelImage = NULL;
|
||||
UINTN KernelSize = 0;
|
||||
UINT64 KernelEntry = 0;
|
||||
BootInfo Boot;
|
||||
KernelEntryFn EntryFn = NULL;
|
||||
|
||||
/* Initialise the GNU-EFI library */
|
||||
InitializeLib(ImageHandle, SystemTable);
|
||||
|
||||
Print(L"Loading kernel...\n\r");
|
||||
|
||||
Status = read_file_to_buffer(ImageHandle, L"\\kernel.elf", &KernelImage, &KernelSize);
|
||||
...
|
||||
Status = load_elf_kernel(KernelImage, KernelSize, &KernelEntry);
|
||||
...
|
||||
/* Populate the BootInfo struct with generic UEFI-backed services */
|
||||
Boot.print = Print;
|
||||
Boot.clear_screen = loader_clear_screen;
|
||||
...
|
||||
Boot.free_pages = loader_free_pages;
|
||||
Boot.firmware_vendor = SystemTable->FirmwareVendor;
|
||||
...
|
||||
|
||||
/* Jump to the kernel – this should not return */
|
||||
EntryFn = (KernelEntryFn)(UINTN)KernelEntry;
|
||||
EntryFn(&Boot);
|
||||
|
||||
Print(L"Kernel returned. Halting.\n\r");
|
||||
return EFI_SUCCESS;
|
||||
}
|
||||
```
|
||||
|
||||
Key steps performed by `efi_main`:
|
||||
|
||||
- **Load kernel image**: `read_file_to_buffer` opens `\\kernel.elf` from the EFI System Partition and reads it into a pool buffer.
|
||||
- **Parse ELF64**: `load_elf_kernel` verifies ELF headers and iterates PT\_LOAD segments, mapping each to its requested virtual/physical address via UEFI `AllocatePages`, then zero-fills `.bss`:
|
||||
|
||||
```168:223:/home/lochlan/Documents/Coding/c/os/main.c
|
||||
static EFI_STATUS load_elf_kernel(VOID *Image, UINTN Size, UINT64 *EntryOut)
|
||||
{
|
||||
Elf64_Ehdr *Ehdr = (Elf64_Ehdr *)Image;
|
||||
...
|
||||
for (Index = 0; Index < Ehdr->e_phnum; Index++) {
|
||||
Elf64_Phdr *Segment = (Elf64_Phdr *)((UINT8 *)Phdr + (Index * Ehdr->e_phentsize));
|
||||
...
|
||||
Address = (EFI_PHYSICAL_ADDRESS)SegmentStart;
|
||||
if (EFI_ERROR(uefi_call_wrapper(BS->AllocatePages, 4, AllocateAddress,
|
||||
EfiLoaderData, SegmentPages, &Address))) {
|
||||
return EFI_OUT_OF_RESOURCES;
|
||||
}
|
||||
|
||||
CopyMem((VOID *)(UINTN)Segment->p_vaddr,
|
||||
(UINT8 *)Image + Segment->p_offset,
|
||||
(UINTN)Segment->p_filesz);
|
||||
if (Segment->p_memsz > Segment->p_filesz) {
|
||||
SetMem((VOID *)(UINTN)(Segment->p_vaddr + Segment->p_filesz),
|
||||
(UINTN)(Segment->p_memsz - Segment->p_filesz), 0);
|
||||
}
|
||||
}
|
||||
|
||||
*EntryOut = Ehdr->e_entry;
|
||||
return EFI_SUCCESS;
|
||||
}
|
||||
```
|
||||
|
||||
- **Prepare the kernel ABI**: `BootInfo` is a compact struct of function pointers and metadata shared between loader and kernel:
|
||||
|
||||
```21:62:/home/lochlan/Documents/Coding/c/os/boot_info.h
|
||||
typedef struct {
|
||||
/* Console I/O */
|
||||
KernelPrintFn print;
|
||||
ConsoleClearFn clear_screen;
|
||||
ConsoleSetAttrFn set_attribute;
|
||||
KeyReadFn read_key;
|
||||
KeyReadFn try_read_key;
|
||||
|
||||
/* System control */
|
||||
void (*shutdown)(void);
|
||||
|
||||
/* Physical memory */
|
||||
KSTATUS (*alloc_pages)(UINTN pages, UINT64 *addr);
|
||||
KSTATUS (*free_pages)(UINT64 addr, UINTN pages);
|
||||
|
||||
/* Firmware metadata (for informational commands only). */
|
||||
const CHAR16 *firmware_vendor;
|
||||
UINT32 firmware_major;
|
||||
UINT32 firmware_minor;
|
||||
} BootInfo;
|
||||
```
|
||||
|
||||
UEFI-specific calls (console I/O, page allocation, shutdown) are wrapped in small adapter functions (`loader_clear_screen`, `loader_alloc_pages`, etc.) and stored in this struct. The kernel never calls firmware entry points directly; instead it depends only on `BootInfo`.
|
||||
|
||||
Finally, the loader:
|
||||
|
||||
- Casts the ELF entry point to `KernelEntryFn`.
|
||||
- Invokes `EntryFn(&Boot)`, transferring control to the kernel.
|
||||
|
||||
---
|
||||
|
||||
## Kernel entry and subsystem initialisation (`kernel.c`)
|
||||
|
||||
The C-level kernel entry point is `kmain` in `kernel.c`. It receives a single `BootInfo *` argument from the loader:
|
||||
|
||||
```169:254:/home/lochlan/Documents/Coding/c/os/kernel.c
|
||||
void kmain(BootInfo *Boot)
|
||||
{
|
||||
KSTATUS Status;
|
||||
Task *terminal_task = NULL;
|
||||
StarlingContext *ctx = NULL;
|
||||
|
||||
if (Boot == NULL) {
|
||||
return;
|
||||
}
|
||||
|
||||
if (Boot->clear_screen != NULL) {
|
||||
Status = Boot->clear_screen();
|
||||
...
|
||||
}
|
||||
|
||||
if (Boot->set_attribute != NULL) {
|
||||
Status = Boot->set_attribute(TEXT_ATTR(COLOR_LIGHTGREEN, COLOR_BLACK));
|
||||
...
|
||||
}
|
||||
|
||||
/* ---- Subsystem initialisation ---- */
|
||||
idt_init(Boot);
|
||||
memory_init(Boot);
|
||||
task_init(Boot);
|
||||
|
||||
/* ---- Welcome banner ---- */
|
||||
SAFE_PRINT(Boot, L" Welcome to Simple 64-bit Operating System!\n\r");
|
||||
...
|
||||
SAFE_PRINT(Boot, L"Type 'help' for a list of commands.\n\r\n\r");
|
||||
|
||||
/* ---- Spawn Starling Terminal as its own task ---- */
|
||||
ctx = (StarlingContext *)kmalloc(sizeof(StarlingContext));
|
||||
...
|
||||
terminal_task = task_create(L"starling-term", starling_terminal_task, ctx);
|
||||
if (terminal_task == NULL) {
|
||||
...
|
||||
starling_terminal_task(Boot);
|
||||
return;
|
||||
}
|
||||
|
||||
SAFE_PRINT(Boot, L"[core] Started Starling Terminal (PID %d).\n\r", terminal_task->pid);
|
||||
|
||||
/* Core thread becomes an idle loop, yielding to the terminal and others. */
|
||||
while (TRUE) {
|
||||
task_yield();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`kmain` performs three major duties:
|
||||
|
||||
1. **Console setup** – clears the screen and sets a green-on-black colour scheme using firmware-backed services from `BootInfo`.
|
||||
2. **Subsystem initialisation** – calls:
|
||||
- `idt_init(Boot)` to install the kernel's Interrupt Descriptor Table and exception handlers.
|
||||
- `memory_init(Boot)` to bring up the physical allocator, paging helpers, and heap.
|
||||
- `task_init(Boot)` to bootstrap the cooperative scheduler and register the current thread as task 0.
|
||||
3. **User interface** – prints a banner and spawns the Starling Terminal as a separate task via `task_create`, then turns the core thread into an idle loop that continuously `task_yield`s to allow other tasks to run.
|
||||
|
||||
At this point, the system has:
|
||||
|
||||
- A working IDT for CPU exceptions and IRQs.
|
||||
- A memory stack providing page allocation, virtual mappings, and heap.
|
||||
- A cooperative scheduler with at least two tasks: the core thread (task 0) and the terminal.
|
||||
|
||||
---
|
||||
|
||||
## Starling Terminal and command dispatch
|
||||
|
||||
Interactive user input is handled by the Starling Terminal task in `kernel.c`. It runs a read–eval–print loop that delegates command execution to `commands.c`:
|
||||
|
||||
```47:163:/home/lochlan/Documents/Coding/c/os/kernel.c
|
||||
static void starling_terminal_task(void *arg)
|
||||
{
|
||||
StarlingContext *ctx = (StarlingContext *)arg;
|
||||
BootInfo *Boot = NULL;
|
||||
KeyEvent Key;
|
||||
...
|
||||
|
||||
if (ctx == NULL || ctx->Boot == NULL) {
|
||||
return;
|
||||
}
|
||||
|
||||
Boot = ctx->Boot;
|
||||
depth = ctx->depth;
|
||||
|
||||
SAFE_PRINT(Boot, L"\n\r[Starling Terminal depth %d] ready.\n\r\n\r", depth);
|
||||
SAFE_PRINT(Boot, L"starling> ");
|
||||
|
||||
while (TRUE) {
|
||||
/* Try non-blocking read first; yield to other tasks while idle */
|
||||
if (Boot->try_read_key != NULL) {
|
||||
Status = Boot->try_read_key(&Key);
|
||||
if (Status != 0) {
|
||||
task_yield();
|
||||
continue;
|
||||
}
|
||||
} else if (Boot->read_key != NULL) {
|
||||
Status = Boot->read_key(&Key);
|
||||
} else {
|
||||
SAFE_PRINT(Boot, L"Console input unavailable.\n\r");
|
||||
break;
|
||||
}
|
||||
...
|
||||
if (Key.unicode_char == L'\r' || Key.unicode_char == L'\n') {
|
||||
/* Enter pressed: execute the buffered command */
|
||||
line[len] = L'\0';
|
||||
SAFE_PRINT(Boot, L"\n\r");
|
||||
trim_spaces_inplace(line);
|
||||
...
|
||||
} else {
|
||||
Task *cmd_task = execute_command(Boot, line);
|
||||
|
||||
/* If a command task was spawned, wait for it to finish. */
|
||||
if (cmd_task != NULL) {
|
||||
task_wait(cmd_task);
|
||||
}
|
||||
|
||||
/* Reset for next command */
|
||||
len = 0;
|
||||
SAFE_PRINT(Boot, L"starling> ");
|
||||
}
|
||||
}
|
||||
...
|
||||
}
|
||||
|
||||
/* Free our context on exit (allocated by the spawner). */
|
||||
kfree(ctx);
|
||||
}
|
||||
```
|
||||
|
||||
Key points:
|
||||
|
||||
- **Non-blocking idle**: when `try_read_key` returns no key, the terminal calls `task_yield()` so other tasks can run while the user is idle.
|
||||
- **Line editing**: handles printable ASCII and backspace to maintain a simple line buffer (`line[128]`).
|
||||
- **Command execution**: on Enter, the line is trimmed and passed to `execute_command(Boot, line)` in `commands.c`. If that function spawns a dedicated command task, the terminal waits for it via `task_wait`.
|
||||
- **Nested terminals**: entering `starling` recursively spawns another Starling Terminal task with increased `depth`, demonstrating multi-level shells.
|
||||
|
||||
The command registry and dispatch path are documented in detail in `commands-and-terminal.md`.
|
||||
|
||||
---
|
||||
|
||||
## Subsystem overview
|
||||
|
||||
The kernel is organised into focused subsystems, each in its own translation unit:
|
||||
|
||||
- **Type layer** (`kernel_types.h`): defines fixed-width and utility types such as `UINT8`, `UINT64`, `UINTN`, and `CHAR16`, deliberately avoiding firmware headers so that core kernel code remains decoupled from UEFI.
|
||||
- **Boot ABI** (`boot_info.h`): defines `BootInfo`, `KeyEvent`, and function pointer types (`KernelPrintFn`, `ConsoleClearFn`, etc.) forming the sole contract between loader and kernel.
|
||||
- **Memory management** (`memory.c` + `memory.h`):
|
||||
- PMM – bitmap-based page-frame allocator over a 16 MB pool obtained from the loader.
|
||||
- Paging – helpers to walk and extend the 4-level x86-64 page tables, map/unmap pages, and translate virtual to physical addresses.
|
||||
- Heap – first-fit free-list allocator with block splitting and coalescing, backed by PMM pages.
|
||||
- **Tasks and scheduler** (`task.c` + `task.h`):
|
||||
- Static process control block (PCB) pool.
|
||||
- Cooperative round-robin scheduler.
|
||||
- Stack management and context switch support (via an external `context_switch` assembly routine).
|
||||
- **Interrupts and exceptions** (`idt.c` + `idt.h`):
|
||||
- IDT mirroring of firmware entries.
|
||||
- Replacement of CPU exception vectors 0–31 with kernel stubs.
|
||||
- Central `isr_handler` that prints diagnostics and halts on unrecoverable faults.
|
||||
- **Commands and shell** (`commands.c` + `commands.h`):
|
||||
- Command registry and help/man system.
|
||||
- System control commands (`shutdown`, `about`, `mem`, `ps`).
|
||||
- Test commands (`memtest`, `tasktest`, `spawn`) that exercise memory and scheduler subsystems in isolation.
|
||||
|
||||
Each of these subsystems is covered in a dedicated document:
|
||||
|
||||
- `memory-and-allocation.md` – PMM, paging, and heap internals.
|
||||
- `tasks-and-scheduler.md` – task lifecycle, stacks, context switching, and scheduling.
|
||||
- `interrupts-and-exceptions.md` – IDT construction, ISRs, and fault handling.
|
||||
- `commands-and-terminal.md` – command pipeline from user input to handler execution.
|
||||
|
||||
Reference in New Issue
Block a user