Skip to content

Boot Sequence

Overview

When you execute a FlatImage binary, it goes through a carefully orchestrated boot sequence that transforms a single executable file into a fully functional, sandboxed container environment. This document explains the conceptual flow of how FlatImage starts up.

High-Level Boot Flow

flowchart LR Start([User Executes
FlatImage Binary]) --> Initialization subgraph Initialization["Binary Relocation"] direction TB I1{Binary
in /tmp?} I2[Copy to /tmp
and re-execute] I3[Initialize System] I1 -->|No| I2 I1 -->|Yes| I3 I2 --> I3 end Initialization --> SystemSetup subgraph SystemSetup["System Setup"] direction TB S1[Load Configuration
from Binary] S2[Mount Filesystems
DwarFS layers] S3[Start Portal
Daemon] S4[Parse Command
Arguments] S1 --> S2 S2 --> S3 S3 --> S4 end SystemSetup --> Router{Command
Type?} Router -->|Config Change| ConfigPath Router -->|Execute Command| ExecutionPath subgraph ConfigPath["Configuration Path"] direction TB CP1[Modify Reserved
Space in Binary] end subgraph ExecutionPath["Container Execution Path"] direction TB E1[Setup Container
Environment] E2[Apply Permissions] E3[Apply Bind Mounts] E4[Set Environment
Variables] E5[Launch Bubblewrap
Container] E6[Run Command
Inside Container] E7[Cleanup & Unmount
Filesystems] E1 --> E2 E2 --> E3 E3 --> E4 E4 --> E5 E5 --> E6 E6 --> E7 end ConfigPath --> Done([Exit]) ExecutionPath --> Done style Initialization fill:#E3F2FD style SystemSetup fill:#FFF3E0 style Router fill:#FFD700 style ConfigPath fill:#87CEEB style ExecutionPath fill:#FFE4B5 style Start fill:#90EE90 style Done fill:#FFB6C6
Hold "Shift" to enable pan & zoom

Phase 1: Binary Relocation

Purpose: Ensure the binary runs from /tmp where FUSE filesystems can be properly mounted.

flowchart LR A[Binary executed
from any location] --> B{Already
in /tmp?} B -->|Yes| C[Continue to
next phase] B -->|No| D[Copy binary
to /tmp] D --> E[Re-execute from
new location] E --> F[Original process
exits] style A fill:#E3F2FD style C fill:#90EE90 style F fill:#FFB6C6
Hold "Shift" to enable pan & zoom

Why this is necessary:

  • FUSE filesystems require specific mount point permissions
  • Simplifies permission handling for mounts
  • Ensures consistent behavior regardless of where binary is stored

What happens:

  1. Binary checks its own path
  2. If not in /tmp, copies itself there
  3. Executes the copy with the same arguments
  4. Original process terminates
  5. New process continues from /tmp

Phase 2: System Initialization

Purpose: Set up the runtime environment and verify system requirements.

flowchart TD Init[System Initialization] --> Logger[Setup Logging
System] Logger --> Metadata[Set Metadata
Environment Variables] Metadata --> MetaDetails["FIM_VERSION
FIM_COMMIT
FIM_DIST
FIM_TIMESTAMP"] MetaDetails --> CheckFUSE{FUSE Module
Available?} CheckFUSE -->|No| Error([Error: FUSE
not available]) CheckFUSE -->|Yes| CalcOffset[Calculate Filesystem
Offsets in Binary] CalcOffset --> Ready[System Ready] style Init fill:#E3F2FD style Ready fill:#90EE90 style Error fill:#FFB6C6
Hold "Shift" to enable pan & zoom

Key concepts:

  • Logging: Debug output controlled by FIM_DEBUG environment variable
  • Metadata: Build-time information embedded in binary (version, commit hash, distribution type, timestamp)
  • FUSE verification: Ensures kernel module is loaded for filesystem operations
  • Offset calculation: Determines where DwarFS filesystems begin in the binary

Phase 3: Configuration Loading

Purpose: Extract all configuration from the binary's reserved space and set up runtime paths.

flowchart LR Start([Configuration Loading]) --> ReservedSpace subgraph ReservedSpace["Reserved Space Reading"] direction TB R1[Read Reserved Space
from Binary] R2["Permissions Bitfield
━━━━━━━━━━━━━━
Runtime Flags
casefold, overlay type
━━━━━━━━━━━━━━
Boot Command
Default executable
━━━━━━━━━━━━━━
Environment Variables
━━━━━━━━━━━━━━
Bind Mount Definitions"] R1 --> R2 end ReservedSpace --> PathSetup subgraph PathSetup["Directory Structure"] direction TB P1[Build Directory
Structure Paths] P2["Create Directories:
/tmp/fim/app/BUILD_ID/
instance/PID/"] P1 --> P2 end PathSetup --> BinaryExtract subgraph BinaryExtract["Binary Extraction"] direction TB B1[Extract Embedded
Binaries] B2["bash
━━━━━━━━━━━━━━
fim_janitor
━━━━━━━━━━━━━━
fim_portal
━━━━━━━━━━━━━━
fim_portal_daemon"] B1 --> B2 end BinaryExtract --> EnvSetup subgraph EnvSetup["Environment Setup"] direction TB E1[Set Environment
Variables] E2["FIM_DIR_GLOBAL
━━━━━━━━━━━━━━
FIM_DIR_APP
━━━━━━━━━━━━━━
FIM_DIR_INSTANCE
━━━━━━━━━━━━━━
...and more"] E1 --> E2 end EnvSetup --> Ready([Configuration Ready]) style ReservedSpace fill:#E3F2FD style PathSetup fill:#FFD700 style BinaryExtract fill:#FFE4B5 style EnvSetup fill:#E8F5E9 style Start fill:#E3F2FD style Ready fill:#90EE90
Hold "Shift" to enable pan & zoom

Configuration sources:

  1. Reserved space (3-4 MB embedded in binary)
  2. Permissions, boot command, environment, binds, desktop integration
  3. Environment variables (can override reserved space)
  4. FIM_OVERLAY, FIM_CASEFOLD, FIM_LAYERS, etc.
  5. Compiled defaults (fallback values)

Directory structure created:

  • Global directory: /tmp/fim
  • Build-specific: /tmp/fim/app/{COMMIT}_{TIMESTAMP}
  • Instance-specific: /tmp/fim/app/.../instance/{PID}
  • Binaries extracted to: /tmp/fim/app/.../bin/

Phase 4: Filesystem Setup

Purpose: Mount the layered filesystem stack that becomes the container's root.

flowchart LR Start([Filesystem Setup]) --> LayerMount subgraph LayerMount["DwarFS Layer Mounting"] direction TB L1[Mount DwarFS
Compressed Layers] L2["Layer 0
Base Filesystem
━━━━━━━━━━━━━━
Layer 1..N
Committed Layers
━━━━━━━━━━━━━━
External Layers
via FIM_LAYERS"] L1 --> L2 end LayerMount --> OverlaySetup subgraph OverlaySetup["Overlay Configuration"] direction TB O1{Select
Overlay Type} O2["UnionFS-FUSE
Writable Layer
━━━━━━━━━━━━━━
FUSE-OverlayFS
Writable Layer
━━━━━━━━━━━━━━
Bubblewrap
Native Overlay"] O1 --> O2 end OverlaySetup --> CasefoldCheck subgraph CasefoldCheck["Case-Insensitive Layer"] direction TB C1{Casefold
Enabled?} C2[Mount CIOPFS
Case-Insensitive Layer] C3[Skip CIOPFS] C1 -->|Yes| C2 C1 -->|No| C3 end CasefoldCheck --> JanitorSetup subgraph JanitorSetup["Cleanup Daemon"] direction TB J1[Spawn Janitor
Cleanup Daemon] end JanitorSetup --> Ready([Merged Root
Filesystem Ready]) style LayerMount fill:#E3F2FD style OverlaySetup fill:#FFF3E0 style CasefoldCheck fill:#E8F5E9 style JanitorSetup fill:#FFE4B5 style Start fill:#E3F2FD style Ready fill:#90EE90
Hold "Shift" to enable pan & zoom

Filesystem layers (bottom to top):

  1. Base layer: Core filesystem (Alpine/Arch/Blueprint)
  2. Committed layers: Previously saved changes
  3. External layers: Mounted from host filesystem
  4. Writable overlay: Temporary changes (UnionFS/OverlayFS/BWRAP native)
  5. Case-insensitive layer (optional): CIOPFS for Wine/Proton compatibility

Janitor daemon:

  • Monitors parent process health
  • Automatically unmounts filesystems if parent crashes
  • Prevents stale FUSE mounts

Phase 5: Portal Daemon Startup

Purpose: Enable inter-process communication between host and container.

flowchart TD Start([Portal Daemon
Startup]) --> FIFOSetup subgraph FIFOSetup["FIFO Creation"] direction TB F1[Create FIFO
Communication Channels] F2[daemon.host.fifo] F3[daemon.guest.fifo] F1 --> F2 F1 --> F3 end FIFOSetup --> DaemonSpawn subgraph DaemonSpawn["Daemon Processes"] direction TB D1[Spawn Host Daemon
Outside Container] D2[Spawn Guest Daemon
Inside Container] end DaemonSpawn --> PollingLoop subgraph PollingLoop["Request Handling"] direction TB P1[Host: Poll for Requests
from Container] P2[Guest: Poll for Requests
from Host] end PollingLoop --> Ready([Portal System
Ready]) style FIFOSetup fill:#E3F2FD style DaemonSpawn fill:#FFF3E0 style PollingLoop fill:#E8F5E9 style Start fill:#E3F2FD style Ready fill:#90EE90
Hold "Shift" to enable pan & zoom

Portal capabilities:

  • Container can execute commands on host
  • Host can execute commands in container (via fim-instance)
  • Full stdin/stdout/stderr redirection
  • Signal forwarding (Ctrl+C, etc.)

Phase 6: Command Parsing

Purpose: Determine what action to take based on command-line arguments.

flowchart TD Parse[Command Parsing] --> CheckArgs{Arguments
Provided?} CheckArgs -->|No args| DefaultBoot[Default Boot
Read boot command
from reserved space] CheckArgs -->|fim-* command| ParseFim[Parse fim-*
Command] DefaultBoot --> BootCmd[Execute Boot
Command] ParseFim --> ConfigCmd{Configuration
Command?} ConfigCmd -->|Yes| Examples1["fim-perms
fim-env
fim-boot
fim-overlay
fim-bind"] ConfigCmd -->|No| ExecCmd{Execution
Command?} Examples1 --> ModifyConfig[Modify Reserved
Space] ExecCmd -->|Yes| Examples2["fim-exec
fim-root
default boot"] ExecCmd -->|No| OtherCmd["fim-layer commit
fim-version
fim-help"] Examples2 --> SetupContainer[Setup Container] OtherCmd --> SpecialAction[Special Action] style Parse fill:#E3F2FD style ModifyConfig fill:#87CEEB style SetupContainer fill:#FFD700
Hold "Shift" to enable pan & zoom

Command categories:

  1. Configuration commands: Modify binary without starting container
  2. fim-perms add network,gpu
  3. fim-env set VAR=value
  4. fim-boot set firefox
  5. Execution commands: Start container and run command
  6. fim-exec firefox (run as user)
  7. fim-root apk add vim (run as root)
  8. Default boot (no arguments)
  9. Layer commands: Manage filesystem layers
  10. fim-layer commit binary (compress overlay to new layer)
  11. Utility commands: Information and help
  12. fim-version, fim-help

Phase 7: Container Setup (for Execution Commands)

Purpose: Configure Bubblewrap to create the sandboxed environment.

flowchart LR Start([Container Setup]) --> PermConfig subgraph PermConfig["Permission Configuration"] direction TB P1[Read Permissions
from Reserved Space] P2[Check Enabled
Permissions] P3["Apply Binds:
HOME → --bind $HOME
GPU → --dev-bind /dev/dri
NETWORK → network files
XORG → X11 socket
WAYLAND → wayland socket
AUDIO → audio devices
...and more"] P1 --> P2 P2 --> P3 end PermConfig --> CustomBinds subgraph CustomBinds["Custom Bind Mounts"] direction TB C1[Read Custom
Binds from Database] C2["Apply by Type:
RO → --ro-bind
RW → --bind
DEV → --dev-bind"] C1 --> C2 end CustomBinds --> EnvConfig subgraph EnvConfig["Environment Configuration"] direction TB E1[Read Environment
Variables] E2[Apply Each with
--setenv] E1 --> E2 end EnvConfig --> NSIdentity subgraph NSIdentity["Namespace & Identity"] direction TB N1["Configure Namespaces:
Mount namespace
PID namespace
Network namespace
IPC namespace"] N2[Set User Identity
uid/gid] N1 --> N2 end NSIdentity --> Ready([Configuration Ready]) style PermConfig fill:#FFE4B5 style CustomBinds fill:#E8F5E9 style EnvConfig fill:#E3F2FD style NSIdentity fill:#FFF3E0 style Start fill:#E3F2FD style Ready fill:#90EE90
Hold "Shift" to enable pan & zoom

Isolation mechanisms:

  • Mount namespace: Container sees isolated filesystem
  • Network namespace: No network unless network permission granted
  • PID namespace: Process isolation (optional)
  • IPC namespace: No inter-process communication with host
  • User namespace: Unprivileged containers (no root required)

Permission system:

  • Default: Full isolation (deny-all)
  • Opt-in: Each permission explicitly grants access
  • Granular: 15+ different permissions available
  • Composable: Permissions can be combined

Phase 8: Container Execution

Purpose: Launch Bubblewrap and execute the command inside the sandbox.

flowchart LR Start([Container Execution]) --> BwrapSetup subgraph BwrapSetup["Bubblewrap Setup"] direction TB B1[Launch Bubblewrap
Process] B2[Mount Root
Filesystem] B3[Apply All
Bind Mounts] B4[Set All
Environment Variables] B1 --> B2 B2 --> B3 B3 --> B4 end BwrapSetup --> ContainerRuntime subgraph ContainerRuntime["Container Runtime"] direction TB R1[Spawn Guest Portal
Daemon in Background] R2[Execute Command
in Sandbox] R1 --> R2 end ContainerRuntime --> Running[Command Running] Running --> Monitoring subgraph Monitoring["Runtime Monitoring"] direction TB M1[Portal Monitors
for IPC Requests] M2{Command
Exits?} M1 --> M2 M2 -->|No| M1 end M2 -->|Yes| Cleanup([Cleanup Phase]) style BwrapSetup fill:#E3F2FD style ContainerRuntime fill:#FFF3E0 style Running fill:#90EE90 style Monitoring fill:#E8F5E9 style M2 fill:#FFD700
Hold "Shift" to enable pan & zoom

What happens inside:

  1. Guest portal daemon starts (optional, for host→container commands)
  2. Bubblewrap creates isolated namespaces
  3. Root filesystem mounted (merged layers + overlay)
  4. All bind mounts applied
  5. Environment variables set
  6. Command executed (bash, firefox, etc.)
  7. Portal system active for IPC
  8. Command runs until completion or signal

During execution:

  • Container has access only to granted permissions
  • Portal enables host communication if needed
  • Standard I/O works normally
  • Signals forwarded (Ctrl+C terminates cleanly)

Phase 9: Cleanup

Purpose: Properly shut down the container and unmount filesystems.

flowchart LR Start([Cleanup Phase]) --> ContainerCleanup subgraph ContainerCleanup["Container Cleanup"] direction TB C1[Signal handler
catches SIGTERM/SIGINT] C2[Shutdown portal daemon] C3[Wait for container
process to exit] C4[Bubblewrap exits
with child exit code] C1 --> C2 C2 --> C3 C3 --> C4 end ContainerCleanup --> FilesystemCleanup subgraph FilesystemCleanup["Filesystem Cleanup"] direction TB F1[Destroy Filesystem
Controller] F2[Kill janitor daemon] F3[Un-mount overlay
and DwarFS layers] F1 --> F2 F2 --> F3 end FilesystemCleanup --> Clean[Clean temporary
directories] Clean --> Return([Return exit code
to parent]) style ContainerCleanup fill:#FFF3E0 style FilesystemCleanup fill:#E8F5E9 style Start fill:#90EE90 style Return fill:#FFB6C6
Hold "Shift" to enable pan & zoom

Cleanup order:

  1. Command finishes execution
  2. Guest portal daemon stopped
  3. Bubblewrap exits
  4. Filesystem controller destroyed
  5. Janitor daemon signaled to terminate
  6. Filesystems unmounted (CIOPFS → overlay → DwarFS layers)
  7. Temporary directories cleaned
  8. Exit code returned to shell

Graceful vs. crash handling:

  • Graceful exit: All cleanup happens in order
  • Crash/kill: Janitor daemon detects parent death and unmounts filesystems
  • Result: No stale FUSE mounts in either case

Boot Sequence Timeline

gantt title FlatImage Boot Sequence Timeline (Typical) dateFormat X axisFormat %L ms section Phase 1 Binary Relocation (if needed) :0, 50 section Phase 2 System Initialization :50, 100 section Phase 3 Configuration Loading :100, 200 section Phase 4 Filesystem Setup :200, 700 section Phase 5 Portal Startup :700, 750 section Phase 6 Command Parsing :750, 755 section Phase 7 Container Setup :755, 1055 section Phase 8 Container Execution :milestone, 1055, 0
Hold "Shift" to enable pan & zoom

Performance factors:

  • Number of DwarFS layers (more = slower)
  • Overlay type (BWRAP fastest, UnionFS slowest)
  • Casefold enabled (adds CIOPFS overhead)
  • External layers (network/disk latency)