tfgrid/zosbuilder
11
0
Fork 1
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity
Files
acc2fce84ae0523a63e4eb113f192c6d6602ce92
zosbuilder/scripts/functionlist.md
Jan De Landtsheer b9f94105cf fix: major build system improvements and container output issues 
- Fix container output visibility with proper TTY handling and debug mode
- Fix build order: kernel modules built before initramfs creation
- Implement two-stage kernel build to resolve chicken-and-egg dependency
- Fix sed command issues in kernel configuration with direct execution
- Add diffutils package to container for proper kernel build support
- Enhance NIC module/firmware correlation with intelligent selection
- Fix module staging logic: all NICs loaded in stage1 before network up
- Add smart firmware installation based on module requirements
- Create comprehensive function documentation (scripts/functionlist.md)
- Add debug container script for troubleshooting

Major fixes:
* Container builds now show real-time output
* Kernel builds work with proper GNU diff support
* Module/firmware selection optimized for common hardware
* Build process handles dependencies correctly
* Documentation provides complete function reference
2025-09-03 14:06:44 +02:00

167 lines
8.9 KiB
Markdown
Raw Blame History

# Function List - scripts/lib Library
This document provides a comprehensive description of all functions available in the `scripts/lib` library that are to be sourced by build scripts.
## **alpine.sh** - Alpine Linux Operations
### Core Functions
- [`alpine_extract_miniroot()`](lib/alpine.sh:14) - Downloads and extracts Alpine miniroot to target directory
- [`alpine_setup_chroot()`](lib/alpine.sh:70) - Sets up chroot environment with essential filesystem mounts
- [`alpine_cleanup_chroot()`](lib/alpine.sh:115) - Unmounts and cleans up chroot environment
- [`alpine_install_packages()`](lib/alpine.sh:142) - Installs packages from packages.list (excludes OpenRC)
- [`alpine_aggressive_cleanup()`](lib/alpine.sh:211) - Removes documentation, locales, dev files for size optimization
- [`alpine_configure_repos()`](lib/alpine.sh:302) - Configures Alpine package repositories
- [`alpine_configure_system()`](lib/alpine.sh:320) - Sets up basic system configuration (hostname, hosts, timezone)
- [`alpine_install_firmware()`](lib/alpine.sh:374) - Installs firmware packages for hardware support
## **common.sh** - Core Utilities
### Logging Functions
- [`log_info()`](lib/common.sh:31) - Log informational messages with timestamp and color
- [`log_warn()`](lib/common.sh:36) - Log warning messages with timestamp and color
- [`log_error()`](lib/common.sh:41) - Log error messages with timestamp and color
- [`log_debug()`](lib/common.sh:46) - Log debug messages (only when DEBUG=1)
### Execution and System Functions
- [`safe_execute()`](lib/common.sh:54) - Execute commands with error handling and logging
- [`section_header()`](lib/common.sh:76) - Creates formatted section headers for output
- [`command_exists()`](lib/common.sh:86) - Check if command is available in PATH
- [`in_container()`](lib/common.sh:91) - Detect if running inside a container
- [`check_dependencies()`](lib/common.sh:96) - Verify required tools are installed
### File System Operations
- [`safe_mkdir()`](lib/common.sh:139) - Create directories safely with error handling
- [`safe_rmdir()`](lib/common.sh:146) - Remove directories safely with error handling
- [`safe_copy()`](lib/common.sh:155) - Copy files/directories safely with error handling
- [`resolve_path()`](lib/common.sh:168) - Convert relative to absolute paths
- [`get_file_size()`](lib/common.sh:178) - Get human-readable file size
- [`wait_for_file()`](lib/common.sh:188) - Wait for file to exist with timeout
- [`cleanup_on_exit()`](lib/common.sh:202) - Cleanup function for exit traps
## **components.sh** - ThreeFold Component Management
### Component Processing
- [`components_parse_sources_conf()`](lib/components.sh:13) - Parse and build all components from sources.conf
- [`components_download_git()`](lib/components.sh:72) - Clone Git repositories with specific versions
- [`components_download_release()`](lib/components.sh:104) - Download pre-built release binaries
- [`components_process_extra_options()`](lib/components.sh:144) - Handle rename/extract options for components
- [`components_build_component()`](lib/components.sh:183) - Build component using specified build function
### Build Environment
- [`components_setup_rust_env()`](lib/components.sh:217) - Configure Rust environment for musl builds
### Component-Specific Build Functions
- [`build_zinit()`](lib/components.sh:252) - Build zinit init system from source (Rust)
- [`build_rfs()`](lib/components.sh:304) - Build rfs (rootfs) from source (Rust)
- [`build_mycelium()`](lib/components.sh:356) - Build mycelium networking from source (Rust, subdirectory)
- [`install_rfs()`](lib/components.sh:401) - Install pre-built rfs binary
- [`install_corex()`](lib/components.sh:427) - Install pre-built corex binary
### Verification and Cleanup
- [`components_verify_installation()`](lib/components.sh:457) - Verify all components were installed correctly
- [`components_cleanup()`](lib/components.sh:493) - Clean build artifacts
## **docker.sh** - Container Runtime Management
### Runtime Detection and Setup
- [`docker_detect_runtime()`](lib/docker.sh:14) - Detect available container runtime (Docker/Podman)
- [`docker_verify_rootless()`](lib/docker.sh:33) - Verify rootless container setup works
- [`docker_check_capabilities()`](lib/docker.sh:209) - Check container runtime capabilities
- [`docker_setup_rootless()`](lib/docker.sh:240) - Setup rootless environment (subuid/subgid)
### Container Image Management
- [`docker_build_container()`](lib/docker.sh:47) - Build container image with build tools
- [`docker_create_dockerfile()`](lib/docker.sh:65) - Create optimized Dockerfile for build environment
- [`docker_commit_builder()`](lib/docker.sh:178) - Commit container state for reuse
- [`docker_cleanup()`](lib/docker.sh:191) - Clean up container images
### Container Execution
- [`docker_start_rootless()`](lib/docker.sh:116) - Start rootless container for building
- [`docker_run_build()`](lib/docker.sh:154) - Run build command in container with proper mounts
## **initramfs.sh** - Initramfs Assembly
### Core Assembly Functions
- [`initramfs_setup_zinit()`](lib/initramfs.sh:13) - Setup zinit as init system (replaces OpenRC completely)
- [`initramfs_install_init_script()`](lib/initramfs.sh:71) - Install critical /init script for initramfs boot
- [`initramfs_setup_modules()`](lib/initramfs.sh:98) - Setup 2-stage module loading with dependencies
### Module Management
- [`initramfs_resolve_module_dependencies()`](lib/initramfs.sh:166) - Recursively resolve module dependencies using modinfo
- [`initramfs_create_module_scripts()`](lib/initramfs.sh:236) - Create stage1/stage2 module loading scripts for zinit
### Optimization and Packaging
- [`initramfs_strip_and_upx()`](lib/initramfs.sh:300) - Strip debug symbols and UPX compress binaries for size optimization
- [`initramfs_create_cpio()`](lib/initramfs.sh:383) - Create final compressed initramfs archive (xz/gzip/zstd/uncompressed)
### Validation and Testing
- [`initramfs_validate()`](lib/initramfs.sh:449) - Validate initramfs contents and structure
- [`initramfs_test_archive()`](lib/initramfs.sh:549) - Test initramfs archive integrity
## **kernel.sh** - Kernel Building
### Source Management
- [`kernel_download_source()`](lib/kernel.sh:14) - Download Linux kernel source code from kernel.org
- [`kernel_apply_config()`](lib/kernel.sh:68) - Apply kernel configuration with embedded initramfs path
- [`kernel_modify_config_for_initramfs()`](lib/kernel.sh:116) - Modify kernel config for embedded initramfs support
### Build Functions
- [`kernel_build_with_initramfs()`](lib/kernel.sh:144) - Build kernel with embedded initramfs (complete process)
- [`kernel_build_modules()`](lib/kernel.sh:203) - Build kernel modules for initramfs inclusion
### Cleanup
- [`kernel_cleanup()`](lib/kernel.sh:242) - Clean kernel build artifacts (with option to keep source)
## **testing.sh** - Virtualization Testing
### QEMU Testing
- [`testing_qemu_boot()`](lib/testing.sh:14) - Test kernel boot with QEMU (multiple modes: basic/serial/interactive)
- [`testing_qemu_basic_boot()`](lib/testing.sh:55) - Basic automated QEMU boot test with timeout
- [`testing_qemu_serial_boot()`](lib/testing.sh:90) - QEMU serial console test for debugging
- [`testing_qemu_interactive_boot()`](lib/testing.sh:114) - Interactive QEMU session (no timeout)
### Cloud Hypervisor Testing
- [`testing_cloud_hypervisor_boot()`](lib/testing.sh:135) - Test with cloud-hypervisor VMM
- [`testing_cloud_hypervisor_basic()`](lib/testing.sh:172) - Basic cloud-hypervisor test with timeout
- [`testing_cloud_hypervisor_serial()`](lib/testing.sh:206) - cloud-hypervisor serial console test
### Analysis and Orchestration
- [`testing_analyze_boot_log()`](lib/testing.sh:228) - Analyze boot logs for success/failure indicators
- [`testing_run_all()`](lib/testing.sh:299) - Run comprehensive test suite (QEMU + cloud-hypervisor)
## Usage Notes
### Function Availability
All functions are exported for sourcing and can be called from any script that sources the respective library file. The common pattern is:
```bash
# Source the library
source "${SCRIPT_DIR}/lib/common.sh"
source "${SCRIPT_DIR}/lib/alpine.sh"
# ... other libraries as needed
# Use the functions
alpine_extract_miniroot "/path/to/target"
components_parse_sources_conf "/path/to/sources.conf" "/path/to/components"
```
### Error Handling
All functions follow consistent error handling patterns:
- Return non-zero exit codes on failure
- Use [`safe_execute()`](lib/common.sh:54) for command execution
- Provide detailed logging via [`log_*()`](lib/common.sh:31) functions
- Clean up resources on failure
### Dependencies
Functions have dependencies on:
- External tools (checked via [`check_dependencies()`](lib/common.sh:96))
- Other library functions (noted in function descriptions)
- Configuration files and environment variables
- Proper directory structures
### Configuration
Most functions respect environment variables for configuration:
- `DEBUG=1` enables debug logging
- `ALPINE_VERSION`, `KERNEL_VERSION` set versions
- `RUST_TARGET` configures Rust builds
- Various `*_DIR` variables set paths
Reference in New Issue View Git Blame Copy Permalink