feat: Add support for virt package

- Add sal-virt package to the workspace members
- Update MONOREPO_CONVERSION_PLAN.md to reflect the
  completion of sal-process and sal-virt packages
- Update src/lib.rs to include sal-virt
- Update src/postgresclient to use sal-virt instead of local
  virt module
- Update tests to use sal-virt

virt/src/nerdctl/README.md

@@ -0,0 +1,223 @@
+# SAL `nerdctl` Module (`sal::virt::nerdctl`)
+
+## Overview
+
+The `sal::virt::nerdctl` module provides a comprehensive Rust interface for interacting with `nerdctl`, a command-line tool for `containerd`.
+It allows for managing container lifecycles, images, and other `nerdctl` functionalities programmatically from Rust and through Rhai scripts via `herodo`.
+
+This module offers two primary ways to interact with `nerdctl`:
+1.  A fluent **`Container` builder pattern** for defining, creating, and managing containers with detailed configurations.
+2.  **Direct static functions** that wrap common `nerdctl` commands for quick operations on containers and images.
+
+## Core Components
+
+### 1. `NerdctlError` (in `mod.rs`)
+
+An enum defining specific error types for `nerdctl` operations:
+- `CommandExecutionFailed(io::Error)`: `nerdctl` command failed to start (e.g., not found).
+- `CommandFailed(String)`: `nerdctl` command executed but returned an error.
+- `JsonParseError(String)`: Failure to parse JSON output from `nerdctl`.
+- `ConversionError(String)`: Error during data type conversions.
+- `Other(String)`: Generic errors.
+
+### 2. `execute_nerdctl_command` (in `cmd.rs`)
+
+The core function for executing `nerdctl` commands. It takes an array of string arguments, runs the command, and returns a `CommandResult` or `NerdctlError`.
+
+```rust
+// Example (internal usage)
+// use sal::virt::nerdctl::execute_nerdctl_command;
+// let result = execute_nerdctl_command(&["ps", "-a"]);
+```
+
+### 3. `Container` Struct (defined in `container_types.rs`, builder in `container_builder.rs`, operations in `container_operations.rs`)
+
+Represents a `nerdctl` container and is the centerpiece of the builder pattern.
+
+**Fields (Configuration):**
+- `name: String`: Name of the container.
+- `container_id: Option<String>`: ID of the container (populated after creation).
+- `image: Option<String>`: Base image for the container.
+- `ports: Vec<String>`: Port mappings (e.g., `"8080:80"`).
+- `volumes: Vec<String>`: Volume mounts (e.g., `"/host/path:/container/path"`).
+- `env_vars: HashMap<String, String>`: Environment variables.
+- `network: Option<String>`: Network to connect to.
+- `network_aliases: Vec<String>`: Network aliases.
+- `cpu_limit: Option<String>`, `memory_limit: Option<String>`, `memory_swap_limit: Option<String>`, `cpu_shares: Option<String>`: Resource limits.
+- `restart_policy: Option<String>`: Restart policy (e.g., `"always"`).
+- `health_check: Option<HealthCheck>`: Health check configuration.
+- `detach: bool`: Whether to run in detached mode (default: `false`, but Rhai `container_build` implies `true` often).
+- `snapshotter: Option<String>`: Snapshotter to use.
+
+**Builder Methods (Fluent Interface - `impl Container` in `container_builder.rs`):**
+These methods configure the `Container` object and return `Self` for chaining.
+- `Container::new(name: &str, image: &str)`: Constructor (Note: Rhai uses `nerdctl_container_new(name)` and `nerdctl_container_from_image(name, image)` which call underlying Rust constructors).
+- `reset()`: Resets configuration, stops/removes existing container with the same name.
+- `with_port(port: &str)`, `with_ports(ports: &[&str])`
+- `with_volume(volume: &str)`, `with_volumes(volumes: &[&str])`
+- `with_env(key: &str, value: &str)`, `with_envs(env_map: &HashMap<&str, &str>)`
+- `with_network(network: &str)`
+- `with_network_alias(alias: &str)`, `with_network_aliases(aliases: &[&str])`
+- `with_cpu_limit(cpus: &str)`
+- `with_memory_limit(memory: &str)`
+- `with_memory_swap_limit(memory_swap: &str)`
+- `with_cpu_shares(shares: &str)`
+- `with_restart_policy(policy: &str)`
+- `with_health_check(cmd: &str)`
+- `with_health_check_options(cmd, interval, timeout, retries, start_period)`
+- `with_snapshotter(snapshotter: &str)`
+- `with_detach(detach: bool)`
+
+**Action Methods (on `Container` instances):
+- `build()` (in `container_builder.rs`): Assembles and executes `nerdctl run` with all configured options. Populates `container_id` on success.
+- `start()` (in `container_operations.rs`): Starts the container. If not yet built, it attempts to pull the image and build the container first. Verifies the container is running and provides detailed logs/status on failure.
+- `stop()` (in `container_operations.rs`): Stops the container.
+- `remove()` (in `container_operations.rs`): Removes the container.
+- `exec(command: &str)` (in `container_operations.rs`): Executes a command in the container.
+- `copy(source: &str, dest: &str)` (in `container_operations.rs`): Copies files/folders. `source`/`dest` must be formatted like `container_name_or_id:/path` or `/local/path`.
+- `status()` (in `container_operations.rs`): Returns `ContainerStatus` by parsing `nerdctl inspect`.
+- `health_status()` (in `container_operations.rs`): Returns the health status string from `nerdctl inspect`.
+- `logs()` (in `container_operations.rs`): Fetches container logs.
+- `resources()` (in `container_operations.rs`): Returns `ResourceUsage` by parsing `nerdctl stats`.
+- `commit(image_name: &str)` (in `container_operations.rs`): Commits the container to a new image.
+- `export(path: &str)` (in `container_operations.rs`): Exports the container's filesystem as a tarball.
+
+### 4. `HealthCheck` Struct (in `container_types.rs`)
+
+Defines health check parameters:
+- `cmd: String`: Command to execute.
+- `interval: Option<String>`
+- `timeout: Option<String>`
+- `retries: Option<u32>`
+- `start_period: Option<String>`
+
+### 5. `prepare_health_check_command` (in `health_check_script.rs`)
+
+A helper function that takes a health check command string. If it's multi-line, it attempts to save it as an executable script in `/root/hero/var/containers/healthcheck_<container_name>.sh` and returns the script path. Otherwise, returns the command as is. The path `/root/hero/var/containers` implies this script needs to be accessible from within the target container at that specific location if a multi-line script is used.
+
+### 6. `Image` Struct (in `images.rs`)
+
+Represents a `nerdctl` image, typically from `nerdctl images` output.
+- `id: String`
+- `repository: String`
+- `tag: String`
+- `size: String`
+- `created: String`
+
+### 7. Static Image Functions (in `images.rs`)
+
+These functions operate on images:
+- `images() -> Result<CommandResult, NerdctlError>`: Lists images (`nerdctl images`).
+- `image_remove(image: &str)`: Removes an image (`nerdctl rmi`).
+- `image_push(image: &str, destination: &str)`: Pushes an image (`nerdctl push`).
+- `image_tag(image: &str, new_name: &str)`: Tags an image (`nerdctl tag`).
+- `image_pull(image: &str)`: Pulls an image (`nerdctl pull`).
+- `image_commit(container: &str, image_name: &str)`: Commits a container to an image (`nerdctl commit`).
+- `image_build(tag: &str, context_path: &str)`: Builds an image from a Dockerfile (`nerdctl build -t <tag> <context_path>`).
+
+### 8. Static Container Functions (in `container_functions.rs`)
+
+Direct wrappers for `nerdctl` commands, an alternative to the builder pattern:
+- `run(image: &str, name: Option<&str>, detach: bool, ports: Option<&[&str]>, snapshotter: Option<&str>)`: Runs a container.
+- `exec(container: &str, command: &str)`: Executes a command in a running container.
+- `copy(source: &str, dest: &str)`: Copies files.
+- `stop(container: &str)`: Stops a container.
+- `remove(container: &str)`: Removes a container.
+- `list(all: bool)`: Lists containers (`nerdctl ps`).
+- `logs(container: &str)`: Fetches logs for a container.
+
+### 9. `ContainerStatus` and `ResourceUsage` Structs (in `container_types.rs`)
+- `ContainerStatus`: Holds parsed data from `nerdctl inspect` (state, status, created, started, health info).
+- `ResourceUsage`: Holds parsed data from `nerdctl stats` (CPU, memory, network, block I/O, PIDs).
+
+## Usage Examples
+
+### Rust Example (Builder Pattern)
+
+```rust
+use sal::virt::nerdctl::{Container, NerdctlError};
+use std::collections::HashMap;
+
+fn main() -> Result<(), NerdctlError> {
+    let mut envs = HashMap::new();
+    envs.insert("MY_VAR", "my_value");
+
+    let container_config = Container::new("my_nginx_container", "nginx:latest") // Assuming a constructor like this exists or is adapted
+        .with_port("8080:80")
+        .with_envs(&envs)
+        .with_detach(true)
+        .with_restart_policy("always");
+
+    // Build (create and run) the container
+    let built_container = container_config.build()?;
+    println!("Container {} created with ID: {:?}", built_container.name, built_container.container_id);
+
+    // Perform operations
+    let status = built_container.status()?;
+    println!("Status: {}, State: {}", status.status, status.state);
+
+    // Stop and remove
+    built_container.stop()?;
+    built_container.remove()?;
+    println!("Container stopped and removed.");
+
+    Ok(())
+}
+```
+
+*Note: The direct `Container::new(name, image)` constructor isn't explicitly shown in the provided Rust code snippets for `Container` itself, but the Rhai bindings `nerdctl_container_new` and `nerdctl_container_from_image` imply underlying Rust constructors that initialize a `Container` struct. The `build()` method is the primary way to run it after configuration.*
+
+### Rhai Script Example (using `herodo`)
+
+```rhai
+// Create and configure a container using the builder pattern
+let c = nerdctl_container_from_image("my_redis", "redis:alpine")
+    .with_port("6379:6379")
+    .with_restart_policy("unless-stopped");
+
+// Build and run the container
+let running_container = c.build();
+
+if running_container.is_ok() {
+    print(`Container ${running_container.name} ID: ${running_container.container_id}`);
+    
+    // Get status
+    let status = running_container.status();
+    if status.is_ok() {
+        print(`Status: ${status.state}, Health: ${status.health_status}`);
+    }
+    
+    // Stop the container (example, might need a mutable borrow or re-fetch)
+    // running_container.stop(); // Assuming stop is available and works on the result
+    // running_container.remove();
+} else {
+    print(`Error building container: ${running_container.error()}`);
+}
+
+// Direct command example
+let images = nerdctl_images();
+print(images.stdout);
+
+nerdctl_image_pull("alpine:latest");
+```
+
+## Key Design Points
+
+- **Fluent Builder**: The `Container` struct uses a builder pattern, allowing for clear and chainable configuration of container parameters before execution.
+- **Comprehensive Operations**: Covers most common `nerdctl` functionalities for containers and images.
+- **Error Handling**: `NerdctlError` provides typed errors. The Rhai layer adds more descriptive error messages for common scenarios.
+- **Dual API**: Offers both a detailed builder pattern and simpler static functions for flexibility.
+- **Health Check Scripting**: Supports multi-line shell scripts for health checks by saving them to a file, though care must be taken regarding the script's accessibility from within the target container.
+- **Resource Parsing**: Includes parsing for `nerdctl inspect` (JSON) and `nerdctl stats` (tabular text) to provide structured information.
+
+## File Structure
+
+- `src/virt/nerdctl/mod.rs`: Main module file, error definitions, sub-module declarations.
+- `src/virt/nerdctl/cmd.rs`: Core `execute_nerdctl_command` function.
+- `src/virt/nerdctl/container_types.rs`: Definitions for `Container`, `HealthCheck`, `ContainerStatus`, `ResourceUsage`.
+- `src/virt/nerdctl/container_builder.rs`: Implements the builder pattern methods for the `Container` struct.
+- `src/virt/nerdctl/container_operations.rs`: Implements instance methods on `Container` (start, stop, status, etc.).
+- `src/virt/nerdctl/images.rs`: `Image` struct and static functions for image management.
+- `src/virt/nerdctl/container_functions.rs`: Static functions for direct container commands.
+- `src/virt/nerdctl/health_check_script.rs`: Logic for `prepare_health_check_command`.
+- `src/rhai/nerdctl.rs`: Rhai script bindings for `herodo`.

virt/src/nerdctl/cmd.rs

@@ -0,0 +1,36 @@
+// File: /root/code/git.threefold.info/herocode/sal/src/virt/nerdctl/cmd.rs
+
+// Basic nerdctl operations for container management
+use super::NerdctlError;
+use sal_process::CommandResult;
+use std::process::Command;
+
+/// Execute a nerdctl command and return the result
+pub fn execute_nerdctl_command(args: &[&str]) -> Result<CommandResult, NerdctlError> {
+    let output = Command::new("nerdctl").args(args).output();
+
+    match output {
+        Ok(output) => {
+            let stdout = String::from_utf8_lossy(&output.stdout).to_string();
+            let stderr = String::from_utf8_lossy(&output.stderr).to_string();
+
+            let result = CommandResult {
+                stdout,
+                stderr,
+                success: output.status.success(),
+                code: output.status.code().unwrap_or(-1),
+            };
+
+            if result.success {
+                Ok(result)
+            } else {
+                Err(NerdctlError::CommandFailed(format!(
+                    "Command failed with code {}: {}",
+                    result.code,
+                    result.stderr.trim()
+                )))
+            }
+        }
+        Err(e) => Err(NerdctlError::CommandExecutionFailed(e)),
+    }
+}

virt/src/nerdctl/container.rs

@@ -0,0 +1,82 @@
+// File: /root/code/git.threefold.info/herocode/sal/src/virt/nerdctl/container.rs
+
+use super::container_types::Container;
+use crate::nerdctl::{execute_nerdctl_command, NerdctlError};
+use sal_os as os;
+use std::collections::HashMap;
+
+impl Container {
+    /// Create a new container reference with the given name
+    ///
+    /// # Arguments
+    ///
+    /// * `name` - Name for the container
+    ///
+    /// # Returns
+    ///
+    /// * `Result<Self, NerdctlError>` - Container instance or error
+    pub fn new(name: &str) -> Result<Self, NerdctlError> {
+        // Check if required commands exist
+        match os::cmd_ensure_exists("nerdctl,runc,buildah") {
+            Err(e) => {
+                return Err(NerdctlError::CommandExecutionFailed(std::io::Error::new(
+                    std::io::ErrorKind::NotFound,
+                    format!("Required commands not found: {}", e),
+                )))
+            }
+            _ => {}
+        }
+
+        // Check if container exists
+        let result = execute_nerdctl_command(&["ps", "-a", "--format", "{{.Names}} {{.ID}}"])?;
+
+        // Look for the container name in the output
+        let container_id = result
+            .stdout
+            .lines()
+            .filter_map(|line| {
+                if line.starts_with(&format!("{} ", name)) {
+                    Some(line.split_whitespace().nth(1)?.to_string())
+                } else {
+                    None
+                }
+            })
+            .next();
+
+        Ok(Self {
+            name: name.to_string(),
+            container_id,
+            image: None,
+            config: HashMap::new(),
+            ports: Vec::new(),
+            volumes: Vec::new(),
+            env_vars: HashMap::new(),
+            network: None,
+            network_aliases: Vec::new(),
+            cpu_limit: None,
+            memory_limit: None,
+            memory_swap_limit: None,
+            cpu_shares: None,
+            restart_policy: None,
+            health_check: None,
+            detach: false,
+            snapshotter: None,
+        })
+    }
+
+    /// Create a container from an image
+    ///
+    /// # Arguments
+    ///
+    /// * `name` - Name for the container
+    /// * `image` - Image to create the container from
+    ///
+    /// # Returns
+    ///
+    /// * `Result<Self, NerdctlError>` - Container instance or error
+    pub fn from_image(name: &str, image: &str) -> Result<Self, NerdctlError> {
+        let mut container = Self::new(name)?;
+        container.image = Some(image.to_string());
+        Ok(container)
+    }
+}

virt/src/nerdctl/container_builder.rs

@@ -0,0 +1,517 @@
+// File: /root/code/git.threefold.info/herocode/sal/src/virt/nerdctl/container_builder.rs
+
+use super::container_types::{Container, HealthCheck};
+use super::health_check_script::prepare_health_check_command;
+use crate::nerdctl::{execute_nerdctl_command, NerdctlError};
+use std::collections::HashMap;
+
+impl Container {
+    /// Reset the container configuration to defaults while keeping the name and image
+    /// If the container exists, it will be stopped and removed.
+    ///
+    /// # Returns
+    ///
+    /// * `Self` - The container instance for method chaining
+    pub fn reset(self) -> Self {
+        let name = self.name;
+        let image = self.image.clone();
+
+        // If container exists, stop and remove it
+        if let Some(container_id) = &self.container_id {
+            println!(
+                "Container exists. Stopping and removing container '{}'...",
+                name
+            );
+
+            // Try to stop the container
+            let _ = execute_nerdctl_command(&["stop", container_id]);
+
+            // Try to remove the container
+            let _ = execute_nerdctl_command(&["rm", container_id]);
+        }
+
+        // Create a new container with just the name and image, but no container_id
+        Self {
+            name,
+            container_id: None, // Reset container_id to None since we removed the container
+            image,
+            config: std::collections::HashMap::new(),
+            ports: Vec::new(),
+            volumes: Vec::new(),
+            env_vars: std::collections::HashMap::new(),
+            network: None,
+            network_aliases: Vec::new(),
+            cpu_limit: None,
+            memory_limit: None,
+            memory_swap_limit: None,
+            cpu_shares: None,
+            restart_policy: None,
+            health_check: None,
+            detach: false,
+            snapshotter: None,
+        }
+    }
+
+    /// Add a port mapping
+    ///
+    /// # Arguments
+    ///
+    /// * `port` - Port mapping (e.g., "8080:80")
+    ///
+    /// # Returns
+    ///
+    /// * `Self` - The container instance for method chaining
+    pub fn with_port(mut self, port: &str) -> Self {
+        self.ports.push(port.to_string());
+        self
+    }
+
+    /// Add multiple port mappings
+    ///
+    /// # Arguments
+    ///
+    /// * `ports` - Array of port mappings (e.g., ["8080:80", "8443:443"])
+    ///
+    /// # Returns
+    ///
+    /// * `Self` - The container instance for method chaining
+    pub fn with_ports(mut self, ports: &[&str]) -> Self {
+        for port in ports {
+            self.ports.push(port.to_string());
+        }
+        self
+    }
+
+    /// Add a volume mount
+    ///
+    /// # Arguments
+    ///
+    /// * `volume` - Volume mount (e.g., "/host/path:/container/path")
+    ///
+    /// # Returns
+    ///
+    /// * `Self` - The container instance for method chaining
+    pub fn with_volume(mut self, volume: &str) -> Self {
+        self.volumes.push(volume.to_string());
+        self
+    }
+
+    /// Add multiple volume mounts
+    ///
+    /// # Arguments
+    ///
+    /// * `volumes` - Array of volume mounts (e.g., ["/host/path1:/container/path1", "/host/path2:/container/path2"])
+    ///
+    /// # Returns
+    ///
+    /// * `Self` - The container instance for method chaining
+    pub fn with_volumes(mut self, volumes: &[&str]) -> Self {
+        for volume in volumes {
+            self.volumes.push(volume.to_string());
+        }
+        self
+    }
+
+    /// Add an environment variable
+    ///
+    /// # Arguments
+    ///
+    /// * `key` - Environment variable name
+    /// * `value` - Environment variable value
+    ///
+    /// # Returns
+    ///
+    /// * `Self` - The container instance for method chaining
+    pub fn with_env(mut self, key: &str, value: &str) -> Self {
+        self.env_vars.insert(key.to_string(), value.to_string());
+        self
+    }
+
+    /// Add multiple environment variables
+    ///
+    /// # Arguments
+    ///
+    /// * `env_map` - Map of environment variable names to values
+    ///
+    /// # Returns
+    ///
+    /// * `Self` - The container instance for method chaining
+    pub fn with_envs(mut self, env_map: &HashMap<&str, &str>) -> Self {
+        for (key, value) in env_map {
+            self.env_vars.insert(key.to_string(), value.to_string());
+        }
+        self
+    }
+
+    /// Set the network for the container
+    ///
+    /// # Arguments
+    ///
+    /// * `network` - Network name
+    ///
+    /// # Returns
+    ///
+    /// * `Self` - The container instance for method chaining
+    pub fn with_network(mut self, network: &str) -> Self {
+        self.network = Some(network.to_string());
+        self
+    }
+
+    /// Add a network alias for the container
+    ///
+    /// # Arguments
+    ///
+    /// * `alias` - Network alias
+    ///
+    /// # Returns
+    ///
+    /// * `Self` - The container instance for method chaining
+    pub fn with_network_alias(mut self, alias: &str) -> Self {
+        self.network_aliases.push(alias.to_string());
+        self
+    }
+
+    /// Add multiple network aliases for the container
+    ///
+    /// # Arguments
+    ///
+    /// * `aliases` - Array of network aliases
+    ///
+    /// # Returns
+    ///
+    /// * `Self` - The container instance for method chaining
+    pub fn with_network_aliases(mut self, aliases: &[&str]) -> Self {
+        for alias in aliases {
+            self.network_aliases.push(alias.to_string());
+        }
+        self
+    }
+
+    /// Set CPU limit for the container
+    ///
+    /// # Arguments
+    ///
+    /// * `cpus` - CPU limit (e.g., "0.5" for half a CPU, "2" for 2 CPUs)
+    ///
+    /// # Returns
+    ///
+    /// * `Self` - The container instance for method chaining
+    pub fn with_cpu_limit(mut self, cpus: &str) -> Self {
+        self.cpu_limit = Some(cpus.to_string());
+        self
+    }
+
+    /// Set memory limit for the container
+    ///
+    /// # Arguments
+    ///
+    /// * `memory` - Memory limit (e.g., "512m" for 512MB, "1g" for 1GB)
+    ///
+    /// # Returns
+    ///
+    /// * `Self` - The container instance for method chaining
+    pub fn with_memory_limit(mut self, memory: &str) -> Self {
+        self.memory_limit = Some(memory.to_string());
+        self
+    }
+
+    /// Set memory swap limit for the container
+    ///
+    /// # Arguments
+    ///
+    /// * `memory_swap` - Memory swap limit (e.g., "1g" for 1GB)
+    ///
+    /// # Returns
+    ///
+    /// * `Self` - The container instance for method chaining
+    pub fn with_memory_swap_limit(mut self, memory_swap: &str) -> Self {
+        self.memory_swap_limit = Some(memory_swap.to_string());
+        self
+    }
+
+    /// Set CPU shares for the container (relative weight)
+    ///
+    /// # Arguments
+    ///
+    /// * `shares` - CPU shares (e.g., "1024" for default, "512" for half)
+    ///
+    /// # Returns
+    ///
+    /// * `Self` - The container instance for method chaining
+    pub fn with_cpu_shares(mut self, shares: &str) -> Self {
+        self.cpu_shares = Some(shares.to_string());
+        self
+    }
+
+    /// Set restart policy for the container
+    ///
+    /// # Arguments
+    ///
+    /// * `policy` - Restart policy (e.g., "no", "always", "on-failure", "unless-stopped")
+    ///
+    /// # Returns
+    ///
+    /// * `Self` - The container instance for method chaining
+    pub fn with_restart_policy(mut self, policy: &str) -> Self {
+        self.restart_policy = Some(policy.to_string());
+        self
+    }
+
+    /// Set a simple health check for the container
+    ///
+    /// # Arguments
+    ///
+    /// * `cmd` - Command to run for health check (e.g., "curl -f http://localhost/ || exit 1")
+    ///
+    /// # Returns
+    ///
+    /// * `Self` - The container instance for method chaining
+    pub fn with_health_check(mut self, cmd: &str) -> Self {
+        // Use the health check script module to prepare the command
+        let prepared_cmd = prepare_health_check_command(cmd, &self.name);
+
+        self.health_check = Some(HealthCheck {
+            cmd: prepared_cmd,
+            interval: None,
+            timeout: None,
+            retries: None,
+            start_period: None,
+        });
+        self
+    }
+
+    /// Set a health check with custom options for the container
+    ///
+    /// # Arguments
+    ///
+    /// * `cmd` - Command to run for health check
+    /// * `interval` - Optional time between running the check (e.g., "30s", "1m")
+    /// * `timeout` - Optional maximum time to wait for a check to complete (e.g., "30s", "1m")
+    /// * `retries` - Optional number of consecutive failures needed to consider unhealthy
+    /// * `start_period` - Optional start period for the container to initialize before counting retries (e.g., "30s", "1m")
+    ///
+    /// # Returns
+    ///
+    /// * `Self` - The container instance for method chaining
+    pub fn with_health_check_options(
+        mut self,
+        cmd: &str,
+        interval: Option<&str>,
+        timeout: Option<&str>,
+        retries: Option<u32>,
+        start_period: Option<&str>,
+    ) -> Self {
+        // Use the health check script module to prepare the command
+        let prepared_cmd = prepare_health_check_command(cmd, &self.name);
+
+        let mut health_check = HealthCheck {
+            cmd: prepared_cmd,
+            interval: None,
+            timeout: None,
+            retries: None,
+            start_period: None,
+        };
+
+        if let Some(interval_value) = interval {
+            health_check.interval = Some(interval_value.to_string());
+        }
+
+        if let Some(timeout_value) = timeout {
+            health_check.timeout = Some(timeout_value.to_string());
+        }
+
+        if let Some(retries_value) = retries {
+            health_check.retries = Some(retries_value);
+        }
+
+        if let Some(start_period_value) = start_period {
+            health_check.start_period = Some(start_period_value.to_string());
+        }
+
+        self.health_check = Some(health_check);
+        self
+    }
+
+    /// Set the snapshotter
+    ///
+    /// # Arguments
+    ///
+    /// * `snapshotter` - Snapshotter to use
+    ///
+    /// # Returns
+    ///
+    /// * `Self` - The container instance for method chaining
+    pub fn with_snapshotter(mut self, snapshotter: &str) -> Self {
+        self.snapshotter = Some(snapshotter.to_string());
+        self
+    }
+
+    /// Set whether to run in detached mode
+    ///
+    /// # Arguments
+    ///
+    /// * `detach` - Whether to run in detached mode
+    ///
+    /// # Returns
+    ///
+    /// * `Self` - The container instance for method chaining
+    pub fn with_detach(mut self, detach: bool) -> Self {
+        self.detach = detach;
+        self
+    }
+
+    /// Build the container
+    ///
+    /// # Returns
+    ///
+    /// * `Result<Self, NerdctlError>` - Container instance or error
+    pub fn build(self) -> Result<Self, NerdctlError> {
+        // If container already exists, return it
+        if self.container_id.is_some() {
+            return Ok(self);
+        }
+
+        // If no image is specified, return an error
+        let image = match &self.image {
+            Some(img) => img,
+            None => {
+                return Err(NerdctlError::Other(
+                    "No image specified for container creation".to_string(),
+                ))
+            }
+        };
+
+        // Build the command arguments as strings
+        let mut args_strings = Vec::new();
+        args_strings.push("run".to_string());
+
+        if self.detach {
+            args_strings.push("-d".to_string());
+        }
+
+        args_strings.push("--name".to_string());
+        args_strings.push(self.name.clone());
+
+        // Add port mappings
+        for port in &self.ports {
+            args_strings.push("-p".to_string());
+            args_strings.push(port.clone());
+        }
+
+        // Add volume mounts
+        for volume in &self.volumes {
+            args_strings.push("-v".to_string());
+            args_strings.push(volume.clone());
+        }
+
+        // Add environment variables
+        for (key, value) in &self.env_vars {
+            args_strings.push("-e".to_string());
+            args_strings.push(format!("{}={}", key, value));
+        }
+
+        // Add network configuration
+        if let Some(network) = &self.network {
+            args_strings.push("--network".to_string());
+            args_strings.push(network.clone());
+        }
+
+        // Add network aliases
+        for alias in &self.network_aliases {
+            args_strings.push("--network-alias".to_string());
+            args_strings.push(alias.clone());
+        }
+
+        // Add resource limits
+        if let Some(cpu_limit) = &self.cpu_limit {
+            args_strings.push("--cpus".to_string());
+            args_strings.push(cpu_limit.clone());
+        }
+
+        if let Some(memory_limit) = &self.memory_limit {
+            args_strings.push("--memory".to_string());
+            args_strings.push(memory_limit.clone());
+        }
+
+        if let Some(memory_swap_limit) = &self.memory_swap_limit {
+            args_strings.push("--memory-swap".to_string());
+            args_strings.push(memory_swap_limit.clone());
+        }
+
+        if let Some(cpu_shares) = &self.cpu_shares {
+            args_strings.push("--cpu-shares".to_string());
+            args_strings.push(cpu_shares.clone());
+        }
+
+        // Add restart policy
+        if let Some(restart_policy) = &self.restart_policy {
+            args_strings.push("--restart".to_string());
+            args_strings.push(restart_policy.clone());
+        }
+
+        // Add health check
+        if let Some(health_check) = &self.health_check {
+            args_strings.push("--health-cmd".to_string());
+            args_strings.push(health_check.cmd.clone());
+
+            if let Some(interval) = &health_check.interval {
+                args_strings.push("--health-interval".to_string());
+                args_strings.push(interval.clone());
+            }
+
+            if let Some(timeout) = &health_check.timeout {
+                args_strings.push("--health-timeout".to_string());
+                args_strings.push(timeout.clone());
+            }
+
+            if let Some(retries) = &health_check.retries {
+                args_strings.push("--health-retries".to_string());
+                args_strings.push(retries.to_string());
+            }
+
+            if let Some(start_period) = &health_check.start_period {
+                args_strings.push("--health-start-period".to_string());
+                args_strings.push(start_period.clone());
+            }
+        }
+
+        if let Some(snapshotter_value) = &self.snapshotter {
+            args_strings.push("--snapshotter".to_string());
+            args_strings.push(snapshotter_value.clone());
+        }
+
+        // Add flags to avoid BPF issues
+        args_strings.push("--cgroup-manager=cgroupfs".to_string());
+
+        args_strings.push(image.clone());
+
+        // Convert to string slices for the command
+        let args: Vec<&str> = args_strings.iter().map(|s| s.as_str()).collect();
+
+        // Execute the command
+        let result = execute_nerdctl_command(&args)?;
+
+        // Get the container ID from the output
+        let container_id = result.stdout.trim().to_string();
+
+        Ok(Self {
+            name: self.name,
+            container_id: Some(container_id),
+            image: self.image,
+            config: self.config,
+            ports: self.ports,
+            volumes: self.volumes,
+            env_vars: self.env_vars,
+            network: self.network,
+            network_aliases: self.network_aliases,
+            cpu_limit: self.cpu_limit,
+            memory_limit: self.memory_limit,
+            memory_swap_limit: self.memory_swap_limit,
+            cpu_shares: self.cpu_shares,
+            restart_policy: self.restart_policy,
+            health_check: self.health_check,
+            detach: self.detach,
+            snapshotter: self.snapshotter,
+        })
+    }
+}

virt/src/nerdctl/container_functions.rs

@@ -0,0 +1,141 @@
+// File: /root/code/git.threefold.info/herocode/sal/src/virt/nerdctl/container_functions.rs
+
+use crate::nerdctl::{execute_nerdctl_command, NerdctlError};
+use sal_process::CommandResult;
+
+/// Run a container from an image
+///
+/// # Arguments
+///
+/// * `image` - Image to run
+/// * `name` - Optional name for the container
+/// * `detach` - Whether to run in detached mode
+/// * `ports` - Optional port mappings
+/// * `snapshotter` - Optional snapshotter to use
+///
+/// # Returns
+///
+/// * `Result<CommandResult, NerdctlError>` - Command result or error
+pub fn run(
+    image: &str,
+    name: Option<&str>,
+    detach: bool,
+    ports: Option<&[&str]>,
+    snapshotter: Option<&str>,
+) -> Result<CommandResult, NerdctlError> {
+    let mut args = vec!["run"];
+
+    if detach {
+        args.push("-d");
+    }
+
+    if let Some(name_value) = name {
+        args.push("--name");
+        args.push(name_value);
+    }
+
+    if let Some(ports_value) = ports {
+        for port in ports_value {
+            args.push("-p");
+            args.push(port);
+        }
+    }
+
+    if let Some(snapshotter_value) = snapshotter {
+        args.push("--snapshotter");
+        args.push(snapshotter_value);
+    }
+
+    // Add flags to avoid BPF issues
+    args.push("--cgroup-manager=cgroupfs");
+
+    args.push(image);
+
+    execute_nerdctl_command(&args)
+}
+
+/// Execute a command in a container
+///
+/// # Arguments
+///
+/// * `container` - Container name or ID
+/// * `command` - Command to execute
+///
+/// # Returns
+///
+/// * `Result<CommandResult, NerdctlError>` - Command result or error
+pub fn exec(container: &str, command: &str) -> Result<CommandResult, NerdctlError> {
+    execute_nerdctl_command(&["exec", container, "sh", "-c", command])
+}
+
+/// Copy files between container and local filesystem
+///
+/// # Arguments
+///
+/// * `source` - Source path (can be container:path or local path)
+/// * `dest` - Destination path (can be container:path or local path)
+///
+/// # Returns
+///
+/// * `Result<CommandResult, NerdctlError>` - Command result or error
+pub fn copy(source: &str, dest: &str) -> Result<CommandResult, NerdctlError> {
+    execute_nerdctl_command(&["cp", source, dest])
+}
+
+/// Stop a container
+///
+/// # Arguments
+///
+/// * `container` - Container name or ID
+///
+/// # Returns
+///
+/// * `Result<CommandResult, NerdctlError>` - Command result or error
+pub fn stop(container: &str) -> Result<CommandResult, NerdctlError> {
+    execute_nerdctl_command(&["stop", container])
+}
+
+/// Remove a container
+///
+/// # Arguments
+///
+/// * `container` - Container name or ID
+///
+/// # Returns
+///
+/// * `Result<CommandResult, NerdctlError>` - Command result or error
+pub fn remove(container: &str) -> Result<CommandResult, NerdctlError> {
+    execute_nerdctl_command(&["rm", container])
+}
+
+/// List containers
+///
+/// # Arguments
+///
+/// * `all` - Whether to list all containers (including stopped ones)
+///
+/// # Returns
+///
+/// * `Result<CommandResult, NerdctlError>` - Command result or error
+pub fn list(all: bool) -> Result<CommandResult, NerdctlError> {
+    let mut args = vec!["ps"];
+
+    if all {
+        args.push("-a");
+    }
+
+    execute_nerdctl_command(&args)
+}
+
+/// Get container logs
+///
+/// # Arguments
+///
+/// * `container` - Container name or ID
+///
+/// # Returns
+///
+/// * `Result<CommandResult, NerdctlError>` - Command result or error
+pub fn logs(container: &str) -> Result<CommandResult, NerdctlError> {
+    execute_nerdctl_command(&["logs", container])
+}

virt/src/nerdctl/container_operations.rs

@@ -0,0 +1,509 @@
+// File: /root/code/git.threefold.info/herocode/sal/src/virt/nerdctl/container_operations.rs
+
+use super::container_types::{Container, ContainerStatus, ResourceUsage};
+use crate::nerdctl::{execute_nerdctl_command, NerdctlError};
+use sal_process::CommandResult;
+use serde_json;
+
+impl Container {
+    /// Start the container and verify it's running
+    /// If the container hasn't been created yet, it will be created automatically.
+    ///
+    /// # Returns
+    ///
+    /// * `Result<CommandResult, NerdctlError>` - Command result or error with detailed information
+    pub fn start(&self) -> Result<CommandResult, NerdctlError> {
+        // If container_id is None, we need to create the container first
+        let container = if self.container_id.is_none() {
+            // Check if we have an image specified
+            if self.image.is_none() {
+                return Err(NerdctlError::Other(
+                    "No image specified for container creation".to_string(),
+                ));
+            }
+
+            // Clone self and create the container
+            println!("Container not created yet. Creating container from image...");
+
+            // First, try to pull the image if it doesn't exist locally
+            let image = self.image.as_ref().unwrap();
+            match execute_nerdctl_command(&["image", "inspect", image]) {
+                Err(_) => {
+                    println!("Image '{}' not found locally. Pulling image...", image);
+                    if let Err(e) = execute_nerdctl_command(&["pull", image]) {
+                        return Err(NerdctlError::CommandFailed(format!(
+                            "Failed to pull image '{}': {}",
+                            image, e
+                        )));
+                    }
+                    println!("Image '{}' pulled successfully.", image);
+                }
+                Ok(_) => {
+                    println!("Image '{}' found locally.", image);
+                }
+            }
+
+            // Now create the container
+            match self.clone().build() {
+                Ok(built) => built,
+                Err(e) => {
+                    return Err(NerdctlError::CommandFailed(format!(
+                        "Failed to create container from image '{}': {}",
+                        image, e
+                    )));
+                }
+            }
+        } else {
+            // Container already has an ID, use it as is
+            self.clone()
+        };
+
+        if let Some(container_id) = &container.container_id {
+            // First, try to start the container
+            let start_result = execute_nerdctl_command(&["start", container_id]);
+
+            // If the start command failed, return the error with details
+            if let Err(err) = &start_result {
+                return Err(NerdctlError::CommandFailed(format!(
+                    "Failed to start container {}: {}",
+                    container_id, err
+                )));
+            }
+
+            // Verify the container is actually running
+            match container.verify_running() {
+                Ok(true) => start_result,
+                Ok(false) => {
+                    // Container started but isn't running - get detailed information
+                    let mut error_message =
+                        format!("Container {} started but is not running.", container_id);
+
+                    // Get container status
+                    if let Ok(status) = container.status() {
+                        error_message.push_str(&format!(
+                            "\nStatus: {}, State: {}, Health: {}",
+                            status.status,
+                            status.state,
+                            status.health_status.unwrap_or_else(|| "N/A".to_string())
+                        ));
+                    }
+
+                    // Get container logs
+                    if let Ok(logs) = execute_nerdctl_command(&["logs", container_id]) {
+                        if !logs.stdout.trim().is_empty() {
+                            error_message.push_str(&format!(
+                                "\nContainer logs (stdout):\n{}",
+                                logs.stdout.trim()
+                            ));
+                        }
+                        if !logs.stderr.trim().is_empty() {
+                            error_message.push_str(&format!(
+                                "\nContainer logs (stderr):\n{}",
+                                logs.stderr.trim()
+                            ));
+                        }
+                    }
+
+                    // Get container exit code if available
+                    if let Ok(inspect_result) = execute_nerdctl_command(&[
+                        "inspect",
+                        "--format",
+                        "{{.State.ExitCode}}",
+                        container_id,
+                    ]) {
+                        let exit_code = inspect_result.stdout.trim();
+                        if !exit_code.is_empty() && exit_code != "0" {
+                            error_message
+                                .push_str(&format!("\nContainer exit code: {}", exit_code));
+                        }
+                    }
+
+                    Err(NerdctlError::CommandFailed(error_message))
+                }
+                Err(err) => {
+                    // Failed to verify if container is running
+                    Err(NerdctlError::CommandFailed(format!(
+                        "Container {} may have started, but verification failed: {}",
+                        container_id, err
+                    )))
+                }
+            }
+        } else {
+            Err(NerdctlError::Other(
+                "Failed to create container. No container ID available.".to_string(),
+            ))
+        }
+    }
+
+    /// Verify if the container is running
+    ///
+    /// # Returns
+    ///
+    /// * `Result<bool, NerdctlError>` - True if running, false if not running, error if verification failed
+    fn verify_running(&self) -> Result<bool, NerdctlError> {
+        if let Some(container_id) = &self.container_id {
+            // Use inspect to check if the container is running
+            let inspect_result = execute_nerdctl_command(&[
+                "inspect",
+                "--format",
+                "{{.State.Running}}",
+                container_id,
+            ]);
+
+            match inspect_result {
+                Ok(result) => {
+                    let running = result.stdout.trim().to_lowercase() == "true";
+                    Ok(running)
+                }
+                Err(err) => Err(err),
+            }
+        } else {
+            Err(NerdctlError::Other("No container ID available".to_string()))
+        }
+    }
+
+    /// Stop the container
+    ///
+    /// # Returns
+    ///
+    /// * `Result<CommandResult, NerdctlError>` - Command result or error
+    pub fn stop(&self) -> Result<CommandResult, NerdctlError> {
+        if let Some(container_id) = &self.container_id {
+            execute_nerdctl_command(&["stop", container_id])
+        } else {
+            Err(NerdctlError::Other("No container ID available".to_string()))
+        }
+    }
+
+    /// Remove the container
+    ///
+    /// # Returns
+    ///
+    /// * `Result<CommandResult, NerdctlError>` - Command result or error
+    pub fn remove(&self) -> Result<CommandResult, NerdctlError> {
+        if let Some(container_id) = &self.container_id {
+            execute_nerdctl_command(&["rm", container_id])
+        } else {
+            Err(NerdctlError::Other("No container ID available".to_string()))
+        }
+    }
+
+    /// Execute a command in the container
+    ///
+    /// # Arguments
+    ///
+    /// * `command` - The command to run
+    ///
+    /// # Returns
+    ///
+    /// * `Result<CommandResult, NerdctlError>` - Command result or error
+    pub fn exec(&self, command: &str) -> Result<CommandResult, NerdctlError> {
+        if let Some(container_id) = &self.container_id {
+            execute_nerdctl_command(&["exec", container_id, "sh", "-c", command])
+        } else {
+            Err(NerdctlError::Other("No container ID available".to_string()))
+        }
+    }
+
+    /// Copy files between container and local filesystem
+    ///
+    /// # Arguments
+    ///
+    /// * `source` - Source path (can be container:path or local path)
+    /// * `dest` - Destination path (can be container:path or local path)
+    ///
+    /// # Returns
+    ///
+    /// * `Result<CommandResult, NerdctlError>` - Command result or error
+    pub fn copy(&self, source: &str, dest: &str) -> Result<CommandResult, NerdctlError> {
+        if self.container_id.is_some() {
+            execute_nerdctl_command(&["cp", source, dest])
+        } else {
+            Err(NerdctlError::Other("No container ID available".to_string()))
+        }
+    }
+
+    /// Export the container to a tarball
+    ///
+    /// # Arguments
+    ///
+    /// * `path` - Path to save the tarball
+    ///
+    /// # Returns
+    ///
+    /// * `Result<CommandResult, NerdctlError>` - Command result or error
+    pub fn export(&self, path: &str) -> Result<CommandResult, NerdctlError> {
+        if let Some(container_id) = &self.container_id {
+            execute_nerdctl_command(&["export", "-o", path, container_id])
+        } else {
+            Err(NerdctlError::Other("No container ID available".to_string()))
+        }
+    }
+
+    /// Commit the container to an image
+    ///
+    /// # Arguments
+    ///
+    /// * `image_name` - Name for the new image
+    ///
+    /// # Returns
+    ///
+    /// * `Result<CommandResult, NerdctlError>` - Command result or error
+    pub fn commit(&self, image_name: &str) -> Result<CommandResult, NerdctlError> {
+        if let Some(container_id) = &self.container_id {
+            execute_nerdctl_command(&["commit", container_id, image_name])
+        } else {
+            Err(NerdctlError::Other("No container ID available".to_string()))
+        }
+    }
+
+    /// Get container status
+    ///
+    /// # Returns
+    ///
+    /// * `Result<ContainerStatus, NerdctlError>` - Container status or error
+    pub fn status(&self) -> Result<ContainerStatus, NerdctlError> {
+        if let Some(container_id) = &self.container_id {
+            let result = execute_nerdctl_command(&["inspect", container_id])?;
+
+            // Parse the JSON output
+            match serde_json::from_str::<serde_json::Value>(&result.stdout) {
+                Ok(json) => {
+                    if let Some(container_json) = json.as_array().and_then(|arr| arr.first()) {
+                        let state = container_json
+                            .get("State")
+                            .and_then(|state| state.get("Status"))
+                            .and_then(|status| status.as_str())
+                            .unwrap_or("unknown")
+                            .to_string();
+
+                        let status = container_json
+                            .get("State")
+                            .and_then(|state| state.get("Running"))
+                            .and_then(|running| {
+                                if running.as_bool().unwrap_or(false) {
+                                    Some("running")
+                                } else {
+                                    Some("stopped")
+                                }
+                            })
+                            .unwrap_or("unknown")
+                            .to_string();
+
+                        let created = container_json
+                            .get("Created")
+                            .and_then(|created| created.as_str())
+                            .unwrap_or("unknown")
+                            .to_string();
+
+                        let started = container_json
+                            .get("State")
+                            .and_then(|state| state.get("StartedAt"))
+                            .and_then(|started| started.as_str())
+                            .unwrap_or("unknown")
+                            .to_string();
+
+                        // Get health status if available
+                        let health_status = container_json
+                            .get("State")
+                            .and_then(|state| state.get("Health"))
+                            .and_then(|health| health.get("Status"))
+                            .and_then(|status| status.as_str())
+                            .map(|s| s.to_string());
+
+                        // Get health check output if available
+                        let health_output = container_json
+                            .get("State")
+                            .and_then(|state| state.get("Health"))
+                            .and_then(|health| health.get("Log"))
+                            .and_then(|log| log.as_array())
+                            .and_then(|log_array| log_array.last())
+                            .and_then(|last_log| last_log.get("Output"))
+                            .and_then(|output| output.as_str())
+                            .map(|s| s.to_string());
+
+                        Ok(ContainerStatus {
+                            state,
+                            status,
+                            created,
+                            started,
+                            health_status,
+                            health_output,
+                        })
+                    } else {
+                        Err(NerdctlError::JsonParseError(
+                            "Invalid container inspect JSON".to_string(),
+                        ))
+                    }
+                }
+                Err(e) => Err(NerdctlError::JsonParseError(format!(
+                    "Failed to parse container inspect JSON: {}",
+                    e
+                ))),
+            }
+        } else {
+            Err(NerdctlError::Other("No container ID available".to_string()))
+        }
+    }
+
+    /// Get the health status of the container
+    ///
+    /// # Returns
+    ///
+    /// * `Result<String, NerdctlError>` - Health status or error
+    pub fn health_status(&self) -> Result<String, NerdctlError> {
+        if let Some(container_id) = &self.container_id {
+            let result = execute_nerdctl_command(&[
+                "inspect",
+                "--format",
+                "{{.State.Health.Status}}",
+                container_id,
+            ])?;
+            Ok(result.stdout.trim().to_string())
+        } else {
+            Err(NerdctlError::Other("No container ID available".to_string()))
+        }
+    }
+
+    /// Get container logs
+    ///
+    /// # Returns
+    ///
+    /// * `Result<CommandResult, NerdctlError>` - Command result or error
+    pub fn logs(&self) -> Result<CommandResult, NerdctlError> {
+        if let Some(container_id) = &self.container_id {
+            execute_nerdctl_command(&["logs", container_id])
+        } else {
+            Err(NerdctlError::Other("No container ID available".to_string()))
+        }
+    }
+
+    /// Get container resource usage
+    ///
+    /// # Returns
+    ///
+    /// * `Result<ResourceUsage, NerdctlError>` - Resource usage or error
+    pub fn resources(&self) -> Result<ResourceUsage, NerdctlError> {
+        if let Some(container_id) = &self.container_id {
+            let result = execute_nerdctl_command(&["stats", "--no-stream", container_id])?;
+
+            // Parse the output
+            let lines: Vec<&str> = result.stdout.lines().collect();
+            if lines.len() >= 2 {
+                let headers = lines[0];
+                let values = lines[1];
+
+                let headers_vec: Vec<&str> = headers.split_whitespace().collect();
+                let values_vec: Vec<&str> = values.split_whitespace().collect();
+
+                // Find indices for each metric
+                let cpu_index = headers_vec
+                    .iter()
+                    .position(|&h| h.contains("CPU"))
+                    .unwrap_or(0);
+                let mem_index = headers_vec
+                    .iter()
+                    .position(|&h| h.contains("MEM"))
+                    .unwrap_or(0);
+                let mem_perc_index = headers_vec
+                    .iter()
+                    .position(|&h| h.contains("MEM%"))
+                    .unwrap_or(0);
+                let net_in_index = headers_vec
+                    .iter()
+                    .position(|&h| h.contains("NET"))
+                    .unwrap_or(0);
+                let net_out_index = if net_in_index > 0 {
+                    net_in_index + 1
+                } else {
+                    0
+                };
+                let block_in_index = headers_vec
+                    .iter()
+                    .position(|&h| h.contains("BLOCK"))
+                    .unwrap_or(0);
+                let block_out_index = if block_in_index > 0 {
+                    block_in_index + 1
+                } else {
+                    0
+                };
+                let pids_index = headers_vec
+                    .iter()
+                    .position(|&h| h.contains("PIDS"))
+                    .unwrap_or(0);
+
+                let cpu_usage = if cpu_index < values_vec.len() {
+                    values_vec[cpu_index].to_string()
+                } else {
+                    "unknown".to_string()
+                };
+
+                let memory_usage = if mem_index < values_vec.len() {
+                    values_vec[mem_index].to_string()
+                } else {
+                    "unknown".to_string()
+                };
+
+                let memory_limit = if mem_index + 1 < values_vec.len() {
+                    values_vec[mem_index + 1].to_string()
+                } else {
+                    "unknown".to_string()
+                };
+
+                let memory_percentage = if mem_perc_index < values_vec.len() {
+                    values_vec[mem_perc_index].to_string()
+                } else {
+                    "unknown".to_string()
+                };
+
+                let network_input = if net_in_index < values_vec.len() {
+                    values_vec[net_in_index].to_string()
+                } else {
+                    "unknown".to_string()
+                };
+
+                let network_output = if net_out_index < values_vec.len() {
+                    values_vec[net_out_index].to_string()
+                } else {
+                    "unknown".to_string()
+                };
+
+                let block_input = if block_in_index < values_vec.len() {
+                    values_vec[block_in_index].to_string()
+                } else {
+                    "unknown".to_string()
+                };
+
+                let block_output = if block_out_index < values_vec.len() {
+                    values_vec[block_out_index].to_string()
+                } else {
+                    "unknown".to_string()
+                };
+
+                let pids = if pids_index < values_vec.len() {
+                    values_vec[pids_index].to_string()
+                } else {
+                    "unknown".to_string()
+                };
+
+                Ok(ResourceUsage {
+                    cpu_usage,
+                    memory_usage,
+                    memory_limit,
+                    memory_percentage,
+                    network_input,
+                    network_output,
+                    block_input,
+                    block_output,
+                    pids,
+                })
+            } else {
+                Err(NerdctlError::ConversionError(
+                    "Failed to parse stats output".to_string(),
+                ))
+            }
+        } else {
+            Err(NerdctlError::Other("No container ID available".to_string()))
+        }
+    }
+}

virt/src/nerdctl/container_test.rs

@@ -0,0 +1,307 @@
+// File: /root/code/git.threefold.info/herocode/sal/src/virt/nerdctl/container_test.rs
+
+#[cfg(test)]
+mod tests {
+    use super::super::container_types::Container;
+    use std::process::Command;
+    use std::thread;
+    use std::time::Duration;
+
+    // Helper function to check if nerdctl is available
+    fn is_nerdctl_available() -> bool {
+        match Command::new("which").arg("nerdctl").output() {
+            Ok(output) => output.status.success(),
+            Err(_) => false,
+        }
+    }
+
+    #[test]
+    fn test_container_builder_pattern() {
+        // Skip test if nerdctl is not available
+        if !is_nerdctl_available() {
+            println!("Skipping test: nerdctl is not available");
+            return;
+        }
+
+        // Create a container with builder pattern
+        let container = Container::new("test-container")
+            .unwrap()
+            .with_port("8080:80")
+            .with_volume("/tmp:/data")
+            .with_env("TEST_ENV", "test_value")
+            .with_detach(true);
+
+        // Verify container properties
+        assert_eq!(container.name, "test-container");
+        assert_eq!(container.ports.len(), 1);
+        assert_eq!(container.ports[0], "8080:80");
+        assert_eq!(container.volumes.len(), 1);
+        assert_eq!(container.volumes[0], "/tmp:/data");
+        assert_eq!(container.env_vars.len(), 1);
+        assert_eq!(container.env_vars.get("TEST_ENV").unwrap(), "test_value");
+        assert_eq!(container.detach, true);
+    }
+
+    #[test]
+    fn test_container_from_image() {
+        // Skip test if nerdctl is not available
+        if !is_nerdctl_available() {
+            println!("Skipping test: nerdctl is not available");
+            return;
+        }
+
+        // Create a container from image
+        let container = Container::from_image("test-container", "alpine:latest").unwrap();
+
+        // Verify container properties
+        assert_eq!(container.name, "test-container");
+        assert_eq!(container.image.as_ref().unwrap(), "alpine:latest");
+    }
+
+    #[test]
+    fn test_container_health_check() {
+        // Skip test if nerdctl is not available
+        if !is_nerdctl_available() {
+            println!("Skipping test: nerdctl is not available");
+            return;
+        }
+
+        // Create a container with health check
+        let container = Container::new("test-container")
+            .unwrap()
+            .with_health_check("curl -f http://localhost/ || exit 1");
+
+        // Verify health check
+        assert!(container.health_check.is_some());
+        let health_check = container.health_check.unwrap();
+        assert_eq!(health_check.cmd, "curl -f http://localhost/ || exit 1");
+        assert!(health_check.interval.is_none());
+        assert!(health_check.timeout.is_none());
+        assert!(health_check.retries.is_none());
+        assert!(health_check.start_period.is_none());
+    }
+
+    #[test]
+    fn test_container_health_check_options() {
+        // Skip test if nerdctl is not available
+        if !is_nerdctl_available() {
+            println!("Skipping test: nerdctl is not available");
+            return;
+        }
+
+        // Create a container with health check options
+        let container = Container::new("test-container")
+            .unwrap()
+            .with_health_check_options(
+                "curl -f http://localhost/ || exit 1",
+                Some("30s"),
+                Some("10s"),
+                Some(3),
+                Some("5s"),
+            );
+
+        // Verify health check options
+        assert!(container.health_check.is_some());
+        let health_check = container.health_check.unwrap();
+        assert_eq!(health_check.cmd, "curl -f http://localhost/ || exit 1");
+        assert_eq!(health_check.interval.as_ref().unwrap(), "30s");
+        assert_eq!(health_check.timeout.as_ref().unwrap(), "10s");
+        assert_eq!(health_check.retries.unwrap(), 3);
+        assert_eq!(health_check.start_period.as_ref().unwrap(), "5s");
+    }
+
+    #[test]
+    #[ignore] // Ignore by default as it requires nerdctl to be installed and running
+    fn test_container_runtime_and_resources() {
+        // Check if nerdctl is available and properly configured
+        let nerdctl_check = super::super::execute_nerdctl_command(&["info"]);
+        if nerdctl_check.is_err() {
+            println!("Skipping test: nerdctl is not available or properly configured");
+            println!("Error: {:?}", nerdctl_check.err());
+            return;
+        }
+
+        // Create a unique container name for this test
+        let container_name = format!(
+            "test-runtime-{}",
+            std::time::SystemTime::now()
+                .duration_since(std::time::UNIX_EPOCH)
+                .unwrap()
+                .as_secs()
+        );
+
+        // Create and build a container that will use resources
+        // Use a simple container with a basic command to avoid dependency on external images
+        let container_result = Container::from_image(&container_name, "busybox:latest")
+            .unwrap()
+            .with_detach(true)
+            .build();
+
+        // Check if the build was successful
+        if container_result.is_err() {
+            println!("Failed to build container: {:?}", container_result.err());
+            return;
+        }
+
+        let container = container_result.unwrap();
+        println!("Container created successfully: {}", container_name);
+
+        // Start the container with a simple command
+        let start_result =
+            container.exec("sh -c 'for i in $(seq 1 10); do echo $i; sleep 1; done'");
+        if start_result.is_err() {
+            println!("Failed to start container: {:?}", start_result.err());
+            // Try to clean up
+            let _ = container.remove();
+            return;
+        }
+
+        println!("Container started successfully");
+
+        // Wait for the container to start and consume resources
+        thread::sleep(Duration::from_secs(3));
+
+        // Check container status
+        let status_result = container.status();
+        if status_result.is_err() {
+            println!("Failed to get container status: {:?}", status_result.err());
+            // Try to clean up
+            let _ = container.stop();
+            let _ = container.remove();
+            return;
+        }
+
+        let status = status_result.unwrap();
+        println!("Container status: {:?}", status);
+
+        // Verify the container is running
+        if status.status != "running" {
+            println!("Container is not running, status: {}", status.status);
+            // Try to clean up
+            let _ = container.remove();
+            return;
+        }
+
+        // Check resource usage
+        let resources_result = container.resources();
+        if resources_result.is_err() {
+            println!("Failed to get resource usage: {:?}", resources_result.err());
+            // Try to clean up
+            let _ = container.stop();
+            let _ = container.remove();
+            return;
+        }
+
+        let resources = resources_result.unwrap();
+        println!("Container resources: {:?}", resources);
+
+        // Verify the container is using memory (if we can get the information)
+        if resources.memory_usage == "0B" || resources.memory_usage == "unknown" {
+            println!(
+                "Warning: Container memory usage is {}",
+                resources.memory_usage
+            );
+        } else {
+            println!("Container is using memory: {}", resources.memory_usage);
+        }
+
+        // Clean up - stop and remove the container
+        println!("Stopping container...");
+        let stop_result = container.stop();
+        if stop_result.is_err() {
+            println!("Warning: Failed to stop container: {:?}", stop_result.err());
+        }
+
+        println!("Removing container...");
+        let remove_result = container.remove();
+        if remove_result.is_err() {
+            println!(
+                "Warning: Failed to remove container: {:?}",
+                remove_result.err()
+            );
+        }
+
+        println!("Test completed successfully");
+    }
+
+    #[test]
+    fn test_container_with_custom_command() {
+        // Skip test if nerdctl is not available
+        if !is_nerdctl_available() {
+            println!("Skipping test: nerdctl is not available");
+            return;
+        }
+
+        // Create a container with a custom command
+        let container = Container::new("test-command-container")
+            .unwrap()
+            .with_port("8080:80")
+            .with_volume("/tmp:/data")
+            .with_env("TEST_ENV", "test_value")
+            .with_detach(true);
+
+        // Verify container properties
+        assert_eq!(container.name, "test-command-container");
+        assert_eq!(container.ports.len(), 1);
+        assert_eq!(container.ports[0], "8080:80");
+        assert_eq!(container.volumes.len(), 1);
+        assert_eq!(container.volumes[0], "/tmp:/data");
+        assert_eq!(container.env_vars.len(), 1);
+        assert_eq!(container.env_vars.get("TEST_ENV").unwrap(), "test_value");
+        assert_eq!(container.detach, true);
+
+        // Convert the container to a command string that would be used to run it
+        let command_args = container_to_command_args(&container);
+
+        // Verify the command arguments contain all the expected options
+        assert!(command_args.contains(&"--name".to_string()));
+        assert!(command_args.contains(&"test-command-container".to_string()));
+        assert!(command_args.contains(&"-p".to_string()));
+        assert!(command_args.contains(&"8080:80".to_string()));
+        assert!(command_args.contains(&"-v".to_string()));
+        assert!(command_args.contains(&"/tmp:/data".to_string()));
+        assert!(command_args.contains(&"-e".to_string()));
+        assert!(command_args.contains(&"TEST_ENV=test_value".to_string()));
+        assert!(command_args.contains(&"-d".to_string()));
+
+        println!("Command args: {:?}", command_args);
+    }
+
+    // Helper function to convert a container to command arguments
+    fn container_to_command_args(container: &Container) -> Vec<String> {
+        let mut args = Vec::new();
+        args.push("run".to_string());
+
+        if container.detach {
+            args.push("-d".to_string());
+        }
+
+        args.push("--name".to_string());
+        args.push(container.name.clone());
+
+        // Add port mappings
+        for port in &container.ports {
+            args.push("-p".to_string());
+            args.push(port.clone());
+        }
+
+        // Add volume mounts
+        for volume in &container.volumes {
+            args.push("-v".to_string());
+            args.push(volume.clone());
+        }
+
+        // Add environment variables
+        for (key, value) in &container.env_vars {
+            args.push("-e".to_string());
+            args.push(format!("{}={}", key, value));
+        }
+
+        // Add image if available
+        if let Some(image) = &container.image {
+            args.push(image.clone());
+        }
+
+        args
+    }
+}

virt/src/nerdctl/container_types.rs

@@ -0,0 +1,97 @@
+// File: /root/code/git.threefold.info/herocode/sal/src/virt/nerdctl/container_types.rs
+
+use std::collections::HashMap;
+
+/// Container struct for nerdctl operations
+#[derive(Clone)]
+pub struct Container {
+    /// Name of the container
+    pub name: String,
+    /// Container ID
+    pub container_id: Option<String>,
+    /// Base image (if created from an image)
+    pub image: Option<String>,
+    /// Configuration options
+    pub config: HashMap<String, String>,
+    /// Port mappings
+    pub ports: Vec<String>,
+    /// Volume mounts
+    pub volumes: Vec<String>,
+    /// Environment variables
+    pub env_vars: HashMap<String, String>,
+    /// Network to connect to
+    pub network: Option<String>,
+    /// Network aliases
+    pub network_aliases: Vec<String>,
+    /// CPU limit
+    pub cpu_limit: Option<String>,
+    /// Memory limit
+    pub memory_limit: Option<String>,
+    /// Memory swap limit
+    pub memory_swap_limit: Option<String>,
+    /// CPU shares
+    pub cpu_shares: Option<String>,
+    /// Restart policy
+    pub restart_policy: Option<String>,
+    /// Health check
+    pub health_check: Option<HealthCheck>,
+    /// Whether to run in detached mode
+    pub detach: bool,
+    /// Snapshotter to use
+    pub snapshotter: Option<String>,
+}
+
+/// Health check configuration for a container
+#[derive(Debug, Clone)]
+pub struct HealthCheck {
+    /// Command to run for health check
+    pub cmd: String,
+    /// Time between running the check (default: 30s)
+    pub interval: Option<String>,
+    /// Maximum time to wait for a check to complete (default: 30s)
+    pub timeout: Option<String>,
+    /// Number of consecutive failures needed to consider unhealthy (default: 3)
+    pub retries: Option<u32>,
+    /// Start period for the container to initialize before counting retries (default: 0s)
+    pub start_period: Option<String>,
+}
+
+/// Container status information
+#[derive(Debug, Clone)]
+pub struct ContainerStatus {
+    /// Container state (e.g., running, stopped)
+    pub state: String,
+    /// Container status
+    pub status: String,
+    /// Creation time
+    pub created: String,
+    /// Start time
+    pub started: String,
+    /// Health status (if health check is configured)
+    pub health_status: Option<String>,
+    /// Health check output (if health check is configured)
+    pub health_output: Option<String>,
+}
+
+/// Container resource usage information
+#[derive(Debug, Clone)]
+pub struct ResourceUsage {
+    /// CPU usage percentage
+    pub cpu_usage: String,
+    /// Memory usage
+    pub memory_usage: String,
+    /// Memory limit
+    pub memory_limit: String,
+    /// Memory usage percentage
+    pub memory_percentage: String,
+    /// Network input
+    pub network_input: String,
+    /// Network output
+    pub network_output: String,
+    /// Block input
+    pub block_input: String,
+    /// Block output
+    pub block_output: String,
+    /// PIDs
+    pub pids: String,
+}

virt/src/nerdctl/health_check.rs

@@ -0,0 +1,40 @@
+// File: /root/code/git.threefold.info/herocode/sal/src/virt/nerdctl/health_check.rs
+
+use super::container_types::HealthCheck;
+
+impl HealthCheck {
+    /// Create a new health check with the given command
+    pub fn new(cmd: &str) -> Self {
+        Self {
+            cmd: cmd.to_string(),
+            interval: None,
+            timeout: None,
+            retries: None,
+            start_period: None,
+        }
+    }
+    
+    /// Set the interval between health checks
+    pub fn with_interval(mut self, interval: &str) -> Self {
+        self.interval = Some(interval.to_string());
+        self
+    }
+    
+    /// Set the timeout for health checks
+    pub fn with_timeout(mut self, timeout: &str) -> Self {
+        self.timeout = Some(timeout.to_string());
+        self
+    }
+    
+    /// Set the number of retries for health checks
+    pub fn with_retries(mut self, retries: u32) -> Self {
+        self.retries = Some(retries);
+        self
+    }
+    
+    /// Set the start period for health checks
+    pub fn with_start_period(mut self, start_period: &str) -> Self {
+        self.start_period = Some(start_period.to_string());
+        self
+    }
+}

virt/src/nerdctl/health_check_script.rs

@@ -0,0 +1,79 @@
+// File: /root/code/git.threefold.info/herocode/sal/src/virt/nerdctl/health_check_script.rs
+
+use std::fs;
+use std::path::Path;
+use std::os::unix::fs::PermissionsExt;
+
+/// Handles health check scripts for containers
+/// 
+/// This module provides functionality to create and manage health check scripts
+/// for containers, allowing for more complex health checks than simple commands.
+
+/// Converts a health check command or script to a usable command
+/// 
+/// If the input is a single-line command, it is returned as is.
+/// If the input is a multi-line script, it is written to a file in the
+/// /root/hero/var/containers directory and the path to that file is returned.
+/// 
+/// # Arguments
+/// 
+/// * `cmd` - The command or script to convert
+/// * `container_name` - The name of the container, used to create a unique script name
+/// 
+/// # Returns
+/// 
+/// * `String` - The command to use for the health check
+pub fn prepare_health_check_command(cmd: &str, container_name: &str) -> String {
+    // If the command is a multiline script, write it to a file
+    if cmd.contains("\n") {
+        // Create the directory if it doesn't exist
+        let dir_path = "/root/hero/var/containers";
+        if let Err(_) = fs::create_dir_all(dir_path) {
+            // If we can't create the directory, just use the command as is
+            return cmd.to_string();
+        }
+        
+        // Create a unique filename based on container name
+        let script_path = format!("{}/healthcheck_{}.sh", dir_path, container_name);
+        
+        // Write the script to the file
+        if let Err(_) = fs::write(&script_path, cmd) {
+            // If we can't write the file, just use the command as is
+            return cmd.to_string();
+        }
+        
+        // Make the script executable
+        if let Ok(metadata) = fs::metadata(&script_path) {
+            let mut perms = metadata.permissions();
+            perms.set_mode(0o755);
+            if let Err(_) = fs::set_permissions(&script_path, perms) {
+                // If we can't set permissions, just use the script path with sh
+                return format!("sh {}", script_path);
+            }
+        } else {
+            // If we can't get metadata, just use the script path with sh
+            return format!("sh {}", script_path);
+        }
+        
+        // Use the script path as the command
+        script_path
+    } else {
+        // If it's a single line command, use it as is
+        cmd.to_string()
+    }
+}
+
+/// Cleans up health check scripts for a container
+/// 
+/// # Arguments
+/// 
+/// * `container_name` - The name of the container whose health check scripts should be cleaned up
+pub fn cleanup_health_check_scripts(container_name: &str) {
+    let dir_path = "/root/hero/var/containers";
+    let script_path = format!("{}/healthcheck_{}.sh", dir_path, container_name);
+    
+    // Try to remove the script file if it exists
+    if Path::new(&script_path).exists() {
+        let _ = fs::remove_file(script_path);
+    }
+}

virt/src/nerdctl/images.rs

@@ -0,0 +1,84 @@
+// File: /root/code/git.threefold.info/herocode/sal/src/virt/nerdctl/images.rs
+
+use super::NerdctlError;
+use crate::nerdctl::execute_nerdctl_command;
+use sal_process::CommandResult;
+use serde::{Deserialize, Serialize};
+
+/// Represents a container image
+#[derive(Debug, Clone, Serialize, Deserialize)]
+pub struct Image {
+    /// Image ID
+    pub id: String,
+    /// Image repository
+    pub repository: String,
+    /// Image tag
+    pub tag: String,
+    /// Image size
+    pub size: String,
+    /// Creation timestamp
+    pub created: String,
+}
+
+/// List images in local storage
+pub fn images() -> Result<CommandResult, NerdctlError> {
+    execute_nerdctl_command(&["images"])
+}
+
+/// Remove one or more images
+///
+/// # Arguments
+///
+/// * `image` - Image ID or name
+pub fn image_remove(image: &str) -> Result<CommandResult, NerdctlError> {
+    execute_nerdctl_command(&["rmi", image])
+}
+
+/// Push an image to a registry
+///
+/// # Arguments
+///
+/// * `image` - Image name
+/// * `destination` - Destination registry URL
+pub fn image_push(image: &str, destination: &str) -> Result<CommandResult, NerdctlError> {
+    execute_nerdctl_command(&["push", image, destination])
+}
+
+/// Add an additional name to a local image
+///
+/// # Arguments
+///
+/// * `image` - Image ID or name
+/// * `new_name` - New name for the image
+pub fn image_tag(image: &str, new_name: &str) -> Result<CommandResult, NerdctlError> {
+    execute_nerdctl_command(&["tag", image, new_name])
+}
+
+/// Pull an image from a registry
+///
+/// # Arguments
+///
+/// * `image` - Image name
+pub fn image_pull(image: &str) -> Result<CommandResult, NerdctlError> {
+    execute_nerdctl_command(&["pull", image])
+}
+
+/// Commit a container to an image
+///
+/// # Arguments
+///
+/// * `container` - Container ID or name
+/// * `image_name` - New name for the image
+pub fn image_commit(container: &str, image_name: &str) -> Result<CommandResult, NerdctlError> {
+    execute_nerdctl_command(&["commit", container, image_name])
+}
+
+/// Build an image using a Dockerfile
+///
+/// # Arguments
+///
+/// * `tag` - Tag for the new image
+/// * `context_path` - Path to the build context
+pub fn image_build(tag: &str, context_path: &str) -> Result<CommandResult, NerdctlError> {
+    execute_nerdctl_command(&["build", "-t", tag, context_path])
+}

virt/src/nerdctl/mod.rs

@@ -0,0 +1,57 @@
+mod images;
+mod cmd;
+mod container_types;
+mod container;
+mod container_builder;
+mod health_check;
+mod health_check_script;
+mod container_operations;
+mod container_functions;
+#[cfg(test)]
+mod container_test;
+
+use std::fmt;
+use std::error::Error;
+use std::io;
+
+/// Error type for nerdctl operations
+#[derive(Debug)]
+pub enum NerdctlError {
+    /// The nerdctl command failed to execute
+    CommandExecutionFailed(io::Error),
+    /// The nerdctl command executed but returned an error
+    CommandFailed(String),
+    /// Failed to parse JSON output
+    JsonParseError(String),
+    /// Failed to convert data
+    ConversionError(String),
+    /// Generic error
+    Other(String),
+}
+
+impl fmt::Display for NerdctlError {
+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+        match self {
+            NerdctlError::CommandExecutionFailed(e) => write!(f, "Failed to execute nerdctl command: {}", e),
+            NerdctlError::CommandFailed(e) => write!(f, "Nerdctl command failed: {}", e),
+            NerdctlError::JsonParseError(e) => write!(f, "Failed to parse JSON: {}", e),
+            NerdctlError::ConversionError(e) => write!(f, "Conversion error: {}", e),
+            NerdctlError::Other(e) => write!(f, "{}", e),
+        }
+    }
+}
+
+impl Error for NerdctlError {
+    fn source(&self) -> Option<&(dyn Error + 'static)> {
+        match self {
+            NerdctlError::CommandExecutionFailed(e) => Some(e),
+            _ => None,
+        }
+    }
+}
+
+pub use images::*;
+pub use cmd::*;
+pub use container_types::{Container, HealthCheck, ContainerStatus, ResourceUsage};
+pub use container_functions::*;
+pub use health_check_script::*;

virt/src/nerdctl/nerdctl-essentials.md

@@ -0,0 +1,503 @@
+# nerdctl Essentials
+
+This guide provides a comprehensive overview of essential nerdctl functionality to help you get started quickly. nerdctl is a Docker-compatible CLI for containerd, with additional features specifically designed for containerd environments.
+
+## Introduction
+
+nerdctl is a Docker-compatible CLI for containerd. It provides the same user experience as the Docker CLI (`docker`) but leverages the more efficient containerd container runtime. Key differences and advantages include:
+
+- Direct integration with containerd (no extra daemon required)
+- Support for containerd-specific features
+- First-class support for rootless mode
+- Compatibility with Docker commands
+- Additional nerdctl-specific commands
+
+## Basic Configuration
+
+nerdctl can be configured using the `nerdctl.toml` configuration file:
+
+- Rootful mode: `/etc/nerdctl/nerdctl.toml`
+- Rootless mode: `~/.config/nerdctl/nerdctl.toml`
+
+Example configuration:
+
+```toml
+debug = false
+debug_full = false
+address = "unix:///run/containerd/containerd.sock"
+namespace = "default"
+snapshotter = "overlayfs"
+cgroup_manager = "systemd"
+hosts_dir = ["/etc/containerd/certs.d", "/etc/docker/certs.d"]
+```
+
+Common configuration properties:
+
+| Property            | CLI Flag                          | Description                |
+|---------------------|-----------------------------------|----------------------------|
+| `address`           | `--address`, `--host`, `-a`, `-H` | containerd address         |
+| `namespace`         | `--namespace`, `-n`               | containerd namespace       |
+| `snapshotter`       | `--snapshotter`                   | containerd snapshotter     |
+| `cni_path`          | `--cni-path`                      | CNI binary directory       |
+| `data_root`         | `--data-root`                     | Persistent state directory |
+| `insecure_registry` | `--insecure-registry`             | Allow insecure registry    |
+
+## Container Management
+
+### Running Containers
+
+**Run a container**:
+```
+nerdctl run [OPTIONS] IMAGE [COMMAND] [ARG...]
+```
+
+Common options:
+- `-i, --interactive`: Keep STDIN open
+- `-t, --tty`: Allocate a pseudo-TTY
+- `-d, --detach`: Run container in background
+- `--name`: Assign a name to the container
+- `-p, --publish`: Publish container's port to the host
+- `-v, --volume`: Bind mount a volume
+- `-e, --env`: Set environment variables
+- `--rm`: Automatically remove the container when it exits
+- `--restart=(no|always|on-failure|unless-stopped)`: Restart policy
+- `--net, --network`: Connect container to a network
+
+Examples:
+```bash
+# Run an interactive container and automatically remove it when it exits
+nerdctl run -it --rm alpine sh
+
+# Run a detached container with port mapping
+nerdctl run -d --name nginx -p 8080:80 nginx
+
+# Run a container with a volume mount
+nerdctl run -it --rm -v $(pwd):/data alpine ls /data
+```
+
+### Managing Containers
+
+**List containers**:
+```
+nerdctl ps [OPTIONS]
+```
+
+Options:
+- `-a, --all`: Show all containers (default shows just running)
+- `-q, --quiet`: Only display container IDs
+- `-s, --size`: Display total file sizes
+
+**Stop a container**:
+```
+nerdctl stop [OPTIONS] CONTAINER [CONTAINER...]
+```
+
+**Start a container**:
+```
+nerdctl start [OPTIONS] CONTAINER [CONTAINER...]
+```
+
+**Remove a container**:
+```
+nerdctl rm [OPTIONS] CONTAINER [CONTAINER...]
+```
+
+Options:
+- `-f, --force`: Force removal of running container
+- `-v, --volumes`: Remove anonymous volumes
+
+**View container logs**:
+```
+nerdctl logs [OPTIONS] CONTAINER
+```
+
+Options:
+- `-f, --follow`: Follow log output
+- `--since`: Show logs since timestamp
+- `-t, --timestamps`: Show timestamps
+- `-n, --tail`: Number of lines to show from the end of logs
+
+**Execute a command in a running container**:
+```
+nerdctl exec [OPTIONS] CONTAINER COMMAND [ARG...]
+```
+
+Options:
+- `-i, --interactive`: Keep STDIN open
+- `-t, --tty`: Allocate a pseudo-TTY
+- `-d, --detach`: Detached mode
+- `-w, --workdir`: Working directory
+- `-e, --env`: Set environment variables
+
+## Image Management
+
+### Working with Images
+
+**List images**:
+```
+nerdctl images [OPTIONS]
+```
+
+Options:
+- `-a, --all`: Show all images
+- `-q, --quiet`: Only show numeric IDs
+- `--digests`: Show digests
+
+**Pull an image**:
+```
+nerdctl pull [OPTIONS] NAME[:TAG|@DIGEST]
+```
+
+Options:
+- `--platform=(amd64|arm64|...)`: Pull content for specific platform
+- `-q, --quiet`: Suppress verbose output
+
+**Push an image**:
+```
+nerdctl push [OPTIONS] NAME[:TAG]
+```
+
+**Build an image**:
+```
+nerdctl build [OPTIONS] PATH
+```
+
+Options:
+- `-t, --tag`: Name and optionally tag the image
+- `-f, --file`: Name of the Dockerfile
+- `--build-arg`: Set build-time variables
+- `--no-cache`: Do not use cache when building
+
+**Remove an image**:
+```
+nerdctl rmi [OPTIONS] IMAGE [IMAGE...]
+```
+
+Options:
+- `-f, --force`: Force removal
+
+**Tag an image**:
+```
+nerdctl tag SOURCE_IMAGE[:TAG] TARGET_IMAGE[:TAG]
+```
+
+**Save an image to a tar archive**:
+```
+nerdctl save [OPTIONS] IMAGE [IMAGE...]
+```
+
+Options:
+- `-o, --output`: Write to a file
+
+**Load an image from a tar archive**:
+```
+nerdctl load [OPTIONS]
+```
+
+Options:
+- `-i, --input`: Read from a tar archive file
+
+## Network Management
+
+### Working with Networks
+
+**List networks**:
+```
+nerdctl network ls [OPTIONS]
+```
+
+**Create a network**:
+```
+nerdctl network create [OPTIONS] NETWORK
+```
+
+Common options:
+- `-d, --driver=(bridge|macvlan|ipvlan)`: Driver to manage the network
+- `--subnet`: Subnet in CIDR format (e.g., "10.5.0.0/16")
+- `--gateway`: Gateway for the subnet
+- `--ipam-driver=(default|host-local|dhcp)`: IP address management driver
+
+**Remove a network**:
+```
+nerdctl network rm NETWORK [NETWORK...]
+```
+
+**Inspect a network**:
+```
+nerdctl network inspect [OPTIONS] NETWORK [NETWORK...]
+```
+
+**Prune networks**:
+```
+nerdctl network prune [OPTIONS]
+```
+
+### Network Types
+
+nerdctl supports the following network types:
+- `bridge` (default on Linux): Creates a bridge interface on the host
+- `host`: Uses the host's network stack
+- `none`: No networking
+- `macvlan`: Connects container interfaces directly to host interfaces
+- `ipvlan`: Similar to macvlan but shares host's IP address
+
+Example creating a macvlan network:
+```bash
+nerdctl network create macnet --driver macvlan \
+  --subnet=192.168.5.0/24 \
+  --gateway=192.168.5.1 \
+  -o parent=eth0
+```
+
+## Volume Management
+
+### Working with Volumes
+
+**List volumes**:
+```
+nerdctl volume ls [OPTIONS]
+```
+
+**Create a volume**:
+```
+nerdctl volume create [OPTIONS] [VOLUME]
+```
+
+**Remove a volume**:
+```
+nerdctl volume rm [OPTIONS] VOLUME [VOLUME...]
+```
+
+**Inspect a volume**:
+```
+nerdctl volume inspect [OPTIONS] VOLUME [VOLUME...]
+```
+
+**Prune volumes**:
+```
+nerdctl volume prune [OPTIONS]
+```
+
+### Volume Flags for Containers
+
+Volume-related flags when running containers:
+- `-v, --volume`: Bind mount a volume (format: `SRC:DST[:OPTIONS]`)
+- `--mount`: Attach a filesystem mount to the container
+- `--tmpfs`: Mount a tmpfs directory
+
+Volume options:
+- `rw`: Read/write (default)
+- `ro`: Read-only
+- `rro`: Recursive read-only (kernel >= 5.12)
+- `shared`, `slave`, `private`: Non-recursive propagation
+- `rshared`, `rslave`, `rprivate`: Recursive propagation
+
+Examples:
+```bash
+# Mount a host directory
+nerdctl run -it --rm -v /host/path:/container/path:ro alpine ls /container/path
+
+# Use tmpfs
+nerdctl run -it --rm --tmpfs /tmp:size=64m,exec alpine ls /tmp
+```
+
+## Compose
+
+nerdctl includes Docker Compose compatibility, allowing you to define and run multi-container applications.
+
+**Run Compose applications**:
+```
+nerdctl compose up [OPTIONS]
+```
+
+Options:
+- `-d, --detach`: Run containers in the background
+- `--build`: Build images before starting containers
+- `--no-build`: Don't build images, even if they're missing
+- `--force-recreate`: Force recreation of containers
+
+**Stop Compose applications**:
+```
+nerdctl compose down [OPTIONS]
+```
+
+Options:
+- `-v, --volumes`: Remove named volumes and anonymous volumes
+
+**View Compose logs**:
+```
+nerdctl compose logs [OPTIONS] [SERVICE...]
+```
+
+Other Compose commands:
+- `nerdctl compose build`: Build service images
+- `nerdctl compose ps`: List containers
+- `nerdctl compose pull`: Pull service images
+- `nerdctl compose exec`: Execute a command in a running container
+- `nerdctl compose restart`: Restart services
+
+Example `compose.yml`:
+```yaml
+version: "3.8"
+services:
+  web:
+    image: nginx
+    ports:
+      - "8080:80"
+    volumes:
+      - ./html:/usr/share/nginx/html
+  db:
+    image: postgres
+    environment:
+      POSTGRES_PASSWORD: example
+    volumes:
+      - db-data:/var/lib/postgresql/data
+
+volumes:
+  db-data:
+```
+
+## Rootless Mode
+
+nerdctl supports rootless containers, allowing unprivileged users to create and manage containers. This provides better security isolation compared to running everything as root.
+
+### Setup Rootless Mode
+
+1. Install required dependencies (see https://rootlesscontaine.rs/getting-started/common/)
+2. Set up rootless containerd:
+   ```
+   containerd-rootless-setuptool.sh install
+   ```
+3. Enable lingering for your user (to keep services running after logout):
+   ```
+   sudo loginctl enable-linger $(whoami)
+   ```
+4. For building images, install BuildKit in rootless mode:
+   ```
+   containerd-rootless-setuptool.sh install-buildkit
+   ```
+
+When running in rootless mode, nerdctl automatically uses the appropriate socket and configuration.
+
+### Limitations and Considerations
+
+- Resource limits require cgroup v2 and systemd
+- By default, ports below 1024 cannot be published (use slirp4netns port driver or configure capabilities)
+- Some file system operations might be restricted
+- Network performance can be slower (consider using bypass4netns to improve performance)
+
+## Registry Authentication
+
+nerdctl uses the same authentication configuration as Docker, located in `${DOCKER_CONFIG}/config.json` (default: `$HOME/.docker/config.json`).
+
+**Log in to a registry**:
+```
+nerdctl login [OPTIONS] [SERVER]
+```
+
+Options:
+- `-u, --username`: Username
+- `-p, --password`: Password
+- `--password-stdin`: Take the password from stdin
+
+**Log out from a registry**:
+```
+nerdctl logout [SERVER]
+```
+
+### Registry Certificates
+
+For private registries with custom certificates, place certificates in:
+- Rootful: `/etc/containerd/certs.d/<HOST:PORT>/` or `/etc/docker/certs.d/<HOST:PORT>/`
+- Rootless: `~/.config/containerd/certs.d/<HOST:PORT>/` or `~/.config/docker/certs.d/<HOST:PORT>/`
+
+## Advanced Features
+
+### GPU Support
+
+nerdctl supports NVIDIA GPU passthrough to containers:
+
+```
+nerdctl run -it --rm --gpus all nvidia/cuda:12.3.1-base-ubuntu20.04 nvidia-smi
+```
+
+Options for `--gpus`:
+- `all`: Use all available GPUs
+- Custom configuration: `--gpus '"capabilities=utility,compute",device=GPU-UUID'`
+
+### BuildKit Integration
+
+BuildKit provides advanced image building capabilities:
+
+1. Set up BuildKit (different for rootful and rootless):
+   ```
+   # Rootless with containerd worker
+   CONTAINERD_NAMESPACE=default containerd-rootless-setuptool.sh install-buildkit-containerd
+   ```
+
+2. Use advanced build features:
+   ```
+   nerdctl build --output=type=local,dest=./output --platform=linux/amd64,linux/arm64 .
+   ```
+
+### Namespace Management
+
+**Create a namespace**:
+```
+nerdctl namespace create NAMESPACE
+```
+
+**List namespaces**:
+```
+nerdctl namespace ls
+```
+
+**Remove a namespace**:
+```
+nerdctl namespace remove NAMESPACE
+```
+
+### Security Features
+
+nerdctl supports various security features:
+
+- `--security-opt seccomp=profile.json`: Apply a seccomp profile
+- `--security-opt apparmor=profile`: Apply an AppArmor profile
+- `--cap-add`/`--cap-drop`: Add or drop Linux capabilities
+- `--privileged`: Give extended privileges to the container
+
+## Typical Workflow Example
+
+```bash
+# Create a container from an existing image
+container=$(nerdctl run -d --name my-nginx nginx:latest)
+
+# Execute a command in the container
+nerdctl exec $container apt-get update
+nerdctl exec $container apt-get install -y curl
+
+# Copy local configuration files to the container
+nerdctl cp ./nginx.conf $container:/etc/nginx/nginx.conf
+
+# Commit the container to create a new image
+nerdctl commit $container my-custom-nginx:latest
+
+# Stop and remove the container
+nerdctl stop $container
+nerdctl rm $container
+
+# Create a new container from our custom image
+nerdctl run -d --name nginx-custom -p 8080:80 my-custom-nginx:latest
+
+# Build an image using a Dockerfile
+nerdctl build -t my-app:latest .
+
+# Push the image to a registry
+nerdctl push my-custom-nginx:latest docker.io/username/my-custom-nginx:latest
+
+# List images
+nerdctl images
+
+# List containers
+nerdctl ps -a
+```

virt/src/nerdctl/nerdctldocs/build.md

@@ -0,0 +1,103 @@
+# Setting up `nerdctl build` with BuildKit
+
+`nerdctl build` (and `nerdctl compose build`) relies on [BuildKit](https://github.com/moby/buildkit).
+To use it, you need to set up BuildKit.
+
+BuildKit has 2 types of backends.
+
+- **containerd worker**: BuildKit relies on containerd to manage containers and images, etc. containerd needs to be up-and-running on the host.
+- **OCI worker**: BuildKit manages containers and images, etc. containerd isn't needed. This worker relies on runc for container execution.
+
+You need to set up BuildKit with either of the above workers.
+
+Note that OCI worker cannot access base images (`FROM` images in Dockerfiles) managed by containerd.
+Thus you cannot let `nerdctl build` use containerd-managed images as the base image.
+They include images previously built using `nerdctl build`.
+
+For example, the following build `bar` fails with OCI worker because it tries to use the previously built and containerd-managed image `foo`.
+
+```console
+$ mkdir -p /tmp/ctx && cat <<EOF > /tmp/ctx/Dockerfile
+FROM ghcr.io/stargz-containers/ubuntu:20.04-org
+RUN echo hello
+EOF
+$ nerdctl build -t foo /tmp/ctx
+$ cat <<EOF > /tmp/ctx/Dockerfile
+FROM foo
+RUN echo bar
+EOF
+$ nerdctl build -t bar /tmp/ctx
+```
+
+This limitation can be avoided using containerd worker as mentioned later.
+
+## Setting up BuildKit with containerd worker
+
+### Rootless
+
+| :zap: Requirement | nerdctl >= 0.18, BuildKit >= 0.10 |
+|-------------------|-----------------------------------|
+
+```
+$ CONTAINERD_NAMESPACE=default containerd-rootless-setuptool.sh install-buildkit-containerd
+```
+
+`containerd-rootless-setuptool.sh` is aware of `CONTAINERD_NAMESPACE` and `CONTAINERD_SNAPSHOTTER` envvars.
+It installs buildkitd to the specified containerd namespace.
+This allows BuildKit using containerd-managed images in that namespace as the base image.
+Note that BuildKit can't use images in other namespaces as of now.
+
+If `CONTAINERD_NAMESPACE` envvar is not specified, this script configures buildkitd to use "buildkit" namespace (not "default" namespace).
+
+You can install an additional buildkitd process in a different namespace by executing this script with specifying the namespace with `CONTAINERD_NAMESPACE`.
+
+BuildKit will expose the socket at `$XDG_RUNTIME_DIR/buildkit-$CONTAINERD_NAMESPACE/buildkitd.sock` if `CONTAINERD_NAMESPACE` is specified.
+If `CONTAINERD_NAMESPACE` is not specified, that location will be `$XDG_RUNTIME_DIR/buildkit/buildkitd.sock`.
+
+### Rootful
+
+```
+$ sudo systemctl enable --now buildkit
+```
+
+Then add the following configuration to `/etc/buildkit/buildkitd.toml` to enable containerd worker.
+
+```toml
+[worker.oci]
+  enabled = false
+
+[worker.containerd]
+  enabled = true
+  # namespace should be "k8s.io" for Kubernetes (including Rancher Desktop)
+  namespace = "default"
+```
+
+## Setting up BuildKit with OCI worker
+
+### Rootless
+
+```
+$ containerd-rootless-setuptool.sh install-buildkit
+```
+
+As mentioned in the above, BuildKit with this configuration cannot use images managed by containerd.
+They include images previously built with `nerdctl build`.
+
+BuildKit will expose the socket at `$XDG_RUNTIME_DIR/buildkit/buildkitd.sock`.
+
+### rootful
+
+```
+$ sudo systemctl enable --now buildkit
+```
+
+## Which BuildKit socket will nerdctl use?
+
+You can specify BuildKit address for `nerdctl build` using `--buildkit-host` flag or `BUILDKIT_HOST` envvar.
+When BuildKit address isn't specified, nerdctl tries some default BuildKit addresses the following order and uses the first available one.
+
+- `<runtime directory>/buildkit-<current namespace>/buildkitd.sock`
+- `<runtime directory>/buildkit-default/buildkitd.sock`
+- `<runtime directory>/buildkit/buildkitd.sock`
+
+For example, if you run rootless nerdctl with `test` containerd namespace, it tries to use `$XDG_RUNTIME_DIR/buildkit-test/buildkitd.sock` by default then try to fall back to `$XDG_RUNTIME_DIR/buildkit-default/buildkitd.sock` and `$XDG_RUNTIME_DIR/buildkit/buildkitd.sock`

virt/src/nerdctl/nerdctldocs/cni.md

@@ -0,0 +1,166 @@
+# Using CNI with nerdctl
+
+nerdctl uses CNI plugins for its container network, you can set network by
+either `--network` or `--net` option.
+
+## Basic networks
+
+nerdctl support some basic types of CNI plugins without any configuration
+needed(you should have CNI plugin be installed), for Linux systems the basic
+CNI plugin types are `bridge`, `portmap`, `firewall`, `tuning`, for Windows
+system, the supported CNI plugin types are `nat` only.
+
+The default network `bridge` for Linux and `nat` for Windows if you
+don't set any network options.
+
+Configuration of the default network `bridge` of Linux:
+
+```json
+{
+  "cniVersion": "1.0.0",
+  "name": "bridge",
+  "plugins": [
+    {
+      "type": "bridge",
+      "bridge": "nerdctl0",
+      "isGateway": true,
+      "ipMasq": true,
+      "hairpinMode": true,
+      "ipam": {
+        "type": "host-local",
+        "routes": [{ "dst": "0.0.0.0/0" }],
+        "ranges": [
+          [
+            {
+              "subnet": "10.4.0.0/24",
+              "gateway": "10.4.0.1"
+            }
+          ]
+        ]
+      }
+    },
+    {
+      "type": "portmap",
+      "capabilities": {
+        "portMappings": true
+      }
+    },
+    {
+      "type": "firewall",
+      "ingressPolicy": "same-bridge"
+    },
+    {
+      "type": "tuning"
+    }
+  ]
+}
+```
+
+## Bridge isolation
+
+nerdctl >= 0.18 sets the `ingressPolicy` to `same-bridge` when `firewall` plugin >= 1.1.0 is installed.
+This `ingressPolicy` replaces the CNI `isolation` plugin used in nerdctl <= 0.17.
+
+When `firewall` plugin >= 1.1.0 is not found, nerdctl does not enable the bridge isolation.
+This means a container in `--net=foo` can connect to a container in `--net=bar`.
+
+## macvlan/IPvlan networks
+
+nerdctl also support macvlan and IPvlan network driver.
+
+To create a `macvlan` network which bridges with a given physical network interface, use `--driver macvlan` with
+`nerdctl network create` command.
+
+```
+# nerdctl network create mac0 --driver macvlan \
+  --subnet=192.168.5.0/24
+  --gateway=192.168.5.2
+  -o parent=eth0
+```
+
+You can specify the `parent`, which is the interface the traffic will physically go through on the host,
+defaults to default route interface.
+
+And the `subnet` should be under the same network as the network interface,
+an easier way is to use DHCP to assign the IP:
+
+```
+# nerdctl network create mac0 --driver macvlan --ipam-driver=dhcp
+```
+
+Using `--driver ipvlan` can create `ipvlan` network, the default mode for IPvlan is `l2`.
+
+## DHCP host-name and other DHCP options
+
+Nerdctl automatically sets the DHCP host-name option to the hostname value of the container.
+
+Furthermore, on network creation, nerdctl supports the ability to set other DHCP options through `--ipam-options`.
+
+Currently, the following options are supported by the DHCP plugin:
+```
+dhcp-client-identifier
+subnet-mask
+routers
+user-class
+vendor-class-identifier
+```
+
+For example:
+```
+# nerdctl network create --driver macvlan \
+    --ipam-driver dhcp \
+    --ipam-opt 'vendor-class-identifier={"type": "provide", "value": "Hey! Its me!"}' \
+    my-dhcp-net
+```
+
+## Custom networks
+
+You can also customize your CNI network by providing configuration files.
+
+When rootful, the expected root location is `/etc/cni/net.d`.
+For rootless, the expected root location is `~/.config/cni/net.d/`
+
+Configuration files (like `10-mynet.conf`) can be placed either in the root location,
+or under a subfolder.
+If in the root location, this network will be available to all nerdctl namespaces.
+If placed in a subfolder, it will be available only to the identically named namespace.
+
+For example, you have one configuration file(`/etc/cni/net.d/10-mynet.conf`)
+for `bridge` network:
+
+```json
+{
+  "cniVersion": "1.0.0",
+  "name": "mynet",
+  "type": "bridge",
+  "bridge": "cni0",
+  "isGateway": true,
+  "ipMasq": true,
+  "ipam": {
+    "type": "host-local",
+    "subnet": "172.19.0.0/24",
+    "routes": [
+      { "dst": "0.0.0.0/0" }
+    ]
+  }
+}
+```
+
+This will configure a new CNI network with the name `mynet`, and you can use
+this network to create a container in any namespace:
+
+```console
+# nerdctl run -it --net mynet --rm alpine ip addr show
+1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
+    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
+    inet 127.0.0.1/8 scope host lo
+       valid_lft forever preferred_lft forever
+    inet6 ::1/128 scope host
+       valid_lft forever preferred_lft forever
+3: eth0@if6120: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue state UP
+    link/ether 5e:5b:3f:0c:36:56 brd ff:ff:ff:ff:ff:ff
+    inet 172.19.0.51/24 brd 172.19.0.255 scope global eth0
+       valid_lft forever preferred_lft forever
+    inet6 fe80::5c5b:3fff:fe0c:3656/64 scope link tentative
+       valid_lft forever preferred_lft forever
+```

virt/src/nerdctl/nerdctldocs/compose.md

@@ -0,0 +1,44 @@
+# nerdctl compose
+
+| :zap: Requirement | nerdctl >= 0.8 |
+|-------------------|----------------|
+
+## Usage
+
+The `nerdctl compose` CLI is designed to be compatible with `docker-compose`.
+
+```console
+$ nerdctl compose up -d
+$ nerdctl compose down
+```
+
+See the Command Reference in [`../README.md`](../README.md).
+
+## Spec conformance
+
+`nerdctl compose` implements [The Compose Specification](https://github.com/compose-spec/compose-spec),
+which was derived from [Docker Compose file version 3 specification](https://docs.docker.com/compose/compose-file/compose-file-v3/).
+
+### Unimplemented YAML fields
+- Fields that correspond to unimplemented `docker run` flags, e.g., `services.<SERVICE>.links` (corresponds to `docker run --link`)
+- Fields that correspond to unimplemented `docker build` flags, e.g., `services.<SERVICE>.build.extra_hosts` (corresponds to `docker build --add-host`)
+- `services.<SERVICE>.credential_spec`
+- `services.<SERVICE>.deploy.update_config`
+- `services.<SERVICE>.deploy.rollback_config`
+- `services.<SERVICE>.deploy.resources.reservations`
+- `services.<SERVICE>.deploy.placement`
+- `services.<SERVICE>.deploy.endpoint_mode`
+- `services.<SERVICE>.healthcheck`
+- `services.<SERVICE>.stop_grace_period`
+- `services.<SERVICE>.stop_signal`
+- `configs.<CONFIG>.external`
+- `secrets.<SECRET>.external`
+
+### Incompatibility
+#### `services.<SERVICE>.build.context`
+- The value must be a local directory path, not a URL.
+
+#### `services.<SERVICE>.secrets`, `services.<SERVICE>.configs`
+- `uid`, `gid`: Cannot be specified. The default value is not propagated from `USER` instruction of Dockerfile.
+  The file owner corresponds to the original file on the host.
+- `mode`: Cannot be specified. The file is mounted as read-only, with permission bits that correspond to the original file on the host.

virt/src/nerdctl/nerdctldocs/config.md

@@ -0,0 +1,62 @@
+# Configuring nerdctl with `nerdctl.toml`
+
+| :zap: Requirement | nerdctl >= 0.16 |
+|-------------------|-----------------|
+
+This document describes the configuration file of nerdctl (`nerdctl.toml`).
+This file is unrelated to the configuration file of containerd (`config.toml`) .
+
+## File path
+- Rootful mode:  `/etc/nerdctl/nerdctl.toml`
+- Rootless mode: `~/.config/nerdctl/nerdctl.toml`
+
+The path can be overridden with `$NERDCTL_TOML`.
+
+## Example
+
+```toml
+# This is an example of /etc/nerdctl/nerdctl.toml .
+# Unrelated to the daemon's /etc/containerd/config.toml .
+
+debug          = false
+debug_full     = false
+address        = "unix:///run/k3s/containerd/containerd.sock"
+namespace      = "k8s.io"
+snapshotter    = "stargz"
+cgroup_manager = "cgroupfs"
+hosts_dir      = ["/etc/containerd/certs.d", "/etc/docker/certs.d"]
+experimental   = true
+```
+
+## Properties
+
+| TOML property       | CLI flag                           | Env var                   | Description                                                                                                                                                      | Availability \*1 |
+|---------------------|------------------------------------|---------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------|
+| `debug`             | `--debug`                          |                           | Debug mode                                                                                                                                                       | Since 0.16.0     |
+| `debug_full`        | `--debug-full`                     |                           | Debug mode (with full output)                                                                                                                                    | Since 0.16.0     |
+| `address`           | `--address`,`--host`,`-a`,`-H`     | `$CONTAINERD_ADDRESS`     | containerd address                                                                                                                                               | Since 0.16.0     |
+| `namespace`         | `--namespace`,`-n`                 | `$CONTAINERD_NAMESPACE`   | containerd namespace                                                                                                                                             | Since 0.16.0     |
+| `snapshotter`       | `--snapshotter`,`--storage-driver` | `$CONTAINERD_SNAPSHOTTER` | containerd snapshotter                                                                                                                                           | Since 0.16.0     |
+| `cni_path`          | `--cni-path`                       | `$CNI_PATH`               | CNI binary directory                                                                                                                                             | Since 0.16.0     |
+| `cni_netconfpath`   | `--cni-netconfpath`                | `$NETCONFPATH`            | CNI config directory                                                                                                                                             | Since 0.16.0     |
+| `data_root`         | `--data-root`                      |                           | Persistent state directory                                                                                                                                       | Since 0.16.0     |
+| `cgroup_manager`    | `--cgroup-manager`                 |                           | cgroup manager                                                                                                                                                   | Since 0.16.0     |
+| `insecure_registry` | `--insecure-registry`              |                           | Allow insecure registry                                                                                                                                          | Since 0.16.0     |
+| `hosts_dir`         | `--hosts-dir`                      |                           | `certs.d` directory                                                                                                                                              | Since 0.16.0     |
+| `experimental`      | `--experimental`                   | `NERDCTL_EXPERIMENTAL`    | Enable  [experimental features](experimental.md)                                                                                                                 | Since 0.22.3     |
+| `host_gateway_ip`   | `--host-gateway-ip`                | `NERDCTL_HOST_GATEWAY_IP` | IP address that the special 'host-gateway' string in --add-host resolves to. Defaults to the IP address of the host. It has no effect without setting --add-host | Since 1.3.0      |
+| `bridge_ip`         | `--bridge-ip`                      | `NERDCTL_BRIDGE_IP`       | IP address for the default nerdctl bridge network, e.g., 10.1.100.1/24                                                                                           | Since 2.0.1      |
+| `kube_hide_dupe`    | `--kube-hide-dupe`                 |                           | Deduplicate images for Kubernetes with namespace k8s.io, no more redundant <none> ones are displayed    | Since 2.0.3      |
+
+The properties are parsed in the following precedence:
+1. CLI flag
+2. Env var
+3. TOML property
+4. Built-in default value (Run `nerdctl --help` to see the default values)
+
+\*1: Availability of the TOML properties
+
+## See also
+- [`registry.md`](registry.md)
+- [`faq.md`](faq.md)
+- https://github.com/containerd/containerd/blob/main/docs/ops.md#base-configuration (`/etc/containerd/config.toml`)

virt/src/nerdctl/nerdctldocs/cosign.md

@@ -0,0 +1,214 @@
+# Container Image Sign and Verify with cosign tool
+
+| :zap: Requirement | nerdctl >= 0.15 |
+|-------------------|-----------------|
+
+[cosign](https://github.com/sigstore/cosign) is tool that allows you to sign and verify container images with the
+public/private key pairs or without them by providing
+a [Keyless support](https://github.com/sigstore/cosign/blob/main/KEYLESS.md).
+
+Keyless uses ephemeral keys and certificates, which are signed automatically by
+the [fulcio](https://github.com/sigstore/fulcio) root CA. Signatures are stored in
+the [rekor](https://github.com/sigstore/rekor) transparency log, which automatically provides an attestation as to when
+the signature was created.
+
+Cosign would use prompt to confirm the statement below during `sign`. Nerdctl added `--yes` to Cosign command, which says yes and prevents this prompt.
+Using Nerdctl push with signing by Cosign means that users agree the statement.
+
+
+```
+Note that there may be personally identifiable information associated with this signed artifact.
+This may include the email address associated with the account with which you authenticate.
+This information will be used for signing this artifact and will be stored in public transparency logs and cannot be removed later.
+
+By typing 'y', you attest that you grant (or have permission to grant) and agree to have this information stored permanently in transparency logs.
+```
+
+You can enable container signing and verifying features with `push` and `pull` commands of `nerdctl` by using `cosign`
+under the hood with make use of flags `--sign` while pushing the container image, and `--verify` while pulling the
+container image.
+
+> * Ensure cosign executable in your `$PATH`.
+> * You can install cosign by following this page: https://docs.sigstore.dev/cosign/installation
+
+Prepare your environment:
+
+```shell
+# Create a sample Dockerfile
+$ cat <<EOF | tee Dockerfile.dummy
+FROM alpine:latest
+CMD [ "echo", "Hello World" ]
+EOF
+```
+
+> Please do not forget, we won't be validating the base images, which is `alpine:latest` in this case, of the container image that was built on,
+> we'll only verify the container image itself once we sign it.
+
+```shell
+
+# Build the image
+$ nerdctl build -t devopps/hello-world -f Dockerfile.dummy .
+
+# Generate a key-pair: cosign.key and cosign.pub
+$ cosign generate-key-pair
+
+# Export your COSIGN_PASSWORD to prevent CLI prompting
+$ export COSIGN_PASSWORD=$COSIGN_PASSWORD
+```
+
+Sign the container image while pushing:
+
+```
+# Sign the image with Keyless mode
+$ nerdctl push --sign=cosign devopps/hello-world
+
+# Sign the image and store the signature in the registry
+$ nerdctl push --sign=cosign --cosign-key cosign.key devopps/hello-world
+```
+
+Verify the container image while pulling:
+
+> REMINDER: Image won't be pulled if there are no matching signatures in case you passed `--verify` flag.
+
+> REMINDER: For keyless flows to work, you need to set either --cosign-certificate-identity or --cosign-certificate-identity-regexp, and either --cosign-certificate-oidc-issuer or --cosign-certificate-oidc-issuer-regexp. The OIDC issuer expected in a valid Fulcio certificate for --verify=cosign, e.g. https://token.actions.githubusercontent.com or https://oauth2.sigstore.dev/auth.
+
+```shell
+# Verify the image with Keyless mode
+$ nerdctl pull --verify=cosign --certificate-identity=name@example.com --certificate-oidc-issuer=https://accounts.example.com devopps/hello-world
+INFO[0004] cosign:
+INFO[0004] cosign: [{"critical":{"identity":...}]
+docker.io/devopps/nginx-new:latest:                                               resolved       |++++++++++++++++++++++++++++++++++++++|
+manifest-sha256:0910d404e58dd320c3c0c7ea31bf5fbfe7544b26905c5eccaf87c3af7bcf9b88: done           |++++++++++++++++++++++++++++++++++++++|
+config-sha256:1de1c4fb5122ac8650e349e018fba189c51300cf8800d619e92e595d6ddda40e:   done           |++++++++++++++++++++++++++++++++++++++|
+elapsed: 1.4 s                                                                    total:  1.3 Ki (928.0 B/s)
+
+# You can not verify the image if it is not signed
+$ nerdctl pull --verify=cosign --cosign-key cosign.pub devopps/hello-world-bad
+INFO[0003] cosign: Error: no matching signatures:
+INFO[0003] cosign: failed to verify signature
+INFO[0003] cosign: main.go:46: error during command execution: no matching signatures:
+INFO[0003] cosign: failed to verify signature
+```
+
+## Cosign in Compose
+
+> Cosign support in Compose is also experimental and implemented based on Compose's [extension](https://github.com/compose-spec/compose-spec/blob/master/spec.md#extension) capibility.
+
+cosign is supported in `nerdctl compose up|run|push|pull`. You can use cosign in Compose by adding the following fields in your compose yaml. These fields are _per service_, and you can enable only `verify` or only `sign` (or both).
+
+```yaml
+# only put cosign related fields under the service you want to sign/verify.
+services:
+  svc0:
+    build: .
+    image: ${REGISTRY}/svc0_image # replace with your registry
+    # `x-nerdctl-verify` and `x-nerdctl-cosign-public-key` are for verify
+    # required for `nerdctl compose up|run|pull`
+    x-nerdctl-verify: cosign
+    x-nerdctl-cosign-public-key: /path/to/cosign.pub
+    # `x-nerdctl-sign` and `x-nerdctl-cosign-private-key` are for sign
+    # required for `nerdctl compose push`
+    x-nerdctl-sign: cosign
+    x-nerdctl-cosign-private-key: /path/to/cosign.key
+    ports:
+    - 8080:80
+  svc1:
+    build: .
+    image: ${REGISTRY}/svc1_image # replace with your registry
+    ports:
+    - 8081:80
+```
+
+Following the cosign tutorial above, first set up environment and prepare cosign key pair:
+
+```shell
+# Generate a key-pair: cosign.key and cosign.pub
+$ cosign generate-key-pair
+
+# Export your COSIGN_PASSWORD to prevent CLI prompting
+$ export COSIGN_PASSWORD=$COSIGN_PASSWORD
+```
+
+We'll use the following `Dockerfile` and `docker-compose.yaml`:
+
+```shell
+$ cat Dockerfile
+FROM nginx:1.19-alpine
+RUN uname -m > /usr/share/nginx/html/index.html
+
+$ cat docker-compose.yml
+services:
+  svc0:
+    build: .
+    image: ${REGISTRY}/svc1_image # replace with your registry
+    x-nerdctl-verify: cosign
+    x-nerdctl-cosign-public-key: ./cosign.pub
+    x-nerdctl-sign: cosign
+    x-nerdctl-cosign-private-key: ./cosign.key
+    ports:
+    - 8080:80
+  svc1:
+    build: .
+    image: ${REGISTRY}/svc1_image # replace with your registry
+    ports:
+    - 8081:80
+```
+
+For keyless mode, the `docker-compose.yaml` will be:
+```
+$ cat docker-compose.yml
+services:
+  svc0:
+    build: .
+    image: ${REGISTRY}/svc1_image # replace with your registry
+    x-nerdctl-verify: cosign
+    x-nerdctl-sign: cosign
+    x-nerdctl-cosign-certificate-identity: name@example.com # or x-nerdctl-cosign-certificate-identity-regexp
+    x-nerdctl-cosign-certificate-oidc-issuer: https://accounts.example.com # or x-nerdctl-cosign-certificate-oidc-issuer-regexp
+    ports:
+    - 8080:80
+  svc1:
+    build: .
+    image: ${REGISTRY}/svc1_image # replace with your registry
+    ports:
+    - 8081:80
+```
+
+> The `env "COSIGN_PASSWORD="$COSIGN_PASSWORD""` part in the below commands is a walkaround to use rootful nerdctl and make the env variable visible to root (in sudo). You don't need this part if (1) you're using rootless, or (2) your `COSIGN_PASSWORD` is visible in root.
+
+First let's `build` and `push` the two services:
+
+```shell
+$ sudo nerdctl compose build
+INFO[0000] Building image xxxxx/svc0_image
+...
+INFO[0000] Building image xxxxx/svc1_image
+[+] Building 0.2s (6/6) FINISHED
+
+$ sudo env "COSIGN_PASSWORD="$COSIGN_PASSWORD"" nerdctl compose --experimental=true push
+INFO[0000] Pushing image xxxxx/svc1_image
+...
+INFO[0000] Pushing image xxxxx/svc0_image
+INFO[0000] pushing as a reduced-platform image (application/vnd.docker.distribution.manifest.v2+json, sha256:4329abc3143b1545835de17e1302c8313a9417798b836022f4c8c8dc8b10a3e9)
+INFO[0000] cosign: WARNING: Image reference xxxxx/svc0_image uses a tag, not a digest, to identify the image to sign.
+INFO[0000] cosign:
+INFO[0000] cosign: This can lead you to sign a different image than the intended one. Please use a
+INFO[0000] cosign: digest (example.com/ubuntu@sha256:abc123...) rather than tag
+INFO[0000] cosign: (example.com/ubuntu:latest) for the input to cosign. The ability to refer to
+INFO[0000] cosign: images by tag will be removed in a future release.
+INFO[0000] cosign: Pushing signature to: xxxxx/svc0_image
+```
+
+Then we can `pull` and `up` services (`run` is similar to up):
+
+```shell
+# ensure built images are removed and pull is performed.
+$ sudo nerdctl compose down
+$ sudo env "COSIGN_PASSWORD="$COSIGN_PASSWORD"" nerdctl compose --experimental=true pull
+$ sudo env "COSIGN_PASSWORD="$COSIGN_PASSWORD"" nerdctl compose --experimental=true up
+$ sudo env "COSIGN_PASSWORD="$COSIGN_PASSWORD"" nerdctl compose --experimental=true run svc0 -- echo "hello"
+# clean up compose resources.
+$ sudo nerdctl compose down
+```
+
+Check your logs to confirm that svc0 is verified by cosign (have cosign logs) and svc1 is not. You can also change the public key in `docker-compose.yaml` to a random value to see verify failure will stop the container being `pull|up|run`.

virt/src/nerdctl/nerdctldocs/cvmfs.md

@@ -0,0 +1,89 @@
+# Lazy-pulling using CernVM-FS Snapshotter
+
+CernVM-FS Snapshotter is a containerd snapshotter plugin. It is a specialized component responsible for assembling
+all the layers of container images into a stacked file system that containerd can use. The snapshotter takes as input the list
+of required layers and outputs a directory containing the final file system. It is also responsible to clean up the output
+directory when containers using it are stopped.
+
+See the official [documentation](https://cvmfs.readthedocs.io/en/latest/cpt-containers.html#how-to-use-the-cernvm-fs-snapshotter) to learn further information.
+
+## Prerequisites
+
+- Install containerd remote snapshotter plugin (`cvmfs-snapshotter`) from [here](https://github.com/cvmfs/cvmfs/tree/devel/snapshotter).
+
+- Add the following to `/etc/containerd/config.toml`:
+```toml
+# Ask containerd to use this particular snapshotter
+[plugins."io.containerd.grpc.v1.cri".containerd]
+    snapshotter = "cvmfs-snapshotter"
+    disable_snapshot_annotations = false
+
+# Set the communication endpoint between containerd and the snapshotter
+[proxy_plugins]
+    [proxy_plugins.cvmfs]
+        type = "snapshot"
+        address = "/run/containerd-cvmfs-grpc/containerd-cvmfs-grpc.sock"
+```
+- The default CernVM-FS repository hosting the flat root filesystems of the container images is `unpacked.cern.ch`.
+  The container images are unpacked into the CernVM-FS repository by the [DUCC](https://cvmfs.readthedocs.io/en/latest/cpt-ducc.html)
+  (Daemon that Unpacks Container Images into CernVM-FS) tool.
+  You can change the repository adding the following line to `/etc/containerd-cvmfs-grpc/config.toml`:
+```toml
+repository = "myrepo.mydomain"
+```
+- Launch `containerd` and `cvmfs-snapshotter`:
+```console
+$ systemctl start containerd cvmfs-snapshotter
+```
+
+## Enable CernVM-FS Snapshotter for `nerdctl run` and `nerdctl pull`
+
+| :zap: Requirement | nerdctl >= 1.6.3 |
+| ----------------- | ---------------- |
+
+- Run `nerdctl` with `--snapshotter cvmfs-snapshotter` as in the example below:
+```console
+$ nerdctl run -it --rm --snapshotter cvmfs-snapshotter clelange/cms-higgs-4l-full:latest
+```
+
+- You can also only pull the image with CernVM-FS Snapshotter without running the container:
+```console
+$ nerdctl pull --snapshotter cvmfs-snapshotter clelange/cms-higgs-4l-full:latest
+```
+
+The speedup for pulling this 9 GB (4.3 GB compressed) image is shown below:
+- #### with the snapshotter:
+```console
+$ nerdctl --snapshotter cvmfs-snapshotter pull clelange/cms-higgs-4l-full:latest
+docker.io/clelange/cms-higgs-4l-full:latest:                                      resolved       |++++++++++++++++++++++++++++++++++++++|
+manifest-sha256:b8acbe80629dd28d213c03cf1ffd3d46d39e573f54215a281fabce7494b3d546: done           |++++++++++++++++++++++++++++++++++++++|
+config-sha256:89ef54b6c4fbbedeeeb29b1df2b9916b6d157c87cf1878ea882bff86a3093b5c:   done           |++++++++++++++++++++++++++++++++++++++|
+elapsed: 4.7 s                                                                    total:  19.8 K (4.2 KiB/s)
+
+$ nerdctl images
+REPOSITORY                    TAG       IMAGE ID        CREATED           PLATFORM       SIZE     BLOB SIZE
+clelange/cms-higgs-4l-full    latest    b8acbe80629d    20 seconds ago    linux/amd64    0.0 B    4.3 GiB
+```
+- #### without the snapshotter:
+```console
+$ nerdctl pull clelange/cms-higgs-4l-full:latest
+docker.io/clelange/cms-higgs-4l-full:latest:                                      resolved       |++++++++++++++++++++++++++++++++++++++|
+manifest-sha256:b8acbe80629dd28d213c03cf1ffd3d46d39e573f54215a281fabce7494b3d546: exists         |++++++++++++++++++++++++++++++++++++++|
+config-sha256:89ef54b6c4fbbedeeeb29b1df2b9916b6d157c87cf1878ea882bff86a3093b5c:   exists         |++++++++++++++++++++++++++++++++++++++|
+layer-sha256:e8114d4b0d10b33aaaa4fbc3c6da22bbbcf6f0ef0291170837e7c8092b73840a:    done           |++++++++++++++++++++++++++++++++++++++|
+layer-sha256:a3eda0944a81e87c7a44b117b1c2e707bc8d18e9b7b478e21698c11ce3e8b819:    done           |++++++++++++++++++++++++++++++++++++++|
+layer-sha256:8f3160776e8e8736ea9e3f6c870d14cd104143824bbcabe78697315daca0b9ad:    done           |++++++++++++++++++++++++++++++++++++++|
+layer-sha256:22a5c05baa9db0aa7bba56ffdb2dd21246b9cf3ce938fc6d7bf20e92a067060e:    done           |++++++++++++++++++++++++++++++++++++++|
+layer-sha256:bfcf9d498f92b72426c9d5b73663504d87249d6783c6b58d71fbafc275349ab9:    done           |++++++++++++++++++++++++++++++++++++++|
+layer-sha256:0563e1549926b9c8beac62407bc6a420fa35bcf6f9844e5d8beeb9165325a872:    done           |++++++++++++++++++++++++++++++++++++++|
+layer-sha256:6fff5fd7fb4eeb79a1399d9508614a84191d05e53f094832062d689245599640:    done           |++++++++++++++++++++++++++++++++++++++|
+layer-sha256:25c39bfa66e1157415236703abc512d06cc1db31bd00fe8c3030c6d6d249dc4e:    done           |++++++++++++++++++++++++++++++++++++++|
+layer-sha256:3cc0a0eb55eb3fb7ef0760c6bf1e567dfc56933ba5f11b5415f89228af751b72:    done           |++++++++++++++++++++++++++++++++++++++|
+layer-sha256:a8850244786303e508b94bb31c8569310765e678c9c73bf1199310729209b803:    done           |++++++++++++++++++++++++++++++++++++++|
+layer-sha256:32cdf5fc12485ac061347eb8b5c3b4a28505ce8564a7f3f83ac4241f03911176:    done           |++++++++++++++++++++++++++++++++++++++|
+elapsed: 181.8s                                                                   total:  4.3 Gi (24.2 MiB/s)
+
+$ nerdctl images
+REPOSITORY                    TAG       IMAGE ID        CREATED          PLATFORM       SIZE       BLOB SIZE
+clelange/cms-higgs-4l-full    latest    b8acbe80629d    4 minutes ago    linux/amd64    9.0 GiB    4.3 GiB
+```

virt/src/nerdctl/nerdctldocs/dir.md

@@ -0,0 +1,73 @@
+# nerdctl directory layout
+
+## Config
+**Default**: `/etc/nerdctl/nerdctl.toml` (rootful), `~/.config/nerdctl/nerdctl.toml` (rootless)
+
+The configuration file of nerdctl. See [`config.md`](./config.md).
+
+Can be overridden with environment variable `$NERDCTL_TOML`.
+
+This file is unrelated to the daemon config file `/etc/containerd/config.toml`.
+
+## Data
+### `<DATAROOT>`
+**Default**: `/var/lib/nerdctl` (rootful), `~/.local/share/nerdctl` (rootless)
+
+Can be overridden with `nerdctl --data-root=<DATAROOT>` flag.
+
+The directory is solely managed by nerdctl, not by containerd.
+The directory has nothing to do with containerd data root `/var/lib/containerd`.
+
+### `<DATAROOT>/<ADDRHASH>`
+e.g. `/var/lib/nerdctl/1935db59`
+
+`1935db9` is from `$(echo -n "/run/containerd/containerd.sock" | sha256sum | cut -c1-8)`
+
+This directory is also called "data store" in the implementation.
+
+### `<DATAROOT>/<ADDRHASH>/containers/<NAMESPACE>/<CID>`
+e.g. `/var/lib/nerdctl/1935db59/containers/default/c4ed811cc361d26faffdee8d696ddbc45a9d93c571b5b3c54d3da01cb29caeb1`
+
+Files:
+- `resolv.conf`: mounted to the container as `/etc/resolv.conf`
+- `hostname`: mounted to the container as `/etc/hostname`
+- `log-config.json`: used for storing the `--log-opts` map of `nerdctl run`
+- `<CID>-json.log`: used by `nerdctl logs`
+- `oci-hook.*.log`: logs of the OCI hook
+- `lifecycle.json`: used to store stateful information about the container that can only be retrieved through OCI hooks
+
+### `<DATAROOT>/<ADDRHASH>/names/<NAMESPACE>`
+e.g. `/var/lib/nerdctl/1935db59/names/default`
+
+Files:
+- `<NAME>`: contains the container ID (CID). Represents that the name is taken by that container. 
+
+Files must be operated with a `LOCK_EX` lock against the `<DATAROOT>/<ADDRHASH>/names/<NAMESPACE>` directory.
+
+### `<DATAROOT>/<ADDRHASH>/etchosts/<NAMESPACE>/<CID>`
+e.g. `/var/lib/nerdctl/1935db59/etchosts/default/c4ed811cc361d26faffdee8d696ddbc45a9d93c571b5b3c54d3da01cb29caeb1`
+
+Files:
+- `hosts`: mounted to the container as `/etc/hosts`
+- `meta.json`: metadata
+
+Files must be operated with a `LOCK_EX` lock against the `<DATAROOT>/<ADDRHASH>/etchosts` directory.
+
+### `<DATAROOT>/<ADDRHASH>/volumes/<NAMESPACE>/<VOLNAME>/_data`
+e.g. `/var/lib/nerdctl/1935db59/volumes/default/foo/_data`
+
+Data volume
+
+## CNI
+
+### `<NETCONFPATH>`
+**Default**: `/etc/cni/net.d` (rootful), `~/.config/cni/net.d` (rootless)
+
+Can be overridden with `nerdctl --cni-netconfpath=<NETCONFPATH>` flag and environment variable `$NETCONFPATH`.
+
+At the top-level of <NETCONFPATH>, network (files) are shared accross all namespaces.
+Sub-folders inside <NETCONFPATH> are only available to the namespace bearing the same name,
+and its networks definitions are private.
+
+Files:
+- `nerdctl-<NWNAME>.conflist`: CNI conf list created by nerdctl

virt/src/nerdctl/nerdctldocs/gpu.md

@@ -0,0 +1,85 @@
+# Using GPUs inside containers
+
+| :zap: Requirement | nerdctl >= 0.9 |
+|-------------------|----------------|
+
+nerdctl provides docker-compatible NVIDIA GPU support.
+
+## Prerequisites
+
+- NVIDIA Drivers
+  - Same requirement as when you use GPUs on Docker. For details, please refer to [the doc by NVIDIA](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#pre-requisites).
+- `nvidia-container-cli`
+  - containerd relies on this CLI for setting up GPUs inside container. You can install this via [`libnvidia-container` package](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/arch-overview.html#libnvidia-container).
+
+## Options for `nerdctl run --gpus`
+
+`nerdctl run --gpus` is compatible to [`docker run --gpus`](https://docs.docker.com/engine/reference/commandline/run/#access-an-nvidia-gpu).
+
+You can specify number of GPUs to use via `--gpus` option.
+The following example exposes all available GPUs.
+
+```
+nerdctl run -it --rm --gpus all nvidia/cuda:12.3.1-base-ubuntu20.04 nvidia-smi
+```
+
+You can also pass detailed configuration to `--gpus` option as a list of key-value pairs. The following options are provided.
+
+- `count`: number of GPUs to use. `all` exposes all available GPUs.
+- `device`: IDs of GPUs to use. UUID or numbers of GPUs can be specified.
+- `capabilities`: [Driver capabilities](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/user-guide.html#driver-capabilities). If unset, use default driver `utility`, `compute`.
+
+The following example exposes a specific GPU to the container.
+
+```
+nerdctl run -it --rm --gpus '"capabilities=utility,compute",device=GPU-3a23c669-1f69-c64e-cf85-44e9b07e7a2a' nvidia/cuda:12.3.1-base-ubuntu20.04 nvidia-smi
+```
+
+## Fields for `nerdctl compose`
+
+`nerdctl compose` also supports GPUs following [compose-spec](https://github.com/compose-spec/compose-spec/blob/master/deploy.md#devices).
+
+You can use GPUs on compose when you specify some of the following `capabilities` in `services.demo.deploy.resources.reservations.devices`.
+
+- `gpu`
+- `nvidia`
+- all allowed capabilities for `nerdctl run --gpus`
+
+Available fields are the same as `nerdctl run --gpus`.
+
+The following exposes all available GPUs to the container.
+
+```
+version: "3.8"
+services:
+  demo:
+    image: nvidia/cuda:12.3.1-base-ubuntu20.04
+    command: nvidia-smi
+    deploy:
+      resources:
+        reservations:
+          devices:
+          - capabilities: ["utility"]
+            count: all
+```
+
+## Trouble Shooting
+
+### `nerdctl run --gpus` fails when using the Nvidia gpu-operator
+
+If the Nvidia driver is installed by the [gpu-operator](https://github.com/NVIDIA/gpu-operator).The `nerdctl run` will fail with the error message `(FATA[0000] exec: "nvidia-container-cli": executable file not found in $PATH)`.
+
+So, the `nvidia-container-cli` needs to be added to the PATH environment variable.
+
+You can do this by adding the following line to your $HOME/.profile or /etc/profile (for a system-wide installation):
+```
+export PATH=$PATH:/usr/local/nvidia/toolkit
+```
+
+The shared libraries also need to be added to the system.
+```
+echo "/run/nvidia/driver/usr/lib/x86_64-linux-gnu" > /etc/ld.so.conf.d/nvidia.conf
+ldconfig
+```
+
+And then, the `nerdctl run --gpus` can run successfully.

virt/src/nerdctl/nerdctldocs/ipfs.md

@@ -0,0 +1,292 @@
+# Distribute Container Images on IPFS (Experimental)
+
+| :zap: Requirement | nerdctl >= 0.14 |
+|-------------------|-----------------|
+
+You can distribute container images without registries, using IPFS.
+
+IPFS support is completely optional. Your host is NOT connected to any P2P network, unless you opt in to [install and run IPFS daemon](https://docs.ipfs.io/install/).
+
+## Prerequisites
+
+### ipfs daemon
+
+Make sure an IPFS daemon such as [Kubo](https://github.com/ipfs/kubo) (former go-ipfs) is running on your host.
+For example, you can run Kubo using the following command.
+
+```
+ipfs daemon
+```
+
+In rootless mode, you need to install ipfs daemon using `containerd-rootless-setuptool.sh`.
+
+```
+containerd-rootless-setuptool.sh -- install-ipfs --init
+```
+
+> NOTE: correctly set IPFS_PATH as described in the output of the above command.
+
+:information_source: If you want to expose some ports of ipfs daemon (e.g. 4001), you can install rootless containerd using `containerd-rootless-setuptool.sh install` with `CONTAINERD_ROOTLESS_ROOTLESSKIT_FLAGS="--publish=0.0.0.0:4001:4001/tcp"` environment variable.
+
+:information_source: If you don't want IPFS to communicate with nodes on the internet, you can run IPFS daemon in offline mode using `--offline` flag or you can create a private IPFS network as described [here](https://github.com/containerd/stargz-snapshotter/blob/main/docs/ipfs.md#appendix-1-creating-ipfs-private-network).
+
+:information_source: Instead of locally launching IPFS daemon, you can specify the address of the IPFS API using `--ipfs-address` flag.
+
+## IPFS-enabled image and OCI Compatibility
+
+Image distribution on IPFS is achieved by OCI-compatible *IPFS-enabled image format*.
+nerdctl automatically converts an image to IPFS-enabled when necessary.
+For example, when nerdctl pushes an image to IPFS, if that image isn't an IPFS-enabled one, it converts that image to the IPFS-enabled one.
+
+Please see [the doc in stargz-snapshotter project](https://github.com/containerd/stargz-snapshotter/blob/v0.10.0/docs/ipfs.md) for details about IPFS-enabled image format.
+
+## Using nerdctl with IPFS
+
+nerdctl supports an image name prefix `ipfs://` to handle images on IPFS.
+
+### `nerdctl push ipfs://<image-name>`
+
+For `nerdctl push`, you can specify `ipfs://` prefix for arbitrary image names stored in containerd.
+When this prefix is specified, nerdctl pushes that image to IPFS.
+
+```console
+> nerdctl push ipfs://ubuntu:20.04
+INFO[0000] pushing image "ubuntu:20.04" to IPFS
+INFO[0000] ensuring image contents
+bafkreicq4dg6nkef5ju422ptedcwfz6kcvpvvhuqeykfrwq5krazf3muze
+```
+
+At last line of the output, the IPFS CID of the pushed image is printed.
+You can use this CID to pull this image from IPFS.
+
+You can also specify `--estargz` option to enable [eStargz-based lazy pulling](https://github.com/containerd/stargz-snapshotter/blob/v0.10.0/docs/ipfs.md) on IPFS.
+Please see the later section for details.
+
+```console
+> nerdctl push --estargz ipfs://fedora:36
+INFO[0000] pushing image "fedora:36" to IPFS
+INFO[0000] ensuring image contents
+INFO[0011] converted "application/vnd.docker.image.rootfs.diff.tar.gzip" to sha256:cd4be969f12ef45dee7270f3643f796364045edf94cfa9ef6744d91d5cdf2208
+bafkreibp2ncujcia663uum25ustwvmyoguxqyzjnxnlhebhsgk2zowscye
+```
+
+### `nerdctl pull ipfs://<CID>` and `nerdctl run ipfs://<CID>`
+
+You can pull an image from IPFS by specifying `ipfs://<CID>` where `CID` is the CID of the image.
+
+```console
+> nerdctl pull ipfs://bafkreicq4dg6nkef5ju422ptedcwfz6kcvpvvhuqeykfrwq5krazf3muze
+bafkreicq4dg6nkef5ju422ptedcwfz6kcvpvvhuqeykfrwq5krazf3muze:                      resolved       |++++++++++++++++++++++++++++++++++++++|
+index-sha256:28bfa1fc6d491d3bee91bab451cab29c747e72917efacb0adc4e73faffe1f51c:    done           |++++++++++++++++++++++++++++++++++++++|
+manifest-sha256:f6eed19a2880f1000be1d46fb5d114d094a59e350f9d025580f7297c8d9527d5: done           |++++++++++++++++++++++++++++++++++++++|
+config-sha256:ba6acccedd2923aee4c2acc6a23780b14ed4b8a5fa4e14e252a23b846df9b6c1:   done           |++++++++++++++++++++++++++++++++++++++|
+layer-sha256:7b1a6ab2e44dbac178598dabe7cff59bd67233dba0b27e4fbd1f9d4b3c877a54:    done           |++++++++++++++++++++++++++++++++++++++|
+elapsed: 1.2 s                                                                    total:  27.2 M (22.7 MiB/s)
+```
+
+`nerdctl run` also supports the same image name syntax.
+When specified, this command pulls the image from IPFS.
+
+```console
+> nerdctl run --rm -it ipfs://bafkreicq4dg6nkef5ju422ptedcwfz6kcvpvvhuqeykfrwq5krazf3muze echo hello
+hello
+```
+
+You can also push that image to the container registry.
+
+```
+nerdctl tag ipfs://bafkreicq4dg6nkef5ju422ptedcwfz6kcvpvvhuqeykfrwq5krazf3muze ghcr.io/ktock/ubuntu:20.04-ipfs
+nerdctl push ghcr.io/ktock/ubuntu:20.04-ipfs
+```
+
+The pushed image can run on other (IPFS-agnostic) runtimes.
+
+```console
+> docker run --rm -it ghcr.io/ktock/ubuntu:20.04-ipfs echo hello
+hello
+```
+
+:information_source: Note that though the IPFS-enabled image is OCI compatible, some runtimes including [containerd](https://github.com/containerd/containerd/pull/6221) and [podman](https://github.com/containers/image/pull/1403) had bugs and failed to pull that image. Containerd fixed this since v1.5.8, podman fixed this since commit [`b55fb86c28b7d743cf59701332cd78d4294c7c54`](https://github.com/containers/image/commit/b55fb86c28b7d743cf59701332cd78d4294c7c54).
+
+### `nerdctl build` and `localhost:5050/ipfs/<CID>` image reference
+
+You can build images using base images on IPFS.
+BuildKit >= v0.9.3 is needed.
+
+In Dockerfile, instead of `ipfs://` prefix, you need to use the following image reference to point to an image on IPFS.
+
+```
+localhost:5050/ipfs/<CID>
+```
+
+Here, `CID` is the IPFS CID of the image.
+
+:information_source: In the future version of nerdctl and BuildKit, `ipfs://` prefix should be supported in Dockerfile.
+
+Using this image reference, you can build an image on IPFS.
+
+```dockerfile
+FROM localhost:5050/ipfs/bafkreicq4dg6nkef5ju422ptedcwfz6kcvpvvhuqeykfrwq5krazf3muze
+RUN echo hello > /hello
+```
+
+Make sure that `nerdctl ipfs registry serve` is running.
+This allows `nerdctl build` to pull images from IPFS.
+
+```
+$ nerdctl ipfs registry serve &
+```
+
+Then you can build this Dockerfile using `nerdctl build`.
+
+```console
+> nerdctl build -t hello .
+[+] Building 5.3s (6/6) FINISHED
+ => [internal] load build definition from Dockerfile                                                                                              0.0s
+ => => transferring dockerfile: 146B                                                                                                              0.0s
+ => [internal] load .dockerignore                                                                                                                 0.0s
+ => => transferring context: 2B                                                                                                                   0.0s
+ => [internal] load metadata for localhost:5050/ipfs/bafkreicq4dg6nkef5ju422ptedcwfz6kcvpvvhuqeykfrwq5krazf3muze:latest                           0.1s
+ => [1/2] FROM localhost:5050/ipfs/bafkreicq4dg6nkef5ju422ptedcwfz6kcvpvvhuqeykfrwq5krazf3muze@sha256:28bfa1fc6d491d3bee91bab451cab29c747e72917e  3.8s
+ => => resolve localhost:5050/ipfs/bafkreicq4dg6nkef5ju422ptedcwfz6kcvpvvhuqeykfrwq5krazf3muze@sha256:28bfa1fc6d491d3bee91bab451cab29c747e72917e  0.0s
+ => => sha256:7b1a6ab2e44dbac178598dabe7cff59bd67233dba0b27e4fbd1f9d4b3c877a54 28.57MB / 28.57MB                                                  2.1s
+ => => extracting sha256:7b1a6ab2e44dbac178598dabe7cff59bd67233dba0b27e4fbd1f9d4b3c877a54                                                         1.7s
+ => [2/2] RUN echo hello > /hello                                                                                                                 0.6s
+ => exporting to oci image format                                                                                                                 0.6s
+ => => exporting layers                                                                                                                           0.1s
+ => => exporting manifest sha256:b96d490d134221ab121af91a42b13195dd8c5bf941012d7bfe07eabcf5259eda                                                 0.0s
+ => => exporting config sha256:bd706574eab19009585b98826b06e63cf6eacf8d7193504dae75caa760332ca2                                                   0.0s
+ => => sending tarball                                                                                                                            0.5s
+unpacking docker.io/library/hello:latest (sha256:b96d490d134221ab121af91a42b13195dd8c5bf941012d7bfe07eabcf5259eda)...done
+> nerdctl run --rm -it hello cat /hello
+hello
+```
+
+> NOTE: `--ipfs` flag has been removed since v1.2.0. You need to launch the localhost registry by yourself using `nerdctl ipfs registry serve`.
+
+#### Details about `localhost:5050/ipfs/<CID>` and `nerdctl ipfs registry`
+
+As of now, BuildKit doesn't support `ipfs://` prefix so nerdctl achieves builds on IPFS by having a read-only local registry backed by IPFS.
+This registry converts registry API requests to IPFS operations.
+So IPFS-agnostic tools can pull images from IPFS via this registry.
+
+This registry is provided as a subcommand `nerdctl ipfs registry`.
+This command starts the registry backed by the IPFS repo of the current `$IPFS_PATH`
+By default, nerdctl exposes the registry at `localhost:5050` (configurable via flags).
+
+<details>
+<summary>Creating systemd unit file for `nerdctl ipfs registry`</summary>
+
+Optionally you can create systemd unit file of `nerdctl ipfs registry serve`.
+An example systemd unit file for `nerdctl ipfs registry serve` can be the following.
+`nerdctl ipfs registry serve` is aware of environment variables for configuring the behaviour (e.g. listening port) so you can use `EnvironmentFile` for configuring it.
+
+```
+[Unit]
+Description=nerdctl ipfs registry serve
+
+[Service]
+EnvironmentFile-=/run/nerdctl-ipfs-registry-serve/env
+ExecStart=nerdctl ipfs registry serve
+
+[Install]
+WantedBy=default.target
+```
+
+</details>
+
+The following example starts the registry on `localhost:5555` instead of `localhost:5050`.
+
+```
+nerdctl ipfs registry serve --listen-registry=localhost:5555
+```
+
+> NOTE: You'll also need to restart the registry when you change `$IPFS_PATH` to use.
+
+> NOTE: `nerdctl ipfs registry [up|down]` has been removed since v1.2.0. You need to launch the localhost registry using `nerdctl ipfs registry serve` instead.
+
+### Compose on IPFS
+
+`nerdctl compose` supports same image name syntax to pull images from IPFS.
+
+```yaml
+version: "3.8"
+services:
+  ubuntu:
+    image: ipfs://bafkreicq4dg6nkef5ju422ptedcwfz6kcvpvvhuqeykfrwq5krazf3muze
+    command: echo hello
+```
+
+When you build images using base images on IPFS, you can use `localhost:5050/ipfs/<CID>` image reference in Dockerfile as mentioned above.
+
+```
+nerdctl compose up --build
+```
+
+```
+nerdctl compose build
+```
+
+> NOTE: `--ipfs` flag has been removed since v1.2.0. You need to launch the localhost registry by yourself using `nerdctl ipfs registry serve`.
+
+### Encryption
+
+You can distribute [encrypted images](./ocicrypt.md) on IPFS using OCIcrypt.
+Please see [`/docs/ocicrypt.md`](./ocicrypt.md) for details about how to encrypt and decrypt an image.
+
+Same as normal images, the encrypted image can be pushed to IPFS using `ipfs://` prefix.
+
+```console
+> nerdctl image encrypt --recipient=jwe:mypubkey.pem ubuntu:20.04 ubuntu:20.04-encrypted
+sha256:a5c57411f3d11bb058b584934def0710c6c5b5a4a2d7e9b78f5480ecfc450740
+> nerdctl push ipfs://ubuntu:20.04-encrypted
+INFO[0000] pushing image "ubuntu:20.04-encrypted" to IPFS
+INFO[0000] ensuring image contents
+bafkreifajsysbvhtgd7fdgrfesszexdq6v5zbj5y2jnjfwxdjyqws2s3s4
+```
+
+You can pull the encrypted image from IPFS using `ipfs://` prefix and can decrypt it in the same way as described in [`/docs/ocicrypt.md`](./ocicrypt.md).
+
+```console
+> nerdctl pull --unpack=false ipfs://bafkreifajsysbvhtgd7fdgrfesszexdq6v5zbj5y2jnjfwxdjyqws2s3s4
+bafkreifajsysbvhtgd7fdgrfesszexdq6v5zbj5y2jnjfwxdjyqws2s3s4:                      resolved       |++++++++++++++++++++++++++++++++++++++|
+index-sha256:73334fee83139d1d8dbf488b28ad100767c38428b2a62504c758905c475c1d6c:    done           |++++++++++++++++++++++++++++++++++++++|
+manifest-sha256:8855ae825902045ea2b27940634673ba410b61885f91b9f038f6b3303f48727c: done           |++++++++++++++++++++++++++++++++++++++|
+config-sha256:ba6acccedd2923aee4c2acc6a23780b14ed4b8a5fa4e14e252a23b846df9b6c1:   done           |++++++++++++++++++++++++++++++++++++++|
+layer-sha256:e74a9a7749e808e4ad1e90d5a81ce3146ce270de0fbdf22429cd465df8f10a13:    done           |++++++++++++++++++++++++++++++++++++++|
+elapsed: 0.3 s                                                                    total:  22.0 M (73.2 MiB/s)
+> nerdctl image decrypt --key=mykey.pem ipfs://bafkreifajsysbvhtgd7fdgrfesszexdq6v5zbj5y2jnjfwxdjyqws2s3s4 ubuntu:20.04-decrypted
+sha256:b0ccaddb7e7e4e702420de126468eab263eb0f3c25abf0b957ce8adcd1e82105
+> nerdctl run --rm -it ubuntu:20.04-decrypted echo hello
+hello
+```
+
+## Running containers on IPFS with eStargz-based lazy pulling
+
+nerdctl supports running eStargz images on IPFS with lazy pulling using Stargz Snapshotter.
+
+In this configuration, Stargz Snapshotter mounts the eStargz image from IPFS to the container's rootfs using FUSE with lazy pulling support.
+Thus the container can startup without waiting for the entire image contents to be locally available.
+You can see faster container cold-start.
+
+To use this feature, you need to enable Stargz Snapshotter following [`/docs/stargz.md`](./stargz.md).
+You also need to add the following configuration to `config.toml` of Stargz Snapshotter (typically located at `/etc/containerd-stargz-grpc/config.toml`).
+
+```toml
+ipfs = true
+```
+
+You can push an arbitrary image to IPFS with converting it to eStargz using `--estargz` option.
+
+```
+nerdctl push --estargz ipfs://fedora:36
+```
+
+You can pull and run that eStargz image with lazy pulling.
+
+```
+nerdctl run --rm -it ipfs://bafkreibp2ncujcia663uum25ustwvmyoguxqyzjnxnlhebhsgk2zowscye echo hello
+```
+
+- See [the doc in stargz-snapshotter project](https://github.com/containerd/stargz-snapshotter/blob/v0.10.0/docs/ipfs.md) for details about lazy pulling on IPFS.
+- See [`/docs/stargz.md`](./stargz.md) for details about the configuration of nerdctl for Stargz Snapshotter.

virt/src/nerdctl/nerdctldocs/multi-platform.md

@@ -0,0 +1,72 @@
+# Multi-platform
+
+| :zap: Requirement | nerdctl >= 0.13 |
+|-------------------|-----------------|
+
+nerdctl can execute non-native container images using QEMU.
+e.g., ARM on Intel, and vice versa.
+
+## Preparation: Register QEMU to `/proc/sys/fs/binfmt_misc`
+
+```console
+$ sudo systemctl start containerd
+
+$ sudo nerdctl run --privileged --rm tonistiigi/binfmt:master --install all
+
+$ ls -1 /proc/sys/fs/binfmt_misc/qemu*
+/proc/sys/fs/binfmt_misc/qemu-aarch64
+/proc/sys/fs/binfmt_misc/qemu-arm
+/proc/sys/fs/binfmt_misc/qemu-mips64
+/proc/sys/fs/binfmt_misc/qemu-mips64el
+/proc/sys/fs/binfmt_misc/qemu-ppc64le
+/proc/sys/fs/binfmt_misc/qemu-riscv64
+/proc/sys/fs/binfmt_misc/qemu-s390x
+```
+
+The `tonistiigi/binfmt` container must be executed with `--privileged`, and with rootful mode (`sudo`).
+
+This container is not a daemon, and exits immediately after registering QEMU to `/proc/sys/fs/binfmt_misc`.
+Run `ls -1 /proc/sys/fs/binfmt_misc/qemu*` to confirm registration.
+
+See also https://github.com/tonistiigi/binfmt
+
+## Usage
+### Pull & Run
+
+```console
+$ nerdctl pull --platform=arm64,s390x alpine
+
+$ nerdctl run --rm --platform=arm64 alpine uname -a
+Linux e6227935cf12 5.13.0-19-generic #19-Ubuntu SMP Thu Oct 7 21:58:00 UTC 2021 aarch64 Linux
+
+$ nerdctl run --rm --platform=s390x alpine uname -a
+Linux b39da08fbdbf 5.13.0-19-generic #19-Ubuntu SMP Thu Oct 7 21:58:00 UTC 2021 s390x Linux
+```
+
+### Build & Push
+```console
+$ nerdctl build --platform=amd64,arm64 --output type=image,name=example.com/foo:latest,push=true .
+```
+
+Or
+
+```console
+$ nerdctl build --platform=amd64,arm64 -t example.com/foo:latest .
+$ nerdctl push --all-platforms example.com/foo:latest
+```
+
+### Compose
+See [`../examples/compose-multi-platform`](../examples/compose-multi-platform)
+
+## macOS + Lima
+
+As of 2025-03-01, qemu seems to be broken in most Apple-silicon setups.
+This might be due to qemu handling of host vs. guest page sizes
+(unconfirmed, see https://github.com/containerd/nerdctl/issues/3948 for more information).
+
+It should also be noted that Linux 6.11 introduced a change to the VDSO (on ARM)
+that does break Rosetta.
+
+The take-away here is that presumably your only shot at running non-native binaries
+on Apple-silicon is to use an older kernel for your guest (<6.11), typically as shipped by Debian stable,
+and also to use VZ+Rosetta and not qemu (eg: `limactl create --vm-type=vz --rosetta`).

virt/src/nerdctl/nerdctldocs/notation.md

@@ -0,0 +1,81 @@
+# Container Image Sign and Verify with notation tool
+
+| :zap: Requirement | nerdctl >= 1.3.0 |
+|-------------------|------------------|
+
+[notation](https://github.com/notaryproject/notation) is a project to add signatures as standard items in the registry ecosystem, and to build a set of simple tooling for signing and verifying these signatures.
+
+You can enable container signing and verifying features with `push` and `pull` commands of `nerdctl` by using `notation`
+under the hood with make use of flags `--sign` while pushing the container image, and `--verify` while pulling the
+container image.
+
+* Ensure notation executable in your `$PATH`.
+* You can install notation by following this page: https://notaryproject.dev/docs/user-guides/installation/cli/
+* Notation follows the RC of OCI spec v1.1.0. Follow the [instruction](https://notaryproject.dev/docs/quickstart/#create-an-oci-compatible-registry) to set up the local registry with the compliance for testing purpose.
+
+Prepare your environment:
+
+```shell
+# Create a sample Dockerfile
+$ cat <<EOF | tee Dockerfile.dummy
+FROM alpine:latest
+CMD [ "echo", "Hello World" ]
+EOF
+```
+
+> Please do not forget, we won't be validating the base images, which is `alpine:latest` in this case, of the container image that was built on,
+> we'll only verify the container image itself once we sign it.
+
+```shell
+
+# Build the image
+$ nerdctl build -t localhost:5000/my-test -f Dockerfile.dummy .
+
+# Generate a key-pair in notation's key store and trust store
+$ notation cert generate-test --default "test"
+
+# Confirm the signing key is correctly configured. Key name with a * prefix is the default key.
+$ notation key ls
+
+# Confirm the certificate is stored in the trust store.
+$ notation cert ls
+```
+
+Sign the container image while pushing:
+
+```
+# Sign the image and store the signature in the registry
+$ nerdctl push --sign=notation --notation-key-name test localhost:5000/my-test
+```
+
+Verify the container image while pulling:
+
+> REMINDER: Image won't be pulled if there are no matching signatures with the cert in the [trust policy](https://github.com/notaryproject/specifications/blob/main/specs/trust-store-trust-policy.md#trust-policy) in case you passed `--verify` flag.
+
+```shell
+# Create `trustpolicy.json` under $XDG_CONFIG_HOME/notation (XDG_CONFIG_HOME is ~/.config below)
+cat <<EOF | tee ~/.config/notation/trustpolicy.json
+{
+    "version": "1.0",
+    "trustPolicies": [
+        {
+            "name": "test-images",
+            "registryScopes": [ "*" ],
+            "signatureVerification": {
+                "level" : "strict"
+            },
+            "trustStores": [ "ca:test" ],
+            "trustedIdentities": [
+                "*"
+            ]
+        }
+    ]
+}
+EOF
+
+# Verify the image
+$ nerdctl pull --verify=notation localhost:5000/my-test
+
+# You can not verify the image if it is not signed by the cert in the trust policy
+$ nerdctl pull --verify=notation localhost:5000/my-test-bad
+```

virt/src/nerdctl/nerdctldocs/nydus.md

@@ -0,0 +1,37 @@
+# Lazy-pulling using Nydus Snapshotter
+
+| :zap: Requirement | nerdctl >= 0.22 |
+| ----------------- | --------------- |
+
+Nydus snapshotter is a remote snapshotter plugin of containerd for [Nydus](https://github.com/dragonflyoss/image-service) image service which implements a chunk-based content-addressable filesystem that improves the current OCI image specification, in terms of container launching speed, image space, and network bandwidth efficiency, as well as data integrity with several runtime backends: FUSE, virtiofs and in-kernel EROFS (Linux kernel 5.19+).
+
+## Enable lazy-pulling for `nerdctl run`
+
+- Install containerd remote snapshotter plugin (`containerd-nydus-grpc`) from https://github.com/containerd/nydus-snapshotter
+
+- Add the following to `/etc/containerd/config.toml`:
+```toml
+[proxy_plugins]
+  [proxy_plugins.nydus]
+    type = "snapshot"
+    address = "/run/containerd-nydus-grpc/containerd-nydus-grpc.sock"
+```
+
+- Launch `containerd` and `containerd-nydus-grpc`
+
+- Run `nerdctl` with `--snapshotter=nydus`
+```console
+# nerdctl --snapshotter=nydus run -it --rm ghcr.io/dragonflyoss/image-service/ubuntu:nydus-nightly-v5
+```
+
+For the list of pre-converted Nydus images, see https://github.com/orgs/dragonflyoss/packages?page=1&repo_name=image-service
+
+## Build Nydus image using `nerdctl image convert`
+
+Nerdctl supports to convert an OCI image or docker format v2 image to Nydus image by using the `nerdctl image convert` command.
+
+Before the conversion, you should have the `nydus-image` binary installed, which is contained in the ["nydus static package"](https://github.com/dragonflyoss/image-service/releases). You can run the command like `nerdctl image convert --nydus --oci --nydus-builder-path <the_path_of_nydus_image_binary> <source_image> <target_image>` to convert the `<source_image>` to a Nydus image whose tag is `<target_image>`.
+
+By now, the converted Nydus image cannot be run directly. It shoud be unpacked to nydus snapshotter before `nerdctl run`, which is a part of the processing flow of `nerdctl image pull`. So you need to push the converted image to a registry after the conversion and use `nerdctl --snapshotter nydus image pull` to unpack it to the nydus snapshotter before running the image.
+
+Optionally, you can use the nydusify conversion tool to check if the format of the converted Nydus image is valid. For more details about the Nydus image validation and how to build Nydus image, please refer to [nydusify](https://github.com/dragonflyoss/image-service/blob/master/docs/nydusify.md) and [acceld](https://github.com/goharbor/acceleration-service).

virt/src/nerdctl/nerdctldocs/ocicrypt.md

@@ -0,0 +1,90 @@
+# OCIcrypt
+
+| :zap: Requirement | nerdctl >= 0.7 |
+|-------------------|----------------|
+
+nerdctl supports encryption and decryption using [OCIcrypt](https://github.com/containers/ocicrypt)
+(aka [imgcrypt](https://github.com/containerd/imgcrypt) for containerd).
+
+## JWE mode
+
+### Encryption
+
+Use `openssl` to create a private key (`mykey.pem`) and the corresponding public key (`mypubkey.pem`):
+```bash
+openssl genrsa -out mykey.pem
+openssl rsa -in mykey.pem -pubout -out mypubkey.pem
+```
+
+Use `nerdctl image encrypt` to create an encrypted image:
+```bash
+nerdctl image encrypt --recipient=jwe:mypubkey.pem --platform=linux/amd64,linux/arm64 foo example.com/foo:encrypted
+nerdctl push example.com/foo:encrypted
+```
+
+:warning: CAUTION: This command only encrypts image layers, but does NOT encrypt [container configuration such as `Env` and `Cmd`](https://github.com/opencontainers/image-spec/blob/v1.0.1/config.md#example).
+To see non-encrypted information, run `nerdctl image inspect --mode=native --platform=PLATFORM example.com/foo:encrypted` .
+
+### Decryption
+
+#### Configuration
+Put the private key files to `/etc/containerd/ocicrypt/keys` (for rootless `~/.config/containerd/ocicrypt/keys`).
+
+<details>
+<summary>Extra step for containerd 1.4 and older</summary>
+
+<p>
+
+containerd 1.4 and older requires adding the following configuration to `/etc/containerd/config.toml`
+(for rootless `~/.config/containerd/config.toml`):
+
+```toml
+version = 2
+
+[stream_processors]
+  [stream_processors."io.containerd.ocicrypt.decoder.v1.tar.gzip"]
+    accepts = ["application/vnd.oci.image.layer.v1.tar+gzip+encrypted"]
+    returns = "application/vnd.oci.image.layer.v1.tar+gzip"
+    path = "ctd-decoder"
+    args = ["--decryption-keys-path", "/etc/containerd/ocicrypt/keys"]
+  [stream_processors."io.containerd.ocicrypt.decoder.v1.tar"]
+    accepts = ["application/vnd.oci.image.layer.v1.tar+encrypted"]
+    returns = "application/vnd.oci.image.layer.v1.tar"
+    path = "ctd-decoder"
+    args = ["--decryption-keys-path", "/etc/containerd/ocicrypt/keys"]
+
+# NOTE: On rootless, ~/.config/containerd is mounted as /etc/containerd in the namespace.
+```
+
+</p>
+
+</details>
+
+#### Running nerdctl
+
+No flag is needed for running encrypted images with `nerdctl run`, as long as the private key is stored
+in `/etc/containerd/ocicrypt/keys` (for rootless `~/.config/containerd/ocicrypt/keys`).
+
+Just run `nerdctl run example.com/encrypted-image`.
+
+To decrypt an image without running a container, use `nerdctl image decrypt` command:
+```bash
+nerdctl pull --unpack=false example.com/foo:encrypted
+nerdctl image decrypt --key=mykey.pem example.com/foo:encrypted foo:decrypted
+```
+
+## PGP (GPG) mode
+(Undocumented yet)
+
+## PKCS7 mode
+(Undocumented yet)
+
+## PKCS11 mode
+(Undocumented yet)
+
+## More information
+- https://github.com/containerd/imgcrypt (High-level library for containerd, using `containers/ocicrypt`)
+- https://github.com/containers/ocicrypt (Low-level library, used by `containerd/imgcrypt`)
+- https://github.com/opencontainers/image-spec/pull/775 (Proposal for OCI Image Spec)
+- https://github.com/containerd/containerd/blob/main/docs/cri/decryption.md (configuration guide)
+  - The `plugins."io.containerd.grpc.v1.cri"` section does not apply to nerdctl, as nerdctl does not use CRI

virt/src/nerdctl/nerdctldocs/overlaybd.md

@@ -0,0 +1,35 @@
+# Lazy-pulling using OverlayBD Snapshotter
+
+| :zap: Requirement | nerdctl >= 0.15.0 |
+| ----------------- | --------------- |
+
+OverlayBD is a remote container image format base on block-device which is an open-source implementation of paper ["DADI: Block-Level Image Service for Agile and Elastic Application Deployment. USENIX ATC'20".](https://www.usenix.org/conference/atc20/presentation/li-huiba)
+
+See https://github.com/containerd/accelerated-container-image to learn further information.
+
+## Enable lazy-pulling for `nerdctl run`
+
+- Install containerd remote snapshotter plugin (`overlaybd`) from https://github.com/containerd/accelerated-container-image/blob/main/docs/BUILDING.md
+
+- Add the following to `/etc/containerd/config.toml`:
+```toml
+[proxy_plugins]
+  [proxy_plugins.overlaybd]
+    type = "snapshot"
+    address = "/run/overlaybd-snapshotter/overlaybd.sock"
+```
+
+- Launch `containerd` and `overlaybd-snapshotter`
+
+- Run `nerdctl` with `--snapshotter=overlaybd`
+```console
+nerdctl run --net host -it --rm --snapshotter=overlaybd registry.hub.docker.com/overlaybd/redis:6.2.1_obd
+```
+
+For more details about how to build overlaybd image, please refer to [accelerated-container-image](https://github.com/containerd/accelerated-container-image/blob/main/docs/IMAGE_CONVERTOR.md) conversion tool.
+
+## Build OverlayBD image using `nerdctl image convert`
+
+Nerdctl supports to convert an OCI image or docker format v2 image to OverlayBD image by using the `nerdctl image convert` command.
+
+Before the conversion, you should have the `overlaybd-snapshotter` binary installed, which build from [accelerated-container-image](https://github.com/containerd/accelerated-container-image). You can run the command like `nerdctl image convert --overlaybd --oci <source_image> <target_image>` to convert the `<source_image>` to a OverlayBD image whose tag is `<target_image>`.

virt/src/nerdctl/nerdctldocs/registry.md

@@ -0,0 +1,471 @@
+# registry authentication
+
+nerdctl uses `${DOCKER_CONFIG}/config.json` for the authentication with image registries.
+
+`$DOCKER_CONFIG` defaults to `$HOME/.docker`.
+
+## Using insecure registry
+
+If you face `http: server gave HTTP response to HTTPS client` and you cannot configure TLS for the registry, try `--insecure-registry` flag:
+
+e.g.,
+```console
+$ nerdctl --insecure-registry run --rm 192.168.12.34:5000/foo
+```
+
+## Specifying certificates
+
+
+| :zap: Requirement | nerdctl >= 0.16 |
+|-------------------|-----------------|
+
+
+Create `~/.config/containerd/certs.d/<HOST:PORT>/hosts.toml` (or `/etc/containerd/certs.d/...` for rootful) to specify `ca` certificates.
+
+```toml
+# An example of ~/.config/containerd/certs.d/192.168.12.34:5000/hosts.toml
+# (The path is "/etc/containerd/certs.d/192.168.12.34:5000/hosts.toml" for rootful)
+
+server = "https://192.168.12.34:5000"
+[host."https://192.168.12.34:5000"]
+  ca = "/path/to/ca.crt"
+```
+
+See https://github.com/containerd/containerd/blob/main/docs/hosts.md for the syntax of `hosts.toml` .
+
+Docker-style directories are also supported.
+The path is `~/.config/docker/certs.d` for rootless, `/etc/docker/certs.d` for rootful.
+
+## Accessing 127.0.0.1 from rootless nerdctl
+
+Currently, rootless nerdctl cannot pull images from 127.0.0.1, because
+the pull operation occurs in RootlessKit's network namespace.
+
+See https://github.com/containerd/nerdctl/issues/86 for the discussion about workarounds.
+
+- - -
+
+# Using managed registry services
+<!-- START doctoc generated TOC please keep comment here to allow auto update -->
+<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
+
+
+- [Amazon Elastic Container Registry (ECR)](#amazon-elastic-container-registry-ecr)
+  - [Logging in](#logging-in)
+  - [Creating a repo](#creating-a-repo)
+  - [Pushing an image](#pushing-an-image)
+- [Azure Container Registry (ACR)](#azure-container-registry-acr)
+  - [Creating a registry](#creating-a-registry)
+  - [Logging in](#logging-in-1)
+  - [Creating a repo](#creating-a-repo-1)
+  - [Pushing an image](#pushing-an-image-1)
+- [Docker Hub](#docker-hub)
+  - [Logging in](#logging-in-2)
+  - [Creating a repo](#creating-a-repo-2)
+  - [Pushing an image](#pushing-an-image-2)
+- [GitHub Container Registry (GHCR)](#github-container-registry-ghcr)
+  - [Logging in](#logging-in-3)
+  - [Creating a repo](#creating-a-repo-3)
+  - [Pushing an image](#pushing-an-image-3)
+- [GitLab Container Registry](#gitlab-container-registry)
+  - [Logging in](#logging-in-4)
+  - [Creating a repo](#creating-a-repo-4)
+  - [Pushing an image](#pushing-an-image-4)
+- [Google Artifact Registry (pkg.dev)](#google-artifact-registry-pkgdev)
+  - [Logging in](#logging-in-5)
+  - [Creating a repo](#creating-a-repo-5)
+  - [Pushing an image](#pushing-an-image-5)
+- [Google Container Registry (GCR) [DEPRECATED]](#google-container-registry-gcr-deprecated)
+  - [Logging in](#logging-in-6)
+  - [Creating a repo](#creating-a-repo-6)
+  - [Pushing an image](#pushing-an-image-6)
+- [JFrog Artifactory (Cloud/On-Prem)](#jfrog-artifactory-cloudon-prem)
+  - [Logging in](#logging-in-7)
+  - [Creating a repo](#creating-a-repo-7)
+  - [Pushing an image](#pushing-an-image-7)
+- [Quay.io](#quayio)
+  - [Logging in](#logging-in-8)
+  - [Creating a repo](#creating-a-repo-8)
+  - [Pushing an image](#pushing-an-image-8)
+
+<!-- END doctoc generated TOC please keep comment here to allow auto update -->
+
+## Amazon Elastic Container Registry (ECR)
+
+See also https://aws.amazon.com/ecr
+
+### Logging in
+
+```console
+$ aws ecr get-login-password --region <REGION> | nerdctl login --username AWS --password-stdin <AWS_ACCOUNT_ID>.dkr.ecr.<REGION>.amazonaws.com
+Login Succeeded
+```
+
+<details>
+<summary>Alternative method: <code>docker-credential-ecr-login</code></summary>
+
+This methods is more secure but needs an external dependency.
+
+<p>
+
+Install `docker-credential-ecr-login` from https://github.com/awslabs/amazon-ecr-credential-helper , and create the following files:
+
+`~/.docker/config.json`:
+```json
+{
+  "credHelpers": {
+    "public.ecr.aws": "ecr-login",
+    "<AWS_ACCOUNT_ID>.dkr.ecr.<REGION>.amazonaws.com": "ecr-login"
+  }
+}
+```
+
+`~/.aws/credentials`:
+```
+[default]
+aws_access_key_id = ...
+aws_secret_access_key = ...
+```
+
+> **Note**: If you are running nerdctl inside a VM (including Lima, Colima, Rancher Desktop, and WSL2), `docker-credential-ecr-login` has to be installed inside the guest, not the host.
+> Same applies to the path of `~/.docker/config.json` and `~/.aws/credentials`, too.
+
+</p>
+</details>
+
+### Creating a repo
+
+You have to create a repository via https://console.aws.amazon.com/ecr/home/ .
+
+### Pushing an image
+
+```console
+$ nerdctl tag hello-world <AWS_ACCOUNT_ID>.dkr.ecr.<REGION>.amazonaws.com/<REPO>
+$ nerdctl push <AWS_ACCOUNT_ID>.dkr.ecr.<REGION>.amazonaws.com/<REPO>
+```
+
+The pushed image appears in the repository you manually created in the previous step.
+
+## Azure Container Registry (ACR)
+See also https://azure.microsoft.com/en-us/services/container-registry/#overview
+
+### Creating a registry
+
+You have to create a "Container registry" resource manually via [the Azure portal](https://portal.azure.com/).
+
+### Logging in
+```console
+$ nerdctl login -u <USERNAME> <REGISTRY>.azurecr.io
+Enter Password: ********[Enter]
+
+Login Succeeded
+```
+
+The login credentials can be found as "Access keys" in [the Azure portal](https://portal.azure.com/).
+See also https://docs.microsoft.com/en-us/azure/container-registry/container-registry-authentication .
+
+> **Note**: nerdctl prior to v0.16.1 had a bug that required pressing the Enter key twice.
+
+### Creating a repo
+You do not need to create a repo explicitly.
+
+### Pushing an image
+
+```console
+$ nerdctl tag hello-world <REGISTRY>.azurecr.io/hello-world
+$ nerdctl push <REGISTRY>.azurecr.io/hello-world
+```
+
+The pushed image appears in [the Azure portal](https://portal.azure.com/).
+Private as default.
+
+## Docker Hub
+See also https://hub.docker.com/
+
+### Logging in
+```console
+$ nerdctl login -u <USERNAME>
+Enter Password: ********[Enter]
+
+Login Succeeded
+```
+
+> **Note**: nerdctl prior to v0.16.1 had a bug that required pressing the Enter key twice.
+
+### Creating a repo
+You do not need to create a repo explicitly, for public images.
+
+To create a private repo, see https://hub.docker.com/repositories .
+
+### Pushing an image
+
+```console
+$ nerdctl tag hello-world <USERNAME>/hello-world
+$ nerdctl push <USERNAME>/hello-world
+```
+
+The pushed image appears in https://hub.docker.com/repositories .
+**Public** by default.
+
+## GitHub Container Registry (GHCR)
+See also https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry
+
+### Logging in
+
+```console
+$ nerdctl login ghcr.io -u <USERNAME>
+Enter Password: ********[Enter]
+
+Login Succeeded
+```
+
+The `<USERNAME>` is your GitHub username but in lower characters.
+
+The "Password" here is a [GitHub Personal access token](https://github.com/settings/tokens), with `read:packages` and `write:packages` scopes.
+
+> **Note**: nerdctl prior to v0.16.1 had a bug that required pressing the Enter key twice.
+
+### Creating a repo
+You do not need to create a repo explicitly.
+
+### Pushing an image
+
+```console
+$ nerdctl tag hello-world ghcr.io/<USERNAME>/hello-world
+$ nerdctl push ghcr.io/<USERNAME>/hello-world
+```
+
+The pushed image appears in the "Packages" tab of your GitHub profile.
+Private as default.
+
+## GitLab Container Registry
+See also https://docs.gitlab.com/ee/user/packages/container_registry/
+
+### Logging in
+
+```console
+$ nerdctl login registry.gitlab.com -u <USERNAME>
+Enter Password: ********[Enter]
+
+Login Succeeded
+```
+
+The `<USERNAME>` is your GitLab username.
+
+The "Password" here is either a [GitLab Personal access token](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html) or a [GitLab Deploy token](https://docs.gitlab.com/ee/user/project/deploy_tokens/index.html). Both options require minimum scope of `read_registry` for pull access and both `write_registry` and `read_registry` scopes for push access.
+
+> **Note**: nerdctl prior to v0.16.1 had a bug that required pressing the Enter key twice.
+
+### Creating a repo
+Container registries in GitLab are created at the project level. A project in GitLab must exist first before you begin working with its container registry.
+
+### Pushing an image
+
+In this example we have created a GitLab project named `myproject`.
+
+```console
+$ nerdctl tag hello-world registry.gitlab.com/<USERNAME>/myproject/hello-world:latest
+$ nerdctl push registry.gitlab.com/<USERNAME>/myproject/hello-world:latest
+```
+
+The pushed image appears under the "Packages & Registries -> Container Registry" tab of your project on GitLab.
+
+## Google Artifact Registry (pkg.dev)
+See also https://cloud.google.com/artifact-registry/docs/docker/quickstart
+
+### Logging in
+
+Create a [GCP Service Account](https://cloud.google.com/iam/docs/creating-managing-service-accounts#creating), grant
+`Artifact Registry Reader` and `Artifact Registry Writer` roles, and download the key as a JSON file.
+
+Then run the following command:
+
+```console
+$ cat <GCP_SERVICE_ACCOUNT_KEY_JSON> | nerdctl login -u _json_key --password-stdin https://<REGION>-docker.pkg.dev
+WARNING! Your password will be stored unencrypted in /home/<USERNAME>/.docker/config.json.
+Configure a credential helper to remove this warning. See
+https://docs.docker.com/engine/reference/commandline/login/#credentials-store
+
+Login Succeeded
+```
+
+See also https://cloud.google.com/artifact-registry/docs/docker/authentication
+
+
+<details>
+<summary>Alternative method: <code>docker-credential-gcloud</code> (<code>gcloud auth configure-docker</code>)</summary>
+
+This methods is more secure but needs an external dependency.
+
+<p>
+
+Run `gcloud auth configure-docker <REGION>-docker.pkg.dev`, e.g.,
+
+```console
+$ gcloud auth configure-docker asia-northeast1-docker.pkg.dev
+Adding credentials for: asia-northeast1-docker.pkg.dev
+After update, the following will be written to your Docker config file located at [/home/<USERNAME>/.docker/config.json]:
+ {
+  "credHelpers": {
+    "asia-northeast1-docker.pkg.dev": "gcloud"
+  }
+}
+
+Do you want to continue (Y/n)?  y
+
+Docker configuration file updated.
+```
+
+Google Cloud SDK (`gcloud`, `docker-credential-gcloud`) has to be installed, see https://cloud.google.com/sdk/docs/quickstart .
+
+> **Note**: If you are running nerdctl inside a VM (including Lima, Colima, Rancher Desktop, and WSL2), the Google Cloud SDK has to be installed inside the guest, not the host.
+
+</p>
+</details>
+
+### Creating a repo
+
+You have to create a repository via https://console.cloud.google.com/artifacts .
+Choose "Docker" as the repository format.
+
+### Pushing an image
+
+```console
+$ nerdctl tag hello-world <REGION>-docker.pkg.dev/<GCP_PROJECT_ID>/<REPO>/hello-world
+$ nerdctl push <REGION>-docker.pkg.dev/<GCP_PROJECT_ID>/<REPO>/hello-world
+```
+
+The pushed image appears in the repository you manually created in the previous step.
+
+## Google Container Registry (GCR) [DEPRECATED]
+See also https://cloud.google.com/container-registry/docs/advanced-authentication
+
+### Logging in
+
+Create a [GCP Service Account](https://cloud.google.com/iam/docs/creating-managing-service-accounts#creating), grant
+`Storage Object Admin` role, and download the key as a JSON file.
+
+Then run the following command:
+
+```console
+$ cat <GCP_SERVICE_ACCOUNT_KEY_JSON> | nerdctl login -u _json_key --password-stdin https://asia.gcr.io
+WARNING! Your password will be stored unencrypted in /home/<USERNAME>/.docker/config.json.
+Configure a credential helper to remove this warning. See
+https://docs.docker.com/engine/reference/commandline/login/#credentials-store
+
+Login Succeeded
+```
+
+See also https://cloud.google.com/container-registry/docs/advanced-authentication
+
+<details>
+<summary>Alternative method: <code>docker-credential-gcloud</code> (<code>gcloud auth configure-docker</code>)</summary>
+
+This methods is more secure but needs an external dependency.
+
+<p>
+
+```console
+$ gcloud auth configure-docker
+Adding credentials for all GCR repositories.
+WARNING: A long list of credential helpers may cause delays running 'docker build'. We recommend passing the registry name to configure only the registry you are using.
+After update, the following will be written to your Docker config file located at [/home/<USERNAME>/.docker/config.json]:
+ {
+  "credHelpers": {
+    "gcr.io": "gcloud",
+    "us.gcr.io": "gcloud",
+    "eu.gcr.io": "gcloud",
+    "asia.gcr.io": "gcloud",
+    "staging-k8s.gcr.io": "gcloud",
+    "marketplace.gcr.io": "gcloud"
+  }
+}
+
+Do you want to continue (Y/n)?  y
+
+Docker configuration file updated.
+```
+
+Google Cloud SDK (`gcloud`, `docker-credential-gcloud`) has to be installed, see https://cloud.google.com/sdk/docs/quickstart .
+
+> **Note**: If you are running nerdctl inside a VM (including Lima, Colima, Rancher Desktop, and WSL2), the Google Cloud SDK has to be installed inside the guest, not the host.
+
+</p>
+</details>
+
+### Creating a repo
+You do not need to create a repo explicitly.
+
+### Pushing an image
+
+```console
+$ nerdctl tag hello-world asia.gcr.io/<GCP_PROJECT_ID>/hello-world
+$ nerdctl push asia.gcr.io/<GCP_PROJECT_ID>/hello-world
+```
+
+The pushed image appears in https://console.cloud.google.com/gcr/ .
+Private by default.
+
+## JFrog Artifactory (Cloud/On-Prem)
+See also https://www.jfrog.com/confluence/display/JFROG/Getting+Started+with+Artifactory+as+a+Docker+Registry
+
+### Logging in
+```console
+$ nerdctl login <SERVER_NAME>.jfrog.io -u <USERNAME>
+Enter Password: ********[Enter]
+
+Login Succeeded
+```
+
+Login using the default username: admin, and password: password for the on-prem installation, or the credentials provided to you by email for the cloud installation.
+JFrog Platform is integrated with OAuth allowing you to delegate authentication requests to external providers (the provider types supported are Google, OpenID Connect, GitHub Enterprise, and Cloud Foundry UAA)
+
+> **Note**: nerdctl prior to v0.16.1 had a bug that required pressing the Enter key twice.
+
+### Creating a repo
+1. Add local Docker repository
+   1. Add a new Local Repository with the Docker package type via `https://<server-name>.jfrog.io/ui/admin/repositories/local/new`.
+2. Add virtual Docker repository
+   1. Add a new virtual repository with the Docker package type via `https://<server-name>.jfrog.io/ui/admin/repositories/virtual/new`.
+   2. Add the local docker repository you created in Steps 1 (move it from Available Repositories to Selected Repositories using the arrow buttons).
+   3. Set local repository as a default local deployment repository.
+
+### Pushing an image
+```console
+$ nerdctl tag hello-world <SERVER_NAME>.jfrog.io/<VIRTUAL_REPO_NAME>/hello-world
+$ nerdctl push <SERVER_NAME>.jfrog.io/<VIRTUAL_REPO_NAME>/hello-world
+```
+
+The `SERVER_NAME` is the first part of the URL given to you for your environment: `https://<SERVER_NAME>.jfrog.io`
+
+The `VIRTUAL_REPO_NAME` is the name “docker” that you assigned to your virtual repository in 2.i .
+
+The pushed image appears in `https://<SERVER_NAME>.jfrog.io/ui/repos/tree/General/<VIRTUAL_REPO_NAME>` .
+Private by default.
+
+## Quay.io
+See also https://docs.quay.io/solution/getting-started.html
+
+### Logging in
+
+```console
+$ nerdctl login quay.io -u <USERNAME>
+Enter Password: ********[Enter]
+
+Login Succeeded
+```
+
+> **Note**: nerdctl prior to v0.16.1 had a bug that required pressing the Enter key twice.
+
+### Creating a repo
+You do not need to create a repo explicitly.
+
+### Pushing an image
+
+```console
+$ nerdctl tag hello-world quay.io/<USERNAME>/hello-world
+$ nerdctl push quay.io/<USERNAME>/hello-world
+```
+
+The pushed image appears in https://quay.io/repository/ .
+Private as default.

virt/src/nerdctl/nerdctldocs/rootless.md

@@ -0,0 +1,193 @@
+# Rootless mode
+
+See https://rootlesscontaine.rs/getting-started/common/ for the prerequisites.
+
+## Daemon (containerd)
+
+Use [`containerd-rootless-setuptool.sh`](../extras/rootless) to set up rootless containerd.
+
+```console
+$ containerd-rootless-setuptool.sh install
+[INFO] Checking RootlessKit functionality
+[INFO] Checking cgroup v2
+[INFO] Checking overlayfs
+[INFO] Creating /home/testuser/.config/systemd/user/containerd.service
+...
+[INFO] Installed containerd.service successfully.
+[INFO] To control containerd.service, run: `systemctl --user (start|stop|restart) containerd.service`
+[INFO] To run containerd.service on system startup, run: `sudo loginctl enable-linger testuser`
+
+[INFO] Use `nerdctl` to connect to the rootless containerd.
+[INFO] You do NOT need to specify $CONTAINERD_ADDRESS explicitly.
+```
+
+The usage of `containerd-rootless-setuptool.sh` is almost same as [`dockerd-rootless-setuptool.sh`](https://rootlesscontaine.rs/getting-started/docker/) .
+
+Resource limitation flags such as `nerdctl run --memory` require systemd and cgroup v2: https://rootlesscontaine.rs/getting-started/common/cgroup2/
+
+#### AppArmor Profile for Ubuntu 24.04+
+
+Configuring AppArmor is needed only on Ubuntu 24.04+, with RootlessKit installed under a non-standard path: https://rootlesscontaine.rs/getting-started/common/apparmor/
+
+## Client (nerdctl)
+
+Just execute `nerdctl`. No need to specify the socket address manually.
+
+```console
+$ nerdctl run -it --rm alpine
+```
+
+Depending on your kernel version, you may need to enable FUSE-OverlayFS or set `export CONTAINERD_SNAPSHOTTER=native`.
+(See below.)
+
+## Add-ons
+### BuildKit
+To enable BuildKit, run the following command:
+
+```console
+$ containerd-rootless-setuptool.sh install-buildkit
+```
+
+## Snapshotters
+
+### OverlayFS
+
+The default `overlayfs` snapshotter only works on the following hosts:
+- Any distro, with kernel >= 5.13
+- Non-SELinux distro, with kernel >= 5.11
+- Ubuntu since 2015
+
+For other hosts, [`fuse-overlayfs` snapshotter](https://github.com/containerd/fuse-overlayfs-snapshotter) needs to be used instead.
+
+### FUSE-OverlayFS
+
+To enable `fuse-overlayfs` snapshotter, run the following command:
+```console
+$ containerd-rootless-setuptool.sh install-fuse-overlayfs
+```
+
+Then, add the following config to `~/.config/containerd/config.toml`, and run `systemctl --user restart containerd.service`:
+```toml
+[proxy_plugins]
+  [proxy_plugins."fuse-overlayfs"]
+      type = "snapshot"
+# NOTE: replace "1000" with your actual UID
+      address = "/run/user/1000/containerd-fuse-overlayfs.sock"
+```
+
+The snapshotter can be specified as `$CONTAINERD_SNAPSHOTTER`.
+```console
+$ export CONTAINERD_SNAPSHOTTER=fuse-overlayfs
+$ nerdctl run -it --rm alpine
+```
+
+If `fuse-overlayfs` does not work, try `export CONTAINERD_SNAPSHOTTER=native`.
+
+### Stargz Snapshotter
+[Stargz Snapshotter](./stargz.md) enables lazy-pulling of images.
+
+To enable Stargz snapshotter, run the following command:
+```console
+$ containerd-rootless-setuptool.sh install-stargz
+```
+
+Then, add the following config to `~/.config/containerd/config.toml` and run `systemctl --user restart containerd.service`:
+```toml
+[proxy_plugins]
+  [proxy_plugins."stargz"]
+      type = "snapshot"
+# NOTE: replace "1000" with your actual UID
+      address = "/run/user/1000/containerd-stargz-grpc/containerd-stargz-grpc.sock"
+```
+
+The snapshotter can be specified as `$CONTAINERD_SNAPSHOTTER`.
+```console
+$ export CONTAINERD_SNAPSHOTTER=stargz
+$ nerdctl run -it --rm ghcr.io/stargz-containers/alpine:3.10.2-esgz
+```
+
+See https://github.com/containerd/stargz-snapshotter/blob/main/docs/pre-converted-images.md for the image list.
+
+## bypass4netns
+| :zap: Requirement | nerdctl >= 0.17 |
+|-------------------|-----------------|
+
+
+[bypass4netns](https://github.com/rootless-containers/bypass4netns) is an accelerator for rootless networking.
+
+This improves **outgoing or incoming (with --publish option) networking performance.**
+
+The performance benchmark with iperf3 on Ubuntu 21.10 on Hyper-V VM is shown below.
+| iperf3 benchmark  | without bypass4netns | with bypass4netns |
+| ----------------- | -------------------- | ----------------- |
+| container -> host | 0.398 Gbps           | **42.2 Gbps**         |
+| host -> container | 20.6 Gbps             | **47.4 Gbps**         |
+
+This benchmark can be reproduced with [https://github.com/rootless-containers/bypass4netns/blob/f009d96139e9e38ce69a2ea8a9a746349bad273c/Vagrantfile](https://github.com/rootless-containers/bypass4netns/blob/f009d96139e9e38ce69a2ea8a9a746349bad273c/Vagrantfile)
+
+Acceleration with bypass4netns is available with:
+- `--annotation nerdctl/bypass4netns=true` (for nerdctl v2.0 and later)
+- `--label nerdctl/bypass4netns=true` (deprecated form, used in nerdctl prior to v2.0).
+
+You also need to have `bypass4netnsd` (bypass4netns daemon) to be running.
+Example
+```console
+$ containerd-rootless-setuptool.sh install-bypass4netnsd
+$ nerdctl run -it --rm -p 8080:80 --annotation nerdctl/bypass4netns=true alpine
+```
+
+More detail is available at [https://github.com/rootless-containers/bypass4netns/blob/master/README.md](https://github.com/rootless-containers/bypass4netns/blob/master/README.md)
+
+## Configuring RootlessKit
+
+Rootless containerd recognizes the following environment variables to configure the behavior of [RootlessKit](https://github.com/rootless-containers/rootlesskit):
+
+* `CONTAINERD_ROOTLESS_ROOTLESSKIT_STATE_DIR=DIR`: the rootlesskit state dir. Defaults to `$XDG_RUNTIME_DIR/containerd-rootless`.
+* `CONTAINERD_ROOTLESS_ROOTLESSKIT_NET=(slirp4netns|vpnkit|lxc-user-nic)`: the rootlesskit network driver. Defaults to "slirp4netns" if slirp4netns (>= v0.4.0) is installed. Otherwise defaults to "vpnkit".
+* `CONTAINERD_ROOTLESS_ROOTLESSKIT_MTU=NUM`: the MTU value for the rootlesskit network driver. Defaults to 65520 for slirp4netns, 1500 for other drivers.
+* `CONTAINERD_ROOTLESS_ROOTLESSKIT_PORT_DRIVER=(builtin|slirp4netns)`: the rootlesskit port driver. Defaults to "builtin" (this driver does not propagate the container's source IP address and always uses 127.0.0.1. Please check [Port Drivers](https://github.com/rootless-containers/rootlesskit/blob/master/docs/port.md#port-drivers) for more details).
+* `CONTAINERD_ROOTLESS_ROOTLESSKIT_SLIRP4NETNS_SANDBOX=(auto|true|false)`: whether to protect slirp4netns with a dedicated mount namespace. Defaults to "auto".
+* `CONTAINERD_ROOTLESS_ROOTLESSKIT_SLIRP4NETNS_SECCOMP=(auto|true|false)`: whether to protect slirp4netns with seccomp. Defaults to "auto".
+* `CONTAINERD_ROOTLESS_ROOTLESSKIT_DETACH_NETNS=(auto|true|false)`: whether to launch rootlesskit with the "detach-netns" mode.
+  Defaults to "auto", which is resolved to "true" if RootlessKit >= 2.0 is installed.
+  The "detached-netns" mode accelerates `nerdctl (pull|push|build)` and enables `nerdctl run --net=host`,
+  however, there is a relatively minor drawback with BuildKit prior to v0.13:
+  the host loopback IP address (127.0.0.1) and abstract sockets are exposed to Dockerfile's "RUN" instructions during `nerdctl build` (not `nerdctl run`).
+  The drawback is fixed in BuildKit v0.13. Upgrading from a prior version of BuildKit needs removing the old systemd unit:
+  `containerd-rootless-setuptool.sh uninstall-buildkit && rm -f ~/.config/buildkit/buildkitd.toml`
+
+To set these variables, create `~/.config/systemd/user/containerd.service.d/override.conf` as follows:
+```ini
+[Service]
+Environment=CONTAINERD_ROOTLESS_ROOTLESSKIT_DETACH_NETNS="false"
+```
+
+And then run the following commands:
+```bash
+systemctl --user daemon-reload
+systemctl --user restart containerd
+```
+
+## Troubleshooting
+
+### Hint to Fedora users
+- If SELinux is enabled on your host and your kernel is older than 5.13, you need to use [`fuse-overlayfs` instead of `overlayfs`](#fuse-overlayfs).
+
+## Rootlesskit Network Design
+
+In `detach-netns` mode:
+
+- Network namespace is detached and stored in `$ROOTLESSKIT_STATE_DIR/netns`.
+- The child command executes within the host's network namespace, allowing actions like `pull` and `push` to happen in the host network namespace.
+- For creating and configuring the container's network namespace, the child command switches temporarily to the relevant namespace located in `$ROOTLESSKIT_STATE_DIR/netns`. This ensures necessary network setup while maintaining isolation in the host namespace.
+
+![rootlessKit-network-design.png](images/rootlessKit-network-design.png)
+
+- Rootlesskit Parent NetNS and Child NetNS are already configured by the startup script [containerd-rootless.sh](https://github.com/containerd/nerdctl/blob/main/extras/rootless/containerd-rootless.sh)
+- Rootlesskit Parent NetNS is the host network namespace
+- step1: `nerdctl` calls `containerd` in the host network namespace.
+- step2: `containerd` calls `runc` in the host network namespace.
+- step3: `runc` creates container with dedicated namespaces (e.g network ns) in the Parent netns.
+- step4: `runc` nsenter Rootlesskit Child NetNS before triggering nerdctl ocihook.
+- step5: `nerdctl` ocihook module leverages CNI.
+- step6: CNI configures container network namespace: create network interfaces `eth0` -> `veth0` -> `nerdctl0`.

virt/src/nerdctl/nerdctldocs/soci.md

@@ -0,0 +1,47 @@
+# Lazy-pulling using SOCI Snapshotter
+
+SOCI Snapshotter is a containerd snapshotter plugin. It enables standard OCI images to be lazily loaded without requiring a build-time conversion step. "SOCI" is short for "Seekable OCI", and is pronounced "so-CHEE".
+
+See https://github.com/awslabs/soci-snapshotter to learn further information.
+
+## Prerequisites
+
+- Install containerd remote snapshotter plugin (`soci-snapshotter-grpc`) from https://github.com/awslabs/soci-snapshotter/blob/main/docs/getting-started.md
+
+- Add the following to `/etc/containerd/config.toml`:
+```toml
+[proxy_plugins]
+  [proxy_plugins.soci]
+    type = "snapshot"
+    address = "/run/soci-snapshotter-grpc/soci-snapshotter-grpc.sock"
+```
+
+- Launch `containerd` and `soci-snapshotter-grpc`
+
+## Enable SOCI for `nerdctl run` and `nerdctl pull`
+
+| :zap: Requirement | nerdctl >= 1.5.0 |
+| ----------------- | ---------------- |
+
+- Run `nerdctl` with `--snapshotter=soci`
+```console
+nerdctl run -it --rm --snapshotter=soci public.ecr.aws/soci-workshop-examples/ffmpeg:latest
+```
+
+- You can also only pull the image with SOCI without running the container.
+```console
+nerdctl pull --snapshotter=soci public.ecr.aws/soci-workshop-examples/ffmpeg:latest
+```
+
+For images that already have SOCI indices, see https://gallery.ecr.aws/soci-workshop-examples
+
+## Enable SOCI for `nerdctl push`
+
+| :zap: Requirement | nerdctl >= 1.6.0 |
+| ----------------- | ---------------- |
+
+- Push the image with SOCI index. Adding `--snapshotter=soci` arg to `nerdctl pull`, `nerdctl` will create the SOCI index and push the index to same destination as the image.
+```console
+nerdctl push --snapshotter=soci --soci-span-size=2097152 --soci-min-layer-size=20971520 public.ecr.aws/my-registry/my-repo:latest
+```
+--soci-span-size and --soci-min-layer-size are two properties to customize the SOCI index. See [Command Reference](https://github.com/containerd/nerdctl/blob/377b2077bb616194a8ef1e19ccde32aa1ffd6c84/docs/command-reference.md?plain=1#L773) for further details.

virt/src/nerdctl/nerdctldocs/stargz.md

@@ -0,0 +1,187 @@
+# Lazy-pulling using Stargz Snapshotter
+
+| :zap: Requirement | nerdctl >= 0.0.1 |
+|-------------------|------------------|
+
+Lazy-pulling is a technique to running containers before completion of pulling the images.
+
+See https://github.com/containerd/stargz-snapshotter to learn further information.
+
+[![asciicast](https://asciinema.org/a/378377.svg)](https://asciinema.org/a/378377)
+
+## Enable lazy-pulling for `nerdctl run`
+
+> **NOTE**
+> For rootless installation, see [`rootless.md`](./rootless.md#stargz-snapshotter)
+
+- Install Stargz plugin (`containerd-stargz-grpc`) from https://github.com/containerd/stargz-snapshotter 
+
+- Add the following to `/etc/containerd/config.toml`:
+```toml
+[proxy_plugins]
+  [proxy_plugins.stargz]                                                                                  
+    type = "snapshot"
+    address = "/run/containerd-stargz-grpc/containerd-stargz-grpc.sock"
+```
+
+- Launch `containerd` and `containerd-stargz-grpc`
+
+- Run `nerdctl` with `--snapshotter=stargz`
+```console
+# nerdctl --snapshotter=stargz run -it --rm ghcr.io/stargz-containers/fedora:30-esgz
+```
+
+For the list of pre-converted Stargz images, see https://github.com/containerd/stargz-snapshotter/blob/main/docs/pre-converted-images.md
+
+### Benchmark result (Dec 9, 2020)
+For running `python3 -c print("hi")`, eStargz with Stargz Snapshotter is 3-4 times faster than the legacy OCI with overlayfs snapshotter.
+
+Legacy OCI with overlayfs snapshotter:
+```console
+# time nerdctl --snapshotter=overlayfs run -it --rm ghcr.io/stargz-containers/python:3.7-org python3 -c 'print("hi")'
+ghcr.io/stargz-containers/python:3.7-org:                                         resolved       |++++++++++++++++++++++++++++++++++++++|
+index-sha256:6008006c63b0a6043a11ac151cee572e0c8676b4ba3130ff23deff5f5d711237:    done           |++++++++++++++++++++++++++++++++++++++|
+manifest-sha256:48eafda05f80010a6677294473d51a530e8f15375b6447195b6fb04dc2a30ce7: done           |++++++++++++++++++++++++++++++++++++++|
+layer-sha256:f860607a6cd9751ac8db2f33cbc3ce1777a44eb3c04853e116763441a304fbf6:    done           |++++++++++++++++++++++++++++++++++++++|
+layer-sha256:96b2c1e36db5f5910f58da2ca4f9311b0690810c7107fb055ee1541498b5061f:    done           |++++++++++++++++++++++++++++++++++++++|
+layer-sha256:c495e8de12d26c9843a7a2bf8c68de1e5652e66d80d9bc869279f9af6f86736a:    done           |++++++++++++++++++++++++++++++++++++++|
+layer-sha256:33382189822a108b249cf3ccd234d04c3a8dfe7d593df19c751dcfab3675d5f2:    done           |++++++++++++++++++++++++++++++++++++++|
+config-sha256:94c9a318e47ab8a318582e2712bb495f92f17a7c1e50f13cc8a3e362c1b09290:   done           |++++++++++++++++++++++++++++++++++++++|
+layer-sha256:6eaa0b6b8562fb4a02e140ae53b3910fc4d0db6e68660390eaef993f42e21102:    done           |++++++++++++++++++++++++++++++++++++++|
+layer-sha256:adbdcbacafe93bf0791e49c8d3689bb78d9e60d02d384d4e14433aedae39f52c:    done           |++++++++++++++++++++++++++++++++++++++|
+layer-sha256:756975cb9c7e7933d824af9319b512dd72a50894232761d06ef3be59981df838:    done           |++++++++++++++++++++++++++++++++++++++|
+layer-sha256:d77915b4e630d47296770ce4cf481894885978072432456615172af463433cc5:    done           |++++++++++++++++++++++++++++++++++++++|
+layer-sha256:5f37a0a41b6b03489dd7de0aa2a79e369fd8b219bbc36b52f3f9790dc128e74b:    done           |++++++++++++++++++++++++++++++++++++++|
+elapsed: 41.9s                                                                    total:  321.3  (7.7 MiB/s)                                       
+hi                                                                                                        
+                                                     
+real    0m51.754s                                                                                         
+user    0m2.687s
+sys     0m5.533s 
+```
+
+eStargz with Stargz Snapshotter:
+```console
+# time nerdctl --snapshotter=stargz run -it --rm ghcr.io/stargz-containers/python:3.7-esgz python3 -c 'print("hi")'
+fetching sha256:2ea0dd96... application/vnd.oci.image.index.v1+json
+fetching sha256:9612ff73... application/vnd.docker.distribution.manifest.v2+json
+fetching sha256:34e5920e... application/vnd.docker.container.image.v1+json
+hi
+
+real    0m13.589s
+user    0m0.132s
+sys     0m0.158s
+```
+
+## Enable lazy-pulling for pulling base images during `nerdctl build`
+
+- Launch `buildkitd` with `--oci-worker-snapshotter=stargz` (or `--containerd-worker-snapshotter=stargz` if you use containerd worker)
+- Launch `nerdctl build`. No need to specify `--snapshotter` for `nerdctl`.
+
+## Building stargz images using `nerdctl build`
+
+```console
+$ nerdctl build -t example.com/foo .
+$ nerdctl image convert --estargz --oci example.com/foo example.com/foo:estargz
+$ nerdctl push example.com/foo:estargz
+```
+
+NOTE: `--estargz` should be specified in conjunction with `--oci`
+
+Stargz Snapshotter is not needed for building stargz images.
+
+## Tips for image conversion
+
+### Tips 1: Creating smaller eStargz images
+
+`nerdctl image convert` allows the following flags for optionally creating a smaller eStargz image.
+The result image requires stargz-snapshotter >= v0.13.0 for lazy pulling.
+
+- `--estargz-min-chunk-size`: The minimal number of bytes of data must be written in one gzip stream. If it's > 0, multiple files and chunks can be written into one gzip stream. Smaller number of gzip header and smaller size of the result blob can be expected. `--estargz-min-chunk-size=0` produces normal eStargz.
+
+- `--estargz-external-toc`: Separate TOC JSON metadata into another image (called "TOC image"). The result eStargz doesn't contain TOC so we can expect a smaller size than normal eStargz. This is an [experimental](./experimental.md) feature.
+
+#### `--estargz-min-chunk-size` usage
+
+conversion:
+
+```console
+# nerdctl image convert --oci --estargz --estargz-min-chunk-size=50000 ghcr.io/stargz-containers/ubuntu:22.04 registry2:5000/ubuntu:22.04-chunk50000
+# nerdctl image ls
+REPOSITORY                          TAG                 IMAGE ID        CREATED           PLATFORM       SIZE        BLOB SIZE
+ghcr.io/stargz-containers/ubuntu    22.04               20fa2d7bb4de    14 seconds ago    linux/amd64    83.4 MiB    29.0 MiB
+registry2:5000/ubuntu               22.04-chunk50000    562e09e1b3c1    2 seconds ago     linux/amd64    0.0 B       29.2 MiB
+# nerdctl push --insecure-registry registry2:5000/ubuntu:22.04-chunk50000
+```
+
+Pull it lazily:
+
+```console
+# nerdctl pull --snapshotter=stargz --insecure-registry registry2:5000/ubuntu:22.04-chunk50000
+# mount | grep "stargz on"
+stargz on /var/lib/containerd-stargz-grpc/snapshotter/snapshots/1/fs type fuse.rawBridge (rw,nodev,relatime,user_id=0,group_id=0,allow_other)
+```
+
+#### `--estargz-external-toc` usage
+
+convert:
+
+```console
+# nerdctl image convert --oci --estargz --estargz-external-toc ghcr.io/stargz-containers/ubuntu:22.04 registry2:5000/ubuntu:22.04-ex
+INFO[0005] Extra image(0) registry2:5000/ubuntu:22.04-ex-esgztoc
+sha256:3059dd5d9c404344e0b7c43d9782de8cae908531897262b7772103a0b585bbee
+# nerdctl images
+REPOSITORY                          TAG                 IMAGE ID        CREATED           PLATFORM       SIZE        BLOB SIZE
+ghcr.io/stargz-containers/ubuntu    22.04                20fa2d7bb4de    9 seconds ago    linux/amd64    83.4 MiB    29.0 MiB
+registry2:5000/ubuntu               22.04-ex             3059dd5d9c40    1 second ago     linux/amd64    0.0 B       30.8 MiB
+registry2:5000/ubuntu               22.04-ex-esgztoc     18c042b6eb8b    1 second ago     linux          0.0 B       151.3 KiB
+```
+
+Then push eStargz(`registry2:5000/ubuntu:22.04-ex`) and TOC image(`registry2:5000/ubuntu:22.04-ex-esgztoc`) to the same registry (`registry2` is used in this example but you can use arbitrary registries):
+
+```console
+# nerdctl push --insecure-registry registry2:5000/ubuntu:22.04-ex
+# nerdctl push --insecure-registry registry2:5000/ubuntu:22.04-ex-esgztoc
+```
+
+Pull it lazily:
+
+```console
+# nerdctl pull --insecure-registry --snapshotter=stargz registry2:5000/ubuntu:22.04-ex
+```
+
+Stargz Snapshotter automatically refers to the TOC image on the same registry.
+
+##### optional `--estargz-keep-diff-id` flag for conversion without changing layer diffID
+
+`nerdctl image convert` supports optional flag `--estargz-keep-diff-id` specified with `--estargz-external-toc`.
+This converts an image to eStargz without changing the diffID (uncompressed digest) so even eStargz-agnostic gzip decompressor (e.g. gunzip) can restore the original tar blob.
+
+```console
+# nerdctl image convert --oci --estargz --estargz-external-toc --estargz-keep-diff-id ghcr.io/stargz-containers/ubuntu:22.04 registry2:5000/ubuntu:22.04-ex-keepdiff
+# nerdctl push --insecure-registry registry2:5000/ubuntu:22.04-ex-keepdiff
+# nerdctl push --insecure-registry registry2:5000/ubuntu:22.04-ex-keepdiff-esgztoc
+# crane --insecure blob registry2:5000/ubuntu:22.04-ex-keepdiff@sha256:2dc39ba059dcd42ade30aae30147b5692777ba9ff0779a62ad93a74de02e3e1f | jq -r '.rootfs.diff_ids[]'
+sha256:7f5cbd8cc787c8d628630756bcc7240e6c96b876c2882e6fc980a8b60cdfa274
+# crane blob ghcr.io/stargz-containers/ubuntu:22.04@sha256:2dc39ba059dcd42ade30aae30147b5692777ba9ff0779a62ad93a74de02e3e1f | jq -r '.rootfs.diff_ids[]'
+sha256:7f5cbd8cc787c8d628630756bcc7240e6c96b876c2882e6fc980a8b60cdfa274
+```
+
+### Tips 2: Using zstd instead of gzip (a.k.a. zstd:chunked)
+
+You can use zstd compression with lazy pulling support (a.k.a zstd:chunked) instead of gzip.
+
+- Pros
+  - [Faster](https://github.com/facebook/zstd/tree/v1.5.2#benchmarks) compression/decompression.
+- Cons
+  - Old tools might not support. And unsupported by some tools yet.
+    - zstd supported by OCI Image Specification is still under rc (2022/11). will be added to [v1.1.0](https://github.com/opencontainers/image-spec/commit/1a29e8675a64a5cdd2d93b6fa879a82d9a4d926a).
+    - zstd supported by [docker >=v23.0.0](https://github.com/moby/moby/releases/tag/v23.0.0).
+    - zstd supported by [containerd >= v1.5](https://github.com/containerd/containerd/releases/tag/v1.5.0).
+  - `min-chunk-size`, `external-toc` (described in Tips 1) are unsupported yet.
+
+```console
+$ nerdctl build -t example.com/foo .
+$ nerdctl image convert --zstdchunked --oci example.com/foo example.com/foo:zstdchunked
+$ nerdctl push example.com/foo:zstdchunked
+```