...

2025-06-15 21:24:50 +02:00 · 2025-06-15 20:55:06 +02:00 · 2025-06-15 20:41:33 +02:00 · 2025-06-15 19:57:32 +02:00 · 2025-06-15 19:42:01 +02:00 · 2025-06-15 19:31:47 +02:00
14 changed files with 3522 additions and 2 deletions
--- a/README.md
+++ b/README.md
@@ -6,6 +6,7 @@
 ```bash
 # Download and execute in one command
 apt update -y && apt install curl -y
-curl -s https://git.threefold.info/ourworld_web2/itenv_tools/raw/branch/main/tools/git_checkout.sh | bash
+curl -s https://git.threefold.info/ourworld_web/itenv_tools/raw/branch/main/tools/git_checkout.sh > /root/git_checkout.sh
+bash /root/git_checkout.sh
 ```

--- a/tools/VM_README.md
+++ b/tools/VM_README.md
@@ -0,0 +1,295 @@
+# Ubuntu VM Management with Cloud Hypervisor
+
+This directory contains scripts for creating and managing Ubuntu VMs using Cloud Hypervisor with btrfs thin provisioning.
+
+## Scripts Overview
+
+1. **`ubuntu_vm_start.sh`** - Creates and starts new VMs
+2. **`ubuntu_vm_manage.sh`** - Manages existing VMs (list, status, console, stop, etc.)
+3. **`setup_vm_network.sh`** - Sets up networking for SSH access to VMs
+
+## Prerequisites
+
+- Root access
+- Btrfs filesystem (script will detect and guide setup)
+- Cloud Hypervisor installed
+- Basic networking tools
+
+## Quick Start
+
+### 1. Set up networking (one-time setup)
+```bash
+sudo ./setup_vm_network.sh
+```
+
+### 2. Create and start a VM
+```bash
+sudo ./ubuntu_vm_start.sh my-vm 2048 2
+```
+This creates a VM named "my-vm" with 2GB RAM and 2 CPU cores.
+
+### 3. List all VMs
+```bash
+sudo ./ubuntu_vm_manage.sh list
+```
+
+### 4. Connect to VM console
+```bash
+sudo ./ubuntu_vm_manage.sh console my-vm
+```
+
+## Detailed Usage
+
+### Creating VMs
+
+```bash
+sudo ./ubuntu_vm_start.sh <vm_name> <memory_mb> <cpu_cores>
+```
+
+**Examples:**
+```bash
+# Create a small VM
+sudo ./ubuntu_vm_start.sh test-vm 1024 1
+
+# Create a larger VM
+sudo ./ubuntu_vm_start.sh dev-vm 4096 4
+```
+
+**What happens:**
+- Downloads Ubuntu 24.04 cloud image (first time only)
+- Creates btrfs subvolumes for thin provisioning
+- Sets up cloud-init with default user
+- Creates network interfaces
+- Starts the VM with Cloud Hypervisor
+
+### Managing VMs
+
+```bash
+sudo ./ubuntu_vm_manage.sh <command> [vm_name]
+```
+
+**Available commands:**
+
+| Command | Description | Example |
+|---------|-------------|---------|
+| `list` | Show all VMs and their status | `./ubuntu_vm_manage.sh list` |
+| `status <vm>` | Detailed status of specific VM | `./ubuntu_vm_manage.sh status my-vm` |
+| `console <vm>` | Connect to VM console | `./ubuntu_vm_manage.sh console my-vm` |
+| `stop <vm>` | Stop a running VM | `./ubuntu_vm_manage.sh stop my-vm` |
+| `delete <vm>` | Delete VM completely | `./ubuntu_vm_manage.sh delete my-vm` |
+| `ssh <vm>` | Show SSH connection info | `./ubuntu_vm_manage.sh ssh my-vm` |
+
+### Accessing VMs
+
+#### Method 1: Console Access (Always Available)
+```bash
+sudo ./ubuntu_vm_manage.sh console my-vm
+```
+- Direct serial console access
+- No network required
+- Press `Ctrl+A` then `X` to exit
+- Default login: `ubuntu` / `ubuntu`
+
+#### Method 2: SSH Access (Requires Network Setup)
+1. Set up networking first:
+   ```bash
+   sudo ./setup_vm_network.sh
+   ```
+
+2. Find VM IP address:
+   ```bash
+   vm-ips
+   # or
+   arp -a | grep 192.168.100
+   ```
+
+3. SSH to the VM:
+   ```bash
+   ssh ubuntu@192.168.100.X
+   ```
+   Default password: `ubuntu`
+
+## Network Configuration
+
+The `setup_vm_network.sh` script configures:
+
+- **Bridge interface**: `br0` with IP `192.168.100.1/24`
+- **DHCP server**: Assigns IPs `192.168.100.10-100` to VMs
+- **NAT**: Enables internet access for VMs
+- **DNS**: Uses Google DNS (8.8.8.8, 8.8.4.4)
+
+### Network Troubleshooting
+
+1. **Check bridge status:**
+   ```bash
+   ip addr show br0
+   ```
+
+2. **Check VM IP assignments:**
+   ```bash
+   vm-ips
+   ```
+
+3. **Check DHCP leases:**
+   ```bash
+   cat /var/lib/dhcp/dhcpd.leases
+   ```
+
+4. **Restart networking:**
+   ```bash
+   systemctl restart dnsmasq
+   ```
+
+## Storage (Btrfs)
+
+VMs use btrfs subvolumes for efficient storage:
+
+- **Base image**: Shared read-only Ubuntu image
+- **VM snapshots**: Copy-on-write clones for each VM
+- **Thin provisioning**: Only changed blocks use disk space
+
+### Storage locations:
+- Base images: `/var/lib/vms/base/`
+- VM instances: `/var/lib/vms/vms/<vm-name>/`
+
+### Storage commands:
+```bash
+# List subvolumes
+btrfs subvolume list /var/lib/vms
+
+# Show space usage
+btrfs filesystem usage /var/lib/vms
+
+# Show individual VM sizes
+du -sh /var/lib/vms/vms/*
+```
+
+## VM Configuration
+
+### Default VM Setup:
+- **OS**: Ubuntu 24.04 LTS
+- **User**: `ubuntu` (password: `ubuntu`)
+- **Sudo**: Passwordless sudo enabled
+- **SSH**: Enabled with password authentication
+- **Packages**: curl, wget, git, htop, vim pre-installed
+
+### Customizing VMs:
+Edit the cloud-init configuration in `ubuntu_vm_start.sh` to:
+- Add SSH keys
+- Install additional packages
+- Set custom passwords
+- Configure services
+
+## Troubleshooting
+
+### VM won't start:
+1. Check if Cloud Hypervisor is installed:
+   ```bash
+   which cloud-hypervisor
+   ```
+
+2. Check btrfs filesystem:
+   ```bash
+   btrfs filesystem show /var/lib/vms
+   ```
+
+3. Check available resources:
+   ```bash
+   free -h
+   nproc
+   ```
+
+### Can't connect to VM:
+1. Verify VM is running:
+   ```bash
+   ./ubuntu_vm_manage.sh status my-vm
+   ```
+
+2. Try console access first:
+   ```bash
+   ./ubuntu_vm_manage.sh console my-vm
+   ```
+
+3. Check network setup:
+   ```bash
+   ip addr show br0
+   vm-ips
+   ```
+
+### VM is slow or unresponsive:
+1. Check host resources:
+   ```bash
+   top
+   iostat
+   ```
+
+2. Adjust VM resources:
+   - Stop VM: `./ubuntu_vm_manage.sh stop my-vm`
+   - Delete and recreate with different specs
+
+## Security Notes
+
+- Default password is `ubuntu` - change after first login
+- Consider adding SSH keys instead of password auth
+- VMs have internet access through NAT
+- Console access requires root privileges on host
+
+## Examples
+
+### Create a development environment:
+```bash
+# Set up networking
+sudo ./setup_vm_network.sh
+
+# Create VM
+sudo ./ubuntu_vm_start.sh dev-env 4096 4
+
+# Wait for boot, then connect
+sudo ./ubuntu_vm_manage.sh console dev-env
+
+# Inside VM, change password and install tools
+passwd
+sudo apt update && sudo apt install -y build-essential nodejs npm
+```
+
+### Create multiple VMs:
+```bash
+# Create web server
+sudo ./ubuntu_vm_start.sh web-server 2048 2
+
+# Create database server
+sudo ./ubuntu_vm_start.sh db-server 4096 2
+
+# Create load balancer
+sudo ./ubuntu_vm_start.sh lb 1024 1
+
+# List all VMs
+sudo ./ubuntu_vm_manage.sh list
+```
+
+## Performance Tips
+
+1. **Use appropriate VM sizing** - Don't over-allocate resources
+2. **Monitor host resources** - Ensure sufficient RAM/CPU
+3. **Use btrfs compression** - Add `compress=zstd` mount option
+4. **Regular cleanup** - Delete unused VMs to free space
+5. **SSD storage** - Use SSD for better I/O performance
+
+## Backup and Recovery
+
+### Backup a VM:
+```bash
+# Stop VM first
+sudo ./ubuntu_vm_manage.sh stop my-vm
+
+# Create snapshot
+sudo btrfs subvolume snapshot /var/lib/vms/vms/my-vm /var/lib/vms/backups/my-vm-$(date +%Y%m%d)
+```
+
+### Restore from backup:
+```bash
+# Delete current VM
+sudo ./ubuntu_vm_manage.sh delete my-vm
+
+# Restore from snapshot
+sudo btrfs subvolume snapshot /var/lib/vms/backups/my-vm-20231215 /var/lib/vms/vms/my-vm
--- a/tools/erase.sh
+++ b/tools/erase.sh
@@ -82,6 +82,113 @@ list_ssd_disks() {
    AVAILABLE_DISKS=("${disks[@]}")
 }

+# Check for RAID arrays and stop them
+check_and_stop_raid() {
+    log "Checking for active RAID arrays..."
+    
+    if [[ ! -f /proc/mdstat ]]; then
+        log "No RAID support detected"
+        return 0
+    fi
+    
+    # Get list of active RAID devices
+    local raid_devices=()
+    while IFS= read -r line; do
+        if [[ "$line" =~ ^(md[0-9]+) ]]; then
+            raid_devices+=("/dev/${BASH_REMATCH[1]}")
+        fi
+    done < /proc/mdstat
+    
+    if [[ ${#raid_devices[@]} -eq 0 ]]; then
+        log "No active RAID arrays found"
+        return 0
+    fi
+    
+    log "Found active RAID arrays:"
+    for raid in "${raid_devices[@]}"; do
+        echo "  - $raid"
+    done
+    
+    # Check if any RAID devices are mounted
+    log "Checking for mounted RAID filesystems..."
+    for raid in "${raid_devices[@]}"; do
+        if mountpoint -q "$raid" 2>/dev/null; then
+            log "Unmounting $raid"
+            umount "$raid" 2>/dev/null || warn "Failed to unmount $raid"
+        fi
+    done
+    
+    # Stop RAID arrays
+    log "Stopping RAID arrays..."
+    for raid in "${raid_devices[@]}"; do
+        log "Stopping $raid"
+        if mdadm --stop "$raid" 2>/dev/null; then
+            log "Successfully stopped $raid"
+        else
+            warn "Failed to stop $raid"
+        fi
+    done
+    
+    # Wait a moment for arrays to fully stop
+    sleep 2
+    
+    # Verify arrays are stopped
+    local remaining_arrays=()
+    while IFS= read -r line; do
+        if [[ "$line" =~ ^(md[0-9]+) ]]; then
+            remaining_arrays+=("/dev/${BASH_REMATCH[1]}")
+        fi
+    done < /proc/mdstat
+    
+    if [[ ${#remaining_arrays[@]} -gt 0 ]]; then
+        warn "Some RAID arrays are still active:"
+        for raid in "${remaining_arrays[@]}"; do
+            echo "  - $raid"
+        done
+        return 1
+    else
+        log "All RAID arrays successfully stopped"
+        return 0
+    fi
+}
+
+# Remove RAID superblocks from disk partitions
+remove_raid_superblocks() {
+    local disk="$1"
+    
+    log "Checking for RAID superblocks on $disk..."
+    
+    # Get all partitions on this disk
+    local partitions=()
+    while IFS= read -r partition; do
+        if [[ -n "$partition" ]]; then
+            partitions+=("/dev/$partition")
+        fi
+    done < <(lsblk -ln -o NAME "$disk" | tail -n +2)
+    
+    # Check each partition for RAID superblocks
+    for partition in "${partitions[@]}"; do
+        if mdadm --examine "$partition" &>/dev/null; then
+            log "Found RAID superblock on $partition, removing..."
+            if mdadm --zero-superblock "$partition" 2>/dev/null; then
+                log "Successfully removed RAID superblock from $partition"
+            else
+                warn "Failed to remove RAID superblock from $partition"
+            fi
+        fi
+    done
+    
+    # Also check the whole disk for superblocks
+    if mdadm --examine "$disk" &>/dev/null; then
+        log "Found RAID superblock on $disk, removing..."
+        if mdadm --zero-superblock "$disk" 2>/dev/null; then
+            log "Successfully removed RAID superblock from $disk"
+        else
+            warn "Failed to remove RAID superblock from $disk"
+        fi
+    fi
+}
+
 # Function to erase a single disk
 erase_disk() {
    local disk="$1"
@@ -93,6 +200,14 @@ erase_disk() {
    
    log "Starting erasure of $disk..."
    
+    # Check if disk is part of any active RAID arrays and stop them if needed
+    local disk_basename=$(basename "$disk")
+    if grep -q "$disk_basename" /proc/mdstat 2>/dev/null; then
+        warn "Disk $disk appears to be part of an active RAID array"
+        log "Stopping RAID arrays that use this disk..."
+        check_and_stop_raid
+    fi
+    
    # Unmount any mounted partitions on this disk
    log "Unmounting any mounted partitions on $disk..."
    for partition in $(lsblk -ln -o NAME "$disk" | tail -n +2); do
@@ -103,11 +218,19 @@ erase_disk() {
        fi
    done
    
+    # Remove RAID superblocks if present
+    remove_raid_superblocks "$disk"
+    
    # Get disk size for progress indication
    local disk_size_bytes=$(blockdev --getsize64 "$disk" 2>/dev/null || echo "0")
    local disk_size_gb=$((disk_size_bytes / 1024 / 1024 / 1024))
    
    log "Disk size: ${disk_size_gb}GB"
+    
+    # Wipe partition table first
+    log "Wiping partition table on $disk..."
+    wipefs -af "$disk" 2>/dev/null || warn "Failed to wipe filesystem signatures"
+    
    log "Erasing first 1GB of $disk (this will destroy the partition table and filesystem headers)..."
    
    # Use dd to zero out the first 1GB of the disk
@@ -157,6 +280,10 @@ erase_all_disks() {
    
    log "Starting disk erasure process..."
    
+    # First, stop any RAID arrays that might be using these disks
+    log "Checking and stopping RAID arrays before disk erasure..."
+    check_and_stop_raid
+    
    local success_count=0
    local total_count=${#AVAILABLE_DISKS[@]}
    
--- a/tools/example_autoconfig/autoconfig
+++ b/tools/example_autoconfig/autoconfig
@@ -0,0 +1,166 @@
+## ======================================================
+##  Hetzner Online GmbH - installimage - standard config
+## ======================================================
+
+
+## ====================
+##  HARD DISK DRIVE(S):
+## ====================
+
+
+# Device Model: SAMSUNG MZQLB1T9HAJR-00007, Serial Number: S439NA0R200855
+DRIVE1 /dev/nvme0n1
+# Device Model: SAMSUNG MZQLB1T9HAJR-00007, Serial Number: S439NA0R200886
+DRIVE2 /dev/nvme1n1
+
+
+
+## ===============
+##  SOFTWARE RAID:
+## ===============
+
+## activate software RAID?  < 0 | 1 >
+
+SWRAID 1
+
+## Choose the level for the software RAID < 0 | 1 | 10 >
+
+SWRAIDLEVEL 1
+
+## ==========
+##  HOSTNAME:
+## ==========
+
+## which hostname should be set?
+##
+
+HOSTNAME Ubuntu-2404-noble-amd64-base
+
+
+## ================
+##  NETWORK CONFIG:
+## ================
+
+# IPV4_ONLY no
+
+
+## =============
+##  MISC CONFIG:
+## =============
+
+USE_KERNEL_MODE_SETTING yes
+
+## ==========================
+##  PARTITIONS / FILESYSTEMS:
+## ==========================
+
+## define your partitions and filesystems like this:
+##
+## PART  <mountpoint/lvm/btrfs.X>  <filesystem/VG>  <size in MB>
+##
+## * <mountpoint/lvm/btrfs.X>
+##            mountpoint for this filesystem *OR*
+##            keyword 'lvm' to use this PART as volume group (VG) for LVM *OR*
+##            identifier 'btrfs.X' to use this PART as volume for
+##            btrfs subvolumes. X can be replaced with a unique
+##            alphanumeric keyword
+##            NOTE: no support btrfs multi-device volumes
+##            NOTE: reiserfs support is deprecated and will be removed in a future version
+## * <filesystem/VG>
+##            can be ext2, ext3, ext4, btrfs, reiserfs, xfs, swap  *OR*  name
+##            of the LVM volume group (VG), if this PART is a VG.
+## * <size>
+##            you can use the keyword 'all' to assign all the
+##            remaining space of the drive to the *last* partition.
+##            you can use M/G/T for unit specification in MiB/GiB/TiB
+##
+## notes:
+##   - extended partitions are created automatically
+##   - '/boot' cannot be on a xfs filesystem
+##   - '/boot' cannot be on LVM!
+##   - when using software RAID 0, you need a '/boot' partition
+##
+## example without LVM (default):
+## -> 4GB   swapspace
+## -> 1024MB /boot
+## -> 10GB  /
+## -> 5GB   /tmp
+## -> all the rest to /home
+#PART swap   swap        4G
+#PART /boot  ext2      1024M
+#PART /      ext4       10G
+#PART /tmp   xfs         5G
+#PART /home  ext3       all
+#
+##
+## to activate LVM, you have to define volume groups and logical volumes
+##
+## example with LVM:
+#
+## normal filesystems and volume group definitions:
+## -> 1024MB boot  (not on lvm)
+## -> all the rest for LVM VG 'vg0'
+#PART /boot  ext3     1024M
+#PART lvm    vg0       all
+#
+## logical volume definitions:
+#LV <VG> <name> <mount> <filesystem> <size>
+#
+#LV vg0   root   /        ext4         10G
+#LV vg0   swap   swap     swap          4G
+#LV vg0   home   /home    xfs          20G
+#
+##
+## to use btrfs subvolumes, define a volume identifier on a partition
+##
+## example with btrfs subvolumes:
+##
+## -> all space on one partition with volume 'btrfs.1'
+#PART btrfs.1    btrfs       all
+##
+## btrfs subvolume definitions:
+#SUBVOL <volume> <subvolume> <mount>
+#
+#SUBVOL btrfs.1  @           /
+#SUBVOL btrfs.1  @/usr       /usr
+#SUBVOL btrfs.1  @home       /home
+#
+## your system has the following devices:
+#
+# Disk /dev/nvme0n1: 1.93 TB (=> 1.75 TiB)
+# Disk /dev/nvme1n1: 1.93 TB (=> 1.75 TiB)
+#
+## Based on your disks and which RAID level you will choose you have
+## the following free space to allocate (in GiB):
+# RAID  0: ~3576
+# RAID  1: ~1788
+#
+
+PART swap swap 4G
+PART /boot ext3 1024M
+PART / ext4 all
+
+
+## ========================
+##  OPERATING SYSTEM IMAGE:
+## ========================
+
+## full path to the operating system image
+##   supported image sources:  local dir,  ftp,  http,  nfs
+##   supported image types: tar, tar.gz, tar.bz, tar.bz2, tar.xz, tgz, tbz, txz
+## examples:
+#
+# local: /path/to/image/filename.tar.gz
+# ftp:   ftp://<user>:<password>@hostname/path/to/image/filename.tar.bz2
+# http:  http://<user>:<password>@hostname/path/to/image/filename.tbz
+# https: https://<user>:<password>@hostname/path/to/image/filename.tbz
+# nfs:   hostname:/path/to/image/filename.tgz
+#
+# for validation of the image, place the detached gpg-signature
+# and your public key in the same directory as your image file.
+# naming examples:
+#  signature:   filename.tar.bz2.sig
+#  public key:  public-key.asc
+
+IMAGE /root/.oldroot/nfs/install/../images/Ubuntu-2404-noble-amd64-base.tar.gz
+
--- a/tools/install_cloudhypervisor.sh
+++ b/tools/install_cloudhypervisor.sh
@@ -0,0 +1,180 @@
+#!/bin/bash
+
+# Cloud Hypervisor Installation Script
+# Downloads and installs Cloud Hypervisor from GitHub releases
+# Compatible with Ubuntu/Debian systems
+
+set -e
+
+# Colors for output
+RED='\033[0;31m'
+GREEN='\033[0;32m'
+YELLOW='\033[1;33m'
+NC='\033[0m' # No Color
+
+# Cloud Hypervisor configuration
+CH_VERSION="v46.0"
+CH_URL="https://github.com/cloud-hypervisor/cloud-hypervisor/releases/download/${CH_VERSION}/cloud-hypervisor-static"
+CH_BINARY="cloud-hypervisor"
+MIN_SIZE_MB=4
+
+log() {
+    echo -e "${GREEN}[$(date +'%Y-%m-%d %H:%M:%S')] $1${NC}"
+}
+
+warn() {
+    echo -e "${YELLOW}[$(date +'%Y-%m-%d %H:%M:%S')] WARNING: $1${NC}"
+}
+
+error() {
+    echo -e "${RED}[$(date +'%Y-%m-%d %H:%M:%S')] ERROR: $1${NC}"
+    exit 1
+}
+
+# Check if running as root
+if [ "$EUID" -eq 0 ]; then
+    log "Running as root. Cloud Hypervisor will be installed system-wide."
+    INSTALL_DIR="/usr/local/bin"
+else
+    log "Installing Cloud Hypervisor for current user"
+    INSTALL_DIR="$HOME/.local/bin"
+    # Create local bin directory if it doesn't exist
+    mkdir -p "$INSTALL_DIR"
+fi
+
+# Check if Cloud Hypervisor is already installed
+if command -v cloud-hypervisor &> /dev/null; then
+    CH_CURRENT_VERSION=$(cloud-hypervisor --version 2>/dev/null | head -n1 || echo "")
+    if [ -n "$CH_CURRENT_VERSION" ]; then
+        warn "Cloud Hypervisor is already installed: $CH_CURRENT_VERSION"
+        read -p "Do you want to update/reinstall? (y/N): " -n 1 -r
+        echo
+        if [[ ! $REPLY =~ ^[Yy]$ ]]; then
+            log "Installation cancelled"
+            exit 0
+        fi
+    fi
+fi
+
+# Check for required dependencies
+log "Checking system dependencies..."
+MISSING_DEPS=()
+
+# Check for curl
+if ! command -v curl &> /dev/null; then
+    MISSING_DEPS+=("curl")
+fi
+
+# Install missing dependencies
+if [ ${#MISSING_DEPS[@]} -gt 0 ]; then
+    log "Installing missing dependencies: ${MISSING_DEPS[*]}"
+    if [ "$EUID" -eq 0 ]; then
+        apt update
+        apt install -y "${MISSING_DEPS[@]}"
+    else
+        log "Please install the following packages and run this script again:"
+        log "sudo apt update && sudo apt install -y ${MISSING_DEPS[*]}"
+        exit 1
+    fi
+fi
+
+log "Starting Cloud Hypervisor installation..."
+log "Version: $CH_VERSION"
+log "Install directory: $INSTALL_DIR"
+
+# Create temporary directory for download
+TEMP_DIR=$(mktemp -d)
+TEMP_FILE="$TEMP_DIR/$CH_BINARY"
+
+# Cleanup function
+cleanup() {
+    rm -rf "$TEMP_DIR"
+}
+trap cleanup EXIT
+
+log "Downloading Cloud Hypervisor from $CH_URL..."
+
+# Download with size verification
+if ! curl -L --fail --progress-bar -o "$TEMP_FILE" "$CH_URL"; then
+    error "Failed to download Cloud Hypervisor from $CH_URL"
+fi
+
+# Check file size
+FILE_SIZE=$(stat -c%s "$TEMP_FILE" 2>/dev/null || stat -f%z "$TEMP_FILE" 2>/dev/null || echo "0")
+FILE_SIZE_MB=$((FILE_SIZE / 1024 / 1024))
+
+log "Downloaded file size: ${FILE_SIZE_MB}MB"
+
+if [ "$FILE_SIZE_MB" -lt "$MIN_SIZE_MB" ]; then
+    error "Downloaded file is too small (${FILE_SIZE_MB}MB). Expected at least ${MIN_SIZE_MB}MB. Download may be corrupted."
+fi
+
+log "✅ Download size verification passed (${FILE_SIZE_MB}MB >= ${MIN_SIZE_MB}MB)"
+
+# Make the binary executable
+chmod +x "$TEMP_FILE"
+
+# Test the binary before installation
+log "Testing downloaded binary..."
+if ! "$TEMP_FILE" --version &> /dev/null; then
+    error "Downloaded binary is not working correctly"
+fi
+
+log "✅ Binary test passed"
+
+# Install the binary
+log "Installing Cloud Hypervisor to $INSTALL_DIR..."
+if [ "$EUID" -eq 0 ]; then
+    cp "$TEMP_FILE" "$INSTALL_DIR/$CH_BINARY"
+else
+    cp "$TEMP_FILE" "$INSTALL_DIR/$CH_BINARY"
+    # Add to PATH if not already there
+    if [[ ":$PATH:" != *":$INSTALL_DIR:"* ]]; then
+        warn "Adding $INSTALL_DIR to PATH in ~/.bashrc"
+        echo "export PATH=\"$INSTALL_DIR:\$PATH\"" >> ~/.bashrc
+        export PATH="$INSTALL_DIR:$PATH"
+    fi
+fi
+
+# Verify installation
+log "Verifying Cloud Hypervisor installation..."
+
+# Set PATH for verification
+if [ "$EUID" -ne 0 ]; then
+    export PATH="$INSTALL_DIR:$PATH"
+fi
+
+if command -v cloud-hypervisor &> /dev/null; then
+    CH_VERSION_OUTPUT=$(cloud-hypervisor --version 2>/dev/null || echo "")
+    if [ -n "$CH_VERSION_OUTPUT" ]; then
+        log "✅ Cloud Hypervisor installation successful!"
+        log "Version: $CH_VERSION_OUTPUT"
+        log "Location: $(which cloud-hypervisor)"
+        
+        # Test basic functionality
+        log "Testing basic functionality..."
+        if cloud-hypervisor --help &> /dev/null; then
+            log "✅ Basic functionality test passed"
+        else
+            warn "Basic functionality test failed, but binary is installed"
+        fi
+        
+        log "✅ Cloud Hypervisor installation completed successfully!"
+        log ""
+        log "Next steps:"
+        if [ "$EUID" -eq 0 ]; then
+            log "- Cloud Hypervisor is available system-wide"
+        else
+            log "- Run 'source ~/.bashrc' to update PATH in this session"
+            log "- Or restart your terminal"
+        fi
+        log "- Run 'cloud-hypervisor --help' to see available options"
+        log "- Check the documentation at: https://github.com/cloud-hypervisor/cloud-hypervisor"
+    else
+        error "Cloud Hypervisor was installed but version check failed"
+    fi
+else
+    error "Cloud Hypervisor installation failed. Binary not found in PATH."
+fi
+
+apt update && apt install -y genisoimage
--- a/tools/install_cloudhypervisor_extract_kernel.sh
+++ b/tools/install_cloudhypervisor_extract_kernel.sh
@@ -0,0 +1,61 @@
+#!/bin/bash
+
+# Extract kernel and initrd from Ubuntu cloud image
+# This script ensures kernel boot works automatically
+
+set -e
+
+# Colors for output
+RED='\033[0;31m'
+GREEN='\033[0;32m'
+YELLOW='\033[1;33m'
+NC='\033[0m' # No Color
+
+log() {
+    echo -e "${GREEN}[$(date +'%Y-%m-%d %H:%M:%S')] $1${NC}"
+}
+
+error() {
+    echo -e "${RED}[$(date +'%Y-%m-%d %H:%M:%S')] ERROR: $1${NC}"
+    exit 1
+}
+
+BASE_SUBVOL="/var/lib/vms/base"
+BASE_IMAGE_PATH="$BASE_SUBVOL/ubuntu-24.04-server-cloudimg-amd64.raw"
+KERNEL_PATH="$BASE_SUBVOL/vmlinuz"
+INITRD_PATH="$BASE_SUBVOL/initrd.img"
+
+# Check if kernel and initrd already exist
+if [ -f "$KERNEL_PATH" ] && [ -f "$INITRD_PATH" ]; then
+    log "Kernel and initrd already extracted"
+    exit 0
+fi
+
+# Check if base image exists
+if [ ! -f "$BASE_IMAGE_PATH" ]; then
+    error "Base image not found at $BASE_IMAGE_PATH"
+fi
+
+log "Extracting kernel and initrd from base image..."
+
+# Create temporary mount point
+TEMP_MOUNT=$(mktemp -d)
+trap "umount '$TEMP_MOUNT' 2>/dev/null || true; losetup -d /dev/loop1 2>/dev/null || true; rmdir '$TEMP_MOUNT'" EXIT
+
+# Mount the image
+losetup -P /dev/loop1 "$BASE_IMAGE_PATH"
+
+# Mount the boot partition (partition 16 contains kernel/initrd)
+mount /dev/loop1p16 "$TEMP_MOUNT"
+
+# Copy kernel and initrd
+if [ -f "$TEMP_MOUNT/vmlinuz-6.8.0-60-generic" ] && [ -f "$TEMP_MOUNT/initrd.img-6.8.0-60-generic" ]; then
+    cp "$TEMP_MOUNT/vmlinuz-6.8.0-60-generic" "$KERNEL_PATH"
+    cp "$TEMP_MOUNT/initrd.img-6.8.0-60-generic" "$INITRD_PATH"
+    log "✓ Kernel and initrd extracted successfully"
+else
+    error "Kernel or initrd not found in boot partition"
+fi
+
+# Cleanup is handled by trap
+log "Kernel extraction completed"
--- a/tools/install_rust.sh
+++ b/tools/install_rust.sh
@@ -0,0 +1,193 @@
+#!/bin/bash
+
+# Rust Installation Script
+# Installs Rust programming language using rustup
+# Compatible with Ubuntu/Debian systems
+
+set -e
+
+# Colors for output
+RED='\033[0;31m'
+GREEN='\033[0;32m'
+YELLOW='\033[1;33m'
+NC='\033[0m' # No Color
+
+log() {
+    echo -e "${GREEN}[$(date +'%Y-%m-%d %H:%M:%S')] $1${NC}"
+}
+
+warn() {
+    echo -e "${YELLOW}[$(date +'%Y-%m-%d %H:%M:%S')] WARNING: $1${NC}"
+}
+
+error() {
+    echo -e "${RED}[$(date +'%Y-%m-%d %H:%M:%S')] ERROR: $1${NC}"
+    exit 1
+}
+
+# Check if running as root
+if [ "$EUID" -eq 0 ]; then
+    warn "Running as root. Rust will be installed system-wide."
+    INSTALL_DIR="/usr/local"
+    CARGO_HOME="/usr/local/cargo"
+    RUSTUP_HOME="/usr/local/rustup"
+else
+    log "Installing Rust for current user"
+    INSTALL_DIR="$HOME"
+    CARGO_HOME="$HOME/.cargo"
+    RUSTUP_HOME="$HOME/.rustup"
+fi
+
+# Check if Rust is already installed
+if command -v rustc &> /dev/null; then
+    RUST_VERSION=$(rustc --version 2>/dev/null || echo "")
+    if [ -n "$RUST_VERSION" ]; then
+        warn "Rust is already installed: $RUST_VERSION"
+        read -p "Do you want to update/reinstall? (y/N): " -n 1 -r
+        echo
+        if [[ ! $REPLY =~ ^[Yy]$ ]]; then
+            log "Installation cancelled"
+            exit 0
+        fi
+    else
+        warn "Rust tools found but no default toolchain set. Will configure default toolchain."
+    fi
+elif command -v rustup &> /dev/null; then
+    warn "Rustup found but no default toolchain set. Will configure default toolchain."
+fi
+
+# Check for required dependencies
+log "Checking system dependencies..."
+MISSING_DEPS=()
+
+# Check for curl
+if ! command -v curl &> /dev/null; then
+    MISSING_DEPS+=("curl")
+fi
+
+# Check for build essentials
+if ! dpkg -l | grep -q build-essential; then
+    MISSING_DEPS+=("build-essential")
+fi
+
+# Install missing dependencies
+if [ ${#MISSING_DEPS[@]} -gt 0 ]; then
+    log "Installing missing dependencies: ${MISSING_DEPS[*]}"
+    if [ "$EUID" -eq 0 ]; then
+        apt update
+        apt install -y "${MISSING_DEPS[@]}"
+    else
+        log "Please install the following packages and run this script again:"
+        log "sudo apt update && sudo apt install -y ${MISSING_DEPS[*]}"
+        exit 1
+    fi
+fi
+
+# Set environment variables for installation
+export CARGO_HOME="$CARGO_HOME"
+export RUSTUP_HOME="$RUSTUP_HOME"
+
+log "Starting Rust installation..."
+log "CARGO_HOME: $CARGO_HOME"
+log "RUSTUP_HOME: $RUSTUP_HOME"
+
+# Check if rustup is already installed but needs default toolchain
+if command -v rustup &> /dev/null; then
+    log "Rustup already installed. Setting up default toolchain..."
+    if [ "$EUID" -eq 0 ]; then
+        /usr/local/bin/rustup default stable || rustup default stable
+    else
+        "$CARGO_HOME/bin/rustup" default stable || rustup default stable
+    fi
+else
+    # Download and run rustup installer
+    log "Downloading rustup installer..."
+    if [ "$EUID" -eq 0 ]; then
+        # For root installation, we need to handle it differently
+        curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --no-modify-path
+    
+    # Create symlinks for system-wide access
+    ln -sf "$CARGO_HOME/bin/cargo" /usr/local/bin/cargo
+    ln -sf "$CARGO_HOME/bin/rustc" /usr/local/bin/rustc
+    ln -sf "$CARGO_HOME/bin/rustup" /usr/local/bin/rustup
+    ln -sf "$CARGO_HOME/bin/rustdoc" /usr/local/bin/rustdoc
+    ln -sf "$CARGO_HOME/bin/rust-gdb" /usr/local/bin/rust-gdb
+    ln -sf "$CARGO_HOME/bin/rust-lldb" /usr/local/bin/rust-lldb
+    
+    # Set up environment for all users
+    cat > /etc/profile.d/rust.sh << 'EOF'
+export CARGO_HOME="/usr/local/cargo"
+export RUSTUP_HOME="/usr/local/rustup"
+export PATH="/usr/local/cargo/bin:$PATH"
+EOF
+    
+    log "Rust installed system-wide. All users can now use Rust."
+    log "Environment variables set in /etc/profile.d/rust.sh"
+else
+    # Standard user installation
+    curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
+    
+    # Source the cargo environment
+    source "$CARGO_HOME/env"
+    
+    log "Rust installed for current user."
+    log "To use Rust in new shells, run: source ~/.cargo/env"
+    log "Or add ~/.cargo/bin to your PATH in your shell profile"
+    fi
+fi
+
+# Verify installation
+log "Verifying Rust installation..."
+if [ "$EUID" -eq 0 ]; then
+    # Source the environment first
+    export PATH="/usr/local/cargo/bin:$PATH"
+    RUST_VERSION=$(/usr/local/bin/rustc --version 2>/dev/null || echo "")
+    CARGO_VERSION=$(/usr/local/bin/cargo --version 2>/dev/null || echo "")
+    RUSTUP_VERSION=$(/usr/local/bin/rustup --version 2>/dev/null || echo "")
+else
+    # Source the cargo environment
+    if [ -f "$CARGO_HOME/env" ]; then
+        source "$CARGO_HOME/env"
+    fi
+    RUST_VERSION=$("$CARGO_HOME/bin/rustc" --version 2>/dev/null || rustc --version 2>/dev/null || echo "")
+    CARGO_VERSION=$("$CARGO_HOME/bin/cargo" --version 2>/dev/null || cargo --version 2>/dev/null || echo "")
+    RUSTUP_VERSION=$("$CARGO_HOME/bin/rustup" --version 2>/dev/null || rustup --version 2>/dev/null || echo "")
+fi
+
+if [ -n "$RUST_VERSION" ] && [ -n "$CARGO_VERSION" ]; then
+    log "✅ Rust installation successful!"
+    log "Rust compiler: $RUST_VERSION"
+    log "Cargo package manager: $CARGO_VERSION"
+    if [ -n "$RUSTUP_VERSION" ]; then
+        log "Rustup toolchain manager: $RUSTUP_VERSION"
+    fi
+    
+    # Set default toolchain and install common components
+    log "Setting default Rust toolchain..."
+    if [ "$EUID" -eq 0 ]; then
+        /usr/local/bin/rustup default stable
+        log "Installing common Rust components..."
+        /usr/local/bin/rustup component add clippy rustfmt
+    else
+        "$CARGO_HOME/bin/rustup" default stable
+        log "Installing common Rust components..."
+        "$CARGO_HOME/bin/rustup" component add clippy rustfmt
+    fi
+    
+    log "✅ Rust installation completed successfully!"
+    log ""
+    log "Next steps:"
+    if [ "$EUID" -eq 0 ]; then
+        log "- Rust is available system-wide"
+        log "- Users may need to log out and back in for PATH changes to take effect"
+    else
+        log "- Run 'source ~/.cargo/env' to use Rust in this session"
+        log "- Restart your terminal or add ~/.cargo/bin to your PATH"
+    fi
+    log "- Create a new Rust project: cargo new my_project"
+    log "- Build and run: cd my_project && cargo run"
+else
+    error "Rust installation failed. Please check the output above for errors."
+fi
+
+rustup default stable
--- a/tools/install_rust_sal.sh
+++ b/tools/install_rust_sal.sh
@@ -0,0 +1,231 @@
+#!/bin/bash
+
+# Rust and SAL Installation Script
+# Defensively installs Rust and clones/updates SAL repository
+# Compatible with Ubuntu/Debian systems
+
+set -e
+
+# Colors for output
+RED='\033[0;31m'
+GREEN='\033[0;32m'
+YELLOW='\033[1;33m'
+BLUE='\033[0;34m'
+NC='\033[0m' # No Color
+
+log() {
+    echo -e "${GREEN}[$(date +'%Y-%m-%d %H:%M:%S')] $1${NC}"
+}
+
+warn() {
+    echo -e "${YELLOW}[$(date +'%Y-%m-%d %H:%M:%S')] WARNING: $1${NC}"
+}
+
+error() {
+    echo -e "${RED}[$(date +'%Y-%m-%d %H:%M:%S')] ERROR: $1${NC}"
+    exit 1
+}
+
+info() {
+    echo -e "${BLUE}[$(date +'%Y-%m-%d %H:%M:%S')] INFO: $1${NC}"
+}
+
+# Get the directory where this script is located
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+log "Script directory: $SCRIPT_DIR"
+
+# Define target directory for SAL repository
+SAL_TARGET_DIR="/root/code/git.threefold.info/herocode/sal"
+SAL_REPO_URL="git@git.threefold.info:herocode/sal.git"
+
+# Function to check if ssh-agent is loaded and has keys
+check_ssh_agent() {
+    log "Checking SSH agent status..."
+    
+    if [ -z "$SSH_AUTH_SOCK" ]; then
+        warn "SSH_AUTH_SOCK is not set. SSH agent may not be running."
+        return 1
+    fi
+    
+    if ! ssh-add -l &>/dev/null; then
+        warn "SSH agent is running but no keys are loaded or agent is not accessible."
+        info "You may need to run: ssh-add ~/.ssh/id_rsa (or your key file)"
+        return 1
+    fi
+    
+    log "✅ SSH agent is loaded and has keys available"
+    return 0
+}
+
+# Function to test SSH connection to git server
+test_ssh_connection() {
+    log "Testing SSH connection to git.threefold.info..."
+    
+    if ssh -T -o ConnectTimeout=10 -o StrictHostKeyChecking=no git@git.threefold.info 2>&1 | grep -q "successfully authenticated"; then
+        log "✅ SSH connection to git.threefold.info successful"
+        return 0
+    else
+        warn "SSH connection test failed. Continuing anyway..."
+        return 1
+    fi
+}
+
+# Function to defensively install Rust
+install_rust_defensively() {
+    log "Starting defensive Rust installation..."
+    
+    # Check if install_rust.sh exists in the same directory
+    RUST_INSTALLER="$SCRIPT_DIR/install_rust.sh"
+    
+    if [ ! -f "$RUST_INSTALLER" ]; then
+        error "install_rust.sh not found at: $RUST_INSTALLER"
+    fi
+    
+    if [ ! -x "$RUST_INSTALLER" ]; then
+        log "Making install_rust.sh executable..."
+        chmod +x "$RUST_INSTALLER"
+    fi
+    
+    log "Calling install_rust.sh from: $RUST_INSTALLER"
+    
+    # Run the Rust installer defensively
+    if ! "$RUST_INSTALLER"; then
+        error "Rust installation failed"
+    fi
+    
+    log "✅ Rust installation completed successfully"
+}
+
+# Function to clone or update SAL repository
+handle_sal_repository() {
+    log "Handling SAL repository at: $SAL_TARGET_DIR"
+    
+    # Create parent directories if they don't exist
+    SAL_PARENT_DIR="$(dirname "$SAL_TARGET_DIR")"
+    if [ ! -d "$SAL_PARENT_DIR" ]; then
+        log "Creating parent directory: $SAL_PARENT_DIR"
+        mkdir -p "$SAL_PARENT_DIR"
+    fi
+    
+    if [ -d "$SAL_TARGET_DIR" ]; then
+        log "SAL directory exists. Checking if it's a git repository..."
+        
+        if [ -d "$SAL_TARGET_DIR/.git" ]; then
+            log "Found existing git repository. Pulling latest changes..."
+            cd "$SAL_TARGET_DIR"
+            
+            # Check if we're in a clean state
+            if ! git status --porcelain | grep -q .; then
+                log "Repository is clean. Pulling changes..."
+                if git pull origin main 2>/dev/null || git pull origin master 2>/dev/null; then
+                    log "✅ Successfully pulled latest changes"
+                else
+                    warn "Pull failed. Repository may be up to date or have conflicts."
+                fi
+            else
+                warn "Repository has uncommitted changes. Skipping pull."
+                git status --short
+            fi
+        else
+            warn "Directory exists but is not a git repository. Removing and cloning fresh..."
+            rm -rf "$SAL_TARGET_DIR"
+            clone_sal_repository
+        fi
+    else
+        log "SAL directory does not exist. Cloning repository..."
+        clone_sal_repository
+    fi
+}
+
+# Function to clone SAL repository
+clone_sal_repository() {
+    log "Cloning SAL repository from: $SAL_REPO_URL"
+    log "Target directory: $SAL_TARGET_DIR"
+    
+    # Ensure parent directory exists
+    mkdir -p "$(dirname "$SAL_TARGET_DIR")"
+    
+    if git clone "$SAL_REPO_URL" "$SAL_TARGET_DIR"; then
+        log "✅ Successfully cloned SAL repository"
+        cd "$SAL_TARGET_DIR"
+        log "Repository cloned to: $(pwd)"
+        log "Current branch: $(git branch --show-current 2>/dev/null || echo 'unknown')"
+    else
+        error "Failed to clone SAL repository. Check SSH keys and network connectivity."
+    fi
+}
+
+# Function to verify SAL repository
+verify_sal_repository() {
+    log "Verifying SAL repository..."
+    
+    if [ ! -d "$SAL_TARGET_DIR" ]; then
+        error "SAL directory does not exist: $SAL_TARGET_DIR"
+    fi
+    
+    if [ ! -d "$SAL_TARGET_DIR/.git" ]; then
+        error "SAL directory is not a git repository: $SAL_TARGET_DIR"
+    fi
+    
+    cd "$SAL_TARGET_DIR"
+    
+    # Get repository information
+    REPO_URL=$(git remote get-url origin 2>/dev/null || echo "unknown")
+    CURRENT_BRANCH=$(git branch --show-current 2>/dev/null || echo "unknown")
+    LAST_COMMIT=$(git log -1 --format="%h - %s (%cr)" 2>/dev/null || echo "unknown")
+    
+    log "✅ SAL repository verification successful"
+    log "Repository URL: $REPO_URL"
+    log "Current branch: $CURRENT_BRANCH"
+    log "Last commit: $LAST_COMMIT"
+}
+
+# Main execution
+main() {
+    log "Starting Rust and SAL installation process..."
+    log "=========================================="
+    
+    # Step 1: Install Rust defensively
+    install_rust_defensively
+    
+    # Step 2: Check SSH agent
+    if ! check_ssh_agent; then
+        warn "SSH agent issues detected. Continuing anyway..."
+        warn "You may need to set up SSH keys and agent before git operations."
+    fi
+    
+    # Step 3: Test SSH connection (optional, continues on failure)
+    test_ssh_connection
+    
+    # Step 4: Handle SAL repository (clone or pull)
+    handle_sal_repository
+    
+    # Step 5: Verify SAL repository
+    verify_sal_repository
+    
+    log "=========================================="
+    log "✅ Installation process completed successfully!"
+    log ""
+    log "Summary:"
+    log "- Rust: Installed and verified"
+    log "- SAL repository: Available at $SAL_TARGET_DIR"
+    log ""
+    log "Next steps:"
+    log "- Source Rust environment: source ~/.cargo/env (if not root)"
+    log "- Navigate to SAL: cd $SAL_TARGET_DIR"
+    log "- Build SAL project: cargo build"
+}
+
+# Trap to handle script interruption
+trap 'error "Script interrupted"' INT TERM
+
+# Run main function
+main "$@"
+
+
+# Install OpenSSL development headers and pkg-config
+sudo apt update
+sudo apt install -y pkg-config libssl-dev
+
+cd ~/code/git.threefold.info/herocode/sal
+./build_herodo.sh
--- a/tools/readme.md
+++ b/tools/readme.md
@@ -0,0 +1,89 @@
+# ITenv Tools - Tool Documentation
+
+This directory contains various utility scripts for server management, disk operations, and git repository management. Each tool is designed for specific infrastructure tasks.
+
+## Tools Overview
+
+### 1. erase.sh
+
+**Purpose**: Secure disk erasure utility for SSD drives
+
+**Usage**: 
+```bash
+sudo ./erase.sh
+```
+
+**⚠️ WARNING**: This tool permanently destroys all data on selected disks. Use with extreme caution!
+
+### 2. git_checkout.sh
+
+**Purpose**: Git repository management for itenv projects
+
+**What it does**:
+
+- Verifies SSH agent has loaded keys for git authentication
+- Checks and configures git user.name and user.email if not set
+- Creates the standard directory structure: `/root/code/git.threefold.info/ourworld_web`
+- Clones or updates two repositories:
+  - `itenv_web2` - Main web application repository
+  - `itenv_tools` - Tools and utilities repository
+
+**Prerequisites**:
+- SSH key must be loaded in ssh-agent
+- Git must be installed
+- Access to git.threefold.info repositories
+
+**Usage**:
+```bash
+./git_checkout.sh
+```
+
+### 3. git_push.sh
+
+**Purpose**: Quick git commit and push utility
+
+```bash
+./git_push.sh
+```
+**Note**: This is a simple automation script for quick commits. For production use, consider more descriptive commit messages.
+
+### 4. ubuntu_install.sh
+**Purpose**: Automated Ubuntu 24.04 installation for Hetzner dedicated servers
+
+> only do this on rescue system for hetzner servers
+
+**What it does**:
+- Detects available NVMe drives (requires minimum 2 drives)
+- Creates Hetzner installimage configuration for Ubuntu 24.04
+- Sets up btrfs filesystem with RAID 1 across two drives
+- Configures btrfs subvolumes for better organization:
+  - `@` (root filesystem)
+  - `@home` (/home)
+  - `@var` (/var)
+  - `@var/log` (/var/log)
+  - `@tmp` (/tmp)
+  - `@opt` (/opt)
+  - `@srv` (/srv)
+  - `@snapshots` (/.snapshots)
+- Creates post-installation scripts for:
+  - btrfs optimization and maintenance
+  - Automatic snapshot management with snapper
+  - RAID monitoring utilities
+
+**Features**:
+- RAID 1 for data redundancy
+- btrfs compression (zstd:3) for space efficiency
+- Automated weekly balance and scrub operations
+- Snapshot management with configurable retention
+
+**Prerequisites**:
+- Must be run from Hetzner rescue system
+- Requires at least 2 NVMe drives
+- installimage must be available
+
+**Usage**:
+```bash
+./ubuntu_install.sh
+```
+
+**⚠️ WARNING**: This script will completely wipe the selected drives and install a fresh Ubuntu system.
--- a/tools/setup_vm_network.sh
+++ b/tools/setup_vm_network.sh
@@ -0,0 +1,151 @@
+#!/bin/bash
+
+# VM Network Setup Script
+# This script sets up networking for VMs to enable SSH access
+
+set -e
+
+# Colors for output
+RED='\033[0;31m'
+GREEN='\033[0;32m'
+YELLOW='\033[1;33m'
+BLUE='\033[0;34m'
+NC='\033[0m' # No Color
+
+log() {
+    echo -e "${GREEN}[$(date +'%Y-%m-%d %H:%M:%S')] $1${NC}"
+}
+
+warn() {
+    echo -e "${YELLOW}[$(date +'%Y-%m-%d %H:%M:%S')] WARNING: $1${NC}"
+}
+
+error() {
+    echo -e "${RED}[$(date +'%Y-%m-%d %H:%M:%S')] ERROR: $1${NC}"
+    exit 1
+}
+
+info() {
+    echo -e "${BLUE}[$(date +'%Y-%m-%d %H:%M:%S')] INFO: $1${NC}"
+}
+
+# Check if running as root
+if [ "$EUID" -ne 0 ]; then
+    error "This script must be run as root"
+fi
+
+BRIDGE_NAME="br0"
+BRIDGE_IP="192.168.100.1/24"
+NETWORK="192.168.100.0/24"
+
+log "Setting up VM networking..."
+
+# Check if bridge already exists and has IP
+if ip link show "$BRIDGE_NAME" &>/dev/null; then
+    if ip addr show "$BRIDGE_NAME" | grep -q "192.168.100.1"; then
+        info "Bridge $BRIDGE_NAME already configured with IP"
+    else
+        log "Adding IP address to existing bridge..."
+        ip addr add "$BRIDGE_IP" dev "$BRIDGE_NAME"
+    fi
+else
+    log "Bridge $BRIDGE_NAME not found. It will be created when VMs start."
+fi
+
+# Enable IP forwarding
+log "Enabling IP forwarding..."
+echo 1 > /proc/sys/net/ipv4/ip_forward
+
+# Make IP forwarding persistent
+if ! grep -q "net.ipv4.ip_forward=1" /etc/sysctl.conf; then
+    echo "net.ipv4.ip_forward=1" >> /etc/sysctl.conf
+    log "IP forwarding made persistent in /etc/sysctl.conf"
+fi
+
+# Set up NAT for VM network
+log "Setting up NAT for VM network..."
+
+# Remove existing rules to avoid duplicates
+iptables -t nat -D POSTROUTING -s "$NETWORK" -j MASQUERADE 2>/dev/null || true
+iptables -D FORWARD -i "$BRIDGE_NAME" -o "$BRIDGE_NAME" -j ACCEPT 2>/dev/null || true
+iptables -D FORWARD -i "$BRIDGE_NAME" -j ACCEPT 2>/dev/null || true
+iptables -D FORWARD -o "$BRIDGE_NAME" -j ACCEPT 2>/dev/null || true
+
+# Add new rules
+iptables -t nat -A POSTROUTING -s "$NETWORK" -j MASQUERADE
+iptables -A FORWARD -i "$BRIDGE_NAME" -o "$BRIDGE_NAME" -j ACCEPT
+iptables -A FORWARD -i "$BRIDGE_NAME" -j ACCEPT
+iptables -A FORWARD -o "$BRIDGE_NAME" -j ACCEPT
+
+log "NAT rules configured"
+
+# Install and configure dnsmasq for DHCP
+if ! command -v dnsmasq &>/dev/null; then
+    log "Installing dnsmasq for DHCP..."
+    apt update && apt install -y dnsmasq
+fi
+
+# Configure dnsmasq for VM network
+DNSMASQ_CONF="/etc/dnsmasq.d/vm-network.conf"
+log "Configuring DHCP for VM network..."
+
+cat > "$DNSMASQ_CONF" << EOF
+# VM Network DHCP Configuration
+# Only bind to the bridge interface to avoid conflicts with systemd-resolved
+interface=$BRIDGE_NAME
+bind-interfaces
+
+# Disable DNS server functionality (only DHCP)
+port=0
+
+# DHCP configuration
+dhcp-range=192.168.100.10,192.168.100.100,12h
+dhcp-option=3,192.168.100.1
+dhcp-option=6,8.8.8.8,8.8.4.4
+
+# Disable reading /etc/hosts and /etc/resolv.conf
+no-hosts
+no-resolv
+EOF
+
+# Restart dnsmasq
+systemctl restart dnsmasq
+systemctl enable dnsmasq
+
+log "DHCP server configured and started"
+
+# Create a script to show VM IPs
+cat > "/usr/local/bin/vm-ips" << 'EOF'
+#!/bin/bash
+echo "VM DHCP Leases:"
+echo "==============="
+if [ -f /var/lib/dhcp/dhcpd.leases ]; then
+    awk '/lease/ { ip = $2 } /client-hostname/ { hostname = $2; gsub(/[";]/, "", hostname) } /binding state active/ { print ip " - " hostname }' /var/lib/dhcp/dhcpd.leases
+elif [ -f /var/lib/dhcpcd5/dhcpcd.leases ]; then
+    cat /var/lib/dhcpcd5/dhcpcd.leases
+else
+    echo "DHCP lease file not found. Checking dnsmasq leases..."
+    if [ -f /var/lib/dhcp/dhcpd.leases ]; then
+        cat /var/lib/dhcp/dhcpd.leases
+    else
+        echo "No lease information available"
+        echo "Try: arp -a | grep 192.168.100"
+    fi
+fi
+EOF
+
+chmod +x /usr/local/bin/vm-ips
+
+log "Network setup completed!"
+echo ""
+info "Network Configuration Summary:"
+info "- Bridge: $BRIDGE_NAME with IP $BRIDGE_IP"
+info "- DHCP range: 192.168.100.10 - 192.168.100.100"
+info "- DNS servers: 8.8.8.8, 8.8.4.4"
+info "- NAT configured for internet access"
+echo ""
+info "To see VM IP addresses: vm-ips"
+info "To check bridge status: ip addr show $BRIDGE_NAME"
+info "To see DHCP leases: cat /var/lib/dhcp/dhcpd.leases"
+echo ""
+warn "Note: VMs need to be restarted to get DHCP IP addresses"
--- a/tools/ubuntu_install.sh
+++ b/tools/ubuntu_install.sh
@@ -0,0 +1,286 @@
+#!/bin/bash
+
+# Ubuntu 24.04 Unattended Installation Script for Hetzner Dedicated Servers
+# Uses btrfs with RAID 1 over 2 disks
+# Based on Hetzner installimage system
+
+set -e
+
+# Colors for output
+RED='\033[0;31m'
+GREEN='\033[0;32m'
+YELLOW='\033[1;33m'
+NC='\033[0m' # No Color
+
+log() {
+    echo -e "${GREEN}[$(date +'%Y-%m-%d %H:%M:%S')] $1${NC}"
+}
+
+warn() {
+    echo -e "${YELLOW}[$(date +'%Y-%m-%d %H:%M:%S')] WARNING: $1${NC}"
+}
+
+error() {
+    echo -e "${RED}[$(date +'%Y-%m-%d %H:%M:%S')] ERROR: $1${NC}"
+    exit 1
+}
+
+# Check if running in rescue system
+INSTALLIMAGE_PATH=""
+if [ -f /root/.oldroot/nfs/install/installimage ]; then
+    INSTALLIMAGE_PATH="/root/.oldroot/nfs/install/installimage"
+elif [ -f /usr/bin/installimage ]; then
+    INSTALLIMAGE_PATH="/usr/bin/installimage"
+else
+    error "This script must be run from Hetzner rescue system (installimage not found)"
+fi
+
+# Additional check for rescue system
+if [ ! -f /etc/hostname ] || [ "$(cat /etc/hostname 2>/dev/null)" != "rescue" ]; then
+    warn "Hostname is not 'rescue' - make sure you're running this in Hetzner rescue system"
+fi
+
+log "Found installimage at: $INSTALLIMAGE_PATH"
+
+# Detect available NVMe drives
+DRIVES=($(lsblk -d -n -o NAME | grep nvme | head -2))
+if [ ${#DRIVES[@]} -lt 2 ]; then
+    error "At least 2 drives are required for RAID 1 setup. Found: ${#DRIVES[@]}"
+fi
+
+DRIVE1="/dev/${DRIVES[0]}"
+DRIVE2="/dev/${DRIVES[1]}"
+
+log "Detected drives: $DRIVE1, $DRIVE2"
+
+# Get drive information
+DRIVE1_MODEL=$(lsblk -d -n -o MODEL $DRIVE1 2>/dev/null || echo "Unknown")
+DRIVE2_MODEL=$(lsblk -d -n -o MODEL $DRIVE2 2>/dev/null || echo "Unknown")
+DRIVE1_SERIAL=$(lsblk -d -n -o SERIAL $DRIVE1 2>/dev/null || echo "Unknown")
+DRIVE2_SERIAL=$(lsblk -d -n -o SERIAL $DRIVE2 2>/dev/null || echo "Unknown")
+
+log "Drive 1: $DRIVE1 - Model: $DRIVE1_MODEL, Serial: $DRIVE1_SERIAL"
+log "Drive 2: $DRIVE2 - Model: $DRIVE2_MODEL, Serial: $DRIVE2_SERIAL"
+
+# Create the installimage configuration file
+# Note: installimage expects /autosetup to be a file, not a directory
+cat > /autosetup << EOF
+## ======================================================
+##  Hetzner Online GmbH - installimage - Ubuntu 24.04 with btrfs RAID
+## ======================================================
+
+## ====================
+##  HARD DISK DRIVE(S):
+## ====================
+
+# Device Model: $DRIVE1_MODEL, Serial Number: $DRIVE1_SERIAL
+DRIVE1 $DRIVE1
+# Device Model: $DRIVE2_MODEL, Serial Number: $DRIVE2_SERIAL
+DRIVE2 $DRIVE2
+
+## ===============
+##  SOFTWARE RAID:
+## ===============
+
+## activate software RAID?  < 0 | 1 >
+SWRAID 1
+
+## Choose the level for the software RAID < 0 | 1 | 10 >
+SWRAIDLEVEL 1
+
+## ==========
+##  HOSTNAME:
+## ==========
+
+## which hostname should be set?
+HOSTNAME Ubuntu-2404-noble-amd64-btrfs
+
+## ================
+##  NETWORK CONFIG:
+## ================
+
+# IPV4_ONLY no
+
+## =============
+##  MISC CONFIG:
+## =============
+
+USE_KERNEL_MODE_SETTING yes
+
+## ==========================
+##  PARTITIONS / FILESYSTEMS:
+## ==========================
+
+## Using btrfs with subvolumes for better snapshot and management capabilities
+## RAID 1 provides redundancy across both drives
+
+# Boot partition (cannot be on btrfs RAID)
+PART /boot ext4 1024M
+
+# Swap partition
+PART swap swap 4G
+
+# Main btrfs volume for root filesystem
+PART btrfs.main btrfs all
+
+## btrfs subvolume definitions:
+## Using @ prefix convention for subvolumes
+SUBVOL btrfs.main @           /
+SUBVOL btrfs.main @home       /home
+SUBVOL btrfs.main @var        /var
+SUBVOL btrfs.main @var/log    /var/log
+SUBVOL btrfs.main @tmp        /tmp
+SUBVOL btrfs.main @opt        /opt
+SUBVOL btrfs.main @srv        /srv
+SUBVOL btrfs.main @snapshots  /.snapshots
+
+## ========================
+##  OPERATING SYSTEM IMAGE:
+## ========================
+
+## Ubuntu 24.04 LTS Noble Numbat
+IMAGE /root/.oldroot/nfs/install/../images/Ubuntu-2404-noble-amd64-base.tar.gz
+EOF
+
+log "Created installimage configuration at /autosetup"
+
+# Create autosetup directory for additional scripts
+mkdir -p /autosetup_scripts
+
+# # Create post-installation script for btrfs optimization
+# cat > /autosetup_scripts/post_install.sh << 'EOF'
+# #!/bin/bash
+
+# # Post-installation script for btrfs optimization
+
+# log() {
+#     echo "[$(date +'%Y-%m-%d %H:%M:%S')] $1"
+# }
+
+# log "Starting post-installation btrfs optimization..."
+
+# # Install btrfs-progs if not already installed
+# if ! command -v btrfs &> /dev/null; then
+#     log "Installing btrfs-progs..."
+#     apt-get update
+#     apt-get install -y btrfs-progs
+# fi
+
+# # Set btrfs mount options for better performance and features
+# log "Configuring btrfs mount options..."
+
+# # Backup original fstab
+# cp /etc/fstab /etc/fstab.backup
+
+# # Update fstab with optimized btrfs mount options
+# sed -i 's/btrfs\s\+defaults/btrfs defaults,noatime,compress=zstd:3,space_cache=v2,autodefrag/' /etc/fstab
+
+# # Create btrfs maintenance scripts
+# mkdir -p /etc/cron.weekly
+
+# # Weekly balance script
+# cat > /etc/cron.weekly/btrfs-balance << 'BALANCE_EOF'
+# #!/bin/bash
+# # Weekly btrfs balance to optimize space usage
+# /usr/bin/btrfs balance start -dusage=50 -musage=50 / 2>/dev/null || true
+# BALANCE_EOF
+
+# chmod +x /etc/cron.weekly/btrfs-balance
+
+# # Weekly scrub script for data integrity
+# cat > /etc/cron.weekly/btrfs-scrub << 'SCRUB_EOF'
+# #!/bin/bash
+# # Weekly btrfs scrub for data integrity check
+# /usr/bin/btrfs scrub start / 2>/dev/null || true
+# SCRUB_EOF
+
+# chmod +x /etc/cron.weekly/btrfs-scrub
+
+# # Install and configure snapper for automatic snapshots
+# log "Installing and configuring snapper for automatic snapshots..."
+# apt-get install -y snapper
+
+# # Create snapper config for root
+# snapper -c root create-config /
+
+# # Configure snapper for reasonable snapshot retention
+# snapper -c root set-config TIMELINE_CREATE=yes
+# snapper -c root set-config TIMELINE_CLEANUP=yes
+# snapper -c root set-config NUMBER_CLEANUP=yes
+# snapper -c root set-config NUMBER_MIN_AGE=1800
+# snapper -c root set-config NUMBER_LIMIT=50
+# snapper -c root set-config NUMBER_LIMIT_IMPORTANT=10
+
+# # Enable snapper timer
+# systemctl enable snapper-timeline.timer
+# systemctl enable snapper-cleanup.timer
+
+# log "Post-installation btrfs optimization completed"
+# EOF
+
+# chmod +x /autosetup_scripts/post_install.sh
+
+# log "Created post-installation script at /autosetup_scripts/post_install.sh"
+
+# # Create a script to monitor RAID status
+# cat > /autosetup_scripts/raid_monitor.sh << 'EOF'
+# #!/bin/bash
+
+# # RAID monitoring script for btrfs
+
+# check_btrfs_raid() {
+#     echo "=== Btrfs RAID Status ==="
+#     btrfs filesystem show
+#     echo
+#     echo "=== Btrfs Device Stats ==="
+#     btrfs device stats /
+#     echo
+#     echo "=== Btrfs Filesystem Usage ==="
+#     btrfs filesystem usage /
+# }
+
+# # Check if btrfs tools are available
+# if command -v btrfs &> /dev/null; then
+#     check_btrfs_raid
+# else
+#     echo "btrfs-progs not installed. Install with: apt-get install btrfs-progs"
+# fi
+# EOF
+
+# chmod +x /autosetup_scripts/raid_monitor.sh
+# log "Created RAID monitoring script at /autosetup_scripts/raid_monitor.sh"
+
+# Verify configuration
+log "Verifying configuration..."
+if [ ! -f /autosetup ]; then
+    error "Failed to create /autosetup configuration file"
+fi
+
+log "Configuration files created successfully:"
+log "  - /autosetup (main installation config)"
+
+# Run installimage with the configuration
+log "Starting Ubuntu 24.04 installation with btrfs RAID 1..."
+log "This will DESTROY ALL DATA on $DRIVE1 and $DRIVE2"
+
+read -p "Are you sure you want to continue? (yes/no): " confirm
+if [ "$confirm" != "yes" ]; then
+    log "Installation cancelled by user"
+    exit 0
+fi
+
+# Execute installimage
+log "Executing installimage..."
+$INSTALLIMAGE_PATH -a -c /autosetup
+
+if [ $? -eq 0 ]; then
+    log "Installation completed successfully!"
+    log "The system will reboot automatically."
+    log "After reboot, run /autosetup_scripts/post_install.sh to optimize btrfs settings"
+    log "Use /autosetup_scripts/raid_monitor.sh to monitor RAID status"
+else
+    error "Installation failed. Check the logs for details."
+fi
+
+log "Installation process completed."
+
--- a/tools/ubuntu_vm_delete.sh
+++ b/tools/ubuntu_vm_delete.sh
@@ -0,0 +1,420 @@
+#!/bin/bash
+
+# Ubuntu VM Delete Script with Comprehensive Cleanup
+# Usage: ubuntu_vm_delete.sh <vm_number|all>
+# Use 'all' to delete all VMs
+
+set -e
+
+# Colors for output
+RED='\033[0;31m'
+GREEN='\033[0;32m'
+YELLOW='\033[1;33m'
+BLUE='\033[0;34m'
+NC='\033[0m' # No Color
+
+# Configuration
+VM_BASE_DIR="/var/lib/vms"
+BTRFS_MOUNT_POINT="/var/lib/vms"
+BASE_SUBVOL="$BTRFS_MOUNT_POINT/base"
+VMS_SUBVOL="$BTRFS_MOUNT_POINT/vms"
+
+log() {
+    echo -e "${GREEN}[$(date +'%Y-%m-%d %H:%M:%S')] $1${NC}"
+}
+
+warn() {
+    echo -e "${YELLOW}[$(date +'%Y-%m-%d %H:%M:%S')] WARNING: $1${NC}"
+}
+
+error() {
+    echo -e "${RED}[$(date +'%Y-%m-%d %H:%M:%S')] ERROR: $1${NC}"
+    exit 1
+}
+
+info() {
+    echo -e "${BLUE}[$(date +'%Y-%m-%d %H:%M:%S')] INFO: $1${NC}"
+}
+
+show_usage() {
+    echo "Ubuntu VM Delete Script"
+    echo ""
+    echo "Usage: $0 <vm_number|all|list>"
+    echo ""
+    echo "Arguments:"
+    echo "  vm_number  - Number of the VM to delete (1-200)"
+    echo "  all        - Delete ALL VMs (use with extreme caution)"
+    echo "  list       - List all existing VMs"
+    echo ""
+    echo "Examples:"
+    echo "  $0 1           # Delete VM number 1"
+    echo "  $0 42          # Delete VM number 42"
+    echo "  $0 all         # Delete all VMs"
+    echo "  $0 list        # List all VMs"
+    echo ""
+    echo "This script will:"
+    echo "  - Stop running VM processes"
+    echo "  - Remove TAP network interfaces"
+    echo "  - Clean up sockets and temporary files"
+    echo "  - Delete btrfs subvolumes"
+    echo "  - Remove all VM data permanently"
+}
+
+# Check if running as root
+if [ "$EUID" -ne 0 ]; then
+    error "This script must be run as root for btrfs and network operations"
+fi
+
+# Parse arguments
+if [ $# -ne 1 ]; then
+    show_usage
+    exit 1
+fi
+
+VM_TARGET="$1"
+
+# Validate VM number (unless it's 'all' or 'list')
+if [ "$VM_TARGET" != "all" ] && [ "$VM_TARGET" != "list" ]; then
+    if ! [[ "$VM_TARGET" =~ ^[0-9]+$ ]]; then
+        error "VM number must be a number"
+    fi
+    
+    if [ "$VM_TARGET" -lt 1 ] || [ "$VM_TARGET" -gt 200 ]; then
+        error "VM number must be between 1 and 200"
+    fi
+fi
+
+# Check if VMs directory exists
+if [ ! -d "$VMS_SUBVOL" ]; then
+    warn "VMs directory not found at $VMS_SUBVOL"
+    info "No VMs to delete"
+    exit 0
+fi
+
+# Check if the base directory is on btrfs
+FILESYSTEM_TYPE=$(stat -f -c %T "$VM_BASE_DIR" 2>/dev/null)
+if [ "$FILESYSTEM_TYPE" != "btrfs" ]; then
+    error "Base directory $VM_BASE_DIR is not on a btrfs filesystem (detected: $FILESYSTEM_TYPE)"
+fi
+
+# Function to safely stop a VM process
+stop_vm_process() {
+    local vm_pid="$1"
+    local vm_name="$2"
+    
+    if [ -z "$vm_pid" ]; then
+        return 0
+    fi
+    
+    # Check if process exists
+    if ! kill -0 "$vm_pid" 2>/dev/null; then
+        info "VM process $vm_pid for '$vm_name' is not running"
+        return 0
+    fi
+    
+    log "Stopping VM process $vm_pid for '$vm_name'..."
+    
+    # Try graceful shutdown first
+    if kill -TERM "$vm_pid" 2>/dev/null; then
+        # Wait up to 10 seconds for graceful shutdown
+        local count=0
+        while [ $count -lt 10 ] && kill -0 "$vm_pid" 2>/dev/null; do
+            sleep 1
+            count=$((count + 1))
+        done
+        
+        # Force kill if still running
+        if kill -0 "$vm_pid" 2>/dev/null; then
+            warn "Graceful shutdown failed, forcing termination..."
+            kill -KILL "$vm_pid" 2>/dev/null || true
+            sleep 1
+        fi
+    fi
+    
+    # Final check
+    if kill -0 "$vm_pid" 2>/dev/null; then
+        warn "Failed to stop VM process $vm_pid"
+    else
+        log "VM process $vm_pid stopped successfully"
+    fi
+}
+
+# Function to clean up network interfaces
+cleanup_network() {
+    local tap_name="$1"
+    local bridge_name="$2"
+    local vm_name="$3"
+    
+    # Remove TAP interface
+    if [ -n "$tap_name" ]; then
+        if ip link show "$tap_name" &>/dev/null; then
+            log "Removing TAP interface '$tap_name' for VM '$vm_name'"
+            ip link delete "$tap_name" 2>/dev/null || warn "Failed to remove TAP interface '$tap_name'"
+        else
+            info "TAP interface '$tap_name' not found (already removed)"
+        fi
+    fi
+    
+    # Check if bridge still has any TAP interfaces
+    if [ -n "$bridge_name" ] && ip link show "$bridge_name" &>/dev/null; then
+        local tap_count=$(ip link show master "$bridge_name" 2>/dev/null | grep "tap-" | wc -l)
+        if [ "$tap_count" -eq 0 ]; then
+            info "Bridge '$bridge_name' has no remaining TAP interfaces"
+            # Note: We don't automatically remove the bridge as it might be used by other services
+        fi
+    fi
+}
+
+# Function to clean up VM files and sockets
+cleanup_vm_files() {
+    local vm_socket="$1"
+    local vm_number="$2"
+    
+    # Remove VM socket
+    if [ -n "$vm_socket" ] && [ -e "$vm_socket" ]; then
+        log "Removing VM socket '$vm_socket'"
+        rm -f "$vm_socket" || warn "Failed to remove VM socket '$vm_socket'"
+    fi
+    
+    # Remove log files
+    local log_file="/tmp/cloud-hypervisor-vm$vm_number.log"
+    if [ -f "$log_file" ]; then
+        log "Removing VM log file '$log_file'"
+        rm -f "$log_file" || warn "Failed to remove log file '$log_file'"
+    fi
+    
+    # Remove any other temporary files
+    rm -f "/tmp/cloud-hypervisor-vm$vm_number"* 2>/dev/null || true
+}
+
+# Function to delete a single VM
+delete_single_vm() {
+    local vm_number="$1"
+    local vm_dir="$VMS_SUBVOL/vm$vm_number"
+    local vm_info_file="$vm_dir/vm-info.txt"
+    
+    if [ ! -d "$vm_dir" ]; then
+        warn "VM number '$vm_number' not found at '$vm_dir'"
+        return 1
+    fi
+    
+    log "Deleting VM number: $vm_number"
+    
+    # Initialize variables with defaults
+    local VM_PID=""
+    local VM_SOCKET=""
+    local TAP_NAME=""
+    local BRIDGE_NAME=""
+    local VM_IMAGE_PATH=""
+    local CLOUD_INIT_PATH=""
+    
+    # Load VM info if available
+    if [ -f "$vm_info_file" ]; then
+        # Safely source the file with error handling
+        if source "$vm_info_file" 2>/dev/null; then
+            info "Loaded VM configuration from '$vm_info_file'"
+        else
+            warn "Failed to load VM configuration from '$vm_info_file', proceeding with cleanup anyway"
+        fi
+    else
+        warn "VM info file not found at '$vm_info_file', proceeding with best-effort cleanup"
+        # Try to guess some values
+        TAP_NAME="tap-vm$vm_number"
+        BRIDGE_NAME="br0"
+        VM_SOCKET="/tmp/cloud-hypervisor-vm$vm_number.sock"
+    fi
+    
+    # Stop VM process
+    if [ -n "$VM_PID" ]; then
+        stop_vm_process "$VM_PID" "vm$vm_number"
+    else
+        # Try to find the process by name
+        local found_pids=$(pgrep -f "cloud-hypervisor.*vm$vm_number" 2>/dev/null || echo "")
+        if [ -n "$found_pids" ]; then
+            warn "Found VM process(es) by name: $found_pids"
+            # Process each PID separately
+            echo "$found_pids" | while read -r pid; do
+                if [ -n "$pid" ]; then
+                    stop_vm_process "$pid" "vm$vm_number"
+                fi
+            done
+        fi
+    fi
+    
+    # Clean up network interfaces
+    cleanup_network "$TAP_NAME" "$BRIDGE_NAME" "vm$vm_number"
+    
+    # Clean up VM files and sockets
+    cleanup_vm_files "$VM_SOCKET" "$vm_number"
+    
+    # Verify the directory is a btrfs subvolume before attempting deletion
+    if btrfs subvolume show "$vm_dir" &>/dev/null; then
+        log "Deleting btrfs subvolume '$vm_dir'"
+        if ! btrfs subvolume delete "$vm_dir"; then
+            error "Failed to delete btrfs subvolume '$vm_dir'"
+        fi
+        log "Btrfs subvolume '$vm_dir' deleted successfully"
+    else
+        warn "Directory '$vm_dir' is not a btrfs subvolume, removing as regular directory"
+        if ! rm -rf "$vm_dir"; then
+            error "Failed to remove directory '$vm_dir'"
+        fi
+        log "Directory '$vm_dir' removed successfully"
+    fi
+    
+    log "VM number '$vm_number' deleted successfully"
+    return 0
+}
+
+# Function to list all VMs
+list_all_vms() {
+    local vm_list=()
+    
+    if [ ! -d "$VMS_SUBVOL" ]; then
+        return 0
+    fi
+    
+    for vm_dir in "$VMS_SUBVOL"/vm*; do
+        if [ -d "$vm_dir" ]; then
+            local vm_name=$(basename "$vm_dir")
+            # Extract number from vm directory name (vm1, vm2, etc.)
+            local vm_number=${vm_name#vm}
+            if [[ "$vm_number" =~ ^[0-9]+$ ]]; then
+                vm_list+=("$vm_number")
+            fi
+        fi
+    done
+    
+    # Sort numerically
+    if [ ${#vm_list[@]} -gt 0 ]; then
+        printf '%s\n' "${vm_list[@]}" | sort -n
+    fi
+}
+
+# Function to show detailed VM list
+show_vm_list() {
+    local vm_numbers=($(list_all_vms))
+    
+    if [ ${#vm_numbers[@]} -eq 0 ]; then
+        info "No VMs found"
+        return 0
+    fi
+    
+    echo ""
+    echo "Existing VMs:"
+    echo "============="
+    
+    for vm_number in "${vm_numbers[@]}"; do
+        local vm_dir="$VMS_SUBVOL/vm$vm_number"
+        local vm_info_file="$vm_dir/vm-info.txt"
+        local vm_ip="192.168.100.$vm_number"
+        
+        echo -n "VM $vm_number: "
+        
+        # Check if VM is running
+        if [ -f "$vm_info_file" ]; then
+            local VM_PID=""
+            local VM_NAME=""
+            local STARTED=""
+            source "$vm_info_file" 2>/dev/null || true
+            
+            if [ -n "$VM_PID" ] && kill -0 "$VM_PID" 2>/dev/null; then
+                echo -e "${GREEN}RUNNING${NC} (PID: $VM_PID, IP: $vm_ip)"
+                if [ -n "$VM_NAME" ]; then
+                    echo "         Name: $VM_NAME"
+                fi
+                if [ -n "$STARTED" ]; then
+                    echo "         Started: $STARTED"
+                fi
+            else
+                echo -e "${YELLOW}STOPPED${NC} (IP: $vm_ip)"
+                if [ -n "$VM_NAME" ]; then
+                    echo "         Name: $VM_NAME"
+                fi
+            fi
+        else
+            echo -e "${RED}UNKNOWN${NC} (no info file)"
+        fi
+    done
+    echo ""
+}
+
+# Main logic
+if [ "$VM_TARGET" = "list" ]; then
+    # List all VMs
+    show_vm_list
+    exit 0
+    
+elif [ "$VM_TARGET" = "all" ]; then
+    # Delete all VMs
+    warn "You are about to delete ALL VMs!"
+    echo ""
+    
+    # List all VMs
+    vm_numbers=($(list_all_vms))
+    
+    if [ ${#vm_numbers[@]} -eq 0 ]; then
+        info "No VMs found to delete"
+        exit 0
+    fi
+    
+    echo "VM numbers to be deleted:"
+    for vm_number in "${vm_numbers[@]}"; do
+        echo "  - VM $vm_number (IP: 192.168.100.$vm_number)"
+    done
+    echo ""
+    
+    warn "Deleting ALL VMs without confirmation..."
+    log "Proceeding with deletion of all VMs..."
+    
+    success_count=0
+    failure_count=0
+    
+    for vm_number in "${vm_numbers[@]}"; do
+        if delete_single_vm "$vm_number"; then
+            success_count=$((success_count + 1))
+        else
+            failure_count=$((failure_count + 1))
+        fi
+    done
+    
+    echo ""
+    log "Deletion summary:"
+    log "  Successfully deleted: $success_count VMs"
+    if [ $failure_count -gt 0 ]; then
+        warn "  Failed to delete: $failure_count VMs"
+    fi
+    
+    # Clean up any remaining orphaned processes
+    log "Checking for orphaned cloud-hypervisor processes..."
+    orphaned_pids=$(pgrep -f "cloud-hypervisor" 2>/dev/null || echo "")
+    if [ -n "$orphaned_pids" ]; then
+        warn "Found orphaned cloud-hypervisor processes: $orphaned_pids"
+        echo "$orphaned_pids" | xargs -r kill -TERM 2>/dev/null || true
+        sleep 2
+        echo "$orphaned_pids" | xargs -r kill -KILL 2>/dev/null || true
+    fi
+    
+    # Clean up any remaining TAP interfaces
+    log "Checking for orphaned TAP interfaces..."
+    orphaned_taps=$(ip link show | grep "tap-" | cut -d: -f2 | tr -d ' ' || echo "")
+    if [ -n "$orphaned_taps" ]; then
+        warn "Found orphaned TAP interfaces: $orphaned_taps"
+        echo "$orphaned_taps" | xargs -r -I {} ip link delete {} 2>/dev/null || true
+    fi
+    
+    log "All VMs deletion completed"
+    
+else
+    # Delete single VM
+    vm_number="$VM_TARGET"
+    
+    if [ ! -d "$VMS_SUBVOL/vm$vm_number" ]; then
+        error "VM number '$vm_number' not found"
+    fi
+    
+    log "Deleting VM number '$vm_number' without confirmation..."
+    delete_single_vm "$vm_number"
+fi
+
+log "VM deletion script completed successfully"
--- a/tools/ubuntu_vm_manage.sh
+++ b/tools/ubuntu_vm_manage.sh
@@ -0,0 +1,453 @@
+#!/bin/bash
+
+# Ubuntu VM Management Script
+# Usage: ubuntu_vm_manage.sh <command> [vm_number]
+
+set -e
+
+# Colors for output
+RED='\033[0;31m'
+GREEN='\033[0;32m'
+YELLOW='\033[1;33m'
+BLUE='\033[0;34m'
+NC='\033[0m' # No Color
+
+# Configuration
+VM_BASE_DIR="/var/lib/vms"
+VMS_SUBVOL="$VM_BASE_DIR/vms"
+
+log() {
+    echo -e "${GREEN}[$(date +'%Y-%m-%d %H:%M:%S')] $1${NC}"
+}
+
+warn() {
+    echo -e "${YELLOW}[$(date +'%Y-%m-%d %H:%M:%S')] WARNING: $1${NC}"
+}
+
+error() {
+    echo -e "${RED}[$(date +'%Y-%m-%d %H:%M:%S')] ERROR: $1${NC}"
+    exit 1
+}
+
+info() {
+    echo -e "${BLUE}[$(date +'%Y-%m-%d %H:%M:%S')] INFO: $1${NC}"
+}
+
+show_usage() {
+    echo "Ubuntu VM Management Script"
+    echo ""
+    echo "Usage: $0 <command> [vm_number]"
+    echo ""
+    echo "Commands:"
+    echo "  list                    - List all VMs and their status"
+    echo "  status <vm_number>      - Show detailed status of a specific VM"
+    echo "  console <vm_number>     - Connect to VM console (serial)"
+    echo "  ssh <vm_number>         - SSH to VM (requires network setup)"
+    echo "  stop <vm_number>        - Stop a running VM"
+    echo "  start <vm_number>       - Start a stopped VM"
+    echo "  delete <vm_number>      - Delete a VM completely"
+    echo "  ip <vm_number>          - Show VM IP address"
+    echo "  logs <vm_number>        - Show VM logs"
+    echo ""
+    echo "Examples:"
+    echo "  $0 list"
+    echo "  $0 status 1"
+    echo "  $0 console 1"
+    echo "  $0 ssh 1"
+}
+
+list_vms() {
+    log "Listing all VMs..."
+    echo ""
+    printf "%-8s %-15s %-15s %-10s %-8s %-15s %-20s\n" "VM #" "IP ADDRESS" "VM NAME" "STATUS" "PID" "MEMORY" "STARTED"
+    printf "%-8s %-15s %-15s %-10s %-8s %-15s %-20s\n" "----" "----------" "-------" "------" "---" "------" "-------"
+    
+    if [ ! -d "$VMS_SUBVOL" ]; then
+        warn "No VMs directory found at $VMS_SUBVOL"
+        return
+    fi
+    
+    for vm_dir in "$VMS_SUBVOL"/vm*; do
+        if [ -d "$vm_dir" ]; then
+            vm_dirname=$(basename "$vm_dir")
+            # Extract number from vm directory name (vm1, vm2, etc.)
+            vm_number=${vm_dirname#vm}
+            if [[ "$vm_number" =~ ^[0-9]+$ ]]; then
+                vm_ip="192.168.100.$vm_number"
+                vm_info_file="$vm_dir/vm-info.txt"
+                
+                if [ -f "$vm_info_file" ]; then
+                    # Safely source the file with error handling
+                    if source "$vm_info_file" 2>/dev/null; then
+                        # Check if VM is running
+                        if [ -n "$VM_PID" ] && kill -0 "$VM_PID" 2>/dev/null; then
+                            status="${GREEN}RUNNING${NC}"
+                            pid="$VM_PID"
+                        else
+                            status="${RED}STOPPED${NC}"
+                            pid="N/A"
+                        fi
+                        
+                        # Handle missing or malformed STARTED field
+                        if [ -z "$STARTED" ]; then
+                            STARTED="Unknown"
+                        fi
+                        
+                        # Use VM_NAME from config, fallback to vm number
+                        display_name="${VM_NAME:-vm$vm_number}"
+                        
+                        printf "%-8s %-15s %-15s %-18s %-8s %-15s %-20s\n" "$vm_number" "$vm_ip" "$display_name" "$(echo -e "$status")" "$pid" "${MEMORY_MB}MB" "$STARTED"
+                    else
+                        printf "%-8s %-15s %-15s %-18s %-8s %-15s %-20s\n" "$vm_number" "$vm_ip" "vm$vm_number" "$(printf "%b" "${RED}ERROR${NC}")" "N/A" "N/A" "Config Error"
+                    fi
+                else
+                    printf "%-8s %-15s %-15s %-18s %-8s %-15s %-20s\n" "$vm_number" "$vm_ip" "vm$vm_number" "$(printf "%b" "${YELLOW}UNKNOWN${NC}")" "N/A" "N/A" "N/A"
+                fi
+            fi
+        fi
+    done
+    echo ""
+}
+
+show_vm_status() {
+    local vm_number="$1"
+    local vm_dir="$VMS_SUBVOL/vm$vm_number"
+    local vm_info_file="$vm_dir/vm-info.txt"
+    
+    # Validate VM number
+    if ! [[ "$vm_number" =~ ^[0-9]+$ ]]; then
+        error "VM number must be a number"
+    fi
+    
+    if [ ! -d "$vm_dir" ]; then
+        error "VM number '$vm_number' not found"
+    fi
+    
+    if [ ! -f "$vm_info_file" ]; then
+        error "VM info file not found for VM number '$vm_number'"
+    fi
+    
+    source "$vm_info_file"
+    
+    log "VM Status for VM number: $vm_number"
+    echo ""
+    echo "VM Number:      $vm_number"
+    echo "VM Name:        ${VM_NAME:-vm$vm_number}"
+    echo "Static IP:      192.168.100.$vm_number"
+    echo "Memory:         ${MEMORY_MB}MB"
+    echo "CPU Cores:      $CPU_CORES"
+    echo "Started:        $STARTED"
+    echo "Image Path:     $VM_IMAGE_PATH"
+    echo "Cloud-init:     $CLOUD_INIT_PATH"
+    echo "Socket:         $VM_SOCKET"
+    echo "TAP Interface:  $TAP_NAME"
+    echo "Bridge:         $BRIDGE_NAME"
+    
+    # Check if VM is running
+    if [ -n "$VM_PID" ] && kill -0 "$VM_PID" 2>/dev/null; then
+        echo -e "Status:         ${GREEN}RUNNING${NC} (PID: $VM_PID)"
+        
+        # Show network info
+        if ip link show "$TAP_NAME" &>/dev/null; then
+            echo -e "Network:        ${GREEN}TAP interface active${NC}"
+        else
+            echo -e "Network:        ${RED}TAP interface not found${NC}"
+        fi
+        
+        # Show socket info
+        if [ -S "$VM_SOCKET" ]; then
+            echo -e "API Socket:     ${GREEN}Available${NC}"
+        else
+            echo -e "API Socket:     ${RED}Not available${NC}"
+        fi
+    else
+        echo -e "Status:         ${RED}STOPPED${NC}"
+    fi
+    echo ""
+}
+
+connect_console() {
+    local vm_number="$1"
+    local vm_dir="$VMS_SUBVOL/vm$vm_number"
+    local vm_info_file="$vm_dir/vm-info.txt"
+    
+    # Validate VM number
+    if ! [[ "$vm_number" =~ ^[0-9]+$ ]]; then
+        error "VM number must be a number"
+    fi
+    
+    if [ ! -f "$vm_info_file" ]; then
+        error "VM number '$vm_number' not found"
+    fi
+    
+    source "$vm_info_file"
+    
+    if [ -z "$VM_PID" ] || ! kill -0 "$VM_PID" 2>/dev/null; then
+        error "VM number '$vm_number' is not running"
+    fi
+    
+    info "Connecting to console for VM number '$vm_number'"
+    info "Press Ctrl+A then X to exit console"
+    echo ""
+    
+    # Connect to the VM's console via the API socket
+    if [ -S "$VM_SOCKET" ]; then
+        # Use socat to connect to the console
+        if command -v socat &>/dev/null; then
+            socat - UNIX-CONNECT:"$VM_SOCKET"
+        else
+            warn "socat not found. Installing..."
+            apt update && apt install -y socat
+            socat - UNIX-CONNECT:"$VM_SOCKET"
+        fi
+    else
+        error "VM socket not found at $VM_SOCKET"
+    fi
+}
+
+ssh_to_vm() {
+    local vm_number="$1"
+    local vm_ip="192.168.100.$vm_number"
+    
+    # Validate VM number
+    if ! [[ "$vm_number" =~ ^[0-9]+$ ]]; then
+        error "VM number must be a number"
+    fi
+    
+    info "Attempting SSH connection to VM number $vm_number at $vm_ip"
+    info "Default login: ubuntu / ubuntu"
+    echo ""
+    
+    # Try to SSH directly
+    if command -v sshpass &>/dev/null; then
+        info "Using sshpass for automatic login..."
+        sshpass -p 'ubuntu' ssh -o StrictHostKeyChecking=no ubuntu@"$vm_ip"
+    else
+        info "Manual SSH connection (enter password 'ubuntu'):"
+        ssh -o StrictHostKeyChecking=no ubuntu@"$vm_ip"
+    fi
+}
+
+stop_vm() {
+    local vm_number="$1"
+    local vm_dir="$VMS_SUBVOL/vm$vm_number"
+    local vm_info_file="$vm_dir/vm-info.txt"
+    
+    # Validate VM number
+    if ! [[ "$vm_number" =~ ^[0-9]+$ ]]; then
+        error "VM number must be a number"
+    fi
+    
+    if [ ! -f "$vm_info_file" ]; then
+        error "VM number '$vm_number' not found"
+    fi
+    
+    source "$vm_info_file"
+    
+    if [ -z "$VM_PID" ] || ! kill -0 "$VM_PID" 2>/dev/null; then
+        warn "VM number '$vm_number' is not running"
+        return
+    fi
+    
+    log "Stopping VM number '$vm_number' (PID: $VM_PID)..."
+    
+    # Graceful shutdown first
+    kill -TERM "$VM_PID" 2>/dev/null || true
+    
+    # Wait a bit for graceful shutdown
+    sleep 3
+    
+    # Force kill if still running
+    if kill -0 "$VM_PID" 2>/dev/null; then
+        warn "Forcing shutdown..."
+        kill -KILL "$VM_PID" 2>/dev/null || true
+    fi
+    
+    # Cleanup network
+    if [ -n "$TAP_NAME" ]; then
+        ip link delete "$TAP_NAME" 2>/dev/null || true
+    fi
+    
+    # Remove socket
+    if [ -n "$VM_SOCKET" ]; then
+        rm -f "$VM_SOCKET"
+    fi
+    
+    log "VM number '$vm_number' stopped"
+}
+
+delete_vm() {
+    local vm_number="$1"
+    local vm_dir="$VMS_SUBVOL/vm$vm_number"
+    
+    # Validate VM number
+    if ! [[ "$vm_number" =~ ^[0-9]+$ ]]; then
+        error "VM number must be a number"
+    fi
+    
+    if [ ! -d "$vm_dir" ]; then
+        error "VM number '$vm_number' not found"
+    fi
+    
+    # Stop VM first if running
+    if [ -f "$vm_dir/vm-info.txt" ]; then
+        source "$vm_dir/vm-info.txt"
+        if [ -n "$VM_PID" ] && kill -0 "$VM_PID" 2>/dev/null; then
+            log "Stopping VM before deletion..."
+            stop_vm "$vm_number"
+        fi
+    fi
+    
+    # Confirm deletion
+    echo -e "${RED}WARNING: This will permanently delete VM number '$vm_number' and all its data!${NC}"
+    read -p "Are you sure? (yes/no): " confirm
+    
+    if [ "$confirm" = "yes" ]; then
+        log "Deleting VM number '$vm_number'..."
+        btrfs subvolume delete "$vm_dir"
+        log "VM number '$vm_number' deleted successfully"
+    else
+        info "Deletion cancelled"
+    fi
+}
+
+start_vm() {
+    local vm_number="$1"
+    local vm_dir="$VMS_SUBVOL/vm$vm_number"
+    local vm_info_file="$vm_dir/vm-info.txt"
+    
+    # Validate VM number
+    if ! [[ "$vm_number" =~ ^[0-9]+$ ]]; then
+        error "VM number must be a number"
+    fi
+    
+    if [ ! -d "$vm_dir" ]; then
+        error "VM number '$vm_number' not found"
+    fi
+    
+    if [ ! -f "$vm_info_file" ]; then
+        error "VM info file not found for VM number '$vm_number'"
+    fi
+    
+    source "$vm_info_file"
+    
+    # Check if VM is already running
+    if [ -n "$VM_PID" ] && kill -0 "$VM_PID" 2>/dev/null; then
+        warn "VM number '$vm_number' is already running (PID: $VM_PID)"
+        return
+    fi
+    
+    log "Starting VM number '$vm_number'..."
+    
+    # Create TAP interface
+    if ! ip link show "$TAP_NAME" &>/dev/null; then
+        log "Creating TAP interface $TAP_NAME..."
+        ip tuntap add dev "$TAP_NAME" mode tap
+        ip link set dev "$TAP_NAME" up
+        ip link set dev "$TAP_NAME" master "$BRIDGE_NAME"
+    fi
+    
+    # Remove existing socket if it exists
+    rm -f "$VM_SOCKET"
+    
+    # Start Cloud Hypervisor in background with kernel boot
+    KERNEL_PATH="/var/lib/vms/base/vmlinuz"
+    INITRD_PATH="/var/lib/vms/base/initrd.img"
+    
+    cloud-hypervisor \
+        --api-socket "$VM_SOCKET" \
+        --memory "size=${MEMORY_MB}M" \
+        --cpus "boot=$CPU_CORES" \
+        --kernel "$KERNEL_PATH" \
+        --initramfs "$INITRD_PATH" \
+        --cmdline "root=LABEL=cloudimg-rootfs ro console=tty1 console=ttyS0" \
+        --disk "path=$VM_IMAGE_PATH" "path=$CLOUD_INIT_PATH,readonly=on" \
+        --net "tap=$TAP_NAME,mac=52:54:00:$(printf '%02x:%02x:%02x' $((RANDOM%256)) $((RANDOM%256)) $((RANDOM%256)))" \
+        --serial tty \
+        --console off \
+        --log-file /tmp/cloud-hypervisor-$VM_NAME.log &
+    
+    NEW_VM_PID=$!
+    
+    # Update VM info file with new PID
+    sed -i "s/VM_PID=.*/VM_PID=$NEW_VM_PID/" "$vm_info_file"
+    
+    log "VM number '$vm_number' started with PID $NEW_VM_PID"
+    log "VM socket: $VM_SOCKET"
+    log "TAP interface: $TAP_NAME"
+    log "To connect to console: ./ubuntu_vm_manage.sh console $vm_number"
+}
+
+# Main script logic
+if [ $# -eq 0 ]; then
+    show_usage
+    exit 1
+fi
+
+COMMAND="$1"
+VM_NUMBER="$2"
+
+case "$COMMAND" in
+    "list")
+        list_vms
+        ;;
+    "status")
+        if [ -z "$VM_NUMBER" ]; then
+            error "VM number required for status command"
+        fi
+        show_vm_status "$VM_NUMBER"
+        ;;
+    "console")
+        if [ -z "$VM_NUMBER" ]; then
+            error "VM number required for console command"
+        fi
+        connect_console "$VM_NUMBER"
+        ;;
+    "ssh")
+        if [ -z "$VM_NUMBER" ]; then
+            error "VM number required for ssh command"
+        fi
+        ssh_to_vm "$VM_NUMBER"
+        ;;
+    "stop")
+        if [ -z "$VM_NUMBER" ]; then
+            error "VM number required for stop command"
+        fi
+        stop_vm "$VM_NUMBER"
+        ;;
+    "start")
+        if [ -z "$VM_NUMBER" ]; then
+            error "VM number required for start command"
+        fi
+        start_vm "$VM_NUMBER"
+        ;;
+    "delete")
+        if [ -z "$VM_NUMBER" ]; then
+            error "VM number required for delete command"
+        fi
+        delete_vm "$VM_NUMBER"
+        ;;
+    "ip")
+        if [ -z "$VM_NUMBER" ]; then
+            error "VM number required for ip command"
+        fi
+        echo "VM $VM_NUMBER IP address: 192.168.100.$VM_NUMBER"
+        ;;
+    "logs")
+        if [ -z "$VM_NUMBER" ]; then
+            error "VM number required for logs command"
+        fi
+        log_file="/tmp/cloud-hypervisor-vm$VM_NUMBER.log"
+        if [ -f "$log_file" ]; then
+            log "Showing logs for VM number $VM_NUMBER:"
+            tail -f "$log_file"
+        else
+            warn "Log file not found: $log_file"
+        fi
+        ;;
+    *)
+        error "Unknown command: $COMMAND"
+        show_usage
+        exit 1
+        ;;
+esac
--- a/tools/ubuntu_vm_start.sh
+++ b/tools/ubuntu_vm_start.sh
@@ -0,0 +1,867 @@
+#!/bin/bash
+
+# Ubuntu VM Start Script with Cloud Hypervisor and Btrfs Thin Provisioning
+# Usage: ubuntu_vm_start.sh $vm_number $name $mem $cores
+
+set -e
+
+# Colors for output
+RED='\033[0;31m'
+GREEN='\033[0;32m'
+YELLOW='\033[1;33m'
+BLUE='\033[0;34m'
+NC='\033[0m' # No Color
+
+# Configuration
+BASE_IMAGE_NAME="ubuntu-24.04-server-cloudimg-amd64"
+BASE_IMAGE_URL="https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img"
+FIRMWARE_URL="https://github.com/cloud-hypervisor/rust-hypervisor-firmware/releases/download/0.5.0/hypervisor-fw"
+VM_BASE_DIR="/var/lib/vms"
+BTRFS_MOUNT_POINT="/var/lib/vms"
+BASE_SUBVOL="$BTRFS_MOUNT_POINT/base"
+VMS_SUBVOL="$BTRFS_MOUNT_POINT/vms"
+
+# Network configuration
+BRIDGE_NAME="br0"
+BRIDGE_IP="192.168.100.1/24"
+NETWORK="192.168.100.0/24"
+
+log() {
+    echo -e "${GREEN}[$(date +'%Y-%m-%d %H:%M:%S')] $1${NC}" >&2
+}
+
+warn() {
+    echo -e "${YELLOW}[$(date +'%Y-%m-%d %H:%M:%S')] WARNING: $1${NC}" >&2
+}
+
+error() {
+    echo -e "${RED}[$(date +'%Y-%m-%d %H:%M:%S')] ERROR: $1${NC}" >&2
+    
+    # If VM_NUMBER is set and we're in VM creation phase, clean up
+    if [ -n "$VM_NUMBER" ] && [ -n "$VM_PID" ]; then
+        cleanup_failed_vm "$VM_NUMBER"
+    fi
+    
+    exit 1
+}
+
+info() {
+    echo -e "${BLUE}[$(date +'%Y-%m-%d %H:%M:%S')] INFO: $1${NC}" >&2
+}
+
+# Test functions
+test_step() {
+    local step_name="$1"
+    local test_command="$2"
+    
+    info "Testing: $step_name"
+    if eval "$test_command"; then
+        log "✓ Test passed: $step_name"
+        return 0
+    else
+        error "✗ Test failed: $step_name"
+        return 1
+    fi
+}
+
+test_file_exists() {
+    local file_path="$1"
+    local description="$2"
+    test_step "$description" "[ -f '$file_path' ]"
+}
+
+test_directory_exists() {
+    local dir_path="$1"
+    local description="$2"
+    test_step "$description" "[ -d '$dir_path' ]"
+}
+
+test_command_exists() {
+    local command="$1"
+    local description="$2"
+    test_step "$description" "command -v '$command' &> /dev/null"
+}
+
+test_network_interface() {
+    local interface="$1"
+    local description="$2"
+    test_step "$description" "ip link show '$interface' &>/dev/null"
+}
+
+test_process_running() {
+    local pid="$1"
+    local description="$2"
+    test_step "$description" "kill -0 '$pid' 2>/dev/null"
+}
+
+# Cleanup function for failed VM creation
+cleanup_failed_vm() {
+    local vm_number="$1"
+    warn "VM creation failed, cleaning up..."
+    
+    # Call the delete script to clean up
+    local delete_script="$(dirname "$0")/ubuntu_vm_delete.sh"
+    if [ -f "$delete_script" ]; then
+        log "Running cleanup script: $delete_script"
+        "$delete_script" "$vm_number" || warn "Cleanup script failed, manual cleanup may be required"
+    else
+        warn "Delete script not found at $delete_script, manual cleanup required"
+    fi
+}
+
+# Generate a proper password hash for 'ubuntu'
+generate_password_hash() {
+    # Generate salt and hash for password 'ubuntu'
+    python3 -c "import crypt; print(crypt.crypt('ubuntu', crypt.mksalt(crypt.METHOD_SHA512)))"
+}
+
+# Wait for VM to boot and verify static IP
+wait_for_vm_boot() {
+    local vm_name="$1"
+    local expected_ip="$2"
+    local max_wait=180  # 3 minutes, increased from 120
+    local count=0
+    
+    log "Waiting for VM '$vm_name' to boot with static IP $expected_ip..."
+    
+    while [ $count -lt $max_wait ]; do
+        # Check if VM process is still running
+        if ! kill -0 "$VM_PID" 2>/dev/null; then
+            error "VM process died while waiting for boot. Check log: $VM_LOG_FILE"
+        fi
+        
+        # Try to ping the expected static IP
+        if ping -c 2 -W 3 "$expected_ip" >/dev/null 2>&1; then
+            log "VM is responding at static IP address: $expected_ip"
+            echo "$expected_ip"
+            return 0
+        fi
+        
+        # Also check ARP table for our MAC address
+        local vm_ip=$(arp -a | grep "$VM_MAC" | sed 's/.*(\([^)]*\)).*/\1/' | head -1)
+        if [ -n "$vm_ip" ] && [ "$vm_ip" = "$expected_ip" ]; then
+            log "VM MAC address found in ARP table with expected IP: $expected_ip"
+            echo "$expected_ip"
+            return 0
+        fi
+        
+        sleep 3
+        count=$((count + 3))
+        if [ $((count % 15)) -eq 0 ]; then
+            info "Still waiting for VM to boot... ($count/$max_wait seconds)"
+            info "VM process PID $VM_PID is still running"
+            info "Expected static IP: $expected_ip"
+            # Show recent log entries
+            if [ -f "$VM_LOG_FILE" ]; then
+                info "Recent VM log entries:"
+                tail -3 "$VM_LOG_FILE" 2>/dev/null || true
+            fi
+        fi
+    done
+    
+    warn "VM did not respond at expected IP $expected_ip within $max_wait seconds"
+    warn "VM may still be booting - check manually with: ping $expected_ip"
+    return 1
+}
+
+# Test IP connectivity
+test_ip_connectivity() {
+    local vm_ip="$1"
+    local max_attempts=10
+    local attempt=1
+    
+    log "Testing IP connectivity to $vm_ip..."
+    
+    while [ $attempt -le $max_attempts ]; do
+        info "Ping attempt $attempt/$max_attempts to $vm_ip"
+        
+        # Test ping connectivity with timeout
+        if ping -c 3 -W 2 "$vm_ip" >/dev/null 2>&1; then
+            log "✓ IP connectivity successful to $vm_ip"
+            return 0
+        fi
+        
+        sleep 3
+        attempt=$((attempt + 1))
+    done
+    
+    error "✗ IP connectivity failed after $max_attempts attempts to $vm_ip"
+    return 1
+}
+
+# Test SSH connectivity
+test_ssh_connection() {
+    local vm_ip="$1"
+    local max_attempts=10
+    local attempt=1
+    
+    log "Testing SSH connectivity to $vm_ip..."
+    
+    while [ $attempt -le $max_attempts ]; do
+        info "SSH attempt $attempt/$max_attempts"
+        
+        # Test SSH connection with timeout, using key-based auth, with verbose output for diagnostics
+        info "Attempting SSH with key: timeout 10 ssh -i \"$SELECTED_HOST_ROOT_SSH_PRIV_KEY_PATH\" -v -o StrictHostKeyChecking=no -o ConnectTimeout=5 ubuntu@\"$vm_ip\" 'echo \"SSH connection successful\"'" >&2
+        if timeout 10 ssh -i "$SELECTED_HOST_ROOT_SSH_PRIV_KEY_PATH" -v -o StrictHostKeyChecking=no -o ConnectTimeout=5 ubuntu@"$vm_ip" 'echo "SSH connection successful"'; then
+            log "✓ SSH connection successful to ubuntu@$vm_ip with key $SELECTED_HOST_ROOT_SSH_PRIV_KEY_PATH" >&2
+            return 0
+        fi
+        
+        sleep 5
+        attempt=$((attempt + 1))
+    done
+    
+    error "✗ SSH connection failed after $max_attempts attempts"
+    return 1
+}
+
+# Check if running as root
+if [ "$EUID" -ne 0 ]; then
+    error "This script must be run as root for btrfs operations"
+fi
+
+# Parse arguments
+if [ $# -ne 4 ]; then
+    error "Usage: $0 <vm_number> <vm_name> <memory_mb> <cpu_cores>"
+fi
+
+VM_NUMBER="$1"
+VM_NAME="$2"
+MEMORY_MB="$3"
+CPU_CORES="$4"
+
+# Validate arguments
+if ! [[ "$VM_NUMBER" =~ ^[0-9]+$ ]]; then
+    error "VM number must be a number"
+fi
+
+if [ "$VM_NUMBER" -lt 1 ] || [ "$VM_NUMBER" -gt 200 ]; then
+    error "VM number must be between 1 and 200"
+fi
+
+if ! [[ "$MEMORY_MB" =~ ^[0-9]+$ ]]; then
+    error "Memory must be a number (in MB)"
+fi
+
+if ! [[ "$CPU_CORES" =~ ^[0-9]+$ ]]; then
+    error "CPU cores must be a number"
+fi
+
+if [[ "$VM_NAME" =~ [^a-zA-Z0-9_-] ]]; then
+    error "VM name can only contain alphanumeric characters, hyphens, and underscores"
+fi
+
+# Calculate static IP address based on VM number
+VM_STATIC_IP="192.168.100.$VM_NUMBER"
+
+log "Starting VM: $VM_NAME (number $VM_NUMBER) with ${MEMORY_MB}MB RAM and $CPU_CORES CPU cores" >&2
+log "VM will be assigned static IP: $VM_STATIC_IP" >&2
+
+# SSH Key Configuration for Host Root to VM Ubuntu User
+HOST_ROOT_SSH_DIR="/root/.ssh"
+HOST_ROOT_SSH_PUB_KEY_PATH_ED25519="$HOST_ROOT_SSH_DIR/id_ed25519.pub"
+HOST_ROOT_SSH_PRIV_KEY_PATH_ED25519="$HOST_ROOT_SSH_DIR/id_ed25519"
+HOST_ROOT_SSH_PUB_KEY_PATH_RSA="$HOST_ROOT_SSH_DIR/id_rsa.pub"
+HOST_ROOT_SSH_PRIV_KEY_PATH_RSA="$HOST_ROOT_SSH_DIR/id_rsa"
+
+SELECTED_HOST_ROOT_SSH_PUB_KEY_PATH=""
+SELECTED_HOST_ROOT_SSH_PRIV_KEY_PATH=""
+
+mkdir -p "$HOST_ROOT_SSH_DIR"
+chmod 700 "$HOST_ROOT_SSH_DIR"
+
+if [ -f "$HOST_ROOT_SSH_PUB_KEY_PATH_ED25519" ]; then
+    SELECTED_HOST_ROOT_SSH_PUB_KEY_PATH="$HOST_ROOT_SSH_PUB_KEY_PATH_ED25519"
+    SELECTED_HOST_ROOT_SSH_PRIV_KEY_PATH="$HOST_ROOT_SSH_PRIV_KEY_PATH_ED25519"
+    log "Using existing ED25519 SSH key: $SELECTED_HOST_ROOT_SSH_PUB_KEY_PATH" >&2
+elif [ -f "$HOST_ROOT_SSH_PUB_KEY_PATH_RSA" ]; then
+    SELECTED_HOST_ROOT_SSH_PUB_KEY_PATH="$HOST_ROOT_SSH_PUB_KEY_PATH_RSA"
+    SELECTED_HOST_ROOT_SSH_PRIV_KEY_PATH="$HOST_ROOT_SSH_PRIV_KEY_PATH_RSA"
+    log "Using existing RSA SSH key: $SELECTED_HOST_ROOT_SSH_PUB_KEY_PATH" >&2
+else
+    log "No existing SSH key found for root. Generating ED25519 key..." >&2
+    ssh-keygen -t ed25519 -N "" -f "$HOST_ROOT_SSH_PRIV_KEY_PATH_ED25519" -q
+    if [ $? -eq 0 ] && [ -f "$HOST_ROOT_SSH_PUB_KEY_PATH_ED25519" ]; then
+        SELECTED_HOST_ROOT_SSH_PUB_KEY_PATH="$HOST_ROOT_SSH_PUB_KEY_PATH_ED25519"
+        SELECTED_HOST_ROOT_SSH_PRIV_KEY_PATH="$HOST_ROOT_SSH_PRIV_KEY_PATH_ED25519"
+        log "Generated ED25519 SSH key: $SELECTED_HOST_ROOT_SSH_PUB_KEY_PATH" >&2
+        chmod 600 "$HOST_ROOT_SSH_PRIV_KEY_PATH_ED25519"
+        chmod 644 "$HOST_ROOT_SSH_PUB_KEY_PATH_ED25519"
+    else
+        error "Failed to generate ED25519 SSH key."
+    fi
+fi
+
+if [ -z "$SELECTED_HOST_ROOT_SSH_PUB_KEY_PATH" ]; then
+    error "Could not find or generate an SSH public key for root user."
+fi
+
+HOST_ROOT_SSH_PUB_KEY_CONTENT=$(cat "$SELECTED_HOST_ROOT_SSH_PUB_KEY_PATH")
+if [ -z "$HOST_ROOT_SSH_PUB_KEY_CONTENT" ]; then
+    error "SSH public key content is empty at $SELECTED_HOST_ROOT_SSH_PUB_KEY_PATH"
+fi
+
+# Comprehensive prerequisite checks
+log "Performing prerequisite checks..."
+
+# Check if cloud-hypervisor is available
+test_command_exists "cloud-hypervisor" "Cloud Hypervisor installation"
+
+# Check if qemu-img is available (for image conversion)
+if ! command -v qemu-img &> /dev/null; then
+    warn "qemu-img not found. Installing qemu-utils..."
+    apt update && apt install -y qemu-utils
+    test_command_exists "qemu-img" "QEMU tools installation"
+fi
+
+# Check for required tools
+test_command_exists "curl" "curl installation"
+test_command_exists "btrfs" "btrfs tools installation"
+test_command_exists "ip" "iproute2 tools installation"
+
+# Check if ethtool is available (for TAP interface optimization)
+if ! command -v ethtool &> /dev/null; then
+    log "Installing ethtool for network interface optimization..."
+    apt update && apt install -y ethtool
+    test_command_exists "ethtool" "ethtool installation"
+fi
+
+# Check if sshpass is available (for SSH testing)
+if ! command -v sshpass &> /dev/null; then
+    log "Installing sshpass for SSH testing..."
+    apt update && apt install -y sshpass
+    test_command_exists "sshpass" "sshpass installation"
+fi
+
+# Check if python3 is available (for password hashing)
+test_command_exists "python3" "Python3 installation"
+
+# Check if genisoimage or mkisofs is available
+if ! command -v genisoimage &> /dev/null && ! command -v mkisofs &> /dev/null; then
+    log "Installing genisoimage for cloud-init ISO creation..."
+    apt update && apt install -y genisoimage
+fi
+
+log "✓ All prerequisites checked"
+
+# Create base directory structure
+log "Setting up storage structure..."
+mkdir -p "$VM_BASE_DIR"
+test_directory_exists "$VM_BASE_DIR" "VM base directory creation"
+
+# Check if the base directory is on btrfs
+FILESYSTEM_TYPE=$(stat -f -c %T "$VM_BASE_DIR" 2>/dev/null)
+if [ "$FILESYSTEM_TYPE" != "btrfs" ]; then
+    error "Base directory $VM_BASE_DIR is not on a btrfs filesystem (detected: $FILESYSTEM_TYPE). Please create a btrfs filesystem first."
+fi
+
+log "✓ Btrfs filesystem detected at $VM_BASE_DIR"
+
+# Create base and vms subvolumes if they don't exist
+if [ ! -d "$BASE_SUBVOL" ]; then
+    log "Creating base subvolume at $BASE_SUBVOL"
+    btrfs subvolume create "$BASE_SUBVOL"
+    test_directory_exists "$BASE_SUBVOL" "Base subvolume creation"
+else
+    log "✓ Base subvolume already exists"
+fi
+
+if [ ! -d "$VMS_SUBVOL" ]; then
+    log "Creating VMs subvolume at $VMS_SUBVOL"
+    btrfs subvolume create "$VMS_SUBVOL"
+    test_directory_exists "$VMS_SUBVOL" "VMs subvolume creation"
+else
+    log "✓ VMs subvolume already exists"
+fi
+
+# Verify subvolumes are properly created
+test_step "Base subvolume verification" "btrfs subvolume show '$BASE_SUBVOL' &>/dev/null"
+test_step "VMs subvolume verification" "btrfs subvolume show '$VMS_SUBVOL' &>/dev/null"
+
+# Define paths
+BASE_IMAGE_PATH="$BASE_SUBVOL/${BASE_IMAGE_NAME}.raw"
+FIRMWARE_PATH="$BASE_SUBVOL/hypervisor-fw"
+VM_SUBVOL_PATH="$VMS_SUBVOL/vm$VM_NUMBER"
+VM_IMAGE_PATH="$VM_SUBVOL_PATH/vm$VM_NUMBER.raw"
+CLOUD_INIT_PATH="$VM_SUBVOL_PATH/cloud-init.img"
+
+# Download and prepare base image if it doesn't exist
+log "Preparing base image and firmware..."
+if [ ! -f "$BASE_IMAGE_PATH" ]; then
+    log "Base image not found. Downloading Ubuntu cloud image..."
+    
+    # Download the qcow2 image
+    TEMP_QCOW2="/tmp/${BASE_IMAGE_NAME}.img"
+    if ! curl -L --fail --progress-bar -o "$TEMP_QCOW2" "$BASE_IMAGE_URL"; then
+        error "Failed to download Ubuntu cloud image from $BASE_IMAGE_URL"
+    fi
+    test_file_exists "$TEMP_QCOW2" "Ubuntu cloud image download"
+    
+    log "Converting qcow2 image to raw format..."
+    qemu-img convert -p -f qcow2 -O raw "$TEMP_QCOW2" "$BASE_IMAGE_PATH"
+    test_file_exists "$BASE_IMAGE_PATH" "Base image conversion"
+    
+    # Verify the converted image
+    image_info=$(qemu-img info "$BASE_IMAGE_PATH" 2>/dev/null)
+    if echo "$image_info" | grep -q "file format: raw"; then
+        log "✓ Base image successfully converted to raw format"
+    else
+        error "Base image conversion verification failed"
+    fi
+    
+    # Cleanup temporary file
+    rm -f "$TEMP_QCOW2"
+    
+    log "✓ Base image created at $BASE_IMAGE_PATH"
+else
+    log "✓ Base image already exists at $BASE_IMAGE_PATH"
+    test_file_exists "$BASE_IMAGE_PATH" "Base image verification"
+fi
+
+# Download firmware if it doesn't exist
+if [ ! -f "$FIRMWARE_PATH" ]; then
+    log "Downloading Cloud Hypervisor firmware..."
+    if ! curl -L --fail --progress-bar -o "$FIRMWARE_PATH" "$FIRMWARE_URL"; then
+        error "Failed to download firmware from $FIRMWARE_URL"
+    fi
+    test_file_exists "$FIRMWARE_PATH" "Firmware download"
+    chmod +x "$FIRMWARE_PATH"
+    test_step "Firmware executable check" "[ -x '$FIRMWARE_PATH' ]"
+    log "✓ Firmware downloaded to $FIRMWARE_PATH"
+    else
+        log "✓ Firmware already exists at $FIRMWARE_PATH"
+        test_file_exists "$FIRMWARE_PATH" "Firmware verification"
+    fi
+    
+    # Extract kernel and initrd from base image
+    log "Extracting kernel and initrd for kernel boot..."
+    EXTRACT_SCRIPT="$(dirname "$0")/extract_kernel.sh"
+    if [ -f "$EXTRACT_SCRIPT" ]; then
+        "$EXTRACT_SCRIPT"
+    else
+        warn "Kernel extraction script not found, attempting manual extraction..."
+        # Fallback manual extraction
+        KERNEL_PATH="$BASE_SUBVOL/vmlinuz"
+        INITRD_PATH="$BASE_SUBVOL/initrd.img"
+        if [ ! -f "$KERNEL_PATH" ] || [ ! -f "$INITRD_PATH" ]; then
+            log "Extracting kernel and initrd manually..."
+            TEMP_MOUNT=$(mktemp -d)
+            losetup -P /dev/loop1 "$BASE_IMAGE_PATH"
+            mount /dev/loop1p16 "$TEMP_MOUNT"
+            cp "$TEMP_MOUNT/vmlinuz-6.8.0-60-generic" "$KERNEL_PATH" 2>/dev/null || true
+            cp "$TEMP_MOUNT/initrd.img-6.8.0-60-generic" "$INITRD_PATH" 2>/dev/null || true
+            umount "$TEMP_MOUNT"
+            losetup -d /dev/loop1
+            rmdir "$TEMP_MOUNT"
+        fi
+    fi
+    
+    test_file_exists "$BASE_SUBVOL/vmlinuz" "Kernel extraction"
+    test_file_exists "$BASE_SUBVOL/initrd.img" "Initrd extraction"
+
+# Create VM subvolume by cloning from base
+log "Setting up VM-specific storage..."
+if [ -d "$VM_SUBVOL_PATH" ]; then
+    warn "VM subvolume $VM_NAME already exists. Removing it..."
+    if btrfs subvolume show "$VM_SUBVOL_PATH" &>/dev/null; then
+        btrfs subvolume delete "$VM_SUBVOL_PATH"
+    else
+        rm -rf "$VM_SUBVOL_PATH"
+    fi
+    test_step "VM subvolume cleanup" "[ ! -d '$VM_SUBVOL_PATH' ]"
+fi
+
+log "Creating VM subvolume by cloning base subvolume..."
+btrfs subvolume snapshot "$BASE_SUBVOL" "$VM_SUBVOL_PATH"
+test_directory_exists "$VM_SUBVOL_PATH" "VM subvolume creation"
+test_step "VM subvolume verification" "btrfs subvolume show '$VM_SUBVOL_PATH' &>/dev/null"
+
+# Copy the base image to VM subvolume (this will be a CoW copy initially)
+log "Creating VM disk image (thin provisioned)..."
+cp --reflink=always "$BASE_IMAGE_PATH" "$VM_IMAGE_PATH"
+test_file_exists "$VM_IMAGE_PATH" "VM disk image creation"
+
+# Verify the image copy
+vm_image_size=$(stat -c%s "$VM_IMAGE_PATH" 2>/dev/null)
+base_image_size=$(stat -c%s "$BASE_IMAGE_PATH" 2>/dev/null)
+if [ "$vm_image_size" = "$base_image_size" ]; then
+    log "✓ VM disk image successfully created (size: $vm_image_size bytes)"
+else
+    error "VM disk image size mismatch (VM: $vm_image_size, Base: $base_image_size)"
+fi
+
+# Create cloud-init image for first boot
+log "Creating cloud-init configuration..."
+
+# Generate a random MAC address for the VM (used in cloud-init and hypervisor)
+VM_MAC="52:54:00:$(printf '%02x:%02x:%02x' $((RANDOM%256)) $((RANDOM%256)) $((RANDOM%256)))"
+log "Generated MAC address for VM: $VM_MAC"
+
+# Generate proper password hash for 'ubuntu'
+# PASSWORD_HASH=$(generate_password_hash) # No longer needed for ubuntu user
+# test_step "Password hash generation" "[ -n '$PASSWORD_HASH' ]"
+
+cat > "/tmp/user-data" << EOF
+#cloud-config
+users:
+  - name: ubuntu
+    sudo: ALL=(ALL) NOPASSWD:ALL
+    shell: /bin/bash
+    # lock_passwd: false # Not needed with key auth
+    # passwd: $PASSWORD_HASH # Using SSH key instead
+    groups: sudo
+    home: /home/ubuntu
+    ssh_authorized_keys:
+      - ${HOST_ROOT_SSH_PUB_KEY_CONTENT}
+
+# ssh_pwauth: true # Disabling password auth for ubuntu user by not setting a password
+disable_root: false # Keep root disabled for direct login, sudo is available
+chpasswd: # This section might be irrelevant now for ubuntu user
+  expire: false
+
+# SSH configuration
+# ssh_authorized_keys: [] # This was a global setting, moved to user-specific
+
+# Network configuration with static IP
+network:
+  config: disabled # This disables cloud-init's direct network config, relying on write_files + runcmd
+
+write_files:
+  - path: /etc/netplan/50-cloud-init.yaml
+    content: |
+      network:
+        version: 2
+        ethernets:
+          ens3:
+            dhcp4: false
+            addresses:
+              - $VM_STATIC_IP/24 # Use variable for IP
+            # gateway4: 192.168.100.1 # Deprecated
+            routes: # Use modern routes syntax
+              - to: default
+                via: 192.168.100.1
+            nameservers:
+              addresses:
+                - 8.8.8.8
+                - 1.1.1.1
+
+# Package updates and installs
+package_update: true
+package_upgrade: false
+packages:
+  - openssh-server
+  - curl
+  - wget
+  - git
+  - htop
+  - vim
+  - net-tools
+
+# Ensure SSH service is enabled and started
+# Also configure static IP as backup
+runcmd:
+  - systemctl enable ssh
+  - systemctl start ssh
+  - systemctl status ssh
+  - netplan apply
+  - chmod 0600 /etc/netplan/50-cloud-init.yaml || echo "Failed to chmod /etc/netplan/50-cloud-init.yaml"
+  - sleep 2 # Allow netplan apply to settle
+  #- ip addr flush dev ens3
+  #- ip addr add $VM_STATIC_IP/24 dev ens3
+  #- ip route add default via 192.168.100.1
+  - ip addr show ens3
+  - ip route show
+  - ping 192.168.100.2 -c 3
+  - echo "WERE THERE"
+
+
+# Final message
+final_message: "Cloud-init setup complete. VM is ready for SSH access!"
+EOF
+
+test_file_exists "/tmp/user-data" "Cloud-init user-data creation"
+
+# Create meta-data file
+cat > "/tmp/meta-data" << EOF
+instance-id: $VM_NAME
+local-hostname: $VM_NAME
+EOF
+
+test_file_exists "/tmp/meta-data" "Cloud-init meta-data creation"
+
+# Create cloud-init ISO
+log "Creating cloud-init ISO..."
+if command -v genisoimage &> /dev/null; then
+    genisoimage -output "$CLOUD_INIT_PATH" -volid cidata -joliet -rock /tmp/user-data /tmp/meta-data
+elif command -v mkisofs &> /dev/null; then
+    mkisofs -o "$CLOUD_INIT_PATH" -V cidata -J -r /tmp/user-data /tmp/meta-data
+else
+    error "Neither genisoimage nor mkisofs found. Please install genisoimage or cdrtools."
+fi
+
+test_file_exists "$CLOUD_INIT_PATH" "Cloud-init ISO creation"
+
+# Verify the ISO was created properly
+iso_size=$(stat -c%s "$CLOUD_INIT_PATH" 2>/dev/null)
+if [ "$iso_size" -gt 0 ]; then
+    log "✓ Cloud-init ISO created successfully (size: $iso_size bytes)"
+else
+    error "Cloud-init ISO creation failed or resulted in empty file"
+fi
+
+# Cleanup temporary files
+rm -f /tmp/user-data /tmp/meta-data
+
+log "✓ Cloud-init ISO created at $CLOUD_INIT_PATH"
+
+# Resize the VM disk to give it more space (optional, expand to 20GB)
+log "Resizing VM disk to 20GB..."
+qemu-img resize "$VM_IMAGE_PATH" 20G
+
+# Verify disk resize
+new_size=$(qemu-img info "$VM_IMAGE_PATH" | grep "virtual size" | awk '{print $3}')
+if echo "$new_size" | grep -q "20"; then
+    log "✓ VM disk successfully resized to 20GB"
+else
+    warn "VM disk resize verification failed, but continuing..."
+fi
+
+# Create network configuration
+TAP_NAME="tap-$VM_NAME"
+
+log "Setting up network configuration..."
+
+# Check if bridge exists, create if not
+if ! ip link show "$BRIDGE_NAME" &>/dev/null; then
+    log "Creating bridge interface $BRIDGE_NAME..."
+    ip link add name "$BRIDGE_NAME" type bridge
+    ip link set dev "$BRIDGE_NAME" up
+    
+    # Configure bridge with IP address for VM network
+    log "Configuring bridge IP address..."
+    ip addr add "$BRIDGE_IP" dev "$BRIDGE_NAME"
+    
+    test_network_interface "$BRIDGE_NAME" "Bridge interface creation"
+    test_step "Bridge IP configuration" "ip addr show '$BRIDGE_NAME' | grep -q '192.168.100.1'"
+else
+    log "✓ Bridge interface $BRIDGE_NAME already exists"
+    # Ensure bridge has IP configured
+    if ! ip addr show "$BRIDGE_NAME" | grep -q "192.168.100.1"; then
+        log "Adding IP address to existing bridge..."
+        ip addr add "$BRIDGE_IP" dev "$BRIDGE_NAME" 2>/dev/null || true
+    fi
+fi
+
+# Create TAP interface for the VM
+TAP_NAME="tap-vm$VM_NUMBER"
+log "Creating TAP interface $TAP_NAME..."
+
+# Remove existing TAP interface if it exists
+if ip link show "$TAP_NAME" &>/dev/null; then
+    warn "TAP interface $TAP_NAME already exists, removing it..."
+    ip link delete "$TAP_NAME" 2>/dev/null || true
+    sleep 1  # Give time for cleanup
+fi
+
+# Create TAP interface with proper configuration for Cloud Hypervisor
+ip tuntap add dev "$TAP_NAME" mode tap user root
+test_network_interface "$TAP_NAME" "TAP interface creation"
+
+# Set TAP interface up
+ip link set dev "$TAP_NAME" up
+sleep 1  # Give the interface a moment to come up
+test_step "TAP interface up" "ip link show '$TAP_NAME' | grep -q 'UP'"
+
+# Attach to bridge
+ip link set dev "$TAP_NAME" master "$BRIDGE_NAME"
+sleep 1  # Give the bridge attachment a moment to complete
+test_step "TAP interface bridge attachment" "ip link show '$TAP_NAME' | grep -q 'master'"
+
+# Disable offloading features that can cause issues with Cloud Hypervisor
+ethtool -K "$TAP_NAME" tx off rx off tso off gso off gro off lro off 2>/dev/null || warn "Could not disable TAP interface offloading (ethtool not available)"
+
+log "✓ Network interfaces configured successfully"
+
+# Ensure basic networking is set up (simplified version of setup_vm_network.sh)
+log "Ensuring basic VM networking is configured..."
+
+# Enable IP forwarding
+echo 1 > /proc/sys/net/ipv4/ip_forward
+
+# Set up basic NAT rules (remove existing first to avoid duplicates)
+iptables -t nat -D POSTROUTING -s "$NETWORK" -j MASQUERADE 2>/dev/null || true
+iptables -D FORWARD -i "$BRIDGE_NAME" -j ACCEPT 2>/dev/null || true
+iptables -D FORWARD -o "$BRIDGE_NAME" -j ACCEPT 2>/dev/null || true
+
+# Add new rules
+iptables -t nat -A POSTROUTING -s "$NETWORK" -j MASQUERADE
+iptables -A FORWARD -i "$BRIDGE_NAME" -j ACCEPT
+iptables -A FORWARD -o "$BRIDGE_NAME" -j ACCEPT
+
+log "✓ Basic NAT and forwarding rules configured"
+
+# Check if dnsmasq is running for DHCP
+if ! systemctl is-active --quiet dnsmasq 2>/dev/null; then
+    warn "dnsmasq is not running. VMs may not get IP addresses automatically."
+    warn "Consider running: sudo ./setup_vm_network.sh"
+fi
+
+# Start the VM with Cloud Hypervisor
+log "Starting VM $VM_NAME..."
+VM_SOCKET="/tmp/cloud-hypervisor-vm$VM_NUMBER.sock"
+VM_LOG_FILE="/tmp/cloud-hypervisor-vm$VM_NUMBER.log"
+
+# Remove existing socket and log file if they exist
+rm -f "$VM_SOCKET" "$VM_LOG_FILE"
+
+
+# Start Cloud Hypervisor in background with error handling
+log "Launching Cloud Hypervisor..."
+
+# Try to start Cloud Hypervisor and capture any error output
+log "Starting Cloud Hypervisor with kernel boot:"
+log "cloud-hypervisor --memory size=${MEMORY_MB}M --cpus boot=$CPU_CORES --kernel $KERNEL_PATH --initramfs $INITRD_PATH --cmdline 'root=LABEL=cloudimg-rootfs ro console=tty1 console=ttyS0' --disk path=$VM_IMAGE_PATH path=$CLOUD_INIT_PATH,readonly=on --net tap=$TAP_NAME,mac=$VM_MAC --serial file=$VM_LOG_FILE --console off --event-monitor path=${VM_LOG_FILE}.events"
+
+# Use kernel boot instead of firmware boot to properly pass root device
+KERNEL_PATH="$BASE_SUBVOL/vmlinuz"
+INITRD_PATH="$BASE_SUBVOL/initrd.img"
+
+cloud-hypervisor \
+    --api-socket "$VM_SOCKET" \
+    --memory "size=${MEMORY_MB}M" \
+    --cpus "boot=$CPU_CORES" \
+    --kernel "$KERNEL_PATH" \
+    --initramfs "$INITRD_PATH" \
+    --cmdline "root=LABEL=cloudimg-rootfs ro console=tty1 console=ttyS0" \
+    --disk "path=$VM_IMAGE_PATH" "path=$CLOUD_INIT_PATH,readonly=on" \
+    --net "tap=$TAP_NAME,mac=$VM_MAC" \
+    --serial tty \
+    --console off \
+    --event-monitor "path=${VM_LOG_FILE}.events" &
+
+VM_PID=$!
+
+# Check if the process started successfully
+if [ -z "$VM_PID" ]; then
+    error "Failed to get VM process ID"
+fi
+
+# Verify VM process started
+sleep 2
+if ! test_process_running "$VM_PID" "VM process startup"; then
+    error "VM process failed to start or died immediately. Check log: $VM_LOG_FILE"
+fi
+
+log "✓ VM $VM_NAME started successfully with PID $VM_PID"
+log "VM socket: $VM_SOCKET"
+log "VM log file: $VM_LOG_FILE"
+log "TAP interface: $TAP_NAME"
+log "Bridge interface: $BRIDGE_NAME"
+log "VM MAC address: $VM_MAC"
+
+# Wait for VM to initialize and check if it's running properly
+log "Waiting for VM to initialize..."
+init_wait_count=0
+while [ $init_wait_count -lt 10 ]; do
+    sleep 1
+    init_wait_count=$((init_wait_count + 1))
+    
+    # Check if VM process is still running
+    if ! kill -0 "$VM_PID" 2>/dev/null; then
+        error "VM process died during initialization. Check log: $VM_LOG_FILE"
+    fi
+    
+    if [ $((init_wait_count % 3)) -eq 0 ]; then
+        info "VM initializing... ($init_wait_count/10 seconds)"
+    fi
+done
+
+log "✓ VM initialization completed"
+
+# Save VM information for management
+log "Saving VM configuration..."
+VM_INFO_FILE="$VM_SUBVOL_PATH/vm-info.txt"
+cat > "$VM_INFO_FILE" << EOF
+VM_NUMBER=$VM_NUMBER
+VM_NAME=$VM_NAME
+VM_PID=$VM_PID
+VM_SOCKET=$VM_SOCKET
+TAP_NAME=$TAP_NAME
+BRIDGE_NAME=$BRIDGE_NAME
+MEMORY_MB=$MEMORY_MB
+CPU_CORES=$CPU_CORES
+VM_IMAGE_PATH=$VM_IMAGE_PATH
+CLOUD_INIT_PATH=$CLOUD_INIT_PATH
+VM_MAC=$VM_MAC
+VM_LOG_FILE=$VM_LOG_FILE
+VM_STATIC_IP=$VM_STATIC_IP
+STARTED="$(date '+%Y-%m-%d %H:%M:%S')"
+EOF
+
+test_file_exists "$VM_INFO_FILE" "VM info file creation"
+log "✓ VM information saved to $VM_INFO_FILE"
+
+# Function to cleanup on exit (only for interactive mode)
+cleanup_on_exit() {
+    log "Cleaning up VM $VM_NAME..."
+    if [ -n "$VM_PID" ] && kill -0 "$VM_PID" 2>/dev/null; then
+        kill "$VM_PID"
+        wait "$VM_PID" 2>/dev/null
+    fi
+    ip link delete "$TAP_NAME" 2>/dev/null || true
+    rm -f "$VM_SOCKET"
+}
+
+# Test VM boot and connectivity
+log "Testing VM boot and connectivity..."
+
+# Show the static IP that will be used
+log "VM $VM_NAME will use static IP address: $VM_STATIC_IP"
+
+# Wait for VM to boot and verify static IP
+VM_IP=$(wait_for_vm_boot "$VM_NAME" "$VM_STATIC_IP")
+if [ $? -ne 0 ] || [ -z "$VM_IP" ]; then
+    error "VM failed to boot or respond at static IP $VM_STATIC_IP. Check log: $VM_LOG_FILE"
+fi
+
+log "✓ VM booted successfully and is using IP: $VM_IP"
+
+# Test IP connectivity first
+log "Testing IP connectivity before SSH..."
+if test_ip_connectivity "$VM_IP"; then
+    log "✓ IP connectivity test passed for $VM_IP"
+else
+    error "IP connectivity test failed for $VM_IP"
+fi
+
+# Test SSH connectivity
+if test_ssh_connection "$VM_IP"; then
+    log "🎉 SUCCESS: VM $VM_NAME is fully operational!" >&2
+    log "✓ VM is running with PID $VM_PID" >&2
+    log "✓ VM has IP address: $VM_IP" >&2
+    log "✓ SSH is working with key: ssh -i \"$SELECTED_HOST_ROOT_SSH_PRIV_KEY_PATH\" ubuntu@$VM_IP" >&2
+    log "✓ VM info saved to: $VM_INFO_FILE" >&2
+    
+    # Display relevant IP configuration lines from VM log
+    if [ -f "$VM_LOG_FILE" ]; then
+        info "Relevant IP configuration from VM log ($VM_LOG_FILE):" >&2
+        grep -E "inet .*ens3|default via" "$VM_LOG_FILE" | tail -n 5 || true # Output of grep goes to stdout, which is fine here
+    fi
+    
+    echo "" >&2 # Direct echo to stderr
+    info "VM $VM_NAME is ready for use!" >&2
+    info "Connect via SSH: ssh -i \"$SELECTED_HOST_ROOT_SSH_PRIV_KEY_PATH\" ubuntu@$VM_IP" >&2
+    # info "Default password: ubuntu (please change after first login)" # No longer relevant
+    info "To stop the VM: sudo $(dirname "$0")/ubuntu_vm_delete.sh $VM_NUMBER"
+    echo ""
+    
+    # Don't set trap for successful VMs - let them run
+    log "VM $VM_NAME will continue running in the background."
+    log "Use 'sudo $(dirname "$0")/ubuntu_vm_delete.sh $VM_NUMBER' to stop and delete it."
+    
+else
+    error "SSH connectivity test failed. VM will be deleted for retry."
+fi
+
+# If we reach here, the VM is working properly
+log "VM startup and testing completed successfully!"
Author	SHA1	Message	Date
kristof	cda492a4be	...	2025-06-15 21:24:50 +02:00
kristof	0bae1ed774	...	2025-06-15 20:55:06 +02:00
kristof	e701615e53	...	2025-06-15 20:41:33 +02:00
kristof	dc9e75704d	...	2025-06-15 19:57:32 +02:00
kristof	3dbd0d0aea	...	2025-06-15 19:42:01 +02:00
kristof	d336f36929	...	2025-06-15 19:31:47 +02:00
kristof	63f4f77366	...	2025-06-15 19:20:32 +02:00
kristof	7b859d274c	...	2025-06-15 19:16:04 +02:00
kristof	06786aed90	...	2025-06-15 19:03:13 +02:00
kristof	bcef07baa4	...	2025-06-15 18:49:18 +02:00
kristof	69c6a57fa4	...	2025-06-15 18:33:35 +02:00
kristof	13839041ff	...	2025-06-15 18:09:34 +02:00
kristof	d6362a5152	...	2025-06-15 17:59:14 +02:00
despiegk	ec41d8a740	Update README.md	2025-06-15 15:44:36 +00:00
despiegk	2eb6f45631	...	2025-06-15 17:37:19 +02:00
despiegk	41b445fdba	...	2025-06-15 17:35:24 +02:00
despiegk	df2b1253fa	Merge branch 'main' of git.threefold.info:ourworld_web/itenv_tools	2025-06-15 17:21:38 +02:00
despiegk	6767e2b4e4	...	2025-06-15 17:21:36 +02:00