...
This commit is contained in:
		
							
								
								
									
										77
									
								
								specs/architecture.md
									
									
									
									
									
										Normal file
									
								
							
							
						
						
									
										77
									
								
								specs/architecture.md
									
									
									
									
									
										Normal file
									
								
							@@ -0,0 +1,77 @@
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
## per user
 | 
			
		||||
 | 
			
		||||
runs in container or VM, one per user
 | 
			
		||||
 | 
			
		||||
- zinit
 | 
			
		||||
- herocoordinator
 | 
			
		||||
  - think about like DAG worklflow manager
 | 
			
		||||
  - manage jobs who are send around to different nodes
 | 
			
		||||
- mycelium address range (part of mycelium on host)
 | 
			
		||||
- herodb
 | 
			
		||||
    - state manager
 | 
			
		||||
    - redis protocol / primitives
 | 
			
		||||
    - fs backend (mem and allways append in future)
 | 
			
		||||
    - encryption & decryption primitives
 | 
			
		||||
    - key mgmt for encryption (creation, deletion)
 | 
			
		||||
    - openrpc admin features: user management, role-based access control
 | 
			
		||||
- postgresql + postgrest
 | 
			
		||||
- AI Agent TBD
 | 
			
		||||
 | 
			
		||||
```mermaid
 | 
			
		||||
%%{init: {"theme":"dark"}}%%
 | 
			
		||||
graph TD
 | 
			
		||||
    subgraph Per Node System
 | 
			
		||||
        N[Node] --> OS(Run on top of ZOS4 or Ubuntu or in a VM)
 | 
			
		||||
 | 
			
		||||
        subgraph On Node
 | 
			
		||||
            OS --> SV(Supervisors)
 | 
			
		||||
            OS --> ZN(Zinit)
 | 
			
		||||
            OS --> R(Runners)
 | 
			
		||||
            OS --> PGN(Some Nodes: PostgreSQL + Postgrest)
 | 
			
		||||
            OS --> HDN(Each Node: Herodb)
 | 
			
		||||
 | 
			
		||||
            subgraph Supervisors Responsibilities
 | 
			
		||||
                SV --> SV_MR(Manage runners & scheduling for the node)
 | 
			
		||||
                SV --> SV_MJ(Monitor & schedule jobs)
 | 
			
		||||
                SV --> SV_RU(Check resource usage)
 | 
			
		||||
                SV --> SV_TO(Checks on timeout)
 | 
			
		||||
            end
 | 
			
		||||
 | 
			
		||||
            subgraph Runners Characteristics
 | 
			
		||||
                R --> R_LV(V/Python & Rust)
 | 
			
		||||
                R --> R_FORK(Uses fork per runner for scalability)
 | 
			
		||||
                R --> R_COUNT(Some runners can only run 1, others more)
 | 
			
		||||
                R --> R_CONTEXT(Some runners are per context)
 | 
			
		||||
            end
 | 
			
		||||
        end
 | 
			
		||||
 | 
			
		||||
        SV -- "Manage" --> R
 | 
			
		||||
        SV -- "Schedule jobs via" --> ZN
 | 
			
		||||
        ZN -- "Starts" --> R
 | 
			
		||||
        R -- "Interacts with" --> PGN
 | 
			
		||||
        R -- "Interacts with" --> HDN
 | 
			
		||||
    end
 | 
			
		||||
```
 | 
			
		||||
 | 
			
		||||
## per node
 | 
			
		||||
 | 
			
		||||
- run on top of ZOS4 or Ubuntu or in a VM
 | 
			
		||||
- supervisors
 | 
			
		||||
  - manage runners and scheduling for the node of these runners
 | 
			
		||||
  - monitor & schedule jobs, check resource usage, checks on timout
 | 
			
		||||
- zinit
 | 
			
		||||
- runners (are scheduled in zinit by supervisor)
 | 
			
		||||
    - V/Python & Rust
 | 
			
		||||
    - uses fork per runner (process) for scalability
 | 
			
		||||
    - some runners can only run 1, others more  
 | 
			
		||||
    - some runners are per context
 | 
			
		||||
- some nodes will have postgresql + postgrest
 | 
			
		||||
- each node has herodb
 | 
			
		||||
 | 
			
		||||
REMARK
 | 
			
		||||
 | 
			
		||||
- each rhaj or heroscript running on a node can use herodb if needed (careful, because can and will be lost), but cannot communicate with anyone else outside of the node
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
		Reference in New Issue
	
	Block a user