266 lines
6.8 KiB
Markdown
266 lines
6.8 KiB
Markdown
# Migration Plan: Restructuring herodb to Use ourdb as Backend
|
|
|
|
This document outlines the plan to restructure herodb to use ourdb as the backend, completely removing all sled references and better aligning with ourdb's design patterns.
|
|
|
|
## Overview
|
|
|
|
```mermaid
|
|
graph TD
|
|
A[Current herodb with sled] --> B[Define new core traits]
|
|
B --> C[Implement ourdb backend]
|
|
C --> D[Create new DB manager]
|
|
D --> E[Implement transaction system]
|
|
E --> F[Update model implementations]
|
|
F --> G[Final restructured herodb with ourdb]
|
|
```
|
|
|
|
## New Architecture
|
|
|
|
```mermaid
|
|
classDiagram
|
|
class Model {
|
|
+get_id() u32
|
|
+db_prefix() &'static str
|
|
}
|
|
class Storable {
|
|
+serialize() Result<Vec<u8>>
|
|
+deserialize() Result<Self>
|
|
}
|
|
class DB {
|
|
-path: PathBuf
|
|
-type_map: HashMap<TypeId, Arc<dyn DbOperations>>
|
|
-transaction: Arc<RwLock<Option<TransactionState>>>
|
|
+new(config: DbConfig) Result<Self>
|
|
+begin_transaction() Result<()>
|
|
+commit_transaction() Result<()>
|
|
+rollback_transaction() Result<()>
|
|
+set<T: Model>(model: &T) Result<()>
|
|
+get<T: Model>(id: u32) Result<T>
|
|
+delete<T: Model>(id: u32) Result<()>
|
|
+list<T: Model>() Result<Vec<T>>
|
|
+register<T: Model>() Result<()>
|
|
+get_history<T: Model>(id: u32, depth: u8) Result<Vec<T>>
|
|
}
|
|
class DbOperations {
|
|
<<interface>>
|
|
+delete(id: u32) Result<()>
|
|
+get(id: u32) Result<Box<dyn Any>>
|
|
+list() Result<Box<dyn Any>>
|
|
+insert(model: &dyn Any) Result<()>
|
|
+get_history(id: u32, depth: u8) Result<Vec<Box<dyn Any>>>
|
|
}
|
|
class OurDbStore~T~ {
|
|
-db: OurDB
|
|
-model_type: PhantomData<T>
|
|
+new(config: OurDBConfig) Result<Self>
|
|
+insert(model: &T) Result<()>
|
|
+get(id: u32) Result<T>
|
|
+delete(id: u32) Result<()>
|
|
+list() Result<Vec<T>>
|
|
+get_history(id: u32, depth: u8) Result<Vec<T>>
|
|
}
|
|
|
|
Model --|> Storable
|
|
OurDbStore ..|> DbOperations
|
|
DB o-- DbOperations
|
|
```
|
|
|
|
## Detailed Restructuring Steps
|
|
|
|
### 1. Define New Core Traits and Types
|
|
|
|
1. Create a new `Model` trait to replace `SledModel`
|
|
2. Create a new `Storable` trait for serialization/deserialization
|
|
3. Define a new error type hierarchy based on ourdb's error types
|
|
4. Create a `DbOperations` trait for database operations
|
|
|
|
### 2. Implement ourdb Backend
|
|
|
|
1. Create an `OurDbStore<T>` type that wraps ourdb
|
|
2. Implement the `DbOperations` trait for `OurDbStore<T>`
|
|
3. Add support for history tracking
|
|
|
|
### 3. Create New DB Manager
|
|
|
|
1. Create a new `DB` struct that manages multiple model types
|
|
2. Implement a builder pattern for configuration
|
|
3. Add methods for CRUD operations
|
|
|
|
### 4. Implement Transaction System
|
|
|
|
1. Create a transaction system that works with ourdb
|
|
2. Implement transaction operations (begin, commit, rollback)
|
|
3. Handle transaction state tracking
|
|
|
|
### 5. Update Model Implementations
|
|
|
|
1. Update all models to use `u32` IDs
|
|
2. Implement the new `Model` trait for all models
|
|
3. Update model constructors and builders
|
|
|
|
## Implementation Details
|
|
|
|
### 1. Core Traits and Types
|
|
|
|
```rust
|
|
// Error types
|
|
pub enum DbError {
|
|
IoError(std::io::Error),
|
|
SerializationError(bincode::Error),
|
|
NotFound(u32),
|
|
TransactionError(String),
|
|
// Map to ourdb error types
|
|
OurDbError(ourdb::Error),
|
|
// Other error types as needed
|
|
}
|
|
|
|
// Result type alias
|
|
pub type DbResult<T> = Result<T, DbError>;
|
|
|
|
// Storable trait
|
|
pub trait Storable: Serialize + for<'de> Deserialize<'de> + Sized {
|
|
fn serialize(&self) -> DbResult<Vec<u8>> {
|
|
// Default implementation using bincode
|
|
Ok(bincode::serialize(self)?)
|
|
}
|
|
|
|
fn deserialize(data: &[u8]) -> DbResult<Self> {
|
|
// Default implementation using bincode
|
|
Ok(bincode::deserialize(data)?)
|
|
}
|
|
}
|
|
|
|
// Model trait
|
|
pub trait Model: Storable + Debug + Clone + Send + Sync + 'static {
|
|
fn get_id(&self) -> u32;
|
|
fn db_prefix() -> &'static str;
|
|
}
|
|
```
|
|
|
|
### 2. ourdb Backend Implementation
|
|
|
|
```rust
|
|
pub struct OurDbStore<T: Model> {
|
|
db: OurDB,
|
|
_phantom: PhantomData<T>,
|
|
}
|
|
|
|
impl<T: Model> OurDbStore<T> {
|
|
pub fn new(config: OurDBConfig) -> DbResult<Self> {
|
|
let db = OurDB::new(config)?;
|
|
Ok(Self {
|
|
db,
|
|
_phantom: PhantomData,
|
|
})
|
|
}
|
|
|
|
// Implementation of CRUD operations
|
|
}
|
|
|
|
impl<T: Model> DbOperations for OurDbStore<T> {
|
|
// Implementation of DbOperations trait
|
|
}
|
|
```
|
|
|
|
### 3. DB Manager Implementation
|
|
|
|
```rust
|
|
pub struct DB {
|
|
path: PathBuf,
|
|
type_map: HashMap<TypeId, Arc<dyn DbOperations>>,
|
|
transaction: Arc<RwLock<Option<TransactionState>>>,
|
|
}
|
|
|
|
impl DB {
|
|
pub fn new(config: DbConfig) -> DbResult<Self> {
|
|
// Implementation
|
|
}
|
|
|
|
// CRUD operations and other methods
|
|
}
|
|
```
|
|
|
|
### 4. Transaction System
|
|
|
|
```rust
|
|
pub struct TransactionState {
|
|
operations: Vec<DbOperation>,
|
|
active: bool,
|
|
}
|
|
|
|
enum DbOperation {
|
|
Set {
|
|
model_type: TypeId,
|
|
serialized: Vec<u8>,
|
|
},
|
|
Delete {
|
|
model_type: TypeId,
|
|
id: u32,
|
|
},
|
|
}
|
|
|
|
impl DB {
|
|
pub fn begin_transaction(&self) -> DbResult<()> {
|
|
// Implementation
|
|
}
|
|
|
|
pub fn commit_transaction(&self) -> DbResult<()> {
|
|
// Implementation
|
|
}
|
|
|
|
pub fn rollback_transaction(&self) -> DbResult<()> {
|
|
// Implementation
|
|
}
|
|
}
|
|
```
|
|
|
|
### 5. Model Implementation Updates
|
|
|
|
```rust
|
|
// Example for Product model
|
|
impl Model for Product {
|
|
fn get_id(&self) -> u32 {
|
|
self.id
|
|
}
|
|
|
|
fn db_prefix() -> &'static str {
|
|
"product"
|
|
}
|
|
}
|
|
|
|
impl Storable for Product {}
|
|
```
|
|
|
|
## Key Technical Considerations
|
|
|
|
1. **Clean Architecture**: The new design provides a cleaner separation of concerns.
|
|
|
|
2. **Incremental IDs**: All models will use `u32` IDs, and ourdb will be configured in incremental mode.
|
|
|
|
3. **History Tracking**: The new API will expose ourdb's history tracking capabilities.
|
|
|
|
4. **Transaction Support**: We'll implement a custom transaction system on top of ourdb.
|
|
|
|
5. **Error Handling**: New error types will map directly to ourdb's error types.
|
|
|
|
6. **Serialization**: We'll use bincode for serialization/deserialization by default.
|
|
|
|
## Migration Risks and Mitigations
|
|
|
|
| Risk | Mitigation |
|
|
|------|------------|
|
|
| Breaking API changes | Create a compatibility layer if needed |
|
|
| Data migration complexity | Develop a data migration utility |
|
|
| Performance impact | Benchmark before and after |
|
|
| Implementation complexity | Implement in phases with thorough testing |
|
|
| Integration issues | Create comprehensive integration tests |
|
|
|
|
## Implementation Phases
|
|
|
|
1. **Phase 1**: Define core traits and types
|
|
2. **Phase 2**: Implement ourdb backend
|
|
3. **Phase 3**: Create DB manager
|
|
4. **Phase 4**: Implement transaction system
|
|
5. **Phase 5**: Update model implementations
|
|
6. **Phase 6**: Create tests and benchmarks
|
|
7. **Phase 7**: Develop data migration utility |