136 lines
4.0 KiB
Markdown
136 lines
4.0 KiB
Markdown
# OurDB
|
|
|
|
OurDB is a lightweight, efficient key-value database implementation that provides data persistence with history tracking capabilities. This Rust implementation offers a robust and performant solution for applications requiring simple but reliable data storage.
|
|
|
|
## Features
|
|
|
|
- Simple key-value storage with history tracking
|
|
- Data integrity verification using CRC32
|
|
- Support for multiple backend files for large datasets
|
|
- Lookup table for fast data retrieval
|
|
- Incremental mode for auto-generated IDs
|
|
- Memory and disk-based lookup tables
|
|
|
|
## Limitations
|
|
|
|
- Maximum data size per entry is 65,535 bytes (~64KB) due to the 2-byte size field in the record header
|
|
|
|
## Usage
|
|
|
|
### Basic Example
|
|
|
|
```rust
|
|
use ourdb::{OurDB, OurDBConfig, OurDBSetArgs};
|
|
use std::path::PathBuf;
|
|
|
|
fn main() -> Result<(), ourdb::Error> {
|
|
// Create a new database
|
|
let config = OurDBConfig {
|
|
path: PathBuf::from("/tmp/ourdb"),
|
|
incremental_mode: true,
|
|
file_size: None, // Use default (500MB)
|
|
keysize: None, // Use default (4 bytes)
|
|
};
|
|
|
|
let mut db = OurDB::new(config)?;
|
|
|
|
// Store data (with auto-generated ID in incremental mode)
|
|
let data = b"Hello, OurDB!";
|
|
let id = db.set(OurDBSetArgs { id: None, data })?;
|
|
println!("Stored data with ID: {}", id);
|
|
|
|
// Retrieve data
|
|
let retrieved = db.get(id)?;
|
|
println!("Retrieved: {}", String::from_utf8_lossy(&retrieved));
|
|
|
|
// Update data
|
|
let updated_data = b"Updated data";
|
|
db.set(OurDBSetArgs { id: Some(id), data: updated_data })?;
|
|
|
|
// Get history (returns most recent first)
|
|
let history = db.get_history(id, 2)?;
|
|
for (i, entry) in history.iter().enumerate() {
|
|
println!("History {}: {}", i, String::from_utf8_lossy(entry));
|
|
}
|
|
|
|
// Delete data
|
|
db.delete(id)?;
|
|
|
|
// Close the database
|
|
db.close()?;
|
|
|
|
Ok(())
|
|
}
|
|
```
|
|
|
|
### Key-Value Mode vs Incremental Mode
|
|
|
|
OurDB supports two operating modes:
|
|
|
|
1. **Key-Value Mode** (`incremental_mode: false`): You must provide IDs explicitly when storing data.
|
|
2. **Incremental Mode** (`incremental_mode: true`): IDs are auto-generated when not provided.
|
|
|
|
### Configuration Options
|
|
|
|
- `path`: Directory for database storage
|
|
- `incremental_mode`: Whether to use auto-increment mode
|
|
- `file_size`: Maximum file size (default: 500MB)
|
|
- `keysize`: Size of lookup table entries (2-6 bytes)
|
|
- 2: For databases with < 65,536 records
|
|
- 3: For databases with < 16,777,216 records
|
|
- 4: For databases with < 4,294,967,296 records (default)
|
|
- 6: For large databases requiring multiple files
|
|
|
|
## Architecture
|
|
|
|
OurDB consists of three main components:
|
|
|
|
1. **Frontend API**: Provides the public interface for database operations
|
|
2. **Lookup Table**: Maps keys to physical locations in the backend storage
|
|
3. **Backend Storage**: Manages the actual data persistence in files
|
|
|
|
### Record Format
|
|
|
|
Each record in the backend storage includes:
|
|
- 2 bytes: Data size
|
|
- 4 bytes: CRC32 checksum
|
|
- 6 bytes: Previous record location (for history)
|
|
- N bytes: Actual data
|
|
|
|
## Documentation
|
|
|
|
Additional documentation is available in the repository:
|
|
|
|
- [API Reference](API.md): Detailed API documentation
|
|
- [Migration Guide](MIGRATION.md): Guide for migrating from the V implementation
|
|
- [Architecture](architecture.md): Design and implementation details
|
|
|
|
## Examples
|
|
|
|
The repository includes several examples to demonstrate OurDB usage:
|
|
|
|
- `basic_usage.rs`: Simple operations with OurDB
|
|
- `advanced_usage.rs`: More complex features including both operation modes
|
|
- `benchmark.rs`: Performance benchmarking tool
|
|
|
|
Run an example with:
|
|
|
|
```bash
|
|
cargo run --example basic_usage
|
|
cargo run --example advanced_usage
|
|
cargo run --example benchmark
|
|
```
|
|
|
|
## Performance
|
|
|
|
OurDB is designed for efficiency and minimal overhead. The benchmark example can be used to evaluate performance on your specific hardware and workload.
|
|
|
|
Typical performance metrics on modern hardware:
|
|
|
|
- **Write**: 10,000+ operations per second
|
|
- **Read**: 50,000+ operations per second
|
|
|
|
## License
|
|
|
|
This project is licensed under the MIT License.
|