add data packages and remove empty submodule
This commit is contained in:
Timur Gordon
277
packages/data/ourdb/API.md
Normal file
277
packages/data/ourdb/API.md
Normal file
@@ -0,0 +1,277 @@
|
||||
# OurDB API Reference
|
||||
|
||||
This document provides a comprehensive reference for the OurDB Rust API.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Configuration](#configuration)
|
||||
2. [Database Operations](#database-operations)
|
||||
- [Creating and Opening](#creating-and-opening)
|
||||
- [Setting Data](#setting-data)
|
||||
- [Getting Data](#getting-data)
|
||||
- [Deleting Data](#deleting-data)
|
||||
- [History Tracking](#history-tracking)
|
||||
3. [Error Handling](#error-handling)
|
||||
4. [Advanced Usage](#advanced-usage)
|
||||
- [Custom File Size](#custom-file-size)
|
||||
- [Custom Key Size](#custom-key-size)
|
||||
5. [Performance Considerations](#performance-considerations)
|
||||
|
||||
## Configuration
|
||||
|
||||
### OurDBConfig
|
||||
|
||||
The `OurDBConfig` struct is used to configure a new OurDB instance.
|
||||
|
||||
```rust
|
||||
pub struct OurDBConfig {
|
||||
pub path: PathBuf,
|
||||
pub incremental_mode: bool,
|
||||
pub file_size: Option<usize>,
|
||||
pub keysize: Option<u8>,
|
||||
}
|
||||
```
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `path` | `PathBuf` | Path to the database directory |
|
||||
| `incremental_mode` | `bool` | Whether to use auto-incremented IDs (true) or user-provided IDs (false) |
|
||||
| `file_size` | `Option<usize>` | Maximum size of each database file in bytes (default: 500MB) |
|
||||
| `keysize` | `Option<u8>` | Size of keys in bytes (default: 4, valid values: 2, 3, 4, 6) |
|
||||
|
||||
Example:
|
||||
```rust
|
||||
let config = OurDBConfig {
|
||||
path: PathBuf::from("/path/to/db"),
|
||||
incremental_mode: true,
|
||||
file_size: Some(1024 * 1024 * 100), // 100MB
|
||||
keysize: Some(4), // 4-byte keys
|
||||
};
|
||||
```
|
||||
|
||||
## Database Operations
|
||||
|
||||
### Creating and Opening
|
||||
|
||||
#### `OurDB::new`
|
||||
|
||||
Creates a new OurDB instance or opens an existing one.
|
||||
|
||||
```rust
|
||||
pub fn new(config: OurDBConfig) -> Result<OurDB, Error>
|
||||
```
|
||||
|
||||
Example:
|
||||
```rust
|
||||
let mut db = OurDB::new(config)?;
|
||||
```
|
||||
|
||||
### Setting Data
|
||||
|
||||
#### `OurDB::set`
|
||||
|
||||
Sets a value in the database. In incremental mode, if no ID is provided, a new ID is generated.
|
||||
|
||||
```rust
|
||||
pub fn set(&mut self, args: OurDBSetArgs) -> Result<u32, Error>
|
||||
```
|
||||
|
||||
The `OurDBSetArgs` struct has the following fields:
|
||||
|
||||
```rust
|
||||
pub struct OurDBSetArgs<'a> {
|
||||
pub id: Option<u32>,
|
||||
pub data: &'a [u8],
|
||||
}
|
||||
```
|
||||
|
||||
Example with auto-generated ID:
|
||||
```rust
|
||||
let id = db.set(OurDBSetArgs {
|
||||
id: None,
|
||||
data: b"Hello, World!",
|
||||
})?;
|
||||
```
|
||||
|
||||
Example with explicit ID:
|
||||
```rust
|
||||
db.set(OurDBSetArgs {
|
||||
id: Some(42),
|
||||
data: b"Hello, World!",
|
||||
})?;
|
||||
```
|
||||
|
||||
### Getting Data
|
||||
|
||||
#### `OurDB::get`
|
||||
|
||||
Retrieves a value from the database by ID.
|
||||
|
||||
```rust
|
||||
pub fn get(&mut self, id: u32) -> Result<Vec<u8>, Error>
|
||||
```
|
||||
|
||||
Example:
|
||||
```rust
|
||||
let data = db.get(42)?;
|
||||
```
|
||||
|
||||
### Deleting Data
|
||||
|
||||
#### `OurDB::delete`
|
||||
|
||||
Deletes a value from the database by ID.
|
||||
|
||||
```rust
|
||||
pub fn delete(&mut self, id: u32) -> Result<(), Error>
|
||||
```
|
||||
|
||||
Example:
|
||||
```rust
|
||||
db.delete(42)?;
|
||||
```
|
||||
|
||||
### History Tracking
|
||||
|
||||
#### `OurDB::get_history`
|
||||
|
||||
Retrieves the history of values for a given ID, up to the specified depth.
|
||||
|
||||
```rust
|
||||
pub fn get_history(&mut self, id: u32, depth: u8) -> Result<Vec<Vec<u8>>, Error>
|
||||
```
|
||||
|
||||
Example:
|
||||
```rust
|
||||
// Get the last 5 versions of the record
|
||||
let history = db.get_history(42, 5)?;
|
||||
|
||||
// Process each version (most recent first)
|
||||
for (i, version) in history.iter().enumerate() {
|
||||
println!("Version {}: {:?}", i, version);
|
||||
}
|
||||
```
|
||||
|
||||
### Other Operations
|
||||
|
||||
#### `OurDB::get_next_id`
|
||||
|
||||
Returns the next ID that will be assigned in incremental mode.
|
||||
|
||||
```rust
|
||||
pub fn get_next_id(&self) -> Result<u32, Error>
|
||||
```
|
||||
|
||||
Example:
|
||||
```rust
|
||||
let next_id = db.get_next_id()?;
|
||||
```
|
||||
|
||||
#### `OurDB::close`
|
||||
|
||||
Closes the database, ensuring all data is flushed to disk.
|
||||
|
||||
```rust
|
||||
pub fn close(&mut self) -> Result<(), Error>
|
||||
```
|
||||
|
||||
Example:
|
||||
```rust
|
||||
db.close()?;
|
||||
```
|
||||
|
||||
#### `OurDB::destroy`
|
||||
|
||||
Closes the database and deletes all database files.
|
||||
|
||||
```rust
|
||||
pub fn destroy(&mut self) -> Result<(), Error>
|
||||
```
|
||||
|
||||
Example:
|
||||
```rust
|
||||
db.destroy()?;
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
OurDB uses the `thiserror` crate to define error types. The main error type is `ourdb::Error`.
|
||||
|
||||
```rust
|
||||
pub enum Error {
|
||||
IoError(std::io::Error),
|
||||
InvalidKeySize,
|
||||
InvalidId,
|
||||
RecordNotFound,
|
||||
InvalidCrc,
|
||||
NotIncrementalMode,
|
||||
DatabaseClosed,
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
All OurDB operations that can fail return a `Result<T, Error>` which can be handled using Rust's standard error handling mechanisms.
|
||||
|
||||
Example:
|
||||
```rust
|
||||
match db.get(42) {
|
||||
Ok(data) => println!("Found data: {:?}", data),
|
||||
Err(ourdb::Error::RecordNotFound) => println!("Record not found"),
|
||||
Err(e) => eprintln!("Error: {}", e),
|
||||
}
|
||||
```
|
||||
|
||||
## Advanced Usage
|
||||
|
||||
### Custom File Size
|
||||
|
||||
You can configure the maximum size of each database file:
|
||||
|
||||
```rust
|
||||
let config = OurDBConfig {
|
||||
path: PathBuf::from("/path/to/db"),
|
||||
incremental_mode: true,
|
||||
file_size: Some(1024 * 1024 * 10), // 10MB per file
|
||||
keysize: None,
|
||||
};
|
||||
```
|
||||
|
||||
Smaller file sizes can be useful for:
|
||||
- Limiting memory usage when reading files
|
||||
- Improving performance on systems with limited memory
|
||||
- Easier backup and file management
|
||||
|
||||
### Custom Key Size
|
||||
|
||||
OurDB supports different key sizes (2, 3, 4, or 6 bytes):
|
||||
|
||||
```rust
|
||||
let config = OurDBConfig {
|
||||
path: PathBuf::from("/path/to/db"),
|
||||
incremental_mode: true,
|
||||
file_size: None,
|
||||
keysize: Some(6), // 6-byte keys
|
||||
};
|
||||
```
|
||||
|
||||
Key size considerations:
|
||||
- 2 bytes: Up to 65,536 records
|
||||
- 3 bytes: Up to 16,777,216 records
|
||||
- 4 bytes: Up to 4,294,967,296 records (default)
|
||||
- 6 bytes: Up to 281,474,976,710,656 records
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
For optimal performance:
|
||||
|
||||
1. **Choose appropriate key size**: Use the smallest key size that can accommodate your expected number of records.
|
||||
|
||||
2. **Configure file size**: For large databases, consider using smaller file sizes to improve memory usage.
|
||||
|
||||
3. **Batch operations**: When inserting or updating many records, consider batching operations to minimize disk I/O.
|
||||
|
||||
4. **Close properly**: Always call `close()` when you're done with the database to ensure data is properly flushed to disk.
|
||||
|
||||
5. **Reuse OurDB instance**: Creating a new OurDB instance has overhead, so reuse the same instance for multiple operations when possible.
|
||||
|
||||
6. **Consider memory usage**: The lookup table is loaded into memory, so very large databases may require significant RAM.
|
||||
Reference in New Issue
Block a user