Expand description
Apache OpenDAL™ is an Open Data Access Layer that enables seamless interaction with diverse storage services.
OpenDAL’s development is guided by its vision of One Layer, All Storage and its core principles: Open Community, Solid Foundation, Fast Access, Object Storage First, and Extensible Architecture. Read the explained vision at OpenDAL Vision.
§Quick Start
OpenDAL’s API entry points are Operator
and BlockingOperator
. All
public APIs are accessible through the operator. To utilize OpenDAL, you
need to:
§Init a service
The first step is to pick a service and init it with a builder. All supported
services could be found at services
.
Let’s take services::S3
as an example:
use opendal::services;
use opendal::Operator;
use opendal::Result;
fn main() -> Result<()> {
// Pick a builder and configure it.
let mut builder = services::S3::default().bucket("test");
// Init an operator
let op = Operator::new(builder)?.finish();
Ok(())
}
§Compose layers
The next setup is to compose layers. Layers are modules that provide extra
features for every operation. All builtin layers could be found at layers
.
Let’s use layers::LoggingLayer
as an example; this layer adds logging to
every operation that OpenDAL performs.
use opendal::layers::LoggingLayer;
use opendal::services;
use opendal::Operator;
use opendal::Result;
#[tokio::main]
async fn main() -> Result<()> {
// Pick a builder and configure it.
let mut builder = services::S3::default().bucket("test");
// Init an operator
let op = Operator::new(builder)?
// Init with logging layer enabled.
.layer(LoggingLayer::default())
.finish();
Ok(())
}
§Use operator
The final step is to use the operator. OpenDAL supports both async Operator
and blocking BlockingOperator
. Please pick the one that fits your use case.
Every Operator API follows the same pattern, take read
as an example:
read
: Execute a read operation.read_with
: Execute a read operation with additional options, likerange
andif_match
.reader
: Create a reader for streaming data, enabling flexible access.reader_with
: Create a reader with advanced options.
use opendal::layers::LoggingLayer;
use opendal::services;
use opendal::Operator;
use opendal::Result;
#[tokio::main]
async fn main() -> Result<()> {
// Pick a builder and configure it.
let mut builder = services::S3::default().bucket("test");
// Init an operator
let op = Operator::new(builder)?
// Init with logging layer enabled.
.layer(LoggingLayer::default())
.finish();
// Fetch this file's metadata
let meta = op.stat("hello.txt").await?;
let length = meta.content_length();
// Read data from `hello.txt` with range `0..1024`.
let bs = op.read_with("hello.txt").range(0..1024).await?;
Ok(())
}
Modules§
- docs
docsrs
This module holds documentation for OpenDAL. - executors module provides implementations for the
Execute
trait for widely used runtimes. Layer
is the mechanism to intercept operations.- Functions provides the functions generated by
BlockingOperator
- Futures provides the futures generated by
Operator
- Raw modules provide raw APIs that used by underlying services
- Services will provide builders to build underlying backends.
Structs§
- BlockingDeleter is designed to continuously remove content from storage.
- BlockingLister is designed to list entries at given path in a blocking manner.
- BlockingOperator is the entry for all public blocking APIs.
- BlockingReader is designed to read data from given path in an blocking manner.
- BlockingWriter is designed to write data into given path in an blocking manner.
- Buffer is a wrapper of contiguous
Bytes
and non-contiguous[Bytes]
. - BufferIterator is an iterator of buffers.
- BufferSink is the adapter of [
futures::Sink
] generated byWriter
. - Capability defines the supported operations and their constraints for a storage Operator.
- DeleteInput is the input for delete operations.
- Deleter is designed to continuously remove content from storage.
- Entry returned by
Lister
orBlockingLister
to represent a path and it’s relative metadata. - Error is the error struct returned by all opendal functions.
- Executor that runs futures in background.
- FuturesAsyncReader is the adapter of [
AsyncRead
], [AsyncBufRead
] and [AsyncSeek
] generated byReader::into_futures_async_read
. - FuturesIoAsyncWriter is the adapter of [
AsyncWrite
] forWriter
. - FuturesBytesSink is the adapter of [
futures::Sink
] generated byWriter::into_bytes_sink
. - FuturesBytesStream is the adapter of [
Stream
] generated byReader::into_bytes_stream
. - FuturesDeleteSink is a sink that generated by
Deleter
- Lister is designed to list entries at given path in an asynchronous manner.
- Metadata carries all metadata associated with a path.
- Operator is the entry for all public async APIs.
- OperatorBuilder is a typed builder to build an Operator.
- Metadata for operator, users can use this metadata to get information of operator.
- Reader is designed to read data from given path in an asynchronous manner.
- StdIterator is the adapter of
Iterator
forBlockingReader
. - StdWriter is the adapter of
std::io::Write
forBlockingWriter
. - Writer is designed to write data into given path in an asynchronous manner.
Enums§
- EntryMode represents the mode.
- ErrorKind is all kinds of Error of opendal.
- Services that OpenDAL supports
Traits§
- Builder is used to set up underlying services.
- Configurator is used to configure the underlying service.
- Execute trait is used to execute task in background.
- IntoDeleteInput is a helper trait that makes it easier for users to play with
Deleter
.
Type Aliases§
- Result that is a wrapper of
Result<T, opendal::Error>