Crate opendal

Expand description

Apache OpenDAL™ is an Open Data Access Layer that enables seamless interaction with diverse storage services.

OpenDAL’s development is guided by its vision of One Layer, All Storage and its core principles: Open Community, Solid Foundation, Fast Access, Object Storage First, and Extensible Architecture. Read the explained vision at OpenDAL Vision.

§Quick Start

OpenDAL’s API entry points are Operator and blocking::Operator. All public APIs are accessible through the operator. To utilize OpenDAL, you need to:

Init a service
Compose layers
Use operator

§Init a service

The first step is to pick a service and init it with a builder. All supported services could be found at services.

Let’s take services::S3 as an example:

use opendal::services;
use opendal::Operator;
use opendal::Result;

fn main() -> Result<()> {
    // Pick a builder and configure it.
    let mut builder = services::S3::default().bucket("test");

    // Init an operator
    let op = Operator::new(builder)?.finish();
    Ok(())
}

§Compose layers

The next setup is to compose layers. Layers are modules that provide extra features for every operation. All builtin layers could be found at layers.

Let’s use layers::LoggingLayer as an example; this layer adds logging to every operation that OpenDAL performs.

use opendal::layers::LoggingLayer;
use opendal::services;
use opendal::Operator;
use opendal::Result;

#[tokio::main]
async fn main() -> Result<()> {
    // Pick a builder and configure it.
    let mut builder = services::S3::default().bucket("test");

    // Init an operator
    let op = Operator::new(builder)?
        // Init with logging layer enabled.
        .layer(LoggingLayer::default())
        .finish();

    Ok(())
}

§Use operator

The final step is to use the operator. OpenDAL supports both async Operator and blocking blocking::Operator. Please pick the one that fits your use case.

Every Operator API follows the same pattern, take read as an example:

read: Execute a read operation.
read_with: Execute a read operation with additional options, like range and if_match.
reader: Create a reader for streaming data, enabling flexible access.
reader_with: Create a reader with advanced options.

use opendal::layers::LoggingLayer;
use opendal::services;
use opendal::Operator;
use opendal::Result;

#[tokio::main]
async fn main() -> Result<()> {
    // Pick a builder and configure it.
    let mut builder = services::S3::default().bucket("test");

    // Init an operator
    let op = Operator::new(builder)?
        // Init with logging layer enabled.
        .layer(LoggingLayer::default())
        .finish();

    // Fetch this file's metadata
    let meta = op.stat("hello.txt").await?;
    let length = meta.content_length();

    // Read data from `hello.txt` with range `0..1024`.
    let bs = op.read_with("hello.txt").range(0..1024).await?;

    Ok(())
}

§Useful Links

Modules§

blockingblocking: blocking module provides blocking APIs for OpenDAL.
docsdocsrs: This module holds documentation for OpenDAL.
executors: executors module provides implementations for the Execute trait for widely used runtimes.
layers: Layer is the mechanism to intercept operations.
operator_futures: Futures provides the futures generated by Operator
options: Options module provides options definitions for operations.
raw: Raw modules provide raw APIs that used by underlying services
services: Services will provide builders to build underlying backends.

Structs§

Buffer: Buffer is a wrapper of contiguous Bytes and non-contiguous [Bytes].
BufferSink: BufferSink is the adapter of [futures::Sink] generated by Writer::into_sink
BufferStream: BufferStream is a stream of buffers, created by Reader::into_stream
Capability: Capability defines the supported operations and their constraints for a storage Operator.
DeleteInput: DeleteInput is the input for delete operations.
Deleter: Deleter is designed to continuously remove content from storage.
Entry: Entry returned by Lister or [BlockingLister] to represent a path and it’s relative metadata.
Error: Error is the error struct returned by all opendal functions.
Executor: Executor that runs futures in background.
FuturesAsyncReader: FuturesAsyncReader is the adapter of [AsyncRead], [AsyncBufRead] and [AsyncSeek] generated by Reader::into_futures_async_read.
FuturesAsyncWriter: FuturesIoAsyncWriter is the adapter of [AsyncWrite] for Writer.
FuturesBytesSink: FuturesBytesSink is the adapter of [futures::Sink] generated by Writer::into_bytes_sink.
FuturesBytesStream: FuturesBytesStream is the adapter of [Stream] generated by Reader::into_bytes_stream.
FuturesDeleteSink: FuturesDeleteSink is a sink that generated by Deleter
Lister: Lister is designed to list entries at given path in an asynchronous manner.
Metadata: Metadata contains all the information related to a specific path.
Operator: The Operator serves as the entry point for all public asynchronous APIs.
OperatorBuilder: OperatorBuilder is a typed builder to build an Operator.
OperatorInfo: Metadata for operator, users can use this metadata to get information of operator.
Reader: Reader is designed to read data from given path in an asynchronous manner.
Writer: Writer is designed to write data into given path in an asynchronous manner.

Enums§

EntryMode: EntryMode represents the mode.
ErrorKind: ErrorKind is all kinds of Error of opendal.
Scheme: Services that OpenDAL supports

Traits§

Builder: Builder is used to set up underlying services.
Configurator: Configurator is used to configure the underlying service.
Execute: Execute trait is used to execute task in background.
IntoDeleteInput: IntoDeleteInput is a helper trait that makes it easier for users to play with Deleter.

Type Aliases§

Result: Result that is a wrapper of Result<T, opendal::Error>

Crate opendalCopy item path