Crate opendal

Source
Expand description

Apache OpenDAL™ is an Open Data Access Layer that enables seamless interaction with diverse storage services.

OpenDAL’s development is guided by its vision of One Layer, All Storage and its core principles: Open Community, Solid Foundation, Fast Access, Object Storage First, and Extensible Architecture. Read the explained vision at OpenDAL Vision.

§Quick Start

OpenDAL’s API entry points are Operator and BlockingOperator. All public APIs are accessible through the operator. To utilize OpenDAL, you need to:

§Init a service

The first step is to pick a service and init it with a builder. All supported services could be found at services.

Let’s take services::S3 as an example:

use opendal::services;
use opendal::Operator;
use opendal::Result;

fn main() -> Result<()> {
    // Pick a builder and configure it.
    let mut builder = services::S3::default().bucket("test");

    // Init an operator
    let op = Operator::new(builder)?.finish();
    Ok(())
}

§Compose layers

The next setup is to compose layers. Layers are modules that provide extra features for every operation. All builtin layers could be found at layers.

Let’s use layers::LoggingLayer as an example; this layer adds logging to every operation that OpenDAL performs.

use opendal::layers::LoggingLayer;
use opendal::services;
use opendal::Operator;
use opendal::Result;

#[tokio::main]
async fn main() -> Result<()> {
    // Pick a builder and configure it.
    let mut builder = services::S3::default().bucket("test");

    // Init an operator
    let op = Operator::new(builder)?
        // Init with logging layer enabled.
        .layer(LoggingLayer::default())
        .finish();

    Ok(())
}

§Use operator

The final step is to use the operator. OpenDAL supports both async Operator and blocking BlockingOperator. Please pick the one that fits your use case.

Every Operator API follows the same pattern, take read as an example:

  • read: Execute a read operation.
  • read_with: Execute a read operation with additional options, like range and if_match.
  • reader: Create a reader for streaming data, enabling flexible access.
  • reader_with: Create a reader with advanced options.
use opendal::layers::LoggingLayer;
use opendal::services;
use opendal::Operator;
use opendal::Result;

#[tokio::main]
async fn main() -> Result<()> {
    // Pick a builder and configure it.
    let mut builder = services::S3::default().bucket("test");

    // Init an operator
    let op = Operator::new(builder)?
        // Init with logging layer enabled.
        .layer(LoggingLayer::default())
        .finish();

    // Fetch this file's metadata
    let meta = op.stat("hello.txt").await?;
    let length = meta.content_length();

    // Read data from `hello.txt` with range `0..1024`.
    let bs = op.read_with("hello.txt").range(0..1024).await?;

    Ok(())
}

Modules§

docsdocsrs
This module holds documentation for OpenDAL.
executors
executors module provides implementations for the Execute trait for widely used runtimes.
layers
Layer is the mechanism to intercept operations.
operator_functions
Functions provides the functions generated by BlockingOperator
operator_futures
Futures provides the futures generated by Operator
raw
Raw modules provide raw APIs that used by underlying services
services
Services will provide builders to build underlying backends.

Structs§

BlockingDeleter
BlockingDeleter is designed to continuously remove content from storage.
BlockingLister
BlockingLister is designed to list entries at given path in a blocking manner.
BlockingOperator
BlockingOperator is the entry for all public blocking APIs.
BlockingReader
BlockingReader is designed to read data from given path in an blocking manner.
BlockingWriter
BlockingWriter is designed to write data into given path in an blocking manner.
Buffer
Buffer is a wrapper of contiguous Bytes and non-contiguous [Bytes].
BufferIterator
BufferIterator is an iterator of buffers.
BufferSink
BufferSink is the adapter of [futures::Sink] generated by Writer::into_sink
BufferStream
BufferStream is a stream of buffers, created by Reader::into_stream
Capability
Capability defines the supported operations and their constraints for a storage Operator.
DeleteInput
DeleteInput is the input for delete operations.
Deleter
Deleter is designed to continuously remove content from storage.
Entry
Entry returned by Lister or BlockingLister to represent a path and it’s relative metadata.
Error
Error is the error struct returned by all opendal functions.
Executor
Executor that runs futures in background.
FuturesAsyncReader
FuturesAsyncReader is the adapter of [AsyncRead], [AsyncBufRead] and [AsyncSeek] generated by Reader::into_futures_async_read.
FuturesAsyncWriter
FuturesIoAsyncWriter is the adapter of [AsyncWrite] for Writer.
FuturesBytesSink
FuturesBytesSink is the adapter of [futures::Sink] generated by Writer::into_bytes_sink.
FuturesBytesStream
FuturesBytesStream is the adapter of [Stream] generated by Reader::into_bytes_stream.
FuturesDeleteSink
FuturesDeleteSink is a sink that generated by Deleter
Lister
Lister is designed to list entries at given path in an asynchronous manner.
Metadata
Metadata contains all the information related to a specific path.
Operator
The Operator serves as the entry point for all public asynchronous APIs.
OperatorBuilder
OperatorBuilder is a typed builder to build an Operator.
OperatorInfo
Metadata for operator, users can use this metadata to get information of operator.
Reader
Reader is designed to read data from given path in an asynchronous manner.
StdBytesIterator
StdIterator is the adapter of Iterator for BlockingReader.
StdReader
StdReader is the adapter of Read, Seek and BufRead for BlockingReader.
StdWriter
StdWriter is the adapter of std::io::Write for BlockingWriter.
Writer
Writer is designed to write data into given path in an asynchronous manner.

Enums§

EntryMode
EntryMode represents the mode.
ErrorKind
ErrorKind is all kinds of Error of opendal.
Scheme
Services that OpenDAL supports

Traits§

Builder
Builder is used to set up underlying services.
Configurator
Configurator is used to configure the underlying service.
Execute
Execute trait is used to execute task in background.
IntoDeleteInput
IntoDeleteInput is a helper trait that makes it easier for users to play with Deleter.

Type Aliases§

Result
Result that is a wrapper of Result<T, opendal::Error>