Struct Operator

pub struct Operator { /* private fields */ }

The Operator serves as the entry point for all public asynchronous APIs.

For more details about the Operator, refer to the concepts section.

All cloned Operator instances share the same internal state, such as HttpClient and Runtime. Some layers may modify the internal state of the Operator too like inject logging and metrics for HttpClient.

Build

Users can initialize an Operator through the following methods:

• Operator::new: Creates an operator using a services builder, such as services::S3.
• Operator::from_config: Creates an operator using a services configuration, such as services::S3Config.
• Operator::from_iter: Creates an operator from an iterator of configuration key-value pairs.

use opendal::services::Memory;
use opendal::Operator;
async fn test() -> Result<()> {
    // Build an `Operator` to start operating the storage.
    let _: Operator = Operator::new(Memory::default())?.finish();

    Ok(())
}

Layer

After the operator is built, users can add the layers they need on top of it.

OpenDAL offers various layers for users to choose from, such as RetryLayer, LoggingLayer, and more. Visit layers for further details.

Please note that Layer can modify internal contexts such as HttpClient and Runtime for all clones of given operator. Therefore, it is recommended to add layers before interacting with the storage. Adding or duplicating layers after accessing the storage may result in unexpected behavior.

use opendal::layers::RetryLayer;
use opendal::services::Memory;
use opendal::Operator;
async fn test() -> Result<()> {
    let op: Operator = Operator::new(Memory::default())?.finish();

    // OpenDAL will retry failed operations now.
    let op = op.layer(RetryLayer::default());

    Ok(())
}

Operate

After the operator is built and the layers are added, users can start operating the storage.

The operator is Send, Sync, and Clone. It has no internal state, and all APIs only take a &self reference, making it safe to share the operator across threads.

Operator provides a consistent API pattern for data operations. For reading operations, it exposes:

• Operator::read: Executes a read operation.
• Operator::read_with: Executes a read operation with additional options using the builder pattern.
• Operator::read_options: Executes a read operation with extra options provided via a options::ReadOptions struct.
• Operator::reader: Creates a reader for streaming data, allowing for flexible access.
• Operator::reader_with: Creates a reader with advanced options using the builder pattern.
• Operator::reader_options: Creates a reader with extra options provided via a options::ReadOptions struct.

The Reader created by Operator supports custom read control methods and can be converted into [futures::AsyncRead] or [futures::Stream] for broader ecosystem compatibility.

use opendal::layers::LoggingLayer;
use opendal::options;
use opendal::services;
use opendal::Operator;
use opendal::Result;

#[tokio::main]
async fn main() -> Result<()> {
    // Pick a builder and configure it.
    let mut builder = services::S3::default().bucket("test");

    // Init an operator
    let op = Operator::new(builder)?
        // Init with logging layer enabled.
        .layer(LoggingLayer::default())
        .finish();

    // Fetch this file's metadata
    let meta = op.stat("hello.txt").await?;
    let length = meta.content_length();

    // Read data from `hello.txt` with options.
    let bs = op
        .read_with("hello.txt")
        .range(0..8 * 1024 * 1024)
        .chunk(1024 * 1024)
        .concurrent(4)
        .await?;

    // The same to:
    let bs = op
        .read_options("hello.txt", options::ReadOptions {
            range: (0..8 * 1024 * 1024).into(),
            chunk: Some(1024 * 1024),
            concurrent: 4,
            ..Default::default()
        })
        .await?;

    Ok(())
}

Implementations

impl Operator

Operator basic API.

pub fn inner(&self) -> &Accessor

Fetch the internal accessor.

pub fn from_inner(accessor: Accessor) -> Self

Convert inner accessor into operator.

pub fn into_inner(self) -> Accessor

Convert operator into inner accessor.

pub fn info(&self) -> OperatorInfo

Get information of underlying accessor.

Examples

use opendal::Operator;

let info = op.info();

pub fn executor(&self) -> Executor

Get the executor used by current operator.

pub fn update_executor(&self, f: impl FnOnce(Executor) -> Executor)

Update executor for the context.

All cloned Operator instances share the same internal state, such as HttpClient and Runtime. Some layers may modify the internal state of the Operator too like inject logging and metrics for HttpClient.

Note

Tasks must be forwarded to the old executor after the update. Otherwise, features such as retry, timeout, and metrics may not function properly.

pub fn http_client(&self) -> HttpClient

👎Deprecated since 0.54.0: Use HttpClientLayer instead. This method will be removed in next version.

Get the http client used by current operator.

pub fn update_http_client(&self, f: impl FnOnce(HttpClient) -> HttpClient)

👎Deprecated since 0.54.0: Use HttpClientLayer instead. This method will be removed in next version

Update http client for the context.

All cloned Operator instances share the same internal state, such as HttpClient and Runtime. Some layers may modify the internal state of the Operator too like inject logging and metrics for HttpClient.

Note

Tasks must be forwarded to the old executor after the update. Otherwise, features such as retry, timeout, and metrics may not function properly.

Deprecated

This method is deprecated since v0.54.0. Use HttpClientLayer instead.

Migration Example

Instead of:

let operator = Operator::new(service)?;
operator.update_http_client(|_| custom_client);

Use:

use opendal::layers::HttpClientLayer;

let operator = Operator::new(service)?
    .layer(HttpClientLayer::new(custom_client))
    .finish();

impl Operator

Operator async API.

pub async fn check(&self) -> Result<()>

Check if this operator can work correctly.

We will send a list request to path and return any errors we met.

use opendal::Operator;

op.check().await?;

pub async fn stat(&self, path: &str) -> Result<Metadata>

Retrieve the metadata for the specified path.

Notes

Extra Options

Operator::stat is a wrapper around Operator::stat_with that uses no additional options. To specify extra options such as if_match and if_none_match, please use Operator::stat_with instead.

Examples

Check if file exists

use opendal::ErrorKind;
if let Err(e) = op.stat("test").await {
    if e.kind() == ErrorKind::NotFound {
        println!("file not exist")
    }
}

pub fn stat_with( &self, path: &str, ) -> FutureStat<impl Future<Output = Result<Metadata>>>

Retrieve the metadata of the specified path with additional options.

Options

Check options::StatOptions for all available options.

Examples

Get metadata while ETag matches

stat_with will

• return Ok(metadata) if ETag matches
• return Err(error) and error.kind() == ErrorKind::ConditionNotMatch if file exists but ETag mismatch
• return Err(err) if other errors occur, for example, NotFound.

use opendal::ErrorKind;
if let Err(e) = op.stat_with("test").if_match("<etag>").await {
    if e.kind() == ErrorKind::ConditionNotMatch {
        println!("file exists, but etag mismatch")
    }
    if e.kind() == ErrorKind::NotFound {
        println!("file not exist")
    }
}

pub async fn stat_options( &self, path: &str, opts: StatOptions, ) -> Result<Metadata>

Retrieve the metadata of the specified path with additional options.

Examples

Get metadata while ETag matches

stat_with will

• return Ok(metadata) if ETag matches
• return Err(error) and error.kind() == ErrorKind::ConditionNotMatch if file exists but ETag mismatch
• return Err(err) if other errors occur, for example, NotFound.

use opendal::options;
use opendal::ErrorKind;
let res = op
    .stat_options("test", options::StatOptions {
        if_match: Some("<etag>".to_string()),
        ..Default::default()
    })
    .await;
if let Err(e) = res {
    if e.kind() == ErrorKind::ConditionNotMatch {
        println!("file exists, but etag mismatch")
    }
    if e.kind() == ErrorKind::NotFound {
        println!("file not exist")
    }
}

pub async fn exists(&self, path: &str) -> Result<bool>

Check whether this path exists.

Example

use anyhow::Result;
use futures::io;
use opendal::Operator;

async fn test(op: Operator) -> Result<()> {
    let _ = op.exists("test").await?;

    Ok(())
}

pub async fn create_dir(&self, path: &str) -> Result<()>

Create a directory at the specified path.

Notes

To specify that a path is a directory, you must include a trailing slash (/). Omitting the trailing slash may cause OpenDAL to return a NotADirectory error.

Behavior

• Creating a directory that already exists will succeed.
• Directory creation is always recursive, functioning like mkdir -p.

Examples

op.create_dir("path/to/dir/").await?;

pub async fn read(&self, path: &str) -> Result<Buffer>

Read the entire file into bytes from given path.

Notes

Additional Options

Operator::read is a simplified method that does not support additional options. To access features like range and if_match, please use Operator::read_with or Operator::read_options instead.

Streaming Read

This function reads all content into memory at once. For more precise memory management or to read big file lazily, please use Operator::reader.

Examples

let bs = op.read("path/to/file").await?;

pub fn read_with( &self, path: &str, ) -> FutureRead<impl Future<Output = Result<Buffer>>>

Read the entire file into bytes from given path with additional options.

Notes

Streaming Read

This function reads all content into memory at once. For more precise memory management or to read big file lazily, please use Operator::reader.

Options

Visit options::ReadOptions for all available options.

Examples

Read the first 10 bytes of a file:

let bs = op.read_with("path/to/file").range(0..10).await?;

pub async fn read_options( &self, path: &str, opts: ReadOptions, ) -> Result<Buffer>

Read the entire file into bytes from given path with additional options.

Notes

Streaming Read

This function reads all content into memory at once. For more precise memory management or to read big file lazily, please use Operator::reader.

Examples

Read the first 10 bytes of a file:

use opendal::options;
let bs = op
    .read_options("path/to/file", options::ReadOptions {
        range: (0..10).into(),
        ..Default::default()
    })
    .await?;

pub async fn reader(&self, path: &str) -> Result<Reader>

Create a new reader of given path.

Notes

Extra Options

Operator::reader is a simplified method without any options. To use additional options such as concurrent or if_match, please use Operator::reader_with or Operator::reader_options instead.

Examples

let r = op.reader("path/to/file").await?;
// Read the first 10 bytes of the file
let data = r.read(0..10).await?;

pub fn reader_with( &self, path: &str, ) -> FutureReader<impl Future<Output = Result<Reader>>>

Create a new reader of given path with additional options.

Options

Visit options::ReaderOptions for all available options.

Examples

Create a reader with a specific version ID:

let r = op.reader_with("path/to/file").version("version_id").await?;
// Read the first 10 bytes of the file
let data = r.read(0..10).await?;

pub async fn reader_options( &self, path: &str, opts: ReaderOptions, ) -> Result<Reader>

Create a new reader of given path with additional options.

Examples

Create a reader with a specific version ID:

use opendal::options;
let r = op
    .reader_options("path/to/file", options::ReaderOptions {
        version: Some("version_id".to_string()),
        ..Default::default()
    })
    .await?;
// Read the first 10 bytes of the file
let data = r.read(0..10).await?;

pub async fn write(&self, path: &str, bs: impl Into<Buffer>) -> Result<Metadata>

Write all data to the specified path at once.

Notes

Visit performance::concurrent_write for more details on concurrent writes.

Extra Options

Operator::write is a simplified method that does not include additional options. For advanced features such as chunk and concurrent, use Operator::write_with or Operator::write_options instead.

Streaming Write

This method executes a single bulk write operation. For more precise memory management or to write data in a streaming fashion, use Operator::writer instead.

Multipart Uploads

OpenDAL offers multipart upload capabilities through the Writer abstraction, automatically managing all upload details for you. You can fine-tune the upload process by adjusting the chunk size and the number of concurrent operations using Operator::writer_with.

Examples

use bytes::Bytes;

op.write("path/to/file", vec![0; 4096]).await?;

pub fn write_with( &self, path: &str, bs: impl Into<Buffer>, ) -> FutureWrite<impl Future<Output = Result<Metadata>>>

Write all data to the specified path at once with additional options.

Notes

Visit performance::concurrent_write for more details on concurrent writes.

Streaming Write

This method executes a single bulk write operation. For more precise memory management or to write data in a streaming fashion, use Operator::writer instead.

Multipart Uploads

OpenDAL offers multipart upload capabilities through the Writer abstraction, automatically managing all upload details for you. You can fine-tune the upload process by adjusting the chunk size and the number of concurrent operations using Operator::writer_with.

Options

Visit options::WriteOptions for all available options.

Examples

Write data to a file only when it does not already exist:

use bytes::Bytes;

let _ = op
    .write_with("path/to/file", vec![0; 4096])
    .if_not_exists(true)
    .await?;

pub async fn write_options( &self, path: &str, bs: impl Into<Buffer>, opts: WriteOptions, ) -> Result<Metadata>

Write all data to the specified path at once with additional options.

Notes

Visit performance::concurrent_write for more details on concurrent writes.

Streaming Write

This method executes a single bulk write operation. For more precise memory management or to write data in a streaming fashion, use Operator::writer instead.

Multipart Uploads

OpenDAL offers multipart upload capabilities through the Writer abstraction, automatically managing all upload details for you. You can fine-tune the upload process by adjusting the chunk size and the number of concurrent operations using Operator::writer_with.

Examples

Write data to a file only when it does not already exist:

use opendal::options;

let _ = op
    .write_options("path/to/file", vec![0; 4096], options::WriteOptions {
        if_not_exists: true,
        ..Default::default()
    })
    .await?;

pub async fn writer(&self, path: &str) -> Result<Writer>

Create a new writer of given path.

Notes

Writer Features

The writer provides several powerful capabilities:

• Streaming writes for continuous data transfer
• Automatic multipart upload handling
• Memory-efficient chunk-based writing

Extra Options

Operator::writer is a simplified version that does not include additional options. For advanced features such as chunk and concurrent, use Operator::writer_with or Operator::writer_options instead.

Examples

use bytes::Bytes;

let mut w = op.writer("path/to/file").await?;
w.write(vec![0; 4096]).await?;
w.write(vec![1; 4096]).await?;
w.close().await?;

pub fn writer_with( &self, path: &str, ) -> FutureWriter<impl Future<Output = Result<Writer>>>

Create a new writer of given path with additional options.

Notes

Writer Features

The writer provides several powerful capabilities:

• Streaming writes for continuous data transfer
• Automatic multipart upload handling
• Memory-efficient chunk-based writing

Chunk Size Handling

Storage services often have specific requirements for chunk sizes:

• Services like s3 may return EntityTooSmall errors for undersized chunks
• Using small chunks in cloud storage services can lead to increased costs

OpenDAL automatically determines optimal chunk sizes based on the service’s Capability. However, you can override this by explicitly setting the chunk parameter.

Visit performance::concurrent_write for more details on concurrent writes.

Examples

use bytes::Bytes;

let mut w = op
    .writer_with("path/to/file")
    .chunk(4 * 1024 * 1024)
    .concurrent(8)
    .await?;
w.write(vec![0; 4096]).await?;
w.write(vec![1; 4096]).await?;
w.close().await?;

pub async fn writer_options( &self, path: &str, opts: WriteOptions, ) -> Result<Writer>

Create a new writer of given path with additional options.

Notes

Writer Features

The writer provides several powerful capabilities:

• Streaming writes for continuous data transfer
• Automatic multipart upload handling
• Memory-efficient chunk-based writing

Chunk Size Handling

Storage services often have specific requirements for chunk sizes:

• Services like s3 may return EntityTooSmall errors for undersized chunks
• Using small chunks in cloud storage services can lead to increased costs

OpenDAL automatically determines optimal chunk sizes based on the service’s Capability. However, you can override this by explicitly setting the chunk parameter.

Visit performance::concurrent_write for more details on concurrent writes.

Examples

Write data to a file in 4MiB chunk size and at 8 concurrency:

use bytes::Bytes;

let mut w = op
    .writer_with("path/to/file")
    .chunk(4 * 1024 * 1024)
    .concurrent(8)
    .await?;
w.write(vec![0; 4096]).await?;
w.write(vec![1; 4096]).await?;
w.close().await?;

pub async fn copy(&self, from: &str, to: &str) -> Result<()>

Copy a file from from to to.

Notes

• from and to must be a file.
• to will be overwritten if it exists.
• If from and to are the same, an IsSameFile error will occur.
• copy is idempotent. For same from and to input, the result will be the same.

Examples

op.copy("path/to/file", "path/to/file2").await?;

pub async fn rename(&self, from: &str, to: &str) -> Result<()>

Rename a file from from to to.

Notes

• from and to must be a file.
• to will be overwritten if it exists.
• If from and to are the same, an IsSameFile error will occur.

Examples

op.rename("path/to/file", "path/to/file2").await?;

pub async fn delete(&self, path: &str) -> Result<()>

Delete the given path.

Notes

• Deleting a file that does not exist won’t return errors.

Examples

op.delete("test").await?;

pub fn delete_with( &self, path: &str, ) -> FutureDelete<impl Future<Output = Result<()>>>

Delete the given path with additional options.

Notes

• Deleting a file that does not exist won’t return errors.

Options

Visit options::DeleteOptions for all available options.

Examples

Delete a specific version of a file:

op.delete_with("path/to/file").version(version).await?;

pub async fn delete_options( &self, path: &str, opts: DeleteOptions, ) -> Result<()>

Delete the given path with additional options.

Notes

• Deleting a file that does not exist won’t return errors.

Examples

Delete a specific version of a file:

use opendal::options;

op.delete_options("path/to/file", options::DeleteOptions {
    version: Some(version.to_string()),
    ..Default::default()
})
.await?;

pub async fn delete_iter<I, D>(&self, iter: I) -> Result<()>

where I: IntoIterator<Item = D>, D: IntoDeleteInput,

Delete an infallible iterator of paths.

Also see:

• Operator::delete_try_iter: delete a fallible iterator of paths.
• Operator::delete_stream: delete an infallible stream of paths.
• Operator::delete_try_stream: delete a fallible stream of paths.

pub async fn delete_try_iter<I, D>(&self, try_iter: I) -> Result<()>

where I: IntoIterator<Item = Result<D>>, D: IntoDeleteInput,

Delete a fallible iterator of paths.

Also see:

• Operator::delete_iter: delete an infallible iterator of paths.
• Operator::delete_stream: delete an infallible stream of paths.
• Operator::delete_try_stream: delete a fallible stream of paths.

pub async fn delete_stream<S, D>(&self, stream: S) -> Result<()>

where S: Stream<Item = D>, D: IntoDeleteInput,

Delete an infallible stream of paths.

Also see:

• Operator::delete_iter: delete an infallible iterator of paths.
• Operator::delete_try_iter: delete a fallible iterator of paths.
• Operator::delete_try_stream: delete a fallible stream of paths.

pub async fn delete_try_stream<S, D>(&self, try_stream: S) -> Result<()>

where S: Stream<Item = Result<D>>, D: IntoDeleteInput,

Delete a fallible stream of paths.

Also see:

• Operator::delete_iter: delete an infallible iterator of paths.
• Operator::delete_try_iter: delete a fallible iterator of paths.
• Operator::delete_stream: delete an infallible stream of paths.

pub async fn deleter(&self) -> Result<Deleter>

Create a Deleter to continuously remove content from storage.

It leverages batch deletion capabilities provided by storage services for efficient removal.

Users can have more control over the deletion process by using Deleter directly.

pub async fn remove_all(&self, path: &str) -> Result<()>

Remove the path and all nested dirs and files recursively.

Notes

If underlying services support delete in batch, we will use batch delete instead.

Examples

op.remove_all("path/to/dir").await?;

pub async fn list(&self, path: &str) -> Result<Vec<Entry>>

List entries in the parent directory that start with the specified path.

Notes

Recursively List

This function only reads the immediate children of the specified directory. To list all entries recursively, use Operator::list_with("path").recursive(true) instead.

Streaming List

This function reads all entries in the specified directory. If the directory contains many entries, this process may take a long time and use significant memory.

To prevent this, consider using Operator::lister to stream the entries instead.

Examples

This example will list all entries under the dir path/to/dir/.

use opendal::EntryMode;
use opendal::Operator;
let mut entries = op.list("path/to/dir/").await?;
for entry in entries {
    match entry.metadata().mode() {
        EntryMode::FILE => {
            println!("Handling file")
        }
        EntryMode::DIR => {
            println!("Handling dir {}", entry.path())
        }
        EntryMode::Unknown => continue,
    }
}

pub fn list_with( &self, path: &str, ) -> FutureList<impl Future<Output = Result<Vec<Entry>>>>

List entries in the parent directory that start with the specified path with additional options.

Notes

Streaming List

This function reads all entries in the specified directory. If the directory contains many entries, this process may take a long time and use significant memory.

To prevent this, consider using Operator::lister to stream the entries instead.

Options

Visit options::ListOptions for all available options.

Examples

This example will list all entries recursively under the prefix path/to/prefix.

use opendal::EntryMode;
use opendal::Operator;
let mut entries = op.list_with("path/to/prefix").recursive(true).await?;
for entry in entries {
    match entry.metadata().mode() {
        EntryMode::FILE => {
            println!("Handling file")
        }
        EntryMode::DIR => {
            println!("Handling dir like start a new list via meta.path()")
        }
        EntryMode::Unknown => continue,
    }
}

pub async fn list_options( &self, path: &str, opts: ListOptions, ) -> Result<Vec<Entry>>

List entries in the parent directory that start with the specified path with additional options.

Notes

Streaming List

This function reads all entries in the specified directory. If the directory contains many entries, this process may take a long time and use significant memory.

To prevent this, consider using Operator::lister to stream the entries instead.

Options

Visit options::ListOptions for all available options.

Examples

This example will list all entries recursively under the prefix path/to/prefix.

use opendal::options;
use opendal::EntryMode;
use opendal::Operator;
let mut entries = op
    .list_options("path/to/prefix", options::ListOptions {
        recursive: true,
        ..Default::default()
    })
    .await?;
for entry in entries {
    match entry.metadata().mode() {
        EntryMode::FILE => {
            println!("Handling file")
        }
        EntryMode::DIR => {
            println!("Handling dir like start a new list via meta.path()")
        }
        EntryMode::Unknown => continue,
    }
}

pub async fn lister(&self, path: &str) -> Result<Lister>

Create a new lister to list entries that starts with given path in parent dir.

Notes

Recursively list

This function only reads the immediate children of the specified directory. To retrieve all entries recursively, use Operator::lister_with with recursive(true) instead.

Examples

use futures::TryStreamExt;
use opendal::EntryMode;
use opendal::Operator;
let mut ds = op.lister("path/to/dir/").await?;
while let Some(mut de) = ds.try_next().await? {
    match de.metadata().mode() {
        EntryMode::FILE => {
            println!("Handling file")
        }
        EntryMode::DIR => {
            println!("Handling dir like start a new list via meta.path()")
        }
        EntryMode::Unknown => continue,
    }
}

pub fn lister_with( &self, path: &str, ) -> FutureLister<impl Future<Output = Result<Lister>>>

Create a new lister to list entries that starts with given path in parent dir with additional options.

Options

Visit options::ListOptions for all available options.

Examples

List all files recursively

use futures::TryStreamExt;
use opendal::EntryMode;
use opendal::Operator;
let mut lister = op.lister_with("path/to/dir/").recursive(true).await?;
while let Some(mut entry) = lister.try_next().await? {
    match entry.metadata().mode() {
        EntryMode::FILE => {
            println!("Handling file {}", entry.path())
        }
        EntryMode::DIR => {
            println!("Handling dir {}", entry.path())
        }
        EntryMode::Unknown => continue,
    }
}

pub async fn lister_options( &self, path: &str, opts: ListOptions, ) -> Result<Lister>

Create a new lister to list entries that starts with given path in parent dir with additional options.

Examples

List all files recursively

use futures::TryStreamExt;
use opendal::options;
use opendal::EntryMode;
use opendal::Operator;
let mut lister = op
    .lister_options("path/to/dir/", options::ListOptions {
        recursive: true,
        ..Default::default()
    })
    .await?;
while let Some(mut entry) = lister.try_next().await? {
    match entry.metadata().mode() {
        EntryMode::FILE => {
            println!("Handling file {}", entry.path())
        }
        EntryMode::DIR => {
            println!("Handling dir {}", entry.path())
        }
        EntryMode::Unknown => continue,
    }
}

impl Operator

Operator presign API.

pub async fn presign_stat( &self, path: &str, expire: Duration, ) -> Result<PresignedRequest>

Presign an operation for stat(head).

Example

use anyhow::Result;
use futures::io;
use opendal::Operator;
use std::time::Duration;

async fn test(op: Operator) -> Result<()> {
    let signed_req = op.presign_stat("test",Duration::from_secs(3600)).await?;
    let req = http::Request::builder()
        .method(signed_req.method())
        .uri(signed_req.uri())
        .body(())?;

pub fn presign_stat_with( &self, path: &str, expire: Duration, ) -> FuturePresignStat<impl Future<Output = Result<PresignedRequest>>>

Presign an operation for stat(head).

Example

use anyhow::Result;
use futures::io;
use opendal::Operator;
use std::time::Duration;

async fn test(op: Operator) -> Result<()> {
    let signed_req = op.presign_stat_with("test",Duration::from_secs(3600)).override_content_disposition("attachment; filename=\"othertext.txt\"").await?;

pub async fn presign_stat_options( &self, path: &str, expire: Duration, opts: StatOptions, ) -> Result<PresignedRequest>

Presign an operation for stat(head) with additional options.

Options

Visit options::StatOptions for all available options.

Example

use anyhow::Result;
use opendal::Operator;
use opendal::options;
use std::time::Duration;

async fn test(op: Operator) -> Result<()> {
    let signed_req = op.presign_stat_options(
        "test",
        Duration::from_secs(3600),
        options::StatOptions {
            if_match: Some("<etag>".to_string()),
            ..Default::default()
        }
    ).await?;
    let req = http::Request::builder()
        .method(signed_req.method())
        .uri(signed_req.uri())
        .body(())?;

pub async fn presign_read( &self, path: &str, expire: Duration, ) -> Result<PresignedRequest>

Presign an operation for read.

Notes

Extra Options

presign_read is a wrapper of Self::presign_read_with without any options. To use extra options like override_content_disposition, please use Self::presign_read_with or `Self::presign_read_options instead.

Example

use anyhow::Result;
use futures::io;
use opendal::Operator;
use std::time::Duration;

async fn test(op: Operator) -> Result<()> {
    let signed_req = op.presign_read("test.txt", Duration::from_secs(3600)).await?;

• signed_req.method(): GET
• signed_req.uri(): https://s3.amazonaws.com/examplebucket/test.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=access_key_id/20130721/us-east-1/s3/aws4_request&X-Amz-Date=20130721T201207Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&X-Amz-Signature=<signature-value>
• signed_req.headers(): { "host": "s3.amazonaws.com" }

We can download this file via curl or other tools without credentials:

curl "https://s3.amazonaws.com/examplebucket/test.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=access_key_id/20130721/us-east-1/s3/aws4_request&X-Amz-Date=20130721T201207Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&X-Amz-Signature=<signature-value>" -O /tmp/test.txt

pub fn presign_read_with( &self, path: &str, expire: Duration, ) -> FuturePresignRead<impl Future<Output = Result<PresignedRequest>>>

Presign an operation for read with extra options.

Options

Visit options::ReadOptions for all available options.

Example

use std::time::Duration;

use anyhow::Result;
use futures::io;
use opendal::Operator;

async fn test(op: Operator) -> Result<()> {
    let signed_req = op
        .presign_read_with("test.txt", Duration::from_secs(3600))
        .override_content_type("text/plain")
        .await?;
    Ok(())
}

pub async fn presign_read_options( &self, path: &str, expire: Duration, opts: ReadOptions, ) -> Result<PresignedRequest>

Presign an operation for read with additional options.

Options

Visit options::ReadOptions for all available options.

Example

use anyhow::Result;
use opendal::Operator;
use opendal::options;
use std::time::Duration;

async fn test(op: Operator) -> Result<()> {
    let signed_req = op.presign_read_options(
        "file",
        Duration::from_secs(3600),
        options::ReadOptions {
            override_content_disposition: Some("attachment; filename=\"othertext.txt\"".to_string()),
            ..Default::default()
        }
    ).await?;
    let req = http::Request::builder()
        .method(signed_req.method())
        .uri(signed_req.uri())
        .body(())?;

pub async fn presign_write( &self, path: &str, expire: Duration, ) -> Result<PresignedRequest>

Presign an operation for write.

Notes

Extra Options

presign_write is a wrapper of Self::presign_write_with without any options. To use extra options like content_type, please use Self::presign_write_with or Self::presign_write_options instead.

Example

use std::time::Duration;

use anyhow::Result;
use opendal::Operator;

async fn test(op: Operator) -> Result<()> {
    let signed_req = op
        .presign_write("test.txt", Duration::from_secs(3600))
        .await?;
    Ok(())
}

• signed_req.method(): PUT
• signed_req.uri(): https://s3.amazonaws.com/examplebucket/test.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=access_key_id/20130721/us-east-1/s3/aws4_request&X-Amz-Date=20130721T201207Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&X-Amz-Signature=<signature-value>
• signed_req.headers(): { "host": "s3.amazonaws.com" }

We can upload file as this file via curl or other tools without credential:

curl -X PUT "https://s3.amazonaws.com/examplebucket/test.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=access_key_id/20130721/us-east-1/s3/aws4_request&X-Amz-Date=20130721T201207Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&X-Amz-Signature=<signature-value>" -d "Hello, World!"

pub fn presign_write_with( &self, path: &str, expire: Duration, ) -> FuturePresignWrite<impl Future<Output = Result<PresignedRequest>>>

Presign an operation for write with extra options.

Options

Visit options::WriteOptions for all available options.

Example

use std::time::Duration;

use anyhow::Result;
use opendal::Operator;

async fn test(op: Operator) -> Result<()> {
    let signed_req = op
        .presign_write_with("test", Duration::from_secs(3600))
        .cache_control("no-store")
        .await?;
    let req = http::Request::builder()
        .method(signed_req.method())
        .uri(signed_req.uri())
        .body(())?;

    Ok(())
}

pub async fn presign_write_options( &self, path: &str, expire: Duration, opts: WriteOptions, ) -> Result<PresignedRequest>

Presign an operation for write with additional options.

Options

Check options::WriteOptions for all available options.

Example

use anyhow::Result;
use opendal::Operator;
use opendal::options;
use std::time::Duration;

async fn test(op: Operator) -> Result<()> {
    let signed_req = op.presign_write_options(
        "file",
        Duration::from_secs(3600),
        options::WriteOptions {
            content_type: Some("application/json".to_string()),
            cache_control: Some("max-age=3600".to_string()),
            if_not_exists: true,
            ..Default::default()
        }
    ).await?;
    let req = http::Request::builder()
        .method(signed_req.method())
        .uri(signed_req.uri())
        .body(())?;

pub async fn presign_delete( &self, path: &str, expire: Duration, ) -> Result<PresignedRequest>

Presign an operation for delete.

Notes

Extra Options

presign_delete is a wrapper of Self::presign_delete_with without any options.

Example

use std::time::Duration;

use anyhow::Result;
use opendal::Operator;

async fn test(op: Operator) -> Result<()> {
    let signed_req = op
        .presign_delete("test.txt", Duration::from_secs(3600))
        .await?;
    Ok(())
}

• signed_req.method(): DELETE
• signed_req.uri(): https://s3.amazonaws.com/examplebucket/test.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=access_key_id/20130721/us-east-1/s3/aws4_request&X-Amz-Date=20130721T201207Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&X-Amz-Signature=<signature-value>
• signed_req.headers(): { "host": "s3.amazonaws.com" }

We can delete file as this file via curl or other tools without credential:

curl -X DELETE "https://s3.amazonaws.com/examplebucket/test.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=access_key_id/20130721/us-east-1/s3/aws4_request&X-Amz-Date=20130721T201207Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&X-Amz-Signature=<signature-value>"

pub fn presign_delete_with( &self, path: &str, expire: Duration, ) -> FuturePresignDelete<impl Future<Output = Result<PresignedRequest>>>

Presign an operation for delete without extra options.

pub async fn presign_delete_options( &self, path: &str, expire: Duration, opts: DeleteOptions, ) -> Result<PresignedRequest>

Presign an operation for delete with additional options.

Options

Visit options::DeleteOptions for all available options.

Example

use anyhow::Result;
use opendal::Operator;
use opendal::options;
use std::time::Duration;

async fn test(op: Operator) -> Result<()> {
    let signed_req = op.presign_delete_options(
        "path/to/file",
        Duration::from_secs(3600),
        options::DeleteOptions {
            ..Default::default()
        }
    ).await?;
    let req = http::Request::builder()
        .method(signed_req.method())
        .uri(signed_req.uri())
        .body(())?;

impl Operator

Operator build API

Operator should be built via OperatorBuilder. We recommend to use Operator::new to get started:

use opendal::services::Fs;
use opendal::Operator;
async fn test() -> Result<()> {
    // Create fs backend builder.
    let builder = Fs::default().root("/tmp");

    // Build an `Operator` to start operating the storage.
    let op: Operator = Operator::new(builder)?.finish();

    Ok(())
}

pub fn new<B: Builder>(ab: B) -> Result<OperatorBuilder<impl Access>>

Create a new operator with input builder.

OpenDAL will call builder.build() internally, so we don’t need to import opendal::Builder trait.

Examples

Read more backend init examples in examples.

use opendal::services::Fs;
use opendal::Operator;
async fn test() -> Result<()> {
    // Create fs backend builder.
    let builder = Fs::default().root("/tmp");

    // Build an `Operator` to start operating the storage.
    let op: Operator = Operator::new(builder)?.finish();

    Ok(())
}

pub fn from_config<C: Configurator>( cfg: C, ) -> Result<OperatorBuilder<impl Access>>

Create a new operator from given config.

Examples

use std::collections::HashMap;

use opendal::services::MemoryConfig;
use opendal::Operator;
async fn test() -> Result<()> {
    let cfg = MemoryConfig::default();

    // Build an `Operator` to start operating the storage.
    let op: Operator = Operator::from_config(cfg)?.finish();

    Ok(())
}

pub fn from_iter<B: Builder>( iter: impl IntoIterator<Item = (String, String)>, ) -> Result<OperatorBuilder<impl Access>>

Create a new operator from given iterator in static dispatch.

Notes

from_iter generates a OperatorBuilder which allows adding layer in zero-cost way.

Examples

use std::collections::HashMap;

use opendal::services::Fs;
use opendal::Operator;
async fn test() -> Result<()> {
    let map = HashMap::from([
        // Set the root for fs, all operations will happen under this root.
        //
        // NOTE: the root must be absolute path.
        ("root".to_string(), "/tmp".to_string()),
    ]);

    // Build an `Operator` to start operating the storage.
    let op: Operator = Operator::from_iter::<Fs>(map)?.finish();

    Ok(())
}

pub fn via_iter( scheme: Scheme, iter: impl IntoIterator<Item = (String, String)>, ) -> Result<Operator>

Create a new operator via given scheme and iterator of config value in dynamic dispatch.

Notes

via_iter generates a Operator which allows building operator without generic type.

Examples

use std::collections::HashMap;

use opendal::Operator;
use opendal::Scheme;
async fn test() -> Result<()> {
    let map = [
        // Set the root for fs, all operations will happen under this root.
        //
        // NOTE: the root must be absolute path.
        ("root".to_string(), "/tmp".to_string()),
    ];

    // Build an `Operator` to start operating the storage.
    let op: Operator = Operator::via_iter(Scheme::Fs, map)?;

    Ok(())
}

pub fn from_map<B: Builder>( map: HashMap<String, String>, ) -> Result<OperatorBuilder<impl Access>>

👎Deprecated: use from_iter instead

Create a new operator from given map.

Notes

from_map is using static dispatch layers which is zero cost. via_map is using dynamic dispatch layers which has a bit runtime overhead with an extra vtable lookup and unable to inline. But from_map requires generic type parameter which is not always easy to be used.

Examples

use std::collections::HashMap;

use opendal::services::Fs;
use opendal::Operator;
async fn test() -> Result<()> {
    let map = HashMap::from([
        // Set the root for fs, all operations will happen under this root.
        //
        // NOTE: the root must be absolute path.
        ("root".to_string(), "/tmp".to_string()),
    ]);

    // Build an `Operator` to start operating the storage.
    let op: Operator = Operator::from_map::<Fs>(map)?.finish();

    Ok(())
}

pub fn via_map(scheme: Scheme, map: HashMap<String, String>) -> Result<Operator>

👎Deprecated: use via_iter instead

Create a new operator from given scheme and map.

Notes

from_map is using static dispatch layers which is zero cost. via_map is using dynamic dispatch layers which has a bit runtime overhead with an extra vtable lookup and unable to inline. But from_map requires generic type parameter which is not always easy to be used.

Examples

use std::collections::HashMap;

use opendal::Operator;
use opendal::Scheme;
async fn test() -> Result<()> {
    let map = HashMap::from([
        // Set the root for fs, all operations will happen under this root.
        //
        // NOTE: the root must be absolute path.
        ("root".to_string(), "/tmp".to_string()),
    ]);

    // Build an `Operator` to start operating the storage.
    let op: Operator = Operator::via_map(Scheme::Fs, map)?;

    Ok(())
}

pub fn layer<L: Layer<Accessor>>(self, layer: L) -> Self

Create a new layer with dynamic dispatch.

Please note that Layer can modify internal contexts such as HttpClient and Runtime for the operator. Therefore, it is recommended to add layers before interacting with the storage. Adding or duplicating layers after accessing the storage may result in unexpected behavior.

Notes

OperatorBuilder::layer() is using static dispatch which is zero cost. Operator::layer() is using dynamic dispatch which has a bit runtime overhead with an extra vtable lookup and unable to inline.

It’s always recommended to use OperatorBuilder::layer() instead.

Examples

use opendal::layers::LoggingLayer;
use opendal::services::Fs;
use opendal::Operator;

let op = Operator::new(Fs::default())?.finish();
let op = op.layer(LoggingLayer::default());
// All operations will go through the new_layer
let _ = op.read("test_file").await?;

Trait Implementations

impl Clone for Operator

fn clone(&self) -> Operator

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Operator

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl From<Operator> for Operator

Available on crate feature blocking only.

fn from(val: Operator) -> Self

Converts to this type from the input type.