Struct Operator

pub struct Operator { /* private fields */ }

Available on crate feature blocking only.

Use OpenDAL in blocking context.

Notes

blocking::Operator is a wrapper around AsyncOperator. It calls async runtimes’ block_on API to spawn blocking tasks. Please avoid using blocking::Operator in async context.

Examples

Init in async context

blocking::Operator will use current async context’s runtime to handle the async calls.

This is just for initialization. You must use blocking::Operator in blocking context.

#[tokio::main]
async fn main() -> Result<()> {
    // Create fs backend builder.
    let mut builder = services::S3::default().bucket("test").region("us-east-1");
    let op = Operator::new(builder)?.finish();

    // Build an `blocking::Operator` with blocking layer to start operating the storage.
    let _: blocking::Operator = blocking::Operator::new(op)?;

    Ok(())
}

In async context with blocking functions

If blocking::Operator is called in blocking function, please fetch a [tokio::runtime::EnterGuard] first. You can use [Handle::try_current] first to get the handle and then call [Handle::enter]. This often happens in the case that async function calls blocking function.

#[tokio::main]
async fn main() -> Result<()> {
    let _ = blocking_fn()?;
    Ok(())
}

fn blocking_fn() -> Result<blocking::Operator> {
    // Create fs backend builder.
    let mut builder = services::S3::default().bucket("test").region("us-east-1");
    let op = Operator::new(builder)?.finish();

    let handle = tokio::runtime::Handle::try_current().unwrap();
    let _guard = handle.enter();
    // Build an `blocking::Operator` to start operating the storage.
    let op: blocking::Operator = blocking::Operator::new(op)?;
    Ok(op)
}

In blocking context

In a pure blocking context, we can create a runtime and use it to create the blocking::Operator.

The following code uses a global statically created runtime as an example, please manage the runtime on demand.

static RUNTIME: LazyLock<tokio::runtime::Runtime> = LazyLock::new(|| {
    tokio::runtime::Builder::new_multi_thread()
        .enable_all()
        .build()
        .unwrap()
});

fn main() -> Result<()> {
    // Create fs backend builder.
    let mut builder = services::S3::default().bucket("test").region("us-east-1");
    let op = Operator::new(builder)?.finish();

    // Fetch the `EnterGuard` from global runtime.
    let _guard = RUNTIME.enter();
    // Build an `blocking::Operator` with blocking layer to start operating the storage.
    let _: blocking::Operator = blocking::Operator::new(op)?;

    Ok(())
}

Implementations

impl Operator

pub fn new(op: AsyncOperator) -> Result<Self>

Create a new BlockingLayer with the current runtime’s handle

pub fn info(&self) -> OperatorInfo

Get information of underlying accessor.

Examples

use opendal::blocking;
use opendal::blocking::Operator;

let info = op.info();

impl Operator

Operator blocking API.

pub fn stat(&self, path: &str) -> Result<Metadata>

Get given path’s metadata.

Behavior

Services that support create_dir

test and test/ may vary in some services such as S3. However, on a local file system, they’re identical. Therefore, the behavior of stat("test") and stat("test/") might differ in certain edge cases. Always use stat("test/") when you need to access a directory if possible.

Here are the behavior list:

Case	Path	Result
stat existing dir	abc/	Metadata with dir mode
stat existing file	abc/def_file	Metadata with file mode
stat dir without /	abc/def_dir	Error NotFound or metadata with dir mode
stat file with /	abc/def_file/	Error NotFound
stat not existing path	xyz	Error NotFound

Refer to RFC: List Prefix for more details.

Services that not support create_dir

For services that not support create_dir, stat("test/") will return NotFound even when test/abc exists since the service won’t have the concept of dir. There is nothing we can do about this.

Examples

Check if file exists

use opendal::blocking;
use opendal::ErrorKind;
if let Err(e) = op.stat("test") {
    if e.kind() == ErrorKind::NotFound {
        println!("file not exist")
    }
}

pub fn stat_options(&self, path: &str, opts: StatOptions) -> Result<Metadata>

Get given path’s metadata with extra options.

Behavior

Services that support create_dir

test and test/ may vary in some services such as S3. However, on a local file system, they’re identical. Therefore, the behavior of stat("test") and stat("test/") might differ in certain edge cases. Always use stat("test/") when you need to access a directory if possible.

Here are the behavior list:

Case	Path	Result
stat existing dir	abc/	Metadata with dir mode
stat existing file	abc/def_file	Metadata with file mode
stat dir without /	abc/def_dir	Error NotFound or metadata with dir mode
stat file with /	abc/def_file/	Error NotFound
stat not existing path	xyz	Error NotFound

Refer to RFC: List Prefix for more details.

Services that not support create_dir

For services that not support create_dir, stat("test/") will return NotFound even when test/abc exists since the service won’t have the concept of dir. There is nothing we can do about this.

pub fn exists(&self, path: &str) -> Result<bool>

Check if this path exists or not.

Example

use anyhow::Result;
use opendal::blocking;
use opendal::blocking::Operator;
fn test(op: blocking::Operator) -> Result<()> {
    let _ = op.exists("test")?;

    Ok(())
}

pub fn create_dir(&self, path: &str) -> Result<()>

Create a dir at given path.

Notes

To indicate that a path is a directory, it is compulsory to include a trailing / in the path. Failure to do so may result in NotADirectory error being returned by OpenDAL.

Behavior

• Create on existing dir will succeed.
• Create dir is always recursive, works like mkdir -p

Examples

use opendal::blocking;
op.create_dir("path/to/dir/")?;

pub fn read(&self, path: &str) -> Result<Buffer>

Read the whole path into a bytes.

This function will allocate a new bytes internally. For more precise memory control or reading data lazily, please use blocking::Operator::reader

Examples

use opendal::blocking;
let bs = op.read("path/to/file")?;

pub fn read_options(&self, path: &str, opts: ReadOptions) -> Result<Buffer>

Read the whole path into a bytes with extra options.

This function will allocate a new bytes internally. For more precise memory control or reading data lazily, please use blocking::Operator::reader

pub fn reader(&self, path: &str) -> Result<Reader>

Create a new reader which can read the whole path.

Examples

use opendal::blocking;
let r = op.reader("path/to/file")?;

pub fn reader_options(&self, path: &str, opts: ReaderOptions) -> Result<Reader>

Create a new reader with extra options

pub fn write(&self, path: &str, bs: impl Into<Buffer>) -> Result<Metadata>

Write bytes into given path.

Notes

• Write will make sure all bytes has been written, or an error will be returned.

Examples

use opendal::blocking;
use bytes::Bytes;

op.write("path/to/file", vec![0; 4096])?;

pub fn write_options( &self, path: &str, bs: impl Into<Buffer>, opts: WriteOptions, ) -> Result<Metadata>

Write data with options.

Notes

• Write will make sure all bytes has been written, or an error will be returned.

pub fn writer(&self, path: &str) -> Result<Writer>

Write multiple bytes into given path.

Notes

• Write will make sure all bytes has been written, or an error will be returned.

Examples

use bytes::Bytes;

let mut w = op.writer("path/to/file")?;
w.write(vec![0; 4096])?;
w.write(vec![1; 4096])?;
w.close()?;

pub fn writer_options(&self, path: &str, opts: WriteOptions) -> Result<Writer>

Create a new writer with extra options

pub fn copy(&self, from: &str, to: &str) -> Result<()>

Copy a file from from to to.

Notes

• from and to must be a file.
• to will be overwritten if it exists.
• If from and to are the same, nothing will happen.
• copy is idempotent. For same from and to input, the result will be the same.

Examples

use opendal::blocking;

op.copy("path/to/file", "path/to/file2")?;

pub fn rename(&self, from: &str, to: &str) -> Result<()>

Rename a file from from to to.

Notes

• from and to must be a file.
• to will be overwritten if it exists.
• If from and to are the same, a IsSameFile error will occur.

Examples

use opendal::blocking;

op.rename("path/to/file", "path/to/file2")?;

pub fn delete(&self, path: &str) -> Result<()>

Delete given path.

Notes

• Delete not existing error won’t return errors.

Examples

use opendal::blocking;
op.delete("path/to/file")?;

pub fn delete_options(&self, path: &str, opts: DeleteOptions) -> Result<()>

Delete given path with options.

Notes

• Delete not existing error won’t return errors.

pub fn delete_iter<I, D>(&self, iter: I) -> Result<()>

where I: IntoIterator<Item = D>, D: IntoDeleteInput,

Delete an infallible iterator of paths.

Also see:

• blocking::Operator::delete_try_iter: delete an fallible iterator of paths.

pub fn delete_try_iter<I, D>(&self, try_iter: I) -> Result<()>

where I: IntoIterator<Item = Result<D>>, D: IntoDeleteInput,

Delete a fallible iterator of paths.

Also see:

• blocking::Operator::delete_iter: delete an infallible iterator of paths.

pub fn deleter(&self) -> Result<Deleter>

Create a [BlockingDeleter] to continuously remove content from storage.

It leverages batch deletion capabilities provided by storage services for efficient removal.

Users can have more control over the deletion process by using [BlockingDeleter] directly.

pub fn remove_all(&self, path: &str) -> Result<()>

Remove the path and all nested dirs and files recursively.

Notes

We don’t support batch delete now.

Examples

use opendal::blocking;
op.remove_all("path/to/dir")?;

pub fn list(&self, path: &str) -> Result<Vec<Entry>>

List entries that starts with given path in parent dir.

Notes

Recursively List

This function only read the children of the given directory. To read all entries recursively, use blocking::Operator::list_options("path", opts) instead.

Streaming List

This function will read all entries in the given directory. It could take very long time and consume a lot of memory if the directory contains a lot of entries.

In order to avoid this, you can use blocking::Operator::lister to list entries in a streaming way.

Examples

use opendal::blocking;
use opendal::blocking::Operator;
use opendal::EntryMode;
let mut entries = op.list("path/to/dir/")?;
for entry in entries {
    match entry.metadata().mode() {
        EntryMode::FILE => {
            println!("Handling file")
        }
        EntryMode::DIR => {
            println!("Handling dir {}", entry.path())
        }
        EntryMode::Unknown => continue,
    }
}

pub fn list_options(&self, path: &str, opts: ListOptions) -> Result<Vec<Entry>>

List entries that starts with given path in parent dir. with options.

Notes

Streaming List

This function will read all entries in the given directory. It could take very long time and consume a lot of memory if the directory contains a lot of entries.

In order to avoid this, you can use blocking::Operator::lister to list entries in a streaming way.

pub fn lister(&self, path: &str) -> Result<Lister>

List entries that starts with given path in parent dir.

This function will create a new [BlockingLister] to list entries. Users can stop listing via dropping this Lister.

Notes

Recursively List

This function only read the children of the given directory. To read all entries recursively, use [blocking::Operator::lister_with] and delimiter("") instead.

Examples

use futures::TryStreamExt;
use opendal::blocking;
use opendal::blocking::Operator;
use opendal::EntryMode;
let mut ds = op.lister("path/to/dir/")?;
for de in ds {
    let de = de?;
    match de.metadata().mode() {
        EntryMode::FILE => {
            println!("Handling file")
        }
        EntryMode::DIR => {
            println!("Handling dir like start a new list via meta.path()")
        }
        EntryMode::Unknown => continue,
    }
}

pub fn lister_options(&self, path: &str, opts: ListOptions) -> Result<Lister>

List entries within a given directory as an iterator with options.

This function will create a new handle to list entries.

An error will be returned if given path doesn’t end with /.

pub fn check(&self) -> Result<()>

Check if this operator can work correctly.

We will send a list request to path and return any errors we met.

use opendal::blocking;
use opendal::blocking::Operator;
use opendal::ErrorKind;

op.check()?;

Trait Implementations

impl Clone for Operator

fn clone(&self) -> Operator

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Operator

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl From<Operator> for Operator

fn from(val: Operator) -> Self

Converts to this type from the input type.