Struct Writer

Source

pub struct Writer { /* private fields */ }

Expand description

Writer is designed to write data into given path in an asynchronous manner.

§Notes

Please make sure either close or abort has been called before dropping the writer otherwise the data could be lost.

§Usage

§Write Multiple Chunks

Some services support to write multiple chunks of data into given path. Services that doesn’t support write multiple chunks will return ErrorKind::Unsupported error when calling write at the second time.

use opendal_core::Operator;
use opendal_core::Result;

async fn test(op: Operator) -> Result<()> {
    let mut w = op.writer("path/to/file").await?;
    w.write(vec![1; 1024]).await?;
    w.write(vec![2; 1024]).await?;
    w.close().await?;
    Ok(())
}

§Write like `Sink`

use anyhow::Result;
use futures::SinkExt;
use opendal_core::Operator;

async fn test(op: Operator) -> Result<()> {
    let mut w = op.writer("path/to/file").await?.into_bytes_sink();
    w.send(vec![1; 1024].into()).await?;
    w.send(vec![2; 1024].into()).await?;
    w.close().await?;
    Ok(())
}

§Write like `AsyncWrite`

use anyhow::Result;
use futures::AsyncWriteExt;
use opendal_core::Operator;

async fn test(op: Operator) -> Result<()> {
    let mut w = op.writer("path/to/file").await?.into_futures_async_write();
    w.write(&vec![1; 1024]).await?;
    w.write(&vec![2; 1024]).await?;
    w.close().await?;
    Ok(())
}

§Write with append enabled

Writer also supports to write with append enabled. This is useful when users want to append some data to the end of the file.

If file doesn’t exist, it will be created and just like calling write.
If file exists, data will be appended to the end of the file.

Possible Errors:

Some services store normal file and appendable file in different way. Trying to append on non-appendable file could return ErrorKind::ConditionNotMatch error.
Services that doesn’t support append will return ErrorKind::Unsupported error when creating writer with append enabled.

Implementations§

Source §

impl Writer

Source

pub async fn write(&mut self, bs: impl Into<Buffer>) -> Result<()>

Write Buffer into writer.

This operation will write all data in given buffer into writer.

§Examples

use bytes::Bytes;
use opendal_core::Operator;
use opendal_core::Result;

async fn test(op: Operator) -> Result<()> {
    let mut w = op.writer("hello.txt").await?;
    // Buffer can be created from continues bytes.
    w.write("hello, world").await?;
    // Buffer can also be created from non-continues bytes.
    w.write(vec![Bytes::from("hello,"), Bytes::from("world!")])
        .await?;

    // Make sure file has been written completely.
    w.close().await?;
    Ok(())
}

Source

pub async fn write_from(&mut self, bs: impl Buf) -> Result<()>

Write [bytes::Buf] into inner writer.

This operation will write all data in given buffer into writer.

§Notes

This function iterates over each chunk in the Buf and collects them into a Buffer. For Bytes chunks, copy_to_bytes is zero-copy (only a ref-count increment). For other chunk types (e.g., &[u8]), the data will be copied.

This approach minimizes copies for chained buffers (Chain<Bytes, Bytes>) where each individual chunk can be extracted without copying.

Source

pub async fn abort(&mut self) -> Result<()>

Abort the writer and clean up all written data.

§Notes

Abort should only be called when the writer is not closed or aborted, otherwise an unexpected error could be returned.

Source

pub async fn close(&mut self) -> Result<Metadata>

Close the writer and make sure all data have been committed.

§Notes

Close should only be called when the writer is not closed or aborted, otherwise an unexpected error could be returned.

Source

pub fn into_sink(self) -> BufferSink

Convert writer into BufferSink which implements [Sink<Buffer>].

§Notes

BufferSink is a zero-cost abstraction. The underlying writer will reuse the Bytes and won’t perform any copy operation over data.

§Examples

§Basic Usage

use std::io;

use bytes::Bytes;
use futures::SinkExt;
use opendal_core::Buffer;
use opendal_core::Operator;
use opendal_core::Result;

async fn test(op: Operator) -> io::Result<()> {
    let mut s = op.writer("hello.txt").await?.into_sink();
    let bs = "Hello, World!".as_bytes();
    s.send(Buffer::from(bs)).await?;
    s.close().await?;

    Ok(())
}

§Concurrent Write

use std::io;

use bytes::Bytes;
use futures::SinkExt;
use opendal_core::Buffer;
use opendal_core::Operator;
use opendal_core::Result;

async fn test(op: Operator) -> io::Result<()> {
    let mut w = op
        .writer_with("hello.txt")
        .concurrent(8)
        .chunk(256)
        .await?
        .into_sink();
    let bs = "Hello, World!".as_bytes();
    w.send(Buffer::from(bs)).await?;
    w.close().await?;

    Ok(())
}

Source

pub fn into_futures_async_write(self) -> FuturesAsyncWriter

Convert writer into FuturesAsyncWriter which implements [futures::AsyncWrite],

§Notes

FuturesAsyncWriter is not a zero-cost abstraction. The underlying writer requires an owned Buffer, which involves an extra copy operation.

FuturesAsyncWriter is required to call close() to make sure all data have been written to the storage.

§Examples

§Basic Usage

use std::io;

use futures::io::AsyncWriteExt;
use opendal_core::Operator;
use opendal_core::Result;

async fn test(op: Operator) -> io::Result<()> {
    let mut w = op.writer("hello.txt").await?.into_futures_async_write();
    let bs = "Hello, World!".as_bytes();
    w.write_all(bs).await?;
    w.close().await?;

    Ok(())
}

§Concurrent Write

use std::io;

use futures::io::AsyncWriteExt;
use opendal_core::Operator;
use opendal_core::Result;

async fn test(op: Operator) -> io::Result<()> {
    let mut w = op
        .writer_with("hello.txt")
        .concurrent(8)
        .chunk(256)
        .await?
        .into_futures_async_write();
    let bs = "Hello, World!".as_bytes();
    w.write_all(bs).await?;
    w.close().await?;

    Ok(())
}

Source

pub fn into_bytes_sink(self) -> FuturesBytesSink

Convert writer into FuturesBytesSink which implements [futures::Sink<Bytes>].

§Notes

FuturesBytesSink is a zero-cost abstraction. The underlying writer will reuse the Bytes and won’t perform any copy operation.

§Examples

§Basic Usage

use std::io;

use bytes::Bytes;
use futures::SinkExt;
use opendal_core::Operator;
use opendal_core::Result;

async fn test(op: Operator) -> io::Result<()> {
    let mut w = op.writer("hello.txt").await?.into_bytes_sink();
    let bs = "Hello, World!".as_bytes();
    w.send(Bytes::from(bs)).await?;
    w.close().await?;

    Ok(())
}

§Concurrent Write

use std::io;

use bytes::Bytes;
use futures::SinkExt;
use opendal_core::Operator;
use opendal_core::Result;

async fn test(op: Operator) -> io::Result<()> {
    let mut w = op
        .writer_with("hello.txt")
        .concurrent(8)
        .chunk(256)
        .await?
        .into_bytes_sink();
    let bs = "Hello, World!".as_bytes();
    w.send(Bytes::from(bs)).await?;
    w.close().await?;

    Ok(())
}