Struct opendal::Reader

pub struct Reader { /* private fields */ }

Reader is designed to read data from given path in an asynchronous manner.

Usage

Reader provides multiple ways to read data from given reader. Please note that it’s undefined behavior to use Reader in different ways.

Direct

Reader provides public API including Reader::read, [Reader:read_range], and [Reader::read_to_end]. You can use those APIs directly without extra copy.

Implementations

impl Reader

pub async fn read(&self, range: impl RangeBounds<u64>) -> Result<Buffer>

Read give range from reader into Buffer.

This operation is zero-copy, which means it keeps the [Bytes] returned by underlying storage services without any extra copy or intensive memory allocations.

Notes

• Buffer length smaller than range means we have reached the end of file.

pub async fn read_into( &self, buf: &mut impl BufMut, range: impl RangeBounds<u64> ) -> Result<usize>

Read all data from reader into given [BufMut].

This operation will copy and write bytes into given [BufMut]. Allocation happens while [BufMut] doesn’t have enough space.

Notes

• Returning length smaller than range means we have reached the end of file.

pub async fn fetch(&self, ranges: Vec<Range<u64>>) -> Result<Vec<Buffer>>

Fetch specific ranges from reader.

This operation try to merge given ranges into a list of non-overlapping ranges. Users may also specify a gap to merge close ranges.

The returning Buffer may share the same underlying memory without any extra copy.

pub fn into_futures_async_read(self, range: Range<u64>) -> FuturesAsyncReader

Convert reader into FuturesAsyncReader which implements [futures::AsyncRead], [futures::AsyncSeek] and [futures::AsyncBufRead].

Notes

FuturesAsyncReader is not a zero-cost abstraction. The underlying reader returns an owned Buffer, which involves an extra copy operation.

Examples

Basic Usage

use std::io;

use futures::io::AsyncReadExt;
use opendal::Operator;
use opendal::Result;

async fn test(op: Operator) -> io::Result<()> {
    let mut r = op
        .reader("hello.txt")
        .await?
        .into_futures_async_read(1024..2048);
    let mut bs = Vec::new();
    r.read_to_end(&mut bs).await?;

    Ok(())
}

Concurrent Read

The following example reads data in 256B chunks with 8 concurrent.

use std::io;

use futures::io::AsyncReadExt;
use opendal::Operator;
use opendal::Result;

async fn test(op: Operator) -> io::Result<()> {
    let mut r = op
        .reader_with("hello.txt")
        .concurrent(8)
        .chunk(256)
        .await?
        .into_futures_async_read(1024..2048);
    let mut bs = Vec::new();
    r.read_to_end(&mut bs).await?;

    Ok(())
}

pub fn into_bytes_stream( self, range: impl RangeBounds<u64> ) -> FuturesBytesStream

Convert reader into FuturesBytesStream which implements [futures::Stream].

Examples

Basic Usage

use std::io;

use bytes::Bytes;
use futures::TryStreamExt;
use opendal::Operator;
use opendal::Result;

async fn test(op: Operator) -> io::Result<()> {
    let mut s = op.reader("hello.txt").await?.into_bytes_stream(1024..2048);
    let bs: Vec<Bytes> = s.try_collect().await?;

    Ok(())
}

Concurrent Read

The following example reads data in 256B chunks with 8 concurrent.

use std::io;

use bytes::Bytes;
use futures::TryStreamExt;
use opendal::Operator;
use opendal::Result;

async fn test(op: Operator) -> io::Result<()> {
    let mut s = op
        .reader_with("hello.txt")
        .concurrent(8)
        .chunk(256)
        .await?
        .into_bytes_stream(1024..2048);
    let bs: Vec<Bytes> = s.try_collect().await?;

    Ok(())
}

Trait Implementations

impl Clone for Reader

fn clone(&self) -> Reader

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.