Module opendal::docs::rfcs::rfc_1477_remove_object_concept

source ·
Available on docsrs only.
Expand description

Remove object concept

§Summary

Eliminating the Object concept to enhance the readability of OpenDAL.

§Motivation

OpenDAL introduces Object Native API to resolve the problem of not being easy to use:

- let reader = SeekableReader::new(op, path, stream_len);
+ let reader = op.object(&path).reader().await?;

- op.stat(&path).run().await
+ op.object(&path).stat().await

However, times are changing. After the list operation has been moved to the Object level, Object is now more like a wrapper for Operator. The only meaningful API of Operator now is Operator::object.

Writing op.object(&path) repeatedly is boring. Let’s take real example from databend as an example:

if let Some(dir) = dir_path {
    match op.object(&dir).stat().await {
        Ok(_) => {
            let mut ds = op.object(&dir).scan().await?;
            while let Some(de) = ds.try_next().await? {
                if let Some(fi) = stat_file(de).await? {
                    files.push(fi)
                }
            }
        }
        Err(e) => warn!("ignore listing {path}/, because: {:?}", e),
    };
}

We designed Object to make users can reuse the same Object. However, nearly no users use our API this way. Most users just build a new Object every time. There are two problems:

§Extra cost

Object::new() is not zero cost:

pub(crate) fn with(op: Operator, path: &str, meta: Option<ObjectMetadata>) -> Self {
    Self {
        acc: op.inner(),
        path: Arc::new(normalize_path(path)),
        meta: meta.map(Arc::new),
    }
}

The Object must contain an Operator and an Arc of strings. With the introduction of Query Based Metadata, there is no longer a need to perform operations on the object.

§Complex concepts

The term Object is applied in various fields, making it difficult to provide a concise definition of opendal::Object. Moreover, this could potentially confuse our users who may assume that opendal::Object is primarily intended for object storage services.

I propose eliminating the intermediate API layer of Object and enabling users to directly utilize Operator.

§Guide-level explanation

After this RFC is implemented, our users can:

op.read("file").await?;
op.range_read("file", 0..1024).await?;
op.reader("file").await?;

op.write("file", bs).await?;
op.writer("file").await?;

op.stat("path").await?;

op.delete("path").await?;
op.remove(vec!["path_a"]).await?;
op.remove_all("path").await?;

op.create_dir("dir/").await?;

op.list("dir/").await?;
op.scan("dir/").await?;

We will include the BlockingOperator for enhanced ease of use while performing blocking operations.

let bop = op.blocking();

bop.read("file")?;
bop.write("file", bs)?;

The scan/list result will be an Entry or BlockingEntry that contains the same fields as Object, but is only used for scan/list entries.

The public API should look like:

impl Entry {
    pub fn mode(&self) -> EntryType;
    pub fn path(&self) -> &str;
    pub async fn stat(&self) -> Result<Metadata>;
    pub async fn metadata(&self, key: impl Into<MetaKey>) -> Result<Metadata>;
    ...
}

§Reference-level explanation

We will remove Object entirely and move all Object APIs to Operator instead:

- op.object("path").read().await
+ op.read("path").await

Along with this change, we should also rename the ObjectXxx structs, such as ObjectReader to Reader.

§Drawbacks

§Breaking Changes

This RFC proposes a major breaking change that will require almost all current usage to be rewritten.

§Rationale and alternatives

None

§Prior art

None

§Unresolved questions

None

§Future possibilities

None