Skip to content

Layers#

This page documents all layers in OpenDAL.

Layer#

Layers are used to intercept the operations on the underlying storage.

RetryLayer#

Bases: Layer

A layer that retries operations that fail with temporary errors.

Operations are retried if they fail with an error for which Error.is_temporary returns True. If all retries are exhausted, the error is marked as persistent and then returned.

Notes#

After an operation on a Reader or Writer has failed through all retries, the object is in an undefined state. Reusing it can lead to exceptions.

__new__(max_times=None, factor=None, jitter=False, max_delay=None, min_delay=None) #

Create a new RetryLayer.

Parameters#

max_times : Optional[int] Maximum number of retry attempts. Defaults to 3. factor : Optional[float] Backoff factor applied between retries. Defaults to 2.0. jitter : bool Whether to apply jitter to the backoff. Defaults to False. max_delay : Optional[float] Maximum delay (in seconds) between retries. Defaults to 60.0. min_delay : Optional[float] Minimum delay (in seconds) between retries. Defaults to 1.0.

Returns#

RetryLayer

ConcurrentLimitLayer#

Bases: Layer

A layer that limits the number of concurrent operations.

Notes#

All operators wrapped by this layer will share a common semaphore. This allows you to reuse the same layer across multiple operators, ensuring that the total number of concurrent requests across the entire application does not exceed the limit.

__new__(limit) #

Create a new ConcurrentLimitLayer.

Parameters#

limit : int Maximum number of concurrent operations allowed.

Returns#

ConcurrentLimitLayer

MimeGuessLayer#

Bases: Layer

A layer that guesses MIME types for objects based on their paths or content.

This layer uses the mime_guess crate (see https://crates.io/crates/mime_guess) to infer the Content-Type.

Notes#

This layer will not override a Content-Type that has already been set, either manually or by the backend service. It is only applied if no content type is present.

A Content-Type is not guaranteed. If the file extension is uncommon or unknown, the content type will remain unset.

__new__() #

Create a new MimeGuessLayer.

Returns#

MimeGuessLayer