Layers#
This page documents all layers in OpenDAL.
Layer #
Base class for all layers.
A layer wraps an operator to intercept operations on the underlying storage, adding behavior such as retries, timeouts, concurrency limits, or content-type detection.
This class is not meant to be instantiated directly. Use a concrete
subclass such as RetryLayer or TimeoutLayer and apply it with
Operator.layer or AsyncOperator.layer.
RetryLayer #
Bases: Layer
A layer that retries operations that fail with temporary errors.
Operations are retried if they fail with an error for which
Error.is_temporary returns True. If all retries are exhausted,
the error is marked as persistent and then returned.
Notes
After an operation on a Reader or Writer has failed through
all retries, the object is in an undefined state. Reusing it
can lead to exceptions.
__new__(max_times=None, factor=None, jitter=False, max_delay=None, min_delay=None)
#
Create a new RetryLayer.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
max_times
|
Optional[int]
|
Maximum number of retry attempts. Defaults to |
None
|
factor
|
Optional[float]
|
Backoff factor applied between retries. Must be a finite value
|
None
|
jitter
|
bool
|
Whether to apply jitter to the backoff. Defaults to |
False
|
max_delay
|
Optional[float]
|
Maximum delay (in seconds) between retries. Must be finite and
non-negative. Defaults to |
None
|
min_delay
|
Optional[float]
|
Minimum delay (in seconds) between retries. Must be finite and
non-negative. Defaults to |
None
|
Returns:
| Type | Description |
|---|---|
RetryLayer
|
|
Raises:
| Type | Description |
|---|---|
ConfigInvalid
|
If |
ConcurrentLimitLayer #
Bases: Layer
A layer that limits the number of concurrent operations.
Notes
All operators wrapped by this layer will share a common semaphore. This allows you to reuse the same layer across multiple operators, ensuring that the total number of concurrent requests across the entire application does not exceed the limit.
__new__(limit)
#
Create a new ConcurrentLimitLayer.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
limit
|
int
|
Maximum number of concurrent operations allowed. |
required |
Returns:
| Type | Description |
|---|---|
ConcurrentLimitLayer
|
|
MimeGuessLayer #
Bases: Layer
A layer that guesses MIME types for objects based on their paths or content.
This layer uses the mime_guess crate
(see https://crates.io/crates/mime_guess) to infer the
Content-Type.
Notes
This layer will not override a Content-Type that has already
been set, either manually or by the backend service. It is only
applied if no content type is present.
A Content-Type is not guaranteed. If the file extension is
uncommon or unknown, the content type will remain unset.
__new__()
#
TimeoutLayer #
Bases: Layer
A layer that adds timeouts to operations.
Timeouts prevent slow or stalled work from hanging indefinitely, for
example when a TCP connection stops emitting IO events. Control
operations (such as stat and delete) and IO operations (such as
read and write) are bounded by separate timeouts.
Notes
A small amount of overhead is added to IO operations to implement the timeout correctly.
__new__(timeout=None, io_timeout=None)
#
Create a new TimeoutLayer.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
timeout
|
Optional[float]
|
Timeout (in seconds) for control operations like |
None
|
io_timeout
|
Optional[float]
|
Timeout (in seconds) for IO operations like |
None
|
Returns:
| Type | Description |
|---|---|
TimeoutLayer
|
|
Raises:
| Type | Description |
|---|---|
ConfigInvalid
|
If |