Apache OpenDAL™ Node.js Binding

Note: This binding has its own independent version number, which may differ from the Rust core version. When checking for updates or compatibility, always refer to this binding's version rather than the core version.

Useful Links

• Documentation
• Upgrade Guide

Installation

npm install opendal
Copy

Docs

To build the docs locally, please run the following commands:

# Only need to run once unless you want to update the docs theme
pnpm run build:theme

# Build the docs
pnpm run docs
Copy

Tests

Services behavior tests read necessary configs from env vars or the .env file.

You can copy .env.example to $(pwd)/.env and change the values on need, or directly set env vars with export KEY=VALUE.

Take fs for example, we need to enable bench on fs on /tmp:

OPENDAL_TEST=fs
OPENDAL_FS_ROOT=/tmp
Copy

You can run service behavior tests of enabled with the following command:

pnpm build && pnpm test
Copy

Usage

import { Operator } from "opendal";

async function main() {
  const op = new Operator("fs", { root: "/tmp" });
  await op.write("test", "Hello, World!");
  const bs = await op.read("test");
  console.log(new TextDecoder().decode(bs));
  const meta = await op.stat("test");
  console.log(`contentLength: ${meta.contentLength}`);
}

main();
Copy

Layers

OpenDAL provides layers to add additional functionality to operators. You can chain multiple layers together to build powerful data access pipelines.

Available Layers

• RetryLayer - Retry failed operations automatically
• ConcurrentLimitLayer - Limit concurrent requests
• TimeoutLayer - Add timeout for operations
• LoggingLayer - Log all operations
• ThrottleLayer - Limit bandwidth usage

TimeoutLayer

Prevents operations from hanging indefinitely:

import { Operator, TimeoutLayer } from "opendal";

const op = new Operator("fs", { root: "/tmp" });

const timeout = new TimeoutLayer();
timeout.timeout = 10000; // 10 seconds for non-IO operations (ms)
timeout.ioTimeout = 3000; // 3 seconds for IO operations (ms)
op.layer(timeout.build());
Copy

Default values: timeout: 60s, ioTimeout: 10s

LoggingLayer

Add structured logging for debugging and monitoring:

import { Operator, LoggingLayer } from "opendal";

const op = new Operator("fs", { root: "/tmp" });

const logging = new LoggingLayer();
op.layer(logging.build());

// All operations will be logged
await op.write("test.txt", "Hello World");
Copy

Enable logging output:

# Show debug logs
RUST_LOG=debug node app.js

# Show only OpenDAL logs
RUST_LOG=opendal::services=debug node app.js
Copy

ThrottleLayer

Control bandwidth usage with rate limiting:

import { Operator, ThrottleLayer } from "opendal";

const op = new Operator("s3", {
  bucket: "my-bucket",
  region: "us-east-1",
});

// Limit to 10 KiB/s with 10 MiB burst
const throttle = new ThrottleLayer(10 * 1024, 10 * 1024 * 1024);
op.layer(throttle.build());
Copy

Parameters:

• bandwidth: Maximum bytes per second
• burst: Maximum burst size (must be larger than any operation size)

RetryLayer

Automatically retry temporary failed operations:

import { Operator, RetryLayer } from "opendal";

const op = new Operator("s3", {
  bucket: "my-bucket",
  region: "us-east-1",
});

const retry = new RetryLayer();
retry.maxTimes = 3;
retry.jitter = true;
op.layer(retry.build());
Copy

ConcurrentLimitLayer

Limit concurrent requests to storage services:

import { Operator, ConcurrentLimitLayer } from "opendal";

const op = new Operator("s3", {
  bucket: "my-bucket",
  region: "us-east-1",
});

// Allow max 1024 concurrent operations
const limit = new ConcurrentLimitLayer(1024);
limit.httpPermits = 512; // Limit HTTP requests separately
op.layer(limit.build());
Copy

Combining Multiple Layers

Stack multiple layers for comprehensive control:

import {
  Operator,
  LoggingLayer,
  TimeoutLayer,
  RetryLayer,
  ThrottleLayer,
} from "opendal";

const op = new Operator("s3", {
  bucket: "my-bucket",
  region: "us-east-1",
});

// Layer 1: Logging for observability
const logging = new LoggingLayer();
op.layer(logging.build());

// Layer 2: Timeout protection
const timeout = new TimeoutLayer();
timeout.timeout = 30000;
timeout.ioTimeout = 10000;
op.layer(timeout.build());

// Layer 3: Retry on failures
const retry = new RetryLayer();
retry.maxTimes = 3;
retry.jitter = true;
op.layer(retry.build());

// Layer 4: Bandwidth throttling
const throttle = new ThrottleLayer(100 * 1024, 10 * 1024 * 1024);
op.layer(throttle.build());

// Now the operator has full production-ready protection
await op.write("data.json", JSON.stringify(data));
Copy

Usage with Next.js

Config automatically be bundled by Next.js.

/** @type {import('next').NextConfig} */
const nextConfig = {
  serverExternalPackages: ["opendal"],
};

module.exports = nextConfig;
Copy

Contributing

• Start with Contributing Guide.
• Submit Issues for bug report or feature requests.
• Asking questions in the Discussions.
  • Talk to community at Discord.

License and Trademarks

Licensed under the Apache License, Version 2.0: http://www.apache.org/licenses/LICENSE-2.0

Apache OpenDAL, OpenDAL, and Apache are either registered trademarks or trademarks of the Apache Software Foundation.