Struct opendal::services::Huggingface

pub struct Huggingface { /* private fields */ }

Available on crate feature services-huggingface only.

Huggingface’s API support. This service will visit the Huggingface API to access the Huggingface File System. Currently, we only support the model and dataset types of repositories, and operations are limited to reading and listing/stating.

Huggingface doesn’t host official HTTP API docs. Detailed HTTP request API information can be found on the huggingface_hub Source Code.

Capabilities

This service can be used to:

• stat
• read
• write
• create_dir
• delete
• copy
• rename
• list
• scan
• presign
• blocking

Configurations

• repo_type: The type of the repository.
• repo_id: The id of the repository.
• revision: The revision of the repository.
• root: Set the work directory for backend.
• token: The token for accessing the repository.

Refer to HuggingfaceBuilder’s public API docs for more information.

Examples

Via Builder

use std::sync::Arc;

use anyhow::Result;
use opendal::services::Huggingface;
use opendal::Operator;

#[tokio::main]
async fn main() -> Result<()> {
    // Create Huggingface backend builder
    let mut builder = Huggingface::default();
    
    // set the type of Huggingface repository
    builder.repo_type("dataset");
    // set the id of Huggingface repository
    builder.repo_id("databricks/databricks-dolly-15k");
    // set the revision of Huggingface repository
    builder.revision("main");
    // set the root for Huggingface, all operations will happen under this root
    builder.root("/path/to/dir");
    // set the token for accessing the repository
    builder.token("access_token");

    let op: Operator = Operator::new(builder)?.finish();

    Ok(())
}

Implementations

impl HuggingfaceBuilder

pub fn repo_type(&mut self, repo_type: &str) -> &mut Self

Set repo type of this backend. Default is model.

Available values:

• model
• dataset

Currently, only models and datasets are supported. Reference

pub fn repo_id(&mut self, repo_id: &str) -> &mut Self

Set repo id of this backend. This is required.

Repo id consists of the account name and the repository name.

For example, model’s repo id looks like:

• meta-llama/Llama-2-7b

Dataset’s repo id looks like:

• databricks/databricks-dolly-15k

pub fn revision(&mut self, revision: &str) -> &mut Self

Set revision of this backend. Default is main.

Revision can be a branch name or a commit hash.

For example, revision can be:

• main
• 1d0c4eb

pub fn root(&mut self, root: &str) -> &mut Self

Set root of this backend.

All operations will happen under this root.

pub fn token(&mut self, token: &str) -> &mut Self

Set the token of this backend.

This is optional.

Trait Implementations

impl Builder for HuggingfaceBuilder

fn build(&mut self) -> Result<Self::Accessor>

Build a HuggingfaceBackend.

const SCHEME: Scheme = Scheme::Huggingface

Associated scheme for this builder. It indicates what underlying service is.

type Accessor = HuggingfaceBackend

The accessor that built by this builder.

fn from_map(map: HashMap<String, String>) -> Self

Construct a builder from given map which contains several parameters needed by underlying service.

impl Clone for HuggingfaceBuilder

fn clone(&self) -> HuggingfaceBuilder

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for HuggingfaceBuilder

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for HuggingfaceBuilder

fn default() -> HuggingfaceBuilder

Returns the “default value” for a type.