Skip to main content

HuggingFace Passport Sync

Automatically import models from HuggingFace Hub as Lucid passports. The sync pulls model metadata (downloads, likes, pipeline type, library) and creates a passport with full provenance tracking.

Sync a Single Model

const passport = await lucid.passports.sync({
  source: "huggingface",
  modelId: "meta-llama/Llama-3-70b",
  tags: ["llama", "open-source", "text-generation"],
});

console.log("Passport:", passport.id);
console.log("Pipeline:", passport.metadata.pipelineTag);    // text-generation
console.log("Downloads:", passport.metadata.downloads);      // 1234567
console.log("Library:", passport.metadata.libraryName);      // transformers

Bulk Sync

Import multiple models at once:
const models = [
  "meta-llama/Llama-3-70b",
  "mistralai/Mistral-7B-v0.1",
  "google/gemma-2-9b",
  "deepseek-ai/DeepSeek-Coder-V2",
  "Qwen/Qwen2-72B",
];

const passports = await Promise.all(
  models.map(modelId =>
    lucid.passports.sync({ source: "huggingface", modelId })
  )
);

console.log(`Synced ${passports.length} models`);

Metadata Mapping

HuggingFace metadata is mapped to passport fields:
HuggingFace FieldPassport FieldExample
modelIdnamemeta-llama/Llama-3-70b
pipeline_tagmetadata.pipelineTagtext-generation
library_namemetadata.libraryNametransformers
downloadsmetadata.downloads1234567
likesmetadata.likes4521
tagsmetadata.tags["llama", "pytorch"]
lastModifiedmetadata.lastModified2025-06-15T...

Auto-Refresh

Set up periodic syncs to keep passport metadata current:
// Re-sync a passport to update metadata (downloads, likes, etc.)
await lucid.passports.resync(passport.id);
For automated updates, use a webhook or n8n workflow on a daily schedule.

Authentication

For gated models (e.g., Llama 3), provide your HuggingFace token:
const passport = await lucid.passports.sync({
  source: "huggingface",
  modelId: "meta-llama/Llama-3-70b",
  credentials: { hfToken: process.env.HF_TOKEN },
});
Ensure you’ve accepted the model’s license on HuggingFace before syncing.