OpenDelta’s documentation!

OpenDelta is a Plug-and-play Library of the parameter-efficient fine-tuning (delta-tuning) technology for pre-trained models.

Essential Advantages:

• Clean: No need to edit the backbone PTM’s codes.

• Simple: Migrating from full-model tuning to delta-tuning needs as little as 3 lines of codes.

• Sustainable: Most evolution in external library doesn’t require a new OpenDelta.

• Extendable: Various PTMs can share the same delta-tuning codes.

• Flexible: Able to apply delta-tuning to (almost) any position of the PTMs.

Getting Started

• What is Delta-tuning and Why OpenDelta?
• Installation
• Quick Start
• Custom Usage

Advanced Usage

• AutoDelta Mechanism
• DeltaCenter
• Composition of delta models
• Multitask Modeling using OpenDelta
• OpenDelta + BMTrain
• OpenDelta + Huggingface Accelerate
• Examples

Utilities

• Visualize the Parameters
• Inspect the optimizer

Mechanisms

• Philosophy and Key Features
• Name-based Addressing
• Common Structure Mapping

Information

• Citation
• Update Logs and Known Issues
• FAQs

Documentation

• Base Classes
  • BaseDeltaConfig
  • DeltaBase
• Delta Models
  • Lora
  • BitFit
  • Adapter
  • LowRankAdapter
  • Compacter
  • Prefix tuning
  • Soft Prompt Tuning
• Auto Classes
  • AutoDeltaConfig
  • AutoDeltaModel
• Utils
  • SaveLoadMixin
  • Visualization
  • Structure Map
  • Utility Functions

Indices and tables

• Index