Update Logs and Known Issues¶
Version 0.3.2¶
We improve the docs.
We support BMTrain to accelerate the training, and parallelize the training of models that are hard to fit in a single GPU. Check tutorial/2_with_bmtrain.py
We add a functionality to inspect the optimizer. The user can see the number of trainable parameters in the optimizer and verify that opendelta is being used correctly.
We move the functions to inspect the delta models into inspect.py
Version 0.3.1¶
We update must_try.py for a simple introduction of the core functionality of OpenDelta.
Thanks to Weilin Zhao We merge a long-developed branch parallel_adapter into the main branch.
Version 0.3.0¶
Updates:¶
Add this changelog for a granular record of updates.
The default configuration of delta models can be applied to more wrapped models.
There is less need to configure ‘modified_modules’ for wrapped models like BertForSequenceClassification or even OpenMatch.DRModel, as long as it has a model we support default configuration inside. Note that if you customize
modified_modules
by yourself, most pytorch models are supported.
LoRA and BitFit models now does not need pseudo data to instantiate the model.
BitFit models can now support Conv1D using default configuration.
Improve type hint for AutoDeltaModel.
Fix bugs in documentation.
Fix small bugs when saving a model without a config attributes.
Make the default modified modules of adapter-like methods more accurate: attach the adapter-like modules after the output of attention layer and second feed-forward layer, both before the layernorm layers.
A simple unit test folder containing development-time tests has been added for interested users.
Known Issues¶
SoftPrompt is still not supported for wrapped model if the model has no attribute
get_input_embeddings
.Prefix Tuning is still limited to T5, GPT2, Bart, Bert, Roberta.
Version 0.2.4¶
Updates¶
examples/examples_seq2seq and examples/examples_text-classification is depreciated and moved to legacy
Thanks to Zhen Zhang, we provide examples_prompt, as a cleaner and more general framework, which unifies the delta tuning paradigm and the prompt-tuning paradigm. It is still based on Huggingface Trainers. In this example framework, the running pipeline is a unified script, the differences in tasks, models, delta tuning models, and even prompt-tuning paradigms are more modular and be more independent . Please try it out!