Most of this release is about training. It adds a pair-similarity trainer, lets fit take Hugging Face datasets columns directly, and adds focal loss, token dropout and faster training. It also adds a public ONNX export API that can export classifier and regressor pipelines, and fixes a number of tokenization and padding bugs.
StaticModel.tokenize no longer takesmax_length . Truncation now follows the model’smax_length property, which has a new setter (#356, #363).encode_as_sequence ignores the model’smax_length . It only truncates if you passmax_length to the call yourself (#363).scripts/export_to_onnx.py is gone, so usemodel2vec.onnx.export_model_to_onnx(model, save_path, ...) instead (#347).pad_token argument offrom_pretrained andfrom_static_model and thepad_id constructor argument have been removed (#390).model.w directly will see different values.freeze_weights now defaults toNone . WithNone , a model that has token weights trains them, and a model without token weights takes the unweighted mean of its token embeddings. Previously, a model without weights always learned new ones. To keep that behavior, passfreeze_weights=False (#390).huggingface-hub>=1.0.0 is now a core dependency (#345).distill extra no longer needsscikit-learn , because PCA is now implemented in NumPy (#344). It now requiresskeletoken>=0.6.0,<0.7.0 .train extra now requiresdatasets>=4.0.0 (#387).onnx extra now includesonnxruntime ,onnxscript andskeletoken (#347).StaticModelForPairSimilarity is a new trainer that embeds related pairs of texts close together, such as queries and their documents (#372). It’s trained with an InfoNCE loss over in-batch negatives, and itstemperature argument defaults to0.05 (#381). Withn_layers=0 , pair and similarity models have no head, soto_static_model returns exactly the trained embeddings (#383).fit accepts dataset columns such asdataset["text"] directly. Data is read and tokenized per batch, so the dataset isn’t loaded into memory. Without an explicit validation set, the held-out split is capped at 10,000 rows, and single-label classification splits are stratified (#387).StaticModelForClassification.fit(..., focal_gamma=...) down-weights texts the model already classifies confidently. It supports single-label and multi-label classification and can be combined withclass_weight . The default,0.0 , is plain (binary) cross-entropy (#391).fit(..., token_dropout=...) randomly drops a fraction of the tokens of each training text. A non-empty text always keeps at least one token. It’s available for all trainers, and for pair similarity it applies to both texts of a pair (#373, #393).from_pretrained andfrom_static_model on every trainable class now list and document their arguments (max_length ,n_layers ,hidden_dim ,out_dim ,freeze ,normalize ,freeze_weights ). You no longer have to pass them blindly through**kwargs (#388, #389).source_model_name records which model was exported (#349).StaticModel has amax_length property with a setter. ONNX export respects it too (#356).StaticModel tokenizes (#359).max_length no longer depends on the static model’s tokenizer truncation (#374).random_seed now also controls the train/validation split. Before, the split always used the default seed (#396).subfolder is now respected when downloading from the Hub, and a cached model is only reused if it actually contains a valid model (#392).token argument is now passed through when a pipeline loads its encoder (#362).T (#357), and some encoding code was refactored to remove duplication (#365).