Video — Deep dive: model merging

Julien Simon
Mar 18, 2024

Model merging is an increasingly popular technique that makes it possible to add or remove capabilities to transformer models, without the need for any additional training.

In this video, we first introduce what model merging is. Then, we discuss different merging algorithms implemented in the mergekit library: model soups, SLERP, Task Arithmetic, TIES, DARE, and Franken-merging.

#opensource #ai