SLM in Action: Arcee Lite, a powerful 1.5B distilled model

Julien Simon
Aug 20, 2024

In this video, you will learn about Arcee-Lite, a small yet powerful 1.5B model created with Distilkit, an open-source project for model distillation. Arcee-Lite outperforms Qwen2 1.5B and is currently the best 1.5B model.

First, I run an 8-bit version on my M3 MacBook with ollama and OpenWeb UI. Then, I deploy the model on AWS with Amazon SageMaker. I run both synchronous and streaming inference. I also show you how to use the OpenAI Messages API, allowing you to invoke the model with the OpenAI prompting format.

Sign up to discover human stories that deepen your understanding of the world.

Free

Distraction-free reading. No ads.

Organize your knowledge with lists and highlights.

Tell your story. Find your audience.

Membership

Read member-only stories

Support writers you read most

Earn money for your writing

Listen to audio narrations

Read offline with the Medium app

Julien Simon
Julien Simon

No responses yet