Video: Training Transformers with AWS Trainium and the Hugging Face Neuron AMI

In this video, I show you how to accelerate Transformer training with AWS Trainium, a new custom chip designed by AWS, and the brand new Hugging Face Neuron Amazon Machine Image (AMI).

First, I walk you through the setup of an Amazon EC2 trn1.32xlarge instance, equipped with 16 Trainium chips. Then, I run a natural language processing job, accelerating a BERT model to classify the Yelp review datatset on 32 Neuron cores.



Chief Evangelist, Hugging Face (

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store