Python Transformers By Huggingface Hands On: 101 practical implementation hands-on of ALBERT/ViT/BigBird and other latest models with huggingface transformers

Table of contents :
Table of Contents
Introduction
Latest Trend in Deep Learning
Cautions
Disclaimer
Trademarks
Feedback
Jupyter Notebook
Chapter 1 pipeline
１：Set up Google’s Colaboratory Environment
２：Sentiment Analysis
３：Question Answering
Chapter 2 Fine-tuning and Evaluation of DistilBERT using real data
Preparation: GPU preparation
４：IMDB Data Set
５：Label Encoding
６：Split training and validation data
７：Tokenize and Encoding
８：Creating your own dataset class
９：Load Pre-trained Model(DistilBertForSequenceClassification)
１０：Define TrainingArguments
１１：Transfer to GPU
１２：Fine-tuning by Trainer class
１３：Fine-Tuning by Pytorch
Chapter 3 Model Performance Evaluation
１４：Accuracy
１５：Recall/Precision/F1-Score
１６：Classification Report
Chapter4 Composition using GPT series
１７：Preparing a writing environment with GPT Neo
１８：Tokenize by GPT-Neo
１９：Composition by GPT-Neo
２０：distilgpt2 environment setting
２１：Composition by distilgpt2
２２：DialoGPT Environment Setting
２３：Composition by DialoGPT
Chapter 5 MLM(Masked Language Model)
２４：MLM pipleline loading BERT
２５：MLM pipleline loading DistilBERT
２６：MLM pipleline loading ALBERT
Chapter6 CLIP～Bridging Image Recognition and Natural Language Processing～
２７：CLIP module install
２８：Sample Image Dataset
２９：Load CLIP based pre-trained model
３０：Check the network of CLIP based pre-trainedmodel
３１：CLIP Preprocessing
３２：Check the image after preprocessing
３３：Encode and Decode
３４：inference by CLIP
３５：Get the logit of CLIP inference
３６：Display the CLIP caption prediction result
Chapter7 Wave2Vec2 Automatic Speech Recognition
３７：Wav2Vec module install
３８：Load Pre-trained Wav2Vec2
３９：Preparing a Data Set for Automatic Speech Recognition(TIMIT_ASR)
４０：Check the audio data in Colab
４１：Wav2Vec2 Pre-processing
４２：ASR by Wav2Vec2
Chapter 8 Multi-class classification in BERT
４３：Load the pre-trained BERT for Multi-class classification
４４：Pre-pare our own dataset for three-class classification of BERT
４５：BERT Classification before fine-tuning
４６：BERT fine-tuning for 3 class classification
４７：Visualizing the learning process of Fine-tuning BERT for Three-Class Classification
４８：BERT Classification after fine-tuning
４９：Classification accuracy
Chapter9 Automatic Summarization by BART
５０：Setting up the BART library and loading the pre-training model
５１：Preprocessing using regular expressions
５２：Tokenizing with the BART prior learning model
５３：Cast the BART tokenize output to numpy array
５４：BART Inference
５５：Decode the BART inference’s result
Chapter10 Ensemble learning with two BERTs
５６：Setting up the BERT ensemble learning library
５７：Preparation of dataset of your own for BERT ensemble
５８：Definition of BERT ensemble network
５９：Load the pretrained BERT for ensemble training
６０：BERT ensemble learning Data Augmentation
６１：BERT ensemble learning Defining a custom dataset
６２：BERT Ensemble Learning: DataLoader
６３：BERT ensemble fine-tuning
６４：BERT ensemble learning prediction using training data
６５：BERT ensemble learning Prediction outside of training data
Chapter11 BigBird
６６：Setting up the BigBird library and loading the pre-training model
６７：Preparation of Data for BigBird inference
６８：BigBird tokenization and encoding
６９：BigBird inference
Chapter12 PEGASUS
７０：PEGASUS library setup and pre-training model loading
７１：Tokenization and Encode
７２：PEGASUS Automatic Summarization
Chapter 13 M2M100
７３：Install the M2M100 library and load the pre-training model
７４：Preparation of M2M100 translation source (Chinese text)
７５：M2M100 Tokenize in source language
７６：M2M100 automatic translation
７７：M2M100 Decode the output of generate method
７８：M2M100 Specify source language (Japanese) and create text
７９：M2M100 Japanese text tokenization
８０：M2M100 Japanese/English translation
８１： M2M100 Japanese to English Translation Decode
Chapter14 Mobile BERT
８２：Install the MobileBERT library and load the pre-training model
Code（MOBILE BERT）
Code(BERT)
８３：Mobile BERT vs. BERT Tokenizer
８４：Last hidden layer during Mobile BERT inference
８５：Mobile BERT Fill-in-the-Blanks Quiz
Chapter15 GPT, DialoGPT, DistilGPT2
８６：Setting up the DistilGPT2 library and loading the pre-training model
８７：Visualization with distilgpt2 tool
８８：distilgpt2 text generation
８９：Loading DialoGPT (Dialogue Text Pre-Learning Model)
９０：Text Generation by DialoGPT
Chapter16 Practical exercise Moderna v.s. Pfizer (compare with BERT and tSNE)
９１：Wikipediaからキーワード検索
９２：Retrieved from Wikipedia "Moderna COVID-19 vaccine" full text
９３：Retrieved from Wikipedia, Pfizer–BioNTech COVID-19 vaccine
９４：Installing a module to handle document vectors in BERT
９５：Load the pre-trained BERT to pipeline
９６：Get document vector representations by BERT
９７：Meaning of Vector Dimensionality in BERT
９８： Definition of the function getting the document vector representation of BERT [CLS] token and Simple Preprocessing for BERT
９９：Get BERT [CLS] vectors of Moderna/Pfizer Covid-19 vaccine
１００： Frequency aggregation by tokenizer
１０１： Visualization by t-SNE "Moderna" v.s. "Pfizer".
Reference
In Closing