Mastering NLP from Foundations to LLMs 9781804619186

Table of contents :
Mastering NLP from Foundations to LLMs
Foreword
Contributors
About the authors
About the reviewers
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example code files
Conventions used
Get in touch
Reviews
Share Your Thoughts
Download a free PDF copy of this book
1
Navigating the NLP Landscape: A Comprehensive Introduction
Who this book is for
What is natural language processing?
The history and evolution of natural language processing
Initial strategies in the machine processing of natural language
A winning synergy – the coming together of NLP and ML
Introduction to math and statistics in NLP
Understanding language models – ChatGPT example
Summary
Questions and answers
2
Mastering Linear Algebra, Probability, and Statistics for Machine Learning and NLP
Introduction to linear algebra
Basic operations on matrices and vectors
Matrix definitions
Eigenvalues and eigenvectors
Numerical methods for finding eigenvectors
Eigenvalue decomposition
Singular value decomposition
Basic probability for machine learning
Statistically independent
Discrete random variables and their distribution
Probability density function
Bayesian estimation
Summary
Further reading
References
3
Unleashing Machine Learning Potentials in Natural Language Processing
Technical requirements
Data exploration
Data visualization
Data cleaning
Feature selection
Feature engineering
Common machine learning models
Linear regression
Logistic regression
Decision trees
Random forest
Support vector machines (SVMs)
Neural networks and transformers
Model underfitting and overfitting
Splitting data
Hyperparameter tuning
Ensemble models
Bagging
Boosting
Stacking
Random forests
Gradient boosting
Handling imbalanced data
SMOTE
The NearMiss algorithm
Cost-sensitive learning
Data augmentation
Dealing with correlated data
Summary
References
4
Streamlining Text Preprocessing Techniques for Optimal NLP Performance
Technical requirements
Lowercasing in NLP
Removing special characters and punctuation
Stop word removal
NER
POS tagging
Rule-based methods
Statistical methods
Deep learning-based methods
Regular expressions
Tokenization
Explaining the preprocessing pipeline
Code for NER and POS
Summary
5
Empowering Text Classification: Leveraging Traditional Machine Learning Techniques
Technical requirements
Types of text classification
Supervised learning
Unsupervised learning
Semi-supervised learning
Sentence classification using one-hot encoding vector representation
Text classification using TF-IDF
Text classification using Word2Vec
Word2Vec
Model evaluation
Overfitting and underfitting
Hyperparameter tuning
Additional topics in applied text classification
Topic modeling – a particular use case of unsupervised text classification
LDA
Real-world ML system design for NLP text classification
Implementing an ML solution
Reviewing our use case – ML system design for NLP classification in a Jupyter Notebook
The pipeline
Code settings
Feature selection
Generating the chosen model
Summary
6
Text Classification Reimagined: Delving Deep into Deep Learning Language Models
Technical requirements
Understanding deep learning basics
What is a neural network?
The basic design of a neural network
Neural network common terms
The architecture of different neural networks
The challenges of training neural networks
Language models
Semi-supervised learning
Unsupervised learning
Transfer learning
Understanding transformers
Architecture of transformers
Applications of transformers
Learning more about large language models
The challenges of training language models
Specific designs of language models
Challenges of using GPT-3
Reviewing our use case – ML/DL system design for NLP classification in a Jupyter Notebook
The business objective
The technical objective
The pipeline
Summary
7
Demystifying Large Language Models: Theory, Design, and Langchain Implementation
Technical requirements
What are LLMs and how are they different from LMs?
n-gram models
Hidden Markov models (HMMs)
Recurrent neural networks (RNNs)
How LLMs stand out
Motivations for developing and using LLMs
Improved performance
Broad generalization
Few-shot learning
Understanding complex contexts
Multilingual capabilities
Human-like text generation
Challenges in developing LLMs
Amounts of data
Computational resources
Risk of bias
Model robustness
Interpretability and debugging
Environmental impact
Different types of LLMs
Transformer models
Example designs of state-of-the-art LLMs
GPT-3.5 and ChatGPT
LM pretraining
Training the reward model
How to fine-tune the model using reinforcement learning
GPT-4
LLaMA
PaLM
Open-source tools for RLHF
Summary
References
8
Accessing the Power of Large Language Models: Advanced Setup and Integration with RAG
Technical requirements
Setting up an LLM application – API-based closed source models
Choosing a remote LLM provider
Prompt engineering and priming GPT
Experimenting with OpenAI’s GPT model
Setting up an LLM application – local open source models
About the different aspects that distinguish between open source and closed source
Hugging Face’s hub of models
Employing LLMs from Hugging Face via Python
Exploring advanced system design – RAG and LangChain
LangChain’s design concepts
Data sources
Data that is not pre-embedded
Chains
Agents
Long-term memory and referring to prior conversations
Ensuring continuous relevance through incremental updates and automated monitoring
Reviewing a simple LangChain setup in a Jupyter notebook
Setting up a LangChain pipeline with Python
LLMs in the cloud
AWS
Microsoft Azure
GCP
Concluding cloud services
Summary
9
Exploring the Frontiers: Advanced Applications and Innovations Driven by LLMs
Technical requirements
Enhancing LLM performance with RAG and LangChain – a dive into advanced functionalities
LangChain pipeline with Python – enhancing performance with LLMs
Advanced methods with chains
Asking the LLM a general knowledge question
Requesting output structure – making the LLM provide output in a particular data format
Evolving to a fluent conversation – inserting an element of memory to have previous interactions as reference and context for follow-up prompts
Retrieving information from various web sources automatically
Retrieving content from a YouTube video and summarizing it
Prompt compression and API cost reduction
Prompt compression
Experimenting with prompt compression and evaluating trade-offs
Multiple agents – forming a team of LLMs that collaborate
Potential advantages of multiple LLM agents working simultaneously
Concluding thoughts on the multiple-agent team
Summary
10
Riding the Wave: Analyzing Past, Present, and Future Trends Shaped by LLMs and AI
Key technical trends around LLMs and AI
Computation power – the engine behind LLMs
The future of computational power in NLP
Large datasets and their indelible mark on NLP and LLMs
Purpose – training, benchmarking, and domain expertise
Value – robustness, diversity, and efficiency
Impact – democratization, proficiency, and new concerns
Evolution of large language models – purpose, value, and impact
Purpose – why the push for bigger and better LLMs?
Value – the LLM advantage
Impact – changing the landscape
NLP and LLMs in the business world
Business sectors
Customer interactions and service – the early adopter
Change management driven by AI’s impact
Behavioral trends induced by AI and LLMs – the social aspect
Personal assistants becoming indispensable
Ease in communication and bridging language barriers
Ethical implications of delegated decisions
Ethics and risks – growing concerns around the implementation of AI
Summary
11
Exclusive Industry Insights: Perspectives and Predictions from World Class Experts
Overview of our experts
Nitzan Mekel-Bobrov, PhD
David Sontag, PhD
John D. Halamka, M.D., M.S.
Xavier Amatriain, PhD
Melanie Garson, PhD
Our questions and the experts’ answers
Nitzan Mekel-Bobrov
Q1.1 – Future of LLM – hybrid learning paradigms: In light of the evolving landscape of learning schemes, what do you envision as the next breakthrough in combining different learning paradigms within LLMs?
Q2.1 – As the Chief AI Officer becomes more integral to the corporate hierarchy, what unique challenges do you foresee in bridging the gap between AI potential and practical business applications, and how should the CAIO’s role evolve to meet these challenges?
Q3 – How do foundation models and the strategies of major tech companies toward open sourcing affect data ownership and its value for businesses?
David Sontag
Q1 – As we progress toward creating more equitable and unbiased datasets, what strategies do you believe are most effective in identifying and mitigating implicit biases within large datasets?
Q2 – How do you see these strategies evolving with the advancement of NLP technologies, and what do you envision as the next breakthrough in combining different learning paradigms within LLMs?
Q3 – We are witnessing an ongoing evolution of regulations around AI from the aspects of training data and model usage. What are the implications for the future development of LLMs in this regulated landscape?
John D. Halamka
Q2 – AI-driven organizational structure – in what ways do you predict AI will continue to reshape companies’ organizational structures to maximize the benefits of AI?
Q3 – Ethical concerns and strategies to combat over-delegation – as AI continues to penetrate daily decision-making processes, what strategies do you recommend to prevent overreliance on AI systems and to maintain a healthy level of human critical thinking and autonomy?
Xavier Amatriain
Q1.1 – The future of LLM – hybrid learning paradigms: In light of the evolving landscape of learning schemes, what do you envision as the next breakthrough in combining different learning paradigms within LLMs?
Q2 – AI-driven organizational structure – in what ways do you predict AI will continue to reshape internal business operations, and how should companies prepare to adapt their organizational structures to maximize the benefits of AI, especially in decision-making and operational efficiency?
Melanie Garson
Q1 – As this book is designed to address technical practitioners in the world of ML and AI, what value would they find in being aware of the various legal and regulatory aspects?
Q2 – From the perspective of a legal expert, how can we categorize the diverse array of risks associated with the burgeoning advancements in AI technology?
Q3 – How can the development and deployment of AI and LLMs be guided to mitigate ethical concerns such as bias and ensure their responsible use in decision-making processes, particularly in high-risk and regulated industries?
Q4 – What strategies can be implemented to transition from traditional roles to collaborative human-AI teams, ensuring the development of human expertise alongside AI integration in the workplace?
Summary
Index
Why subscribe?
Other Books You May Enjoy
Packt is searching for authors like you
Share Your Thoughts
Download a free PDF copy of this book