Python for Everyone: Learn and Polish Your Coding Skills in Python (English Edition) 9789355518156

Citation preview



Python for Everyone Learn and polish your coding skills in Python

Saurabh Chandrakar Dr. Nilesh Bhaskarrao Bahadure

www.bpbonline.com

i

ii



Copyright © 2023 BPB Online All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor BPB Online or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book. BPB Online has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, BPB Online cannot guarantee the accuracy of this information.

First published: 2023 Published by BPB Online WeWork 119 Marylebone Road London NW1 5PU UK | UAE | INDIA | SINGAPORE ISBN 978-93-55518-170

www.bpbonline.com



Dedicated to Dedicated to my Parents

Dr Surendra Kumar Chandrakar and Smt. Bhuneshwari Chandrakar brother

Shri Pranav Chandrakar to my wife

Priyanka Chandrakar and to my lovely son

Yathartha Chandrakar

Saurabh Chandrakar

Dedicated to my Parents

Smt. Kamal B. Bahadure and Late Bhaskarrao M. Bahadure to my in-laws

Smt. Saroj R. Lokhande and Shri. Ravikant A. Lokhande and to

my wife Shilpa N. Bahadure and to beautiful daughters Nishita and Mrunmayee

And to all my beloved students.

Dr. Nilesh Bhaskarrao Bahadure

iii

iv



About the Authors • Saurabh Chandrakar is a Research & Development Engineer (Dy. Manager) at Bharat Heavy Electricals Limited (BHEL) Hyderabad. He is the winner of the best executive award on Operations Division by BHEL Hyderabad. Recently, he has been awarded the prestigious BHEL Excellence Award under Anusandhan category for Redundant Composite Monitoring System of Power Transformers project. He has 20 copyrights and 1 patent granted. Additionally, he has 6 patents filed. Moreover, he has published 3 books in reputed publications such as BPB New Delhi (Programming Techniques using Python), Scitech Publications Chennai (Programming Techniques using matlab) and IK International publishers (Microcontrollers and Embedded System Design). He has also launched 1 video course on BPB titled “First Time Play with Basic, Advanced Python concepts and complete guide for different python certification exams all in one umbrella.” • Dr. Nilesh Bhaskarrao Bahadure received the Bachelor of Engineering degree in Electronics Engineering in 2000, the Master of Engineering degree in Digital Electronics in 2005, and the Ph.D. degree in Electronics in 2017 from KIIT Deemed to be University, Bhubaneswar, India. He is currently an Associate Professor in the Department of Computer Science and Engineering at Symbiosis Institute of Technology (SIT), Nagpur, Symbiosis International (Deemed University) (SIU), Pune, Maharashtra, India. He has more than 20 years of experience. Dr. Bahadure is a life member of IE(I), IETE, ISTE, ISCA, SESI, ISRS, and IAENG professional organizations. He has published more than 40 articles in reputed international journals and conferences, and has 5 books to his credit. He is the reviewer of many indexed journals such as IEEE Access, IET, Springer, Elsevier, Wiley and so on. His research interests are in the areas of Sensor Technology, the Internet of Things, and Biomedical Image Processing



v

Acknowledgements First and foremost, I would like to thank you all for selecting this book. It has been written with the beginner reader in mind. First of all, I take this opportunity to greet and thank my mentor"Prof. Nilesh Bahadure Sir" for motivating me and always communicating his expertise fully on topics related to Python. I am very thankful of being his protégé. I appreciate his belief in mee, for always standing behind me, and pushing me to achieve more. The phrase "Journey of Thousand Miles Begins with a Single Step" is something he always reminds me of. My parents, Dr. Surendra Kumar Chandrakar and Smt. Bhuneshwari Chandrakar, my brother, Shri Pranav Chandrakar, my beloved wife, Mrs. Priyanka Chandrakar, my adorable son, Yathartha Chandrakar, and all of my friends have inspired me and given me confidence over the years. Last but not least, I would like to express my sincere gratitude to the staff at "BPB Publications Private Limited" for their contributions and insights that made parts of this book possible. — Saurabh Chandrakar It was my privilege to thanks Dr. S. B. Mujumdar, Chancellor of the Symbiosis International University, Pune and Shri. Vijay Kumar Gupta, Chairman of Beekay Industries Bhilai and BIT Trust, for his encouragement and support. I would like to thank my mentors Dr. Arun Kumar Ray, Dean, School of Electronics Engineering, KIIT Deemed to be University, Bhubaneswar and Dr. Anupam Shukla, Director, SVNIT Surat. I would like to thank Dr. Vidya Yeravdekar, Principal Director of Symbiosis Society, and the Pro Chancellor of Symbiosis International University, Pune, Dr. Rajani R.Gupte, Vice Chancellor of the Symbiosis International University, Pune, and Dr.Ketan Kotecha, Dean, Faculty of Engineering, Symbiosis International University, Pune, and Dr. Mukesh M. Raguwanshi, Director, SIT Nagpur, for their advice, and encouragement throughout the preparation of the book. I would also like to thank Dr. Sanjeev Khandal, HOD, Department of Aeronautical Engineering, SGU Kolhapur, my well-wisher Dr. Prasenjeet D. Patil, Associate Professor, MIT ADT University, Pune and my colleagues in Symbiosis Institute of Technology Nagpur for providing valuable suggestions and lots of encouragement throughout the project.

vi



I am thankful to Prof. Dr. N. Raju, Sr. Assistant Professor, SASTRA University, Thanjavur, Tamil Nadu, for his support, assistance during writing, and for his valuable suggestions. I would also like to thank Dr. Ravi M. Potdar, Sr. Associate Professor, BIT Durg, and Dr. Md. Khwaja Mohiddin, Associate Professor, BIT Raipur for providing valuable suggestions and lots of encouragement throughout the project. Writing a beautiful, well balanced and acquainted book is not a work of one or two days or a month; it takes a lot of time and patience, as well as hours of hard work. Many thanks to my family members, my parents, wife, children and my well-wishers for their kind support. Without them and their faith and support, writing this classic book, would have remained just a dream. I also like to thank my students, who have always been with me, for relating problems and to finding solutions too. The perfection in any work does not come in a day. It needs a lot of effort, time and hard work, and sometimes, proper guidance. It is my privilege to thank Prof. (Dr.) Ram Dhekekar, Professor, Department of Electronics & Telecommunication Engineering, SSGMCE Shegaon and Dr. C. G. Dethe, Director UGC Staff College Nagpur. Last but not the least, I would like to offer an extra special thanks to the staff at “BPB Publications Private Limited" for their insight and contribution in polishing this book. Most significantly, I want to thank Lord Ganesha for all of the work I was able to put into the book's preparation. I would not be as zealous as I am now if it weren't for God's amazing creation of the universes. For since the creation of the world God’s invisible qualities - his eternal power and divine nature - have been clearly seen, being understood from what has been made, so that men are without excuse." — Dr. Nilesh Bhaskarrao Bahadure



vii

Preface The purpose of this book is to introduce readers with little to no programming experience to Python and provide them with the foundational knowledge and skills necessary to begin writing code in the language. By mastering Python, readers will be able to apply this technology to solve real-world problems and create useful applications. The first part of the book covers core python concepts. Then we shall see some advanced Python concepts along with some data science libraries like numpy, matplotlib and pandas. Finally, in the later part of the book, we shall see some powerful observations while writing python code. This book covers a wide range of topics, from basic syntax and data types to more advanced concepts along with a little touch of data science. Overall, the book provides a solid foundation for beginners to start their journey for getting trained in Python language. This book is divided into 11 chapters. Each chapter's description is listed as follows. Chapter 1: Basic Python Introduction – will cover the basic elements of writing a Python program where readers can gain an understanding of the fundamental topics. Chapter 2: Concept of Strings in Python – will cover strings in detail, as well as different string methods, usage of command-line arguments, and string access with some examples. Chapter 3: Concept of Flow Control Statements in Python – will cover the concept of conditional, iterative, and transfer statements, with loop patterns such as star pattern, alphabet pattern, and number pattern, with examples. Chapter 4: Concept of Exception Handling in Python – will cover the concept of multiple ways of using the try-except block, for preventing errors during runtime. Users will be able to create their exceptions for handling the errors. Moreover, the user will see different use cases and control flow of Python exception hierarchy. Chapter 5: Concept of Regular Expressions in Python – will cover the declarative mechanism for the representation of a group of strings, according to a particular pattern called regular expressions, which is quite difficult to understand, and also

viii



perform its usage. Each regex expression is explained in a very lucid manner with examples. Chapter 6: Concept of Functions in Python – will cover function types, and different types of function arguments such as positional arguments, keyword arguments, default arguments, variable length arguments, and kwargs. Moreover, concepts on local, global and non-local variable are well explained with examples, along with Python closures. Chapter 7: Concept of Data Structures in Python – will cover non-primitive inbuilt data structures such as list, tuple, set and dictionary, which are unique on their own. All the data structures definitions, methods, and different types of comprehensions such as list comprehension, tuple comprehension, set comprehension, and dictionary comprehension are well discussed. Chapter 8: Concept of Packages in Python – will cover the demonstration of package examples, with different approaches to module usage. Chapter 9: Numpy Introduction – will cover different examples of scientific computing and data analysis library, that is, numpy library. Chapter 10: Data Visualization Introduction – will cover Matplotlib data visualization library by creating a line plot with examples. Chapter 11: Pandas Introduction – will cover Pandas Series and Pandas DataFrame with examples.



ix

Code Bundle and Coloured Images Please follow the link to download the Code Bundle and the Coloured Images of the book:

https://rebrand.ly/33uiwxo The code bundle for the book is also hosted on GitHub at https://github.com/bpbpublications/Python-for-Everyone. In case there’s an update to the code, it will be updated on the existing GitHub repository. We have code bundles from our rich catalogue of books and videos available at https://github.com/bpbpublications. Check them out!

Errata We take immense pride in our work at BPB Publications and follow best practices to ensure the accuracy of our content to provide with an indulging reading experience to our subscribers. Our readers are our mirrors, and we use their inputs to reflect and improve upon human errors, if any, that may have occurred during the publishing processes involved. To let us maintain the quality and help us reach out to any readers who might be having difficulties due to any unforeseen errors, please write to us at : [email protected] Your support, suggestions and feedbacks are highly appreciated by the BPB Publications’ Family. Did you know that BPB offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.bpbonline.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at : [email protected] for more details. At www.bpbonline.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on BPB books and eBooks.

x



Piracy If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.

If you are interested in becoming an author If there is a topic that you have expertise in, and you are interested in either writing or contributing to a book, please visit www.bpbonline.com. We have worked with thousands of developers and tech professionals, just like you, to help them share their insights with the global tech community. You can make a general application, apply for a specific hot topic that we are recruiting an author for, or submit your own idea.

Reviews Please leave a review. Once you have read and used this book, why not leave a review on the site that you purchased it from? Potential readers can then see and use your unbiased opinion to make purchase decisions. We at BPB can understand what you think about our products, and our authors can see your feedback on their book. Thank you! For more information about BPB, please visit www.bpbonline.com.

Join our book's Discord space

Join the book's Discord Workspace for Latest updates, Offers, Tech happenings around the world, New Release and Sessions with the Authors: https://discord.bpbonline.com



xi

Table of Contents

1. Basic Python Introduction.................................................................................... 1

Introduction.................................................................................................................. 1 Structure........................................................................................................................ 1 Objectives...................................................................................................................... 2

Benefits of Python........................................................................................................ 2



Uses of Python.............................................................................................................. 4



Limitations of Python.................................................................................................. 4



Keywords and reserved words.................................................................................. 13

Identifiers..................................................................................................................... 15

Line joining methods................................................................................................. 15



Implicit line joining method.............................................................................16



Using curly braces......................................................................................16



Using square brackets................................................................................16



Using parenthesis.......................................................................................16



Explicit joining method....................................................................................17



Example 1..................................................................................................17



Example 2..................................................................................................18



Print function.............................................................................................................. 18



Different styles to use print function...............................................................19

Variables....................................................................................................................... 24

Importance of mnemonic variable names in Python............................................ 27



Concept of immutability vs fundamental data types............................................. 29

Conclusion................................................................................................................... 33

Points to remember.................................................................................................... 33

Questions..................................................................................................................... 33

2. Concept of Strings in Python............................................................................... 35

Introduction................................................................................................................ 35 Structure...................................................................................................................... 36 Objectives.................................................................................................................... 36

Reading dynamic input from the keyboard............................................................ 36

raw_input()......................................................................................................36

xii



input()...............................................................................................................37

Using Try Except..............................................................................................40



Using eval function...........................................................................................41



Command Line arguments....................................................................................... 43



Python sys.argv.................................................................................................43



Python getopt module......................................................................................46



Exception getopt.GetoptError...................................................................47



Short form options.....................................................................................47



Long form options......................................................................................48



Python argparse module..................................................................................49



Python argparse Positional arguments.....................................................50



Python argparse positional arguments default values.............................51



Python argparse argument help................................................................52



Python argparse Data Type......................................................................54



Python argparse optional arguments........................................................55



Short names for optional arguments with argparse.................................56



Python argparse with optional and positional arguments.......................57



Python argparse with required..................................................................57



Python argparse dest action......................................................................58



Python argparse append action................................................................59



Allowing or disallowing abbreviations.....................................................59

Strings.......................................................................................................................... 61

Python multiline strings............................................................................................ 62



Python string access................................................................................................... 64



Using index.......................................................................................................64



Using slice operator...........................................................................................65



In forward direction...................................................................................66



In backward direction................................................................................66

Conclusion................................................................................................................... 71

Points to remember.................................................................................................... 71

Questions..................................................................................................................... 71

3. Concept of Flow Control Statements in Python................................................. 73

Introduction................................................................................................................ 73 Structure...................................................................................................................... 73



xiii

Objectives.................................................................................................................... 74

Flow of execution of the program............................................................................ 74



Selection statements/Conditional statements..................................................74

if..................................................................................................................74 if-else..........................................................................................................77 if-elif-else....................................................................................................79

if-elif-else ladder........................................................................................81



Iterative statements..........................................................................................84

for...............................................................................................................85 while...........................................................................................................88

Transfer statements..........................................................................................91

break...........................................................................................................91 continue......................................................................................................95 pass...........................................................................................................100

in keyword usage...................................................................................................... 100



Loop patterns............................................................................................................ 102



Star pattern.....................................................................................................102



Printing stars in pyramid shape..............................................................102



Alphabet pattern ............................................................................................104



Number pattern .............................................................................................107

Conclusion................................................................................................................. 109

Points to remember.................................................................................................. 109

Questions................................................................................................................... 110

4. Concept of Exception Handling in Python....................................................... 111

Introduction.............................................................................................................. 111 Structure.................................................................................................................... 111 Objectives.................................................................................................................. 112 Errors.......................................................................................................................... 112

Syntax error....................................................................................................112



Runtime Error................................................................................................113



Importance of exception handling......................................................................... 114



Python exception hierarchy.................................................................................... 115



Customized exception handling............................................................................. 116

xiv





Control flow in try-except....................................................................................... 118



Case-1 - No raising of exception....................................................................119

Case-2 - Exception raised at st3 and corresponding except block is matched.............................................................................................119 Case-3 - Exception raised at st3 and corresponding except block is not matched.......................................................................................119

Case-4 - Exception raised at st5.....................................................................119



Case-5 - Exception raised at st6.....................................................................119



Exception information printing to the console.................................................... 120



Try with multiple except blocks.............................................................................. 121



Single except block that can handle multiple exceptions.................................... 122



Default except block................................................................................................. 124



Possible combinations of except block.................................................................. 125



finally except block.........................................................................................126



Control flow in try-except and finally...........................................................129



Case-1 - No exception is raised...............................................................130

Case-2 - Exception raised at st3 and corresponding except block is matched....................................................................................................130 Case-3 - Exception raised at st3 and corresponding except block is not matched.............................................................................................130

Case-4 - Exception raised at st5..............................................................131



Case-5 - Exception raised at st6 or st7....................................................131



Nested try-except finally block.......................................................................131



Control flow in Nested try-except finally block.............................................134



Case-1 - If there is no exception..............................................................135

Case-2 - Exception raised at st3 and corresponding except block is matched.......................................................................................135 Case-3 - Exception raised at st3 and corresponding except block is not matched................................................................................135

Case-4 - Exception raised at st6 and inner except block is matched.....135

Case-5 - Exception raised at st6 and inner except block is not matched but outer except block is matched............................................135 Case-6 - Exception raised at st6 and both inner and outer except block is not matched.....................................................................136 Case-7 - Exception raised at st7 and the corresponding except block is matched...........................................................................136



xv

Case-8 - Exception raised at st7 and the corresponding except block is not matched.....................................................................136 Case-9 - Exception raised at st8 and the corresponding except block is matched...........................................................................136 Case-10 - Exception raised at st8 and the corresponding except block is not matched.....................................................................137 Case-11 - Exception raised at st9 and the corresponding except block is matched...........................................................................137 Case-12 - Exception raised at st9 and the corresponding except block is not matched.....................................................................137

Case-13 - Exception raised at st10..........................................................137



Case-14 - Exception raised at st11 or st12.............................................138



Else block with try-except finally block.........................................................138

Conclusion................................................................................................................. 141

Points to remember.................................................................................................. 141

Questions................................................................................................................... 141

5. Concept of Regular Expressions in Python....................................................... 143

Introduction.............................................................................................................. 143 Structure.................................................................................................................... 144 Objectives.................................................................................................................. 144 compile().................................................................................................................... 144 finditer()..................................................................................................................... 145

Character classes....................................................................................................... 147



Pre-defined character classes.................................................................................. 153

Quantifiers................................................................................................................. 158

Functions of re module............................................................................................ 162

match..............................................................................................................162 fullmatch.........................................................................................................164 search..............................................................................................................166 findall..............................................................................................................167 sub...................................................................................................................168 subn.................................................................................................................169 split..................................................................................................................171 escape..............................................................................................................173 Metacharacters.......................................................................................................... 174

xvi





[] (Square brackets)........................................................................................174



. (Period).........................................................................................................175



^ (Caret).........................................................................................................175



$ (Dollar)........................................................................................................175



* (Star)............................................................................................................176



+ (Plus)...........................................................................................................176



? (Question Mark)..........................................................................................177



{} (Braces).......................................................................................................177



| (Alternation):................................................................................................177



() (Group)......................................................................................................178



\ (Backslash)...................................................................................................178



r prefix........................................................................................................................ 178

Conclusion................................................................................................................. 179

Points to remember.................................................................................................. 179

Questions................................................................................................................... 179

6. Concept of Functions in Python........................................................................ 181

Introduction.............................................................................................................. 181 Structure.................................................................................................................... 181 Objectives.................................................................................................................. 182

Functions in Python................................................................................................. 182



Function types........................................................................................................... 184



Built in functions............................................................................................184



User defined functions....................................................................................185



Function arguments................................................................................................. 186



Positional arguments......................................................................................187



Keyword arguments.......................................................................................189



Default arguments..........................................................................................190



Variable length arguments.............................................................................192



Keyword variable length arguments (kwargs)..............................................194



Nested function......................................................................................................... 197



Python closures......................................................................................................... 199



Function passing as a parameter............................................................................ 201



Local, global, and non-local variables.................................................................... 202



Local variables................................................................................................202



xvii



Global variables..............................................................................................203



Non-local variables.........................................................................................207



Recursive function.................................................................................................... 208



Python Lambda functions....................................................................................... 210



Nested lambda functions................................................................................211



Passing lambda functions to another function.............................................212

Conclusion................................................................................................................. 213

Points of remember.................................................................................................. 213

Questions................................................................................................................... 214

7. Concept of Data Structures in Python.............................................................. 215

Introduction.............................................................................................................. 215 Structure.................................................................................................................... 215 Objectives.................................................................................................................. 216

List data structure..................................................................................................... 216



Creating a list..................................................................................................216



Creating an empty list.............................................................................216



Creating a list when elements are known...............................................217



Creating a list with dynamic input.........................................................217



List creation using list() function............................................................218



List creation using split() function..........................................................218



Lists versus immutability...............................................................................219



Accessing elements of list................................................................................219



By using index..........................................................................................219



By using index and for loop.....................................................................221



By using Index and while loop................................................................221



By using list slicing...................................................................................222



List comprehension.........................................................................................223



List comprehension with for loop............................................................224



List comprehension with for loop and if statement................................225



List comprehension with for loop and nested if statement.....................225



List comprehension with if else statement and for loop.........................226



Nested list comprehension with for loop.................................................227



Tuple data structure.................................................................................................. 228

xviii





Tuple creation.................................................................................................229



An empty tuple creation..........................................................................229



Single valued tuple creation....................................................................230



Multiple valued tuple creation................................................................231



Using tuple() function..............................................................................231



Accessing elements of tuple.............................................................................232



By using index..........................................................................................232



By using index and for loop.....................................................................233



By using index and while loop................................................................234



By using tuple slicing...............................................................................235



Tuple versus immutability..............................................................................236



Tuple comprehension......................................................................................236



List versus tuple comparison..........................................................................237



Set data structure...................................................................................................... 238



Set creation.....................................................................................................239



Set comprehension..........................................................................................240



Set comprehension with for loop.............................................................241



Set comprehension with for loop and if statement.................................241



Set comprehension with for loop and nested if statement......................242



Set comprehension with if else statement and for loop..........................243



Dictionary data structure........................................................................................ 244



Creation of an empty dictionary....................................................................245



By using dict() function...........................................................................245



By using curly braces only.......................................................................245



Creation of a dictionary.................................................................................246



Accessing dictionary.......................................................................................247



Accessing dictionary.......................................................................................249



Deleting dictionary item................................................................................250



Dictionary comprehension.............................................................................252



Dictionary comprehension with for loop................................................252



Dictionary comprehension with for loop and if statement....................253



Dictionary comprehension with for loop and nested if statement.........254



Dictionary comprehension with if else statement and for loop.............255



xix

Conclusion................................................................................................................. 256

Points of remember.................................................................................................. 256



Questions (Long/Short/MCQs).............................................................................. 257



8. Concept of Packages in Python......................................................................... 259

Introduction.............................................................................................................. 259 Structure.................................................................................................................... 259 Objectives.................................................................................................................. 260 Packages..................................................................................................................... 260

Structure for package of games............................................................................... 262



Using “import” in Packages............................................................................262



Accessing objects like variables, functions, classes, and lists..................263



Using “from import” in Packages...................................................................263



Accessing objects like variables, functions, classes, and lists..................264



Using “from import *” in Packages................................................................264



Accessing games package using different approaches......................................... 266

Conclusion................................................................................................................. 287

Points to remember.................................................................................................. 287

Questions................................................................................................................... 287

9. Numpy Introduction.......................................................................................... 289

Introduction.............................................................................................................. 289 Structure.................................................................................................................... 289 Objectives.................................................................................................................. 290

Similarities between list and numpy array............................................................ 290



Differences between list and numpy array............................................................ 290



Numpy arrays creation............................................................................................. 292



1-D array creation using list..........................................................................292



1-D array creation using tuple.......................................................................293



2-D array creation using Nested lists.............................................................294



Array creation with a particular dtype.........................................................295



Object type array creation..............................................................................296



1-D array creation with arange() function...................................................297



Using linspace()..............................................................................................299



Using zeros()...................................................................................................301



Using ones()....................................................................................................303

xx





Using full()......................................................................................................304



Using eye()......................................................................................................305



Using diag()....................................................................................................307



Using empty().................................................................................................309



Comparison between zeros and empty..........................................................310

Conclusion................................................................................................................. 311

Points to remember.................................................................................................. 311

Questions................................................................................................................... 312 10. Data Visualization Introduction....................................................................... 313 Introduction.............................................................................................................. 313 Structure.................................................................................................................... 314 Objectives.................................................................................................................. 314

Python data visualization tools............................................................................... 314



Line plot creation by passing 2 ndarrays............................................................... 315



Adding title, xlabel and ylabel to the line plot...................................................... 317



Advanced line plot.................................................................................................... 318



linestyle property...................................................................................................... 320



color property........................................................................................................... 322



default color............................................................................................................... 323



Peep in a shortcut way to set color, marker and linestyle.................................... 325



mlc form..........................................................................................................326



clm form..........................................................................................................327



If no color is mentioned, then default color is blue.......................................328



alpha property........................................................................................................... 329



linewidth and markersize property........................................................................ 330



markerfacecolor property........................................................................................ 331



Customizing the figure size..................................................................................... 333



Plotting multiple lines in a same plot..................................................................... 335

Conclusion................................................................................................................. 337

Points of remember.................................................................................................. 337

Questions................................................................................................................... 338 11. Pandas Introduction.......................................................................................... 339 Introduction.............................................................................................................. 339 Structure.................................................................................................................... 340



xxi

Objectives.................................................................................................................. 340

Pandas Series............................................................................................................. 340



Pandas Series constructor..............................................................................340



Creating Pandas Series by passing a list.................................................341



Creating Pandas Series by passing a dictionary.....................................342



Creating Pandas Series by passing a numpy array.................................342



Accessing elements in Pandas Series..............................................................343



Pandas Series slicing.......................................................................................344



Pandas Series filtering....................................................................................345



Usage of apply method to Pandas Series.......................................................347



Aggregating of Pandas Series.........................................................................348



Pandas DataFrame.................................................................................................... 349



Pandas DataFrame constructor.....................................................................350



Pandas DataFrame creation..........................................................................350



Accessing data in Pandas DataFrame...........................................................351



Data modification in Pandas DataFrame.....................................................353



Data aggregation in Pandas DataFrame......................................................356

Conclusion................................................................................................................. 358

Points to remember.................................................................................................. 358

Questions................................................................................................................... 359 Index...................................................................................................... 359-367

xxii



Chapter 1

Basic Python Introduction Introduction

In the early 1990s, the Python programming language was created by Guido Van Rossum. Its implementation began in December 1989, in the National Research Institute, Netherlands. The Python language is even older than the Java language. The name Python was coined from one of the most popular British sketch comedy series “Monty Python’s Flying Circus”. The first Python version 0.9.0 was released in February, 1991. The next Python version 1.0 was released in January, 1994. Python version 2.0 was released in the October 2000, and version 3.0 came in December 2008. The latest version at the moment, is 3.11.2, which was released in February, 2023.

Structure

In this chapter, we will discuss the following topics: • Benefits of Python • Uses of Python

• Limitations of Python

• Keywords and reserved words

2



Python for Everyone

• Identifiers

• Line joining methods • Print function • Variables

• Importance of mnemonic variable names in Python

• Concept of Immutability vs Fundamental data types

Objectives

By the end of this chapter, the reader will learn about the Python language and its benefits, limitations and applications. The rules regarding variables and identifiers will be thoroughly explained with examples. It is important to understand different case styles of the print() function, so that the reader can create various Python applications while using it smartly in their code. Finally, the immutability concept with respect to fundamental data types will be elucidated to the reader with examples, for better understanding. Step by step code explanation is also illustrated while explaining various concepts in Python, within the chapter.

Benefits of Python

It is important to understand why we need to study Python, as it comprises of various benefits:

• Python is a freeware and an open-source beginner’s language for newcomers. If anyone is a beginner, Python is the best language to start with. For Java Commercial, the business organization is Oracle; for C#.net, the business organization is Microsoft, but for Python, the business organization is Python Software Foundation. It is a non- profit organization that has IP rights for Python programming language. The website is https://www.python. org. Moreover, the source code is open. Based on our requirement, we can customize the Python requirement itself. To work with Java applications, we are required to go for Jython, and for C#.net, we need Iron Python. Similarly, to work with large data (in Machine Learning, Deep Learning), we require Anaconda Python, and to work with Ruby Applications, we require Ruby Python.

• Python is very simple and easy to understand as it is a high-level language. Its syntax is also very easy to comprehend. We do not need to worry for low level activities such as memory management, providing free space and so on. Internally, Python Virtual Machine (PVM) will take care.

Basic Python Introduction



3

• Python is platform independent. We can run this language on different hardware platforms such as Linux, Mac, Windows, Raspberry Pi and so on. You can write the program once and run anywhere: this is the concept of platform independent nature. PVM is responsible for running the Python code in different platforms. PVM is platform dependent, whereas Python is platform independent.

• In Python, code can be executed as soon as it is written. Thus, it runs on an interpreter system. It is not required to compile explicitly. • A programmer can write a program in fewer lines than other programming languages. Thus, it is a more efficient and concise way of writing the code.

• Python can be used as functional oriented, object oriented or procedure oriented. • Python has tons of libraries which are used for various implementations such as NumPy, SciPy, Pandas and so on.

• Python is extensible. We can use legacy non Python code in our application and fill up the performance gap with other language code. • Python is a portable language. The Python application can be migrated from one platform to another very easily.

• Python is embedded. We can use the Python script inside Java or C#.Net application. The scope of the Python code is improved. Thus, our application will become scalable. • It is dynamically typed. We do not need to declare type explicitly. Based on our provided value, the type will be considered automatically, thus giving more flexibility to the programmer. Refer to the following code: a=10 print(type(a)) a='Python' print(type(a)) a=False print(type(a))

Output:





4



Python for Everyone

Note: The preceding code is covered in (Program Name: Chap01_prog1_ dynamictyped.py) Python language has similarity to the English language and was designed for readability. The command can be completed with a new line, unlike semicolons or parenthesis, as opposed to other programming languages. To define the scope of loop, classes and functions, Python relies heavily on indentation such as whitespace, unlike other programming languages. which use curly space for this purpose.

Uses of Python

The Python language is used for:

• Back end web development.

• To create Artificial Intelligence and scientific computing, Machine Learning, Deep Learning, Internet of Things and so on. • Desktop Applications, 3D graphics, Graphical User Interface Applications.

• Data Analysis, Network applications such as Client Server, Chatting and so on. • For connectivity to database systems; it can read, write, delete or update the data as per need. • It can perform very complex mathematics and can handle big data.

A good Python developer writes effective code for backend components, testing and debugging programs. A good developer creates applications that can be integrated with the present ones. Python is used by different companies such as Google, Facebook, Yahoo, NASA, DropBox, BitTorrent, Netflix, YouTube and so on. In Python, most of the syntax has been borrowed from C language and ABC language.

Limitations of Python

Apart from the benefits, Python also has the following limitations:

• Python is not suitable for developing mobile applications. This is because currently, it does not have library support to develop mobile applications. • Python is not the best choice to develop end to end enterprise applications such as banking, telecom applications and so on, as there is no library support.

• The performance is low because the execution is happening line by line. Thus, JIT compiler is added to Python Virtual Machine, so that a group of lines will

Basic Python Introduction



5

be interpreted only once, and each time, the interpreted code is used directly. The preceding flavor is called PyPy version (Python for speed). The popularity of Python in 2023 is record breaking. According to Stack-Overflow, Python is the most questioned emerging language, and it is far ahead of competitor languages such as JavaScript, C# and so on. GitHub grants Python the top slot of being the most popular language. To install python on the system, go to the website https://www.python.org. Then go to the Download section to download and install the latest Python version (3.11.2, as on today). We have installed Python version on Windows operating system. Keep in mind that we will be doing all our programs in Windows OS only. It is better to learn Python version 3 instead of version 2 because as of today, all the multinational companies who have been using Python 2, have migrated to Python 3. Python 3.x is developed as a completely independent language and is not an extension of 2.x version. The backward compatibility of Python 3.x is not there for Python 2.x, as there is no guarantee that it will support the same. Moreover, Python 2 may become obsolete in the near future as the libraries will not be maintained. During the installation, an important point to be noted is to tick the checkbox Add Python 3.7 to Path (we have installed Python 3.7.3 version in our case). Otherwise, there are chances of error once we try to install our own libraries. Install the setup and you are good to go. Once Python is installed, type cmd and enter the word Python. You will get a screen as shown in Figure 1.1. In this book, we will learn about Python 3.x and not Python 2.x:

Figure 1.1: Python version

6



Python for Everyone

Once we have installed Python, we also get Integrated Development and Learning Environment (IDLE) for Python. Here, we can do our coding. It looks as shown in Figure 1.2:

Figure 1.2: Python 3.7.3 shell

To run the program, follow the given steps: 1. Go to File | New File.

2. Type print('Hello World') in the file and save it into a respective folder. 3. Click Run | Run Module F5, as shown in Figure 1.3:

Figure 1.3: Python file saved as hello.py

Basic Python Introduction



7

The output Hello World will be printed in the IDLE screen, as shown in Figure 1.4:

Figure 1.4: Output in Python 3.7.3. shell

You can also look for other IDEs such as PyCharm, Jupiter and so on. However, we have used Visual Studio Code (VsCode), and integrated gitbash into VSCode. VSCode is a source code editor for making programs which are developed by Microsoft. It has support for debugging, syntax highlighting, code refracting, Intelligent code completion and so on. Before starting with the basics of Python, let us have a brief overview of how to use command line in VSCode. With the help of command line, we can use files/folders, quickly create or remove them, as well as copy and move the files to/from the folder, among others. Let us start by typing the following commands in the bash terminal of VSCode, and thus visualize the output. The commands are typed in bold letters.

• pwd: The pwd command or the Print working directory will display the current location of the working directory. From the terminal output shown in Figure 1.5, we can see that the working directory is E:/python_progs.

Figure 1.5: pwd command

• ls: This command will display the list of files and folders in the terminal. From the terminal output shown in Figure 1.6, we can see the list of files and directories within the file system:

8



Python for Everyone

Figure 1.6: ls command

• ls -l: It will display the list of files and folders in the terminal, using a long listing format. From the terminal output shown in Figure 1.7, we can see the list of files and directories within the file system, along with the owner, permissions. Since the list of contents is so long, only a part of it has been shown.

Figure 1.7: ls -l command

• clear: This command will clear the screen terminal. From the terminal output shown in Figure 1.8, we can see that the screen has been cleared:

Figure 1.8: clear command

Basic Python Introduction •



9

cd: It will change the current working directory in the operating systems.

From the terminal output shown in Figure 1.9, we can see that the directory has been changed to E://democreated:

Figure 1.9: cd command •

mkdir: This command is used to create folders in the operating system. From

the terminal output shown in Figure 1.10, we can see that a new folder namely command folder, has been created:

Figure 1.10: mkdir command

An important point to observe is that if we give space inbetween the folder name, and the folder name is not present in double inverted commas, then separate individual folders will be created. In Figure 1.11,we can see that 3 different folders are created from mkdir command practice 2:

Figure 1.11: mkdir without double inverted commas

Now, let us put double inverted comma between the folder name: mkdir "command practice 2". From Figure 1.12, we can see that a new folder command practice 2 is created:

10



Python for Everyone

Figure 1.12: mkdir with double inverted commas

Let us assume we need to move to a folder command practice 2. Type the command cd "command practice 2 "/, as shown in Figure 1.13:

Figure 1.13: Moving to a folder •

touch: This command is used to set the modification and access times of

files to the current time of day. If the file is not present, then it will create the file with default permissions. In Figure 1.14, a text file namely newfile.txt is created:

Figure 1.14: touch (text file)

We can even create a Python file. A new file namely demo.py is created, in Figure 1.15:

Figure 1.15: touch (python file)

Basic Python Introduction •



11

rm: This command is used to remove objects such as file, directories and so on. In Figure 1.16, let us remove newfile.txt:

Figure 1.16: remove a text file •

cd .. : This command is used to move one folder back. in Figure 1.17, we can

see that one folder has been moved back:

Figure 1.17: command to move one folder back •

rm -rf: This command is used to remove folders completely. rf stands for recursive force. In Figure 1.18, we can see that the folder command practice 2 has been removed completely:

Figure 1.18: rm -rf command •

mv: This command is used to rename a file with a new name or move a file name into a new folder. In Figure 1.19, we can see that the file name file1.py has been renamed to file.py from the current folder cd commandfolder/:

Figure 1.19: mv command (rename)

12



Python for Everyone

In Figure 1.20, we can see that the file name file.py has been moved to the current folder mvfolder. Firstly, we have created a folder named mvfolder. Then, we have moved the file into that folder using command mv file.py ./mvfolder/ On the current path, we typed ls to see the list of files and folders. We can see that the file has moved into the mvfolder since only the folder name is being displayed. We then change the directory into mvfolder. We typed ls to see the list of files and folders. Refer to Figure 1.20:

Figure 1.20: mv command (move file into folder)

Let us now assume that we again want to move the file one folder backwards. In this case, we have to use filename with “..”, as shown in Figure 1.21. After typing the command mv file.py .. , the file file.py has moved to the folder commandfolder as shown:

Figure 1.21: mv command (move file into one folder backward)

Basic Python Introduction



13

Let us now assume that we want to copy a file into the folder mvfolder. In Figure 1.22, we can see that the command cp file.py has been copied into the folder / mvfolder/. We can see that using ls command, the folder mvfolder contains the file name file.py:

Figure 1.22: cp command

If you want to save the list of commands, type in the bash terminal of VsCode into a text file, and then use the following command: history > history_for_print.txt

Here, history is the command and history_for_print.txt is the text filename as shown in Figure 1.23. The command and the file name is joined by the ‘>’ symbol. We can see that the list of commands typed has been saved in the text file name history_for_print.txt.

Figure 1.23: history command

Keywords and reserved words

In any language, whether it may be a general speaking language such as English, or a programming language such as Python or C, there are some reserved words

14



Python for Everyone

to represent some meaning or functionality, and they are called reserved words or keywords. There are tons of English reserved words, each with some specific meaning. It is quite impossible to remember those words. On the other hand, if we look at some programming languages like Java, only 53 reserved words are present. However, in Python, there are only 35 reserved words. Thus, if we understand these 35 keywords, we might become an expert on this language. To know the total keywords in the Python language, type the following command in VsCode: import keyword print(keyword.kwlist)

Note: The preceding code is covered in (Program Name: Chap01_prog2_ keywords.py) Save this to a file named as python_keywords.py. All reserved words in Python contain alphabet symbols. To run the above program, type the command as: python python_keywords.py.

You will get the list of keywords as shown: ['False', 'None', 'True', 'and', 'as', 'assert', 'async', 'await', 'break', 'class', 'continue', 'def', 'del', 'elif', 'else', 'except', 'finally', 'for', 'from', 'global', 'if', 'import', 'in', 'is', 'lambda', 'nonlocal', 'not', 'or', 'pass', 'raise', 'return', 'try', 'while', 'with', 'yield']

We will discuss all the list of keywords as per requirement. However, the important observation that you may get by looking into the keywords is as follows: • All the 35 keywords contain only alphabets.

• True, False and None are the only 3 keywords which are in Uppercase. The remaining 32 keywords are in lower case.

• An important point to note is that there is no switch or do-while concepts in Python.

• Since Python is dynamically typed, there is no reserved words such as int, float, Boolean, complex data types and so on. To check if any word is a keyword or not, type the following command: print(keyword.iskeyword('yield'))

Basic Python Introduction



15

On running the preceding file, you will get a Boolean value as True, indicating that the yield word is a keyword. You can continue to check with other reserved words. The return type of iskeyword is either True or False.

Identifiers

Identifiers are nothing but the user defined names utilized in the programs, to represent a variable, a function, a class or a module. Identifiers can contain a letter, digit or underscore. The first letter of the identifier must be a letter or underscore, and should not start with digit. An example of an identifier is as follows: • var_1=67: Variable name is an example of an identifier.

• _var = 4: The first letter of an identifier can be an underscore. • 3sd = 7: Invalid. SyntaxError: invalid syntax.

• Special characters are excluded from the identifier.

• var@2 = 8: SyntaxError: cannot assign to operator. The allowed characters in Python are alphabets (either upper or lower case), digits (from 0 to 9) and the underscore symbol. As the Python language is case sensitive, identifiers themselves are case sensitive. The variables, for example, sum and SUM are different. If the identifier starts with underscore, then it is private. Reserved words are not allowed in identifiers. There is no length limit for identifiers, but care has to be taken that it is not too lengthy and must be meaningful enough for even a new person to understand.

Line joining methods

Whenever we are trying to divide input into different physical line, there will be an error. For example: print("hello

When we try to write the rest of the words in the next line, the following error will be displayed: print("hello ^ SyntaxError: EOL while scanning string literal

This can be overcome by using implicit and explicit line joining method.

16



Python for Everyone

Implicit line joining method In implicit line joining method, the expression is split into multiple lines using parenthesis, curly lines or square brackets, as discussed.

Using curly braces Refer to the expression: days = {'Mon','Tue','Wed', 'Thu','Fri','Sat', 'Sun'} print(days)

Output {'Tue', 'Sat', 'Fri', 'Sun', 'Mon', 'Wed', 'Thu'}

Note: You may get different output every time here.

Using square brackets Refer to the expression: days = ['Mon','Tue','Wed', 'Thu','Fri','Sat', 'Sun'] print(days)

Output: ['Mon', 'Tue', 'Wed', 'Thu', 'Fri', 'Sat', 'Sun']

Using parenthesis Refer to the expression: days = ('Mon','Tue','Wed', 'Thu','Fri','Sat', 'Sun')

Basic Python Introduction



17

print(days)

Output: ('Mon', 'Tue', 'Wed', 'Thu', 'Fri', 'Sat', 'Sun')

The preceding examples can also be written with comments and blank lines, as shown: • We can write comment after the line:

days = {'Mon','Tue','Wed', # Mon - Wed



'Thu','Fri','Sat', # Thu - Sat



'Sun'} # sun

print(days)

Output

{'Tue', 'Thu', 'Fri', 'Sat', 'Sun', 'Mon', 'Wed'}

• Blank lines can be inserted:

days = { #Blank



'Mon','Tue','Wed', # Mon - Wed



'Thu','Fri','Sat', # Thu - Sat



'Sun'} # sun

print(days)

Output

{'Sun', 'Tue', 'Thu', 'Sat', 'Wed', 'Mon', 'Fri'}

Note: All the implicit joining method examples are covered in (Program Name: Chap01_prog3_implicit_joining_method.py)

Explicit joining method In the explicit line joining method, we are using backward slash to split the input. The backward slash is used to join the logical lines together in such a manner that they are declared in a single line. Refer to the following examples.

Example 1 Refer to the following expression: print("hello\

18



Python for Everyone

welcome beginners")

Output: hello

welcome beginners

Example 2 Instead of writing the logic in one line, we have used backslash and continued the half logic in another line. This way of writing is very much allowed, as shown: x = 12 print(x>10 | \ x=1.5 and my_age =12 and my_age =17 and my_age =30 and my_age total = 0 + 1 total

= 1

2nd Iteration i = 2 total += 2 => total = 1 + 2 total

= 3

3rd Iteration i = 3 total += 3 => total = 3 + 3 total

= 6

88



Python for Everyone

4th Iteration i = 4 total += 4 => total = 6 + 4 total

= 10

5th Iteration i = 5 total += 5 => total = 10 + 5 total

= 15

So, the sum of first 5 natural numbers is 15.

while In while loop, a block of code is executed as long as the test expression is True and the control will come out of the loop if the test expression is False. At the beginning of the loop, the condition is checked every time. The body of while loop is determined through indentation and the first un-indented line marks the end. The syntax of while loop is as follows: While test expression: Body of while

Here, test expression is checked first. If found True, then the control enters into the body of while loop. After one iteration, the control goes again to evaluate test expression. This is a repetitive process until the test expression is evaluated to False. The flow chart of while loop is shown in Figure 3.9:

Concept of Flow Control Statements in Python

Figure 3.9: Flowchart of while loop

Let us see some examples: j = 1 while j 32: pass

Output: Python will do nothing and there will be no output. Note: The preceding code is covered in (Program Name: Chap03_Example3.12. py)

in keyword usage

We have used the ‘in’ keyword in Python code quite often. This special keyword serves 2 purpose in Python. We shall see using the following examples:

• It checks whether value is present in a sequence or not. The sequence includes list, range, set and so on.

name = "Saurabh"

#I1

if 'z' in name: print('z is present in the name')

Concept of Flow Control Statements in Python



101

else:

print('z is not present')



#I2



if 'a' in name:



print('a is present in the name')

else:

print('a is not present')



Output:



z is not present



a is present in the name

Note: The preceding code is covered in (Program Name: Chap03_ Example3.13.py) In I1, we are checking the letter ‘z’ in the string variable name as ‘Saurabh’. Since the letter ‘z’ is not present in the string variable, the output will be z is not present.

In I2, we are checking the letter ‘a’ in the string variable name as ‘Saurabh’. Since letter ‘a’ is present in the string variable at 2 different index positions, the output will be a is present in the name. • It is used to iterate through a sequence in a for loop.

games = ['cricket','football','basketball']



for i in games:



print(i)



Output:

cricket football basketball

Note: The preceding code is covered in (Program Name: Chap03_ Example3.14.py)

102



Python for Everyone

Loop patterns

In C language, we have studied various pattern programming’s using simple for loops. The same can be replicated using Python code also. We will be using nested for loops to handle number of rows and columns. The print statements will be manipulated to display different number patterns, alphabet patterns or star patterns.

Star pattern We will be viewing different star patterns in a simple manner. The basic program will have 2 for loops. The outer for loop will be for number of rows, and inner for loop will be for number of columns in the pattern. The print function will be used for displaying the output and the input function will be used to get the user input. The range function will be used for iterating the loop between the starting number, and ends with integer number which the user inputs. The iteration of inner loop depends on the values of the outer loop. The new line is added after each row, that is, on each iteration of the outer for loop, so that the pattern can be displayed as per the need. Let us see an example.

Printing stars in pyramid shape Refer to the following example: my_num = int(input('Enter the number of rows: ')) for a in range(0,my_num): for b in range(0,my_num-a-1): print(end = " ") for c in range(0,a+1): print('*',end = " ") print()

Output: Enter the number of rows: 4 * * * * * * * * * *

Concept of Flow Control Statements in Python



103

Note: The preceding code is covered in (Program Name: Chap03_Example3.15. py) Refer to the following Table 3.1: (s)

(s)

(s)

*

(s) *

*

(s) *

(s)

*

*

*

a=0 *

*

a=1 a=2

*

Table 3.1: Table depicting star printing in pyramid shape

a=3

The user is first prompted to input the number of rows. Here, the user entered the row number as 4. So, the first for loop is for rows from a = 0 to a=3, as shown in Table 3.1. The next for loop, for b in range(0,my_num-a-1), will print the spaces in each row. The print statement is ended with space using end = “ “. In Table 3.1, the spaces are marked as (s) to identify the number of spaces in each row. Last but not the least, the for loop will print the stars, followed by space in each row. print() function is used to move the cursor onto the next line. We shall see the display of stars in each iteration of number of rows. a=0 3 spaces and one star followed by a space in row = 0. (s)

(s)

(s)

a=0

*

a=1 2 spaces and 2 stars each followed by a space in row = 1 (s)

(s)

(s)

(s)

(s) *

*

a=0

a=1

*

a=2 1 space and 3 stars each followed by a space in row = 2 (s)

(s)

(s)

(s)

(s) *

(s) *

*

*

*

a=3

a=0 *

a=1

a=2

104



Python for Everyone

No space and 4 stars each followed by a space in row = 3 (s) (s)

(s) *

(s)

(s) *

(s) *

*

* *

* *

a=0 *

a=1

*

a=2

a=3

Alphabet pattern The behavior of printing alphabet pattern will be the same as that of using star pattern. So, without any further delay, let us see the example: #Method-1 for a in range(0,6): my_num = 65 for b in range(0,a+1): my_alpha = chr(my_num) print(my_alpha,end = " ") my_num += 1 print() #Method-2 from string import ascii_uppercase for i in range(1, 7): print(" ".join(ascii_uppercase[:i]))

Output: A A B A B C A B C D A B C D E

Concept of Flow Control Statements in Python



105

A B C D E F A A B A B C A B C D A B C D E A B C D E F

Note: The preceding code is covered in (Program Name: Chap03_Example3.16. py) In Method-1, we are using an ASCII value of a letter. The ASCII value will be converted to character, in order to print it on the screen. The number of rows is 6. First, the ASCII value 65 will be converted to character ‘A’ using chr function. Then the current ASCII value will be incremented by one, and the control goes to the next row. Depending on each iteration, the ASCII value will again start with initial value 65 and will be incremented on each iteration. The corresponding characters will be displayed based on the ASCII values. The execution of the program on each iteration is shown as follows. a=0 my_num = 65 for b in range(0,1) A a=1 my_num = 65 for b in range(0,2) A A B

a=2 my_num = 65

106



Python for Everyone

for b in range(0,3) A A B A B C

a=3 my_num = 65 for b in range(0,4) A A B A B C A B C D

a=4 my_num = 65 for b in range(0,5) A A B A B C A B C D A B C D E

a=5 my_num = 65 for b in range(0,5) A A B A B C

Concept of Flow Control Statements in Python



107

A B C D A B C D E A B C D E F

In Method-2, from string, we have imported the ascii_uppercase string. ascii_ uppercase is initialized with value ABCDEFGHIJKLMNOPQRSTUVWXYZ in string module. A new substring is created and concatenated with empty space using join method. The output will be same as Method-1.

Number pattern The behavior of the number pattern is same as that of star pattern. We will be using the same for loops and range function to display the numbers in some pattern. Let us view an example: num = int(input('Enter the number: ')) count = 1 for my_row in range(1,num + 1): for my_col in range(1,my_row+1): print(count,end = ' ') count +=1 print()

Output: Enter the number: 4 1 2 3 4 5 6 7 8 9 10

Note: The preceding code is covered in (Program Name: Chap03_Example3.17. py) In the preceding example, the user is prompted to enter the number. The user has entered the integer value 4. Here, nested for loop is used, in which, the outer for

108

Python for Everyone



loop is for the display of rows, and the inner for loop is for the display of columns. The count value is displayed and incremented by 1 on each loop iteration. The program execution of the preceding code is shown as follows. Enter the number: 4 count = 1

1st iteration my_row = 1 for my_col in range(1,2): print(count,end = " ") count +=1 print()

Output: 1

2nd iteration my_row = 2 for my_col in range(1,3): print(count,end = " ") count +=1 print()

Output: 1 2 3

3rd iteration my_row = 3 for my_col in range(1,4): print(count,end = " ") count +=1 print()

Output: 1 2 3

Concept of Flow Control Statements in Python



109

4 5 6

4th iteration my_row = 4 for my_col in range(1,5): print(count,end = " ") count +=1 print()

Output: 1 2 3 4 5 6 7 8 9 10

Conclusion

In this chapter, we first saw the if statement followed by if-else statement and finally if-elif-else conditional statements. Then we saw if-elif-else ladder concept where there is provision for the user to choose between multiple options. Finally, we saw for and while loop iterative statements for executing group of statements multiple times. We also saw how to change the flow of a normal loop using break and continue statements. We also saw the use case of pass statement for constructing a body that does nothing. The importance of ‘in’ keyword is well explained. Finally, at the end, we saw star, alphabet and loop patterns display with well-illustrated code. In the next chapter, we will discuss about exception handling in Python.

Points to remember

• When a certain condition is met using if statement, then a block of code can be executed.

• When a certain condition is not met using if statement, then a certain block of code can be executed on else statement. • Whenever there is a requirement to check multiple expressions, then we can use if-elif-else statement.

• We can use for loop if there is a requirement to iterate over a sequence of elements.

110



Python for Everyone

• We can use while loop if a block of code is executed with the test expression check.

• The loop can be terminated using break statement and the remaining part of the code can be skipped for the current iteration using continue statement. • pass statement can be used to construct a body which does nothing.

• ‘in’ keyword can be used for checking or iterating through a sequence in a ‘for’ loop. • We can display number, star or alphabet patterns using loops and print statement.

Questions

1. What are selective statements in Python? Explain with their definition, syntax, flow chart with an example code.

2. Explain if-elif-else ladder concept with its syntax and a Python snippet code. 3. Explain for and while iterative statements with its syntax and Python snippet code.

4. Explain break and continue transfer statements with its syntax and Python snippet code. 5. Explain the importance of in statement with a Python snippet code. 6. Explain star loop pattern with a Python snippet code.

7. Explain Alphabet pattern with a Python snippet code. 8. Explain number pattern with a Python snippet code.

Join our book's Discord space

Join the book's Discord Workspace for Latest updates, Offers, Tech happenings around the world, New Release and Sessions with the Authors: https://discord.bpbonline.com

Chapter 4

Concept of Exception Handling in Python Introduction

There are many times when the user intentionally or unintentionally enters a wrong input and our program meets with an error. When the error occurs, an exception is generated by Python that can be handled, thus avoiding the program to crash. We will be covering this topic in a much-detailed manner as the programmer must have the flexibility to use it in their own code.

Structure

In this chapter, we will cover the following topics: • Errors

• Importance of exception handling • Python exception handling

• Customized exception handling • Control flow in try-except

• Exception information printing to the console • Try with multiple except blocks

112



Python for Everyone

• Single except block that can handle multiple exceptions • Default except block

• Possible combinations of except block

Objectives

By the end of this chapter, the reader will have an idea about syntax and runtime errors. They will be able to gracefully terminate the code using try and except block, and the importance of finally block will be known for cleaning up the code. Single except block for handling multiple exceptions will be learned about, along with the default except block. Different use cases along with control flow of try-except, tryexcept finally block and nested try-except and finally block will be illustrated to the reader with examples. The importance of else block will be explained at the end.

Errors

There are generally 2 types of error in any programming language: Syntax and Runtime.

Syntax error Whenever we write invalid syntax in our Python code, the Python interpreter will throw syntax error. The program will not run and it is the responsibility of the programmer to correct the syntax errors. Only if the errors are corrected, will the program execution start. Let us see the following program: if 1 == 1 print("One")

Output: if 1 == 1 ^ SyntaxError: invalid syntax

Note: The preceding code is covered in (Program Name: Chap04_Example4.01. py)

Concept of Exception Handling in Python



113

In the preceding example, we are missing a simple ‘:’ operator. Let us see one more example: print "I am done"

Output: print "I am done" ^ SyntaxError: Missing parentheses in call to 'print'. Did you mean print("I am done")?

Note: The preceding code is covered in (Program Name: Chap04_Example4.02. py) There must be ‘()’ parenthesis in print statement. So, again Interpreter will throw syntax error. So, it is the responsibility of the programmer to correct these mistakes.

Runtime Error Often, we write the code without any syntax mistakes. However, at runtime, while executing the program, there might be chances of entering the wrong input, memory problem or wrong programming logic. In such cases, runtime errors or exceptions will occur. The best example that every programmer know of is the ZeroDivisionError. Just observe the following code: num1 = int(input("Enter the first number: ")) num2 = int(input("Enter the second number: ")) my_div = num1/num2 print("my_div1 is", my_div)

Output: Enter the first number: 12 Enter the second number: 0 Traceback (most recent call last): File "execption_handling.py", line 3, in

114



Python for Everyone

my_div = num1/num2 ZeroDivisionError: division by zero

Note: The preceding code is covered in (Program Name: Chap04_Example4.03. py) In the preceding program, there is no syntax error. However, when the user is prompted to enter the 2 numbers for division, then the second input was purposefully entered as ‘0’, which was due to an end user input. Hence, ZeroDivisionError will be thrown by Python Interpreter. Let us observe the same example with different end user input this time: Enter the first number: Three Traceback (most recent call last): File "execption_handling.py", line 1, in num1 = int(input("Enter the first number: ")) ValueError: invalid literal for int() with base 10: 'Three'

The user has entered the string ‘Three' which is a string and is invalid literal for int() with base 10. Hence, ValueError will be thrown by Python Interpreter. Thus, if something goes wrong while executing the program at runtime, then the program will be terminated by these runtime errors. The concept of exception handling is applicable only for runtime errors and not for syntax errors. We must know now what an exception is.

Importance of exception handling

Suppose we are watching TV and suddenly our stomach gets upset. In such a scenario, we rush to the toilet and attend nature’s call. This unexpected and unwanted event which disturbed the normal flow of watching TV is an exception for us. In our terms, we will call this as ‘NaturesCall’ Exception. Thus, an unwanted unexpected event which disturbs the normal flow of program is called an exception. For example, ZeroDivisionError, TypeError, ValueError, FileNotFoundError and so on. To prevent disruptions, it is highly recommended to handle these exceptions. We need to properly terminate the program if the exceptions are raised during the execution of the program. This can be understood like this.

Concept of Exception Handling in Python



115

Suppose, we are making a Project Initiation Report and are required to submit the same to our HOD by 2 p.m. in the afternoon. By 1:30 p.m., we are modifying the report in our desktop by aligning the data, changing the font, adding some required inputs and outputs and so on. Suddenly, the power is gone and there is no power back up. In this scenario, our entire work goes away in vain since this is an abnormal termination, as well as a non-graceful one. So, to have a graceful termination, exception handling is required. No resources should be missing. Exception handling does not mean repairing. It means to have an alternative backup for the current problem. Therefore, defining an alternative way to continue the rest of the program normally, is exception handling. Just look at the following 2 statements: • S1: You are feeling hungry. You want to eat rice and chapati to satisfy your hunger.

• S2: You am feeling hungry. You want to eat rice and chapati to satisfy your hunger. If rice and chapati are not cooked, then you will try some snacks. If snacks are also not available, then you will go for biscuits. In S2, there are maximum chances of achieving success since there are multiple ways to satisfy the hunger. However, in S1, if there is no rice and chapati, then unfortunately, your hunger will not be unsatisfied. So, there has to be an alternative way to continue rest of the program normally. In Python, every exception is an object and the corresponding class is available for every exception type. Python Virtual Machine (PVM) creates a corresponding exception object whenever an exception occurs. This object checks for handling code. If available, then it will be executed and rest of the program code will be executed normally. If not available, then PVM will terminate the program abnormally and rest of the program code will not be executed. The corresponding exception information will be displayed to the console. This abnormal termination can be handled explicitly using try-except blocks. Thus, in default exception handling, the program will be terminated abnormally and will print exception information to the console.

Python exception hierarchy

We need to have a basic idea of Python’s exception hierarchy. We should have clarity about various exception classes, of which it is a parent class or a child class. In Python, every exception is an object. There are classes that correspond to each exception type. Thus, all exception classes in Python are child classes of BaseException. Every

116



Python for Everyone

exception class extends BaseClass directly or indirectly. So, for Python exception hierarchy, BaseException acts as a root. The hierarchy is shown in Figure 4.1:

Figure 4.1: Python Exception Hierarchy

From the preceding figure, we can see that ZeroDivisionError is a child of Arithmetic Error. Arithmetic Error is a child of Exception, and Exception is a child of BaseException. We will concentrate more on exception and its child classes as a programmer.

Customized exception handling

It is highly recommended to handle exceptions, as we want graceful termination of the program. We do not want to miss or block any resources. We can handle exceptions using 2 keywords try and except. Observe the following code:

Concept of Exception Handling in Python



117

print("St-1") print(2/0) print("St-2")

Output: St-1 Traceback (most recent call last): File "prog74_debugging.py", line 12, in print(2/0) ZeroDivisionError: division by zero

Note: The preceding code is covered in (Program Name: Chap04_Example4.04. py) In the preceding code, we can see that the ZeroDivisionError occurred on the statement print(2/0). This is one of the risky codes which causes abnormal/nongraceful termination of the code. This code which raised an exception must be inside ‘try’ block. While executing the preceding code, there must be a handling code which must be inside ‘except’ block. If there is a problem in the try block, then the corresponding except block will be executed. Refer to the following code: print("St-1") try: print(2/0) except ZeroDivisionError: print("Normal Execution") print("St-2")

Output: St-1 Normal Execution St-2

118



Python for Everyone

Note: The preceding code is covered in (Program Name: Chap04_Example4.05. py) As we can see, the print(2/0) statement is a risky code which will raise ZeroDivisionError. The handling code will be executed in such cases. Hence, the handling code print("Normal Execution") will be executed and the control goes out of except block and print(“St-2”) will be displayed to the console. It is a normal/ graceful termination which the programmer is recommended to write in the code. Thus, the code which may generate exception must be in try block and the handled code should be in except block. So, the entire summary of the custom exception is: try: risky code except: handling code

Control flow in try-except

Here, we must see how the control will be flowing when the exception is raised or not-raised. Consider the following Python snippet: try: st1 st2 st3 st4 except: st5 st6

Concept of Exception Handling in Python



119

Case-1 - No raising of exception If there will be no exception, then all the statements excluding except block will be executed. The except block will be executed only when there is exception will be raised. So, control flow will be from st1  st2  st3  st4  st6. Thus, it is a normal/ graceful termination.

Case-2 - Exception raised at st3 and corresponding except block is matched In the preceding case, the statements from st1 to st4 will be the main flow and the statement st5 is the handled code flow or alternative flow. If there is an exception in st3, then the control goes to the alternative flow. After executing the statement, the control never comes to the main flow. Rest of the code in the try block will not be executed. Inside try block, only the execution of the risky code must be taken into consideration, and not normal code, as there will be no guarantee for the execution of the entire code. Thus, the length of the code in the try block must be as less as possible. Here, the control flow will be from st1  st2  st5  st6. Therefore, it is a normal/graceful termination.

Case-3 - Exception raised at st3 and corresponding except block is not matched Here, the exception will be raised in the try block but there will be no match in the available except block. Since there is no match in the except block, hence abnormal/ non-graceful termination will be there. Therefore, the control flow will be st1  st2  Abnormal termination.

Case-4 - Exception raised at st5 Exception can be raised in except block also. Any exception raised outside try block is an abnormal termination. Hence, the previous case is an abnormal termination.

Case-5 - Exception raised at st6 Any exception raised outside try block is an abnormal termination. Hence, the previous case is an abnormal termination also. Thus, we can conclude that if there are chances of getting an exception anywhere within the try block, then rest of the try block will not be executed even though the

120



Python for Everyone

exception is handled. So, the length of the try block must be as less as possible and only the risky code must be inside the try block. It is no surprise to get an exception inside except block in addition to try block. If the statement is not a part of the try block and the exception is raised, then it is always an abnormal/ non-graceful termination.

Exception information printing to the console

We can print the exception information to the console based on our usage and need. Just observe the following code: try: print("st1") print(2/0) print("st-3") except Exception as e: print(f"The type of exception is {type(e)}") # E1 print(f"The type of exception is {e.__class__}") # E2 print(f"The exception class name is {e.__class__.__name__}") # E3 print(f"The exception description is {e}") # E4

Output: st1 The type of exception is The type of exception is The exception class name is ZeroDivisionError The exception description is division by zero

Note: The preceding code is covered in (Program Name: Chap04_Example4.06. py) In E1 and E2, we are trying to print the type of exception. ‘e’ is a reference variable to the exception object.

Concept of Exception Handling in Python



121

In E3, the exception class name is displayed to the console. Here, ‘e’ is the exception object pointing to the corresponding class object and name of that class. In E4, the exception description is getting printed to the console.

Try with multiple except blocks

Handling an exception always depends on the type of exception; it is always unique. For every exception type, we have to take separate except block. Thus, it is recommended to use the try block with multiple except blocks. Consider the following example: try: num1= int(input(" Enter the first number: ")) num2= int(input(" Enter the second number: ")) total = num1 / num2 print("The division is ", total) except ValueError: print("It is a Value error") except ZeroDivisionError: print("It is a zero division error")

Output: Case-1 Enter the first number: 32 Enter the second number: 2 The division is

16.0

Case-2 Enter the first number: 32 Enter the second number: 2a It is a Value error

Case-3

122



Python for Everyone

Enter the first number: 32 Enter the second number: 0 It is a zero division error

Note: The preceding code is covered in (Program Name: Chap04_Example4.07. py) In the preceding example, we will see that the Python Virtual Machine (PVM) will always move from top to bottom in the except block, until it matches the except block. The order of except blocks is important if you are writing code with multiple except blocks. In Case-1, we prompted the user to enter 2 integer numbers as 32 and 2. Hence, the output will be 16. In Case-2, we prompted the user to enter the numbers 32 and 2a. Since, the value ‘2a’ cannot be converted to integer, PVM will throw ValueError as wrong input is fed by the user. In Case-3, we prompted the user to enter the numbers 32 and 0. Since, we cannot divide a number by 0, PVM will throw ZeroDivisionError, as wrong input is fed by the user. An important point to observe is that PVM will first give preference to ValueError except block and then to ZeroDivisionError except block. Thus, based on the exception raised, the corresponding except block is going to be executed. The order of the except blocks is important if we are using try with multiple except blocks.

Single except block that can handle multiple exceptions

Up until now, we have seen the single try block which can handle multiple except blocks. However, let us now see the single except block, that can handle multiple exceptions. Each exception inside except block will be separated by comma. Thus, parenthesis is mandatory for exceptions. The group of exceptions is internally considered as tuple. If it is single except block, then parenthesis is not mandatory. try: st1 st2

Concept of Exception Handling in Python



123

except (Exception1,Exception2,…):

In order to print the exception information, use the syntax: try: st1 st2 except (Exception1,Exception2,…) as msg1:

An important point to observe is that msg1 is taken outside of the parenthesis. If handling code is same for multiple exceptions, instead of taking multiple except blocks, we should go for single except block. We can give any name instead of msg1. It totally depends on user perception to write the code. Let us see an example: try: num1= int(input(" Enter the first number: ")) num2= int(input(" Enter the second number: ")) total = num1 / num2 print("The division is ", total) except (ArithmeticError,ValueError) as msg1: print(f"The type of exception is {type(msg1)}") print(f"The exception class name is {msg1.__class__.__name__}") print("The input is not valid")

Output: Case-1 Enter the first number: 12 Enter the second number: 2a The type of exception is The exception class name is ValueError The input is not valid

Case-2 Enter the first number: 12

124



Python for Everyone

Enter the second number: 0 The type of exception is The exception class name is ZeroDivisionError The input is not valid

Note: The preceding code is covered in (Program Name: Chap04_Example4.08. py) In Case-1, we prompted the user to enter the numbers 12 and 2a. Since ‘2a’ is an invalid integer number, the exception ValueError will be thrown by PVM. So, the exception block will be responsible for printing the type of exception, the exception class name, and the message information. In Case-2, we prompted the user to enter the numbers 12 and 0. Since, we cannot divide a number by 0, the exception ZeroDivisionError will be thrown by PVM. So, the exception block will be again responsible to print the type of exception, the exception class name, and the message information.

Default except block

We have discussed about multiple except blocks. But what if no except blocks are matched. In such cases, default except block comes handy. It will be executed if not even one except block is matched. The default except block contains just printing exception information to the console. No specific handling code will be there in the default except block. Default except block must be the last except block; otherwise, we will get ‘Syntax Error’. The syntax is shown as follows: try: st1 st2 except :

Any type of exception is handled by default except block. Just observer the following snippet code: try: num1= int(input(" Enter the first number: ")) num2= int(input(" Enter the second number: "))

Concept of Exception Handling in Python



125

total = num1 / num2 print("The division is ", total) except ValueError as msg1: print("The input is not valid") except: print("Default except block")

Output: Case-1 Enter the first number: 10 Enter the second number: 2a The input is not valid

Case-2 Enter the first number: 3 Enter the second number: 0 Default except block

Note: The preceding code is covered in (Program Name: Chap04_Example4.09. py) In Case-1, the user is prompted to enter 2 numbers. The user entered 10 and ‘2a’. Since, 2a is not an integer, ValueError will be thrown by Python. In Case-2, the user entered the numbers 10 and 0. Since, it is a ZeroDivisionError and the except block corresponding to this exception is not present, the default except block will handle the exception.

Possible combinations of except block The valid syntaxes for the except block are as follows: • except ValueError:

• except (ValueError):

• except ValueError as msg1:

• except (ValueError) as msg1:

126



Python for Everyone

• except (ValueError,ZeroDivisionError):

• except (ValueError,ZeroDivisionError) as msg1: • except:

The invalid syntaxes for the except block are as follows: • except (ValueError as msg1):

• except ValueError,ZeroDivisionError:

• except (ValueError,ZeroDivisionError as msg1): For multiple exceptions, parenthesis is mandatory. For only one exception, parenthesis is optional. If parenthesis is used, then ‘as’ must be outside of parenthesis.

finally except block Till now we have explained the ‘try’ and ‘except’ keywords. Now, we will go for the ‘finally’ keyword. There are chances of a situation occurring where once the connection is open for reading database or excel file, it is mandatory to close the connection. If an error occurs during reading or writing of the contents, then the connection will be wasted and will not be closed if the closing connection is written inside try block. In such cases, it is recommended to use the closing connection inside the finally block. The closing connection is called the resource deallocation or clean up code. It is recommended that it is never used inside try block, because there is no guarantee that every statement inside try block will be executed always. Moreover, we cannot write the cleanup code inside except block because unless and until there is an exception, the except block will not be executed. So, it must be always within the finally block, irrespective of whether the exception is raised or not, and whether the exception is handled or not. The finally block purpose is to maintain clean up code. So, the syntax of try-except-finally block is: try: Code generating an exception or risky code except: handled/alternative code finally: clean up code (Resource deallocation code).

Observe the following possible cases with try-except-finally block: try:

Concept of Exception Handling in Python print("inside try") num1= int(input(" Enter the first number: ")) num2= int(input(" Enter the second number: ")) total = num1 / num2 print("The division is ", total) except ValueError: print("inside except") finally: print("inside finally")

Output: Case-1: If there is no exception inside try Enter the first number: 12 Enter the second number: 2 The division is

6.0

inside finally

Case-2: Exception is raised but handled inside try Enter the first number: 12 Enter the second number: 2a inside except inside finally

Case-3: Exception is raised but not handled inside try Enter the first number: 12 Enter the second number: 0 inside finally



127

128



Python for Everyone

Traceback (most recent call last): File "prog74_debugging.py", line 54, in total = num1 / num2 ZeroDivisionError: division by zero

Note: The preceding code is covered in (Program Name: Chap04_Example4.10. py) In Case-1, it is a normal execution of the code. In Case-2, the exception is raised but is handled by the except block. In Case-3, it is an abnormal termination as the except block does not match the exception. But the finally block is executed. Even in the case of an abnormal termination, finally block is executed. So, in all the 3 cases, we came to know that the finally block is executed. However, there is one case when the finally block would not be executed. PVM will exit explicitly by writing the statement os._exit(0). In this case, PVM will be shut down and finally block would not be executed. Just observe the following program: import os try: print("inside try") print("hello there") os._exit(0) except ValueError: print("inside except») finally: print("inside finally")

Output: inside try hello there

Note: The preceding code is covered in (Program Name: Chap04_Example4.11. py)

Concept of Exception Handling in Python



129

In the previous case, os._exit(0) function is called. PVM will exit explicitly. The finally block here would not be executed. We have imported os module. To interact with the operating system, os module is used. In os._exit(0), 0 means status code and is a normal termination. Non ‘0’ means abnormal termination. The effect will always be the same but this status code will be used by Python for internal logging or reporting purpose. We do not need to worry, being a programmer. We can use any value inside the exit function. The effect will always be same and there will be no difference in the result of the program. Just see the following code: import os try: print("inside try") print("hello there") os._exit(100) except ValueError: print("inside except») finally: print("inside finally")

Output: inside try hello there

Note: The preceding code is covered in (Program Name: Chap04_Example4.12. py)

Control flow in try-except and finally Now, we will see what different cases for the occurrence of exception inside either try block or except block or finally block. Just observe the following demo python snippet code: try: st1

130

Python for Everyone



st2 st3 st4 except xxx: st5 finally: st6 st7

Case-1 - No exception is raised Since there is no exception, all the statements in the try block will be executed followed by finally block and the remaining lines of code. The except block would not be executed. The control flow is as follows: st1  st2  st3  st4  st6  st7. Normal termination

Case-2 - Exception raised at st3 and corresponding except block is matched Since, an exception is raised at st3 and the corresponding except block is matched, the control will move to except block, followed by finally block and the remaining lines of code. It will be a normal termination. st1  st2  st5  st6  st7. Normal termination.

Case-3 - Exception raised at st3 and corresponding except block is not matched Since an exception is raised at st3 and the corresponding except block is not matched, the control will skip the except block and will execute the finally block only. It will be an abnormal normal termination. st1  st2  st6. Abnormal termination.

Concept of Exception Handling in Python



131

Case-4 - Exception raised at st5 If any statement is raising an exception which is not a part of try block, then it is always an abnormal termination. But before abnormal termination, finally block is going to be executed.

Case-5 - Exception raised at st6 or st7 Any exception raised outside try block is an abnormal termination. Hence, the previous case is an abnormal termination also.

Nested try-except finally block We can take try except and finally blocks inside either try or except or finally blocks. Just observe the following demo Python snippet code: try: try: st1 except: st2 finally: st3 except: try: st4 except: st5 finally: st6 finally: try: st7

132



Python for Everyone

except: st8 finally: st9

In the previous pattern, we have taken nested try except and finally block inside each try, except and finally blocks. In general, normal risky code is taken outside outer try block and too much risky code is taken outside inner try block. If an exception is raised inside inner try block, then the inner except block is responsible for handling it. If the inner except block is unable to handle it, then it is the responsibility of the outer except block to handle. Let us see the following example: try: print("inside try block") try: num1= int(input(" Enter the first number: ")) num2= int(input(" Enter the second number: ")) total = num1 / num2 print("The division is ", total) except ValueError: print("Inside ValueError") finally: print("Inside finally block") except ZeroDivisionError: print("Outside except block with ZeroDivision Error") finally: print("Outside finally block")

Output: Case-1 inside try block

Concept of Exception Handling in Python



133

Enter the first number: 12 Enter the second number: 2 The division is

6.0

Inside finally block Outside finally block

Case-2 inside try block Enter the first number: 12 Enter the second number: 2a Inside ValueError Inside finally block Outside finally block

Case-3 inside try block Enter the first number: 12 Enter the second number: 0 Inside finally block Outside except block with ZeroDivision Error Outside finally block

Note: The preceding code is covered in (Program Name: Chap04_Example4.13. py) In Case-1, the user is prompted to enter 2 numbers: 12 and 2. Since the input numbers are valid, the division is 6. 0 will be displayed, followed by displaying the message of inner finally block and outer finally block. In Case-2, the user has entered the numbers 12 and 2a. Since, 2a is an invalid integer number, ValueError will be thrown by PVM, which will be handled by the inside except block. So, the message of except block will be displayed, followed by displaying the message of inner finally block and outer finally block.

134



Python for Everyone

In Case-3, the user has entered the numbers 12 and 0. In this case, since it is to be divided by 0, ZeroDivisionError will be thrown by PVM, which will be handled by outside except block. So, first the inside finally block message will be executed, followed by the message of outside except block and outer finally block. An important point to note is that once the control has not entered in the try block, then the corresponding finally block will not be executed. On entering the control in the try block, the finally block must be executed compulsorily.

Control flow in Nested try-except finally block Now, we shall see the control flow in the nested try-except-finally block. Just observe the following demo Python snippet code: try: st1 st2 st3 try: st4 st5 st6 except X: st7 finally: st8 st9 except Y: st10 finally: st11 st12

Concept of Exception Handling in Python



135

Case-1 - If there is no exception If there is no exception, then the except block would not be getting executed. The control flow is as follows: st1  st2  st3  st4  st5  st6  st8  st9  st11  st12. Normal termination

Case-2 - Exception raised at st3 and corresponding except block is matched In the previous case, exception is raised at st3, and the except block, which is ‘except Y’, will handle the block as it is matched. Hence, the control flow is as follows: st1  st2  st10  st11  st12. Normal termination

Case-3 - Exception raised at st3 and corresponding except block is not matched In the previous case, exception is raised at st3, and the except block which is ‘except Y’, is not matched. Only the outside finally block is executed. Since except block is not matched, it is abnormal termination. Hence, the control flow is as follows: st1  st2  st11. Abnormal termination

Case-4 - Exception raised at st6 and inner except block is matched In the previous case, exception is raised at st6 and the inner except block, that is, except X is matched. So, the inside except block will be executed. Hence, the control flow is as follows: st1  st2  st3  st4  st5  st7  st8  st9  st11  st12. Normal termination

Case-5 - Exception raised at st6 and inner except block is not matched but outer except block is matched In the previous case, exception raised at st6 matches the outer except block. It will be a Normal termination and inside finally block will be executed, and then the control moves to outer except block. The control flow is as follows: st1  st2  st3  st4  st5  st8  st10  st11  st12. Normal termination

136



Python for Everyone

Case-6 - Exception raised at st6 and both inner and outer except block is not matched Since both the except blocks are not matched, it will be an abnormal termination. But both inner and outer finally blocks will be executed. Hence, the control flow is as follows: st1  st2  st3  st4  st5  st8  st11. Abnormal termination

Case-7 - Exception raised at st7 and the corresponding except block is matched Since, the exception is raised at st7 and the corresponding except block is matched, that is, ‘except Y’, it is a normal termination. But the exception at st7 can be raised only if there will be an exception either at st4 or st5 or st6. The inner finally block is executed. Hence, the control flow is as follows: st1  st2  st3  st4 (May or may not be)  st5 (May or may not be)  st6 (May or may not be)  st8  st10  st11  st12. Normal termination

Case-8 - Exception raised at st7 and the corresponding except block is not matched Since, the exception is raised at st7 and the corresponding except block is not matched, that is, ‘except Y’, it is an abnormal termination. But the exception at st7 can be raised only if there will be an exception either at st4 or st5 or st6. The inner finally block is executed. Hence, the control flow is as follows: st1  st2  st3  st4 (May or may not be)  st5 (May or may not be)  st6 (May or may not be)  st8  st11. Abnormal termination

Case-9 - Exception raised at st8 and the corresponding except block is matched In the previous case, the exception may or may not happen at st4, st5 and st6. If exception is not raised, then the control can go to st8 from where the outer except block is matched, that is ‘except Y’. It is a normal termination since the outer except block is matched. If exception is raised, then the control goes to st7. So, the control flow is as follows: st1  st2  st3  st4 (May or may not be)  st5 (May or may not be)  st6 (May or may not be)  st7(May or may not be)  st10  st11  st12. Normal termination.

Concept of Exception Handling in Python



137

Case-10 - Exception raised at st8 and the corresponding except block is not matched In the previous case, the exception may or may not happen at st4, st5 and st6. If exception is not raised, then the control can go to st8 from where the outer except block is matched, that is, ‘except Y’. It is an abnormal termination since the outer except block is not matched. If exception is raised, then the control goes to st7. So, the control flow is as follows: st1  st2  st3  st4 (May or may not be)  st5 (May or may not be)  st6 (May or may not be)  st7(May or may not be)  st11. Abnormal termination.

Case-11 - Exception raised at st9 and the corresponding except block is matched In the previous case, exception is raised at st9 and the outer except block is matched, that is, ‘except Y’, so it is a normal termination. The exception may or may not happen at st4, st5 and st6. If raised, then inner except block will be executed and the control goes to st8. So, the control flow is as follows: st1  st2  st3  st4 (May or may not be)  st5 (May or may not be)  st6 (May or may not be)  st7(May or may not be)  st8  st10  st11  st12. Normal termination.

Case-12 - Exception raised at st9 and the corresponding except block is not matched In the previous case, exception is raised at st9 and the outer except block is not matched, that is, ‘except Y’, so it is an abnormal termination. The exception may or may not happen at st4, st5 and st6. If raised, then inner except block will be executed and the control goes to st8. So, the control flow is as follows: st1  st2  st3  st4 (May or may not be)  st5 (May or may not be)  st6 (May or may not be)  st7(May or may not be)  st8  st11. Abnormal termination.

Case-13 - Exception raised at st10 Exception raised at st10 is an abnormal termination but before abnormal termination, the outer finally block (st-11) is going to be executed.

138



Python for Everyone

Case-14 - Exception raised at st11 or st12 If an exception is raised either at st11 or st12, then it is always an abnormal termination as it is not a part of try block and there is no except block for the exception handling at these statements.

Else block with try-except finally block The else block is a python specific concept. else block will be executed inside tryexcept-finally block. So, the combination will be try-except-else-finally block. Within the try block, if there is no exception, then else block will be going to be executed. Observe the demo Python snippet code: try: risky code (code which generates an exception) except: handled code/alternative code (will be executed if exception is raised in try block) else: will be executed if there is no exception inside try block finally: clean up code or resource deallocation code (will be executed if exception is raised or not If exception is handled or not )

An important point to note is that both except and else are not going to execute simultaneously. If except is going to execute, then else part will not be executing or vice-versa. Just observe the following example: try: print("inside try") num1= int(input(" Enter the first number: ")) num2= int(input(" Enter the second number: "))

Concept of Exception Handling in Python total = num1 / num2 print("The division is ", total) except: print("inside except block") else: print("inside else part") finally: print("inside finally part")

Output: Case-1 inside try Enter the first number: 14 Enter the second number: 2 The division is

7.0

inside else part inside finally part

Case-2 inside try Enter the first number: 14 Enter the second number: 2a inside except block inside finally part

Case-3 inside try Enter the first number: 14 Enter the second number: 0



139

140



Python for Everyone

inside except block inside finally part

Note: The preceding code is covered in (Program Name: Chap04_Example4.14. py) In Case-1, the user is prompted to enter 2 numbers: 14 and 2. Since they are valid inputs, the output “The division is 7.0” will be displayed to the console. As there is no exception inside try part, else block will be executed. At last, the finally block execution will occur. In Case-2, the user entered 2 numbers: 14 and 2a. Since, ‘2a’ is not a valid integer, exception will occur in try block. So, except block will be executed. At last, the finally block execution will occur. In Case-3, the user entered 2 numbers: 14 and 0. Since, any number divided by 0 will result in ZeroDivisionError, exception will occur in try block. So, except block will be executed. At last, the finally block execution will occur. We must know that if we are using else block, the except block must compulsorily be there. Otherwise, PVM will generate syntax error as shown below: try: print("inside try") num1= int(input(" Enter the first number: ")) num2= int(input(" Enter the second number: ")) total = num1 / num2 print("The division is ", total) else: print("inside else part") finally: print("inside finally part")

Output: File "prog74_debugging.py", line 93 else:

Concept of Exception Handling in Python



141

^ SyntaxError: invalid syntax

Note: The preceding code is covered in (Program Name: Chap04_Example4.15. py) We can see that else without except block is invalid.

Conclusion

In this chapter, we first initially learned about syntax and runtime error. Then we learned the importance of exception handling. We learned about handling exceptions using try and except keywords for graceful termination of the program. We saw different use cases for the control flow in try-except, try-except finally block and nested try-except finally block. We also saw an example of single except block handling multiple exceptions with all possible combinations of except block. The default and finally except block is well explained too. Thus, we saw exception handling in this chapter. In the next chapter, we will learn about the concept of regular expressions in Python in greater detail.

Points to remember

• Syntax error is thrown whenever we write invalid syntax in our code, whereas Runtime error is thrown on entering the wrong input or wrong programming logic or memory problem. • Use try and except block for graceful termination of the program.

• It is recommended to use the try block with multiple except blocks. • Moreover, a single except block can handle multiple exceptions. • For cleaning up the code, finally block is used.

• else block can be used inside try-except and finally block.

• The default except block is the last except block where there is no specific handling code.

Questions

1. Explain syntax and runtime error with a Python snippet code. 2. What is the need of exception handling?

142



Python for Everyone

3. Explain with the help of flow chart of Python, exception hierarchy. 4. Explain customized exception handling with an example.

5. Explain the control flow in try-except block with different use cases.

6. Explain try with multiple except blocks with the help of Python example. 7. Explain default except block with the help of Python snippet code.

8. Explain finally except block with the help of Python snippet code. 9. Define possible combinations of except block.

10. Explain the control flow in try-except and finally block with different use cases. 11. Explain the control flow in nested try-except and finally block with different use cases. 12. Explain else block with try-except and finally block.

Join our book's Discord space

Join the book's Discord Workspace for Latest updates, Offers, Tech happenings around the world, New Release and Sessions with the Authors: https://discord.bpbonline.com

Chapter 5

Concept of Regular Expressions in Python Introduction

We all know that any mobile number comes with 10 digits. The numbers may be starting from 9, 8, 7 or even 6. The rest of the 9 digits can be anything. There is some kind of pattern which is followed by these mobile numbers. So, we can say that if there is any requirement to represent a group of strings according to a particular pattern/format, then we can do so with Regular Expressions. So, it is a declarative mechanism for representation of a group of strings, according to a particular pattern. As stated, the regular expressions can be written to represent the mobile numbers, email ids, passwordand so on. The regular expressions findapplications in the areas of validation of logic, translators such as assemblers, compilers (lexical analysis) and so on, as well as areas related to pattern matching, digital circuits, connection, and connection less communication protocols such as TCP/IP, UDP and many more. If we want to use regular expressions, Python uses some special modules known as re module. Several inbuilt functions are contained in this module, to use regular expression very easily in our applications. A regex pattern is a character sequence and is a combination of literals and meta-characters. The literals are the ordinary characters which have no special meaning and is processedas it is. On the other hand, meta-characters are the special characters with some meaning. In this chapter, we will be discussing about the inbuilt functions of regular expressions in details.

144



Python for Everyone

Structure

In this chapter, we will cover the following topics: • compile() • finditer()

• Character classes

• Pre-defined character classes • Quantifiers

• Functions of re module • Metacharacters • r prefix

Objectives

By the end of this chapter, the reader will have an idea about various use cases of regular expression, which will include the concept and syntax of character classes, allowable pre-defined character classes and metacharacters depicting its special meaning. We shall also see the concept of quantifiers, where the number of occurrences to match shall be viewed with various examples in Python. Different functions of re module will also be explained with examples. Important functions such as compile(), finditer() and raw string using ‘r’ or ‘R’ prefix will be neatly explained. By the end of this chapter, you all will have a better understanding of regex expressions.

compile()

This function will compile pattern into pattern objects, which can perform various operations such as string substitution performing or searching for matching of pattern. Refer to the following: import re mypattern=re.compile('re') print(type(mypattern)) # C1 myans=mypattern.findall('regular expressions') print(myans) # C2 print(mypattern.findall('recursive')) # C3

Concept of Regular Expressions in Python



145

chk_name = re.compile(r"[^A-Za-zs.]") myname = input("Kindly, enter the name: ") while chk_name.search(myname): print("Kindly enter the name correctly!") myname = input("Kindly, enter the name: ") # C4

Output:

['re', 're'] ['re'] Kindly, enter the name: saurabh123 Kindly enter the name correctly! Kindly, enter the name: saurabh

Note: The preceding code is covered in (Program Name: Chap05_prog1_compile. py) In C1, the type of variable mypattern is . In C2, a list of string re is returned containing all matches where each string is a single match. Hence, output is ['re', 're']. In C3, only one match is found. Hence, output is ['re']. In C4, we are checking if the user input contains only letters, spaces or (no digits). Any other character is not allowed. We are initially entering the name as ‘saurabh123’. Since the name entered contains digits, the user id prompter to enter the name again. On the 2ndattempt, the correct name ‘saurabh’ without any digit was entered.

finditer()

This function will return an iterator object which yields match object for all nonoverlapping matches. The scanning will be done from left to right. If a match is found, then it will be returned in the order. The following methods can be called on each match object:

146



Python for Everyone

• start(): It will return start index of the match. • end(): It will return end+1 index of the match. • group(): It will return the matched string

In the following example, we are trying to find the pattern ‘10’ in the string ‘1000010000100101’. mymatcher is a callable_iterator object. So, we are using the for loop to get the starting index of the match, ending index of the match as well as the matched string. Refer to the following program: import re mycount = 0 mypattern = re.compile('10') mymatcher = mypattern.finditer('1000010000100101') for loop in mymatcher: mycount += 1 print("starting {}: ,ending index {} and group is {} " .format(loop. start(), loop.end(), loop.group())) print("The total occurrences of pattern '10' is: ", mycount)

Output: starting 0: ,ending index 2 and group is 10 starting 5: ,ending index 7 and group is 10 starting 10: ,ending index 12 and group is 10 starting 13: ,ending index 15 and group is 10 The total occurrences of pattern 10 is:

4

Note: The preceding code is covered in (Program Name: Chap05_prog2_finditer. py) Here, we are searching for only one pattern, which is, ‘10’. Thus, here the group will always return the matched string which is ‘10’. We can simplify the code as shown here: import re mycount = 0

Concept of Regular Expressions in Python



147

mymatcher = re.finditer('10','1000010000100101') for loop in mymatcher: mycount += 1 print("starting {}: ,ending index {} and group is {} start(), loop.end(), loop.group()))

" .format(loop.

print("The total occurrences of pattern '10' is: ", mycount)

Output: starting 0: ,ending index 2 and group is 10 starting 5: ,ending index 7 and group is 10 starting 10: ,ending index 12 and group is 10 starting 13: ,ending index 15 and group is 10 The total occurrences of pattern '10' is:

4

Note: The preceding code is covered in (Program Name: Chap05_prog3_finditer_ code_simplification.py) Here, we are searching for the pattern ‘10’ in the string ‘1000010000100101’. The callable_iterator object is looped to get the result. An important point to note is that we cannot use filename in the secondargument of finditer function. There is a separate way to find the pattern in a file, which we will discuss later.

Character classes

If there is a requirement to search a group of characters, then we should go for character classes. Refer to the following Table 5.1: S No.

Syntax

1

[xyz]

2

[^xyz]

3

[a-z]

4

[A-Z]

Description

A match is returned where one of the specified characters (x, y, or z) is present. A match is returned for any character except x, y, and z.

A match is returned for any lower case character alphabetically between a and z. A match is returned for any upper case character alphabetically between A and Z.

148



Python for Everyone

S No.

Syntax

5

[a-zA-Z]

6

[0-9]

7

[aeiou]

9

[^aeiou]

10

[a-zA-Z0-9]

12

[0123]

11

A match is returned for either upper case or lower case character alphabetically between a and z. A match is returned for any digit between 0 and 9.

[0-6][0-9]

8

Description

A match is returned for any 2 digit numbers from 00 and 69. A match is returned for any one lower case vowel.

A match is returned for character other than a lowercase vowel. A match is returned for any alphanumeric characters.

[^a-zA-Z0-9] A match is returned for special characters (other than alphanumeric characters). A match is returned for any of the digits between 0 and 3.

Table 5.1: Table depicting match for different character classes

Let us see some examples of different character classes. import re mymatcher = re.finditer('[xyz]','djx&AI#82%3U') for loop in mymatcher: print(loop.start(), '.....', loop.group()) # CC1 print("------------------") mymatcher = re.finditer('[^xyz]','djx&AI#82%3U') for loop in mymatcher: print(loop.start(), '.....', loop.group()) # CC2 print("------------------") mymatcher = re.finditer('[a-z]','djx&AI#82%3U') for loop in mymatcher: print(loop.start(), '.....', loop.group()) # CC3 print("------------------") mymatcher = re.finditer('[A-Z]','djx&AI#82%3U') for loop in mymatcher: print(loop.start(), '.....', loop.group()) # CC4

Concept of Regular Expressions in Python print("------------------") mymatcher = re.finditer('[a-zA-Z]','djx&AI#82%3U') for loop in mymatcher: print(loop.start(), '.....', loop.group()) # CC5 print("------------------") mymatcher = re.finditer('[0-9]','djx&AI#82%3U') for loop in mymatcher: print(loop.start(), '.....', loop.group()) # CC6 print("------------------") mymatcher = re.finditer('[0-6][0-9]','djx&AI#82%3U1059') for loop in mymatcher: print(loop.start(), '.....', loop.group()) # CC7 print("------------------") mymatcher = re.finditer('[aeiou]','djx&AI#82%3U') for loop in mymatcher: print(loop.start(), '.....', loop.group()) # CC8 print("------------------") mymatcher = re.finditer('[^aeiou]','djx&AI#82%3U') for loop in mymatcher: print(loop.start(), '.....', loop.group()) # CC9 print("------------------") mymatcher = re.finditer('[a-zA-Z0-9]','djx&AI#82%3U') for loop in mymatcher: print(loop.start(), '.....', loop.group()) # CC10 print("------------------") mymatcher = re.finditer('[^a-zA-Z0-9]','djx&AI#82%3U')



149

150



Python for Everyone

for loop in mymatcher: print(loop.start(), '.....', loop.group()) # CC11

Code explanation with output: In CC1, the characters x, y or z are searched for in the string ‘djx&AI#82%3U’ and is looped (since iterators are always iterables) to get the starting index and the matched items. Hence, output is 2 ..... x In CC2, any character except x, y and z is searched in the string ‘djx&AI#82%3U’ and is looped to get the starting index and the matched items. Hence, output is: 0 ..... d 1 ..... j 3 ..... & 4 ..... A 5 ..... I 6 ..... # 7 ..... 8 8 ..... 2 9 ..... % 10 ..... 3 11 ..... U

In CC3, the lowercase characters alphabetically between a and z are searched in the string ‘djx&AI#82%3U’ and is looped to get the starting index and the matched items. Hence, output is: 0 ..... d 1 ..... j 2 ..... x

In CC4, the uppercase characters alphabetically between a and z are searched in the string ‘djx&AI#82%3U’ and is looped to get the starting index and the matched items. Hence, output is: 4 ..... A

Concept of Regular Expressions in Python



151

5 ..... I 11 ..... U

In CC5, both the uppercase and lowercase characters alphabetically between a and z are searched for, in the string ‘djx&AI#82%3U’ and is looped to get the starting index and the matched items. Hence, output is: 0 ..... d 1 ..... j 2 ..... x 4 ..... A 5 ..... I 11 ..... U

In CC6, any digit between 0 and 9 is searched in the string ‘djx&AI#82%3U’ and is looped to get the starting index and the matched items. Hence, output is: 7 ..... 8 8 ..... 2 10 ..... 3

In CC7, any two digit number between 00 and 69 is searched in the string ‘djx&AI#82%3U1059’ and is looped to get the starting index and the matched items. Hence, output is: 12 ..... 10 14 ..... 59

In CC8, the vowels a, e, i, o and u are searched in the string ‘djx&AI#82%3U’ and is looped to get the starting index and the matched items. Hence, output is None. In CC9, any character other than vowels are searched in the string ‘djx&AI#82%3U’ and is looped to get the starting index and the matched items. Hence, output is: 0 ..... d 1 ..... j 2 ..... x 3 ..... &

152



Python for Everyone

4 ..... A 5 ..... I 6 ..... # 7 ..... 8 8 ..... 2 9 ..... % 10 ..... 3 11 ..... U

In CC10, any alphanumeric character is searched in the string ‘djx&AI#82%3U’ and is looped to get the starting index and the matched items. Hence, output is: 0 ..... d 1 ..... j 2 ..... x 4 ..... A 5 ..... I 7 ..... 8 8 ..... 2 10 ..... 3 11 ..... U

In CC11, the special characters (other than alphanumeric character) are searched in the string ‘djx&AI#82%3U’ and is looped to get the starting index and the matched items. Hence, output is: 3 ..... & 6 ..... # 9 ..... %

Note: The preceding code is covered in (Program Name: Chap05_prog4_ character_classes.py)

Concept of Regular Expressions in Python



153

Pre-defined character classes

Now, we will be discussing about pre-defined character classes. Table 5.2 shows the allowable Pre-defined Character Classes: S No. Character 1

\s

2

\S

3

\d

4

\D

5

\w

6

\W

7

\A

8

\b

9

\B

10

\Z

11

.

Description

A match is returned where the string contains a whitespace character. A match is returned where the string contains any character other than whitespace character. A match is returned where the string contains digits from 0 to 9.

A match is returned where the string contains any character other than digits.

A match is returned where the string contains any word character (characters from 0-9, a-z, A-Z, and underscore).

A match is returned where the string does not contain word characters (special characters). A match is returned if the specified characters are at the beginning of a string. A match is returned if the specified characters are at the beginning or at the end of a word.

A match is returned where the specified characters are present (not at the beginning or at the end of a word).

A match is returned if the specified characters are present at the end of a string. A match is returned where the string contains any character except newline. Table 5.2: Allowable Pre-defined Character Classes

Let us see some examples: import re mymatcher = re.finditer('\s','fx&A I#82%3') for loop in mymatcher: print(loop.start(), '.....', loop.group()) # PC1 print("------------------") mymatcher = re.finditer('\S','fx&A I#82%3') for loop in mymatcher:

154



Python for Everyone

print(loop.start(), '.....', loop.group()) # PC2 print("------------------") mymatcher = re.finditer('\d','fx&A I#82%3') for loop in mymatcher: print(loop.start(), '.....', loop.group()) # PC3 print("------------------") mymatcher = re.finditer('\D','fx&A I#82%3') for loop in mymatcher: print(loop.start(), '.....', loop.group()) # PC4 print("------------------") mymatcher = re.finditer('\w','fx&A I#82%3_') for loop in mymatcher: print(loop.start(), '.....', loop.group()) # PC5 print("------------------") mymatcher = re.finditer('\W','fx&A I#82%3_') for loop in mymatcher: print(loop.start(), '.....', loop.group()) # PC6 print("------------------") mymatcher = re.finditer('\AThe','The Corona Virus in world') for loop in mymatcher: print(loop.start(), '.....', loop.group()) # PC7 print("------------------") mymatcher = re.finditer(r"rld\b",'The Corona Virus in world') for loop in mymatcher: print(loop.start(), '.....', loop.group()) # PC8 print("------------------")

Concept of Regular Expressions in Python



155

mymatcher = re.finditer(r'\Bect','A Temporary effect') for loop in mymatcher: print(loop.start(), '.....', loop.group()) # PC9 print("------------------") mymatcher = re.finditer('effect\Z','A Temporary effect') for loop in mymatcher: print(loop.start(), '.....', loop.group()) # PC10 print("------------------") mymatcher = re.finditer('.','fx&A\nI#82%3') for loop in mymatcher: print(loop.start(), '.....', loop.group()) # PC11

Code explanation with output: In PC1, a whitespace character is searched for in the string ‘‘fx&A I#82%3” and is looped to get back the starting index and the matched items. Hence, output is 4 ..... . In PC2, any character apart from whitespace character is searched in the string ‘‘fx&A I#82%3” and is looped to get back the starting index and the matched items. Hence, output is: 0 ..... f 1 ..... x 2 ..... & 3 ..... A 5 ..... I 6 ..... # 7 ..... 8 8 ..... 2 9 ..... % 10 ..... 3

156



Python for Everyone

In PC3, digits from 0-9 is searched in the string ‘‘fx&A I#82%3” and is looped to get back the starting index and the matched items. Hence, output is: 7 ..... 8 8 ..... 2 10 ..... 3

In PC4, any character apart from the digits 0-9 is searched in the string ‘‘fx&A I#82%3” and is looped to get back the starting index and the matched items. Hence, output is: 0 ..... f 1 ..... x 2 ..... & 3 ..... A 4 ..... 5 ..... I 6 ..... # 9 ..... %

In PC5, alphanumeric characters including _ (underscore) is searched in the string ‘‘fx&A I#82%3_” and is looped to get back the starting index and the matched items. Hence, output is: 0 ..... f 1 ..... x 3 ..... A 5 ..... I 7 ..... 8 8 ..... 2 10 ..... 3 11 ..... _

In PC6, any character apart from alphanumeric characters and underscore is searched in the string ‘‘fx&A I#82%3_” and is looped to get back the starting index and the matched items. Hence, output is:

Concept of Regular Expressions in Python



157

2 ..... & 4 ..... 6 ..... # 9 ..... %

In PC7, the specified characters ‘The’ will be checkedat the beginning of the string “The Corona Virus in the world” and is looped to get back the starting index and the matched item. Hence, output is: 0 ..... The

In PC8, the specified characters ‘rld’ will be checkedat the beginning or at the end of the string “The Corona Virus in the world” and is looped to get back the starting index and the matched item. Hence, output is: 22 ..... rld

In PC9, the specified character ‘ect’ is checked if present but not at the beginning of the string “A temporary effect” and is looped to get back the starting index and the matched item. Hence, output is: 15 ..... ect

In PC10, the specified characters ‘effect’ will be checked if it ends in the string “A temporary effect” and is looped to get back to the starting index and the matched item. Hence, output is: 12 ..... effect

In PC11, any character except new line is searched in the string ‘‘fx&A\nI#82%3_” and is looped to get back the starting index and the matched items. Hence, output is: 0 ..... f 1 ..... x 2 ..... & 3 ..... A 5 ..... I 6 ..... # 7 ..... 8 8 ..... 2

158



Python for Everyone

9 ..... % 10 ..... 3

Note: The preceding code is covered in (Program Name: Chap05_prog5_pre_ defined_character_classes.py)

Quantifiers

Whenever there is a requirement to specify the number of occurrences to match, we will be using quantifiers. In other words, it is a mechanism to define how a character, character set and metacharacter can be repeated in the pattern. Suppose a regular expression pattern is definedand we specify that a character should be repeated 4 times. In this scenario, we can either write the character 4 times or we can use the concept of quantifiers. Refer to the following Table 5.3: SNo.

Syntax

Description

1

x

A match is returned for exactly single x.

2

x+

A match is returned for the expression to its left 1 or more times. In other words, it is searching for at least single x.

3

x*

A match is returned for the expression to its left 0 or more times. In other words, it is searching for any number of x including zero number of x’s.

4

x?

A match is returned for the expression to its left 0 or 1 time. In other words, it is searching for at most one ‘x’, that is, either single x or zero number of x’s.

5

x{m}

A match is returned for the expression to its left m times and not less. In other words, exactly m number of x’s.

6

x{m,n}

A match is returned for the expression to its left m to n times and not less. In other words, minimum m number of x’s and maximum n number of x’s.

7

^x

A match is returned for the expression to its right at the start of a string. Every such instance is matched before each \n in the string. In other words, it checks whether the target string starts with x or not.

8

x$

A match is returned for the expression to its left at the start of a string. Every such instance is matched before each \n in the string. In other words, it checks whether the target string ends with x or not. Table 5.3: Table depicting quantifiers

Concept of Regular Expressions in Python

Let us see some examples of quantifiers: import re mymatcher = re.finditer('x','xyxxyxxxyxxxxy') for loop in mymatcher: print(loop.start(), '.....', loop.group()) #Q1 print("------------------") mymatcher = re.finditer('x+','xyxxyxxxyxxxxy') for loop in mymatcher: print(loop.start(), '.....', loop.group()) #Q2 print("------------------") mymatcher = re.finditer('x*','xyxxyxxxyxxxxy') for loop in mymatcher: print(loop.start(), '.....', loop.group()) #Q3 print("------------------") mymatcher = re.finditer('x?','xyxxyxxxyxxxxy') for loop in mymatcher: print(loop.start(), '.....', loop.group()) #Q4 print("------------------") mymatcher = re.finditer('x{3}','xyxxyxxxyxxxxy') for loop in mymatcher: print(loop.start(), '.....', loop.group()) #Q5 print("------------------") mymatcher = re.finditer('x{3,5}','xyxxyxxxyxxxxy') for loop in mymatcher: print(loop.start(), '.....', loop.group()) #Q6



159

160



Python for Everyone

print("------------------") mymatcher = re.finditer('^x','xyxxyxxxyxxxxy') for loop in mymatcher: print(loop.start(), '.....', loop.group()) #Q7 print("------------------") mymatcher = re.finditer('y$','xyxxyxxxyxxxxy') for loop in mymatcher: print(loop.start(), '.....', loop.group()) #Q8

Code explanation with output: In Q1, exactly a single character ‘x’ is searched in the string ‘‘xyxxyxxxyxxxxy” and is looped to get back the starting index and the matched items. Hence, output is: 0 ..... x 2 ..... x 3 ..... x 5 ..... x 6 ..... x 7 ..... x 9 ..... x 10 ..... x 11 ..... x 12 ..... x

In Q2, an occurrence of a character ‘x’ one or more times (at least one x) is searched in the string ‘‘xyxxyxxxyxxxxy” and is looped to get back the starting index and the matched items. Hence, output is: 0 ..... x 2 ..... xx 5 ..... xxx 9 ..... xxxx

Concept of Regular Expressions in Python



161

In Q3, an occurrence for any number of character x’s including 0 number of x is searched in the string ‘‘xyxxyxxxyxxxxy” and is looped to get back the starting index and the matched items. Hence, output is: 0 ..... x 1 ..... 2 ..... xx 4 ..... 5 ..... xxx 8 ..... 9 ..... xxxx 13 ..... 14 .....

An important point to note is that even though the string is terminated with character ‘y’ at index position 13, it will also check for index 14 (last index position +1) which is nothing, that is, 0 number of x. That is why, here the index position 14 is displayed. In Q4, an occurrence for, at most, one number of character x is searched in the string ‘‘xyxxyxxxyxxxxy” and is looped to get back the starting index and the matched items. Hence, output is: 0 ..... x 1 ..... 2 ..... x 3 ..... x 4 ..... 5 ..... x 6 ..... x 7 ..... x 8 ..... 9 ..... x 10 ..... x

162



Python for Everyone

11 ..... x 12 ..... x 13 ..... 14 .....

In Q5, exactly three numbers of character x is searched in the string ‘‘xyxxyxxxyxxxxy” and is looped to get back the starting index and the matched items. Hence, output is: 5 ..... xxx 9 ..... xxx

In Q6, minimum 3 numbers of character x and maximum 5 numbers of x is searched in the string ‘‘xyxxyxxxyxxxxy” and is looped to get back the starting index and the matched items. Hence, output is: 5 ..... xxx 9 ..... xxxx

In Q7, a string ‘‘xyxxyxxxyxxxxy” is checked whether it starts with character x or not, and is looped to get back the starting index and the matched item. Hence, output is 0 ..... x. In Q8, a string ‘‘xyxxyxxxyxxxxy” is checked whether it ends with character y or not, and is looped to get back the starting index and the matched item. Hence, output is 13 ..... y. Note: The preceding code is covered in (Program Name: Chap05_prog6_ quantifiers.py)

Functions of re module

We have already discussed compile() and finditer() function. Apart from the previous two functions, we will be discussing some more important functions.

match This function will check for matching of the given pattern only at the beginning of the target string. A match object is returned if the match is available. Otherwise, we will get None.

Concept of Regular Expressions in Python



163

The syntax of re.match() is: re.match(pattern, string, flags=0)

The first parameter is the pattern whose regular expression is to be matched. The second parameter is the string where the pattern is to be matched. The third parameter is optional flags, where we can specify different flags such as re.IGNORECASE. Refer to the following code: import re mypattern=input("Please write the pattern to check: ") mymatch=re.match(mypattern,"rstuvwxyz") if mymatch!= None: print("We have found match at the beginning of the string") p rint("Start Index is :",mymatch.start(), "and End Index is :",mymatch.end()) else: print("We have not found match at the beginning of the string")

Output: Case-1 Please write the pattern to check: rstu We have found match at the beginning of the string Start Index is : 0 and End Index is : 4

Case-2 Please write the pattern to check: rstuvwxyz We have found match at the beginning of the string Start Index is : 0 and End Index is : 9

164



Python for Everyone

Case-3 Please write the pattern to check: qrst We have not found match at the beginning of the string

Note: The preceding code is covered in (Program Name: Chap05_prog7_re_ match.py) In Case-1, we have entered the pattern ‘rstu’. It is attempted to match the previous pattern at the beginning of the target string ‘rstuvwxyz’. It is matchedand the match object is returnedand the start index (0), and end index (4), are displayed. Hence, output is displayedas shown above. In Case-2, we have entered the pattern ‘rstuvwyxyz’. It is attempted to match the previous pattern at the beginning of the target string ‘rstuvwxyz’. It is matchedand the match object is returnedand the start index (0), and end index (9), are displayed. Hence, output is displayedas shown above. In Case-3, we have entered the pattern ‘qrst’. It is attempted to match the previous pattern at the beginning of the target string ‘rstuvwxyz’. It is not matched. Hence, output is displayedas shown above. An important point to note is that the pattern is matched only at the beginning of the string and not at the beginning of each line even in MULTILINE mode.

fullmatch This function will check for matching of the given pattern to the whole string. A match object is returned if the match is available. Otherwise, we will get None. The syntax of re.fullmatch() is: re.fullmatch(pattern, string, flags=0) import re mypattern=input("Please write the pattern to check: ") mymatch=re.fullmatch(mypattern,"rstuvwxyz") if mymatch!= None: print("We have found match at the whole string")

Concept of Regular Expressions in Python



165

p rint("Start Index is :",mymatch.start(), "and End Index is :",mymatch.end()) else: print("We have not found match at the whole string")

Output: Case-1 Please write the pattern to check: rstu We have not found match at the whole string

Case-2 Please write the pattern to check: rstuvwxyz We have found match at the whole string Start Index is : 0 and End Index is : 9

Case-3 Please write the pattern to check: rstuvwxyza We have not found match at the whole string

Note: The preceding code is covered in (Program Name: Chap05_prog8_re_ fullmatch.py) In Case-1, we have entered the pattern ‘rstu’. It is attempted to match the previous pattern at the entire string ‘rstuvwxyz’. It is not matched. Hence, output is displayedas shown above. In Case-2, we have entered the pattern ‘rstuvwyxyz’. It is attempted to match the previous pattern at the entire string ‘rstuvwxyz’. It is matchedand the match object is returnedand the start index (0) and end index (9) is displayed. Hence, output is displayedas shown above. In Case-3, we have entered the pattern ‘rstuvwxyza’. It is attempted to match the previous pattern at the entire string ‘rstuvwxyz’. It is not matched. Hence, output is displayedas shown above.

166



Python for Everyone

search This function will check the given pattern in the target string. A match object is returned if the match is available, representing the first occurrence of the match. None is returned if no match is found. It is best suited for regular expression, testing more than data extraction as it stops after the first match. The syntax of re.search() is re.search(pattern, string, flags=0) import re mypattern=input("Please write the pattern to check: ") mymatch=re.search(mypattern,"101101110") if mymatch!= None: print("We have found match") p rint("Start Index is :",mymatch.start(), "and End Index is :",mymatch.end()) else: print("We have not found the match")

Output: Case-1 Please write the pattern to check: 111 We have found match Start Index is : 5 and End Index is : 8

Case-2 Please write the pattern to check: 1010 We have not found the match

Case-3 Please write the pattern to check: 0110

Concept of Regular Expressions in Python



167

We have found match Start Index is : 1 and End Index is : 5

Note: The preceding code is covered in (Program Name: Chap05_prog9_re_ search.py) In Case-1, we have entered the pattern ‘111’. It attempts to match the first occurrence of the pattern in the target string ‘101101110’. It is matchedand the match object is returned, and the start index (5), and end index (8) is displayed. Hence, output is displayedas shown above. In Case-2, we have entered the pattern ‘1010’. It is attempted to match the first occurrence of the pattern in the target string ‘101101110’. It is not matched. Hence, output is displayedas shown above. In Case-3, we have entered the pattern ‘0110’. It is attempted to match the first occurrence of the pattern in the target string ‘101101110’. It is matchedand the match object is returnedand the start index (1) and end index (5) are displayed. Hence, output is displayedas shown above.

findall This function will return a list object containing all the non-overlapping match pattern. The scanning of the string is done left to right. If match is found, then it will be returned in the order. The syntax of re.findall() is: re.findall(pattern, string, flags=0) import re myl1=re.findall("[a-z]","b4j5k67ap0") print(myl1)

Output: ['b', 'j', 'k', 'a', 'p']

Note: The preceding code is covered in (Program Name: Chap05_prog10_re_ findall.py)

168



Python for Everyone

In the previous example, all the lowercase characters occurrences of the match are returnedas a list object.

sub This function stands for substitution or replacement. The syntax of re.sub() is: re.sub(pattern, repl, string, count=0, flags=0)

A certain regular expression pattern (1st parameter) is searched in the string mentioned (3rd parameter) and upon finding the match, occurrences are replaced with the replace variable content (2nd parameter). The count (4th variable) is then checkedand the number of times this occurred, is maintained. The original string is returned if the pattern is not found. Refer to the following example: import re mytxt = '123 \n123 gfh' mypattern = '\s' myreplace = '' mystr = re.sub(mypattern, myreplace, mytxt) print(mystr) # SUB1 print("-------------") mystr = re.sub(mypattern, myreplace, mytxt,count =1) print(mystr) # SUB2 print("-------------") print(re.sub('st', '*#' , 'Stay safe, stay healthy', flags = re.IGNORECASE)) # SUB3 print("-------------") print(re.sub('st', '*#' , 'Stay safe, stay healthy')) # SUB4 print("-------------") print(re.sub('st', '*#' , 'Stay safe, stay healthy', count = 1, flags = re.IGNORECASE)) # SUB5

Concept of Regular Expressions in Python



169

print("-------------") print(re.sub(r'\sAND\s', ' & ' , 'The prince and the pauper', flags = re.IGNORECASE)) # SUB6

Code explanation with Output: In SUB1, we are trying to match all the whites space characters in the target string ‘123 \n123 gfh’ and is replaced by an empty string ‘’ when match found. The count parameter is default 0 and will replace all the matched occurrences. Hence, output will be 123123gfh. In SUB2, we are trying to match all the whites space characters in the target string ‘123 \n123 gfh’, and they are replaced by an empty string ‘’ when match is found. The count parameter is 1, and so the maximum times that replacement occurs, is 1. Due to presence of \n, the output after replacement of whitespace once only is: 123 123 gfh

In SUB3, the regular expression pattern ‘st’ matches the substring at “Stay” and “stay”. The case has been ignored due to the flag “st”, will match twice with the string. The “st” will be replaced by “*#” in both the substrings. Hence, output is *#ay safe, *#ay healthy. In SUB4, we are considering the case sensitivity “st” in “Stay” which will not be replaced. Hence, output is Stay safe, *#ay healthy. In SUB5, the count parameter is 1. So, the maximum time that replacement can be done is 1. Also, the case is ignored. Hence, output is *#ay safe, stay healthy. In SUB6, the letter before the pattern denotes re, \s is for start and end of a string. AND is replaced by ‘ & ‘. Hence, output is The prince & the pauper. Note: The preceding code is covered in (Program Name: Chap05_prog11_re_sub. py)

subn This function is like the sub function, except that it returns a tuple of 2 arguments. The first argument is the new string, rather than just the string after the replacement and the other is the number of substitutions or replacements done (count value).

170



Python for Everyone

Refer to the following example: import re mytxt = '123 \n123 gfh' mypattern = '\s' myreplace = '' mystr = re.subn(mypattern, myreplace, mytxt) print(mystr) # SUBN1 print(type(mystr)) print(mystr[0]) print(mystr[1]) print("-------------") mystr = re.subn(mypattern, myreplace, mytxt,count =1) print(mystr) # SUBN2 print("-------------") print(re.subn('st', '*#' , 'Stay safe, stay healthy', flags = re.IGNORECASE)) # SUBN3 print("-------------") print(re.subn('st', '*#' , 'Stay safe, stay healthy')) # SUBN4 print("-------------") print(re.subn('st', '*#' , 'Stay safe, stay healthy', count = 1, flags = re.IGNORECASE)) # SUBN5 print("-------------") print(re.subn('\sAND\s', ' & ' , 'The prince and the pauper', count = 1, flags = re.IGNORECASE)) # SUBN6

Code explanation with Output: In SUBN1, we are trying to match all the whites space characters in the target string ‘123 \n123 gfh’, and they are replaced by an empty string ‘’ when match is found. The count parameter is default 0 and will replace all the matched occurrences. The 1st

Concept of Regular Expressions in Python



171

parameter will be the new string after replacement and the 2nd parameter is the total number of substitutions done, which is 3. Hence, output will be ('123123gfh', 3). Here, the class is tuple and the 1st and 2nd element is displayedas shown:

123123gfh 3

In SUBN2, we are trying to match all the whites space characters in the target string ‘123 \n123 gfh’ and they are replaced by an empty string ‘’ when match found. The count parameter is 1, so the maximum time replacement occurs is 1. Hence, output is ('123\n123 gfh', 1). In SUBN3, the regular expression pattern ‘st’ matches the substring at “Stay” and “stay”. The case has been ignored due to the flag; “st” will match twice with the string. The “st” will be replaced by “*#” in both the substrings. The number of times substitution done is 2. Hence, output is ('*#ay safe, *#ay healthy', 2). In SUBN4, we are considering the case sensitivity “st” in “Stay” which will not be replaced. The number of times substitution is done, is 1. Hence, output is ('Stay safe, *#ay healthy', 1). In SUBN5, the count parameter is 1. So, the maximum time replacement can be done is 1. Also, the case is ignored. The number of times substitution done is 1. Hence, output is ('*#ay safe, stay healthy', 1). In SUBN6, the letter before the pattern denotes re, \s is for start and end of a string. AND is replaced by ‘& ‘. The number of times substitution done is 1. Hence, output is ('The prince & the pauper', 1). Note: The preceding code is covered in (Program Name: Chap05_prog12_re_ subn.py)

split This function will spilt the mentioned target string according to a particular pattern. The syntax of re.split() is: re.split(pattern, string, maxsplit=0, flags=0)

The first parameter is the regular expression pattern.

172



Python for Everyone

The second parameter is the target string where the pattern will be searched for and in which splitting will occur. The third parameter is the maxsplit. If provided, then almost that many splits will occur. Otherwise, its default value is 0. Let us assume that the maxsplit parameter is mentionedas 1. In that case, the string will be splitting once, resulting in a list of length 2. The fourth parameter is optional flags, in which we can specify different flags such as re.IGNORECASE. Refer to the following example: import re mymatch=re.split('\s',"Python is awesome") print(type(mymatch)) # SP1 print(mymatch) # SP2 for loop in mymatch: # SP3 print(loop) mytxt = "www.abc.com" mymatch=re.split('\.',mytxt) print(mymatch) # SP4 print(re.split('\W+', 'Fruits, fruits , Fruits')) # SP5 print(re.split('\W+', "Fruit's fruits Fruits")) # SP6 print(re.split('\W+', '25th March 2020, at 12:02 PM')) # SP7 print(re.split('\d+', '25th March 2020, at 12:02 PM')) # SP8

Code explanation with Output: In SP1, the type of mymatch object is . In SP2, a list is returned where the string has been split at each white space character. Hence, output is ['Python', 'is', 'awesome']. In SP3, we are using the for loop to iterate each element of the list object. Hence, output is:

Concept of Regular Expressions in Python



173

Python is awesome

In SP4, a list is returned where the string has been split at each ‘.’ character. Hence, output is: ['www', 'abc', 'com']

In SP5, a list is returned where the string has been split at ‘,’ or ‘ ‘ (whitespace) character. Hence, output is ['Fruits', 'fruits', 'Fruits']. In SP6, a list is returned where the string has been split at Non-Alphanumeric Characters. Here, at ‘’’ (apostrophe) and ‘ ‘ (white space). Hence, output is ['Fruit', 's', 'fruits', 'Fruits']. In SP7, a list is returned where the string has been split at Non-Alphanumeric Characters at ‘:’, ‘,’ and ‘ ‘ . Hence, output is ['25th', 'March', '2020', 'at', '12', '02', 'PM']. In SP8, a list is returned where splitting occurs at numeric characters. Here, at ‘25’, ‘2020’, ‘12’ and ‘02’. Hence, output is ['', 'th March ', ', at ', ':', ' PM']. Note: The preceding code is covered in (Program Name: Chap05_prog13_re_ split.py)

escape This function will return a string with backslash before every non-alphanumeric character. It comes handy when there is an arbitrary literal string which may have regular expression metacharacters in it. The syntax of re.escape() is: re.escape(pattern)

Refer to the following example: import re print(re.escape("Congratulations to all the team members")) # ES1 print(re.escape("https://www.webmail.in")) # ES2

174



Python for Everyone

print(re.escape("One of the set in regex is [a-z]. I am ^ /n flatterred")) # ES3

Code explanation with output: In ES1, a whitespace character is not alphanumeric. So, a backslash character will be used before every white space. Hence, output is Congratulations\ to\ all\ the\ team\ members. In ES2, ‘.’ is not alphanumeric. So, a backslash character will be used before every ‘.’ character. Hence, output is https://www\.webmail\.in. In ES3, ‘ ‘, caret ‘^’, ‘-’ and ‘[]’ are not alphanumeric. So, a backslash character will be used before these characters. Hence, output is One\ of\ the\ set\ in\ regex\ is\ \[a\-z\]\.\ I\ am\ \^\ /n\ flattered. Note: The preceding code is covered in (Program Name: Chap05_prog14_re_ escape.py)

Metacharacters

Metacharacters are characters with a special meaning andare interpreted in a special way by a RegEx engine. We will be discussing about the metacharacters one by one.

[] (Square brackets) A set of characters can be specified that we wish to match. Refer to Table 5.4: Expression

String x

[xyz]

xz Hello James xyz abc zx

Matched ?

1 Match

2 Matches No Match

5 Matches

Table 5.4: Table depicting square brackets meta character

In the previous example, [xyz] will match if the target string contains any of the x, y, or z. A range of characters can be specified inside square brackets using -, as follows: • [d-g] is same as [defg]

• [0-8] is same as [012345678]

Concept of Regular Expressions in Python



175

• The character set can be inverted using ^ symbol at the start of the square bracket. • [^xyz] means any character except x , y, or z.

• [^0-9] means any non-digit character (any character except digits from 0 to 9)

. (Period) This metacharacter will match any character except new line (‘\n’). Refer to Table 5.5: Expression

String x

1 Match

xz

..

No Match

Matched ?

1 Match

xyz

2 Matches (containing 4 characters)

wxyz

Table 5.5: Table depicting period meta character

^ (Caret) This metacharacter will check whether the target string starts with the provided pattern or not. A match object is returned if the target strings starts with the provided pattern. Otherwise, it returns None. Refer to Table 5.6: Expression

String x

^x

xyz yzx wxyz

^xy

xyz xzy

1 Match

Matched ?

1 Match

No Match No Match 1 Match

No Match (starts with x but not followed by y)

Table 5.6: Table depicting caret meta character

$ (Dollar) This metacharacter will check whether the target string ends with the provided pattern or not. A match object is returned if the target strings ends with the provided pattern, otherwise, it returns None. Refer to Table 5.7:

176



Python for Everyone Expression x$ xy$

String x xyz xyz zxy

Matched ?

1 Match

No Match No Match 1 Match

Table 5.7: Table depicting dollar meta character

* (Star) This metacharacter will match the expression to its left, 0 or more times. Refer to Table 5.8: Expression

String spn

spa*n

1 Match

Matched ?

1 Match

span

1 Match

spaaan

No Match (character a is not followed by character n)

spain

Table 5.8: Table depicting star meta character

+ (Plus) This metacharacter will match the expression to its left, 1 or more times. Refer to Table 5.9: Expression

String spn

spa+n

span spaaan spain

Matched ?

No Match (as no ‘a’ character) 1 Match 1 Match

No Match (character a is not followed by character n)

Table 5.9: Table depicting plus meta character

Concept of Regular Expressions in Python



177

? (Question Mark) This metacharacter will match the expression to its left, 0 or 1 time. Refer to Table 5.10: Expression

String spn

spa?n

span spaaan spain

Matched ?

1 Match 1 Match

No Match (more than one a character)

No Match (character a is not followed by character n)

Table 5.10: Table depicting question mark meta character

{} (Braces) This metacharacter {a,b} will match the expression to its left, a to b times and not less. Refer to Table 5.11: Expression

String

No match

xyz abc

1 Match (at xxy)

vw xxy

x{2,3}

Matched ?

2 Matches (at xxyz and wxxxy)

xxyz wxxxy

2 Matches (at xxyz and wxxxxy)

xxyz wxxxxy [0-9]{2,3} (will match at least 2 xy34abs digits but not more than 3 digits) 76 and 765432

1 Match at xy34abs

2 Matches (at 76 and 765432) No match

3 and 4

Table 5.12: Table depicting braces meta character

| (Alternation): This metacharacter vertical bar is used for alteration (or operator). Refer to Table 5.12: Expression

String

x|y (a string is matched which contains either x wst or y) xza

xzaybc

Matched ?

No match

1 Match (at xza)

2 Matches (at xzaybc)

Table 5.13: Table depicting alternation meta character

178



Python for Everyone

() (Group) This metacharacter parenthesis is used to group sub-patterns. Refer to Table 5.13: Expression

String

(x|y|z)ac (a string is matched which xy ac contains either x or y or z followed by xyac ac) xac and zyxac

Matched ?

No match

1 Match (at xyac)

2 Matches (at xac and zyxac)

Table 5.14: Table depicting group meta character

\ (Backslash) The character \ will escape various characters including metacharacters. For example, \$x will match if a string has $, followed by character x. Here, interpretation of $ will not be done in a special way. So, just put backslash and the character will not be treated in a special way. We have already discussedabout pre-defined character classes using backslash.

r prefix

We have seen the use of r or R prefix before a regular expression, quite often. It means raw string. For example, \n means new line whereas r'\n' means there are 2 characters, one backslash followed by character n. Refer to the following example: import re mystr = '\n and \t are escape sequences.' myres = re.findall(r'[\n\t]', mystr) print(myres)

Output: ['\n', '\t']

Note: The preceding code is covered in (Program Name: Chap05_prog15_r_ prefix.py)

Concept of Regular Expressions in Python



179

Conclusion

In this chapter, we started with 2 important functions: compile() and finditer() function. Then we saw about different character classes syntax followed by allowable pre-defined character classes with well-illustrated Python code. We dig up into the concept of quantifiers and saw different functions of ‘re’ module. Then we learnedabout different metacharacters’ importance with its applicational usage. Finally, at the end, we saw the use of ‘r’ raw string prefix before a regular expression. In the next chapter, we shall be learning about functions concept importance with various examples in detail

Points to remember

• compile() function is used for compiling pattern into pattern objects.

• finditer() function will return an iterator object which yields a match object for all non-overlapping matches. • Applicational usage of character andallowable pre-defined character classes is a must to know for various Python code applicational requirements.

• Whenever there is a need for specifying the number of occurrences to match, then we shall be using quantifiers. • Observe various examples and concepts of re module functions which was explained elegantly with a Python snippet code.

• Make sure that metacharacters are the characters with a special meaning and has a special way of interpretation by a RegEx engine. • raw string uses either ‘r’ or ‘R’ prefix before a regular expression.

Questions

1. Name the function which can compile pattern into pattern objects such as matching pattern or string substitution. Explain with a Python code. 2. Name the function which returns an iterable object which yields match object for all non-overlapping matches. Explain with a Python code. 3. Explain different syntax of character classes with a Python snippet code.

4. Explain different allowable pre-defined character classes with a Python snippet code. 5. Explain different syntax of quantifiers with a Python snippet code.

180



Python for Everyone

6. Explain the following functions with syntax anda Python snippet code. a. match() b. fullmatch() c. findall() d. search() e. sub() f. subn() g. split() h. escape() 7. Explain different metacharacters with a Python snippet code.

8. What is the importance of r-prefix in regular expressions? Explain with a Python snippet code.

Join our book's Discord space

Join the book's Discord Workspace for Latest updates, Offers, Tech happenings around the world, New Release and Sessions with the Authors: https://discord.bpbonline.com

Chapter 6

Concept of Functions in Python Introduction

Sometimes during programming a code, we are required to use the same code again and again. As per the Don’t Repeat Yourself (DRY) concept, the same code should not be repeated multiple times during programming. Nonetheless, sometimes we need to use a block of code multiple times. In such cases, functions in Python come handy. Functions play a huge role in our Python code. An important point to note is that functions must be defined initially before being later called in Python with proper indentation. Functions in Python are important as a coder, because DRY concept must be respected, while creating any big project makes that any 3rd party coder understand its importance.

Structure

In this chapter, we will discuss the following topics: • Functions in Python • Function types

• Function arguments

182



Python for Everyone

• Nested function • Python closures

• Function passing as a parameter

• Local, global, and non-local variables • Recursive function

• Python lambda functions

Objectives

By the end of this chapter, the reader will have an idea about what exactly function is in Python. They will have a taste of various arguments such as positional, keyword, default, *args and **kwargs. The concept of local, global, and non-local variables will be well explained. We shall see how to bind the data to a function without passing them as parameters called closures. One function will be defined inside another function using nested function concept. We shall see recursive function, that is, a function calling itself. Anonymous functions taking any number of arguments with one expression will be seen by exploring the concept of lambda functions.

Functions in Python

It is not recommended to write group of statements repeatedly to execute the program. The group of statements will be defined as a single unit, thus breaking our program into smaller and modular chunks. We name this group of statements as functions, which performs a specific task. Code reusability is possible and repetition is avoided. We will be writing function only once, but the function will be called multiple times. The basic syntax of function is as follows: def func_name(parameters): """ docstring """ statement(s)

Let us explain the significance of each component in the preceding syntax: • def is a keyword which marks the function header start.

• A unique name to identify function which follows the same rule of writing identifiers in Python. • Parameters (arguments) are optional, through which we pass values to a function. • A colon operator (:) to mark the end of the function header.

Concept of Functions in Python



183

• Docstring is optional to give a small description of what the function does. A documentation string is used to explain in brief about the function. Generally, docstring is written using triple quotes which can be extended up to multiple lines. The docstring is also available to us as __doc__ attribute of the function. • Function body contains one or more valid Python statements. An important point to note is that the statements must have the same indentation level (usually 4 spaces).

• An optional return statement at the end so that a value can be returned from the function. This statement will exit from the function and the control will come back to the place from where it was called. If there is no return statement itself inside a function, then the function will return the None object. An example of a function is as follows: def myfun_add_two(a,b): ''' adding two numbers functions''' return a + b # return is optional you can even write print here only. print(myfun_add_two(3,4)) print(myfun_add_two.__doc__)

Output: 7 adding two numbers functions

Note: The preceding code is covered in (Program Name: Chap06_Example6.1.py) In the preceding example, we can see that a function my_fun_add_two is defined using the def keyword. The function has 2 arguments: a and b. A documentation string about this function is written in triple inverted commas. Addition of 2 values a and b is returned. The function name my_fun_add_two is called with the appropriate parameters 3 and 4. At the end, we are displaying the docstring using __doc__ attribute of the function. The output after the execution of the preceding code is selfexplanatory. If we will be defining the function name after calling, then the error will be displayed as the function name is not defined. Refer to the following program:

184



Python for Everyone

print(myfun_add_two(3,4)) print(myfun_add_two.__doc__) def myfun_add_two(a,b): ''' adding two numbers functions''' return a + b # return is optional you can even write print here only.

Output: print(myfun_add_two(3,4)) NameError: name 'myfun_add_two' is not defined

Note: The preceding code is covered in (Program Name: Chap06_Example6.2.py) The function works as follows in Python: def func_name():

……………………………..



……………………………..

func_name() …………………………….. ……………………………..

Function types

There are 2 types of function that Python supports.

Built in functions The functions that are built into Python and comes along with the software automatically are called built in functions or pre-defined functions. These built-in functions can be used in our code according to the requirements, as shown in the following Table 6.1:

Concept of Functions in Python



185

abs()

all()

any()

ascii()

bin()

bool()

bytearray()

bytes()

callable()

chr()

classmethod()

compile()

complex()

delattr()

dict()

dir()

divmod()

enumerate()

eval()

exec()

filter()

float()

format()

frozenset()

getattr()

hasattr()

hash()

help()

hex()

id()

input()

int()

isinstance()

issubclass()

iter()

len()

list()

map()

max()

memoryview()

min()

next()

object()

oct()

open()

ord()

pow()

print()

property()

range()

repr()

reversed()

round()

set()

setattr()

slice()

sorted()

str()

sum()

super()

tuple()

type()

vars()

zip()

_ _import _ _()

Table 6.1: Built-in functions in Python

User defined functions The functions which are explicitly defined by the programmer to perform certain specific task as per need, is termed as user-defined functions. The user-defined function could be a library function to someone else (Library functions are written by others and we are using them in the form of library). With the help of user-defined function, programmer can decompose a large program into smaller segments for better understanding and debugging. There can be no repetition of the codes as we can execute such codes by calling them on function. There can be multiple number of people working on a single project. So, a large project can be divided by making different functions. Thus, the advantages of these functions help us to reach to our business requirement as early as possible. The syntax to create user-defined functions is as follows: def functionname(parameters): ''' docstring ''' Code 1 Code 2 return value

So, while creating functions, 2 keywords are used. The first keyword is def which is mandatory and another keyword is return, which is optional. If we are not returning

186



Python for Everyone

any value, then default value is None. Moreover, we can return multiple values from a function in Python. Just observe the following example: def my_mult(num1, num2): ''' The above user defined function will multiply 2 numbers and return their product''' return num1 * num2 my_result = my_mult(3,6) print("The product of 2 numbers 3 and 6 is: ", my_result)

Output: 18

Note: The preceding code is covered in (Program Name: Chap06_Example6.3.py) The function my_mult() will multiply 2 numbers and return the result. It is always a better practice to name functions according to the task we perform. Moreover, we can see that the print() is a built-in function. So, it is a choice of the programmer to make user-defined functions as per the requirement.

Function arguments

Arguments are input to the function which are passed during function call. But before discussing about types of arguments, let us see view about formal and actual arguments. Formal arguments are the arguments which come into action when we are defining a function. On the other hand, actual arguments are the arguments which come into action when we are calling a function. In the example discussed as follows, num1 and num2 are the formal arguments and “3,6” are the actual arguments. def my_add(num1, num2): ''' The above user defined function will add 2 numbers and return the result''' return num1 + num2

Concept of Functions in Python



187

my_result = my_add(3,6) print("The addition of 2 numbers 3 and 6 is: ", my_result)

Note: The preceding code is covered in (Program Name: Chap06_Example6.4.py) There are different ways in which a function can be defined, which can take variable number of arguments. We can also say different types of actual arguments.

Positional arguments The arguments passed to the function in correct positional order are called positional parameters. The ‘nth’ positional arguments always need to be listed at nth position when the function is called. Whenever the function is called, the first positional argument will always be listing first, the second positional argument will be listing second and so on. The arguments number and their positions in the function definition must be equal to the arguments number and position in the function call. If we change the order, there are chances that the result may change. Refer to the following program: def my_complex(my_real, my_imag): my_result = complex(my_real,my_imag) return my_result print(my_complex(6,8)) # P1 print(my_complex(8,6)) # P2 print(my_complex(8,6,10)) # P3

Output: (6+8j) (8+6j) TypeError: my_complex() takes 2 positional arguments but 3 were given

Note: The preceding code is covered in (Program Name: Chap06_Example6.5.py) In P1, the actual arguments 6 and 8 are passed to the function my_complex(my_real, my_imag). The values 6 will be listed first in my_real and 8 will be listed second in

188



Python for Everyone

my_imag. A complex number is created using the arguments and the result is returned and displayed. So, output will be 6 + 8j.

In P2, the actual arguments 8 and 6 are passed to the function my_complex(my_real, my_imag). The values 8 will be listed first in my_real and 6 will be listed second in my_imag. A complex number is created using the arguments and the result is returned with change of order as compared to P1. Hence, output will be 8+6j. In P3, the formal arguments are 2 only but the actual arguments which we are passing to the function is 3. So, Python will generate TypeError : my_complex() which takes 2 positional arguments, but 3 were given. An important point to note is that we can also pass positional arguments to function using an iterable object such as list, tuple, set and so on, as shown in the following program: my_list = [5,3] print(pow(*my_list)) # F1 my_tuple = (3,4) print(complex(*my_tuple)) # F2 import math my_set = {4} print(math.sqrt(*my_set)) # F3

Output: 125 (3+4j) 2.0

Note: The preceding code is covered in (Program Name: Chap06_Example6.6.py) In F1, the iterable list is preceded by * and power of 2 numbers is calculated. Hence, output will be 5**3 = 125. In F2, the iterable tuple is preceded by * and complex number is returned where the real part will be the first element and the imaginary part is the second element. Hence, output will be 3 +4j.

Concept of Functions in Python



189

In F3, the iterable set is preceded by * and the square root of the element inside set is being calculated which is 4. Hence, output will be 2.0 as the return value will be of float type.

Keyword arguments The arguments passed to the function as name-value pair such that the keyword arguments can identify the formal arguments by their names is called keyword arguments. It is important to match the keyword argument’s name and formal argument’s name. The order of one keyword argument compared to another keyword argument does not matter. But the number of arguments must be matched. Refer to the following program: def my_complex(my_real, my_imag): my_result = complex(my_real,my_imag) return my_result print(my_complex(my_real = 2,my_imag = 8)) # K1 print(my_complex(my_imag = 8, my_real = 2)) # K2 print(my_complex(2,my_imag = 8)) # K3

Output: (2+8j) (2+8j) (2+8j)

Note: The preceding code is covered in (Program Name: Chap06_Example6.7.py) In K1, the keyword arguments my_real is assigned with the value 2 and my_imag is assigned with the value 8. The formal argument my_real will have value as 2 and my_imag as value 8. The complex number is created and returned. So, output will be 2 + 8j. In K2, the order of keyword argument is different than K1. However, still the formal argument my_real will have value as 2 and my_imag as value 8. The complex number is created and returned. So, output will be same as K1 which is 2 + 8j.

190



Python for Everyone

In K3, both positional and keyword arguments are used simultaneously. However, positional arguments are taken first, followed by keyword arguments. Otherwise, Python will raise an error. Here, the value 2 will be assigned to my_real, followed by keyword argument my_imag which is 8. The complex number is created and returned. So, output will be same as K1 which is 2 + 8j. Refer to the following program: def my_complex(my_real, my_imag): my_result = complex(my_real,my_imag) return my_result print(my_complex(my_imag = 8, 2))

Note: The preceding code is covered in (Program Name: Chap06_Example6.8.py) In the preceding example, Python will raise SyntaxError: positional argument follows keyword argument. Hence, never try to make this mistake. An important point to note is that we can also pass keyword arguments to functions using a Python dictionary, as shown in the following program: my_dict = {'real': 6, 'imag': 8} print(complex(**my_dict))

Output: 6+8j

Note: The preceding code is covered in (Program Name: Chap06_Example6.9.py) In the preceding example, my_dict is the name of the dictionary containing keywords (real,imag) and values (6,8) preceded by ** character. Hence, output will be 6 + 8j.

Default arguments Many a times, there may be cases when the default value is mentioned to the formal argument in the function definition and there is no requirement to provide actual argument. So, in such cases, the default argument will be used by the formal argument. The formal argument will use default value if the actual argument is not provided for the formal argument explicitly while calling the function. If we provide actual argument, then it will use the provided value. Refer to the following example:

Concept of Functions in Python



191

def my_details(name,age): print(f'My name is {name} and age is {age} ') def my_details1(name,age = 31): print(f'My name is {name} and age is {age} ') my_details('Mukesh',31) # D1 my_details1(name = 'Mukesh') # D2 my_details1('Ramesh',32) # D3

Output: My name is Mukesh and age is 31 My name is Mukesh and age is 31 My name is Ramesh and age is 32

Note: The preceding code is covered in (Program Name: Chap06_Example6.10. py) In D1, actual arguments are passed to the function my_details. These are behaving like positional arguments. Hence, output will be My name is Mukesh and age is 31. In D2, the keyword argument name with value as ‘Mukesh’ is passed while calling the function my_details1. But the second argument is not provided for the formal argument explicitly. The default value of age will be 31. Hence, output will be My name is Mukesh and age is 31. In D3, the first argument ‘Ramesh’ is given and second argument is provided with value 32 for the formal argument explicitly. Hence, output will be My name is Ramesh and age is 32. We should not take non-default arguments after default arguments, as shown: def my_details2(age = 31, name): print(f'My name is {name} and age is {age} ')

Note: The preceding code is covered in (Program Name: Chap06_Example6.11. py)

192



Python for Everyone

In the above example, Python will raise SyntaxError: non-default argument follows default argument.

Variable length arguments Many a times while executing big projects, we will not know in advance, the number of arguments which will be passed into a function. So, in Python, there is something called the variable length arguments, that can accept any number of values. An asterisk (*) symbol is used before the parameter name to denote the preceding argument. Variable number of arguments can be passed to a function by using special syntax *args. All the values are represented in the form of a tuple. The number of arguments here will not be maintained. We may use the for loop to retrieve all the arguments back. The variable length arguments can be mixed with positional arguments. A non-keyworded variable length argument list is passed. To current formal arguments, any number of extra arguments can be tackled (including zero extra arguments). If we are taking any other arguments after variable length arguments, then values should be provided as keyword arguments. Refer to the following program: def mymul(*my_num): myresult = my_num[0]*my_num[1]*my_num[2] return myresult def mymul2(first, *my_num): myresult = first*my_num[0]*my_num[1] return myresult def mymul3(*my_num): myresult = 1 for loop in my_num: myresult *= loop return myresult print(mymul(2,3,4)) # V1

Concept of Functions in Python



193

print(mymul(2,3,4,5)) # V2 print(mymul2(2,3,4)) # V3 print(mymul2(2,3,4,5)) # V4 print(mymul3(2,3,4,5)) # V5

Output: 24 24 24 24 120

Note: The preceding code is covered in (Program Name: Chap06_Example6.12. py) In V1, we are passing actual arguments 2,3 and 4 to the function mymul. We are accessing the preceding arguments using index number. So, num[0] will have value 2, num[1] as 3 and num[2] as 4. Multiplication of the preceding values are done and the result is returned. The output is 24. In V2, we are passing actual arguments 2, 3, 4 and 5 to the function mymul. We are accessing the preceding arguments using index number. So, num[0] will have value 2, num[1] as 3 and num[2] as 4. We are not accessing the argument 5 using index number. It is completely the programmer’s choice. Multiplication of the preceding values are done and the result is returned. The output is 24. In V3, we are passing actual arguments 2,3 and 4 to the function mymul2. The value 2 will be accessed by formal argument first and 3,4 using index number. Multiplication of the preceding values are done and the result is returned. The output is 24. In V4, we are passing actual arguments 2, 3, 4 and 5 to the function mymul2. We are accessing the preceding arguments by formal argument first and 3,4 using index number using index number. So, formal argument first will have value 2, num[0] as 3 and num[1] as 4. We are not accessing the argument 5 using index number. Multiplication of the preceding values are done and the result is returned. The output is 24.

194



Python for Everyone

In V5, we are passing actual arguments 2, 3, 4 and 5 to the function mymul3. A for loop is used to retrieve all the arguments back. Multiplication of the preceding values are done and the result is returned. The output is 120.

Keyword variable length arguments (kwargs) The arguments which can accept any number of values provided in the form of keyvalue pair is called keyword variable length argument. The variable length keyword argument dictionary is passed to function by Python using **kwargs and on which dictionary operation can be performed. The double asterisk sign ** is used before the parameter name to denote the above argument. We will be passing the arguments as dictionary and these arguments will make a dictionary inside function with the name same as parameter name but excluding **. Both *args and **kwargs make the function flexible. Refer to the following program: def func1(**kwargs): print(kwargs) # F1 print(type(kwargs)) # F2 for my_key,my_value in kwargs.items(): print(f"{my_key}:{my_value}") # F3 def func2(name1,**kwargs): print(kwargs) # F4 print(name1) # F5 print(type(kwargs)) # F6 for my_key,my_value in kwargs.items(): print(f"{my_key}:{my_value}") # F7 def func3(**kwargs): for my_key,my_value in kwargs.items(): print(f"{my_key}:{my_value}") # F9

Concept of Functions in Python



195

func1(fname = "Saurabh",lname = "chandrakar", phone_number = 9876543210) # F10 func2(1,fname = "Priyanka",lname = "chandrakar", phone_number = 8987676543) # F11 #dictionary unpacking d1 = dict(name ='Saurabh',age = 31) func3(**d1) # F12

Output: {'fname': 'Saurabh', 'lname': 'chandrakar', 'phone_number': 9876543210}

fname:Saurabh lname:chandrakar phone_number:9876543210 {'fname': 'Priyanka', 'lname': 'chandrakar', 'phone_number': 8987676543} 1

fname:Priyanka lname:chandrakar phone_number:8987676543 name:Saurabh age:31

Note: The preceding code is covered in (Program Name: Chap06_Example6.913py) From F1 to F3, we have a function func1 with **kwargs as a parameter. The dictionary with variable length argument F10 is being passed to the func1() function. The variable length arguments passed and its type is displayed followed by a for loop inside func1() function working on the data of passed dictionary and printing the value of dictionary. Hence:

196



Python for Everyone

Output of F1 is: {'fname': 'Saurabh', 'lname': 'chandrakar', 'phone_number': 9876543210} Output of F2 is Output of F3 is fname:Saurabh lname:chandrakar phone_number:9876543210

From F4 to F7, we are illustrating kwargs for variable number of keyword arguments with one extra argument. We have a function func2 with name1, and **kwargs as a parameter. The dictionary with variable length argument F11 is being passed to the func2() function. Inside func2() function, we have a for loop working on the data of extra argument, passed dictionary and prints its value. Hence, Output of F4 is: {'fname': 'Priyanka', 'lname': 'chandrakar', 'phone_number': 8987676543}. Output of F5 is 1 Output of F6 is Output of F7 is

fname:Priyanka

lname:chandrakar phone_number:8987676543

In F12, we have a function func3 with **kwargs as a parameter. The dictionary with variable length argument F12 is being passed to the func3() function. We are unpacking the dictionary. So, output will be: name:Saurabh age:31 Note: A group of statements is termed as functions. A group of functions is termed as modules. A group of modules is termed as package. A group of package is termed as library.

Concept of Functions in Python



197

Nested function

A Nested function or inner function or function nesting is the term used when we define one function inside another function. We can access the variables within the enclosing scope by using the inner function. We can create an inner function to protect it from everything which is happening outside of the function. This process is called as encapsulation and can be seen in the following program: def outside_func(): def inner_func(): print("Inside Inner Function") print("Inside Outer Function") inner_func() outside_func()

Output: Inside Outer Function Inside Inner Function

Note: The preceding code is covered in (Program Name: Chap06_Example6.14. py)ss In the preceding example, inner_func() has been defined inside outside_func(), thus making it an inner function. To call inner_func(), we must first call outside_ func(). The outside_func() will then go ahead and call inner_func() as it has been defined inside it. An important point to observe is that outside_func() has been called to execute inner_func(). The inner_func() will never execute if the outside_func() is not called. Just execute the following code: def outside_func(): def inner_func(): print("Inside Inner Function") print("Inside Outer Function") inner_func()

198



Python for Everyone

Nothing will be returned by Python when executed. We can also pass a parameter to the function: def outside_func(str1): def inner_func(): print(str1 + "Inner Function") print("Inside Outer Function") inner_func() outside_func("Hello ")

Output: Inside Outer Function Hello Inner Function

Note: The preceding code is covered in (Program Name: Chap06_Example6.15. py) In the preceding example, we were passing a string parameter say “Hello” to outside_func function. The string variable will be accessed inside inner_func() as shown in the output. We can change the variables of the outer function from inside the inner function, as shown in the following program: def outer(a): b = 3 b += a def inner(c): b = 6 print(c**b) print(b) inner(3) outer(1)

Concept of Functions in Python



199

Output: 4 729

Note: The preceding code is covered in (Program Name: Chap06_Example6.16. py) In the preceding example, outer(1) function is called storing the value of a as 1. The expression b+=a will yield the value of b as 4 and will depict a variable defined within the outer() function. print(b) will display the value of b of the outer function. A new variable ‘b’ is defined within the inner() function rather than changing the value of ‘b’ of the outer() function. Finally print(c **b) will provide us the result as 729 (3 **6).

Python closures

If there is a requirement of binding data to a function without passing them as parameters, then it is called as closures. It will help to invoke function outside their scope. It is a function object which remembers values in their enclosing scopes, even if they are absent in memory, which means that we have a closure when a nested function references a value which is in its enclosing scope. To create a closure in Python, the following conditions are to be met: • A nested function must be present.

• The nested function will refer to a variable defined in the enclosing function. • The enclosing function should return the nested function. Just observe the following example: def outer_func(str1): var1 = " is awesome" def nested_func(): print(str1 + var1) return nested_func myobj = outer_func('Python') myobj()

200



Python for Everyone

Output: Python is awesome

Note: The preceding code is covered in (Program Name: Chap06_Example6.17. py) Till now, we have just seen calling the nested function inside the function. In the preceding example, we have returned the nested function instead of calling it. We have returned the whole functionality of the nested function and bind it to a variable instead for further usage. The scope of nested_func() is only inside outer_func() . But with the help of closures, we have extended the scope and invoked it from outside its scope. The Python Virtual Machine (PVM) will detect the dependency of nested function on outer function, and will make sure that the variables in which the nested function depends on, is available even if the outer function goes away. So, the previous example will generate the output as Python is awesome. The advantage of closures is that since they provide some sort of data hiding, it helps to reduce the use of global variables. Closures prove to be effective when we have few functions in our code. But we can go for class if we need to have many functions. There is a function which creates another object called factory functions. The inner functions will help us in defining factory functions, as shown: def outer(mynum): def inner(a1): return mynum ** a1 return inner obj1 = outer(6) obj2 = outer(5) print(obj1(2)) print(obj2(3))

Output: 36 125

Concept of Functions in Python



201

Note: The preceding code is covered in (Program Name: Chap06_Example6.18. py) In the preceding Python script, 2 objects are created from inner(a1) function: obj1 and obj2 which makes inner(a1) a factory function, since it generates the obj1 and obj2 functions for us, using the parameter we pass it. Hence, the output displayed here will be (6**2) = 36 and 125 respectively.

Function passing as a parameter

Multiple arguments can be taken by a function. The arguments can be objects, variables (of same or different data types) or functions. A function which accepts other functions as arguments are called higher-order functions. Refer to the following: def myupper(mystr): return mystr.upper() def mylower(mystr): return mystr.lower() def world_virus(myfunc): virus = myfunc("CoronA") print(virus) world_virus(myupper) # FF1 world_virus(mylower) # FF2

Output: CORONA corona

Note: The preceding code is covered in (Program Name: Chap06_Example6.19. py) In the preceding example, a function world_virus, is created which takes a function as an argument.

202



Python for Everyone

In FF1, we are passing myupper as an argument to world_virus(myfunc) function. myupper("CoronA") will be function called, which returns the string value as “CORONA” and will be stored in a variable for displaying the output. Hence, output will be “CORONA”. In FF2, we are passing mylower as an argument to world_virus(myfunc) function. mylower("CoronA") will be function called which returns the string value as “corona” and will be stored in a variable for displaying the output. Hence, output will be “corona”. So, a function can also be passed as argument to another function.

Local, global, and non-local variables

We have unknowingly used local, global, and non-local variables in the past. But it is the correct time to know what these variables are and how to use them in our code.

Local variables The variables which are declared inside a function and is not accessible outside the function, is called local variables. The value of the local variable is available only in that function itself and not outside of the function, as the scope is limited to that function, where it is created. Just observe the following Python script: def func1(num2): num1 = 1 print(num1) print(num2 + num1) func1(12) print(num1)

Output: 1 13 NameError: name 'num1' is not defined

Note: The preceding code is covered in (Program Name: Chap06_Example6.20. py)

Concept of Functions in Python



203

In the preceding Python script, a local variable num1 is created by declaring a variable inside the function. The local variable is used inside the function by displaying it and adding it with the formal argument variable. Whenever we are displaying the local variable outside its scope, Python will result in NameError. In the preceding example, num1 is a local variable for the func1() function and is not accessible outside of it. So, Python raises NameError: name 'num1' is not defined.

Global variables Whenever a variable is declared outside of the function, it is called as global variable. These variables will be available to all the function which are written after it. The scope of global variable will be the entire program body written below it. Refer to the following Python script: g1 = 1 # GL1 def display(): l1 = 2 # GL2 print(l1) # GL3 print(g1) # GL4 display() print(g1) # GL5 print(l1) # GL6

Output: 2 1 1 NameError: name 'l1' is not defined

Note: The preceding code is covered in (Program Name: Chap06_Example6.21. py) In GL1, the variable g1 is assigned with value 1 and is a global variable. In GL2, the variable l1 is assigned with value 2 and is a local variable.

204



Python for Everyone

In GL3, we are using local variable inside display() function. In GL4, we are using global variable inside display() function. In GL5, we are using global variable outside function. In GL6, we are using local variable outside function and it will show NameError. We can access the global variable g1 as it has been defined out of a function. A new local variable is created in the function’s namespace if another value is assigned to a globally declared variable inside the function. The value of the global variable will not be changed. Refer to the following script: g1 = 1 def display(): g1 = 2 print(g1) print("inside",id(g1)) display() print(g1) print("outside",id(g1))

Output: 2 inside 140733773341360 1 outside 140733773341328

Note: The preceding code is covered in (Program Name: Chap06_Example6.22. py) In the preceding example, a new variable g1 is created inside the display() function. We can see from the display of id(g1) inside the function which is 140733773341360. The global variable g outside the function has the id as 140733773341328 . Clearly, the 2 variables are having different ids.

Concept of Functions in Python



205

Now, to change the global variable value from within a function, the global keyword will be used. So, if there is any requirement of accessing the global variable inside the function, it can be accessed using ‘global’ keyword followed by variable name. Refer to the following code: g1 = 1 def display(): global g1 g1 = 2 print(g1) print("inside",id(g1)) display() print(g1) print("outside",id(g1))

Output: 2 inside 140733773341360 2 outside 140733773341360

Note: The preceding code is covered in (Program Name: Chap06_Example6.23. py) From the preceding Python script, we can see that the global variable g1 value has been modified from 1 to 2. Moreover, identity of the global variable inside the function and outside the function remains same. An important point to note is that whenever the name of global and local variable is same, the function by default refers to local variable and ignores the global variable. It is mandatory to reference the local variable before assignment. Refer to the following code: g1 = 1 def display():

206



Python for Everyone

g1 = g1 -1 print(g1) display()

Output: UnboundLocalError: local variable 'g1' referenced before assignment

Note: The preceding code is covered in (Program Name: Chap06_Example6.24. py) There is something called globals() function which returns a table of global variables in the form of dictionary. Using variable name as key, its value can be accessed and modified. So, the preceding function will give the programmer flexibility to use a global and local variable with the same name simultaneously. Refer to the following script: num1 = 1 def display(): num1 = 2 print("local variable: ", num1) num2 = globals()['num1'] print(num2) num2 = 3 print(num2) display() print('global variable: ', num1)

Output: local variable:

2

1 3 global variable:

1

Concept of Functions in Python



207

Note: The preceding code is covered in (Program Name: Chap06_Example6.25. py) Here, we can see that the same variable name num1 is used both locally and globally. The variable name num1 is used as key to access the global value and stored in variable num2. So, num2 is assigned with value 1. Then we are assigning new variable to num2 with value 3. The original global variable num1 has value 1.

Non-local variables The non-local variable will be used in nested functions whose local scope is not defined, which means that the variable can neither be in the local or global scope. Refer to the following program: def outer(): a1 = 20 b1 = 40 def inner(): # nonlocal binding nonlocal a1 a1 = 50 # will update b1 = 60 # will not update, # it will be considered as a local variable inner() print("a1 : ", a1) print("b1 : ", b1) # main code # calling the function i.e. outer() outer()

208

Python for Everyone



Output: a1 :

50

b1 :

40

Note: The preceding code is covered in (Program Name: Chap06_Example6.26. py) In the preceding example, a1 and b1 are the variables of the outer() function and in the inner() function, we are using the variable a1 as a non-local variable. However, variable b1 is the local variable for inner() function, and if the value of b1 has changed, then it will be considered a new assigned for local variable (for inner() ) b1. So, a nonlocal keyword is used to create a non-local variable. Hence, the output of the preceding Python snippet will be: a1 :

50

b1 :

40

Recursive function

The process of defining itself is called recursion. So, a function which calls itself is called recursive function. These recursive functions will reduce the code length and improve readability. The complex problems can be solved very easily by breaking down into simpler sub-problems. Sequence generation is easier than using some nested iteration with recursion. But recursive calls are inefficient as they take up a lot of memory and time. The logic in recursion is quite hard to follow through, and hard to debug. The best example in the real world is when we stand between 2 parallel mirrors facing each other. Our face would be reflected recursively. The best example that we have learned from our basic C language is to calculate the factorial of a number. Refer to the following program: def factorial(a): if a == 1: return 1 else: return (a * factorial(a-1))

Concept of Functions in Python



209

num = 7 print("The factorial of", num, "is", factorial(num))

Output: The factorial of 7 is 5040

Note: The preceding code is covered in (Program Name: Chap06_Example6.27. py) In the preceding example, factorial is a recursive function which calls itself. When the preceding function is called with a positive integer, it will recursively call itself by decreasing the number. The recursive call is explained as follows: factorial(7) #1st call with 7 7 * factorial(6)

#2nd call with 6

7 * 6 * factorial(5)

#3rd call with 5

7 * 6 * 5 * factorial(4)

#4th call with 4

7 * 6 * 5 * 4 * factorial(3)

#5th call with 3

7 * 6 * 5 * 4 * 3 * factorial(2)

#6th call with 2

7 * 6 * 5 * 4 * 3 * 2 * factorial(1)

#7th call with 1

7 * 6 * 5 * 4 * 3 * 2 *1

#return from 7th call as number 1

7 * 6 * 5 * 4 * 3 * 2

#return from 6th call as number 2

7 * 6 * 5 * 4 * 6

#return from 5th call as number 6

7 * 6 * 5 * 24

#return from 4th call as number 24



7 * 6 * 120



#return from 3rd call as number 120

7 * 720

#return from 2nd call as number 720

5040

# final output





The recursion will end when the control reaches to the base condition, that is, when the number reduces to 1. It is mandatory to have the base condition when we are using recursive function since it stops the recursion otherwise the function will call itself infinitely.

210



Python for Everyone

Python Lambda functions

Till now, we have declared functions with some name given to it. However, we can also declare a function without any name. Such type of functions which are nameless, are called anonymous or lambda functions. Normal functions are defined using the def keyword, whereas anonymous functions are defined using lambda keyword. For one-time usage or for instant use, the anonymous function comes into the picture. A lambda function can take any number of arguments but can have only one expression. The syntax of lambda function is as follows: lambda arguments: expression

Here, lambda is a keyword that defines a lambda expression. The second parameter arguments is a comma separated list of arguments as found in the function definition (kindly note the lack of parenthesis). The third parameter is a single Python expression which cannot be a complete statement. The expression is returned after evaluation. The : represents the function beginning. Lambda function returns a function. There is no need to write return statement. All the type of actual arguments can be used. Here, the arguments can be many but the expression must be single. We can use the lambda expression wherever function objects are required. Lambda functions are used in built-in functions such as map, filter and reduce. We will see the examples of each one for better understanding and evaluation. Refer to the following program: def add(x,y): return x + y print("Normal function: ", add(2,3)) # L1 add2 = lambda a,b: a+b # L2 print("Lambda function: ", add2(2,3))# L3

Output: Normal function:

5

Lambda function:

5

Concept of Functions in Python



211

Note: The preceding code is covered in (Program Name: Chap06_Example6.28. py) Both the normal and lambda functions will yield the same output. In L1, we are calling the function with arguments 2 and 3 and returning the output as 5. Hence, output will be Normal function: 5. In L2, lambda a,b: a + b is the lambda function . We have given the arguments a and b and the expression a + b gets evaluated and returned. The preceding function has no name and returns a function object assigned to the identifier add2. So, in L3, the output will be Lambda function:

5.

Nested lambda functions A lambda function written inside another lambda function is called nested lambda functions. Observe the example shown: #M-1 def mul1(a1): return lambda b1:b1*a1 myresult = mul1(3) print(myresult(7)) #M-2 mul = lambda a = 3: (lambda b: a*b) myres = mul() print(myres) print(myres(7))

Output: 21

21

212



Python for Everyone

Note: The preceding code is covered in (Program Name: Chap06_Example6.29. py) In M1, lambda appears inside a function mul1 and the value can be accessed that the argument a1 has in the function’s scope at the time when we are calling enclosing function. So, a1 value will be 3 when mul1(3) is being called. myresult is now a function object and when calling the preceding function with argument 7, the b1 value will 7. Hence, the output 7*3 = 21 is returned. In M2, lambda can access the names in the enclosing lambda. Here, lambda structure is nested to make a function that makes a function when called. The preceding approach is quite convoluted and should be avoided. Moreover, the function object myres is printed which yields output:

The overall output using the preceding approach yields value 21 as M1.

Passing lambda functions to another function Lambda function can be passed as an argument to another function. Let us see an example. def mypow(num): print(num) print(num(3)) mypow(lambda a: a**4)

Output:

81

Note: The preceding code is covered in (Program Name: Chap06_Example6.30. py) In the preceding example, we are passing the lambda function as an argument in the function call. So, num is now a function argument which is displayed. The value 3 is passed as a parameter to variable a. Hence, the output will be 3**4 = 81.

Concept of Functions in Python



213

Conclusion

Thus, in the first half of this chapter, we initially witnessed the concept of functions with its syntax. Then we saw various built-in functions, followed by an example of user defined functions. We saw important concept of function arguments which are positional arguments, keyword arguments, default arguments, *args and **kwargs. Then we defined one function inside another function which is nested function. We also saw the requirement of binding data to a function without passing them as parameters, called Python closures. The usage of local, global, and non-local variables was also illustrated. The best example of factorial was seen using recursive function. Finally, we touched on the concept of Python lambda function with various Python examples.

Points of remember

• Functions must be defined initially before being later called in Python with proper indentation.

• Built-in functions are the functions that are already built in Python, whereas user defined functions are explicitly defined by the programmer to perform specific task as per need.

• Arguments which come into action when we are defining a function are called formal arguments, whereas arguments which come into action when we are calling a function is called actual arguments.

• Positional arguments are the arguments passed to the function in the correct positional order. Keyword arguments are the arguments passed to the function as name-value pair, such that the keyword arguments can identify the formal arguments by their names. The default value will be used by the formal argument if the actual argument is not provided for the formal argument explicitly while calling the function. • We can pass the variable number of arguments to a function by using special syntax *args. When used as a function parameter, *args gathers all the function’s unmatched positional arguments into a single tuple.

• When a function’s **kwargs parameter is specified, all the function’s unmatched keyword arguments are collected into a dictionary (where the parameter name is the key). • We can bind the data to a function without passing them as parameters called closures.

214



Python for Everyone

• Variables defined inside a function but not accessible outside the function, are called local variables. Variables declared outside of the function are called global variables. When the variable is neither in local or global scope, then it is called non-local variables.

• Normal functions are defined using def keyword, whereas anonymous functions are defined using lambda keyword.

Questions

1. Explain the concept of functions in Python with its syntax and example. 2. How many built-in functions are there in Python? 3. Explain user defined function with an example.

4. Explain the concept of positional, keyword, default, variable length arguments and keyword variable length arguments with a Python snippet code. 5. What are nested functions in Python?

6. What are Python closures? Explain with a Python snippet code.

7. Explain the concept of functions passing as a parameter with a Python snippet code.

8. Explain the concept of local, global, and non-local variables in Python with an example. 9. What are recursive functions in Python? Explain with a Python snippet code. 10. What are Python lambda functions? Explain with a Python snippet code.

Join our book's Discord space

Join the book's Discord Workspace for Latest updates, Offers, Tech happenings around the world, New Release and Sessions with the Authors: https://discord.bpbonline.com

Chapter 7

Concept of Data Structures in Python Introduction

A way of organizing and storing data in order such that it can be accessed and modified efficiently, iscalled data structures. There is always a provision for data structures in every programming language. We have seen some primitive data types such as strings, integers, Boolean and float. Now, we will study some nonprimitive inbuilt data structures such as list, tuple, set, dictionary, files and arrays. Each of these data structures are unique on its own. These data structures are an indispensable part in programming. Without these data structures, we cannot think of writing any Python program. Let us see some of these data structures one by one.

Structure

In thischapter, we will discuss the following topics: • List data structure

• Tuple data structure • Set data structure

• Dictionary data structure

216



Python for Everyone

Objectives

By the end of thischapter, the reader will have an idea about 4 data structures, viz, list, tuple, set and dictionary creation, as well as the elements accessing using indexing and slicing except for set, list, set, and dictionary comprehension using for loops, if/if-else/nested if statement. We shall also see which data structures are mutable, whose insertion order is preserved, and whose heterogenous objects are allowed with a Python snippet code.

List data structure

A list can store different types of elements. It represents a group of elements; heterogeneous objects are allowed. List is dynamic as the size can be increased or decreased based on our requirement. We can modify the element in a list as it is mutable. Here, duplicate objects are allowed. In lists, the insertion order is preserved by using index. Duplicate elements are differentiated by using index in list. So, index play a vital role. All the elements in a list are separated by comma and are placed within square brackets. So, whenever there is a requirement to represent a group of individual objects as a single entity where we need to preserve the insertion order and duplicate objects are allowed, we should go for lists.

Creating a list The syntax of list object creation is as follows: listname = [element1, element2, element3, …]

For example: myl1 = [10, 20, -30.1, 'Hello'] The preceding example creates a list object containing positive numbers, negative numbers, string and so on. We can either create a list object with integer numbers, float numbers, or string, depending on our need and requirement.

Creating an empty list The syntax of empty list creation is as follows: listname = [] myl1 = [] print(myl1) # L1

Concept of Data Structures in Python



217

print(type(myl1)) #L2

Output: []

Note: The preceding code iscovered in (Program Name: Chap07_Example7.1.py) In L1, myl1 is an empty list. In L2, myl1 object is of type list.

Creating a list when elements are known If we already know the elements in a list, then it can be created as shown: myl1 = [1,2,3,4,5]

Creating a list with dynamic input The list elements will be provided from the command prompt, as shown: mylist = eval(input("Enter the list: ")) print(mylist) # LI1 print(type(mylist)) # LI2

Output: Enter the list: [1,2,3,4,5] [1, 2, 3, 4, 5]

Note: The preceding code iscovered in (Program Name: Chap07_Example7.2.py) In the preceding example, user is prompted to enter the list: In LI1, the user entered the list as [1,2,3,4,5] In LI2, mylist object is of type list.

218



Python for Everyone

List creation using list() function For a given sequence, we can provide either set, string, range, and so on, as shown here: mylist = list(range(1,10,2)) print(mylist)# LS1 print(type(mylist))# LS2

Output: [1, 3, 5, 7, 9]

Note: The preceding code iscovered in (Program Name: Chap07_Example7.3.py) Here, the sequence is a range function with values start as 1, stop as 10 and step as 2. It isconverted into a list object using list(). So, output of LS1 is [1, 3, 5, 7, 9]. In LS2, mylist object is of type list.

List creation using split() function In Python, string to list conversion is possible, as shown: st1 = "Python is awesome" mylist = st1.split(' ') print(mylist) # SP1 print(type(mylist))# SP2

Output: ['Python', 'is', 'awesome']

Note: The preceding code iscovered in (Program Name: Chap07_Example7.4.py)

Concept of Data Structures in Python



219

In SP1, the string isconverted into list object. Hence, output is ['Python', 'is', 'awesome']. In SP2, mylist object is of type list.

Lists versus immutability Once we have created a list object, the contentscan be modified. So, list objects are mutable, as shown: myl1 = [11,12,13,14] print(myl1)# MU1 myl1[1] = 20 print(myl1)# MU2

Output: [11, 12, 13, 14] [11, 20, 13, 14]

Note: The preceding code iscovered in (Program Name: Chap07_Example7.5.py) In MU1, the list elements are displayed as [11, 12, 13, 14]. In MU2, we have modified the element at index/position 1 to 20. Hence, output after the modification will be [11, 20, 13, 14].

Accessing elements of list The list elementscan be accessed by using index or by using slicing operator.

By using index The position number of a list element will be represented by an index. The index will be written inside square brackets and starts from 0 onwards. For example: myl1 = [10, 20, -30.1, 'Hello']

220



Python for Everyone

Positive indexing The positive index means from left to right. From the preceding example, the positive index numbering is as shown in Table 7.1: list object with index

Element

myl1[1]

20

myl1[0] myl1[2] myl1[3]

10

-30.1

Hello

Table 7.1: Positive indexing

Negative indexing The negative index implies right to left. From the preceding example, the negative index numbering is as shown Table 7.2: list object with index

Element

myl1[-3]

20

myl1[-4] myl1[-2] myl1[-1]

10

-30.1

Hello

Table 7.2: Negative indexing

Refer to the following program: myl1 = [10, 20, -30.1, 'Hello'] print(myl1[3])# LI1 print(myl1[-1])# LI2 print(myl1[4])# LI3

Output: Hello Hello IndexError: list index out of range

Note: The preceding code iscovered in (Program Name: Chap07_Example7.6.py)

Concept of Data Structures in Python



221

In LI1, the element at index/position 3 is Hello. In L12, the element at index/position -1 is Hello. In LI3, the index/position 4 is unavailable as there are only 4 elements. Hence, Python will raise IndexError : list index out of range.

By using index and for loop The elements of list can be accessed using index and for loop as shown: myl1 = [10, 20, -30.1, 'Hello'] list_length = len(myl1) for loop in range(list_length): print(loop, myl1[loop])

Output: 0 10 1 20 2 -30.1 3 Hello

Note: The preceding code iscovered in (Program Name: Chap07_Example7.7.py) In the preceding example, the number of elements in a list is returned and is iterated using a for loop. The index number along with the element will be displayed. Here, we are sequentially accessing each element of the list which iscalled as traversal. The elements are accessed using for loop by using index.

By using Index and while loop The elements of list can be accessed using index and while loop as shown: myl1 = [10, 20, -30.1, 'Hello'] list_length = len(myl1) count = 0 while count 35 else 'young' for (key, value) in mydict_age.items()} print(new_dict)

Output: {'Ankt': 'young', 'Saurabh': 'young', 'Nilesh': 'old', 'Mr. Ben': 'old'} {'Ankt': 'young', 'Saurabh': 'young', 'Nilesh': 'old', 'Mr. Ben': 'old'}

Note: The preceding code iscovered in (Program Name: Chap07_Example7.48. py) In both the cases, a new dictionary will be created using dictionary comprehension. The items those having value >35 are old, while others will have the value of ‘young’.

256



Python for Everyone

Despite these advantages, dictionary comprehension must be carefully used as it consumes more memory, makes the code run slow and decreases the code readability.

Conclusion

In thischapter, we saw about 4 data structures viz list, tuple, set and dictionary. We also saw how to create all these data structures, and how to access these data structures wherever possible (except set). The difference between list and tuple is well explained. We need to make sure that we use the list or tuple correctly while creating big projects. List, set and dictionary comprehension were also touched upon, using the for loop and if/if-else/nested if statement.

Points of remember

• List data structure allows heterogeneous objects. Its size is dynamic. Element can be modified as it is mutable. Insertion order is preserved using indexing. Both positive and negative indexing are supported. We can differentiate duplicate elements using index. Representation: [] • Tuple data structure allows heterogeneous objects. Its size is dynamic. Element cannot be modified as it is immutable. Insertion order is preserved using indexing. Both positive and negative indexing is supported. We can differentiate duplicate elements using index. Representation: ()

• Set data structure is an unordered collection of items where order of elements is not maintained. Heterogeneous elements are allowed. Duplicates are not allowed. We can sort the elements. No provision of indexing and slicing. Representation: {} • Whenever there is a requirement to represent group of objects using keyvalue pairs, we need to choose dictionary. Here, duplicate keys are not allowed but duplicate values are allowed. The heterogeneous objects are allowed for both key and values. Insertion order is not preserved. Slicing and indexing concepts are not applicable. Representation: {}

Concept of Data Structures in Python



257

Questions (Long/Short/MCQs)

1. Explain different ways of creating a list with Python snippet code.

2. Explain different ways of accessing elements of list with Python snippet code.

3. What is list comprehension? Explain different approaches of creating a list using list comprehension with Python snippet code. 4. Explain the different ways of creating a tuple with Python snippet code.

5. Explain the different ways of accessing elements of tuple with Python snippet code. 6. Explain list and tuple object with respect to immutability. 7. Compare list and tuple differences.

8. Explain different ways of creating a set with Python snippet code.

9. What is set comprehension? Explain different approaches of creating a set using set comprehension with Python snippet code. 10. Explain different ways of creating a dictionary with Python snippet code.

11. Explain different ways of accessing elements of dictionary with Python snippet code.

12. What is dictionary comprehension? Explain different approaches of creating a dictionary using dictionary comprehension with Python snippet code.

Join our book's Discord space

Join the book's Discord Workspace for Latest updates, Offers, Tech happenings around the world, New Release and Sessions with the Authors: https://discord.bpbonline.com

Chapter 8

Concept of Packages in Python Introduction

In this chapter, we shall discuss about packages. Package is an encapsulation mechanism to group modules which are related to a single unit. It is a way of structuring module namespace of Python, by using “dotted module names”. So, X,Y means that Y is a sub module which is under package name X. It is a collection of modules and packages. A package can contain subpackages. It is a folder or directory. Just like modules can handle functions and namespace in an effective way, Python packages can handle more than one module in a structured way.

Structure

In this chapter, we will discuss the following topics: • Packages

• Structure for package of Games

o Using “import” in Packages



o Using “from import *” in Packages



o Using “from import” in Packages

• Accessing Games package using different approaches

260



Python for Everyone

Objectives

By the end of this chapter, the reader will know about the main difference between modules and packages and how to use Python. A package name ‘Games’ folder will be created, which will itself consist of 3 packages namely Cricket (subpackage: India), Football and Kabaddi package. These packages will consist of various submodules, which will be explained in this chapter. We will be viewing different approaches of accessing the structure for package of Games. In the end of this chapter, the reader will also understand the approach to import one package module inside another package module.

Packages

The following Figure 8.1 depicts that a package can consist of modules and packages. Modules in simple terms mean files and packages as folders. However, every folder must contain a file called __init__.py, ensuring that folder be treated as a Python package:

Figure 8.1: Package Overview

Concept of Packages in Python



261

So, the package should contain a special file called __init__.py. This file can be empty. It can also be an executable initialization code. Package statements resolve naming conflicts, the components can be identified uniquely, and the modularity of the application is improved providing flexibility benefit in usage. To create a package in Python, there are 3 basic steps which need to be followed:

1. The first step is to create a folder and give it a package name, mainly related to operation. 2. Then put the classes, variables and required functions in it.

3. The __init__.py file is created inside the folder to consider it as a Python package. Now, we will discuss a collection of modules and packages that are designed for the Games. In Figure 8.2, we have created a package named Games. This package consists of following items:

• Cricket package consisting of modules (othercountrybatsman.py and othercountrybowler.py), package India (indianbatsman.py, indianbowler. py and __init__.py) and __init__.py. • Football package consisting of modules (goalkeeper.py and forward.py) and __init__.py.

• Kabaddi package consisting of modules (raider.py and defender.py) and __init__.py. • games.py.

• __init__.py.

262



Python for Everyone

Refer to the following Figure 8.2:

Figure 8.2: Items under Games package

Structure for package of games Now, we will see how to access the package.

Using “import” in Packages The syntax is: import packName.modName

Here, packName is the name of the package and modName is the module Name. For example: import Cricket.othercountrybatsman The Syntax is: import packName.subPackName.modName

Concept of Packages in Python



263

Here, packName is the name of the package, subPackName is package name insidepack, and Name and modName is the module Name. For example, import Cricket.India.indianbatsman

Accessing objects like variables, functions, classes, and lists The Syntax is: packName.modName.funcName()

Here, packName is the name of the package, modName is the module Name and funcName is the function Name. For example: Cricket.othercountrybatsman.name_othercountrybatsman() The syntax is: packName.subPackName.modName.funcName()

Here, packName is the name of the package, subPackName is package name inside packName , modName is the module Name and funcName is the function Name. For example, Cricket.India.indianbatsman.name_indianbatsman()

Using “from import” in Packages The syntax is: from packName.modName import funcName,

Here, packName is the name of the package, modName is the module name and funcName is the function name. For example: from Cricket.othercountrybatsman import name_ othercountrybatsman. The syntax is: from packName.subPackName.modName import funcName

Here, packName is the name of the package, subPackName is package name inside packName, modName is the module name and funcName is the function name. For example, from Cricket.India.indianbatsman import name_indianbatsman.

264



Python for Everyone

Accessing objects like variables, functions, classes, and lists The Syntax is: funcName()

Here, funcName is the function Name. For example: name_othercountrybatsman() The syntax is: funcName()

Here, funcName is the function Name. For example, name_indianbatsman()

Using “from import *” in Packages If packages __init__.py code defines a list named __all__, it would take into consideration module name list which should be imported when from Cricket import * is encountered. __all__ = ['othercountrybatsman',' othercountrybowler']

Now, we have made a folder name Games, under which different packages (folders) have been made, along with module names, as shown in the Figure 8.3:

Figure 8.3: Games Package

Concept of Packages in Python



265

The Cricket package has the modules and packages shown in Figure 8.4:

Figure 8.4: Cricket Package India is a subpackage in the Cricket package, as shown in Figure 8.5:

Figure 8.5: India Package

The Football package has the following modules and package, as shown in Figure 8.6:

Figure 8.6: Football Package

266



Python for Everyone

The Kabaddi package has the following modules and packages, as shown in Figure 8.7:

Figure 8.7: Kabaddi Package

Note: The previous package example was compiled and run in the path E:\ python_progs\Games\ folder and the details were captured and presented here. All the source code present in the code bundle is stored in the respective folder of this chapter, and a separate Games folder is also present for your practice. The reference of the program is mentioned against each code. So, observe the pattern and behavior of the code and practice yourself.

Accessing games package using different approaches

We have made a module named “games.py”. First, we will see how to access the module name of different packages and subpackages, such as Cricket, India, Football and so on, using the previous file for cases from 1 to 24. The code inside different module names and their functions are shown as follows. Modulename: othercountrybatsman.py (Present in \Games\Cricket) #Cricket Package --- othercountrybatsman module def name_othercountrybatsman(): '''Other Country Batsman Names are''' print("Other Country Batsman Function") print("Batsman1: Mr. E") print("Batsman2: Mr. F") print()

Concept of Packages in Python

Modulename: othercountrybowler.py (Present in \Games\Cricket) #Cricket Package --- othercountrybowler module def name_othercountrybowler(): '''Other Country Bowler Names are''' print("Other Country Bowler Function") print("Bowler1: Mr. G") print("Bowler2: Mr. H") print()

Modulename: indianbatsman.py (Present in \Games\Cricket\India) #India subpackage ----- indianbatsmann module def name_indianbatsman(): '''Indian Batsman Names are''' print("Indian Batsman Function") print("Batsman1: Mr. A") print("Batsman2: Mr. B") print()

Modulename: indianbowler.py (Present in \Games\Cricket\India) #India subpackage --- indianbowler module def name_indianbowler(): '''Indian Bowler Names are''' print("Indian Bowler Function") print("Bowler1: Mr. C") print("Bowler2: Mr. D") print()



267

268



Python for Everyone

Modulename: goalkeeper.py (Present in \Games\Football) #Football Package --- goalkeeper module def name_goalkeeper(): '''Football goalkeeper names are''' print("GoalKeeper Function") print("GoalKeeper1: Mr. R") print("GoalKeeper2: Mr. S") print()

Modulename: forward.py (Present in \Games\Football) #Football Package --- forward module def name_forward(): '''Football forward names are''' print("Forward Function") print("Forward1: Mr. T") print("Forward2: Mr. U") print()

Modulename: raider.py (Present in \Games\Kabaddi) #Kabaddi Package --- raider module def name_raider(): '''Kabaddi raider names are''' print("Raider Function") print("Raider1: Mr. W") print("Raider2: Mr. X") print()

Concept of Packages in Python



269

Modulename: defender.py (Present in \Games\Kabaddi) #Kabaddi Package --- defender module def name_defender(): '''Kabaddi defender names are''' print("Defender Function") print("Defender1: Mr. Y") print("Defender2: Mr. Z") print()

Now, we will be viewing different cases of accessing multiple modules, using the games.py file. We are writing different types of code, as shown below, in different case scenarios.

• games.py module and accessing othercountrybatsman module using import only:

#games module



import Cricket.othercountrybatsman

Cricket.othercountrybatsman.name_othercountrybatsman()

Output: On running Python games.py from the console, we have:

Other Country Batsman Function



Batsman1: Mr. E



Batsman2: Mr. F

Note: The preceding code is covered in (Program Name: Chap8_ Example8.1.py)

In the previous code, we are accessing the othercountrybatsman module of Cricket package from the file. Under this module, we are accessing the name_ othercountrybatsman function. So, output is displayed as shown above.

• games.py module and accessing othercountrybowler module using import only:

#games module



import Cricket.othercountrybowler

Cricket.othercountrybowler.name_othercountrybowler()

270



Python for Everyone

Output: On running Python games.py from the console, we have:

Other Country Bowler Function



Bowler1: Mr. G



Bowler2: Mr. H

Note: The preceding code is covered in (Program Name: Chap8_ Example8.2.py)

In the previous code, we are accessing the othercountrybowler module of Cricket package from the file. Under this module, we are accessing the name_ othercountrybowler function. So, output is displayed as shown above.

• games.py module and accessing indianbatsman module inside India package using import only:

#games module



import Cricket.India.indianbatsman

Cricket.India.indianbatsman.name_indianbatsman()

Output: On running Python games.py from the console, we have:

Indian Batsman Function



Batsman1: Mr. A



Batsman2: Mr. B

Note: The preceding code is covered in (Program Name: Chap8_ Example8.3.py)

In the previous code, we are accessing the indianbatsman module inside India subpackage of Cricket Package from the file. Under this module, we are accessing the name_indianbatsman function. So, output is displayed as shown above.

• games.py module and accessing indianbowler module inside India package using import only:

#games module



import Cricket.India.indianbowler



Cricket.India.indianbatsman.name_indianbowler ()

Output: On running Python games.py from the console, we have:

Indian Bowler Function

Concept of Packages in Python

Bowler1: Mr. C



Bowler2: Mr. D



271

Note: The preceding code is covered in (Program Name: Chap8_ Example8.4.py) In the previous code, we are accessing the indianbowler module inside India subpackage of Cricket Package from the file. Under this module, we are accessing the name_indianbowler function. So, output is displayed as shown above. • games.py module and accessing goalkeeper module inside Football package using import only:

#games module



import Football.goalkeeper



Football.goalkeeper.name_goalkeeper()

Output: On running Python games.py from the console, we have:

GoalKeeper Function



GoalKeeper1: Mr. R



GoalKeeper2: Mr. S

Note: The preceding code is covered in (Program Name: Chap8_ Example8.5.py)

In the previous code, we are accessing the goalkeeper module of Football Package from the file. Under this module, we are accessing the name_ goalkeeper function. So, output is displayed as shown above.

• games.py module and accessing forward module inside Football package using import only:

#games module



import Football.forward

Football.forward.name_forward()

Output: On running Python games.py from the console:

Forward Function



Forward1: Mr. T



Forward2: Mr. U

272



Python for Everyone

Note: The preceding code is covered in (Program Name: Chap8_ Example8.6.py) In the previous code, we are accessing the forward module of Football Package from the file. Under this module, we are accessing the name_forward function. So, output is displayed as shown above. • games.py module and accessing raider module Kabaddi package using import inside only:

#games module



import Kabaddi.raider

Kabaddi.raider.name_raider()

Output: On running Python games.py from the console, we have:

Raider Function



Raider1: Mr. W



Raider2: Mr. X

Note: The preceding code is covered in (Program Name: Chap8_ Example8.7.py)

In the previous code, we are accessing the raider module of Kabaddi Package from the file. Under this module, we are accessing the name_raider function. So, output is displayed as shown above.

• games.py module and accessing defender module inside Kabaddi package using import only:

#games module



import Kabaddi.defender

Kabaddi.defender.name_defender()

Output: On running Python games.py from the console, we have:

Defender Function



Defender1: Mr. Y



Defender2: Mr. Z

Note: The preceding code is covered in (Program Name: Chap8_ Example8.8.py)

Concept of Packages in Python



273

In the previous code, we are accessing the defender module of Kabaddi Package from the file. Under this module, we are accessing the name_defender function. So, output is displayed as shown above.



However, there is also another approach to access multiple modules of different packages using from packname import modulename approach.

• games.py module and accessing othercountrybatsman module inside Cricket package using from … import:

#games module



from Cricket import othercountrybatsman

othercountrybatsman.name_othercountrybatsman()

Output: On running Python games.py from the console, we have:

Other Country Batsman Function



Batsman1: Mr. E



Batsman2: Mr. F

Note: The preceding code is covered in (Program Name: Chap8_ Example8.9.py) • games.py module and accessing othercountrybowler module inside Cricket package using from … import:

#games module



from Cricket import othercountrybowler

othercountrybowler.name_othercountrybowler()

Output: On running Python games.py from the console, we have:

Other Country Bowler Function



Bowler1: Mr. G



Bowler2: Mr. H

Note: The preceding code is covered in (Program Name: Chap8_ Example8.10.py) • games.py module and accessing indianbatsman module inside India subpackage present in Cricket package using from … import:

#games module



from Cricket.India import indianbatsman

274



Python for Everyone

indianbatsman.name_indianbatsman()

Output: On running Python games.py from the console, we have:

Indian Batsman Function



Batsman1: Mr. A



Batsman2: Mr. B

Note: The preceding code is covered in (Program Name: Chap8_ Example8.11.py) • games.py module and accessing indianbowler module inside India subpackage present in Cricket package using from … import:

#games module



from Cricket.India import indianbowler

indianbowler.name_indianbowler()

Output: On running Python games.py from the console:

Indian Bowler Function



Bowler1: Mr. C



Bowler2: Mr. D

Note: The preceding code is covered in (Program Name: Chap8_ Example8.12.py) • games.py module and accessing goalkeeper module inside Football package using from … import:

#games module



from Football import goalkeeper



goalkeeper.name_goalkeeper()

Output: On running Python games.py from the console, we have:

GoalKeeper Function



GoalKeeper1: Mr. R



GoalKeeper2: Mr. S

Note: The preceding code is covered in (Program Name: Chap8_ Example8.13.py)

Concept of Packages in Python



275

• games.py module and accessing forward module inside Football package using from … import:

#games module



from Football import forward

forward.name_forward()

Output: On running Python games.py from the console, we have:

Forward Function



Forward1: Mr. T



Forward2: Mr. U

Note: The preceding code is covered in (Program Name: Chap8_ Example8.14.py) • games.py module and accessing raider module inside Kabaddi package using import from … inside:

#games module



from Kabaddi import raider

raider.name_raider()

Output: On running Python games.py from the console, we have:

Raider Function



Raider1: Mr. W



Raider2: Mr. X

Note: The preceding code is covered in (Program Name: Chap8_ Example8.15.py) • games.py module and accessing defender module inside Kabaddi package using from … import:

#games module



from Kabaddi import defender

defender.name_defender()

Output: On running Python games.py from the console, we have:

Defender Function



Defender1: Mr. Y



Defender2: Mr. Z

276



Python for Everyone

Note: The preceding code is covered in (Program Name: Chap8_ Example8.16.py)

There is a third approach to access the objects (here functionnames only) inside multiple modules of different packages using from packname. modulename import funcname approach.

• games.py module and accessing name_othercountrybatsman function:

#games module



from Cricket.othercountrybatsman import name_othercountrybatsman

name_othercountrybatsman()

Output: On running Python games.py from the console, we have:

Other Country Batsman Function



Batsman1: Mr. E



Batsman2: Mr. F

Note: The preceding code is covered in (Program Name: Chap8_ Example8.17.py) • games.py module and accessing name_othercountrybowler function:

#games module



from Cricket.othercountrybowler import name_othercountrybowler

name_othercountrybowler()

Output: On running Python games.py from the console, we have:

Other Country Bowler Function



Bowler1: Mr. G



Bowler2: Mr. H

Note: The preceding code is covered in (Program Name: Chap8_ Example8.18.py) • games.py module and accessing name_indianbatsman function, we have:

#games module



from Cricket.India.indianbatsman import name_indianbatsman

name_indianbatsman()

Output: On running Python games.py from the console, we have:

Concept of Packages in Python

Indian Batsman Function



Batsman1: Mr. A



Batsman2: Mr. B



277

Note: The preceding code is covered in (Program Name: Chap8_ Example8.19.py) • games.py module and accessing name_indianbowler function:

#games module



from Cricket.India.indianbowler import name_indianbowler

name_indianbowler()

Output: On running Python games.py from the console, we have:

Indian Bowler Function



Bowler1: Mr. C



Bowler2: Mr. D

Note: The preceding code is covered in (Program Name: Chap8_ Example8.20.py) • games.py module and accessing name_goalkeeper function:

#games module



from Football.goalkeeper import name_goalkeeper



name_goalkeeper()

Output: On running Python games.py from the console, we have:

GoalKeeper Function



GoalKeeper1: Mr. R



GoalKeeper2: Mr. S

Note: The preceding code is covered in (Program Name: Chap8_ Example8.21.py) • games.py module and accessing name_forward function:

#games module



from Football.forward import name_forward

name_forward()

Output: On running Python games.py from the console, we have:

278



Python for Everyone



Forward Function



Forward1: Mr. T



Forward2: Mr. U

Note: The preceding code is covered in (Program Name: Chap8_ Example8.22.py) • games.py module and accessing name_raider function:

#games module



from Kabaddi.raider import name_raider

name_raider()

Output: On running Python games.py from the console, we have:

Raider Function



Raider1: Mr. W



Raider2: Mr. X

Note: The preceding code is covered in (Program Name: Chap8_ Example8.23.py) • games.py module and accessing name_defender function:

#games module



from Kabaddi.defender import name_defender

name_defender()

Output: On running Python games.py from the console, we have:

Defender Function



Defender1: Mr. Y



Defender2: Mr. Z

Note: The preceding code is covered in (Program Name: Chap8_ Example8.24.py)

Now, suppose there is a requirement to import all the modules in a package. This can be done using the following 2 methods.

• Method-1

#games module



from Cricket import othercountrybatsman, othercountrybowler

Concept of Packages in Python



279

othercountrybatsman.name_othercountrybatsman() othercountrybowler.name_othercountrybowler()

Output: On running Python games.py from the console, we have:

Batsman1: Mr. E



Batsman2: Mr. F



Other Country Bowler Function



Bowler1: Mr. G



Bowler2: Mr. H

Note: The preceding code is covered in (Program Name: Chap8_ Example8.25.py) • Method-2

Most of you might be thinking of the following approach:



#games module



from Cricket import *

othercountrybatsman.name_othercountrybatsman() othercountrybowler.name_othercountrybowler()

Output: On running Python games.py from the console, we have:

Traceback (most recent call last): File "games.py", line 3, in othercountrybatsman.name_othercountrybatsman() NameError: name 'othercountrybatsman' is not defined

Note: The preceding code is covered in (Program Name: Chap8_ Example8.26.py)

In the previous method, we have used * to import the modules. But in the package, the preceding method is not feasible as we will get the NameError, as shown. We were using import * in modules. So, we will be specifying all the above module names which shall be loaded in the __init__.py file of Cricket package.

• Cricket package __init__.py file

# Cricket package ---- __init__ module

280



Python for Everyone __all__ = ['othercountrybatsman','othercountrybowler']

games.py file for loading Cricket package modules:

#games module



from Cricket import *

othercountrybatsman.name_othercountrybatsman() othercountrybowler.name_othercountrybowler()

Output: On running Python games.py from the console, we have:

Batsman1: Mr. E



Batsman2: Mr. F



Other Country Bowler Function



Bowler1: Mr. G



Bowler2: Mr. H

Note: The preceding code is covered in (Program Name: Chap8_ Example8.27.py) • India subpackage __init__.py file

# Cricket Package --- India subpackage ---- __init__ module



__all__ = ['indianbatsman','indianbowler']

• Football package __init__.py file

# Football Package

---- __init__ module



__all__ = ['goalkeeper','forward']

• Kabaddi package __init__.py file

# Kabaddi Package

---- __init__ module



__all__ = ['raider','defender']

• games.py file for loading Cricket package, Indiasubpackage, Football package and Kabaddi package modules:

#games module



from Cricket import *



from Cricket.India import *



from Football import *



from Kabaddi import *

Concept of Packages in Python othercountrybatsman.name_othercountrybatsman() othercountrybowler.name_othercountrybowler() indianbatsman.name_indianbatsman() indianbowler.name_indianbowler()

goalkeeper.name_goalkeeper()

forward.name_forward() raider.name_raider() defender.name_defender()

Output: On running Python games.py from the console, we have:

Other Country Batsman Function



Batsman1: Mr. E



Batsman2: Mr. F



Other Country Bowler Function



Bowler1: Mr. G



Bowler2: Mr. H



Indian Batsman Function



Batsman1: Mr. A



Batsman2: Mr. B



Indian Bowler Function



Bowler1: Mr. C



Bowler2: Mr. D



GoalKeeper Function



GoalKeeper1: Mr. R



GoalKeeper2: Mr. S



Forward Function



Forward1: Mr. T



Forward2: Mr. U





281

282

Python for Everyone



Raider Function



Raider1: Mr. W



Raider2: Mr. X



Defender Function



Defender1: Mr. Y



Defender2: Mr. Z

Note: The preceding code is covered in (Program Name: Chap8_ Example8.28.py) Suppose there is a requirement to use the Football package forward module inside Kabaddi package defender module. Then the code inside defender.py file will be: #Kabaddi Package --- defender module from Football import forward def name_defender(): '''Kabaddi defender names are''' print("Defender Function") print("Defender1: Mr. Y") print("Defender2: Mr. Z") print()

forward.name_forward()

Output: The output can be seen in the following Figure 8.8:

Figure 8.8: Output

Concept of Packages in Python



283

Note: The preceding code is covered in (Program Name: Chap8_Example8.29.py) Using the previous approach on running, we are getting an error ModuleNotFoundError: No module named 'Football'. We are inside Kabaddi folder and trying to access the module of Football package. So, we will go back by one folder and then try to execute the previous file, as shown in Figure 8.9:

Figure 8.9: File Execution inside Games folder

However, we again received an error as there is no such file or directory. So, we will be using flags as we must run library module as a script. Just observe the following highlighted python help in Figure 8.10:

Figure 8.10: Highlighting Python help

Refer to the following snippet: usage: C:\Users\SAURABH\AppData\Local\Programs\Python\Python37\python. exe [option] ... [-c cmd | -m mod | file | -] [arg] ... Options and arguments (and corresponding environment variables): -b

:  issue warnings about str(bytes_instance), str(bytearray_ instance) and comparing bytes/bytearray with str. (-bb: issue errors)

-B

: don't write .pyc files on import; also PYTHONDONTWRITEBYTECODE=x

-c cmd : program passed in as string (terminates option list) -d

: debug output from parser; also PYTHONDEBUG=x

-E

: ignore PYTHON* environment variables (such as PYTHONPATH)

-h

: print this help message and exit (also --help)

284



Python for Everyone

-i

:  inspect interactively after running script; forces a prompt even if stdin does not appear to be a terminal; also PYTHONINSPECT=x

-I

: isolate Python from the user's environment (implies -E and -s)

-m mod : run library module as a script (terminates option list) -O

:  remove assert and __debug__-dependent statements; add .opt-1 before .pyc extension; also PYTHONOPTIMIZE=x

-OO

:  do -O changes and discard docstrings; add .opt-2 before .pyc extension

-q

:  don't print version and copyright messages on interactive startup

-s

:  don't add user site directory to sys.path; also PYTHONNOUSERSITE

-S

: don't imply 'import site' on initialization

-u

:  force the stdout and stderr streams to be unbuffered; this option has no effect on stdin; also PYTHONUNBUFFERED=x

-v

:  verbose (trace import statements); also PYTHONVERBOSE=x can be supplied multiple times to increase verbosity

-V

:  print the Python version number and exit (also --version) when given twice, print more information about the build

-W arg :  warning control; arg is action:message:category:module:lineno also PYTHONWARNINGS=arg -x

:  skip first line of source, allowing use of non-Unix forms of #!cmd

-X opt :  set implementation-specific option --check-hash-based-pycs always|default|never: control how Python invalidates hash-based .pyc files file

: program read from script file

-

:  program read from stdin (default; interactive mode if a tty)

arg ...: arguments passed to program in sys.argv[1:]

Concept of Packages in Python



285

Other environment variables: PYTHONSTARTUP

: file executed on interactive startup (no default)

PYTHONPATH

:  ';'-separated list of directories prefixed to the default module search path. The result is sys. path.

PYTHONHOME

:  alternate directory (or ;). The default module search path uses \python{major}{minor}.

PYTHONCASEOK

: ignore case in 'import' statements (Windows).

PYTHONIOENCODING

: Encoding[:errors] used for stdin/stdout/stderr.

PYTHONFAULTHANDLER : dump the Python traceback on fatal errors. PYTHONHASHSEED :  if this variable is set to 'random', a random value is used to seed the hashes of str, bytes and datetime objects. It can also be set to an integer in the range [0,4294967295] to get hash values with a predictable seed. PYTHONMALLOC

:  set the Python memory allocators and/or install debug hooks on Python memory allocators. Use PYTHONMALLOC=debug to install debug hooks.

PYTHONCOERCECLOCALE : if this variable is set to 0, it disables the locale coercion behavior. Use PYTHONCOERCECLOCALE=warn to request display of locale coercion and locale compatibility warnings on stderr. PYTHONBREAKPOINT :  if this variable is set to 0, it disables the default debugger. It can be set to the callable of your debugger of choice. PYTHONDEVMODE

: enable the development mode.

So, we will be using “python -m packageName.moduleName” to run the module from Games folder, as shown in Figure 8.11:

286



Python for Everyone

Figure 8.11: Using “python -m packageName.moduleName”

There is also one more method to get the same output. Just observe the code inside defender.py file, as shown: #Kabaddi Package --- defender module import sys sys.path.append("E:/python_progs/Games/Football") import forward def name_defender(): '''Kabaddi defender names are''' print("Defender Function") print("Defender1: Mr. Y") print("Defender2: Mr. Z") print() forward.name_forward()

Output: The output can be seen in Figure 8.12:

Figure 8.12: Output

Note: The preceding code is covered in (Program Name: Chap8_Example8.30.py)

Concept of Packages in Python



287

Conclusion

In this chapter, we first saw the meaning of modules and packages. Then we saw the basic steps to create a package in Python. We created a package, named Games folder, having 3 different packages as Cricket (subpackage: India), Football and Kabaddi package. Then we saw how to access multiple modules of different packages using ‘import', ‘from packname import modulename’ and ‘from packname.modulename import funcname’ approaches. Then we saw the method to import all the modules in a package. Finally, in the end, we saw the approach to import one package module inside another package module.

Points to remember

• Package can consist of modules and packages.

• Module means files and packages means folders.

• __init__.py file is created inside the folder to consider it as a Python package.

• For the syntax, import packName.modName, packName is the name of the package and modName is the module Name. • For the syntax, from packName.modName import funcName, packName is the name of the package, modName is the module name and funcName is the function name. • We can import all the modules in a package using *.

• To run the module from the main package folder, use the syntax python -m packageName.moduleName.

Questions

1. Explain the difference between module and packages in Python. 2. What are the basic steps to create a package in Python?

3. How to access multiple modules of different packages using ‘import' only?

4. How to access multiple modules of different packages using ‘from packname import modulename’ approach?

5. How to access multiple modules of different packages using ‘from packname. modulename import funcname' approach? 6. What are the methods to import all the modules in a package?

7. What is the approach to import one package module inside another package module?

Join our book's Discord space

Join the book's Discord Workspace for Latest updates, Offers, Tech happenings around the world, New Release and Sessions with the Authors: https://discord.bpbonline.com

Chapter 9

Numpy Introduction

Introduction

One of the essential libraries for data science and machine learning is numpy, which stands for numerical python library. We know that basic mathematical operations are performed by normal Python. Complex mathematical operations such as array creation, performing several operations on arrays, differential calculations, statistical operations, integral calculus operations are not at all supported by normal Python. So, all these operations will be performed by numpy. The basic data structure in numpy is n-dimensional array (ndarray). Numpy is written in the Python and C language. So, performance is high and due to high speed, it is the best choice for ML algorithms that has built-in data structures of Python-like list. Data is stored in array form in numpy. Numpy has an array data structure that we can use to hold data. Numpy-created arrays are referred to as “nd arrays.”

Structure

In this chapter, we will discuss the following topics: • Similarities between list and numpy array • Differences between list and numpy array • Numpy arrays creation

290



Python for Everyone

Objectives

By the end of this chapter, the reader will have an idea about what exactly numpy array is, and the similarities and dissimilarities between list and numpy array. We will be also looking into creating nd array creation using various approaches such as list, tuple data structures along with various functions with a Python snippet code. So, by the end of this chapter, the reader will be able to appreciate the concept of numpy arrays by looking into its properties and ways of creation and its importance for Machine Learning algorithms.

Similarities between list and numpy array The similarities between list and numpy array are as follows: • Data storage is possible with both.

• Both will preserve the order. The concepts of indexing and slicing are so applicable. • We can change the content of both, thus making both mutable.

Differences between list and numpy array The differences between list and numpy array are as follows:

• We have to install numpy explicitly whereas list is in-built type of Python.

• Homogeneous elements are present in array whereas list contains heterogeneous elements.

• Vector operations are performed on ndarray whereas we cannot perform vector operations on list. Refer to the following code snippet:

myl1 = [100,200,300,400]



myl1+2 # we cannot perform vector operations on list

Output:

TypeError: can only concatenate list (not "int") to list

Note: The preceding code is covered in (Program Name: Chap09_ Example9.1.py)

Observe the following code:



import numpy as mynp



mynp1_array = mynp.array([100,200,300,400])

Numpy Introduction



291

print(mynp1_array+200) # we can perform vector operations on ndarray

Output:

[300 400 500 600]

Note: The preceding code is covered in (Program Name: Chap09_ Example9.2.py) • Less memory is consumed by ndarray than list, as shown:

import numpy as mynp



import sys



myl1 = [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20]

myndarray = mynp. array([1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20])

print('List size is

=> ',sys.getsizeof(myl1))



print('ndarray size is a => ',sys.getsizeof(myndarray))

Output:

List size is

=>

224



ndarray size is a =>

184

Note: The preceding code is covered in (Program Name: Chap09_ Example9.3.py) • Arrays are faster compared to list, as shown:

import numpy as mynp



from datetime import datetime



myarr1 = mynp.array([4,5,6])



myarr2 = mynp.array([1,2,3])



#conventional python code



def dot_product(myarr1,myarr2):



myresult = 0



for x,y in zip(myarr1,myarr2):



myresult +=



return myresult



x*y

mybefore = datetime.now()

292



Python for Everyone for loop in range(1000000): dot_product(myarr1,myarr2)



myafter = datetime.now()



print('Total Time taken by conventional python:',myafter-mybefore)



#numpy library code



mybefore = datetime.now()



for loop in range(1000000):



mynp.dot(myarr1,myarr2) # numpy



myafter = datetime.now()



print('Total Time taken by Numpy Library:',myafter-mybefore)

Output:

Total Time taken by conventional python: 0:00:01.650488



Total Time taken by Numpy Library: 0:00:01.345035

Note: The preceding code is covered in (Program Name: Chap09_ Example9.4.py)

Numpy arrays creation

In this section, we shall see how ndarrays are created.

1-D array creation using list Observe the following code: # 1-D array creation from list import numpy as mynp myl1 = [100,200,300] print(f'myl1 type is: {type(myl1)}') myndarray = mynp.array(myl1) print(f'myndarray type is:  {type(myndarray)}') print(f"myndarray  {myndarray}")

Numpy Introduction



293

# nd-array properties print(f'Array dimensions is: {myndarray.ndim}') print(f'Data type of elements of array is :

{myndarray.dtype}')

print(f'Array size is : {myndarray.size}') print(f'Array shape is : {myndarray.shape}')

Output: myl1 type is: myndarray type is:  myndarray  [100 200 300] Array dimensions is: 1 Data type of elements of array is :

int32

Array size is : 3 Array shape is : (3,)

Note: The preceding code is covered in (Program Name: Chap09_Example9.5.py)

1-D array creation using tuple Observe the following code: # 1-D array creation from tuple import numpy as mynp myt1 = ('Ramesh','Mahesh','Suresh') print(f'myt1 type is: {type(myt1)}') myndarray = mynp.array(myt1) print(f'myndarray type is:  {type(myndarray)}') print(f"myndarray  {myndarray}")

Output: myt1 type is:

294



Python for Everyone

myndarray type is:  myndarray  ['Ramesh' 'Mahesh' 'Suresh']

Note: The preceding code is covered in (Program Name: Chap09_Example9.6.py)

2-D array creation using Nested lists Observe the following code: # 2-D array creation from list import numpy as mynp myl1 = [[100,200,300],[400,500,600],[700,800,900]] myndarray = mynp.array(myl1) print(f'myndarray type is:  {type(myndarray)}') print(f"myndarray  {myndarray}") # nd-array properties print(f'Array dimensions is: {myndarray.ndim}') print(f'Data type of elements of array is : print(f'Array size is : {myndarray.size}') print(f'Array shape is : {myndarray.shape}')

Output: myndarray type is:  myndarray  [[100 200 300] [400 500 600] [700 800 900]] Array dimensions is: 2 Data type of elements of array is : Array size is : 9 Array shape is : (3, 3)

int32

{myndarray.dtype}')

Numpy Introduction



295

Note: The preceding code is covered in (Program Name: Chap09_Example9.7.py) It is important to note that ndarray contains homogeneous elements. Ndarray upcasting will be performed while creation, if the list contains heterogeneous elements as shown: # list containing heterogenous elements import numpy as mynp myl1 = [100,200,310.5] myndarray = mynp.array(myl1) # float upcasting print(f'myndarray : {myndarray}') print(f'Elements data type is: {myndarray.dtype}')

Output: myndarray : [100.

200.

310.5]

Elements data type is: float64

Note: The preceding code is covered in (Program Name: Chap09_Example9.8.py)

Array creation with a particular dtype Observe the following code: import numpy as mynp # type int myndarray = mynp.array([100,200,310.5],dtype=int) print(myndarray) # type float myndarray = mynp.array([100,200,310.5],dtype=float) print(myndarray) # type bool: For number and non-empty string  True and for 0 and empty string  False

296

Python for Everyone



myndarray = mynp.array([100,200,310.5, 0, "","Hello"],dtype=bool) print(myndarray) # type complex myndarray = mynp.array([100,200,310.5],dtype=complex) print(myndarray) # type str myndarray = mynp.array([100,200,310.5],dtype=str) print(myndarray)

Output: [100 200 310] [100.

200.

[ True

True

310.5] True False False

True]

[100. +0.j 200. +0.j 310.5+0.j] ['100' '200' '310.5']

Note: The preceding code is covered in (Program Name: Chap09_Example9.9.py)

Object type array creation The parent of data types such as int, float, bool, complex and str are of object type. Here, the elements look like heterogeneous but elements data type is of ‘object’ type, as shown: import numpy as mynp myndarray = mynp.array([100,'Saurabh',110.5,False,6+8j], dtype=object) print(myndarray) print(f"Elements data type of myndarray is: {myndarray.dtype}")

Output: [100 'Saurabh' 110.5 False (6+8j)]

Numpy Introduction



297

Elements data type of myndarray is: object

Note: The preceding code is covered in (Program Name: Chap09_Example9.10. py)

1-D array creation with arange() function 1-D array can be created with arange() function, as shown: import numpy as mynp myndarray = mynp.arange(9) print(f'myndarray is: {myndarray}') print(f'Array dimensions is: {myndarray.ndim}') print(f'Data type of elements of array is :

{myndarray.dtype}')

print(f'Array size is : {myndarray.size}') print(f'Array shape is : {myndarray.shape}') print('-'*50) myndarray1 = mynp.arange(1,9) print(f'myndarray is: {myndarray1}') print(f'Array dimensions is: {myndarray1.ndim}') print(f'Data type of elements of array is :

{myndarray1.dtype}')

print(f'Array size is : {myndarray1.size}') print(f'Array shape is : {myndarray1.shape}') print('-'*50) myndarray2 = mynp.arange(1,9,2) print(f'myndarray is: {myndarray2}') print(f'Array dimensions is: {myndarray2.ndim}') print(f'Data type of elements of array is : print(f'Array size is : {myndarray2.size}')

{myndarray2.dtype}')

print(f'Array shape is : {myndarray2.shape}')

298



Python for Everyone

print('-'*50) myndarray3 = mynp.arange(1,9,3,dtype=float) print(f'myndarray is: {myndarray3}') print(f'Array dimensions is: {myndarray3.ndim}') print(f'Data type of elements of array is : print(f'Array size is : {myndarray3.size}')

{myndarray3.dtype}')

print(f'Array shape is : {myndarray3.shape}')

Output: myndarray is: [0 1 2 3 4 5 6 7 8] Array dimensions is: 1 Data type of elements of array is :

int32

Array size is : 9 Array shape is : (9,) myndarray is: [1 2 3 4 5 6 7 8] Array dimensions is: 1 Data type of elements of array is :

int32

Array size is : 8 Array shape is : (8,) myndarray is: [1 3 5 7] Array dimensions is: 1 Data type of elements of array is :

int32

Array size is : 4 Array shape is : (4,) myndarray is: [1. 4. 7.] Array dimensions is: 1 Data type of elements of array is : Array size is : 3

float64

Numpy Introduction



299

Array shape is : (3,)

Note: The preceding code is covered in (Program Name: Chap09_Example9.11. py)

Using linspace() This function will return linearly spaced values but in the specified interval. The syntax is: linspace(start, stop, num = 50, endpoint = True, retstep = False, dtype = None)

where, start: It is optional whose default value is 0 and depicts start of interval range. Stop: It depicts interval range end. Num: It is optional and depicts samples number to generate. Endpoint: If endpoint is False, then stop is excluded.

When endpoint is True, spacing calculation: (stop-start)/(num-1) When endpoint is False, (stop-start)/num retstep: The spacing between the points is denoted by retstep, whose default value

is False. If True, then value will be returned. Dtype: It is the output array type.

Let us see some examples: import numpy as mynp # evenly spaced values of no. of samples as 50 between 0 and 1 which includes both 0 and 1 print(mynp.linspace(0,1))

300



Python for Everyone

print('-'*50) # 3 evenly spaced values between 0 and 1 and also including both 0 and 1 print(mynp.linspace(0,1,3)) print('-'*50) # 3 evenly spaced values between 0 and 1 and also including 0 but excluding 1 print(mynp.linspace(0,1,3, endpoint=False)) print('-'*50) # 3 evenly spaced values between 0 and 1 and also including 0 but excluding 1 and returning spacing print(mynp.linspace(0,1,3, endpoint=False, retstep=True)) print('-'*50) # 10 values between 1 to 50 including both 1 and 50 with equally spacing int type values print(mynp.linspace(1,50,5, dtype=int, retstep=True)) print('-'*50)

Output: [0.

0.02040816 0.04081633 0.06122449 0.08163265 0.10204082

0.12244898 0.14285714 0.16326531 0.18367347 0.20408163 0.2244898 0.24489796 0.26530612 0.28571429 0.30612245 0.32653061 0.34693878 0.36734694 0.3877551

0.40816327 0.42857143 0.44897959 0.46938776

0.48979592 0.51020408 0.53061224 0.55102041 0.57142857 0.59183673 0.6122449

0.63265306 0.65306122 0.67346939 0.69387755 0.71428571

0.73469388 0.75510204 0.7755102

0.79591837 0.81632653 0.83673469

0.85714286 0.87755102 0.89795918 0.91836735 0.93877551 0.95918367 0.97959184 1. [0. [0.

]

0.5 1. ] 0.33333333 0.66666667]

Numpy Introduction (array([0.



301

, 0.33333333, 0.66666667]), 0.3333333333333333)

(array([ 1, 13, 25, 37, 50]), 12.25)

Note: The preceding code is covered in (Program Name: Chap09_Example9.12. py) Here, arange(): Based on the step value, elements will be considered in the specified range. Linspace(): The specified number of values will be included for the given range.

Keep in mind that:

• 0-D array: Single value: Scalar.

• 1-D array: 0-D arrays collection: Vector

• 2-D array: 1-D arrays collection: Matrix • 3-D array: 2-D arrays collection

• (20,): 1-D array containing 20 elements.

• (10,2): 2-D array containing 10 rows and 2 columns • (1,2,3): 3-D array containing • 1 number of 2-D array

• 2 number of rows in every 2-D array.

• 3 number of columns in every 2-D array. • Size: 1*2*3 = 6

Using zeros() Observe the following code: import numpy as mynp # 1-D array with zeros print(mynp.zeros(5)) print('-'*50) # 2-D array with zeros print(mynp.zeros((3,3)))

302



Python for Everyone

print('-'*50) # 3-D array with zeros print(mynp.zeros((1,2,3))) print('-'*50) # 4-D array with zeros print(mynp.zeros((1,2,3,4))) print('-'*50) # 1 no. of 3-D arrays # 2 no. of every 3-D 2 2-D arrays # 3 no. of every 2-D array containing 3 rows # 4 no. of every 2-D array containing 4 columns

Output: [0. 0. 0. 0. 0.] [[0. 0. 0.] [0. 0. 0.] [0. 0. 0.]] [[[0. 0. 0.] [0. 0. 0.]]] [[[[0. 0. 0. 0.] [0. 0. 0. 0.] [0. 0. 0. 0.]] [[0. 0. 0. 0.] [0. 0. 0. 0.] [0. 0. 0. 0.]]]]

Numpy Introduction



303

Note: The preceding code is covered in (Program Name: Chap09_Example9.13. py)

Using ones() This function is exactly the same as zeros function. A new array of given shape and data type is returned where the element’s value is set to 1, as shown: import numpy as mynp # 1-D array with ones print(mynp.ones(5)) print('-'*50) # 2-D array with ones print(mynp.ones((5,2), dtype = int)) print('-'*50) # 3-D array with ones print(mynp.ones((1,2,3), dtype = int)) print('-'*50)

Output: [1. 1. 1. 1. 1.] [[1 1] [1 1] [1 1] [1 1] [1 1]] [[[1 1 1] [1 1 1]]]

Note: The preceding code is covered in (Program Name: Chap09_Example9.14. py)

304



Python for Everyone

Using full() The full function of numpy will return a new array of given shape and type, which is filled with fill_value. Just type the following command: import numpy as mynp print(help(mynp.full))

To view the parameters and its functionality, type: full(shape, fill_value, dtype=None, order='C', *, like=None)

Observe the following code: import numpy as mynp # 1-D array print(mynp.full(shape=8,fill_value=2)) print('-'*50) # 2-D array print(mynp.full((2,3),fill_value=2)) print('-'*50) # 3-D array print(mynp.full((2,2,3),2)) # fill_value = 2 print('-'*50)

Output: [2 2 2 2 2 2 2 2] [[2 2 2] [2 2 2]] [[[2 2 2] [2 2 2]] [[2 2 2] [2 2 2]]]

Numpy Introduction



305

Note: The preceding code is covered in (Program Name: Chap09_Example9.15. py)

Using eye() An identity matrix is generated and a 2-D array is returned with value as 1 on the diagonal and 0 elsewhere. Just type the following: import numpy as mynp print(help(mynp.eye))

To view the parameters and its functionality, type: eye(N, M=None, k=0, dtype=, order='C', *, like=None)

Here, N = Row number in the output. M = Column number in the output. K = Determines which diagonal should have a value filled with 1s. Default value is

0.

Here, 2-D arrays are always returned. It is not necessary to have the same number of rows and columns. So, row and column number need not be same. The value will be same as ‘N’ if ‘M’ value is omitted. Observe the following code: import numpy as mynp # 2-D array with default value k = 0 print(mynp.eye(3,4)) print('-'*50) # 2-D array with k = 1 print(mynp.eye(4, k=1)) print('-'*50) # 2-D array with k = 2 print(mynp.eye(4, k=2))

306



Python for Everyone

print('-'*50) # 2-D array with k = -1 print(mynp.eye(4, k=-1)) print('-'*50) # 2-D array with k = -2 print(mynp.eye(4, k=-2)) print('-'*50)

Output: [[1. 0. 0. 0.] [0. 1. 0. 0.] [0. 0. 1. 0.]] [[0. 1. 0. 0.] [0. 0. 1. 0.] [0. 0. 0. 1.] [0. 0. 0. 0.]] [[0. 0. 1. 0.] [0. 0. 0. 1.] [0. 0. 0. 0.] [0. 0. 0. 0.]] [[0. 0. 0. 0.] [1. 0. 0. 0.] [0. 1. 0. 0.] [0. 0. 1. 0.]] [[0. 0. 0. 0.] [0. 0. 0. 0.] [1. 0. 0. 0.] [0. 1. 0. 0.]]

Numpy Introduction



307

Note: The preceding code is covered in (Program Name: Chap09_Example9.16. py)

Using diag() A diagonal array is constructed or a diagonal is extracted using diag() function. Just type the following command: import numpy as mynp print(help(mynp.diag))

To view the parameters and its functionality, type: diag(v, k=0)

Observe the following code: import numpy as mynp # 2-D diagonal elements are extracted myndarray = mynp.arange(1,17).reshape(4,4) print(f"2-D Original array is : \n {myndarray}") print(f"Elements which diag(myndarray,k=0)}")

are

present

at

0-diagonal

:

{mynp.

print(f"Elements which diag(myndarray,k=1)}")

are

present

at

1-diagonal

:

{mynp.

print(f"Elements which diag(myndarray,k=2)}")

are

present

at

2-diagonal

:

{mynp.

print(f"Elements which diag(myndarray,k=-1)}")

are

present

at

-1-diagonal

:

{mynp.

print(f"Elements which diag(myndarray,k=-2)}")

are

present

at

-2-diagonal

:

{mynp.

print(f"Elements which diag(myndarray,k=3)}")

are

present

3-diagonal

:

{mynp.

print(f"Elements which diag(myndarray,k=-3)}")

are

present

-3-diagonal

:

{mynp.

print('-'*50)

at at

308

Python for Everyone



# 1-D construct a 2-D array using the provided elements and a diagonal array; # the remaining elements are all filled with zeros. Myndarray1 = mynp.array([100,200,300,400]) print(mynp.diag(myndarray1,k=0)) print('-'*50) myndarray2 = mynp.array([100,200,300,400]) print(mynp.diag(myndarray2,k=1)) print('-'*50) myndarray3 = mynp.array([100,200,300,400]) print(mynp.diag(myndarray3,k=-1)) print('-'*50)

Output: 2-D Original array is : [[ 1 [ 5

2

3

6

7

4] 8]

[ 9 10 11 12] [13 14 15 16]] Elements which are present at 0-diagonal : [ 1

6 11 16]

Elements which are present at 1-diagonal : [ 2

7 12]

Elements which are present at 2-diagonal : [3 8] Elements which are present at -1-diagonal : [ 5 10 15] Elements which are present at -2-diagonal : [ 9 14] Elements which are present at 3-diagonal : [4] Elements which are present at -3-diagonal : [13] [[100

0

0

0]

Numpy Introduction [

0 200

[ [ [[

0

0]

0

0 300

0]

0

0

0 100



309

0 400]] 0

0

0]

0

0] 0]

[

0

0 200

[

0

0

0 300

[

0

0

0

0 400]

[

0

0

0

0

0]]

[[

0

0

0

0

0]

[100

0

0

0

0]

[

0 200

0

0

0]

[

0

0 300

0

0]

[

0

0

0 400

0]]

Note: The preceding code is covered in (Program Name: Chap09_Example9.17. py)

Using empty() A new array of given shape and type is created without initializing entries. Use the empty() function to generate a dummy array. Type the following command: import numpy as mynp print(help(mynp.empty))

To view the parameters and its functionality, type: empty(shape, dtype=float, order='C', *, like=None)

Observe the following snippet: import numpy as mynp print(mynp.empty((3,3)))

310



Python for Everyone

Output: [[0.00000000e+000 0.00000000e+000 0.00000000e+000] [0.00000000e+000 0.00000000e+000 2.13436359e-321] [1.06811015e-306 1.24611470e-306 0.00000000e+000]]

Note: The preceding code is covered in (Program Name: Chap09_Example9.18. py)

Comparison between zeros and empty If we compare zeros and empty:

• We should choose zeroes if we simply need an array of zeros().

• If we only need an empty array for future use and do not care about data, we should choose empty().

• As compared to zeros array, the empty() array creation takes a very small amount of time. In other words, if data is not a concern, the empty() function is preferred over zeros() from a performance standpoint. Refer to the following code: # Performance comparison of zeros() and empty() function of numpy import numpy as mynp from datetime import datetime mybegin = datetime.now() myndarray = mynp.zeros((10000,200,300)) myafter = datetime.now() print('Time taken by zeros function of numpy is:',myafter-mybegin) print('-'*50) myndarray= None mybegin = datetime.now() myndarray = mynp.empty((10000,200,300))

Numpy Introduction



311

myafter = datetime.now() print('Time taken by empty function of numpy is:',myafter-mybegin)

Output: Time taken by zeros function of numpy is: 0:00:00.200177 -------------------------------------------------Time taken by empty function of numpy is: 0:00:00.010974

Note: The preceding code is covered in (Program Name: Chap09_Example9.19. py) Now, we know how to create ndarray using numpy. There are lot of other concepts in numpy such as creation of ndarrays with random data, indexing, slicing, advanced indexing, concept of view versus copy, iterating elements of ndarray, arithmetic operators, broadcasting, array manipulation functions, multiple arrays joining into a single array, arrays splitting, elements sorting of ndarrays , elements insertion into ndarray, elements deletion from ndarray, matrix class importance in numpy library and so on, that are unfortunately, out of scope of this book.

Conclusion

In this chapter, we initially saw what numpy array is. Then we discussed about similarities and dissimilarities between list and numpy array with examples. We have also seen various approaches to create ndarrays. 1-D array creation was done using list, and tuple with a Python code. 2-D array creation was done using nested lists, array creation with a particular dtype, 1-D array creation with arange function. We also saw various other functions such as linspace(), zeros(), ones(), full(), eye(), diag(), empty() functions with their meanings, followed by a Python snippet code. In the next chapter, we shall see about introduction to data visualization.

Points to remember

• Numpy array is written in C and Python language.

• Numpy array performance is high and is the best choice for ML algorithms. • Numpy arrays are faster and consumes less memory as compared to list.

• Homogeneous elements are present in numpy array where heterogeneous elements are present in list.

312



Python for Everyone

• We can perform vector operations on ndarray, whereas the same cannot be performed on list. • 1-D arrays can be created using list, tuple, arange(), linspace(), zeros(), ones(), full(), eye(), diag() and empty() functions

Questions

1. What is numpy array? Is it the best choice of ML?

2. Explain the similarities between list and numpy array.

3. Explain the dissimilarities between list and numpy array.

4. Explain 1-D array creation using list, tuple with a Python snippet code.

5. Explain 2-D array creation using nested lines with a Python snippet code.

6. Explain array creation with a particular dtype and with a Python snippet code. 7. Explain 1-D array creation with arange() function.

8. Explain linspace() function with a Python snippet code.

9. Explain zeros(), ones(), full(), eye(), diag(), empty() functions with a Python snippet code.

Join our book's Discord space

Join the book's Discord Workspace for Latest updates, Offers, Tech happenings around the world, New Release and Sessions with the Authors: https://discord.bpbonline.com

Chapter 10

Data Visualization Introduction Introduction

The representation of data in a graphical or visual format is known as data visualization. Data visualization aims to increase viewers’ access to, understanding of, and connection to complicated data sets. Patterns, trends, and relationships can be easily seen and expressed when data is presented visually. Charts, graphs, tables, maps, and other types of data visualization are only a few examples. Each kind of visualization works best with particular kinds of information or ideas. For instance, scatterplots can be used to depict the relationship between two variables, and line charts are frequently used to show patterns over time. In a variety of disciplines, including business, finance, science, and social sciences, data visualization is a crucial tool. It lets decision-makers swiftly grasp and analyze data, which can lead to more informed judgements. Data visualization is a vital tool for researchers and educators because it can make complicated ideas understandable to a wide audience.

314



Python for Everyone

Structure

In this chapter, we will discuss the following topics: • Python data visualization tools

• Line plot creation by passing 2 ndarrays

• Adding title, xlabel and ylabel to the line plot • Advanced Line Plot • linestyle property • color property • default color

• Peep in a shortcut way to set color, marker and linestyle • alpha property

• linewidth and markersize property • markerfacecolor property

• Customizing the figure size

• Plotting multiple lines in a same plot

Objectives

By the end of this chapter, the reader will know about different python data visualization tools available for creating plots out of which, we will learn about the most common matplotlib library. We will investigate creating line plots by using 2 ndarrays followed by addition of title, xlabel and ylabel to the code. We can set the marker, color and linestyle property with various available options. Also, in a shortcut way, we can set these properties using either mlc or clm form. Finally, we will be able to adjust the transparency using alpha attribute, followed by changing the width of the line, denoting the marker size in a plot and setting the face color of markers in plots by using lw, ms and mfc property.

Python data visualization tools

The most well-liked Python data visualization tools are as follows:

• Matplotlib: Matplotlib is a well-liked Python package for making animated, interactive, and static visualizations. Line charts, scatter plots, bar charts, histograms, and other visualizations are among the many that are offered.

Data Visualization Introduction



315

• Seaborn: Based on Matplotlib, Seaborn is a Python data visualization package that offers a more advanced interface for producing statistical visualizations. It offers a variety of sophisticated visualization techniques, including cluster maps, heatmaps, and violin plots.

• Plotly: Plotly is a Python visualization toolkit that offers customizable and interactive visualizations. Several other chart formats, such as scatter plots, bar charts, line charts, and more, are supported. • Bokeh: Bokeh is a Python library for building interactive web browser visualizations. It handles streaming and real-time data and offers a variety of visualization formats, such as scatter plots, line charts, and bar charts.

• Altair: With just a few lines of code, you can build interactive visualizations with Altair, a declarative visualization toolkit for Python. It enables you to effortlessly modify and alter your visualizations and supports a variety of chart styles, such as scatter plots, bar charts, and line charts. We will discuss about matplotlib in this chapter, to give you all a taste of it. To install matplotlib, type: pip install matplotlib

We will first use the following statement: import matplotlib.pyplot as myplt

A group of functions in Python called matplotlib.pyplot offers a high-level interface for making different visualizations, such as line charts, scatter plots, bar charts, histograms, and more. Python code frequently uses the import matplotlib.pyplot as myplt statement to import the pyplot module from the matplotlib library and alias it as myplt. This enables calling functions from the pyplot module with shorter and more convenient names. For example, calling myplt.plot() instead of matplotlib.pyplot.plot(). So, matplotlib is a library in which pyplot is a module name. With the help of pyplot module, we can create plots such as plot, bar, pie, hist and scatter for creating line plot, bar chart, pie chart, histograms, and scatter plot.

Line plot creation by passing 2 ndarrays Observe the following code:

# Line plot creation by passing 2 ndarrays import matplotlib.pyplot as myplt

316



Python for Everyone

import numpy as mynp x_axis = mynp.arange(1,11) y_axis = x_axis**3 myplt.plot(x_axis,y_axis) #(1,1),(2,8),(3,27) myplt.show()

Output: The output is shown in Figure 10.1:

Figure 10.1: Output of line plot creation using 2 ndarrays

Note: The preceding code is covered in (Program Name: Chap10_Example10.1.py) Figure 10.1 is a distinct window on the screen that Matplotlib uses to display the graphs, serving as the output container for graphics. To generate this figure object, use the plot() function. Container for plot | figure Figure 10.1 object includes multiple plots.

Data Visualization Introduction



317

Adding title, xlabel and ylabel to the line plot

We may add a title to the line plot using the myplt.title() function (“Cube function line plot”). To get the information related to title, type the following command: import matplotlib.pyplot as myplt help(myplt.title)

To study various parameters, type: title(label, fontdict=None, loc='center', pad=None, **kwargs)

Observe the following code: # Line plot creation by passing 2 ndarrays with title, xlabel and ylabel import matplotlib.pyplot as myplt import numpy as mynp x_axis = mynp.arange(1,11) y_axis = x_axis**3 myplt.plot(x_axis,y_axis) # line plot is drawn myplt.title('Cube Function Line Plot') # title is provided to the line plot myplt.xlabel('X axis-Value -------') # information is described about x-axis data myplt.ylabel('Cube of Y axis-Value -------') # information is described about y-axis data myplt.show() # Display the line plot

318



Python for Everyone

Output: The output is shown in Figure 10.2:

Figure 10.2: Output depicting xlabel, ylabel and title to the line plot

Note: The preceding code is covered in (Program Name: Chap10_Example10.2.py)

Advanced line plot

A line drawn on a graph can be distinguished by its color, style, width, transparency, and so on. They can be customized to meet our requirements. Data points on the line plot can be highlighted using the marker property. We must provide a keyword argument for the marker as shown: myplt.plot(x_axis,y_axis,marker='o')

Here, o stands for circle. The details against each marker character can be seen in the following Table 10.1: S No.

Character

2

,

1 3

Description

.

point marker

o

circle marker

pixel marker

Data Visualization Introduction S No.

Character

5

^

7

>

4 6 8 9

10 11

12

triangle_down marker


1, the graph plot becomes less transparent.

linewidth and markersize property

The linewidth property denotes the thickness/width of the line in a plot by using lw property. The markersize property denotes the marker size in a plot by using ms property. Observe the following code: # mlc form with lw and ms import matplotlib.pyplot as myplt import numpy as mynp x_axis = mynp.arange(1,11) y_axis = x_axis**3

Data Visualization Introduction



331

myplt.plot(x_axis,y_axis,'o:g', lw=5, ms = 10) myplt.title('Cube Function Line Plot') myplt.xlabel('X axis-Value -------') myplt.ylabel('Cube of Y axis-Value -------') myplt.show()

Output: The output is shown in Figure 10.11:

Figure 10.11: Output of mlc form with lw and ms property to the line plot

Note: The preceding code is covered in (Program Name: Chap10_Example10.12. py)

markerfacecolor property

This property will set the face color of markers in plots. Observe the following code: # mfc property import matplotlib.pyplot as myplt

332



Python for Everyone

import numpy as mynp x_axis = mynp.arange(1,11) y_axis = x_axis**3 myplt.plot(x_axis,y_axis,'o:g', lw=5, ms = 10, mfc='yellow') myplt.title('Cube Function Line Plot') myplt.xlabel('X axis-Value -------') myplt.ylabel('Cube of Y axis-Value -------') myplt.show()

Output: The output is shown in Figure 10.12:

Figure 10.12: Output with mfc property to the line plot

Note: The preceding code is covered in (Program Name: Chap10_Example10.13. py)

Data Visualization Introduction



333

Customizing the figure size

To display all default settings in matplotlib, we can use: • print(myplt.rcParams)

• print(myplt.rcParams.get('figure.figsize')) Observe the following code: # Getting the default figure size import matplotlib.pyplot as myplt print(myplt.rcParams.get('figure.figsize')) print(myplt.rcParams['figure.figsize'])

Output: [6.4, 4.8] [6.4, 4.8]

Note: The preceding code is covered in (Program Name: Chap10_Example10.14. py) We can customize the size of the figure by setting the figsize parameter by using the myplt.figure() function. A new figure can be created or an existing figure can be activated by using:

• num: num can be either int or str and specifies a unique identifier for the figure. • figsize: figsize is a tuple that specifies the width and height of the figure in inches.

Observe the following code: import matplotlib.pyplot as myplt import numpy as mynp myplt.figure(num=1,figsize=(8,4),facecolor='green') myndarray = mynp.arange(1,6) myplt.plot(myndarray,myndarray,'o-r') myplt.show()

334



Python for Everyone

Output: The output is shown in Figure 10.13:

Figure 10.13: Output of customizing the figure size

Note: The preceding code is covered in (Program Name: Chap10_Example10.15. py) Observe the following code: import matplotlib.pyplot as myplt import numpy as mynp myplt.figure(figsize=(4,4),facecolor='green') myndarray = mynp.arange(1,6) myplt.plot(myndarray,myndarray,'o-r') myplt.show()

Output:

Data Visualization Introduction



335

The output is shown in Figure 10.14:

Figure 10.14: Output with change of figure size to the line plot

Note: The preceding code is covered in (Program Name: Chap10_Example10.16. py)

Plotting multiple lines in a same plot Observe the following code:

# plotting multiple lines in the same plot import matplotlib.pyplot as myplt import numpy as mynp x_axis = mynp.arange(1,11) y_axis1 = x_axis y_axis2 = x_axis**2 y_axis3 = x_axis**3 myplt.plot(x_axis,y_axis1,'o:r') myplt.plot(x_axis,y_axis2,'o:g')

336



Python for Everyone

myplt.plot(x_axis,y_axis3,'o:b') myplt.title('Plotting multiple lines in the same plot') myplt.xlabel('X axis-Value -------') myplt.ylabel('Y axis-Value -------') myplt.show()

Output: The output is shown in Figure 10.15:

Figure 10.15: Output lotting multiple lines in the same plot

Note: The preceding code is covered in (Program Name: Chap10_Example10.17. py) There is lot to discuss in the matplotlib but in this book, we have only provided the taste of line plot using matplotlib.

Data Visualization Introduction



337

Conclusion

In this chapter, we initially saw various Python data visualization tools. Then we created a simple line plot by passing 2 ndarrays. We added code for title, xlabel and ylabel to the line plot. We investigated different marker, linestyle and color character while creating line plot. We saw Python code to view default color. Then we peeped in a shortcut way to set color, marker and linestyle property using either mlc or clm form with a python code. For transparency, we saw alpha property and then played by changing the thickness of the line and marker size in a plot by using linewidth and markersize property. We set the facecolor of markers in plots by using markerfacecolor property. We have learned how to customize the figure size. Finally, we learned about plotting multiple lines in a same plot.

Points of remember

• Matplotlib, seaborn , plotly , bokeh and altair are some of the data visualization tools. • pyplot is a module name in matplotlib library.

• To generate the figure object, use plot() function.

• Use title() function to add a title to the line plot.

• Use xlabel and ylabel function to add label on x and y axis. • Marker property to add marker to the line plot.

• We can create plots as solid, dotted and dashed suing linestyle property. • We can add colors to our plot using color property.

• The default color will be selected from the style circle if we do not specify a color. • We can set color, marker and linestyle to our plot using shortcut way as either mlc or clm form. • Transparency can be adjusted in our plot using alpha attribute

• The width of a line and market size in a plot can be denoted by lw and ms property. • The facecolor of markers in plots can be set by using mfc property.

338



Python for Everyone

Questions

1. Explain different types of Python data visualization tools.

2. Write a Python snippet code to create line plots by passing 2 nd arrays.

3. Write a Python snippet code to add title, xlabel and ylabel to the line plot. 4. What are different types of marker characters available?

5. Explain linestyle and color property with a Python code. 6. Write a python code to get the default color.

7. How to set color, marker and linestyle property in a shortcut way. Explain with a Python code.

8. Explain alpha, linewidth, markersize and markerfacecolor property with a Python snippet code. 9. Write a Python code for customizing the figure size.

10. Write a Python code to plot multiple lines in a same plot.

Join our book's Discord space

Join the book's Discord Workspace for Latest updates, Offers, Tech happenings around the world, New Release and Sessions with the Authors: https://discord.bpbonline.com

Chapter 11

Pandas Introduction

Introduction

We shall see our last chapter which is on Pandas. Pandas is a robust and popular Python package for data analysis and manipulation. For working with structured data, including tabular, time series, and heterogeneous data, it offers simple-to-use data structures and data analysis capabilities. With Matplotlib, SciPy, and Scikitlearn among the many additional data analysis libraries which Pandas is compatible with, it is built on top of NumPy. Series and DataFrame are the two main data structures in Pandas. Like a column in a spreadsheet or a SQL table, a “Series” is a one-dimensional named array that can carry any data type (integers, floats, texts, and so on). A two-dimensional labelled data structure called a “DataFrame” can carry a variety of data kinds in its columns of potentially different types. Data cleansing, merging and joining, reshaping, filtering, sorting, grouping, and aggregation are just a few of the many data manipulation and analysis features offered by Pandas. Additionally, it supports complex data analysis activities including machine learning, statistical modelling, and time series analysis.

340



Python for Everyone

Data science, machine learning, finance, economics, social science, and many more fields, all use Pandas extensively. Its popularity is a result of its simplicity, adaptability, and potent capabilities, which make it a crucial tool for Python data analysis. We can install pandas by typing the following pip command: pip install pandas

Let us learn some concepts on Pandas Series and Pandas DataFrame.

Structure

In this chapter, we will discuss the following topics: • Pandas Series

• Pandas DataFrame

Objectives

By the end of this chapter, the reader will know about Pandas introduction and why it is important in data science. We shall see the data structures in Pandas, which is series and DataFrame. We will be looking into their constructor, their methods of creation, the elements/data accessing and their aggregation with a Python snippet code. By the end of the chapter, we will be able to appreciate the importance of pandas and its application.

Pandas Series

A Pandas Series is a one-dimensional labelled array that may store any sort of data, including Python objects, floats, strings or integers. We can compare Pandas Series to a column in the excel sheet. We shall see some of the key concepts on Pandas Series in Python.

Pandas Series constructor We can create a Pandas series using the following constructor: pandas.Series(data=None, index=None, dtype=None, name=None, copy=False, fastpath=False)

Here,

Pandas Introduction



341

• data: This is the information that will be kept in the Series. It could be a scalar value, a dictionary, a list, or a NumPy array.

• index: This option details the Series’ index labels. It could be a dictionary, list, array, or range. The Series will be given a default integer index starting at 0, if no index is given. • dtype: The data type for the values in the Series is specified by this argument. Pandas will attempt to determine the data type from the input data, if it is not provided. • name: The Series’ name is specified by this argument.

• Copy: With this argument, you can decide whether or not the input data should be copied.

• fastpath: Most users should not use this argument because it is used for internal optimization only.

Creating Pandas Series by passing a list Observe the following code: import pandas as mypd # creating a Pandas Series from a list mylist = [1, 2, 3, 4, 5] mypd_series = mypd.Series(mylist) print(mypd_series)

Output: 0

1

1

2

2

3

3

4

4

5

dtype: int64

Note: The preceding code is covered in (Program Name: Chap11_Example11.1.py)

342



Python for Everyone

Creating Pandas Series by passing a dictionary Observe the following code: import pandas as mypd # creating a Pandas Series from a dictionary mydict = {'key1':1, 'key2':2, 'key3':3, 'key4':4, 'key5':5} mypd_series = mypd.Series(mydict) print(mypd_series)

Output: key1

1

key2

2

key3

3

key4

4

key5

5

dtype: int64

Note: The preceding code is covered in (Program Name: Chap11_Example11.2.py)

Creating Pandas Series by passing a numpy array Observe the following code: import pandas as mypd import numpy as mynp # creating a Pandas Series from a numpy array myndarray = mynp.array([11,12,13,14,15]) mypd_series = mypd.Series(myndarray) print(mypd_series)

Pandas Introduction



343

Output: 0

11

1

12

2

13

3

14

4

15

dtype: int32

Note: The preceding code is covered in (Program Name: Chap11_Example11.3.py)

Accessing elements in Pandas Series The individual elements of a Pandas Series can be accessed using either index label or by using the element’s integer location, as shown: import pandas as mypd # creating a Pandas Series mylist = [11, 12, 13, 14, 15] mypd_series = mypd.Series(mylist, index=['r', 's', 't', 'u', 'v']) print(mypd_series) # element access using index label print(mypd_series['u']) # element access using integer location print(mypd_series[3])

Output: r

11

s

12

t

13

344



u

14

v

15

Python for Everyone

dtype: int64 14 14

Note: The preceding code is covered in (Program Name: Chap11_Example11.4.py)

Pandas Series slicing A Pandas Series can be sliced using either index labels or integer locations, as shown: import pandas as mypd

# creating a Pandas Series mylist = [11, 12, 13, 14, 15] mypd_series = mypd.Series(mylist, index=['r', 's', 't', 'u', 'v']) print(mypd_series) # slicing the Pandas Series using index labels print(mypd_series['r':'u']) # slicing the Pandas Series using integer locations print(mypd_series[1:])

Output: r

11

s

12

t

13

u

14

v

15

Pandas Introduction



345

dtype: int64 r

11

s

12

t

13

u

14

dtype: int64 s

12

t

13

u

14

v

15

dtype: int64

Note: The preceding code is covered in (Program Name: Chap11_Example11.5.py)

Pandas Series filtering A Pandas Series can be filtered using Boolean indexing, as shown: import pandas as mypd # creating a Pandas Series mylist = [11, 12, 13, 14, 15] mypd_series = mypd.Series(mylist, index=['r', 's', 't', 'u', 'v']) print(mypd_series) print('-'*50) # filtering the Pandas Series using Boolean indexing my_bool_filter = mypd_series > 3 print(my_bool_filter) print('-'*50)

346



Python for Everyone

my_filtered_mypdseries = mypd_series[my_bool_filter] print(my_filtered_mypdseries)

Output: r

11

s

12

t

13

u

14

dtype: int64 -------------------------------------------------r

True

s

True

t

True

u

True

v

True

dtype: bool -------------------------------------------------r

11

s

12

t

13

u

14

v

15

dtype: int64

Note: The preceding code is covered in (Program Name: Chap11_Example11.6.py)

Pandas Introduction



347

Usage of apply method to Pandas Series A function can be applied to each element of a Pandas Series using the ‘apply’ method as shown: import pandas as mypd # creating a Pandas Series mylist = [11, 12, 13, 14, 15] mypd_series = mypd.Series(mylist, index=['r', 's', 't', 'u', 'v']) print(mypd_series) print('-'*50) # applying a function to each element of the Pandas Series def mycube(mynum): return mynum ** 3 mycube_pdseries = mypd_series.apply(mycube) print(mycube_pdseries)

Output: r

11

s

12

t

13

u

14

v

15

dtype: int64 -------------------------------------------------r

1331

s

1728

t

2197

348



Python for Everyone

u

2744

v

3375

dtype: int64

Note: The preceding code is covered in (Program Name: Chap11_Example11.7.py)

Aggregating of Pandas Series A Pandas Series can be aggregated using ‘min’, ‘max’, ‘sum’, ‘mean’ and ‘median’, as shown: import pandas as mypd

# creating a Pandas Series mylist = [11, 12, 13, 14, 15] mypd_series = mypd.Series(mylist, index=['r', 's', 't', 'u', 'v']) print(mypd_series) print('-'*50)

# Minimum value calculation of the Series print(f" Minimum value is: {mypd_series.min()}")

# Maximum value calculation of the Series print(f" Maximum value is: {mypd_series.max()}")

# Calculating the sum of the Series print(f" Sum is: {mypd_series.sum()}")

# Calculating the mean of the Series print(f" Mean is: {mypd_series.mean()}")

Pandas Introduction



349

# Calculating the median of the Series print(f" Median is: {mypd_series.median()}")

Output: r

11

s

12

t

13

u

14

v

15

dtype: int64 -------------------------------------------------Minimum value is: 11 Maximum value is: 15 Sum is: 65 Mean is: 13.0 Median is: 13.0

Note: The preceding code is covered in (Program Name: Chap11_Example11.8.py) These are only a few of the methods that the user can use to work with the Pandas Series. There are numerous additional ways to perform operation on Series, including filling in missing values, sorting, grouping, and more.

Pandas DataFrame

A two-dimensional size-mutable, potentially heterogeneous tabular data structure with labelled axes (rows and columns) is called a Pandas DataFrame . Data is aligned in a tabular fashion in rows and columns in a data frame, which is a two-dimensional data structure. The data, rows, and columns are the 3 primary components of a Pandas DataFrame. The data representation is like a SQL table or a spreadsheet. We shall see some of the key concepts on Pandas DataFrame in Python.

350



Python for Everyone

Pandas DataFrame constructor We can create a Pandas DataFrame by the usage of the following constructor: pandas.DataFrame(data=None, copy=False)

index=None,

columns=None,

dtype=None,

Here,

• data: The DataFrame’s data are specified using this argument. It might be a dictionary, a list of dictionaries, a NumPy array, or another DataFrame, among other data structures. An empty DataFrame is created if no data is provided. • index: The row labels for the DataFrame are specified using this argument. It could be a list, an array, or another DataFrame, among other data structures. A default integer index is created if index is not specified. • columns: The DataFrame’s column labels are specified using this option. It could be a list, an array, or another DataFrame, among other data structures. A default set of column labels is created if columns are not specified. • dtype: The data type for the columns in the DataFrame is specified using the dtype option. A dictionary that maps column names to data types or a single data type, such as int or float, are both acceptable.

• copy: To specify whether to copy the data, use the copy parameter. A new copy of the data is made if copy=True. The original data is used if copy=False.

Pandas DataFrame creation Observe the following code: import pandas as mypd # creating a DataFrame from a dictionary of lists mydict = {'myname': ['Alex', 'John', 'Michael', 'Tom'], 'myage': [35, 45, 55, 65], 'mycountry': ['UK', 'USA', 'Australia', 'Finland']} mypd_dataframe = mypd.DataFrame(mydict) print(mypd_dataframe) print('-'*50)

Pandas Introduction



351

# creating a DataFrame from a list of dictionaries mylist = [{'myname': 'Alex', 'myage': 35, 'mycountry': 'UK'}, {'myname': 'John', 'myage': 45, 'mycountry': 'USA'}, {'myname': 'Michael', 'myage': 55, 'mycountry': 'Australia'}, {'myname': 'Tom', 'myage': 65, 'mycountry': 'Finland'}] my_pd_dataframe2 = mypd.DataFrame(mylist) print(my_pd_dataframe2)

Output: myname

myage

mycountry

0

Alex

35

UK

1

John

45

USA

2

Michael

55

Australia

3

Tom

65

Finland

-------------------------------------------------myage

mycountry

myname

0

35

UK

Alex

1

45

USA

John

2

55

Australia

Michael

3

65

Finland

Tom

Note: The preceding code is covered in (Program Name: Chap11_Example11.9.py)

Accessing data in Pandas DataFrame Using indexing and slicing, we can access data in Pandas DataFrame, as shown: import pandas as mypd # creating a DataFrame from a dictionary of lists

352

Python for Everyone



mydict = {'myname': ['Alex', 'John', 'Michael', 'Tom'], 'myage': [35, 45, 55, 65], 'mycountry': ['UK', 'USA', 'Australia', 'Finland']} mypd_dataframe = mypd.DataFrame(mydict) print(mypd_dataframe) print('-'*50) # getting the second row of the DataFrame print(mypd_dataframe.loc[1]) print('-'*50) # getting the mycountry column of the DataFrame print(mypd_dataframe['mycountry']) print('-'*50) # getting the subset of the DataFrame print(mypd_dataframe.loc[0:2, ['myname', 'myage']]) print('-'*50)

Output: myname

myage

mycountry

0

Alex

35

UK

1

John

45

USA

2

Michael

55

Australia

3

Tom

65

Finland

-------------------------------------------------myname myage mycountry

John 45 USA

Pandas Introduction



353

Name: 1, dtype: object -------------------------------------------------0

UK

1

USA

2

Australia

3

Finland

Name: mycountry, dtype: object -------------------------------------------------myname

myage

0

Alex

35

1

John

45

2

Michael

55

--------------------------------------------------

Note: The preceding code is covered in (Program Name: Chap11_Example11.10. py)

Data modification in Pandas DataFrame Using indexing and slicing with various other methods, we can modify the data in Pandas DataFrame, as shown: import pandas as mypd

# creating a DataFrame from a dictionary of lists mydict = {'myname': ['Alex', 'John', 'Michael', 'Tom'], 'myage': [35, 45, 55, 65], 'mycountry': ['UK', 'USA', 'Australia', 'Finland']} mypd_dataframe = mypd.DataFrame(mydict) print(mypd_dataframe)

354

Python for Everyone



print('-'*50)

# modifying the mycountry of the second row mypd_dataframe.loc[1, 'mycountry'] = 'Russia' print(mypd_dataframe) print('-'*50)

# adding a new column to the DataFrame mypd_dataframe['Hobby'] = ['Playing Cricket', 'Listening Music', 'Reading Books', 'Cooking'] print(mypd_dataframe) print('-'*50)

# dropping the Hobby column from the DataFrame my_df_drop = mypd_dataframe.drop(columns=['Hobby']) print(my_df_drop) print('-'*50)

# sorting the DataFrame by myname in descending order my_df_sort = mypd_dataframe.sort_values(by='myname', ascending=False) print(my_df_sort) print('-'*50)

Output: myname

myage

mycountry

0

Alex

35

UK

1

John

45

USA

2

Michael

55

Australia

Pandas Introduction 3

Tom

65

Finland

-------------------------------------------------myname

myage

mycountry

0

Alex

35

UK

1

John

45

Russia

2

Michael

55

Australia

3

Tom

65

Finland

-------------------------------------------------myname

myage

mycountry

Hobby

0

Alex

35

UK

Playing Cricket

1

John

45

Russia

Listening Music

2

Michael

55

Australia

Reading Books

3

Tom

65

Finland

Cooking

-------------------------------------------------myname

myage

mycountry

0

Alex

35

UK

1

John

45

Russia

2

Michael

55

Australia

3

Tom

65

Finland

-------------------------------------------------myname

myage

mycountry

Hobby

3

Tom

65

Finland

Cooking

2

Michael

55

Australia

Reading Books

1

John

45

Russia

Listening Music

0

Alex

35

UK

Playing Cricket

--------------------------------------------------



355

356



Python for Everyone

Note: The preceding code is covered in (Program Name: Chap11_Example11.11. py)

Data aggregation in Pandas DataFrame Data can be aggregated in a Pandas DataFrame using methods such as count, sum, mean, median, and so on. Refer to the following: import pandas as mypd

# creating a DataFrame from a dictionary of lists mydict = {'myname': ['Alex', 'John', 'Michael', 'Tom'], 'myage': [35, 45, 55, 65], 'mycountry': ['UK', 'USA', 'Australia', 'Finland']} mypd_dataframe = mypd.DataFrame(mydict) print(mypd_dataframe) print('-'*50)

# getting the mean age of the DataFrame my_mean_age = mypd_dataframe['myage'].mean() print(my_mean_age) print('-'*50)

# getting the total number of rows in the DataFrame my_num_row_count = len(mypd_dataframe.index) print(my_num_row_count) print('-'*50)

Pandas Introduction



357

# grouping the DataFrame by country and getting the mean age for each group mygroup = mypd_dataframe.groupby(['mycountry']).mean() print(mygroup) print('-'*50)

Output: myname

myage

mycountry

0

Alex

35

UK

1

John

45

USA

2

Michael

55

Australia

3

Tom

65

Finland

-------------------------------------------------50.0 -------------------------------------------------4 -------------------------------------------------myage mycountry Australia

55

Finland

65

UK

35

USA

45

--------------------------------------------------

Note: The preceding code is covered in (Program Name: Chap11_Example11.12. py) We have only discussed about few methods of Pandas DataFrame and there are many more. You may check them out if interested.

358



Python for Everyone

Conclusion

In this chapter, we initially saw the introduction of Pandas package. Then we saw introduction to Pandas series, its constructor, its creation by passing a list, dictionary and numpy array. We learned how to access the elements in Pandas series using slicing, filtering. We used the apply method to each element of Pandas series, followed by aggregating of Pandas series. Then we learned about an introduction to Pandas DataFrame, its constructor, the DataFrame creation, accessing data using indexing and slicing in Pandas DataFrame, modifying the data in Pandas DataFrame. Finally, we learned about data aggregation in Pandas DataFrame.

Points to remember • Pandas is a robust and popular Python package for data analysis and manipulation. • A Pandas series is a 1-D labelled array for storing any sort of data including Python objects, floats, strings or integers. • A Pandas series can be created by passing a list, dictionary or a numpy array. • The individual elements of Pandas series can be accessed by using either index label or by using element’s integer location. It can be filtered by using Boolean indexing. • A Pandas series can be aggregated using min, max, sum, mean and median. • A Pandas DataFrame is a 2-D size-mutable, potentially heterogeneous tabular data structure with labelled axes (rows and columns). • We can access the data in Pandas DataFrame using indexing and slicing. • The data can be modified in Pandas DataFrame. • Data can be aggregated using methods such as count, sum, mean, median and so on.

Pandas Introduction



359

Questions

1. What is Pandas in data science? 2. What is Pandas series? Explain the syntax of Pandas series constructor? 3. How to create Pandas series? Explain with help of Python snippet code. 4. How to access elements in Pandas series? 5. Write a Python snippet code to aggregate Pandas series? 6. What is Pandas DataFrame? Explain the syntax of Pandas DataFrame constructor? 7. How to create Pandas DataFrame? Explain with help of Python snippet code. 8. How to access data in Pandas DataFrame? 9. How to modify data in Pandas DataFrame? 10. How data can be aggregated in Pandas DataFrame?

Join our book's Discord space

Join the book's Discord Workspace for Latest updates, Offers, Tech happenings around the world, New Release and Sessions with the Authors: https://discord.bpbonline.com

Index A

advanced line plot 318-320 alphabet pattern 104-107 alpha property 329, 330 Altair 315 B

Bokeh 315

break statement 91 examples 92-95 flowchart 92

built-in functions 184 C

cd command 9-11

character classes 147 examples 148-152

clear command 8 closures 199

creating 199-201

color property 322, 323

command line arguments 43

Python argparse module 49 Python getopt module 46 sys.argv 43-46

compile() function 144, 145 continue statement 95 examples 96-99 flowchart 96

control flow, in nested try-except finally block 134-138 cases 135

control flow, in try-except 118 cases 119, 120

control flow, in try-except and finally block cases 129-131

362



Python for Everyone

cp command 13

dynamic input

D

E

customized exception handling 116-118 data structures

dictionary data structure 244 list data structure 216 set data structure 238

tuple data structure 228, 229

data visualization 313

data visualization tools 314 Altair 315

Bokeh 315

Matplotlib 314

reading, from keyboard 36

else block

with try-except finally block 138-140

errors 112

runtime error 113, 114 syntax error 112, 113

eval function 41, 42 except block

possible combinations 125, 126

exception handling

customized exception handling 116118

Plotly 315

Seaborn 315

default arguments 190, 191

importance 114, 115

exception hierarchy 115

default color 323-325

exception information

dictionary comprehension

explicit line joining method

default except block 124, 125

printing, to console 120, 121 examples 17, 18

with for loop 252, 253

with for loop and if statement 253, 254 with for loop and nested if statement 254

with if else statement and for loop 255

dictionary creation

curly braces, using 245

dict() function, using 245

dictionary data structure 244

dictionary, accessing 247-249

dictionary comprehension 252 dictionary, creating 246, 247

dictionary, deleting 250, 251

empty dictionary, creating 245 key rules 244

F

figure size

customizing 333-335

finally except block 126-129 finditer() function 145-147

flow control statements 74

iterative statements 84, 85

selection or conditional statements 74 transfer statements 91

for loop 85

examples 85-88 flowchart 85

function arguments 186, 187

default arguments 190, 191

Index keyword arguments 189, 190

keyword variable length arguments (kwargs) 194-196 positional arguments 187, 188

variable length arguments 192, 193

functions 182

built-in functions 184 example 183, 184

passing as parameter 201, 202

user defined functions 185, 186

if-else statement 77 flowchart 77, 78

if statement 74

flowchart 75, 76

immutability 29

with reference to, fundamental data types 29-32

implicit line joining method 16 curly braces, using 16

parenthesis, using 16, 17

square brackets, using 16

findall 167

in keyword

match 162-164

input() function 37-40

fullmatch 164, 165 search 166, 167 split 171-173 sub 168, 169

usage 100, 101

Integrated Development and Learning Environment (IDLE) 6 iterative statements 84 for loop 85-88

subn 169-171

while loop 88-91

fundamental data types 29 G

games package structure

K

keyword arguments 189, 190

accessing, with different approaches 266-286

keywords 13, 14

import, using 262

L

from import, using 263-266 objects, accessing 263, 264

Garbage Collector (GC) 30 global variable 203-207

363

flowchart 79-81

functions, of re module escape 173, 174



keyword variable length arguments (kwargs) 194-196 lambda functions

passing, to another function 212

line joining methods 15

explicit line joining method 17

I

identifiers 15

if-elif-else ladder 81-83 flowchart 84

if-elif-else statement

implicit line joining method 16

line plot

advanced line plot 318-320

creating, by passing 2 ndarrays 315, 316

364



Python for Everyone

title, adding 317, 318

xlabel, adding 317, 318

ylabel, adding 317, 318

linestyle property 320, 321

linewidth property 330, 331 list array and numpy array differences 290-292 similarities 290

list comprehension 223

lists, versus immutability 219 nested list comprehension, with for loop 227, 228

versus, tuple data structure 237

local variables 202, 203 loop patterns 102

alphabet pattern 104-107 number pattern 107, 108 start pattern 102

with for loop 224

ls command 7

with for loop and nested if statement 225, 226

M

with for loop and if statement 225

with if else statement and for loop 226, 227

list data structure 216

empty list, creating 216, 217

ls -l command 8 markerfacecolor property 331, 332 markersize property 330, 331 Matplotlib 314

metacharacters 174

list comprehension 223, 224

alteration (|) 177

list, creating when elements are known 217

braces {} 177

list, creating with list() function 218

group () 178

list elements, accessing 219

plus (+) 176

list, creating 216

backslash (\) 178 caret (^) 175

list, creating with dynamic input 217

dollar ($) 175

list, creating with split() function 218

period (.) 175

list elements, accessing by index 219

question mark (?) 177

list elements, accessing by index and for loop 221

list elements, accessing by index and while loop 221, 222 list elements, accessing by list slicing 222, 223

list elements, accessing by negative index 220 list elements, accessing by positive index 220

R prefix 178

square brackets [] 174, 175 star (*) 176

mkdir command 9

mnemonic variable names 27, 28 multiple except blocks trying with 121, 122

multiple lines

plotting, in same plot 335, 336

Index mv command 11, 12 N

Nested function 197-199

nested lambda functions 211, 212

nested try-except finally block 131-134 control flow 134

non changeable behavior 29 non-local variable 207, 208 number pattern 107-109 numpy 289

numpy arrays creation

1-D array creation, with arrange() function 297

1-D array creation, with list 292

1-D array creation, with tuple 293 2-D array creation, with nested lists 294, 295

diag() function, using 307-309 empty() function, using 309 eye() function, using 305

full() function, using 304

linspace() function, using 299-301 object type array creation 296 ones() function, using 303

with particular dtype 295, 296 zeros() function, using 301

zeros, versus empty 310, 311



creating 350, 351

creating, with constructor 350 data, accessing 351-353

data aggregation 356, 357

data modification 353-356

Pandas Series 340

aggregating 348, 349

apply method, using 347

creating, by passing dictionary 342 creating, by passing list 341

creating, by passing numpy array 342 creating, with constructor 340, 341 elements, accessing 343 filtering 345, 346 slicing 344, 345

pass statement 100 Plotly 315

positional arguments 187, 188

pre-defined character classes 153 examples 153-157

print function 18, 19 using styles 19-24

pwd command 7 PyPy version 5 Python 1

benefits 2-4

installation 5, 6 limitations 4, 5

P

packages 260

creating 261

games package structure 262 overview 260

Pandas 339

Pandas DataFrame 349

365

program, running 6, 7 URL 2

usage 4

Python argparse module 49, 50 abbreviations, allowing 59-61

abbreviations, disallowing 59-61

366



Python for Everyone

append action 59

recursive function 208, 209

data type 54

rm command 11

argument help 52, 53 dest action 58

optional and positional arguments, using 57 optional arguments 55

positional arguments 50, 51 positional arguments default values 51, 52

short names, for optional arguments 56 with required 57, 58

Python exception hierarchy 115, 116 Python getopt module 46

Exception getopt.GetoptError 47 long form options 48, 49

short form options 47, 48

Python lambda functions 210, 211

nested lambda functions 211, 212 passing, to another function 212

Python multiline strings 62-64 Python string access 64 index, using 64, 65

slice operator, using 65, 66

Python sys.argv 43-46

Python Virtual Machine (PVM) 2, 115, 122 Q

quantifiers

examples 159-162 using 158

R

raw_input function 36 recursion 208

reserved words 13, 14 rm -rf command 11

runtime error 113, 114 S

Seaborn 315

selection or conditional statements if-elif-else ladder 81-84

if-elif-else statement 79-81 if-else statement 77, 78 if statement 74-76

set comprehension 240 with for loop 241

with for loop and if statement 241, 242 with for loop and nested if statement 242, 243 with if else statement and for loop 243, 244

set data structure 238

set comprehension 240 set creation 239, 240

shortcut notation 325 clm form 327 mlc form 326

output with default color 328

single except block

multiple exceptions, handling with 122-124

slice operator

in backward direction 66-70 in forward direction 66 using 65

Stack-Overflow 5 star patterns 102

Index stars, printing in pyramid shape 102-104

strings 35, 61

syntax error 112, 113 T

touch command 10 transfer statements

break statement 91-95

continue statement 95-99 pass statement 100

try except block 40, 41 tuple creation 229

empty tuple creation 229, 230

multiple valued tuple creation 231 single valued tuple creation 230 tuple() function, using 231, 232

tuple data structure 228, 229

tuple comprehension 236, 237 tuple creation 229

tuple elements, accessing 232

tuple, versus immutability 236 versus, list data structure 237



367

tuple elements

accessing, with index 232

accessing, with index and for loop 233, 234 accessing, with index and while loop 234, 235

accessing, with negative index 232, 233 accessing, with positive index 232

accessing, with tuple slicing 235, 236 U

user-defined functions 185, 186 V

variable 24, 25 rules 25, 26

variable length arguments 192, 193 Visual Studio Code (VsCode) 7 W

while loop 88

examples 89-91 flowchart 89

368



Python for Everyone