Programming and Scientific Computing in Python [6.0 ed.]

Citation preview

AE1205

Programming and Scientific Computing in Python

Faculty of Aerospace Engineering

J.M. Hoekstra and J. Ellerbroek

Version 6.0

Preface The first version of this reader was developed for, and during the pilot of, the Programming course in the first year of the BSc program Aerospace Engineering at the Delft University of Technology in 2012. Originally it was written for Python 2 and then converted to Python 3. The goal of the Python programming course is to enable the student to: • write a program for scientific computing • develop models • analyse behaviour of the models, for instance using plots • visualise models by animating graphics The course assumes some mathematical skills, but no programming experience whatsoever. This document is provided as a reference for the elaboration of the assignments. The reader is encouraged to read through the relevant chapters applicable to a particular problem. For later reference, many tables, as well as some appendices with quick reference guides, have been included. These encompass the most often used functions and methods. For a complete overview, there is the excellent documentation as provided with Python in the IDLE Help menu, as well as the downloadable and on-line documentation for the Python modules NumPy, SciPy, Matplotlib and Pygame. Also, the set-up of the present course is to show the appeal of programming. Having this powerful tool at hand allows the reader to use the computer as a ‘mathematical slave’. And by making models, one basically has the universe in a sandbox at one’s disposal: Any complex problem can be programmed and displayed, from molecular behaviour to the motion in a complex gravity field in space. An important ingredient at the beginning of the course is the ability to solve mathematical puzzles and numerical problems. Also the very easy to use graphics module Pygame module has been included in this reader. This allows, next to the simulation of a physical problem, a real-time visualization and some control (mouse and keyboard) for the user, which also adds some fun for the beginning and struggling programmer in the form of visual feedback. Next to the mathematical puzzles, challenges (like Project Euler and the Python challenge) and simulations and games, there is a programming contest included in the last module of the course for which there is a prize for the winners. Often students surprise me with their skills and creativity in such a contest by submitting impressive simulations and games. Also check out the accompanying videos: Search for “AE1205” on Youtube. Many thanks to the students and teaching assistants, whose questions, input and feedback formed the foundation for this reader. J.M. Hoekstra and J. Ellerbroek Delft, March 2022

iii

Reading guide This reader is intended to be used as reference material during the course, as well as during the exams. To guide you through the reader, the following is useful to know: Throughout the reader, most material counts as exam material. Some parts, however, are optional. You can recognise optional parts as follows: • Within chapters, small optional parts (that for instance dive deeper into a concept, or provide an alternative approach) are encapsulated in blue boxes that are marked extra: Extra: The explanation in this box isn’t exam material Very deep contemplations go here! • When an entire chapter is optional, its chapter number will be marked with a red asterisk: ∗ . Coloured boxes are also used to mark or reiterate something that is important: NB: An important remark Make sure not to skip these boxes! In each chapter, several sections have exercises related to the presented material. The answers to these numbered exercises can be found in several places: • In Appendix A. • As Python files on https://github.com/TUDelft-AE-Python/ae1205-exercises/ in the folder pyfiles. • As Jupyter notebooks on https://github.com/TUDelft-AE-Python/ae1205-exercises/ in the folder notebooks. The exam of the Python course is an open-book exam: you are allowed to use an original bound copy of the reader during the entire exam. Sometimes it’s useful to read something back in detail in one of the chapters, but often just a quick reminder will be enough. In the latter case you can use the course cheat sheet in Appendix C.

v

Contents 1 Getting started 1.1 What is programming? . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 What is Python? . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Installing Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3.1 Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3.2 macOS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3.3 Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3.4 Explanation of the installed modules . . . . . . . . . . . . . 1.3.5 Configuring the IDLE editor . . . . . . . . . . . . . . . . . . 1.3.6 Working environments: IDLE, PyCharm, Spyder, and others 1.3.7 Documentation . . . . . . . . . . . . . . . . . . . . . . . . . 1.4 Sneak preview: Try the Python language yourself . . . . . . . . . . 1.5 Variables and the assignment statement . . . . . . . . . . . . . . . 1.6 Finding your way around: many ways in which you can get help . . 1.6.1 Method 1: Using help(”text”) or interactive help() . . 1.6.2 Method 2: Python documentation in Help Pull-down menu . 1.6.3 Method 3: Get help from the huge Python community . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

1 1 2 3 3 5 6 6 6 7 9 10 11 13 13 14 14

2 Variable assignment and types 2.1 Assignment and implicit type declaration . . . . . . . . . . 2.2 Short-hand when using original value with an operator. . . 2.3 Number operations in Python . . . . . . . . . . . . . . . . 2.4 Float: variable type for floating point values. . . . . . . . . 2.5 Int: variable type for round numbers like counter or index). 2.6 Complex numbers . . . . . . . . . . . . . . . . . . . . . . 2.7 String operations . . . . . . . . . . . . . . . . . . . . . . . 2.8 Booleans and logical expressions . . . . . . . . . . . . . . 2.9 The list type: an ordered collection of items. . . . . . . . . 2.9.1 What are lists? . . . . . . . . . . . . . . . . . . . . 2.9.2 Indexing and slicing, list functions and delete . . . . 2.9.3 Quick creation of lists with repeating values . . . . 2.9.4 List methods . . . . . . . . . . . . . . . . . . . . . 2.10 Some useful built-in functions . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

17 18 19 19 20 20 22 22 23 25 25 26 27 28 29

3 Python syntax: Statements 3.1 Assignment . . . . . . . . . . . . . . . . . . . 3.2 The print statement . . . . . . . . . . . . . . . 3.2.1 Using print . . . . . . . . . . . . . . . 3.2.2 Formatting your output . . . . . . . . . 3.3 The input function . . . . . . . . . . . . . . . 3.4 Checking conditions: the if-statement . . . . . 3.5 Iteration using a for-loop . . . . . . . . . . . . 3.5.1 Additional tools for for-loops: itertools . 3.6 Iteration using a while-loop. . . . . . . . . . . 3.7 Controlling a loop: break and continue . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

31 31 33 33 34 35 36 38 39 40 41

4 Making your code reusable and readable

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

43 vii

viii

Contents

5 Extending Python: Using modules 5.1 The import statement. . . . . . . . . . . . . . . . . . . . . . . . 5.1.1 Importing from a module . . . . . . . . . . . . . . . . . . 5.1.2 Renaming imports within the import statement . . . . . . 5.2 Math functions in Python: the math module. . . . . . . . . . . . 5.2.1 Overview of functions and constants in the math module 5.3 The random module . . . . . . . . . . . . . . . . . . . . . . . . 5.4 Exploring other modules . . . . . . . . . . . . . . . . . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

47 47 47 48 48 49 50 50

6 Defining your own functions and modules 6.1 Defining a function: The def statement . . . . . . 6.1.1 Accepting arguments in your function . . . 6.1.2 Providing default values . . . . . . . . . . 6.1.3 Passing arguments by order or by name . 6.1.4 Returning data from your function . . . . . 6.2 Variable names: Arguments and return variables. 6.3 Returning a result by updating an argument . . . 6.4 Managing a project: defining your own modules .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

51 51 52 53 53 53 55 57 57

7 File input/output and string handling 7.1 Opening and closing files . . . . . . . . . . . . . . . 7.1.1 Easy opening and closing: the with statement 7.2 Reading data from the file object . . . . . . . . . . . 7.2.1 Iteratively reading data . . . . . . . . . . . . . 7.2.2 If you need more control . . . . . . . . . . . . 7.2.3 A full example: Reading a formatted file . . . 7.3 Writing to files. . . . . . . . . . . . . . . . . . . . . . 7.4 Useful string methods . . . . . . . . . . . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

61 61 61 61 62 63 63 65 66

8 Matplotlib: Data visualisation in Python 8.1 The basics: plotting a 2D XY-graph . . . . . . . . . . . . 8.2 Multiple plots and setting their scaling: subplot and axis . 8.3 Interactive plots . . . . . . . . . . . . . . . . . . . . . . . 8.4 3D and contour plots . . . . . . . . . . . . . . . . . . . . 8.5 Plotting on a map . . . . . . . . . . . . . . . . . . . . . . 8.6 Overview of essential pyplot functions . . . . . . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

69 69 71 74 74 76 79

. . . . . . . .

9 Numerical integration 81 9.1 Euler’s method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 9.2 Adding dimensions and the atan2 function . . . . . . . . . . . . . . . . . . . . . 84 10 NumPy and SciPy: Scientific programming with arrays and matrices 10.1 Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.1.1 Creating NumPy arrays . . . . . . . . . . . . . . . . . . . . . 10.1.2 Indexing and slicing NumPy arrays . . . . . . . . . . . . . . . 10.2 Logical expressions using arrays . . . . . . . . . . . . . . . . . . . . 10.3 Speeding it up: Vectorising your code with NumPy . . . . . . . . . . . 10.4 Matrix operations: Linear algebra with NumPy . . . . . . . . . . . . . 10.5 Genfromtxt: Easy data-file reading using NumPy. . . . . . . . . . . . 10.6 SciPy: a toolbox for scientists and engineers . . . . . . . . . . . . . . 10.6.1 SciPy example: Polynomial fit on noisy data . . . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

87 87 88 89 90 92 95 97 98 99

103 11 Tuples, dictionaries, and sets 11.1 Tuples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 11.2 Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 11.3 Dictionaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 12 Pygame: Animation, visualisation and controls 107 12.1 The basics: importing and initialising . . . . . . . . . . . . . . . . . . . . . . . . . 107 12.2 Setting up the Pygame window . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

Contents

ix

12.3 12.4 12.5 12.6 12.7

Surfaces and Rectangles . . . . . . . . . . . . . . . . Bitmaps and images . . . . . . . . . . . . . . . . . . . Drawing shapes and lines . . . . . . . . . . . . . . . . When your drawing is ready: flipping the display . . . . Timing and the game loop . . . . . . . . . . . . . . . . 12.7.1 Method 1: Fixed time-step . . . . . . . . . . . . 12.7.2 Method 2: Variable time-step . . . . . . . . . . 12.7.3 The termination condition of the game loop . . . 12.8 Processing inputs: Keyboard, mouse, and other events 12.9 Overview of basic Pygame functions . . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. 109 . 110 . 111 . 112 . 113 . 113 . 114 . 115 . 116 . 117

13 Creating your own types: Object-Oriented Programming 119 13.1 Implementing operator behaviour for your type . . . . . . . . . . . . . . . . . . . . 121 14 Graphical User Interface building in Python 14.1 TkInter. . . . . . . . . . . . . . . . . . . . 14.2 PyQt. . . . . . . . . . . . . . . . . . . . . 14.3 wxPython . . . . . . . . . . . . . . . . . . 14.4 PyGtk . . . . . . . . . . . . . . . . . . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

123 . 123 . 125 . 125 . 126

15 Reading and writing data files: beyond the basics 15.1 Tabular data: CSV files. . . . . . . . . . . . . . . . . . . . . . . . 15.1.1 Doing it yourself . . . . . . . . . . . . . . . . . . . . . . . 15.1.2 Doing it with a library . . . . . . . . . . . . . . . . . . . . . 15.2 Tabular data: Excel spreadsheets . . . . . . . . . . . . . . . . . . 15.2.1 Openpyxl: Reading and writing directly to XLSX Excel files 15.2.2 Xlrd and xlwt: Reading and writing the old .xls format . . . 15.2.3 Reading and writing Excel files with Pandas . . . . . . . . 15.3 Hierarchical data: JSON files . . . . . . . . . . . . . . . . . . . . 15.4 Hierarchical data: XML files . . . . . . . . . . . . . . . . . . . . . 15.5 Binary data: MATLAB mat files . . . . . . . . . . . . . . . . . . . 15.6 Binary data: PDF files . . . . . . . . . . . . . . . . . . . . . . . . 15.7 Binary data: Any binary file . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

127 . 127 . 128 . 130 . 131 . 131 . 132 . 133 . 134 . 136 . 137 . 138 . 139

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

16 Exception handling in Python

143

17 Creating self-contained bundles of your program using PyInstaller 17.1 Making an executable of your program . . . . . . . . . . . . . . . 17.1.1 Example Mazeman game . . . . . . . . . . . . . . . . . . 17.1.2 Adding additional files to make your program run. . . . . . 17.2 Making a setup installer for your program . . . . . . . . . . . . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

147 . 147 . 147 . 148 . 148

18 Version Control 18.1 Basic concepts of version control . . . 18.1.1 Repository, working copy. . . . 18.1.2 Revision, commit, check-out . . 18.1.3 Comparing source code . . . . 18.1.4 Distributed vs centralised VCS. 18.1.5 Branches and merging . . . . . 18.2 Overview of Git commands . . . . . . 18.3 Overview of Subversion commands . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

149 . 149 . 149 . 149 . 150 . 151 . 152 . 153 . 154

19 Beyond the course: Exploring applications of Python 19.1 Alternatives for Pygame for (2D) visualisation . . . . 19.1.1 TkInter Canvas (included in Python). . . . . . 19.1.2 Arcade . . . . . . . . . . . . . . . . . . . . . 19.2 Alternatives for 3D visualisation . . . . . . . . . . . . 19.2.1 Visual Python: easy 3D graphics . . . . . . . 19.2.2 Panda3D . . . . . . . . . . . . . . . . . . . . 19.2.3 OpenGL . . . . . . . . . . . . . . . . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

157 . 157 . 157 . 158 . 161 . 161 . 162 . 162

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

x

Contents 19.3 Animating your graphics: Physics . . 19.4 Creating graphics. . . . . . . . . . . 19.4.1 GIMP and its Python console 19.4.2 Blender . . . . . . . . . . . . 19.5 Interfacing with hardware . . . . . . 19.5.1 The Raspberry Pi . . . . . . . 19.5.2 MicroPython . . . . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. 162 . 163 . 163 . 164 . 165 . 165 . 165

20 Examples 167 20.1 Logicals and loops: Bubblesort . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167 A Answers to exercises

169

B Jupyter: the interactive Python notebook

193

C Programming and Scientific Computing in Python - Cheat sheet C.1 Getting started . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C.2 Variable assignment and types . . . . . . . . . . . . . . . . . . . . C.3 Python syntax: Statements . . . . . . . . . . . . . . . . . . . . . . C.4 Making your code reusable and readable . . . . . . . . . . . . . . . C.5 Extending Python: using modules . . . . . . . . . . . . . . . . . . . C.6 Defining your own functions and modules. . . . . . . . . . . . . . . C.7 File IO and string handling . . . . . . . . . . . . . . . . . . . . . . . C.8 Matplotlib: Data visualisation in Python . . . . . . . . . . . . . . . . C.9 Numerical integration. . . . . . . . . . . . . . . . . . . . . . . . . . C.10 NumPy and SciPy: Scientific programming with arrays and matrices C.11 Tuples, dictionaries, and sets . . . . . . . . . . . . . . . . . . . . . C.12 Pygame: Animation, visualisation and controls . . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

195 . 195 . 196 . 197 . 198 . 198 . 200 . 200 . 201 . 201 . 203 . 204 . 204

1 Getting started 1.1. What is programming? Ask a random individual what programming is and you will get a variety of answers. Some love it. Some hate it. Some call it mathematics, others philosophy, and making models in Python is mostly a part of physics. More interestingly, many different opinions exist on how a program should be written. Many experienced programmers tend to believe they see the right solution in a flash, while others say it always has to follow a strict phased design process, starting with thoroughly analysing the requirements (the ‘bookkeeper’ style). It definitely is a skill and personally I think a well written, easy to read and effective piece of code with a brilliant algorithm, is also a piece of art. Programming does not require a lot of knowledge, it is a way of thinking and it becomes an intuition after a lot of experience. This also means that learning to program is very different from the learning you do in most other courses. In the beginning, there is a very steep learning curve, but once you have taken the first big step, it will become much easier and a lot of fun. But how and when you take that first hurdle is very personal. Of course, you first need to achieve the right rate of success over failure, something you can achieve by testing small parts during the development. For me, there aren’t many things that give me more pleasure than to see my program (finally) work. The instant, often visual, feedback makes it a very rewarding activity. And even though at some stage you will also see the right solution method in a flash, at the same time your program will almost certainly not work the first time you run it. A lot of time is spent understanding why it will not work and fixing this. Therefore some people call the art of programming: “solving puzzles created by your own stupidity”! While solving these puzzles, you will learn about logic, you will learn to think about thinking. The first step towards a program is always to decompose a problem into smaller steps, into ever smaller building blocks to describe the so-called algorithm. An algorithm is a list of actions and decisions that a computer (or a person) has to go through chronologically to solve a problem. This is often schematically presented in the form of a flow chart. For instance, the algorithm of a thermostat that has to control the room temperature is shown in Figure 1.1.

1

start

Get desired temperature

Measure room temperature

?

True

Switch on heating

False Switch off heating

Figure 1.1: Flow chart of ‘thermostat’ algorithm

2

1. Getting started

Another way to design and represent algorithms is using simplified natural language. Let’s take as an example the algorithm to “find the maximum value of four numbers”. We can detail this algorithm as a number of steps: Algorithm: Find the maximum of four numbers Let's call the 4 numbers a, b, c and d if a > b then make x equal to a, else make x equal to b if c < d then make y equal to c, else make y equal to d if x < y then make x equal to y show result x on screen Going through these steps, the result will always be that the maximum value of the four numbers is shown on the screen. This kind of description in natural language is called “pseudo-code”. This pseudo-code is already very close to how Python looks, as this was one the goals of Python: it should read just as clear as pseudo-code. But before we can look at some real Python code, we need to know what Python is and how you can install it. After that, we will have a look at some simple programs in Python, which you can try out in your freshly installed Python environment.

1.2. What is Python? Python is a general purpose programming language. And even though until a few years ago Python was used more in the USA than in Europe, it has been developed by a Dutchman, Guido van Rossum. It all started as a hobby project, which he pursued in his spare time while still employed at the so-called Centrum Wiskunde & Informatica (CWI) in Amsterdam in 1990. Python was named after Monty Python and references to Monty Python in comments and examples are still appreciated. The goals of Python, as Guido van Rossum has formulated them in a 1999 DARPA proposal, are: • an easy and intuitive language just as powerful as major competitors • open source, so anyone can contribute to its development • code that is as understandable as plain English • suitable for everyday tasks, allowing for short development times Guido van Rossum was employed by Google for many years, as this is one of the many companies that use Python. He briefly retired in 2019, only to return back from retirement to join the Development Division at Microsoft. As moderator of the Python language he used to be called the “benevolent dictator for life” until he stepped down from the position in July 2018. A practical advantage of Python is that it supports many platforms (Windows, Apple, Linux) and that it is free, and so are all add-ons. Many add-ons have been developed by the large (academic) Python community. Some have become standards of their own, such as the scientific package NumPy/SciPy/Matplotlib. These scientific libraries (or modules as they are called in Python), in syntax(grammar) heavily inspired by the software package MATLAB, are now the standard libraries for scientific computing in Python. IEEE has named Python as the de facto standard programming language for data analysis. Up to Python 3 all versions were downwards compatible. So all past programs and libraries will still work in a newer Python 2.x versions. However, at some point, Guido van Rossum, being the BDFL (Benevolent Dictator For Life) of Python, wanted to correct some fundamental issues, which could only be done by breaking the downward compatibility. This started the development of Python 3.0. For a long time Python 2.x was also still maintained and updated. This has stopped around 2018. An example of a difference between the two versions is the syntax of the print-function, which shows a text or a number on the screen during runtime of the program:

1.3. Installing Python

3

Python 2 vs. Python 3 # Python 2.x: print ”Hello world” # Python 3.x: print(”Hello World”) (Other major differences relate to integer division and the input() function. You will learn more about these concepts during the course). Python is called a scripting language. This means that the Python source is interpreted when you run the program as opposed to needing to compile all program files first, then link them together with the correct libraries to make an executable and then run this. This is very user-friendly: there is no need to compile or link files before you can run the same program on any operating system: Windows, Apple’s macOS or Linux. While many Python libraries are platform dependent, most are available on all three supported platforms. The penalty for this is, depending on your program structure, sometimes some execution speed. Some runtime, often seconds or milliseconds, are traded for shorter development times, which saves time in the order of days or weeks. Note that Python libraries like NumPy and Scipy use very fast low-level modules, resulting in extremely fast execution times for scientific computing. By using a similar syntax, this is replacing MATLAB (which also uses a small scripting language) worldwide. Using vectorised programming and the NumPy library makes runtimes comparable with optimised C++ and Fortran for the scientific computing. The same mechanism allows fast graphics. For example for 2D graphics the Pygame graphics library, which is a layer over the very fast and famous SDL library, is also extremely fast. Using an interpreter instead of a compiler, means you need to have Python, with its interpreter, installed on the computer where you run the Python program. Would you ever want to make an executable to avoid this requirement, there are modules, like distutils, or add-ons, like called Py2exe creating selfcontained application which does not need Python installed. Such an executable will only run on the OS under which it was compiled. How this should be done, is covered by a separate chapter of the reader, called ‘Distributing your Python program’. Using the Inno Setup tool, you can make a installer for your application on Windows for example. This tool converts the complete folder with data and the program into one setup executable file, which is also covered in this chapter.

1.3. Installing Python For each platform there are several ways to install Python. In this reader we’ll give instructions for Windows, macOS, and Linux for which we’ve found that they result in the least number of problems.

1.3.1. Windows In Windows there are three main ways to install Python; starting out with a plain package from the official distributor, using a bundle like Anaconda, and installing python through the Microsoft Store. We’ve found that the most robust way to install Python is by starting out with a plain installation of Python, from the official python website. Go to Python.org and download the latest version of Python 3 64-bits. (You can also check out the steps at Youtube: https://youtu.be/-P7dVLJPows) If you’re installing Python a second time, then make sure you have uninstalled/removed any other versions of Python, using the Add/Remove Apps/Programs from the Settings/Configuration panel of Windows. Once you’ve downloaded the installer, run it, and:

4

1. Getting started 1. During the installation, when you get the dialog indicated below, make sure you tick the box ‘Add Python 3.X to PATH’! (otherwise the Python package installer pip will not work)

2. Choose ‘Customize installation’ 3. In the dialog that follows, check all options, especially ‘install for all users’ 4. Click ‘Install’ to start a full installation of Python When the installation finishes you’ll not only have Python, but also the Python package manager pip installed, which makes it relatively simple to install the other packages. Pip makes sure to download the exactly right version for your installation of Python and OS and will install it. The next step is to install the Python packages you’ll need during the course. In the Windows Start Menu, search for command prompt:

Right-click on the result, and click ‘Run as Administrator’. In the command window that has opened, type pip3 install numpy scipy matplotlib pygame

1.3. Installing Python

5

1.3.2. macOS By default, macOS already has Python installed, on recent macOS versions even Python 3.8. However, these preinstalled versions have some issues, specifically with the installation of packages needed for the course. It is therefore required to install an additional version of Python on your Mac. The currently recommended method for this is to use the official Python installer from python.org: 1. In your browser, navigate to python.org, and in the ‘Downloads’ menu, click on the download link of the current Python version for macOS. 2. Once downloaded, open the installer, which will give you the following dialog:

Follow the dialog steps. Unless you want a different installation location (not recommended), there’s no need to make any changes to the default settings. 3. Once the installation is finished, a finder window is opened, showing the results of the Python installation. 4. To install the packages required for this course, you need to open a terminal. Simultaneously press the ‘command’ and space keys, type the word terminal, and press enter:

5. In the terminal that has opened, type pip3 install numpy scipy matplotlib pygame A dialog will open, requesting you to provide your user password (required to execute a command as Administrator).

6

1. Getting started

1.3.3. Linux Python is usually installed by default on modern Linux distributions. To ensure that you have all the required packages, execute the following installation command: • For debian-based distributions (Debian, Ubuntu, Mint Linux, ...): sudo apt-get install python3 python3-numpy python3-scipy \ python3-matplotlib python-pygame • For RedHat-based distributions (RedHat, openSUSE, Fedora, CentOS, ...): sudo zypper install python3 python3-numpy python3-scipy \ python3-matplotlib python-pygame

1.3.4. Explanation of the installed modules The description above installs a number of modules: • SciPy and NumPy, which give you many tools for scientific calculations, also with large table or matrices. • Matplotlib is a very useful plotter, which contains an interactive plot window from which you can save image to be used in your publications • Pygame is a 2D graphics library, which allows you to make real-time animations using both shapes and image file, read individual keys from the keyboard, process mouse input, generate sound and play music files. It is compatible with NumPy, and many other packages. It is easy to use and uses graphics acceleration hardware if available, but will also work without. • Spyder is a working environment mainly used for interactively doing scientific computing (iPython). If you ever want to install a package which is not found by pip, it might be useful to download the .whl (wheel file) which is used by pip, yourself from this site: https://www.lfd.uci.edu/~gohlke/ pythonlibs/. Download the correct wheel file yourself, open a Command prompt, go to the downloads folder (with cd command) and type “pip3 install” in the folder with this file followed by the filename, for example: pip3 install example-py39-x64-filename.whl

1.3.5. Configuring the IDLE editor The IDLE file editor is provided with all Python installations by default, and is therefore also the default editor for the course. There are a couple of settings that make using IDLE easier and safer: • Change Working Folder to My Documents\Python In Windows, IDLE will by default start in the Python program directory (folder) and this will therefore also be your default working directory. This is dangerous because you may overwrite parts of Python when you save your own programs. Therefore make a shortcut on your Desktop in which we change the working folder to a more appropriate one. Right-click in the Start Menu on IDLE, select Copy and then Paste it on to your Desktop. Then right click Properties of this Shortcut and change the working directory to the folder where you want to keep your Python code (e.g. My Documents\Python). • Add links to your documentation of the Help menu Start IDLE. Then if you have not already done so, select in the menu of the IDLE window, Options>Configure IDLE > General to add additional Help sources in the lower part of the property

1.3. Installing Python

7

sheet. Download the documentation for Scipy (CHM files) and use the link to the pygame/docs site as well as the Matplotlib gallery. • Using IDLE: your first program In the IDLE window, named Python Shell, select File > New Window. This creates a second window in which you can edit your first program. You will see that this window has a different pull-down menu. It contains “Run”, indicating this is a Program window, still called Untitled at first. Enter the following lines in this window:

print(”Hello World”) print(”This is my first Python program”) Note: Python is case sensitive, so it matters that you use lower case for the print command. Now select Run > Run Module. You will get a dialog box telling you that you have to Save it and then asking whether it is Ok to Save this, so you click Ok. Then you enter a name for your program like hello.py and save the file. The extension .py is important for Python, the name is only important for you. Then you will see the text being printed by the program which runs in the Python Shell window. • Switching off the dialog box “Ok to Save?” By default, IDLE will ask confirmation for Saving the file every time you run it. To have this dialog box only the first time, goto Options>Configure IDLE>General and Select “No Prompt” in the line: At Start of Run (F5). Now you can run your programs by pressing the function key F5. Now only the first time you run your program, it will prompt you for a locations and filename to save it, the next time it will use the same name automatically. Note: Be careful not to use program names which are identical to modules that you import. So avoid calling your program “python.py”, “numpy.py”, “math.py”, or any other name that also refers to an existing module. Also for every new assignment or set of exercises make a new folder to keep your files well organised.

1.3.6. Working environments: IDLE, PyCharm, Spyder, and others A working environment, in which you edit and run a program is called an IDE, which stands for Integrated Development Environment. Which one you use, is very much a matter of taste. In the course we will use as an editor and working environment the IDLE program, because of its simplicity. This is provided with Python and it is easy to use for beginners and advanced programmers. Since it comes with Python, it is hence also available in both distributions. For more larger projects, there are many other IDEs, for example PyCharm. You try any of these. At the exam IDLE will be sufficient and Spyder is also available. So it is advised to learn how to use IDLE for the exam. Though IDLE is a very useful IDE (Interactive Development Environment), there are some limitations: • With large projects and many files it can become cumbersome to switch between different files • Debugging facilities are limited For this reason often other IDEs are used for larger projects. There are many on the web. The screenshot in Figure 1.2 shows PyCharm, a popular IDE, aimed specifically at Python. The community version, which we use, is freely available from Jetbrains: https://www.jetbrains.com/ pycharm/download/. Other options are, e.g., VS Code and Atom, which support multiple programming and typesetting languages either directly or through plugins. For scientific purposes a very popular one is Spyder (see Figure 1.3), as it reminds many older users of the Matlab tool for scientific computing which they used before. Other features of Spyder include inspecting data arrays, plotting them and many other advanced debugging tools.

8

1. Getting started

Figure 1.2: The PyCharm editor.

Figure 1.3: The Spyder editor.

Make sure to change the settings of file in the dialog box which will pop up the first time you run the file to allow interaction with the Shell. Then you have similar features to which IDLE allows: checking your variables in the shell after running your program or simply to testing a few lines of code. Spyder has chosen to stop offering a standard Python console, so they only have an iPython console. By tweaking the run setting per file, it is still possible to run your script in an external Python console. But this would still be a reason to prefer PyCharm as it is more versatile and focused on Standard Python applications not just the interactive iPython. My advice would be to first keep it simple and use IDLE for the basics. Use the print() function and the shell (to inspect variables) as debugger and occasionally www.pythontutor.com. Then later, and only for larger or more complex problems switch to PyCharm or one of the multi-language IDE’s (or optionally Spyder). On the exam, only IDLE and Spyder will be available, unless specified otherwise in the course announcements. A different environment especially for Scientific Computing, using iPython is Jupyter, which creates

1.3. Installing Python

9

Python Notebooks, a beautiful blend between a (LaTeX) document, spreadsheet and Python.

1.3.7. Documentation IDLE, our default editor supplied with Python, has an option to Configure IDLE and add items to the Help menu. Here a link to a file or a URL can be added as an item in the Help pull down menu. The Python language documentation is already included,select them with F1, on-line they can be found at https://docs.python.org/3/ For Scipy and NumPy, downloading the .HTML files of the reference guides onto your hard disk and linking to these files is recommended. They are available for download at: http://docs.scipy.org/doc/ For Matplotlib both an online manual as well as a pdf is available at: http://matplotlib.sourceforge.net/contents.html Also check out the Gallery for examples but most important: with the accompanying source code for plotting with Matplotlib: http://matplotlib.sourceforge.net/gallery.html For Pygame, use the online documentation, with the URL: http://www.pygame.org/docs/ Another useful help option is entering ‘python’ in your favourite search engine, followed by what you want to do. Since there is a large Python user community, you will easily find answers to your questions as well as example source codes. Other useful resources: Search on Youtube for the course code AE1205 for some videos or the Python 3 playlist: https://www.youtube.com/results?search_query=ae1205 Stackoverflow contains many answers and will often show up in your Google results: https:// stackoverflow.com/questions/tagged/python-3.x A nice tutorial and reference can also be found at: https://www.tutorialspoint.com/python3/index.htm

10

1. Getting started

1.4. Sneak preview: Try the Python language yourself In the IDLE window, named Python Shell, select File > New Window to create a window in which you can edit your program. You will see that this window has a different pull-down menu. It contains “Run”, indicating this is a Program window, still called Untitled at first. Enter (or copy) the following four lines in this window and follow the example literally. Temperature conversion tool print(”Temperature conversion tool”) tempf = float(input(”Enter temperature in degrees Fahrenheit: ”)) tempc = (tempf - 32.0) * 5.0 / 9.0 print(tempf, ”degrees Fahrenheit is about”, round(tempc), ”degrees ↪ Celsius”) Now select Run > Run Module. Depending on your settings, you might get a dialog box telling you that you have to Save it and then asking whether it is Ok to Save this, so you click Ok. Then you enter a name for your program like temperature.py and save the file. The extension .py is important for Python, the name is only important for you. Then you will see the text being printed by the program, which runs in the window named “Python Shell”: Python shell output >>> Temperature conversion tool Enter temperature in degrees Fahrenheit: 72 72.0 degrees Fahrenheit is about 22 degrees Celsius >>> Let’s have a closer look at this program. It is important to remember that the computer will step through the program line by line. So the first line says: print(”Temperature conversion tool”) The computer sees a print() function, which means it will have to output anything that is entered between the brackets, separated by commas, to the screen. In this case it is a string of characters, marked by a ” at the beginning and the end. Such a string of characters is called a text string or just string. When the computer prints this to the screen, it will also automatically add a newline character to

1.5. Variables and the assignment statement

11

jump to the next line for any next print statement (unless you specify otherwise, which will be explained later). This is all that the first line does. The next line is slightly more complicated, and will be analysed in the next section. Exercises section 1.4 Exercise 1.4.1: If you want to make it a bit more interesting, and harder for yourself, you could make a variation on this program. In much the same fashion, you could try to make a saving/debt interest calculator where you enter start amount, interest rate in percentage and number of years. To raise x to the power y, you use x**y) Exercise 1.4.2: Make a program that computes the overall resistance for the circuit shown below, where your inputs are the three resistances (𝑅𝐴 , 𝑅𝐵 , 𝑅𝐶 ).

(Solutions to all exercises can be found in Appendix A)

1.5. Variables and the assignment statement The second line in our temperature convertor is:

tempf = float(input(”Enter temperature in degrees Fahrenheit:”)) Different things are happening in this statement. The first part of the line, tempf =, is a so called Assignment statement, indicated by the equals sign (=). It should be read as: “let tempf be”. In general it has the following structure:

variablename = expression In our example it tells the computer that in the computer’s memory a variable has to be created with the name tempf. (If the variable name already exists, it will be just be overwritten by the result of the expression at the end.) To be able to do this, the computer first evaluates the expression on the other side of the “=” sign to see what the type of this variable has to be. It could for example be a floating point value (float type) or a round number (integer type), a series of characters (string) or a switch (boolean or logical). It then reserves the required number of bytes, stores the type and the name. If the name already exists, then this old value and type are cleared after the expression is evaluated. The computer evaluates the expression. The outcome is stored in memory and can be used later in other expressions by using the variable name on the left side of the equals sign. To do this, the computer saves the name in a table associated with this module. This table contains information on the type of variable and points to a computer memory address where the value is stored. For the assignment statement a = 2, this table would look something like:

12

1. Getting started name type memory address ⋯ Address 1625981072 1625981073 1625981074 1625981075

’a’ 4 byte Integer 1625981072 ⋯ Byte value 0 0 0 2

For numbers, three basic types of variables exist, each stored binary in a different way: • Integers: 2 • Floats: 2.0 • Complex: 2.0+0j Integers are whole numbers, used to count something or as an index in a table or grid. Floats are numbers with a floating point and can have any real value. Complex numbers refer to the mathematical definition of ‘complex’, which have a real part and an imaginary part. They are like floats but can have an imaginary part next to the real part. Each type is stored in a different way. Python looks at the expression to determine the type: 2 -4 3 * 4 2.0 0. 1e6 3 * 4.0 3 + 4j

=> => => => => => => =>

integer type integer type integer type float type float type float type float type complex type

In chapter 2 the assignment statement, as well as the different types of variables are discussed in detail. Now let us have a look at the expression in our example program. This is not a simple one. The expression in our example has a combination of twice the following structure: functionname(argument) Python knows this is a function because of the parentheses. In our example case, there are two functions: float() and input(). As the result of the input-function is needed as input of the floatfunction, the input function is called first. In general, when functions are called in a nested way (a function is placed inside of the parentheses of another function), the order of execution is from inside to outside. Both float() and input() are standard functions included in the Python language. (Later we will also use functions which we have defined ourselves!) Most functions do some calculations and yield a value. For example, the abs(x) function returns the absolute value (modulus) of x, and the function int(x) will try to convert the variable x to an integer number. The int() function is one of the type conversion functions: int(3.4) int(-4.315) float(2) float(0)

=> => => =>

integer with value 3 integer with value -4 float with value 2.0 float with value 0.0

But some functions are complete little programs in themselves. The input() function for example does several things: it can be called with one argument (you can pass it a string with some text), which will be printed on the screen, before the user is prompted to enter a value. When the user presses enter, the value entered by the user is read, and this is returned in a string variable as the result by the input function. So in our case, the text Enter temperature in degrees Fahrenheit: is printed, and the

1.6. Finding your way around: many ways in which you can get help

13

user enters something (hopefully a number). The text entered by the user is returned by input() as a string, which in our example gets passed on to the float() function which tries to convert this string to a floating-point value, to be stored in a memory location. We call this variable tempf. The next line is again an assignment statement as the computer sees from the equal sign “=”: tempc = (tempf - 32.0) * 5.0 / 9.0 Here a variable with the name tempc is created. The value is deduced from the result of the expression. Because the numbers in the expression on the left side of the equals sign are spelled like “5.0” and “32.0”, the computer sees we have to use floating point calculations. We could also have left out the zero as long as we use the decimal point: 5. / 9. would have been sufficient to indicate that we want to use floating point values. Extra: Regular division and Integer division When performing a division, we could also use only integers, so using 5 / 9. In Python 3, the result of a division is always a float (this didn’t use to be the case in Python 2). Note that integer division is available in Python: in this case two slashes are needed: 5 // 9 will return an integer. Integer arithmetic always rounds down, so in this case the result would be zero. (See sections 2.4 and 2.5 for a detailed discussion on the difference between integers and floats). When this expression has been evaluated, a variable of the right type (float) has been created and named tempc, the computer can continue with the next line: print(tempf,”degrees Fahrenheit is”,round(tempc),”degrees Celsius”) This line prints four things: a variable value, a text string, an expression which needs to be evaluated and another text string, which are all printed on the same line as with each comma a space character is automatically inserted as well. The round() function means the result will be rounded off. Try running the program a few times. See what happens if you enter your name instead of a value.

1.6. Finding your way around: many ways in which you can get help 1.6.1. Method 1: Using help(”text”) or interactive help() If you wonder how you could ever find all these Python-functions and function modules, here is how to do that. There are many ways to get help. For instance if you need help on the range function, in the Python shell, you can type: help(”range”) Which calls the help-function and uses the string to find appropriate help-information. You can also directly pass a function or other object to help: help(list) You can also use help interactively by typing help(), without arguments, and then type the keywords at the “help>” prompt to get help, e.g. to see which modules are currently installed. >>>help() help>math And you will see an overview of all methods in the math module. There are some limitations to this help. When you will type append, you will not get any results because this is not a separate function but a part of the list object, so you should have typed

14

1. Getting started

>>> help(list.append) Help on method_descriptor in list: list.append = append(...) L.append(object) -- append object to end

1.6.2. Method 2: Python documentation in Help Pull-down menu Another way to get help in Python is to use the supplied CHM-file, (compiled HMTL) via the Help-menu of the IDLE-windows: Select Help>Python Docs and you will find a good set of documentation, which you search in the “Index” or do a full text search (“Search”), see the following screenshots:

1.6.3. Method 3: Get help from the huge Python community Python has a very large user community: conservative estimates say there are 3 to 5 million Python users and it’s growing fast as MIT, Stanford and many others use Python in their programming courses and exercises. It is also the most popular language among PhDs. IEEE calls it the standard language for scientific computing and data analysis. So simply Googling a question (or error message) with the keyword Python (or later NumPy) in front of it as an extra search term will quickly bring you to a page with an example or an answer. For basic questions this might bring you to the same documentation as in the Help-menu which is at https:// docs.python.org. You will also see that https://www.stackoverflow.com is a site that often pops up, and where most questions you might have, have already been posed and answered:

1.6. Finding your way around: many ways in which you can get help

15

For instance, for the search query “python append to a list” you will find: https://stackoverflow. com/questions/252703/python-append-vs-extend (Which shows that next to append() there apparently is another method called extend() available, which works with a list as an argument and apparently appends each element to the list.) In general, thanks to the interactive Python Shell window, simply trying statements or program lines in the Python shell is a way to try what works as you intended. Or to check the value of variables after your still incomplete program has run. To find bugs in your program, for small programs https://www.pythontutor.com can be helpful to see what is going on inside the computer when your program runs. And of course using the print statement to see what a variable is, or where the computer gets stuck in your program, is always the first thing to try. Be prepared that in the beginning the level of frustration might be high, but so is the reward when your program runs. And when you get experience, you will see how fast you can make working programs, not because you won’t create bugs, but because you will immediately recognise what went wrong. The nice thing of software is that it can always be fixed, so “trial and error” is an accepted way to develop programs. So do not be afraid to try things and play around with Python to get acquainted with the way it works. Appendix C gives an overview of all of the Python statements and most important functions that are covered in this course, with a short example for each item. Having a copy of these pages at hand may be handy when starting Python. It will provide the keywords for you to look into the help-functions of Google searches. Although we already touched upon most basic elements of the Python language, we have not seen all statements and types yet, so first go through the next chapters (discover the very useful while-loop, the string type and its methods and the Boolean/logical and slicing of strings and lists etc.). Just glance over these chapters to get the essence and then start making simple programs and study the paragraphs in more detail when you (think you) need them. Programming has a steep learning curve at the beginning, so good luck with making it through this first phase. Once you have passed this, things will be a lot easier and more rewarding. Good luck!

2 Variable assignment and types In this chapter we describe the building blocks which you need to write a program. First we have a look at variables, which represent the way in which a computer stores data. But there are different types of data: numbers or bits of text for instance. There are also different kinds of numbers, and of course there are more data types than just text and numbers, like switches (called “booleans” or “logicals”) and tables (“lists”). The type of variable is defined by the assignment statement; the programming line giving the variable its name, type and value. Therefore we first concentrate the assignment statement and then the different data types are discussed. In this reader we’ll often use examples to introduce and explain programming concepts. The example we’ll start out with in this chapter is a program to solve a second order polynomial equation. We will use this example to discuss the use of variables in systematic fashion. ABC solver import math print(”To solve ax2 + bx + c = 0

:”)

a = float(input(”Enter the value of a: ”)) b = float(input(”Enter the value of b: ”)) c = float(input(”Enter the value of c: ”)) D = b ** 2 - 4.0 * a * c x1 = (-b - math.sqrt(D)) / (2.0 * a) x2 = (-b + math.sqrt(D)) / (2.0 * a) print(”x1 =”, x1) print(”x2 =”, x2) Before we start some notes about this program. Run this program and you will see the effect. • note how ** is used to indicate the power function. So 5 ** 2 will yield 25, and 16 ** 0.5 will yield 4. • the program uses a function called sqrt(); the square root function. This function is not a standard Python function. It is part of the math module, supplied with Python. When you need functionality of such external modules in your code, you can use the import statement at the beginning of your code to make these modules accessible. The expression math.sqrt() tells Python that the sqrt() function should be obtained from the imported math module. You’ll learn more about using modules in chapter 5. 17

18

2. Variable assignment and types • After you have run the program, you can type D in the shell to see the value of the discriminant. All variables can be checked this way. • Note the difference between text input and output. There is no resulting value returned by the print function, while the input function does return a value, which is then stored in a variable. The argument of the input-function calls is between the brackets: it’s a prompt text, which will be shown to the user before he enters his input.

2.1. Assignment and implicit type declaration A variable is a reference by name to a memory location with a value and a type. You define a variable when you assign a value or expression to it. This expression also determines the type. The assignment statement is very straightforward: variablename = expression Example assignment statements for the four main types (which will be discussed in detail in the following sections): Integers, only whole numbers, using digits and the sign: Integers i = 200 a = -5 Floats, contain a decimal point or “e” to indicate the exponent of the power of 10: Floating point values inch = 2.54 ft = .3048 lbs = 0.453592 spd = 4. alt = 1e5 Logicals: Logicals swfound = True alive = False forward = (v > 0.0) onground = h i += So the examples above can also be coded like this: i j n x a

+= -= *= /= *=

1 1 2 2 k

# # # # #

increase i with one decrease j with one doubles n halves x a will be multiplied by k

This short-hand is the same as in the programming language C (and C++), a language which is well known for how short code can be written (often with an adverse effect on the readability). Also in this example, the benefit is mainly for the programmer who saves a few keystrokes, (in Python) there is no execution time benefit as the operations done are the same. This short-hand hence increases the distance from pseudo-code. Whether you use it is a matter of personal preference.

2.3. Number operations in Python For the number types float, integer (and complex) the following operators can be used + * / // % ** -x +x

Addition Subtraction Multiplication Division Floored or integer division (rounded off to lower round number) Modulo, remainder of division Power operator x**y is x to power y, so x**2 is x2 Negated value of x Value of x with sign unchanged

As well as the following standard functions: abs(x) pow(x, y) divmod(x, y)

Absolute value of x; |x| Power function, same as x ** y Returns both x // y and x % y

20

2. Variable assignment and types

2.4. Float: variable type for floating point values Floating point variables, commonly referred to as floats, are used for real numbers with decimals. These are the numbers as you know them from your calculator. Operations are similar to what you use on a calculator. For power you should use the asterisk twice, so 𝑥 𝑦 would be x ** y. They are stored in a binary exponential way with base 2 (binary). This means that values which are round values in a decimal systems sometimes need to be approximated. For instance if you try typing in the shell: >>> 0.1 + 0.2 The result will be: >>> 0.1 + 0.2 0.30000000000000004 Often this is not a problem, but it should be kept in mind, that float are always approximations! One special rule with floats in programming is that you never test for equality with floats. So never use the condition “when x is equal to y” with floats, because a minute difference in how the float is stored can result an inaccuracy, making this condition False when you expected otherwise. This may not even be visible when you print it, but can still cause two variables to be different according to the computer while you expected them to be the same. For example: adding 0.1 several times and then checking whether it is equal to 10.0 might fail because the actual result is approximated by the computer as 9.99999999 when it passes the 10. So always test for smaller than (). You can also use “smaller or equal to” (> round(10 3 >>> round(10 3.3333 >>> round(10 ↪ return a 3.0

/ 3) / 3, 4) / 3, 0) # Explicitly specify zero decimals lets round float

Extra: Try for yourself Try adding 0.1 to a variable for more than 1000 times (use a for-loop or while-loop) and check the variable value to see this effect.

2.5. Int: variable type for round numbers like counter or index) Integers are variables used for whole numbers. They are used for example as counters, loop variables and indices in lists and strings. They work similar to floats but division results will be of the type float.

2.5. Int: variable type for round numbers like counter or index)

21

For example: a = 4 b = 5 c = a / b d = a // b print(c) print(d) In this example, the first print statement will print 0.8, but the second statement prints 0. The difference, as mentioned before, is the difference between regular division (/) and integer division (//). The former returns a float, whereas the latter returns an integer. Note also that with integer division, the result is always rounded down. A useful operator, commonly used with integers, is the ‘%’ operator. This operator is called the modulo function, and calculates the remainder of a division. For example: 27 % 4 = 3 4 % 5 = 4 32 % 10 = 2 128 % 100 = 28 So to check whether a number is divisible by another, checking for a remainder of zero is sufficient: if a % b == 0: print(”a is a multiple of b”) You can convert integers to floats and vice versa (since they are both numbers) with the functions int() and float(): a = float(i) j = int(b) k = int(round(x))

# Conversion cuts of behind decimal point # Rounded off to nearest whole number # ..and then converted to an integer type

The int() function will cut off anything behind the decimal point, so int(b) will give the largest integer not greater than b (in other words, it rounds down). These functions int() and float() can also take strings (variables containing text) as an argument, so you can also use them to convert text to a number like this: txt = ”100” i = int(txt) a = float(txt) Note the different functions related to division in the example below: n = 20 i = 3 a = n / i print(”a =”,a) j = n // i k = int(n / i) m = round(n / i) print(”j,k,m =”,

# floor division # rounded down and converted to int # rounded to nearest integer value j, k, m)

The output of this program will be:

22

2. Variable assignment and types

a = 6.666666666666667 j,k,m = 6 6 7 As we can see from the missing decimal points in the second output line, the variable j, k, m all are integers. This can also be verified in the shell using the type() function, which prints the type of a variable: >>> type(m)

2.6. Complex numbers Python has complex numbers built-in as a standard type. In many other languages one would have to treat this as two float numbers, the real part and the imaginary part, but in Python the j directly after a number signals an imaginary art and will change the resulting type to complex. c = 3 + 4j print (”Length of”, c, ”is”, abs(c)) Will give the following output: Length of (3+4j) is 5.0 To access the different components, use the following method: c = 3 + 0j a = c.real b = c.imag print(”c has”, a, ”as real part and”, b, ”as imaginary part”) Will give the following output: c has 3.0 as real part and 0.0 as imaginary part All other operations work as expected: >>> e = 2.718281828459045 >>> pi = 3.141592653589793 >>> e ** (1j * pi) (-1+1.2246467991473532e-16j) Which demonstrates Euler’s equation, and also shows the rounding off problems in the imaginary part, which is practically zero. Both e and pi were binary approximations, as are all floats and complex numbers.

2.7. String operations Strings are variables used to store text. They contain a string of characters (and often work similar to a list of characters). They are defined by a text surrounded by quotes. These quotes can be single quotes or double quotes, as long as you use the same quote-character to start and end the string. It also means you can use the other type of quote-symbol inside the text. A useful operator on strings is the “+” which glues them together. This can be very useful, e.g., in functions which expect one string variable (like the input function). Example string assignments: txt = ”abc” s = ””

2.8. Booleans and logical expressions

23

name = ”Graham” + ”Chapman” # (so the + concatenates strings) words = 'def ghi' a = input(”Give value at row” + str(i)) Some useful basic functions for strings are: len(txt) str(a) eval(str) chr(i) ord(ch)

returns the length of a string converts an integer or float to a string evaluates the expression in the string and returns the number converts ASCII code i to corresponding character (see table 2.1) returns ASCII code of the character variable named ch

Using indices in square brackets [ ] allows you to take out parts of a string. This is called slicing. You cannot change a part of string but you can concatenate substrings to form a new string yielding the same effect: c = ”hello world” c[0] c[1:4] c[:4] c[4:1:-1] c[::2] c[-1] c[-2] c = c[:3] + c[-1]

is then ”h” (indices start with zero) is then ”ell” (so when 4 is end, it means until and not including 4) is then ”hell” is then ”oll” so from index 4 until 1 with step -1 is then ”hlowrd” will return last character ”d” will give one but last character ”l” will change c into ”hel” + ”d” = ”held”

The string variable also has many functions built-in, the so-called string methods. See some examples below (more on this in chapter 8). a = c.upper() a = c.lower() a = c.strip() sw = c.isalpha() sw = c.isdigit() i = c.index(”b”)

returns copy of the string but then in upper case returns copy of the string but then in lower case returns copy of the string with leading and trailing spaces removed returns True if all characters are alphabetic returns True if all characters are digits returns index for substring in this case “b”

For more methods see chapter 8, or section 5.6.1 of the Python supplied reference documentation in the Help file. Strings are stored as lists of characters. Internally, Characters are stored by the computer as numbers, using the Unicode encoding. This encoding is an extension of the ASCII codes. See table 2.1: ASCII codes can be derived from the corresponding character using the ord(char) function. Similarly, the character for a given ASCII code i can be retrieved using the chr(i) function. As you may have noticed in table 2.1, the first 31 and the last entry are ‘special’ characters. Most of these special characters you will likely never need. Exceptions to this are the newline character, and the tab character. These special can be easily entered into a string using an ‘escape character’: a backslash followed by a character. In this way, you can add a newline to a string by adding '\n', and a tab by adding '\t'. When you start reading text files from disk (see chapter 8), you might notice that newlines can also be indicated with other characters: In Windows/DOS text files the newline is indicated by both a Carriage Return and a Line Feed: chr(13) + chr(10). On Linux and Apple systems only the newline character chr(10) is used.

2.8. Booleans and logical expressions Booleans (or logicals) are two names for the variable type in which we store conditions. You can see them as switches inside your program. Conditions can be either True or False, so these are the only two possible values of a logical. Mind the capitals at the beginning of True and False when you use these words: Python is case sensitive. Examples of assignments are given below:

24 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

2. Variable assignment and types NUL (Null) SOH (Start of Header) STX (Start of Text) ETX (End of Text) EOT (End of Transmission) ENQ (Enquiry) ACK (Acknowledgement) BEL (Bell) BS (Backspace) HT (Horizontal Tab) LF (Line Feed) VT (Vertical Tab) FF (Form Feed) CR (Carriage Return) SO (Shift Out) SI (Shift In) DLE (Data Link Escape) DC1 (Device Control 1 (XON)) DC2 (Device Control 2) DC3 (Device Control 3 (XOFF)) DC4 (Device Control 4) NAK (Negative Acknowledgement) SYN (Synchronous Idle) ETB (End of Transmission Block) CAN (Cancel) EM (End of Medium) SUB (Substitute) ESC (Escape) FS (File Separator) GS (Group Separator) RS (Record Separator) US (Unit Separator)

32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63

SP ! ” # $ % & ' ( ) * + , . / 0 1 2 3 4 5 6 7 8 9 : ; < = > ?

64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95

@ A B C D E F G H I J K L M N O P Q R S T U V W X Y Z [ \ ] ^ _

96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127

‘ a b c d e f g h i j k l m n o p q r s t u v w x y z { | } ~ DEL

Table 2.1: The ASCII table.

found = False prime = True swlow = a < b outside = a >= b swfound = a == b notfound = a != b # ( != means: not equal to) outside = x < xmin or x > xmax or y < ymin or y > ymax inside = not outside out = outside and (abs(vx) > vmax or abs(vy) > vmax) inbetween = 6.0 < x xmin and x < xmax) or (y > ymin and y < ymax)) You can use logicals as conditions in if and while statements: if inside: print(”(x,y) is inside rectangle”) while not found: i = i + 1 found = (a == lst[i]) Note that if inside basically means: if inside == True and similarly while not found means while found == False. This is why Booleans are never compared with True or False as this serves no function (for more information on if-statements, see section 3.4, for while-statements, see section 3.6).

2.9. The list type: an ordered collection of items 2.9.1. What are lists? Lists are a way to group variables together. This allows you to repeat something for all elements of a list by using an index as a counter or by iterating through the list with the for-statement (see section 3.5). This could be a repetitive operation, but also a search or a sorting action. Often it is useful to have a series of variables that are grouped together. You can therefore look at a list as if it is a table. Each element of a list can be of any type: it can be an integer, float, logical, string or even a list! Special for Python is that you can even use different types in one list; in most programming languages this would not be possible. Both to define a list, as well as to specify an index, square brackets are used as in strings: Creating lists a b c d

= [2.3 , 4.5 , -1.0, .005, 2200.] = [20, 45, 100, 1, -4, 5] = [”Adama”, ”Roslin”, ”Starbuck”, ”Apollo”] =[[”Frederik”,152000.0], [”Alexander”,193000.0], [”Victor”,110000.0]] e = [] This would make the variable a a list of floats, b a list of integers, and c a list of strings. A list of lists as defined in d is basically a way to create a two-dimensional list. Finally e is an empty list. Accessing elements from the list is done as indicated below. Another way to make a list of numbers is using the so-called range function. This is a so-called iterator function used primarily in loops, but it can also be converted to a list. The range function can contain one two or even three arguments: range(stop) range(start, stop)) range(start, stop, step))

26

2. Variable assignment and types

In all these cases start is the start value (included), stop is the value for which the block is not executed because for will stop when this value is reached. So it is an inclusive start but excludes stop. Another option is to enter a third argument, to indicate a stepsize. The default start is 0, the default step is 1. Note that range() only works with integers as arguments: list(range(5)) list(range(5,11)) list(range(5,11,2))

gives gives gives

[0, 1, 2, 3, 4] [5, 6, 7, 8, 9, 10] [5, 7, 9]

2.9.2. Indexing and slicing, list functions and delete Let’s use the following list: lst = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] If we now print elements we can use the indices in many ways. Using one index, indicating a single element: lst[0] lst[1] lst[9] lst[-1] lst[-2]

which holds value 1 which holds value 2 which holds value 10 last element, which holds the value 10 one but last element which holds the value 9

Here it can be seen that indices start at 0, just as with strings. And similar to strings, the function len() will give the length of a list and thus the not to reach maximum index for the list. Slicing lists with indices works in the same way as it does for strings, with three possible arguments: start, stop and step. Only one index refers to one single element. Two arguments separated by a colon refer to start and stop. A third can be used as step. If you look at the lists that were made in the previous paragraph about lists you can see that: a[1] b[1:4] d[1][0]

will return a float 4.5 will return a list [45, 100, 1] will return ”Alexander” (do you see the logic of the order of the indices for the different dimensions: second element of main list, then first element of that list?)

Adding and removing elements to a list can be done in two ways: the “+” operator, or the append and extend functions: a = a + [3300.0] a = a[:-2]

a = a[:i]+a[i+1:] a = b + c

will add 3300.0 at the end of the list will remove the last two elements of the list (by first copying the complete list without the last two elements, so not very efficient for large lists) will remove element i from the list, when it’s not the last one in the list will concatenate (glue together) two lists if b and c are lists

Another way is to use the del (delete command) and/or functions which are a part of the list class. You call these functions using the dot notation: variablename.functionname() so a period between variablename and functionname. Some examples: a.append(x) a.extend(b) a.remove(3300.0) del a[-1] del a[3:5] a = a[:3] + a[4:] Slicing

add x as a list element at the end of the list named a, equivalent to a = a + [x] extend the list with another list b, equivalent to a = a + b removes the first element with value 3300.0 removes the last element of a list named a removes the 4th till the 6th element of a list named a will remove the 4th element of the list named a

2.9. The list type: an ordered collection of items

27

The last example line uses the slicing notation (which can also be used on strings!). Slicing, or cutting out parts of a list is done with the colon notation. Slicing notation is similar to the range arguments: start:stop and optionally a step can be added as third argument: start:stop:step. If no value is entered as start, the default value is zero. If no value is added as end the default value is the last element. The default step size is 1. Negative values can be used to point to the end (-1 = last element, -2 is the one-but-last, etc.). Using the original definition of the lst variable: lst = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10], this will give: lst[:3] lst[:-2] lst[4:] lst[::2]

first three element index 0,1,2: [1,2,3] all except last two elements all elements except the first four: except elements nr 0,1,2,3 every second element so with index 0,2,4, until the end

Other useful functions for lists: b.index(45) len(d) min(a) max(a) sum(a)

will return the index of the first element with value 45 will return the length of the list will return the smallest element of the list variable a will return the largest element of the list variable a will return the sum of the elements in the list variable a

2.9.3. Quick creation of lists with repeating values If you need to create a one-dimensional list with multiple times the same value, Python offers the following short-hand: a = 5 * [0] print(a) # Would print [0, 0, 0, 0, 0] a[1] = 1 print(a) # Would print [0, 1, 0, 0, 0] This notation, however, does not work for nested (two-dimensional) lists. When you need this, you can use a loop to construct your list: a=[] for i in range(3): a.append(3*[0]) a[0][1] = 1 print(a) # Would print [[0, 1, 0], [0, 0, 0], [0, 0, 0]] This is a safe way of constructing nested lists, as each new element of a is constructed from scratch. The following would also work: # if your list is small, just do it entirely by hand a = [[0, 0], [0, 0], [0, 0]] # You can also initialise your list with dummy values, which you replace ↪ afterwards: a = [0, 0, 0] for i in range (3): a[i] = [0, 0] If you are not sure, test a simple example by constructing your nested list, changing one of its elements, and see the effect by printing the entire list. You can also use www.pythontutor.com to visualise what is going on in the computer’s memory. In the above example we use a for-loop to execute a line of code a fixed (in this case 3) number of times. You’ll learn more about for-loops in section 3.5.

28

2. Variable assignment and types

Extra: The tricky nature of mutable types Although the short-hand notation described above doesn’t give you what you’d expect, it doesn’t return an error either. What happens is the following: a = 3 * [3 * [0]] print(a) # Would print [[0, 0, 0], [0, 0, 0], [0, 0, 0]] a[0][1] = 1 print(a) # Would print [[0, 1, 0], [0, 1, 0], [0, 1, 0]] The result of the second print statement is probably not what you expected! In line 3 we only change the second element of the first row, but surprisingly all rows (all lists) have their second element changed into 1! What happens is related to the fact that some types are mutable (lit. ‘can be changed’), and others immutable (lit. ‘cannot be changed’). In Python, all basic types, like floats, integers, and booleans are immutable. So in the following, variable a is copied into b: a = 1 # Create an integer variable in memory, with value 1 b = [a, a, a] # Create a list, with three copies of a as content However, aggregate types (i.e., combinations of multiple data, such as a list) are mutable, which in Python means that they are passed by reference. This results in the following: a = [0, 0] # Create a list b = [a, a, a] # Create a second list, with three references to a as ↪ content Here, b literally contains three times the original variable a. Changing the contents of a therefore results in the same behaviour as the short-hand construction of nested lists: a[0] = 1 print(b)

# Would print [[1, 0], [1, 0], [1, 0]]

Have a look at this Python-tutor example for a visualisation of such construction and referencing!

2.9.4. List methods Another powerful attribute of Python’s list is the number of methods that come with it. Methods are functions which you call with the dot-notation, as you already saw in section 2.9.2, and in the previous section with the append() method. An example of this is the sort method: a = [2, 10, 1, 0, 8, 7] a.sort() print(a) # Would print [0, 1, 2, 7, 8, 10] As we can see the method sort() (you can recognise functions and methods by the presence of parentheses) sorts the list a. This is a special function in that it changes the actual source list instead of creating a sorted copy (this is called an in-place operation). Most methods will return the result of the function as a new variable, and leave the original intact. Examples are the methods index and count. Index will give the index of the first occurrence of the given value. Count will give the total number of occurrences: >>> a = [3,1,8,0,1,7] >>> a.count(1) 2 >>> a.index(1) 1

2.10. Some useful built-in functions

29

>>> a.index(9) Traceback (most recent call last): File ””, line 1 a.index(9) ValueError: 9 is not in list >>> How would you make a program which will print the index of the first occurrence but also avoids the error message if it is not found by simply printing -1? Exercises section 2.9 Exercise 2.9.1: Make a program which calculates effect of a yearly interest rate (entered by the user) on start sum (also entered by the user). Print the amount after each year (ask the user how many years). Exercise 2.9.2: Make a list of words input by the user (e.g. aeroplane company names) by using the append() function, then print them below each other with the length of each word. Exercise 2.9.3: You can also make a list which consists of different lists, each with a different length, as shown here: values = [[10, 20], [30, 40, 50, 60, 70]] Print the length of each list and all the values it contains. Exercise 2.9.4: We can also go in the other direction, and construct a list of lists from a list of values and a list of lengths: values = [10, 20, 30, 40, 50, 60, 70] lengths = [2, 2, 3] Construct a list of lists, with each list i containing lengths[i] values from values. Print the constructed list.

2.10. Some useful built-in functions As we have already seen in the examples, Python has a number of built-in functions which you can use in the right side of assignment statements on any expression using values and variables: float(i) int(x) round(x) round(x,n) str(x) eval(txt) bool(x) repr(x) list(txt) chr(i) ord(c) len(txt) len(lst) abs(x) range(stop)

converts an integer or string(text) to a float converts a float to an integer (whole number rounded to lowest integer) rounds off to nearest round number and converts to integer rounds off a float n decimals, result is still float converts a number into a string (text) converts a text to a number convert x to a Boolean (typically non zero is True) generates a string with a printable content of x converts x to a list type (e.g. splits string in single characters) gives character for ascii code i (look up ascii table to see codes) gives acsii code of character c (a string with length one) gives length of the string txt gives length of the list lst absolute value of x (modulo/length for complex numbers) give a list [0,1,...stop-1] so stop is not included

30

2. Variable assignment and types range(start,stop) range(start,stop,step) sum(a,b,c,d,...) sum(lst) max(a,b,c,d,...) max(lst) min(a,b,c,d,...) min(lst) input(txt) help(txt)

give a list [start, start+1,...,stop-1] so start is included! give a list [start, start+step,...] stop is not included total sum of variables a,b,c,d,... total sum of the list elements in lst maximum value of a,b,c,d,... maximum value in the list lst minimum value of a,b,c,d,... minimum value in the list lst prints txt, asks user for input, returns user input as a text string for use in shell: prints help on Python or installed function, module or statement, e.g. help(’math’) or help(’input’)

3 Python syntax: Statements 3.1. Assignment In this chapter we discuss the basic Python statements: commands that you use in a program which often do something with the data. A summary of all of the basic python statements covered in this course can be found in Appendix C. We already discussed the assignment statement at the beginning of the previous chapter. The assignment statement is one of the simplest statements in Python:

Ë

Ê

variablename = expression This general statement is executed in the following order: 1. Evaluate expression, and store the result in memory 2. Point variablename to the area of memory created in step 1. The type and value of the variable variablename is determined by the expression. Note that it is possible to overwrite a variable, even when it is used in the expression on the right-hand side of the equals sign. This is because the computer first evaluates the expression and only then stores the result in the variable, overwriting anything that was previously stored in that variable. Some examples of how you can use the assignment are given below: a = 4.5 i = 100 total = 0 i = i + 1 i += 1 total = total + i total += i txt = ”ea”

will: (1) create a float with value 4.5, and (2) point variable name a to it will: (1) create an integer with value 100, and (2) point variable name i to it will: (1) create an integer with value 0, and (2) point variable name total to it will: (1) calculate the result of i + 1, store it in a new integer, and (2) point variable name i to it equal to previous line will: (1) calculate the result of total + i, store it in a new integer, and (2) point variable name i to it equal to previous line will: (1) create a string containing ”ea”, and (2) point variable name txt to it

31

32

3. Python syntax: Statements txt = txt + ”sy”

series = [] series = series + [i]

will: (1) evaluate the expression txt + ”sy”, store the result (”easy”) in a new string, and (2) point variable name txt to it will: (1) create an empty list, and (2) point variable name series to it will: (1) create a new list, containing the contents of series, appended with an additional element i, and (2) point variable name series to it

Here it should be noted that a statement such as i = i + 1 may seem mathematically incorrect, but in programming this simply means that the expression i + 1 is evaluated with the old value of i, after which the variable i is updated to point to the result of the expression. This order of actions basically means that ‘=’ means ‘will become’ instead of ‘is equal to’ (remember that for the ‘is equal to’ comparison ‘==’ in Python). Also note the short-hand notation “+=”, which has been discussed extensively in section 2.1 already. Adapting our example program from the previous chapter, we add some logic. Edit the program to match this example precisely, note the margin jumps (use TAB-key) in the if statement, which is called indentation. Having the same margin, or indentation level, is what Python uses to mark blocks of code that belong together, for example because they are part of an if-branch or a loop (see later sections on for and while loops). ABC solver with discriminant check from math import sqrt print(”To solve ax2 + bx + c = 0

:”)

a = float(input(”Enter the value of a: ”)) b = float(input(”Enter the value of b: ”)) c = float(input(”Enter the value of c: ”)) D = b ** 2 - 4.0 * a * c if D < 0.0: print(”This equation has no solutions”) else: x1 = (-b - sqrt(D)) / (2.0 * a) x2 = (-b + sqrt(D)) / (2.0 * a) print(”x1 =”, x1) print(”x2 =”, x2) Now the program first checks whether D is negative. If so, it will tell you that there are no solutions. • Note the structure and syntax(=grammar) of the if-else statement. A colon ( : ) indicates that a block of code will be given, so it acts as a ‘then’. The block is marked by the increased level of indentation of the code. • When it jumps back to the margin of the beginning to indicate the end of this block of code, an ‘else’ follows, again with a colon and an indented block of code. • It also uses a different way to import the sqrt function from the math module. Note the difference in syntax in both the line with the import statement as well as the line where the sqrt function is used. Chapter 5 will explain the different ways to import modules.

3.2. The print statement

33

Exercises section 3.1 Exercise 3.1.1: Adapt the program so that it calculates the hypotenuse c of a rectangular triangle for rectangular sides a and b as entered by the user, using Pythagoras formula. Exercise 3.1.2: Using the flow chart depicted below, make a program that determines the maximum value of given numbers a, b, c and d as entered by the user, using a number of ifelse-statements to find the maximum value of these four variables. start

Enter values

a, b, c, d

a > b?

max1 = a

max1 = b

c > d?

max2 = c

max2 = d

max1 > max2?

print max1

print max2

end

Exercise 3.1.3: Write a program that clamps a float input by the user to [-10, 10] and prints it. Additionally, if the input is negative, the program should print a warning message.

3.2. The print statement The print function generates text output in the console. It can print text, variables and (the result of) expressions. To print several things on one line, pass them to the print function, separated by commas. In the resulting printed line of text, a space will automatically be inserted between each argument. The space is the default so-called ‘separator’ symbol. After each call to print automatically a newline character is printed, so the next print call will start at a new line. A print-statement without any arguments, prints an empty line, or a newline character to be exact.

3.2.1. Using print Below is an example of four different ways of using the print command, which will result in exactly the same output (printing the text ‘Hello world’ to the console, followed by a newline): # Method 1 print(”Hello world”) # Method 2 print(”Hello”, ”world”, sep=” ”) # Method 3

34

3. Python syntax: Statements

print(”Hello”, ”world”, end=”\n”) # Method 4 (using two print statements) print(”Hello”,end=” ”) print(”world”) The above examples also use the use of the ‘sep’ and ‘end’ keyword, which can be used to use a different separator character or a different end of line. Note how the hash symbol (#), indicates the start of a comment: this symbol and everything after it is ignored by the Python interpreter when the program is executed. The print statement can print one or more texts, variables expressions and the outcome of functions. Basically anything can be printed. Apart from generating output for the user, print is also very useful when trying to find bugs in your program. Experienced programmers often put temporary print statements after every so many lines of code to check if everything works as intended. These are then later deleted or commented out with the hash symbol. Other examples of valid print statements with other arguments than strings: print(”Program to solve 2nd order equations. ”) print(”The solutions are”, x1, ”and”, x2) print(”Discriminant is”, b * b - 4.0 * a * c) print(”List of names”, lst) print() print(”Ready. ”) Extra: Using print to output to a file Next to ‘sep=’ and ‘end=’, there is also the keyword ‘file=’, which can be used to print to a file instead of to the screen (see chapter 8 for more information on how to read and write text files). This file needs to be opened first with the open function. This function returns a file-object. The next example shows how this can be used to print to: # Open file for writing (see ch 8) fout = open('test.dat','w') print(”abc”, file=fout) print(”def”, file=fout) f.close() # Close file

3.2.2. Formatting your output Sometimes more control is needed over how data is printed, or stored in a string of text. For instance how many decimals are printed or to line up a series of numbers using the decimal point in a more table-like fashion. Or when you want to right-align a number of integers, or add leading zeros. For that you can use the format function of shorter, the f-string. Let’s first look at the f-string as this has become the easiest and most commonly-used method ever,since it was introduced for the first time in Python 3.6: Formatting using f-strings from math import sqrt

i = j = x = txt

5 10 sqrt(7) = ”Hello”

3.3. The input function

35

#--------- Default method since Python 3.6: f-strings -------------print(f”An example of a float is: {x}”) print(f”An example of a formatted float is: {x:5.2f}”) # 5 places, 2 ↪ decimals print(f”An example of formatted floats is: {x:.2f}”) # 2 decimals print(f”An example of a formatted integer is: {i:3d}”) # 3 places print(f”A formatted integer with leading zeros is: {i:03d}”) # leading ↪ zeros print(f”And a string stays the same: {txt}”) When executed, this code generates the following output: An example of float is: 2.6457513110645907 An example of formatted float is: 2.65 An example of formatted floats is: 2.65 An example of a formatted integer is: 5 An example of a formatted integer with leading zeros is: 005 And a string stays the same: Hello The above syntax is actually a short-hand version for using the format function which you can also use. See the example below, using the same data: #--------- Method 1: Format-function (value, formatstring) # ”5.2f” means: length = 5 characters, 2 decimals, floating point # (so for x=1.2 there will be one leading space, as it is right aligned) y = sqrt(3) print(”Result is”, format(x, ”5.2f”)) print(”Position is (” + format(x, ”5.2f”) + ”,” + format(y, ”7.4f”) + ↪ ”).”) print(”Number of iterations:”, format(j, ”5d”))# Length 5 so 2 leading ↪ spaces #--------- Method 2: Format string method: print(”i is{:>2d}”.format(i)) # Adds space print(f”i is{i:>2d}”) # Adds space on left print(”j is {:04d}”.format(j)) # Add zeros

” ...{:f7.3}....”.format(y) on left side: i is 5 side: i is 5 on left side: j is 0010

# Floating point: number of digits (optionally: dot number of decimals) print(”x is {:4.2f}”.format(x)) # x is 2.65 print(”x,y is {:3.2f},{:3.2f}”.format(x, y)) # x,y is 2.65 , 1.7 # Strings print(”Look at this: {}”.format(txt)) print(”Look {} times at this:{}”.format(i, txt)) # Format can also use names to refer to variables: print(”x,y is {xcoor:3.2f},{ycoor:3.2f}”.format(xcoor=x, ycoor=y))

3.3. The input function With the function input() the user can be asked to input values during runtime of the program. Input is not really a statement, but a function in an assignment line. It returns the text as entered by the user during runtime. The input function takes one argument; a string, which is used as the prompt by the function: it will print this text just before the user can give the input. Some examples:

36

3. Python syntax: Statements

Using the input function to get data from the user name ans

= input(” What is your name: ”) = input(” Do you want to continue? (Y/N) ”)

The input function will always return a string. When another type of variable is needed as input, the converter functions can be used to get the right type: x = float(input(”Give value for x: ”)) i = int(input(”Give number of values: ”)) In some cases this might not work, for instance when we want to enter an expression (maybe even using variable names). It will also not work when a list of values is entered by the user. In both cases successful conversion can be achieved by using an alternative method, the evaluation function eval() can be used for the conversion of the string. A call to eval() will evaluate any expression as an actual Python expression, and determine the type in the same as the assignment function would do it: The eval function xlst = list(eval(input(”Give values for x: ”))) a, b, c = eval(input(”Enter a,b,c: ”)) The eval function could also be used in the examples above, but there is a risk of the type ending up different from what you expected. Users will not always add a decimal point when entering a float for instance. Note the difference between print and input, in terms of the arguments they accept. While print takes any number of arguments, which it combines together in a single printed output, separated by spaces, input only takes one argument. There are different ways to get around this. The first is to use print in combination with input: print(”Enter the value of coordinate”, i, ”:”, end=” ”) x[i] = input() Another approach would be to generate a single string of text before passing it to input: # Join strings together with + x[i] = input(”Enter the value of coordinate” + str(i) + ”: ”) # Or create a combined string using f-strings x[i] = input(f”Enter the value of coordinate {i} : ”) The second option in the box above uses f-strings, see the previous section on formatting your print output. This method is even more powerful than joining strings with the ‘+’ operator.

3.4. Checking conditions: the if-statement Almost any program relies at some point on the ability to test a certain condition: have we reached the end of a list?, has our simulated Mars lander hit the ground? and so on. Such condition checking is done using the if-statement. This statement has the following syntax:

Ê

Ë

if condition: # Execute only when condition is true Ì statements Here, the elements are: 1. The statement keyword if

3.4. Checking conditions: the if-statement

37

2. The condition to be checked 3. The code to execute only when condition is true An example: Condition checking using if if condition: statement 1 statement 2 ... statement 3 In this example above, only if the condition is True, the statements statement 1 and statement 2 are executed. If the condition is False it will automatically jump to statement 3 without executing statement 1 and statement 2. The conditional lines of code are marked by increasing the margin, which is called indenting the lines after the colon. Lines with the same indentation level, so the same margin, are together called a block of code. In Python a code block is marked only by this same margin. One indentation step is set to four spaces by default in IDLE but in principle any number of spaces will do, as long as all statements in one block have the same indentation. Extra: Code blocks in other languages Note that Python is relatively unique in its use of indentation levels to distinguish related blocks of code. Many other languages, such as Java, C, and C++, use curly braces around a block of code to mark the beginning and end of each block. Because statement 3 starts at the beginning of the line, at the previous level of indentation again, Python knows that this is the end of the conditional block. Note that one of the statements inside the if-statement could be another if-statement, which would mean that another set of conditional statements would be indented one more time (nested if-statements). Optionally the if-statement can be expanded with an else-branch or one or more else-ifs which are spelled as elif in Python. As many elif-branches can be added as necessary. Only one else can be added, always at the end of the if/elif branch(es):

if condition: # Execute only when condition is true statements elif condition2: # Execute only when condition2 is true statements else: # Execute only when all conditions are false statements An example: If/elif/else branching if x < xmin: x = xmin vx = -vx print(” Ball left via left side of screen. ”)

38

3. Python syntax: Statements

elif x > xmax: x = xmax vx = -vx print(”Ball left via right side of screen.”) else: print(”Ball still on screen.”) x = x + vx * dt In this example the condition x > xmax is only checked if x < xmin is False. Which probably is not a problem in this case. In the above example the assignment statement to update the variable x is always executed because it is not indented.

3.5. Iteration using a for-loop The for loop is a very convenient loop, and is often used in combination with the range() function. For example: The for loop for i in range(10): print(i) print(”Ready. ”) If you try this code, you will see that it prints the numbers 0 to 9.

Ê

Ë

for i in range(10): # Print Hello! Ì print(”Hello!”) A for-loop statement consists of three elements: 1. The statement keyword for 2. An iterable (often a counter) 3. Code to execute iteratively The for keyword is how Python recognises that a for-loop is being specified. The iterable determines how often the loop should run, and gives the current position of the iteration in the iterator i (variable name i is often used when iterating over a range of integers, but any variable name is possible!). Finally, the indented code block that comes after the colon (:) will be called on each iteration of the loop. The example above uses the range(10) function as an iterable. This function generates a sequence of numbers (in this case 0-9), through which the variable i is iterated. This means that the indented block of code (one statement in this case) is executed ten times, once for each value of i. When using the range function, the variable i can therefore be seen as the counter of the for-loop. Another example: total = 0 for i in range(1,11): print(i) total = total + i * i

3.5. Iteration using a for-loop

39

print(total) In this example, a variable total is initialised as zero just before the loop. Then, in every iteration of the loop i * i is added to total. Since i runs from 1 until but, not including, 11 in this code, the result is that total will be equal to 1*1 + 2*2 +3*3+ ... + 9*9 + 10*10. Instead of using range() to generate a range of integers, you can also use a for-loop to iterate directly over the contents of a list: Iterating directly over a list fellowship = [”Frodo”, ”Sam”, ”Merry”, ”Pippin”, ”Gandalf”, \ ”Legolas”, ”Gimli”, ”Aragorn”, ”Boromir”] for name in fellowship: print(name) If you need to iterate over a range of floats, e.g., from 0.0 to 10.0, with a step of 0.1, you can also use the following construction. This example will print these values: for i in range(101): x = i * 0.1 print(x) Tip: When setting up loops (while or for), use print statements first (e.g. with a small number of iterations) to see whether your loop does what you intend it to do.

3.5.1. Additional tools for for-loops: itertools The module itertools has a lot of iteration tools which are very useful in combination with a forloop. It for instance provides functions to get any permutation or combination of elements in a loop. This can save a lot of programming time compared to when you would have to make that yourself. For example, this bit of code uses the permutations iterator function, which takes a list as input: Using itertools’ permutations from itertools import permutations lst = [1, 2, 3] for a in permutations(lst): print (a, end=” ”) When executed, this code will generate the following output: (1, 2, 3) (1, 3, 2) (2, 1, 3) (2, 3, 1) (3, 1, 2) (3, 2, 1) The combinations(lst, n) function can be used to generate all n-sized combinations from the values provided in lst: from itertools import combinations lst = [1, 2, 3, 4, 5] for a in combinations(lst, 2): print (a, end=” ”) When run, this code will print:

40

3. Python syntax: Statements (1, 2) (1, 3) (1, 4) (1, 5) (2, 3) (2, 4) (2, 5) (3, 4) (3, 5) (4, 5)

There are many more useful functions in itertools, such as cycle, product, etc. In the python shell, type help(”itertools”) to see a complete overview. Exercises section 3.5 Exercise 3.5.1: Make a program that prints all the prime numbers until a certain value, (recall that a prime number is not divisible by any number except itself and 1). Exercise 3.5.2: * * * * * * * * *

* * * * * * *

Write a Python program to construct the pattern shown below:

* * * * * * * * *

Exercise 3.5.3: Write a Python program that accepts a string and counts the number of digits and letters in the string.

3.6. Iteration using a while-loop A more versatile loop is the while-loop. It is a sort of a combination between an if-statement and a for-loop. A block of statements is executed for as long as a given condition is True. The block is repeated until the condition becomes False. The syntax of such a while-loop is: Syntax of the while loop while condition: statement 1 statement 2 statement 3 Because it evaluates a condition, while is just like if: it is used to control the flow of a program based on a condition. The difference with if is that the condition checking in the case of while is automatically repeated, until the condition becomes False. Below is an example of a while-loop: h0 = [0.0, 11000.0, 20000.0, 32000.0, 47000.0, 51000.0, 71000.0] hmax = h0[-1] h = float(input(”Enter altitude”)) h = min(h, hmax) i = 0 while h > h0[i + 1]: i = i + 1 When you use this as a sort of “repeat until”, you have to prepare the condition which is tested by while in such a way that the first time it will be True, so the loop is entered at least once. You can see this in the second example below, where found is set to False first. Another example:

3.7. Controlling a loop: break and continue

41

import math n = int(input(”Give a value of n: ”)) i = 1 vn = int(math.sqrt(n)) found = False while i 0 if𝑥 < 0 and 𝑦 ≥ 0 if𝑥 < 0 and 𝑦 < 0 if𝑥 = 0 and 𝑦 > 0 if𝑥 = 0 and 𝑦 < 0 if𝑥 = 0 and 𝑦 = 0

Exercise 9.2.1 Make a small simulation of a cannon ball which has a mass of 10 kg, that is fired with a starting speed of 100 m/s, but it is shot at an angle of 30∘ . Besides that, the canon is placed one meter above the ground and you can assume that this is also the starting height of the cannon ball. a) Make the simulation as described above. Show a plot of the vertical distance against the horizontal distance. So you can see the trajectory of the cannon ball. b) Now extend this program, because this model has too many simplifications. For example a cannon ball is not frictionless, so we should include the drag. For now we assume it is constant and N is set to for example 80. Try to find the accelerations with force equilibrium and plot the trajectory of the ball. c) The last step to finalise our program is that the drag is not constant. It is actually a function of speed, in this program use the relation that 𝐹𝑑 = 0.05𝑉 2 . From Intro-I you know that the gravitational constant changes with altitude, also include this effect in your program. Hint: 𝑅 𝑔 = 𝑔0 𝐸𝑎𝑟𝑡ℎ , where 𝑅𝐸𝑎𝑟𝑡ℎ = 6371 km. 𝑅𝐸𝑎𝑟𝑡ℎ +ℎ

10 NumPy and SciPy: Scientific programming with arrays and matrices The modules NumPy and SciPy have provided users of Python with an enormous range of engineering and scientific computing tools. Many of this has been inspired by the functionality of MATLAB and its provided toolboxes. The syntax and names of functions are often identical. Of the two, NumPy can really be seen as Python’s foundation for scientific computing. With array- and matrix-types as well as a large range of linear algebra functions it forms the basis for SciPy, Matplotlib and many other modules. On top of NumPy, SciPy adds a whole range of sometimes very dedicated scientific computing and engineering functions. Thanks to NumPy and SciPy, Python has become the default language of choice for scientific computing, according to IEEE and many others. Let’s explore the capabilities of these modules, even though we can only scratch the surface in this course. To use these modules, they need to be imported. It has become standard practice to rename them when importing them. NumPy becomes “np” and SciPy becomes “sp”. This means we will often see the following header in our scientific applications. In this course, we assume you have imported the modules as follows: import import import import

numpy as np scipy as sp matplotlib as mpl matplotlib.pyplot as plt

In the NumPy and SciPy documentation it is often even assumed that you have imported everything using from numpy import *. So do not forget to type np. before the NumPy functions and sp. before the SciPy functions, even though you don’t see this in the NumPy and SciPy documentation. Extra: Using NumPy and SciPy together For a long time, the SciPy module also included all of NumPy; so importing SciPy was enough to use both. For current versions of SciPy this is no longer the case! This would also be a good time to add a link to the Help chm-files containing the NumPy and Scipy reference to your IDLE Help menu, if you have not already done so. Go to Options>Configure IDLE, click on the “General”-sheet on the top right. Then click “Add” in the lower part of the window to add the menu item and the link to these files.

10.1. Arrays So far we have seen lists and even lists of lists. When we used lists as tables we often kept separate columns in separate one-dimensional lists, so we could select them individually (like xtab, ytab in the 87

88

10. NumPy and SciPy: Scientific programming with arrays and matrices

previous chapter). And if we used a two dimensional table, we could add small lists of two elements to the lists like this: for i in range(1000): x = x + vx * dt y = y + vy * dt postab.append([x, y]) print(”Last x-position: ”,postab[-1][0]) print(”Last y-position: ”,postab[-1][1]) But how can we now select only the first column? Unfortunately, postab[:][0] does not give this result. This indicates, as well as many other examples, that two dimensional lists are, as a variable type, despite their versatility, not always the most suitable type for scientific computing and working with large tables in a fast and user-friendly way. The module NumPy has an advanced ‘table’-type which solves this: the array. This type forms the foundation for NumPy and SciPy. With it you can do computations with entire tables, as easy as if they are one scalar variable. Look at the example below: import numpy as np import matplotlib.pyplot as plt # The function np.arange() creates an array of floats # using the given start, stop and step x = np.arange(0.0, 10.0, 0.1) # With arrays you can do calculations on complete arays with one ↪ expression y1 = np.sin(x) y2 = np.cos(x) y3 = y1 + y2 # adds element by element # Plot all of them plt.plot(x, y1) plt.plot(x, y2) plt.plot(x, y3) plt.show() As you can see, NumPy arrays can be used integrally in equations: they can be multiplied with each other, or with a scalar, and they can be used in NumPy functions like np.sin(), np.exp(), np.sqrt(), etc. All functions and operators, work on an element-by-element basis with arrays.

10.1.1. Creating NumPy arrays There are different ways to create a NumPy array. The easiest examples are arrays filled with a single, repeating value: np.zeros(shape) np.ones(shape)

Create an array of zeros with given shape: e.g. np.zeros((2,3)) Create an array of ones with given shape: e.g. np.ones((2,3)). Note that you can also use this function to create an array with other values than one: np.ones((2,3)) * 3.1415

Similar to the built-in Python range() function, NumPy can also create arrays with evenly (or logarith-

10.1. Arrays

89

mically) spaced values: np.arange(start, stop, step)

np.linspace(start, end, nelem)

np.logspace(start, end, nelem)

Create an array with floats evenly spaced with step (stop not included). Unlike the range() function it works with floats and the result is a NumPy array. Example: arange(0.,0.6,0.1) will result in [0.0, 0.1, 0.2, 0.3, 0.4, 0.5] Create an array ranging from start to, and including, end with nelem elements, results in a NumPy array. Example: np.linspace(1, 3, 5) will result in [1.0, 1.5, 2.0, 2.5, 3.0] Create an array ranging from basestart to, and including, baseend with nelem elements, results in a NumPy array. The default base is 10, which you can change with keyword argument base. Example: np.logspace(1, 4, 4, base=2) will result in [2.0, 4.0, 8.0, 16.0]

You can also create an array from an existing list, or grow an array with its append() function: np.array(lst) np.append(array_in, values)

Create a NumPy array from an existing (nested) list. Create a new array based on array_in, with values appended at the end.

Note that in contrast to list.append(), the np.append() function copies the entire original array for each append. This has as a drawback that it is slower, especially for large arrays. The difference between NumPy and list append import numpy as np # List append xtab.append(x) # NumPy array append xtab = np.append(xtab, x) The effect is more than a different syntax. The np.append() function makes a copy of the array with the value appended to it. In the example above, because xtab is on both sides of the equals sign, the original variable is overwritten. The end effect is therefore equal to list.append(). The real penalty however is the decrease in execution speed. With large quantities of data, making a copy takes extra time which can slow down your code immensely. There are two better alternatives, that you can use, depending on the situation: 1. Start out by creating your data in a (nested) list. When it is complete convert it to a NumPy array: arr = np.array(lst). 2. When you already know the size of the array, generate an array of the right size with np.zeros(shape) (e.g. np.zeros(10) or np.zeros((4, 4)), and change its individual values using indexing: arr[1] = 10.

10.1.2. Indexing and slicing NumPy arrays Note that slicing and indexing works slightly different for NumPy arrays, but that this does allow you to do things that are not possible in nested lists: single element whole row whole column

list a[i][j] a[i][:] a[:][j]

array a[i,j] a[i,:] a[:,j]

90

10. NumPy and SciPy: Scientific programming with arrays and matrices

As you can see, with nested lists you add a set of square brackets in your expression for each nested list you want to access: a[i] accesses the outermost list to obtain row i, after which you get the j𝑡ℎ element of the row: [j]. With NumPy arrays on the other hand, you index all dimensions of your array within one set of square brackets. For a two-dimensional array this is arr[i_row, j_column]. Because with lists Python stores two-dimensional tables as lists of rows, it is possible to access whole rows, but not whole columns. NumPy arrays do allow accessing whole columns, Using the [:,j] slice illustrated above. In the same way you can also slice very specific subsets from an array: Slicing for NumPy arrays import numpy as np # Create a random 10x10 array arr = np.random.random((10, 10)) # Get the bottom-right 2x2 corner sub = arr[-2:,-2:] # Get every second element of the fourth row els = arr[3, ::2] Besides the slightly different notation for multi-dimensional arrays, compared to lists, slicing otherwise works in exactly the same way. Have a look at section 2.9.2 if you feel you could use a refresher on slicing! Exercises section 10.1 Exercise 10.1.1: As you have seen in the reader, arrays can make your life a lot easier. Therefore in this first exercise we want you to plot a somewhat harder function, but by making use of arrays, you can be much quicker. So make a plot of a hexagon in python, which fits inside a circle with radius 2. Exercise 10.1.2:

Make a function, when you give a certain n x n array (2D) you get the main 2 5 diagonal, so e.g. if you have 𝐴 = [ ], we want our function to print 2, -1. Make sure this 4 −1 function can work for all n x n matrices. Exercise 10.1.3: Make a small program that gives the first column of a 2-dimensional list and also make a program that does this for an array. Do you see the difference and how much easier arrays are for this use.

10.2. Logical expressions using arrays The previous section already described how you can perform mathematical operations like multiplication, addition, and subtraction, but also mathematical functions like sine, cosine, and square root, on entire arrays in a single expression. This allows you to, for instance, calculate a full period of the sine function using just two lines of code: import numpy as np phi = np.linspace(0, 2 * np.pi, 1000) sinphi = np.sin(phi) Logical expressions, which you know from sections 2.8 and 3.4, can also be expressed using array comparisons, and arrays of booleans. Take for example two floating point arrays a and b:

10.2. Logical expressions using arrays

91

a = np.array([0, 6, 3, -1, -10, 22]) b = np.array([10, 20, 30, 40, 50, 60]) The following logical expressions result in arrays of boolean values, as shown in the middle column: Expression a > 0

a[a > 0]

Resulting array [False, True, True, False, False, True] [False, True, True, False, False, False] [False, False, False, True, True, True] [6, 3, 22]

b[a > 0]

[20, 30, 60]

np.where(a > 0)[0]

[1, 2, 5]

(a > 0) * (a < 10) (a < 0) + (a >= 10)

Explanation Per-element greater than zero comparison for elements in a Multiplication with booleans works as AND Summing booleans works as OR Using a boolean array as an index: Selects elements of a where a>0 is True Selects elements of b where a>0 is True. For this to work, a and b need to have the same size. Return a list of indices of where condition (a > 0) was True. (The [0] refers to the first, and in this case only, dimension of indices. In twodimensional tables there would also be a list of indices which can be retrieved by [1])

NB: Logical and, or, not: doing the math NumPy has functions for and, or, and not: np.logical_and(), np.logical_or(), and np.logical_not(). But you can also use boolean arithmetic to achieve the same: Operator

Example

Arithmetic

AND &, *

True * True = True True * False = False False * False = False

1 * 1 = 1 1 * 0 = 0 0 * 0 = 0

OR |, +

True + True = True True + False = True False + False = False

1 + 1 = 2 1 + 0 = 1 0 + 0 = 0

NOT !, 1−

1 - True = False 1 - False = True not True = False

1 - 1 = 0 1 - 0 = 1 1 - 1 = 0

There are different ways to use logic in a program which you have vectorised with NumPy. Imagine we want a behaviour comparable with the unvectorised code below: import numpy as np h tropo

= np.arange(0, 20.0, 1.0) = 288.15 - 6.5 * h

92

10. NumPy and SciPy: Scientific programming with arrays and matrices

strato = 216.65 * np.ones(len(h)) temp

= []

for i in range(len(h)): if h[i] < 11: temp.append(tropo[i]) else: temp.append(strato[i]) There are different ways to do this vectorised: You can use multiplication, making use of the fact that a True-value behaves as one and a False value as zero: # This ... temp = (h < 11) * tropo + (h >= 11) * strato # does the same as this: swtropo = h < 11 temp = swtropo * tropo + (1 - swtropo) * strato Using the conditional indexing to only select the values for which the condition is True and gluing these arrays together with append: temp = np.append(tropo[h < 11], strato[h > 11]) Using the np.where() function to switch between two arrays: # where(condition, value_if_true, value_if_false) temp = np.where(h < 11, tropo, strato) Do not confuse this use with the other function of the where-function: when used with only a condition it returns arrays with indices of where it was found to be true. Exercises section 10.2 Exercise 10.2.1: Expand the program above so you can calculate the temperature until 32.000 m. Note that the temperature gradient between 20 km and 32 km is 1.0 K/km. Make a plot of the temperature against the height.

Exercise 10.2.2:

Plot the discontinuous function 𝑓(𝑥) = {

𝑥 2 ∀𝑥 ≤ 1 6 − 𝑥∀𝑥 > 1

Exercise 10.2.3: Extract from the array [3, 4, 6, 10, 24, 89, 45, 43, 46, 99, 100] with Boolean masking all the numbers: • which are not divisible by 3 • which are divisible by 5 • which are divisible by 3 and 5 • which are divisible by 3 and set them to 42

10.3. Speeding it up: Vectorising your code with NumPy Check out the two variants of the same program below. These examples use two different ways to decompose a list of speeds and headings into northern speeds and eastern speeds (assuming no sideslip and no wind).

10.3. Speeding it up: Vectorising your code with NumPy

93

The examples below also time the execution time of both approaches. Decomposition using loops ''' Calculating V North and V East the ”list way” ''' import math from random import random as rnd from time import clock t0 = clock() # Generate lists with random speeds & headings n = 10000 vetab = [] vntab = [] for i in range(n): V = 100.0 + 250.0 * rnd() hdg = 360.0 * rnd() vntab.append(V * cos(radians(hdg))) vetab.append(V * sin(radians(hdg))) dt = clock() - t0 print(”Lists took”, dtlst, ”seconds”) Decomposition using NumPy vectorisation ''' Calculating V North and V East the ”Numpy array way” ''' import numpy as np from time import clock # Generate lists with random speeds & headings t0 = clock() n = 10000 # Create random V and hdg arrays V = 100.0 + 250.0 * np.random.random(n) hdg = 360.0 * np.random.random(n) # Decompose using NumPy array expressions vntab = V * np.cos(np.radians(hdg)) vetab = V * np.sin(np.radians(hdg)) dtarr = clock() - t0 print(”Arrays took”, dtarr, ”seconds”) The result show that with 10000 elements the NumPy method is about 11 times faster than the list method! And also the NumPy code looks cleaner. The difference between the two approaches is that when you apply a function or an operator on the complete array, or the complete vector, the looping over the elements is handled by low-level code inside NumPy. The reason why this is faster is twofold: • the loop takes place in the fast, compiled low-level code of NumPy • list elements can have different types, while all array elements always have the same type, this saves the computer from checking the type for each element and this saves execution time

94

10. NumPy and SciPy: Scientific programming with arrays and matrices

Changing the code from treating each element to a complete array (or vector) at once, is called vectorising your code. In general, it is a good rule of thumb that when you can vectorise your code, you should. Sometimes lists are easier or the lists are not that large that it is needed. But for large quantities of data with the same type, vectorising is nearly always a good idea. Using two-dimensional arrays and the transpose function, vectorising can be really powerful for geometrical calculations, see the example below: import numpy as np # n x y

Some random coordinates = 4 = np.array([12, 2, 35, 11]) = np.array([1, 54, 23, 7])

# Make two dimensional x = x.reshape((n,1)) y = y.reshape((n,1)) # Calculate distance matrix dx = x-x.T dy = y-y.T print(”dx = ”) print(dx) dist = np.sqrt(dx * dx + dy * dy) del dx, dy # Free up memory print(”Dist = ”) print(dist) The output of this program is: dx = [[ 0 10 -23 1] [-10 0 -33 -9] [ 23 33 0 24] [ -1 9 -24 0]] Dist = [[ 0. 53.93514624 [ 53.93514624 0. [ 31.82766093 45.27692569 [ 6.08276253 47.85394446

31.82766093 45.27692569 0. 28.8444102

6.08276253] 47.85394446] 28.8444102 ] 0. ]]

The reshape operation of x and y is necessary to turn these arrays into two-dimensional arrays, with a size of one along one of its dimensions. While this doesn’t seem to change these arrays (they are still sized 4 × 1), without this the transpose operation on the following lines (x.T and y.T) would actually not work: in a one-dimensional ‘world’, transposing has no meaning! To illustrate the transposition done by x.T: >>>print(x) array([[12], [ 2], [35], [11]])

10.4. Matrix operations: Linear algebra with NumPy

95

>>>print(x.T) array([[12, 2, 35, 11]]) Note that after transposition, x.T is still a two-dimensional array (you can tell this from the double square brackets). As you can see in the output of the program, the expression dx = x - x.T results in a 𝑛 × 𝑛 matrix, in which dx[i,j] equals x[i] - x[j]. The downside of using vectors this way with a lot of data is that all intermediate answers also become vectors. With large quantities of data, you can easily consume a lot of memory this way. In that sense, vectorising sometimes means exchanging speed for memory usage. A good way to avoid running out of memory is therefore to delete arrays that are used only for intermediate results. But together with some linear algebra, you can speed things up enormously.

10.4. Matrix operations: Linear algebra with NumPy So far, the type of arithmetic we’ve seen for NumPy arrays always worked in an element-wise manner: for instance, multiplying two arrays together is done element by element. So for array 𝑎 = [𝑎1 , 𝑎2 , ..., 𝑎𝑛 ] and array 𝑏 = [𝑏1 , 𝑏2 , ..., 𝑏𝑛 ], the product between the two becomes 𝑎 × 𝑏 = [𝑎1 ∗ 𝑏1 , ..., 𝑎𝑛 ∗ 𝑏𝑛 ]. This works in the same way for other operations (addition, subtraction, division). On many occasions this is fine, unless you expect vector or matrix multiplication in a linear algebraic sense. With the multiplication of two vectors you could for instance expect the dot product (resulting in a scalar) or the cross product. With matrix multiplication the results are similarly unexpected: Element-wise multiplication with NumPy arrays: 𝑎1,1 𝑎2,1

𝑎1,2 𝑎2,2

×

𝑏1 𝑏2

=

𝑎1,1 ∗ 𝑏1 𝑎2,1 ∗ 𝑏2

𝑎1,2 ∗ 𝑏1 𝑎2,2 ∗ 𝑏2

What you’d expect with matrix multiplication: 𝑎1,1 𝑎2,1

𝑎1,2 𝑎2,2

×

𝑏1 𝑏2

=

𝑎1,1 ∗ 𝑏1 + 𝑎1,2 ∗ 𝑏2 𝑎2,1 ∗ 𝑏1 + 𝑎2,2 ∗ 𝑏2

There are two ways to achieve the type of multiplication you’d expect when you’re doing linear algebra. The first is by using NumPy’s dot() function: import numpy as np a = np.array([[1, 2], [3, 4]]) b = np.array([[10], [20]]) # # x x

Calculate the matrix-vector multiplication a * b. The following two expressions do the same thing: = a.dot(b) = np.dot(a, b)

You can also calculate the dot-product between two vectors. The result is a scalar, which is equal to ⃗ cos(∠𝑎, 𝑏). 𝑎⃗ ⋅ 𝑏⃗ = |𝑎|| ⃗ 𝑏| Since Python 3.5, there is also the @ operator, with which you can tell Python to calculate a matrix- or dot-product: import numpy as np a = np.array([[1, 2], [3, 4]]) b = np.array([[10], [20]]) # Calculate the matrix-vector multiplication a * b.

96

10. NumPy and SciPy: Scientific programming with arrays and matrices

x = a@b You can use any of the above three methods that you prefer most. Note, though, that when doing matrix multiplications, the shape of a vector (and therefore the placement of the nested square brackets) matters: np.array([[1, 2]]) creates a row vector, while np.array([[1], [2]]) creates a column vector. If you need to convert between the two you can use the transpose attribute (.T) of an array: # Convert a row-vector into a column vector: x = np.array([[1, 2]]).T Below is an overview of the most common matrix operations and their NumPy implementation:

Matrix product: 𝑦⃗ = 𝐴 ⋅ 𝑥⃗ Dot product: 𝑠 = 𝑣⃗1 ⋅ 𝑣⃗2

y = A.dot(x) y = A @ x s = v1.dot(v2) s = v1 @ v2

# or

# or

Cross product: 𝑣⃗3 = 𝑣⃗1 × 𝑣⃗2

v3 = np.cross(v1, v2)

Outer product: 𝐴 = 𝑣⃗1 ⊗ 𝑣⃗2

A = np.outer(v1, v2)

Inverse: 𝐴𝑖 = 𝐴−1 Transpose: 𝐴𝑡𝑟𝑎𝑛𝑠 = 𝐴𝑇 Determinant: 𝑑 = det(𝐴)

Ai = np.linalg.inv(A)

A_trans = A.T

d = np.linalg.det(A)

Extra: np.matrix(): the old way of doing linear algebra with NumPy Previously, you would define your data as a NumPy matrix (np.matrix()) instead of an array to indicate that arithmetic with this variable should behave in the algebraic way, instead of elementwise. However, this type has been deprecated since several years. It is currently still present in NumPy, but users are advised to use arrays instead. Exercises section 10.4 Exercise 10.4.1:

2 −5 Make a program to solve the linear system 𝐴 = 𝑏, where 𝐴 = [4 3 5 −2

6 8], and 9

3 𝑏 = [ 5 ]. −1

Exercise 10.4.2: A motorboat makes an upstream trip (i.e., against the current) of 29.5 km on a river in 3 hours and 10 minutes. Returning the same trip with the current takes 2 hours and 6 minutes. Find the speed of the motorboat and the current speed. Try to solve this with a system of equations and python.

10.5. Genfromtxt: Easy data-file reading using NumPy

97

Exercise 10.4.3: Make a linear trend line 𝑦 = 𝑏1 + 𝑏2 𝑥 from the ten points with coordinates shown below. Use the least squares method to calculate the coefficients for this trend line. 𝑥𝑖

= 0.1𝑖, 𝑖 = 1, 2, 3, … , 10

𝑦𝑖

= 5𝑒 −𝑥𝑖 + 2𝑥𝑖

10.5. Genfromtxt: Easy data-file reading using NumPy You will probably often want to use NumPy to process and analyse things like measurement data, and simulation data. Almost without exception such data will come in large data files, which have very structured contents. We know some ways from chapter 7 to read such files, but lucky for you, NumPy has an even easier way to get your data from its file, directly into an array with just one statement! This function is called genfromtxt(). For example, to read a comma-separated file, with comments indicated with the hash character: table = np.genfromtxt(”test.dat”, delimiter=”,”, comments=”#”) x = table[:, 0] # x in first column y1 = table[:, 1] # y1 in 2nd column y2 = table[:, 2] # y2 in third column Here we see that the different columns of the two-dimensional array table are copied in independent one-dimensional arrays x, y1 and y2. There are many arguments you can pass to numpy.genfromtxt(): numpy.genfromtxt(fname, dtype=, comments='#', ↪ delimiter=None, skiprows=0, skip_header=0, skip_footer=0, ↪ converters=None, missing='', missing_values=None, ↪ filling_values=None, usecols=None, names=None, excludelist=None, ↪ deletechars=None, replace_space='_', autostrip=False, ↪ case_sensitive=True, defaultfmt='f%i', unpack=None, usemask=False, ↪ loose=True, invalid_raise=True) As you can see from this list of arguments, numpy.genfromtxt() provides the ability to load data from a text file, with missing values handled as specified. Above the default values for the keywords are shown. Each line past the first skiprows line is split at the delimiter character, and characters following the comments character are discarded. A selection of useful arguments with explanation: fname

file object or str File or filename to read. If the filename extension is gz or bz2, the file is first decompressed.

dtype

data type, optional Data type of the resulting array. If None, the dtypes will be determined by the contents of each column, individually.

comments

str, optional The character used to indicate the start of a comment. All the characters occurring on a line after a comment are discarded

delimiter

str, int, or sequence, optional The string used to separate values. By default, any consecutive whitespaces act as delimiter. An integer or sequence of integers can also be provided as width(s) of each field.

98

10. NumPy and SciPy: Scientific programming with arrays and matrices

skip_header

int, optional The numbers of lines to skip at the beginning of the file.

skip_footer

int, optional The numbers of lines to skip at the end of the file

missing_values

variable or None, optional The set of strings corresponding to missing data.

filling_values

variable or None, optional The set of values to be used as default when the data are missing.

usecols

sequence or None, optional Which columns to read, with 0 being the first. For example, usecols=(1, 4, 5) will extract the 2nd, 5th and 6th columns.

10.6. SciPy: a toolbox for scientists and engineers The previous sections introduced NumPy, the foundation for scientific computing in Python. SciPy is a package on top of NumPy, provinding a large amount of additional functionality, often more specific for a given purpose. The following lists give you an overview of the functionality in NumPy and SciPy respectively: NumPy functionality • Array creation routines

• Window functions

• Array manipulation routines

• Floating point error handling

• Indexing routines

• Masked array operations

• Data type routines

• Numpy-specific help functions

• Input and output

• Miscellaneous routines

• Fast Fourier Transform (numpy.fft)

• Test Support (numpy.testing)

• Linear algebra (numpy.linalg)

• Asserts

• Random sampling (numpy.random)

• Mathematical functions with automatic domain (numpy.emath)

• Sorting and searching • Logic functions • Binary operations

• Matrix library (numpy.matlib) • Optionally SciPy-accelerated routines (numpy.dual)

• Statistics

• Numarray compatibility (numpy.numarray)

• Mathematical functions

• Old Numeric compatibility

• Functional programming

(numpy.oldnumeric)

• Polynomials • Financial functions • Set routines SciPy functionality

• C-Types Foreign Function Interface (numpy.ctypeslib) • String operations

10.6. SciPy: a toolbox for scientists and engineers • Clustering package (scipy.cluster) • Constants (scipy.constants)

99 • Optimization and root finding (scipy.optimize)

• Fourier transforms (scipy.fftpack)

• Signal processing (scipy.signal)

• Integration and ODEs (scipy.integrate)

• Sparse matrices (scipy.sparse)

• Interpolation (scipy.interpolate)

• Sparse linear algebra

• Input and output (scipy.io) • Linear algebra (scipy.linalg) • Maximum entropy models (scipy.maxentropy) • Miscellaneous routines (scipy.misc) • Multi-dimensional image processing (scipy.ndimage) • Orthogonal distance regression (scipy.odr)

(scipy.sparse.linalg) • Spatial algorithms and data structures (scipy.spatial) • Distance computations (scipy.spatial.distance) • Special functions (scipy.special) • Statistical functions (scipy.stats) • C/C++ integration (scipy.weave)

On the internet, many more modules, which use SciPy and NumPy can be found. For nearly all fields of science and engineering, modules with many tools are available for free.

10.6.1. SciPy example: Polynomial fit on noisy data Suppose we receive the following data file containing a time history:

C C Output theta in radians to block response of elevator C Elevator = 0.2 radians between 1 and 2 seconds C Boeing 737 C timestamp in seconds C 0.0 = -0.00122299949023 0.1 = -0.0148544598502 0.2 = -0.00128081998763 0.3 = -0.00912089119957 ... 19.7 = 0.0375150505001 19.8 = 0.0133852241026 19.9 = 0.0195944297302

When we plot this data we get this figure, showing a very noisy signal:

100

10. NumPy and SciPy: Scientific programming with arrays and matrices

Noisy data 0.150 0.125 0.100 0.075 0.050 0.025 0.000 0.025 0.0

2.5

5.0

7.5

10.0

12.5

15.0

17.5

20.0

In this plot we can see the original signal, to get this line we can try a polynomial fit, to get a signal more close to the original signal. Note how in this program we use the function genfromtxt(from NumPy to read the data into a two-dimensional array with one program line! Then we select the two columns. To fit the data we use the function polyfit, which returns the polynomial coefficients, in this case set to a 10th order polynomial. The result is then written to a file. Polyfitting using SciPy import numpy as np import scipy as sp import matplotlib.pyplot as plt # Read file into tables: table = np.genfromtxt(”data/flightlog.dat”, delimiter=”=”, ↪ comments=”C”) xtab = table[:, 0] ytab = table[:, 1] plt.title('Noisy data') plt.plot(xtab,ytab,”r+”) plt.show() # Polyfitting 10th order polynomial coefficients = sp.polyfit(xtab, ytab, 10) polynomial = sp.poly1d(coefficients) ysmooth = sp.polyval(polynomial,xtab) plt.plot(xtab, ysmooth) plt.plot(xtab, ytab, ”r+”) plt.title('Polynomial 10th order') plt.show() with open(”filtered.log”,”w”) as g: g.write(”C\n”) g.write(”C Data smoothed by fitting a 10th order polynomial\n”) g.write(”C\n”) for i in range(len(xtab)): line = f”{xtab[i]} = {ysmooth[i]}\n” g.write(line) The resulting plot shows both the power and limitation of polynomial fitting:

10.6. SciPy: a toolbox for scientists and engineers

101

Polynomial 10th order 0.150 0.125 0.100 0.075 0.050 0.025 0.000 0.025 0.0

2.5

5.0

7.5

10.0

12.5

15.0

17.5

20.0

The curve is indeed a smooth polynomial. The disadvantage is that there are limitations to how a polynomial can fit the data points. Even when a higher order is chosen (10th order seems to be working for a smooth curve with 8 local minima/maxima) it does not give a useful result outside the interval. So while it is very useful for interpolation, it should never be used for extrapolation. More advanced methods take into account a certain assumed relation of which the parameters will then be estimated. The least squares method, which minimises the square of the error, can be used for this and is made available by SciPy.

∗

11

Tuples, dictionaries, and sets In the previous chapters we learned about different kinds of variables. Some relate to single (scalar) values, such as float, int, and bool, other variables are actually collections of variables. The collections we’ve already seen are Python’s list, str (text string), and the NumPy array. In this chapter we’ll extend this with the built-in collection types tuple, dict, and set.

11.1. Tuples Python’s built-in type tuple we’ve actually already glanced at in chapter 6, where it was shown that in Python, tuples can be used to return multiple values from a function. A tuple can be seen as a list that cannot be modified after it is created: it is immutable (just like strings). The variable can be overwritten by a new tuple, but individual elements stored in a tuple cannot be assigned a different value, nor can elements be added or removed. This is the only difference between lists and tuples! When creating lists and tuples, Python distinguishes between them as follows: to create a tuple, enclose the values that should be stored in the tuple by round (normal) parentheses. As you already knew, a list is created when square brackets are used. With a tuple it is also possible to omit the parentheses, so two valid ways to create a tuple are: origin = (0, 0) pos = 3, 4 If you want to call a function, passing an anonymous tuple (d1 in the example below) as one of the arguments you always need to use the syntax with parentheses. For named tuple variables this is not necessary (d2 in the example below): d1 = dist((3, 4), (-3, 6)) d2 = dist(origin, pos) If you would omit the brackets in the second line, Python would think you call the function dist with four arguments. It seems like a tuple is a list with a limitation, so what are they used for? Tuples can be seen as multi-dimensional constants. So for instance if you want to specify a named RGB-colour by its redgreen-blue components you could use the following assignment to defines these colours for later calls to a graphical library: black = (0, 0, 0) white = (255, 255, 255) brightred = (255, 0, 0) red = (127, 0, 0) cyan = (0, 255, 255) 103

104

11. Tuples, dictionaries, and sets

We’ve also already seen that matplotlib.pyplot.legend() uses a tuple for the legend text, Although this legend()-function can also be called with a list. In the next chapter about Pygame, tuples are used for colors and positions.

11.2. Sets There are many more list-like types. In most cases the list type will work for you. But sometimes a special list-like type can be convenient. Sets are lists used for unordered collections of unique elements. It is used to check membership of a collection, overlaps of collections, etc. Sets can be defined using curly brackets: myset = {1, 6, 3, 88, 10} Lists, tuples, and (one-dimensional) arrays can also be converted to sets: s1 = set([1, 1, 2, 3, 4]) # from a list s2 = set(np.array([4, 5, 100])) # from an array s3 = set((4, 4, 4)) # from a tuple In this conversion, order is lost, and repeating values are discarded. These properties make sets mostly useful when assessing membership of a value in a group. Such operations belong to what is called mathematical set theory. Imagine you have two lists with a set of numbers, and you simply want to know whether they use the same digits, than converting them to sets and comparing it will do this for you. There are a number of methods which can be used on sets: a = {1, 2, 5, 3, 6, 7, 0} b = a.copy() # returns contents of a as a set (shallow copy) b.add(100) # adds 100 to set b (but not to set a) b.remove(7) # removes 7 from set b xdif = b.difference(a) # return elements in b which are not in a xdif = b - a # does the same as the difference method overlap = a.intersection(b) # return set items which are in a and in b overlap = a & b # does the same as the intersection method. together = a.union(b) # return a set with elements in a or in b together = a | b # does the same as the union method print(a.issubset(b)) # Prints True if b is completely contained in a For a complete overview, including the operators, type “help(set)” in the Python shell.

11.3. Dictionaries Another special variant of the list that can sometimes be very useful is the dictionary. Dictionaries are like lists, where instead of using an integer index to access a value, you use a key to look up a value. Dictionary keys can variables of any immutable type (e.g., str, int, float, or tuple), although string keys are most common. An example of using a dictionary is given below: >>> ages = { ”Bob”: 20 , ”Alice”: 18 ,”Jacco”: 29 } >>> ages[”Jacco”] 29 >>> ages[”Jet”] Traceback (most recent call last): File ””, line 1

11.3. Dictionaries

105

ages[”Jet”] KeyError: 'Jet' >>> This error can be prevented by first checking whether the “Jet” key is present in this dictionary: if ”Jet” in ages: print(ages[”Jet”]) We can also add entries in the way that you would intuitively expect: ages[”Jet”] = 30 Even though the order is not defined you can still iterate over dictionaries. The following code for a in ages: print(a) will print all keys. Then again, if you wanted not the keys but the entries, you can use one of the following: # Only print all of the values: for age in ages.values(): print(age) # Use items() instead of values() to get both key # and corresponding value in a loop: for name, age in ages.items(): print(f'{name} is {age} years old')

12 Pygame: Animation, visualisation and controls The Pygame module contains a set of user-friendly modules for building arcade-like games in Python. But because it has several drawing routines as well as keyboard controls, it is also a very convenient library to make any 2D graphics, especially moving graphics with some keyboard controls. This has made Pygame an often-used graphics library, also for other purposes than games, like animation, visualization of simulation and all other non-moving drawings which need something more versatile than Matplotlib, which produces graphs but is not a graphics library. Also key and mouse controls are part of Pygame and can be added to pan, zoom or control a simulation. In this chapter we will explore different modules inside Pygame like display, event, key, mouse, image, transform and draw, as well as the new types (classes to be exact): surface and rect. There are many more modules inside Pygame for you to explore like, music, joystick, etc. but these will not be discussed here. Next to Pygame, some basics of setting up a game or simulation will be discussed such as the ‘game-loop’ and timing issues. Pygame has a very good online reference manual at https://pygame.org/docs. At the top of this page there is a clickable overview of each module inside Pygame. Click on these names to get a list of functions in that module, and click on the function name in this list to get a full description.

12.1. The basics: importing and initialising Just as with NumPy and Matplotlib, Pygame is an external module. So before you can use Pygame, you need to import its module. In addition to this, you should also add two calls to Pygame at the beginning and end of your program for initialisation and clean-up, resulting in the following three mandatory lines: import pygame as pg # Initialise the pygame environment pg.init() # Your (game) code here ... # At the end of your program call pygame's quit() function to # properly clean up pg.quit() The init()-call has no immediately visible effect, but it is required to avoid having to initialise each 107

108

12. Pygame: Animation, visualisation and controls

module of Pygame independently. So calling pygame.display.init(), pygame.key.init() is no longer necessary when you start your program with pygame.init(). pygame.quit() will have a visible effect: it will close any Pygame windows still open. If during the development your program crashes due to an error and the Pygame window is still open, type pygame.quit() in the shell to close it. (or pg.quit() if you used import pygame as pg).

12.2. Setting up the Pygame window Before you can draw anything, you need a window to draw in. Creation of this window is provided by the set_mode() function of the pygame.display module. Although it has more arguments, you’ll often call it with just a tuple containing the requested window size in pixels: import pygame as pg # Create a window 600 pixels wide, 500 pixels high reso = (600, 500) screen = pg.display.set_mode(reso) Or the same in one line (mind the double brackets, see section ?? for more information on tuples): screen = pg.display.set_mode((600, 500)) The function pygame.display.set_mode() returns a Surface, which is stored here in the variable name screen (could have been any name). In this case the surface refers to the video memory of the screen, so we have called this variable screen in our example but it could have been any other name like win, window1, scr, etc. In our example screen is now a surface we can draw on. In computer graphics, a coordinate system different from mathematics is used, mainly for historical reasons. So for Pygame get used to the following: The top left corner is the origin, x-coordinates are from left (x=0) to right (x=horizontal pixel size of window), and y-coordinates are from top (y=0) to bottom (y=vertical pixel size of the window). This coordinate system stems from the old text terminals, which had line zero and column zero in the top left. So the window, which we just created, has the following coordinate system: x-coordinate

y-coordinate

(0, 0)

(0, 500)

(600, 0)

screen

(600, 500)

Extra: When two dimensions aren’t enough In the computer graphics world things get really complicated in three dimensions. In the graphics coordinate system, the Z-axis points positive to the viewer, in effect creating a left-handed (!) axes reference system, where cross products and other functions work just the other way around. So screen coordinates work in a non-standard way and are also always integers. For these two reasons, it is common to define your own so-called world coordinates (with a right-handed reference frame), which you use for your model. Only at the final drawing step (just before you draw) you convert them to

12.3. Surfaces and Rectangles

109

screen coordinates. An added benefit is that your drawing code doesn’t rely on a particular window size in pixels anymore, but can scale with a resized window, or when changing from windowed to full-screen mode. An example of a simple 2D world coordinate system is: x-coordinate

(0, 0)

y-coordinate

(0, 1)

(600, 0) (1.2, 1)

screen

(0, 0) (0, 500)

(1.2, 0) (600, 500)

In this case we can do calculations using any position in floats, like do numerical integration in floats and then only when we draw convert our float world coordinates to the integer screen coordinates just before plotting with the following lines of code. In the example below the world coordinates are (𝑥, 𝑦) and are converted to screen coordinates (𝑥𝑠 , 𝑦𝑠 ): xmax = ymax = reso = screen

600 500 (xmax, ymax) = pygame.display.set_mode(reso)

... while running: x = x + vx * dt y = y + vy * dt xs = int(x / 1.2 * xmax) ys = ymax - int(y * ymax) NB: The aspect ratio of your window Note that if you use such a custom coordinate system, and you allow varying window sizes, you have to keep in mind that the aspect ratio of your window (the ratio between its width and height) cannot be assumed constant! If you ignore this circles might turn out as ellipses, and squares as rectangles.

12.3. Surfaces and Rectangles There are two new concepts to grasp in Pygame: they are called Surface and Rect. They each have their own section in the https://pygame.org/docs documentation. A Surface is a type to hold an image, a bitmap. It can be an entire screen, an image loaded from a file or (often) a smaller part. You can draw on a Surface; you can copy and paste any surface, or parts of it, on another surface. A Surface can also have a transparent background so that when you paste it over another surface some pixels will keep the color of the original surface. All in all, it is a very versatile type allowing you to manipulate or transform (scale move, copy/paste, rotate) arrays of pixels (bitmaps). When you paste one surface onto another, this is called blitting in the computer graphics world. It stems from the “block of bits” of the video memory which is transferred with one call to a blit function. Before we can do this we also need to specify where we want this rectangular image to be pasted.

110

12. Pygame: Animation, visualisation and controls

When we have loaded an image from the disk, we need to know how large it is and where we should be able to position it onto another surface later. This is where Pygame’s Rect (rectangle) type comes in. A rectangle is not the actual surface but rather an object that contains the dimensions and position of a Surface. It has several members, which we can assign values to. The neat thing is that we do not need to worry about the bookkeeping: when you change one value, the other ones will be changed automatically when needed. One simple example is given below. To clear the screen, we draw a black rectangle. For this we need to specify the position and scale first. If we get the Rect from a surface to measure the size, the position is by default set to (0,0) for the top left corner. To do this, we use the method get_rect() of a Surface (see the section on Surface in the Pygame documentation for a full description of the methods of surface). black = (0, 0, 0) scrrect = screen.get_rect() pygame.draw.rect(screen, black, scrrect) A Rect has the following members, which you can read or assign a value to: top, left, bottom, right topleft, bottomleft, topright, bottomright midtop, midleft, midbottom, midright center, centerx, centery size, width, height w, h Center is a tuple equal to (centerx, centery). The user can choose to position the rectangle using any combination of these members. Another example where the Rect type is used is given below, where an image is loaded from a file on the hard disk. Generally, we do this loading from a file only once at the beginning of our program, because accessing the hard disk has a huge execution speed penalty. The example shows how to use the rectangle: first we get the size from the surface object and then we position it and use it to blit the surface on the screen surface: ship shiprect

= pygame.image.load(”rocket.gif”) = ship.get_rect()

... while running: shipx = shipx + vx * dt shipy = shipy + vy * dt shiprect.centerx = shipx shiprect.centery = shipy In the code we can see how the rectangle is used to position the bitmap, which was read from rocket.gif, on the screen. The same call can be used with a tuple containing the coordinates of the top-left corner of the surface. As visible in the syntax, blit is a method from the Surface type, hence it is called as a method, so with the dot-notation, combined with the destination surface (i.e., the variable named “screen” in our example).

12.4. Bitmaps and images In the previous paragraphs we have seen that surfaces allow you to load images from the disk and blit them to the screen with the following two functions:

12.5. Drawing shapes and lines

111

scr = pygame.display.set_mode((500, 500)) ship = pygame.image.load('lander.gif') shiprect = ship.get_rect() ... while running: ... shiprect.center = (xs, ys) scr.blit(ship, shiprect)

A few remarks about using these functions: As said before, it is important to load the bitmaps before the actual game loop. In general, accessing files (e.g., to load an image) takes a lot of time. This means it might cause hick-ups or delays in your program, if you do this during the loop. Sometimes bitmaps do need some editing before they can be used. When blitting images on a surface a transparent background is often required. This can be edited with a painting program such as Paint.net, Gimp, Paint Shop, Corel Draw or Photo shop and deleting the background so it becomes transparent. Save the file in the PNG format (or GIF) as these formats allow transparent backgrounds. Also use these programs to change colors, size or perform rotations. If you need one bitmap in a lot of different orientations, editing it with a paint program can be cumbersome. These operations can also be done much easier with pygame in your program. This can be done using the transform module which allows you to manipulate bitmaps/surfaces. Some examples of functions available in pygame.transform: flip vertically and horizontally, returns new surface pg.transform.scale(surf, (w, h)) resize to new resolution, returns new surface pg.transform.rotate(surf, angle) rotate an image, angle is float in degrees, rotate an image, returns new surface filtered scale and rotation, angle float in depg.transform.rotozoom(surf, angle, scale) grees, scale factor also float, returns new surface pg.transform.scale2x(surf) specialized image doubler, returns new surface pg.transform.smoothscale(surf, (w, h)) scale a surface to an arbitrary size smoothly, returns new surface pg.transform.flip(surf, xs, ys)

Be aware that you do not scale or rotate inside the game loop unless it is absolutely necessary. In most cases it pays off to generate different surfaces for all possible orientations beforehand, store them in a list and just use the appropriate one during the game loop by setting the index with the angle rounded off to 45, 20 or 5 degrees. The same goes for scaling. Transforming bitmaps is rather computationally intensive and in general it is the goal to do any time consuming operation as much as possible before the actual running of the game loop.

12.5. Drawing shapes and lines The draw module contains functions to draw lines and filled or unfilled shapes such as rectangles, circles, ellipses and polygons:

112 pygame.draw.rect pygame.draw.polygon pygame.draw.circle pygame.draw.ellipse pygame.draw.arc pygame.draw.line pygame.draw.aaline

12. Pygame: Animation, visualisation and controls draw a rectangle shape draw a shape with any number of sides draw a circle around a point draw a round shape inside a rectangle draw a partial section of an ellipse draw a straight line segment draw fine anti-aliased lines

A few notes on using these functions: The coordinate system used is explained in the section on setting up your screen. It runs from (0, 0) in the top left corner to the maximum x and y in the bottom right. All sizes and positions are integers. Colours are specified with a tuple (red, green, blue), three numbers each ranging from 0 – 255 to indicate the amount of each colour channel present in the mixed color. For readability it helps to define a few colours at the beginning of your program and use these in the calls: black = (0, 0, 0) cyan = (0, 255, 255) white = (255, 255, 255) background = (0, 0, 63) foreground = white Some draw functions allow to switch on what is called “antialiasing”. This means a pixel on the edge of the line will get a colour in between the foreground colour and background colour depending on the ‘amount of line’ which is in the pixel. This avoids the jagged lines you will get with normal bitmapped shapes. One disadvantage of this is that it takes more time and memory: the shape is first drawn in a higher resolution and then “anti-aliased”. Another disadvantage is that you later on cannot select the pixels which are part of the background and not a part of the shape. It depends on your application whether you want to use antialiasing.

Aliased

Anti-aliased

12.6. When your drawing is ready: flipping the display Animation consists of nothing else than redrawing every frame over and over in a simulation or game. To avoid flickering images when the screen is cleared for the next drawing, this is first done in a part of the video memory which is not visible. This area is first cleared to the background colour, and either with drawing (rectangles, circles and lines) or blitting the next image is created. Once it is finished, we can show it on the screen. This also means we will not see anything until we copy this video memory to the screen. This is done with the following call, to be used at end of all your draw and blit calls in the loop. pygame.display.flip() If you have used the default settings when creating the window with pygame.display.set_mode(), a call to flip updates the window with the video memory. When you use the option double buffering, you have two areas of video memory, so two surfaces which will be used simultaneously: one will be shown on screen, the other you can draw on. Once you’re done drawing the frame, you swap the function of both surfaces. The advantage is that you do not need to start with an empty frame then, but can use the one-but-last frame as a start. In practice, to take advantage of this you might need a lot of bookkeeping but it might result in a higher execution speed. Using the full-screen option is often only done when the game or simulation is ready, debugged and tested, since debugging is severely hampered by a crashing full screen application!

12.7. Timing and the game loop

113

12.7. Timing and the game loop Imagine we want to let a rocket take off on the screen. We could use the following code to do so: import pygame pygame.init() reso = (600, 500) screen = pygame.display.set_mode(reso) scrrect = screen.get_rect() black =(0, 0, 0) ship = pygame.image.load(”rocket.jpg”) shiprect = ship.get_rect() shiprect.centerx = 250 for y in range(500, -100, -2): shiprect.centery = y pygame.draw.rect(screen, black, scrrect) screen.blit(ship, shiprect) pygame.display.flip() pygame.quit() Now when we run this, it could turn out that our ship moves too fast or too slow. To fix this, we will then adjust the third argument of the range function in the for-loop, currently set to -2. However, on another computer the speed would again be different. Also, when another application in the background needs the CPU for a short period, our rocket will hamper before continuing. This is because we have no control over the real speed of the rocket. For this we need more than controlling the position, we need to control the timing in our game as well. Or at least measure it and calculate the elapsed time (time step) since we last drew our ship, so that we can calculate the required new position with the speed and right time step based on the elapsed time since the last frame. There are two principles we can use to make sure our simulated time runs in accordance with the real time. These will be discussed in the following sub-sections.

12.7.1. Method 1: Fixed time-step We use a constant time step similar to our previous numerical integration examples. We check whether the real time is equal to or larger than our next to be simulated time, if so, we make a time step dt. This is how this could look when coded in Python: Fixed timestep gameloop import pygame as pg # initialise clock pg.init() # Define constants tsim = 0.0 tstart = 0.001 * pg.time.get_ticks() dt = 0.1 ... running = True while running: trun = 0.001 * pg.time.get_ticks() - tstart if trun + dt >= tsim:

114

12. Pygame: Animation, visualisation and controls

tsim = tsim + dt vx = vx + ax * dt vy = vy + ay * dt x = x + vx * dt y = y + vy * dt The advantage of this method is that you have a fixed and known time-step. This allows for more advanced numerical integration methods and guarantees the stability of your simulation. When using a variable time step (see method II) there is a risk of getting a too large time step, which can result in overshoots and even a runaway of your simulation. A disadvantage of this method is that it needs a time-step of which you can be sure that the computer can always keep up with in terms of computing power. If you make this time step too small, the simulation may lag behind or run at a variable speed, resulting in quirky speeds and movements. However, the opposite, having a time-step that is much larger than the computer needs to perform a single simulation cycle is also unwanted: in this case, the code above would repeatedly and very rapidly run through the while-loop, until it reaches the desired target time. While this effectively amounts to doing nothing for a while, it does mean that your CPU is loaded to its maximum while running over the while-loop. To solve this you can use a built-in function that asks your CPU to sleep (literally do nothing) for a while, making it available for other processes on your computer. The adapted example below uses the sleep() function from the built-in time module to achieve this: Fixed timestep gameloop with CPU sleep import time import pygame as pg # initialise clock pg.init() # Define constants MINSLEEP = 0.001 # Minimum interval to sleep the CPU tsim = 0.0 # Simulation time tstart = 0.001 * pg.time.get_ticks() dt = 0.1 ... running = True while running: # First increment the simulation time tsim = tsim + dt # To avoid high CPU usage, calculate remainder to sleep. remainder = tsim - (0.001 * pg.time.get_ticks() - tstart) if remainder > MINSLEEP: time.sleep(remainder) # Perform simulation/game calculations/actions ...

12.7.2. Method 2: Variable time-step In this case we simply measure the time elapsed since the last time we updated the simulation. This difference is our actual time-step dt, which we then use to update everything including the time. The benefit of this approach is that your time-step is always correct, and when the computer is fast enough it will also use this for higher update frequencies. A downside is that the reverse is also true: if the computer is too slow or occasionally too slow, the simulation might cause problems because of a very large dt. It is possible to safeguard this and catch up later, but then the code gets more complex than we want to use for now. (It becomes basically a mix of the two methods.) Another downside is that with this method your simulation/game will always load your computer to its maximum, even when this

12.7. Timing and the game loop

115

is not strictly necessary to obtain the required accuracy in your simulation. This is how the code looks in Python when using the method of the variable time step. The variable t0 here keeps the previous time when everything was updated: import pygame as pg # initialise clock pg.init() t0 = 0.001 * pg.time.get_ticks() ... maxdt = 0.5 # time step limit to avoid jumps running = True while running: t = 0.001 * pg.time.get_ticks() dt = min(t - t0, maxdt) # set maximum limit to dt if dt>0.0: t0 = t vx = vx + ax * dt vy = vy + ay * dt x = x + vx * dt y = y + vy * dt ...

12.7.3. The termination condition of the game loop One of the above mechanisms will provide the basis of your loop that is executed while running. This is generally called the game-loop. In principle, it runs indefinitely until a quit event is triggered. This quit event can be multiple things: • The escape-key is pressed • A specific condition in your simulation or game is reached (e.g., the rocket has landed (𝑦 = 0), the player is dead, the level is complete, etc.) • An abort or quit event is emitted by the operating system. This for instance happens when the user clicks the close button of your window. In our example so far, just setting running to False will make sure the loop ends: running = True while running: ... ... if y “Save as...” and then for the Save as Filetype underneath where we enter the filename: we choose in the box “Save as type:” “CSV (Comma delimited) *.csv”. There are several options for CSV for Mac and for MS-DOS, but for now we choose the default. You will get a warning the some features are lost, but choose to go ahead (by clicking Yes). In Numbers you can do the same by selecting “File”>“Export to”>“CSV...”. The dialog that follows lets you choose to combine all sheets in one file, or to generate a separate file for each sheet. The figure below gives a typical example of tabular data in a spreadsheet. The remainder of this section will describe how to read this data when it’s provided as a CSV file. The next section will show you how to get this data straight from an excel file. 127

128

15. Reading and writing data files: beyond the basics

When exported to CSV, the resulting file will look like this: MP0101a,,20/07/1969,Exam Introduction to Python's Flying Circus,,,, ,,,max 35 pts,max 15 pts,max 25 pts,75, Student nr,Rounded,Name,Q1,Q2,Q3,Total,Grade 25167,9.3,John Cleese,35,12,22,69,9.28 28785,9.5,Michael Palin,35,15,20.5,70.5,9.46 25194,5.7,Terry Jones,28,11,0,39,5.68 27111,6.6,Terry Gilliam,25,12,10,47,6.64 25872,7.2,Eric Idle,27,10,15,52,7.24 25493,6.3,Graham Chapman,32,10,2,44,6.28 Note how each line contains 7 comma’s as a separator between the eight cells per line, even when the cells are empty. Note also that only data is saved by value, so other features such as lines, formulae etc are lost. Depending on your (regional) settings the comma separator could even be replaced by another character. Also the encoding of character beyond the standard ASCII character set for special characters might vary depending on your settings in Excel and your OS. The following methods handle this for you in different ways.

15.1.1. Doing it yourself In Chapter 7 we’ve already seen how to open files in Python, and process their data using Python’s string methods. Processing the CSV data described above can be done in very much the same way as the examples in Chapter 7. To read the data into a two-dimensional text array (a list of lists containing strings), we remove the newline character (”\n”) and split each line based on the separator character: ””” Reading a full CSV file into a two dimension text array ””” # Start with empty table, where we will append each line with data table = [] with open(”ExamOct.csv”) as f: for line in f: table.append(line.strip(”\n”).split(”,”)) print(table[4][2],”has an unrounded grade of”, table[4][7])

15.1. Tabular data: CSV files

129

When executed, this will print: Michael Palin has an unrounded grade of 9.46 As a result, the sheet will be stored in the list called table, where the first index is the line number, starting with index zero, and the second index is the cell number, also starting with index zero. This means that table[4][2] will give us the value from cell C5: line 5 and column 3 (“C”). Note how Excel by calling it C5, basically uses the column,row indexing while normally data is accessed in the row, column indexing format. Extra: Smarter storing of data using dictionaries If you take some more steps to process the file’s data, removing its header and selecting only the columns of interest, you can also easily build up your own database structure directly from the file: ””” Reading grades directly into dictionary with names as keys ””” gradebook = dict() # dictionary for unrounded grades with open(”ExamOct.csv”) as f: lines = f.readlines() for line in lines[3:]: # Skip header of three lines cells = line.strip(”\n”).split(”,”) if cells[2] != ””: # avoid reading empty line(s) name = cells[2] grade = float(cells[7]) gradebook[name] = grade for student in gradebook: print(student, ”has grade”, gradebook[student]) When executed, this will print: John Cleese has grade 9.28 Michael Palin has grade 9.46 Terry Jones has grade 5.68 Terry Gilliam has grade 6.64 Eric Idle has grade 7.24 Graham Chapman has grade 6.28 Writing to a csv file can be done in much the same way, as it is a text file with a comma as separator. For instance, if we want to write the text array table from the first example, we can use the join method of Python’s str type. For instance, ”,”.join(lst), will return a string with all elements of lst, separated by commas. Our code to save a CSV then becomes: with open(”OutFile.csv”, ”w”) as g: # First write the header g.write(”Student nr,Rounded grade,Name,Q1,Q2,Q3,Total,Grade\n”) # Then write the data in table. # Each line is ended with a newline character for row in table: g.write(”,”.join(row) + ”\n”) The resulting CSV file can in turn be easily opened in Excel, Numbers, or your spreadsheet editor of choice.

130

15. Reading and writing data files: beyond the basics

Extra: Using print to write to files To write our CSV file, Python’s print-function is also very suitable, since in addition to writing to the screen, it can also be asked to print text to other (text-based) outputs: with open(”OutFile.csv”, ”w”) as g: print(”Name,Grade”, file=g) for student in gradebook: print(student, gradebook[student], sep=”,”, file=g)

15.1.2. Doing it with a library The previous method simply does the job and is easy to remember as it uses commonly used string methods. However, Python also comes with a built-in module called csv, which you can use to automate some steps. However, to be honest, for such a simple file format its added value is limited. A simple example of using the reader from the csv module to take care of the string stripping and splitting is given below. It does the same as the first approach in the previous section: from csv import reader table = [] with open(”ExamOct.csv”) as f: datareader = reader(f) for row in datareader(): table.append(row) As you can see this approach is not much different from the first example above. What can save you some work, however, is csv.DictReader, which automatically names your columns using the labels in the header row, and outputs this in a dictionary. However, since the CSV module expects a simple file with the column names on the first row, we need to skip the first two lines in our example, and explicitly tell the DictReader where to find the column names: import csv names, grades = [], [] with open(”ExamOct.csv”) as f: dummy = f.readline(),f.readline() #skip two lines colnames = f.readline().strip().split(”,”) reader = csv.DictReader(f, delimiter = ”,”, fieldnames=colnames) for row in reader: if row[”Name”]!=””: names.append(row[”Name”]) grades.append(float(row[”Rounded”])) print(names) print(grades) When executed this will print: ['John Cleese', 'Michael Palin', 'Terry Jones', 'Terry Gilliam', 'Eric Idle', 'Graham Chapman'] [9.3, 9.5, 5.7, 6.6, 7.2, 6.3] For writing, the same logic can be used with csv.DictWriter(). The level of automation of the csv module is very limited, compared to the already extensive string methods provide by Python. The next section therefore looks at how to directly interface with Excel

15.2. Tabular data: Excel spreadsheets

131

files.

15.2. Tabular data: Excel spreadsheets 15.2.1. Openpyxl: Reading and writing directly to XLSX Excel files To use this method, first use pip to install openpyxl (and automatically its dependency et-xmlfile). So in a command prompt/console (with administrator rights on Windows), run: pip install openpyxl In Openpyxl, contrary to the python numbering logic, the indexing of rows and columns starts with one not with zero (to match the numbering in Excel): import openpyxl # Reading xls file allsheets = openpyxl.load_workbook(”ExamOct.xlsx”, data_only=True)) gradesheet = allsheets.active # get the active (first) sheet gradebook = dict() # Row 4-9 according to Excel numbering is also 4-9 here for irow in range(4, 9): cell = gradesheet.cell(row=irow, column=3) name = cell.value grade = gradesheet.cell(row=irow, column=2).value gradebook[name] = grade print(gradebook) This prints the dictionary gradebook: {'John Cleese': 9.3, 'Michael Palin': 9.5, 'Terry Jones': 5.7, 'Terry Gilliam': 6.6, 'Eric Idle': 7.2, 'Graham Chapman': 6.3} Note that to get the actual values, you need to open the file with the workbook with the data_only switch set to True: allsheets = openpyxl.load_workbook(”ExamOct.xlsx”, data_only=True) This is because in openpyxl, by default we will get what is in the top box of Excel, so in the case for the grades, the formulae which will not be evaluated. So if you leave this option out, you will get the formulae: {'John Cleese': '=ROUND(H4,1)', 'Michael Palin': '=ROUND(H5,1)', 'Terry Jones': '=ROUND(H6,1)', 'Terry Gilliam': '=ROUND(H7,1)', 'Eric Idle': '=ROUND(H8,1)'} This allows accessing the formula, but it is rarely what you intend when interfacing with Excel files. To write to an Excel xlsx-file, you can use openpyxl.Workbook() to construct an Excel workbook, with one or more sheets. The following piece of example code demonstrates how to write two columns of data with a header row to a new Excel-file: # Create a workbook and write the results to it newsheets = openpyxl.Workbook()

132

15. Reading and writing data files: beyond the basics

sheet = newsheets.active # First row: header A1 = sheet.cell(row=1, column=1) A2 = sheet.cell(1, 2) # row, column A1.value = ”Name” A2.value = ”Grade” # Row 2 and down: data row = 2 # Keep track of the row number to write to for name in gradebook: column1 = sheet.cell(row, 1) column2 = sheet.cell(row, 2) column1.value = name column2.value = gradebook[name] row = row + 1 newsheets.save(”Output.xlsx”)

15.2.2. Xlrd and xlwt: Reading and writing the old .xls format Similar to openpyxl, xlrd and xlwt can be used to respectively read and write Excel workbooks, but not in the newer xlsx format, but in the older xls format. Like openpyxl, xlrd and xlwt can read and write workbooks with multiple sheets. To use these modules, first use pip to install them, by opening a command prompt/console (with administrator rights), and running: pip install xlrd xlwt An example of its usage can be found below. Different from openpyxl, the index of the first column and first row is 0, when you use xlrd, following the standard indexing in Python. import xlrd # Reading xls file allsheets = xlrd.open_workbook(”ExamOct.xls”) gradesheet = allsheets.sheet_by_name(”Uncorrected”) gradebook = dict() for irow in range(3,9): cell = gradesheet.cell(irow, 2) name = cell.value grade = gradesheet.cell(irow, 1).value gradebook[name] = grade allsheets.release_resources() # comparable with closing a file print(gradebook) To save to an Excel xls-file, we can use the following piece of code:

15.2. Tabular data: Excel spreadsheets

133

# Writing the results import xlwt newsheets = xlwt.Workbook() sheet = newsheets.add_sheet(”Checked”) sheet.write(0,0,”Name”) sheet.write(0,1,”Grade”) row = 1 for name in gradebook: sheet.write(row, 0, name) sheet.write(row, 1, gradebook[name]) row = row + 1 newsheets.save(”Output.xls”) This stores the results in a new file Output.xls, with two columns of data.

15.2.3. Reading and writing Excel files with Pandas Pandas provides a read_excel function, which is actually a wrapper around the previous modules. So to use it, first install openpyxl or xlrd/xlwt. Similar to readfromtxt in NumPy, pandas excel reader has many options that you can set by passing keyword arguments to the function: pandas.read_excel = read_excel(iofileorstring, sheet_name=0, header=0, names=None, index_col=None, usecols=None, squeeze=False, dtype: ↪ ↪ 'DtypeArg | None' = None, engine=None, converters=None, ↪ true_values=None, false_values=None, skiprows=None, nrows=None, ↪ na_values=None, keep_default_na=True, na_filter=True, ↪ verbose=False, parse_dates=False, date_parser=None, thousands=None, ↪ comment=None, skipfooter=0, convert_float=None, ↪ mangle_dupe_cols=True, storage_options: 'StorageOptions' = None) In our example, we would like to skip the first three rows before we get to the actual data: import pandas table = pandas.read_excel(”ExamOct.xlsx”, skiprows=2) # Result is a pandas dataframe print(table) The result is a pandas dataframe, which looks as follows: # 0 1 2 3 4 5

Student nr 25167 28785 25194 27111 25872 25493

Rounded 9.3 9.5 5.7 6.6 7.2 6.3

Name John Cleese Michael Palin Terry Jones Terry Gilliam Eric Idle Graham Chapman

Q1 35 35 28 25 27 32

Q2 12 15 11 12 10 10

Q3 22.0 20.5 0.0 10.0 15.0 2.0

Total 69.0 70.5 39.0 47.0 52.0 44.0

Grade 9.28 9.46 5.68 6.64 7.24 6.28

The columns are accessible via the labels on the first row read from the file. The lines, which follow, are by default indexed using integers: import pandas

134

15. Reading and writing data files: beyond the basics

table = pandas.read_excel(”ExamOct.xlsx”, skiprows=2) # Result is a pandas dataframe # Use labels of first row to get columns names = table[”Name”] grades = table[”Rounded”] gradebook = {} row = 0 for name in names: gradebook[name] = grades[row] row = row + 1 print(gradebook) This results in the same gradebook dictionary as before: {'John Cleese': 9.3, 'Michael Palin': 9.5, 'Terry Jones': 5.7, 'Terry Gilliam': 6.6, 'Eric Idle': 7.2, 'Graham Chapman': 6.3} To write to an excel file using pandas, we need to make a dataframe and then write this using the Excelwriter function. For our example we can add the following code: import pandas ... # Get columns as list from gradebook dictionary names = list(gradebook.keys()) grades = [] for name in names: grades.append(gradebook[name]) # Make new dataframe using dictionary # with columns names as keys and lists as columns tabledict = {”Name”: names, ”Grade”: grades} df = pandas.DataFrame(data=tabledict) # Write dataframe to a sheet in an Excel-file xlwriter = pandas.ExcelWriter(”Outputpd.xlsx”, engine=”openpyxl”) df.to_excel(xlwriter, sheet_name=”Corrected”) xlwriter.close() Note how we need to specify to engine in the writer, to enable the writing of xlsx-files. Otherwise, as pandas uses the xlwt module by default, we are limited to writing xls-file (Excel 2007).

15.3. Hierarchical data: JSON files JSON is short for a JavaScript Object Notation. It is a text-based method to write data in a structured way. JSON files are a very useful way to read and write data in a readable format. Let’s take this small JSON data file for example: { ”__comment”: { ”name”: ”aircraft full name”, ”wa”: ”wing area (m^2)”,

15.3. Hierarchical data: JSON files

135

”engine_type”: ”Engine typer: TF, TP”, ”mtow”: ”max take-of weight (kg)”, ”mlw”: ”max landing weight (kg)”, ”oew”: ”operating empty weight (kg)”, ”mfc”: ”max fuel capacity (L)”, }, ”A319”: { ”name”: ”Airbus A-319”, ”n_engines”: 2, ”engine_type”: ”TF”, ”wa”: 122.4, ”mtow”: 75500, ”mlw”: 62500, ”oew”: 40800, ”mfc”: 30190, }, ”A320”: { ”name”: ”Airbus A-320”, ”n_engines”: 2, ”engine_type”: ”TF”, ”wa”: 122.6, ”mtow”: 78000, ”mlw”: 66000, ”oew”: 42600, ”mfc”: 24210, } } As you can see, these data are comparable to dictionaries (or rather nested dictionaries: a dictionary of dictionaries). The standard Python dictionary type therefore provides a good interface with JSON files. This is used by the JSON module built in to Python. See the example below on how to read JSON files. The reading is very simple: import json with open(”aircraftdata.json”) as f: datadict = json.load(f) print(list(datadict.keys())) The datadict dictionary contains three keys, which will be printed by the program above: ['__comment', 'A319', 'A320'] Equally simple as reading is the writing with the dump function from the json-module, e.g. with the gradebook dictionary from our previous example. Use the indent keyword gives us a readable lay-out, and avoids the file being one long line: import json with open(”grades.json”, ”w”) as g: json.dump(gradebook, g, indent=3) Which results in the following file grades.json:

136

15. Reading and writing data files: beyond the basics

{ ”John Cleese”: 9.3, ”Michael Palin”: 9.5, ”Terry Jones”: 5.7, ”Terry Gilliam”: 6.6, ”Eric Idle”: 7.2, ”Graham Chapman”: 6.3 }

15.4. Hierarchical data: XML files XML is a text-based file format, and falls in the category of mark-up languages (similar to HTML). It is used to represent the data and its structure. It can thus be used as a structured data file format similar to JSON. There is a standard xml module in Python which allows you to work dom and minidom and with tree data structures (etree), but this is beyond our scope here. In our case we will use dictionaries, much like the json module. For this we will use the module xmltodict: pip install xmltodict In XML format, the same data from the earlier JSON file would look like this:

Engine typer: TF, TP max fuel capacity (L) max landing weight (kg) max take-of weight (kg) aircraft full name operating empty weight (kg) wing area (m^2)

TF 30190 62500 75500 2 Airbus A-319 40800 122.4

TF 24210 66000 78000 2 Airbus A-320 42600 122.6

To read this with xmltodict is quite straightforward: use the parse() function from the module on

15.5. Binary data: MATLAB mat files

137

the text data read from the file: import xmltodict with open('aircraftdata.xml') as f: xmldata = f.read() acdata = xmltodict.parse(xmldata)[”root”] print(acdata) This program prints the following ordered dictionary: OrderedDict([('A319', OrderedDict([('engine_type', 'TF'), ('engines', ↪ OrderedDict([('element', ['CFM56-5B5', 'CFM56-5B6', 'CFM56-5B7', ↪ 'CFM56-5A4', 'CFM56-5A5', 'V2522-A5', 'V2524-A5', 'V2527M-A5'])])), ↪ ('mfc', '30190'), ('mlw', '62500'), ('mtow', '75500'), ↪ ('n_engines', '2'), ('name', 'Airbus A-319'), ('oew', '40800'), ↪ ('wa', '122.4')])), ('A320', OrderedDict([('engine_type', 'TF'), ↪ ('engines', OrderedDict([('element', ['CFM56-5B4', 'CFM56-5B5', ↪ 'CFM56-5B6', 'V2500-A1', 'V2527-A5'])])), ('mfc', '24210'), ('mlw', ↪ '66000'), ('mtow', '78000'), ('n_engines', '2'), ('name', 'Airbus ↪ A-320'), ('oew', '42600'), ('wa', '122.6')])), ('__comment', ↪ OrderedDict([('engine_type', 'Engine typer: TF, TP'), ('engines', ↪ '[list of engines]'), ('mfc', 'max fuel capacity (L)'), ('mlw', ↪ 'max landing weight (kg)'), ('mtow', 'max take-of weight (kg)'), ↪ ('name', 'aircraft full name'), ('oew', 'operating empty weight ↪ (kg)'), ('wa', 'wing area (m^2)')]))]) Extra: Ordered dictionaries As can be seen, this is an ordered dictionary, where the keys have been used to sort it. An OrderedDict is a special kind of dictionary provided by the built-in module collections. The original dict type is unordered, and iterating over it might give you an unexpected ordering. OrderedDict, on the other hand, remembers the order in which items were added to it, so when you iterate over an OrderedDict you will get its items in the same order. So for example: from collections import OrderedDict ages = OrderedDict() ages['Terry'] = 20 ages['Eric'] = 7 ages['John'] = 86 for name, age in ages.items(): print(name, 'is', age, 'years old') When executed, this will print: Terry is 20 years old Eric is 7 years old John is 86 years old

15.5. Binary data: MATLAB mat files Nowadays we use Python for many things for which we previously needed the expensive MATLAB package. Migrating between them has its share of difficulties, one of which relates to the proprietary,

138

15. Reading and writing data files: beyond the basics

binary format of datafiles in MATLAB. Luckily, SciPy’s io module provides the functionality to load and save MATLAB .mat files. The function to load .mat files is scipy.io.loadmat(). It can take several arguments, but the most important ones are the filename, and optionally the mdict keyword argument. When this argument is not passed, the data loaded from the .mat file are put into a new dictionary with the MATLAB variable names as keys, and the corresponding matrices as values. This dictionary is returned by the function. When the optional mdict argument is given, however, the data is appended to the indicated dictionary instead. For example, loading a .mat file containing a single array called ‘testdata’: from scipy.io import loadmat data = loadmat('test.mat') print(data['testdata']) When exectuted, this would load and print the testdata array: [1, 2, 3, 40, 50] Similarly, scipy.io.savemat() can be used to save data from Python to a MATLAB .mat file. Its arguments are similar to those of loadmat(), but now, mdict is not optional, but should contain a dictionary of all the data to be stored in the .mat file: from scipy.io import savemat data = { ”testdata”: [1, 2, 3, 4, 5], ”moredata”: [100, 200, 300] } savemat('output.mat', data)

15.6. Binary data: PDF files Manipulating PDF files is not always possible without a licensed, paid version of the Acrobat suite. But you can do this in Python too. Using the module PyPDF2 you can manipulate pdf files. You can use pip to install this module from the console or command prompt (run as administrator): pip install pypdf2 Two examples of its usage are given below. The first example combines several separate PDF files from the folder PDFin and puts the result in PDFcombined. ””” Combines all pdf-files in the PDFin folder, make sure sorting by name works out right into one PDF file ”merged.pdf” om the PDFout folder. ””” import os from PyPDF2 import PdfFileMerger, PdfFileReader files = sorted(os.listdir(”PDFin”)) merger = PdfFileMerger()

15.7. Binary data: Any binary file

139

for fname in files: if fname[-4:].lower() == ”.pdf”: with open(”PDFin/” + fname, ”b”) as g: print(”Adding”, fname) merger.append(PdfFileReader(g)) if len(files) > 0: print(”Writing ./PDFcombined/merged.pdf”) merger.write(”PDFcombined/merged.pdf”) print(”Ready.”) The next program rotates any pdf files in the PDFin folder clockwise and writes these to PDFrotated folder. ””” Rotates all pdf-files in the PDFin folder, make sure sorting by name works out right Output to pdfrotated folder ””” import os from PyPDF2 import PdfFileMerger, PdfFileReader, PdfFileWriter files = sorted(os.listdir(”PDFin”)) for fname in files: if fname[-4:].lower() == ”.pdf”: with open(”PDFin/” + fname, ”b”) as g: print(”Adding”, fname) reader = PdfFileReader(g) writer = PdfFileWriter() for pagenum in range(reader.numPages): page = reader.getPage(pagenum) page.rotateClockwise(90) writer.addPage(page) with open(”PDFrotated/” + fname, ”wb”) as h: writer.write(h) print(”Ready.”)

15.7. Binary data: Any binary file So far we have used a module or read a file as text file. But what if a file is not a text file and you also cannot find a module to read it? Well, as long as you know the structure of the file, you can always make a reader function yourself. For this we need to open the file in binary mode: f = open(”icon.bmp”, mode=”b”) This allows us to access and read the file byte by byte. Every byte is composed of 8 bits, so can take a value between 0-255. When an 4-byte integer format is used, we need to multiply the following three bytes by higher powers of 256. Sometimes the highest power (most significant byte) might be the first

140

15. Reading and writing data files: beyond the basics

in the file. When conversion function are used, this order needs to be specified. As an example we’ll look at the BMP format, a device independent image bitmap format. Looking at https://docs.fileformat.com/image/bmp/ we can find the following information. It starts off with a header of 14 bytes with some general information: Offset hex 00

Offset dec 0

Size

Purpose

2 bytes

02 06

2 6

4 bytes 2 bytes

08

8

2 bytes

0A

10

4 bytes

The header field used to identify the BMP and DIB file is 0x42 0x4D in hexadecimal, same as BM in ASCII. It can following possible values.* BM - Windows 3.1x, 95, NT, ... etc. * BA OS/2 struct bitmap array * CI - OS/2 struct color icon * CP OS/2 const color pointer * IC - OS/2 struct icon * PT - OS/2 pointer The size of the BMP file in bytes Reserved; actual value depends on the application that creates the image Reserved; actual value depends on the application that creates the image The offset, i.e. starting address, of the byte where the bitmap image data (pixel array) can be found.

Let’s use the following program to inspect it, byte by byte (so reading with size of 1). A single byte can be converted to an integer using the ord() function: with open(”icon.bmp”, mode=”b”) as f: for i in range(14): byte = f.read(1) print(”Byte”, i, ”:”, ord(byte), ” = ”, byte) This shows the header byte by byte: Byte Byte Byte Byte Byte Byte Byte Byte Byte Byte Byte Byte Byte Byte

0 : 66 = b'B' 1 : 77 = b'M' 2 : 50 = b'2' 3 : 6 = b'\x06' 4 : 0 = b'\x00' 5 : 0 = b'\x00' 6 : 0 = b'\x00' 7 : 0 = b'\x00' 8 : 0 = b'\x00' 9 : 0 = b'\x00' 10 : 50 = b'2' 11 : 2 = b'\x02' 12 : 0 = b'\x00' 13 : 0 = b'\x00'

Using the info from this table, we can see that this is a BMP file made on Windows. Then a 4 byte integer shows the size of the file, least significant byte first, so little value first: 50 + 6 ∗ 256 + 0 ∗ 256 ∗ 256 + 0 ∗ 256 ∗ 256 ∗ 256 = 1586 bytes We can use this in our BMP analyser: # Inspect the BMP file with open(”icon.bmp”, mode=”b”) as f: # Check type identifier:

15.7. Binary data: Any binary file

141

# convert 2 bytes to string with decode() method ident = f.read(2).decode() # Filesize: int*4 format, least signficant byte first (”little”) size_bytes = f.read(4) # is now of bytes type filesize = int.from_bytes(size_bytes, ”little”) #conversion to int, LSB first ↪ # Go to byte 10 (OA) skip = f.read(4) # Get start of pixel data offset_bytes = f.read(4) offset = int.from_bytes(offset_bytes, ”little”) print(”Filetype identifier:”, ident) print(”BMP file is:”, filesize) print(”Pixel data starts at:”, offset) This results in the following output: Filetype identifier: BM BMP file is: 1586 Pixel data starts at: 562 Opening files as binary files, whether for reading or for writing gives you full control of the meaning of every byte. Using the int methods int.from_bytes() and int.to_bytes(), you can convert back and forth from integers, as long as you specify the order of the bytes “little” or “big”, depending on whether the least significant bytes (LSB) first or the most significant (MSB) is first in the sequence of bytes.

16 Exception handling in Python A powerful feature of Python is that it allows you to trap runtime errors using the try and except statements. This functionality is very useful to catch and deal with unwanted cases (e.g., a user giving letters to the input() command where the code only expects numbers: int(input('Give a number: '))). You should be aware, however, that this is also a way to obscure errors, especially when you catch too general an exception (more about this later). This can quickly make your program impossible to debug. In general it is better to prevent errors than to catch them. Still, it can sometimes be a very useful feature, so we show a few examples here of how to use try and except. A try-except block is structured as follows:

Ê

try: # Block of code to test Ë ... Ì

Í

Î

except ExceptionType as e: # An error has occurred Ï print(e) The six elements in this block have the following meaning: 1. The statement keyword try: Indicates to Python that exceptions in the following indented code can be caught by the following except-block(s). 2. The indented block of code for which exceptions can be caught. 3. The statement keyword except: Indicates the start of a block of code to process one or more kinds of exceptions. 4. Optional: The exception type. When an exception type is specified, only exceptions of this type are caught by this except-block. 5. Optional: Variable name assignment for the caught exception. In this case, the exception object (which contains information about the error) can be used in the except block by the variable name e. 143

144

16. Exception handling in Python

6. The block of code to execute when an exception (of the specified type) is caught. When nothing needs to be done when catching an exception you can put the pass keyword here. Note that in the except block, it is not mandatory to provide a specific exception type; you can also just write except:, in which case all exceptions will be caught by this except block. The benefit of this is that you don’t need to know the type of errors that can occur in advance, the downside is that all errors in your code (aside from indentation errors and syntax errors) will be caught! This can make it difficult to find bugs in your program. Note also that you can add more than one except block; one for each type of exception. That way you can implement different kinds of responses for different kinds of errors in the same code. Finally, you can also catch more than one type of exception (instead of just one, or all) in one except block. You do this by providing the list of exception types you want to catch in a tuple: try: ... except (Type1, Type2) as e: ... Now for some examples. For these we revisit the 𝑎, 𝑏, 𝑐 formula solver, and make it check for problems in the equation coefficients by simply catching the error. Let’s first look at the original abc program: import math print(”To solve ax2 + bx + c = 0 ,”) a = float(input(”Enter the value of a:”)) b = float(input(”Enter the value of b:”)) c = float(input(”Enter the value of c:”)) D = b**2 - 4.0 * a * c x1 = (-b - math.sqrt(D)) / (2.0 * a) x2 = (-b + math.sqrt(D)) / (2.0 * a) print(”x1 =”, x1) print(”x2 =”, x2) When there are no solutions to the quadratic equation, 𝐷 will be negative, which leads to the following exception: >>> x1 = (-b - math.sqrt(D)) / (2.0 * a) Traceback (most recent call last): File ””, line 1 x1 = (-b - math.sqrt(D)) / (2.0 * a) ValueError: math domain error >>> It is also possible that the user has entered values for a first-order equation (𝑎 = 0). In this case there is a division by zero, leading to the following exception: >>> x1 = (-b - math.sqrt(D)) / (2.0 * a) Traceback (most recent call last): File ””, line 1 x1 = (-b - math.sqrt(D)) / (2.0 * a)

145

ZeroDivisionError: division by zero >>> The easiest way to avoid your program stopping with an error is to use a general try-except block: import math print(”To solve ax2 + bx + c = 0 ,”) a = float(input(”Enter the value of a:”)) b = float(input(”Enter the value of b:”)) c = float(input(”Enter the value of c:”)) D = b**2 - 4.0 * a * c try: x1 = (-b - math.sqrt(D)) / (2.0 * a) x2 = (-b + math.sqrt(D)) / (2.0 * a) print(”x1 =”, x1) print(”x2 =”, x2) except: print(”There was a problem with the specified polynomial values!”) This is easy to use, but it doesn’t allow you to make a distinction between the different kind of problems that can occur, and as described before, it can also hide other errors in your code. You have more control and knowledge over the caught exceptions when you explicitly specify the type of exception to catch. In this case there are two types of exception we are interested in: when a is zero, we get a ZeroDivisionError exception, and when D is negative we get a ValueError. If we catch these exceptions separately we can do the following: import math print(”To solve ax2 + bx + c = 0 ,”) a = float(input(”Enter the value of a:”)) b = float(input(”Enter the value of b:”)) c = float(input(”Enter the value of c:”)) D = b**2 - 4.0 * a * c try: x1 = (-b - math.sqrt(D)) / (2.0 * a) x2 = (-b + math.sqrt(D)) / (2.0 * a) print(”x1 =”, x1) print(”x2 =”, x2) except ValueError: print(”This polynomial has no roots”) except ZeroDivisionError: print(”This is a first order equation: a=0”) try: x = -c / b print(”Solution x =”, x) except: print(”No x found.”) There are several types of exceptions. Some examples:

146

16. Exception handling in Python ZeroDivisionError IndexError SyntaxError KeyError

OverflowError TypeError SystemExit ValueError

NameError KeyboardInterrupt FloatingPointError ...

There is also the basic exception type Exception: Catching this exception type in an except block catches all exceptions. When an error occurs, the name of the error is always given in the shell. A complete list can also be found in the Python (the documentation provided with Python) under the header Exceptions.

17

∗

Creating self-contained bundles of your program using PyInstaller Many students have asked me how they can share their Python programs with people who haven’t got Python installed on their computer, for example by converting your Python program into an executable. There is a tool for this, called PyInstaller. PyInstaller freezes Python programs into standalone applications, which don’t require an installation of Python to be present on the computers where the applications are used. PyInstaller can make such bundles for Windows, macOS, and Linux computers.

17.1. Making an executable of your program Before you can start making your own bundled application you need to install PyInstaller. You can do this using pip: pip install pyinstaller Note that macOS and Linux users might have to put sudo in front of pip. The next step is to go to your project folder with the terminal / command prompt with the cd command (in Windows you can also do this by right-clicking on your project folder in the windows explorer, while holding the right shift button). You can then convert your program by pointing PyInstaller at all the python files of your project you wish to include: pyinstaller myprogram.py myfunc.py tools.py This will create an executable called myprogram.exe (.exe on windows, .app on macOS, no extension on Linux) in a subfolder dist, accompanied by all the necessary files to make it run without Python. The folder and executable will be named after the the first script (myprogam) and this script will be started when the executable is started. Manually you need to add any data files or data folders which you want included because the program needs those.

17.1.1. Example Mazeman game Mazeman is a game written in Python using the third-party module Pygame and the Tkinter module, which is included in Python. It is a variation on Pac-man, but it allows the user to edit simple text files containing the maze lay-out including an unlimited number of ghosts (and a player starting position). Many creative mazes are possible, just right-click Edit in the File Open menu to create and save your own. The program reads the file the user selects and starts the game with the maze as specified in this file. The program scales all graphics to a window size, which covers max 80% of the screen. The sprites are non-animated, continuous movement. The game has two python files: mazeman.py and toolsmaze.py, the latter of which all functions are imported in mazeman.py. The are three subfolders with data files named bitmaps, sounds and choose-your-maze. The program uses fonts to display the score text on the top line. The main module, with which the program is started, is called mazeman.py. So we convert it using: 147

148

17. Creating self-contained bundles of your program using PyInstaller pyinstaller mazeman.py toolsmaze.py

If it runs successfully, it will create a subfolder called dist\mazeman containing your program (mazeman.exe in our example) and a lot of other files needed by the executable. Still we also need to add some files manually before we’re ready.

17.1.2. Adding additional files to make your program run First of all we need to copy our data files to this dist folder, so our mazeman.exe will find them. In our example, the three subfolders named: bitmaps, sounds and choose-your-maze. To see whether our program needs any additional files, it may be useful to start our program mazeman.exe once, so we can read warnings and error messages in the console. In Windows it’s helps to create a batch file for this with a pause command. Let’s call this runme.bat for example (in the dist folder as well). Create it with notepad or idle): mazeman.exe pause Then run it to test it and see if you need to add any other files. Sometimes you may need other DLLs from third party module you use.

17.2. Making a setup installer for your program To distribute your program you only need to archive your folder e.g to a zip file. You can rename the dist subfolder to mazeman and then create the zip file mazeman.zip, which includes all files and subfolder. In our example about 10 Mb. The user can then unzip this in the Program Files folder and add a shortcut to mazeman.exe on the desktop or on the Start Menu as desired. There are also more fancy ways to distribute your program and create an installer, which will do these same things for you and create an executable for installation. Just download such a program e.g. from download.com. Some are free like NSIS or Inno Setup (last one is recommended). With Inno Setup, you need to select the dist folder as root and add this (with all files and subfolders). Also you need to indicate which executable is the program to start and perhaps allows an entry in the Start menu. Inno set-up then compiles everything into one neat setup program as a executable e.g. setupmazeman.exe. This is often quite large. Many mail programs or mail servers will prevent sending executables, because that’s the way viruses are distributed. So to mail it, you may need to convert this one file to a compressed folder/ archive (ZIP or ARJ extension).

∗

18

Version Control At some point in the development of software, especially when collaborating in a team, you will find yourself handling many versions of the same program. You will copy files back and forth, zip up the state of a project and label it with date and time, etc. Anyone who has worked like this for a while will have found out that this is an error-prone process, important fixes might get lost, changes inadvertently overwritten, collaborators work using older versions of files, and at some point it becomes almost impossible to combine all code contributions into a final, correct version. This is such a common problem in software development that people have created Version Control Systems (VCS) for this. With a VCS you can automate the process of keeping different versions of your software project, and the VCS will help you on many tasks, like find differences between versions, revert to an older version if you accidentally botched things up, up to incorporating automatic testing of the software in the project. Although these systems are not specific to Python, we decided to add this section on VCS after hearing from 2nd and 3rd year students, who eventually picked up using VCS at some point, that they would really have liked to start with VCS earlier, and that they would have appreciated an introduction and some pointers on where to learn more. Many VCS exist now, we will discuss two commonly used ones; subversion and git.

18.1. Basic concepts of version control 18.1.1. Repository, working copy VCS work with “repositories”, which can be considered as master copies of your software. This repository may be locally on the computer where you are working, but often it is installed remotely on a server. When you want to work on your project, you need a local check-out or working copy of the software. Using a “check-out”, you can get a copy of the latest files from the repository (if you wish, you can also get an older version from the repository). After creating changes to that copy – and of course you should have tested these changes too! – you can do a “commit”, and the repository is updated with your latest software.

18.1.2. Revision, commit, check-out A VCS only tracks the software when you indicate you have a new version, through a “commit” action. This creates what is commonly termed as a revision, a version of your software known to the VCS. It is customary to also add a commit message to this revision, so that you can later trace back what you worked on, and why you made specific changes. A VCS also maintains revision numbers (or “hashes”, large strings, in case of some VCS, e.g., git), that uniquely identify the revisions. You can check out any of the revisions later with a checkout command. Modern VCS use revisions as labels defining the state of the entire repository. Each change to the repository (by a commit, or a branch, see later), creates a new revision number or hash, keeping the history of the repository traceable, and enabling you to go back to and compare all the old versions of the software. 149

150

18. Version Control

18.1.3. Comparing source code Some of the precursor tools for version control are diff and patch. The basics of these tools are still used in modern VCS. diff compares two text files and determines the difference between these two, outputting this difference again as text file, in a compact fashion. To make this output human-readable, some “context” can be added, here is a small example of some of this output: Version A

Version B

Comparing source code

Comparing source code

The oldest tools for version control are diff and patch. The basics of

The oldest tools for version control are patch and diff. The basics of

these tools are still used in modern ↪ VCS.

these tools are still used in modern ↪ VCS.

The output of calling diff text1.txt text2.txt from the command prompt 4c4 < are diff and patch. The basics of --> are patch and diff. The basics of This output is commonly called a diff or a patch. From a diff, you can quickly see what changes were applied. patch is a program that is the companion to diff, it can create the newer file, based on the old file and the output from diff (it can also work in the other direction, creating the old file from the new file and patch). VCS work on this basis. Each time you tell the VCS that there is a new version of your project, the VCS compares the new version to the older version it already has, then determines and stores the difference between the existing version and the new one. In that way the VCS keeps the complete history of versions and changes. Asking the VCS for a diff between two revisions of your software can be a quick way of identifying where changes were made. To an unsuspecting user, it may seem like the version control system simply has stored every copy ever committed. However, using a diff, rather than simply storing the latest version and labeling it, has a side effect. A diff will fail if the file in the working copy does not match the file in the repository, because, when you perform a commit, the VCS determines how your files have changed using a diff, and tries to implement that diff on the code in the repository. As an example, if you make a checkout to a file with following line: # A program by Lisa Then you are going to collaborate with Max, who also checks out a copy of the software from the repository, and corrects to: # A program by Max and Lisa Max quickly checks this in by doing a commit, so now the repository has Max’s updated version. In the meantime you recognise Max’s contribution, and change the line to: # A program by Lisa and Max Now, if you commit your version, the VCS tries to get the latest, final version of the software in the repository by applying a patch from your old version (# a program by Lisa), to your new version (# a program by Lisa and Max), to the version that is currently in the repository. However, that will fail with a “conflict”, because the repository version has changed at that specific location. You have to then manually fix the file (the VCS helps you there by marking the conflicts), and tell the VCS that you resolved the problem. If you work together in a reasonable fashion (typically by not all working on the same code at the same time), conflicts can be avoided, and if they occur, it is usually easy

18.1. Basic concepts of version control

151

enough to resolve them. This side effect may seem to be a drawback of using version control systems over copying files, but there is actually a significant advantage; it prevents you from silently overwriting previous work. On Windows, a tools called Winmerge can be used to compare source in a GUI and merge different versions. Many similar tools exist on other platforms.

18.1.4. Distributed vs centralised VCS The “simplest” model of version control is centralised version control. The VCS “subversion” (or SVN) uses this model. There is one central software repository, usually on an internet server. When doing a check-out, the latest version of the software (or a version defined by means of a version tag supplied in the check-out) is copied to the user’s local folders, and when doing a commit, the changes are pushed back to that single repository.

repository

e up da t

m m co

it m

working

copy

m co

Local pc 1

te da up

it

Server

Local pc 2

working

copy

Centralised version control

The alternative is a distributed or de-centralised VCS, of which “git” is a prime example. When starting to work on the code, you must “clone” a git repository from the web. The clone copies the entire repository, including information about its history, to the local computer. In a second step you can check out any revision from the now local copy of this repository; by default the VCS checks out the latest development version. Then any commit will be done to the local clone of the repository, creating a new revision , and you will not need internet access for that. In a second step these changes can be copied over to the remote repository, with a “push”. To get changes that may have been added by others to the remote repository, you need a “pull” command, which gets all the revisions added to the remote repository from the time you cloned it, or did your last pull. The local copy you are working on is linked to one particular remote repository, but it is also possible to link multiple remote repositories to the local development copy, and switch between these. We will not further explore this here, interested readers are referred to https://git-scm.com/book/en/v2/Git-Basics-Working-with-Remotes. When you are working with only one remote repository, there is conceptually not much difference between using a centralised or a distributed version control system; the only steps you should not forget are the pull and push steps, to synchronise the local clone with the remote repository.

152

18. Version Control

repository

Server

sh

pu

ll

pu

ll

sh

pu pu

Local

repository

repository clone

Local

repository

repository clone

commit

commit update

update

working

copy

working

copy

Local pc 1

Local pc 2

Distributed version control

18.1.5. Branches and merging The version control facilities we discussed here show you how to use a VCS as a fancy back-up facility, remembering all previous versions and preventing you from accidentally overwriting your (or your team’s) precious work. When the software you are working on is being used by yourself or others, and you need to improve or expand it further, two other concepts, branching and merging, become very important. commits on main branch Main branch (svn: trunk, git: master) merge branch back to main

create development branch

Additional (development) branch commits on development branch Illustration of branching and merging, with a case where a development branch is split off from the main branch, and fixes and development occur in parallel. With branching, you can split off a version of the software (i.e., as a complete set of files and folders) that can from then on be independently developed from the main “branch”. To illustrate this, let’s take the scenario where your teammates are already running low-speed analysis of the flight performance using the initial version of the software you developed in your (3𝑟𝑑 year) DSE project, and you now want to add compressibility effects to properly calculate cruise performance. By branching off, you will be able to simultaneously have and maintain two versions of the program. You can work on the new version without risking breaking the version that the others are using. When the work on the branch is complete, the two versions can be merged again, applying all development from the development branch to the main version in one step. For this, the VCS can apply all the commits you added to the

18.2. Overview of Git commands

153

new branch to the old branch that people have been using. Also if the main branch has been moving on, for example when bugs had to be fixed there, the merge will combine the changes on both branches, keeping both the addition of new capability and the bug fixes.

18.2. Overview of Git commands For most purposes you can stick with using a graphical tool like Github desktop, or the git integration in your IDE, to do your git interactions. But sometimes, for the more exotic interactions with git you might want to use the command-line tool. Below is an overview of some of its commands: • Clone (check out to your own computer) the latest version of a project git clone https://myserver.nl/GIT/MyProject.git MyProject This clones the repository abd checks out the “master” branch into the local folder “MyProject”. • Update local repository to match remote one git pull Run this from the working copy folder. • Commit your changes to the local repository git commit Note that this commits to the local repository, see “git push” to transfer these commits to the remote repository. • Create a new local repository git init Run this from within the folder where you want the repository. • Copy the local repository changes over to the remote repository git push • Create a branch, based on the working copy folder git branch • Check which branches there are: git branch -av • Start work on another branch: git checkout Note that there may not be uncommitted changes in the working directory. • Merge branch into the current working directory git merge • Add a file for tracking in the repository git add • Determine the difference between the current working copy and a repository commit git diff • See the history git log

154

18. Version Control

18.3. Overview of Subversion commands While with git, branches are switched between with the git command-line tool, or in any of the git gui applications, if you use subversion, branches will look like folders on a file system. The typical layout of a subversion repository uses “trunk” as the main development version, and the “folders” branches and tags. The folder “branches” is for branched off (development) versions, and “tags” is typically used to remember released versions of the software. Here is an example of a repository layout: MyProject/trunk MyProject/tags/v0.9 MyProject/tags/v1.0 MyProject/branches/add-mach-effects At each branch (trunk, v0.9, etc.) the subversion repository show all the files and folders in your project. Superficially, it looks as if everything is copied, internally, subversion maintains an inventory of differences and changes in the files. On Windows TortoiseSVN is an svn-program which works as a Windows shell extension: right clicking any folder allows you to sync it with a repository. Then simply right clicking a file or folder an selecting commit or update will do the same as above. The following has a summary of the most-used actions in subversion. • Check out the latest version of your software svn co https://myserver.nl/SVN/MyProject/trunk newfolder Assumes standard repository layout, checks out project into new folder “newfolder”. • Update working copy to include changes committed to the repository svn update Run this from the working copy folder! • Commit changes to repository svn commit Run from working copy folder. • Create a branch svn cp . https://myserver.nl/SVN/MyProject/branches/newbranch Assumes you run from working copy (indicated by “.”). • Merge a branch svn merge -r4:6 https://myserver.nl/SVN/MyProject/branches/abranch Merges the commits (i.e. changes) from revision 4 to revision 6 on branch abranch, into the current checked out folder. • Revert changes to match repository (e.g., after goofing up) svn revert • Resolve a conflict svn resolved After fixing a conflict (marked by ««« markers in the file), subversion needs to be told you have fixed this. • Add a new file or folder to the version control svn add

18.3. Overview of Subversion commands

155

Do not forget this, subversion does not automatically recognise or add new files. • Log the commit messages on a working copy svn log . or svn log . –stop-on-copy the –stop-on-copy flag is useful to figure out when you started a branch. You will need to use that revision number for a merge (see the merge above). • Check for changes with a previous copy svn diff -r • Check which repository is connected to your working folder: svn info Note that these git and subversion command summaries are by no means complete, they represent a minimum set of commands to handle the most common cases. There are also many GUI tools to help you navigate the git or subversion commands, check out the pages at: https://git-scm.com/downloads/guis https://en.wikipedia.org/wiki/Comparison_of_Subversion_clients

∗

19

Beyond the course: Exploring applications of Python The purpose of this course is to get you started with writing programs in Python, to explain its basics, and the basics of scientific computing in Python, and to give you a brief overview of what kinds of things are possible in Python. Here, many of the things we show you are just examples, with plenty of alternatives out there as well. A good example of this is Pygame; we introduce you to Pygame in this course because it is easy to get started with, and allows you to very quickly get visual (and visually pleasing) output of your work. But there are of course many alternatives! Below are just some examples.

19.1. Alternatives for Pygame for (2D) visualisation There are really tons of alternatives to Pygame out there. Sometimes it makes more sense to just stick with Pygame, but on other occasions an alternative may offer a significant benefit over Pygame. Below are just a few examples, there are many more! Also note that gui libraries (see chapter 14) often also have their own (vector) drawing implementation, e.g., the Cairo library in Gtk, but also Qt and WxWidgets have similar functionality.

19.1.1. TkInter Canvas (included in Python) Tkinter is primarily meant to design dialog boxes, but it also has the so-called canvas object to draw custom shapes. See an example below which demonstrates its ease of use: from tkinter import * master = Tk() w = Canvas(master, width=200, height=100) w.pack() w.create_line(0, 0, 200, 100) w.create_line(0, 100, 200, 0, fill=”red”, dash=(4, 4)) w.create_rectangle(50, 25, 150, 75, fill=”blue”) mainloop() Alternatively, even LOGO like graphics have been included in Python using the turtle module. Not te be used for serious applications but it’s really a lot of fun, like a nice, spirograph-like toy and also a very good way to teach younger people how to program! from turtle import * setup (width=200, height=200, startx=0, starty=0) 157

158

19. Beyond the course: Exploring applications of Python

speed (”fastest”) # important! turtle is intolerably slow otherwise tracer (False) # This too: rendering the 'turtle' wastes time for i in range(200): forward(i) right(90.5) done()

19.1.2. Arcade The Arcade library provides much of the same functionality as Pygame, and is specifically aimed at making games, for which it also has specific functionality to make your life easier. Have a look at https://arcade.academy/ for more information. Can easily be installed with pip: pip install arcade. An example: # Import the Arcade library. If this fails, then try following the ↪ instructions # for how to install arcade: # https://arcade.academy/installation.html import arcade import os # Set the working directory (where we expect to find files) to the same # directory this .py file is in. You can leave this out of your own # code, but it is needed to easily run the examples using ”python -m” # as mentioned at the top of this program. file_path = os.path.dirname(os.path.abspath(__file__)) os.chdir(file_path) # Open the window. Set the window title and dimensions (width and ↪ height) arcade.open_window(600, 600, ”Drawing Primitives Example”) # Set the background color to white # For a list of named colors see # http://arcade.academy/arcade.color.html # Colors can also be specified in (red, green, blue) format and # (red, green, blue, alpha) format. arcade.set_background_color(arcade.color.WHITE) # Start the render process. This must be done before any drawing ↪ commands. arcade.start_render() # Draw a grid # Draw vertical lines every 120 pixels for x in range(0, 601, 120): arcade.draw_line(x, 0, x, 600, arcade.color.BLACK, 2) # Draw horizontal lines every 200 pixels for y in range(0, 601, 200): arcade.draw_line(0, y, 800, y, arcade.color.BLACK, 2) # Draw a point

19.1. Alternatives for Pygame for (2D) visualisation

159

arcade.draw_text(”draw_point”, 3, 405, arcade.color.BLACK, 12) arcade.draw_point(60, 495, arcade.color.RED, 10) # Draw a set of points arcade.draw_text(”draw_points”, 123, 405, arcade.color.BLACK, 12) point_list = ((165, 495), (165, 480), (165, 465), (195, 495), (195, 480), (195, 465)) arcade.draw_points(point_list, arcade.color.ZAFFRE, 10) # Draw a line arcade.draw_text(”draw_line”, 243, 405, arcade.color.BLACK, 12) arcade.draw_line(270, 495, 300, 450, arcade.color.WOOD_BROWN, 3) # Draw a set of lines arcade.draw_text(”draw_lines”, 363, 405, arcade.color.BLACK, 12) point_list = ((390, 450), (450, 450), (390, 480), (450, 480), (390, 510), (450, 510) ) arcade.draw_lines(point_list, arcade.color.BLUE, 3) # Draw a line strip arcade.draw_text(”draw_line_strip”, 483, 405, arcade.color.BLACK, 12) point_list = ((510, 450), (570, 450), (510, 480), (570, 480), (510, 510), (570, 510) ) arcade.draw_line_strip(point_list, arcade.color.TROPICAL_RAIN_FOREST, ↪ 3) # Draw a polygon arcade.draw_text(”draw_polygon_outline”, 3, 207, arcade.color.BLACK, 9) point_list = ((30, 240), (45, 240), (60, 255), (60, 285), (45, 300), (30, 300)) arcade.draw_polygon_outline(point_list, arcade.color.SPANISH_VIOLET, 3) # Draw a filled in polygon arcade.draw_text(”draw_polygon_filled”, 123, 207, arcade.color.BLACK, ↪ 9) point_list = ((150, 240), (165, 240), (180, 255),

160

19. Beyond the course: Exploring applications of Python

(180, 285), (165, 300), (150, 300)) arcade.draw_polygon_filled(point_list, arcade.color.SPANISH_VIOLET) # Draw an outline of a circle arcade.draw_text(”draw_circle_outline”, 243, 207, arcade.color.BLACK, ↪ 10) arcade.draw_circle_outline(300, 285, 18, arcade.color.WISTERIA, 3) # Draw a filled in circle arcade.draw_text(”draw_circle_filled”, 363, 207, arcade.color.BLACK, 10) ↪ arcade.draw_circle_filled(420, 285, 18, arcade.color.GREEN) # Draw an ellipse outline, and another one rotated arcade.draw_text(”draw_ellipse_outline”, 483, 207, arcade.color.BLACK, ↪ 10) arcade.draw_ellipse_outline(540, 273, 15, 36, arcade.color.AMBER, 3) arcade.draw_ellipse_outline(540, 336, 15, 36, arcade.color.BLACK_BEAN, 3, 45) # Draw a filled ellipse, and another one rotated arcade.draw_text(”draw_ellipse_filled”, 3, 3, arcade.color.BLACK, 10) arcade.draw_ellipse_filled(60, 81, 15, 36, arcade.color.AMBER) arcade.draw_ellipse_filled(60, 144, 15, 36, arcade.color.BLACK_BEAN, 45) # Draw an arc, and another one rotated arcade.draw_text(”draw_arc/filled_arc”, 123, 3, arcade.color.BLACK, 10) arcade.draw_arc_outline(150, 81, 15, 36, arcade.color.BRIGHT_MAROON, 90, 360) arcade.draw_arc_filled(150, 144, 15, 36, arcade.color.BOTTLE_GREEN, 90, 360, 45) # Draw an rectangle outline arcade.draw_text(”draw_rect”, 243, 3, arcade.color.BLACK, 10) arcade.draw_rectangle_outline(295, 100, 45, 65, arcade.color.BRITISH_RACING_GREEN) arcade.draw_rectangle_outline(295, 160, 20, 45, arcade.color.BRITISH_RACING_GREEN, 3, 45) # Draw a filled in rectangle arcade.draw_text(”draw_filled_rect”, 363, 3, arcade.color.BLACK, 10) arcade.draw_rectangle_filled(420, 100, 45, 65, arcade.color.BLUSH) arcade.draw_rectangle_filled(420, 160, 20, 40, arcade.color.BLUSH, 45) # Load and draw an image to the screen # Image from kenney.nl asset pack #1 arcade.draw_text(”draw_bitmap”, 483, 3, arcade.color.BLACK, 12) texture = arcade.load_texture(”images/playerShip1_orange.png”) scale = 0.6 arcade.draw_texture_rectangle(540, 120, scale * texture.width, scale * texture.height, texture, 0) arcade.draw_texture_rectangle(540, 60, scale * texture.width,

19.2. Alternatives for 3D visualisation

161

scale * texture.height, texture, 45) # Finish the render. # Nothing will be drawn without this. # Must happen after all draw commands (comparable with ↪ pygame.display.flip() ) arcade.finish_render() # Keep the window up until someone closes it. arcade.run()

19.2. Alternatives for 3D visualisation Next to dominating the world of scientific computing and web development, Python is also very large in the professional computer graphics world. Therefore there are many modules to edit images, movies, 3D scenes and sound files.

19.2.1. Visual Python: easy 3D graphics There are a few options to generate moving 3D graphics, depending on to which level you want to take it. Simple, basic 3D shapes can be programmed using VPython (https://www.vpython.org/ where you can also find the documentation). This builds on Pygame and OpenGL (See further below). It however provides a ‘layer’ over OpenGL to allow even less experienced programmers to build fancy 3D worlds in a relatively simple way. Two starter examples of the code, from the excellent VPython tutorial, are given below, next to their resulting graphical output window. In this window, you can rotate the camera with your mouse when you hold the right mouse button down and you can zoom in/out with both mouse buttons pressed. For instance these two lines will already result in the window below to be shown: from visual import * sphere()

Objects can be added a simple way (see the axes below to understand the positioning): from visual import * ball = sphere(pos=(-5, 0, 0), radius=0.5, color=color.cyan) wallR = box(pos=(6, 0, 0), size=(0.2, 12, 12), color=color.green)

162

19. Beyond the course: Exploring applications of Python

y

x z

19.2.2. Panda3D Panda 3D is a very complete 3D graphics and game engine developed originally and hosted by Carnegie Mellon. This includes very advanced graphics functions. It is compatible with the PyODE physics engine. Check out their website for some impressive demos: https://www.panda3d.org and for documentation and download. Panda3D is Open Source and free for any purpose.

19.2.3. OpenGL OpenGL has been the standard for 3D graphics for a long time and is at the lowest layer of most other 3D packages. For the Microsoft gamers: it is a sort of cross-platform DirectX. You can then directly control the hardware of OpenGl compatible graphics cards. Only to be used by the more daring, demanding and experienced programmer, it provides the possibility to call OpenGL directly from Python with the PyOpenGL module. PyOpenGL is compatible with Pygame: so you can use it in a pygame window (and it often is). PyOpenGL is very fast and interoperable with a large number of external GUI libraries for Python including wxPython, PyGTK, and Qt. It can also use the GLUT library to provide basic windowing and user interface mechanisms (pulldown menus). As using OpenGL requires many parameters to be set, example source code tends to be a bit too long to included here as an example. But check out for example: https://www.willmcgugan.com/blog/tech/ 2007/6/4/opengl-sample-code-for-pygame/ Or google some examples yourself. The PyOpenGL website has documentation at https: //pyopengl.sourceforge.net/documentation/, which will help you to understand the code. You can of course always use some samples as a starting point for your own applications. Several students have explored this option and used it successfully to make a 3D solar system visualization, a 3D asteroids game and even a Moonlander simulation on a hilly moon surface. Working with OpenGL is quite complex, so it can be hard, but it is also very powerful.

19.3. Animating your graphics: Physics To add even more realism, you can also use the PyODE physics engine as your development environment (http://pyode.sourceforge.net/). It provides a simulation of the physics of an extremely rich physical environment. It takes care of realistic mechanics and dynamics of your scenery including gravity, friction, inertia, collisions, etc.. PyODE can be used for simulation, visualisation or gaming.

19.4. Creating graphics

163

Relatively easy to use, but realise that the closer you get to reality with your simulation, the more complex your model and your program will become: you’ll need to set a lot of parameters, although many defaults are supplied. To get an impression of what it can do check put youtube videos like ‘ragdoll demo python ODE’.

See also the ragdoll demo of PyODE: https://youtu.be/rBolkg1bq4k

19.4. Creating graphics If you want to create (or just edit) your own graphics you will need editors for that. There are plenty of commercial products out there, but some good ones are also free!

19.4.1. GIMP and its Python console Gimp is a very good example of an extensive open-source image editor. And it even has a Python console built-in! This allows you to do batch operations in GIMP. You can find it in the pull down menu “Filters” > “Python FU”. To get an impression of the code an example is given below:

g = gimp.pdb images = gimp.image_list() my_image = images[0] layers = my_image.layers w = g.gimp_image_width(my_image) h = g.gimp_image_height(my_image) print(”Image Resolution: w=%d,h=%d”%(w,h)) new_layer = g.gimp_layer_new( my_image, w, h, \ RGBA_IMAGE, ”LeopardLayer”, 100, NORMAL_MODE) my_image.add_layer( new_layer ) g.gimp_context_set_pattern(”Leopard”) g.gimp_edit_fill(new_layer, PATTERN_FILL) g.gimp_layer_set_opacity(new_layer, 20) g.gimp_layer_set_mode(new_layer, SCREEN_MODE)

Instruction and a tutorial for Python-FU can be found at : https://www.gimp.org/docs/python/.

164

19. Beyond the course: Exploring applications of Python

19.4.2. Blender No matter which 3D library you use, you will need to create 3D objects with complex shapes, surfaces and textures. For this most Python programmers (and many others) use the freeware program Blender (https://www.blender.org). Blender a a very advanced, yet easy to use, 3D content creation suite. You can import objects made in Blender in Python using OpenGL. Even without Python, Blender can make interactive animations. Make sure to visit the website to check out the beautiful gallery. Blender is used a lot by professionals in the Computer graphics world (for commercials, movies and games). Also check out Youtube for some impressive Blender examples. Blender also has a powerful game engine.

19.5. Interfacing with hardware

165

19.5. Interfacing with hardware 19.5.1. The Raspberry Pi

The Raspberry Pi (https://www.raspberrypi.org/) is a basic (but complete), low cost, very small (credit card size) Linux computer which comes with Python installed and it also runs pygame. For around 50 euros you have a complete computer with the basic accessories like power supply, OS on SD-card, etc. You can connect it to any screen via HDMI. Because it is small and consumes not much power, it is used for many purposes like internet of things, mobile webcam, robotics, education projects, programming/gaming console, webserver, twitter-monitoring and other desktop applications. Some specifications: • CPU: 1.2 GHZ quad-core ARM Cortex A53 (ARMv8 Instruction Set) • GPU: Broadcom VideoCore IV @ 400 MHz • Memory: 1 GB LPDDR2-900 SDRAM • USB ports: 4 • Network: 10/100 MBPS Ethernet, 802.11n Wireless LAN, Bluetooth 4.0 And IDLE runs on the Raspberry and it is provided with it.

19.5.2. MicroPython A relatively new initiative (from MIT) is called MicroPython. This is a special version of Python 3 for a microcontroller. The Micro Python board, called pyboard, is shown below.

By simply copying the text files with your Python code from your PC onto the microboard, you can run the programs. To get an impression of the code, there are several examples on the website. For example, controlling LEDs:

166

led1 = pyb.Led(1) led2 = pyb.Led(2) while True: mma = pyb.mma()[0] if mma < -10: led1.on() led2.off() elif mma > 10: led1.off() led2.on() else: led1.off() led2.off() pyb.delay(20)

19. Beyond the course: Exploring applications of Python

# # # # # # # # # # # # # #

Get LED 1 object Get LED 2 object loop forever get the x-axis angle if angled enough one way turn led 1 on turn led 2 off if angled the other way turn led 1 off turn led 2 on if flat: turn led 1 off turn led 2 off wait 20 milliseconds

More information can be found on the website https://micropython.org/.

20 Examples 20.1. Logicals and loops: Bubblesort In the first example in this chapter we’ll investigate Bubblesort; a classical algorithm that relies on several logical statements and loops to implement a surprisingly efficient sorting algorithm. To set up our program, our first task is to generate a collection of random numbers under 100, which we can then sort with our Bubblesort function. We’ll use randint (see section 5.3) to generate such a list: from random import randint # Make an unsorted list numbers = [] for i in range(50): numbers.append(randint(1, 100)) print(”Before sort:”, numbers) When run, this code will print an unsorted array of 50 random integers under 100. Now we will sort it with the famous Bubblesort algorithm. The logic of this algorithm is based on three steps: • iteratively check all consecutive pairs in the list • if a pair is in the wrong order swap the places of the elements • keep checking all pairs until no swap is needed anymore This logic is implemented in the following code: The Bubblesort algorithm # Make sure we enter while loop first time swapped = True while swapped: # Now assume everything is done unless proven otherwise swapped = False # Check all pairs by running index to one but last for i in range(len(numbers)-1): # If they are in the wrong order: swap 167

168

20. Examples

if numbers[i] > numbers[i+1]: a = numbers[i] numbers[i] = numbers[i+1] numbers[i+1] = a # and set switch indicating that we swapped swapped = True # The loop ends when swapped is False and no swaps were performed ↪ anymore print() print(”After sort:”) print(numbers) Let’s investigate this code: The key line is this: if numbers[i] > numbers[i+1]: a = numbers[i] numbers[i] = numbers[i+1] numbers[i+1] = a First we check whether these elements are in the wrong order. When they are the same we should not swap them (it would keep doing this forever then). If wrong, we save the value of numbers[i] in the variable a, then we overwrite this with the value of numbers[i+1]. And then we use our saved value in a to put in numbers[i+1] resulting in a swap of places. For this to work we need to let i run from 0 till the last place minus 1. Otherwise numbers[i+1] would result in an error for the last step in the loop. The for-loop therefore reads: for i in range(len(numbers)-1): Now all we need to do is add some logic to detect whether we are done. We can use a logical swap to detect any swapping. For this we first set it to False outside the for-loop and as soon as we swap two variables we set it to True. Then we add a while-loop around this which will keep repeating our for-loop until swap remains False and hence no swap was performed. To make sure we enter the while-loop the first time, we have to initialise our logical to True outside the while-loop. This is the order in which this program was made: from the inside out. First putting in elementary operations and then using a logical to add some logic. The end result is an already complex looking construction. Look at the three indent levels: while, for and if are inside each other or nested as we call this in programming.

A Answers to exercises Exercise 1.4.1 For our savings/debt calculator we need three inputs from our user: the starting sum, the number of years to calculate the interest for, and the interest rate. We can use the input command for this. Because input returns a string variable, we need to convert the user input to numbers before we can use them: startsum = float(input(”What is your starting amount to save?”)) nyears = int(input(”How many years do you want to save your money?”)) interest = float(input(”What is the yearly interest rate (in ↪ percent)?”)) With the three numbers known we can calculate the resulting sum, and print it as a result back to the user: endsum = startsum * (1.0 + interest / 100.0) ** nyears print(endsum) Exercise 1.4.2 In this exercise we also need three inputs from our user, which we will convert to floating-point numbers before using them to calculate the total resistance: Ra = float(input(”What is the resistance of resistor A?”)) Rb = float(input(”What is the resistance of resistor B?”)) Rc = float(input(”What is the resistance of resistor C?”)) Rab = (1.0 / Ra + 1.0 / Rb) ** -1 Rabc = Rab + Rc print(Rabc) Exercise 2.9.1 As we can see in the question, we need to ask our user three numbers: the interest rate, the start sum, and the number of years to calculate the interest for. The number of years can be interpreted as an int (let’s assume only whole years), the other two values could be floating point values: v_start = float(input(”Enter the start amount: ”)) interest = float(input(”Enter the interest rate in percent: ”)) 169

170

A. Answers to exercises

nyears = int(input(”For how many years do you want to calculate the ↪ interest? ”)) We can use the interest rate to calculate the growth rate; a multiplication factor with which our total sum grows every year. Because our interest rate is a percentage we need to divide by 100 to obtain 𝑖𝑛𝑡𝑒𝑟𝑒𝑠𝑡 the correct fraction: 𝑟𝑔𝑟𝑜𝑤𝑡ℎ = 1 + . If we don’t add or subtract any money aside from the 100 interest, we should multiply the starting sum with this growth rate for each year we consider: 𝑣𝑒𝑛𝑑 = 𝑛 𝑣𝑠𝑡𝑎𝑟𝑡 ⋅ ∏ 𝑦𝑒𝑎𝑟𝑠 𝑟𝑔𝑟𝑜𝑤𝑡ℎ . We can implement this in code with a for-loop (for more info on this see section 3.5): # Calculate growth rate rate = 1.0 + interest / 100.0 # Calculate compound interest: start with initial sum v_end = v_start for i in range(1, nyears + 1): # Each year we should multiply the accumulated sum with the # growth rate v_end = v_end * rate # Print the accumulated sum after each year print('Total sum after', i, 'years is:', v_end) Exercise 2.9.2 In this exercise we want to ask our user a number of words, but we don’t know how many. To fix this we could for instance first ask our user how many words (s)he wants to enter, and then use the input() function that many times to ask each word using a for-loop: nwords = int(input(”How many words do you want in your list? ”)) # Store the words in a list. Start with an empty one words = [] # Use a for-loop to call input() nwords times for i in range(nwords): word = input(”Enter the next word in the list: ”) # Use list.append() to add the word to the list words.append(word) The next step is to print the words, as well as the length of each word. We can again use a forloop: for i in range(nwords): print(words[i], 'has length', len(words[i])) Exercise 2.9.3 To print the contents of a list, and the size of each item in a list we can use the same approach with a for-loop as in the previous exercise. To make our loop with the right number of iterations we need to find the length of our list. We can obtain this with the len() function: values = [[10, 20], [30, 40, 50, 60, 70]] for i in range(len(values)): print(len(values[i]), values[i]) Exercise 2.9.4

171 To construct a new list, we start off with an empty one. Looking at lengths, we can see that the first list will contain two values, the second list will contain two values, and the third list will contain three values. As in the previous exercise, we can use a for-loop to go over these lengths, and select an appropriate number of values from values. There’s one catch: we need to make sure we remember where we left off in values when we re-enter the loop. This can be done with a counter, as shown here: values = [10, 20, 30, 40, 50, 60, 70] lengths = [2, 2, 3] result = [] counter = 0 for i in range(len(lengths)): result.append(values[counter:counter+lengths[i]]) # select a list ↪ of lengths[i] values counter += lengths[i] # increment the counter by the number of ↪ values we just added print(result) Exercise 3.1.1 In this exercise we need to get two numbers (floats) from the user (with input() and float()), and calculate the hypotenuse using Pythagoras formula. The equation contains a square root, which we can implement using sqrt() from the math module, or with the built-in power-of operator: (a**2 + b**2)**0.5. from math import sqrt a = float(input(”Enter the length of right side a: ”)) b = float(input(”Enter the length of right side b: ”)) # c # c

Option 1 = sqrt(a * a + b * b) Option 2 = (a**2 + b**2)**0.5

print(”The length hypotenuse c is”, c) Exercise 3.1.2 In this exercise we’ll again use input() and float() to obtain the numbers a, b, c, and d from the user: a b c d

= = = =

float(input('Enter float(input('Enter float(input('Enter float(input('Enter

a a a a

value value value value

a: b: c: d:

')) ')) ')) '))

Next we’ll follow the steps from the diagram. The first step is to calculate the largest value of a and b. We’ll store the result in a new variable m1: # Find maximum of a and b if a > b: m1 = a else:

172

A. Answers to exercises

m1 = b In the second step we’ll do the same for c and d: # Find if c > m2 else: m2

maximum of c and d d: = c = d

Finally in a third comparison we’ll compare the new variables m1 and m2 to find the largest of all four variables: # Print largest of all four values if m1 > m2: print(”The maximum value is:”, m1) else: print(”The maximum value is:”, m2) Note that the table in section2.10 contains a nice built-in function which gives us a much easier approach, the max() function. It allows us to do the entire comparison in one line!: # Print largest of all four values print(”The maximum value is:”, max(a, b, c, d)) Exercise 3.1.3 First, we need to get a number from the user (with input() and float()) and set the correct bounds. Next, we can take one of two approaches: using if-else-statements, or using min and max. Approach 1: using if-else-statements: a = float(input(”Enter a value: ”)) minimum = -10.0 maximum = 10.0 if a < minimum: # too small a = minimum elif a > maximum: # too big a = maximum else: pass # means 'do nothing' print(a) if a < 0: print(”The value is negative!”) Approach 2: using min and max: a = float(input(”Enter a value: ”)) minimum = -10.0 maximum = 10.0 # max(a, minimum) clamps the lower bound

173

# min(..., maximum) then immediately clamps the upper bound a = min(max(a, minimum), maximum) print(a) if a < 0: print(”The value is negative!”) Exercise 3.5.1 To find out whether a number is a prime number we need to check if it’s divisible by any other number than itself or 1. This answer will take a simple approach: For each potential prime ‘n’ it will use a simple loop that checks for each number smaller than n whether n is divisible by it. We’ll start by asking the user until which value we should check for primes, and then loop over all these numbers: maxvalue = int(input(”Until which value do you want to calculate the ↪ prime numbers?”)) # The first prime, 2, we skip, so that afterwards we can make our loop ↪ faster by skipping all even numbers: print(”The prime numbers under”, maxvalue, ”are:”) print(2) # Now make the loop over all odd values smaller or equal than maxvalue: # We start at 3, until maxvalue, in steps of 2 # Remember that to include the final value, we need # to set the end value in the range function one higher! for n in range(3, maxvalue + 1, 2): To calculate if the current n is divisible by any other number than itself and one we need to check a variable number of values. We can do this with a nested loop (a second for-loop inside the loop of n values): # The outer for-loop from above: for n in range(3, maxvalue + 1, 2): # Let's start out by assuming n is a prime: isprime = True # For each n we want to look at all values larger than one, and ↪ smaller than n: for x in range(2, n): # If n is divisible by x, the remainder of this division is 0. ↪ We can check this with the modulo operator: if n % x == 0: # If we get here the current n is not a prime. isprime = False # If isprime is still True this is indeed a prime if isprime: print(n) Exercise 3.5.2 Although you can generate this pattern also with two consecutive loops (first: for i in range(1,6), then loop back with a second loop: for i in range(5, 0, -1)), a solution with only one loop is of course neatest. A nice way to obtain the requested triangular shape is by making use of numbers around zero, in combination with the pyabs() function. To generate the sequence (1, 2, 3, 4, 5, 4, 3,

174

A. Answers to exercises

2, 1), we can use the following equation: 𝑥 = 5 − |𝑖|, following for-loop:

𝑖 ∈ [−4, 4]. In Python, this is done with the

for i in range(-4, 5): x = 5 - abs(i) # Print string of 'x' stars. # Remember that 'num * list' creates a combined # list with num times list joined together: print(x * '* ') Instead of the print(x * '* ') you can also use a nested loop: for i in range(-4, 5): for j in range(5 - abs(i)): # To avoid the newline that is normally added by print, use the ↪ 'end' keyword argument: print('* ', end='') print() # Add the newline after each finished nested loop Exercise 3.5.3 In this program we’ll let the user enter a string with the input() function, and count the number of digits and letters in that string. For this we can use two built-in methods of Python’s string type: str.isdigit() and str.isalpha(). We’ll use two integers to store these two numbers, and do the counting in a for-loop: print('Hello! This is a program that counts the number of digits and letters in a string you provide.') ↪ text = input('Give your string: ') # counters for the digits and letters digits = 0 letters = 0 # We can loop directly over the characters in 'text' for c in text: # Check if c is a digit if c.isdigit(): digits = digits + 1 # Else check if c is a letter elif c.isalpha(): letters = letters + 1 # After the loop print the results: print('Your string has', digits, 'digits, and', letters, 'letters!') Exercise 3.7.1 In the previous version of our prime number checker from exercise 3.5.1, we used a nested loop: an outer for-loop to go over all values below a certain target value, and a second, nested for-loop to, for each value of the outer loop, find if there exists a value with which it is divisible. We can make this inner loop faster by breaking out of it as soon as we’ve found a number with which our current potential prime is divisible: # The outer for-loop from above: for n in range(3, maxvalue + 1, 2): # Let's start out by assuming n is a prime: isprime = True

175

# For each n we want to look at all values larger than one, and ↪ smaller than n: for x in range(2, n): # If n is divisible by x, the remainder of this division is 0. ↪ We can check this with the modulo operator: if n % x == 0: # If we get here the current n is not a prime. isprime = False # we don't need to check further numbers break # If isprime is still True this is indeed a prime if isprime: print(n) An alternative approach which avoids the isprime logical makes use of a for-else combination: when else is used connected to a for-loop, its contents are only executed when the for-loop completely runs. So when a break prematurely stops the loop, the code in the else block is not run. for n in range(3, maxvalue + 1, 2): for x in range(2, n): if n % x == 0: # In this case we only need to break break else: # When this else block is reached n is a prime print(n) Exercise 3.7.2 We first need to get the input from the user and construct sets of requirements to check against. We can check whether a character is in a string or list using in. When a criterion is met, we set the indicator for that criterion to True. At the end, if all criteria are met, the password is valid. password = input(”Enter a password: ”) abc = ”abcdefghijklmnopqrstuvwxyz” ABC = ”ABCDEFGHIJKLMNOPQRSTUVWXYZ” digits = ”0123456789” special = ”!@#$%&*+-=?^_~” length_indicator = False abc_indicator = False ABC_indicator = False digits_indicator = False special_indicator = False # check length if len(password) >= 8 and len(password) 0 or y2 > 0: t = t + dt F = m * g D = 0.5 * CD * rho * vy2 ** 2 * S # Without drag a1 = (-1.0 * F) / m vy1 = vy1 + a1 * dt y1 = y1 + vy1 * dt # With drag a2 = (-1.0 * F + D) / m vy2 = vy2 + a2 * dt y2 = y2 + vy2 * dt ttab.append(t) y1tab.append(y1) y2tab.append(y2) plt.plot(ttab, y1tab) plt.plot(ttab, y2tab) plt.show() Exercise 9.2.1 In this exercise we make a two-dimensional simulation. As you’ve seen in the corresponding chapter, the difference with a one-dimensional simulation is not big, so the steps in this exercise are the same as in the previous one: Set the initial values, create the empty lists for later plotting, and simulate until the desired end condition using a while-loop. In the first step we make this simulation without drag: import matplotlib.pyplot as plt import math delta = 0.01 s = 0 h = 1.0 theta = 30.0 * math.pi / 180.0 vx = 100 * math.cos(theta) vy = 100 * math.sin(theta) g = 9.81 stab = [] htab = [] while h >= 0: # Acceleration only dependent on gravity

184

A. Answers to exercises

ay = -1.0 * g ax = 0 vy = vy + ay * delta vx = vx + ax * delta h = h + vy * delta s = s + vx * delta htab.append(h) stab.append(s) plt.plot(stab, htab) plt.ylim(0, 140) plt.show()

In the second step we add a constant drag. This adds two lines of initialisation, and changes the calculation of 𝑎𝑥 and 𝑎𝑦 . Also theta needs to be updated now:

delta = 0.01 s = 0 h = 1.0 theta = 30.0 * math.pi / 180.0 vx = 100 * math.cos(theta) vy = 100 * math.sin(theta) g = 9.81 Fd = 80 m = 10 stab = [] htab = [] while h >= 0: # Acceleration dependent on gravity # and constant drag ay = (-1 * m * g -1 * Fd * math.sin(theta)) / m ax = (-1 * Fd * math.cos(theta)) / m vy = vy + ay * delta vx = vx + ax * delta h = h + vy * delta s = s + vx * delta theta = math.atan(vy / vx) htab.append(h) stab.append(s) plt.plot(stab, htab) plt.ylim(0, 140) plt.show()

In the final step we acknowledge that drag depends on airspeed. For this we need a calculation of total drag, and a decomposition of this value into x/y components:

185

delta = 0.01 s = 0 h = 1.0 theta = 30.0 * math.pi / 180.0 vx = 100 * math.cos(theta) vy = 100 * math.sin(theta) g = 9.81 CD = 0.47 rho = 1.225 R = 0.15 S = math.pi * R**2 m = 10 stab = [] htab = [] while h >= 0: # Calculate the actual drag V2 = (vx**2 + vy**2) V = math.sqrt(V2) D = 0.5 * CD * rho * V2 * S # Decompose in x/y components, # making use of the ratio between vx and vy Dx = -D * vx / V Dy = -D * vy / V # Acceleration dependent on gravity # and drag ay = (-1 * m * g + Dy) / m ax = Dx / m vy = vy + ay * delta vx = vx + ax * delta h = h + vy * delta s = s + vx * delta htab.append(h) stab.append(s) plt.plot(stab, htab) plt.ylim(0, 140) plt.show() Exercise 10.1.1 In this exercise we’ll plot a hexagon inside a circle, using NumPy. This can be done very easily with numpy.linspace(), when we consider that a hexagon is just a very coarse circle: import matplotlib.pyplot as plt import numpy as np

# Create our circle: r = 2 phi1 = np.linspace(0, 2 * np.pi, 200) x1 = r * np.sin(phi1) y1 = r * np.cos(phi1)

186

A. Answers to exercises

# A hexagon is no more than a circle drawn with just 6 points: phi2 = np.linspace(0, 2 * np.pi, 6) x2 = r * np.sin(phi2) y2 = r * np.cos(phi2) plt.plot(x1, y1) plt.plot(x2, y2) plt.show() Exercise 10.1.2 There are different ways to get the diagonal of an array. One way that you will already be familiar with is to use a for-loop: import numpy as np a = np.array([[1, 2], [3, 4]]) def printdiag(arr): # Use end=' ' to avoid the newline of print print('The numbers on the diagonal are:', end=' ') for i in range(len(arr)): print(arr[i,i], end=' ') # A final print for the newline print() However, if we look further in the documentation of NumPy we find that the array type already provides a method to return its diagonal: import numpy as np a = np.array([[1, 2], [3, 4]]) print('The numbers on the diagonal are:', a.diagonal()) Exercise 10.1.3 Multi-dimensional slicing is one of the aspects where NumPy arrays really shine compared to nested lists. While it is still easy enough to get rows from a nested list, its impossible to directly slice columns. For arrays this is possible. First the list approach: lst = [[1, 2], [3, 4]] # Get the first column, start with an empty one col = [] # Iterate over all rows in our nested list, and add # the first element to col to recreate the entire # first column for row in lst: col.append(row[0]) As you can see, this is quite tedious, since a for-loop is needed here. Because NumPy arrays support multi-dimensional slicing, getting the first column of an array is much easier: import numpy as np arr = np.array([[1, 2], [3, 4]]) col = arr[:,0]

187 Exercise 10.2.1 To extend the calculation of the temperature vector to 32 km altitude, one layer needs to be added to the logical multiplication expression. With the given temperature gradient, and the starting temperature of 216.65 K, the temperature equation for this layer becomes: 𝑇(ℎ) = 216.65 + 1 ⋅ (ℎ − 20). The code with one layer extra becomes: import numpy as np import matplotlib.pyplot as plt h = np.arange(0, 32.0, 1.0) to11 = 288.15 - 6.5 * h to20 = 216.65 to32 = 216.65 + 1.0 * (h - 20.0) temp = (h < 11) * to11 + (h >= 11) * to20 + (h >= 20) * (to32 - to20) plt.plot(h, temp) plt.show() Exercise 10.2.2 We have to plot a discontinuous function that is equal to 𝑥 2 for 𝑥 < 1, and equal to 6 − 𝑥 for 𝑥 > 1. We can again solve this with boolean masks: import numpy as np import matplotlib.pyplot as plt # First create our x array x = np.arange(-3, 3, 0.001) # Then create our discontinuous function # with masks for x 1 fx = (x1) * (6 - x) plt.plot(x, fx) plt.show() Exercise 10.2.3 For the array given in the exercise, we want to retrieve the numbers: (1) not divisible by 3 (arr%3 != 0), (2) divisible by 5 (arr%5 == 0), (3) divisible by 3 and 5 ((arr%3 == 0) * (arr%5 == 0)). Finally we want to set all numbers that are divisible by three to 42. This can be done as follows: import numpy as np arr = np.array([3, 4, 6, 10, 24, 89, 45, 43, 46, 99, 100]) # Not divisible by 3: print('Elements not divisible by 3:', arr[arr%3 != 0]) # Divisible by 5: print('Elements divisible by 5:', arr[arr%5 == 0]) # Divisible by 3 and 5: print('Elements divisible by 3 and 5:', arr[(arr%3 == 0) * (arr%5 == ↪ 0)]) # Set elements that are divisible by 3 to 42 arr[arr%3 == 0] = 42

188

A. Answers to exercises

print('Our new array becomes:', arr) Exercise 10.4.1 To solve the linear system 𝐴𝑥 = 𝑏, we need to right-multiply with the inverse of A: 𝐴−1 𝐴𝑥 = 𝑥 = 𝐴−1 𝑏. We can invert a matrix using numpy.linalg.inv(): import numpy as np A = np.array([[2, -5, 6], [4, 3, 8], [5, -2, 9]]) b = np.array([[3, 5, -1]]).T # .T because we need a column vector # Calculate x using the inverse of A Ainv = np.linalg.inv(A) x = Ainv.dot(b) # You can also use Ainv @ b Exercise 10.4.2 We want to find the speed of a motorboat, from the time it takes to make a trip up- and downstream. If the motorboat speed is 𝑢, and the river current speed is 𝑣, we know that: 3 ∶ 10 ⋅ (𝑢 − 𝑣)

= 29.5[km]

2 ∶ 00 ⋅ (𝑢 + 𝑣)

= 29.5[km]

or rewritten: 𝑢−𝑣

=

𝑢+𝑣

=

29.5 [km/h] 3 ∶ 10 29.5 [km/h] 2 ∶ 00

1 −1 𝑢 29.5/3.16667 This can be written as a linear system 𝐴𝑥 = 𝑏, where 𝐴 = [ ], 𝑥 = [ ], and 𝑏 = [ ]. 1 1 𝑣 29.5/2.1 This linear system can be solved in the same way as the previous exercise: import numpy as np A = np.array([[1, -1], [1, 1]]) b = np.array([[29.5 / 3.16667, 29.5 / 2.1]]).T # .T because we need a column vector ↪ # Calculate x using the inverse of A Ainv = np.linalg.inv(A) x = Ainv.dot(b) # You can also use Ainv @ b # Print the results print('The motorboat speed is', x[0], 'km/h') print('The current speed is', x[1], 'km/h') Exercise 10.4.3 We want to find a linear trend 𝑦 = 𝑏1 + 𝑏2 𝑥, which best fits the ten datapoints from the function from the exercise. This means that the following needs to be minimised: 𝑦1

− (𝑏1 + 𝑏2 𝑥1 )

𝑦2

− (𝑏1 + 𝑏2 𝑥2 ) ⋮

𝑦1 0 −

(𝑏1 + 𝑏2 𝑥1 0)

This can be rewritten as 𝑌 = 𝑋𝑏, where 𝑏 = [𝑏1 , 𝑏2 ], 𝑌 is a 𝑛 ×1 column vector with all values of y, and 𝑋 is a 𝑛 ×2 matrix, with on its first column all ones, and on its second column all values of 𝑥 corresponding to the values in vector 𝑌.

189 The linear least squares method involves finding the minimum of the squared differences: ||𝑌 − 𝑋𝑏||2 . This can be done by setting the derivative to zero, and solving for 𝑏: 𝜕(||𝑌 − 𝑋𝑏||2 ) 𝜕(𝑌 𝑇 𝑌 − 𝑌 𝑇 𝑋𝑏 − 𝑏 𝑇 𝑋 𝑇 𝑌 + 𝑏 𝑇 𝑋 𝑇 𝑋𝑏) = = −2𝑋 𝑇 𝑌 + 2𝑋 𝑇 𝑋𝑏 = 0 𝜕𝑏 𝜕𝑏 This is again a linear system that we can solve, where the solution is: 𝑏 = (𝑋 𝑇 𝑋)−1 𝑋 𝑇 𝑌. Note that 𝑋 𝑇 𝑋 becomes a 2 × 2 matrix, and 𝑋 𝑇 𝑌 a 2 × 1 column vector. In code this becomes: import numpy as np x = np.arange(0.1, 1.1, 0.1) y = 5 * np.exp(-x) + 2 * x # Reshape x and y to obtain 2D vectors # We want a linear trend, so y = b1 + b2x = Xb # If b = [b1, b2], then X should be [[1, x_0], [1, x_1], ...] X = np.ones((len(x), 2)) X[:,1] = x Y = y.reshape((len(y), 1)) # Solve using b = (XTX)^-1XTY XTXi = np.linalg.inv(X.T @ X) XTY = X.T @ Y b = XTXi @ XTY # Print the result print(b) Note that if you dive into the NumPy docs you’ll find that the linalg sub module has a built-in least squares function. So you can also do the following: import numpy as np x = np.arange(0, 1.1, 0.1) y = 5 * np.exp(-x) + 2 * x X = np.ones((len(x), 2)) X[:,1] = x Y = y.reshape((len(y), 1)) print(np.linalg.lstsq(X, Y)) Exercise 12.9.1 In this exercise we’re building a game in four steps: a) The basics of initialising PyGame are always the same: import the module, initialise it, and set up the screen: import pygame as pg ## INITIALISATION CODE BLOCK pg.init() reso = (750, 500) blue = (0, 0, 255)

190

A. Answers to exercises

screen = pg.display.set_mode(reso) ## DRAWING CODE BLOCK # Event pump to keep the window responsive pg.event.pump() screen.fill(blue) pg.display.flip() pg.quit() b) In the next step we add a picture of an airplane to the screen. This involves loading the image, giving it an appropriate scale, and a position on the screen. The final step is to blit the image to the screen: # Add these lines to the initialisation code block plane = pg.image.load('plane.jpg') plane = pg.transform.scale(plane, (150, 60)) planerect = plane.get_rect() # Add these lines to the drawing code block planerect.center = (75, 250) screen.blit(plane, planerect) c) In step 3 we add a while-loop to allow us to update the screen with time. In this loop we use numerical integration to give the airplane a horizontal speed on the screen. For the numerical integration we need some more initialisation: # Add these lines to the initialisation code block xs = 75 ys = 250 vx = 50 tsim = 0.0 tstart = 0.001 * pg.time.get_ticks() dt = 0.01 running = True The drawing code block needs to be changed to have all drawing inside a while-loop, together with the numerical integration: # The new drawing code block while running: # update current time trun = 0.001 * pg.time.get_ticks() - tstart # Perform update when dt time has elapsed if trun + dt >= tsim: tsim = tsim + dt xs = xs + vx * dt planerect.center = (xs, ys) # Event pump to keep the window responsive pg.event.pump() screen.fill(blue) screen.blit(plane, planerect)

191

pg.display.flip() # Add a terminating condition for when the # airplane leaves the screen if xs > 750 + 75: running = False # Quit after the loop finishes pg.quit() d) In the final step we enable moving the airplane vertically over the screen using the up and down keys. This involves processing the pygame events using pg.event.get(): # Add this in the while loop, right after pump: for event in pg.event.get(): if event.type == pg.QUIT: running = False if event.type == pg.KEYDOWN: if event.key == pg.K_ESCAPE: running = False elif event.key == pg.K_DOWN: ys = ys + 5 elif event.key == pg.K_UP: ys = ys - 5 After these four steps you should have a complete (and very simple) game, with an airplane moving over the screen from left to right, where you can control its height on the screen with the arrow keys!

B Jupyter: the interactive Python notebook

You can also install jupyter with pip install. With Jupyter you can edit so-called iPython Notebooks. These are a blend between Python and LateX with an excel-like functionality. The result are cells with Python code, with their output and documentation in-between the cells. Each cell can run individually, but they all access the same namespace, so one Notebook is like one module. It is accessed in a server-like way, so with the browser.

The iPy Notebook is very convenient user interface for Scientific Computing tasks. It allows you to edit snippets of Python/NumPy code (“Cells”) which you run but also can still edit later. You can access all variables between cells, as if you are in one program, but you can also see the output (including plots) in between the code. The result is a blend of a graphical calculator, Python and a spreadsheet program like Excel. You can run an individual cell or all cells and see how the output changes. You can also add cells with raw text to create a living document. Each notebook page can be saved (“downloaded” as it is called in this client-server set-up) as either a .py Python file or a .ipynb Python notebook file. The Notebook page is both a scratchpad as well as a publication.

An example of how this looks is shown below. As you can see all functions from math, numpy scipy and matplotlib.pyplot have already been imported: 193

194

B. Jupyter: the interactive Python notebook

C Programming and Scientific Computing in Python - Cheat sheet This appendix gives a quick overview of the concepts, modules, types, functions, and methods described in this reader. Each of the following sections covers one chapter in the reader.

C.1. Getting started • Printing text to the screen:

• Calling a function: functionname(arguments)

print(”This is a line of ↪ text”) a = 1 b = 2 print(f”An f-string with ↪ variables a={a}, b={b}”) # Printing multiple items: print(”Some text, and”, a, b, ↪ ”!”)

# examples of # built-in functions print() input() num = round(3.14) • Getting help:

• Getting input from the user:

# String argument help(”math”) # imported module as argument help(math) # type as argument help(int) # method/function as argument help(str.split)

text = input(”Give a line of text: ”) ↪ i = int(input(”Give a whole ↪ number: ”)) f = float(input(”Give any ↪ number: ”))

195

196

C. Programming and Scientific Computing in Python - Cheat sheet

C.2. Variable assignment and types • Variable assignment: linking a variable name to the stored outcome of an expression: variablename = expression i = 1 # Integer inch = 2.54 # Floating point running = True # Logical name = ”John” # String # Same variable on left and # right side of equals sign i = i + 1 # Multiple assignments a, b, c = 1, 2, ”Graham” # compound expression c = (a**2 + b**2)**0.5 • Mathematical short-hands: i j f g

+= -= *= /=

1 # Increment i 10 # Decrement j 0.5 # Scale f 2 # Scale g

• Number operations: + * / // % ** -x abs(x) pow(x, y) divmod(x, y)

Addition Subtraction Multiplication Division Integer division Modulo/division remainder Exponentiation (power of) Negation (reverse sign) Absolute value Same as x**y Returns both x // y and x % y

• Type conversion: # float to int: rounds down, # sets i to 3 i = int(3.14) # Change int type to float f = float(12) txt = ”120” # Convert string to int count = int(txt) • Complex numbers: x = 1 + 2j x.real # real part x.imag # imaginary part

• String operations: # concatenation name = ”John” + ”Cleese” # String length n = len(name) # convert variable to string s = str(a) # Get ith ASCII character c = chr(i) # Get ASCII index of # character c i = ord(c) # convert to uppercase c.upper() # convert to lowercase c.lower() # Remove leading/trailing # whitespace c.strip() # Split c after every 'ch' c.split(ch) # True if c is in a-zA-Z c.isalpha() # True if c is in 0-9 c.isdigit() # get index of first # occurrence of 'b' c.index('b') • Logical expressions: a = False b = True # a is smaller than b? sw = a < b # a is larger or equal to b? sw = a >= b # a equal to b? sw = a == b # a not equal to b? sw = a != b # True if at least a or b is True ↪ sw = a or b # True if a and b are both True ↪ sw = a and b # True if a is not True sw = not a # Combined inequality ↪ comparison sw = 1 8s”)

• String inserting with format: ”i is {:5d}”.format(i) ”Coords: ↪ ({:.2f},{:.2f})”.format(x, ↪ y) ”The text is: ↪ {}”.format(text) ”A named value ↪ {ref:2.2f}”.format( ref=3.1415) • String inserting with f-strings:

198

C. Programming and Scientific Computing in Python - Cheat sheet

f”Your age is {age:5d}” f”Coconut weight: {w_c:.2f}”

• Combinations and permutations: from itertools import ↪ permutations, ↪ combinations

• Condition checking: # Check a condition if a > b: # When true, perform this print(”a is bigger”) elif a < b: # Perform a second check ↪ if # the first was false print(”a is smaller”) else: # If all previous conditions ↪ # were false, do this print(”a and b are ↪ equal”)

lst = [1, 2, 3] # Print (1, 2, 3), (1, 3, 2), ↪ (3, 1, 2) # etc. for a in permutations(lst): print(a)

# Print (1, 2), (1, 3), (2, ↪ 3) for a in combinations(lst, ↪ 2): print(a) • Controlling the loop

• Fixed-range iteration: # Iterate over range [0-9] # current value is available # as variable i in loop for i in range(10): print(i) # Iterate directly over list lst = [1, 12, ”Word”, False] for el in lst: print(”el is:”, el)

# continue: jump to the next ↪ iteration # of the loop for i in range(10): if i == 5: continue # break: exit the entire loop for j in range(50): if j % 7 == 0: break

• Variable-range iteration: while condition: # Do something as long as # condition is true

C.4. Making your code reusable and readable • Use clear variable names • Use comments to explain your code • Use docstrings to document: – Your file: start the file with a docstring – Your functions: start functions with a docstring – Your classes: start a class definition with a docstring

C.5. Extending Python: using modules

C.5. Extending Python: using modules • Importing modules: import math import pygame as pg • Importing from modules: from math import sqrt from numpy import * • The math module: constants import math math.pi # 3.1415 math.e # 2.718 • The math module: log/exp math.exp(x) # e^x math.log(x) # ln(x) math.log(x, a) # a-log(x) math.log1p(x) # ln(x) precise ↪ around 0 math.log10(x) # 10log(x) math.pow(x, y) # x**y math.sqrt(x) # x**0.5 • The math module: trigonometry # [cst] of x in radians math.sin(x) math.cos(x) math.tan(x) # arc[cst] in radians math.acos(x) math.asin(x) math.atan(x) math.atan2(y, x) # (x**2+y**2)**0.5 math.hypot(x,y) math.degrees(x) # rad to deg math.radians(x) # deg to rad • The math module: Hyperbolic # Hyperbolic functions math.sinh(x) math.cosh(x) math.tanh(x) math.asinh(x) math.acosh(x) math.atanh(x) • The math module: number functions

199

math.ceil(x) # round up math.floor(x) # round down math.copysign(x) # -1 or 1 math.factorial(n) # x%y for floats math.fmod(x, y) # more accurate summing math.fsum(lst) # return x//1 and x%1 math.modf(x) # truncate real x to integer math.trunc(x) • The random module: import random as rnd # Uniform sample from [0,1> rnd.random() # uniform int selection # from [a, b] rnd.randint(a, b) # uniform selection from # range(start,stop,step) rnd.randrange(start, stop, ↪ step)

200

C. Programming and Scientific Computing in Python - Cheat sheet

C.6. Defining your own functions and modules • Defining a custom function: A function has a name, and zero or more arguments. Good practice is to document your function with a docstring. All indented code after the def statement will be executed when the function is called. You can return values to the caller with the return statement. def functionname(arguments): ''' A docstring ''' function.contents() return returnvalues

• Creating your own modules If you have code that you use in many different places it makes sense to put it in its own file, which you can then import in other places: constants.py rho0 = 1.225 ft = 0.3048 inch = 2.54 kts = 0.51444 main.py import constants

• Providing argument defaults def myfun(a=1, b=2, c=3): print(a, b, c)

# Speed of 250 knots in m/s spd = 250 * constants.kts

myfun() # this is possible myfun(10) # this too # You can also pass by name myfun(c=100)

C.7. File IO and string handling • Opening and closing a file: # For reading f = open(filename) f = open(filename, 'r') # For writing f = open(filename, 'w') # For appending f = open(filename, 'a') # Closing a file f.close() • Easy management of open files. You can avoid having to keep track of open files by using the with statement: with open(fname) as fin: # Do stuff with file ... • Read a single line line = fin.readline() • Iterating over file contents:

with open(fname) as fin: for line in ↪ fin.readlines(): print(line) # or with open(fname) as fin: for line in fin: print(line) • Reading characters (across newlines): # Get the next 300 characters ↪ from file spartans = fin.read(300) • Writing to file with open(fname, 'w') as ↪ fout: for row in table: line = ','.join(row) ↪ + '\n' fout.write(line)

C.8. Matplotlib: Data visualisation in Python

201

C.8. Matplotlib: Data visualisation in Python • Importing and basic plotting. Basic plotting in matplotlib with pyplot:

import matplotlib.pyplot as ↪ plt x = [1, 2, 3, 4] y = [1, 4, 9, 16] plt.plot(x, y) plt.show()

plt.plot(x, y) # Label the axes plt.xlabel('x [sec]') plt.ylabel('y [m/s]') # Add a title plt.title('Speed as a ↪ function of time') # Add a legend plt.legend(('Matlab', ↪ 'Python')) plt.show() • Multiple plots in one figure

• Making it look good:

• Interactive plots

plt.subplot(211) plt.plot(t, x) plt.subplot(212) plt.subplot(t, v) • 3D surface plot:

# Enable interactive mode plt.ion()

from mpl_toolkits.mplot3d ↪ import Axes3D

for i in range(100): plt.plot(i / 100, math.sin(i / 100)) ↪ plt.draw()

fig = plt.figure() ax = Axes3D(fig) surf = ax.plot_surface(X, Y, ↪ Z)

plt.close() • Contour plot:

plt.show()

X, Y, Z = ... im = plt.imshow(Z) cset = plt.contour(Z, ↪ [levels]) plt.show()

C.9. Numerical integration • For numerical integration in the course we typically use the Backward Euler method, because of its ease of use. It allows us to do this: # Increment time with # fixed timestep t = t + dt

# Determine acceleration # from forces a = Ftot / m # Integrate acceleration # to obtain next speed

202

C. Programming and Scientific Computing in Python - Cheat sheet

v = v + a * dt # Integrate speed to # obtain next position x = x + v * dt

• Adding dimensions is easy, but take care for non-linear relationships between dimensions, such as (total) drag and airspeed!

# Calculate the total ↪ airspeed from its x,y ↪ components V = sqrt(vx * vx + vy * vy) # Calculate the total drag D = cd * 0.5 * rho * V**2 * S # Calculate the direction of # the velocity vector angle = atan2(-vy, -vx) # Use this direction to # decompose the total drag # into x,y components Dx = D * cos(angle) Dy = D * sin(angle) • math.atan2() can help you with quadrantaware angle calculations!

C.10. NumPy and SciPy: Scientific programming with arrays and matrices

203

C.10. NumPy and SciPy: Scientific programming with arrays and matrices • (Multi-dimensional) array calculations are simpler (and faster) with NumPy! Creating an array: import numpy as np arr = np.array([[1, 2], [3, ↪ 4]]) • You can perform mathematical operations on entire arrays in one go: a = np.array([1, 2, 3]) b = np.array([4, 5, 6]) c = a * b # [4, 10, 18]

• Indexing and logical operations # Using a logical expression ↪ as index a[a > 0] # only works if len(a) == ↪ len(b) a[b < 3] # Return indices where ↪ condition is true np.where(a > 0) # Logical and np.logical_and(a, b) a * b

s = np.sqrt(c) • Functions to create arrays: # Create array filled with zeros ↪ np.zeros(shape) np.ones(shape) # ones np.ones(shape) * 10 # tens # range() but also for floats np.arange(start, stop, step) # linspace: end included! np.linspace(start, end, ↪ nelems) # logarithmic scale np.logspace(start, end, ↪ nelems) x = np.append(x, 10) # append ↪ element • Slicing arrays Slicing works the same as regular Python, but the syntax for multi-dimensional slices is slightly different (only one set of brackets: a[i, j]), and you can also slice over columns.

# Logical or np.logical_or(a, b) a + b # Logical not np.logical_not(a) 1 - a • Linear algebra: # Vector dot product v1.dot(v2) v1 @ v2 # Matrix multiplication x = A.dot(b) x = A @ b # Cross product v3 = np.cross(v1, v2) # Outer product A = np.outer(v1, v2) # Inverse Ai = np.linalg.inv(A) # Transpose At = A.T # Determinant det = np.linalg.det(A)

204

C. Programming and Scientific Computing in Python - Cheat sheet

C.11. Tuples, dictionaries, and sets • Tuples: immutable lists # Defining a tuple a = (1, 2, 3) b = 4, 5, 6 name = tuple(”John”, ↪ ”Cleese”) # The following is not ↪ possible: b[1] = 10 • Sets: Unordered collection of unique items # a b #

Creating sets = {1, 2, 6, 8} = set((1, 2, 3, 4, 9)) Sets have special ↪ operations: # difference: in b but not in ↪ a b.difference(a) b - a # union: combination of a and ↪ b a.union(b) a + b # intersection: in a and b overlap = a.intersection(b) # true if all values in a are ↪ also in b a.issubset(b)

• Dictionaries: unordered list-like container, with key as lookup value (often strings). # Creating a dict a = {”John”: 10, ”Graham”: ↪ 20} b = dict(fish=10, dairy=0) # Accessing values print(a[”John”]) # Testing if key in dict if ”fish” in b: print(b[”fish”]) # Iterating over all values ↪ in dict for v in b.values(): print(v) # Iterating with both key and ↪ value for key, val in a.items(): print(key, ”is”, val)

C.12. Pygame: Animation, visualisation and controls • Initialisation and wrapping up:

import pygame as pg pg.init() # setting up the screen scr = pg.display.set_mode( (w, h)) # Draw in a loop while running: # All of the drawing ... # Done? flip the screen pg.display.flip() # Wrapping up pg.quit()

• Drawing to the screen: # Color is an RGB tuple black = (0, 0, 0) # Fill screen with color scr.fill(black) pg.draw.rect(scr, color, ↪ pg.Rect(left, top, width, ↪ height)) # Load and draw image img = ”enterprise.jpg” ship = pg.image.load(img) shiprect = ship.get_rect() # Set position shiprect.centerx = 10 shiprect.centery = 20 # Copy image to screen scr.blit(ship, shiprect)

C.12. Pygame: Animation, visualisation and controls • Image transforms: pg.transform.flip(surf, xs, ↪ ys) pg.transform.scale(surf, (w, ↪ h)) # Rotate, angle in degrees pg.transform.rotate(surf, ↪ angle) pg.transform.rotozoom(surf, ↪ angle, scale) # shorthand scale 2x pg.transform.scale2x(surf) # smooth scale pg.transform.smoothscale(surf, ↪ (w, h)) • Timing the pygame loop: # Fixed timestep while running: trun = 0.001 * ↪ pg.time.get_ticks() if trun + dt >= tsim: # do next step tsim = tsim + dt ... # Variable timestep while running: t = 0.001 * ↪ pg.time.get_ticks() dt = min(t - t0, maxdt) if dt > 0.0: # do next step

205 • Drawing shapes: pg.draw.rect() pg.draw.polygon() pg.draw.circle() pg.draw.ellipse() pg.draw.arc() pg.draw.line() pg.draw.aaline() • Processing inputs # Let pygame process events pg.event.pump() # Iterate over events for e in pg.event.get(): if e.type == pg.QUIT: # user clicked close ↪ btn running = False elif e.type == ↪ pg.KEYDOWN: # Keyboard press elif e.type == ↪ pg.MOUSEBUTTONDOWN: # mouse click x, y = e.pos # Get list (with true/false) ↪ of keypresses keys = pg.key.get_pressed() if keys[pg.K_ESCAPE]: running = False

Some blank pages for your notes - 1

Some blank pages for your notes - 2

Some blank pages for your notes - 3

Some blank pages for your notes - 4

Some blank pages for your notes - 5

Some blank pages for your notes - 6

Some blank pages for your notes - 7