Interactive Dashboards and Data Apps with Plotly and Dash: Harness the power of a fully fledged frontend web framework in Python – no JavaScript required [1 ed.] 1800568916, 9781800568914

Citation preview

Interactive Dashboards and Data Apps with Plotly and Dash Harness the power of a fully fledged frontend web framework in Python – no JavaScript required

Elias Dabbas

BIRMINGHAM—MUMBAI

Interactive Dashboards and Data Apps with Plotly and Dash Copyright © 2021 Packt Publishing All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book. Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information. Group Product Manager: Kunal Parikh Publishing Product Manager: Reshma Raman Senior Editor: David Sugarman Content Development Editor: Joseph Sunil Technical Editor: Devanshi Deepak Ayare Copy Editor: Safis Editing Project Coordinator: Aparna Nair Proofreader: Safis Editing Indexer: Manju Arasan Production Designer: Nilesh Mohite First published: May 2021 Production reference: 1210421 Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK. ISBN 978-1-80056-891-4 www.packt.com

Contributors About the author Elias Dabbas is an online marketing and data science practitioner. He produces open source software for building dashboards and data apps, as well as software for online marketing, with a focus on SEO, SEM, crawling, and text analysis.

About the reviewer Leonardo Ferreira is an accountant and self-taught data scientist with the title of Kaggle Grandmaster and also acts as a data platform developer. He started his studies in data science in 2017 and started working in the field a few months after starting his apprenticeship. Since then, he has worked for large companies in Brazil and worldwide, with more than 100 open source projects adding GitHub, Kaggle, and their portfolio of web applications. He is currently a Top Rated Plus freelancer on the Upwork platform, where he has already carried out more than 20 data science and data platform projects. He is also enthusiastic about solutions on the Cardano blockchain.

Table of Contents Preface

Section 1: Building a Dash App

1

Overview of the Dash Ecosystem Technical requirements Setting up your environment Exploring Dash and other supporting packages The different packages that Dash contains

Understanding the general structure of a Dash app Creating and running the simplest app

4 4

Adding HTML and other components to the app

6

Learning how to structure the layout and managing themes

7

8 9

11

Adding HTML components to a Dash app11

14

Themes14 Grid system and responsiveness 15 Prebuilt components 18 Encoded colors 18 Adding Dash Bootstrap components to our app 19

Summary20

2

Exploring the Structure of a Dash App Technical requirements 22 Using Jupyter Notebooks to run Dash apps 22 Isolating functionality for better management and debugging

23

Creating a standalone pure Python function 25 The id parameter of Dash components26

ii Table of Contents

Dash inputs and outputs

28

Determining your inputs and outputs Specifying your callback function Implementing the callback

28 30 30

Incorporating the function into the app

32

Properties of Dash's callback functions

39

Summary42

3

Working with Plotly's Figure Objects Technical requirements 44 Understanding the Figure object 44 Getting to know the data attribute47 Getting to know the layout attribute49 Interactively exploring the Figure object 51 Configuration options for the Figure object 52

Exploring the different ways of converting figures

Converting figures into HTML Converting figures into images Plotting using a real dataset Data manipulation as an essential part of the data visualization process Making the chart interactive with a callback function Adding the new functionality to our app Theming your figures

53 54 54 58 59 62 65

Summary66

53

4

Data Manipulation and Preparation, Paving the Way to Plotly Express Technical requirements Understanding long format (tidy) data

68

Plotly Express example chart Main attributes of long format (tidy) data

69

68

73

Understanding the role of data manipulation skills

74

Exploring the data files Melting DataFrames

74 85

Pivoting DataFrames Merging DataFrames

88 90

Learning Plotly Express

92

Plotly Express and Figure objects 98 Creating a Plotly Express chart using the dataset 100 Adding new data and columns to our dataset103

Summary103

Table of Contents iii

Section 2: Adding Functionality to Your App with Real Data

5

Interactively Comparing Values with Bar Charts and Dropdown Menus Technical requirements Plotting bar charts vertically and horizontally Creating vertical bar charts with many values

108

Incorporating the functionality into our app

109

Using facets to split charts into multiple sub-charts – horizontally, vertically, or wrapped Exploring additional features of dropdowns

116

Linking bar charts and dropdowns117 Exploring different ways of displaying multiple bar charts (stacked, grouped, overlaid, and relative) 122 Creating the income share DataFrame 123

128

130 134

Adding placeholder text to dropdowns 134 Modifying the app's theme 135 Resizing components 137

Summary139

6

Exploring Variables with Scatter Plots and Filtering Subsets with Sliders Technical requirements Learning about the different ways of using scatter plots: markers, lines, and text Markers, lines, and text

Creating multiple scatter traces in a single plot Mapping and setting colors with scatter plots Discrete and continuous variables Using color with continuous variables

142 142 143

148 151 152 152

Manually creating color scales Using color with discrete variables

156 159

Handling over-plotting and outlier values by managing opacity, symbols, and scales

162

Controlling the opacity and size of markers Using logarithmic scales

Introducing sliders and range sliders

163 165

168

iv Table of Contents

Customizing the marks and values of sliders 

171

Summary176

7

Exploring Map Plots and Enriching Your Dashboards with Markdown Technical requirements 180 Exploring choropleth maps 180 Utilizing animation frames to add a new layer to your plots 183 Using callback functions with maps 186 Creating a Markdown component190

Understanding map projections195 Using scatter map plots 197 Exploring Mapbox maps 199 Exploring other map options and tools 206 Incorporating an interactive map into our app 206 Summary208

8

Calculating Data Frequency and Building Interactive Tables Technical requirements Creating a histogram Customizing the histogram by modifying its bins and using multiple histograms

210 210

Using color to further split the data Exploring other ways of displaying multiple bars in histograms

214

212

215

Adding interactivity to histograms218

Creating a 2D histogram 224 Creating a DataTable 225 Controlling the look and feel of the table (cell width, height, text display, and more) 227 Adding histograms and tables to the app 229 Summary232 What we have covered so far 232

Section 3: Taking Your App to the Next Level

9

Letting Your Data Speak for Itself with Machine Learning Technical requirements

238

Understanding clustering

238

Table of Contents v

Finding the optimal number of clusters 242 Clustering countries by population246 Preparing data with scikit-learn 249

Handling missing values Scaling data with scikit-learn

249 250

Creating an interactive KMeans clustering app 252 Summary257

10

Turbo-charge Your Apps with Advanced Callbacks Technical requirements Understanding State

260 261

Understanding the difference between Input and State 262

Creating components that control other components

Allowing users to add dynamic components to the app 272 Introducing pattern-matching callbacks275 Summary281

266

11

URLs and Multi-Page Apps Technical requirements Getting to know the Location and Link components Getting to know the Link component

Parsing URLs and using their components to modify parts of the app

284

Restructuring your app to cater to multiple layouts

285

Displaying content based on the URL

286

289

290 293

Adding dynamically generated URLs to the app 294 Incorporating the new URL interactivity into the app 296 Summary298

12

Deploying Your App Technical requirements Establishing the general development, deployment, and update workflow

302 302

Creating a hosting account and virtual server Connecting to your server with SSH

305 307

vi Table of Contents

Running the app on the server 310 Setting up and running the app with a WSGI 313 Setting up and configuring the web server 314

Managing maintenance and updates Fixing bugs and making changes Updating Python packages Maintaining your server

317 318 318 320

Summary320

13

Next Steps Technical requirements 324 Expanding your data manipulation and preparation skills325 Exploring more data visualization techniques 325 Exploring other Dash components326

Other Books You May Enjoy Index

Creating your own Dash component327 Operationalizing and visualizing machine learning models 327 Enhancing performance and using big data tools 328 Going large scale with Dash Enterprise 332 Summary333 Why subscribe? 335

Preface Plotly's Dash framework allows Python programmers to develop complete analytic data apps and interactive dashboards. This book will help you explore the functionalities of Dash for visualizing data in different ways and getting the most out of your data. You'll start with an overview of the Dash ecosystem, its main packages, and the available third-party packages crucial for structuring and building different parts of your apps. Next, you will learn how to create a basic Dash app and add different features to it. You'll then integrate controls such as dropdowns, checkboxes, sliders, date pickers, and more in the app, and then link them to charts and other outputs. Depending on the data you are visualizing, you will also add several types of charts, including scatter plots, line plots, bar charts, histograms, maps, and more, as well as exploring the options available for customizing them. By the end of this book, you will have developed the skills you need to create and deploy an interactive dashboard, be able to handle complexity and code refactoring, and understand the process of improving your application.

Who this book is for This Plotly Dash book is for data professionals and data analysts who want to gain a better understanding of their data with the help of different visualizations and dashboards. Basic to intermediate-level knowledge of the Python programming language is expected to grasp the concepts covered in this book more effectively.

What this book covers Chapter 1, Overview of the Dash Ecosystem, will help you attain a good understanding of the Dash ecosystem, the main packages used, as well as the available third-party packages. By the end of this chapter, you will be able to differentiate between the different elements of an app and what each is responsible for, and you will have built a minimal app.

viii

Preface

Chapter 2, Exploring the Structure of a Dash App, demonstrates how to add some interactive functionality to the app we created earlier. We will go through the app callbacks and see how they allow your users to connect visual elements of your app, and how you can let users use some of those elements to control others by creating special callback functions. Chapter 3, Working with Plotly's Figure Objects, provides an in-depth introduction to the figure object, its components, how it can be manipulated, and how it can be converted to different formats. Later on, we will use this understanding to build specific types of charts as needed in our application. Chapter 4, Data Manipulation and Preparation - Paving the Way to Plotly Express, introduces an overview of tidy data, as well as the high-level Plotly Express package, and shows how it uses the grammar of graphics to easily produce charts and map data to visual elements. Chapter 5, Interactively Comparing Values with Bar Charts and Dropdown Menus, goes in some depth into the available options for graphs and explores further possibilities. Then, we will see how to allow users to select which values they want to compare using drop-down menus. Chapter 6, Exploring Variables with Scatter Plots and Filtering Subsets with Sliders, moves on to one of the most frequently used chart types: scatter plots. Like we did with bar charts, we will see many different ways to customize them. Scatter plots provide even more options that we will explore, such as mapping the size of the points to a certain variable, dealing with overplotting, and handling a large number of points. Chapter 7, Exploring Map Plots and Enriching Your Dashboards with Markdown, explores a new type of chart that we see in many situations. There are many ways to plot data on maps, and we will explore two of the most frequently used types: scatter maps and choropleth maps. Chapter 8, Calculating Data Frequency and Building Interactive Tables, explores the different ways of creating histograms and customizing them, as well as splitting the data in different ways and then counting the resulting values. Chapter 9, Letting Your Data Speak for Itself with Machine Learning, shows us how clustering works, and also uses a test model to evaluate performance. We will also explore a technique to evaluate various clusters and finally, we will design an interactive app using Kmeans. Chapter 10, Turbo-charge Your Apps with Advanced Callbacks, introduces the pattern-matching callback to achieve dynamic app modification based on user interaction and various other conditions.

Preface

ix

Chapter 11, URLs and Multi-Page Apps, introduces a new architecture that allows us to incorporate multiple apps in one (each corresponding to a page in the app). The other interesting feature we will explore is using URLs either as inputs or outputs, interacting with different elements in the app. Chapter 12, Deploying Your App, shows you how to deploy your app on a server, where people can access it from anywhere so you can share it with the world. Different options are available, and we will go through two simple options that might be useful. Chapter 13, Next Steps, showcases the different options in taking your app(s) to the next level. This chapter provides some pointers on things you might want to explore.

To get the most out of this book You will need a system with a good internet connection and an AWS account. If you are using the digital version of this book, we advise you to type the code yourself or access the code via the GitHub repository (link available in the next section). Doing so will help you avoid any potential errors related to the copying and pasting of code.

Download the example code files You can download the example code files for this book from GitHub at https://github.com/PacktPublishing/Interactive-Dashboards-andData-Apps-with-Plotly-and-Dash. In case there's an update to the code, it will be updated on the existing GitHub repository. We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Download the color images We also provide a PDF file that has color images of the screenshots/diagrams used in this book. You can download it here: https://static.packt-cdn.com/ downloads/9781800568914_ColorImages.pdf.

Code in Action Code in Action videos for this book can be viewed at (https://bit.ly/3vaXYQJ).

x

Preface

Conventions used There are a number of text conventions used throughout this book. Code in text: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: "Our dataset will consist of the files in the data folder in the root of the repository."

A block of code is set as follows: import plotly.express as px gapminder = px.data.gapminder() gapminder

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold: import os import pandas as pd pd.options.display.max_columns = None os.listdir(‚data') [‚PovStatsSeries.csv', 'PovStatsCountry.csv', 'PovStatsCountry-Series.csv', 'PovStatsData.csv', 'PovStatsFootNote.csv']

Bold: Indicates a new term, an important word, or words that you see onscreen. For example, words in menus or dialog boxes appear in the text like this. Here is an example: "Another important column is the Limitations and exceptions column." Tips or important notes Appear like this.

Get in touch Feedback from our readers is always welcome. General feedback: If you have questions about any aspect of this book, mention the book title in the subject of your message and email us at [email protected].

Preface

xi

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata, selecting your book, clicking on the Errata Submission Form link, and entering the details. Piracy: If you come across any illegal copies of our works in any form on the Internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material. If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.

Reviews Please leave a review. Once you have read and used this book, why not leave a review on the site that you purchased it from? Potential readers can then see and use your unbiased opinion to make purchase decisions, we at Packt can understand what you think about our products, and our authors can see your feedback on their book. Thank you! For more information about Packt, please visit packt.com.

Section 1: Building a Dash App This section provides a general overview of the Dash ecosystem, and shows how to get started with a minimal functional app. This section comprises the following chapters: • Chapter 1, Overview of the Dash Ecosystem • Chapter 2, Exploring the Structure of a Dash App • Chapter 3, Working with Plotly's Figure Objects • Chapter 4, Data Manipulation and Preparation - Paving the Way to Plotly Express

1

Overview of the Dash Ecosystem One of the few constants in our work with data is the amount of change in the volume, sources, and types of data that we deal with. Being able to quickly combine data from different sources and explore them is crucial. Dash is not only for exploring data; it can be used for almost all phases of the data analysis process, from exploration to operational production environments. In this chapter, we will get an overview of Dash's ecosystem and focus on building the layout, or the user-facing part, of the app. By the end of the chapter, you will be able to build a running app with almost any visual component you want, but without interactivity. The following topics will be covered: • Setting up your environment • Exploring Dash and other supporting packages • Understanding the general structure of a Dash app • Creating and running the simplest app • Adding HTML and other components to the app • Learning how to structure the layout and managing themes

4

Overview of the Dash Ecosystem

Technical requirements Every chapter will have slightly different requirements, but there are some that you will need throughout the book. You should have access to Python 3.6 or higher, which can be easily downloaded from https://www.python.org, as well as a text editor or an integrated development environment (IDE) so you can edit code. For this chapter, we will be using Dash, Dash HTML Components, and Dash Bootstrap Components, which can be installed together with all other required packages by following the instructions in the following section. All code and data required for this book can be downloaded from the book's GitHub repository, which can be found at https://github.com/PacktPublishing/Interactive-Dashboards-andData-Apps-with-Plotly-and-Dash. As I just mentioned, the following section will show in detail how to get started with your setup. The code files of this chapter can be found on GitHub at https://github.com/ PacktPublishing/Interactive-Dashboards-and-Data-Apps-withPlotly-and-Dash/tree/master/chapter_01. Check out the following video to see the Code in Action at https://bit.ly/3atXPjc.

Setting up your environment With the fast pace of change in all the packages used in the book, you will most likely come across some differences in functionality, so in order to reproduce the exact outcomes described in the book, you can clone the book's repository, install the packages used (in the specified versions), and use the included dataset. From the command line, go to a folder in which you want to build the project and do the following: 1. Create a Python virtual environment in a folder called dash_project (or any other name you want). This will also create a new folder with the name you chose: python3 –m venv dash_project

2. Activate the virtual environment. On Unix or macOS, run this: source dash_project/bin/activate

On Windows, run this: dash_project\Scripts\activate.bat

Setting up your environment

5

3. Go to the created folder: cd dash_project

4. Clone the book's GitHub repository: git clone   https://github.com/PacktPublishing/ Interactive-Dashboards-and-Data-Apps-with-Plotly-and-Dash

5. You should now have a file containing the required packages and their versions called requirements.txt. You can install those packages by going to the repository's folder and running the install command as follows: cd Interactive-Dashboards-and-Data-Apps-with-Plotly-andDash/ pip install -r requirements.txt

You should find a copy of the dataset in the data folder, which was downloaded from this link: https://datacatalog.worldbank.org/dataset/poverty-andequity-database. You can get the latest version if you want, but as with packages, if you want to get the same results, it's better to work with the provided dataset. In order for Plotly figures and apps to be displayed in JupyterLab, you will need to install Node.js, which can be install from https://nodejs.org. You will also need to install the JupyterLab Plotly extension, which can be done by running the following from the command line in your virtual environment: jupyter labextension install [email protected]

Note that the version number at the end should correspond to the version of Plotly that you are running. You can replace the preceding version numbers if you want to upgrade (making sure to upgrade the Plotly Python package as well). Once you have run the preceding code, you should have everything you need to follow along. You will see that each chapter of this book builds on the previous one: we will be building an app that adds more and more functionality and complexity as we go through the chapters. The main objective is to put you in a practical setting as much as possible. In general, it is straightforward to create any standalone Dash component, but it gets more challenging when you already have a few components in a running app. This becomes clear when you have to decide how to change the layout to accommodate new changes and how to refactor code, focusing on the details without losing sight of the big picture. Now that the environment has been established, let's get an overview of Dash.

6

Overview of the Dash Ecosystem

Exploring Dash and other supporting packages Although not strictly necessary, it's good to know the main components that are used to make Dash and its dependencies, especially for more advanced usage, and in order to know how and where to get more information:

Figure 1.1 – What Dash is made of

Note One of the main advantages of using Dash is that it allows us to create fully interactive data, analytics, and web apps and interfaces, using pure Python, without having to worry about HTML, CSS, or JavaScript.

As you can see in Figure 1.1, Dash uses Flask for the backend. For producing charts, it uses Plotly, although it is not strictly required, but it is the best-supported package for data visualization. React is used for handling all components, and actually a Dash app is rendered as a single-page React app. The most important things for us are the different packages that we will be using to create our app, which we will be covering next. Tip For people who are familiar with or invested in learning Matplotlib, there is a special set of tools to convert Matplotlib figures to Plotly figures. Once you have created your figure in Matplotlib, you can convert it to Plotly with one command: mpl_to_plotly. As of the time of this writing, this is supported for Matplotlib