Test-driven web development with Python: [obey the testing goat: using Django, Selenium, and JavaScript] [1. ed] 9781449364823, 2112122142, 1449364829

Table of contents :
Cover......Page 1
Copyright......Page 4
Table of Contents......Page 5
Why I Wrote a Book About Test-Driven Development......Page 15
Outline......Page 17
Conventions Used in This Book......Page 18
Safari® Books Online......Page 19
Contacting O’Reilly......Page 20
Python 3 and Programming......Page 21
Required Software Installations......Page 22
Required Python Modules......Page 25
Acknowledgments......Page 27
Part I. The Basics of TDD and Django......Page 29
Obey the Testing Goat! Do Nothing Until You Have a Test......Page 31
Getting Django Up and Running......Page 34
Starting a Git Repository......Page 36
Using a Functional Test to Scope Out a Minimum Viable App......Page 41
The Python Standard Library’s unittest Module......Page 44
Commit......Page 46
Chapter 3. Testing a Simple Home Page with Unit Tests......Page 49
Unit Tests, and How They Differ from Functional Tests......Page 50
Unit Testing in Django......Page 51
Django’s MVC, URLs, and View Functions......Page 52
At Last! We Actually Write Some Application Code!......Page 54
urls.py......Page 55
Unit Testing a View......Page 58
The Unit-Test/Code Cycle......Page 59
Chapter 4. What Are We Doing with All These Tests?......Page 63
Programming Is like Pulling a Bucket of Water up from a Well......Page 64
Using Selenium to Test User Interactions......Page 65
Refactoring to Use a Template......Page 68
On Refactoring......Page 72
A Little More of Our Front Page......Page 73
Recap: The TDD Process......Page 75
Wiring Up Our Form to Send a POST Request......Page 79
Processing a POST Request on the Server......Page 82
Passing Python Variables to Be Rendered in the Template......Page 83
Three Strikes and Refactor......Page 87
The Django ORM and Our First Model......Page 88
Our First Database Migration......Page 90
The Test Gets Surprisingly Far......Page 91
A New Field Means a New Migration......Page 92
Saving the POST to the Database......Page 93
Better Unit Testing Practice: Each Test Should Test One Thing......Page 96
Rendering Items in the Template......Page 97
Creating Our Production Database with migrate......Page 99
Ensuring Test Isolation in Functional Tests......Page 105
Running Just the Unit Tests......Page 108
Small Design When Necessary......Page 109
REST......Page 110
Implementing the New Design Using TDD......Page 111
Iterating Towards the New Design......Page 114
Testing Views, Templates, and URLs Together with the Django Test Client......Page 115
A New URL......Page 116
A New View Function......Page 117
A Separate Template for Viewing Lists......Page 118
Another URL and View for Adding List Items......Page 120
A Test Class for New List Creation......Page 121
A URL and View for New List Creation......Page 122
Removing Now-Redundant Code and Tests......Page 123
Pointing Our Forms at the New URL......Page 124
Adjusting Our Models......Page 125
A Foreign Key Relationship......Page 127
Adjusting the Rest of the World to Our New Models......Page 128
Each List Should Have Its Own URL......Page 130
Capturing Parameters from URLs......Page 131
Adjusting new_list to the New World......Page 132
One More View to Handle Adding Items to an Existing List......Page 133
The Last New URL......Page 134
The Last New View......Page 135
But How to Use That URL in the Form?......Page 136
A Final Refactor Using URL includes......Page 138
Part II. Web Development Sine Qua Nons......Page 141
What to Functionally Test About Layout and Style......Page 143
Prettification: Using a CSS Framework......Page 146
Django Template Inheritance......Page 148
Integrating Bootstrap......Page 149
Static Files in Django......Page 150
Switching to StaticLiveServerCase......Page 152
Large Inputs......Page 153
Using Our Own CSS......Page 154
What We Glossed Over: collectstatic and Other Static Directories......Page 155
A Few Things That Didn’t Make It......Page 158
Chapter 8. Testing Deployment Using a Staging Site......Page 159
TDD and the Danger Areas of Deployment......Page 160
As Always, Start with a Test......Page 161
Getting a Domain Name......Page 163
Choosing Where to Host Our Site......Page 164
User Accounts, SSH, and Privileges......Page 165
Installing Nginx......Page 166
Using the FT to Confirm the Domain Works and Nginx Is Running......Page 167
Deploying Our Code Manually......Page 168
Adjusting the Database Location......Page 169
Creating a Virtualenv......Page 170
Simple Nginx Configuration......Page 172
Creating the Database with migrate......Page 175
Switching to Gunicorn......Page 176
Getting Nginx to Serve Static Files......Page 177
Switching to Using Unix Sockets......Page 178
Using Upstart to Make Sure Gunicorn Starts on Boot......Page 179
Automating......Page 180
“Saving Your Progress”......Page 184
Chapter 9. Automating Deployment with Fabric......Page 185
Breakdown of a Fabric Script for Our Deployment......Page 186
Trying It Out......Page 190
Deploying to Live......Page 191
Nginx and Gunicorn Config Using sed......Page 193
Further Reading......Page 194
Validation FT: Preventing Blank Items......Page 197
Skipping a Test......Page 198
Splitting Functional Tests out into Many Files......Page 199
Fleshing Out the FT......Page 202
Refactoring Unit Tests into Several Files......Page 203
Unit Testing Model Validation and the self.assertRaises Context Manager......Page 205
Surfacing Model Validation Errors in the View......Page 206
Checking Invalid Input Isn’t Saved to the Database......Page 209
Django Pattern: Processing POST Requests in the Same View as Renders the Form......Page 211
Refactor: Transferring the new_item Functionality into view_list......Page 212
Enforcing Model Validation in view_list......Page 214
Refactor: Removing Hardcoded URLs......Page 215
Using get_absolute_url for Redirects......Page 216
Moving Validation Logic into a Form......Page 221
Exploring the Forms API with a Unit Test......Page 222
Switching to a Django ModelForm......Page 223
Testing and Customising Form Validation......Page 224
Using the Form in a View with a GET Request......Page 226
A Big Find and Replace......Page 229
Adapting the Unit Tests for the new_list View......Page 231
Using the Form in the View......Page 232
Using the Form in the Other View......Page 233
A Helper Method for Several Short Tests......Page 234
Using the Form’s Own Save Method......Page 236
Another FT for Duplicate Items......Page 239
Preventing Duplicates at the Model Layer......Page 240
A Little Digression on Queryset Ordering and String Representations......Page 242
Rewriting the Old Model Test......Page 244
Some Integrity Errors Do Show Up on Save......Page 245
Experimenting with Duplicate Item Validation at the Views Layer......Page 246
A More Complex Form to Handle Uniqueness Validation......Page 247
Using the Existing List Item Form in the List View......Page 249
Starting with an FT......Page 253
Setting Up a Basic JavaScript Test Runner......Page 254
Using jQuery and the Fixtures Div......Page 257
Building a JavaScript Unit Test for Our Desired Functionality......Page 260
Columbo Says: Onload Boilerplate and Namespacing......Page 262
A Few Things That Didn’t Make It......Page 263
Live Deploy......Page 265
Wrap-Up: git tag the New Release......Page 266
Part III. More Advanced Topics......Page 267
Chapter 15. User Authentication, Integrating Third-Party Plugins, and Mocking with JavaScript......Page 269
Exploratory Coding, aka “Spiking”......Page 270
Frontend and JavaScript Code......Page 271
The Browser-ID Protocol......Page 272
The Server Side: Custom Authentication......Page 273
De-spiking......Page 279
A Common Selenium Technique: Explicit Waits......Page 281
Reverting Our Spiked Code......Page 283
Housekeeping: A Site-Wide Static Files Folder......Page 284
Mocking: Who, Why, What?......Page 285
A Simple Mock to Unit Tests Our initialize Function......Page 286
More Advanced Mocking......Page 292
Checking Call Arguments......Page 295
QUnit setup and teardown, Testing Ajax......Page 296
More Nested Callbacks! Testing Asynchronous Code......Page 300
A Look at Our Spiked Login View......Page 305
Testing Our View by Mocking Out authenticate......Page 306
Checking the View Actually Logs the User In......Page 309
De-spiking Our Custom Authentication Backend: Mocking Out an Internet Request......Page 313
1 if = 1 More Test......Page 314
Patching at the Class Level......Page 315
Beware of Mocks in Boolean Comparisons......Page 318
The get_user Method......Page 319
A Minimal Custom User Model......Page 321
A Slight Disappointment......Page 323
Tests as Documentation......Page 324
Users Are Authenticated......Page 325
The Moment of Truth: Will the FT Pass?......Page 326
Finishing Off Our FT, Testing Logout......Page 327
Skipping the Login Process by Pre-creating a Session......Page 331
Checking It Works......Page 333
The Proof Is in the Pudding: Using Staging to Catch Final Bugs......Page 334
Setting Up Logging......Page 335
Fixing the Persona Bug......Page 337
A Django Management Command to Create Sessions......Page 339
Getting the FT to Run the Management Command on the Server......Page 340
An Additional Hop via subprocess......Page 342
Baking In Our Logging Code......Page 345
Using Hierarchical Logging Config......Page 346
Wrap-Up......Page 348
Why Prefer “Outside-In”?......Page 351
The FT for “My Lists”......Page 352
The Outside Layer: Presentation and Templates......Page 353
Moving Down One Layer to View Functions (the Controller)......Page 354
A Quick Restructure of the Template Inheritance Hierarchy......Page 355
Designing Our API Using the Template......Page 356
Moving Down to the Next Layer: What the View Passes to the Template......Page 357
The Next “Requirement” from the Views Layer: New Lists Should Record Owner......Page 358
Moving Down to the Model Layer......Page 359
Final Step: Feeding Through the .name API from the Template......Page 361
Revisiting Our Decision Point: The Views Layer Depends on Unwritten Models Code......Page 365
A First Attempt at Using Mocks for Isolation......Page 366
Using Mock side_effects to Check the Sequence of Events......Page 367
Listen to Your Tests: Ugly Tests Signal a Need to Refactor......Page 369
Keep the Old Integrated Test Suite Around as a Sanity Check......Page 370
Thinking in Terms of Collaborators......Page 371
Moving Down to the Forms Layer......Page 375
Keep Listening to Your Tests: Removing ORM Code from Our Application......Page 376
Finally, Moving Down to the Models Layer......Page 379
Back to Views......Page 381
The Moment of Truth (and the Risks of Mocking)......Page 382
Thinking of Interactions Between Layers as “Contracts”......Page 383
Identifying Implicit Contracts......Page 384
Fixing the Oversight......Page 385
One More Test......Page 386
Removing Redundant Code at the Forms Layer......Page 387
Removing the Old Implementation of the View......Page 388
Removing Redundant Code at the Forms Layer......Page 389
Conclusions: When to Write Isolated Versus Integrated Tests......Page 390
Onwards!......Page 391
Installing Jenkins......Page 393
Configuring Jenkins Security......Page 395
Adding Required Plugins......Page 396
Setting Up Our Project......Page 397
First Build!......Page 399
Setting Up a Virtual Display so the FTs Can Run Headless......Page 400
Taking Screenshots......Page 402
A Common Selenium Problem: Race Conditions......Page 406
Running Our QUnit JavaScript Tests in Jenkins with PhantomJS......Page 409
Installing node......Page 410
Adding the Build Steps to Jenkins......Page 411
More Things to Do with a CI Server......Page 412
An FT with Multiple Users, and addCleanup......Page 415
Implementing the Selenium Interact/Wait Pattern......Page 417
The Page Pattern......Page 418
Extend the FT to a Second User, and the “My Lists” Page......Page 421
An Exercise for the Reader......Page 423
Chapter 22. Fast Tests, Slow Tests, and Hot Lava......Page 425
Faster Tests Mean Faster Development......Page 426
Don’t Take It from Me......Page 427
Mocky Tests Can Become Closely Tied to Implementation......Page 428
Clean, Maintainable Code......Page 429
Architectural Solutions......Page 430
Functional Core, Imperative Shell......Page 431
Conclusion......Page 432
Obey the Testing Goat!......Page 435
Running Firefox Selenium Sessions with Xvfb......Page 437
Setting Up Django as a PythonAnywhere Web App......Page 438
The Deployment Chapter......Page 439
Class-Based Generic Views......Page 441
The Home Page as a FormView......Page 442
Using form_valid to Customise a CreateView......Page 443
The Tests Guide Us, for a While......Page 445
Until We’re Left with Trial and Error......Page 446
Back on Track......Page 447
Best Practices for Unit Testing CBGVs?......Page 448
Take-Home: Having Multiple, Isolated View Tests with Single Assertions Helps......Page 449
Installing System Packages and Nginx......Page 451
Configuring Gunicorn, and Using Handlers to Restart Services......Page 453
Use Vagrant to Spin Up a Local VM......Page 454
An Attempted Deploy to Staging......Page 455
Copying Test Data from the Live Site......Page 456
Inserting a Data Migration......Page 457
Testing the New Migrations Together......Page 458
Conclusions......Page 459
Switch to Postgres......Page 461
Investigate a BDD Tool......Page 462
Async and Websockets......Page 463
Your Suggestion Here......Page 464
The Basic TDD Workflow......Page 465
Moving Beyond dev-only Testing......Page 466
Selenium/Functional Testing Best Practices......Page 467
Outside-In, Test Isolation Versus Integrated Tests, and Mocking......Page 468
Appendix G. Bibliography......Page 469
Index......Page 471
About the Author......Page 478