Test-driven web development with Python: [obey the testing goat: using Django, Selenium, and JavaScript] [1. ed] 9781449364823, 1131161181, 1449364829

Table of contents :
Cover......Page 1
Copyright......Page 3
Table of Contents......Page 4
Why I wrote a book about Test-Driven Development......Page 14
Outline......Page 16
Conventions Used in This Book......Page 17
Safari® Books Online......Page 18
Version history......Page 19
Acknowledgments......Page 21
Python 3 & programming......Page 22
Required software installations:......Page 23
Git’s default editor, and other basic Git config......Page 26
Required Python modules:......Page 27
Part I. The basics of TDD and Django......Page 30
Obey the Testing Goat! Do nothing until you have a test......Page 32
Getting Django up and running......Page 35
Starting a Git repository......Page 37
Using the Functional Test to scope out a minimum viable app......Page 40
The Python standard library’s unittest module......Page 43
Commit......Page 45
Chapter 3. Testing a simple home page with unit tests......Page 48
Unit Tests, and how they differ from Functional Tests......Page 49
Unit testing in Django......Page 50
Django’s MVC, URLs and view functions......Page 51
At last! We actually write some application code!......Page 53
urls.py......Page 55
Unit testing a view......Page 57
The unit test / code cycle......Page 58
Chapter 4. What are we doing with all these tests?......Page 62
Programming is like pulling a bucket of water up from a well......Page 63
Using Selenium to test user interactions......Page 64
Refactoring to use a template......Page 67
On refactoring......Page 71
A little more of our front page......Page 73
Recap: the TDD process......Page 74
Wiring up our form to send a POST request......Page 80
Processing a POST request on the server......Page 83
Passing Python variables to be rendered in the template......Page 84
Three strikes and refactor......Page 88
The Django ORM & our first model......Page 89
Our first database migration......Page 91
The test gets surprisingly far......Page 92
Saving the POST to the database......Page 93
Redirect after a POST......Page 96
Better unit testing practice: each test should test one thing......Page 97
Rendering items in the template......Page 98
Creating our production database with migrate......Page 100
Ensuring test isolation in functional tests......Page 106
Running just the unit tests......Page 109
Small Design When Necessary......Page 110
REST......Page 111
Implementing the new design using TDD......Page 112
Iterating towards the new design......Page 115
Testing views, templates and URLs together with the Django Test Client......Page 116
A new test class......Page 117
A new view function......Page 118
A separate template for viewing lists......Page 119
Another URL and view for adding list items......Page 121
A test class for new list creation......Page 122
A URL and view for new list creation......Page 123
Removing now-redundant code and tests......Page 124
Adjusting our models......Page 125
A foreign key relationship......Page 127
Adjusting the rest of the world to our new models......Page 128
Each list should have its own URL......Page 130
Capturing parameters from URLs......Page 131
Adjusting new_list to the new world:......Page 132
One more view to handle adding items to an existing list......Page 133
The last new URL......Page 134
The last new view......Page 135
But how to use that URL in the form?......Page 136
A final refactor using URL includes......Page 138
Part II. Web development sine qua non’s......Page 140
What to functionally test about layout and style......Page 142
Prettification: Using a CSS framework......Page 145
Django template inheritance......Page 147
Rows and columns......Page 149
Static files in Django......Page 150
Switching to StaticLiveServerCase......Page 151
Jumbotron!......Page 152
Using our own CSS......Page 153
What we glossed over: collectstatic and other static directories......Page 155
A few things that didn’t make it......Page 157
Chapter 8. Testing deployment using a staging site......Page 160
TDD and the Danger Areas of deployment......Page 161
As always, start with a test......Page 162
Getting a domain name......Page 164
Choosing where to host our site......Page 165
User accounts, SSH and privileges......Page 166
Installing Nginx......Page 167
Configuring domains for staging and live......Page 168
Deploying our code manually......Page 169
Adjusting the database location......Page 170
Creating a virtualenv......Page 172
Simple nginx configuration......Page 174
Creating the database with migrate......Page 177
Switching to Gunicorn......Page 178
Getting Nginx to serve static files......Page 179
Switching to using Unix sockets......Page 180
Using upstart to make sure gunicorn starts on boot......Page 181
Saving our changes: adding Gunicorn to our requirements.txt......Page 182
Automating:......Page 183
“Saving your progress”......Page 186
Chapter 9. Automating deployment with Fabric......Page 188
Breakdown of a fabric script for our deployment......Page 189
Trying it out......Page 193
Deploying to live......Page 194
Nginx and gunicorn config using sed......Page 196
Further reading:......Page 197
Validation FT: preventing blank items......Page 200
Skipping a test......Page 201
Splitting functional tests out into many files......Page 202
Fleshing out the FT......Page 205
Refactoring unit tests into several files......Page 206
Unit testing model validation and the self.assertRaises context manager......Page 208
Surfacing model validation errors in the view:......Page 209
Checking invalid input isn’t saved to the database......Page 212
Django pattern: processing POST request in the same view as renders the form......Page 213
Refactor: Transferring the new_item functionality into view_list......Page 214
Enforcing model validation in view_list......Page 216
The {% url %} template tag......Page 218
Using get_absolute_url for redirects......Page 219
Moving validation logic into a form......Page 222
Exploring the forms API with a unit test......Page 223
Switching to a Django ModelForm......Page 224
Testing and customising form validation......Page 225
Using the form in a view with a GET request......Page 227
A big find & replace......Page 230
Adapting the unit tests for the new list view......Page 232
Using the form in the view......Page 233
Using the form in the other view......Page 234
A helper method for several short tests......Page 235
Using the form’s own save method......Page 237
Another FT for duplicate items......Page 240
Preventing duplicates at the model layer......Page 241
A little digression on Queryset ordering and string representations......Page 243
Rewriting the old model test......Page 245
Some integrity errors do show up on save......Page 246
Experimenting with duplicate item validation at the views layer......Page 247
A more complex form to handle uniqueness validation......Page 248
Using the existing lists item form in the list view......Page 250
Starting with an FT......Page 254
Setting up a basic JavaScript test runner......Page 255
Using jquery and the fixtures div......Page 258
Building a JavaScript unit test for our desired functionality......Page 261
Columbo says: onload boilerplate and namespacing......Page 263
A few things that didn’t make it......Page 264
Live deploy......Page 266
Wrap-up: git tag the new release......Page 267
Part III. More advanced topics......Page 268
Chapter 15. User authentication, integrating 3rd party plugins, and Mocking with JavaScript......Page 270
Exploratory coding, aka “spiking”......Page 271
Front-end and JavaScript code......Page 272
The Browser-ID protocol......Page 273
The server-side: custom authentication......Page 274
De-Spiking......Page 280
A common Selenium technique: waiting for......Page 283
Reverting our spiked code......Page 284
Housekeeping: a site-wide static files folder......Page 285
Namespacing......Page 287
A simple mock to unit tests our initialize function......Page 288
More advanced mocking......Page 294
Checking call arguments......Page 297
Qunit setup and teardown, testing Ajax......Page 298
More nested callbacks! Testing asynchronous code......Page 302
A look at our spiked login view......Page 306
Testing our view by mocking out authenticate......Page 307
Checking the view actually logs the user in......Page 310
De-spiking our custom authentication back-end: mocking out an Internet request......Page 314
1 if = 1 more test......Page 315
patching at the Class level......Page 316
Beware of Mocks in boolean comparisons......Page 319
Creating a user if necessary......Page 320
Tests the get_user method by mocking the Django ORM......Page 321
Testing exception handling......Page 322
A minimal custom user model......Page 324
A slight disappointment......Page 326
Users are authenticated......Page 327
The moment of truth: will the FT pass?......Page 328
Finishing off our FT, testing logout......Page 329
Skipping the login process by pre-creating a session......Page 334
Checking it works......Page 336
Staging finds an unexpected bug (that’s what it’s for!)......Page 337
Setting up logging......Page 338
Fixing the Persona bug......Page 340
Managing the test database on staging......Page 341
A Django management command to create sessions......Page 342
Getting the FT to run the management on the server......Page 343
An additional hop via subprocess......Page 344
Using hierarchical logging config......Page 348
Wrap-up......Page 351
Why prefer “outside-in”?......Page 354
The FT for “My Lists”......Page 355
The outside layer: presentation & templates......Page 356
Moving down one layer to view functions (the controller)......Page 357
A quick re-structure of the template inheritance hierarchy......Page 358
Designing our API using the template......Page 359
Moving down to the next layer: what the view passes to the template......Page 360
The next “requirement” from the views layer: new lists should record owner......Page 361
Moving down to the model layer......Page 362
Final step: feeding through the .name API from the template......Page 364
Revisiting our decision point: the views layer depends on unwritten models code......Page 368
A first attempt at using mocks for isolation......Page 369
Using mock side_effects to check the sequence of events......Page 370
Listen to your tests: ugly tests signal a need to refactor......Page 372
Keep the old integrated test suite around as a sanity-check......Page 373
Thinking in terms of collaborators......Page 374
Moving down to the forms layer......Page 378
Keep listening to your tests: removing ORM code from our application......Page 379
Finally, moving down to the models layer......Page 382
Back to views......Page 384
The moment of truth (and the risks of mocking)......Page 385
Thinking of interactions between layers as “contracts”......Page 386
Identifying implicit contracts......Page 387
Fixing the oversight......Page 388
One more test......Page 389
Removing redundant code at the forms layer......Page 390
Removing the old implementation of the view......Page 391
Removing redundant code at the forms layer......Page 392
Let complexity be your guide......Page 393
Onwards!......Page 394
Installing Jenkins......Page 396
Configuring Jenkins security......Page 397
Adding required plugins......Page 398
Setting up our project......Page 400
First build!......Page 401
Setting up a virtual display so the FTs can run headless......Page 403
Taking screenshots......Page 405
A common Selenium problem: race conditions......Page 408
Running our Qunit JavaScript tests in Jenkins with PhantomJS......Page 411
Installing node......Page 412
Adding the build steps to Jenkins......Page 413
More things to do with a CI server......Page 414
An FT with multiple users, and addCleanup......Page 416
Implementing the Selenium interact/wait pattern......Page 418
The Page pattern......Page 419
Extend the FT to a second user, and the My Lists page......Page 422
An exercise for the reader......Page 424
Chapter 22. Fast tests, slow tests and Hot Lava......Page 426
Faster tests mean faster development......Page 427
Don’t take it from me......Page 428
Mocky tests can become closely tied to implementation......Page 429
Clean, maintainable code......Page 430
Architectural solutions......Page 431
Functional Core, Imperative Shell......Page 432
Conclusion:......Page 433
Obey the Testing Goat!......Page 434
Running Firefox Selenium sessions with Xvfb......Page 436
Cleaning up /tmp......Page 437
The deployment chapter......Page 438
Class-based generic views......Page 440
The home page as a FormView......Page 441
Using form_valid to customise a CreateView......Page 442
The tests guide us, for a while......Page 444
Until we’re left with trial and error......Page 445
Back on track......Page 446
Best practices for unit testing CBGVs?......Page 447
Take-home: having multiple, isolated view test with single assertions helps......Page 448
Appendix C. Provisioning with Ansible......Page 450
Installing system packages and nginx......Page 451
Configuring gunicorn, and using handlers to restart services......Page 452
Use Vagrant to spin up a local VM......Page 453
Switch to Postgres......Page 454
Investigate a BDD tool......Page 455
Async and websockets......Page 456
Your suggestion here......Page 457
An attempted deploy to staging......Page 458
Copying test data from the live site......Page 459
Inserting a data migration......Page 460
Testing the new migrations together......Page 461
Conclusions......Page 462
Appendix F. Bibliography......Page 464
Index......Page 466
About the Author......Page 467