Testing in Production

@benjiweber and @pr0bablyfine

Testing
Checking

Discrete Delivery

Pair Programming

Who you gonna call?

Structure

  • Testing in Production
  • Monitoring is Checking
  • Sabotaging Production

Testing in Production

User Testing

Customer Signoff

Exploratory Testing

Data Toggles

Product Experiments

Acceptance Testing…

Monitoring is Checking

Monitoring Value

over

Server Monitoring

Run Tests in Production


  class adunit_monitoring {
    common::nrpe::config_cmd { 'check_sponsored_text':
      cmd => "java -jar acceptancetests.jar ad-sponsored-text"
    }

    common::nrpe::config_cmd { 'check_sharing_icons':
      cmd => "java -jar acceptancetests.jar ad-sharing"
    }
    
    common::nrpe::config_cmd { 'check_click_through':
      cmd => " protractor /path/protractor-config.js --specs clickthrough.js"
    }
  }
                    

Many reasons for

System changes, Not just Deploys

Time

Users

Third Parties

Traffic

Infra

Ticking Timebomb

Monitoring Driven Development

#MDD

Smells

Private State

Repetition

Non-Determinism

Heisentests

Failure Injection

Prod is the only source of truth

Throughput

Latency

Bottleneck detection

Isn't that Risky?

Checks vs Properties

Wrap-Up

Monitoring
==
Tests

#mdd

If you don't break your system, someone else will

Thanks for Listening!

@benjiweber @pr0bablyfine

Any questions?