Testing Without Testing

Testing Without Testing by James Bach 26.10.2012

This time on the road were Kjell Lauren (@klauren69) and Ismo Paukamainen (@IsmoPaukamainen)


– Testing without… what?
– The appearance of testing without the substance of testing?
– Noooo….
– The substance of testing without the appearance* of testing
*appearance means “appearance to people who aren’t testers”

– Testing is….

  • Creating the conditions necessary to…
  • question a product in order to evaluate it, including by…
  • operating a product to check specific facts about it
  • …so that our clients can make informed decisions about risk
  • perhaps help make the product better, too

– Call this “Checking” not testing

  • “operating a product to check specific facts about it” means
  • Operate – Perform specific, pre-defined actions with the product
  • Compare – Check product behaviour against specific expected result
  • Report – Report any failed tests
  • Test automation is actually not about test, but checking. You cannot automate tests, but only checks. Machine cannot make decisions that requires human thinking.

Things that testers do that non-testers don’t know or see:
– Inner circle

  • Playing with the product
  • Tool operation and maintenance
  • Bug investigation
  • Interpretive test reporting
  • Discovery of non-bugs
  • Productive failure
  • Galumphing, see definition here
  • Guiding helpers
  • Consistency oracles
  • Tacit test procedures
  • Configuring product
  • Designing tests

– Outer circle

  • Prospective, simulated, or simulation testing
  • Testing logistics management
  • Archival documentation
  • Bug triage
  • Project Mortems
  • Study customer feedback
  • Schedule management
  • Building the test team
  • Test tooling and artifact development
  • Relationship building
  • Commitment management (inc estimation)
  • Learning and teaching
  • Recruiting helpers

– How do yo know you’re testing?
– How to fake a test project presentation, see presentation here
– Testing books are full of non-sense
– Testers usually have an intuition and don’t need the testing books
– Testers should read books but not necessary testing books, e.g. art books are good to read, or books about different models. Testers usually have an intuition to do the right things without knowing what it is called in the literature.
– Here you can find some books recommended to read by James Bach
– Don’t say you assume, say you don’t know
– Instead of assuming, do tests to find out how it works
– If I have evidence I don’t need to make an assumption
– Assumption vs. Inference
– assume = to treat something as true without sufficient evidence
– inference = the act or process of deriving logical conclusions from premises known or assumed to be true
– Not all differences matter to the same degree  = equivalence partitioning in other words = in practise
– Equivalence class partitioning, see definitions here and here
– Testing is all about different models  (=mental models, modelling and learning)
– Novices need experiences

You need to learn about things to be able to see them clearly!
Treat your profession with respect!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s