Advanced testing tools and development techniques

What is Testing?

Testing can be stated as the process of validating and verifying that a computer program/application/product works as expected

How people used to test

• You write your code according to your requirements
• After it is done, you write a test to ensure that your code does what it is supposed to do
• A test is a piece of code that checks the functionality of other code
• If anything inside your code changes, the tests will fail und you know what to fix

Problems with this approach

• Tests are often neglected
• Not everything is tested. Therefore test coverage is often bad
• This means that not every code change can be guaranteed to have no negative effects
• After 3 years of working on a piece of software you will not be able to tell if a small change will break a lot of your code
• And no, you will never add these missing tests "later"

What is the solution?

Behavior-Driven-Development (BDD)

BDD is a software development technique that encourages collaboration between developers, quality assurance and non-technical or business participants in a software project

• You define what your code needs to do before you start coding (Behavior)
• All stakeholders should be involved in this process
• You write a test for your code before you code
• You then proceed to write the code according to the test
• Test-First approach

We have a tool for that: Cucumber

Cucumber reads plain-text descriptions of application features (tests) with example scenarios and uses the scenario steps to automate interaction with the code being developed.

It helps us write great tests before we start coding!

• Use customer acceptance tests to drive the development of code
• Ideally, these are the result of a collaborative effort between the customer and the delivery team
• Cucumber gives us a language and format to write them down
• A test in Cucumber is called a feature

Example Feature

 1Feature: pay bill on-line
 2
 3  In order to reduce the time I spend paying bills
 4  As a bank customer with a checking account
 5  I want to pay my bills on-line
 6
 7  Scenario: pay a bill
 8    Given checking account with $50
 9    And a payee named Acme
10    And an Acme bill for $37
11    When I pay the Acme bill
12    Then I should have $13 remaining in my checking account
13    And the payment of $37 to Acme should be listed in Recent Payments

How does it work?

• Every line of the written out feature defines a function our code needs to run
• The lines also provide an expected outcome (e.g. $13 remaining)
• Cucumber is smart enough to translate this text into a test scenario
• This way we not only have great tests, but they are fully automated as well

"Besides the application code, the infrastructure deserves tests too!"

How people used to test infrastructure

• Building a server was to follow a checklist
• A quality assurance engineer would verify that everything was okay
• Basically by checking config files or running services

Speeding it up

• Configuration Management or Infrastructure as a Service decreased the time needed to build machines
• They also increased the quality of the systems built
• But there are still manual tests involved

How can we automate this to avoid mistakes?

Test Kitchen for testing infrastructure code

What is Test Kitchen?

• A test tool to execute your configured code on one or more platforms in isolation
• Automates the testing process, to verify that your infrastructure code has done the right thing
• Works with Chef Cookbooks
• Lets you run your infrastructure tests on OpenStack

Example

• You use BDD to write tests for your Chef Cookbooks (e.g. DB Server )
• When starting a test, Test Kitchen will provision a fresh VM
• It will then setup the Cookbook on this VM
• Then it will check if the DB is working correctly, if the correct ports are listening, if user rights were granted correctly etc.

Example Postgres Serverspec

 1  # Req. 19: ssl_ciphers = 'ALL:!ADH:!LOW:!EXP:!MD5:@STRENGTH'
 2  describe file(postgres_config_file) do
 3    its(:content) do
 4      should match_key_value('ssl_ciphers', "'ALL:!ADH:!LOW:!EXP:!MD5:@STRENGTH'")
 5    end
 6  end
 7
 8  # Req. 6: password_encryption = on
 9  describe file(postgres_config_file) do
10    its(:content) { should match_key_value('password_encryption', 'on') }
11  end

"With Test Kitchen we can make sure that our server configurations are also tested: No manual testing anymore!"