As part of the process of implementing Jenkins into our workflows at work, I have set up two sets of automated end to end tests.

The first is a small batch of 10 orders which runs every morning after an automated deployment to the test environment (using a post-build action on the deployment job). This batch of 10 contains 2 which should cause alerts to be raised, and 8 which should succeed and contains locations which have caused us issues in the past. These should all have completed before our working day begins and so gives us the confidence that all is well in our test environment.

The second is a batch of 50 orders, which runs on a weekend to act as more of a bulk testing mechanism. The 50 orders were mostly statically generated across the country. I was concerned that this data was concentrating on the wrong areas as it was using a fairly even distribution of points across the country, rather than concentrating on the hot spots of actual use.

So, the other week I took a few hours to map where in the country the bulk of our live orders are coming from. I made use of the GMap.Net library to do this plotting latitude and longitudes (converted from our native eastings and northings with help from this script). Here’s what it looked like:

Live Orders

Having seen this, and the density of some of the areas, I confirmed my suspicion that there were some parts of the country that should be targeted.

So, I adjusted my test scripts to allow me to make more targeted random selections - if that makes any sense at all. I’ve broken the country down into rectangles (with a lot of help from the excellent UK Grid Reference Finder), and have now got the script chosing a number of random points within that rectangle - so, central London now gets 6 test orders, whilst a rectangle covering a northern England area gets 5 orders. Over time this should mean that our coverage becomes closer to the live usage patterns, which should help us to be surer that our systems are doing what they should be against a more realistic dataset.

Here’s the points used over the weekend:

Test Orders

which has made me realise that I’m still over-targeting Wales and Scotland and could do with some more around Newcastle and Birmingham. Those changes will be made before the next batch is run through.