The Future of Software Testing?

Well it’s Xmas, and there are a lot of programmes on the TV about the best of 2005 as well as people speculating about the market trends, house prices and economics of 2006. Who am I to leave a good band-wagon behind.......?

Part of my role as a Managing Director is to look forward and see what 'might be', where TCL could fit into it and where we can add value to our clients. For the last 6 years we have done relatively well in this regard, but I am posed by more of a challenge this year...in fact I would say that my answer is still a work in progress!

Primarily, my views are that the convergence of Companies, Commercial Propositions and subsequently systems is driving a lot of the IT change we are involved with. It seems to be a world wide, multi sector trend, and areas such as VoIP, Mobile, Data, Fixed line and ISP convergence is a good example of how things are unifying. Let me give you a highlight from BT. You can now take out a Mobile tariff and join it with an ISP, landline and VoIP bill. The systems you have at home enable you to have a wireless (and wired) LAN, and when you come home your mobile will act as your home (fixed line) phone - routing calls through VoIP or your fixed line as desired. It’s brilliant because its simple (almost seamless) for the user, it’s cost effective and it’s easy to buy (online).

The challenges we face as testers of this kind of technology include:
The size of it all - things are getting smaller. Usability requirements are very different. The scale of the interfaces and joined up thinking across the organisation though is much much bigger. Needing good solid design work to make it a reality rather than a travesty.
The Complexity - lots of things in the same place. To test it you have to know a lot more, about a lot more things
Time to Market - the competition is fiercer than it used to be and time to market drivers are more acute in a lot more of the sectors. A statistic I was given at a recent networking dinner was that a new software product is outdated/superseded or competitively out priced within 16 months of its launch. Projects cannot last a year or two in domains like that
Data - lots more data in even embedded systems. The performance of the systems handling this data and the security / integrity of the data in the system and more often now across the Internet is on everyone’s minds - although not necessarily in the design!
Price - Excessive pricing is no longer tolerated. Quality and Innovation may be differentiators but in markets such as mobile telcos where there is near customer saturation the cost mark (even for very innovative products) is essential to get right.

So how does this shape the future of testing? It would point towards a necessity for more Non Functional Testing, for better, quicker design and for testers that can do more and know more about a lot more things. The future of software testing therefore is rooted in proactive, professional development. Significant specialism in some areas and amazingly competent all-rounders for others.

The thing is though; there is a doubt in my mind generated by History. Love it or loathe it, you just can’t ignore history. You see when I first started in testing I was lucky enough to work in defence. We had a very mature product, in a very mature industry and the testing processes had been evolved over 20+ years to something akin to clockwork. Many of the key features brought up at today’s conferences were already common practise even then. Regression testing, testing of requirements, TTRM/VCRIs etc all sorted. So where's the new stuff? Where's our evolution? How are so many people working in 'testing' by doing exactly the same thing every day?

We thought that automation was the key to the future, and indeed I have heard that said many times in the last ten years. But look around and you see that circa 85% of all testing is still manual. So what happened? Were there no market drivers to encourage time to market initiatives, innovative approaches or change? Did no-one do this? Well a lot of us did, but why didnt it become a uniform approach?

And finally, we have all seen the need for better professional development within testing but after all this time there are still very few Universities who even mention it as part of a Computer Science Degree let alone Research and Develop it. Certification courses really are in their infancy and we spend a great deal of time reminding people that testers have to be able to test not just push paper. There isn’t an industry body, a globally accepted and common practise standard and there are commercial interests getting in the way of the true development of better tools, techniques and practises.

So if History has taught me anything its that Software Testing has been pretty much stagnant over the last 10 years (In a way like the surface of the earth seems flat from space) . An industry of silos for a long time and in fact the last 10 to 15 years haven’t really got us substantially further forward as a community.

This leads me to an interesting thought....
For me Software Testing will only have a different future from what we do now if there is a period of commercial consolidation and consistent governance/control of standardisation. This isn’t needed just in the UK, but worldwide. There is enough financial reward in software testing for this to start happening, and indeed some companies in software testing have now floated - generating capital to invest in such consolidation. When this starts to happen we might just see the merging of practices across industries and sectors generating a new level of collective competence and standards which will drive the way we work, the skills we need and the development of people that the industry needs to get on a par with other areas of IT, and perhaps start to lead the way.

The future of Software Testing in my eyes is in Unification and collective advancement. Let's see if 2006 proves me right.

Merry Xmas 2005 Everyone. I wish you all the best for 2006, and look forward to meeting up with many of you at StarEAST in May.
Stew

Certification Programmes

Well my colleagues seem to be debating the pros and cons of certification programmes at the moment, and I have a couple of views on that so here goes......

I feel that the primary reason certification programmes exist is to set a standard, and this is a good thing. Testing has historically been seen as the IT backwater - where you go if you cant do anything well, or they cant think where else to put you. This is / was a reputation that was not good for testing as a whole, and any move to systematically remove that poor perception of what we do is something I welcome.

There are three areas that I feel are a bit astray at the moment with regards to certification:

1. Practical Skills - Most importantly testing is a practical subject. It can be argued therefore that any certification programme without practical, useful, applicable skills is not really good enough. At present exams like ISEB Foundation and Practitioner don't seem to contain any 'hands on' stuff - and this diminishes the value of the qualification and thus the very standard (or 'bar') we were trying to set is perceived as lower.
2. Standards Body - Any standards body will be having a headache trying to make a one size fits all programme that incorporates aptitude, practical and theory across development methodologies, industries and even countries. The key here however is consultation and I don't know how you can get involved in influencing and changing the certification programmes for the better.
3. Commercial Interruption - My feelings on free trade and the removal of commercial barriers are very positive in general, but when it comes to certification and qualification programmes I feel that commercial interests should take a back seat. The current state of our market place, the money to be gained from training programmes and the commercial interests of some people who are involved with the certification side of things has, in my opinion corrupted areas of the usefulness of the current courses. There are areas of study within these courses devoted to commercial, trade marked methods and this goes a long way to creating an improper social construct of reality that these methods are a standard in their own right. Also for a majority of training providers the focus is on churn rather than quality of delivery and this generates content to the courses and behaviors in teaching that are improper.

So, what I would like to see happening is for the certification programmes to be more deeply routed into a more academic mind set. One where the standards would be set to achieve the right balance between Aptitude, Practical Skills and Theory - and they should link directly to / or perhaps from - qualifications such as University Degree courses. This will help to set a uniform bar without as much commercial influence.

Most importantly though, it must be time for the certification groups to extend consultation groups into centres of testing excellence and get some practical skills and measures into the programmes. Let people learn to test and show how good they are. Lets build on the idea of being an expert tester and show people how to approach anything, absolutely anything, with a testing mindset. Lets see more people adopt the tactics of James Bach, John Bach, James Lyndsay, Mark Garnett and James Whittaker in bringing people down this path and lets see some measure of this skill that shows how valuable it is. After all it isn't everyone that can do it!

In summary then, I feel that these certification programmes are a good start,
but they just aren't good enough right now. Time for a change. Time to
prove that testing is BRILLIANT and Expert Testers have a real,
valuable place in the IT landscape.

What do you think? Let me know your opinion - or take a look at the Blogs of Mark Garnett and/or James Bach to see some more opinions on this topic.

Security Testing Training - Public Course (March 2006)

Just spent the day out in Paris today, which has been good fun.

This time of year seems to agree with Paris. Lots of hussle and bussle and the Xmas lights look great.

While I have been out here I have made a provisional agreement for TCL to host a public training course in Security Testing, located in London (UK), in March 2006.

If anyone is interested then there will be something on the TCL website early next week (www.TransitionConsulting.co.uk) and if you are really keen to get involved then you can contact Katie at our Exeter office (+44 1392 262 343) to register your interest and provisionally book a place.

I am very excited about the course because it is something we are putting together as part of a joint venture, and it is the first part of our journey into the security testing space in 2006. Its an area I personally find very interesting, but dont currently have much ability in :) Hopefully I will learn something as the course will be delivered by experts!

A public course means that there will be a mixed group of people to meet with and share ideas. With a bit of luck it might feel a bit like a mini conference, over two days.