Archive for the ‘Testing’ Category

Throughout industry, whether business led or IT led, the mantra still seems to be if you are not ‘agile’ then you can’t survive or rather succeed in today’s market place. But what does ‘agile’ mean? In my role, I speak to numerous companies, whether in a business development capacity or networking at industry events or after presenting at conferences. The views of the World are polar opposite, sometimes disingenuous or contradictory to manifesto guidelines. Some of the questions and statements that I hear on a regular basis include:

  • “Do you develop products or technology in an agile way?”
  • “Does your company adopt business agility models?”
  • “Are you a Scrum fan or Kanban? “
  • “Actually we operate lean principles “
  • “We’re not quite there yet, so we are more scrumfall than anything”
  • “It sounds good, but it’s just so difficult to be heard”
  • “We have a daily stand up every day”
  • “The business/IT Management just don’t get it”
  • “The pressure of multiple releases is just intense”
  • And my personal favourite “We’re so agile, we go beyond agile”

The objective of agile is no different to the objective of more traditional approaches I.e. a working product (Quality), delivered on or under budget (Cost) in an acceptable timeframe (Time) to gain advantage over competition, by increasing company reputation and trust as the ideal solution provider.

There is a lot of talk in blogs regarding how ‘agile’ must adapt to keep up with the ever changing demands of the marketplace. I’m not sure that I completely agree with that, from the examples above there is already confusion in the marketplace, so change for the sake of change may make the gulf in ‘maturity’ even wider. A good example of change, is the kudos that DevOps brings. If you ask 10 different people how to describe agile, you’ll get many different views. If you ask the same people to describe DevOps you’ll likely receive more than 10 views, as was proven in my office when we discussed what our testing offering could offer to a DevOps environment. Everyone had a view about what it was, though the biggest consensus gained was how to spell it!

For me, that is the bottom line, if your project, company or industry wants to operate in an agile way or adopt a certain flavour of agile, you need to lay the foundations, in affect define your individual manifesto and publicise it

“Working software over documentation” could translate to “working software with just enough documentation to meet our regulatory demands”

I could go on, but I think the message here is clear, to be agile, you need to collaborate. To collaborate, you need to know what you are working towards, define a top to bottom understanding within your organisation or project team to enable change, transform your approach, and deliver success, and if you want to badge this as ‘agile’ that’s fine too.

Working for a Global consultancy like Sogeti UK, where the majority of our business is in the testing space, when we are asked to assist with introducing agile testing to engagements, we look at the bigger picture. Trying to understand the landscape of the project and the company will determine the type of consultants that we deploy to the engagement. If a client needs to set their own manifesto, there isn’t much reason to deploy highly technical testers, as reputation could suffer if collaboration or rather the mechanics of the project are not smooth and delays occur or releases dates missed as the end game wasn’t known. Instead we’d look to utilise more rounded consultants that can embed into teams and coach and steer the team or company to defining “what looks good for us?” Setting the manifesto and expectations for all to understand and commit and collaborate too.

At Sogeti, we look to help companies achieve their own manifesto, through the use of workshops, involving senior stakeholders through to junior testers, with the objective of aligning objectives, and embedding consultants to assist companies with their transformation journey, to define an individual working ‘agile’ approach, irrespective of ‘flavour’ applied. At every step of the way, looking to support the needs as maturity I.e. Success is gained, to move to the seeming utopia of DevOps (I think that was the agreed spelling!)

If you’d like to find out more about the services Sogeti UK can offer, visit our website or drop me a line.


A nice read on the benefits of Testing in production from InfoQ and shared by

Having spent two days at London’s ExCel conference centre at AppsWorld; it was great to see the level of advancement and constant change that is prevalent across various industries. Technology is playing an ever increasing role in industry, and the panel discussions that pulled various contributors from different areas was a great insight to how similar the challenges of keeping up with the consumerisation affect is having on IT programmes.

The conference focussed on open content areas; such as:

  • Developer World
  • Droid World
  • Cloud World
  • Connected Car
  • Gaming World
  • Enterprise World

In addition there were a series of premium tracks that covered a wide array of topics:

  • Mobil Payments & Retail
  • Mobile Strategy & Marketing
  • TV & Multi-Screen Apps
  • API Strategies
  • Wearable App Tech
  • HTML5 Workshops

The use of APIs

As I mentioned, the panel discussions were very beneficial. On Day One, listening to the discussion around “Exploring the business value of APIs – Opening data as a channel for Innovation” was very thought provoking.  With speakers from councils, retailers and product companies; there was a very balanced feel to proceedings. The main takeaway from the session centred on the consistency and availability of message transport. Oliver Ogg (Product Owner of APIs for M&S) focussed on how the company are not only providing digital solutions for their customers; but also, providing solutions for their staff to use in-store to ensure there is a ‘single source of proof’ for customer enquiries. The digital; omni-channel experience; though focussed on the consumer, needs to consider how staff interact with consumers. How the in-store display of message is translated to the consumer on their mobile device or at home on their desktop is key to converting enquiries into product sales.

API conversation; specifically publically available APIs were present across multiple tracks over the two days. Companies are making APIs available in the public domain to encourage innovation in the market. Providing the tools (or guidelines) for developers to be creative in designing new or better ways of completing transactions is actively being encouraged. This was epitomised for me during the open track session presented by Mark Dearnley; HMRC’s Chief Digital and Information Officer. During his session; Mark provided an outline of the Government Digital Strategy, and how over the course of 2015, HMRC’s APIs will be made publically available for developers to ‘make things easy’. HMRC have no desire to control the market; preferring to adopt a natural selection process.

What this means is that if we take the example of Self-Assessment (SA); over the course of 2015; there may be a number of privately developed apps; across mobile platforms that attempt to make SA submission more efficient. During the course of consumerisation, only the best apps will survive. Natural selection will ensue, as app store ratings take effect. Only the most user friendly and easy to use apps will survive, thus reducing the need to control the marketplace. As the APIs are in the public domain; HMRC can control the integration.

 CRM Strategies, Push Notifications and App Usage

The session hosted by Patrick Mareuil, the Chief Innovation Officer of Accengage; provided a very good overview of the brand loyalty of consumers with respect to app usage. Some highlights of the statistics shared, provide interesting reading:

  • 20% of users access mobile apps only once
  • 40% of users access the app between 1 and 3 times
  • 40% access apps 11 times or more

These statistics are interesting, as looking at them at face value; we can see a lot of missed potential in terms of consumer engagement.

From a testing perspective, the overview of the way push notifications are used outlined a number of use case scenarios that companies; such as Sogeti can assist with productionising apps ready for general use.

Concentrating on the right message, at the right time, in the right place on the right channel is an important feature of maximising conversion of message. From a testing perspective, being able to replicate these scenarios; will provide customers with the right data to complement their digital marketing strategy. As with all approaches, there is a fine-line between optimising the interaction with customers and over-kill. Too much interaction and prompting the user, can also have a negative effect on a consumers willingness to buy.

In addition, the way in which this marketing activity interacts with other applications on the mobile device should be considered. If a user is playing level 105 of Candy Crush, and at that key moment of completing the level, a push notification interrupts their enjoyment, this again could cause negative feedback. Balancing the need for interacting and promoting offers with not interfering with the consumers day to day use of smartphones will need to be covered by the test scenarios that constitute the scope of a release.  Throw into the equation the different approaches to notifications across device platforms, and the scope of testing increases exponentially to ensure the maximum consistency of message across platforms to keep the user experience standard.

Proximity Beacons

A number of the tracks either showcased or made reference to the use of beacon technology as a means to delivering up to date messages and special offers to customers based on their location within a store or theme park.

The use of the technology does; in my opinion counteract the intrusive nature of the audience; as the consumer will be captive and in the right mindset to take on board the advertising messages. Some of the challenges that were outlined during the various talks centred upon the proximity challenges of the technology. In an expansive space; such as a theme park, there are unlikely to be beacons that interfere with each other’s transmission; however; how do companies ensure that in relatively small retail stores, the use of beacons is appropriate and displays the right message at the right time?

It was this example that outlined to me one of the key challenges to the testing of the beacon. How do you replicate on a large scale? If you set-up a test lab, with a selection of beacons; do you lose the desired proximity locations in the live; store environment? Is it sufficient to test using a small selection of beacons, to conduct interruption testing scenarios?

This is a very real consideration that companies need to consider when introducing new technologies to their digital marketing strategies.

Wearable Technology

References to the Internet of Things and wearable’s, brought some interesting viewpoints; but for me the best and perhaps not unsurprising summary of this area came from the session on “Privacy and the adoption of wearable technology”.  In this session, the key message was that most; if not all policies around data security and protection apply to all devices. Securing the transport of message from back-end system through to ‘thing’ must follow the same policy and legislation.

For me; the same can be said in terms of the development of the ‘thing’ and also the testing of such devices. Validating the message transport, identifying weaknesses and vulnerabilities remains the challenge. Validating the display and user experience will require testing; developing Omni channel automation frameworks that maximise coverage, whilst controlling the amount of maintenance will appeal to companies as the industry matures. This is certainly a key area of development that I am overseeing in the Sogeti Studio.  In the coming months; we at Sogeti hope to be able to demonstrate these service innovations, to provide customers with an alternative approach to the current mode of operation.

The rise of Crowdsourcing – Testing device compatibility

Device fragmentation; specifically within the Android landscape, raises a number of challenges with regards the age old question of “How much testing is enough?”

Speaking to a number of companies; including app development agencies at the conference the challenges were very similar. How do you make sure that the apps released are compatible with the devices? Some of the answers were the use of emulators, to provide the breadth of coverage complimenting this with a top 10 physical device list to perform the depth of coverage.

Others mentioned the reliance on crowdsourcing, booking slots to open up the scope of testing on real devices, seems to be becoming a popular supplementary approach to release testing.

When we add in to the mix operating system platforms and screen resolution, there needs to be a more robust way of achieving the right level of quality. Tools vendors need to look at ways of replicating the user interactions in a standard manner, to provide options in the marketplace.

All in all, the conference was very thought provoking, and has certainly provided a number of takeaways regarding how we at Sogeti can answer some of these challenges through the extension of the current offerings within the Sogeti Studio and developing models that can improve testing coverage on devices, through complimenting emulation with physical device testing and creating Omni channel automation frameworks that promote efficiency in the test cycles.

If you have ever wondered which is the best smartphone OS to use then look no further than this article from Digital Trends which takes a look at the key wins for each of the Mobile OS.

The article covers many angles, and rates the OS accordingly in areas such as:

  • Affordability
  • Interface
  • Apps
  • App Store Usability
  • Battery Life
  • Updates
  • Cloud backups
  • Security

providing a balance view and highlights the key features in each section that the others provide in comparison. Though the article is behind in terms of the new releases of Apple, it still shows some useful analysis.

Through my work with Sogeti, and indeed through the promotion of the Sogeti Studio, our on-demand mobile lab, keeping up with the demand in the market place is a key challenge to ensure existing apps are compatible with the advances in devices and OS.

With the imminent release of iOS 8.1 due later this month and the suggested launch of Apple Pay; mobile decisions and utilisation of the features by businesses and app designers will provide another important rung on the mobile device market share ladder.

Having worked in and around the testing industry for over 15 years, I have seen many changes in the World of testing. The importance of the tester over these years has gone through peaks and troughs. Potential business failures such as Y2K and the implementation of the Euro have seen companies invest heavily in “Career Testers”, paying increased salaries and bonuses to get the job done. Both Y2K and Euro had issues, but not the catastrophic impact that analysts predicted. The immediate aftermath of both events led to questions being raised around why testers were needed in such numbers and at such cost to projects.

That to and fro question continues, market events often play a big role in the answer to the question.

In more recent years, the popularity of agile, and in particular the rise of approaches such as TDD has raised the question of a testers value still further, with some companies relying on development led activity to provide the necessary quality. Other companies; such as Microsoft have invested in engineering roles for testers, or “Developers in Test”, people who can bridge the gap between testing and development.

So, with all these changes in approach, where does the humble tester sit? Do they become coders? Do they become business analysts? Some will, some won’t. Others may increase their skills by learning the basics of both in order that they can continue to contribute to ensure a development area covers the ‘what if’s’. We hear terms such as ‘Shift Left’, testers in this arena can still add value, static testing with respect to documentation or architecture diagrams will highlight risk areas and contribute to coverage models, environment management and performance strategies.

The technology World is becoming more service based, enterprise architects are looking at SOA services to provide efficiency. Testing is playing catch up again, defining a framework that provides optimal coverage of service calls is imperative, striking a balance between manual and automation coverage is key, supporting continuous integration is another way that testers can bring value back to the project life cycle. Complimenting development and business approaches, builds confidence and should contribute to companies meeting the business objectives around the cost, time and quality triangle.

But what if the tester doesn’t want to code? Doesn’t want to use automation frameworks? Well the digital age is here, and growing. The rise of consumerization in the market place is an important factor for companies when committing budget spend to projects. Consumers hold the key, demands for information to be displayed on multiple channels is leading companies to define their digital strategies. The technology race to bring newer and better wearable devices raises the question of consistency and quality again. Each device, whether static, mobile or wearable will need to portray consistent design and functionality to keep the consumer interested. Deviation in level of service will lead to the consumer voting with their feet (or fingers !) and move to a competitor.

Usability is key, device interaction and gesture support is vital to maintain market share. Can automation tools cover this? Not entirely, therefore the need for the humble tester is raised again, filling the gaps that automation frameworks cannot support, operating in an assurance role to keep products relevant and maintain consumer interest.

In short, the manual tester is dead, long live the manual tester !

Having spoken to various clients across various industries; there seems to be a general shift away from offshore test centres of excellence towards a more proximate solution. This is especially apparent with respect to mobile device testing. Companies want to call upon services, but don’t want to rely on ‘follow the sun approaches’.

Couple the instantly accessible device solutions with the experience of offshore partners and you have the foundation of a solution that covers the main elements of today’s society with respect to new device tests being performed onshore; with older; more unusual device configurations being performed by offshore partners; via a cloud solution.

With regard automation approaches, we are looking for frameworks that look at the multi-channel aspect of the digital transformation age.

Finding an automation framework that can be used across multiple devices, channels and browsers will provide stakeholders with great solace; as; in theory any expenditure in automation approach will result in greater ROI returns as, “one script fits all” across various desktop and mobile browsers.

Providing companies with a “script once” approach will be very appealing.

Tools and frameworks are trying to readdress the balance, providing efficiency in the framework space, but until verified; there will be the need for maintenance to ensure current framework is compatible.

However, for me the most important thing is that as testers, we need to be constantly innovating, and developing our skills. Technology is constantly changing and so must our testing methods, frameworks that limit maintenance and maximise channel coverage are a must; but the most important (for now) is that the individual must develop and keep pace with the ever changing technology landscape.

Travelling around different clients, with different objectives; different risk appetites; the one question that always comes up is related to Governance.

Irrespective of what Methodology your project is following; be it SCRUM, Waterfall, SCRUMFALL, there will always be a stakeholder or two that will want to know exactly what return on their investment they are getting. The title of this post includes the phrases “Too Much, Too Little, Too Late”; I am going to outline my views on each of these situations and the potential impact each has on the project lifecycle and perception of the stakeholder.


“Too Much”

At first glance this seems a little counter intuitive; surely demonstrating full control of the project via the medium of metrics can only be good? To a point yes; but this is a case of knowing your audience. What happens if your project is only one of a wider portfolio that your stakeholder has budgetary responsibility for? Too much detail, may lead to disinterest or confusion. Key messages may become lost in the countless graphics and metrics.

To avoid this situation; it is imperative that boundaries and expectations are set as early in the project lifecycle. Agree the format and detail level with your stakeholder, ensure that they are engaged from the outset so that week on week you are providing decision grade information to assis the stakeholder to manage risk within the project.

“Too Little”

Again this seems an obvious statement, but sometimes when you are in the detail or under mounting pressure to deliver; these relatively “simple” steps can be overlooked. So what can happen in this situation? Well, depending on the vision and focus of your stakeholder, a lack of governance, can lead to the perception of panic or rather lack of control in your project space. Not having to hand the latest defect trends, the latest execution numbers will not install confidence that there is control within the project. If governance meetings are scheduled and there are conflicting messages being delivered from each of the project teams, the battle is lost. Stakeholders losing faith in their project team could be an ominous precursor to budget reductions and smaller releases, to reduce their “perceived” risk exposure.

Setting the boundary and expectation upfront, will provide the foundation to provide key messages during the project lifecycle. Delivering bad news in a controlled, consistent manner; though not necessarily welcomed; will still maintain confidence in the project team, decision grade information will allow options to be explored, mitigating risk where appropriate.

“Too Late”

Trying to apply governance when things start to go wrong, or after stakeholders have raised concerns with delivery is risky. Introducing governance in this situation could lead to the perception of the team being “Reactionary”. the stakeholder may feel that they need to follow closely ‘progress’.

In this situation, I believe the key is too start small. Introducing one or two measures to overcome the ‘reactionary’ tag, to increase confidence with stakeholder is key. Once the value is demonstrated, the team can work with the stakeholder to increase the level of decision grade information, to regain faith that project team is working in partnership, and is in control.

In all of these situations, the key is communication; which is often overlooked as a matter of course. Setting the expectation upfront, will ensure the level of governance is acceptable. Agreeing format and data collection tools is also key. Governance should be useful project tool, rather than a massive overhead and an invitation for criticism. providing good and bad messages on a project are par for the course; both messages need to be delivered with confidence and consistency to ensure stakeholders remain confident; giving projects the best chance to succeed