What cost a lack of acceptance?

Having worked over many years on various sized projects, similarities can be drawn irrespective of who runs the project, what the business priority is and how much budget is available. Wherever you go you are likely to face the same issues or anti-patterns when it comes to process and approach.

The focus of this blog post will be the Agile sprint process, moreover what prevents us achieving near “Production Quality” code at the end of a sprint and how in turn that code becomes “Production Ready”. I will be looking at some of the main barriers to achieving this status, and outlining where things should change, to give the project team the best chance of success.

Before going into detail, I’d like to start by explaining what I mean by “Production Quality” and “Production Readiness”:

Production Quality – I see this as the minimum acceptable level for any code package delivered at the end of a sprint cycle. To achieve this production quality, I assume that before the sprint, an inclusive game-planning session was conducted, where the scope of each story was agreed, and the acceptance criteria (inclusive of tests) were defined in agreement with the Product Owner. The “Quality” is determined, by the fact that the project team has developed and reviewed the agreed stories within the sprint, meeting all of the acceptance criteria. By achieving this, the project team are suggesting that there is available a Production candidate that can be handed over to pre-release QA and / or Business Acceptance Teams (UAT) for validation.

Production Readiness – Once the release candidate has been subjected to ‘an appropriate’ level of QA and UAT analysis, if no blocking defects have been found, and all of the functional and non-functional requirements have been validated, we can label the package as “Production Ready”. The package can now be handed over to the Release Management team, in order to schedule a deployment to Production.

Now that I have clarified what I mean regarding quality and readiness, I’d like to address the title of this post “What cost a lack of acceptance?”. If elements of the sprint process are ignored, or overlooked, whether to save time, money or to get things done; the impact can be quite high. In my opinion, these thought processes are a false economy, as you are potentially shifting the issues later in the development or release cycle; which as we know from traditional analysis; the later in the lifecycle you detect a defect, the greater the cost is to fix it. So what pressures can be experienced, that can lead to a lack of acceptance?

1. Development Versus QA Ratios - Inequality within a project team with respect to the ratio of QA personnel to that of the development team can have  a significant impact on the ability of the QA team to keep pace with in-sprint tasks (such as defining acceptance tests for future sprints and providing daily feedback during the current sprint) whilst conducting more traditional QA tasks; such as release candidate testing. If this situation arises, it is likely that a trade-off may need to happen. One such trade-off could be to reduce the amount of acceptance criteria on a story. If this happens, we could be adding risk to our release schedule as only limited validation has taken place before entering final QA; leading to more bugs being found during QA, requiring additional builds and retest cycles. Having to pull developers off of in-sprint duties to fix defects, raises the cost to the project. Not only in terms of time to fix (and merge to code branch(es)), but also the potential impact their withdrawal has on the current sprint.

2. Waterfall in a sprint – I have worked on projects, which claim to be Agile, which on inspection stretches the truth just a little! All that happens is work packages are split in to 3 weeks, the developers code for 2 – 2 1/2 weeks then ‘throw over the fence’ to the in-sprint QA for testing. Adopting this approach, does not maximise the potential gains of providing regular (daily?) feedback from the QA team. The earlier in the sprint issues are found; the quicker they can be fixed, raising the perception of quality on a daily basis.

3. Lack of Product Owner buyin -  If your product owner is not fully engaged with the project, this can lead to a lack of acceptance. If the Product Owner does not provide (or at least provides input) the minimum acceptance criteria at the beginning of a sprint, the team will be fighting a losing battle, as come the walkthrough at the end of the sprint, any vagueness will more than likely have been interpreted incorrectly, and thus the Product Owner may insist on instance fixes to resolve before providing sign-off for the feature in question.

5. How integrated should you be? -  If during a sprint the acceptance criteria, overlooks particular integration dependencies; including the target go-live dates of dependent systems, this could impact the projects attempts to achieve production readiness. A lack of understanding of release schedules for dependent systems can lead to reputational loss of the project team with sponsors; as features will be unusable in production, thus preventing the company from rolling the product or feature out to prospective clients.

I have experienced all of these pressures on various projects over the years. They are always likely to come up during software delivery projects; but it is how far they are allowed to manifest themself; that will determine the size of the impact that they will have. Keeping control of these factors, should increase confidence, and promote knowledge within the team. In order for the team to succeed, it is how these influences are handled, and how close the team sticks to the agreed approach, which will play a significant part in determining whether the delivered product is fit for purpose. Keeping control on acceptance criteria, from clear definition through to implementation will provide one of the key factors of project success.

 

Pragmatism Vs Optimism – Releasing Software

With the recent downgrading of the US credit rating, and the on going commentary regarding the potential double dip recession; businesses; including banks are feeling the heat and are displaying caution; particularly with regard to IT projects and the risk releases have on existing revenue streams.

What affect does this caution have on IT? Well, if the cautionary stance is a prolonged activity and you are working in a fairly established project team; budgets are likely to be reviewed. Rather than new development, the emphasis may switch to that of a maintenance agreement.

If you work on a greenfield project, these market conditions will have various impacts depending on:

- Risk Appetite of your sponsor(s)
- Stability and availability of your non-Prod environments
- Value proposition of upcoming feature delivery I.e. Can the company sell it!

If your company has invested heavily in your project, the optimist will come to the front and lead, direct and hopefully motivate your team to deliver high quality software, that when marketed has the potential to make the sponsor money, whether through shares or bonuses.

If you are already ‘live’ and are working on a feature release, stakeholders may adopt a more pragmatic approach to software delivery. They may decide to delay releases to protect reputational risk position. An undetected bug in the new feature may dissuade clients from using the platform, and thus impact potential revenues. Other forms of pragmatism may include soft or internal launches, to minimise exposure and thus reduce risk; whilst maintaining traction on delivery pipelines.

It is a fine balancing act for project teams in these periods of volatility, too cautious an approach, and you lose ground against competitors, or your stakeholders reduce budget due to lack of delivery. Too aggressive an approach, and if things go wrong the product loses its reputation, and prospective clients are cautious of using the platform; due to a perceived lack of quality.

To ensure projects survive and maintain velocity; open communication between project team and sponsors is essential. Clear messages outlining progress; coupled with a concise outline of risk and issues (with associated impact) will provide essential data for sponsors to make decisions in line with their risk appetite. Taking the view from various parties; such as service management, should build a clear picture of the risk to deployment. Agreeing various SLA’s for production candidates, should be a matter of course, however these become imperative, when seeking approvals.

Can Agile beat the ‘inevitable’ Death March…Discuss

Having worked on various projects over the years, with the main driver being time to market pressures, I thought that working as part of an Agile team, the days of the death march were over as we agree our work packages upfront.

Currently, I am working on a project that is going all out to meet the deadline, at the expense of agile approach. Though it has it’s moments of chaos, the job is getting done.

This leads me to the topic of this post. What are people’s views on the death march approach, is it an inevitable consequence of changing business demands? Or can agile fight it’s corner and prove it’s flexibility, to deliver to change in focus?

All comments / views welcome

Continuous (Process) Improvement – How Often & How Much?

Reading and practising agile within a team, the aim is to promote collaboration and commitment to delivery on a sprint basis. Process improvement is what all team’s should strive for, making the process ‘lean’ will allow the team to take on more work or deliver increased complexity. I’ve come across a couple of interesting articles, that highlight the case for doing something and the case for doing nothing until you have a question to answer.

The first article from tool vendor VersionOne – http://bit.ly/drDGUG highlights the need for constant improvement to keep the process fresh.

The second article My Flexible Pencil – http://bit.ly/9dj2pD shares a different view, that sometimes doing nothing is the ‘best’ thing that could happen to the process.

From my point of view, I’ll take a few splinters, and sit on the fence with this one; though if I had to commit, I’d probably fall on the side of My Flex Pencil.

The important thing for me is that process change has to be worthwhile, and display a tangible benefit to the team; whether that be reducing admin overhead or encouraging collaboration through workshops. The outcome of retrospectives is an important one, small; achievable tweaks to a process on a sprint by sprint basis should easily show tangible benefits. Vast overhauls of process, can lead to the dangerous position where team members become lost; in turn affecting delivery.

So in summary, ‘how often’ should be driven by the outcome of retrospectives. ‘How much’ depends on the evolution of the team; has the team ‘outgrown’ the current process? If yes then a major overhaul maybe the answer. If no, the retrospective tweaks may be sufficient, but the team will decide…..

Sprint Cycles, Production Deployments and are we done yet?

In an earlier post, we highlighted “The Definition of Done” concentrating of what factors make up a complete story, that is potentially shippable at the end of a sprint.

In this post, I want to focus on Production releases, integration headaches and how to account for Production Support in later sprints.

Recently, a situation arose where a 3rd party vendor could not react quick enough to the requirements of our sprint. Rightly or wrongly, the decision was made to create fake services to test out the code that was being written, all was merry and bright and there was a good feeling within the team, until reality dawned that there was an imminent production rollout, and our application was far from shippable; due to the lack of integration.

What followed was Death March city, as the team worked through the raft of integration defects, where message contracts between the systems were either incorrectly applied or uncovered edge cases that had previously gone undefined.

A harsh lesson learnt by all concerned. This situation got me thinking about post-production; specifically support, the inevitable patch releases, as end users uncover missing business rules or fine grained entitlement issues etc.

How do we support these in an Agile sprint? The main problem that I foresee is that the developers will have moved on to the next feature delivery in-line with the roadmap; therefore if production issues do arise, that codebase is likely to be 4-6 weeks old. There is the danger that fixes become ‘over engineered’ as the developer may be working on new functions in that area as part of the current sprint, which may affect the fix on the ‘old’ code branch.

Also, there is the issue of delivery, production support estimates are always finger in the air estimates, however if you do not provide sufficient cover, your current sprint may be at risk. The counter argument to this is that if you take too cautious approach, and assign too many points to support, your Product Owner will loose faith, as the team will not be delivering much new functionality.

The approach currently being taken, is to agree a proportion of points for production support (up to 20%-25% of capability), splitting these between our development teams. Within the team there is a rota of who is on support each day, the objective here is to ensure everyone contributes to the current sprint whilst also being mindful of the need to support an old code branch.

In line with my previous post, we are looking to improve our ‘doneness’ relying less on the fakery, or if fakery is required, ensure that future tasks are captured and estimated to reduce pain points in the run up to subsequent production releases. This includes testing considerations such as multi-region testing, external network and compatibility.

Implementing Agile throws up a number of challenges, but I guess the testament to how well you are doing lies in how well you can react and implement new strategies and considerations……

A good outline of the role of an Agile tester

http://www.agile-community.com/profiles/blog/show?id=4633438%3ABlogPost%3A1764

The Definition of Done

There have been some interesting discussions within the agile team that I work. Retrospectives have suggested that we are currently guilty of operating more Wagile rather than Agile. In some circumstances, there is a return to the handing over of work items for the next team to use. The sprints are still completing, but are we working as a team?

Having attended the CSM course, it reiterated the need for collective agreement, specifically agreeing what tasks are needed to complete before a story is ‘Done’. This brings me to the subject of this blog post; ‘The Definition of Done’. More specifically, I want to talk about when tasks within a story are done.

As we know one of the main principle’s of agile is that at the end of each sprint you have a ‘potentially’ shippable work package. So what constitutes ‘Shippable’ and what can you really achieve during your 1 to 4 week sprint?

In terms of shippable, I do not believe that all activities need to be completed by the end of each sprint. If you have a feature that is broken down into manageable stories, this feature may be delivered over the course of three sprints; therefore all tasks that are required to ensure the feature is ‘done’ should also be split over the course of the three sprints. If we work on the assumption of a main release every 3-4 months in line with the project roadmap, to determine when a story really is ‘done’ we need to identify what activities need to be performed before we reach production readiness.

The main considerations for any story or feature are likely to be:

1. Performance testing considerations
2. Resiliency/Failover considerations
3. Entitlements
4. Integration or End to End workflows with legacy systems
5. Logging and Monitoring to ensure support teams have the tools available to manage a production system

These are technical considerations, and forward looking to production readiness for our stories, as a team we may include more sprint specific considerations:

1. Have the stories been Automated / added to our Regression pack
2. Continuous Integration builds are stable
3. Design documents have been signed off and application mirrors design?
4. All Acceptance tests have validated their criteria
5. A successful Product Owner walkthrough has been conducted
6. No defects have been left unaddressed

The options are unlimited. The important thing to remember about agreeing the ‘definition of done’ is that it is no different from other aspects of the agile process.

1. The team must come to a collective agreement
2. The team is making a commitment as part of the sprint, so these checkpoints must be achievable and relevant to the objective.

Collabnet (Danube) Certified Scrum Master Course; London

Last week I attended a Certified Scrum Master course. I was keen to gain perspective on the role of a Scrum Master; specifically coming from a testing / QA background, I wanted to understand who ‘best’ fitted the role. Going into the course, I had the pre-conceived idea that Developers were best suited to operating in this role; as if the team raised blockers during a stand-up, they are likely to understand the problem and be better placed to work towards a solution.

The course provided a good mix of BA’s, PM’s, developers and test; the trainer herself (Tamara Sulaiman) came from a project management background. Due to this, the classroom discussions provided a number of different viewpoints. The training material provided a good balance of theory and practical application.

In terms of meeting my objective for this course, I think this was achieved. The role of Scrum Master is varied; coaching teams to reach collective decisions; ensuring that Product Owners are engaged and maintain roadmaps and product backlogs are fundamental to a scrum teams success. With this in mind; the role does not need to be limited to a development team member.

All I need to do now is pass my exam

What’s Driving YOU to Better Testing? Consider the Story of Amy and Morgan

What’s Driving YOU to Better Testing? Consider the Story of Amy and Morgan.

Very interesting comments made by Jean Tabaka. I don’t think that there is an Agile team in existence that has not been guilty of ‘hiding’ defects at the end of the sprint by adding them as a backlog item.  Reading Tabaka’s blog post, and having worked on iterative and agile projects, I see where there is a need to focus on the testing phase. Since I have been working in Agile teams; I have been very conscious of not applying ‘quality gates’ or too much process. I have bought in to the collaborative model and the need for self-organisation.

But as time goes on, and reading such a viewpoint, it is also important to keep to the other objectives of agile; specifically that at the end of a sprint, you have a shippable product. Pushing testing earlier in the sprint, working with developers to provide feedback is one thing. Not acting on this feedback, adds pressure to the team towards the end of the sprint as you have a bottleneck of bugs that need to be addressed to get to ‘Done’.

I’ll certainly be looking hard at what can be changed during the sprint cycle, by applying more focus on testing (QA) to minimise the bottleneck of bugs, that cause issue at the end of sprint, so that we avoid the build up of technical backlog in the form of outstanding defects.

There will always be feedback during the sprint that is clearly a ‘missed’ requirement or a logical enhancement to a story; this is the beauty of agile, and these items quite rightly are added to the backlog to be prioritised by the Product owner. But as the linked blog suggests teams do need to avoid adding to future pain and get it right first time. Committing to a sprint is exactly that, and all members of the team have to remember that they are responsible for the process and the quality of the deliverable.

What is the ‘right’ Developer Vs Test Ratio in an Agile Team?

A number of text’s on the subject of ratio’s suggest a 4:1 developer to tester split. I would agree that if the majority of your development work concentrates on delivering functionality, then you will need to increase the number of testers on the team. But what about the early phases of a project, where infrastructure needs to be installed and integrated or in the run up to that first release; where teams have to tackle additional technical debt to harden the application? Running a 4:1 ratio in this situation may seem to be overkill for your client; or lead to utilisation problems. In these situations I have been working to a 6:1 ratio, anything greater than that and cracks began to appear; leading to resourcing conversations.

No project is going to be the same; and as the project matures; and hopefully the technical backlog/debt mountain reduces; this is where team ratios will get closer to 4:1 or even 3:1 in order that velocity can be maintained and that the testers can provide feedback early in the sprint; whilst maintaining the automation effort as part of continuous build.

I mentioned in my previous post whether the Agile Tester’s input finishes at the end of a sprint; as I outlined; my current thinking is ‘No’; therefore this also needs to be factored in to discussions around team size and ratios.