Suggested Time Box: 60 minutes
Get ready for some big changes in your traditional thinking! Agile introduces radical changes. It is important to understand these changes ahead of time begin to prepare for the mental shift required to successfully implement these new concepts. You do not need to dwell in depth on these concepts as some will be discussed as training workshops later. The idea of this chapter is to introduce the major concept changes to get your mind prepared for the change. The following is a list of major paradigm shifts I have observed in my projects and talked over with other Agile Practitioners.
The following statistic will undoubtedly create the most contentious discussions if you are new to Agile: the average Agile project experiences 20-40% change in scope (Source: Valtech). My personal experience has been increases in the range of 30-40% on every project, after the backlog is stable. The following are real comments I heard when I undertook my first Agile project:
“Requirements should be complete and frozen, so we can build a complete and solid architecture that can handle whatever requirements are thrown at it.”
Reality check: in a traditional project, how often is your architecture the exact same as when you started? Have you ever looked back on your development and thought, “I should have done it this way.” Or have you learned as you went and improved your knowledge of the architecture and tweaked it as you went? Agile embraces these realities. Yes, it’s nice to have a big picture of the requirements (the Product Backlog) so you can create the seeds of good architecture and data structure. But, embrace the reality: users don’t know exactly what they want or how to communicate their picture accurately the first time. Also, as developers you often do not know how to best address the architecture and data structure until you are in the midst of development. Embrace the reality of creating a flexible and changeable architecture. Be willing to adapt the requirements and architecture and data structure as needed to meet the new realities revealed as the project progresses. You are either going to address these realities immediately (in an Agile project) or next year when you finish the project. Wouldn’t you rather delight your customers by being flexible and accepting this right now?
“It’s like building a plane while you are flying it.”
Well, kind of. In a literal sense, it is less risky than building a plane while you are flying it since you are still on the ground. However, in a sense, it feels a lot like that because it surfaces a lot of scary feelings: transparency, constant pressure to produce, high visibility into all the ugly problems of the project, constantly changing requirements, constant change to the data structure and architecture.
This is a key paradigm shift in Agile – change is not locked down and fixed. Agile is flexible and embraces, accepts, and bakes change into the process. And yes, that does feel like flying a plane while you are building it.
Transparency and Micromanagement
The visibility into progress of short one-, two-, three-, or four-week increments suddenly creates high accountability, transparency, and ability to see the details and the risks and issues of a project. This is very scary at first! Understanding that Agile is going to create some very scary visibility is important to acknowledge. The Product Owner and Scrum Master and an Agile Manager all have key roles in supporting and protecting the team when the problems of the project are exposed for all to see. The temptation will be for management to micromanage the details and the problems.
However, as everyone adjusts, this visibility actually creates a protection and builds comfort and confidence in the team by management. Management has visibility into real details and decisions that they can make instead of making decisions with carefully crafted, politically correct statements. Decisions can be made in real time against real data. They can offer visionary course corrections and make real trade-offs based on the details captured in the Product Backlog and the Scope and Capacity Planning. Strategically add a person? Reduce scope? More accurate predictions can be made on budget and timeline. The anchor of scope becomes the key lever that is exercised in an effort to manage to the budget and timeline originally projected. This is normal and good!
Surfacing Organizational and Systemic Impediments
This new transparency creates visibility into the problems the project is facing. The transparency of the impediments provides real data and areas for the management to uncover and provide substantive assistance in removing roadblocks. The Agile process often surfaces organizational (silos, communication barriers) and systemic (communication culture of the organization) impediments. Making these impediments clear can create some politics. These types of impediments are often beyond the purview of the team to overcome. These impediments are best handled by management who has greater influence and role power to effect changes surfaced by the Agile process.
The average encouraged Sprint is recommended to be two weeks. I suggest one-to-two week Sprints. A team has the option of deciding the length of their own Sprint from one to four weeks, with one or two weeks being the ideal time box. At the beginning of an Agile implementation, overhead time spent in planning, administration, tasking, demo preparation is often forgotten. Also, there is a tendency to underestimate the actual complexity of a feature. The initial speed creates anxiety and will feel like a circus for a while until the habit becomes ingrained. The temptation will be to change the Sprint length. However, don’t worry about slow starts, underestimating tasks, forgetting about tasks and overhead or preparation. You will get used to this new rhythm. Eventually this will be part of your habit and you will get used to the speed and constant gentle pressure applied by the Agile Scrum process.
A note of caution: there is often a temptation to change the length of a Sprint because there were a lot of underestimates and forgetting of tasks. Resist the urge! You will catch on. Your estimating will improve with practice and you will begin planning the overhead time into your estimates. Just give it at least 3-4 Sprints.
Test Driven Development / Writing Tests as Requirements
Test Driven Development (TDD), in a nutshell, is when a developer takes the User Story and converts it into an automated test. The developer then takes the test, runs it and watches it fail. Then the developer writes code until the test passes. This test is automated to run with the Continuous Integration build (see below). If the test fails at any point, the team immediately knows that something changed in the code and Stops the Line (see below) to immediately fix anything that broke (or fix the test if the test needs to be changed as a result of code change) while the programming is still fresh in everyone’s memory. If you have ever had to trace a bug months after you developed code, you know how much less expensive it is in time (hence, money) to fix a bug when it happens, not after it happens.
This is the second biggest paradigm shift I have seen on the Agile. In the traditional environment, the testing takes place last and the window for testing usually shrinks as the development takes more time than originally planned. Also, testing tools are often designed to write tests after the code is developed. This habit carries over into Agile teams because the mindset behind TDD is such a significant shift. As the project progresses, and the team has time to digest why this is a best practice, more effort is made to move the team’s maturity level over by testing during the Sprint. Sometimes such a significant testing debt is developed that automated testing is relegated to be implemented during the next release and focus is only on manual testing. Often TDD is implemented in the next project while automating or doing manual tests on the first release. Don’t worry if your team doesn’t achieve perfection the first time.
One of the reasons I think this is the biggest shift, is that learning to write requirements as a test, and learning the automated software tool is very different from the traditional method of writing requirements as a document and doing manual testing later. However, it is well worth the effort to spend time up front with learning the tool, cross-training team members as necessary. This initial learning curve and paradigm shift will save you time and produce higher quality software and happier customers in the long term. In the words of one of my colleagues once he understood the concept of TDD, “Take some time, waste some time, spend time at the beginning of your project (however, your team interprets this concept at this point) and do TDD.” You will be happy you did.
Continuous Integration (CI) is another concept where the individual pieces of code are built continuously and fully integrated as they are developed and checked into the source code repository. The team can choose the increments of when the CI takes place (hourly, daily, as new code is checked in, twice a day, etc.). The point behind CI is similar to TDD, if the build breaks, it is evident that a piece of code introduced a bug or broke another piece of code or a test needs to be modified. The team then implements Stop the Line to immediately fix this issue.
This paradigm shift I find a little easier to adopt as developers can quickly see the value of continuous integration and have actually asked to turn on the Continuous Integration tool to catch any errors introduced in the code that weren’t caught through Unit Testing or Peer Review.
Stop the Line
TDD, CI, and Stop the Line are tightly integrated with each other. When an Automated Test or Continuous Integration build fails, the team should implement Stop the Line mentality, similar to the Toyota’s “Pull the Red Handle” if there is a defect. This is the cheapest time to fix a defect, as soon as it is introduced into the environment.
The concept is relatively easy to understand. It takes a little more effort to develop the habit as it is a change from the traditional development practice.
User Stories – Writing Requirements from a User Point of View
Traditional project development has developed a very document-heavy, system-centric point of view with writing requirements. Agile embraces including business people as Product Owners and Subject Matter Experts as part of the development team, sitting in the same room (if at all possible) and communicating back and forth with direct conversation. (It is possible to implement the Scrum methodology with a distributed team, but with time lags, special concessions regarding velocity should be estimated and accounted for when planning. More on Distributed Teams later.) Because the business is part of the Agile Scrum team, one major change is learning to write requirements as a User Story. Instead of writing “the system shall”, the story will be written from the point of view of a user. It will be necessary to identify the types of roles a user will have: administrator, shopper, customer, you can write the stories from the point of view of the actual performer of the operation: “As a customer, I would like to check out using a Visa credit card.”
Another shift in this thinking is that the stories should be simple. The overall concept of Agile is to keep things as simple and functional as possible. User stories will contain a simple statement of functionality with a simple set of User Acceptance Criteria. A single story should not contain an exhaustive of “should” and “should not” criteria. A very general rule of thumb (that can be broken as needed) is to write the function on the front of a 3” x 5” index card and the details of the acceptance criteria on the back of the card. When you run out of room, it means you need to write a new story. This keeps simplicity and signals when stories should be simplified.
There are a number of techniques and ideas for story writing workshops. Some of the specific techniques will be addressed later. The key idea is to keep things as simple as possible.
Your mind might be swimming with these paradigm shifts. Don’t worry about implementing everything successfully the first time you go through this process. Even the Agile process itself can be implemented incrementally with each release. For example: TDD is one of the difficult concepts to implement right the first time around (although I highly recommend giving reasonable effort to doing this.)
There are three phases of Agile Maturity as the team adopts various practices. Don’t worry as you go through implementing this framework. Practices are implemented incrementally and change is incremental. Accept that your team will go through a process of achieving Agile maturity. Lyssa Adkins highlights these phases in her book Coaching Agile Teams. In short, the phases of maturity you will go through are:
- Learn the framework (learn the basics)
- Break the framework (make adjustments that fit your team and environment)
- Live the framework (internalize the core principles and pass these principles with your key lessons along to others who follow you in the Agile path)
Okay, we discussed the Big, Scary Changes that happen on an Agile Project. So, why should you do Agile? Here is the main reason to keep you motivated about the benefits of Agile and go ahead and dive in and start.
The Best Agile Project
The best thing that can happen on a good Agile Project is you produce a piece of high-quality software that your customers not only love, it creates high demand from your other customers. The challenge, then, is meeting demand because you have delighted your customers so much. In my first project, I had the opportunity to observe this phenomenon: while everyone watched the first rollout with anxiety, once they saw the product in action, they were begging for the product. In fact, the problem then became the politics because everyone wanted it now because it was changing their business process, doing away with home-grown, half-developed tools and creating opportunities to grow the business. That’s a good problem to have!