Why Scrum is Overrated and Overused
Disclaimer
The below represents my opinion, however I’d like to think that this opinion is not based on thin air, what follows is based on my experience of scrum, which you could argue (and you may be right) that perhaps wasn’t the most ideal implementation of Agile. However upon reading the book Accelerate, their research seems to suggest that the technical practices that XP emphasises are generally more conducive to high performance than scrum as a framework centered around management practices. I get the feeling that when many authors in the industry touch this topic they don’t confront it too directly and tip toe around it; perhaps because not wanting to offend. I don’t mean to offend you if you feel scrum is amazing and works for you. I just want to be able to present a different opinion without becoming a pariah, and I want teams and organisations to at least give rituals and practices a second and third thought, rather than implementing just because everyone is saying it’s the great thing to do. After all, Lean and DevOps is all about continuously improving processes, not blindly adopting them.
The Agile Manifesto laid the groundwork for a revolution in software development. It is a set of principles for engineers that enables them to do continuous and short release cycles in a more efficient, faster and nimble way by embracing changing requirements on the fly, and improving dialogue with the customer and the business.
There is no doubt in my mind that agile is a phenomenal approach to software development and that when it is fully understood and properly implemented it can help teams achieve much better results faster.
However the problem with many companies nowadays is that when they think of agile, they are actually thinking of scrum and they don’t seem to see the difference. Scrum is merely an implementation of agile. If agile is the bible, think of scrum as the catholic church. There is nothing particularly wrong with it if used in certain scenarios but just like the catholic implementation of christianity doesn’t work for every christian, I don’t feel that scrum should apply to every agilist.
So what’s wrong with all of this? Why is it a problem that scrum has become the de facto standard for agile across the industry? Let’s look at some of the components of scrum to find out why. But first a little history…
Engineering Practice turned into Business Practice
Back when Agile was born, Extreme Programming (XP) was its most popular implementation. XP inspired Agile, so if you practiced it as described, you would be as close as you could be to what the authors originally intended.
After Agile was founded, scrum master certifications courses became available, but the problem was that only project managers attended these (the business), not coders (engineering).
Because of this, the business and top management began to push their interpretation of agile and effectively created a split between business and engineering, which was the stark opposite of what agile was trying to achieve.
The main difference between XP and Scrum is that scrum is a lot more vague and general purpose and it doesn’t contain any guidelines or programming practices and it is because of this that scrum is sometimes practiced in a way that feels rushed and it doesn’t pay attention to addressing technical debt or coding finesse the way that XP did.
There is a phenomenal talk by Robert C Martin (one of the founders of Agile) that goes over the history of agile and why and how scrum became popular and where it is failing. It is totally worth watching if you haven’t already, here is the link:
The Scrum Master
Most companies that don’t understand agile hire scrum masters as opposed to agile coaches, these scrum masters, who are generally not very technical, are embedded within teams, often as de facto “managers”. The word scrum master in itself is a bit misleading, as Robert C Martin says:
A Scrum master was a coach, explicitly not a manager. He/she was responsible for defending the process, not the schedule, budget, stories or backlog. The role was supposed to rotate through the various team members. The idea of the role was to slowly fade away, so that in the end you would not need a Scrum master.
Although most scrum masters mean well, many of them are often not very technical and do not understand very well what the developers have to do to bring engineering finesse to the product and may inadvertently rush the process along more than they should to make it more “Agile”. Finally, because a scrum master is not really a full time role, a scrum master is somewhat encouraged to create extra meetings and extra processes to validate her time, which can be counterproductive for everyone.
One of the first things Spotify says in their DevOps culture video is “We are not a scrum company, we are an agile company” Spotify does not hire scrum masters and embeds them into teams. Instead what they do is hire agile coaches who can guide teams into implementing agile more efficiently, and the team can chose whatever method works best for them. This is a much more productive alternative than just embedding a scrum master in a team.
The Daily Standup (Or Daily Scrum)
The most popular scrum ritual is by far the daily standup and it is widely adopted even by teams that don’t do scrum. The daily standup is the loose equivalent of the confession booth for catholics or a support group for AA meetings, you standup every morning and you talk about what you did yesterday, what you are doing today, and what your blockers are.
My beef with daily standups is that in my experience, and for what I’ve seen in other teams, they are often less useful than not, and many of the so called benefits are widely overhyped.
These are some benefits attributed to daily standups:
- They remove blockers — In my opinion this incentivises your team to only broach blockers on the daily standup. Blockers should be addressed immediately, not wait for a ritual.
- Improves team communication — This is also dubious, your team should be encouraged to proactively communicate ad hoc, when required in real time, not wait for a ritual to address issues. Quantity of communication is not quality of communication.
- It gives you visibility of what everyone is doing — As opposed to having a well organised and detailed board where you can see this information in a few minutes and not waste fifteen minutes every morning talking about it?
- If someone is not pulling their weight, it shows on the standup — If someone is not pulling their weight you either hired the wrong person or they are not engaged in work they find inspiring. Also my observation is that the laziest people are often the ones that blab away the most at standups, wasting everyone’s time.
- It builds trust — Because nothing builds trust like a daily scrutiny of everyone’s backlog and activities.
- Fifteen minutes every morning saves time down the road — This is also very dubious. Fifteen minutes is often if you are lucky, most daily standups run for a lot longer as people often engage into lengthy debates that yield no action items and may not involve everyone in the team. Also it’s not just Fifteen minutes, it is fifteen minutes per team member, so if you have six team members, that means one hour and a half per day of company time. But wait, it doesn’t end there; usually daily standups take place around half hour after start of the day to give people time to arrive to the office, sometimes a little later. Usually engineers need big blocks of time of uninterrupted thinking (and according to many studies, most people are more productive in the morning, I know I am), if an engineer arrives to the office at 9:00 she is not likely to dive into code right there and then, because she knows that she has to wait for the daily standup, this means that she may or may not do productive bits here and there, or she may just go make a tea and linger around until 9:30, so in that scenario your company time ends up being forty five minutes multiplied by six, that’s four and half hours company time. Colin Breck makes excellent arguments about this here.
It’s not that I think daily standups are always useless. I have been to daily standups that on occasion some useful stuff has come out of it, but for the most part daily standups are very wasteful use of people’s times. In fact many high performing teams do just fine without them.
Here are cases where I feel daily standups can be useful.
- When you are starting a team or enrolling a new team member — Here it could be useful for people to do some thought gathering and initial discussions until team members are more familiar with the work and goals involved.
- Weekly or biweekly standups — whereas I feel that daily standups are often overkill, weekly or biweekly standups can be very useful. But should it be then called a standup? Or would it be more like a retrospective or planning session?
- During a critical time period where urgency is required — here it may be more useful to have daily reports of progress to make sure things are on track.
- When you have a few junior developers that you want to bring up to speed.
Aside from that, I think that meetings are very useful if done right, but I am more keen on having meetings that serve specific purposes and where there is a definite value for everyone involved. I also feel that meetings should require some preparation before attending them and they should create some action items, otherwise you may as well just have a chat at the bar. Amazon has an interesting way of dealing with meetings which I have never tried but it looks intriguing.
The Sprint
The sprint is another one of those scrum practices or ritual that I find rather questionable as a continuous practice. Just like the term scrum master, I don’t like the name sprint at all because it implies that you are constantly running at breakneck speed.
Scrum is designed so you are constantly sprinting, you finish one sprint, you start another sprint. in real life sprinting is a type of running you cannot constantly maintain, you don’t have sprinting marathons, because people are out of breath after very short distances. So if sprinting in real life is not meant to be constant, why is it constant when doing thoughtful and careful development?
I know what you are thinking now. “Oh but agile is all about fast development iterations”. Well, yes and no; Agile is about short iterations of work that come around faster because you have processes in place that make experimenting and deploying safer, it also mean you are following coding practices that make this possible. This doesn’t mean you should rush all your implementations, because if you do, you are going to bring a pile of technical debt to the table. And this is exactly the problem with Dark Scrum that Ron Jeffries, Robert C Martin and Martin Fowler talk about. If you rush your releases without any sort of design objectives and coding practices in mind then you are likely to create shit software.
Another thing about sprints I am not fond of is that they are meant to be fixed and you are not encouraged to add or remove work after they start, so scrum teams often have even more meetings called refinements and sprint plannings where every single ticket that goes into them is debated to see how long it will take. I’ve been to refinement and sprint planning meetings that lasted hours.
Sprinting can be useful if the goals are somewhat flexible and it is clear enough what needs to be delivered in those two weeks, as long as this doesn’t involve rushing through it and not following engineering best practices. The problem is when teams are constantly sprinting through their backlog without any clear feature that needs to be delivered, just for the sake of sprinting. So in that case, kanban style of working would make more sense.
Rushing never saves time, whatever time you save will always come back to bite you in the ass, if you create technical debt here to save one hour, chances are it will cost you four hours or more to fix later, so think about this carefully.
So does scrum suck for everything?
No, I never said that. I think scrum can be a good framework for some cases and provided that at least a good portion of the engineers working in the team understand good coding and engineering practices and actively follow them. Also it works if the business understands these engineering needs and supports developers in their aim to delivery a high quality product.
I remember having a discussion with a talented and experienced engineer, I was discussing with him a blog post by Michael Church. He dismissed it as if it was written by an angry, bitter person. He then added that scrum is just a tool, and a tool doesn’t define the work that you do, the engineers in the team do.
I agree with that to a certain extent, however you could also argue that chopsticks are a good tool for eating. Now imagine if a restaurant enforced you to use chopsticks to eat your minestrone soup or a t-bone steak, and imagine the restaurant assigned a “chopsticks master” to your table to dictate how you should use your chopsticks to drink your soup in sprints and made you stand up to talk about it every five minutes.
I love eating with chopsticks, it is one of my few skills (Not as nice as playing the piano, I know), as soon as I grabbed them for the first time I knew how to use them, it just came very naturally to me, but I wouldn’t think it is the best tool to eat the dishes mentioned above, and I certainly wouldn’t like being forced to eat with them if I don’t feel like it’s helping me eat my food.
The blog post by Michael Church I mentioned earlier elaborates why scrum is poorly implemented so much better than I ever could, I totally recommend that you read it you have a chance. In a nutshell, he concludes that scrum is better suited for special circumstances and emergency situations where constant feedback is required, but it is not a model that’s useful or sustainable to constantly run with. That’s a use for scrum that I can totally agree with.
So if not scrum? What would you use?
Overall I think that XP is a far superior framework for most developing teams because it provides a set of engineering guidelines that all members must at the very least understand. It is therefore more difficult to just rush through features and create a ton of technical debt doing XP than doing scrum.
However depending on the team XP may not work for you, so in that case I believe it is good to have a grasp of kanban and Agile and then apply engineering practices and processes that make sense for your situation.
Some general guidelines apply to every team in my opinion:
First of all, I believe firmly in good communication between team members and across the organisation, and in fact if I were to hire someone to work with me the very first thing I would look for is if the person can communicate clearly and efficiently.
It’s great that there are all these frameworks like scrum encouraging people to talk more, but the aim of communication is not quantity of communication, it is quality. Quality communication is harder to achieve, you have to think more and structure more, but it is a lot more useful.
This is why you shouldn’t force people to have more meetings to improve communication, sometimes meetings are useful but I think more importantly you should train them in how to communicate more efficiently.
Efficient communication can take many shapes and forms, such as speech, writing skills, contextual, useful documentation and clear, well commented code. Your employees should be trained to cater to the next person, to be proud of their communication skills and a little ashamed of handing something over that’s unclear or untidy.
Meetings on the other hand are a great tool to reach consensus, discuss and debate topics, but I think meetings should be planned as they are needed and thought and preparation should go into them, and you should always make sure to have some action items at the end of them. If you do that, then your meetings will be great, otherwise you will just waste time. When you plan recurring meetings, you may not always have significant content to discuss in them, so make sure to think if these are required or not, and also make sure to only include people that will clearly benefit from it. Sometimes it is best to keep just outline the contents of the meeting and share it with other parties that may not be as actively involved on what’s discussed but it’s good they are informed, that way you don’t waste an entire hour of their time.
It is also good to organise teams around design principles and coding practices and refine them constantly so they don’t become stagnant. For example you may have a design principle of your application never deliver a response that takes longer than 40 ms. When you have a design principle in place you are less likely to add too many features or do any work that would introduce technical debt, and as long as everyone knows what should not be violated, then you are all operating under the same value set and frame of reference so you will be more efficient.
Finally try and configure your tools and engineer your code so it is harder to do bad engineering. For example if you want a clean trunk while doing TBD, you could disallow non fast forward merges and require squashing in your repository.
Conclusion
When I worked in a scrum team I felt the implementation was really heavy handed which is what drove me away from it and made me dislike it, however I did learn a few good things from it as well. So I am actually quite happy I got to work in a scrum team.
I would be okay giving scrum another chance in different circumstances, and I am certainly okay following some of these rituals if the whole team feels is bringing definite value.
However given what I’ve seen and all the evidence I am not sold that it is as good as all the scrum evangelists would have you believe and there are other alternatives that are better.
The best thing you can do is to look at various implementations, learn technical practices, implement design principles and then apply what best works for you and your team.
Anything you implement, remember that you should always refine your process, don’t become dogmatic!
