Spinning in Place.
A Retrospective on Returning to the Spirit of the Agile Manifesto (Using the words of the Founders, Elvis Costello, and the Tao)
By: Allan Gross
“It’s a devastated wasteland. The life has been sucked out of it. It’s a few religious rituals carried out by people who don’t understand the purpose that those rituals were intended to serve in the first place.” — Kent Beck, creator of Extreme Programming
If you’ll excuse a brief diversion down the rabbit hole, I can tell you that I started to write a straightforward essay on the state of Agile Software Development, supplemented by some quotes from the writers of the Agile Manifesto themselves. Along the way, I flipped on some music, did a bit of Internet searching, and before I knew it, I headed off in a different direction.
On Elvis Costello’s Punch the Clock, there is a song called Shipbuilding. In many of Mr. Costello’s songs he hides literal meanings and allows for varied interpretations with vague but interesting imagery. Shipbuilding offers a more direct, pointed allegory spawned from the buildup to the war in the Falklands. I’ve read that Costello was particularly proud of the haunting line, “Diving for dear life when we could be diving for pearls.”
The line seems relevant as we consider the choices we make, in what, why, and how, we do any job. We can build ships for war or we can build them for more peaceful or pleasurable pursuits, like diving for pearls. The boy depicted in the song learns… “that people get killed in the result of this shipbuilding”. This also reminds me of verse forty-six of the Tao te Ching as translated by Stephen Mitchell. He modernizes this ancient wisdom as: “When a country is in harmony with the Tao, the factories make trucks and tractors. When a country goes counter to the Tao, warheads are stockpiled outside the cities.”
What does all this have to do with Agile Software Development?
I remember well when I picked up my first edition of Kent Beck’s seminal Extreme Programming Explained back in 1999. I was blown away. It really spoke to me, as it did to many software developers. It was not just a set of techniques to develop software, but it created a feeling that software developers were not hired assembly-line workers generating code. There were human values and principles involved. And fostering these values would also help in building quality code! Great stuff!
It wasn’t surprising that these values and principles were also present at the core of the Agile Manifesto, which was written a couple years later, with Mr. Beck as a participant. For those who don’t have the manifesto tattooed on their body, it is very succinct and goes as follows:
“We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:
- Individuals and interactions over processes and tools
- Working software over comprehensive documentation
- Customer collaboration over contract negotiation
- Responding to change over following a plan
That is, while there is value in the items on the right, we value the items on the left more.”
While the four main bullet points are not numbered, I find it significant that the first one references the value in people. This was a relatively new and important idea at the time when being applied to software development. And the Manifesto’s list structure has a familiar Taoist nature, with the comparisons replete with a sense of duality. You can value the items on the left AND the values on the right, even when they seem at odds. It is surprising how often I encounter developers who have been taught Agile Software Development but really don’t grasp this spirit. As Ron Jeffries said, “And worst of all, agile methods themselves have not been agile. Now there is an irony for you.”
Today, depending on your organization, you may get a feeling that for the most part Agile is trapped in time and space. There are exceptions, but it is fairly static, as pointed out by Jon Kern, who lamented, “Scrum, scrum, scrum, scrum. Part of me is happy that there are legions out there turning the sod under, trying to sow the agile seeds via Scrum. Another part of me thinks it is like Corn-to-Ethanol. A waste of energy for the dumbfounded among us, spurred on by lots of lobbying.”
My experience as an early adopter of Extreme Programming (and later Agile) is still colored by my turn of the millennium experience. Back in 2000, I was much maligned by skeptics (including my government boss) for suggesting many of its tenets. However, I prevailed and used extreme programming and a team of like-minded contractors to build a highly successful reuse library. This was long before Java or Free and Open Source (FOSS) repositories had taken hold and it saved the government tens of millions of dollars. Some of that code is actually still in production. Its robustness I attribute to the power of extreme programming techniques.
However, after a foray into more program management-oriented tasks, I came back to the software development world, to find the overwhelmingly Scrum dominated dogmatic Agile landscape that many have described. It had been accepted and become so mainstream it was dictated for use within contracts, even if it was not an applicable paradigm for the particular task. I heard Martin Fowler call this the “Agile Industrial Complex” and “faux-agile”. It still feels like a Seinfeldian-bizarro world at times. And I see no clear convincing scientific evidence that software developed this way is conclusively more effective. And that’s not to say it doesn’t work and work very well. Some of the concepts have absolutely helped with DevOps and Continuous Integration/Continuous Delivery (CI/CD) success. But we had those general concepts 30 years ago, albeit without as many colorful product names or buzzwords.
The failing with Agile often has to do with the people doing the implementation, not the concepts. I see a lot of what I call “Spinning in Place”. Even with well-meaning, highly qualified teams. Development cycles get a lot done, but don’t move toward a completed set of mission requirements. This can come from both sides not understanding the spirit of “Customer Collaboration over Contract Negotiation”. They become obsessed with the business aspects of getting a product “to market” quickly. As a developer of reusable software, I understand this. But it doesn’t always apply. It seems to come from simply having discarded the larger picture and value of accurate requirements, often exchanged for a ravenous and addictive taste for bite-sized stories.
There are plenty of other Agile trapdoors that can be investigated and avoided. That is not my concern nearly as much as a simple loss of the spirit and understanding of Agile in areas that are not often investigated. I have heard it said by the founders that the name may have been a mistake and that something like Adaptive might have served better. I agree. Simply stated, we need to get back to the point where changing the process is acceptable. A Stand-Up or Sprint Review or Retrospective should really excite the team and add value through improved communication and information sharing. If it’s not working, then I say change it up! Understand why you need it but find a way to make it work. For the team’s sake. I know it sounds like heresy but don’t be afraid to find a way that works better! But you can’t do that until you really understand the spirit behind, “Responding to change over following a plan”.
Andrew Hunt described it this way, “The word ’agile’ has become sloganized, meaningless at best, jingoist at worst. We have large swaths of people doing ‘flaccid agile,’ a half-hearted attempt at following a few select software development practices, poorly. We have scads of vocal agile zealots — as per the definition that a zealot is one who redoubles their effort after they’ve forgotten their aim.” Given this religious fervor, it won’t be easy. The genie rarely goes back into the bottle without a fight, and customers feel like they are getting a great deal of work from the team with this forced structure. There is pressure to continually standardize or reduce the length of sprints, like that is chiseled in a stone tablet or cast in deathless bronze. James Grenning explained, “Generally, I am critical of what most Agile adoptions have become, Agile in name only (AINO). AINO adoptions leave developers feeling like they are being micromanaged and pressured to do poor work.”
Do we see a trend here in where the failings come from?
So, while it is useful to examine the shortcomings of the Agile technical evolution, it is more important to see what the true goals of the Agile Manifesto were, where they went wrong, and what we can do about it. Many of the creators have tried to do this. As Ward Cunningham (creator of the Wiki) said, “Agile is the accepted methodology, and people are ready to find the next variation of it, to master its intricacies and respond to the computing opportunities that are out in front of us and however we need to organize to do that.”
It is also interesting to see all the directions the creators have gone and some of their creations. They have, for the most part, attempted to live the vision, to continue to adapt, to create, to try new things, and make software development interesting and empowering for software developers and the engineers and team members that work together to solve problems. Andrew Hunt has his GROWs method, others have gone on this to expand Agile in more business-oriented areas (business agility), or simply helping corporations adopt Agile practices effectively.
So, all this to say, what if “spinning in place” isn’t just about Agile development losing its way in terms of software development methodology. What if the metaphor runs deeper? Maybe this is more about losing the spirit of humanity that was initially instilled in the movement. Is this all about business practices? Are we back to just shipbuilding again? Have we lost the spirit of diving for pearls? Of finding beauty and perhaps self-improvement? Where is the joy in what we do and what we accomplish through the spirit of a team that communicates well, shares knowledge, and a passion for learning their trade, while taking pride in supporting their product? Does this need to be spelled out in a new Manifesto?
Kent Beck noted fairly recently that the most important skill he was learning was “compassion”.
Interesting. Leave it to Kent to get me spun up again.
That is the type of spinning I can appreciate.
Allan Gross grew up in the Washington D.C. suburbs playing in original and cover bands and attending many Elvis Costello concerts. He still gets on stage from time to time when not “punching the clock” as a Principal Software Engineer or Program Manager.