As part of my series about the “How Businesses Pivot and Stay Relevant In The Face of Disruptive Technologies,” I had the pleasure of interviewing Cliff Berg.
Cliff is a lead consultant for the planning and execution of many Agile or DevOps transformations. Cliff’s former company, Digital Focus, which he co-founded in 1995, was a survivor of the Internet bubble, and was a pioneer in the adoption of agile methods. Cliff has written four books, most recently Value-Driven IT. Cliff believes that business value is central to IT decisions and that IT needs to have a more actionable and tangible view of the concerns of business. Cliff is an experienced public speaker, instructor, and facilitator, as well as an experienced software architect of complex software systems, and has built high-volume Internet and data processing systems as well as compilers and simulators. Cliff has a BS in Engineering Physics and masters degrees in Nuclear Engineering and Operations Research from Cornell University.
Thank you so much for joining us in this interview series. Our readers would love to “get to know you” a bit better. Can you tell us a bit about your ‘backstory’ and how you got started?
After studying nuclear physics at university in the 1970s, the early part of my career was spent as a nuclear engineer. I quickly lost interest in nuclear as a career path following the Three Mile Island incident in 1979 as the industry shifted its focus to health and safety protocols and research, which wasn’t of interest to me. After this I went back to university where I gained a degree in operations research, to become an electrical engineer in the area of designing compilers and tools for designing circuits.
I started my journey as an engineer on the road, speaking to other engineering groups, deriving their knowledge and expertise and learning the tricks of the trade. I then moved on and began working for a company called Intermetrics, which built compilers. I was part of a team that developed VHDL, which is a hardware description language used in electronic design automation to describe digital and mixed-signal systems such as field-programmable gate arrays and integrated circuits. As part of this project, I wrote the first-ever synthesis compiler that could translate any algorithm into hardware level VHDL. I worked in this sector for some time before trying my hand at the graphics realm.
It was around this time that an acquaintance told me about a new programming language called Java. Noticing one another’s ability to identify technologies that would change the world, we decided to go into business together. We got together using our own money to fund a company called Digital Focus, which turned out to be hugely successful. Within five years we boasted 200 staff members and high-profile clients such as IBM and FedEx.
In 2000, while he was CTO of Digital Focus, one of Cliff’s project managers showed him a book that had been recommended by one of their most capable software architects. The book was called Extreme Programming Explained. This marked the beginning of his lifelong relationship with Agile.
Can you share a story about the funniest mistake you made when you were first starting? Can you tell us what lessons or ‘take aways’ you learned from that?
Well, I assume that by “when you were starting” you mean starting my career — not starting any of my current initiatives.
I remember that when I began my first job as a nuclear engineer, I was given a private office — that was the norm in those days — and I had no clue what one did in an office; so the first thing I did was close my door, put my feet up on my desk, and start writing letters to some friends.
That kind of set a pattern. I was not focused, and eventually, I learned the importance of focus. I would say that above all else, one needs to always be examining oneself and making sure that things are not going sideways: that one’s decisions are always thought through, and not ad-hoc, and focused on one’s objectives.
None of us are able to achieve success without some help along the way. Is there a particular person who you are grateful towards who helped get you to where you are? Can you share a story?
Oh there are so many. One is Dianne Houghton, who became the CEO of the company that I co-founded. She was a very valuable mentor. She taught me about leadership, and I learned how to facilitate group discussions by watching her. We hired her because she had written a book about business strategy. She had been trained at Anderson Consulting, and she had enormous discipline and focus, and she always stayed calm and rational.
Extensive research suggests that “purpose-driven businesses” are more successful in many areas. When your company started, what was its vision, what was its purpose?
When we began this project our main vision was to redefine Agile principles to enhance the value of software projects and other kinds of team based initiatives as well. Agile 2 is a collection of ideas derived from the original Agile methodology, which has been transformed and adapted to offer a more modern approach to project management. Unlike the original Agile methodology, Agile 2 focuses on leadership, product design, and collaboration in all areas, as opposed to strict face-to-face communication. It incorporates data into project management, something that is not mentioned in legacy Agile, and rather than focusing exclusively on completed software as proof of value Agile 2 focuses on the business outcomes and demonstrable progress towards a project’s business goal.
Thank you for all that. Let’s now turn to the main focus of our discussion. Can you tell our readers a bit about what your business does? How do you help people?
I currently have a boutique business called Agile Griffin, which provides Agile coaches who also know DevOps methods. That is a rare but very powerful combination. But I’d rather talk about my other non-business initiative, which is Agile 2. Agile 2 is an attempt to realign the Agile movement, around more sound principles. The Agile movement has long ago permeated the executive suite. You might be familiar with Steve Dennings Agile column in Forbes. The movement began as a disruptive rejection of big plan methods that had become commonplace by 1999, and that do not work for building software. Software is always different — you never write the same code twice — and so you can’t know ahead of time what all the issues are. That’s only one reason why planning it all out upfront does not work. Agile says, “Don’t even try to figure it all out. Instead, build a little at a time, and try it as you go”. That’s a simple idea.
Unfortunately, there are other ideas that came with it, which each had a kernel of truth, but which got distorted into absurd extremes. For example, putting everyone in a big room so that they can collaborate. Remember these people — software developers — are building complex things, and they need focus. Who can focus in a big room, with side conversations happening all the time and people coming and going? I know I can’t.
The Agile movement also said “no” to having autocratic project managers tell the programmers how to do their work. But in the process, it tacitly rejected any kind of formal leadership or management. But we need leadership and management. Agile 2 tries to bring those things back in, but in a way that supports true business agility.
Which technological innovation has encroached or disrupted your industry? Can you explain why this has been disruptive?
Commodity virtualization and cloud computing. These were created to scale. Companies like Google and Amazon needed to be able to support hundreds of millions of users. They could not do it without virtualization, and both Amazon and Google invented their own cloud technologies to enable them to scale. Amazon pioneered on-demand provisioning internally and then released it as AWS. Google pioneered containerization and large container cluster management and then rewrote that as Kubernetes. The Linux kernel team saw that containerization, which had been around for a long time, needed to really work, and so they beefed it up, and that made containers what they are today — trustworthy runtimes that start instantly and scale fantastically.
Those innovations led to on-demand integration testing, which became known as continuous delivery and DevOps. Early descriptions of continuous delivery at conferences were well received, but the Agile community largely ignored it, because the community had become obsessively focused on Agile frameworks, and so DevOps became its own movement, apart from Agile. But Agile and DevOps go hand in hand, and so the reality is that to scale Agile, you need DevOps. This has created a dilemma today, in that organizations need Agile coaches who also know DevOps, but very few do. We, therefore, have a kind of bottleneck: the unavailability of people who know both Agile and DevOps.
What did you do to pivot as a result of this disruption?
Agile 2 tries to address a whole range of dysfunctions. The Agile/DevOps chasm is only one. Fifteen people were involved in developing Agile 2 — people with expertise in product design, program management, business, system engineering, human resources, learning theory, Agile and DevOps of course, and other fields, because we knew that all those are elements that are needed for Agile to work. We began by discussing everything we feel is wrong today in the use of Agile ideas, and with those ideas themselves. We then synthesized our views and derived a set of principles. These principles are not absolutes. Agile is about human behavior, and when dealing with humans, there are no absolutes. But the principles apply “in most cases”. Because of that ambiguity, we felt it was important to provide our thinking behind each principle so that the “why” would be present — people can read why we feel that a given principle is true — most of the time.
About the Agile/DevOps chasm, we need to first acknowledge a reality today: that most large businesses operate on a technology platform. Amazon operates on its e-commerce website, which they created and is unique to them. Google operates on its massive search engine. Tesla operates on the advanced technology of their factories, producing cars of its own design that contain batteries of Tesla’s design. Pretty much any large company you name today, runs on a technology platform.
What has also changed is that change is more frequent. Amazon deploys new platform features all day long. They can tell how well those features are received by their customers, and rapidly pivot based on that feedback. That’s business agility. In other words, Amazon is competitive partly because of its business agility; and it has business agility as a result of how its technology platform delivers new features. To spell this out: the way that Amazon delivers new features is a source of strategic value for Amazon. It’s IT function is not a mere cost center: the way that features are created and deployed is one of the things that makes Amazon so competitive. The “how” of its IT operation is what we are talking about — it has enormous value.
Agile 2 states that managers need to try to understand the “how” of how technology services are delivered to customers. Not just the end result, but how the results are delivered. Because if managers do not understand that, then they do not understand an important source of strategic value, and they are unable to make tradeoffs about improving the “how” versus adding new features to the product. For example, suppose you have to choose between two investments: (1) increase the speed of testing product changes; or (2) add a new product feature. Which would you choose? To know, you have to be able to assess the value of each of these. To assess the value of #1 intelligently, you have to know something about the role of testing of the product. You can’t make the choice unless you know how the product is made.
IT is no longer a cost center. It is the goose that lays the golden eggs
Was there a specific “Aha moment” that gave you the idea to start this new path? If yes, we’d love to hear the story.
I had been mulling over an Agile 2 for a very long time. In 2000 I was co-owner of a software development company employing about 200 people. At the advice of a lead engineer, we tried Extreme Programing, which was an early Agile method. It was a success, but I was concerned about the nascent Agile community’s seeming disregard for reliability and security, which as CTO were my main focus. Over the years I did not see this change. Other dysfunctions arose within the community. One of them is that the Agile community evolved an extreme bias in favor of real-time face-to-face collaboration over all other forms of exchange. That is great for highly extroverted people, but it tends to put quietly thoughtful people at a disadvantage. Some people — a lot of people — do not do well in group discussions. Books by Susan Cain, Cal Newport, and many others elaborate on this. The Agile community is far, far too biased in favor of methods that favor extroverts over those who communicate naturally in writing or in the one-on-one discussion instead of group discussion. That needs to change.
COVID created a crisis for Agile. The Agile-favored practice of all meetings in person became impossible. Yet many organizations found that working remotely went very well. Github is an all-remote company, and they use what they call “asynchronous communication”. The basic idea is that with a global workforce, and global time zones, it is no longer feasible to be able to get everyone in a room anyway: so we need to deal with that reality and make it work.
COVID created the watershed moment for Agile 2: it proved that a core tenet of Agile — that people collaborate best face-to-face — was either questionable or impractical. I assembled a team and we developed Agile 2. Agile 2 expresses potent ideas for how to honor different personality types and create effective collaboration and addresses a whole range of dysfunctions within the Agile community. And by the way, the dysfunctions that we uncovered were also uncovered ten years ago by the British Computer Society: nothing had changed in the past ten years.
So, how are things going with this new direction?
Amazing. We have been mentioned in numerous publications. We have a book coming out about Agile 2, through Wiley. So many people are “coming out” and saying — often for the first time — that they have been thinking these things, but were afraid to say so. Just the other day, one of our group, Lisa Cooney, was in a Women In Agile conference, and the topic of Agile 2 came up. According to Lisa, “It seemed to open them up to feel safe to criticize Agile, to name their feelings about the dogma and unthinking.”
It is important to note that none of the ideas in Agile 2 are new. All are battle-tested. What is new is that these ideas are expressed in a single statement about things that are needed for agility, and some of the ideas pivot or add nuance to ideas that are commonplace among the Agile community today.
Can you share the most interesting story that happened to you since you started this pivot?
I would say that the most interesting thing has been silence from those who have a lot invested in the status quo. Agile 2 challenges many ideas that are championed by various interests who make a lot of money from branding and selling those ideas. We have not heard from them. But that was expected given that we are still in the very early days. I was speaking to David Anderson, who launched the Kanban movement, and he told me, “First they ignore you. Then they laugh at you. Then they fight you. Then you win!”
What would you say is the most critical role of a leader during a disruptive period?
When is there not a disruptive period?!
The most important thing is to be relentless: to continuously push things forward. Continuous gentle pressure. People need to be inspired by what they are doing: they need to feel that it is leading to something that they personally embrace, both on a collective and a personal level. But people are pulled in many directions. As a leader, you have to keep them focused and moving forward. You have to check in on things, you have to ask questions, you have to step in if an issue is not getting resolved — not to tell them what to do, but to get the discussion going and nudge toward resolution. If you don’t do these things, you lose your window of opportunity.
When the future seems so uncertain, what is the best way to boost morale? What can a leader do to inspire, motivate and engage their team?
There are different ways to inspire people. Some leaders inspire people on an emotional level. They create a battle cry. Other leaders inspire people intellectually: they instill confidence in others that the leader can lead them to success. I like to think that I am the latter kind. And one dilemma for an organization is that those who respond to one kind of leader usually do not respond to the other kind. That means that most of the time, you will not be able to inspire everyone.
Is there a “number one principle” that can help guide a company through the ups and downs of turbulent times?
Sure. Focus on your core. And to prepare for hard times, which always come, don’t become dependent on one thing — one big customer, or one product. Diversity early, and then when hard times come, focus on those things that are most solid.
Can you share 3 or 4 of the most common mistakes you have seen other businesses make when faced with a disruptive technology? What should one keep in mind to avoid that?
I am going to answer this in a very specific way. Agile is disruptive, and the biggest mistake that I see senior managers make when they decide to “adopt” Agile methods in their organization, is that they assume that Agile is something that only impacts software teams. Senior managers typically think that the patterns that have worked for them throughout their career will work for Agile. But it is not the case. Agile is not a technology insertion. It is a total organizational change. It affects every job function — not so much what the functions are, but how they are done. Everyone needs to learn new ways of working. As such, it is an enormous learning journey, at all levels. And you don’t undertake a full-scale change like that without first learning all you can about it, from multiple sources, because when Agile works, it manifests differently in different kinds of organizations.
Ok. Thank you. Here is the primary question of our discussion. Based on your experience and success, what are the five most important things a business leader should do to pivot and stay relevant in the face of disruptive technologies? Please share a story or an example for each.
- Stay current in things. Things change. Don’t get behind.
- Don’t assume that you do not need to understand how the product is delivered.
- Focus on your products. Don’t treat them like a mere portfolio. Today, if you are not number one, you are vulnerable, and to be number one in a product category, you have to be really focused.
- Don’t separate product definition from product delivery. Today, these need to be joined at the hip. Agile does not tell you how to do that, either; but Agile 2 has guidance for that, and Agile 2 is, therefore, a good model to share with your product groups.
- The top one: define the primary kind or style of leadership that you want in your organization, and take steps to institutionalize that and make it happen over time.
Can you please give us your favorite “Life Lesson Quote”? Can you share how that was relevant to you in your life?
The quote is, “Youth is wasted on the young”.
For people starting out and seeking to have a good life path, I would advise them to talk to others who have been there. There is nothing like hindsight. People with experience know how things tend to go with different approaches. Our experience is tangible, concrete. We know what works.
How can our readers further follow your work?
You can find out more about Agile 2 by connecting with us at https://agile2.net/.
