November 2019 marked the publication of a memoir by Ameritrade CEO and Founder, Joe Ricketts, The Harder You Work, The Luckier You Get. I led the internet initiative at Ameritrade to the internet and spent many hours being interviewed for this book. Chapter 10 (page 245) highlights some of that story. Following is some more detail.
Like many software development teams in the 90s, we were struggling to keep up, as the internet – rather than floppy disks – became the data delivery method of choice and previously successful development models began to break down. This new paradigm was powerful because we could just update all our clients’ software at the flip of a switch. But it was also fraught with danger as it became very inexpensive and easy to launch new code without appropriate quality control.
The Egyptians were not Agile
In the olden days, almost all projects were delivered following a waterfall project management process. Even the pyramids likely started with some sort of primitive blueprint which included every thought and idea that the hoping-to-be-immortalized king might have in his head. As challenges to designing crept-up during construction, they were dealt with via design compromises. As a result, end projects were almost always something less than they were initially intended – even if nonetheless cool.
As people got better at building skyscrapers, ships, and bridges, they got better at estimating the time a project would take. Estimates were created based on the average of past projects scaled up or down to match the one at hand. We can imagine the thinking of an engineer in 1890:
That last bridge took 2 years and was a mile long. This new one is 20% longer so it should take about 29 months.
As our society of builders approached the middle of the 20th century, waterfall project management was very nearly perfected for capital projects. Engineers were aware of the details, pitfalls, and requirements of their projects. They were good enough that we were able to put a man on the moon!
Old practices didn’t fit new products
But around the time of Apollo 11, a new generation was coming of age who would challenge the time-tested results of waterfall concerning software development. Capital projects had been physical-resource-based. Metal, factory space, rivets, steel, and labor had gone into building the monuments of the previous two millennia. The new monuments were to be made of information, communication protocol, mental horsepower, and electricity. They deserved a new method of project management that matched their intangible materials.
Taking cues from Japanese manufacturing, their own experiences, and the changing relationship between programmers and users, computer engineers began to advocate for smaller production cycles that were more flexible. They sought to replace the heavy process of waterfall with something lighter. They promised to increase their accountability and transparency in return for a seat at the planning table and the opportunity to honestly manage executive expectations. Enter Agile.
There is a romantic notion within the software development community that in 2001, a group of 17 computer scientists got together in Utah and invented Agile software development. But that is really not what happened. In truth, engineers all over the world had been wrestling with the problems of building software in executive-driven environments and by the 1990s were coming up with similar solutions. The birth of the internet, and the explosion of users demands, really kicked the discussion into high gear.
Ameritrade
I was in Omaha Nebraska in the 1990s. Omaha might not have much of a silicon reputation, but due to its central proximity and favorable demographics, it was an early hub for military, communications, and technology providers. My company was Ameritrade even though it was called TransTerra at the time. Our technology team was part of the greater Omaha tech community. We knew each other, met regularly to discuss issues and share a beer and traveled to the same conferences. You can always spot a traveler from Nebraska because invariably they will be wearing red or have the state name emblazoned across their chest. It helps when they get lost.
As TransTerra changed its name to Ameritrade and its number of clients grew from thousands to millions, we were under unprecedented pressure to revamp the way we built software. At the time Amazon first bragged about $1 million days, we were already seeing $100 million days. Our development cycles were built around exploding demand, for which our capacity increasing releases were regularly inadequate. And every day we were learning how better to present our interfaces. Clearly, long, waterfall development cycles were not going to work anymore.
In the spring of 1998, following the crush of a few 80-hour/week development cycles, we put a stop to the treadmill and took a step backward. For nearly three weeks, the development team and I locked the doors, sat together, and wrote what was to be Ameritrade’s new software development process. We called it the Cooperative Software Design Process.
I wish I still had a copy of that original document, but it is long gone. What I do have is a PowerPoint simplification, dated a year later (1999), that was presented to executives and new management as part of an initiative I worked on with my buddy, Ronny Gal from Boston Consulting Group (BCG). It is interesting to review and note how many similarities there are to what we now all know as Agile.
I have attached one particularly interesting slide from that deck. Much of it will feel familiar.
- Cooperative acknowledges that the stakeholders were part of the process.
- Concurrent iterations allowed us to deliver features more quickly.
- The overlapping snail-shell arrow that has come to define Agile diagrams
- That spinning idea generator up front was our version of ongoing backlog tasks
- The little note in the bottom of the call-out box “smaller is better”
We missed a lot of important Agile components too. We didn’t think of time boxing, completed software instead of status reports, or daily stand-ups, but those things probably weren’t appropriate to Ameritrade at the time. We were creating a process that allowed our business units – and subsequently the end-users – to understand that they were in control of what we were doing. In retrospect, it was quite appropriate.
The emergence of the internet changed a lot of things in the mid-90s. Companies that didn’t embrace it went away and those that did were forced to re-evaluate the way they did business. Few companies were as shaken and shaped as much as Ameritrade. Every element of the company was put into flux – but nowhere greater than what became the internet development team. We were pioneers out of necessity and lucky that the pre-Agile discussion was one of which we could be a part. These concepts helped our team progress to maturity and paved the road for Ameritrade’s growth and eventual position as one of the largest brokerages in the country.
Additional posts in this series:
- Agile Without a Learning Curve
- Story Points, Agile Estimating, and Today’s Puzzle
- Better Estimating through Fibonacci
- Winning-over Executives with Agile
- Selling Agile, and The Cost of Uncertainty (coming soon)
- Agile and Waterfall: Comparing Unknowns (coming soon)
A Card Carrying Centrist 