At one time software projects were planned like construction projects, marching forward one phase at a time in sequence, ending with a complete delivery of the finished building or piece of software. We called this a "waterfall" process because the effort cascaded out of one phase into the next.

Waterfall projects suffered from two big sources of delay. The first source of delay was in the requirements phase. It was hard to figure out everything that application users wanted to do in the future. Projects tended to have a long "fuzzy front end" period to work out requirements. A project might never get started if requirements kept changing.

The second source of delay was in the integration and debugging phase. The components of a large project rarely worked together as expected, and the integration phase could be longer than expected (and sometimes infinite).

With a waterfall project, you ran the risk that the project would never start, or never finish. That is why the world moved to Agile.

Agile projects get started faster and get to integration faster. The Agile lifecycle starts with a short phase of roadmapping and prototyping. After that the system proceeds directly to a series of short build and release cycles.

Repetition leads to efficiency. Agile projects have higher productivity because a much bigger part of their lifetime is spent in programming iterations. Developers have time to optimize and automate so that each iteration is more productive than the last.

While a waterfall project is intended to be finished and delivered, an Agile project is never really done. Agile projects have an evolutionary lifecycle rather than a plan. The variable scope can be used to make products that are better than the original requirements.

All Agile methodologies depend on frequent releases. This eliminates the risk of non-delivery, and lowers the risk of bad results. Each release resolves integration problems. It also provides an opportunity for end users to see the product and provide feedback, so the development team can receive detailed, up-to-date information on real user requirements.

In order to guarantee frequent releases, most Agile methodologies release on a fixed schedule, rather than when features are finished.

We can imagine the development of a software feature as a sequence of steps, from figuring out what to do (the fuzzy front end), to defining the required tasks and assigning them to team members (task management), to changing code (code contribution), to building, testing and deploying the result.

The many variants of Agile are different types of team and task management, so traditional "Agile" methodologies only take advantage of one piece of the puzzle.

The most popular methodology is Scrum, which engages "self-managing" teams of 5 to 10 people. They plan and deliver software in iterations that are typically 2-6 weeks long. The process can be very motivating for team members, and it provides a framework for regular process improvement. For these reasons, Scrum is extremely popular in large organizations, crowding out other Agile methodologies.

Scrum will probably work well for you if your teams meet these requirements:

• Your teams are all 5 to 10 people.
• Each team is co-located and meets in person every day.
• They implement with iterations that are two to four weeks long.
• You release software every four to 26 weeks.

However, I rarely see these conditions in cloud-based development.

Teams sticking to Scrum will have problems:

• Releasing rapidly.
• Scaling to distributed groups.
• Engaging large numbers of new contributors.

Fortunately, there is a continuous form of Agile development that works better for frequent releases, distributed teams, and large and diverse groups of contributors. Continuous Agile pulls in new ideas from coding workflows, code review, automated testing, and continuous integration. These produce much faster release cycles and more scalable teams. And when we get to build, test, and deploy methods, we find an avalanche of new capabilities from cloud computing providers.