Egg’s iterative development method was a customised version of eXtreme Programming (XP) created by Kent Beck and consists of an outer release planning cycle, with an inner cycle for planning iterations, and developing and testing the code
The key planning practices embedded were:
- Planning Games – the approach for planning releases and iterations (see below)
- Small releases – breaking work down into smaller, more achievable chunks. Minimises risk and impact of change. In Agile Egg, releases were strictly less than three months
- On-site customer – nominating key business representative – called the “Truth” in Egg Agile terminology – to each project
- Sustainable pace – planning and working at a sensible pace that can be maintained, no “death marches”
- Information Radiator – communicating clearly and openly on progress and related project information
You can read an overview of the method here…
Http iframes are not shown in https pages in many major browsers. Please read this post for details.
Planning Games
The Planning Game handled the two cycles, and worked with different levels and accuracy for the work estimates:
- Release plans (Outer Cycle) — Based on rough story estimates allocate stories to iterations culminating in a software release
- Iteration plans (Inner Cycle) — Based on detailed task decomposition allocate stories and recorded team velocity to next iteration
The Truth was a fundamental role and represented the business on a project, accelerating the development cycle by collaborating with the Development team to provide rapid feedback, advice on required outcomes, and was responsible for prioritising work according to the business needs.
Development Practices
Underlying the iterative cycles were a number of development practices drawn from XP:
- Test First – Test cases are written before the code, and based on clear acceptance descriptions from the business users (see Test First Development below)
- Pair Programming
- All code written in pairs at a single workstation – You may need to move furniture, have convex desks, with large/multiple screens
- Developers take turns to drive
- Constant QA
- Pairs work together for less than a day
- Share knowledge by rotating pairs frequently
- Metaphor
- The architecture is captured by a system metaphor, a simple shared system of names and roles for higher level components shared by customer and developers.
- May be an analogy of how the whole system works which guides development.
- This also embodies the principle of “emergent architecture” which evolves and mutates as development proceeds
- Collective Ownership
- Code is owned by the team not by individuals
- Everyone has the right to check out and modify any code
- Developers are empowered to fix problems when they are found
- Continuous Integration
- “Nightly builds are for wimps”
- Pairs aim to integrate code into the baseline several times a day
- Tasks are planned as smallest chunks that can be released
- Code is written test first, all tests must pass before code is released
- Refactoring
- Restructuring the existing code to improve it for development/technical purposes, but without its external behaviour
- Design becomes a process of discovery, rather than invention
- Coding Standards
- Team agrees on a set of coding standards they adhere to
- Reached by consensus
- Everyone agrees to follow them
- No reformatting according to individual tastes!
- Same development tools and preferences on each workstation
- Any person can pair equally well on any machine
- Developers may find this hard – There are advantages of collective ownership in lack of blame culture
- Team agrees on a set of coding standards they adhere to
Test First Development
Test First Development together with Continuous Integration contributed significantly to an overall reduction in defects reaching live.
The essence of Test First Development (TFD) is…quoting from the Agile Egg practice document…
…using the creation of tests to drive development and testing the smallest thing possible in isolation from other things in the system so that if the test fails you know exactly why it failed. Without test driven techniques it is difficult on an agile project to spec out what the code needs to look like and how to be sure it works right. Code is often not documented so it’s hard to understand existing code. It is difficult to add functionality to it or to refactor code safely so it doesn’t break.
TFD is implemented by following these steps:
- Write a test for a new piece of functionality
- Write some code that fails the test for the right reason (i.e. compiles and fails because the test is testing something)
- Write simple code that passes the test (not yet refactored, maybe containing hard coded values, maybe ‘known’ to be wrong)
- Write a test that fails that fails because the code contains hard values or because the code is wrong.
- Write code that passes this test
- Refactor as necessary
- Repeat
Most people miss out steps 2, 3 and 4. You can do this if you are confident but including steps 2, 3 and 4 is better if you are struggling to make progress and can lead to better code, it forces you to take small steps, which is good if you are running into problems by taking steps that are too big.
The outcomes from TFD are manifold:
- Complete automated unit test coverage of the code
- Code that doesn’t contain unnecessary stuff, it only exists because there is a test that means it has to exist. Necessary means ‘needed to pass a test’.
- Higher quality code, simpler, smaller without being obscurely terse. A good example is the CMS rewrite, a lot shorter than the original.
- The tests achieve the side effect of documenting the code
- Easier to add new functionality
- Obvious when new code breaks existing code because tests fail
- Supports collective code ownership, others can edit your code easily
- Proves the correctness of the solution
- Informs the developer when a task is complete
- Focus the developers on the small-scale task at hand
- Drives the design of the solution (by making use of the code before you’ve even written it)
Acceptance Test Driven Planning
In terms of planning, the Iteration cycle is structured as per the diagram below with one working day planning and nine working days for development. The planning day is structured into two main parts: closing the previous iteration and planning the next.
You can read more about the Acceptance Test Driven Planning process in the paper “Acceptance Test Driven Planning” by David Leigh-Fellows, an Egg alumnus, and Richard Watt, of Thoughtworks.
Agile Management Tools
The XP approach to managing agile is/was quite profoundly physical and tangible with real postcards and “stickies” on wallboards, and back in 2003 there were few software tools available back in 2003 to manage agile development electronically. And so some intrepid team members set out to develop their own tools to help manage user stories, called XP Story Studio. The main site for XP Story Studio was taken down some time ago, but you can still see the site if you dig into the Internet Archives’ Wayback Machine here.
The front page looked like this…
And here are the screenshots…
Also, if you feel really intrepid you can also download V0.99 of the software from the archive, but it requires .Net 1.1 support and so an old Windows Server (2008 and back) to run it on…
Feedback on V1.0
In the spirit of iterative working a retrospective view of Agile Egg V1.0 was taken which produced a number of recommendations to be considered in the development of Agile Egg v1.1:
- It is unlikely that a formal development method straight out of the box will be a good fit for Egg. Although, it is possible that some project management techniques could be drawn from DSDM.
- Agile Egg v1.1 should be evolved from what works (via workshops with stakeholder groups), remaining a light-weight framework adaptable to project context.
- Process definition needs to be expanded to an end-to-end lifecycle that includes a wider set of roles (QA, Security, Architects, …).
- Teams across ETS could become more consistent in use of tools such as Test Director and XP Story Studio. Some teams could improve use of existing practices such as Velocity Tracking and Retrospectives, some Governance might assist in this.
- XP development practices (pair programming, continuous integration, refactoring and test driven development) should not be attempted on existing mixed technology stack until obstacles are removed (e.g., by providing appropriate tools and environments)
- Consistent approach to Risk Management
The feedback from business people was, in particular, revealing of the need to spread the understanding of the iterative methods into the business areas, outside of the technology development teams:
- Agile Egg defined by ETS for software development rather than end-to-end lifecycle across the business
- Project initiation process should get stakeholders involved from start
- Non-IT agile roles and responsibilities not clear, e.g. where do process engineers fit?
- Domain experts asked to perform Truth role without analysis skills/training
- Some Truths have operational responsibility so not enough time
- Hard to keep track of big picture progress
- TTL resource constraints often delays delivery
- Content Management sometimes left out of the loop
You can read more about the feedback below
Http iframes are not shown in https pages in many major browsers. Please read this post for details.Part of a bigger picture – facing the Architecture issue
The feedback from business people highlights the point that software development process was in Egg, as it is anywhere, part of a bigger picture…
…and there was a need to ensure that various puzzle parts fitted together properly.
In particular, another key insight from the early experience was the conflict between the concept of emergent architecture within XP and the need to work within the existing and developing architecture of the Egg IT environment. In particular, it was essential to comply with the strict requirements for IT security that are incumbent on a Financial Services organisation.
Therefore the development approach was subsequently aligned with a set of key architectural principles:
- Revolutionary Customer Experience
- Secure by Design
- Operational Excellence
- Fit for Purpose
- Business Led
- Flexible and Agile
Next…