Observations from a Large Scale Agile Project

agilityWell, I’ve been living and working in Bristol 3-4 days a week since last October (2013) and working on a pretty interesting and pretty intense project.

I’m the technical lead on a large scale Agile transformation project that will be used in 22’000 locations across the UK by 60’000 people, with 40 million processes completed per year. It’ll be developed using Open Source technologies and will align with the Government Digital Services assessment criteria.

There are 3 multi-disciplinary scrum teams with around 12 people in each team; and thats just for the software development side of things alone. In total, there’s around 150-200 people on the programme, including a number of additional software and infrastructure suppliers who are looking after software assurance and devops.

Some key observations:

  • Communication is key – with this many people, it’s important that information is shared quickly and accurately. Scrum of Scrums is a decent forum, but I’d also recommend that the multi-disciplinary stand-ups are used too, i.e. Tech leads, UX, BA etc. Share information here across the disciplines, and have it relayed back to the teams and back to ‘management’. People who feel engaged in the process are happier, and it has the bonus effect of both killing rumours and moving progress in the same direction across all teams.
  • Governance of Shared Services – On this programme, there are 3 other parallel projects. Each of these other projects have their own requirements for Authentication / Authorisation, Integration, Document Generation etc. It’s very easy to think ‘lets do it once and share across projects’, but suddenly you are confronted with the functional and non-functional requirements of all these other projects to consider. Don’t under-estimate it; the governance alone is a full time job and it needs customer ownership.
  • Developers argue – Everyone can and often will have a different opinion and sometimes it can get heated. The important point is to listen to all sides and share the concerns. If you don’t lance that boil, it can go septic. This is especially important when you have a multi-supplier delivery team. Have a session across teams to look at best coding practices and share them. Do this frequently.
  • Open Source has it’s problems – What version of MySQL should you use? Oracle Classic or Enterprise, what about the DB Engine? Percona? TokaDB? How’s it covered with licensing? What about the Jasper reports? Can the community edition be automated with Puppet? Start considering early.
  • Build environments early – Your environment code (i.e. Puppet configs) should be centralised and managed under source control. Every single environment should be built from this repository programatically rather than manually. If your scripts get out of line, then your environments get out of line. No-one wants a Dev and Build server that are different. Your environments should evolve in the same way as your application – this means destroy and rebuild per sprint. Doing this ensures that  you are thoroughly testing everything. Make sure your infrastructure people/suppliers are doing this.
  • Performance Test early  – Performance testing needs to be in-sprint. You need the feedback to come back to you for prioritisation for future work. After all, it’s part of the Agile Testing Quadrant.
  • Give People Opportunities – Get a second-in-command. This ensures that you aren’t a single point of failure, and it helps develop them as well. Roll this approach out through the team. Additionally, have good communications directly with the people on your project to find out what they want to do and be, then see what you can do to assist them on that journey. It’s important to make the time to talk to people directly, otherwise you can lose touch with whats actually going at the coal face.
  • Empower and trust your people – Similar to the point above; encourage and develop your teams ability to think for themselves. Better to have a team of individual, intelligent, autonomous individuals that can own decisions, rather than drones that think you have all the answers. However I will say that this approach to be tempered with some governance, to ensure that each team isn’t re-inventing the wheel – communication is key.
  • Review and improve often – It’s important to have retrospectives at each level, be it scrum team, tech leads, UX etc. Look for the rough edges in your processes and see what can be done to streamline things. For example, we had a failing build because all the teams where committing at the same time – a simple thing like having the tech leads own the commit sorted it. Green Build.
  • Don’t assume it’ll get picked up – Self organising teams are fantastic, but when you’re dealing with passing information over to other suppliers, it’s best to ensure that there’s an owner that can pull things together, at least for the initial few iterations. Pull together a quick and basic checklist to ensure things don’t get missed when handing off software to another supplier.
  • Agile vs Waterfall – It’ll happen. Despite the best intentions of Agile communication, you will inevitably hit a part of your project or programme that is waterfall based. For example, your software development might be a lean, mean, agile sprinting machine, but you cant really roll out to 22’000 locations each sprint. It’s important to align your sprints with the wider programme deadlines. Beware of agile frameworks as well – I’m not convinced on that front.


New Year, New Role, New Tech

Open SourceSo, 2014 is here, and it’s been here for nearly 3 months. I’ve been quiet, but thats because I’m now working on a new project based in Bristol. Strangely it’s a million miles from the tech that I normally work with, i.e. principally .NET.

This new project is all open source. We’re talking PHP, MySQL, Apache, Linux, Java, Vagrant, Puppet, shell scripts, Mongodb, Memcached, SAML and potentially cool things like yubikeys. Additionally it’s Agile based and I’ve a big whiteboard with all my tech stories on it. Class stuff.

I cant go into details on the project, except to say that it’s a big one for me and I’m absolutely loving it. This blog will change over the next few weeks or so as I adopt a new colour scheme and theme.

Anyway, it’s good to be back on here. Took me a while as I’ve moved to using a mac and haven’t a clue what tools to use to get access again 🙂