Building a Cloud Platform: Weapons of War

I've just returned home from a big development build. For the last five months we've been in a secret room racing to get version 1.0 of a cumulonimbus size cloud project out. It wasn't easy. Getting to 1.0 was a real battle because... well, you don't know! Because you weren't there... man!

Now that I am back on civilian soil and fighting off night terrors from the horrors of war, I can reflect on what went well and what when badly. This could be a book in its own right; but for this post I'm going to cover The Weapons of War - or the essential tools needed when building any product or platoon... I mean platform.

Essential Dev tools 

This was a .Net, Microsoft Azure project with four teams working in different locations on various components that all had to integrate harmoniously. To add to the challenge there were 14 different environments that all needed configuration, CI and automated deployment.

Source Control and CI 

We used GitHub and Team City for Source Control, CI and Automated Deployment. Team City, GitHub and Azure worked really well together thanks to awesome Powershell scripts created by the Development Operations (Dev Ops) team which hooked into Azure via the Azure Management Services API.

The role of Dev Ops was essential to a cloud project of this size and one that came into sharp focus in a world without using any IT or Hosting Services.

We used the Git Flow model for branching and release management. The ease of switching branches in Git - namely speed, thanks to having a full, local repository to work with rather than doing everything client / server side - combined with a Team City setup that allowed one click deployment of any branch to any environment made Dev and Testers lives infinitely easier. We haven't used the Git flow tool yet but rather manually followed the model manually using native Git commands and GitBash and GitGUI.

There was still the challenge of knowing what goes into releases and coordinating that across the program which unfortunately didn't have a magic tool which made that happen. If you know of any good release managers...

Azure Tools

As this was an Azure build there are two tools which you can't live without: Cloud Storage Studio 2 and Azure Diagnostics Manager from Cerebrata.

CSS2 gives you access to Azure Subscriptions and Storage accounts and a host of management and diagnostic functions that are essential to developers and testers.

ADM is focused purely on diagnostics and provides an intuitive UI for reaching into the cloud and pulling out essential performance, infrastructure, Trace and Azure logs.

Testing and other bits

Something we missed on this project was access to VS.Net Ultimate for its Load Testing features. But with a host of load testing products like Fiddler Stress Simulous - a great load testing plugin for Fiddler - you can and should load test your site and services regularly and easily.

Another nifty tool I discovered in the project was a Code Comparison tool called Beyond Compare.

Other Dev tools which should be baked into your build from the beginning are Style Cop and FX cop which will help keep coding standards maintained through the project.

Essential Communication Tools

One of the biggest challenges in any project which exponentially grows with the team size and distribution is communication. Defining the right channels, forums, groups and timings for communication are essential to hitting the information efficiency sweet spot where the right people are kept in the loop and communication isn't too much or too little.

To facilitate this there are some tools which make the job much easier. It is worthwhile to point out that the key to success here isn't the tools but rather a combination of rigour and discipline to keep communication in check and adaptability to realise when something needs to change to meet the projects needs.

Skype

The success of any communication tool is largely driven by penetration - or how many people have access to it. Skype is ubiquitous in Western Europe which means that any new team members will most likely have a Skype account and can join in the conversation with zero setup and cost. Free Skype offers one on one calls, group calls, IM and screen shareing which is pretty much all you will ever need.

TeamViewer

The weakest part of Skype is its screen sharing capability. For screen sharing and remote control we used Team Viewer which is fantastic if you want to work with others in another location. It works over port 80 so you never have any firewall problems either.

The JIRA Trinity

 Effective Task and Knowledge Management are the cornerstones of any project.
Tracking tasks and time is essential to knowing what needs to be done, who is going do it and when it will be finished. A good tool should work well for both the practitioners (developers, testers) and the managers (PMs) on a project.

Task Management

 We used JIRA as task and issue management system. There are many other great products out there but JIRA does seem to be one of the best on the market. It does well to find the right level of customisability and control to meet the diverse needs of projects and also keep standards and patterns of usage in check. It also had great agile plugins like Greenhopper.

No matter what you system you use you should have a central  hub that everyone on the project should interface with from developers to project managers to testers and even ux and designers.

Knowledge Management

Managing and maintaining knowledge through documentation for a project is essential for ensuring the important information on a project does disappear when people do.
Getting documentation right is very difficult and needs buy in and investment from every team member. It shouldn't be shouldered by one person. Also, Documentation is something which often is either over or under-invested invested in.

When overinvested a lot of time is wasted on writing words and drawing pictures which could more efficiently be communicated through a face to face discussion, a drawing on a piece of paper, a few bullet points or just sitting down and writing code. Documentation goes stale very quickly too.

On the flip side, no documentation means new starters will really struggle to get up to speed with the system and on boarding will become a very inefficient process. Also, people on a project forget stuff so documentation is essential to ensure key information is easily accessible.

Since we're using JIRA it would have made sense to use its Confluence wiki plugin. We didn't but opted for the GitHub wiki. The GitHub wiki is very basic but good for small project. It's lack of functionality like being able to attach files, IA organising tools, search and a WSYWIG editor made managing large knowledge base for multiple teams very difficult. We're now planning to move to Confluence.

So fellow veterans and new recruits. These are tech rash's current top tools for epic scale builds. Being a pragmatist by nature I'm not decreeing that you use them (all) but if you don't your project could end up coming home in a box.

Useful Links

Cerebrata
GitHub
GitFlow
Team City
JIRA
Fiddler StressStimulus
Beyond Compare
Team Viewer