Team Assessments — remotely and without grief…

As Lord Kelvin — and many others, point out — you can’t improve what you don’t measure, and, given the beating heart of any organisations are their teams, most organisations quite rightly perform some kind of assessment of their teams at regular intervals to identify issues, areas for improvement, and see trends over time.

There are many such assessments that are used in different circumstances, and at different levels in the organisation. Some of the most common ones include:

• Spotify Health Check
• 5 Dysfunctions of a Team
• Scrum Master Assessments
• Agile Maturity

…to name but a few. Organisations often even make up their own new ones to measure certain things over time.

Ideally, these assessments would be done in a big room with the questions printed out and stuck on a whiteboard. However, the world of work has changed, and the occasions when everyone is available face-to-face at the same time and place are increasingly hard to schedule. Even before the pandemic, many teams and/or workers were remote, sometimes in different time zones. With the new trend towards increased remote working, and fewer days in the office, scheduling becomes ever more tricky.

The obvious answer answer is to take the assessment online. It can still be done with all the team at the same time over (say) Zoom to get the same discussion and feel, but also non-concurrently if it is difficult to get all the team available at the same time.

However, simply sharing a set of PP slides and getting participants to write their answers in the chat, or similar, is not as good an experience as being round-the-table, so it makes sense to build something a bit more slick to increase the engagement for those doing the assessment. We discuss our approach to this below.

Doing it online does give us benefits, though, in terms of easier data collection and analysis, and also flexibility in the way we run the assessment. As an example, do the team want their answers to be anonymous, or not? Do we want everybody to answer before going to the next question, or can individuals do it in their own time, and we’ll meet up to discuss results later.

Every company I have worked with who do assessments come up against a similar set of issues. They will typically have a spreadsheet to store results per team, but this is a lot of effort to build and maintain, and it is usually a very onerous and time-consuming task for (usually) a Scrum Master who could make better use of their time helping their teams. Yes, I’ve been there, and the assessments usually stop if that person leaves or changes role.

The scope of the results is also very complicated; I would argue it’s far too complicated to be handled reliably by a spreadsheet. As a minimum, the following would usually be required:

1. The results for a single team for a single assessment.
2. The results for a single team over time to see trends; are things gettng better or worse for the team? What are the areas for improvement?
3. The results for all teams in an organisation for a single assessment to see if any of the identified issues are across the organisation, and therefor maybe need addressing globally.
4. The results for all teams in an organisation over time to see if those global issues are improving, or new issues emerging.

Managing all these on a spreadsheet is extremely difficult, particularly if team members change over time, or the organisation decides to add, remove, or change the questions or possible answers.

However, it can, again, be a lot of work to set these up, and to generate the analysis. The results of some assessments are simple to calculate — one question to one answer — but others (like 5 Dysfunctions) take the answers to a number of questions and combine them in some algorithmic way.

We realised, however, that once you’ve worked out how to store things like who is in which team, and how to generate the results, all you need to define an assessment is a set of questions, and a function to generate answers, and the assessment will just work. So, we built an assessment platform that could provide these, and any other assessment. The data analysis is then exactly the same, whatever the assessment; we want to see individual teams, or the whole organisation, and we want to see single assessments, or trends over time. Or, any combination of these.

An Agile Simulations licence gives you access to all the assessments listed above, as well as giving you the ability to add, edit or delete questions, or even define your own assessment. See Licencing for more information.

We believe that using the assessment platform will save enormous time and effort in running assessments, particularly in larger organisations, where (again, in my experience) you may well end up with different parts of the organisation doing their own thing, making cross-organisational analysis impossible. Having a single platform to run assessments and store results will make them much more likely to happen, and will standardise the results across the organisation. This vastly increases the value that can be gained from assessment.

People doing the assessment answer the questions on their own browser; no screen sharing is necessary, and this allows the responses to be anonymous if required. However, like pretty much everything in the assessment apps, this is configurable; you may start off anonymous, but the team may want to see each others’ answers and comments as psychological safety and transparency increases. Indeed, a team suddenly wanting things to be anonymous may be the indication of an issue in itself…

We have found that this kind of “own browser” interaction does increase engagement over screen sharing; here’s what we found playing one of our online games which uses the same technology — What We Learned from Taking Agile Workshops Online

Every assessment comes with all the data analysis as standard; as an example, here is an example graph of data per question over time for a whole organisation. Clearly, there is a n issue with people understanding the mission!

Here is a graph of aggregated question scores over time; looks like Easy To Release got very good, then suddenly very bad, then good again! Note that these graphs can get quite busy, so you can temorararily remove datasets (questions in this case) by simply clicking on the legend.

Here is a table for just the current team, and just the current assessment, for a 5 Dysfunctions of a Team assessment.

There are many options in how the assessment is run to fit in with how your organisation operates. As an example, you can allow (or not) comments on the questions. These can be simply seen by clicking in the results pane.

As with the answers to questions, comments can also be attibutable or anonymous by simply checking a box on the config screen.

In terms of configuration, there are many more options to allow you to fully tailor how you run assessments; do you run them every month? Or quarterly? Do you want everybodty to answer a question before showing them the next one? Do you want everybody to see all the results, or just their own? All can be simply changed as circumstances change in the organisation.

All the questions can be edited, disabled, deleted or shifted in order as required. Note that this will not affect previous assessment results.

We also supply tools to help you easily set up your teams, and monitor who has completed assessments; this latter is, again, often surprisingly tricky to determine in a large organisation.

(This can also identify who has finger trouble — I suspect Edina, in this screenshot, has clicked the wrong year a couple of times…)

As well as all the assessments, analysis tools and configuration, an Agile Simulations licence gives you access to a set of online agile games and workshops that, again, are played in each individuals browser, giving them the closest experience to a face-to-face game they can get. Games include:

• . The No Estimates Game
• The Coin Game
• Agile Battleships
• The Context Switching Game
• The Survival Game

…and others. We are keen to see these used as widely as possible in the agile space, hence we bundle them in. We are keen for Scrum Masters and Agile Coaches to be able to say “we need to strike while the iron is hot and play No Estimates right now with the C-Suite” rather than having to raise a PO to purchase it, delay, and lose the moment. We want them to have the games “in their back pocket” to use when the time is right.

If you would like more information, or a demo of the assessments or the games, ping us on LinkedIn, go the web site, or drop us a mail at info@agilesimulations.co.uk