Here are my thoughts after completing week 2 of the Learn xAPI MOOC.
The strategy track for this week is titled “Building the Business Case for Data”, which suggested that it might hold the answers to some of my questions from week 1. It didn’t.
This week I’ve included some of the discussion points from the MOOC as well as my reaction to them.
The introduction to this week suggests that we can use data to evaluate not just learning activity but the learning experience. However, I have a couple of big doubts about this:
- I love the idea of being able to assess the end user’s learning experience, but I don’t think that is what xAPI is going to do (at least not as it’s being described in this MOOC). What it seems to be talking about here is measuring the learner’s experience of a training intervention. I’m not saying that there is no value in this, but it’s not the same thing.
- Given that the people who complete a learning activity are rarely the people who commission the development of that learning activity I’d be interested to know how successful people have been in getting the go ahead to make changes and further develop the learning activity based on that data. If the need of the person who commissions the activity is to “get something out there that proves we’ve done health and safety training” how much will they care about the learner’s experience?
Which brings us nicely to this question in the MOOC.
Discussion point: Does L&D deserve it’s place as a key influencer in business strategy? Or are we playing second fiddle to other departments and their needs?
In the video for this section Sean Putman suggests we need to think about who are the customers for our learning interventions and who are the customers for your learning data?
This is logical advice, but with very few exceptions the customer for both of these is usually someone other than the learner. Indeed, it’s quite common that the customer is someone far detached from the learner (and thus even further detached from the organisation’s customer’s).
L&D puts itself into the place of playing second fiddle when it sees those other departments as its customers and does nothing more than take and satisfy their orders. If L&D wants to be treated as an equal it needs to behave like one – have an opinion, develop its own plan for supporting the organisation’s strategy. xAPI alone isn’t going to fix this, but it could give L&D more data to work with – if it knows what to do with it.
Discussion point: If you actually wanted to measure the performance impact of your learning solutions, who else would you need to work with? Do you think this would be easy to achieve, or are you likely to face road blocks?
It is scary (although accurate) that this question starts with “if”. The fact there is any doubt that L&D might want to measure the performance impact of learning tells you a lot about the state of L&D today.
However, my own experience is that even when that kind of analysis is offered to stakeholders, they don’t want it. I think that is a result of the order/supply relationship that in many cases exists between other departments and L&D.
Discussion point: How could you use this approach in your organisation? What data would you collect and why?
This question was asked in response to this blog post.
I like this idea of generating xAPI statements from the the software that someone is using. However I think I’d be more interested in how I could use the data to improve the user’s experience of the software rather than to better train the users.
The additional data which xAPI can generate makes it even more important that L&D understands what it is they expect to change through any learning intervention and what actually needs to be measured to see if that change has happened. Defining, collecting and analysing this data is not an easy task – it requires a skillset that few L&D people have and it will be time consuming and costly to do.
As with last week, I’m left with more questions.
- Will L&D be given the opportunity to develop these skills?
- Will their customers be prepared for the additional time and effort required to develop solutions?
- Will an industry pop up around this, with vendors selling promises of systems that do all of the analysis for you?