Learning analytics is too important for L&D to own

I’m clearly not the only one to recognise that L&D lacks the skills needed for the kind of learning analytics enabled by advances such as xAPI.

Mark Aberdour has written a very thoughtful post about the challenges we face and makes this suggestion:

Clearly some of these items require close interaction with the L&D team, but in summary there is a real need to bring experienced data scientists into corporate learning and development, not just to set up analytics programmes but to continually monitor, review and refine the results.

via Building a learning analytics platform | Open Thoughts.

I agree that organisations need people who can interpret this data and make it actionable, but I don’t believe they should sit within L&D. If ownership for learning data remains within L&D we risk continuing the current situation where all we do is measure the most basic elements of our performance (inputs and outputs) rather than the impact of learning on workplace performance.

For learning data to have strategic value it needs to be considered at a higher level, in combination with data from other parts of the organisation. To be objective, ownership for this needs to sit outside any department with a vested interest in the results.

10 thoughts on “Learning analytics is too important for L&D to own

  1. I agree emphatically with this. I believe the best question to ask is “Who cares?” If the answer is learners, their operational managers and the customers of both then that gives a clue as to the sort of data needed and the volume of it too. Just as the yardstick for training providers has always been “how much, how fast and how cheap” so the statistician must be proud to say “how deep, how broad and how organised”.

  2. Thanks for another thought provoking post Barry. As we get deeper into the topic I’m inclined to agree with you. Realistically very few companies are going to hire a data scientist to do this kind of work specifically for learning/performance, but I can see larger, more data savvy organisations, incorporating learning data into their overall strategy. Where they are already building centralised data warehouses to gather MI from across the business the issue is probably more about educating them about the more nuanced data they can now collect with xAPI and how that might be useful when taken in context with the other data sources they have.

    Do you think that smaller organisations, which are unlikely to invest in this area, should even bother with learning analytics, or is there value in products that make it easier to analyse the data for the layperson?

    • Even in large organisations there is a cost to gathering, storing and analysing this data. L&D will need to make a case for including their data in the mix. xAPI may help, but only if what it’s tracking is valuable to the organisation.

      I think smaller organisations have to give careful consideration to the cost vs the benefits of any kind of measurement (let alone analytics) on a per case basis. The greater the expected return on the learning activity the greater the case for measurement.

      I’m sure we’ll see plenty of products offering to make the analysis easier – we’ll have to wait and see if they actually add value.

  3. Interestingly, I think xAPI is L&D’s best kept secret. It’s impact should be felt across all organisational data (performance is ‘owned’ by the whole organisation). So, L&D should be thinking big on this.

  4. Great conversation and thanks for the pingback, Barry. I agree, it makes sense to utilise Business Intelligence units if they already exist, and work in collaboration with them, that’s going to be much more cost effective I’m sure. Am involved with plenty of clients who push LMS data into tools like Qlikview and other BI tools, but that tends to currently be used (within the L&D context, anyway) more for standard ‘bums on seats’ reporting around training delivery and course completion, rather than for predictive analytics.

    Will be interesting to see how xAPI comes into the mix. From the folks I’ve spoken to in the analytics area, once they can get access to source data from multiple systems they can take it from there. xAPI could certainly be a facilitator for that, for sure. It’s often said that analytics efforts are 80% data preparation and 20% analysis. xAPI offers a way of neatly formatting data from multiple sources into a common format, hence reducing the data preparation and cleansing effort, which could save a LOT of time.

  5. The data in xAPI is only as good as as it has been designed. Who is designing learning experiences? Unless L&D is part of the equation and well informed of what can be used and themselves document what they store in xAPI the data is worth nothing.

    Now the only way to ensure that learning was profitable is to compare past and future performance on the workplace. This means it’s a correlation between data collected inside xAPI and outside of it that will be the most convincing. When you reach this stage you will need a few synthetic figures on the learning that took place. What are they, how to get them for the tons of statements collected? Again an L&D job.

    It’s a kind of chicken and egg problem and xAPI is just the wires. It’s what is placed in it that is important.

    Now if it is a smaller organization that is buying some of the learning experience they will have to ensure that the events captured are consistent. Same verbs, same actor ids, same frequency.

    One course is using a long video will have one statement. Another course using micro-videos will generate many statements. How to make aggregates? Mobile consumption is likely to be longer, interrupted, restarted. Does it implies the learner was distracted?

  6. Pingback: Learn xAPI MOOC – Week 3 Reflections | Barry Sampson

  7. Pingback: Learn xAPI MOOC – Week 4 Reflections | Barry Sampson

  8. The scope is greater than L&Ds usual “corporate” remit. Looking at analytics from an instructional viewpoint (by definition) misses the real opportunity to watch and observe performance behavoirs in high performance human capital. No high productive, innovative or quality leader ever gained even 0.01% of their attributes from eLearning. Watching a problem solver, recognising, rewarding and encouraging the sharing of competency is the real name of the game . It’s about the 70;20 stuff

Leave a Reply