Learn xAPI MOOC – Week 2 Reflections

Here are my thoughts after completing week 2 of the Learn xAPI MOOC.

The strategy track for this week is titled “Building the Business Case for Data”, which suggested that it might hold the answers to some of my questions from week 1. It didn’t.

This week I’ve included some of the discussion points from the MOOC as well as my reaction to them.

The introduction to this week suggests that we can use data to evaluate not just learning activity but the learning experience. However, I have a couple of big doubts about this:

  • I love the idea of being able to assess the end user’s learning experience, but I don’t think that is what xAPI is going to do (at least not as it’s being described in this MOOC). What it seems to be talking about here is measuring the learner’s experience of a training intervention. I’m not saying that there is no value in this, but it’s not the same thing.
  • Given that the people who complete a learning activity are rarely the people who commission the development of that learning activity I’d be interested to know how successful people have been in getting the go ahead to make changes and further develop the learning activity based on that data. If the need of the person who commissions the activity is to “get something out there that proves we’ve done health and safety training” how much will they care about the learner’s experience?

Which brings us nicely to this question in the MOOC.

Discussion point: Does L&D deserve it’s place as a key influencer in business strategy? Or are we playing second fiddle to other departments and their needs?

In the video for this section Sean Putman suggests we need to think about who are the customers for our learning interventions and who are the customers for your learning data?

This is logical advice, but with very few exceptions the customer for both of these is usually someone other than the learner. Indeed, it’s quite common that the customer is someone far detached from the learner (and thus even further detached from the organisation’s customer’s).

L&D puts itself into the place of playing second fiddle when it sees those other departments as its customers and does nothing more than take and satisfy their orders. If L&D wants to be treated as an equal it needs to behave like one – have an opinion, develop its own plan for supporting the organisation’s strategy. xAPI alone isn’t going to fix this, but it could give L&D more data to work with – if it knows what to do with it.

Discussion point: If you actually wanted to measure the performance impact of your learning solutions, who else would you need to work with? Do you think this would be easy to achieve, or are you likely to face road blocks?

It is scary (although accurate) that this question starts with “if”. The fact there is any doubt that L&D might want to measure the performance impact of learning tells you a lot about the state of L&D today.

However, my own experience is that even when that kind of analysis is offered to stakeholders, they don’t want it. I think that is a result of the order/supply relationship that in many cases exists between other departments and L&D.

Discussion point: How could you use this approach in your organisation? What data would you collect and why?

This question was asked in response to this blog post.

I like this idea of generating xAPI statements from the the software that someone is using. However I think I’d be more interested in how I could use the data to improve the user’s experience of the software rather than to better train the users.

Conclusions

The additional data which xAPI can generate makes it even more important that L&D understands what it is they expect to change through any learning intervention and what actually needs to be measured to see if that change has happened. Defining, collecting and analysing this data is not an easy task – it requires a skillset that few L&D people have and it will be time consuming and costly to do.

As with last week, I’m left with more questions.

  • Will L&D be given the opportunity to develop these skills?
  • Will their customers be prepared for the additional time and effort required to develop solutions?
  • Will an industry pop up around this, with vendors selling promises of systems that do all of the analysis for you?

Learn xAPI MOOC – Week 1 Reflections

Last week I made a start with the Learn xAPI MOOC, which is being run in Curatr and organised by the tried and trusted team of Ben Betts, Martin Couzins and Sam Burrough. This is the first MOOC that I’ve started which I can actually picture myself completing.

For me, the format is spot on – short, focussed pieces of content delivered by people that really understand the topic (who appear to have a real passion for it) with lots of opportunities to share and interact. They’ve also recognised that not everyone has the same kind of interest in this as a subject and so have separated out the content into two tracks – strategy and technical.

A bit of background

I’ve been aware of the Experience API (xAPI) for a long time, and I understand the basics of what it is intended to do:

The xAPI enables tracking of learning experiences, including traditional records, such as scores or completion. It also stores records of learners’ actions, like reading an article or watching a training video.

That description is sourced from this page on the ADL website and if you want a purely functional description of what xAPI is, it’s a good starting point.

I also understand as much as I currently need and want to about the technical aspects of how it is implemented. What I don’t understand is why we should use it and in what context. I have therefore chosen to go through the strategy track.

Week 1

I’ve come away from the first week with more questions than answers – and that’s great. The MOOC has got me thinking about xAPI in much more detail than before. Here are the things that I’m currently pondering:

  • I get that xAPI allows us to track more things than SCORM, in more ways and with much richer data. What I currently don’t get is why we would track them at all.
  • Is knowing that someone has attended a conference, watched a video or read an article somehow more valuable than knowing that they have completed a piece of elearning?
  • xAPI has the potential to produce a lot of data. Other than storing it in a Learning Record Store (LRS) what do you do with it all? Organisations are generally pretty poor at using the data they have now – will they be able to do a better job with even more of it?
  • For it to have value, data needs to be meaningful and actionable. As far as I can tell the xAPI standard makes no attempt to address this – which makes sense. While it may be possible to standardise the mechanisms for structuring, recording and storing the data, the meaning of that data will be unique.
  • There is much talk about using xAPI to record performance data (which is terrific) – but in that case why is the data kept in a Learning Record Store? That name needs to change if it is to be taken seriously by anyone other than L&D.
  • While I’m thinking about it – I’m not yet sure that an LRS records learning any more than an LMS manages it.
  • Who benefits from the xAPI?
    • I’m pretty sure that so far all of the potential benefits I’ve seen broadly relate to the organisation, L&D or the individual learning designer.
    • I can see that there are benefits to the learner, simply because if the organisation wanted to track activity in a SCORM world, it had to be SCORM content, whereas xAPI seems to be able to track pretty much anything. Whether the learner benefits from that tracking is another question entirely.

I’ve really enjoyed the first week. I certainly know more about xAPI and even if I don’t have the answers yet, I’m getting a better handle on the questions I need to answer.

Noddlepod news

Back in 2011 I wrote about Noddlepod, which at the time I described as the best collaboration tool you’ve never heard of. Since then they’ve gone from strength to strength and I was delighted to see a big announcement from them. Not only have they received a funding package from Finance Wales and four angel investors, they’ve been joined in an advisory capacity by Charles Jennings, Mary McKenna and Nigel Paine.

I’m really delighted for everyone involved. Ollie and Stephen Gardener came up with the idea in 2010 and had already developed it into a company that employs six people – this new investment should help them continue their mission to revolutionise community learning.

You can read the official announcement on the noddlepod blog.

Let’s Talk About Inequality

Facebookinstagramsocialtile2

Today is Blog Action Day and this year the focus is on inequality.

As a father I have hopes and aspirations for my daughter, and one of the most important things is to make sure that she gets a good education. Here in the UK that isn’t really a big problem. Sure, I can agonise over catchment areas and OFSTED reports, but having a choice of schools is a luxury. I don’t have to deal with anyone trying to stop my daughter receiving an education, or limiting the opportunities she has because she is a girl.1

Sadly, the same can’t be said in many other places, in fact according to Plan International 1 in 5 girls is denied an education.2

Making sure that girls get access to good quality education is one of the most significant steps in eradicating inequality. The results are felt far beyond the classroom.

  • It can help give them the knowledge and skills they need to establish a livelihood, and to develop a career.
  • It can give them the opportunity to enter a relationship by choice, when they are ready and as an equal.
  • And it doesn’t just help them as individuals, it can help lift whole communities out of poverty.

There are no limits to what an educated girl can do, they just need to be given the opportunity. Checkout this video from Plan International:

 

It’s apt that I am writing this in the week that Malala Yousafzai was jointly awarded the Nobel Peace Prize for the work she has done to promote child rights, particularly in relation to education. If you have any doubt about the power of an educated girl, watch this video of Malala as she addresses the UN on her 16th birthday.

Take Action

I chose the title of this post, Let’s Talk About Inequality, not just because it is the tagline of this year’s Blog Action Day, but because if we’re going to address inequality we have to talk about it. Spread the word, tell other people how important this is and ask them to tell even more people.

Most importantly, take action:

  • Give your support to charities that work with communities to address inequality and improve education
  • Contact your MP. Politicians aren’t just there to represent your views on local issues, they can have a significant influence on international efforts to address inequality. Visit theyworkforyou.com to find your MP’s contact details and let them know that this issue is important to you.
  1. Just let anyone even try!
  2. http://plan-international.org/what-we-do/education

Packt Publishing $10 offer

10 days 10 years - Home Banner

I’m a big fan of Packt Publishing’s technical books and videos, and it’s 10 years since they embarked on their mission to deliver effective learning and information services to IT professionals. In that time they’ve published over 2000 titles and helped projects become household names, awarding over $400,000 through its Open Source Project Royalty Scheme.

To celebrate this huge milestone, Packt is offering all of its eBooks and Videos at just $10 each – this promotion covers every title and customers can stock up on as many copies as they like until July 5th.

More information is available at www.packtpub.com/10years

Learning Articulate Storyline – Book Review

Learning Articulate StorylineEarlier this year I was delighted to be asked by Packt Publishing to act as a Technical Reviewer on their new book Learning Articulate Storyline. The book was recently published and not long afterwards a copy arrived by post. Even though I hadn’t done the hard work of actually writing the book, it was still exciting to see the finished product after having played a small part in its development.

I’m a big fan of the practical approach adopted in Packt’s books, and Learning Articulate Storyline is no exception. If you’ve never used Storyline before you can be confident that after working through this book you will be able to use it to develop elearning content.

There’s no unnecessary theory or explanations; after a quick introduction to Storyline you get stuck into building the first part of a course that you will continue to develop as you progress through the book. You’re introduced to new concepts at the point at which you use them, so there’s always a clear link between theory and practice.

At less than 280 pages you might think that the book is a little short, particularly when compared to typical IT books, but don’t let that fool you. It’s a testament to both to the simplicity of the tool and the practical approach of the book that it covers everything you need to get started – and I should be clear that getting started is what it’s all about.

The book is aimed squarely at novices and it meets their needs well, but this isn’t the book for you if you’re looking for advanced techniques. I can happily recommend it to anyone who is just getting started with Articulate Storyline.

Open Source LMS – Buyer Beware

In mid July the eLearning Network held an event called ‘The Open Source Revolution’ at which I spoke about my experience helping several organisations select an open source LMS and comparing it to similar experience with commercial systems. I was happy to report that all the projects had gone well and that relationships with suppliers had been mostly positive, but there was one major issue I had come up against which generated a lot of discussion during the Q&A.

Imagine that you have an open source LMS which was implemented by a supplier who specialises in supporting and hosting that particular LMS. Let’s call them a partner… The LMS has been in place for some time and after discussion with the supplier you decide to upgrade to the latest version. After the upgrade you discover that a plugin you have been using for some time has stopped working, not because of a problem with the plugin but because of a bug in the core LMS itself.

When you raise this with the partner who has implemented, hosted, supported and upgraded the LMS you are told that they are not liable for bugs in the core code, although they do of course offer to fix it at your expense.

It was the first time that I had been faced with a situation of this kind, and of course I immediately checked the support contract, and sure enough it absolved the supplier from any responsibility in this area. I was not happy, and judging from the response at the eLN event I’m not the only one who feels that way.

I appreciate that the supplier didn’t develop the software in the first place, but like many others they have built a business around supplying, supporting and hosting it. My view is that if they believe the software is fit for use, they should be willing to provide a warranty against bugs in the core code. If they aren’t willing to do that then any potential customer should be very sceptical about the suitability of the software, or the suitability of the supplier to support it.

For me there has been a simple outcome. I’ve updated my base requirements for procuring any open source solution to include an indemnity against core bugs as a non-negotiable. It will no doubt narrow down the choice of suppliers but I’ll have a great deal more faith in those who are left.

Don’t Annoy Users

If you have even a passing interest in technology and own one or more iOS devices you’ll know that last week Google released their Chrome web browser for iPhone and iPad. It’s likely that you’ll also know that if you want to use Chrome as your default browser, you’ll be disappointed; Apple doesn’t allow this. As a result, even if you choose to use Chrome, any web links from emails or other apps will still open in Safari.

This was a point of considerable frustration for me. My desktop browser of choice is Chrome, but I’d switched to Safari solely because I get some level of integration with iOS. It seemed to me that this was a compromise that I no longer had to make, as long as I could set Chrome as the default on iOS.

There is a solution, if you’re willing to jailbreak your iOS device and install BrowserChooser, which lets you set Chrome (or any other browser) as the default.

This is where the unintended consequences begin.

  1. Until now, neither my iPhone or iPad was jailbroken, as I had never seen the benefit. The opportunity to set Chrome as my default browser was enough of a push and so both devices are now jailbroken.
  2. Now that I’ve applied the jailbreak I realised that I could also install Sparrow+ and use Sparrow as my default email client (it plays better with Gmail).
  3. Which then led me to install Sparrow on my Macs in place of Apple Mail.
  4. Then I found myself thinking, if I’m going to all this trouble to get better integration with Google products, why don’t I just use an Android phone1?

It’s this last point that should be most troubling for Apple.

Jay Harrier suggested that an iPhone user wanting greater integration with Google services is Apple’s worst nightmare, to which John Gruber responded:

No, Apple’s worst nightmare is someone buying an Android phone instead of an iPhone. If you buy an iPhone, Apple wins, that’s all there is to it. Every iOS user who chooses to use a third-party app as their preferred client for web browsing, email, calendaring, etc. is annoyed every single time they click a web/email/event URL and are taken to an iOS system app that they don’t want to use.

“Don’t annoy users” is a good rule of thumb, and the inability to specify third-party apps as default handlers for these things is annoying.

Remember that. Don’t annoy users. It seems to me that we should all keep those three words in mind, no matter what we’re working on lest we bring about our own set of unintended consequences.


  1. Which would not be difficult to do as I have a Nexus S for development and testing. 

This Is Not Showbusiness

In his recent post Market failure? Blame it on the dog food, Clive Shepherd suggests that the reason that elearning content often has high production values but superficial learning design, is down to a market failure; that elearning vendors are selling to the employer, not the learner.

I agree, but how did this situation come to exist? Based on my own experience as an elearning manager within a corporate, and more recently as a consultant, I have some thoughts about why the relationship often works this way.

In his post, Clive touches on the first one:

When employers purchase an e-learning product or engage with a developer, they choose on the basis of production values rather than learning design, because they have neither the time nor the inclination to test out materials with real learners.

I’ve seen many examples of this, but it’s a problem that existed long before elearning. The same can happen when commissioning external providers of face to face training. The focus is on the delivery and collateral rather than the learning and business outcomes. Trainers are often selected, quite literally, on the basis of style over substance.

This is closely related to the second problem. When I was getting started with elearning, I was given all sorts of advice, but one of the more frequent suggestions was to produce “high profile content”; that is, content that people would talk about because it had the “wow factor”. The trouble is, that ends up being “wow those graphics were amazing”, or “wow I think 3D models are cool” when it should be “wow I’ve learned so much by doing that!”.

Who are we trying to impress with this high profile, “wow” content? It’s usually stakeholders and senior managers, because we want their buy in as way to ensure that we can do more of this.

The trouble is, once we’ve opened this box, it’s almost impossible to put thing back in again. We establish a situation where elearning is perceived as something that must have high production values, and anything else is considered to be sub-standard.

This also relates to the problem of procurement models and IT involvement. In the past, it was not unusual to see the commissioning of a piece of elearning treated as a software purchase, and for some organisations that’s still true today.

There are two problems with this;

First of all, a software purchase is usually sourced based on factors such as integration with existing systems, and the availability of support. Effective learning outcomes will not be high on the list. IT then end up making decisions about the choice of vendor; something which should be the domain L&D.

The second issue is how these things are paid for. A software purchase is usually treated as capital expenditure, and in many organisations something can only be a capital expense if it is above a certain cost (perhaps £30-£40,000). So what do you do? You make your elearning content more media rich in order to push up the cost, because you couldn’t get the budget if you wanted to spend less. Sounds crazy, but I’ve got plenty of experience of this!

How do we overcome this? Like Clive, I’m not completely sure, but I do know that it involves L&D keeping their focus on being business people rather than show people.