On Thursday, October 23rd, 2014, we hosted a webinar about how PANDORA and Tribridge are using xAPI to correlate training to employee performance using sales data. As usual, the attendees had more questions that we could answer during the live webinar, so we’ve posted the questions and the answers here.
Pandora
Q: Wondering if Pandora (or other) collects any instructor-led training interactions in Watershed LRS. Think: audience response system (phone) interaction data going to LRS.
A: (PER) At this point in time, no. But the idea sounds interesting! The data we want to collect on ILT is related to attendance (of course), performance and engagement (on assignments, tests, etc.) through LMS data and data on offline LOs.
Q: What would you motivate learners to actually answer polls via apps, etc? They could easily skip it. How would you track whether they’ve actually taken that step (eg used a phone app to answer a poll)?
A: (PER) As for the motivational part, I guess it would depend on the poll. If it’s as part of a training session, it requires your learners to be engaged with the training. There are a variety of strategies you can apply to achieve engagement.
Q: It would seem that much of this approach depends on user involvement. What validations are used to verify user learning if it self-defined?
A: (PER) I agree, the key to successful learning is learner engagement! At least, this is something teachers and l&d people in Europe believe in, while I believe there is a more behaviouristic tradition in the US.
Q: What content authoring tool is Pandora using?
A: (PER) eXactLS LCMS and Packager.
Q: Can I see any Pandora course (for example course about some product)?
A: (PER) Unfortunately PANDORA courses are only available for staff selling our products. The answer is no.
Q: Are you still using CSOD for your LMS?
Q: So, is the Watershed LRS xAPI data migrated into the Cornerstone LMS?
A: (Mike) Yes, Pandora is still using the CSOD LMS and the Tribridge Amplify HR platform helps bridge the data from Watershed and CSOD.
Q: How has xAPI integration changed the way you view your LMS?
A: (PER) Before xAPI, we saw the LMS as a technically limited but absolutely essential centre of all our learning activities. Without the LMS, we couldn’t track any learning activities. With the LMS, we could only track so and so much, which was never enough. Now, with xAPI, we see the LMS as a less important piece of the puzzle. It’s still needed for ILT mgmt and traditional e-learning courses delivery, and for tracking part of those activities. But it’s now only one data source, out of a number of data sources.
xAPI Ecosystem
Q: What is the future of the LMS? Will they be phased out in the future by most organizations?
A: (Mike) It’s hard to say. I don’t think that LMSs will go away anytime soon, they still do so many things that an LRS will never do.
Q: Is Cornerstone’s LMS xAPI compliant?
A: (Mike) Not yet to our knowledge. They did present xAPI support as a goal for the upcoming year in the product roadmap session at their Convergence user conference in May 2014.
Q: We currently utilize the Success Factors LMS, which does not currently support Experience API. Have you heard from or are your currently working with them? We are really interested, but can go no further until it is supported.
A: (Mike) We have had conversations with just about every major LMS vendor regarding adoption but we have to keep most of them in confidence. The best thing you can do to encourage your LMS vendor to adopt is to be specific about why you want xAPI support and what you want to be able to do with it in their system. Many of the large LMS vendors are hearing a loud demand from their customers for xAPI support, but customers aren’t giving details which makes it hard for the vendors to figure out how to best design and prioritize their xAPI plans. In the interim, we started creating Watershed to let people take advantage of the power of xAPI before the big LMS systems are ready for it. We can almost always find a way to integrate the systems and get important data where it needs to be.
Q: Question for Mike: are you going to keep the Experience API aligned with the xAPI or branch?
A: (Mike) The Experience API and xAPI are the exact same thing. xAPI is just a nickname for the officially named xAPI.
Q: Is it cheaper to integrate Experience API in a LRS or LMS than SCORM?
A: (Mike) There are a couple different ways I could interpret this question so I will answer both of them:
Qv1: “Is the cheaper to develop xAPI functionality in my LMS so that my LMS contains an LRS and can accept xAPI Statements than it was to develop a SCORM player in my LMS”?
A: (Mike) It’s a bit too early to tell since xAPI implementations are so new. Early adoptions of any technology always take longer than the mainstream adoptions. I think that just doing the basic implementation of the xAPI plumbing is a lot easier than the basic SCORM plumbing (when you include SCORM 2004’s sequencing). However, adding other functionality to fully embrace the potential of xAPI and make sense of all the new information you received is a very big undertaking and in some places still an unsolved problem.
Qv2: “Is the cheaper to integrate my xAPI content with an LMS than it is to integrate my SCORM content with an LMS?”
A: (Mike) Both specifications are technically free to implement so there is no “cost” per se, so the question is one of effort. Both specifications seek to drive integration costs to zero, that is the point of a specification, to make things plug-and-play. The answer to the question then comes down to which specification comes closer to achieving plug-and-play. SCORM is more mature and so right now it is probably closer to plug-and-play in more cases. In the long term though, I expect xAPI to far surpass SCORM’s interoperability. xAPI is much simpler, leaving less wiggle room for different interpretations. When we drafted the original version of the xAPI spec, we did so having spent the prior 10 years troubleshooting SCORM content. We tried to incorporate as many lessons from that time into the new spec as we could.
Q: Are there already results using xAPI in a company like this? Or is it still in the starting phase?
A: (Mike) Early adopters of the Watershed LRS like AT&T, Google, and the BBC are starting to results with xAPI and the Watershed LRS. We hope to publicize as many of these experimental conclusions as we can.
Q: The training events that a learner completes can be infinite. How can they be indexed so they can be correlated with other data? Similarly, if learners want to take their learning achievements with them from company to company, how can they be tagged or described so that company A has any way of understanding/appreciating what was completed at company B?
A: (Mike) Great question. There are still a lot of problems to be solved in these areas as our industry matures. I can’t claim to have perfect answers to these questions yet. There are a few technical solutions that can help solve part of the problem. The Experience API allows for metadata about learning activities to travel alongside the statements of completion. We are to taking some of the first steps towards sharing activity definitions with the xAPI Registry as well.
Watershed LRS
Q: I have heard a lot about watershed on this and other webinars, but its not clear to me exactly what Watershed is and who provides access to it.
A: (Mike) Watershed LRS is a Learning Record Store that uses the Experience API to do things with training programs that couldn’t be done before the Experience API existed. It’s provided by Rustici Software. More information at http://watershedlrs.com
Q: Does Watershed include the reporting API’s?
Q: What support does Watershed offer to produce the aggregated reporting out of the data in the Watershed LRS?
A: (Mike) Watershed provides a set of out of the box reports and a big part of many of these early projects is Rustici Software configuring Watershed to provide additional analysis for companies. All of the data is available through the Experience API and Watershed provides additional API access to certain sets of data.
Q: Has the employee the “rights” on his registered data? If he wants to export it, to have its possession or to upload it to another LRS, can he do it?
A: (Mike) There is no right answer to this question or an established legal or industry norm yet. Technically this is all possible via xAPI and in Watershed. Whether a company chooses to enable it or not is still an option. In my experience, more organizations are open to this idea than not…but I’m typically talking to L&D folks, not the lawyers.
(PER) At the moment, all the activities we track are related to PANDORA training, and as such owned by PANDORA. But like I said at the webinar, we see this technology as a step in the direction of enabling data transfer to the learners, once a standard or service is established. I’d like to think of a future in which every learning experience you’ve had can be collected in your own personal LRS, enabling you to use that both in future interviews as well as other future career decisions.
Logistics
Q: Will this be recorded and a link sent to us?
Q: Showed up a little late and wanted to make sure i didn’t miss it … Will this be recorded for review later? Thanks.
A: (Mike) Yep! We’ll send an email out to everybody that signed up for the webinar. You’ll get a link to the slides, the Q&A blog, and the recorded webinar.
Q: Just an observation: You can compare Pandora’s experience to any retail chain .. Many thanks .. A big shout out to all of you from Cancun, Mexico!
A: (Mike) Absolutely. We try to extract lessons from each of these use cases that are broadly applicable. I’m pretty sure you need an onsite consultation in Cancun, don’t you? 😉
Q: Is this webinar hosted by Pandora the jewelry company or PANDORA the music application? The logo on the screen is for the jewelry, but I thought this was about web apps???
A: (Mike) This is Pandora the Jewelry company, and how they’re using the Experience API, Watershed LRS, and Tribridge to do things with their training departments that could never been done before. Sorry if there was any confusion, being named “Rustici Software” we not really used to having disambiguate our company name!
Q: Ha!
A: (Mike) Thanks for that. It’s tough making a joke when everybody is muted and you can’t tell if it flopped!
Feedback
Q: Great session!
Q: Great session!
Q: Excellent webinar
Q: Very well done. Thanks!
Q: This webinar was interesting to see from a commercial case study perspective and I enjoyed the slant from a performance perspective.
Q: Thank you for this interesting webinar .. I hope my invited clients were in attendance .. I’m sure this will have piqued their interest far better than I have been able to.
Q: Keep up the great work!
A: (Mike) Thanks! Glad you enjoyed it.
Q: Interesting to hear what people are doing but would like to know more about that how.
Q: Would like to see more of the “how” behind these technologies. Show the admin interface, discuss the integration process.
Q: To be quite honest, this was a waste of time. I knew many things before about xAPI, and I was hoping to see how the data actually looked, and how it tracks it. I saw none of this.
Q: The topics were quite interesting but the points Per made were far from direct. Would prefer a more concise, directed webinar about the integration – such as how did he collect the informal training data, or the physical workbook assignment answers?
Q: I think I came into this webinar with different expectations. I thought this was going to be a demo of xAPI and show how you could track all of these learning event. But I will look online for the demos. If I already had an understanding of xAPI beforehand I’m sure this would have been time well spent.
A: (Mike) Thanks for the feedback. We always struggle with the balance of keeping things at a high level and understandable by a mass audience vs expressing our inner geekdom and providing technical details. I think this is good feedback that perhaps we could go a bit deeper and show some things actually in action.
