Lak11 Week 3 and 4 (and 5): Semantic Web, Tools and Corporate Use of Analytics

Two weeks ago I visited Learning Technologies 2011 in London (blog post forthcoming). This meant I had less time to write down some thoughts on Lak11. I did manage to read most of the reading materials from the syllabus and did some experimenting with the different tools that are out there. Here are my reflections on week 3 and 4 (and a little bit of 5) of the course.

The Semantic Web and Linked Data

This was the main topic of week three of the course. Basically the semantic web has a couple of characteristics. It tries to separate the presentation of the data and the data itself. It does this by structuring the data which then allows linking up all the data. The technical way that this is done is through so-called RDF-triples: a subject, a predicate and an object.

Although he is a better writer than speaker, I still enjoyed this video of Tim Berners-Lee (the inventor of the web) explaining the concept of linked data. His point about the fact that we cannot predict what we are going to make with this technology is well taken: “If we end up only building the things I can imagine, we would have failed“.

[youtube=http://www.youtube.com/watch?v=OM6XIICm_qo]

The benefits of this are easy to see. In the forums there was a lot of discussion around whether the semantic web is feasible and whether it is actually necessary to put effort into it. People seemed to think that putting in a lot of human effort to make something easier to read for machines is turning the world upside down. I actually don’t think that is strictly true. I don’t believe we need strict ontologies, but I do think we could define more simple machine readable formats and create great interfaces for inputting data into these formats.

Use cases for analytics in corporate learning

Weeks ago Bert De Coutere started creating a set of use cases for analytics in corporate learning. I have been wanting to add some of my own ideas, but wasn’t able to create enough “thinking time” earlier. This week I finally managed to take part in the discussion. Thinking about the problem I noticed that I often found it difficult to make a distinction between learning and improving performance. In the end I decided not to worry about it. I also did not stick to the format: it should be pretty obvious what kind of analytics could deliver these use cases. These are the ideas that I added:

  • Portfolio management through monitoring search terms
    You are responsible for the project management portfolio learning portfolio. In the past you mostly worried about “closing skill gaps” through making sure there were enough courses on the topic. In recent years you have switched to making sure the community is healthy and you have switched from developing “just in case” learning intervention towards “just in time” learning interventions. One thing that really helps you in doing your work is the weekly trending questions/topics/problems list you get in your mailbox. It is an ever-changing list of things that have been discussed and searched for recently in the project management space. It wasn’t until you saw this dashboard that you noticed a sharp increase in demand for information about privacy laws in China. Because of it you were able to create a document with some relevant links that you now show as a recommended result when people search for privacy and China.
  • Social Contextualization of Content
    Whenever you look at any piece of content in your company (e.g. a video on the internal YouTube, an office document from a SharePoint site or news article on the intranet), you will not only see the content itself, but you will also see which other people in the company have seen that content, what tags they gave it, which passages they highlighted or annotated and what rating they gave the piece of content. There are easy ways for you to manage which “social context” you want to see. You can limit it to the people in your direct team, in your personal network or to the experts (either as defined by you or by an algorithm). You love the “aggregated highlights view” where you can see a heat map overlay of the important passages of a document. Another great feature is how you can play back chronologically who looked at each URL (seeing how it spread through the organization).
  • Data enabled meetings
    Just before you go into a meeting you open the invite. Below the title of the meeting and the location you see the list of participants of the meeting. Next to each participant you see which other people in your network they have met with before and which people in your network they have emailed with and how recent those engagements have been. This gives you more context for the meeting. You don’t have to ask the vendor anymore whether your company is already using their product in some other part of the business. The list also jogs your memory: often you vaguely remember speaking to somebody but cannot seem to remember when you spoke and what you spoke about. This tools also gives you easy access to notes on and recordings of past conversations.
  • Automatic “getting-to-know-yous”
    About once a week you get an invite created by “The Connector”. It invites you to get to know a person that you haven’t met before and always picks a convenient time to do it. Each time you and the other invitee accept one of these invites you are both surprised that you have never met before as you operate with similar stakeholders, work in similar topics or have similar challenges. In your settings you have given your preference for face to face meetings, so “The Connector” does not bother you with those video-conferencing sessions that other people seem to like so much.
  • “Train me now!”
    You are in the lobby of the head office waiting for your appointment to arrive. She has just texted you that she will be 10 minutes late as she has been delayed by the traffic. You open the “Train me now!” app and tell it you have 8 minutes to spare. The app looks at the required training that is coming up for you, at the expiration dates of your certificates and at your current projects and interests. It also looks at the most popular pieces of learning content in the company and checks to see if any of your peers have recommended something to you (actually it also sees if they have recommended it to somebody else, because the algorithm has learned that this is a useful signal too), it eliminates anything that is longer than 8 minutes, anything that you have looked at before (and haven’t marked as something that could be shown again to you) and anything from a content provider that is on your blacklist. This all happens in a fraction of a second after which it presents you with a shortlist of videos for you to watch. The fact that you chose the second pick instead of the first is of course something that will get fed back into the system to make an even better recommendation next time.
  • Using micro formats for CVs
    The way that a simple structured data format has been used to capture all CVs in the central HR management system in combination with the API that was put on top of it has allowed a wealth of applications for this structured data.

There are three more titles that I wanted to do, but did not have the chance to do yet.

  • Using external information inside the company
  • Suggested learning groups to self-organize
  • Linking performance data to learning excellence

Book: Head First Data Analytics

I have always been intrigued by O’Reilly’s Head First series of books. I don’t know any other publisher who is that explicit about how their books try to implement research based good practices like an informal style, repetition and the use of visuals. So when I encountered Data Analysis in the series I decided to give it a go. I wrote the following review on Goodreads:

The “Head First” series has a refreshing ambition: to create books that help people learn. They try to do this by following a set of evidence-based learning principles. Things like repetition, visual information and practice are all incorporated into the book. This good introduction to data analysis, in the end only scratches the surface and was a bit too simplistic for my taste. I liked the refreshers around hypothesis testing, solver optimisation in Excel, simple linear regression, cleaning up data and visualisation. The best thing about the book is how it introduced me to the open source multi-platform statistical package “R”.

Learning impact measurement and Knowledge Advisers

The day before Learning Technologies, Bersin and KnowledgeAdvisors organized a seminar about measuring the impact of learning. David Mallon, analyst at Bersin, presented their High-Impact Measurement framework.

Bersin High-Impact Measurement Framework
Bersin High-Impact Measurement Framework

The thing that I thought was interesting was how the maturity of your measurement strategy is basically a function of how much your learning organization has moved towards performance consulting. How can you measure business impact if your planning and gap analysis isn’t close to the business?

Jeffrey Berk from KnowledgeAdvisors then tried to show how their Metrics that Matter product allows measurement and then dashboarding around all the parts of the Bersin framework. They basically do this by asking participants to fill in surveys after they have attended any kind of learning event. Their name for these surveys is “smart sheets” (an much improved iteration of the familiar “happy sheets”). KnowledgeAdvisors has a complete software as a service based infrastructure for sending out these digital surveys and collating the results. Because they have all this data they can benchmark your scores against yourself or against their other customers (in aggregate of course). They have done all the sensible statistics for you, so you don’t have to filter out the bias on self-reporting or think about cultural differences in the way people respond to these surveys. Another thing you can do is pull in real business data (think things like sales volumes). By doing some fancy regression analysis it is then possible to see what part of the improvement can be attributed with some level of confidence to the learning intervention, allowing you to calculate return on investment (ROI) for the learning programs.

All in all I was quite impressed with the toolset that they can provide and I do think they will probably serve a genuine need for many businesses.

The best question of the day came from Charles Jennings who pointed out to David Mallon that his talk had referred to the increasing importance of learning on the job and informal learning, but that the learning measurement framework only addresses measurement strategies for top-down and formal learning. Why was that the case? Unfortunately I cannot remember Mallon’s answer (which probably does say something about the quality or relevance of it!)

Experimenting with Needlebase, R, Google charts, Gephi and ManyEyes

The first tool that I tried out this week was Needlebase. This tool allows you to create a data model by defining the nodes in the model and their relations. Then you can train it on a web page of your choice to teach it how to scrape the information from the page. Once you have done that Needlebase will go out to collect all the information and will display it in a way that allows you to sort and graph the information. Watch this video to get a better idea of how this works:

[youtube=http://www.youtube.com/watch?v=58Gzlq4zSDk]

I decided to see if I could use Needlebase to get some insights into resources on Delicious that are tagged with the “lak11” tag. Once you understands how it works, it only takes about 10 minutes to create the model and start scraping the page.

I wanted to get answers to the following questions:

  • Which five users have added the most links and what is the distribution of links over users?
  • Which twenty links were added the most with a “lak11” tag?
  • Which twenty links with a “lak11” tag are the most popular on Delicious?
  • Can the tags be put into a tag cloud based on the frequency of their use?
  • In which week were the Delicious users the most active when it came to bookmarking “lak11” resources?
  • Imagine that the answers to the questions above would be all somebody were able to see about this Knowledge and Learning Analytics course. Would they get a relatively balanced idea about the key topics, resources and people related to the course? What are some of the key things that would they would miss?

Unfortunately after I had done all the machine learning (and had written the above) I learned that Delicious explicitly blocks Needlebase from accessing the site. I therefore had to switch plans.

The Twapperkeeper service keeps a copy of all the tweets with a particular tag (Twitter itself only gives access to the last two weeks of messages through its search interface). I manage to train Needlebase to scrape all the tweets, the username, URL to user picture and userid of the person adding the tweet, who the tweet was a reply to, the unique ID of the tweet, the longitude and latitude, the client that was used and the date of the tweet.

I had to change my questions too:

Another great resource that I re-encountered in these weeks of the course was the Rosling’s Gapminder project:

[youtube=http://www.youtube.com/watch?v=BPt8ElTQMIg]

Google has acquired some part of that technology and thus allows a similar kind of visualization with their spreadsheet data. What makes the data smart is the way that it shows three variables (x-axis, y-axis and size of the bubble and how they change over time. I thought hard about how I could use the Twitter data in this way, but couldn’t find anything sensible. I still wanted to play with the visualization. So at the World Bank’s Open Data Initiative I could download data about population size, investment in education and unemployment figures for a set of countries per year (they have a nice iPhone app too). When I loaded that data I got the following result:

Click to be able to play the motion graph
Click to be able to play the motion graph

The last tool I installed and took a look at was Gephi. I first used SNAPP on the forums of week and exported that data into an XML based format. I then loaded that in Gephi and could play around a bit:

Week 1 forum relations in Gephi
Week 1 forum relations in Gephi

My participation in numbers

I will have to add up my participation for the two (to three) weeks, so in week 3 and week 4 of the course I did 6 Moodle posts, tweeted 3 times about Lak11, wrote 1 blogpost and saved 49 bookmarks to Diigo.

The hours that I have played with all the different tools mentioned above are not mentioned in my self-measurement. However, I did really enjoy playing with these tools and learned a lot of new things.

My Top 10 Tools for Learning 2010

CC-licensed photo by Flickr user yoppy
CC-licensed photo by Flickr user yoppy

For this year’s edition of the Top 100 Tools for Learning (a continuing series started, hosted and curated by JaneDuracell BunnyHart of the Internet Time Alliance) I decided to really reflect on my own Learning Process. I am a knowledge worker and need to learn every single day to be effective in my job. I have agreed with my manager to only do very company-specific formal training. Things like our Leadership development programs or the courses around our project delivery framework are so deeply embedded in our company’s discourse that you miss out if you don’t allow yourself to learn the same vocabulary. All other organised training is unnecessary: I can manage myself and that is the only way in which I can make sure that what I learn is actually relevant for my job.

So what tools do I use to learn?

1. Goodreads in combination with Book Depository
The number one way for me personally to learn is by reading a book. When I started as an Innovation Manager in January I wanted to learn more about innovation as a topic and how you could manage an innovation funnel. I embarked on a mission to find relevant books. Nowadays I usually start at Goodreads, a social network for readers. I like the reviews there more than the ones on Amazon and I love the fact that I can get real recommendations from my friends. Goodreads has an excellent iPhone app making it very easy to keep a tab on your reading habits. I found a bunch of excellent books on innovation (they will get a separate post in a couple of weeks).
My favourite book store to buy these books is Book Depository (please note that this is an affiliate link). They have worldwide free shipping, are about half the price of the book stores in the Netherlands and ship out single books very rapidly.

2. Twitter and its “local” version Yammer
Ever since I got an iPhone I have been a much keener Twitter user (see here and guess when I got the iPhone). I have come to realise that it is a great knowledge management tool. In recent months I have used it to ask direct questions to my followers, I have used it to follow live news events as they unfold, I have searched to get an idea of the Zeitgeist, I have used it to have a dialogue around a book, and I have used it as a note taking tool (e.g. see my notes on the Business-IT fusion book, still available thanks to Twapperkeeper).
Yammer is an enterprise version of Twitter that is slowly taking off in my company. The most compelling thing about it is how it cuts across all organizational boundaries and connects people that can help each other.

3. Google
Google does not need any introduction. It is still my favourite search tool and still many searches start at Google. I have to admit that those searches are often very general (i.e. focused on buying something or on finding a review or a location). If I need structured information I usually default to Wikipedia or Youtube.

4. Google Reader
I have about 300 feeds in Google Reader of which about 50 are in my “first read” category, meaning I follow them religiously. This is the way I keep up with (educational) technology news. What I love about Google Reader is how Google has made a very mature API available allowing people to write their own front-end for it. This means I can access my feeds from a native iPhone app or from the web or from my desktop while keeping the read counts synchronised. Another wonderful thing is that Google indexes and keeps all the feed items once you have added the feeds. This means that you can use it to archive all the tweets with a particular hash tag (Twitter only finds hash tags from the last two weeks or so when you use their search engine). Finally, I have also used Google Reader as a feed aggregator. This Feedburner feed, for example, was created by putting three different feeds in a single Google Reader folder (more about how to do that in a later post).

5. Wikipedia (and Mediawiki)
The scale of Wikipedia is stupefying and the project still does not seem to run out of steam. The Wikimedia organization has just rolled out some enhancements to their Mediawiki software allowing for easier editing. The openness of the project allows for people to build interesting services on top of the project. I love Wikipanion on my iPhone and I have enthusiastically used Pediapress a couple of times to create books from Wikipedia articles. I find Wikipedia very often (not always!) offers a very solid first introduction to a topic and usually has good links to the original articles or official websites.

6. Firefox
Even though I have written earlier that I was a Google Chrome user, I have now switched back and let Mozilla’s Firefox be the “window” through which I access the web. This is mainly due to two reasons. The first being that I am incredibly impressed with the ambitions of Mozilla as an organization. Their strategy for making the web a better place really resonates with me. The other reason is Firefox Sync, allowing me to use my aliased bookmarks and my passwords on multiple computers. I love Sync for its functionality but also for its philosophy: you can also run your own Sync server and do not need to use Mozilla’s and all the sync data is encrypted on the server side, needing a passphrase on the client to get to it.

7. LinkedIn
It took a while before I started to see the true benefits of LinkedIn. A couple of weeks ago I had a couple of questions to ask to people who have experience with implementing SAP Enterprise Learning in large organizations. LinkedIn allowed me to search for and then contact people who have SAP Enterprise Learning in their profile in some way. The very first person that I contacted forwarded me on to a SAP Enterprise Learning discussion group on LinkedIn. I asked a few questions in that forum and had some very good public and private answers to those questions within days. In the past I would only have access to that kind of market information if SAP would have been the broker of this dialogue or if I would buy from analysts like Bersin. LinkedIn creates a lot of transparency in the market place and transparency is a good thing (especially for customers).

8. WordPress (including the WordPress.com network) and FocusWriter
Writing is probably one of the best learning processes out there and writing for other people is even better. WordPress is used to publish this post, while I use a simple cross-platform tool called FocusWriter to give me a completely uncluttered screen with just the words (no menus, window edges or status bars!). WordPress is completely free to use. You can either opt for a free (as in beer) hosted version that you can set up within seconds on http://www.wordpress.com or you can go the free (as in speech) version where you download the application, modify it to your needs and host it where you want. If I was still a teacher now, this would be the one tool that I would let all of my students use as much as possible.

9. Youtube
The quantity of videos posted on Youtube is not comprehensible. It was Rob Hubbard who first showed me how you could use the large amount of great tutorials to great effect. He rightfully thought: Why would I put a lot of effort into developing a course on how to shoot a great video if I can just link to a couple of excellent, well produced, short, free videos that explain all the most important concepts? The most obvious topics to learn about are music (listening to music and learning how to play music) and games (walkthroughs and cheat codes) , but there are already lots of great videos on other topics too.

10. Moodle and the community on Moodle.org
Moodle is slowly slipping to the bottom of my list. In the last few years a lot of my professional development was centred around Moodle and I still owe many of the things I know about educational technology, open source and programming/systems administration to my interactions in the forums at Moodle.org. Two things are the cause for Moodle being less important to my own learning:
1. I now have a job in which I am tasked to try and look ahead and see what is coming in the world of enterprise learning technology. That is a broad field to survey and I have been forced to generalise my knowledge on the topic.
2. I have become increasingly frustrated with the teacher led pedagogical model that all Virtual Learning Environments use. I do believe that VLEs “are dead”: they don’t fully leverage the potential of the net as a connection machine, instead they are usually silos that see themselves as the centre of the learning technology experience and lack capabilities to support a more distributed experience.

Previous versions of my Top 10 list can be found here for 2008 and here for 2009. A big thank you again to Jane for aggregating and freely sharing this hugely valuable resource!