How to engage students in real open source projects

I gave a talk at FOSSA 2012 earlier this week sharing some experiences with teaching students using open source projects in a module at the University of Bolton.

Basically, there are 5 tips:

  1. Start with soft skills, not code
  2. Let students pick their own projects
  3. Teach how to “read” a project
  4. Get students interacting with the project community – not the lecturer
  5. Assess public interactions

Here’s the slides:

Hopefully I’ll be working with the teaching team to develop Year 2 & 3 modules building on this work.

Posted in Uncategorized | Tagged , | 11 Comments

Sharing usage data about web apps between stores

SPAWS logoThe 140 Character Question:

Can different web app stores share usage data such as reviews, ratings, and stats on how often an app has been downloaded or embedded?

Thats the question that we investigated in the SPAWS project. And the answer is:


Building on the Learning Registry and Activity Streams we connected together several web app stores aimed at sharing web widgets and gadgets. Each time a user visits a store and writes a review about a particular widget/gadget, or rates it, or embeds it, that information is syndicated to other stores in the network.

This means that, even if a store is focussed on a niche market, the web apps in the store can include user reviews and information collected from a wider federation. It also means that web app developers can pull together all the reviews and download stats for their apps to display on their own site, even when they are sharing their work in multiple stores.

We created a software library that developers can use to add “paradata sharing” to web app stores, and integrated it into Edukapp, a “white label web app store”. Edukapp is being used by ITEC for sharing web apps between secondary school teachers, and now by SURFNet to share web apps used in research portals in higher education. Both of these stores  should be in production use in 2013, and several other web app stores, both educational and commercial, have also shown strong interest in adopting it.

SPAWS isn’t limited to web apps and app stores – the same approach can be used for all types of resources and repositories, for example to share reviews and usage stats about learning materials, books, 3D printer models or whatever you like.

The SPAWS software library itself is open source and can be readily added to any Java project using Maven or Ivy. You can also fork it on Github.

Thanks go to the JISC/HEA OER Programme for funding this work, and to Amber Thomas for being a great programme manager!


Posted in app stores, apps | Tagged , | 4 Comments

SPAWS: Project Plan

(I’m still wrapping up the JISC part of SPAWS…normal blog service will resume shortly)

Here’s the original project proposal I submitted, including the project plan:


Posted in Uncategorized | Tagged , | Leave a comment

SPAWS: Impact

The SPAWS project – or rather, the funded part – is coming to an end so its time to wrap up the project details.

In terms of impact, what have we achieved?

1. We answered the question we set: “is the LearningRegistry Node infrastructure suitable for sharing paradata between app stores?”. (tl;dr: yes).

2. We created a software library for it

3. We embedded the library into Edukapp, which means stores operated using it can switch on SPAWS and start syndicating paradata. As JISC, SURFNet and ITEC roll out their Edukapp-based stores between now and Q3 2013, we will start to see some real practical sharing benefits

4. We demonstrated it actually works.

5. We developed paradata recipes that have general applicability

6. We made upstream contributions to improve the LRJavaLib library used with the Learning Registry, and got it listed in Maven Central making it much easier to include in other projects.

7. We contributed to the evolution of the LR itself, in particular how it will handle duplicate/outdated records using TTL in future, enabling better scaling of this kind of paradata handling

Future impact now rests on the success of the stores using SPAWS.



Posted in development, widgets | Tagged , | Leave a comment

SPAWS: What have we made?

The SPAWS project has produced:

All of which were in the original project plan.

Some things we didn’t do:

  • We didn’t use CAM for the recipes, as it isn’t really suitable. Its more for individual learning analytics than the kind of data we’re concerned with
Posted in Uncategorized | Tagged , | Leave a comment

PHP: The Pot Noodle of software development

PHP is, despite its huge popularity, one of only a handful of web programming tools I’ve never really used in anger. And thats because, basically, I’m a programming snob. I like Java (despite its annoyances), Ruby, Python, JavaScript and a whole load of other things, but I just can’t bring myself to “do” PHP.

I had a good chat about this with my colleague Mark and I think I figured it out. PHP is the Pot Noodle of programming.

A Pot Noodle

Image courtesy of Poundland

Let me explain. Pot Noodle is, frankly, a bit crap. On the other hand, the only thing it requires to make a meal is a kettle, which is pretty much the only appliance in the kitchen that is always guaranteed to be there, to work, and not be totally disgusting (especially if you’ve ever lived in a shared house). It also requires absolutely zero culinary skills. So as a student, Pot Noodle is a good standby. However, as a proper grown up I never really bother with them.

For me, PHP is a bit similar.  All you need is a fairly normal web server, and it can do the job, and most people, including non-developers, can pick up enough skills in it fairly easily. (Looking at anything substantial built in PHP also looks a bit like a tangled mess of noodles wrapped around some unidentifiable chunks too.)

But its not something I particularly want to touch myself; which is fine, as if you throw a Pot Noodle out of the window of an IT dept there’s a reasonable chance it will land on someone who can do PHP.

So, if you need the job doing, if you can’t install a ton of software on the server, and it needs doing quickly… well there’s a web programming language that can fit.

Long live PHP!

Just don’t make me eat use it.




Posted in development | 11 Comments

Working at OSS Watch, but not leaving CETIS

Some of you may have noticed that I’ve just started a new role as Service Manager at OSS Watch and have been asking whether I’ve left CETIS. Well, the situation is that I’m being seconded to Oxford to work at OSS Watch for 80% of my time, for at least the next year (after that, its hard to say as it depends on funding).

So OSS Watch will be my main focus, but I haven’t stopped working on standards and specifications. You’ll also probably still meet me at some of the same kinds of events I’ve been attending in the past, just in a different role.

(I’ll also, of course, still be contributing to open source projects like Apache Wookie, Apache Rave, Edukapp, SPAWS, Xcri4J, Wave-Node …)


Posted in Uncategorized | 1 Comment

Unhosted – building web apps without servers

For some reason I never came across this before, but it seems there is something of a movement (complete with manifesto) promoting client-side-only web apps.

According to the Unhosted site:

By “unhosted web apps” we mean browser-based apps with no server-side backend. Unlike server-side or client-server apps, unhosted web apps leave users in control of their valuable user data and privacy, by default.

I’m a big fan of writing client-side apps using the web standards stack (JS, CSS, HTML) for many of the reasons stated on the site, including not least the fact I don’t need to provide an actual hosting platform – users can run the app locally, or access it from an app store (e.g. Edukapp) or a shared collaboration environment (e.g. Apache Rave).

Publishing a web application in this way means you can also make it available as open source software, allowing it to be forked and modified by users, and also potentially  bringing in contributions and improvements from a wider community.

So, even when the intention is to create a web app that will work in a particular hosting runtime such as Apache Wookie, I always start with creating a working web app that will run in the browser without any server-side services. So, you can use the following apps directly in the browser or host them in Wookie:

(You can also run them locally in the browser – download the .wgt version, change the file extension to .zip and extract the contents)

I can then add tests for injected features provided by the runtime, and make use of them where available. For example, multi-user collaboration via the Wave API in Wookie, or inter-widget communication in Rave. Also even for things like storage, I can test whether widget.preferences is available, or default to localstorage. (Maybe in future I might want to give remotestorage a go, which provides more storage options for the user.)

I tend to opt wherever for possible for highly generic back-end services such as storage and communication, rather than creating a tightly-coupled set of services for each application. For some, this seems like an odd restriction, as you have so much freedom to do nice things on the server side (e.g. using NodeJS). However, it does make the apps much more portable and reusable, and avoids a lot of configuration hassles.

Even where it is necessary (at east for now) to use a server-based back-end, such as for collaboration or multi-user games, I created WaveNode as a generic backend in NodeJS so you don’t need to run a dedicated server-side component for each app, and the apps themselves will work anywhere there is a Wave Gadget API implementation (e.g. Apache Wookie, Apache Wave etc).

So I guess I’ve been in the Unhosted movement for a while now without realising it!

Posted in apps, development, javascript, mashups, standards, widgets | 8 Comments

SPAWS: Lessons Learned So Far

The funded part of SPAWS is coming to a close, so its time for me to write about some of the things I’ve learned so far from working on this project.

Is the LearningRegistry Node infrastructure suitable for sharing paradata between app stores?

The major question this project posed was whether the LearningRegistry infrastructure could be used to effectively share paradata among a set of app stores. While we had some teething issues with the JLern test server, overall the infrastructure performed well, and the documentation was good enough to answer any questions I had.

There are some limitations we had to overcome; for example, there is no normalization of paradata, and so I had to code this into SPAWS itself. However I suspect its a useful enough capability to be eventually added to LR as an add-on module in the future.

The only aspect of the LR infrastructure we didn’t get to test was the federation/distribution aspect, where paradata is synchronised among a network of nodes rather than through a single node. Although from a SPAWS perspective I suspect it wouldn’t make any actual difference to the way it operates.

Which structures to use for Paradata?

When I started I wasn’t sure whether we would be able to use existing JSON “recipes” for paradata, have to write our own, or whether to use CAM. In the end, I created some very simple recipes for the paradata we needed, published on Github. The basic ActivityStreams-inspired format used by LR seems good enough for the purpose, whereas CAM seemed more suited to more detailed learning analytics than the kind of basic usage information I needed to share, so I decided not to use CAM in SPAWS, at least not for the purpose of the initial project.

Something interesting that came up from our advisory group was the use of contextual information in the paradata – so links back to the page where the original comment was made, or the detail page for the widget, and to the public profile page of the user. This allows for stores (or other kinds of agents) to extract additional information where this may be useful, for example for analytics. This meant that we could keep the “core” paradata recipes very light and functionally-oriented (i.e., sticking to data that would actually be used in the UI or in core functions such as popularity ordering).

Will anyone want to share paradata?

In the proposal I set out a business case for paradata sharing for “niche” app stores, somewhat speculatively at the time. Its good to see that this business case does seem to have some traction, and other commercial app stores have shown an interest as well as the initial set of educational app stores I focussed on for the project.

Timescales as always are something of a problem, with most of the stores I work with not going live (or even beta) for some months yet. However, there was enough of an opportunity to test out the SPAWS library in integration with the development sites of the stores to see it working OK.

Are there any legal or privacy issues with paradata sharing?

There are potential privacy issues with sharing paradata that is personally generated, however I stepped around this by only sharing paradata that was either completely depersonalised (e.g. everyone’s total number of downloads/shares/likes for a widget) or completely public (e.g. published reviews). This is pretty superficial but functional as far as stores are concerned. However, by including a context URL to the original store page and user profile pages, its possible to harvest additional contextual information as needed; again this is all public rather than protected information. More detailed or personalised analytics are either not necessary for app stores at their current stage of development, or would be highly unlikely to be shared. So, the legal and privacy questions proved quite simple to resolve in this case.

How sustainable is the SPAWS code?

I started out the code on Github from the very outset, not particularly with the idea of attracting contributions during the first six months (though that would be very welcome in future). The SPAWS library itself is published on OSSRH so is fairly simple to include in Java projects.

I also ended up contributing a fair number of pull requests to LRJavaLib, a generic LearningRegistry library that SPAWS builds on top of. There is currently quite a small Java community using LR so its especially important to use and contribute to existing libraries.

As it stands, SPAWS is quite a small codebase, with plenty of comments and testcases, so should be quite straightforward to sustain with minimal effort. Its also integrated into Edukapp, so will also be updated as needed for that project.

Which APIs to use?

The LearningRegistry has a range of supported APIs; it provides simple HTTP “obtain” and “publish” services that use JSON, and it also supports OAI-PMH and SWORD. I’m not the biggest fan of OAI-PMH so I was quite relieved to find the simple obtain service was quite sufficient for my purposes; likewise for publishing the basic POST service was enough. I suspect that these protocols only really come into play when synchronising large volumes of metadata and paradata; for the SPAWS scenarios  the amount of paradata will always be relatively small, certainly for each request, and not worth the extra hassle of having to maintain state on the client side.

In the end, I opted for using a cacheing strategy for “external” paradata rather than to actually synchronise it internally with the store’s own data, avoiding any potential conflicts. Perhaps on a larger scale it would make sense to create an actual local repository for external paradata and synch it using OAI-PMH, but that would seem to be quite a way off – the next logical scaling step for Edukapp, for example, would simply be to move from memory cache to disk cache, and increase the cache size.


Given that I didn’t need to use OAI-PMH or SWORD, the standards of interest were principally the widget specs themselves – W3C and OpenSocial. The stores already provide an abstraction of the common metadata for the two specs so it wasn’t really necessary for SPAWS to rely on any particular features of the specifications apart from one rather critical one – there is no “identifier” for an OpenSocial gadget.

This is quite an issue to overcome; I opted for relying on the URL for now while punting the issue to the OpenSocial spec community to see if it could be addressed there.

Historically, each OpenSocial gadget was hosted on its own site rather than packaged and hosted elsewhere; however over time there have been gadgets being packaged up and rehosted and so this would become more of a requirement – perhaps even using the W3C format to package and transport OpenSocial gadgets.

I can also see now that what I’ve worked on here for app/widget stores would also work fine for pretty much any kind of site – so product catalogues or more traditional repositories could also reuse it pretty much as-is, including the recipes. I was originally expecting it to be similar but not directly reusable; this is something worth pursuing .

Now what?

Given what I’ve learned, would I do anything differently? I don’t think so, I think this project has answered the questions we posed initially, so its more now a question of what to do next.

One aspect I didn’t get to investigate was using other platforms and languages – for example I’d like to see if its possible to translate at least the fetching and normalization code into JavaScript; this would be useful for including in blog posts or static pages belonging to the authors themselves – for example I’d like to be able to add a stats-and-reviews panel to my own app download pages (e.g. like this one).

We also really need to come up with a solution for PHP-based stores like ROLE, for example we could use the same algorithms as in the Java version of SPAWS, but build on LRPHP.

However, what I most want to do now is get the code into production in the stores that are being launched and take it from there.

Posted in apps, development, lessonslearnt, standards, widgets | Tagged , | 1 Comment

SPAWS: Sharing a comment between stores

I’ve finally gotten around to creating a quick demo of one of the use cases I wrote for SPAWS – sharing a comment.

Its pretty simple really, although I didn’t want to use a “real” store for this one; so I’ve used a local instance of Edukapp on my own machine, and a demo/developer version of the ITEC store.

So, here are the steps in the use case:

  1. Browse a widget store
  2. Download a widget
  3. Submit a review of the widget in the widget store
  4. Search for the same widget on a different widget store
  5. The description page for the widget should contain the review submitted to the other widget store

Seems easy enough – lets go!

1. Browse a widget store

Here’s a widget I’m going to review in my local Edukapp:

A detail page on Edukapp showing a widget called "Weather"

2. Download a widget

Well, I don’t really need to do this, as I can play with a live preview on the site, but lets take it as read I’ve had a go with the widget, and have formed an opinion.

3. Submit a review of the widget in the widget store

So, I create an account and log into Edukapp, and submit a review; here it is:

The widget detail page for "Weather" widget on edukapp; I've added a comment using my username "scottbw"

4. Search for the same widget on a different widget store

On ITEC, lets find the Weather widget; luckily its pretty obvious on the “featured” page:

The front page of the ITEC App Store; in the browse list there is the same Weather widget as the one we reviewed on Edukapp

5. The description page for the widget should contain the review submitted to the other widget store

Yep, there it is!

Note that, because its an “external” review, ITEC hasn’t provided a link for the user who submitted it. SPAWS does send a URL with the “Agent” paradata that links to the user profile back on the original site, but its up to stores whether they want to use that, or to just style external reviews without any links.

The detail page on the ITEC App Store for "Weather"; below the description is the comment I added on the Edukapp site

Note also that you can set up a similar kind of demo using two different instances of Edukapp (just make sure in you give each one a different value for “” or they won’t show up in the other store as it filters out any paradata it thinks it submitted itself).

Posted in Uncategorized | Tagged | 2 Comments