Design challenges for user-interface mashups: user control and usability in inter-widget communications

I originally wrote this as a paper submission for WSKS 2011 as part of my work on the OMELETTE project, but decided to withdraw it – however I think it might be interesting given there is a lot of discussion at the moment around inter-widget communication and interactive mashups. Its probably worth reading this in conjunction with my post on Web Intents.

Design challenges for user-interface mashups: user control and usability in inter-widget communications

Scott Wilson

Abstract. Mashup platforms can expose a means for widgets to interact with one another; while there are communications protocols suitable for implementing this, a design challenge is how to provide users with the feeling that they are in control without being confronted with complex platform configuration. In this paper we discuss a number of approaches taken by projects to resolve this; either reducing platform complexity using metaphor, or in enhancing user control. One major issue is the lack of usability research in this space, particularly of the design metaphors commonly used by mashup solutions aimed at non-developers.

Keywords. Widgets, Inter-Widget Communication, Choreography, Mashups, Design, Usability, Accessibility

Introduction

Widgets, perhaps taking the place of portlets (Law, Müller and Nguyen-Ngoc, 2009) can be used as a building block of web dashboards and mashups, enabling users to construct personal workspaces consisting of multiple widgets talking to different backend services organized according to personal needs. In the literature of technology-enhanced learning, this is the focus of MashUp Personal Learning Environments (MUPPLE); in other domains we may describe this more generally as a user interface mashup (UI Mashup).

A workspace or dashboard can provide a range of features; for example it can support extension with additional widgets talking to backend services such as enterprise systems and social web services. It can also support user-managed layouts, for example enabling users to position widgets on the screen or to divide them among tabs.

However a number of projects that have used this approach considered that the absence of communication between widgets to be a significant limitation, and have developed solutions that extend either the capabilities of widgets, the capabilities of the containing platform, or both.

Reviewing prior approaches to inter-widget communication

Probably the most comprehensive analysis of inter-widget communication implementations and standards is that undertaken by Ivankovic and Zuzak (2011). However, in this paper we focus on high-level architecture and the challenges for users, particularly non-developers. In Wilson, Daniel, Jugel and Soi (2011, in press) we set out the following characterization of user-interface mashups involving inter- widget communications:

Orchestrated UI mashups, where the interactions between the widgets in the mashup are defined using a central control logic;

Choreographed UI mashups, where the interactions between the widgets in the mashup are not defined, but instead emerge in a distributed fashion from the internal capabilities of the widgets;

Hybrid UI mashups, where the emerging behaviour of a choreographed UI mashup is modified by inhibiting individual behaviours, practically constraining the ad-hoc nature of choreographed UI mashups.

Orchestrated UI Mashups

For an orchestrated mashup, each communication channel between widgets needs to be explicitly declared by the user before there is any information flow. This can be done at design time or at runtime, and can use a range of metaphors, but in implementation results in a set of instructions to the orchestration control logic.

In the EZWeb project (see Lizcano, Soriano, Reyes and Hierro, 2008), the orchestration takes the form of a control screen where users create “channels” that link the outputs of widgets with the inputs of widgets. So, for example, a “Map” widget may have a “location” output, and a “Flickr Photo” widget may have a “location” input. The user can therefore make a connection between them.

The Gepetto prototype system also used a wiring model, but exposed this within events in the widget interface itself, for example allowing users to select the components of the widgets as sources and targets of communications (Zuzak, Krolo and Srbljic, 2008).

The Rooftop Marketplace system created by SAP for the FAST project used a direct wiring mechanism directly in the user interface, exposing input and output points of widgets to be connected by users with wires in much the same manner as the Yahoo! Pipes web interface (Hoyer, Gilles and Stanoevska-Slabeva, 2009).

The key challenge for orchestrated mashups is engaging the user in the active construction of the inter-widget communication. While “wiring” may be a fairly easy to understand metaphor for most engineers (and musicians, for example the Propellerheads Reason music software also uses this metaphor; see Villar, Lindsay, and Gellersen 2005), it is unclear if this would pose a challenge for general users.

Choreographed UI Mashups

In a choreographed UI mashup, the interactions between widgets are emergent rather than designed; for example, when a user adds a widget to a workspace or layout, it starts interacting with the widgets around it without any connections needing to be explicitly defined by the user. Each widget is therefore capable of independently sending and receiving communications and of acting on them

A common model for choreographed communication is a publish-subscribe messaging approach. In this model, widgets can subscribe to topics of interest (e.g. “location”) and send messages to topics. The runtime environment is then responsible for routing messages according to their topic.

The strength of this approach is that there is no need for explicit design, removing the barrier to users constructing mashups; mashups also execute automatically without prior configuration (Isaksson and Palmer, 2010).

However, there are also several drawbacks to this approach.

Without explicit “wiring”, widgets are reliant on sending and receiving messages on topics of interest in a format that they are capable of understanding. Isaksson and Palmer (2010) have proposed an approach using a resource-oriented approach, and by including metadata along with the message to aid widgets in interpreting messages they receive. Wilson, Daniel, Jugel and Soi (2011) proposed the development of a limited reference ontology for the widgets within a particular domain of use, for example to support interoperability in enough common scenarios to satisfy most users, but requiring additional work for less common communication tasks.

Another weakness is that with no predefined “plan” of the mashup, there could be the risk of the emergent behaviour of the widgets being pathological – for example, self-reinforcing loops or hunting. This could be a serious problem where the mashup components have real-world consequences, such as SMS-sending widgets.

User Control versus Ease of Configuration

For inter-widget communication, we can identify a number of parameters for the design space independent;y of the messaging infrastructure used to transport them.

In the previous section we discussed one parameter, orchestration versus choreography. Another set of concerns are user control and ease of configuration; in order for users to feel in control of the mashup and its behaviour there is usually a tradeoff with the level of configuration necessary for the environment (see Table 1).

In orchestrated UI mashups, users are directly involved in configuring the interactions between the widgets. While this sets up a challenge for the platform design in reducing the learning curve for the user, it does offer a fairly high degree of user control over the behaviour of the mashup.

In choreographed UI mashups, there is no configuration necessary, however the control is shared between the user in their choice of widgets, and the designers of the widgets themselves, and the interactions are emergent based on those choices. This means that in practice a choreographed UI mashup may create a situation where the user feels the environment is out of their control as widgets interact with one another, possibly even preventing user interaction.

Table 1: Tradeoffs between ease of configuration and user control

Ease of configuration: High Ease of configuration: Low
User control: High Orchestrated UI mashup
User control: Low Choreographed UI mashups

To progress the state-of-the-art in user interface mashups therefore requires innovation to either reduce the difficulty of configuration in orchestrated mashups, or to improve the level of user control in choreographed mashups.

Improving the usability of orchestrated UI mashups

In the MATURE project, an alternative form of orchestration was used, where the user controlled interactions are managed by proximity in layout. So, for example, to initiate a communication channel between two widgets, the user moves one of them in the layout until it lays adjacent to the other (see Nelkner, 2009). This also enables multiple widgets to communicate. This is an interesting user experience approach, though it isn’t clear how accessible such a solution might be.

In the PALETTE project (See Sire, Paquier, Vagner and Bogaerts, 2009), the platform supported an early form of drag-and-drop interoperability between widgets.

Users could drag elements from one widget onto another, triggering a one-time transmission of data from one widget to another. This work has to some extent been superceded by HTML 5’s drag and drop functionality, however it remains an interesting approach as it demonstrates a case of limited orchestration that builds upon an existing metaphor that is likely to be familiar to users. However, as with drag and drop in the browser generally, there is as yet little user experience or design consistency for drag and drop interfaces, and it can be a challenge providing accessible alternatives.

Each of these prototypes provide an interesting approach to reducing the technical barrier to users in creating explicit channels for the orchestration logic; however in each case they also expose additional challenges for making such solutions accessible.

Improving user control for choreographed UI mashups

We refer to a situation where a set of approaches is taken to constrain a choreographed UI mashup as a hybrid UI mashup.

Perhaps the crudest level of user control that can be added to choreographed mashups is a high level “pause” control so that the user can stop and resume all interactions within a workspace. This would directly control the messaging mechanism (for example, the publish-subscribe hub). Control could also be offered at an individual level, for example to isolate a widget from other widgets in the mashup.

However, each of these mechanisms presents a solution to the “runaway mashup” problem but does not address the general issue of user control in how widgets are supposed to interact before there is a problem.

One approach would be to request explicit user permission before each new interaction; for example, a message such as that in Figure 1 might be raised by a mashup platform the first time that two widgets attempt to share information.

Figure shows Alert box with message: ‘“Map” would like to share “location” with “Flickr”. What would you like to do?”’ with two buttons labeled Allow and Don’t Allow, and a checkbox labelled “Remember this decision.

Fig. 1. Example of a message allowing users to control emergent mashup behaviour.

While this might seem intrusive, enabling this level of user control may be sufficient to shift the balance of user trust for such a system, and could be made less annoying by remembering decisions by users so that they are not asked for subsequent sessions.

Conclusion

There are sufficient projects requiring inter-widget communications in mashups that a need is emerging emerging for a standardized solution, or at least a set of design patterns.

In this paper we looked at two classes of inter-widget communication architecture – orchestration versus choreography – and the design challenges that need to be addressed to overcome their limitations in terms of either ease of configuration or user control.

In the case of orchestration, there have been innovative attempts to reduce the difficulty of configuration for users using different configuration metaphors. In the case of choreography we have suggested several strategies for putting the user back in control of their workspace, however these have not been implemented in a running prototype.

In neither case do we yet have a body of evidence for the usability of these approaches, despite considerable investment in implementations. Al Sarraj and De Troyer (2010) note in reference to visual wiring tools for mashups generally that “These tools target Web users without programming background and promise that creating a Web mashup is just a matter of a few mouse clicks. However, no profound usability evaluations have been performed to justify those claims.” While there has been investment in researching ways to simplify mashup enabling tools for end-users, this seems to be largely on the basis of intuition rather than an understanding of user needs or competence, and in some cases focus on metaphors understood by engineers (e.g. wiring diagrams).

Other researchers have also reported accessibility issues with mashup platforms (Schwerdtfeger and Parente, 2008) that need to be addressed. For example, where changes propagated across widgets take place without notifying the user, potentially also disrupting keyboard navigation or window focus.

Therefore we recommend that, before committing to standardizing approaches to inter-widget communication from an engineering perspective, there should be corresponding work in the usability of user-created mashups taking account of users perceptions of the issues of configuration complexity and degree of user control, and to identify the set of design metaphors and language for mashups that a broad set of users can relate to.

References

Sire, S., Paquier, M.,Vagner, A., and Bogaerts, J.: A Messaging API for Inter-Widgets Communication. Proceedings of the 18th international conference on World Wide Web, 2009. ACM New York, NY, USA. ISBN: 978-1-60558-487-4

Wilson, S., Daniel, F., Jugel, U., and Soi, S.: Orchestrated User Interface Mashups Using W3C Widgets. Proceedings of 3rd International Workshop on Lightweight Integration on the Web (ComposableWeb 2011), Paphos, Cyprus, 20th June, 2011. Springer.

Law, E.L.-C, Muller, D., Nguyen-Ngoc, A.V.: Differentiating and Defining Portlets and Widgets: A survey approach. 2nd Workshop on Mash-Up Personal Learning Environments (MUPPLE-09), Nice, France, September 29, 2009.

Isaksson, E., Palmér, M.:Usability and Inter-widget Communication in PLE. In Proceedings of the 3rd Workshop on Mashup Personal Learning Environments (MUPPLE10), Barcelona, Spain, September 29, 2010, CEUR Workshop Proceedings Vol. 638, 2010.

Nelkner, T.: An Infrastructure for Intercommunication between Widgets in Personal Learning Environments. In WSKS (2)(2009) 41-48

Lizcano, D., Soriano, J., Reyes, M. and Hierro, J.: EzWeb/FAST: reporting on a successful mashup-based solution for developing and deploying composite applications in the upcoming web of services. In Proceedings of the 10th International Conference on Information Integration and Web-based Applications & Services (iiWAS ’08), Gabriele Kotsis, David Taniar, Eric Pardede, and Ismail Khalil (Eds.). ACM, New York, NY, USA, (2008).15-24.

Hoyer, V., Gilles, JT, Stanoevska-Slabeva, K.:SAP Research Rooftop Marketplace: Putting a face on service-oriented architectures. IEEE Congress on Services, pages 107–114, 2009.

Villar, N., Lindsay, A., and Gellersen, H. Pin & Play & Perform: a rearrangeable interface for musical composition and performance. In Proceedings of the 2005 conference on New interfaces for musical expression (NIME ’05). National University of Singapore, Singapore, Singapore, 188-191.

Zuzak, I., Krolo, J., Srblijic, S. Geppeto: Promoting End-users to Gadget Developers. Google Tech Talk, Mountain View, California, USA (2008). Talk: accessed 4th July 2011 from http://www.youtube.com/watch?v=fbUvCdADz_A

Ivankovic, M. and Zuzak, I. List of system that enable inter-window or web worker communication. Website, April 2011: http://code.google.com/p/pmrpc/wiki/IWCProjects. Al Sarraj, W. and De Troyer, O. Web mashup makers for casual users: a user experiment. In

Proceedings of the 12th International Conference on Information Integration and Web-based Applications & Services (2010). ACM, New Y ork, NY , USA, 239-246. DOI=10.1145/1967486.1967526 http://doi.acm.org/10.1145/1967486.1967526

Schwerdtfeger. R., and Parente, P. Web 2.0 mashup accessibility. Website, 2008: http://www- 03.ibm.com/able/resources/mashup.html

About these ads
This entry was posted in apps, development, mashups, standards, widgets. Bookmark the permalink.

4 Responses to Design challenges for user-interface mashups: user control and usability in inter-widget communications

  1. Pingback: Design challenges for user-interface mashups: user control and usability in inter-widget communications « juandon. Innovación y conocimiento

  2. Ivan Zuzak says:

    Hi Scott,
    it’s really cool that you decided to publish the paper as a blog post! Thanks!
    Few comments:

    1) I think that the work you are doing is *very* important and this field has high impact. And I strongly agree with your conclusion that we need more and better usability studies.

    2) A better reference to our systematization of IWC systems would be the paper published on arXiv (http://arxiv.org/abs/1108.4770) since it analyzes a broader list of systems and provides more detail and insight.

    3) Concerning the names “orchestration” and “choreography” – these names already have well defined meaning in distributed systems, so overloading these terms with different meaning might confuse people. In dist. systems, “orchestration” means that a centralized entity exists that communicates with other components in order to achieve some control and data flow, while “choreography” means that no such central entity exists and that parts of the overall logic for achieving the desired control and data flow is contained in the components themselves. If I understand your definition correctly, “orchestration” means “user involvement required for defining the desired IWC interaction” while “choreography” means “no user involvement required”? Or to use an example, let’s say that several widgets exists that use publish-subscribe to communicate. And let’s say that you add those widgets to your dashboard. If, as a user, you need to specify which widgets are actually listening on channels (“turned on”) or need to choose which channels they are listening on from a list – you would call this orchestration, right? And if the widgets are preconfigured to communicate on some channels right away – you would call this choreography, right?

    4) When you say that “Without explicit wiring, widgets are reliant on sending and receiving messages on topics of interest in a format that they are capable of understanding.”, it’s also important to note that they are reliant on using the same channels. In other words, if pubsub widget A uses the “location” channel and pubsub widget B uses the “locations” channel (notice the “s” in the end) – nothing will happen even if both use the same data format. So, someone needs to define/standardize the channel names, which, in a way, WebIntents will try to tackle also.

    5) Furthermore, it think that the problem of the data formatting/understanding issue may be orthogonal to the problem of end-user involvement (i.e. it does not emerge in choreography mashups only). In other words, in orchestration mashups, if a user specifies that widgets A and B should communicate (e.g. specifies that both should listen on the “location” channel), there also could be a formatting issue if those two widgets don’t use the same format for their messages, right?

  3. scottbw says:

    Thanks Ivan!

    I agree that “Orchestration” and “Choreography” can be misinterpreted – I think the distinction is more along the lines of “designed” versus “emergent”. In the “Orchestration” model the mashup as a whole – the widgets and their interactions – is designed or configured explicitly. In the “Choreography” model the widgets are combined but their interactions are allowed to emerge from the behaviour of the individual widgets. Perhaps not so much “choreography” as “after-concert party”: you put the widgets in a room and they start talking to each other, if they have something to say of mutual interest. But you don’t make a design decision beyond who to invite (to stretch the metaphor further…)

    I think Web Intents potentially has a lot to offer in terms of standardising both the message channels/semantics and also in offering a consistent user experience where we can make sure the user is in control; hopefully W3C can make some headway and Mozilla and Google can settle their (seemingly quite minor) differences.

    I’d be interested to see if their are IWC scenarios that wouldn’t suit a Web Intents approach, but can still put the user in control.

  4. Evgeny Bogdanov says:

    Thanks Scott for this classification and hints for UI problems!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s