DevCSI | Developer Community Supporting Innovation » julian-cheal

Google Wave Hackday

julian-cheal — Tue, 24 Nov 2009 10:55:08 +0000

Cameron Neylon organised a Google Wave Hackday at Nature Publishing’s London Offices which was sponsored by DevCSI. The theme for the day was to build a Google Wave gadget to facilitate Scientists using Google Wave.

On the day there was only a small group of us at Nature, about nine developers from Nature, Cameron and a colleague from the STFC Dan Hagon and myself, however there were many more people participating during the day via Wave.

We set up camp in one of the Meeting rooms at Nature. For a venue Nature Publishing was great, we had a nice airy room, with a view over the cannel good wifi and power sockets, the only thing lacking was the coffee, however the was a Costa down stairs in the building and various Coke machines so caffeine wasn’t an issue.

Firstly before telling all about the Hackday a quick history lesson on Google Wave. Google Wave is “a personal communication and collaboration tool” according to Google, the term Wave it’s self was inspired by “Firefly” the Sci-fi television series by Joss Whedon. In Firefly a Wave is a form of electronic communication that was often video call or video message. The inspiration from Firefly not only gave the name Wave, but also the some of it’s error messages. “Everything’s shiny, Cap’n. Not to fret!”, “This wave is experiencing some slight turbulence, and may explode.”, and “Curse your sudden but inevitable betrayal!”

The actual hack day started with with a general discussion on Google Wave (technically the actual discussion for the hackday started before the event over a series of Google Waves) – what people had done before on it, whether they’d used it or not, what they wanted to achieve on the day. There was some good discussions that we had for the first hour or so of the day. This in it’s self was a valuable part finding out what people had done already with Wave and what people wanted to do.

From this discussion it emerged that there were some definite groups that people wanted to split up into, one was building a Wave robot from scratch, another was improving Dan Hagon’s Chemspider molecule gadget, Cameron was working on a demonstration to use Wave as a collaboration tool for drafting Scientific papers. There was also a RDF Robot, EBI reflect Robot and various others.

Throughout the remainder of the day, everyone worked in their groups or on their own, on their specific topics, writing code coming up with errors expressing those errors both verbally and via Wave so as to include the remote hackers. One of the main stumbling blocks of the day turned out to be that currently the only way to create a robot that can interact with Google Wave is to have it hosted on Google Appengine. Now don’t get me wrong Appengine is a great service for developers to upload applications for free on to the web. The problem comes in the fact that you can’t locally debug your Appengine code that your using for Wave as it needs to be in Appengine to talk to Wave as mentioned, also Appengine doesn’t always update it’s log files instantly so it could take a few minuets to discover a bug.

The general workflow went like this:

Write code
Upload to Appengine
Test in Wave
Read logs
Fix bugs
Re-upload to Appengine
Repeat

Which is fine, it’s just it took a little longer than you’d like to write code.

Now for me this is were the excitement of being at a Hackday really comes through. It turned out that the developers at Nature Publishing in London are Ruby developers and I myself have been learning Ruby so we instantly had a bond there, however not only that they are Ruby developers who have to use JRuby to interface to all of Natures Java Applications. Now stepping back a bit going back to Google Appengine. With Appengine you can upload Applications written in either Python or Java.

Now I and the other Ruby developers at the Hackday can code slightly in Python, but someone mentioned that there had been a project on Github to get a JRuby App running on Appengine. So we all independently downloaded the code and tried to build a Wave robot using JRuby and Appengine. However to cut a long story short we all failed. The code just wasn’t up to the task. So during lunch we teamed up and started working together to get a working JRuby implementation working on Appengine that could talk to Google Wave.

Speaking of lunch, as with all good Hackdays lunch was pizza from Firezza.com and instead of writing a paper list of what everyone wanted we started a Google Wave! Unfortunately the pizza restaurant although they allow you to order online, as of yet you can not do this via Wave.

The afternoon carried on with much code writing although everyone was feeling rather full after the mountains of pizza. Then late in the afternoon people who had got a Wave robot working were doing show and tells, explaining the resoning behind their ideas and how they achieved it.

As far as the Rubists were concerned getting a working Wave robot was the last thing on our todo lists. Throughout the afternoon and much coffee and Coke later we had finally got a working set of code to deploy a JRuby app onto Appengine. We would have also gotten a few Wave demos too, but unfortunately the rest of the group felt it was time to put the code down and go to the pub. Now any developer knows if you have to force a developer to stop coding and go to the pub, they must really be in the zone.

Enhancing Institutional Repositories One Line of Code at A Time

julian-cheal — Mon, 23 Mar 2009 11:08:01 +0000

Back in February I met with quiet a few UK HE institutional repository managers over a period of two weeks, the meetings were designed in order to gather some basic requirements on what additional light-weight software could be developed for repositories; that would aid not only repository managers, but also repository users and depositors.Once back in the office I read over my notes that I took down from the meetings and wrote them up in a big list and pasted them through the wordle.net tool. What this gave me was a tag cloud of all of my notes, which looks like the following. (Click the image to view a larger version in a new window.)

Repository Wants List Wordle

However as you can see it’s hard to read due to the relative size of the words repository and repositories. Which you’d expect in a meeting about repositories, but not too helpful in this case. So here is an updated version with the repository words taken out.

Repository Wants List Wordle Without Repositories

In the meeting the main topics we asked about were:

Deposit
User Workflows
Metadata creation
Managing copyright
Preservation
Using the content of repositories
Integrating with institutional software
Integrating with other software, services and websites outside of the institution

Looking at the wordle output, it is easy to notice that the main topics discussed at the meetings where:

Search
REF
SWORD
Import
Arxiv.org (or any other type of subject specific repository)
Funder
Statistics
And so on

These are all good topics to start development on. During our first meeting we had a flip chart and drew out some ideas of a couple of applications, that could be developed for the desktop to aid in repository deposit and then one to deal with publications output.The following are mockups of what such applications could look like.

Software Ideas

Desktop Repository Deposit Tool

The idea behind this is for a simple and easy to use application for academics to simply drag and drop their files into the application, fill out some basic metadata and simply click deposit. That’s all they’d need to do to deposit into their institutional repository. Behind the scenes SWORD will be taking care of talking to the repository and uploading the files, so then the repository manager can check the upload just like any other deposit.

Desktop Repository Deposit Tool

Desktop Repository Deposit Tool Advanced Options

Desktop Repository Deposit Tool Integrated Sherpa RoMEO Lookup

Desktop Repository Deposit Tool Drag and Drop files

Desktop Repository Deposit Tool Browse For Files

Software Ideas

Multiple Feed RSS Publications Output

The idea behind this application is sometimes academics would like to have some control over their publications output and embed them into either their personal pages or departmental pages. This application would run on their desktop the user would copy and paste their own rss feeds from their institutional repository. The application would then show the feed in a list, which the user can then decide which publications they wish to view in their new feed. The idea is for an academic to include multiple input feeds maybe from different repositories, where they have items deposited. Also to create a formated output or a widget for ease of inclusion in an existing website.

Desktop Repository RSS Publication Output Tool

Desktop Repository RSS Publication Output Tool New RSS Feed

Next steps

This post is really the starting point of these ideas. What we really want is for user interaction, so please feel free to comment on any of the ideas mentioned in this post, or to add new ideas and discuss possible applications that can be written for UK repository users.

SWAP – Django Demonstrator

julian-cheal — Thu, 19 Feb 2009 19:12:43 +0000

In the course of working with the Scholarly Works Application Profile at UKOLN it has become apparent that it is often difficult for repository managers to understand how SWAP might work for them, as they have nothing concrete to work with. Because of this, it is difficult to ‘sell’ the idea of SWAP to this important stakeholder-group and it has meant that SWAP is still largely untested in terms of usability. For this reason we have decided to investigate a concrete implementation of SWAP using a demonstrator that can be used to allow users to play with entering real world data to populate a SWAP profile.

I have recently been using Django to make quick prototypes and in this case I realised it would be the perfect tool to create the demonstrator with minimum fuss. Django has many great features including my favourite implementation of a template engine and object relational mapper (ORM) and an automagic administration interface. In this case we are interested in the latter two. The data model is defined in Python code and the database generated from these models. In most cases there is no need to write any SQL with the Django model API providing the functionality to alter objects. Having used several ORM’s in the past including Hibernate, which admittedly has a lot more to do with greater database support and more features, really appreciate the simplicity. The admin interface comes for free and can be used as is or customised with little effort. It is this admin interface that I am using as a SWAP demonstrator.

The different major elements of SWAP can be represented as objects and hence as a database table which in turn appears in the admin interface.

Django admin interface

As we would expect each main element of the profile can be added and built up in order. In this example we are adding an expression to an existing Scholarly Work.

SWAP Expression

We will be running an interactive workshop for an invited group of repository managers to test our SWAP implementation, with a view to getting feedback about the usability of the profile itself. Any comments about this approach will be welcome!

Sherpa RoMEO AJAX Autocomplete

julian-cheal — Mon, 19 Jan 2009 14:16:15 +0000

Depositing papers in an institutional repository usually requires manual entry of data into a web form by the depositor who is either an academic, clerical staff, or the repository manager. The next part of the workflow is for the item to then be validated by the repository manager, before being deposited into the repository.

What problems are there with manual entry?

The problem with manual entry of data by humans is that it leaves room for mistakes, typos and general incorrectness. Also manual entry is quite a laborious task. One textual section prone to mistakes is the publication details of an item. This section usually consists of a journal or publication title, an ISSN and the publisher. Getting these details correct is useful in knowing whether a paper can be deposited or not. Using a journal’s name as reference for looking up the publisher’s details for example a repository manager needs to know that the data is correct.

Can a paper be deposited in a repository?

Repository managers need to know whether copyright polices allow a paper to be deposited in to their institutional repository, or whether there is an embargo on the paper. If a paper can be deposited which version of the paper can they use? The pre-print or the post-print, or any other variation of the paper. Services such as Sherpa’s RoMEO have been created to give managers a tool for searching and retrieving such information.

The Sherpa RoMEO service is maintained by Sherpa and supported by JISC and the Wellcome Trust. It is a development that has grown out of the RoMEO project which aimed to produce a listing of journal publishers. The journal information is provided by the British Library’s Zetoc service which is hosted by MIMAS. Community contributions and Sherpa’s partners update the publisher information.

However even with services like RoMEO this still leaves the situation as it was before; manual entry of publisher information, being manual checked in RoMEO for any restrictions on an item. However there is now a better solution. AJAX autocompletion.

How it works:

The user starts typing in the journal or publication title
Local JavaScript sends the text via a http request to Sherpa RoMEO API
RoEMO returns the results back encoded in XML
Local xScript (either Perl or Java or other) parses the XML results
Local JavaScript then displays the results and the user chooses correct title from them
The publication section then gets filled in with correct data.

RoMEO API Lookup using AJAX

EPrints & DSpace

EPrints (current version 3.1.2) has built in autocomplete functionality using the http://script.aculo.us/ JavaScripts. DSpace (current version 1.5.1) currently does not have autocomplete included out of the box. There is a plugin http://wiki.dspace.org/index.php/Autosuggest_using_AJAX that (technical) users can install. Stuart Lewis is currently working on implementing autocomplete natively into DSPace.

Within the EPrints repository there are a couple of different approaches to autocompletion. One is what EPrints call authority lists, which are simply static files that site on the web server. http://wiki.eprints.org/w/Autocompletion_and_Authority_Files_(Romeo_Autocomplete)The EPrints autocomplete can search either the static authority lists or, functions that execute SQL queries. Using the authority lists can range from a simple implementation to a complex solution. EPrints users can also install a local version of the RoMEO database which it stores in an authority file. Using AJAX autocomplete, EPrints can look up the publisher information for a journal. However this data will need updating to add/remove publisher information as needed.

Another option is to use a Perl script http://lucas.ucs.ed.ac.uk/test/ajax-romeo.html by Ian Stuart at Edina. This is a Perl script which uses the Sherpa RoMEO API unlike the EPrints authority lists. Using the RoMEO API means that this method will have up-to-date publisher information. Installation of Ian’s script is very easy to do and instructions to do so are here http://wiki.iedemonstrator.org/confluence/display/ied/EPrints+Romeo+AJAX+lookup+widget

Why autocomplete is better?

An AJAX autocomplete form asynchronously requests data in the background without the need of submitting the page as is typical of most web forms. Therefore as the depositor types in the journal name or title the AJAX script requests a list of results from the Sherpa/Romeo database or the EPrints authority lists in ‘real time’.

If a depositor does not have JavaScript enabled in their web browser, the AJAX script will not load, therefore failing gracefully. Allowing the depositor to continue to enter the details manually.

Autocomplete is better as it allows the depositor to not have to look up the journal information separately in RoMEO while completing a deposit. Furthermore with manual entry user error can produce typos this method removes user error and should allow for faster more accurate deposits.