DevCSI Stakeholder Report: Results
The structure used for the results follows the same structure of the survey and interview i.e.:
Selected quotes from interviews and open responses on the survey support quantitative data from the survey.
Adding value and fostering innovation
Developers supporting innovation
A number of assumptions were tested with each stakeholder group to see how much they agreed with them. The overall results across all respondents are shown in Figure 1:
Assumption |
Strongly disagree |
Disagree |
Agree |
Strongly agree |
Don’t know |
| Developers understand the local context and act as a bridge between remote service providers/open source software communities and local end-users | 0.7% | 9.7% | 44.2% | 33.7% | 11.6% |
| Developers take the remote service/open source software and add value by integrating it into the local context/systems | 1.1% | 4.5% | 38.6% | 45.3% | 10.5% |
| Developers work closely with end-users to deliver innovation | 1.9% | 17.2% | 43.1% | 28.1% | 9.7%, |
Figure 1: Developers supporting innovation table
Examining the data in more detail using the different stakeholder groups showed similar results, with the majority rating each statement as “agree‟ or “strongly agree‟. The results therefore demonstrate that the majority of respondents are in agreement with each of the assumptions.
It is clear from many of the free text responses to the survey and interviews, that there is value to be gained from local developers:
“Local development is not just a luxury, it is essential to ensure proprietary software fulfils the needs of users”
“We have a number of packaged solutions which have been customised / enhanced in-house. This is much more cost-effective and likely to meet customer needs than requesting such changes from the original solution provider”
“Developers can relate things to the local context, understand the structure and procedures of the institution, and use local terminology to explain possible future developments to end-users”
“Local developer can take … software and tailor it to the specific needs of the institution to help the institution stand out”
“A good developer is far more than a developer – they are your strategic instrument”
However, there were a number of survey responses which suggest that this value may not currently being realised:
“I feel that those statements should all be true, but this doesn’t reflect my own workplace
The answers reflect an ideal world – in the real world the experiences often aren’t so ideal”
Due to the terminology used within the survey, it is not clear whether the responses given reflect the current situation, or opinions of an ideal world situation. However it is clear that there is definite support for these statements though some areas need further investigation.
Upon further examination of the survey responses, it was clear that there were some differences of opinion about the role of developers and their relationship with end-users:
“I agree that developers work closely with end-users, but I think there is often room for improvement in this area”
“Developers are technical people and direct communication with end users is not always helpful”
This view of developers being more removed from end-users was also supported by some of the responses to these questions during the initial semi structured interviews:
“It is sometimes difficult to get a level of engagement so often have to work through a proxy (e.g. research leaders as a way to get through to researchers)”
“Most important thing is to talk to end users but it is difficult to know who to talk to”
“I feel a bit removed from real end-users generally”
A number of responses suggest that the relationship is possibly not directly between developers and end-users, but through a mediator:
“Local developers seldom interact directly with end-users. Very different species. The need for an intermediary ‘translator’ role between the two groups has been identified”
Different approaches are suggested to support the relationship between developers and end-users:
“As the manager I tend to act as the bridge between the developers and the end users”
“We use system librarians to bridge between users and developers, with good results”
It seems that working with end-users is seen as an important part of innovative development work; however this relationship may be more beneficial when it also involves others. This echoes the view of a funder:
“Developers need to be connected to their end-users and working in a team with strong links to other areas within the institution to maintain this relationship”
A number of examples of software/web development work showcasing customisation at an institutional level were given, including:
- VLEs (e.g. Moodle, Blackboard, Sakai)
- Repositories (e.g. DSpace, EPrints)
- Library Management Systems (e.g. Talis extensions, Ex-Libris plugin)
- Content Management Systems (e.g. Drupal, Joomla, WordPress)
Sharing to benefit sector
When asked about local innovation and the work of developers being shared more widely, the responses were overwhelmingly positive with 87.7% who “agree‟ or “strongly agree‟ that local innovation can be shared to benefit the sector (see Figure 2).
Figure 2: Sharing to benefit sector chart
The survey responses and interview data supported this:
“By sharing our work… we would save ourselves a lot of development time, and probably end up with better products in the long run”
“If developers could innovate together, the sector as a whole would clearly benefit”
“The dissemination of knowledge and experience benefits the broader community, reduces the burden on individual institutions and fosters greater interoperability”
“Particularly relevant in HE as many want to support the wider community and raise the profile of themselves and the institution. Universities are usually happy with this code being shared as it’s not commercially sensitive”
“We’re all part of the same troop so should be sharing more. Things are costing much more than they need to because things aren’t being shared”
However, some noted difficulties or barriers to sharing and commented that development work isn‟t shared as much as it could be due to difficulties in sharing innovation, concern that sharing would result in more generic resources, and competitive considerations:
“Sharing is not something that happens easily and the sharing of innovation is particularly difficult”
“Often local variations will be necessary, and if it is thought that there is a team outside of the University already working on something, then no further resources may be allocated to it. The institution is then left with a one-size, fits all resource, which may or may not ‘fit’”
“I get the impression that universities see themselves as competing with each other and are sometimes reluctant to share”
A number of people referred to the work of JISC funded projects as an example of successful sharing within the community:
“Wherever practical, code written within the institution should be made available under an open source license for use elsewhere especially when it is funded by external bodies, such as JISC”
“The work of SWORD… began as a JISC funded project using local innovation at a number of different institutions. This was hugely influential within the sector and is now used by a large number of different systems”
In addition to sharing of code, many responses referred to other areas of work that can benefit from sharing, including:
- Contributing to the wider open source community (not just within education)
- Collaborative development of sector specific products
- Shared best practice in use of particular systems/integration of systems
- General shared best practice (e.g. approaches, workflows)
Value of local developers
Figure 3: Value of local developers chart
As can be seen in Figure 3, the response to the question about the value of developers was not clear. A large number responded “Don‟t know”, with only few having strong views either way. However, of those who expressed a decisive opinion, more were in disagreement than agreement.
Investigation of the survey responses showed that there is a huge variety in the opinion about value being realised by stakeholders, with many remarking that they weren‟t sure how to answer this as they know of many different situations:
“Think it varies, and depends on the extent to which developers align their work to the strategic goals of the institution”
Some respondents felt that developers were valued by key stakeholders such as senior management:
“Within HE I believe there is a fairly clear understanding that local developers are needed to ensure systems are developed in line with institutional policies and guidelines”
“In our institution they do; we need a local developer to do customisation to improve the student experience”
However a number of others feel that the value of local developers is not understood:
“I think they are undervalued, based on evidence of short-term contracts and professional development opportunities for them”
“I think that senior managers often get caught up in the cost benefit trap and tend to undervalue the local resource because it is almost always more costly to sustain”
“I’d say on the strategy level they’re not considering the advantages of having local developers”
“I think the focus from senior management is that developers are there to administer systems, tweak systems, and work on projects. This doesn’t give the scope for innovation and developers’ capacity for innovation can be overlooked”
“Value is badly costed in universities…if you said this developer brings cost benefit to university of £200K a year and we can point to that, then you’d get people to look at it”
“There is a vicious cycle… a lot of the best things you develop are the most seamless things so the developer is invisible and tends not to get credit”
The area of software development and open source software in particular is still a relatively new concept and there is a wide variance in understanding across the sector. As one survey respondent summarised:
“The benefits of local or outsourced developers, cloud computing, hosted software etc, is a complex subject which is constantly changing. It remains a challenge for IT departments to keep senior management up to speed with what is best practice”
Barriers to innovation
This was an optional question in the survey and a question in all interviews; “What, if any, barriers do you perceive there to be to developers adding value and fostering innovation in FE and HE?” 152 survey respondents chose to share their ideas, many referring to similar barriers as mentioned in the interviews. These included:
- Lack of time – full time developers are too busy fire fighting with day-to-day support and maintenance of existing systems, often no time dedicated to innovation (where they would need uninterrupted time to focus)
- Lack of staff (i.e. skilled developers) – developers are often on short-term contracts
- Lack of funding – developers funded through project work do not have scope to innovate outside the project
- Lack of training for developers – many new technologies and programming languages to learn
- Poor internal communication (between developers, managers, users etc.) – developers can be isolated within the institutions and therefore lack an overview of the wider context and do not have an understanding of user needs
- Expectations of developers (e.g. someone who can just do a set task, no scope to innovate) – there is a need to educate non-developers
- Risk-averse management of organisations – organisational structure doesn‟t allow for innovation
- Getting management approval for innovative ideas – this can be too long a process by which point the idea may no longer be relevant
- Institutional policies and bureaucracy (e.g. against use of open source, in-house development or cross-institutional collaboration)
- Competitive advantage – don’t want other institutions to benefit from internal development work
- Cost of proprietary software (if an open source option is not viable)
- Licensing of software – sometimes unable to customise or adapt software due to licensing restrictions
- Lack of awareness of institutional goals – developers need to align innovative work to the aims of the institution
- No forum for discussion or sharing innovation within developer community – not the same level of community as other areas within education (e.g. librarians)
Developers and managers of developers (and those who had worked in either capacity in the past) were invited to give their views on the training and development opportunities for developers. There were 156 survey respondents to this question. Each were asked to state how much they agree with statements addressing career development opportunities; training, development and networking opportunities; and barriers to training and development.
Career development opportunities
Figure 4: Career development opportunities for developers chart
As is clear in Figure 4, the majority of respondents (75%) “agree‟ or “strongly agree‟ that there are limited career development opportunities in FE and HE. Many commented that the only real progression route is into management which removes them from the development work they enjoy. The lack of career development opportunities seems to be a major concern, with some even changing careers due to this:
“I left my previous role as a developer in HE because of a lack of career development opportunities in that environment…despite the fact that I actually had a very understanding manager, who attempted to provide the proper opportunities for me. However, even she was limited in what she could provide based on the constraints placed by upper management, budget and the university as a whole”
However, this is possibly not exclusive to FE and HE and could be a wider issue:
“I don’t think of career development opportunities as being much more limited than any other industry. In fact, in HE and FE there is probably more opportunity to gain seniority while retaining an element of a developer role (rather than moving exclusively to a management role) than in industry”
In one of the interviews with a developer, it was suggested that the fluid nature of the work of a developer and the constant changes in technology mean that job titles and jobs roles are constantly changing, as are the skills necessary to fulfil them.
One area of career progression that emerged as a positive move is to enable more experienced developers to take on more of a strategic role:
“Giving [experienced developers] more of a strategic responsibility…gives a combination of young new developers and more experienced developers with extensive knowledge of the institution and its systems”
There were also some examples of career opportunities for developers in HE, including being able to get involved in working with new technologies and taking advantage of funded research projects.
Training, development and networking opportunities
Figure 5: Training, development and networking opportunities for developers chart
Opinions on the opportunities for training, development and networking were more evenly spread (see Figure 5). There were many who felt that the opportunities for training were good, covering both internal and external training:
“[My organisation] provide numerous opportunities for all staff to train and develop – not just skills restricted to the specific job e.g. I have undertaken project management, bid writing and numerous other ‘non-IT’ courses”
However a common theme through other responses was the difficulty in keeping up-to-date without investing a lot of their own time and money in learning by doing:
“It’s a self-taught, often self-financed (i.e. in your spare time) approach to learning and gaining development skills”
“In terms of developing skills of relevance to software development it is more a case of learning by doing. Developers learn by trying different tools to solve problems (even those relating to non-work related topics – thinking about Google’s employee scheme whereby 20% of time is spent on their own project), and these help develop their skills”
Many expressed the importance of networking to support personal development and build connections within the community to remain informed and begin potential collaborations, and though some gave examples of successful networking opportunities (e.g. Dev8D, IWMW, Mashed Libraries), there appears to be a need for more networking opportunities.
Barriers to training and development
Figure 6: Barriers to training and development for developers chart
Figure 6 demonstrates views on the barriers to training and development for developers. As can be seen, there were differences here; some commenting that they have not experienced any barriers, whilst others were reporting multiple barriers.
The main barriers referred to were funding issues and time to attend events:
“Training budgets are often not formally agreed, and the increasingly high cost of training and conferences prevents them from taking courses or going to relevant conferences”
“Mostly financial, but often limited time – too much stuff to work on / develop, not enough time and emphasis given to improving skills (and therefore productivity)”
“Time is a big factor even if there is opportunity and support. Many developers work in small teams or even alone making time out hard”
“I know of developers who have been unable to attend events that are totally relevant to their job role and based in their own institution because their manager felt that they had too much work to do and couldn’t afford the time away from development”
Some developers commented on the difficulty in getting permission to attend events as their manager couldn‟t see the value of them attending:
“It is difficult to pitch the value of them to a manager…they are very ethereal in content and you can’t be too precise about what’s going to be on the agenda. So it’s hard to be clear to a manager about what exactly you can show as a return on investment for such events”
The interviews also highlighted an issue in discovering relevant training and events:
“I think developers can find it difficult to find relevant events which meet their needs and are within budget (commercial courses are too expensive)”
The next question tested a number of perceived benefits of having a developer community, and was completed by 261 respondents.
Strongly disagree |
Disagree |
Agree |
Strongly agree |
Don’t know |
Rating Average |
|
| Supporting funders such as JISC, for example, through prototyping and testing new initiatives/services/APIs and identifying and framing problems which could benefit from sector-wide intervention from organisations such as JISC | 0.8% | 6.1% | 47.5% | 31.4% | 14.2% | 3.28 |
| Providing career development opportunities for developers | 1.5% | 9.6% | 52.5% | 23.8% | 12.6% | 3.13 |
| Providing a forum to work with vendors | 0.8% | 15.7% | 55.9% | 16.1% | 11.5% | 2.99 |
| Providing peer to peer support for developers | 0.8% | 3.1% | 47.5% | 42.9% | 5.7% | 3.41 |
| Providing extremely cost-effective training to their peer community, especially through ad-hoc training events | 1.9% | 6.5% | 46.7% | 36.0% | 8.8% | 3.28 |
| Fostering innovation, for example through interacting with end users | 1.1% | 9.2% | 45.6% | 34.9% | 9.2% | 3.26 |
Figure 7: Benefits of a developer community table
As can be seen in Figure 7, the most common response for each statement was one of agreement. The rating scores enable us to get more detail and demonstrate that peer to peer support for developers got the highest score (i.e. people agree with this statement the most). All the scores are relatively high, with providing a forum to work with vendors being the lowest – but still a majority agreement on that statement also. The interviews had similar responses, with the vast majority of those interviews agreeing with the stated benefits of a developer community.
We also asked if there were other benefits in addition to those listed; suggested benefits included:
- Raising the profile of developers and providing validation for their work
- Encouraging sharing and collaborative approaches to problem solving in order to reduce duplication and help progress the sector as a whole
- Development and promotion of standards Links with commercial equivalent developers
- Links with international projects
- A skills register/directory to establish a skills pool for support in particular areas of development
Although this exercise was primarily focused on testing assumptions that the DevCSI project has been based on so far, we also took the opportunity to get some feedback on involvement in DevCSI and idea for future areas of focus.
Of the 259 respondents who answered this question, the below shows a breakdown of the level of engagement with DevCSI:
- 80 (30.9%) had attended a DevCSI event (47 developers; 17 users; 7 managers of developers; 6 senior managers; 2 vendors; 2 funders)
- 20 subscribe to the DevCSI blog (9 developers; 6 users; 3 senior managers; 1 manager of developers; 1 vendor)
- 32 are members of the Developer Contact list managed by the Developer Focus Group (22 developers; 6 users; 2 senior managers; 1 manager of developers; 1 vendor)
- 25 had presented at an event (10 developers; 7 users; 4 managers of developers; 3 senior managers; 1 vendor)
- 7 had sponsored an event (2 funders; 2 senior managers; 1 vendor; 1 developer; 1 user)
The majority (59.8%) had not yet been involved in DevCSI activities. Some of the survey responses and interview data suggest that DevCSI could benefit from raising its profile:
“I think more visibility would help”
“I probably should have heard of DevCSI but I haven’t”
“I’d like to see DevCSI become more widespread”
“Main challenge is outreach to the wider community”
“[Challenges include] reaching developers not on JISC projects, reaching a diverse range of developers and fostering an open and welcoming community for newcomers”
Respondents were asked to provide examples of cases where developers had supported innovation of the building of a developer community. A number of examples were provided demonstrating cases where DevCSI has supported innovation and begun to build a developer community. This also included collaborative projects that had begun at DevCSI events, and connections that have been made through DevCSI activity and been built upon since.
The stakeholders interviewed and surveyed were asked whether there were additional activities and roles that DevCSI could undertake to fulfil unmet needs in the community.
Different stakeholder groups such as vendors and users expressed an interest in further events which focus on bringing together different groups for a specific purpose, similar to the previous organised Reading List event:
“It would be good to have more similar events to the reading list one, where different stakeholders are bought together. This is really useful for us [vendors] to listen to the community to find out what their priorities and current concerns are, which can help us to amend our current products and shape our future products”
“Future similar events which bring together the different groups would be useful – it was good to get a mix of people together for a specific event rather than a developer attending a library conference or a librarian attending a developer conference where they may feel out of their depth and not so integrated”
Other suggestions for future of DevCSI, arising from needs which are not currently being met included:
- Case studies of good practice (e.g. joining up the gaps and building links between different products)
- Inclusion of FE rather than a focus on HE
- A developers IRC network/channel or discussion forum
- More updates about outcomes from events
- A directory site or “one stop shop‟ portal to link to relevant projects (a code repository or wiki perhaps) and events
- A roadshow event within institutions to get other stakeholders on board
- Support for not just developers, but other staff (e.g. library staff and managers) who have to carry out technical work but don‟t have the skills of developers
- More focus on problems that need resolving in the next few years, not just immediate future
- Work to define a developer role and clear career path (or wider discussion of these issues to improve recognition)
- International collaboration
- Developer specific job lists
- Aggregation of UK developer communities and activities in a historical progression timeline
- Critical reviews of technology, software and tools including an analysis of trends
- Consolidated directory of events and courses available to developers
- Register/directory of developers and their skills
- Regional chapters and theme related interest groups
- Clearer explanation to managers of the benefit of the events
- A “geek week‟ type idea to bring together the developer community for different events to be held in the same location and week (including getting software providers on board)
- Economic evaluation of the value of the developer community
Stakeholder specific sections
In addition to the more general questions which were aimed at all stakeholders, there were some questions which were specific to particular stakeholder groups. These are examined below.
Senior managers: Outsourcing
Senior managers were asked whether or not they have considered outsourcing any of their software development work, and if so what their reasons were for considering outsourcing and what the outcome had been if they had outsourced.
Of the 82 senior managers who completed this question, 40 (48.8%) had considered outsourcing. The main reasons outsourcing were considered (ranked in order of frequency from the responses) were:
- Lack of relevant skills in-house
- Lack of resources in-house (staff/time)
- Cost savings
- Software development work not core to their business (therefore not worth investing in internally)
- Recruitment of software developers is too costly and unnecessary (e.g. only needing short-term)
- To share development costs with other organisations
Of those 40 that had outsourced software development work, 12 had a mainly positive outcome, 3 a mainly negative outcome, and 25 a mixed outcome.
Below are samples of comments on the outcome of outsourcing software development:
“Entirely beneficial experience. I have a wide network of contacts and thus didn’t have any problems finding suitable contractors”
“Results of outsourcing were poor. It cost more and the results were not well implemented or did not meet the requirement as well as systems developed in house with closer liaison throughout the project”
“Projects delivered on or ahead of time and on budget”
“Difficult to brief an external company on all details as no one person within the uni is in a position to know all the details”
“The advantage is that you know the costs up front and if you manage thing properly, you get the result you expected. The disadvantage is that you can be locked into needing support from the supplier long term”
Vendors/Suppliers: APIs and opening source code
The DevCSI project is also interested in encouraging links with other relevant stakeholders such as vendors and suppliers. Two vendors were interviewed and four responded to the survey. With low number it is difficult to generalise, however some interesting impressions could be gained from the responses.
Three of the vendors had worked with developers within HE, whilst the other three did all their testing and development in-house. Of those that have worked with HE developers, work includes collaborative development of products, utilising APIs to extend products, and working to integrate institutional systems.
Involvement in developer challenges at developer events supports testing:
“We had a sponsored competition at Dev8D last year to use [our software] to encourage developers to write applications and play with it”
Some vendors seem keen to encourage further development of their products through the HE community, whilst others prefer to provide a standard product and not provide APIs. There is obvious conflict with commercial vendors:
“We try to provide APIs and use open source code to share it back with the community, however this depends as sometimes there is tech transfer between our projects and the particular commercial product”
Another link with vendors is sponsorship for developer community activities, through event sponsorship or competitions. One vendor was keen to encourage more of this, through either a broad community or a more specific focused sub-sector:
“An over-reaching developer community in general would be good to get involved in (e.g. sponsorship) but it is less tangible than a sort of meta-community which is really valuable for specific feedback – there are advantages and disadvantages to both types of community”
Funders: Feeding back and taking forward
Another stakeholder group that DevCSI is particularly interested in engaging with is funders. One funder was interviewed and a further 7 completed the survey. Under investigation was the relationship and communication channels between developers and funders, i.e. funders taking software to developers to test/enhance and developers taking ideas to funders to inform future research areas.
It is evident that there are examples of this already:
“Opportunities [are] identified and explored, and then funded in a larger way”
“Developers have helped us conclude that a certain technology in not ready for further funding”
“Funding priorities are underpinned by evidence from funded research… may include software research”
“Most of the recent work has been research to test out ideas and commission developments”
“Software developers are key to most of the projects [my organisation] funds. Their role includes exploring new technical possibilities and opportunities… R&D around established platforms… and embedding and localising commodity technologies into local environments”
“Problems are bought to the event… and attendees are encouraged to try to find a suitable solution… community knowledge is utilised to help solve these problems:
However, it is suggested that there is still progress to be made on building these links:
“Still work to be done to ensure communication is two-way (at the moment it is mainly from us to developers and this could be improved on)”
Recent Comments