Invited Seminar at Open University, 3 Oct 2013

Two's Company, Three's a Crowd: A Case Study of Crowdsourcing Software Development

Crowdsourcing is an emerging and promising approach which

involves delegating a variety of tasks to an unknown workforce i.e.

the crowd. Crowdsourcing has been applied quite successfully in

various contexts from basic tasks on Amazon Mechanical Turk to

solving complex industry problems, e.g. Innocentive. Companies

are increasingly using crowdsourcing to accomplish specific software

development tasks. However, very little research exists on

this specific topic. Our case study highlights a number of challenges

that arise when crowdsourcing software development. For

example, the crowdsourcing development process is essentially a

waterfall model and this must eventually be integrated with the

agile approach used by the company. Crowdsourcing works better

for specific software development tasks that are less complex and

stand-alone without interdependencies. The development cost was

much greater than originally expected, overhead in terms of company

effort to prepare specifications and answer crowdsourcing

community queries was much greater, and the time-scale to complete

contests, review submissions and resolve quality issues was

significant. Finally, quality issues were pushed later in the lifecycle

given the lengthy process necessary to identify quality issues

and have them resolved by the community. Given the emphasis in

software engineering on identifying bugs as early as possible, this

is quite problematic.