Relationship between quality and payment in crowdsourced design

Wu, Hao and Corney, Jonathan and Grant, Michael; Hou, Jiang-Liang, ed. (2014) Relationship between quality and payment in crowdsourced design. In: Proceedings of the 2014 IEEE 18th International Conference on Computer Supported Cooperative Work in Design (CSCWD). IEEE, Piscataway, NJ., pp. 499-504. ISBN 9781479937769 (https://doi.org/10.1109/CSCWD.2014.6846895)

Full text not available in this repository.Request a copy

Abstract

In recent years, the “power of the crowd” has been repeatedly demonstrated and various Internet platforms have been used to applied collaborative intelligence to areas that range from open innovation to image analysis. However, crowdsourcing applications in the fields of design research and creative innovation have been much slower to emerge. So, although there have been reports of systems and researchers using Internet crowdsourcing to carry out generative design, there are still many gaps in knowledge about the capability and limitations of the technology. For example on crowdsourcing platforms, like Amazon’s Mechanical Turk, the relationship between remuneration and the final quality of designs has not been established, so it is unclear how much payment should be offered to ensure a particular standard of result. To investigate the relationship between crowd’s remuneration and the quality of their innovation, this paper reports how payment for a 2D interior design task (living room layout) was systematically varied and the quality of the output assessed by a ranking process designed that was also crowdsourced. Information about individual Mturk workers who participated in the study was also collected. The results suggest that while that average design quality only slowly increases, the quality of the “best” design generated by the crowd improved dramatically with payment levels. In other words, increasing monetary rewards does not improve the average creativity of design but rather it increases the chance of an excellent solution being generated by an individual in the crowd.

ORCID iDs

Wu, Hao ORCID logoORCID: https://orcid.org/0000-0002-4988-3444, Corney, Jonathan ORCID logoORCID: https://orcid.org/0000-0003-1210-3827 and Grant, Michael ORCID logoORCID: https://orcid.org/0000-0002-3337-4653; Hou, Jiang-Liang
CORE (COnnecting REpositories)