Full Text:   <1981>

Summary:  <1580>

CLC number: TP391

On-line Access: 2018-03-10

Received: 2017-12-04

Revision Accepted: 2018-01-20

Crosschecked: 2018-01-25

Cited: 0

Clicked: 6007

Citations:  Bibtex RefMan EndNote GB/T7714

-   Go to

Article info.
Open peer comments

Frontiers of Information Technology & Electronic Engineering  2018 Vol.19 No.1 P.126-138


Crowdsourcing intelligent design

Author(s):  Wei Xiang, Ling-yun Sun, Wei-tao You, Chang-yuan Yang

Affiliation(s):  State Key Lab of CAD & CG, Zhejiang University, Hangzhou 310058, China; more

Corresponding email(s):   wxiang@zju.edu.cn, sunly@zju.edu.cn, weitao_you@zju.edu.cn, changyuan.yangcy@alibaba-inc.com

Key Words:  Crowdsourcing, Flexible crowdsourcing design, Design intelligence

Wei Xiang, Ling-yun Sun, Wei-tao You, Chang-yuan Yang. Crowdsourcing intelligent design[J]. Frontiers of Information Technology & Electronic Engineering, 2018, 19(1): 126-138.

@article{title="Crowdsourcing intelligent design",
author="Wei Xiang, Ling-yun Sun, Wei-tao You, Chang-yuan Yang",
journal="Frontiers of Information Technology & Electronic Engineering",
publisher="Zhejiang University Press & Springer",

%0 Journal Article
%T Crowdsourcing intelligent design
%A Wei Xiang
%A Ling-yun Sun
%A Wei-tao You
%A Chang-yuan Yang
%J Frontiers of Information Technology & Electronic Engineering
%V 19
%N 1
%P 126-138
%@ 2095-9184
%D 2018
%I Zhejiang University Press & Springer
%DOI 10.1631/FITEE.1700810

T1 - Crowdsourcing intelligent design
A1 - Wei Xiang
A1 - Ling-yun Sun
A1 - Wei-tao You
A1 - Chang-yuan Yang
J0 - Frontiers of Information Technology & Electronic Engineering
VL - 19
IS - 1
SP - 126
EP - 138
%@ 2095-9184
Y1 - 2018
PB - Zhejiang University Press & Springer
ER -
DOI - 10.1631/FITEE.1700810

design intelligence, namely, artificial intelligence to solve creative problems and produce creative ideas, has improved rapidly with the new generation artificial intelligence. However, existing methods are more skillful in learning from data and have limitations in creating original ideas different from the training data. crowdsourcing offers a promising method to produce creative designs by combining human inspiration and machines’ computational ability. We propose a crowdsourcing intelligent design method called ‘flexible crowdsourcing design’. Design ideas produced through crowdsourcing design can be unreliable and inconsistent because they rely solely on selection among participants’ submissions of ideas. In contrast, the flexible crowdsourcing design method employs a cultivation procedure that integrates the ideas from crowd participants and cultivates these ideas to improve design quality at the same time. We introduce a series of studies to show how flexible crowdsourcing design can produce original design ideas consistently. Specifically, we will describe the typical procedure of flexible crowdsourcing design, the refined crowdsourcing tasks, the factors that affect the idea development process, the method for calculating idea development potential, and two applications of the flexible crowdsourcing design method. Finally, it summarizes the design capabilities enabled by crowdsourcing intelligent design. This method enhances the performance of crowdsourcing design and supports the development of design intelligence.




Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article


[1]Ball LJ, Ormerod TC, 1995. Structured and opportunistic processing in design: a critical discussion. Int J Hum-Comput Stud, 43(1):131-151.

[2]Chan J, Dang S, Dow SP, 2016. Improving crowd innovation with expert facilitation. Proc 19th ACM Conf on Computer-Supported Cooperative Work & Social Computing, p.1223-1235.

[3]Chang DN, Chen CH, Lee KM, 2014. A crowdsourcing development approach based on a neuro-fuzzy network for creating innovative product concepts. Neuro-computing, 142:60-72.

[4]Cross N, 2006. Designerly Ways of Knowing. Springer, London.

[5]Dontcheva M, Morris RR, Brandt JR, et al., 2014. Combining crowdsourcing and learning to improve engagement and performance. Proc SIGCHI Conf on Human Factors in Computing Systems, p.3379-3388.

[6]Flores RL, Belaud JP, le Lann JM, et al., 2015. Using the collective intelligence for inventive problem solving: a contribution for open computer aided innovation. Expert Syst Appl, 42(23):9340-9352.

[7]Gatys LA, Ecker AS, Bethge M, 2016. Image style transfer using convolutional neural networks. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.2414-2423.

[8]Glickman ME, 1999. Parameter estimation in large dynamic paired comparison experiments. J Roy Stat Soc Ser C, 48(3):377-394.

[9]Goldschmidt G, 2015. Ubiquitous serendipity: potential visual design stimuli are everywhere. In: Gero JS (Ed), Studying Visual and Spatial Reasoning for Design Creativity. Springer Dordrecht Netherlands, p.205-214.

[10]Ikeda K, Morishima A, Rahman H, et al., 2016. Collaborative crowdsourcing with crowd4U. Proc VLDB Endowm, 9(13):1497-1500.

[11]Kim J, Dontcheva M, Li W, et al., 2015. Motif: supporting novice creativity through expert patterns. Proc 33rd Annual ACM Conf on Human Factors in Computing Systems, p.1211-1220.

[12]Lafreniere B, Grossman T, Anderson F, et al., 2016. Crowdsourced fabrication. Proc 29th Annual Symp on User Interface Software and Technology, p.15-28.

[13]Li W, Wu WJ, Wang HM, et al., 2017. Crowd intelligence in AI 2.0 era. Front Inform Technol Electron Eng, 18(1): 15-43.

[14]Michelucci P, Dickinson JL, 2016. The power of crowds. Science, 351(6268):32-33.

[15]O’Donovan P, Agarwala A, Hertzmann A, 2014. Learning layouts for single-pagegraphic designs. IEEE Trans Vis Comput Graph, 20(8):1200-1213.

[16]Pan YH, 2017. Special issue on artificial intelligence 2.0. Front Inform Technol Electron Eng, 18(1):1-2.

[17]Park CH, Son KH, Lee JH, et al., 2013. Crowd vs. crowd: large-scale cooperative design through open team competition. Proc Conf on Computer Supported Cooperative Work, p.1275-1284.

[18]Pauwels P, de Meyer R, van Campenhout J, 2013. Design thinking support: information systems versus reasoning. Des Iss, 29(2):42-59.

[19]Pinel F, Varshney LR, Bhattacharjya D, 2015. A culinary computational creativity system. In: Besold TR, Schorlemmer M, Smaill A (Eds), Computational Creativity Research: Towards Creative Machines. Springer, Paris, p.327-346.

[20]Prats M, Earl CF, 2006. Exploration through drawings in the conceptual stage of product design. In: Gero JS (Ed), Design Computing and Cognition. Springer Dordrecht Netherlands, p.83-102.

[21]Ren J, Nickerson JV, Mason W, et al., 2014. Increasing the crowd’s capacity to create: how alternative generation affects the diversity, relevance and effectiveness of generated ads. Dec Supp Syst, 65:28-39.

[22]Schneider OS, Seifi H, Kashani S, et al., 2016. HapTurk: crowdsourcing affective ratings of vibrotactile icons. Proc CHI Conf on Human Factors in Computing Systems, p.3248-3260.

[23]Sun LY, Xiang W, Chai CL, et al., 2014a. Creative segment: a descriptive theory applied to computer-aided sketching. Des Stud, 35(1):54-79.

[24]Sun LY, Xiang W, Chai CL, et al., 2014b. Designers’ perception during sketching: an examination of creative segment theory using eye movements. Des Stud, 35(6): 593-613.

[25]Sun LY, Xiang W, Chen S, et al., 2015. Collaborative sketching in crowdsourcing design: a new method for idea generation. Int J Technol Des Educat, 25(3):409-427.

[26]Suzuki R, Salehi N, Lam MS, et al., 2016. Atelier: repurposing expert crowdsourcing tasks as micro-internships. Proc CHI Conf on Human Factors in Computing Systems, p.2645-2656.

[27]van der Maaten L, Weinberger K, 2012. Stochastic triplet embedding. IEEE Int Workshop on Machine Learning for Signal Processing, p.1-6.

[28]Wah C, van Horn G, Branson S, et al., 2014. Similarity comparisons for interactive fine-grained categorization. IEEE Conf on Computer Vision and Pattern Recognition, p.859-866.

[29]Warby SC, Wendt SL, Welinder P, et al., 2014. Sleep-spindle detection: crowdsourcing and evaluating performance of experts, non-experts and automated methods. Nat Methods, 11(4):385-392.

[30]Wiltschnig S, Christensen BT, Ball LJ, 2013. Collaborative problem–solution co-evolution in creative design. Des Stud, 34(5):515-542.

[31]Xiang W, Sun LY, Xia SC, et al., 2017. An evolutionary computation method of crowdsourcing ideation that integrates the balanced-exploration pattern. J Mech Eng, 53(15):73-80 (in Chinese).

[32]Xu AB, Rao HM, Dow SP, et al., 2015. A classroom study of using crowd feedback in the iterative design process. Proc 18th ACM Conf on Computer Supported Cooperative Work & Social Computing, p.1637-1648.

[33]Yu LX, Nickerson JV, 2011. Cooks or cobblers?: crowd creativity through combination. Proc SIGCHI Conf on Human Factors in Computing Systems, p.1393-1402.

[34]Yu LX, Kittur A, Kraut RE, 2014. Distributed analogical idea generation: inventing with crowds. Proc SIGCHI Conf on Human Factors in Computing Systems, p.1245-1254.

[35]Yu LX, Kraut RE, Kittur A, 2016. Distributed analogical idea generation with multiple constraints. Proc 19th ACM Conf on Computer-Supported Cooperative Work & Social Computing, p.1236-1245.

[36]Zhao Q, Huang ZH, Harper FM, et al., 2016. Precision crowdsourcing: closing the loop to turn information consumers into information contributors. Proc 19th ACM Conf on Computer-Supported Cooperative Work & Social Computing, p.1615-1625.

[37]Zhu JY, Krähenbühl P, Shechtman E, et al., 2016. Generative visual manipulation on the natural image manifold. Proc 14th European Conf on Computer Vision, p.597-613.

Open peer comments: Debate/Discuss/Question/Opinion


Please provide your name, email address and a comment

Journal of Zhejiang University-SCIENCE, 38 Zheda Road, Hangzhou 310027, China
Tel: +86-571-87952783; E-mail: cjzhang@zju.edu.cn
Copyright © 2000 - 2024 Journal of Zhejiang University-SCIENCE