Human–Computer Interaction and Participation in Software Crowdsourcing

Abstract

Improvements in communication and networking technologies have transformed people’s lives and organizations’ activities. Web 2.0 innovation has provided a variety of hybridized applications and tools that have changed enterprises’ functional and communication processes. People use numerous platforms to broaden their social contacts, select items, execute duties, and learn new things. Context: Crowdsourcing is an internet-enabled problem-solving strategy that utilizes human–computer interaction to leverage the expertise of people to achieve business goals. In crowdsourcing approaches, three main entities work in collaboration to solve various problems. These entities are requestors (job providers), platforms, and online users. Tasks are announced by requestors on crowdsourcing platforms, and online users, after passing initial screening, are allowed to work on these tasks. Crowds participate to achieve various rewards. Motivation: Crowdsourcing is gaining importance as an alternate outsourcing approach in the software engineering industry. Crowdsourcing application development involves complicated tasks that vary considerably from the micro-tasks available on platforms such as Amazon Mechanical Turk. To obtain the tangible opportunities of crowdsourcing in the realm of software development, corporations should first grasp how this technique works, what problems occur, and what factors might influence community involvement and co-creation. Online communities have become more popular recently with the rise in crowdsourcing platforms. These communities concentrate on specific problems and help people with solving and managing these problems. Objectives: We set three main goals to research crowd interaction: (1) find the appropriate characteristics of social crowd utilized for effective software crowdsourcing, (2) highlight the motivation of a crowd for virtual tasks, and (3) evaluate primary participation reasons by assessing various crowds using Fuzzy AHP and TOPSIS method. Conclusion: We developed a decision support system to examine the appropriate reasons of crowd participation in crowdsourcing. Rewards and employments were evaluated as the primary motives of crowds for accomplishing tasks on crowdsourcing platforms, knowledge sharing was evaluated as the third reason, ranking was the fourth, competency was the fifth, socialization was sixth, and source of inspiration was the seventh.

1. Introduction

Individuals contribute their time, expertise, and wealth to help the needy and transform the Earth into a better living place [1]. Technologies like social networking and Web 2.0 are making health and medical care more accessible to businesses, professionals, patients, and laypeople. The innovative tools and applications made publicly available by Web 2.0 innovation have changed how organizations operate and communicate. People utilize a variety of platforms to expand their social networks, purchase items, complete activities, and learn new things. Information retrieval, blogging, tagging, path-finding, text messaging, collaborative online services, and multi-player gaming are some of the activities that are carried by Web 2.0 applications [2]. Individuals may now interact and collaborate more easily because of advances in technology, and this engagement of people is referred as “crowdsourcing” [3]. Crowdsourcing is a task-solving methodology in which human participation is required to solve difficult tasks [4]. Jeff Howe first used the term “crowdsourcing” in Wired magazine. Crowdsourcing, according to him, is “the act of taking a job traditionally performed by a designated agent (usually an employee) and outsourcing it to an undefined, generally large group of people in the form of an open call” [5].

Crowdsourcing is a popular strategy for accomplishing a variety of tasks. Several individuals and their interactions are involved in the process, such as requesters and crowdsourcers [6] that manage, execute, and supervise crowdsourcing initiatives and may post task requests, the crowd (individuals) [7], consisting of virtual employees who participate in outsourcing activities or events, and the platform [8], which serves as a channel for interaction between the crowd and the crowdsourcers. The people on these platforms are connected by means of social media such as Facebook, WhatsApp, Instagram, Twitter, Y-mail, and Gmail accounts. Social networking improvements have encouraged corporations to collect information from individuals all over the world in order to identify the best solutions to unique challenges [9,10]. Crowdsourcing allows enterprises to hire globalized, low-cost, and talented workers through internet platforms [11,12].Crowdsourcing is employed for a wide range of tasks including spelling correction, content creation, coding, pattern recognition, software development, and debugging [13]. The task is advertised on a platform, and the crowd participates in various types of activities [14]. Considering the user interface and their intrinsic motivation, crowdsourcing platforms utilize appealing concepts that promote human–computer interaction in the context of open innovation [15]. In some situations, computers may be used to manage crowdsourced tasks, resulting in human-based computation systems. This type of human-based computing is incorporated in many online systems (crowdsourcing platforms) [16]. Crowdsourcing platforms are websites that work as the interface between job seekers and online people. Both entities are registered on these crowdsourcing platforms. The platform uses various selection methods such as skill testing, profiling, previous participating history, matching, and many others to select an appropriate participant to accomplish requestor tasks [17].

In crowdsourcing, participants from diverse backgrounds have skills, knowledge, abilities, and some expertise in task domains, and they collaborate to tackle various challenges [3,6]. “Crowdsourced Software Engineering is the act of undertaking any external software engineering tasks by an undefined, potentially large group of online workers in an open call format” [18]. Crowdsourcing allows a requestor to tap into a global community of users with various types of expertise and background to facilitate the completion of a task that would be difficult to complete without a large group of individuals [19]. Crowdsourcing has also been utilized in software engineering to resolve coding, validation, and architectural problems. However, most crowdsourcing approaches for engineering are for theoretical concepts and are often implemented and assessed on a comparatively small crowd with a maximum of 20 participants. Considering the nature of the crowdsourcing sector and the cognitive features of participants, there is a great demand for an integrated resource-sharing crowdsourcing environment for real-world solutions [20].

Modern society depends on sophisticated hardware and software systems, many of which are safety-critical, such health monitoring software, which is utilized by medical organizations to detect, monitor, and aid elders and patients. Innovations in mobile computing might change how health interventions are delivered. In order to overcome limitations brought on by a scarcity of clinician timing, poor patient engagement, and the challenge of ensuring appropriate treatments at the correct time, mobile health (mHealth) treatment may be more effective than occasional in-clinic consultations. The delivery of healthcare is being revolutionized by technology that collects, analyzes, and configures patient data across devices. Intelligent and interconnected healthcare will deliver benefits that are safe, more user-centered, economical, effective, and impartial due to advancements in ubiquitous computing.

Crowdsourcing is being utilized more often as it offers the opportunity to mobilize a wide and heterogeneous group through improved communication and collaboration. In the areas of research, crowdsourcing R&D has been successful. Due to “crowdsourcing’s elasticity and mobility”, it is excellent for carrying out research tasks such data processing, surveying, monitoring, and evaluation. By including the public as innovation partners, projects may be improved in terms of quality, cost, and speed while also generating solutions to important research problems [21].

The key contributions of this article are as follows:

• Analysis of the appropriate characteristics of social crowds utilized for effective software crowdsourcing.
• Analysis of the participation reasons of people in carrying out software developmental tasks.
• Development of a decision support system for evaluating primary participation reasons by assessing various crowds using Fuzzy AHP and TOPSIS techniques.

The paper is organized into five sections. Section 2 describes the existing literature on crowdsourcing, Section 3 describes the overall methodology of our study, Section 4, “Experimental setup and results”, provides the description and evaluation of the proposed method, and Section 5, “conclusion”, concludes and summarizes the objectives achieved in this study.

2. Literature Review

The Internet has made it possible for organizations to attract a significant number of individuals. Cell phones, computers, tablets, and smart gadgets are all ways for crowd and requestor organizations to engage people to carry out various crowdsourcing tasks (health monitoring, question and answering, problem solution, recommendations, etc.). Crowds are employed from around the world to undertake various jobs. As the tasks are completed by groups of individuals, quality outcomes may be obtained in less time and with less expenditure [22]. Organizations issue an open call for employees who will satisfy specific standards, along with guidelines if the organization receives a reply from competent workers to carry out the assignment within the specific timelines. The organization provides a confirmation to that individual [23]. Crowds are constituted of a group of skilled people who possess some expertise [7,24]. Organizations recruit people who can provide numerous and diverse suggestions for fixing technical concerns [25]. The company pre-assesses the individual’s capacity to engage in complicated activities [26].

Participants are selected based on their backgrounds. Demographic filtering is used to pick persons from relevant countries/localities. If a worker is ready to begin a task, they must supply demographic information. The audience is drawn from a variety of sources and backgrounds. Not every crowd is suitable for every activity, and different activities need different levels of expertise, field knowledge, and so on. Inaccurate workers can reduce job productivity and increase recruitment costs. Choosing the right employee is a difficult task. Workers are recruited with three goals in mind: maximize test specification scope, maximize recruited worker competence in bug identification, and decrease costs. Workers are classified into five belts based on their enrollment—red, green, yellow, blue, and grey—that indicate their skill levels. Worker reliability is evaluated by qualifying and completing the assigned work [27].

To choose appropriate personnel for the sensing task, many task assignment methodologies are used. Mobile crowd-sensing is a technology that enables a group of people to interact and gather data from devices with sensing and processing capabilities in order to measure and visualize phenomena that are of interest to all. Using smartphones, data can be gathered in everyday life and easily compared to other users of the crowd, especially when taking environmental factors or sensor data into account as well. In the context of chronic diseases, mobile technology can particularly help to empower patients in properly coping with their individual health situations. Employee selection in Mobile Crowd Sensing (MCS) is a difficult problem that has an impact on sensing efficiency and quality. Different standards are used to screen for appropriate employees. Participants in the job scheduling system employ sensors to gather or evaluate details about their actual subject [11].

Many software firms are knowledge-intensive; therefore, knowledge management is critical. The design and execution of software systems need information that is frequently spread across many personnel with diverse areas of experience and capabilities [28]. Software engineering is increasingly taking place in companies and communities involving large numbers of individuals, rather than in limited, isolated groups of developers [29]. The popularity of social media has created new methods of distributing knowledge over websites. We live in an environment dominated by social media and user-generated information. Many social relationships, from pleasure to learning and employment, are the result of people engaging actively with one another. It is hardly surprising that social sites have modified the spread of knowledge [30].

Social media is essential for organizations of all sizes because people must engage to perform tasks. As technology develops, it becomes more user-friendly and incorporates a variety of features, such as an operating system based on social factors, software applications primarily geared toward communication, and a medium of engagement through social networks, which are becoming more and more significant. Social media have altered how individuals engage with and share their perspectives on state policies. Four key objectives are achieved by organizations using social media: engaging with citizens, promoting citizen involvement, advancing open government, and analyzing/monitoring public sentiment and activity [31].

Human–computer interaction (HCI) research and practice are based on the principle of human-centered design. The goal of human-centered design is to develop new technologies that are geared toward the requirements and activities of the users. This design philosophy ensures that user needs are considered throughout the whole development process of a technology, from obtaining required information through its final stages. Crowdsourcing utilizes gamification-based strategy for solving larger problems. The practice of adding features that provide game-like representation is known as gamification. The method seeks to boost overall problem-solving strategy by eliciting users’ intrinsic drives through the development of systems that are similar to game interfaces. The principles and characteristics of games may be utilized to attract and engage users, lowering anticipated constraints to system usage, such as low motivation and low acceptance rates, and transforming game-based activities into successful outcomes [32].

2.1. Stack Overflow

Social media has become an essential source of information for a diverse range of fields as a means to gain a broader knowledge of information processes and community groups [33]. To aid with application creation, the internet provides a plethora of built-in libraries and tools. Developers typically use pre-existing mobile APIs to save time and money. An Application Programming Interface is a group of elements that provide a mechanism for software-to-software interaction. Stack Overflow (SO) is a renowned question-and-answer platform for software developers, engineers, and beginners. The Stack Overflow technique provides a distributed skill set that enables clients worldwide to improve and broaden their knowledge in coding and their communication capabilities [34]. SO enables a user (literally, the problem presenter or question submitter) to initiate a conversation (question), provide an answer, make discussion, rate questions, and accept responses that they believe are beneficial [35].

Choosing the right coding language is usually an important phase in the software development process. Technical factors concerning the coding language’s abilities and flaws in resolving the topic of concern naturally drive this decision. The recent emergence of social networks concerned with technical difficulties has brought professionals into conversations regarding programming languages wherein rigorously technical difficulties usually compete with strongly articulated viewpoints [36]. Practitioners and scholars throughout software engineering are continually focusing on the challenges associated with mobile software development [37]. When coworkers are drawn from various cultures, tackling cultural barriers in software engineering is essential for ensuring appropriate group performance, and the necessity of managing such difficulties has expanded with activities that are reinforced in software development [38].

Some famous platforms that assign workers to requested tasks are mentioned in subsequent sections. Platforms and their working procedures are also briefly highlighted in [39], and their working procedures are also explained.

2.2. Amazon Mechanical Turk

Amazon Mechanical Turk is a crowdsourced recruiting platform that enables requesters to offer online projects in exchange for completion rewards to online people who have the required skills, without the limitations of permanent employment [40,41]. Requesters seeking access to a large pool of individuals have minimal entrance requirements for executing human intelligence task (HITs). The tasks are assigned by requesters and completed by individuals. The HITs vary in complexity from recognizing an image to performing domain-specific tasks such as interpreting source code [42]. Requesters might provide qualifying conditions, such as gender, age, and geography requirements [43]. MTurk is a platform for seeking jobs that may require human intelligence. By displaying Human Intelligence Tasks, employees may examine and decide to complete specific tasks [44]. Workers sign up for tasks on the platform and afterwards according to their credentials and compete on micro projects known as HITs that are advertised by requester organizations that require the accomplishment of such tasks. The MTurk workforce is mostly constituted of individuals from various parts of the globe [45].

2.3. Upwork

Upwork is a platform that connects people selling work with potential employees. Organizations may publish a variety of jobs on Upwork for potential workers to bid on. Accounts are created on Upwork by both job hunters and job posters in order to browse or post jobs and to use the services and functionalities that Upwork delivers. Jobs on Upwork often entail client commitments that last from hours to weeks and necessitate more dynamic interaction than those advertised on micro-tasking websites such as AMT. Jobs advertised on Upwork generally demand a higher degree of implicit understanding and, as a result, a new strategy for planning and coordinating activities between providers and users that goes beyond simple automated monitoring [46]. Upwork enables skilled employees to perform knowledge work ranging from web design to strategic decisions [47].

2.4. Freelancer

Freelancers are self-employed individuals who have a short-term, task-based relationship with employers and hence are not part of the firm workforce. Their relationship with the firm lasts just until the assigned task is finished satisfactorily; therefore, they do not have the long-term obligations with the corporation that full-time workers have. In exchange for payment, they are obligated to finish the assignment with high quality standards by the agreed-upon day. During the period that the freelancer is carrying out the assigned activity, they are permitted to take on additional freelancing projects with various requesters under different contractual circumstances. In other words, freelancers may work on many projects concurrently. As opposed to full-time employees, freelancers are not bound by restrictive and long contractual work [48]. Engagement of freelancers on these websites enables them to pool their collective intellect in order to execute an assignment in a creative and cost-effective way. The lower processing cost makes freelancers a desirable option for completing tasks efficiently and effectively [49]. FaaT (Freelance as a Team) is an approach for professionals to optimize their internal procedures to fulfill the requirements and capabilities of a single programmer [50].

2.5. Top Coder

Many prominent application developers use online communities to enhance the products or solutions they deliver. TopCoder is a digital platform of over 430,000 creative professionals that contest to build and improve software, websites, and mobile applications for subscribers. TopCoder was a pioneer of technology innovation and allows developers and producers from across the world to choose and solve the various problems and difficulties to which they wish to make contributions. TopCoder offers functionality and technology to coordinate and ease the advancement of solution and implementation [51]. Every TopCoder application passes through the following phases: application design, architecture, development, assembly, and delivery.

Each step is advertised as a contest on the TopCoder site. Registered platform users can enter any contest and submit the appropriate solutions. The preceding phase’s successful answer is used as input for the following step. The needs of businesses are gathered and specified during the application specification process. Following that, each application is separated into a collection of components in the architecture stage. Following that, each part passes through the design and development phases. The components are then joined together during the assembly process to develop the application, which is eventually deployed and delivered to corporations [52].

Overall, the registration of social people on crowdsourcing platforms to participate on different tasks is on the rise.

3. Methodology

As the value-creation process of organization has transformed from being centralized to decentralized and from being closed to being open, various operational constraints are currently disappearing with advancements in technology. Due to internal resource constraints and external competitiveness, corporations are searching for crowdsourcing initiatives to direct crowdsourcing towards innovative products and services. Crowd involvement is a key concern in crowdsourcing systems, since it has been found to be essential to the variety and success of crowdsourcing competitions. Crowdsourcers announce tasks on crowdsourcing platforms where virtual crowds are present, and many people believe that the quantity of participants is a good proxy for contest quality when determining the value that can be derived from participating in the contest. Crowdsourcing is one of the numerous digital economy sectors that have emerged as a result of the expansion of internet connectivity and cellular technologies. By linking requesters and employees from all over the world together in a public setting, crowdsourcing tackles the problem of individual employment and advances societal wellbeing [53]. Social and psychological aspects have a significant role in determining how people work, their involvement in their jobs, their well-being, and the sustainability of their employment [54]. The expectancy theory postulates that the task’s accomplishment influences the individual’s decision by transforming his or her mental representation, particularly by their perceptual expectation and valence. The most impact on perceived expectation derives from comments concerning task completion, whereas the major influence on perceived valence comes from narratives about rewards. It has also been observed that task accomplishment and compensation are commonly focused when crowdsourcing organizations post tasks for the general public.

Our research focused on three goals. The first two goals were achieved from a review of the literature, and the third goal was achieved by using a decision support system. The goals are discussed in the Methodology section, and the results are provided in Section 4.

3.1. Analysis of the Appropriate Characteristics of Social Crowds Utilized for Effective Software Crowdsourcing

Corporations are interested in harnessing and learning from individuals. Superior strategies are used to acquire this expertise from outside specialists in order to enhance the effectiveness of diverse processes [25]. Workers are typically divided into two categories: trustworthy and untrustworthy. Trustworthy employees accomplish their jobs honestly; hence, trustworthiness is a favorable characteristic of the populace. Untrustworthy employees are solely concerned in the incentives linked with duties; thus, they do not truly work and are a destructive presence in a crowd [55]. Workers can solve enormous challenges. These challenges need innovation, practical wisdom, and prior knowledge. Non-experts can also do jobs involving geo-referenced data, maps, and atlases. Workers can be classified in accordance with their contributions and previous participation history, and they are chosen exclusively on these categories [56]. Crowd workers are either software engineers with programming abilities or software testers who provide multiple analytical services to help the software development process [57].

The crowd consists of qualified professionals with varied talents (java, Photoshop, accountancy, etc.) [58]. Appropriately diversified crowds engage in the creation of numerous initiatives. Domain specialists are chosen from crowds of inside and outside developer groups in a collaborative setting. Crowd employees are able to work on coding, architectural implementations, unit tests, and debugging [8]. Crowd developers collaborate, exchange, and cooperate with one another to make the software development process more productive. The audience consists of smartphone users who place bids on social site [59]. Through an open call, developers are invited to engage in various developmental stages of software life cycle [60]. Individuals have diverse competences and the capacity to coordinate different tasks, adapt to changes in the workplace, and create their own designs [61]. A corporation picks a group of employees with diverse knowledge, numbers, and heterogeneity.

Complex activities are carried out by people with specialized abilities, such as software developers and engineers. The entire software package or a portion of it is outsourced to a vast pool of possible developers. Professionals work on these projects or components in order to deliver a solution. Workers in crowd-sourced software development collaborate and manage time to build high-quality software. The crowd understands the objective of the activity and collaborates in English [62]. Experienced software testers provide high-quality products [63]. The crowd is recruited from a pool of qualified testing professionals and is employed to perform operational tests. Experts and software engineers are employed for development, testing, and evaluating the results [64]. Some individuals in the pool have defined competence and a high degree of experience to give a response to various tasks [65,66]. The overall crowdsourcing phenomenon is represented in Figure 1.

3.2. Analysis of the Participating Reasons of People in Carrying out Software Developmental Task

When analyzing any type of user community, it is crucial to consider the potential effects that the recruiting strategies and distribution channels may have on the research outcome. It is well-understood that different user demographics have different expectations, preferences, and cultural backgrounds [67]. Crowdsourcing participants may be divided into paid and unpaid crowd labor, depending on the compensation obtained by employees. The term “paid work” refers to crowdsourcing jobs for which participants are compensated financially, generally using a platform that streamlines payments. Crowdwork, Crowd4U, Wikipedia, Test My Brain, Moral Machine, and Zooniverse are platforms on which users participate without greed or expecting anything in return [68]. Crowd workers on other platforms such as fiver and AMT perform and complete various tasks as they have expectations in return. These expectations may include a reward that is associated with task completion, or may involve various other material or non-material things [6].

3.2.1. Gaining Various Types of Rewards

People perform diverse activities (such as development, designing, debugging) in order to get rewarded [57]. Crowds are compensated according to the quality of their work. Employees that create high-quality results are rewarded more. The major motivations for crowd involvement are monetary rewards. Monetary awards may play an important part in engaging and motivating employees, which can lead to considerable returns on the organizational level [69,70]. These benefits might be cash rewards or reimbursement. Rapid payoffs, as well as earning extra money as a result of perks, encourage the crowd to participate. Non-monetary incentives are an excellent way to boost engagement and participation. Players complete numerous tasks in order to earn reputation, credit scores, status enhancement [8], and compensation [71] from job seekers.

3.2.2. Ranking

Several incentive approaches are used to encourage public participation, just as they are used to motivate internal working groups in corporations. Employees are granted various points based on their degree of work engagement, and these scores are converted into numerous presents and awards [25]. Developers join in order to improve their ranking, self-development, and to safeguard the authorship of their unique work [11,63].

3.2.3. Employment Purposes

Employees from the outside world engage in various tasks for professional progress and employment prospects [14,72].

3.2.4. Enhancing or Sharing Knowledge

Crowd involvement is primarily motivated by the desire to obtain information, improve understanding, and education. Participation is important for skill development. The crowd contributes to numerous crowdsourcing jobs in order to raise their degree of competence [73]. They may take part in order to learn or share what they have learned. Crowd participation may be used to seek a suitable response to a question [8,57,74]. The crowd may engage by suggesting and brainstorming [9] ideas [75,76] for developmental projects.

3.2.5. Socialization

Some motivators for participation include social comparison (identity with competitors), social capital (partnership), socialization (making new friends), and connectivity motivations (developing interpersonal or professional relationships) that boost the personality of the engaging crowd. The individual may also carry out various activities in order to acquire specified incentives, through which the worker increases their fame and recognition among peers. Crowd involvement may occur as a result of one’s own exposure, such as exposure, self-advertisement, and identity. Self-efficacy indicates that an individual’s efforts are valued, that they will be rewarded for participating in tasks, and that they are providing the most appropriate solutions. Crowd involvement may be a result of determined motivation to achieve personal goal achievement [77]. Crowdfunding may also be used to raise and collect finances for initiatives, while the audience wishes to contribute with others. Four forms of incentives are present: helping others, encouraging others in completing tasks, providing effective solutions, and ensuring trust [78].

3.2.6. Source of Inspiration

Crowds may be stimulated to encourage people to participate in tasks such as expectancy in which individuals work as volunteer for solving community problems as other workers will also benefit people. Altruistic commitment (acting without expectations) or pure compassion may be contributory factors [79,80]. Funding and campaigning may also be reasons for crowd participation [71].

3.2.7. Evaluating Competency Level

Individuals may engage in order to assess their own skill and aptitude. By competing in several tournaments, workers can acquire a variety of rewards. Workers may also receive feedback on their efforts after they help complete a job. The audience may engage in activities in order to meet requirements by detecting and correcting different errors [81].

3.3. Development of a Decision Support System for Evaluating Primary Participation Reasons by Assessing Various Crowds Using Fuzzy AHP and TOPSIS Techniques

The Analytic Hierarchy Process (AHP), which was invented by Saaty in the late 1970s, is one of the methods for making multi-criteria decisions. In this method, a complicated decision problem is divided into several hierarchical levels. The weight of each criterion and alternative is estimated via pairwise comparisons, and the priority is established using the eigenvector method. Fuzzy AHP is an analytical hierarchy process (AHP) that is based on fuzzy logic. The AHP technique and the fuzzy AHP approach are interchangeable. The fuzzy AHP approach merely transforms the AHP scaling into a fuzzy triangle scaling that may be obtained in a variety of ways. It is frequently used in situations with ambiguity and uncertainty, but it typically tackles concerns employing several criteria. The TOPSIS technique is useful for assessing alternatives based on their proximity to the ideal +ive and ideal -ive solutions in Euclidian space. It is a realistic way for addressing difficulties that need many decision-making procedures. The choice is made after thoroughly considering all the available options in the scenario after comparing the efficacy of several solutions in a transparent and sensible manner. In our proposed method, we used the TOPSIS algorithm to evaluate the audience based on their motivations for involvement. Details of our method are presented in Section 4.

4. Experimental Setup and Results

In order to cope with ambiguity and imprecision throughout the decision-making process, one of the main AI agents, known as fuzzy set theory, was applied to evaluate reasons for participation. This section discusses our evaluation of our proposed method.

4.1. Fuzzy AHP Approach for Finding Criterion Weightage

The weightage of criteria was calculated using a fuzzy scale, as presented in

Table 1. Fuzzy scale.

Equal	Moderate	Strong	Very Strong	Extremely Strong
1	3	5	7	9
(1,1,1)	(2,3,4)	(4,5,6)	(6,7,8)	(9,9,9)
Intermediate values
2		4	6	8
(1,2,3)		(3,4,5)	(5,6,7)	(7,8,9)

.

Our suggested strategy uses the fuzzy AHP approach to assess a crowd based on their participation motives. This method reliably assesses selected qualities and determines their percentage relevance. Seven engagement criteria were taken into consideration in the proposed study. The variables were identified by their titles, which included competency, knowledge-sharing, socialization, ranking, employment, and rewards. The list below is ordered by the procedure results and total numerical effort. The steps of the approach are as follows.

Step 1. Draw a pairwise decision matrix n*n.

$$\mathrm{C}=\left[\begin{array}{ccc}{C}_{11}& \dots & C_{1n}\\ C_{21}& \dots & C_{2n}\\ C_{31}& \dots & C_{3n}\\ C_{41}& \dots & C_{4n}\\ \dots & \dots & \dots \\ \dots & \dots & \dots \\ C_{n1}& \dots & C_{nn}\end{array}\right]$$

(1)

The decision matrix (n*n) may be created by solving the preceding matrix equation and assigning a value from 1 to 10 to each criterion, as shown in

Table 2. Pairwise decision matrix.

Criteria	Competency Purposes	Source of Inspiration	Knowledge Sharing	Socialization	Ranking	Employment	Rewards
Competency purposes	1	8	1/3	5	1/3	2	1/7
Source of Inspiration	1/8	1	1/4	2	3	1/3	1/2
Knowledge sharing	3	4	1	1/2	2	1/6	3
Socialization	1/5	1/2	2	1	1/8	2	3
Ranking	3	1/3	½	8	1	1/3	2
Employment	1/2	3	6	1/2	3	1	1/2
Rewards	7	2	1/3	1/3	½	2	1

.

Step 2. Replacing and offering fuzzy numbers to each criterion. For reciprocals, the equation is

A − 1 = (l, m, u) − 1 = (1/u, 1/m, 1/l),

(2)

where l is a lower number, m is the middle number, and u is the upper number.

Equation (2) may be used to replace specific integers with fuzzy numbers, and the resultant fuzzified matrix is shown in

Table 3. Fuzzified decision matrix.

Criteria	Competency Purposes	Source of Inspiration	Knowledge Sharing	Socialization	Ranking	Employment	Rewards
Competency purposes	(1,1,1)	(7,8,9)	(1/4,1/3,1/2)	(4,5,6)	(1/4,1/3,1/2)	(1,2,3)	(1/8,1/7,1/6)
Source of inspiration	(1/9,1/8,1/7)	(1,1,1)	(1/5,1/4,1/3)	(1,2,3)	(2,3,4)	(1/4,1/3,1/2)	(1/3,1/2,1/1)
Knowledge sharing	(2,3,4)	(3,4,5)	(1,1,1)	(1/3,1/2,1/1)	(1,2,3)	(1/7,1/6,1/5)	(2,3,4)
Socialization	(1/6,1/5,1/4)	(1/3,1/2,1/1)	(1,2,3)	(1,1,1)	(1/9,1/8,1/7)	(1,2,3)	(2,3,4)
Ranking	(2,3,4)	(1/4,1/3,1/2)	(1/3,1/2,1/1)	(7,8,9)	(1,1,1)	(1/4,1/3,1/2)	(1,2,3)
Employment	(1/3,1/2,1/1)	(2,3,4)	(5,6,7)	(1/3,1/2,1/1)	(2,3,4)	(1,1,1)	(1/3,1/2,1/1)
Rewards	(6,7,8)	(1,2,3)	(1/4,1/3,1/2)	(1/4,1/3,1/2)	(1/3,1/2,1/1)	(1,2,3)	(1,1,1)

.

Step 3. We compute the fuzzy geometric mean value (FGMV) by implementing the following equation,

FGMV = Ã₁ * Ã₂…* Ã_n = ((l₁,m₁,u₁) * (l₂,m₂,u₂) * (l₃,m₃,u₃)*…*(l_n, m_n, u_n)) = ((l₁ * l₂ * l₃*…*l_n)^1/n, (m₁ * m₂*…* m_n)^1/n, (u₁ * u₂*…*u_n)^1/n)

(3)

whereas “n” indicates the number of criteria.

The FGMV values are derived using solution (3). In

Table 4. Calculating FGMV.

Criteria	Competency Purposes	Source of Inspiration	Knowledge Sharing	Socialization	Ranking	Employment	Rewards	FGMV
Competency purposes	(1,1,1)	(7,8,9)	(1/4,1/3,1/2)	(4,5,6)	(1/4,1/3,1/2)	(1,2,3)	(1/8,1/7,1/6)	0.805,1.035, 1.314
Source of Inspiration	(1/9,1/8,1/7)	(1,1,1)	(1/5,1/4,1/3)	(1,2,3)	(2,3,4)	(1/4,1/3,1/2)	(1/3,1/2,1/1)	0.449,0.610, 0.836
Knowledge sharing	(2,3,4)	(3,4,5)	(1,1,1)	(1/3,1/2,1/1)	(1,2,3)	(1/7,1/6,1/5)	(2,3,4)	0.923,1.292, 1.739
Socialization	(1/6,1/5,1/4)	(1/3,1/2,1/1)	(1,2,3)	(1,1,1)	(1/9,1/8,1/7)	(1,2,3)	(2,3,4)	0.534,0.763, 1.037
Ranking	(2,3,4)	(1/4,1/3,1/2)	(1/3,1/2,1/1)	(7,8,9)	(1,1,1)	(1/4,1/3,1/2)	(1,2,3)	0.839,1.150, 1.601
Employment	(1/3,1/2,1/1)	(2,3,4)	(5,6,7)	(1/3,1/2,1/1)	(2,3,4)	(1,1,1)	(1/3,1/2,1/1)	0.958,1.314, 1.962
Rewards	(6,7,8)	(1,2,3)	(1/4,1/3,1/2)	(1/4,1/3,1/2)	(1/3,1/2,1/1)	(1,2,3)	(1,1,1)	0.743,1.065, 1.511

, the results of the FGMV are shown.

Step 4. For computing the fuzzy weights (W_i), the formula is as follows:

W_i = r_i * (r₁, r₂, r₃… r₁₀)⁻¹

(4)

Step 5. Defuzzification: average weights are computed by using the formula given below:

Center of Area (w_i) = l + m + u/3

(5)

Using the COA method, we obtain the average weights from fuzzy weights.

Step 6. If the overall sum of the average weightage is greater than one, convert the weights to normalized weights by applying the formula below:

$$\mathrm{Normalized} \mathrm{Weights} \left(\mathrm{Ni}\right)=\frac{wi}{{{\displaystyle \sum }}^i wi}$$

(6)

Using the afore-mentioned Equations (4)–(6), we must determine the FGMV before calculating the fuzzy weights, average weights, and normalized weights of Formula (6). The fuzzy weights are initially calculated using Formula (4). Then, we use Formula (5) to calculate the average weights. Lastly, Formula (6) is used to obtain the normalized weights of the criterion.

Table 5. Fuzzy weights along with normalized weights of criteria.

Criteria	Fuzzy Weights	Average Weights (Mi)	Normalized Weights (Ni)	Ranking
Competency purposes	0.080,0.143,0.250	0.158	0.133	5
Source of inspiration	0.045,0.084,0.159	0.096	0.081	7
Knowledge-sharing	0.092,0.178,0.330	0.200	0.169	3
Socialization	0.053,0.105,0.197	0.119	0.100	6
Ranking	0.084,0.159,0.304	0.182	0.153	4
Employment	0.096,0.181,0.373	0.217	0.182	2
Rewards	0.074,0.147,0.287	0.217	0.183	1
Total		1.188

displays the general results.

Figure 2 indicates the overall weights of the criteria (participation reasons). Here, rewards and employment are the primary motives of crowds for task accomplishment, followed by knowledge-sharing, ranking, competency purpose, socialization, and source of inspiration.

4.2. TOPSIS Technique for Evaluating and Ranking Alternatives

To overcome MCDM difficulties, Hwang and Yoon developed TOPSIS, a method for judging order performance by similarity to the ideal solution. According to the primary premise of the technique, the choice to be picked should be the one that is the furthest from the positive ideal solution and the closest to the negative one. In conventional MCDM techniques, the ratings and weights of criteria are precisely known. The traditional TOPSIS approach also uses real numbers to show the weights of the criteria and the ratings of the options. Several other fields have successfully used the TOPSIS technique. The proposed approach successfully evaluates the options and calculates their percentages. Five alternatives are proposed in the corresponding study. The alternatives are determined by their titles, which include crowd-1, crowd-2, crowd-3, crowd-4, and crowd-5. The following list is organized by the procedure’s outcomes and total numerical computation. The steps of the approach are as follows:

Step 1. Draw a decision matrix.

Develop the decision matrix by applying matrix Equation (7):

$$\mathrm{D}=[{\mathrm{D}}_{\mathrm{ij}}]=\left[\begin{array}{cccc}{\mathrm{D}}_{11}& {\mathrm{D}}_{12}& \dots & {\mathrm{D}}_{1n}\\ {\mathrm{D}}_{21}& {\mathrm{D}}_{22}& \dots & {\mathrm{D}}_{2n}\\ {\mathrm{D}}_{31}& {\mathrm{D}}_{32}& \dots & {\mathrm{D}}_{3n}\\ {\mathrm{D}}_{41}& {\mathrm{D}}_{42}& \dots & {\mathrm{D}}_{4n}\\ \dots & \dots & \dots & \dots \\ \dots & \dots & \dots & \dots \\ {\mathrm{D}}_{m1}& {\mathrm{D}}_{m2}& \dots & {\mathrm{D}}_{mn}\end{array}\right]$$

(7)

Here, i = 1,2,3,4,…,m and j = 1,2,3,4,…,m.

In the given matrix (1), D_ij displays the value of ith alternatives on the jth characteristic.

Using the crowds and criteria listed in

Table 6. Decision matrix.

Criteria	Operating Cost	Reliability	Computational Efficiency	Detection Accuracy	Quality	Numeric Robustness	Performance
Alternatives
Crowd-1	7	9	2	8	5	6	3
Crowd-2	3	2	4	5	7	3	8
Crowd-3	2	5	3	6	9	7	4
Crowd-4	6	4	5	2	8	9	7
Crowd-5	8	3	6	7	4	2	5

as a foundation, the decision matrix can be built for five crowds and provide values between 1 to 10.

Step 2. Draw normalized decision matrix (NDM)

Identify the normalized matrix by using Equation (8):

$$F_{ij}=\frac{D_{ij}}{\sqrt{{{\displaystyle \sum }}_{i=1}^n{d_{ij}}^2}}$$

(8)

Equation (8) is used to normalize the previously provided decision matrix in Table 2; the results are shown in

Table 7. NDM.

Criteria	Operating Cost	Reliability	Computational Efficiency	Detection Accuracy	Quality	Numeric Robustness	Performance
Alternatives
Crowd-1	0.550	0.775	0.211	0.600	0.326	0.448	0.235
Crowd-2	0.236	0.172	0.422	0.375	0.457	0.224	0.627
Crowd-3	0.157	0.430	0.316	0.450	0.587	0.523	0.313
Crowd-4	0.471	0.344	0.527	0.150	0.522	0.673	0.548
Crowd-5	0.629	0.258	0.632	0.525	0.261	0.149	0.392

.

Step 3. Calculate the weighted normalized decision matrix (weighted NDM); recognize the weighted NDM via Equation (9):

N = Nᵢⱼ = Cⱼ ∗ Fᵢⱼ

(9)

$$N=\left[\begin{array}{cccc}{N}_{11}& \cdots & N_{1j}& N_{1n}\\ ⋮& ⋮& ⋮& ⋮\\ N_{i1}& \dots & N_{ij}& N_{in}\\ ⋮& ⋮& ⋮& ⋮\\ N_{m1}& \dots & N_{mi}& N_{mn}\end{array}\right]=\left[\begin{array}{cccc}{C}_1{f}_{11}& C_1{f}_{11}& C_1{f}_{11}& C_1{f}_{11}\\ ⋮& ⋮& ⋮& ⋮\\ C_1{f}_{11}& C_1{f}_{11}& \dots & C_1{f}_{11}\\ ⋮& ⋮& ⋮& ⋮\\ C_1{f}_{11}& C_1{f}_{11}& C_1{f}_{11}& C_1{f}_{11}\end{array}\right]$$

The normalized matrix shown in Table 6 was used to create the weighted NDM via Equation (9); the outputs of the scaled NDM, together with criteria weighting, are shown in

Table 8. Weighted NDM.

Criteria Weights	0.133	0.081	0.169	0.1	0.153	0.182	0.183
Criteria	Operating cost	Reliability	Computational efficiency	Detection accuracy	Quality	Numeric robustness	Performance
Alternatives
Crowd-1	0.073	0.063	0.036	0.060	0.050	0.082	0.043
Crowd-2	0.031	0.014	0.071	0.037	0.070	0.041	0.115
Crowd-3	0.021	0.035	0.053	0.045	0.090	0.095	0.057
Crowd-4	0.063	0.028	0.089	0.015	0.080	0.122	0.100
Crowd-5	0.084	0.021	0.107	0.052	0.040	0.027	0.072

.

Step 4. Calculating ideal +ive and −ive parameters

The ideal +ive and −ive solutions are determined using the given Formulas (10) and (11),

$${\mathrm{I}}_{\mathrm{j}}^{+}=\left\{\left(\max \mathrm{i}\left({\mathrm{I}}_{\mathrm{ij}}\right) \mathrm{if} \mathrm{j} \epsilon \mathrm{J}\right);\left(\min \mathrm{i} \left({\mathrm{I}}_{\mathrm{ij}}\right) \mathrm{if} \mathrm{j} \epsilon {\mathrm{J}}^{/}\right) \right\}$$

(10)

$${\mathrm{I}}_{\mathrm{j}}^{-}=\left\{\left(\min \mathrm{i}\left({\mathrm{I}}_{\mathrm{ij}}\right) \mathrm{if} \mathrm{j} \epsilon \mathrm{J}\right);\left(\max \mathrm{i} \left({\mathrm{I}}_{\mathrm{ij}}\right) \mathrm{if} \mathrm{j} \epsilon {\mathrm{J}}^{/} \right)\right\}$$

(11)

The ideal I_j⁻ and I_j⁺ solutions were determined from the weighted normalized table using Equations (10) and (11), and their results are shown in

Table 9. Beneficial and non-beneficial parameter identification.

Criteria	Operating Cost	Reliability	Computational Efficiency	Detection Accuracy	Quality	Numeric Robustness	Performance
Alternatives
Crowd-1	0.073	0.063	0.036	0.060	0.050	0.082	0.043
Crowd-2	0.031	0.014	0.071	0.037	0.070	0.041	0.115
Crowd-3	0.021	0.035	0.053	0.045	0.090	0.095	0.057
Crowd-4	0.063	0.028	0.089	0.015	0.080	0.122	0.100
Crowd-5	0.084	0.021	0.107	0.052	0.040	0.027	0.072
Ij+	0.084	0.063	0.107	0.060	0.090	0.122	0.115
Ij−	0.021	0.014	0.036	0.015	0.040	0.027	0.043

below. A solution that optimizes the beneficial criterion and reduces the cost or non-beneficial criteria was the positive ideal I_j⁺ solution. The negative ideal I_j⁻ solution optimizes the cost or non-beneficial criteria while minimizing the beneficial criteria. The ideal I_j⁺ was the best crowd-motivation reason, while I_j⁻ was considered to be the worst crowd-motivation reason.

Step 5. Identifying ideal and non-ideal separation

The ideal (S⁺) as well as non-ideal separation (S⁻) was determined using Equations (12) and (13):

$${\mathrm{S}}^{+}=\sqrt{{\displaystyle \sum }_{\mathrm{J}=1}^{\mathrm{n}}{({\mathrm{N}}_{\mathrm{ij}}-{ \mathrm{I}}^{+})}^2}$$

(12)

$${\mathrm{S}}^{-}=\sqrt{{\displaystyle \sum }_{\mathrm{J}=1}^{\mathrm{n}}{({\mathrm{N}}_{\mathrm{ij}}-{ \mathrm{I}}^{-})}^2}$$

(13)

Equations (13) and (14) were used to compute Si⁺ and Si⁻, accordingly, for the ranking of options, and the overall result is shown in

Table 10. Ideal and non-ideal separation measures.

Alternatives	Si+	Si−	Si+ + Si−
Crowd-1	0.117	0.101	0.218
Crowd-2	0.118	0.090	0.208
Crowd-3	0.109	0.095	0.203
Crowd-4	0.066	0.137	0.202
Crowd-5	0.123	0.106	0.230

.

Step 6. Calculate performance score (Pᵢ) and ranking of alternatives

Pᵢ was determined using Equation (14):

$$\mathrm{Performance} \mathrm{score} \left({\mathrm{P}}_i\right)=\frac{{S_i}^{-}}{\left({S_i}^{+}+{S_i}^{-}\right)}$$

(14)

Table 11. P_i and ranking of alternatives.

Alternatives	Performance Score (Pi)	Ranking
Crowd-1	0.464	3
Crowd-2	0.433	5
Crowd-3	0.466	2
Crowd-4	0.676	1
Crowd-5	0.463	4

demonstrates the ordering of options after measuring the ideal and non-ideal separation measures and identifying pi via Equation (14).

The following Figure 3 depicts the performance score and ranking of the evaluated crowds as alternatives.

We concluded from the outcomes that crowd-4 was the best alternative, having the highest performance value at 0.676, and thus, we ranked it 1st among the available crowds.

5. Conclusions

Crowdsourcing is a popular strategy for accomplishing a variety of tasks. Several individuals and their interactions are involved in the process, such as requesters that manage, execute, and supervise crowdsourcing initiatives and may post task requests, the crowd (individuals), consisting of virtual employees who participate in outsourcing activities or events, and the platform, which serves as a channel for interaction between the crowd and the crowdsourcers. The people on these platforms are connected by means of social media such as Facebook, WhatsApp, Instagram, Twitter, Y-mail, and Gmail accounts. By utilizing crowdsourcing, a requestor may access a wide potential audience with a variety of skills and experiences to help with work that would be challenging to do without many people. In the realm of software engineering, crowdsourcing has been used to address coding, validation, and architectural problems. A range of incentive schemes are used to encourage public involvement, just as they are used to motivate internal working groups in businesses. Employees get several types of points, cash awards, and other rewards based on their performance. Our study focused on achieving three goals related to crowdsourcing paradigm:

(1)

Analysis of the appropriate characteristics of social crowd utilized for effective software crowdsourcing.

(2)

Analysis of the participation reasons of people carrying out software developmental tasks.

(3)

Development of a decision support system for evaluating primary participation reasons by assessing various crowds using Fuzzy AHP and TOPSIS techniques.

A decision support system was developed in this study to analyze the appropriate reasons for crowd participation in software development. Rewards and employment were evaluated as the highest motives of crowds for task accomplishment, followed by knowledge-sharing, ranking, competency, socialization, and source of inspiration. As this study evaluated crowd drives and motivations for task accomplishment, it will assist crowdsourcing organizations in enhancing their operations and productivity by providing incentives according to crowd needs and expectations.