Next Article in Journal
A Multi-Source Early Warning System of MEMS Based Wireless Monitoring for Rainfall-Induced Landslides
Previous Article in Journal
Adaptive LOS Path Following for a Podded Propulsion Unmanned Surface Vehicle with Uncertainty of Model and Actuator Saturation
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Value Systems Alignment Analysis in Collaborative Networked Organizations Management

by
Patricia Macedo
1,2,* and
Luis Camarinha-Matos
2,3
1
Polytechnic Institute of Setubal, 2900 Setúbal, Portugal
2
Institute for the Development of New Technologies (Uninova), Centre of Technology and Systems, 2829 Campus de Caparica, Portugal
3
Faculty of Sciences and Technology, Nova University of Lisbon, 2829 Caparica, Portugal
*
Author to whom correspondence should be addressed.
Appl. Sci. 2017, 7(12), 1231; https://doi.org/10.3390/app7121231
Submission received: 19 October 2017 / Revised: 17 November 2017 / Accepted: 20 November 2017 / Published: 28 November 2017

Abstract

:
The assessment of value systems alignment can play an important role in the formation and evolution of collaborative networks, contributing to reduce potential risks of collaboration. For this purpose, an assessment tool is proposed as part of a collaborative networks information system, supporting both the formation and evolution of long-term strategic alliances and goal-oriented networks. An implementation approach for value system alignment analysis is described, which is intended to assist managers in virtual and networked organizations management. The implementation of the assessment and analysis methods is supported by a set of software services integrated in the information system that supports the management of the networked organizations. A case study in the solar energy sector was conducted, and the data collected through this study allow us to confirm the practical applicability of the proposed methods and the software services.

Graphical Abstract

1. Introduction

Over the last years, and motivated by the positive effects experienced by the organizations that collaborate, several studies have been conducted on the scope of Collaborative Networked Organizations (CNO). Distinct research projects [1,2,3,4,5] have focused on gaining a better understanding of the effective approaches to manage collaboration. One of the topics that is still challenging is the analysis of value systems alignment in CNOs management. This is motivated by the fact that, despite the potential benefits of collaboration, consortia often fail due to internal conflicts, and most of the time, conflicts appear due the existence of different prioritization of values and distinct perceptions of the outcomes of collaboration. Because perception of outcomes depends on the preferences of the evaluator, an effort to have network members with aligned value systems can be a decisive contribution for the collaboration sustainability.
However, the literature review on CNOs topics reveals that the analysis of value systems alignment is not usually integrated in CNO management practices, in spite of several authors, e.g., [6,7], having identified the importance of aspects such as compatible goals, complementary skills, and compatible cultures and values to the success and sustainability of an alliance. In this context the following research question was proposed: How can value systems alignment analysis be integrated in CNO management practices, and supported by information systems?
The aim of this work is thus to develop a set of software services that allow the integration of value systems alignment analysis into the management of CNOs. As a theoretical basis for the specification of the intended software services, the V-Align framework [8] is briefly reviewed, allowing to identify how the framework can be applied in the distinct processes of CNOs management. The proposed solution was tested in practice through a case study related to the building of a solar power plant in Charanka, India, in the context of the GloNet European research project.

2. Research Methodology

The constructive research method [9] was followed in this research work. This method is both prescriptive and focused on building one or more artifacts (such as frameworks, diagrams, models, prototypes etc.) to solve a domain problem. These artifacts have the purpose of “creating knowledge on how the problem can be solved” and demonstrating how the found solution is innovative or better than previous ones. An implementation of the solution can be used to demonstrate the usability of the artifacts.
The research is supported on the body of knowledge of the Collaborative Networks discipline [10] and is focused on providing management methods and tools that contribute to the sustainability of the CNOs. Starting with the assumption that the assessment of value systems alignment can play an important role in the formation and evolution of CNOs, contributing to reduce potential risks of collaboration, a set of artefacts is built to allow the integration of this assessment in CNOs’ management practices. A value systems alignment tool that implements a value systems alignment (V-Align) framework [8], and which can be integrated in existing CNOs’ management systems through web services, is developed with the purpose of demonstrating the usability of the proposed artifacts. The explanation and detailed discussion of how the developed artifacts contribute to enrich the body of knowledge of the Collaborative Networks discipline demonstrate the theoretical relevance of the proposed solution. To make evident the practical relevance of the proposed solution, a case study was conducted in the building of solar park in India. This case study allowed collecting a set of evidences confirming that the developed tool can be applied in real world contexts, while providing relevant information to support better decision-making during CNOs’ life-cycle management.

3. Related Work on Value System in CNOs

3.1. Brief Overview of Collaborative Networked Organizations

A CNO (Collaborative Networked Organization), or simply Collaborative Network, is a term used to name “alliances constituted of a variety of organizations that are largely autonomous, geographically distributed, and heterogeneous in terms of their operating environment, culture, social capital and goals, but that collaborate to better achieve common or compatible goals, and whose interactions are supported by a computer network” [11]. The involvement in this kind of alliances is expected to increase the capacity of survival in market turbulence contexts, since enterprises can then acquire a larger apparent dimension, access to extended markets, access to new knowledge, and have a way to share risks and resources, while joining their complementary skills [12]. Examples of business areas that have benefited from the adoption of collaborative networks paradigm include the industrial sector [13,14], the transport sector [15], the agribusiness sector [16], ICT and health [17,18], energy [19] etc. Distinct forms of collaborative networks, including virtual enterprises (VE), virtual organizations (VO), dynamic supply chains, industry clusters, business ecosystems, virtual organizations breeding environments (VBE), professional virtual communities (PVC), collaborative virtual laboratories (VL), etc., can been found in our society. These cases can be classified according to their internal structure and organizational model (structured or ad-hoc), purpose or business model (goal-oriented or long-term strategic networks), duration, and type of participants (enterprises, individual professionals, intelligent machines, etc.) [20].
A CNO evolves through several stages along its life-cycle, each stage being characterized by a set of partial goals to be achieved and thus a set of specific processes to be executed. Therefore, to be able to properly apply the developed value systems analysis method, it is important to understand the specificities of each of these life-cycle phases. For instance, the steps required for VBE management are slightly different from those involved in the management of a VO, as VBE and VO have distinct business goals. The main tasks to be executed for each stage of the CNO life-cycle and according to each network type are described in the model introduced by Camarinha-Matos and Afsarmanesh [21] and summarized in Table 1.
Literature on collaborative networks has extensively pointed out several benefits of collaboration and discussed the main requirements that should be satisfied in order to promote the success and sustainability of the networks [22]. Several studies [23,24] have suggested that sharing goals among members, having set some common infrastructures, having reached a good level of mutual trust, and having agreed (totally or in part) on some business practices and values are mandatory requirements to achieve a sustainable network. Some methods and tools for modeling and assessing the achievement of these requirements have been developed. For instance, in [25], a framework to manage and assess trust in CNOs was proposed, while Camarinha-Matos and Abreu [26] discuss how to evaluate the success of a collaboration by measuring the collaboration achieved benefits. In [27], an approach to assess the preparedness of an enterprise to join a CNO is proposed, where the concept of competences fitness is introduced. This concept is further developed in [28] where “the effects of the behavioral (soft) competencies on the performance of the functional (hard) competencies” is discussed in order to compute the competency levels in face of the requirements of the business opportunities. More recently, Andres and Poler [29] published some achievements related to the importance of the business strategies alignment for the success of CNOs.

3.2. Analysis and Characterization of Value Systems

The term “value system” has been widely used in different contexts and for distinct purposes, but often without a totally clear meaning. In fact, it is not easy to find in literature a consensus on what a value system is, which elements compose it, and how these elements are characterized and inter-related. For instance, in [30] two main approaches for the value systems concept are discussed, namely the approach of economy, where a value system is characterized by the elements that are the enablers of value creation or value exchange, and the approach of socio-psychology, where a value system is characterized by the set of elements that guide the behavior of individuals and/or organizations. Partial models to represent a value system can be found in distinct areas, such as sociology, software engineering, organizational management, psychology, and artificial intelligence. In Table 2 the main contribution to value systems modeling along the last decades are summarized.
The definition of Core Value System adopted in this research work is the one introduced in [41], where a core value system is defined algebraically as a tuple C V S = C E V S , C R V S , being C E V S = C O S , C E S the aggregation of two subsystems that compose the “core value system” and being CRVS the set of relationships between these two sub-systems: (i) the Core Value Objects Subsystem (COS), which is represented by the organization or by the CNO itself; (ii) Core Evaluation Subsystem (CES), which is specified as a tuple C F , C V , C P , where CF is the set of evaluation functions used to evaluate the organization’s core-values; CV is the set of core-values; and CP represents the core-evaluation perspective. CRE is the set of relationships among core-values, evaluation functions and core-evaluation perspectives. A core-evaluation perspective (CP) is defined as a tuple <dvcore, wvcore>, where dvcore defines the vector of core-values of the organization (or network) and wvcore defines the weights vector, in which each element represents the degree of importance of the corresponding core-value. Such weights qualitatively represent the preferences of the holder of the value-system (see Figure 1 for an illustrative example of the main concepts).
Example 1.
A Research Center defined its core-values as: knowledge, innovation, uniqueness, and profit and specify for each core-value its degree of importance. These core-values are, thus part of the Core Value System of the enterprise, and more specifically part of the core-evaluation subsystem (CES).
  • CV = {knowledge, uniqueness, innovation, profit}
  • dvcore = [knowledge, uniqueness, innovation, profit]
  • wvcore = [fair, high, high, low], representing the enterprise’s preferences.

3.3. An Assessment Framework for Value System Alignment Analysis

“Alignment” is a term usually used to describe fitness, consistency and similar ideas. Therefore, the design of methods to assess the alignment between value systems in a collaborative environment requires the identification of the factors that influence either the alignment or the misalignment. The identification of the core values shared among the network members and shared between the network itself and each member, constitute a typical criterion used for the alignment assessment. However, this criterion is not sufficient to assess values alignment, because the sharing of values is not the only factor that contributes to the sustainability of a collaborative process. In a previous work [8], suggested that the assessment of the value systems alignment in collaborative environments should consider not only a comparison between core values and their priorities, but should also involve the estimation of the impact that one value system has into another. Thus, the assessment model needs to consider two levels of assessment, the value systems alignment among network members; and the value systems alignment between the CNO and the network members. In both levels, the alignment analysis shall take in account three aspects: (1) the shared core values between value systems; (2) the positive impacts between the core values of the two value systems; and (3) the negative impacts among the core values of the two value systems. The V-Align framework (see Figure 2), which resorts to graph theory and causal maps, provides an effective way to model these aspects, namely: “the relationships among core-values, to understand how they influence each other, which are represented in Core-values influence map”; “the relationships between core-values and organizations, to know which core-values are held by each organization, which are represented in Organizations’ core-values map.” The relationships “between core-values and the CNO, to understand which core-values are held by the network, which are represented in CNO’s core-values map.”
Considering the set of maps (Core-values influence map, Organizations’ core-values map, and CNO’s core-values map), defined in V-Align, three base indicators have been proposed: “Shared Values Level,” “Potential for Conflict Level,” and “Positive Impact Level” (see [8] for an exhaustive description of these indicators). A brief overview of V-Align is shown in Figure 2. The implementation of mechanisms to evaluate these indicators consider the existence of a “Reference Core-Values Ontology,” shared by the VBE members, and containing a description of all core-values that VBE organizations can hold. The Reference Core-Values Ontology can be built with the information obtained through surveys and interviews or directly provided by experts.
The “Network Value System Alignment Level” indicator [44] aims to give an indication of the overall alignment level among the network members’ value systems. This indicator is calculated as the average of the “Value System Alignment Levels” between each pair of members. On the other hand, the “Value System Alignment Level” represents the level of fitness between two value systems and results from the aggregation of two benefit criteria (Shared Value Level and Synergies Level) and one cost criteria (Potential for conflict).

4. Value System Alignment Analysis in CNO Management Practices

4.1. In Distinct Life-Cycle Phases of Collaborative Networked Organizations

As introduced in Section 3.1, activities performed in each phase of CNO are distinct according to the type of network. Therefore, the application of Value System Alignment analysis in VBEs and VOs is also distinct, as summarized in Table 3 for the 3 main phases. It is assumed that VOs are formed inside a VBE, and that all members that compose the VBE and all VO potential members have their profile entered in the VBE management system, and thus that their value systems were already set on the corresponding profiles. It is assumed that each VBE member is responsible for defining its Value System. The process of identification of core values and priorities of each organization is out of the scope of this research work, although we consider it as an important issue. Badovic and Beatty [45] and Brian Hall [31] have proposed methods to identify core values based in three distinct instrumental techniques, namely interviews, document analysis, and questionnaires, which can be applied to these purposes.
The following sections will detail some stages related to VOs and VBEs.

4.2. In the Creation of Virtual Organizations

During “the formation of goal-oriented networks, which typically assume the form of either a short-term or long-term VO” [21], the selection of adequate partners represents “a crucial step for the success of these networks” [46]. Often, such networks are composed of members of the VBE and also of entities related to the customer (in case co-innovation is pursued) and local suppliers (customer’s-related network), materializing the concepts of glocal enterprise and co-creation.
Due to the large diversity and heterogeneity of organizations present in the global market, the VOs creation deals with many challenges, including identifying a suitable set of partners, building trust, making an adequate division of responsibilities and rights, promoting a cooperation spirit among partners, achieving agreements on working principles, and establishing some base commonality on concepts, models and infrastructures.
At the VO creation stage, the VO planner typically determines the set of lists of potential members to form the VO (according to the required competences), i.e., a list of potential consortia. To select the best consortium out of this list, one criterion can be the analysis of the alignment of the value systems among the potential partners (i.e., organizations that satisfy base requirements in terms of needed competencies, availability, etc.). Therefore, for each set of potential VO members:
  • Calculate the Network Value System Alignment level. This value is used in the risk level estimation, since the higher is the alignment level, the lower is the estimated risk level.
  • Build the aggregate Organizations’ core-values map (a map resulted from the insertion of the influence relationships between the core-values on the Organizations’ core-value map). This map will allow the identification of the synergies between members, i.e., positive impacts, and the conflicts between their values.
The VO planner may then select the most suitable group of partners to constitute the VO consortium based on the obtained values.

4.3. In the Evolution of Virtual Organizations

Particularly for the case of long-term VOs, it might be necessary to reorganize the members’ structure, with some members leaving the network and others joining it. At this life-cycle stage it is important to understand the impact of the admission of a new member to prevent potential conflicts. In case alternative organizations are available, it is convenient to be able to select the partner that in respect to its value system is more aligned with the existing configuration, to guarantee that the new candidate’s value system will fit the value systems of the members already in. The addition of new values can bring new positive synergies and the potential for new conflicts to the network, changing the existing dynamics among VO partners. The identification a-priori of these potential synergies or conflicts may be crucial to guarantee the successful integration of a new member in the network.
Therefore, to evaluate the impact of a new member added to the VO, the following steps can be performed:
  • Specify the value system (core values and priorities) for the new member in its profile.
  • Calculate the “Network Value System Alignment Level” for the new VO configuration and compare the value obtained with the previous value, to appraise if the new configuration is better or worse than the previous one in terms of Value Systems Alignment.
Calculate the “Shared Value Level,” the “Potential for Conflict Level,” and the “Synergies Level” between the new member and each other member that belongs to the network. These results will allow identifying the members that are more aligned with the new member and the ones with who conflicts are more likely to occur.

4.4. In the Operation of Virtual Organizations Breeding Environments

The VBE operation includes activities such as (i) assistance in VO creation, (ii) management of competencies and shared assets, (iii) establishment and management of common domain ontologies, (iv) admission of new members, (v) promotion of good understanding and sharing of principles and rules by all VBE members, (vi) definition of the collaboration processes, and (vii) promotion of the acquisition and management of further common knowledge and shared assets. During VBE operation, it is normal that some conflicts among members emerge, being one of the roles of VBE manager to guarantee the VBE sustainability.
The sustainability of a long-term strategic network depends on the capacity of the VBE to maintain its members satisfied and to be able to attract new members. For this purpose, the value systems alignment analysis can be useful from two perspectives:
  • In case of conflict among members, an analysis of values alignment will allow to calculate the level of conflict between each pair of members. If the obtained level is high, a detailed qualitative analysis can be performed to identify which values are contributing to the conflict. This qualitative analysis involves the generation of Organization’s core-values maps, as proposed in the V-Align framework, which allow an explicit identification of the incompatibilities among members in terms of values.
To promote the VBE sustainability, it is strategic to understand what motivates members to belong to the network to be able to keep them satisfied. For this purpose, an analysis of the alignment between the value system of each member and the value system of the network can provide relevant elements to understand which values are responsible for keeping each member “aligned” with the vision of the network. A qualitative analysis can be conducted using the V-Align framework through the study of the VBE’s core values map.

5. The V-Align Software Tool

5.1. Overview of the Software System

This section focuses on the implementation of the V-Align software tool and its possible integration with other systems. As introduced in previous sections, the value systems alignment assessment is a process that should be integrated in CNOs management processes. Therefore, the V-Align software tool aims at providing a mechanism to help in the assessment of value systems alignment along the distinct phases of the VBE and VO life-cycles.
V-Align is built to support two distinct modes of use:
(i)
as a stand-alone web application, providing a user interface to define VBE and VO value systems;
(ii)
integrated with other CNO information systems, using web services.
Therefore, the implementation of V-Align faced the challenge of both developing a user interface that copes with the characteristics of different devices (e.g., PCs, tablets, smart-phones, etc.), with dynamic graph rendering, and developing a set of web-services that support easy integration with other applications.
To achieve these requirements, Java script with Jason files was adopted to implement the client–side for rendering the V-Align maps; for the server sider JSF 2.0 framework was adopted, and to implement the web services JAX-WS, and JAVA API for XML Web Services were chosen (see Figure 3).

5.2. V-Align Tool Services

The services provided by the V-Align tool can be grouped into three main categories:
  • VBE Value System Alignment Analysis Services
  • Services to produce a partial aggregate map, following the V-Align framework, and a set of indicators related to the alignment level between the value system of a specific member and the VBE value system.
  • Members’ Value System Alignment Analysis Services
  • Services to support the members’ value system alignment analysis according to the V-Align framework. These services deliver information that supports the analysis of the alignment level among a group of network members, more specifically, they calculate a set of alignment indicators, and deliver a complete aggregate map.
  • A web service that receives a list of members’ identifiers, and returns the Network Value System Alignment Level.
  • Core-Values Ontology Management Services
  • A web service that returns the list of core-values adopted in the Reference Core-Values Ontology, and their description.
The above services are not intended to work in standalone mode. Rather, they need to use information provided by some subsystems and generate information that can be consumed by other subsystems.
In the case of the Glonet project, these services are used to interact both with the VBE Management System and VO Management System. Figure 4 illustrates the i* Rationale Strategic model, where the considered actors as well as their dependency with objectives with other sub-systems are shown. Furthermore, the services included within the V-Align boundaries are also represented.
To illustrate how the conceptual integration between the V-Align Tool and the other subsystems represented in the above i* diagram is mapped into the GloNet System architecture, a block diagram is presented in Figure 5.

6. Pilot Demonstrator

6.1. The Solar Energy Plant Case Study Overview

With the purpose of validating the developed functionalities, data from a case study related to a Solar Plant built in Charanka Park, in Gujarat, India, was used. This solar park comprises a total installed capacity of about 500 MWp, which makes it one of the largest of its kind in the world. The case study was conducted in the scope of the GloNet project. GloNet was developed to support highly customized, complex, and service-enhanced products, such as a power plant, pursuing collaboration with customers and some local suppliers. GloNet had the duration of 42 month and started in September 2011, involving several academic and industrial organizations (UNINOVA (Lisbon, Portugal), CAS (Karlsruhe, Germany), University of Amsterdam (Amsterdam, The Netherlands), iPLON (Schwäbisch Hall, Germany), SKILL Estrategia (Tomares, Spain), KOMIX (Praha, Czech Republic), Steinbeis (Stuttgart, Germany), PROLON (Albertslund, Denmark)).
The sector of solar energy is creating new business opportunities for small and medium organizations that can only reach larger markets if involved in CNOs. A solar energy plant requires several business services along its life-cycle and it is an example of a highly customized and complex product, involving a large variety of stakeholders in several distinct roles. Several business services may be added during the “operation and maintenance phase” of a solar plant in order to generate more value to the involved stakeholders, such as warranty enforcement, auditing, training services, monitoring, and preventive maintenance [19]. In most cases, in the solar plant industry, the market offers fragmented services, while clients demand integrated services, which requires support for collaboration among various stakeholders.
The case study was performed in the context of a collaborative R&D project and performed partly in parallel with the development of the power plant. As such, part of the end-users’ team accompanied all phases of the development of V-Align. On the other hand, software developers assisted the end-users in the pilot validation process.
The GloNet pilot initiative proposes a move from traditional sub-contracting practices in the solar energy sector to the establishment of CNOs and (partially) remote service provision. During the life-cycle of a solar plant a variety of stakeholders are involved. Several of these stakeholders do many projects together, and thus they can be seen and organized as a VBE. The creation of proper profiles, including value systems modeling, is likely to facilitate partners’ search and consortia creation for each new project.
The stakeholders involved in our case study, which focuses mainly on the “operation and maintenance phase” of the solar plant and the co-creation of new services, are listed in Table 4. For confidentiality reasons the names of companies were anonymized.
As shown in Figure 4, V-Align services require a set of information provided by the VBE Base Management subsystem. Therefore, during the set-up of the pilot experiment, the VBE Management subsystem was used to manage the Solar VBE profile and the profile of each VBE member. Each relevant stakeholder in this scenario was modeled including member’s roles, values, profile data, competences, and rights. To illustrate distinct applications of the value system alignment in a collaborative context three main cases are explored:
  • Partners’ Selection during consortium formation.
  • Partner’s Selection to replace a member in an existing consortium.
  • Conflict Resolution between two partners during VBE operation.

6.2. Partner Selection during Networked Organizations Formation

In the starting phase of a new solar plant, a detailed specification of the complex product has to be entered in the GloNet System through a specific module [47]. Based on this specification, a VO creation subsystem matches the requirements with the competences existing in the VBE to identify members that can be suitable candidates for the new VO. All possible VO combinations that can satisfy the specified goals are then automatically generated (see Figure 6).
To select from the list of potential consortia the most appropriate one, the VO Planner assesses the network Value System alignment level with the involved partners in each potential consortium. Table 5 shows the list of the potential consortia ranked by the Network Value System Alignment Level. In this case, we can notice that consortium 1 has the lowest value and consortium 4 the highest. Consortium 4 is formed by Organizations KI, AJ, HS, IW, and STS, as shown in Figure 6.
Using V-Align it is also possible to get a report of the value system alignment assessment for each consortium (see Table 6, for the case of Consortium 4). In this example organization ST has a potential of conflicts with organizations KI, IW, and HS.
This potential misalignment is caused by the negative influence between the agility and reliability core-values, as evidenced in the core-values members’ map of Figure 7. However, from a global perspective, it can be noticed that the five members of this consortium are well aligned in terms of values, having high scores in the synergy level and shared values level.
In any case, the final decision regarding the consortium to be chosen will be made by the VO planner. As such, V-Align can be considered as a decision-support tool.

6.3. Partner’s Selection to Replace a Member in an Existing Virtual Organization

During the operation of a virtual organization, namely in the solar plant case study, let us assume that Organization IW asked to be replaced in the consortium and suggests Organization ES to be its substitute. Since ES is a member of the solar VBE, its member’s profile was already set in the GloNet System. Using V-Align services, the “Shared Values Level,” “Potential for Conflicts Level,” and “Synergy Level” can be computed (see Table 7). From the obtained results we can conclude that the new partner is well-aligned with the existing members in the VO consortium in terms of its value system.

6.4. Conflict Resolution between Two Partners during VBE Operation

As explained in Section 4.2, during VBE operation, an analysis of the alignment between the value system of each member and the value system of the VBE could provide interesting elements to understand which the values contribute to keep each member “aligned” with the vision of the network. From the example in Table 8 it can be noticed that Organization ES does not share values with the VBE, and has a null positive impact in its value system. These results indicate that Organization ES is not aligned in terms of its value system with the VBE, which may represent a threat to the sustainability of the relationship. This conclusion may seem contradictory to the results presented in previous section. However, it is not, since in previous case the alignment analysis was performed between Organization ES and the other VO members (a subset of the VBE). In this case, what was studied was the alignment between the value system of Organization ES and the VBE value system. Despite ES being aligned with the VO consortium members in terms of its value system, it is not aligned with the VBE Value System.
On the other hand, we can notice that Organization IP is well-aligned with the vision of the VBE, and looking at the map of Figure 8, we can easily notice that reputation and interdisciplinary are core-values that contribute to keep this organization aligned with the VBE.
In case of any conflict among members, the analysis of the indicators obtained in the Members Alignment Report could give us a better understanding about the base reason for the conflict, allowing the identification of incompatible values or lack of shared values. For instance, Organization LT and Organization AJ have no shared values and have some incompatible values, so there is some risk of potential conflicts between these two members (see Table 9).

7. Discussion

A number of case studies were conducted to demonstrate and validate GloNet project results. The adopted validation methodology included three phases. In phase 1, the GloNet system was discussed with and tested by 11 lead users based on relevant business processes identified by these users. In phase 2, the best practice results achieved in phase 1 were presented to approximately 50 potential users within the same or similar business sectors as the lead users. The users group selected for this validation phase represented different application domains and diverse geographical regions in Europe. In phase 3, an in-depth validation of selected GloNet components was made by 40 potential users using real data from the Charanka power plant in India. In this case, a validation analysis was performed, where the “required functionality” by the end users was compared with the “delivered functionality” by the GloNet system. For each of the main GloNet software subsystems, specific questionnaires were filled in by the evaluators.
The results obtained from these surveys regarding the Glonet VBE Management subsystem and the VO formation and operation support subsystem are presented in Figure 9 and Figure 10. As it can be seen, GloNet solutions were considered positive in all adopted evaluation criteria. The value systems alignment analysis functionalities were not evaluated in isolation. However, as these mentioned subsystems embed the value system alignment analysis, as explained in Section 5.2, it can be inferred that the V-Align features satisfy the needs and expectations of the end-users.

8. Conclusions and Future Work

Risks in a CNO can be reduced if there is a good level of alignment among the value systems of the various members of the network. This assumption is motivated by the fact that the behavior of an organization is, to a large extent, driven or influenced by its value system. Therefore, value systems alignment is a relevant criterion to be considered in the process of CNO formation. As such, it is important to integrate a value systems alignment assessment mechanism with a consortia formation environment.
On the other hand, rapid formation of VOs can better be achieved if done in the context of a long-term strategic alliance, like a VBE, that is established to promote the preparedness of its members to collaborate. As such, if a value systems alignment mechanism is integrated with a “VBE management system,” it can support both the process of acquiring a new member to join the VBE and the processes of VO creation and evolution.
According to these hypotheses, a prototype V-Align system was designed and developed as part of the “VBE management system” and “VO management system” of the GloNet project. The developed functionalities were validated through interaction with a large group of potential end-users and more specifically in a case study in the solar energy domain. Validation results are quite promising and tend to confirm the importance of value systems alignment in CNO management.
In spite of validation results are quite promising, during our research work we identify some significant issues that we would like to address in a future work. A subject that should be discussed in more detail is how to guarantee that the meaning of each core value is the same for all the stakeholders, since it is easy to agree on words, however it is difficult to developing a shared meaning for these words. An issue which also deserves additional work is to construct an adequate core values influence map. A simple map was proposed in this pilot case study, which was considered as plausible for diverse researchers in the area. However, the findings in the Core Value Systems alignment assessment are dependent on the core values influence map adopted. Therefore, methods to construct and evaluate core values influence maps should be explored and validated. Data mining methods to discover core value influence relations seem to be an adequate approach. In spite of the core values and priorities assessment being out of the scope of this research work, this is an issue that deserves additional attention and where the collaboration of social researches can also be useful. Therefore, it is planned to develop methods and tools to assess the core-values of organizations in a more precise way.

Acknowledgments

This work was funded in part by the European Commission through the GloNet project (FP7 program) and in part by the Center of Technology and Systems and FCT-PEST program UID/EEA/00066/2013 (Impactor project).

Author Contributions

Patricia Macedo and Luis Camarinha-Matos conceived and designed the models and methods presented, Patricia Macedo developed the software tools and performed the testes presented in the case study; Patricia Macedo and Luis Camarinha-Matos analyzed the data and wrote the paper.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Capuano, N.; Gaeta, A.; Gaeta, M.; Orciuoli, F.; Brossard, D.; Gusmini, A. Management of virtual organizations. In Service Oriented Infrastructures and Cloud Service Platforms for the Enterprise; Springer: Berlin/Heidelberg, Germany, 2010; pp. 49–73. [Google Scholar]
  2. Chevrier, S. Cross-cultural management in multinational project groups. J. World Bus. 2003, 38, 141–149. [Google Scholar] [CrossRef]
  3. Kelly, M.J.; Schaan, J.-L.; Joncas, H. Managing alliance relationships: Key challenges in the early stages of collaboration. R D Manag. 2002, 32, 11–22. [Google Scholar] [CrossRef]
  4. Martinez, M.; Fouletier, P.; Park, K.; Favrel, J. Virtual enterprise—Organisation, evolution and control. Int. J. Prod. Econ. 2001, 74, 225–238. [Google Scholar] [CrossRef]
  5. Noordin, N.A.; Bititci, U.S.; Van Der Meer, R. Review on collaborative decision making in supply chain: The relationship between e-collaboration technology and development of inter-organizational trust. In Proceedings of the IFIP International Conference on Advances in Production Management Systems, Rhodes, Greece, 24–26 September 2012; Springer: Berlin/Heidelberg, Germany, 2011; pp. 326–341. [Google Scholar]
  6. Bititci, U.; Martinez, V.; Albores, P.; Parung, J. Creating and managing value in collaborative networks. Int. J. Phys. Distrib. Logist. Manag. 2004, 34, 251–268. [Google Scholar] [CrossRef] [Green Version]
  7. Camarinha-Matos, L.M.; Afsarmanesh, H.; Galeano, N.; Molina, A. Collaborative networked organizations—Concepts and practice in manufacturing enterprises. Comput. Ind. Eng. 2009, 57, 46–60. [Google Scholar] [CrossRef]
  8. Macedo, P.; Camarinha-Matos, L.M. A qualitative approach to assess the alignment of Value Systems in collaborative enterprises networks. Comput. Ind. Eng. 2013, 64, 412–424. [Google Scholar] [CrossRef]
  9. Kasanen, E.; Lukka, K.; Siitonen, A. The constructive approach in management accounting research. J. Manag. Account. Res. 1993, 5, 243–264. [Google Scholar]
  10. Camarinha-Matos, L.M.; Afsarmanesh, H. Collaborative networks: A new scientific discipline. J. Intell. Manuf. 2005, 16, 439–452. [Google Scholar] [CrossRef]
  11. Camarinha-Matos, L.M.; Afsarmanesh, H. Classes of collaborative networks. In Encyclopedia of Networked and Virtual Organization; Information Science Reference: Hershey, PA, USA, 2008. [Google Scholar]
  12. Bititci, U.; Turner, T.; Mackay, D.; Kearney, D.; Parung, J.; Walters, D. Managing synergy in collaborative enterprises. Prod. Plan. Control 2007, 18, 454–465. [Google Scholar] [CrossRef]
  13. Cunha, P.F.; Ferreira, P.S.; Macedo, P. Performance Evaluation within Cooperate Networked Production Enterprises. Int. J. Comput. Integr. Manuf. 2008, 21, 174–179. [Google Scholar] [CrossRef]
  14. Westergren, U.H.; Holmström, J. Exploring preconditions for open innovation: Value networks in industrial firms. Inf. Organ. 2012, 22, 209–226. [Google Scholar] [CrossRef]
  15. Osório, L.A.; Camarinha-Matos, L.M.; Afsarmanesh, H. ECoNet Platform for Collaborative Logistics and Transport; Springer: Cham, Switzerland, 2015; pp. 265–276. [Google Scholar]
  16. Volpentesta, S.A. Alternative agrifood networks in a regional area: A case study. Int. J. Comput. Integr. Manuf. 2013, 26, 55–56. [Google Scholar] [CrossRef]
  17. Baldissera, T.A.; Camarinha-Matos, L.M. Services Personalization Approach for a Collaborative Care Ecosystem; Springer: Cham, Switzerland, 2016; pp. 443–456. [Google Scholar]
  18. Bruun-Rasmussen, M.; Bernstein, K.; Chronaki, C. Collaboration—A new IT-service in the next generation of regional health care networks. Int. J. Med. Inform. 2003, 70, 205–214. [Google Scholar] [CrossRef]
  19. Camarinha-Matos, L.M.; Oliveira, A.I.; Ferrada, F.; Thamburaj, V. Collaborative services provision for solar power plants. Ind. Manag. Data Syst. 2017, 117, 946–966. [Google Scholar] [CrossRef]
  20. Afsarmanesh, H.; Camarinha-Matos, L.M. Vbe Reference Framework. In Methods and Tools for Collaborative Networked Organizations; Springer: New York, NY, USA, 2008; pp. 35–68. [Google Scholar]
  21. Camarinha-Matos, L.M.; Afsarmanesh, H. The virtual enterprise concept. In Infrastructures for Virtual Enterprises; Springer: New York, NY, USA, 1999; Volume 153, pp. 15–30. [Google Scholar]
  22. Durugbo, C. Collaborative networks: A systematic review and multi-level framework. Int. J. Prod. Res. 2016, 54, 3749–3776. [Google Scholar] [CrossRef]
  23. Huxham, C.; Vangen, S. Managing to Collaborate: The Theory and Practice of Collaborative Advantage; Routledge: Abingdon, UK, 2013. [Google Scholar]
  24. Cummings, J.L.; Holmberg, S.R. Best-fit Alliance Partners: The Use of Critical Success Factors in a Comprehensive Partner Selection Process. Long Range Plan. 2012, 45, 136–159. [Google Scholar] [CrossRef]
  25. Msanjila, S.S.; Afsarmanesh, H. Trust analysis and assessment in virtual organization breeding environments. Int. J. Prod. Res. 2008, 46, 1253–1295. [Google Scholar] [CrossRef]
  26. Camarinha-Matos, L.M.; Abreu, A. Performance indicators for collaborative networks based on collaboration benefits. Prod. Plan. Control 2007, 18, 592–609. [Google Scholar] [CrossRef]
  27. Rosas, J.; Camarinha-Matos, L.M. A collaboration readiness assessment approach. In Innovation in Manufacturing Networks; Springer: New York, NY, USA, 2008; pp. 77–86. [Google Scholar]
  28. Rosas, J.; Macedo, P.; Camarinha-Matos, L.M. Extended competencies model for collaborative networks. Prod. Plan. Control 2011, 22, 501–517. [Google Scholar] [CrossRef]
  29. Andres, B.; Poler, R. Dealing with the Alignment of Strategies Within the Collaborative Networked Partners. In Technological Innovation for Cloud-Based Engineering Systems; Springer: Cham, Switzerland, 2015; pp. 13–21. [Google Scholar]
  30. Macedo, P.; Sapateiro, C.; Filipe, J. Distinct approaches to Value System in collaborative networks environments. In Network-Centric Collaboration and Supporting Frameworks; Springer: Boston, MA, USA, 2006; pp. 111–120. [Google Scholar]
  31. Hall, B.P. Values Shift: A Guide to Personal and Organizational Transformation; Wipf and Stock Publishers: Eugene, OR, USA, 2006. [Google Scholar]
  32. Filipe, J.; Liu, K. The EDA Model: An Organizational Semiotics Perspective to Norm-Based Agent Design. In Proceedings of the Agents’2000 Workshop on Norms and Institutions in Multi-Agent Systems, Barcelona, Spain, 28 May–1 June 2000. [Google Scholar]
  33. Antunes, L.; Faria, J.; Coelho, H. Improving choice mechanisms within the BVG architecture. In Proceedings of the International Workshop on Agent Theories, Architectures, and Languages, Boston, MA, USA, 7–9 July 2000; Springer: Berlin/Heidelberg, Germany, 2000; pp. 290–304. [Google Scholar]
  34. Alle, V. Reconfiguring the Value Network. J. Bus. Strat. 2000, 21, 36–39. [Google Scholar] [CrossRef]
  35. Goguen, J.A. Semiotics, compassion and value-centered design. In Virtual, Distributed and Flexible Organisations; Springer: Dordrecht, The Netherlands, 2004; pp. 3–14. [Google Scholar]
  36. Kartseva, V.; Gordijn, J. A Design Perspective on Networked Business Models: A Study of Distributed Generation in the Power Industry Sector. In Proceedings of the 12th European Conference on Information Systems, Turku, Finland, 14–16 June 2004. [Google Scholar]
  37. Rodrigues, M.R.; Costa, R.; Bordini, R. A System of Exchange Values to Support Social Interactions in Artifical Societies. In Proceedings of the 2nd International Joint Conference on Autonomous Agents and Multiagent Systems, Melbourne, Australia, 14–18 July 2003. [Google Scholar]
  38. Rodrigues, M.R.; Luck, M. Analysing Partner Selection Through Exchange Values. In Multi-Agent-Based Simulation VI; Lecture Notes in Computer Science; Springer: Berlin/Heidelberg, Germany, 2006; Volume 3891, pp. 24–40. [Google Scholar]
  39. Barrett, R. Building a Vison-Guided, Values-Driven Organization; Paperback; Butterworth-Heinemann: Oxford, UK, 2006. [Google Scholar]
  40. Value Chain Group. Available online: http://www.value-chain.org/ (accessed on 8 May 2017).
  41. Camarinha-Matos, L.M.; Macedo, P. A conceptual model of value systems in collaborative networks. J. Intell. Manuf. 2010, 21, 287–299. [Google Scholar] [CrossRef]
  42. Bourne, H.; Jenkins, M. Organizational values: A dynamic perspective. Organ. Stud. 2013, 34, 495–514. [Google Scholar] [CrossRef] [Green Version]
  43. Laurier, W.; Poels, G. Invariant conditions in value system simulation models. Decis. Support Syst. 2013, 56, 275–287. [Google Scholar] [CrossRef] [Green Version]
  44. Macedo, P.; Cardoso, T.; Matos, L.M.C. Value Systems Alignment in Product Servicing Networks. In Collaborative Systems for Reindustrialization; Springer: Berlin/Heidelberg, Germany, 2013; pp. 71–80. [Google Scholar]
  45. Badovick, G.; Beatty, S. Shared organizational values: Measurement and impact upon strategic marketing implementation. J. Acad. Mark. Sci. 1987, 15, 19–26. [Google Scholar] [CrossRef]
  46. Holmberg, S.R.; Cummings, J.L. Building Successful Strategic Alliances. Long Range Plan. 2009, 42, 164–193. [Google Scholar] [CrossRef]
  47. Afsarmanesh, H.; Shafahi, M.; Sargolzaei, M. On service-enhanced product recommendation guiding users through complex product specification. In Proceedings of the 2015 International Conference on Computing and Communications Technologies (ICCCT), Madras, India, 26–27 February 2015; pp. 43–48. [Google Scholar]
Figure 1. Example of Core Evaluation perspective.
Figure 1. Example of Core Evaluation perspective.
Applsci 07 01231 g001
Figure 2. V-Align Framework overview.
Figure 2. V-Align Framework overview.
Applsci 07 01231 g002
Figure 3. V-Align Software Tool Implementation Architecture and technologies.
Figure 3. V-Align Software Tool Implementation Architecture and technologies.
Applsci 07 01231 g003
Figure 4. V-Align system: i* Rational Strategic model.
Figure 4. V-Align system: i* Rational Strategic model.
Applsci 07 01231 g004
Figure 5. Integration between V-Align Tool and GloNet Subsystems.
Figure 5. Integration between V-Align Tool and GloNet Subsystems.
Applsci 07 01231 g005
Figure 6. Potential Consortium Analysis.
Figure 6. Potential Consortium Analysis.
Applsci 07 01231 g006
Figure 7. Core-Values Members Map—Potential Consortium 4.
Figure 7. Core-Values Members Map—Potential Consortium 4.
Applsci 07 01231 g007
Figure 8. Partial network cores values map.
Figure 8. Partial network cores values map.
Applsci 07 01231 g008
Figure 9. Assessment of subsystems by the solar energy network.
Figure 9. Assessment of subsystems by the solar energy network.
Applsci 07 01231 g009
Figure 10. Assessment of VBE/VO functionalities by the solar energy network.
Figure 10. Assessment of VBE/VO functionalities by the solar energy network.
Applsci 07 01231 g010
Table 1. Virtual organizations breeding environments (VBE) and virtual organization (VO) life-cycle stages.
Table 1. Virtual organizations breeding environments (VBE) and virtual organization (VO) life-cycle stages.
Virtual Organization (VO)Virtual Organizations Breeding Environments (VBE)
Creation
Preparatory Planning—Focused on the identification and description of a business opportunity that triggers the VO creation, and designing a draft structure of VO.VBE Initialization—Focused on the definition of the VBE mission, recruitment of VBE members, establishment of common business models, and implementation of a common ICT infrastructure.
Consortium Formation—Including definition of the appropriate organizational structure, partners’ search and selection, agreements negotiation, and definition of roles of the VO members.
VO Launching—Including the refinement of the draft VO plan, definition of “governance principles”, and formulation, modeling and signing of contracts and collaboration agreements.VBE setting up—Including the configuration of ICT support systems, registration and profiling of founding members, setting of governance principles and business rules.
VO Set-up—Involving the configuration of the “ICT support infrastructures”, the instantiation of the needed “collaboration spaces”, according on VO governance principles, the assignment of tasks, and setting up of VO resources.
Operation
VO members: Execution of the processes that were planned during the VO launch phase, according to assigned roles.
VO coordinator: supervision of status of activities and achievement of objectives, including corrective actions in case of deviations.
Involving the assistance in VO creation and other VBE support activities, such as: (i) management of competencies and shared assets, (ii) management of ontologies for the specific domain, (iii) admission of new members, (iv) definition and application of incentives, and (v) assessment of the collaboration processes.
Evolution
Rescheduling of VO activities and milestones, reassigning tasks and roles, and budget reallocation in case of need. It may also involve changes in the consortium membership.Involving the evolution of governance principles and rules, the admission of new members and small readjustments in the VBE organizational structure, etc.
DissolutionMetamorphosis
Record useful experiences, lessons learned and key performance indicators of the VO, which can be used in the VBE operation and creation of future VOs. Define liabilities and apply inheritance mechanisms.Focused on reorganizing the membership structure and on transferring the knowledge collected during the VBE operation.
Table 2. A summary of perspectives on value systems.
Table 2. A summary of perspectives on value systems.
ModelResearchers/ReferencesApproachScientific Area
Values Map—comprising a conceptual map of the values that characterize an organization or an individual in terms of values profile. The model considers three types of values: Focus Values, Foundation Values, and Vision Values.[31]Socio-PsychologySociology
EDA (Epistemic-Deontic-Axiological)—a formal model based on agent theory, “where it is assumed that an agent is responsible for its values, and an agent can represent a member of an organization or an organization itself”. A value system model is introduced as a component of EDA, where the “agent’s preferences with respect to norms” are represented.[32]Socio-PsychologyArtificial Intelligence
BVG (beliefs, values, goals) Model—a formal agents’ model, supporting “the decision-making process and including goals, candidate actions to be chosen from, beliefs about states of the world, and values”. [33]Socio-PsychologyArtificial Intelligence
Value Network—a model that “represents the value exchanges with each and every member of the business or organizational network”. In this model, value exchanges include: goods, knowledge, and intangible benefits.[34]EconomicsKnowledge Management
Value System tree—introduced as “a formal model, which presents the values of an organization in a hierarchical structure”.[35]Socio-PsychologyArtificial Intelligence
e3-value Model—a model developed for e-commerce cases, and “essentially focused on the representation of economic value of objects and on the activities and actors that create economic value”.[36]EconomicsSoftware Engineering
Value Exchange Model—This model is focused on the “dynamics of value interactions among agents (whereas agents can be people or organizations)” and it is supported on Piaget’s theories on value exchange.[37,38]EconomicsArtificial Intelligence
Seven Levels of Consciousness—The set of the values, categorized according to seven levels, represent the value profile and culture of an organization, team, or individual.[39]Socio-PsychologySociology
Value Reference Model (old VCOR Model)—This model covers all categories of processes required to support and enable the value chain execution under govern and plan categories. This includes processes supporting human capital management, assets management, performance management.[40]EconomicsManagement
Core Value System Conceptual Model—this model is specified using algebraic notation and defined as a pair of two subsystems: “(i) the Core Value objects subsystem, which represents the organizations or networked organization to be valuated, and (ii) the Core evaluation subsystem, which represents the core mechanisms of evaluation”.[41]Socio-PsychologyCollaborative Networks
Organizational Values Framework—this model assumes that organizational values are dynamic entities, and classify the values in exposed, attributed, aspirational and shared values. Moreover, they defend that “the identification of the shifting gaps and overlaps between value forms reveals the dynamic nature of organizational values”.[42]Socio-PsychologyPsychology
Value System Simulation Model—this model is based on the e3-value model, allowing the simulation of virtual enterprises to assess the financial, operational and logistical performance.[43]EconomicsDecision Support Systems
Table 3. Potential application of Value System Alignment Analysis in each life-cycle phase.
Table 3. Potential application of Value System Alignment Analysis in each life-cycle phase.
Phase/CNOVirtual Organization (VO)Virtual Breeding Environments (VBE)
CreationFor partners’ selection: Select the set of partners that have the potential of working well together, since their value systems are aligned.An assessment can be performed when the VBE is first established or each time a new member is added to the VBE. However, this can also be done during the operation phase.
OperationTo support conflict resolution: In case of conflicts between members, identify which set of incompatible values potentiate the conflict.To support conflict resolution: To improve VBE’s membership experience. E.g., identify the set of members that work well together and the pairs of members that will potentially generate conflicts if they work together.
EvolutionTo study the impact of the admission of new partners and changes in network composition, in terms of value systems alignment.To assess the alignment between the value system of each new candidate member and the value system of the current VBE (although this can be supported by the mechanism foreseen for the operation phase).
Table 4. Partial List of Stakeholders of the Charanka Plant Case Study.
Table 4. Partial List of Stakeholders of the Charanka Plant Case Study.
StakeholderArea
KIProject developer
LTEPC Contractor
EFEmbedded systems
ADSoftware development
AJService Company (security)
HSManufacture of electrical power installation
IWEmbedded hardware, FPGA and software
ESSolar energy consultant
REElectricals and panel cleaning services
STConsultant for PV engineering in Germany
Table 5. VO’s Goals and Potential Consortia.
Table 5. VO’s Goals and Potential Consortia.
VO—Charanka Park
Goals/Competences RequiredPotential Consortium Network Value System Alignment LevelRanked Network Value System Alignment Level
OpticalPotential Consortium 496100
String BoxPotential Consortium 38386
Power PlantPotential Consortium 67578
TransformerPotential Consortium 27174
StringPotential Consortium 56871
Cleaning SolutionPotential Consortium 16770
Table 6. Detailed Results for Potential Consortium 4.
Table 6. Detailed Results for Potential Consortium 4.
Member 1Member 2Shared LevelPotential for Conflicts LevelSynergy Level
Organization KIOrganization IW33060
Organization KIOrganization AJ65022
Organization KIOrganization HS54019
Organization KIOrganization ST781612
Organization IWOrganization AJ76098
Organization IWOrganization HS0031
Organization IWOrganization ST543398
Organization AJOrganization HS54034
Organization AJOrganization ST100067
Organization HSOrganization ST543357
Table 7. Detailed results for the integration of a new member in a VO Consortium 4.
Table 7. Detailed results for the integration of a new member in a VO Consortium 4.
Member 1Member 2Shared LevelPotential for Conflict LevelSynergy Level
Organization—KIOrganization—ES80038
Organization—AJOrganization—ES0012
Organization—HSOrganization—ES0012
Organization—STOrganization—ES0038
Table 8. Network Alignment Report.
Table 8. Network Alignment Report.
MemberShared LevelPositive Impact Level
Organization LT066
Organization KI800
Organization IW0100
Organization AJ10033
Organization IP117179
Organization HS11460
Organization AD600
Organization ST12760
Organization ES00
Table 9. Partial results from VBE Members vs Alignment Report.
Table 9. Partial results from VBE Members vs Alignment Report.
Member 1Member 2Normalized Shared LevelNormalized Potential for Conflict Level Normalized Synergy Level
Organization LTOrganization IW10010043
Organization LTOrganization AJ07577
Organization KIOrganization IP050100
Organization KIOrganization AD1005027
Organization KIOrganization RE785088

Share and Cite

MDPI and ACS Style

Macedo, P.; Camarinha-Matos, L. Value Systems Alignment Analysis in Collaborative Networked Organizations Management. Appl. Sci. 2017, 7, 1231. https://doi.org/10.3390/app7121231

AMA Style

Macedo P, Camarinha-Matos L. Value Systems Alignment Analysis in Collaborative Networked Organizations Management. Applied Sciences. 2017; 7(12):1231. https://doi.org/10.3390/app7121231

Chicago/Turabian Style

Macedo, Patricia, and Luis Camarinha-Matos. 2017. "Value Systems Alignment Analysis in Collaborative Networked Organizations Management" Applied Sciences 7, no. 12: 1231. https://doi.org/10.3390/app7121231

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop