Defining Organizational Context for Corporate Sustainability Assessment: Cross-Disciplinary Approach

Abstract

Extensive research has already been conducted on the technical aspects of a Sustainability Assessment (SA) at a company level, i.e., Corporate Sustainability Assessment (CSA). However, previous research paid little attention to the context in which an assessment takes place, i.e., the conditions within the organization that facilitate or inhibit the effectiveness of CSA as a decision support tool. This study seeks to shed more light on the organizational context for CSA. Since the research in this topic is limited within the SA discipline, the knowledge from the Performance Measurement (PM) discipline is used. Using a cross-disciplinary approach, factors that define the organizational context for CSA were proposed. Furthermore, they were ranked by 104 sustainability professionals using online survey. The survey results revealed that the top five factors are “leadership commitment”, “sustainability strategy”, “data collection capabilities”, “understanding of purpose and benefits of SA” and “focus on continuous improvements”. The practical application of the result of this study was illustrated using a multiple case study, in which the readiness to implement CSA by four manufacturing companies was evaluated.

1. Introduction

In the scientific and professional literature, CSA, or SA at a company level, has been embraced as a means to incorporate sustainability concerns into the decision-making process. Profound research on the development of CSA for different types of organizations—e.g., SMEs, large companies, manufacturing and public—has been carried out and the literature on CSA demonstrates that researchers and practitioners learned a great deal about assessment techniques and sustainability indicators, see, e.g., [1,2,3,4,5,6,7,8,9,10,11,12]. It is apparent that the focus is mainly on the technical aspects, such as the choice of indicators, aggregation of data, requirements to an assessment process, the characteristics of indicators, aggregation methods and challenges related to data collection and analysis. However, existing research paid no attention to the context in which CSA takes place, i.e., conditions within the organization that facilitate or inhibit the effectiveness of assessment as a decision support tool.

Although assessment tools are becoming increasingly comprehensive, see, e.g., [13,14], there is little evidence regarding their regular use by companies. Often a new tool is developed by a group of researchers and is tested in a few case companies without further publications on whether the application of assessment is effective, e.g., [15]. The effectiveness of the application is defined not only by the design of the tool itself but also by the context in which assessment takes place. Organizational context plays a vital role in the implementation and use of the tool, just as any initiatives in an organization can be enabled or hindered by the resistance from employees, by organizational culture, by the lack of resources, etc. Bina [16] argues that good information generated by assessment alone will not lead to better choices and planning but it is the context that makes the difference. She emphasizes the importance of the relationship between assessment (as a system) and its context.

A brief review indicates that limited research has been conducted regarding the role of the context in the success and effectiveness of SA. Most of the existing research regarding this aspect is focused on the Strategic Environmental Assessment (SEA). In the 2000s, researchers started to emphasize a need to better understand the influence of the context for SEA, i.e., politics, society and culture and the institutions and organizations, on the effectiveness of the assessment [16]. Runhaar and Driessen [17] investigated the role of the context in the contribution of the SEA to decision-making. Some of the identified factors that contribute to how SEA influences decision-making are ‘openness’ of decision-making to environment/sustainability, adequate resources and effective communication. Guijt and Moiseev [18] who view SA as a process with seven stages, identify factors which determine a successful application of SA. These are (1) a purpose of the assessment; (2) perceived importance of the assessment by participants; (3) time and funding availability; (4) size of the system being assessed; (5) degree of divergence in participants’ needs and worlds; (6) dealing with representation; and (7) experience of the assessment team. Besides the limited research on the role of the context for SA in general, there is no evidence of the research on the role of the organizational context in the effectiveness of CSA.

In this paper, we will study the organizational context for Corporate Sustainability Assessment System (CSAS). CSAS can be understood as a sub-system within the company that evaluates sustainability performance of the organization on a regular basis. The organizational context for CSAS can be seen as a combination of factors within the organization that can influence the effectiveness of the implementation and use of assessment system. A better understanding of the organizational context and factors that influence the implementation and use of CSA will allow practitioners to assess the readiness of an organization to implement and use CSA.

Although the concept of sustainability is a key to sustainability assessment, the definition of sustainability falls outside of the scope of this paper due to the presence of often contradicting discourses and views on sustainability, see, e.g., [19,20,21,22].

The goal of this paper is to define the organizational context for CSA that can potentially influence the effectiveness of the implementation and use of the assessment. Thus, the research question in this study is: What are the organizational factors that can affect the effectiveness of CSA? Since the research on the organizational context and factors that influence the effectiveness of CSA is limited, a cross-disciplinary approach is adopted. The cross-disciplinary approach allows borrowing knowledge, expertise and information from one discipline to another. The research question is approached from the perspective of the PM discipline due to the lack of relevant research in the SA discipline. First, the extrapolation of information from PM discipline to SA discipline is done. Second, validation is performed by conducting a survey among professionals in CSA. The list of factors defined in the first part of this study is ranked by the SA professionals with the goal to identify factors that can most influence the effectiveness of CSA. To illustrate the application of the results, the readiness to implement CSA was evaluated in four case companies.

The rest of the paper is organized as follows. Section 2 describes the methodology used in this study. Section 3 presents why a cross-disciplinary approach is appropriate for this research work. Section 4 sheds the light on the knowledge regarding the role of the organizational context in the effectiveness of performance measurement system (PMS). In Section 5, knowledge from the PM discipline is extrapolated to the SA discipline and a context for CSA and a list of factors that might influence the application of CSA are defined. Section 6 presents the results of the case studies, in which an organizational context was evaluated with the goal to identify whether the case companies are ready to implement CSA, i.e., whether the implementation will be effective. Finally, a conclusion is drawn and possible directions for future research are suggested.

2. Methodology

In this study, a cross-disciplinary approach [23,24,25] was applied to identify factors that influence the effectiveness of the implementation and use of CSA. The methodology of the study is presented in Figure 1. First, after justifying the use of a cross-disciplinary approach, a thorough literature review was conducted to identify the factors that affect the effectiveness of PMS. Afterwards, the list of factors was adjusted to the SA discipline. Second, an online questionnaire was designed and sent to CSA professionals through the LinkedIn network. Third, interviews in the four case companies were conducted with the goal to identify the readiness of each organization to implement CSA.

2.1. Survey

Based on the list of factors that can influence the effectiveness of the CSA defined through the cross-disciplinary study, a questionnaire was designed to collect professionals’ views on the relative significance of factors. The five-point Likert scale method [26] was used to measure the degree of the significance of factors. Respondents were asked to rate the factors from 1 to 5, where “1” refers to “plays no role” in the effectiveness of implementation and use of CSA and “5” refers to “plays a great role”. To ensure the reliability and validity of the survey, the questionnaire was distributed to the professionals in CSA through the LinkedIn networking website. Since no similar study was done before and to receive a high response rate, the questionnaire was sent worldwide and to different sectors including manufacturing, consultancy, service, etc. The questionnaire contained two sections. The first section consisted of three questions regarding information about the respondents, i.e., country, type of an organization and his/her function in a company. Section 2 of the survey contained the list of 51 factors that can influence the effectiveness of CSA.

1009 professionals with more than five years of experience (according to their LinkedIn profiles) were invited to participate in the study. The invitation explained the purpose of the study and the knowledge required to participate in the survey. A total of 476 questionnaires were distributed after respondents agreed to participate in the survey. From 116 received questionnaires, 12 were excluded due to their inconsistency—e.g., some respondents ranked all factors the same, others analysed their companies according to the factors. Therefore, 104 responses were further analysed.

2.2. Case Study

An illustrative multiple case study [27] was used to illustrate the practical application of the research on the organizational context for CSA. Four Norwegian companies that either participated in research projects related to sustainability or proclaimed through the media that they work with sustainability were selected. All case companies are a B2B type of companies.

Table 1. Profile of case companies.

Company	Number of Employees	Turnover (Million Euro)	Type of Industry	Interviewed Functions
Company A	40	7	Production of plastic products	8 managers, covering functions: Production, Maintenance, R&D, Industrialization, Improvement, Logistics, Purchasing, Economy, HMS, Quality, CEO
Company B	169	45	Production of automobile parts	11 managers, covering functions: Tooling, CEO, R&D, Industrialization, Production, Maintenance, HMS, Quality, Logistics, Purchasing, Prototyping.
Company C	47	12	Production of equipment for fishery	8 managers, covering functions: CEO, Supply chain, HR, Marketing, R&D, Purchasing, Service, Sales, Production
Company D	77	27	Production of parts for the public utility industry	8 managers, covering functions: CEO, Marketing, Sales, Maintenance, Finance, Production, R&D, Logistics, Quality, Purchasing

represents a profile of each case company.

35 face-to-face semi-structured interviews were conducted with 8 to 11 managers in each company during the period from September 2016 to June 2017. On AVERAGE, each interview lasted for 1.5 h. Also, four meetings with the contact person from each company were focused on the background information about the company including the revision of the documentation as strategy, products portfolio, yearly plans, indicators, metrics, code of conduct, IT systems, etc. Moreover, a plant tour was conducted in each company that allowed observing the organization at work. Interviewing representatives from different departments allows addressing interviewees’ views and opinions based on his/her experience. Moreover, it ensures that consideration is given to the whole organization instead of the view of a single employee. Four companies from different industry sectors were deliberately chosen to introduce diversity into the sample.

All interviews were recorded and further transcribed. Data from interviews were analysed using NVivo 11 software (www.qsrinternational.com/nvivo). Both manifest and latent content were included in the analysis.

3. Why a Cross-Disciplinary Approach

A cross-disciplinary research is defined as sharing of information across disciplines [28]. In a cross-disciplinary research, resources (e.g., knowledge, methods, concepts, expertise and information) are borrowed from one discipline to serve the aims of the study in another discipline. Since no sufficient attention has yet been paid to the role of organizational context in the effectiveness of CSA, the choice was made toward the use of knowledge from the PM discipline that is similar to SA discipline. The two following sub-sections justify a cross-disciplinary approach for the purpose of this study. In other words, it is demonstrated why it is suitable to apply the knowledge from PM discipline to the research questions within the SA discipline.

3.1. Comparing the Purpose of PMS and CSA

PMS is defined as a balanced and dynamic system that can support the decision-making process by collecting, elaborating and analysing information [29]. It is also seen as a set of metrics used to quantify both the efficiency and effectiveness of actions [30]. Nelly et al. [30] present PMS as a combination of three elements: (1) the individual performance measurements; (2) the set of performance measurements; and (3) the relationship between the PMS and the environment within which it operates. PMS has multiple purposes: (1) assessment information can be used to identify problems and remedial actions and to focus attention on the critical processes [31]; (2) PMS can facilitate organizational learning [32].

SA is mostly defined as a process that directs decision-making toward sustainability [33]. However, due to the lack of the unified terminology, SA can be referred to as either a tool, a process, or a method [34,35,36]. Sustainability assessment can be applied to a variety of objects, e.g., SA of a product, process, organization, supply chain, project, policy, program, or plan. SA is viewed as a process rather than a tool when the assessed object is a policy, project, or program. When the assessed object is a product, process, or organization, SA can be viewed as either a process or a tool. Purposes of SA are defined by Waas et al. [37] as (1) information generation for decision-making; (2) structuring complexity; (3) operationalization and forum for participation, debate and deliberation; and (4) social learning. In addition, Moldavska and Welo [38] formulate the purpose of SA as “to support decision makers, facilitating the identification of actions that they should undertake in the attempt to contribute to sustainable development”. The authors also specified that SA should provide information about issues in the company that require improvements. When SA is applied at the organizational level, it is called CSA. CSA is sometimes argued to be a replacement for financial performance as the sole measure of corporate success [39].

In comparison to PMS, CSA is rarely seen as a system within the organization. Only lately, researchers have discussed “sustainability assessment system” that is more than an assessment process or an assessment tool. Strategic environmental assessment (SEA) as a system has been discussed by Bina [16], who argue that conceptualizing and exploring SEA as a system allows focusing on the relationship between planning processes, assessment and the context in which both are implemented. Furthermore, the definition of CSA in the organizations is given by Moldavska and Welo [38] as “a system that addresses [manufacturing] organization and provides valuable input to decision-makers regarding sustainable development”. The term CSA is used by authors to define any analytical technique (e.g., guideline, algorithm, software) to evaluate organizational sustainability performance and people involved in data collection, data analysis and communication with decision makers. After SA is implemented in the context system, this system, i.e., organization, will be changed since assessment will have some impact on the organization. Moldavska and Welo conclude that focus on the CSAS and its connections with exterior systems may increase the applicability of CSA. Thus, the authors raised the importance of the following questions: (1) what kind of changes may occur in the company due to the application of CSA; and (2) what does the company need to change in order to use CSA [38].

It can be concluded that both PMS and CSA have similar purposes and aim at supplying information for decision-makers regarding organizational performance (even though performance can be defined in different terms), facilitating identification of problems and actions to improve organizational performance.

3.2. Comparing Development of PM and SA Disciplines

Measurements in the PM discipline evolved from cost through productivity and quality to multi-dimensional measurements [40]. Nowadays, PMS includes multi-dimensional measurements, where “sustainability” is seen as one of the dimensions. With sustainability becoming one of the main concerns for stakeholders—e.g., customers, investors and governments—sustainability balanced scorecards were advocated [41]. PM can be used for different domains such as product, process, function (e.g., product development), organization, supply chain and individuals.

SA evolved from the use of solely environmental measurements to the integration of economic, environmental and social ones. The focus was also shifted from solely on the projects, programs and plans to additionally on the products, processes, organizations and supply chains. Thus, one can notice a proliferation of assessment tools with a focus on sustainability in the literature [42]. More than 40 types of assessment are presented within the SA discipline including sustainability assessment, impact assessment, life cycle assessment, integrated impact assessment, sustainability appraisal and integrated assessment [43]. Each of these types has its own extensive literature and body of research that hinders cooperation between the researchers.

When comparing the development of the two research disciplines, SA and PM, many similarities can be found. The summary of the mainstream themes in PM and SA research discipline is presented in

Table 2. Development of the PM discipline, based on [29,40,41,44,45,46,47].

Time Frame	Prevailing Research Focus
1970s	PM was first established as an independent discipline
1990s	Focus on multi-dimensional PMS (sustainability is one of the dimensions)
The mid-1990s	Focus on the design of PMS; what to measure? How to develop PMS?
The late 1990s	Distinguishing design, implementation, use and review of PMS
The late 1990s–Early 2000s	Implementation of PMS and use; how to implement and use?
2000s	Factors of the success of PMS & Context for the PMS; what makes PMS a successful PMS?
The mid-2000s	PMS as a dynamic, holistic system

and

Table 3. Development of the SA discipline, based on [37,48,49,50,51,52,53,54,55,56,57,58,59,60,61].

Time Frame	Prevailing Research Focus
1970	The introduction of environmental impact assessment
Early 1990s	Development of SEA
The mid-1990s	‘Sustainability Assessment’ became evident as the third generation of impact assessment (integration of economic, environmental and social aspects)
2000s–2010s	Focus on the purposes, principles and design of SA, indicators development
2000s–2010s	Development of generic frameworks for SA and case-based tools for different levels (product, process, system, organization, industrial sector, etc.), including CSA (organizational level)
The mid-2010s	“the integration problem” and “the implementation problem” are discussed
2010s	SA as a holistic and complexity-based tool, focus on the technical side

respectively.

The comparison of the development of the two research disciplines shows that they both have been following a similar development path. Both went through the research phases focused on the purposes, principles and design of SA tools and PMS. However, the SA discipline is behind the PM discipline in the application phase and soft aspects related to the application, i.e., an organizational context for SA, factors affecting the effectiveness of the implementation and use of SA. The SA discipline still focuses on the development of the assessment tools and methods. After the first wave of the development of the SA tools, shortcomings and challenges were identified by researchers. Now, the second wave of development focuses on designing new tools, more advanced and holistic. Morrison-Saunders et al. [43] argue that the assessment community “is more inclined to invent new tools rather than to modify existing ones” because it is more appealing than to address “the tool-used dilemma”, i.e., whether to use the original tool, whether to adopt it, whether to integrate it with other tools, or whether to develop a new tool. Moreover, Morrison-Saunders et al. see the growing problem in the fact that the assessment community is dividing itself into ever-smaller slices, thus increasing the risk of isolationism and confusion among both practitioners and the assessment community itself [43]. It can be argued that the silo-based research activities and the lack of a common vocabulary constrain advances in the implementation and use of SA. In contrast, PM practitioners have already conducted extensive research with the focus on the implementation and use of PMS in response to the prevalence of unsuccessful applications of PMSs in the early 2000s. PM practitioners have identified until now the variety of factors which influence both success and failure of a PMS [47]. After studying the similarities in the development of the PM and SA disciplines, it can be expected that SA researchers will turn their attention to the implementation and use phases and the organizational context for SA and CSA in the near future.

The comparison of purposes of both SA and PMS and a study of the development of both disciplines allow advocating the use of knowledge created in PM discipline to address the research questions defined in this paper. Thus, next section will present the knowledge created within the PM discipline, i.e., knowledge related to the effectiveness of PMS and factors that influence the effectiveness of PMS.

4. Organizational Context for PMS

Franco and Bourne [44] argue that often those who develop and implement PMS (i.e., collecting, storing and analysing information) have limited or inaccurate views on how people use information and knowledge in their work. It is not enough to develop an efficient PMS; it is also important to study the context in which this system will be used. Thus, a participation of employees in the development of PMS directly contributes to its effectiveness through the improved communication, knowledge utilization, increased job understanding, less resistance to change and commitment to the PMS [41].

The context for PMS has been described by D.R. Spitzer [62], who argue that a context (i.e., social and organizational aspects) for a measurement system not only determines the effectiveness of the system but is more important than the technical aspects. Figure 2 illustrates the context for PMS defined by Spitzer. It includes seven categories of factors that can influence the effectiveness of PMS.

The effectiveness of PMS can be measured by assessing the extent to which the desired outcomes of PMS have been achieved. The desired outcomes of the implementation and use phases have been identified in the literature before adjusting them to the CSA (

Table 4. Desired outcomes of the implementation and use of PMS and CSA.

Desired Outcomes of the PMS Implementation Phase Include [63]	Desired Outcomes of the CSA Implementation Phase Include
the system is regularly being used to monitor performance and to make decisions; the system is used by the management team regularly to discuss and manage business performance related issues; people see the value of using the system;	the system is being regularly used to monitor performance and to make decisions; the system is used by the management team regularly to discuss and manage business performance related issues; Employees see the value of using the system.
Desired Outcomes of the PMS Use Phase Include [41]	Desired Outcomes of the CSA Use Phase Include
PMS motivates performance; PMS assists in the achievement of goals; PMS is developing a performance-oriented culture; PMS supports change efforts; PMS provides useful performance feedback to employees; PMS implements the organizational strategy; PMS provides an accurate assessment of business (increase the quality of information relevant to decision making, evaluate the organizational progress); PMS ensures staff commitment to organizational objectives; PMS links individual performance to business unit performance; PMS develops individual’s skill and knowledge; PMS addresses the concerns of staff; PMS identifies talented employees; PMS rewards talented employees; PMS identifies poor performing staff; PMS manages poor performing staff; PMS facilitates the improvement of the organization as a whole; PMS highlights areas requiring improvement; PMS facilitates reliable forecasts	the system motivates sustainability performance; the system assists in the achievement of sustainability goals; the system is developing a sustainability-oriented culture; the system supports change efforts; the system provides useful performance feedback to employees; the system increases the quality of information relevant to decision making; the system evaluates the sustainability performance of an organization; the system facilitates the improvement of the organization as a whole; the system highlights areas requiring improvement; the system facilitates reliable forecasts; the system leads to changes in processes, actions and outcomes; the result of the assessment is considered to be worth time and cost involved; the system enables to reverse prevailing unsustainable trends; the system integrates all key intertwined factors affecting sustainability; the system seeks mutually reinforcing gains; the system minimizes trade-offs; the system respects the context in which assessment takes place; the system is open and broadly engaging; the system provides a forum for participation and debates; the system facilitates social learning; the system helps to structure complexity; the system facilitates the identification of actions to contribute to sustainable development

). Moreover, factors that influence the effectiveness of the PMS’ implementation and use phases have been identified through the extensive search of the review papers, see [29,41,44,45,46,47,63,64].

5. Organizational Context for CSA

5.1. Defining Organizational Context by Using a Cross-Disciplinary Approach

To define the context for CSA and factors that can influence its effectiveness, the effectiveness of CSA should be defined first. Bond and Morrison-Saunders [33] state that hundreds of academic papers discuss the effectiveness of SA and yet pluralism of SA complicates the consideration of effectiveness because different views exist on what effectiveness means. Bond et al. [55] propose a framework for evaluating the effectiveness of the SA process. The framework includes six categories of effectiveness: procedural; substantive; transactive; normative; pluralism; and knowledge and learning. Procedural effectiveness reflects whether a process follows institutional and professional standards and procedures. Substantive effectiveness represents how SA led to changes in process, actions and outcomes. Transactive effectiveness shows whether the outcome of conducting an assessment is worth the time and cost involved. Normative effectiveness indicates how assessment satisfies the list of imperatives: (1) reserve prevailing unsustainable trends; (2) integrate all the key factors affecting sustainability; (3) seek mutually reinforcing gains; (4) minimize trade-offs; (5) respect the context in which assessment takes place; (6) is open and broadly engaging. Pluralism represents how affected and concerned parties are accommodated and satisfied with the assessment. And the last category, knowledge and learning, shows how assessment facilitates instrumental and conceptual learning. Bond et al. [56] argue that existing SA tools, while often performing well, fall short when evaluated against these six criteria of effectiveness.

Six criteria of effectiveness developed by Bond et al. [55] have been developed for SA process, not SA system; and as literature review shows, criteria of effectiveness for CSA have not been researched yet. Therefore, using the knowledge created within the PM discipline about the effectiveness of PMS and research on the effectiveness of SA as a process, addressing the purposes of SA, we define the effectiveness of the implementation and use of CSA.

All factors identified for PMS have been adjusted to the sustainability aspect. This resulted in the list of organizational factors that can influence the effectiveness of CSA (

Table 5. Factors that influence the effectiveness of implementation and use of CSA.

1. Organizational Climate: Domination of reactive approach in an organization Level of processes formalization in an organization (tacit knowledge is prevailing) Resistance to measurement/assessment Culture that does not punish people’s errors Culture that encourages discussion and analysis around assessment Joined-up thinking at all levels of the organization (vs. a functional mindset) Employees have a fear of consequences Organization is oriented at ‘organizational learning’ A quality management culture Employee perception of assessment Employee involvement in assessment Focus on continuous improvement in an organization Common language in an organization Employee resistance to measurement/policy resistance Organizational maturity (level of organization’s readiness and experience with people, processes, technologies and consistent measurement practices) Employee empowerment Individual behavioural characteristics Organization has customer/stakeholder focus Active accountability/ownership is a part of organizational culture Use of early warning system/monitoring & evaluation Priority/abandonment of new initiatives Incentive programs in organization Incremental goal setting Understanding of purposes and benefits of assessment Internal interest groups (individuals for whom information from assessment makes a difference, who are willing to share responsibility for assessment, who can use information)	2. Resources for SA: Availability of human resources Availability of capital resources Availability of time Availability of informational/data collection capabilities Availability of knowledge/skills in sustainability assessment Availability of technical skills Information systems support (IT support) Relationship of assessment to daily responsibilities (embedding the assessment)
	3. Leadership: Management capacity Management style Leadership commitment to assessment and support Leadership skills/style Assessment is an initiative pushed by a parent company Action-orientation of the organization Legislative mandates Sustainability strategy
	4. History of Measurement/Assessment: Misconception of performance measurement Unsatisfactory results from assessment The perceived benefits of assessment Previous experience with measurement/assessment State of previous metrics/indicators
	5. Expectations from SA: Expectations from the use of assessment Expectations about the result of assessment
	6. Communication: Communication of expectations/strategy Dissemination of knowledge and results Visible use of the system and results

). The difference between PMS and CSA is taken into account during the definition of the factors that influence the effectiveness of CSA. Since CSA has obviously the focus on sustainability, two additional factors were added to the list identified for PMS, availability of knowledge on CSA, the existence of sustainability strategy. The list of factors is divided into six categories, i.e., organizational climate, the resources for CSA, leadership, history of assessment/measurement, expectations from CSA and communication. The factors can indicate either barriers or enablers to effective implementation and use of CSA.

5.2. Validating Organizational Context for CSA Using Online Survey

The main purpose of the survey was to rank the list of factors that potentially can influence the implementation and use of CSA. A total of 104 respondents from 21 countries participated in the survey, with the majority representing USA, UK, Australia, India and Norway. The profile of respondents is presented in Figure 3. The majority of respondents are from consultancy and manufacturing sectors.

The ranked list of factors that can influence the effectiveness of the implementation and use of CSA is obtained through the survey. The descriptive statistic is presented in

Table 6. Rank of factors that influence the effectiveness of CSA, where 1—plays no role, 5—plays a great role.

Factors	I Do Not Know, %	Average	1	2	3	4	5	6
Leadership commitment to/support of assessment	5.8	4.65
Organization has a sustainability strategy	2.9	4.41
Availability of informational/data collection capabilities	1.9	4.25
Understanding of purposes and benefits of assessment	1.9	4.25
Focus on ‘continuous improvement’ in an organization	6.7	4.05
Communication of expectations/strategy	4.8	4.04
The perceived benefits of assessment	3.8	4.03
Availability of knowledge/skills in SA	1	4.03
Availability of time	1	4.00
Culture that encourages discussion around assessment	3.8	4.00
Visible use of the system and results	3.8	3.99
Dissemination of knowledge and results	4.8	3.98
Availability of human resources	0	3.93
Active accountability/ownership—part of culture	3.8	3.93
A quality management culture	5.8	3.90
Joined-up thinking at all levels of the organization	6.7	3.88
Organization has customer/stakeholder focus	4.8	3.86
Relationship of assessment to daily responsibilities	2.9	3.80
Organizational maturity	1.9	3.76
Managerial capacity	3.8	3.75
Leadership skills/style	8.7	3.75
Legislative mandates	1.9	3.73
Internal interest groups	5.8	3.70
Management style	6.7	3.70
Expectations about the result of assessment	2.9	3.68
Assessment is an initiative pushed by a parent company	14.4	3.66
Expectations from the use of assessment	4.8	3.65
Previous experience with measurement/assessment	1.9	3.63
Organization is oriented at ‘organizational learning’	8.7	3.62
Action-orientation of the organization	4.8	3.62
Employees involvement in assessment	3.8	3.59
Incremental goal setting	6.7	3.59
Level of processes formalization in an organization	13.5	3.56
Employees perception of assessment	5.8	3.51
State of previous metrics/indicators	6.7	3.47
Common language in an organization	3.8	3.45
Availability of technical skills	0	3.43
Employees empowerment	3.8	3.43
Availability of capital resources	11	3.41
Resistance to measurement/ policy resistance	4.8	3.38
Use of early warning system/monitoring & evaluation	8.7	3.37
Resistance to measurement/assessment	1.9	3.36
Domination of reactive approach in an organization	8.7	3.35
Misconception of performance measurement	15.4	3.33
Incentive programs in organization	2.9	3.20
Priority/abandonment of new initiatives	14.4	3.18
Unsatisfactory results from assessment	12.5	3.15
Information systems support (IT support)	1	3.14
Culture that does not punish people’s errors	3.8	3.09
Individual behavioural characteristics	9.6	3.05
Employees have a fear of consequences	9.6	2.84

including (1) the number of “I do not know” answers for each factor as a percentage of all responses; (2) the average for each factor; and (3) the standard deviation presented by the graph, representing the consistency of the answers.

The result shows that some of the respondents did not have an opinion about certain factors. Factors that received over 10% of “I do not know” answers are (1) misconception of performance measurement; (2) priority/abandonment of new initiatives; (3) assessment is an initiative pushed by a parent company; (4) level of processes formalization in the organization (tacit knowledge is prevailing); (5) unsatisfactory results from assessment; and (6) availability of capital resources. The possible explanation is that respondents have not experienced the influence of the above factors on the effectiveness of the assessment. Another explanation is that these factors influence the effectiveness of assessment in some cases and do not influence in other cases.

The most significant factors, with the average over “4”, are the “leadership commitment to/support of assessment”, “organization has a sustainability strategy”, “availability of informational/data collection capabilities”, “understanding of purpose and benefits from assessment”, “focus on ‘continuous improvement’ in an organization”, “communication of expectations/strategy”, “the perceived benefits from assessment”, “availability of knowledge/skills in SA”, “availability of time’ and ‘culture that encourages discussion around assessment”. The ranking of some of the factors varies significantly; this is represented by the standard deviation. The top five factors regarding which respondents had significantly different opinions are “resistance to measurement/policy resistance”, “resistance to measurement/assessment”, “joined-up thinking at all levels of the organization”, “incentive programs in organization” and “culture that does not punish people’s errors”.

According to the survey, “leadership commitment and support” is the factor that plays the greatest role in the effectiveness of the implementation and use of CSA. The number of respondents stated that availability of human resources is “the result of leadership commitment to sustainable values” and commitment is what defines whether the resources will be allocated. Respondents stated that “availability of information/data collection capabilities” depends on the commitment as well. Commitment is defined as a key factor to the success, e.g., “in my experience top management commitment together with an adequate level of maturity within the company are the key factors for the success and effectiveness of a sustainability assessment”. Respondents recognize leadership commitment as the one that can be helpful to overcome existing internal barriers and a must-have for CSA. In addition, respondents stated that the ‘state of previous metrics and indicators’ defines how fast the patterns can be seen, that is important in order to demonstrate the management that “the assessment gives valuable information”.

“Sustainability strategy” and “availability of informational/data collection capabilities” are seen as other important factors. According to respondents, strategy provides “a roadmap” and “makes it easier to determine what to measure and how”, while the availability of collection capabilities is critical “to get the right information for an accurate assessment”.

While respondents rated “culture that encourages discussion around assessment”, “a quality management culture” and “culture of active accountability and ownership” with the average of 4, 3.9 and 3.9 respectively, they rated “culture that does not punish people’s errors” with the average of 3.09. Respondents mention that culture that encourages a discussion around the assessment is an enabler that “makes it much easier for a start” and “culture change is the main barrier to implementation” of CSA. One respondent stated, “the greatest obstacle in my experience, was implementing effective organizational change”. Moreover, a culture change effects the development of organizational maturity as well as incentive programs and reaction on the unsatisfactory results from assessment. It is also stated, “compliance culture does not breed embedded assessment”.

In general, respondents rated resources for SA in the following order: “availability of informational/data collection capabilities”, “availability of knowledge/skills in sustainability assessment”, “availability of time”, “availability of human resources”, “relationship of assessment to daily responsibilities”, “availability of technical skills”, “availability of capital resources”, “information systems support (IT support)”. Information systems support is at the bottom of the list; however, some mentioned that it is a “key to holistic transparency for integrated steering” and setting up a network is important for information sharing. Some respondents advocated outsourcing of assessment, thus removing barriers as a lack of knowledge/skills, time and human resources. Regarding the relationship of assessment to daily responsibilities (i.e., embedding the assessment), respondents stated, “it always helps to position assessments as relevant to both daily responsibilities and corporate goals” and it helps to get “the right information in a timely manner”. Moreover, it is argued that embedded assessment provides the sense of data ownership and “re-thinking, not additional thinking” makes people creative and increases capacity.

Although “joined-up thinking at all levels of the organization” has a relatively high standard deviation (1.19), respondents stated that it is “crucial for sustainable organization” and “it is not actually sustainability” without it. This result can be explained by the view of respondents on CSA, those who advocate for outsourcing of CSA and those who favour “in-house” CSA. The former would not see the effect of the joined-up thinking on the effectiveness of CSA in contrast to the latter.

6. Case Study

To illustrate the application of the research on the organizational context for CSA, four case companies were analysed in regard to whether the possible implementation of CSA will be effective. 16 factors from Table 5 were chosen for the analysis. The choice of the factors was defined by the following criteria. First, at least one factor from each category should be analysed. Second, since the case companies do not practice CSA, only factors related to the implementation phase should be chosen; therefore, factors from the “communication” category were not included in the analysis. Communication of the expectations from the CSA, dissemination of assessment results and visible use of the system and results can be analysed after the assessment system is implemented. The following factors from five categories were chosen: (1) “climate” category: domination of reactive approach, focus on “continuous improvement”, joined-up thinking, a quality management culture, employee perception of assessment, employee resistance to measurement, organization has customer/stakeholder focus; (2) “resources” category: availability of human resources, availability of time, availability of informational/data collection capabilities, availability of knowledge/skills in sustainability assessment; (3) “leadership” category: sustainability strategy; (4) “history” category: previous experience with measurement/assessment, state of previous metrics/indicators; (5) “expectations” category: expectations from the use of assessment, expectations from the result of assessment. The following sections present the analysis of each case company regarding the chosen factors and demonstrate whether the implementation of CSA will be effective.

6.1. Case Company A

6.1.1. Climate

Company A demonstrates a strong focus on continuous improvement. The strategy highlights that the company continuously develops and improves. This is accomplished by the use of a system for improvements, where all detected problems and potential improvements are registered by employees and which is managed by the improvement manager. The improvement manager coordinates projects related to solutions for existing challenges and improvements to various elements of the organization (e.g., products, processes, communication) by applying new approaches and ideas. The majority of managers demonstrate the interest in thinking ahead about improvements. They also stated that improvements are one of the most important aspects of their work. Moreover, the majority recognizes the value of performance measurements for continuous improvement, especially if PMS detects mistakes and negative performance.

The company works very closely with its customers during the whole life cycle of the product and is interested in “creating new knowledge with the customers together”. The managers focus on the creation of a strong bond with their customers through continuous follow-up. The interviewed managers described their main priorities as to understand the customers’ needs, to develop the right product and to deliver the product on time. Most of the managers indicated that they know that they did a good job when the customer is happy.

The characteristics of a proactive approach have been identified during the interviews. The fact that employees see the importance of learning from negative results is an indicator of the proactive leadership. The company is looking continuously for new ideas, new tendencies and possible future trends in research and business through the participation in a variety of conferences, workshops, research projects, etc. In general, managers perceive the company as a very innovative, e.g., “we are constantly looking upwards”.

In general, Company A demonstrates a joined-up thinking. The company is solution-oriented; therefore, managers see the cooperation between departments and interdisciplinary cooperation as pivotal to their success. A close dialogue with colleagues is named as one of their success factors. The relatively small size of the company, as indicated by everyone during the interviews, is one of the factors that contribute to the good communication in the company. However, it was mentioned that communication within some departments still has a potential for improvements, in addition to the challenges the company has with management and archiving the internal documentation.

All managers expressed a positive attitude toward the PM and saw it as something that can help them to be better and make better decisions. The majority prefers measurements at the department or team level. One manager stated that measurements at the individual level are only appropriate for the managers but not for the rest of employees. Some managers think that they need more leading indicators and that is the most important is that everyone understands what is behind PMS and why chosen measurements are vital. All managers stated that they already measure what matters most. Although the majority of managers stated that the company does not need a new measurement system (75%), they also suggested measurements that they think are important but which they do not currently measure. Only one manager strongly advocated a need for a better PMS. According to him, the company does not have a comprehensive understanding of how to measure performance in the context of sustainability and how to use sustainability measurements for both internal decision-making and external reporting. He stated, “there is nothing worse than just saying that we are sustainable without good arguments to prove it”.

The managers do not demonstrate the resistance to measurements and some of them stated that they like to see a negative result because it allows them to find improvements. However, a couple of issues that can potentially cause resistance are identified. First, some employees find it difficult to learn to use new software. Thus, if the measurement system will not be intuitive to use, it can cause a resistance from some users. Second, some employees find it unfear if their performance is measured by indicators that are influenced by others. In this situation, an employee loses the feeling of the control over his/her performance. Third, the majority of managers indicated that measurement at the individual level is perceived negatively by them and most of the employees.

6.1.2. Resources

All managers named time pressure as one of the main challenges in their work. In addition, some departments have too few human resources. The company has been in the process of implementing a new system for documentation and a new system for production, which should enable them to improve data collection—including the availability of data in real time—and automatize the reporting process. Regarding the knowledge on SA, the company does not currently practice CSA and does not have sustainability reporting. All managers are aware of the concept of sustainability due to the involvement of the company in a number of research projects related to sustainability. However, the knowledge varies from manager to manager and is limited to the basic ideas.

6.1.3. Leadership

The company does not have a formal sustainability strategy even though it is concerned with the quality of the natural environment (e.g., set a focus on the renewable sources of energy) and is continuously looking for new knowledge about sustainable production methods. As stated in the strategy, the company pays attention to the environmental impact of a product and recyclability and sustainability aspects. In addition, Company A is ISO14001 certified. The company has a long-term vision, i.e., “the important is not what gives us money for the next month but a stable organization that continuously develops and a long-term thinking”. All interviewed managers stated that it is important to introduce sustainable development in an organization, naming a variety of reasons such as “responsibility for future generations”, “everything affects everything”, “business should survive while society is satisfied” and “a long-term responsibility for society”. It was said that all employees in the company have to have a unified understanding of what sustainable development is if it is to be introduced in the company. Currently the managers associate sustainability with a wide range of issues as three pillars, environment (e.g., “respect environmental boundaries in a long-term”), effectiveness, recycling, closed loop, life cycle perspective, resource use, “it has to pay off for both company and society”, “to live long and make money”, short- and long-term economic sustainability, local community development, respect each other, wellbeing, responsible choice of suppliers. The majority stated that sustainable development is not solely about the environment, e.g., “if it is only environmental aspects, then all industry in Norway should be closed tomorrow”.

6.1.4. History

The company does not have a unified PMS, i.e., different managers are responsible for different measurements and not all managers could name what is measured in other departments. The majority of managers stated that they have a good overview of the organizational performance due to the small size of the company, thus they do not need a unified PMS. However, there is a slight mismatch in what different managers said about what is being measured in the company. Officially Company A has 13 measurements that are based on the company’s goals and are chosen by the management based on experience. These measurements can be divided into two groups, those required by the regulations—e.g., sick leave—and those related to the problem areas. The measurements were updated one year before the interviews and will be used to create a plan for the next year, to prioritize improvement projects and in addition to being a basis for salary. Moreover, some managers have additional measurements that they use internally in their departments.

6.1.5. Expectation

Seven managers had clear expectations of a possible assessment system including (1) the use of language that employees, auditors and customers can understand (e.g., include explanation of terms); (2) possibility for feedbacks between departments; (3) a user friendly system; (4) information received from the system should be used for improvements; (5) system should provide arguments “if we say that we are sustainable”; and (6) automatic data collection from existing systems, without manual work. Although managers stated that it is good to measure something that enables improvements, e.g., “something useful”, most of them could not say which measurements that what would be useful. Moreover, one manager stated that since employees on the production floor often do not understand what key performance indicators are, for a new system to have a positive impact, it should include only a few measurements that everyone can understand.

6.2. Case Company B

6.2.1. Climate

All interviewed managers indicated that continuous improvement is a part of their work, e.g., “we constantly try to be better: if it goes well we want to be better, if it goes bad we want to be better”. Company B uses data analysis to identify problems and potential improvements. This is the reason why the half of the managers sees the important role measurements play in the continuous improvement. However, according to three managers the lack of resources—time and competence—is the biggest barrier to a thorough analysis, deeper problem solving and innovation outside of the customer-oriented projects.

Although some managers stated that it is important that the company is seen as innovative, they also indicated that they do not have time for innovation and “to look a little more into the future”. Thus, the current approach can be defined as more reactive than proactive.

None of the managers named customers as the most important aspect of their work. However, this does not imply that the company does not seek customer satisfaction Customer satisfaction is rather seen as a means to the economic success, i.e., some managers see the effect of their work on the customers and try to ensure that customer gets the right product at the right time since “it affects the economy”. As one of the managers stated, “the most important is to deliver the right quality of work at the right time, do not overdevelop or underdevelop, i.e., fulfil customer requirements but just that, nothing more and nothing less”.

The majority of the managers stated that they have a good cooperation within the departments; however, almost all of them indicated that the cooperation between the departments is not sufficient due to the existing management structure and many strong personalities in the company.

Most of the managers stated that the company has a lack of the link between organizational vision, strategy, goals and measurements for each department. They indicated that organizational goals are not broken down into the goals for each department and some managers stated that current measurements for their departments are meaningless and do not represent their true performance. In addition, employees think that they are measured by things they cannot control or influence directly. If the measurement is too general as “to deliver at the right time”, then “someone else always can be blamed”. One of the managers stated that the challenge the company has is to get all employees to understand why the company has the measurements it has. The majority of the managers argued for a need to change their current measurement system: some advocated for fewer measurements than what is currently used, others suggested more measurements. Even those who stated that their current system is “good enough”, said that they need a better structure. Although managers stated that measurements should be meaningful and useful, most of the managers do not know which measurements are useful. All managers advocated for measurements at the department level instead of the individual level but one manager suggested that in the case of bad performance in the department, measurements at the individual level could be used to identify the reason.

6.2.2. Resources

All managers indicated that one of the main problems they have is a lack of time and this affects the quality of their work and the ability to innovate and causes the need for overwork. During the time of the interviews, the company was working on the implementation of a system for data management. The company has a variety of systems, e.g., Enterprise Resource Planning (ERP), Electronic Data Exchange (EDE), Enterprise Asset Management (EAM), which are currently not connected. In addition, the company does not have an IT department and IT support is provided by their parent company in another country. This affects the efficiency of the IT systems and the interface between them. Some managers stated that the current organizational data collection capabilities are extremely insufficient. Company B does not have any experience with a formal CSA.

6.2.3. Leadership

The company does not have a clear sustainability strategy; however, their mission includes “to ensure strong, competitive and long-term sustainable supply chain”. The interviewed managers have different associations with sustainable development, e.g., do not harm the environment, do not harm people, to be an attractive company, people willing to work in the company, employees are motivated, long-term thinking, environmental friendliness, reuse, recycling, less material and energy use, future generations, a company has a future, reasonable goals, profitability, thinking smart, environmental efficiency, conditions for employees, long-term effects, green value chain, economically sound foundation, renewable, support local suppliers, recyclability and pollution. The managers also stated that (1) working for SD is to avoid fast decisions, to think thoroughly about consequences, to think ahead and plan, to not hide things and to be honest; (2) SD means to look at the overall picture, consider all aspects, including economic and environmental; and (3) SD thinking allows to uncover problems earlier and avoid potential disasters. Even though all managers stated that it is important to introduce SD in an organization, their everyday focus is still mostly on the cost, e.g., “I think the price beats the environment”, even though the company is ISO14001 certified.

6.2.4. History

Company B is a part of an international business company; therefore, measurements are developed based on the objectives set by the management in order to achieve the goals of the parent company. Every year departments receive the list of goals and measurements but some managers said they do not have the overview of all the measurements in the company. Moreover, not all managers know what other departments measure or how often working environment and employees’ satisfaction is evaluated. Also, while measurements in the production department are used for a root-cause analysis, organizational measurements, are mostly used for status reports, management meetings and bonus system.

6.2.5. Expectation

The managers’ expectations for a possible new assessment system for measurements include: (1) a system can evolve, e.g., it is possible to add or change measurements; (2) a system covers all vital processes in the organization, i.e., the holistic system that includes everything―“snowplough instead of snow shovel”; (3) a system helps to avoid sub-optimization; (4) measurements should help to improve and to increase quality of both processes and products; (5) less subjective measurements; (6) data provided by the system should be trustful; (7) meaningful measurements that cover what is important for the company, i.e., “company is innovative and cooperative to the customers and produces with profitability”; (8) a user friendly system that can be used for data analysis to identify improvements and causes of problems; (9) a system that collects data automatically from existing systems; and (10) measurements represent feedback between different factors.

6.3. Case Company C

6.3.1. Climate

The top management in Company C focuses on “being always in the forefront and developing new products, then competitors are always behind”. The company’s philosophy is, “if you see the opportunity to make things easier or better, share it with the R&D department”. Also, the company has a system for registering problems and projects for improvements. All interviewed managers demonstrated the focus on continuous improvements and described improvements as a part of their daily work.

The company demonstrates the strong focus on the customer satisfaction and customer loyalty. Company C keeps a close relationship with customers through continuous support, follow-ups and social activities, which are organized regularly to integrate customers with employees.

In general, interviewed managers either do not have experience with formal PMS or do not see a need in a formal measurement system. Since the company is relatively small, all managers stated that they have a good overview even without a formal system in place. Therefore, it is more likely that the managers will demonstrate resistance if they do not see the benefit from a formal measurement system.

The company demonstrates the domination of proactive approach regarding product development and finding better solutions for existing and potential customers. In addition, they also invest in new equipment and new technologies as a part of their future-oriented thinking.

Due to the small size of the company and since all departments are located in the same building, all managers indicated that they have good communication and cooperation between and within the departments. In addition, the leadership is focused on the social activities as a means to a team building; as a result, employees see the company as “a gang”. The choice to have everything “in the house” is caused by giving priority to delivery time and quality of the products and processes. Despite the priority given to the quality, the company does not have a quality manager position. The top manager presented it as “everyone in the company is responsible for quality” and they “do not control quality but build in quality”. However, some of the managers expressed concern over the lack of a formal quality control. In addition, the company has challenges with formal routines, information flow and coordination between some departments.

The top management expressed a positive attitude toward measurements since “what measures gets done”. The company, in general, has little documentation and formal vision, mission and strategy and most of the managers do not see a strong need in a formal measurement system. It was explained by the small size of the company where everyone gets a feeling of a good overview and understanding of the performance without formally measuring it. Part of the managers think they measure what matters most, while couple of managers think that the current system is sufficient for now but probably has to be more formalized since the company is growing. Most of the managers see economic measurements as the most important and one manager stated, “there is no point in measuring more if you do not have resources to do anything about it anyway”. Apart from the managers that favoured measurements at the individual level, the majority advocated for measurements at the department level.

6.3.2. Resources

The majority of the managers stated that the lack of time and need to multitask and prioritize are their main challenges. Regarding the data collection, some managers stated that they already have a lot of data available in their systems—ERP and CRM—they just do not currently use it. Moreover, some managers also see the CRM system as an example of PMS. The company does not have experience with sustainability assessment and has no knowledge about it.

6.3.3. Leadership

Company C does not have a sustainability strategy or even a formal strategy. Only top management showed the knowledge about the strategy while other managers stated that the company does not have one. Although the company does not have a formal sustainability strategy, they position their product as one that contributes to a sustainable fishery. The managers associate sustainability mostly with environmental―e.g., environmental friendly product, emissions and resource use, waste management, life cycle perspective―and economic issues―e.g., company last long, organizational profitability and surviving, jobs creation. In addition, one of the interviewed managers does not know about the sustainable development concept. Only a few managers see a potential in introducing sustainable development in the company, while others see it (1) as something extra that “the company cannot prioritize now” because they have other challenges; or (2) as something that is already a part of the company because they “survived through tough times”. The managers would introduce SD solely as a marketing argument, since customers can be interested in sustainable product.

6.3.4. History

The company has mostly economic measurements or measurements that are required by the government, e.g., a sick leave. Some managers stated that the company has five measurements for tracking the “pulse of the company”; other managers do not know or do not remember what is measured. In addition, some managers have an impression that the company uses strategically only economic measurements. This can be explained by the management style, i.e., the top manager is involved in all activities and is the one who analyses data and makes strategic decisions. Therefore, since all is done by one person, other managers do not feel like these things are taking place in the company. Measurements are used to report to the board and as a basis for bonuses, which are linked to the economic result of the company.

6.3.5. Expectations

The majority of the managers do not think the company needs to change the current system; therefore, they have no expectations to a possible assessment system. The lack of expectations can also be explained by the fact that most of the managers do not have experience with formal PMS and do not know “how it can be”. Only some managers suggested that the use of measurements as a benchmark with competitors can be a useful application of an assessment system and the measurements that lead to sub-optimization should be avoided. In addition, one manager stated that there is a link between the use of an assessment system and efficient identification of improvement opportunities.

6.4. Case Company D

6.4.1. Climate

All managers demonstrated the focus on continuous improvement, e.g., “We never say that we are satisfied and will not change. The day we do not want to change, we fail.” The managers discussed the improvements in competence and technologies, processes and products, materials and solutions. Moreover, the company has a system for registering problems and improvement projects and the majority of managers stated that a root case analysis is a part of their daily work. However, some managers stated that they “mostly work on extinguishing fires, than having time to work in advance, work ahead and be prepared, work proactively”. One manager said that they are good at looking at the data but “what we are not good at is continuous improvements and improvement initiatives”.

The company prioritizes customers experience with the company, i.e., providing support in a reasonable length of time, delivering an expected product quality and delivering product in time. However, the couple of managers are dissatisfied with the necessity to balance between satisfying customers and satisfying the parent company, which has mostly economic interests.

Although the majority of the managers advocated the importance to have a log-term thinking, the lack of time is often what leads to the domination of a reactive approach in the company, e.g., “too little time to work proactively”. The managers stated that planned improvement projects could be disrupted by small problems that always appear.

Company D demonstrates a tendency to a joined-up thinking, i.e., all managers indicated that they have good cooperation through the meetings and work regularly in interdisciplinary teams. The flat organizational structure also contributes to a sound cooperation between departments. However, that the parent company focuses solely on the economic results can be seen as a barrier for a joined-up culture. Some managers think that while they safeguard product quality, the rest of the organization prioritizes on reducing the cost. The managers stated that they sometimes have to balance between the interests of the parent company, by delivering the expected economic result and a long-term partnership with customers, by prioritizing customers interests, quality and continuous improvements. In addition, a misunderstanding between engineers and non-engineers was named as another challenge.

In general, all interviewed managers have a positive perception of measurements, stating that measurements help to improve and to ensure that everyone moves in the same direction, e.g., “if you choose not to measure, you choose not to know”. Some managers even think that they measure too much and should instead be better in taking actions based on the measurements, while others suggested a list of additional measurements that could be helpful. The managers think that measurements should be meaningful and give value. Thus, some managers stated that there is no value in too much reporting to the parent company―“a waste of time and money”. They also think that the parent company prioritizes the short-term results over the long-term performance. A couple of managers feel a need to balance between measurements centred on the customer and the bonus criteria, e.g., delivery time vs. inventory cost. In addition, managers think that measurements should be clearly communicated to all employees and that employees should be measured by something they feel they can influence directly. The majority of managers think that measurements at the individual level are a stimulus for performing better.

6.4.2. Resources

Only a few managers stated that their main challenge is the lack of time. In addition, the company either already collects a variety of data (including economic, technical, environmental and social) or has the possibility to collect more using existing systems. The company is also currently implementing systems for improving the quality of data collection.

6.4.3. Leadership

Company D does not have a sustainability strategy. The company is ISO14001 certified; however, data that are collected in relation to the certification are not seen by some managers as strategic data and are not usually taken into account when strategic decisions are made. Although R&D department is thinking about the products’ end of life and the company has recycling programs, environmental and social issues are not seen as the indicators of organizational performance. Sustainability is associated with a range of issues like development of technologies to increase productivity, environmental friendliness, long life time, healthy, next generation, responsibility for society, profitable company, paying tax, securing working places, recycling, connection between environment and economy, long-term, surviving, develop the company and exist in a good environment in the future, holistic, resources use, to be reasonable in relation to environment, employees wellbeing, reduce energy loss, reduce wastes of resources, life time of the product, the whole cycle, energy use, CO₂ footprint, design for recycling, generate value, money and jobs. The named reasons to introduce sustainable development in the company include “important for the company to continuously develop and make money,” “it is a sales argument,” “to think long-term and create work places”, “to see the bigger picture” and “serious company should be environmentally conscious for future generations”. One manager stated that it is possible to introduce sustainable development “if the customer is willing to pay”. It was also mentioned that it is important that each employee knows what this big word [sustainable development] means for his or her work.

6.4.4. History

The company does not have a unified measurement system. Since the company has a strong focus on the economic results, economic measurements are known and discussed in the company, e.g., “we have the focus on the economic data”, in contrast to other measurements that are not well known outside of the departments which are responsible for them. Data are used for reporting to the parent company and to the customers (reports are different because interests of the customers and the parent company do not always match), to find the reasons the problems or negative performance, for prioritizing improvement initiatives and for bonuses calculation. Although the company has status meetings with all employees, some managers stated that data usually stop at the upper management meetings and are not distributed down in the company. The company is working on the automation of data reporting and the improvement of data quality—that was a problem before, because some employees registered wrong data.

6.4.5. Expectations

Two managers do not see the need for a new assessment system and one of them is strongly against the system that could combine different types of data (e.g., economic, environmental). The rest of the managers would expect the new assessment system to include social measurements, to be visual and not complex, to depict relations between data, to present data in real-time and to assist in identification of areas of improvements.

6.5. Readiness of the Case Companies to Implement CSA

The goal of the case study was to identify how organizational context can potentially influence the effectiveness of the CSA implementation. In other words, whether the context of case companies can enable or hinder implementation of an assessment system. The effectiveness of the implementation is defined as whether CSA is regularly used to monitor performance and to make decisions; whether CSA is used by the management team regularly to discuss and manage business performance related issues; and whether employees see the value of using CSA. A summary of the organizational context for each case company is presented in

Table 7. Summary of the organizational context for CSA in case companies.

Company A	Company B	Company C	Company D
Climate
Strong focus on continuous improvement, seeing a value of negative results of PM for improvements. Strong focus on the customers. Looking for new ideas and innovation-oriented, proactive approach. Good cooperation between departments and see the value of interdisciplinary cooperation. Positive attitude toward measurements. Do not see a need for a new measurement system. Difficulties with new software. Resistance to measurements that cannot influence.	Strong focus on continuous improvement, use data for identification of improvements. Lack resources for improvement projects. The traditional view on the customers, i.e., satisfy the minimum needs to get profit. Value innovation but lack time.Weak cooperation between departments. Weak link between strategy, goals and measurements. Some measurements are seen as meaningless. See a strong need for a new measurement system. Resistance to measurements that cannot influence.	Great focus on continuous improvements. Strong focus on the customers. No experience with formal measurements. A proactive approach to product development. Close communication within the company. Mostly positive attitude toward measurements	Focus on continuous improvements but lack time for improvement initiatives. Prevailing reactive approach. Balancing customer satisfaction and a parenting company interests. Challenges in cross-department joined-up thinking. In general, positive attitude toward measurements but partly negative toward reporting to parenting company.
Resources
Possibly limited data collection capabilities. Lack of time.	Possibly limited data collection capabilities. Lack of time.	Lack of time. Limited data collection capabilities.	Potentially good data collection capabilities.
Leadership
ISO14001 certified. Positive attitude toward sustainable development. Have basic knowledge about sustainable development. No sustainability strategy. Leadership is sustainability-aware.	ISO14001 certified. See a value in sustainable development. Have some associations with sustainable development but prioritize economy. No sustainability strategy.	No ISO14001 certification. No sustainability strategy or any formal strategy. Very limited knowledge about sustainable development.	No sustainability strategy. ISO 14001 certified. Positive attitude toward sustainable development but mainly from economic benefits.
History
Distributed responsibility for performance measurements. Do not have an overview of all measurements in the company.	Distributed responsibility for performance measurements. Do not have an overview of all measurements in the company. No systematic work with performance measurements.	No formal measurement system No measurement culture.	Distributed responsibility for performance measurements. Do not see value in the integrated measurement system.
Expectations
Expressed clear expectations for a possible assessment system.	Expressed clear expectations for a possible assessment system.	Almost no expectations to assessment system.	Some expectations to a possible assessment system. Partly not seeing a need for a new system, or being strongly against it

.

By analysing the organizational context of Company A, it can be concluded that a new CSA has the potential for being used by the management team to discuss and manage performance. However, lack of time and not seeing the need for a new system can be a barrier. There are signs of the possibly regular use of the system by the management team if the company will see the benefits gained from the assessment system. It is most likely that employees will see the value of using the system if it helps them to identify improvements and improve their sustainability image.

A strong recognition of the need for a new assessment system, clear expectations and a positive attitude toward sustainable development can be great facilitators for the implementation of CSA in Company B. The barriers to an effective implementation are the lack of time, focus on the economic issues, weak cooperation between the departments and a lack of systematic work with measurements. These barriers can hinder the use of CSA by the management team. In order to improve the organizational context, sustainable development concept can be introduced in the company.

The analysis of the organizational context in Company C indicates that it is most likely that CSA will not be effectively implemented. There is no history of a formal measurement system, no experience with formal assessment systems among most of the managers, no expectations to a possible system, lack of time, lack of data collection possibilities, limited knowledge in sustainable development. These indicate that CSA might not be used to monitor performance and not being used to discuss performance by managers. It also seems that employees will not see a value of CSA. This can be explained by the small size of the company and informal management style.

Company D might not have the potential to implement CSA effectively due to strong influence by the parent company (which prioritizes economic benefits solely), almost not seeing a need for a new system and not seeing any value in an integrated system, challenges in joined-up thinking, lack of time in general and specifically for improvements and a proactive approach. To enable an effective implementation of CSA, the communication between departments should be improved and the concept of sustainable development should be introduced in the company.

7. Conclusions and Future Work

Even the best technical solutions do not guarantee that they will be beneficial for a company. A role of the social component is recognized in different fields, from IT to manufacturing. User resistance is seen as a primary factor that prevents users from adopting new solutions, innovative systems and novel technologies.

Much of the literature to date about CSA has focused on the mechanics of measurement with limited emphasis on the softer issues—a context for assessment system and factors influencing the effectiveness of CSA. The failure to recognize the role of organizational context in the application of CSA might hinder the potential of CSA as a decision support tool that leads decision-making toward sustainability. Thus, an understanding of the organizational context will allow practitioners to assess better a company’s readiness to implement CSA.

Using a cross-disciplinary approach and online survey, factors that define organizational context for CSA have been proposed. Some of the factors can have a great impact on the effectiveness of the implementation and use of CSA; it can either enable or hinder the effectiveness of CSA as a decision support tool. Therefore, before implementing an assessment system, it is important to evaluate whether the organization has a favourable context for it, i.e., the readiness of the organization to implement and use the system. If it shows that some of the factors that can influence the effectiveness of CSA are negative, organization can first improve the context and resolve the barriers instead of investing in a system that will not be effective due to, for example, employees’ resistance to assessment, lack of knowledge about sustainable development, or lack of leadership commitment to assessment.

In this paper, it is suggested that the focus of SA research should be expanded from the design to implementation and use of CSA. Moreover, there is a need to conceptualize CSA as a system that will allow focusing on the relationship between assessment and the context in which it is implemented. A multiple case study demonstrated that even before implementation of CSA, the early warning signs, which indicate that a current organizational context might negatively influence the effectiveness of CSA.

One limitation of this research is the number of respondents in the online survey. However, due to the lack of the previous research on this topic, this study serves as exploratory research. A broader survey (the number of respondents) and in-depth interviews are required in the future in order to increase the validity of the study. Moreover, the length of the questionnaire, which included the list of 51 factors, could influence the number of finished questionnaires and the scrupulousness in the answers. The measures to improve the quality of the study were (1) a check of the work experience of the invited participants; (2) an evaluation of the received responses, i.e., consistency of the answers; (3) an inclusion of the answer “I do not know” to avoid a forced answer in case a respondent did not have experience with some factors.

The possible future research direction can be to study how to create an organizational context where CSA can be effective. In addition, further research work should include in-depth case studies of companies that use CSA on a regular basis. Such empirical data will allow testing a theoretical list of factors defined in this work.