2.1. Selection of Areas and Performance Indicators
We have developed the sustainability assessment framework as operational part of the sustainable development strategy with the support of performance indicators.
Application of performance indicators to VET is a phenomenon imported from the higher education sector [
14]. In the field of VET, it is employed by the European Quality Assurance in Vocational Education and Training (EQAVET) quality assurance reference framework [
15], which operates by means of 10 indicators with the main objective to determine the impact of unemployment. In fact, this is the single framework operational for VET providers, which has one area and makes no reference to sustainable development.
In the research, we had to decide the areas of focus in the new proposed framework, the format for description of indicators as well as their content.
In the first phase of the development, we started with establishing the areas of the new framework by collecting relevant ideas from the literature review in order to keep a significant and causal positive relationship between a good institution and sustainability, as demonstrated by Stoever [
16].
In this stage, we have considered the synergies between quality and sustainable development in an organizational framework described by Isaksson [
17]; the capital-based frameworks as part of the statutory responsibility that promote the social, economic, environmental and cultural well-being of their communities [
18]; and the many perspectives for implementation described between the extremes of the ecological sustainability perspective described in the international standard for environmental systems [
19] and the well-being perspective [
20].
Among the so-called three “pillars” of sustainable development, it is necessary to introduce a fourth dimension—institutional component, as demonstrated by Zdravkovic and Radukic [
21], due to the fact that sustainable development cannot be achieved without governance because of its nature: it is normative and requires collective action.
Having analysed the previous conceptual components of a framework evidenced in literature: quality, social, economic, environmental and institutional, in the new developed framework we considered merging institutional capacity with quality management, and adding a fifth component related to training as a core area of a VET organization. As a result, the new developed sustainability assessment framework for VET organizations helps orienting within the following five areas:
institutional capacity and management;
environmental responsibility;
economic performance;
social responsibility; and
training provision.
These areas are seen as key pillars a VET organization will have to consider for both implementing and continuing a sustainable development approach, by interconnection of economics with environmental issues that are of great significance.
In the second phase of the development, we have selected performance indicators in order to cover as wide as possible areas, but also to have a limited number of indicators.
The criteria for selection of sustainable development indicators were built from the basic functionality of the indicator, which is to transfer relevant information from physical space to users through communicable data [
22]. For this purpose, a qualitative study of five case study VET providers was undertaken. The five VET providers represent the landscape types selected because each exhibited a cross-section of characteristics—such as declining, growing and stagnating—as well as differing levels of human capital.
Gathering quality and sustainability approaches developed by each organization was proceeded in a comparative analysis in order to achieve a common reference framework.
We examined how the VET providers implemented the quality assessment for sustainable development and we discovered different approaches to help other VET organizations find a common way to implement sustainable management. The key aspects that informed the methodology are:
- ▪
The vision/mission/goal related to sustainability assessment;
- ▪
Previous experience of the organization;
- ▪
Institutional context;
- ▪
Key issues for “implementation to governance” of a sustainability assessment towards a sustainable quality development (this should describe how organization started, how they implemented it, how they run and manage it, who is involved when/why, what measures have been taken, etc.); and
- ▪
Key success factors.
In addition, from the literature review, in describing the performance indicators, we have used the integrative management concept elaborated by Jabareen [
23], but also the 16 elements contributing to the framework of sustainable technical and vocational education: creativity, innovation, networks and partnerships, staff development program, teaching methods, generic skills, industrial relations and internships, counseling, entrepreneurship, ICT skills, interest, recognition, knowledge, competency based training, articulation, and commitment of management revealed by Minghat
et al. [
24].
Finally, the new and innovative sustainability assessment framework was designed, with the main objective to determine the impact on the environmental, economic, and societal quality of the training system.
The new developed framework is composed of fivve key pillars: institutional capacity, environmental, economic, social, and training provision, which interact in the context of globalization of training, as presented in
Figure 2. Thus, all five areas are considered in their mutual interaction and dependency.
Figure 2.
Interaction of the sustainability framework areas.
Figure 2.
Interaction of the sustainability framework areas.
In the five areas of the new sustainability assessment framework, a total of 40 performance indicators are used to make the assessment. They are designed to cover as wide as possible areas.
2.2. Sustainability Assessment Framework Content
The purpose of the innovative sustainability assessment framework is to assess VET organizations institutional sustainability in terms of the five key pillars (areas): institutional capacity, environmental, economic, social, and training provision.
The map comprising the areas of the sustainability assessment framework and the corresponding performance indicators are presented in
Table 1, in which the four phases of the continuous improvement cycle are bordered.
Figure 3 demonstrates the allocation of indicators from the five areas of the framework on the four phases of the continuous improvement cycle.
Figure 3.
Continuous improvement cycle for the VET organization sustainable development.
Figure 3.
Continuous improvement cycle for the VET organization sustainable development.
The sustainability assessment framework is based on the Plan-Do-Check-Act (PDCA) cycle and consists of four phases:
setting of policy goals/objectives and planning phase;
implementation phase;
evaluation phase, which deals with the design of the mechanisms for evaluation and the assessment of achievements/outcomes at individual, provider and system levels; and
review, based on a combination of internal and external evaluation results, processing of feedback and organization of procedures for change.
Table 1.
Sustainability assessment framework areas and performance indicators.
Table 1.
Sustainability assessment framework areas and performance indicators.
1. Institutional Capacity and Management | 2. Environmental Responsibility | 3. Economic Performance | 4. Social Responsibility | 5. Training Provision |
---|
1.1. Strategy and planning mechanism 1.2. Management system | 2.1. Bio diversity | 3.1. Design and development of VET delivery | 4.1. Community involvement and development | 5.1. Design of training provision |
1.3. Responsibility 1.4. Internal and external communication 1.5. Facilities 1.6. Logistics 1.7. Staff participation 1.8. Participant access | 2.2. Water consumption management 2.3. Non-renewable energy consumption 2.4. Consumption of raw materials | 3.2. Policy of employment and wages 3.3. Marketing and sales of services | 4.2. Equal opportunities 4.3. Social relation in organization 4.4. Fair operating practices | 5.2. Training contents 5.3. Training delivery 5.4. Training resources |
1.9. Relationship with stakeholders 1.10. Supply chain 1.11. Legislation awareness 1.12. Risk management | 2.5. Water pollution 2.6. Soil management and pollution 2.7. Air pollution and greenhouse effect | 3.4. Investment | 4.5. Health and safety 4.6. Consumer issues | 5.5. Citizenship and training |
1.13. Fundamental rights 1.14. Internal audit | 2.8. Waste | 3.5. Economic performance | 4.7. Training and education | 5.6. Training review |
2.3. Performance Indicators Format
In order to enable assessment for each indicator, there are five performance categories: awareness, measures, action, innovation and excellence and five importance categories.
The assessment process is based on the novel approach of “couple values” of each indicator’s performance and importance, both evaluated on specific scales from 1 to 5.
To help understand the above-mentioned areas and indicators, the descriptions below in
Table 2 exemplifies the format in the case of the “Water consumption” performance indicator (code 2.2) from the environmental area, which is composed of “Indicator description”, “Environmental aspects to check” and the couple criteria (Performance.Importance).
For a more synthetic presentation of the other seven indicators from the “Environmental responsibility” area, the following is a summary of: the description, environmental aspects to check and performance criteria for the five rankings: (1) Becoming aware; (2) Measures; (3) Setting up actions; (4) Incomplete mastery of innovations; and (5) Excellence/model behavior.
Non-renewable energy consumption (code 2.3): Lightning and heating as well as specific training activities using energy are elements to be taken into account in energy consumption management. These elements are: information, by type of activity or by sector, and its own consumption; prevention actions and individual and collective sensitization towards staff and learners; being aware of energy needs regarding premises, transport, materials and other kind of electronic equipment and the actions developed in order to optimize them; and using renewable energies.
All organizations are concerned with this indicator; whatever we think of lighting, or heating for some of them, the control of energy consumption is a collective topic to think about.
Environmental aspects to check: information about the energy needs; the cost of the consumption; activity or uses that are the most important consumers of energy; follow up of energy consumption settled in the organization; decisions or measures taken in order to manage and control the energy consumption (control and reduction of heating, hot water temperature, lighting, double glass windows, etc.); and how the organization raises staff members and trainees’ awareness.
Performance: (1) Becoming aware: No management nor control at all of energy consumption. None or very little knowledge about types of energy consumed; (2) Measures: Existence of a detailed report of consumption and evaluation of costs sharing by location and activity; (3) Setting up actions: Actions are implemented in order to reduce energy consumption. Actions are set up to raise and develop staff and trainees’ awareness on saving energy; (4) Incomplete mastery of innovations: An action plan has been settled and formalized in order to reduce energy consumption (including a quantitative objective for consumption reduction). It has been communicated towards staff members and training participants. It includes measures/means that enable reducing consumptions but also optimizing needs (isolation of premises, using systems of low consumption, etc.). Surveys and analyses on premises allow considering what to do in order to optimize energy consumption and use renewable energies; (5) Excellence/model behavior: Building and construction of projects, purchasing or renting new premises taking into account energy consumption aspects (premises certified as being “low consumption”). Concerning type of energy, preference is given to renewable energies.
Table 2.
Performance indicator format.
Table 2.
Performance indicator format.
Indicator Code | Water Consumption 2.2 |
---|
Indicator description | Water has becoming such a rare good that it needs better consumption management by individuals as well as by organizations. To be able to observe better and more efficient consumption, an organization must know its level of consumption, types of use, the origin of water consumed, communication actions developed towards staff members and learners. |
Environmental aspects to check | What is the water resource origin; the water consumption and the amount of the invoice; the uses and sectors most concerned by water consumption; means to follow up water consumption; means and measures taken to reduce water consumption (“saving” taps, use of water rain, etc.); how the staff members and trainees awareness is raised about this subject. |
| Description | |
Scale | Description criteria: Performance | Impact |
1 | Becoming aware: There is no management or control of water consumption. | Assessment result with concrete examples from VET provider organization |
2 | Measures: Control of water consumption without any optimization. | |
3 | Setting up actions: Optimization of water consumption and formalized follow up—optimized networks of used and consumption waters. Activities oriented towards staff members and training course participation in order to develop awareness. | |
4 | Incomplete mastery of innovations: Formalized action plan with a fixed quantitative objective for water consumption reduction and actions developed to reach the objective. Action plans for staff member and participant awareness. | |
5 | Excellence/model behavior: Exclusive use of renewable water resources. Functioning is performed in close circuit. Mode of functioning that gives priority to recycling. | |
Scale | Description criteria: Importance | Impact |
1 | The issue is of little consequence; dealing with it is not on the agenda, and it can be ignored for the moment. | Assessment result with concrete examples from VET provider organization |
2 | Not to master the issue could undermine or penalize—for a limited time and in a limited area—the operating process of the organization. | |
3 | Not to master the issue could jeopardize the success of the organization’s projects. Mastering it is essential in order to accomplish projects. | |
4 | Not to master the issue could jeopardize the successful completion of all the organization’s projects. Mastering it is essential for the successful completion of the whole body of projects | |
5 | Not to master the issue could put the very existence of the organization in jeopardy. It is essential for its survival that it should master the issue. | |
Consumption of raw materials (code 2.4): All raw materials have to be taken into account whenever they are common (used by everyone), such as ink or specific to an activity (e.g., food stuff for a catering and cooking course). It is relevant to know the level of consumption, the origin of products and to take them into account. The raw materials used for training courses provision are paper, ink, etc.
Environmental aspects to check: Disposal to identify and check permanently the raw materials consumption; how the organization raises staff members and trainees awareness.
Performance: (1) Becoming aware: No concern nor management at all of raw materials. None or very little knowledge of raw materials origin and types; (2) Measures: Measure of quantity of raw materials used by location and activity in relation with turnover or training activity level; (3) Setting up actions: Actions are set up in order to reduce raw materials consumption. Actions are implemented to raise staff members and participants’ awareness in saving energy; (4) Incomplete mastery of innovations: An action plan aiming to reduce materials consumption (including a quantitative objective for consumption) is formalized and communicated to all staff members and trainees. Some materials come from recycling. Use of recyclable raw materials; (5) Excellence/model behavior: Consumption is permanently decreasing (for the same level of activity). Raw materials come from recycling. Preference is given to renewable energies when making the selection.
Water pollution (code 2.5): According to its activity, a training organization can produce waste that might have a risk of impacting water pollution. Being aware of this impact, measures taken to reduce it, awareness of staff and learners are points that can be observed to measure how this risk is managed.
Environmental aspects to check: Existence of a reflexion within the organization about environmental impacts on water and if these impacts are taken into account. If yes, how? How much is the water consumption in terms of quantity and costs; the impact of disposal on the environment; and how the organization raises staff and trainees awareness.
Performance: (1) Becoming aware: None or very little knowledge of the activity impact on rivers in the neighborhood and of the potential treatments of polluted waters rejected by the organization; (2) Measures: Quantitative and qualitative analyze of throwing (types and pollution risks); (3) Setting up actions: Treatment of waste; (4) Incomplete mastery of innovations: Limiting wastes and disposal at source. Staff members and trainees awareness; (5) Excellence/model behavior: Technological observation enables identifying, for one’s activity and inherent risks of pollution, what are the most relevant and appropriate techniques for liquid waste disposal. Functioning is performed in closed circuit. Staff member and participants to trainings are developed on new technologies that enable better liquid waste disposal.
Soil management and pollution (code 2.6): Decisions and activities may have an important impact on urban and rural environment and its ecosystems. This impact can be, for example, associated with urban policy, buildings, construction, transport systems, waste, used water and to agricultural techniques.
Environmental aspects to check: The impact of the activities on soil and subsoil (underground).
Performance: (1) Becoming aware: No observations of impacts on soil or subsoil. None or very little knowledge about the impact produced by its activity on soil or subsoil; (2) Measures: Rough knowledge of soil pollution and geology; (3) Setting up actions: According to the law and regulations—storage in hermetic containers of all waste and equipment that may generate soil pollution. Organization of cleaning and decontamination of sites previously polluted; (4) Incomplete mastery of innovations: An action plan is set up and aims at reducing the organization impact on soil and subsoil. Soil geology is taken into account within the training organization activity. Research and implementation of new processes to limits impacts are organized; (5) Excellence/model behavior: The organization implements its staff, training participants, and partners’ awareness on this topic.
Air pollution and greenhouse effect (code 2.7): Gas emissions, in particular greenhouse gas emissions, have an impact on climate evolution. Depending on the knowledge of the atmosphere, this pollution can be taken into account and actions implemented to limit it. The purpose is to identify types of disposal that cause air pollution, and they are quite numerous and depend on the activity: gas released by means of transport used by staff members and by persons attending courses has to be taken into account; and gas issued by training activities or production directly linked to training, and these are quite common.
Environmental aspects to check: The types of gas released into atmosphere; following up on gas released by the organization; how to take into account these emissions and what actions can be set up; and method for measuring the gas throwing out greenhouse effect; actions implemented to reduce these impacts.
Performance: (1) Becoming aware: No idea or very little knowledge of atmospheric emissions. No conformity with legislation or no knowledge of legislation obligations; (2) Measures: Identification of which gases are emitted. Measure and analyze emissions and their impacts (in particular for greenhouse gas). Conforming to legislative obligations; (3) Setting up actions: Setting up of punctual actions in order to reduce greenhouse gas emissions. Reduction of emissions (in particular through improvement of transport policy); (4) Incomplete mastery of innovations: An action plan in order to reduce all throwing out into atmosphere for any kind of activity (personnel’s transport, trainees’ transport, emissions directly linked to training activity) and on processes concerned with the training activity (e.g., purchase policy); (5) Excellence/model behavior: The organization organizes and develops staff members, trainees and partners’ awareness.
Waste (code 2.8): It includes waste produced by all organizations (paper, ink, old IT material) and specific waste directly linked to training activity (oil, food, etc.). The identification (type, quantity) of the activities that produce waste, all the measures taken to reduce waste, reusing materials, recycling, and waste treatment, are elements to be taken into account by the organization in order to improve its waste management policy.
Environmental aspects to check: nature and volume of waste the organization is producing by type of activity or location; the types of use that produce the most waste; the impact of waste on the environment; how the organization deals with its waste; measures and means the organization sets up in order to reduce waste; and how the organization raise staff members and trainees awareness.
Performance: (1) Becoming aware: No idea about legislation or at least some knowledge but no application; no specific waste management; and everything is put into the dustbin; (2) Measures: Partly in conformity with laws and regulations; identification of volume and type of waste and location; (3) Setting up actions: Identification of the impact of waste on the environment; pretreatment of waste (for example, selective waste sorting) organized on some sites; and actions are implemented in order to raise staff and trainees awareness on waste reduction; (4) Incomplete mastery of innovations: An action plan is set up with the aim of reducing waste by changing habits (for example, it could consist of reusing paper); organization of waste that is formalized and communicated to staff members and trainees; waste selection is systematically done; forthcoming waste is checked at “collector’s” or “eliminator’s” location; and when selecting waste “collectors” or “eliminators”, the energy cost is taken into account (e.g., transport); (5) Excellence/model behavior: Volume of wastes decreases (for an equivalent activity); reducing raw material consumption; and agreements have been made with local partners in order to valorize materials when the organization cannot do it by itself internally.
Bio diversity (code 2.1): Biodiversity is life diversity in all its forms, at all levels, in any combination. It includes diversity among species and between species and ecosystems diversity. Knowledge of how the organization’s activities impact fauna and flora, and actions developed to control and limit these impacts.
Environmental aspects to check: Information about the impact of the organization activity upon flora and fauna; existence of emissions into the atmosphere made by the organization; how they are taken into account and what actions are set up.
Performance: (1) Becoming aware: No conformity with legislation; none or very little knowledge about the impact of the activity upon flora and fauna; (2) Measures: Knowledge; (3) Setting up actions: Conformity with legislation; implementation of actions or disposals to correcting negative impacts on biodiversity; (4) Incomplete mastery of innovations: An action plan is set up in order to guarantee the consideration of the impact on biodiversity for any new activity; (5) Excellence/model behavior: The organization improves the awareness of staff members and trainees.