Next Article in Journal
“Addiction Pharmacy” and the Professionalization Process: Technology-Enhanced Assessment of Reflective Practice and Teamwork
Previous Article in Journal
From Workshop to E-Learning: Using Technology-Enhanced “Intermediate Concept Measures” As a Framework for Pharmacy Ethics Education and Assessment
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

The Production of a Framework of Competences for Pharmacy Practice in the European Union

1
Pharmacology Department, Lorraine University, Pharmacolor Consultants Nancy (PCN), 12 rue de Versigny, Villers 54600, France
2
Past-president, European Association of Faculties of Pharmacy (EAFP), Vrije Universiteit Brussel (VUB), Laarbeeklaan 103, Brussels 1090, Belgium
3
Faculty of Pharmacy, University of Granada (UGR), Campus Universitario de la Cartuja s/n, Granada 18701, Spain
4
School of Pharmacy, National and Kapodistrian University Athens, (University of Athens), Panepistimiou 30, Athens 10679, Greece
5
Pharmacy Faculty, University of Tartu (UT), Nooruse 1, Tartu 50411, Estonia
6
Pharmacy Faculty, University of Helsinki (UH), Yliopistonkatu 4, P.O. Box 33-4, Helsinki 00014, Finland
7
Faculty of Pharmacy, University of Ljubljana, Askerceva cesta 7, Ljubljana 1000, Slovenia
8
Pharmacy Faculty, Jagiellonian University (University of Cracow), UL, Golebia 24, Krakow 31-007, Poland
9
Pharmacy Faculty, University of Medicine and Pharmacy "Carol Davila" Bucharest (UMFCD-B), Dionisie Lupu 37, Bucharest 020021, Romania
10
Faculty of Pharmacy, Université de Lille 2 (UL), Lille 59000, France
11
School of Life and Health Sciences, Aston University (AU), Birmingham, B4 7ET, UK
*
Author to whom correspondence should be addressed.
Bart Rombaut passed away in January 2014. This article is dedicated to his memory.
Pharmacy 2014, 2(2), 161-174; https://doi.org/10.3390/pharmacy2020161
Submission received: 18 March 2014 / Revised: 22 April 2014 / Accepted: 23 April 2014 / Published: 9 May 2014

Abstract

:
The goal of the PHAR-QA (quality assurance in European pharmacy education and training) project is the production of a European framework for a quality assurance system based on competences for pharmacy practice. The PHAR-QA framework will be European, consultative and will encompass the various aspects of pharmacy practice. In this review, we describe the methodology to be used in the project and the first stage in the development of this framework. Using the proposals for competences produced by our previous PHARMINE (Pharmacy education in Europe) project, together with those of other sources, three university professors of pharmacy (Authors 2 through 4) produced a list of three major competency domains that reflect the activities of practitioners: Patient Care Competences, Personal Competences and Management and Organizational Structure Competences. Each domain was subdivided into nine, nine and eight competencies, respectively, for a total of 27 major competencies that were further subdivided into an average of five supporting competences per major competence, giving a total of 140 proposals for competences for pharmacy practice. The 27 and 140 proposals were ranked by an expert panel of seven university professors of pharmacy (Authors 5 through 11). The panel also commented on the proposed competences. On the basis of the ranks and comments, a list of 68 proposals for competences was produced. This list was then examined by the expert panel and a new version based on their comments produced. The latter process was repeated twice based on Delphi methodology. This review presents this process and the 68 proposals. We invite the pharmacy community to participate in the second stage of the elaboration of the PHAR-QA competence framework for pharmacy practice by ranking the proposals and adding comments. It is anticipated that this survey will stimulate a productive discussion on pharmacy education and practice by the various stakeholders (department staff and students, community, hospital and industrial pharmacists, as well as pharmacists working in clinical biology and other branches, together with representatives of chambers and associations).

1. Introduction

The goal of the PHAR-QA (quality assurance in European pharmacy education and training) [1] project is the production of a European framework for a quality assurance system based on competences for pharmacy practice.
The European Higher Education Area (EHEA) was launched in March, 2010. Similar to the Bologna Process (started in 1999), the EHEA is meant to ensure more comparable, compatible and coherent systems of higher education in Europe. The establishment of the European Higher Education Area has led universities to harmonize their educational systems in order to amplify the mutual recognition of degree courses needed for student and professional mobility. This applies to all degree courses, not only pharmacy. Within the European Higher Education Area, the Bologna process developed university programs based on competences [2]. The use of competences eliminates ambiguity and clearly establishes what the student is able to do at the end of the program.
The competence approach is also adopted in European directives. All EU pharmacy degree programs follow the European directive [3] for the recognition of qualifications for sectoral professions. In its latest 2013 version, programs are defined in terms of competences given the evolution in the role of pharmacists towards ensuring safety and efficacy in medicine use, reporting of adverse reactions to pharmaceutical products, personalized patient support and contribution to public health campaigns. Thus, the definition of programs in terms of competences is necessary, not only because of the changes brought about by the Bologna process, but also because of changes in the role of the pharmacist as defined in the EU directive.
The elaboration of a list of competences that are harmonized across different countries, sectoral activities and education systems requires the use of an iterative process, such as the Delphi method [4]. The Delphi method is a process for structuring group communication, allowing a group of individuals, as a whole, to deal with a complex problem. This approach has been used for the elaboration of a framework of competences for medical doctors by the consortium “Medical Education in Europe” (MEDINE) [5]. This EU-funded thematic network used a modified Delphi iterative process to produce a series of outcomes for medical education.
In this paper, we describe the results of the first stage in the production of an EU framework for competences for pharmacy practice using a modified Delphi process. We started with an expert panel consisting of university staff that elaborated through several Delphi rounds the framework to be used as a starting point in the second stage of this process. This second stage will consist in the evaluation of the framework by a much wider expert panel consisting of university staff, students, community pharmacists, hospital pharmacists, industrial pharmacists and others (laboratory medicine/clinical biology, wholesalers, pharmacists working in government and other agencies and representatives from pharmacy chambers and associations).

2. Methods

2.1. Elaboration of the List of Competences

A questionnaire was produced by Authors 2 through 4 incorporating the principles outlined in the 2013 EC directive on sectoral professions. The questionnaire was also based on the framework of competences for pharmacists produced by the PHARMINE (Pharmacy education in Europe) [6] work programme 3 [7] supplemented by frameworks developed for:
  • Medical doctors by MEDINE.
  • Dentists by ADEE [8]; the Association for Dental Education in Europe (ADEE) was founded in 1975 as an independent European organisation representing academic dentistry and the community of dental educators (from their website).
  • Community pharmacists in SW England [9]. The Competency Development and Evaluation Group (CoDEG) is a collaborative network of specialist and academic pharmacists, developers, researchers and practitioners. Its aim is to undertake research and evaluation in order to help develop and support pharmacy practitioners and ensure their fitness to practice at all levels. Among its key outputs are the General Level Framework and the Advanced Level Framework (from their website).
In order to account for future developments in the role of pharmacists, trends in healthcare systems, especially those concerning pharmacy, were also taken into account. We used those outlined in the documents from the European Observatory on Health System and Policies [10].
A questionnaire with 27 major competences was developed as indicated below. The major competences were grouped into 3 major domains representing the main activities of the practitioner.
(I)
Domain “Patient Care Competences”, subdivided into 9 major competences:
(1)
Patient consultation
(2)
Need for the drug
(3)
Promote health, engage with the population on health issues and work effectively in a healthcare system
(4)
Selection of drug
(5)
Drug specific issues
(6)
Provision of drug product
(7)
Medicines information and patient education
(8)
Monitoring drug therapy
(9)
Evaluation of outcomes
(II)
Domain “Personal Competences” subdivided into 10 major competences:
(10)
Organisation
(11)
Effective communication skills both orally and in writing
(12)
Teamwork
(13)
Professionalism
(14)
Learning and knowledge
(15)
The global pharmacist
(16)
Problem-solving knowledge
(17)
Problem solving; effective use of information and information technology
(18)
Providing information
(19)
Follow-up
(III)
Management and Organization Competences” subdivided into 8 major competences:
(20)
Clinical governance
(21)
Service provision
(22)
Budget setting and reimbursement
(23)
Organisation
(24)
Training
(25)
Staff management
(26)
Procurement (medicines purchasing)
(27)
Drug product-process development and manufacture
Each of the 27 major competences was further divided into an average of 5 supporting competences per major competence. Thus, for example, the major competence: “Personal Competences: learning and knowledge” was broken down into 7 proposals:
(1)
Capacity to learn, including continuous professional development;
(2)
Ability to teach others;
(3)
Analysis: ability to apply logic to problem solving, evaluating pros and cons and following up on the solution found;
(4)
Synthesis: capacity to gather relevant knowledge and summarise the key points;
(5)
Capacity to evaluate scientific data in line with current scientific and technological progress;
(6)
Ability to interpret pre-clinical and clinical evidence-based medical science and apply the knowledge to pharmaceutical practice;
(7)
Skills in scientific and biomedical research.
This gave altogether a total of 140 proposals for competences that the panel had to rank.
Complete details of the 140 proposals are available on the PHAR-QA website.
The panel was asked to rank the 27 major competences and the 140 competences according to the following Likert [11] scheme:
(1)
Not important
(2)
Quite important
(3)
Very important
(4)
Essential

2.2. Statistical Analysis

Statistical analysis was performed using parametric and non-parametric methods with GraphPad software [12]. We used the following methods:
(1)
Descriptive statistics
i.
Parametric: means and standard deviations
ii.
Non-parametric: medians with 25 and 75% percentiles
(2)
Tests of normality of distribution:
i.
Kolmogorov-Smirnov
ii.
Skewness
iii.
Kurtosis
(3)
Comparisons
i.
Non-parametric
  • Kruskal–Wallis
  • Wilcoxon signed rank test

3. Results

3.1. Panel Members

We analysed the differences amongst the rankings of panel members (Table 1).
Minima (one) and maxima (four) were the same for all seven members. For 6/7, the median was three; means ranged from 3.0 to 3.6 (Table 1). The Kruskal–Wallis test revealed that the median ranking for expert panel Member 7 was different from the other six, but amongst the other six, there were no significant differences. There was a significant correlation between medians and means (test of slope significantly non-zero at p < 0.05). The Kolmogorov–Smirnov (K-S) test showed a significant deviation from the normality of distribution. All distributions showed significant critical values for skewness.
Table 1. Statistical analysis of ranking data of the seven expert panel members.
Table 1. Statistical analysis of ranking data of the seven expert panel members.
Panel member1234567
25% percentile2.03.03.03.03.02.04.0
Median3.03.03.03.03.03.04.0
75% percentile4.04.04.04.04.04.04.0
Mean3.03.23.23.13.23.13.6
Standard deviation0.890.770.720.900.720.960.75
Kolmogorov-Smirnov (K-S) distance0.210.260.240.260.250.270.45
p value<0.0001<0.0001<0.0001<0.0001<0.0001<0.0001<0.0001
Passed K-S normality testNoNoNoNoNoNoNo
Skewness−0.31−0.62−0.46−0.69−0.71−0.70−1.8
Kurtosis−0.91−0.47−0.52−0.510.44−0.632.0

3.2. Major Competences

The rankings of the major competences were analysed (Table 2).
Table 2. Statistical analysis of ranking data of the 27 major competences (ranked by mean). Med, median; SD, standard deviation; CV, coefficient of variation.
Table 2. Statistical analysis of ranking data of the 27 major competences (ranked by mean). Med, median; SD, standard deviation; CV, coefficient of variation.
RankMajor competenceDomain-numberMedMeanSDCV%
1Selection of drugPCC-1.443.90.0952
2Providing informationPC-2.1843.80.3810
4Problem solving: effective use of information and information technologyPC-2.173.43.60.319
5Need for the drugPCC-1.23.33.60.4312
8Provision of drug productPCC-1.63.53.50.5315
3Drug specific issuesPCC-1.53.43.50.236
9Effective communication skills both orally and in writingPC-2.113.63.40.6720
7Procurement (medicines purchasing)MOC-3.263.33.40.5316
6Medicines information and patient educationPCC-1.73.33.30.3716
12Problem-solving knowledgePC-2.163.33.30.26
14ProfessionalismPC-2.133.33.30.134
10Monitoring drug therapyPCC-1.83.13.30.3812
11Follow-upPC-2.1933.30.5215
13TrainingMOC-3.2433.30.5717
15Learning and knowledgePC-3.2433.20.3912
16Patient consultationPCC-1.13.43.10.5718
17Promote health, engage with population on health issues and work effectively in a health care systemPCC-1.33.13.10.4314
18Drug product-process development and manufactureMOC-3.272.92.900
20Clinical governanceMOC-3.2132.90.4114
19OrganisationMOC-3.232.82.80.5519
21TeamworkPC-2.122.82.80.4616
23The global pharmacistPC-2.152.82.70.5420
24Staff managementMOC-3.252.72.70.3915
22Evaluation of outcomesPCC-1.92.52.60.3814
25OrganisationPC-2.102.32.30.913
26Service provisionMOC-3.212.22.10.7234
27Budget setting and reimbursementMOC-3.221.31.50.5135
PCC, Patient Care Competences; PC, Personal Competences; MOC, Management and Organization Competences.
There was a significant correlation between medians and means (test of slope significantly non-zero at p < 0.05). Coefficients of variation ranged from 0–30%. The K-S test showed that for 5/27 major competences (“selection of drug”, “providing information”, “drug specific issues”, “follow-up”, “learning and knowledge”), there were significant deviations from normality; in 5/27 (“providing information”, “drug specific issues”, “effective communication skills both orally and in writing”, “learning and knowledge”, “the global pharmacist”), there were significant degrees of skewness.
The Wilcoxon signed rank test showed that all the medians for the major competences (except that for the 27th) were significantly different from a theoretical median ranking of one (= not important).
Of the six high ranking major competences (rank: 1–4, 8 and 9; median > 3.4), three were Patient Care Competences and three Personal Competences. Of the eight lowest ranking major competences (rank: 19, 21–27; median < 2.8), four were Management and Organization Competences, one a Patient Care Competence and one a Personal Competence.

3.3. Ranking Data for All Competences

Table 3 shows part of the statistical analysis of the ranking of the data of the 140 competences: the median rank for the three lowest ranked competences (left) was not different from a theoretical rank of one (= not important); the medians for the 6 highest ranked competences (right) were significantly different from 1. The 3 lowest ranked competences showed a wide range of scores from 1 through 4 and very high coefficients of variability compared to the 6 competences that had the highest ranks. Although the Wilcoxon signed rank test comparison to a theoretical value of one (= not important) (W value) showed that the three lowest ranks had scores that were not significantly different from one, it should be noted that this is primarily due to the discrepancies of the positive ranks as negative ranks (i.e., ranks < 1) were all zero.
Table 3. Part of the statistical analysis of the ranking of the data of the 140 competences: data (left) for three lower ranked competences not different from a theoretical rank of one (= not important) and data (right) for six highest ranked competences (mean rank = 3.9) (for complete results for the 140 competences). K-S, Kolmogorov-Smirnov; W value, Wilcoxon signed rank test value.
Table 3. Part of the statistical analysis of the ranking of the data of the 140 competences: data (left) for three lower ranked competences not different from a theoretical rank of one (= not important) and data (right) for six highest ranked competences (mean rank = 3.9) (for complete results for the 140 competences). K-S, Kolmogorov-Smirnov; W value, Wilcoxon signed rank test value.
RankLowestLowestLowestHighestHighestHighestHighestHighestHighest
CompetenceDescribes the key drivers for national and local service developmentClaims reimbursement appropriately for services providedEnsures the prescriber’s intentions are clearEnsures appropriate timing of doseSupplies information on documentsAccesses information from appropriate sourcesDemonstrates ability to describe the mechanisms of interactionsProvides information that is appropriate to the recipient’s needsEstablishes the priority of information provision when it is needed
Median224444444
Minimum111333333
Maximum344444444
25% percentile111444444
75% percentile32.54444444
Mean2233.93.93.93.93.93.9
Standard deviation0.821.11.40.380.380.380.380.380.38
Coefficient of variation40.82%54.77%47.14%9.80%9.80%9.80%9.80%9.80%9.80%
K-S distance0.210.330.330.50.50.50.50.50.5
p value0.20.03590.0192<0.0001<0.0001<0.0001<0.0001<0.0001<0.0001
Deviation from normalityYesNoNoNoNoNoNoNoNo
W value151015282828282828
Sum of + ranks151015282828282828
Sum of − ranks000000000
p value0.06250.1250.06250.01560.01560.01560.01560.01560.0156
Skewness01.4−0.99−2.6−2.6−2.6−2.6−2.6−2.6
Kurtosis−1.22.5−1.2777777

4. Discussion

In this paper, we have presented the PHAR-QA survey modus operandi and the use of a Delphi method with a small-sized expert panel. Thus, this article is about how to determine competences; further articles will deal with the definitive competence framework.

4.1. Likert Scales

As the PHARMINE and PHAR-QA consortia work in close association with MEDINE, we used the four-point Likert scale used by the MEDINE consortium. During discussion with the MEDINE statisticians, they explained that their reasoning behind the use of a four-point, rather than a five-point scale, was that with a five-point scale, the middle ranking point, three (“moderately important”), can be taken as an “easy, no-choice option” and, so, bias the data in the case of long questionnaires. With the latter, the fatigue of filling in the questionnaire may lead panel members to consistently choose Rank 3 for the sake of facility. The four-point scale makes answers requisite in that the responder is obliged to make a value judgement.
However, during our development of the PHAR-QA questionnaire, the following question was raised several times: how does one accommodate for the fact that an expert panel member may simply not know and/or have no opinion on the ranking. This begs the question as to whether the possibility to express the absence of reply should be incorporated into the ranking scale and how. This could be done with a two-stage post-hoc analysis: (1) binary for “reply/no reply”; then (2) parametric/non-parametric for “analysis of ranking”. The solution chosen for the Delphi rounds with the European pharmacy community as the expert panel is the use of a four-point Likert scale plus an additional fifth option “I cannot rank this competence”. In future papers, the question of how this response can be analysed will be detailed.

4.2. Statistical Analysis of the Rankings

Examination of the data showed that:
(1)
There is a lack of sphericity: the variances of the differences between all possible pairs of rankings are not equal. The difference between Rank 1 (not important) and 2 (quite important) is not the same as between 3 (very important) and 4 (essential).
(2)
Data are discrete rather than continuous variables: rankings cannot take on any value between two specified values.
(3)
Data are skewed to higher ranking values. They do not follow any pre-specified distribution; some data even show an inverse bell-shaped form. In spite of this, there is a significant correlation between median and mean in several cases. This generally implies the normality of data, but in this case is probably affected by the small number of observations. Finally, it should be noted that there is a wide spread in values for the coefficient of variation.
Although the characteristics above would argue in favour of the use of non-parametric methods, both parametric and non-parametric statistics have been used as follows:
(1)
Descriptive statistics: reduce and summarize complex data to a few comprehensible variables without losing any of the original information. In this case, parametric statistics (means and standard deviations, coefficient of variation, etc.) are more useful, as they are more precise.
(2)
Decisional statistics: reveal the statistical importance of such variables while taking into account errors from known or unknown disturbing influences. In this case, non-parametric tests (Wilcoxon signed rank, Kruskal-Wallis, etc.) are more appropriate in the case of non-continuous variables, such as ranks, although parametric tests (analysis of variance, etc.) do show a certain robustness [13]. It should be noted that in the Wilcoxon comparison of the ranks observed with a theoretical rank of “1” (= not important), the test is affected by the fact that the sum of negative ranks = 0. This arises because there are no observed ranks <1. As Likert ranks are integral, a rank <1 would be equal to “0”, and this would bring in a binary scale of an “important/not important” nature. This is against the philosophy behind the Likert scale, which is used to create nuance in a questionnaire (not important, quite important, very important, essential) and so goes beyond a binary scale.
Albeit, it should be remembered that no statistical analysis can give a correct answer to a wrong question or if used in unfavourable conditions. In order to be valid, the K-S, skewness and kurtosis tests of normality of distribution require a theoretical number of observations >25–30. With seven panel members, this condition is not met, so the results from the tests applied in such circumstances should be treated with caution. This is shown by the fact that in the analysis of the 27 major competences, in some cases (e.g., for “selection of drug”) the data did not pass the Kolmogorov-Smirnov test for normality and, yet, did not show statistical significance in the skewness and kurtosis tests. The opposite was also seen: for example, for “effective communication skills both orally and in writing”, the data showed significant skewness and, yet, passed the normality test. The design will be more balanced in the future when the expert panel will consist of several hundred persons. This will allow for a more accurate description of distributions and their deviation (or not) from normality. It will also allow cluster analysis of specific groups.

4.3. The Modified Delphi Process and the Rankings of the Proposals (Figure 1)

Figure 1. The modified Delphi process.
Figure 1. The modified Delphi process.
Pharmacy 02 00161 g001
The above process has several characteristics:
(1)
It is a two-stage process with the first university staff panel producing a questionnaire that is run through a wider pharmacy community panel at a second stage. This will “even out” discrepancies between the different actors. Thus, for instance, several university staff members gave a low rank to management and organization competences (although there was wide variability on some of these points). It may be that active pharmacists will give such competences a higher rank. The process will ensure that the final framework is universally accepted.
(2)
The initial suggested framework is only a starting point: it may be modified by suggestions to remove or add competences as the process evolves. In the initial stages, it was realised that some competences were duplicates (e.g., major Competences 16 and 17); other competences were not described with sufficient clarity. There have also been changes in the organisation of the questionnaire. At the end of Stage 3 in Figure 1, PHAR-QA will put forward a proposal that represents a consensus (not unanimity) of the opinions of the expert panel members. This constitutes a starting point, and pharmacists around Europe will participate in order to ensure that the final proposed competences will have the widest possible acceptance.
(3)
The final framework will be validated by the large number of responses from pharmacists with widely varying occupations throughout Europe. Thus, the framework will be European, consultative and will encompass pharmacy practice in a wide sense.
(4)
In order to stimulate participation, the time needed to fill in the questionnaire is set at 30 min; thus, the number of questions is limited to 60–70.

4.4. Limitations

The experience of the university staff expert panel outside of academia is unknown. Albeit, in the two-stage process we use, the questionnaire produced by the university staff expert panel is but a consensus starting point for the second stage. It is unknown how far an academically produced starting point questionnaire will influence the final result, i.e., the competence framework produced by the Delphi rounds in the wider EU pharmacy community.
The second stage Delphi rounds will include staff, students, community pharmacists, etc., reflecting the heterogeneity of the EU pharmacy community. The sample of stakeholders should be reflective of the target population. Attention will be paid to collecting a sufficient number of replies from the different stakeholders in different countries in order to allow cluster analysis of possible trends in replies. There is a certain contradiction here with the need to fix numbers in stakeholder categories in terms of factors, such as the population of the country concerned, the relative numbers of staff and students in academia and other factors.

5. Conclusions

PHAR-QA has developed a proposal for a competence framework for pharmacy practice. This represents a consensus (not unanimity) of the opinions of the expert panel members. It constitutes a starting point upon which pharmacists around Europe are invited to comment and validate, giving a final competence framework with the widest possible acceptance.
The questionnaire “The European network evaluation of the PHAR-QA framework of competences for pharmacists” can be found at the website given in [14].

Acknowledgments

PHAR-QA is funded by the European, Education, Audio-visual and Culture Agency (EACEA: [15]); the grant number is 527194-LLP-1-2012-1-BE-ERASMUS-EMCR.

Author Contributions

Bart Rombaut (Vrije Universiteit Brussel) assisted in the design of the project and performance of the research. Jeffrey Atkinson (Pharmacolor Consultants Nancy) helped to design the project and to perform the research, assisted in the data collection, analysis and interpretation, wrote the paper and directed the revision of the paper. Antonio Sanchez-Pozo (University of Granada) helped to design the project and to perform the research and assisted in the data collection and in the revision of the paper. Dimitrios Rekkas (University of Athens) helped to design the project and to perform the research and assisted in the revision of the paper. Peep Veski (University of Tartu) and Jouni Hirvonen helped to design the project and assisted in the revision of the paper. Borut Bozic (University of Ljubljana) helped to design the project and to perform the research and assisted in the revision of the paper. Agnieska Skowron (University of Cracow) helped to perform the research. Constantin Mircioiu (University of Medicine and Pharmacy “Carol Davila” Bucharest) helped to design the project and to perform the research, performed the data analysis and interpretation and assisted in the revision of the paper. Annie Marcincal (University of Lille) helped to perform the research. Keith Wilson (Aston University) helped to design the project and to perform the research and assisted in the data collection and in the revision of the paper.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. PHAR-QA “Quality Assurance in European Pharmacy Education and Training”. 527194-LLP-1-2012-1-BE-ERASMUS-EMCR. Available online: http://www.pharmine.org/PHAR-QA/ (accessed on 18 March 2014).
  2. Bologna process on the website of the European Universities Association. Available online: http://www.eua.be/eua-work-and-policy-area/building-the-european-higher-education-area/bologna-basics.aspx (accessed on 18 March 2014).
  3. EU directive 2013/55/EU on the recognition of professional qualifications. Available online: http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2005:255:0022:0142:EN:PDF (accessed on 18 March 2014).
  4. Hsu, C.C.; Sandford, B.A. The Delphi Technique: Making Sense of Consensus. Practical Assess. Res. Eval. 2007, 12. Available online: http://pareonline.net/getvn.asp?v=12&n=10 (accessed on 18 March 2014).
  5. Cumming, A.D.; Ross, M.T. The Tuning Project for medicine: Learning outcomes for undergraduate medical education in Europe. Med. Teach. 2007, 29, 636–641. [Google Scholar] [CrossRef]
  6. PHARMINE “Pharmacy Education and Training in Europe”. 142078-LLP-1-2008-1-BE-ERASMUS-ECDSP. Available online: http://www.pharmine.org/ (accessed on 18 March 2014).
  7. PHARMINE work programme 3. Available online: http://enzu.pharmine.org/media/filebook/files/PHARMINE%20WP3%20Lisbon%200611.pdf (accessed on 18 March 2014).
  8. Association for Dental Education in Europe. Available online: http://www.adee.org/ (accessed on 18 March 2014).
  9. CoDEG. The Competency Development and Evaluation Group. Available online: http://www.codeg.org/ (accessed on 18 March 2014).
  10. European Observatory on Health System and Policies. Available online: http://www.euro.who.int/en/who-we-are/partners/observatory/eurohealth (accessed on 18 March 2014).
  11. Edmondson, D.R. Likert scales. A history. Available online: http://faculty.quinnipiac.edu/charm/CHARM%20proceedings/CHARM%20article%20archive%20pdf%20format/Volume%2012%202005/127%20edmondson.pdf (accessed on 18 March 2014).
  12. GraphPad®. Available online: http://www.graphpad.com/ (accessed on 18 March 2014).
  13. Raschi, D.; Guiard, V. The robustness of parametric statistical methods. Psychol. Sci. 2004, 46, 175–208. [Google Scholar]
  14. The survey of the European network evaluation of the PHAR-QA framework of competences for pharmacists. Available online: https://www.surveymonkey.com/s/pharqasurvey1 (accessed on 18 March 2014).
  15. European, Education, Audio-visual and Culture Agency (EACEA). Available online: http://eacea.ec.europa.eu/index_en.php (accessed on 18 March 2014).

Share and Cite

MDPI and ACS Style

Atkinson, J.; Rombaut, B.; Pozo, A.S.; Rekkas, D.; Veski, P.; Hirvonen, J.; Bozic, B.; Skowron, A.; Mircioiu, C.; Marcincal, A.; et al. The Production of a Framework of Competences for Pharmacy Practice in the European Union. Pharmacy 2014, 2, 161-174. https://doi.org/10.3390/pharmacy2020161

AMA Style

Atkinson J, Rombaut B, Pozo AS, Rekkas D, Veski P, Hirvonen J, Bozic B, Skowron A, Mircioiu C, Marcincal A, et al. The Production of a Framework of Competences for Pharmacy Practice in the European Union. Pharmacy. 2014; 2(2):161-174. https://doi.org/10.3390/pharmacy2020161

Chicago/Turabian Style

Atkinson, Jeffrey, Bart Rombaut, Antonio Sánchez Pozo, Dimitrios Rekkas, Peep Veski, Jouni Hirvonen, Borut Bozic, Agnieska Skowron, Constantin Mircioiu, Annie Marcincal, and et al. 2014. "The Production of a Framework of Competences for Pharmacy Practice in the European Union" Pharmacy 2, no. 2: 161-174. https://doi.org/10.3390/pharmacy2020161

Article Metrics

Back to TopTop