*2.2. e-Cognocracy*

E-Cognocracy [10–12,23–25] is a new system of democratic representation that combines liberal (representative) democracy and direct (participative) democracy to cognitive ends. It seeks the creation and social diffusion of knowledge and the construction of a more open, transparent, cultured, educated and freer society; a society that is more cohesive and connected, more participative, egalitarian and cooperative.

The new system uses multicriteria decisions as its methodological support, the internet as its communication support and the democratic system as a catalyst for learning. As explained in [16,26], the e-Cognocracy methodology consists of 16 stages grouped in four blocks (Figure 1): (1) Problem Formulation, Stages 1 and 2; (2) Problem Resolution, Stages 3 to 10; (3) Knowledge Extraction and Democratisation, Stages 11 to 14; and (4) Evaluation and Documentation, Stages 15 and 16.

Block 1, Problem Formulation, refers to the initial problem proposed by the representatives or the citizens, its presentation to the actors involved in its resolution and its final setting.

**Figure 1.** Blocks and stages of the e-Cognocracy methodology [16].

Block 2, Problem Resolution, includes the two voting rounds, in which the scientific resolution of the problem is obtained, using the Internet as communication support and the analytic hierarchy process (AHP) as the multicriteria decision making technique. The rounds are interspersed with an e-discussion process, in which the actors justify their preferences. The online discussion is the step prior to the extraction of knowledge that takes place in third block. The discussion allows the creative capacity of all individuals interested in the resolution of the problem to be incorporated into the decision making process and this accords with Dahl's proposals [27,28] on the improvement of democracy based on the use of ICTs. The e-discussion allows the active citizens, a *minipopulus*, to complement the institutions in order to reduce the gap between the representatives/politicians, and the represented/citizens. As argued by Habermas [29], the e-discussion fosters the co-creation of a more cohesive, fair, educated and e ffective society.

Block 3, Knowledge Democratisation, provides the arguments that support the di fferent positions [14,15], identifies the social leaders (the citizens whose arguments are followed by the majority of the citizens) and shares this knowledge in order to generate individual and social learning.

Finally, and as recommended for any procedure that makes use of public funds, Block 4 analyses the e ffectiveness (doing what is right), the e fficacy (achieving goals) and the e fficiency (doing things correctly) of the e-Cognocracy public policy making process.

The methodology for the conjoint design (politicians and citizens) of local public policies comprises the following phases or steps [11,23]: Step1: Project presentation; Step 2: Problem presentation; Step 3: Identification of the actors, factors and alternatives; Step 4: Modelling the problem; Step 5: Evaluation; Step 6: Identification of the initial positions; Step 7: Citizen debate and discussion. Step 8: Evaluation II; Step 9: Identification of new positions; Step 10: System behaviour; Step 11: Assignation of messages to alternatives and justification of positions; Step 12: Evaluation of individual and collective learning; Step 13: Identification of arguments that support the decisions; Step 14: Extraction and di ffusion of knowledge; Step 15: E ffectiveness of e-Cognocracy; and Step 16: Project documentation (final report).

The new methodology for the evaluation of the behaviour of models of citizen participation in the taking of public decisions uses an EF3 framework that allows the simultaneous evaluation of the model's e fficiency, e fficacy and e ffectiveness.

#### *2.3. The Evaluation of e-Participation*

E-participation is still evolving and there have only been a few proposals on evaluation. Bagozzi and Warshaw [30] advanced a secure and stable technology model for predicting users' acceptance of a range of new technologies that has been widely employed and studied in the last decades. Fred Davis [18] suggested a technology acceptance model (TAM) that explains the process of acceptation of information technology at an individual level. The Davis model is probably the most recognised and utilised in the scientific literature, it is commonly referenced and has been the inspiration behind a number of other, similar, models.

Delone and MacLean [19] designed the 'Information System Success' model as a conceptual framework for measuring the complex dependent variable in research on information technology systems. Ten years later, the same authors updated the original model based on changes in the managemen<sup>t</sup> of information systems [31].

Rowe and Frewer [32] put forward a framework to evaluate participation in general (it was not specific to e-participation); they defined a number of theoretical criteria which are essential for e ffective public participation and divided them into two types: acceptance criteria and process criteria. Acceptance criteria refer to representativeness, independence, early involvement, influence and transparency that o ffer a measure of acceptability to the wider public; process criteria refer to resource accessibility, task definition, structured decision making and cost-e ffectiveness that o ffer a measure of e ffectiveness.

Henderson and Henderson [33] constructed a model for evaluating on-line consultations, e-petitions and internet live broadcasting of parliamentary initiatives. There are seven evaluation dimensions and indicators: E ffectiveness; Equity; Quality; E fficiency; Appropriateness; Sustainability; and Process.

Macintosh and Whyte [3] developed an evaluation methodology with criteria that cover three perspectives of an e-participation experience: (i) democratic—the overarching democratic criteria addressed by the experience; (ii) project—the identification of the aims and objectives; and (iii) socio-technical—the extent to which the ICTs directly a ffect the outcomes. Each evaluation perspective is linked to a number of criteria:

• Democratic criteria: representation, engagement, transparency, conflict and consensus, political equality, community control;


In 2009, [4] published an e-Participation model as part of the DEMO-net project, undertaken in cooperation with other European researchers [34]. They reviewed and analysed applied methods for the evaluation of e-participation and gave core criteria and indicators relevant to e-participation activities such as consultation and deliberation.

Mamaqui and Moreno-Jiménez [35] formulated a methodology based on the utilisation of Structural Equation Models (SEM) for the evaluation of the e ffectiveness of e-Cognocracy.

Luna-Reyes, Gil-García and Romero [36] devised a multidimensional system for measuring and evaluating electronic government. The model seeks to incorporate the approaches currently used to measure electronic governmen<sup>t</sup> and ideas from published literature related on the issue.

Finally, Wimmer and Bicking [37] gave us the impact evaluation framework, based on evaluation methods of empirical research thereby reflecting the programmatic contexts of the projects. Evaluation is based on the interaction of the elements of a holistic e-participation solution: the participation process; the topics to be discussed; the policy; and the technology and tools employed.

This current paper proposes a theoretical framework based on the three dimensions for evaluating e-participation processes: e ffectiveness, e fficacy and e fficiency. The framework is designed for the evaluation of citizen participation experiences and projects, particularly those that involve e-Cognocracy.

#### **3. The Evaluation of e-Cognocracy**
