Strasbourg, 7 December 2016

CEPEJ(2016)12

ERUOPEAN COMMISSION FOR THE EFFICIENCY OF JUSTICE

(CEPEJ)

Measuring the quality of justice

As adopted on 7 December 2016, at the 28th plenary meeting of the CEPEJ

Table of contents

Measuring the quality of justice- 2

1.       Introduction- 2

2.       A reference model 2

3.       Measuring the quality of processes and decisions- 4

4.       Measuring the performance of the judicial services- 6

5.       Customer satisfaction surveys- 14

6.       Concluding remarks- 17

Appendices- 18

Appendix 1- 18

Appendix 2- 29

Measuring the quality of justice

1.            Introduction

1.            The term "Quality" has a very broad meaning. In everyday life, it refers to all those features that best describe and identify objects, products, or experiences. The colour of a wine, for example, is an element of its quality, or the shape is an important feature of a piece of furniture. This being so, the competence of staff, the functionality of the services and the cleanliness of the rooms are just some of the elements which determine the quality of an office, a shop, a railway station and of any place that  we come across as users or visitors.

2.            When we move from the ordinary meaning to a more technical concept of quality of goods and services, it becomes necessary to adopt a method of classification of the level of quality assigned. In this case, we enter the field of "quality measurement".

3.            It can be assumed that there are two ways of measuring the degree of quality of a product or a service:

a)            measuring the extent to which a product or service has a certain number of features and pre-defined indicators (conformity with requirements);

b)            measuring the gap between the expectations that the user had before using the goods or services and the assessment made following the use or consumption of these goods or services (conformity with expectations).

4.            It is important to note that the definition of quality contained in a common dictionary makes explicit reference to the two ways of measuring indicated above: "Quality is a notion to which are brought aspects of reality susceptible of classification or judgment."

5.            There are two preconditions for measuring quality in accordance with the principle of conformity with requirements described above:

I.              there must be (pre) defined quality parameters;

II.             there must be fixed standards of quality.

6.            The parameters for measuring the quality of a product or service can be numerous. They should be selected in accordance with the purpose of the measurement. If a railway company aims to measure the comfort of its customers during their journey, they will focus on the parameters that have a direct influence on the experience of travellers (such as punctuality, service, cleanliness, etc.). Different indicators will be used however if the purpose of the measurement concerns issues of management and internal organisation. In this case, for example, the measurement will concern the energy consumption and maintenance costs in order to assess the quality of its trains in relation to costs.

7.            Like parameters, standards of quality may vary depending on the goals and needs of the measurement. A Michelin-starred restaurant will set the parameters of ingredients to be used in its kitchen at the highest quality level, while lower standards will be acceptable for common restaurants.

8.            As mentioned previously, quality is a broad concept and many options are possible as regards its measurement, on the basis of the needs and objectives of the evaluation.

2.            A reference model

9.            The concept of quality measurement applied to justice is the subject of a lively debate in Europe. The different positions cannot be summarised because the concept of quality has been based, until today, on a general approach as to what is meant by good justice. There is the fear that the issue would be solved too swiftly by defining a few indicators focused on specific aspects of the judicial system, which would be partial and misleading.

10.          In Europe and elsewhere in the world, there is a growing number of judicial systems, which are working to develop or improve mechanisms to measure the quality of their services. These programmes, very often, are aimed at improving the existing statistical systems to measure and manage performance in a more detailed and sophisticated way. Sometimes this approach has led to modern information systems but limited in scope if compared with the range of indicators needed to measure quality for a service as complex as justice. The consequence is that when quality aspects are not included in the measurement systems, the absence of quality indicators can lead to management and organisational decisions focusing mainly on aspects of efficiency while neglecting other important aspects of quality.^[1]

11.          A certain number of quality and performance indicators, most commonly applied in European countries and in the rest of the world are monitored by international organisations including OECD and the World Bank.

12.          Also, since 2013, the EU Justice Scoreboard which the European Commission is published annually and provides information on the quality, independence and efficiency of the justice systems of all EU member States.[2]

13.          Measuring the quality of justice is an activity that may give little satisfaction and reward. The potential risk among justice operators is that very few perceive the benefits while others will say that the data will not tell them anything they do not know already. Others will say it is a waste of money and time, just adding another layer of bureaucracy instead of focusing on doing the job well.

14.          These pages do not intend to provide a universal definition of quality of justice. In fact, it would not make sense, given the different legal systems and the many specific features of each judicial system, to formulate a trans-national methodology. Moreover, the concept is so large that it cannot be reduced to a unique technique or methodology. The complex and multifaceted character of the quality of justice is also reflected in the variety of instruments which have been elaborated by the dedicated CEPEJ Working Group on quality of justice (CEPEJ-GT-QUAL). The latter has stressed on many occasions that the different aspects of a justice system which functions well are strongly connected with the characteristics of each national judicial system. It is therefore impossible to give an aprioristic view of what a good system should look like, and it would not make any sense either to compare different judicial systems between or among themselves with a view to determining which one is of best quality.

15.          In a narrow sense, the “quality of justice” is often understood exclusively as the “quality of judicial decisions”.[3] In a broader sense, it also covers key aspects of the way judicial services are provided. In this case, policy makers and reformers aiming at measuring this kind of quality will focus on aspects that go beyond the quality of the decision solely and will include parameters such as timeliness, activity rates, clearance rates, etc. In any event, it would seem simplistic to focus on only one of these concepts, even two would not give a complete picture of the situation.

16.          In a more comprehensive sense, “quality of justice” can be understood as comprising not only the quality of judicial decisions and key aspects of judicial service delivery, but all aspects that are relevant for the good functioning of the justice system, typically assessed through the user perception. Measuring in this way means considering the quality aspects that go beyond the quality of the decisions and include a variety of elements such as the clarity of the procedure and judicial decisions, on-time individual procedural steps, the accessibility of the offices and the ease of use of available tools.^[4]

17.          Measuring users’ perception in the justice field does not mean that justice shall be administered with a view to satisfying users’ expectations, as there are objectives and institutional constraints which need to be taken into account in justice administration. User’s trust is of course a legitimate objective, but it is not the only one.

18.          The model for quality measurement proposed here intends to cover all aspects of the justice system as well as the rigorous methodologies normally available on the market for assessing the quality of goods and services. It is structured on the three levels described above, which need to be considered together for measurement purposes. Focusing only on one or two of these levels would allow only for a partial assessment of the quality of a justice system. Once quality is measurement is performed based on the three above-mentioned levels, and depending on the outcomes of the assessment and the priorities of intervention, it will be of course possible to take the necessary steps and measures to strengthen quality at each individual level.

19.          Before proceeding to the description of the methodology and reviewing the indicators for each of the three levels of quality measurement, it can be helpful to list possible areas where there can apply:

a.            the entire judicial system of a country;

b.            a component or sector (in civil, criminal, administrative justice);

c.            an area that includes one or more courts (region, district);

d.            a court or branch of such court;

e.            a combination of the above elements.

3.            Measuring the quality of processes and decisions

20.          Measuring the quality of justice with the data collected periodically through the systems for statistical measurement is a common practice which is solidly established. This information enables to know the duration of proceedings, the flows and workloads, and many other aspects related to performance. This exercise becomes more complex when wishing to identify areas of quality within the "production process" of the procedure or on the substance of decisions made. In many legal systems, this type of analysis is carried out by the inspection bodies and by the different degrees of judgment. However, the purpose of these activities is not to give a certification of quality but to ensure compliance with the rules and obligations applied to the judiciary, tested as regards their compliance with the constitutional and European framework, with consequences both for the professional appraisal and for the disciplinary action.

21.          To avoid any unfortunate misunderstanding, it is important to clarify the scope of the method. The proposed method does not intend to enter into the merits of decisions and does not claim to invent strange algorithms that are able to measure the intrinsic quality of the decisions taken by a magistrate.

22.          This does not however exclude the possibility to measure what is around the contours of the decision and could lower the risks of a poor quality of the latter. Thus, the methodology proposed in this document is intended to be applied to a wide number of elements of “justice as a service”, that are complementary to the decisional process itself.

23.          Having said that, and taking into account that in a democratic society a fixed level of quality is determined on the basis of a common denominator, which is the applicable rules of the European Convention on Human Rights and the related case law of the Court, as well as the fundamental principles which stem from States’ traditional constitutions, the issue of the quality of proceedings and decisions must be sought in the light of these objectives and institutional constraints. More specifically, in article 6[5] of the European Convention on Human Rights  we can identify all the essential elements of the intrinsic quality of the jurisdiction can be identified:

i.              The fairness of the proceedings

ii.             The reasonable duration of the proceedings

iii.            The publicity of the judgment / decision and transparency of the process

iv.           The protection of minors (and other subjects for whom it is appropriate to provide a form of assistance)

v.            The comprehensibility of the prosecution, the course of the procedure, and of judgments / decisions

vi.           The right to legal assistance and access to justice in general

vii.          The legal aid (when all the conditions are met)

The Quality Checklist drawn up by the European Commission for the Efficiency of Justice (CEPEJ)

24.          Most of the quality elements mentioned in Article 6 of the European Convention on Human Rights is contained in the "Checklist for Promoting the Quality of Justice", prepared by the CEPEJ-GT-QUAL and reported in full in Appendix 1.

25.          This list contains about 250 essential questions concerning all components of a judicial system to assess the quality of judicial services. Verifying that each of the principles within the checklist is adopted by a judicial system is a good way to proceed with a deep and detailed X-ray of the "production process" leading to quality decisions.

26.          The Checklist on quality can be used as an indicator by calculating the percentage of positive responses compared to the total number of selected questions. Of course, not all the 250 items of the Checklist have the same weight. Certain aspects such as the independence of the judiciary and fair trial are of fundamental importance and must be given a higher priority. Other items of the Checklist such as the presence of information signs in court, the availability of waiting rooms for opposite parties to prevent them from waiting together, are also important but can be given a lower priority.

27.          A possible solution could be to choose a certain number of items included in the complete list, according to the objectives and the priorities of the measurement from the policy makers/courts perspective. This approach would allow for the weighting of the selected elements and therefore clearly establishing if the most important criteria are met.

28.          Here under is an extract of the checklist to show the comprehensiveness of questions concerning the criteria of fairness of the proceedings which is at the core of article 6.

Abstract from the Checklist for Promoting the Quality of Justice and the courts

29.          The table above only covers 15 of the 250 questions of the checklist. They give an idea of the importance of verifying these facts, and to assess in practice the quality of an entire judicial system or part of it. Selecting a certain number of these questions and answering them in an objective manner allows to measure concretely the quality of processes and of the basic conditions leading to decisions of the judicial system.

4.            Measuring the performance of the judicial services

30.          Measuring the performance of the judicial services is of particular importance for the various judicial services as it constitutes a source of information of major interest for society as a whole. In particular, interest for statistics concerning judicial performance comes from the following elements:

·                     they are a source of information for the various aspects of judicial services, assessed according to a scientific and analytical method;

·                     they are of high social and economic use, given the importance that judicial procedures have in the lives of people, families and businesses and in their relationships;

·                     they are one of the instruments used by international bodies to assess systemic efficiency of a country ,  which ensure the effective implementation of reforms and, in particular, to provide an objective measure of the level of respect for human rights;

·                     they constitute a fundamental instrument for judicial organisation - as the analysis of the number of civil and criminal proceedings is used – and should be used even more to optimise organisational choices relating to means and resources.

31.          The CourTools^[6] system developed in the United States summarises five reasons why performance measurement is important: (1) The perception of courts performance, even those of insiders, tends to be inaccurate. The availability of empirical data helps to hold a more objective debate. (2) The multiplicity of indicators defined with external actors enables courts to take into account the concerns of a variety of constituents, including parties, lawyers, witnesses, the public and funding authorities. (3) Clarity about service delivery targets can encourage staff to be more creative to reach them. (4) Empirical data is invaluable for the preparation, justification, and presentation of the budget. (5) If courts do not measure their performance or present their goals, this undermines the legitimacy of the judiciary in running its own affairs. The Ministry of Justice and the judiciary, each for its own competence and responsibility, must be able to defend its own budget and its independence showing their level of performance.

32.          A good evaluation system of the performance of the justice system cannot be limited to the principal indicators (duration, backlog, etc.) which describe services (duration, backlog, etc.), even if at the international level these are important interest for numerous assessment bodies, which consider them to be essential to describe the relationship between citizens and public administration. The system must be complete and based on the assumption that there are various forms of performance, not all of which are related to the “final product”, which inthe justice system, refers to the judgment. Certain service indicators are of general nature and may therefore concern activities and functions that are not directly related to judicial activity. Therefore, one of the many service indicators concerning courts’ registries could be based on the average period of time necessary to make payments for services and supplies.

33.          The level of detail of the system of indicators must be addressed in accordance to their relevance and selectiveness. Indicators are relevant if they provide useful information for decision making in fields that are critical for the management of court and relate to phenomena which have the heaviest impact on performance. They must provide concise yet significant information to facilitate the identification of alternative solutions. The indicators, generally speaking, must also be selective, since the existence of too many variables for monitoring creates a system that is difficult to manage, which may even be useless. On the other hand, it is appropriate to focus on variables which are considered as enabling the realisation of predefined objectives, meaning those which improve the description of the studied phenomenon.

ü     Case flows: new cases, resolved cases and pending cases[7]

34.          The flows of civil, administrative and criminal cases dealt with by a judicial system or a part thereof (geographic area, court, section) represent the workload basic indicator[8] of the measured unit. In particular, the number of new cases corresponds to the demand for justice, whilst resolved cases are the responses provided by the system. Pending proceedings are the number of cases that still have to be dealt with by the court, or a judge, at a certain time (for example at a specific date or at the end of a given period).

35.          The unit of measurement may be a single area of law, a division within a court, a court in its entirety or an entire judicial system of a country. In any case, the applied formula is:

36.          It is important to precise that pending cases at the end of the period (PE) for period Xt, are the pending cases at the start of the period for the following period Xt+1.

37.          Systems in which a large number of pending cases are not accumulated at the end of the period are considered to be of good quality.

ü     Clearance Rate (CR): the ratio between the number of resolved cases (R) and the number of incoming cases (I).[9]

38.          This indicator, which is derived from logistics, measures the level of turnover of products transiting through a store. In relation to justice, the indicator measures the capacity of the measured unit, such as a division of a court, a court in its entirety or an entire national judicial system, to resolve the cases submitted for judgment compared with the number of cases presented within a given period of time.

39.          For example, if in a given calendar year, 500 new cases are registered in a court, and this court solves at the same time 550 cases, its CR would be 110%. If the number of resolved cases is 400, its CR would be 80%. A CR above 100 % means that the number of pending cases decreases. A ratio lower than 100% indicates that, during the unit of time considered, the court (or other unit of production) is unable to process a caseload equal to the number of new proceedings, with the result that the court tends to generate a backlog. Conversely, a ratio with a value greater than 100% suggests that productivity is ahead of demand, and therefore that the production unit has a capacity to work through a number of cases equal to incoming cases during the previous period in addition to a share of pending cases from previous periods. As a general rule, a judicial system, or a part thereof, is efficient if it maintains a turnover ratio over the long term that is equal to 100%.

ü     Case Turnover ratio:[10]

40.          This ratio measures the relation between the number of resolved cases and the number of unresolved cases at the end of period. It measures the frequency with which a judicial system or a court replaces the number of received cases. .

ü     Duration of proceedings

41.          The duration of proceedings is the most important and significant indicator for measuring the performance of judicial systems and their components.

42.          There are two important indicators concerning the duration of judicial proceedings (both for civil cases and for criminal or administrative cases):

A)            Actual duration

B)            Prospective duration

43.          The actual duration measures the laps of time between the date on which a new case is initiated and the date where a judgment is issued. In this regard, the completion of a case or trial may be regarded as the time when the operative part is read by the court (where such a procedure is provided for), or alternatively the later date on which the details of the judgment are filed. Both points in time have a statistical significance, as the reading of the operative part is useful for the purposes of the practical consequences for the parties, who at that time become aware of the outcome of their dispute, i.e. become officially aware of who has won and who has lost the case. With regard to criminal trials, it is the time when the defendant becomes aware of whether he or she has been found innocent or guilty. The second point in time is significant, in particular with respect to the activity of the judiciary and of court registries as the legal and administrative requirements relating to a procedure are all completed with the drafting and filing of the judgment.

44.          The effective duration is a final indicator which is very precise concerning the time used in order to address a case. By extrapolating from historical data using specific statistical techniques, which rely on sophisticated modern software programs, it is possible to estimate the expected duration of “open” proceedings, i.e. those that have not yet been concluded.

45.          However, legal systems do not always have a database containing information relating to the effective duration of all resolved proceedings. In order to avoid this challenge and ensure that a comparative indicator can be calculated for all judicial systems, , we generally use a formula from the logistics’ theory. It measures the time a product stays in stock by its input and output flows.. Such formula is reported below as Disposition Time (DT).[11]

46.          DT indicator determines the number of days necessary for a pending case to be solved in court and provides further insight into how a judicial system manages its flow of cases. This indicator compares the number of resolved cases during the observed period and the number of unresolved cases at the end of the observed period. 365 is divided by the number of resolved cases divided by the number of unresolved cases at the end, so as to be able to express it in number of days.

47.          The result of the operation indicates the average prospective duration[12] in days. DT can also be calculated as the ratio between the sets of proceedings pending at the end of the period [PE] and the sets of proceedings resolved during the reference period [R]. This formula corresponds to an intuitive logic. Where 100 cases are pending at the end of a given year and 50 cases are resolved during that year, it follows that the pending cases will be solved within two years.

ü     Rate of resolved cases within the established timeframes:[13] percentage of completed cases within a certain period of time. The period of time is the period regarded as the desirable, reasonable length of proceedings, or which is stipulated as such by law.

48.          For instance, the calculation of the average duration of 1,000 proceedings of cases resolved within one year by a given court shows that the average duration is of one year. Considering the breakdown of the respective durations and the matters considered in the individual cases, there will be a considerable range of issues discussed and a large spread of time-scales. There will be very complicated cases, which may take longer that the average time of one year, and other cases, typically non-litigious which are handled very quickly in a few days. The mere arithmetical average is therefore not representative of the reality and of the variety of the situations observed. In order to avoid this inconvenience, a very common practice consists in indicating the resolution times for proceedings both according to similar categories of cases, and according to categories of maximum duration, as in the table provided below which stands as an example and is non-prescriptive.

Percentage of cases resolved:	%
Within 12 months	90%
Between 12 and 18 months	5%
More than 18 months	5%

49.          A judicial system, or an element thereof, provides a service of quality if most cases are handled within a reasonable time, which may be established by law or by practice; a very limited proportion of cases with a longer duration is permitted as a tolerance category including only a few number of cases which, for imponderable reasons, cannot be processed within the pre-determined time limits^[14].

ü     Age of pending proceedings:[15] number of cases that are pending on a given date, grouped according to the year in which they started. Alternatively, the percentage of pending cases that have been open for longer than a set time limit (for example, percentage of cases pending for more than three years).

50.          Since they are undifferentiated, the problem of global figures is to make individual units appear identical. In fact, a correct statistical analysis of the figures in question will reveal increasingly detailed information. Let us consider the civil or criminal backlog of a court or a judicial system as a whole. The backlog has a varied and anomalous structure, which differs from court to court; it is particularly pronounced for courts with an insufficient number of judges and registry staff. It is possible to take into consideration the different types of cases included in this stock by introducing specific legal subject matters (going beyond the simple difference between “litigious cases” and “non litigious cases”).

51.          For example, the following table is an illustrative example of how the backlog of cases (pending at 31 December 2016) within a court or entire system may be graphically represented.

52.          In contrast to the previous indicator - which is closely related to this one - measuring the duration of proceedings that have already been resolved, this indicator measures the period of time for which proceedings that are still pending have been on-going, and therefore enables the general duration of cases to be assessed, distinguishing between structurally normal pending proceedings (for instance, in the case above it would be the class of up to 2 years) and structurally abnormal pending proceedings (the sum total of proceedings that have been pending for more than two years).

ü     Number of adjournments during the proceedings: number of adjournments decided prior to the solving of the case.

53.          This indicator may be calculated as the simple average of the adjournments recorded for each case.

54.          There can be many reasons for adjournments. If we examine them all it is clear that responsibility may be, to varying degrees, equally shared between all participants in the trial. These range from service errors committed by registries to absences of judges, which are not always justified and mean that panels cannot be convened by using the agreements of counsels - which are necessary in some cases although, in others, result merely from delaying tactics. Obviously, there is also no lack of cases in which witnesses or the accused themselves fail to appear.

55.          The adjournment of a hearing is not merely an intrinsic element of inefficiency within a judicial system. It is also a shortcoming which causes harm to users. There is a large number of cases in which witnesses have to appear at hearings which are held hundreds of kilometres from where they live, and only find out about the adjournment upon arrival. The number of lawyers who spend time and money in order to reach the location of the trial only to see their efforts thwarted due to an adjournment is even greater.

ü     Efficiency rate (ER indicator): relation between the number of personnel used in a court in a year and the output of cases from the same court at the end of the year.[16] Typically, this indicator is applied to homogeneous categories of activity such as civil justice, criminal sector, administrative tasks, etc.

56.          ER measures the quantity of output produced by a productive unit over a unit of time. The most productive of the two divisions of a court dealing with similar matters and with the same number of judges and administrative staff will be the division that resolves the greater number of cases within the specified period.

57.          The implementation of the concept of efficiency and productivity in the judicial sector requires, however, several considerations.

58.          The result of the activity of the judiciary is not the production of bolts or other consumer goods that are, to a greater or lesser extent, non-essential. Within the civil sector, the activity of the judiciary relates to the protection and exercise of the rights of individuals. In the criminal sector, it involves investigation and trial with the aim of establishing the guilt or innocence of a defendant accused of an offence.

59.          Any reasoning concerning the concept of productivity in justice, as for any other concept derived from business that is aimed at ensuring an efficient judicial service, must at all times take account the difficulty of the task assigned of the judiciary and the importance of protecting human rights.

60.          However, this does not alter the fact that justice may be considered as a service provided to citizens, and that as such is comparable with a large number of other services provided by the public administration, which are no less important or essential for living in a community. It is therefore from the service perspective that the efficiency of justice can and should be measured.

61.          The efficiency of the judiciary must relate to measurable acts within homogeneous categories and without considering the underlying activity which makes up the routine work of the profession.

62.          The main indicator used at the international level focuses on the number of pending cases which were solved, in general without considering the composition of any intermediate acts or decisions. It is consequently the number of decisions (judgment or other definitive act) made by an individual judge, division or court within the reference period that is significant for the purposes of productivity.

ü     Structural and organisational indicators: number of judges in relation to the population, judges’ workload, number of lawyers in comparison with the number of judges, administrative staff in comparison with to judges

63.          There are a series of very useful structural and organisational indicators to assess the adequacy of resources – both human and material – with regard to the efficiency and efficacy of judicial services. These include, for example:

·                     the ratio of judges reported to the population;^[17]

·                     the ratio between the number of judges  working on a specific category of cases and the number of resolved cases per category;

·                     the ratio of judges reported  to lawyers;

·                     the number of judges in comparison with the number of non judicial staff.^[18]

64.           If a performance indicator  is directly indicative of the quality of service,  for example, in the case of the duration of proceedings, when the time to process a case is short, the increase or decrease of structural indicators do not necessarily mean that justice is of better quality.

65.          However, a comparison between these indicators and the search for any homogeneity or correct balance between them undoubtedly reveals an intention to improve quality. If it is discovered that some courts within a judicial system have a ratio of 1:2 of judges to administrative staff (meaning that there are two administrative staff members for each judge), whilst other courts have a ratio of 1:4, this should encourage those responsible for the organisation to act in such a manner as to bring the ratio in both courts towards an average figure of 1:3.

ü      Efficiency of management, costs of justice and cost per procedure

66.          The concept of the efficient management of resources is cited with increasing frequency by governments and administrative authorities. The efficiency of the judicial system is associated in a very simple manner  to the mere reduction of costs. In reality, a service or activity is more efficient when, assuming equal efficacy (i.e. equal results), it costs less or uses fewer resources. Consequently, in order to achieve efficient management, it is not sufficient to reduce costs, but it is necessary to maintain, if not improve, the level and the quality of the service delivered.

67.          In any case, any approach that seeks to measure the quality of justice, in addition to the performance indicators examined above, must be based on a good understanding of the administration machinery and on the analysis of costs, of its products and services.

68.          The exercise is complex and involves an approach that is not very widespread within the different public administrations, namely the creation of a management control system that makes it possible to:

·                     measure judicial phenomena in time and space, with space being understood as both national space, with a comparison between courts, and  the international dimension with comparative criteria between judicial systems;

·                     ensure the optimal allocation of resources;

·                     set objectives and measure disparities between objectives and results obtained;

·                     suggest adapted measures necessary  to correct these disparities.

69.          In this regard, the cost of each handled case represents an interesting indicator to measure the efficiency of a judicial system. A fundamental indicator to measure the efficiency of the judicial system is the cost of each case processed. There are two calculation methods. The first one, relatively straightforward, involves dividing the total cost of a productive unit (division, court, judicial system) by the number of cases resolved, thereby obtaining a generic average cost per procedure. The second one, which is much more complex for many public administrations (which historically did not keep detailed accounts and do not have a modern management control system), involves calculating the effective cost generated by each case. This involves linking up each case and each trial with all of the costs of the resources dedicated to it the number of hours spent by the officiating judge and the administrative staff involved to the expenses incurred and so on.

70.          It is very clear that an important trial spread over several years, which has involved various hearings with the participation of defendants, witnesses, parties’ experts, interpreters, etc., will certainly cost the State more than a short trial under summary proceedings that is swiftly concluded by a judgment.

71.          The adoption of a cost accounting system which can show all of the costs relating to an individual case is an expensive activity both financially and in terms of human resources. Even if such an information system  existed, it would constantly need to be fed with information not only by those directly involved, who would have to record the time spent on a particular case, but also  by a  specialised staff responsible for allocating all fixed costs (rent, consumables, etc.) and variable costs (expert appraisals, travel expenses, safe custody charges, etc.)  for the different cases . Particular attention should obviously be paid to ensuring that the same cost units are used for comparison purposes.

ü     Effectiveness of the appeal system

72.          The effectiveness of the appeal system is an indicator to be “handled with care" as in some circumstances it may be taken as an indirect measure of the quality of decisions, assuming that poor quality decisions would be more likely to bring an appeal.

73.          This principle has ignited a debate among those who expressed strong criticisms on all forms of quality measurement based on numerical relations between the degrees of judgment and those who consider it as a valid objective criterion. The opponents argue that in many cases the appeal is not always based on matters of error or poor quality of first instance decision but that an appeal may be the result of a tactical behaviour independent from the quality of the decision.^[19] The numerous cases of appeals instituted only in order to cause delays are an example of that.

74.          If we agree that the ratio of appeals is not in itself an indicator of quality, on the other hand it must be recognised that a strong difference in this value between comparable areas of measurement is at least an indicator of anomaly. For example, two sections in a court that deal with the same matters showing very different ratios of appeals in a period of time do represent a situation that requires further investigation. It should not be forgotten that the difference in treatment is in itself an element of poor quality of the system as it undermines the principle of fairness of trial.

75.          We propose two indicators: the first measures the percentage of appeals against the total number of judgments at first instance, a calculation to make taking into consideration that the denominator of the fraction will be the number of those only decisions for which the appeal is admissible. In addition, we recommend taking data of several years as a reference, as a single’s year results could be influenced by external or temporary factors having a strong impact on the indicator.

       Appeal Rate (AR)

76.          Where A is the number of appeals recorded in a period of time and DFI the number of decisions in first instance recorded in the same time interval.

77.          The second indicator measures the  proportion of appeals that confirm the decision at first instance.

       Held Appeal Rate (HAR)

78.          Where AH is the number of appeals held i.e. that confirm the decision in first instance and A the number of appeals recorded in the time reference.

79.          It may be noted that the judgments of  second degree, both in  civil ,and criminal law, are often structured so as not to determine uniqueness if it is a full confirmation of the decision of first instance or a full or partial reform of it. It is evident that the indicator refers to full confirmations or even partial reforms that confirm the substance of the decision of the previous degree. This concept, which would require the establishment of additional criteria, is not simple for its application to concrete cases.

80.          It also appears significant to use both indicators together, since the interpretation of only one of them could lead to inappropriate conclusions. In this regard, the observation of the values relating to the number of appeals together with the outcome of the same ones - whether of reform or confirmation of the decision taken the first instance - appears to offer more useful elements for the analysis of quality. Both TA and TDCA should be considered in conjunction to be really useful, and not in an isolated fashion.

5.            Customer satisfaction surveys

81.          Regardless of the method adopted to measure the quality of a service, it is important to consider the opinion of users.

82.          The concepts of quality and customer satisfaction are usually associated with the production of goods and the provision of services in the private sector. A company will find it difficult to survive on the market if it does not question its clients and if it does not satisfy their needs as best it can, setting itself apart from its competitors and thereby achieving a competitive advantage.

83.          It may appear to be rather strange to refer to the public administration in such terms, the primary characteristic of which is that it does not have to distinguish with competitors on a daily basis. And yet a stance that does not give primary importance to the interest and satisfaction of the “citizen-user” traditionally has clear negative consequences. The perception of a growing distance between the needs of citizens and services actually provided means that it is essential to consider this aspect.

84.          In this perspective, the CEPEJ hopes that satisfaction surveys will be carried out at the national level or at the level of individual courts within a political framework aimed at introducing a culture of quality into European judicial systems. This approach reflects a concept of justice that is focused on users of the “service” in addition to the performance of the judicial system.

85.          From a previous experience at the Court of Geneva, the CEPEJ Working Group on Quality of Justice (CEPEJ-GT-QUAL) produced in 2010 a “Handbook for conducting satisfaction surveys aimed at court users in Council of Europe member States” which was updated in 2016. This handbook, which is available on the website of the Council of Europe along with other documents drawn up by that body, contains a model questionnaire (see Appendix II to this document), which may be used, subject to appropriate adjustments, by any court or judicial system that wishes to test the level of satisfaction of the individuals who make contact with it in various capacities.

86.          Several member States within the Council of Europe have used the opportunity to carry out several surveys, among them Albania, Italy, Finland, Germany, Lithuania, Poland and Slovenia.

87.          From a methodological point of view, the Italian Ministry of Justice has developed the project following two main approaches: the CEPEJ handbook and the SERVQUAL method (Parasuraman A., Zeithaml V.A., Berry L.L., 1985), and the latter still remains one of the main reference within the academic and professional domain for measuring the quality of service.

88.          The SERVQUAL method, which uses ad hoc or ongoing quantitative market research concerning clients, enables the quality of a service or of a group of services provided by an undertaking to be measured precisely and monitored over time (Visconti G., 2007). With the appropriate considerations and adjustments, the instrument may easily be applied within the public domain.

89.          The SERVQUAL model is based on the concept of a gap between the expectations and perceptions of users. The measurement of the disparity between expectations, i.e. what the client expects to receive, and perceptions, i.e. what he or she considers to have received, is fundamental in order to assess customer satisfaction (Figini M., 2003).

90.         

The greater the disparity between expectations and perception, the lower the user satisfaction will be, and vice versa.

91.          In order to be complete, a customer satisfaction survey should take into account two types of clients:

A.            external clients or users, for instance in our case the end users of the judicial system (citizens, the parties to a trial, etc.);

B.            internal clients, i.e. the employees of a company who through their work and efforts represent the main players involved in the provision of the service.

92.          In our case, the category of internal clients includes the employees of the courts, including first and foremost administrative staff.

93.          Lawyers may fall into both categories as they perform a dual role, on the one hand with their active role in the “judicial system” and, on the other, as users of the justice system as a service. For this reason, so far in Italy, while the survey carried out for lawyers kept the questionnaire essentially unchanged, the process by which it was carried out was different from the surveys directed at users and administrative staff.

94.          In fact, internal clients should be subject to the same survey as external clients in order to assess how much the internal culture i.e. that of the service provider differs from the expectations and perceptions of the user. It is in fact clear that, for example, if the perception of employees concerning the quality of the service provided is low and users consider the service received to be unsatisfactory, then there will be ample room for improvement. It would be a serious matter to discover that employees consider that they are providing an excellent service while citizens remain unsatisfied. In this case the employee would not take any steps to achieve improvements on his or her own initiative as he or she would be convinced that the service provided was already satisfactory^[20].

95.          A customer satisfaction survey is pointless if there is no will or possibility to implement the necessary changes to meet the users’ expectations that have been identified. In this case, in addition to being a waste of resources, it would end up being a further source of frustration for the (internal and external) “client”, who would see him- or herself involved in an activity that was an end to itself. It is therefore necessary,  to start with, to disseminate broadly  the results of this type of survey  for users to gain feedback on the results obtained of the surveys in which they took part and  then to take at least some of the necessary operational  measures to fill the gaps identified between expectations and perceptions.

96.          The operational measures which a customer satisfaction survey may normally establish are directed both at external customers, i.e. users, and at internal customers, i.e. service providers.

97.          With respect to users, improvements may be broken down  by sector where improvement groups could be constituted to address processes and their problems by analysing all aspects in order to find an optimal solution (Figini M., 2003).

98.          Similarly, as regards internal customers, including first and foremost employees, it is possible to consider creating improvement groups focusing on training and raising the awareness of external clients.  Naturally, since such cases concern public administration, it is very often impossible to use one of the principal levers: the financial incentive. It is nevertheless possible to attempt to increase the level of staff involvement, at minimum, to start with, by disseminating widely results of the surveys by implementing changes in the management of resources, including above all human resources, and putting in place ad hoc training activities focused on issues such as relations with users, communication, internal relations and the organisation of roles.

99.          A very delicate aspect of user satisfaction surveys is the collection of data which may be carried out using various methods, with varying degrees of relevance and of costs. The methods used can range from face-to-face interviews – a method most commonly used for surveys in the judicial sector – to questionnaires to be filled in independently by visitors to court buildings, or even opinion surveys sent to users by e-mail.

100.        The questionnaire proposed by the CEPEJ and other models used in Europe and the United States are typically structured around the following variables:

vService

ØGeneral organisation

ØDelays / red tape

ØOpening hours to the public

ØCosts

ØAccess to information

vJudges / Public Prosecutors

ØCompetence

ØImpartiality

ØAbility to listen to the parties

ØTime available before the judge

ØClarity of judgments

vStaff (registrars / administrative staff)

ØCompetence

ØCourtesy

ØAbility to listen

vStructure

ØOrganisation of the court (registries, offices,…)

ØBuilding (external/internal accessibility,…)

ØOther (noise, cleanliness,…)

101.        The grading scale proposed by the CEPEJ contains 5 points and is in line with the one proposed by other bodies (1 Not satisfied, 2 Little satisfied, 3 Average satisfied, 4 Satisfied, 5 Very Satisfied). The core issue is to decide whether to adopt a scale with an odd number of options, which has the special characteristic of having a central point of average satisfaction, or a scale with an even number of options, which forces interviewees to decide whether they are satisfied or dissatisfied.

102.        Here is a extract of a model questionnaire for a court where socio-demographic questions have been removed along with those used in order to identify the service received by the user interviewed (civil or criminal sector, location of the court visited, type of procedure or service as reason for visiting the court). It is worth noting that the questionnaire developed by the CEPEJ in the Handbook on conducting satisfaction surveys focuses on the same issues (see Appendix 2).

6.            Concluding remarks

103.        As noted above, “quality” is a term with a very broad range of meanings as it can be used to describe objects, individuals and experiences. When moving from the everyday common meaning to a more technical concept of quality, with the objective of assessing goods or services to compare them with others, one enters into the difficult field of “measuring quality”.

104.        The concept of measuring quality is of particular interest in the field of justice and has been subject  to a lively debate at the European level, as there is a strong demand for systems of measurement that take account of multiple  aspects. More and more judicial systems have developed new information systems in an attempt to respond to this demand. These have very often improved existing statistical systems for measuring and managing performance in a more detailed and sophisticated manner. On some occasions, this approach has led to information systems that are modern but limited in scope compared with the broad spread of indicators which quality measurement requires.

105.        This document has accordingly attempted to offer a broad and comprehensive approach to the issue of the quality of the judicial service, and a measurement model that is not only theoretical but also concrete, which does not seek to enter into the merits of decisions but takes account of a large number of factors relating to judicial organisation and performance: from self-verification of the core structure of the organisation and the judicial order (quality checklist) to service performance (service indicators) through to the opinions expressed by users – whether internal (employees) or external (users) – in customer satisfaction surveys.

106.        This document therefore offers an approach, a method and a very rich set of indicators which are made available to readers who may wish to draw on all, or some, of the proposals made here in order to measure the quality of judicial services.

Appendices

Appendix 1

Checklist for promoting the quality of justice and the courts

(S: State; R: Region; C: Court;J: Judge, N.A.:not applicable)

	S	R	C	J	N.A.
I. STRATEGY AND POLICY
I.1. Judicial organisation and policy
1.   Is there a public authority (Ministry of Justice or High Council for the Judiciary) responsible for drafting general policies and strategic documents concerning the judiciary?
2.   Is there legislation supporting the courts or court organisation?
3.   Is there a guarantee at constitutional level (or at the highest level of the hierarchy of norms) to protect the independence of the judiciary vis-à- vis the executive and legislative powers?
4.   Is there a policy regarding the specialisation of courts and/or certain categories of judges?
5.   Are (performance) targets defined for courts?
6.   Is there a strategy and policy regarding the needs and planning of court resources?
7.   Is there a policy concerning the structure and competence of courts, including geographical court location policy?
I.2. Mission, strategy, objectives
1.   Has the court management defined a mission/vision and a strategy (basic characteristics of the judiciary are to be incorporated in this, such as impartiality, independence, legal certainty and access)?
2.   Does the court management give wide publicity to the mission/vision and strategy among stakeholders, judges and prosecutors and court staff?
3.   Does the court management translate the mission/vision into concrete and measurable objectives and priorities? Does it have performance indicators?
4.   Does the court management determine critical success factors for achieving these objectives?
5.   Does the court management take the expectations of the legitimate needs and wishes of the internal and external stakeholders into account when drafting a court policy?
6.   Does the court management maintain a systematic contact with the internal and external stakeholders?
7.   Does the court management ensure a culture that is  aimed at stimulating and inspiring improvements in the overall organisation?
8.   Has the court management determined the priorities on which court policies should be developed?
9.     Has the court management described how the decision-making process on these priorities should take place?
I.3. Allocation of cases and delegation of responsibilities from judges to non-judges staff
1.   Does it exist a system to monitor the workload of each judge continuously?
2.   Does the court have the possibility to reassign cases or assignments in order to increase efficiency in the court? Is the court able to establish a flexibility among judges that allows such reassignments?
3.   Has the court management drafted a policy regarding the delegation of responsibilities from judges to non-judge staff?
4.   Has the court management defined an objective method for allocating cases between judges?
5.   Is the information on the allocation of cases made available to the whole court organisation?
6.   Has the court management determined the main tasks, role and standards for the court clerk office?
I.4. Evaluation of the strategy
1.   Is there a system for assessing the management of strategic risks?
2.   Is the implementation of policies concerning changes in the structure of the court organisation regularly evaluated?
3.   Is the implementation of changes in legislation regularly evaluated?
4.   Are changes in legislation and their impact on courts and/or judges / prosecutors evaluated?
5.   Are the effectiveness and efficiency of judicial and ADR-proceedings systematically evaluated?

II. JOB AND OPERATIONAL PROCESS
II.1.Legislation
1.   Are quality standards and guidelines used for drafting new legislation or changes in current legislation?
2.   Is the impact of the introduction of new legislation or changes in current legislation on the workload of courts assessed? If yes, does this lead to changes in the (staff) capacity of courts?
3.   Are legislative proposals presented by the executive to Parliament reviewed and commented by independent authorities and the judiciary as a part of the legislation process?
4.   Are procedural laws (civil, criminal, administrative) regularly reviewed and modified to increase effectiveness and efficiency of court proceedings?
5.   Is there specific legislation (formal and procedural laws) for the use of ADR?
II.2. Court proceedings
1.   Are measures taken to ensure a fair and efficient allocation of workload between judges (i.e. follow up of case flows, of the number of cases entrusted to each judge, of the speed of processing a case, stock-taking of external activities, etc.)?
2.   Are measures taken to ensure adequacy between the judges’ functions and the files entrusted to them (training periods, specialisation, regrouping of cases, “test files”, etc.)?
3.   Are measures taken to ensure transparency in the allocation of files to judges (i.e. initial and public objective criteria)?
4.   Is there an established policy concerning the processing of cases by a single judge or by a panel of judges?
II.3 Legal certainty
1.   Is there a policy regarding the promotion of legal certainty?
2.   Are there specific instruments used to promote legal certainty, for example an internal system for jurisprudence or the organisation of meetings to discuss relevant jurisprudence?

II.4. Management of cases
1.   Does each judge have specific tools which enable him/her to know - in real time - the state of the pending cases within his/her department?
2.   Is he/she able to share this information with his/her administrative staff?
3.   Is this information shared within the court?
4.   Can judges take alternative, yet non-coercive measures to solve conflicts during a pending proceeding?
5.   Are court proceedings (in principle) open to the public?
6.   Are the proceedings organised in an expedient manner to solve the conflict?
7.   Are the proceedings arranged and carried out in such a manner that the expenses for the parties and others involved in the proceedings are minimalised?

	S		R	C		J	N.A.
8.   Are measures taken so that the parties and lawyers have confidence that judges are preparing their cases properly, have sufficient expertise to address the case and that their position has fully been understood?
9.   Do judges /prosecutors have the competence/authority to hand over certain disputes to mediators?
10.  Does it exist routines to safeguard that mediation does not delay the case unnecessarily?
II.5. Management of hearings
1.   Is there a policy for preparing the hearings?
2.   Is a court hearing scheduled within some days after having received the case, in cooperation with the counsels of the parties, to decide on the duration of the proceedings and the time needed to prepare for the main court hearing?
3.   Is there a system for measuring the timely start of hearings?
4.   Are parties informed when the hearing is adjourned or delayed?
5.   Is there an information system which is used for determining an efficient schedule of court sessions?
6.   As regards judges :
-    do they prepare court files in an appropriate way, bearing in mind oral investigation?
-    do they have the ability to improve the understanding of their role by the various players in the proceeding?
-    do they control the allocation of the parties and witnesses’ speaking time?
-    do they control the police of the trial in an appropriate way?
-    do they take into account the parties and witnesses’ expectations within the oral phase of the proceeding?
-    do they control the timetable of the proceeding?
-    are they punctual?
7.   Are summonses for hearings sent at the earliest period possible, to avoid unnecessary waiting time (scheduled appointments, time slot, etc) ?
8.   Have the parties the opportunity to request priority treatment of the case if there are legitimate reasons given?
II.6. Management of timeframes
1.   Is there a policy for setting foreseeable and optimum timeframes?
2.   Are standards or norms concerning the acceptable length of judicial proceedings defined?
3.   Is there a policy for managing case flows preventing delays?
4.   Are measures taken to speed up delayed cases and to reduce the backlog?
5.   Is there an active role for the judge in the management of the timeliness of the proceedings?
6.   Can parties negotiate with the court on the timeframes that will be used
7.   Is there a timeframe set for delivering the decision after the court hearing?
II.7. Execution of judicial decisions
1.   Is there a policy concerning the execution of judicial decisions?
2.   Is there a system of notification of judicial decisions ?
3.   Is there a maximum timeframe defined between the final decision of a judge and the notification of the decision to the parties?
4.   Is the timeframe between the final decision of a judge and the execution of the judicial decision periodically monitored?
5.   If the execution of decisions is entrusted to members of a specific profession (bailiffs, etc), are they supervised by the judicial authorities?

	S	R		C	J		N.A.
II.8. Partners of justice
1.   Is there an up-to-date list of court experts, interpreters that can be consulted?
2.   Does the court collaborate with other institutions (police, lawyers, public prosecutors, social workers, custodians, experts, etc.)?
3.   Is there an up-to-date list of custodians?
4.   Is there a system of quality control for experts and court interpreters?
5.   Are fixed deadlines defined for receiving an expert report?
6.   Is there a possibility of challenging the result of an expert report?
7.   Are the experts and court interpreters certificated?
II.9. Management of files and archiving
1.   Does a case management information system exist for the recording and monitoring of court files and cases?
2.   Is there a specific policy concerning archiving of court files and court decisions?
3.   Does an (electronic) information system for archiving court cases and decisions exist?
4.   Does a court system of electronic files exist?
5.   Is it possible to submit documents to the court in electronic form?
II.10 Evaluation of performance
1.   Is there a system for assessing operational risks and the quality of the internal supervision of courts by court managers?
2.   Is the management of operational risks :
-     risks of loss of public trust in the judiciary (relationship with the media, communication management by judges/prosecutors, etc.)?
-     risks linked to the reliability of procedures (in particular concerning information systems)?
taken into account in the (court) policies?
3.   Does the court management periodically evaluate court performance?
4.   Is there a policy on the publication of the evaluation results?
5.   Are quality regulations and standards periodically evaluated?
6.   Following the evaluation results, have measures been identified and implemented to improve the situation? are these improvements monitored?
7.   Is the percentage of cases with a full-bench division (panel of judges) recorded and published?
8.   Is the number of successful challenges recorded and published?
9.   Is the percentage of appeals recorded and published?
10.  Is the productivity of judges and court staff recorded and published?
11.  Is the percentage of quashes recorded?
12.  Is the length of proceedings systematically recorded and published?
13.  Is it possible to determine the total number of incoming, pending and decided cases in a given period?
14.  Is the nature of pending cases systematically analysed?
15.  Have objectives been determined for reducing the backlog of cases?
16.  Does a quantitative and qualitative evaluation system regarding the activity of each judge exist?
17.  Is it possible to present information on the number of pending cases and decided cases by an individual judge at any given time?
18.  Is each judge granted access to the information regarding his/her own court department, his/her colleagues’ department as well as to the data regarding the whole court?
19.  Are the qualitative aspects of the performance of individual judges also part of the court human resources policy?

		S		R		C		J		N.A.
III. ACCESS TO JUSTICE, COMMUNICATION TO CITIZENS AND PUBLIC
III.1. Access to legal and court information
1.   Are laws published in such a manner that they are easily accessible?
2.   Are there free (non-fee paying) Internet sites providing access to legal texts?
3    Is the reception staff trained to explain the working methods, rules of procedure and other practical information to court visitors and users?
4.   Are court judgments and decisions accessible on court internet sites?
5.   Is there a policy regarding the publication of court decisions?
6.   Do people speaking/reading minority languages have access to an official version of the legal texts in their own language?
7.   Are persons who do not understand the official language used in judicial proceedings entitled to an interpreter (free of charge)?
8.   Do the courts have an interpreting service or can interpreters be called upon rapidly?
9.   Is information on the functioning of courts available and easily accessible to citizens?
10.  Is information concerning the rights and obligations of citizens (as stated in the law) widely available to them (for example via a general telephone number)?
11.  Is this information adapted in its content to the wide range of existing situations (children at risk, divorces, criminal proceedings, detention locations, etc.)?
12.  Does the court have an information desk for court visitors?
13.  Is there an up-to-date list of lawyers/barristers available at the court reception and/or on its website?
14.  Are any information leaflets available for the users in the court?
15.  Can a litigant be present or be represented during all levels of proceedings?
16.  When a litigant is represented by a lawyer, is this representation a monopoly of lawyers?
17.  When lawyers do not have the monopoly of representation, is there a possibility that associations or trade unions offer legal advice and assistance to litigants (for example in social matters or consumer law)?
III.2. Financial access
1.   Are litigants without the necessary financial means entitled to free legal consultations or consultations at a reduced price in order to be informed on their (civil) rights and duties? If yes, is this the case in all areas of the law?
2.   Are litigants able to receive free legal representation or legal representation at a reduced price (financed by the governments’ legal aid budget) of a lawyer? Is this applicable only to criminal matters or does it apply to all the other areas?
3.   Are the costs/fees for a proceeding transparent?
4.   Is there a system which guarantees the moderation of the costs/fees for a proceeding?
5.   Are there general rules concerning the payment of court fees or court taxes in the criminal proceedings? other than criminal proceedings?
6.   Do members of the bar association hold free legal consultations?
7.   In an effort to ensure the public is aware of the cost of proceedings:       -     are lawyers/barristers required to publicise the fees they will charge and to       establish contracts with their clients?
-     are there legal procedures for challenging excessive fees charged by lawyers/barristers?
-     are there legal procedures for challenging excessive fees charged by experts?
8.   Is there a (legal) possibility to challenge the fees charged by lawyers/barristers and experts processed?

		S		R		C		J		N.A.
III.3. Physical and virtual access
1.   Are courts located so that they are effectively accessible?
2.   Is there a provision to hold hearings in other locations away from the main seat of the court?
3.   Are reception staff properly trained to take the stress of persons summoned into account?
4.   Has the court drafted a special charter to improve the reception of visitors?
5.   Do people with disabilities or elderly people have easy access to
-     reserved parking spaces?
-     access ramps into buildings?
6.   If necessary, is there a possibility that someone may accompany them to the courtrooms?
7.   Are the waiting and hearing rooms properly equipped and of a reasonable standard?
8.   Are there rooms in the court where the lawyers can meet with their clients?
9.   Are the waiting rooms organised so that the opposite parties do not have to wait together?
10.  Are there clear signs for visitors entering court buildings?
11.  Is there a policy for the use of ADR?
12.  Are mediators easy accessible to resolve certain disputes?
III.4. Treatment of parties
1.   When a litigant appears in person, do judges and other staff have sufficient time and training to provide parties with basic explanations about the disputes to which they are a party?
2.   Is appropriate advice provided to the participants in the proceedings, while still maintaining the impartiality and fairness of the court?
3    Are the participants in proceedings as well as the public treated so that their dignity is preserved?
4.   Are judges capable of ensuring the needs of persons summoned understand the legal language of the proceedings?
5.   Do judges take into account the cost of proceedings for the parties by:
-     limiting the measures to be taken (expert report, payment into court etc)?
-     giving priority to cases which have a direct impact on the parties’ resources   (dismissal, alimony etc)?
6.   Do judges ask those present at the hearing to indicate any reasons why they should be given priority or if they have any special requirements (e.g. people unable to stand)?
7.   Do judges organise their hearings in such a manner that people can be heard at specific times?
8.   Are parties allowed to intervene, in particular to ask for explanations?
9.   Is there a public complaints procedure?
III.5. Presentation of decisions
1.   Are the pronouncement and the reasons for the decision made by the judge comprehensible?
2.   Are the reasons for the decision detailed and systematic?
3.   Do the reasons for the decisions demonstrate a clear guidance for the parties and legal professionals of the fairness and lawfulness of the decision?
4.   Are there specific rules and standards used for the presentation of judicial decisions?
5.   Are the expectations of the parties, the lawyers, the lower or higher courts sufficiently taken into account when drafting judicial decisions?
6.   Are “standard” decisions and rules used for “bulk” cases?

S

R

C

J

N.A.

III.6. Legitimacy and public trust

1.   Is there an annual report presented to citizens on the quality and functioning of the judicial system?

2.   Is this report debated in parliament?

3.   Is there a regular evaluation of the public trust in the judiciary?

4.   Is there a regular public report on the functioning (court performance) and quality of the court?

5.   Are special enquiry committees established to conduct studies on the difficulties of the functioning of the judiciary? Is the work of these committees public?

6.   Does a court users’ charter presenting their rights and duties exist?

7.   Do parties have the possibility of receiving, at any given moment, information about the stage their proceedings have reached?

-     directly (through the reception of information or Internet)?

-     indirectly through their legal counsel (i.e. lawyer or legal representative)?

8.   Is information on the system of disciplinary measures and sanctions imposed at the judiciary available to the general public and the court users’ and are figures made public?

9.   Do citizens play a consultative role in discussing the priorities of the judicial system (financing, priority given to certain disputes, etc.)?

10.  Are associations whose social role relates to the judicial system (victims, consumers, etc.) able to play a particular role in improving the functioning of justice?

11.  Are there regular exchanges of views on the functioning and quality of the courts at local level (public debates, meetings with associations), reception of school children, etc.) ?

12.  Does the court have a special officer trained in dealing with the press?

13.  Are any relevant documents of consensus which are the result of consultations between court judges and other legal professionals setting out rules of conduct or organisational arrangements agreed by all published?

14.  Are there open days organised for citizens to visit the courts?

III.7. Evaluation

1.   Is there an assessment/evaluation system for measuring a (potential) loss of public trust in the judiciary?

2.   Is a potential risk of loss of public trust in the judiciary taken into account in the court policies (relationship with the media, communication management by the judges/prosecutors, etc.)?

3.   Have the relevant users been identified (users include litigants, lawyers, public prosecutors, probation and after-care service, interpreters, the Child Protection Board, experts, etc.).

4.   Is the court users’ satisfaction periodically evaluated?

5.   Are the evaluation results of the users’ satisfaction surveys made public?

6.   Is progress on this subject monitored on the basis of the results of such assessments (the topics on which the user could be questioned could be for example: treatment by the judge and the latter’s behaviour, the court’s infrastructure and services, delay before the trial, impression of legal certainty and readability of the decision)? Are these made use of to improve on the functioning of the courts?

IV. HUMAN RESSOURCES AND STATUS OF THE JUDICIARY AND STAFF

IV.1. Human Resources Policy

1.   Is there a long-term strategy and policy concerning the recruitment, selection, training, evaluation, career development and salary of the judiciary and court staff?

2.   Is there a short term policy concerning the recruitment, selection, training, evaluation, career development, salary mobility of the judiciary and the staff?

		S		R		C		J		N.A.
3.   Does an independent national training institute for judges and prosecutors exist (judicial school)?
4.   Is there a policy concerning knowledge sharing between courts and judges?
5.   Is the remuneration of judges and prosecutors regulated by law?
6.   Does an evaluation system for judges and prosecutors exist?
7.   Do judges and prosecutors know the evaluation criteria applicable to them?
8.   Are the following topics included in the evaluation criteria of judges and prosecutors?
-     personal and professional integrity of judges and prosecutors;
-     appropriate behaviour when dealing with the media;
-     appropriate behaviour regarding political and trade union activities;
-     independence vis-à-vis media and politics;
-     treatment of parties;
-     professional competencies.
9.   Are the evaluation criteria for judges and prosecutors clear enough?
10.  Are the skills of candidates for the position of a judge or a prosecutor evaluated when entering the judiciary?
11.  Is the personal ethical behaviour of future judges and prosecutors evaluated before entering the judiciary?
12.  Are there objective criteria for selecting future judges and prosecutors?
13.  Are these criteria known to the candidates?
14.  Does an evaluation system of non judge / prosecutor staff exist?
15.  Are the criteria applied in this system known to the staff?
IV.2 Status and competences of the judiciary
1.   Is the status and position of judges and prosecutors established in legislation?
2.   Are the main competences of judges and prosecutors described in general policy documents or laid down in legislation?
3.   Are judges and prosecutors encouraged to establish best practices and codes of conduct?
4.   Is the protection of the independent position of judges described in legislation?
5.   Is there a Council for the judiciary? Does this Council play a role in strengthening the independence of the judiciary?
IV.3. Training and development of competences
1.   Does the court management stimulate co-operation between the departments within a court?
2.   Does the court management keep track of the requirements regarding professional knowledge and skills of judges and prosecutors and court staff?
3.   Has the court management developed a policy for the expertise and attitude of all the court staff members?
4.   Is there a court policy to strengthen the culture of co-operation and integrity?
5.   Does the court management conduct a policy for maintaining and stimulating judicial integrity in all levels of the court?
6.   Is there a policy regarding the deployment of deputy judges?
7.   Has the court management developed a policy regarding the specialisation of judges?
8.   Has the court management described the core competences of the staff?
9.   Are organisational skills and the techniques for managing hearings part of an initial training course when entering the judiciary?
10.  Do judges and prosecutors follow an initial and/or continuous training?
11.  Is there a standard for initial and/or continuous training?
12.  Is the personal ethical behaviour of future judges and prosecutors taught before entering the judiciary?
13.  Are questions of ethics dealt with during continuous training?
14.  Are ethics specific to particular work – like juvenile courts – dealt with in a particular way?

		S		R		C		J		N.A.
15.  Is sufficient importance given to the competencies concerning the treatment of judges and prosecutors and their attitude?
16.  Are organisational competencies and techniques for dealing with hearings dealt with in continuous training?
17.  Are drafting techniques the subject of initial training prior to/before entering the judiciary?
18.  Are drafting techniques included in the continuous training?
19.  Is the mobility of judges and prosecutors linked to acquiring the necessary knowledge for a new function?
20.  Are the specific functions – such as the chairmanship of a chamber or of a court – linked to a special training programme?
21.  Are the specific functions – such as those linked to juvenile or commercial issues – linked to a specific training programme?
22.  Does the court arrange regular judges’ meetings, quality improvement conferences and other occasions in which all judges participate and in which they have the opportunity of discussing - in addition to administrative matters - judicial matters, in particular those proposed by the judges themselves?
IV.4. Knowledge sharing, quality and ADR
1.   Does the court management promote a culture of knowledge-sharing?
2.   Are sources of legal knowledge available and easily accessible?
3.   Do judges and prosecutors practice a form of peer review (discussion of cases between colleagues) or of supervision (discussion of cases with a more qualified colleague)?
4.   Is in-camera recording acceptable as a source of information during peer reviews?
5.   Do judges participate in “quality groups” within their own court to discuss their jurisprudence in the light of the jurisprudence of higher courts?
6.   Do judges take part in discussion fora on their own rulings:
-     with colleagues from other courts?
-     with regular players, such as lawyers?
-     with other third parties?
7.   Is there a policy for discussing quashed or overruled decisions?
8.   Is there periodic consultation between lower courts and courts of appeal?
9.   Is there sufficient opportunity for the self- training of judges and prosecutors?
10.  Is there sufficient opportunity for reflecting on the decisions taken by the judges?
11.  Is there sufficient attention paid to the issue of impartiality and integrity of judges? (for instance workshops on moral dilemma or the implementation of an ethics committee).
12.  Are judges taught ADR techniques (such as mediation)?
13.  Are personal development discussions (methodical and planned) held annually with judges / prosecutors and members of staff? Are the objectives set out during these discussions achieved and followed up?
IV.5. Evaluation of the Human Resources policy
1.   Are there criteria to monitor the HR policies (for example, the criteria concerning sick leave, the efficiency of studies or training, the respect of the level of required training, and productivity) and is the HR policy regularly evaluated ?
2.   Is the judge / prosecutor and staff satisfaction periodically evaluated (for example via surveys)?
3.   Are the results of these evaluations published?

		S		R		C		J		N.A.
4.   Is the progress achieved through the human resources evaluation studies, monitored (staff satisfaction regarding, for example, workload, evaluation and performance recognition, training opportunities, career development and the supervision manner)?
5.   Is there a systematic evaluation of training and competency development policies of the judiciary and staff?
V. MEANS OF JUSTICE
V.1. Finances
1.   Is there a budgetary process in place to guarantee an adequate funding of the judicial system?
2.   Are the financial resources available for the judiciary sufficient to protect the independence of the judiciary?
3.   Are objective quality standards/norms formulated concerning the financial needs of courts, court buildings, offices in courts, technical equipment, and court security?
4.   Have operation and financial standards been set for the efficiency of the court?
5.   Is there an objective policy for the distribution of budgetary items (for example staff costs, material costs) in the court?
6.   Is there a specific budgetary item for the quality system of the court?
V.2. Information systems
1.   Is there a policy on the use of information and communication technologies in courts (e- justice, video-conferencing, electronic data exchange, etc.)?
2.   Are the court information systems regularly reviewed and improved?
3.   Are the developments of human resources- information systems in line and in conformity with the (technical) specifications of the other operational court systems (i.e. case management information systems, financial information systems, etc.)?
4.   Does the information recorded in the court management information system give an overall picture of the court’s performance?
5.   Can the analysis of the data recorded in the court management information system be performed by all the court’s staff (or authorised staff) or can only specialised staff (for example IT professionals) exploit these data?
6.   Have rules been set out concerning the confidentiality of the treatment of information (for example: prohibition to enter data in the system from one’s home)?
7.   Is the security of the information contained in the system assured (against the risk of introducing hackers into the system)?
8.   Is a rational budgetary process set up to monitor court performance and funding allocation?
V.3. Logistics and security
1.   Is there a facility for the procurement of goods and services for courts?
2.   Is there an outsourcing policy?
3.   Does the court management apply a standard purchasing procedure?
4.   Does the court management use a standard control procedure for all incoming goods and services?
5.   Does the court management periodically evaluate suppliers?
6.   Does the court management have a long-term office allocation plan?
7.   Has the court management drafted a policy regarding physical and IT security of the court?

		S		R		C		J		N.A.
8.   Has the court management drafted a policy regarding the security of all court stakeholders?
9.   Has the court management formulated a policy regarding working conditions and (in- house) emergency services?
10.  Is there a facility for the security of court buildings?
11.  Is there a facility for the security of parties at hearings?
V.4. Evaluation of means, information systems, logistics and security
1.   Is there a system of control of financial and other risks linked to information systems and support activities?
2.   Is the quality and integrity of information, in particular financial information, guaranteed?
3.   Is there a history of incidents involving the security of access, people and data?
4.   Is the security of information systems guaranteed?
5.   Is the risk of loss and material damage covered?
6.   Is the risk of fraud and embezzlement managed?
7.   Is there an annual assessment of the expenses and the impact of these expenses?
8.   Does the court management examine annually whether the expected results have been achieved (results may involve production, quality and staff)?
9.   Does the court management use the results for adapting its policy and/or amending working procedures?

Appendix 2

CEPEJ – Handbook for the conducting satisfaction surveys aimed at court users in Council of Europe member States INFORMATION for Survey managers The shaded parts are optional.

The basic questionnaire, made up of 20 closed-ended questions and one open-ended question, constitutes a standard format common to all courts in the Council of Europe member States.More specific or locally oriented questions can be added in the second section, for which a number of models are suggested. It is important to note that a usable survey must comprise a limited number of questions that users can answer quickly.

Arrangements for distributing and returning the questionnaire. Several wordings can be used:

1) If distributed within the court

Please answer this questionnaire, then place it in the box provided at the court’s reception desk, using the sealed envelope provided.

2) If sent with the court summons

Please answer this questionnaire and return it to the address on the postage-paid envelope.

Note: if the questionnaire is made available by electronic means:

You may submit your reply online to the website address appearing on the document. This site is secure and your anonymity is guaranteed.

MODEL QUESTIONNAIRE FOR COURT USERS

Dear Sir/Madam

This questionnaire is part of an assessment of the quality of the justice system, focusing more specifically on the quality of services and operation of the [type of court] in ………………………..

Your opinion and suggestions are important to us and we would be grateful if you would take a little time to reply to the questions below. The questionnaire is anonymous and we guarantee that your replies will be dealt with in the strictest confidence.

Please tick the appropriate boxes

1.   In what capacity are you [were you] at the court in ……………………………..

r₁ As a party to proceedings

r₂ As a witness

r₃ As a member of the jury

r₄ Other (e.g. family of one of the parties, requesting information, visitor, etc.)

2.   On what type of procedure was the case for which you went to the court based?

r₁ Civil procedure

r₂ Administrative procedure

r₃ Commercial procedure

r₄ Labour law

r₅ Criminal procedure

r₆ Other (minors, guardianship, pensions, register, etc.). Please specify: ………………………………………………………..

r₇ Don’t know

[If the questionnaire is specifically intended for users of court registry services]

2a Which court registry services have you used in the course of the past year?

r₁ Information on legal aid services

r Information on forms of legal action

r₃ Access to documents (e.g. copy of evidence)

r₄ Information on the court’s decisions

r₅ Information on the execution of decisions

r₆ Other; please specify: ………………………………………………………..

2b. What means of communication have you used to contact the court registry?

r₁ in person

r₂ post

r₃ telephone

r₄ fax

r₅ email

r₆ online via the court’s website

3.   Were you assisted by a lawyer?

r₁ yes

r₂ no

[optional question]

4.   What level of confidence do you have in the justice system?

Very low confidence	Low confidence	Average confidence	High confidence	Very high confidence
r1	r2	r3	r4	r5

5. If you were a party, and the decision was delivered, did the court find partially or fully in your favour?

r₁ yes, fully

r₂ yes, partly

r₃ no

r₄ I was not a party

6. Were the hearings held in your mother tongue?

r₁ yes (go to 8)

r₂ no

7. If the hearing was not held in your mother tongue, were you given an interpreter?

r₁ yes

r₂ no

7a. Was the conduct of the oral proceedings in …………………. (language) a disadvantage for you?

r₁ yes

r₂ no

8. Assess the importance you attach to the following elements:

	Not important	Not very important	Average importance	Important	Very important	No reply
8.1 Conditions of access to the court	r1	r2	r3	r4	r5	r6
8.2 Signposting in the court building	r1	r2	r3	r4	r5	r6
8.3 Waiting conditions	r1	r2	r3	r4	r5	r6
8.4 Courtroom furnishing	r1	r2	r3	r4	r5	r6
8.5 Clarity of summonses	r1	r2	r3	r4	r5	r6
8.6 The time lapse between the summons and the hearing	r1	r2	r3	r4	r5	r6
8.7 Punctuality of hearings	r1	r2	r3	r4	r5	r6
8.8 Attitude and courtesy of court staff	r1	r2	r3	r4	r5	r6
8.9 Level of competence of non-judicial court staff	r1	r2	r3	r4	r5	r6
8.10 Attitude and courtesy of judges and prosecutors	r1	r2	r3	r4	r5	r6
8.11 The language used by the judges and prosecutors	r1	r2	r3	r4	r5	r6
8.12 Time allowed to set out your arguments at the hearing	r1	r2	r3	r4	r5	r6
8.13 Timeframe for the delivery of judgments	r1	r2	r3	r4	r5	r6
8.14 Clarity of judgments	r1	r2	r3	r4	r5	r6

[Optional elements:]

8.15 Information provided by the court’s information service

r1

r2

r3

r4

r5

r6

9. Assess your degree of satisfaction with regard to the following elements

Not satisfied

Not very satisfied

Average satisfaction

Satisfied

Very satisfied

No reply

9.1 Conditions of access to the court

r1

r2

r3

r4

r5

r6

9.2 Signposting in the court building

r1

r2

r3

r4

r5

r6

9.3 Waiting conditions

r1

r2

r3

r4

r5

r6

9.4 Courtroom furnishing

r1

r2

r3

r4

r5

r6

9.5 Clarity of summonses

r1

r2

r3

r4

r5

r6

9.6 The time lapse between the summons and the hearing

r1

r2

r3

r4

r5

r6

9.7 Punctuality of hearings

r1

r2

r3

r4

r5

r6

9.8 Attitude and courtesy of court staff

r1

r2

r3

r4

r5

r6

9.9 Level of competence of non-judicial court staff

r1

r2

r3

r4

r5

r6

9.10 Attitude and courtesy of judges and prosecutors

r1

r2

r3

r4

r5

r6

9.11 The language used by the judges and prosecutors

r1

r2

r3

r4

r5

r6

9.12 The time allowed to set out your arguments at the hearing

r1

r2

r3

r4

r5

r6

9.13 Timeframe for the delivery of judgments

r1

r2

r3

r4

r5

r6

9.14 Clarity of judgments

r1

r2

r3

r4

r5

r6

[Optional elements:]

9.15 Information provided by the court’s information service

r1

r2

r3

r4

r5

r6

[[Optional questions:]

10. In general terms, what is your assessment of the operation of the courts?

Too opaque	opaque	Clear	Very clear
r1	r2	r3	r4

11. What is your assessment of the judges’ impartiality in conducting oral proceedings?

Not at all impartial	Not very impartial	Fairly impartial	Completely impartial
r1	r2	r3	r4

12. What is your assessment of the speed at which your case was dealt with by the court?

Too slow	Slow	Normal	Fast	Very fast
r1	r2	r3	r4	r5

13. Without taking into account lawyer’s fees, what is your assessment of the costs of access to justice?

Costs very low	Costs low	Costs average	High costs	Very high costs
r1	r2	r3	r4	r5

14. Based on your experience, what is your assessment of the resources available to the courts?

Very insufficient	Insufficient	Sufficient	Broadly sufficient
r1	r2	r3	r4

15. In general, how do you assess the possibility of finding out about one’s rights?

Very difficult	Quite difficult	Fairly easy	Very easy
r1	r2	r3	r4

Personal data

16. Did you use legal protection insurance? 

r₁ yes

r₂ no

17. Did you receive legal aid?

r₁ yes

r₂ no

18. Have you already been in contact with another court than the court in…………………………………..?

r₁ yes, (specify which) …………………………………………………………………………………………………….

r₂ non

19. Gender

r₁ Male

r₂ Female

20. Age

r₁ 18-30

r₂ 31-50

r₂ 51-65

r₂ 66 and over

21. Do you have any remarks or suggestions to make in connection with the operation of the court in ……………………………………………… and the justice system more generally?

MODEL QUESTIONNAIRE FOR LAWYERS Note to local survey managers. The questionnaire intended for lawyers should if possible be emailed to all members of the Bar association.

ASSESSMENT OF THE OPERATION OF THE COURTHOUSE IN …………………………………………..

BY LAWYERS OF THE BAR ASSOCIATION OF …………………………………………………………………………………………………….

Your opinions and suggestions are important to us and we would be grateful if you would take a little time to reply to the questions below. The questionnaire is anonymous and we guarantee that your replies will be dealt with in the strictest confidence.

Assess the importance you attach to the following elements

1.   General services

	Not important	Not very important	Average importance	Important	Very important	No reply
1.1 Co-ordination in setting hearing times	r1	r2	r3	r4	r5	r6
1.2 Access to the case-law of the courts of the judicial area	r1	r2	r3	r4	r5	r6
1.3 Communication between the court and lawyers	r1	r2	r3	r4	r5	r6
1.4 Clarity in terms of organisation and administrative responsibilities	r1	r2	r3	r4	r5	r6
1.5 Quality of the court’s website	r1	r2	r3	r4	r5	r6
1.6 Signposting in the court building	r1	r2	r3	r4	r5	r6

For the next questions, please only choose the court or service with which you have had the most contact (legal aid office, Family Division, juvenile court, criminal hearings department, etc.).

2.   Relations with the court or service

	Not important	Not very important	Average importance	Important	Very important	No reply
2.1 Attitude and courtesy of judges and prosecutors	r1	r2	r3	r4	r5	r6
2.2 Attitude and courtesy of court officers	r1	r2	r3	r4	r5	r6
2.3 Judges’/prosecutors’ professional competence	r1	r2	r3	r4	r5	r6
2.4 Court officers’ professional competence	r1	r2	r3	r4	r5	r6
2.5 Judges’/prosecutors’ approachability and availability	r1	r2	r3	r4	r5	r6
2.6 Court officers’ approachability and availability	r1	r2	r3	r4	r5	r6
2.7 Speed of replies to requests	r1	r2	r3	r4	r5	r6
2.8 Quality and reliability of registry’s responses	r1	r2	r3	r4	r5	r6
2.9 Computerised management of proceedings	r1	r2	r3	r4	r5	r6
2.10 Ease of file consultation	r1	r2	r3	r4	r5	r6
2.11 Clarity of responsibilities and organisation	r1	r2	r3	r4	r5	r6
2.12 Costs of/fees for access to justice	r1	r2	r3	r4	r5	r6

3.   Preparation and conduct of hearings

	Not important	Not very important	Average importance	Important	Very important	No reply
3.1 Conditions of meetings with clients	r1	r2	r3	r4	r5	r6
3.2 Furnishing and equipment of the courtroom	r1	r2	r3	r4	r5	r6
3.3 Punctuality of hearings	r1	r2	r3	r4	r5	r6
3.4 Organisation and conduct of hearings	r1	r2	r3	r4	r5	r6

4.   Judges’ decisions

	Not important	Not very important	Average importance	Important	Very important	No reply
4. Clear and comprehensible judgments	r1	r2	r3	r4	r5	r6
4.2 Rapid handling of cases	r1	r2	r3	r4	r5	r6
4.3 Decisions easy to enforce	r1	r2	r3	r4	r5	r6

Assess your degree of satisfaction with regard to the following elements

5.   General services

	Not satisfied	Not very satisfied	Average satisfaction	Satisfied	Very satisfied	No reply
5.1 Co-ordination in setting the times of hearings	r1	r2	r3	r4	r5	r6
5.2 Access to the case-law of the courts of the judicial area	r1	r2	r3	r4	r5	r6
5.3 Communication between the court and lawyers	r1	r2	r3	r4	r5	r6
5.4 Clarity in terms of organisation and administrative responsibilities	r1	r2	r3	r4	r5	r6
5.5 Quality of the court’s website	r1	r2	r3	r4	r5	r6
5.6 Signposting in the court building	r1	r2	r3	r4	r5	r6

6.   Relations with the court or service

	Not satisfied	Not very satisfied	Average satisfaction	Satisfied	Very satisfied	No reply
6.1 Attitude and courtesy of judges and prosecutors	r1	r2	r3	r4	r5	r6
6.2 Attitude and courtesy of court officers	r1	r2	r3	r4	r5	r6
6.3 Judges’/prosecutors’ professional competence	r1	r2	r3	r4	r5	r6
6.4 Court officers’ professional competence	r1	r2	r3	r4	r5	r6
6.5 Judges’/prosecutors’ approachability and availability	r1	r2	r3	r4	r5	r6
6.6 Court officers’ approachability and availability	r1	r2	r3	r4	r5	r6
6.7 Speed of replies to requests	r1	r2	r3	r4	r5	r6
6.8 Quality and reliability of registry’s responses	r1	r2	r3	r4	r5	r6
6.9 Computerised management of proceedings	r1	r2	r3	r4	r5	r6
6.10 Ease of file consultation	r1	r2	r3	r4	r5	r6
6.11 Clarity of responsibilities and organisation	r1	r2	r3	r4	r5	r6
6.12 Costs of/fees for access to justice	r1	r2	r3	r4	r5	r6

7.   Preparation and conduct of hearings

	Not satisfied	Not very satisfied	Average satisfaction	Satisfied	Very satisfied	No reply
7.1 Conditions of meetings with clients	r1	r2	r3	r4	r5	r6
7. Courtroom furnishing and equipment	r1	r2	r3	r4	r5	r6
7.3 Punctuality of hearings	r1	r2	r3	r4	r5	r6
7.4 Organisation and conduct of hearings	r1	r2	r3	r4	r5	r6

8.   Judges’ decisions

	Not satisfied	Not very satisfied	Average satisfaction	Satisfied	Very satisfied	No reply
8.1 Clear and comprehensible judgments	r1	r2	r3	r4	r5	r6
8.2 Rapid handling of cases	r1	r2	r3	r4	r5	r6
8.3 Decisions easy to enforce	r1	r2	r3	r4	r5	r6

[Optional questions:]

9. In general terms, what is your assessment of the operation of the court (service)?

Too opaque	opaque	Clear	Very clear
r1	r2	r3	r4

10. What is your assessment of the judges’ impartiality in conducting oral proceedings?

Not at all impartial	Not very impartial	Fairly impartial	Completely impartial
r1	r2	r3	r4

11. What is your assessment of the judges’ independence?

Not at all independent	Not very independent	Fairly independent	Completely independent
r1	r2	r3	r4

12. In your opinion, how has the operation of the court (service) changed over the last five years?

Has considerably deteriorated	Has deteriorated	Has not changed	Has improved	Has considerably improved
r1	r2	r3	r4	r5

13. What is your assessment of any changes in the court’s workload during this period (five years),?

r₁ The workload has increased faster than the resources available

r₂ The workload has increased in proportion to the resources available

r₂ The resources available have increased faster than the workload

14. In your opinion, are the material resources available to the court?

Very insufficient	Insufficient	Sufficient	Broadly sufficient
r1	r2	r3	r4

15. In your opinion, are the human resources available to the court?

Very insufficient	Insufficient	Sufficient	Broadly sufficient
r1	r2	r3	r4

Personal data

16. How many years have you been a member of the Bar in …………….?

r₁ Less than 5 years

r₂ 5-10 years

r₂ 11-20 years

r₂ More than 20 years

17. How do you exercise your profession as a lawyer?

r₁ Alone

r₂ As a member of a group (company)

18. Gender

r₁ Male

r₂ Female

19. Age

r₁ Under 30

r₂ 31-50

r₂ 51-65

r₂ 66 and over

20. Do you have any remarks or suggestions to make in connection with the operation of the court and, more generally, the justice system?