The assessment of court quality: a breach of the independence of the judiciary or a promising development?
Special advisor CEPEJ
Council of Europe
In the private sector there is a continuous awareness of the need for quality improvement and quality measurement of the delivery of goods and services. As a part of the market principles many companies are seeking for the best product and ultimate satisfied customers. The attention for quality was not at all times high on the agenda in companies. The main idea of quality-thinking derived back to the ’50ties when the management boards of Japanese companies recognised the importance of changing the negative image of the products into a positive one by introducing a quality policy. It was Feigenbaum who introduced the term ‘total quality control’ in 1951. Total quality control is defined an “effective system for integrating quality development, quality maintenance, and quality improvement efforts of the various groups in an organization to enable production and service at the most economical levels which allow full customer satisfaction”. At a later stage (period 1980 – 1990) the term total quality control was replaced by principles of total quality management (TQM). In a reaction to the quality revolution in Japan the United States introduced this concept for the first time at a US Naval Aviation Depot. Rather relying purely on product inspection, in the TQM approach it is necessary to focus on all relevant aspects of an organization and organizational processes. To encourage quality improvement further and to underline the importance of quality thinking for the US national economy the Congress adopted in 1987 a special Act for the creation of the Malcolm Baldridge National Quality Award. The purpose of this act is “to provide for the establishment and conduct of national quality improvement program under which (1) awards are given to selected companies and other organizations in the United States that practice effective quality management and as a result make significant improvements in the quality of their goods and services and (2) information is disseminated about successful strategies and programs” (Section 2 (b) purpose). From 1988 to current many organizations, varying from small businesses to public sector (healthcare, education, etc), received the award for their achievements in the area of quality. Four years later a comparable initiative was launched in Europe with the introduction of the European Foundation on Quality Management model (EFQM). The EFQM excellence model can be used by organizations to: assess, to benchmark with similar types of organizations and to improve their level of quality. The model is based on the principle that “excellent results with respect to Performance, Customers, People and Society are achieved through Leadership driving Policy and Strategy, that is delivered through People, Partnerships and Resources, and Processes”. As is the case with the Malcolm Badridge Quality Award, an organization is yearly awarded for the achievements made in the field of quality.
In addition to these two ‘quality award’ approaches several other quality concepts and techniques has been developed over the past years. Examples are the Six Sigma Model, focusing on the elimination of ‘defects’ in the product of services or goods, the Balanced Scorecard Principle and Quality Management Systems (ISO).
As far as the private sector and some parts of the public sector concerned the notion of quality belongs to the core values of how companies or organizations are run. The needs and wishes of clients are in the center of importance, when it comes to the determination of the quality. What about a specific part of the public sector: the judiciary? Is there a similar trend and history in quality thinking visible or are we just at the beginning? In this article the main trends and developments in the judiciary at a global level and regional levels are described. Before I start with illustrating examples of quality approaches and quality systems I will shortly introduce the concepts of ‘judicial quality’ and ‘court quality’.
2 Judicial quality and court quality
In many legal studies the quality of the judiciary is related to decisions made by judges and the possibility to appeal these decisions. Also the existence of a High Court, Supreme Court and/or Constitutional Court must be seen in the light of judicial quality. These high level courts form a guarantee for a high quality judicial decision making process, where there exist the possibility to correct legal errors made by judges and to protect the unity of law i.e. the fact that similar cases are decided in a similar manner. However, judicial quality is not only limited to the quality of the decisions or the existences of appeal and higher courts. It is also brought in relation to the notion of independence of the judiciary. Guarantees for independence are connected with the recruitment and nomination of judges, their terms of office, the remuneration and the freedom to decide in the courtroom without direct influence from the executive or legislative power.
For a long time quality was only seen in the light of judicial quality. It did not take into account other aspects that may influence the quality of the administration of justice. As was the case with the development of quality models and quality systems in the past for companies the perspective on quality was limited to the final quality of the product or services or in judicial terms: the quality of the decision rendered by the judge. A client perspective or other notions that can be related to the quality of courts were for many years not taken into account when it comes to the determination or improvement of the quality in the judicial field. Nowadays, there seems to be a trend that in certain parts of the world a wider notion on quality is appearing as a result of a growing number of countries which are developing integral quality systems for courts. This trend started in the United States with the creation of the Trial Court Performance Standards (TCPS).
3 US Trial Court Performance Standards
In 1983 the Conference of State Court Administrators of the United States and separately in 1987 the American Bar Association published case processing time standards for the courts. It was the first recognition of the need to be accountable and responsive to the public demands for a proper case management. 1987 was also the year that a commission of leading trial judges, court managers and (legal) scholars took the initiative to develop a measurement system for enhancing the ability of US State Courts to provide fair and efficient adjudication and disposition of cases. This initiative resulted in the Trial Court Performance Standards (TCPS). TCPS articulates the fundamental purposes of courts and provides a tool for the community of courts to discuss with each other and with the environment of the courts the quality of the services delivered by the court. In the system 68 measures for 22 standards are identified within a framework of five areas, connected with the fundamental purposes and responsibilities of courts. These areas are:
- Access to justice: courts should be open and accessible.
- Expedition and timeliness: decisions made by the courts should be delivered in a timely manner without any undue delay.
- Equality, fairness and integrity: courts should provide due process and equal protection of the law. Concerning integrity the decisions and actions of a court should adhere to the duties and obligations imposed on the court by relevant law as well as administrative rules, policies, and ethical and professional standards.
- Independence and accountability: trial courts must establish their legal and organizational boundaries, monitor and control their operations, and account publicly for their performance.
- Public trust and confidence: courts should work in an accessible, fair and accountable way where there is a high level of public trust.
To check the feasibility of TCPS it was tested in twelve courts. For the collection of data different methods were used (observations and simulations, structured interviews and focus groups, case and administrative record reviews, surveys and appellate court performance standards). Despite the fact that the system was endorsed by many trial courts in the United States, experiences showed that the self-assessment of the courts - by making use of all the 68 measures and 22 standards - was difficult to realize. Only a very limited number of courts, for example the Los Angeles Municipal Court (in 1996), were able to evaluate all the 68 measures. Schauffler (2007) explained in his article that several factors contributed to the inability of state courts to institutionalize the TCPS system. First of all the number of 68 measures was too great and too complex. Secondly, the courts’ information systems were not originally designed for the data that was necessary to provide the information for TCPS. Thirdly, due to an improvement of the economic climate in the United States the need to invest in performance measurement declined. Fourthly, as a result of the institutional separation of the judiciary from other branches of government the pressure to adopt performance measures in the courts were reduced and lastly, a lack of consistent leadership in the courts for drawing attention to the need for implementing performance measurement systems lead to a decreasing attention to TCPS.
A ‘revival’ of the quality approach seems to appear when in 2005 the Courtools where published. These tools, a list of 10 practical measures, are based on the integration of the major performance areas defined in TCPS with general quality models such as the Malcom Baldridge Quality Award and the Balanced Score card. The measures/indicators are: (1) access and fairness, (2) clearance rates, (3) time to disposition, (4) age of active pending caseload, (5) trial data certainty, (6) reliability and integrity of case files, (7) collection of monetary penalties, (8) effective use of jurors, (9) court employee satisfaction, and (10) costs per case. In 2008 the list 10 practical measures is connected with each other through the development of a ‘framework for court performance measurement’ by the US National Center for State Courts. In this framework the measures/indicators are arranged in four quadrants (comparable with the balanced scorecard): effectiveness (measures 5, 7 and 8), procedural satisfaction (measure 1), efficiency (measures 2, 4 and 6) and productivity (measures 3 and 10). For the quadrants ‘procedural satisfaction’ and ‘productivity’ the framework for court performance includes two new indicators: transaction time, respectively value added time (actual time spends on a case). The essence of the framework is that adequate operating courts are scoring ‘good’ in all four quadrants. It must be noted that, in contrast with the court tools, the framework for court performance is still in the development phase.
4 European initiatives
In the beginning of the nineties initiatives were undertaken in Europe to improve the public services of governmental institutions. This resulted in the United Kingdom in the creation of so-called ‘Citizens charters’. The former UK Prime Minister John Major promised in 1991 better quality for consumers through the publication of service standards and specific rights for consumers of public services. The most successful providers of public services were/are awarded with ‘charter marks’. Specific citizen’s charters were also available for the justice sector. For example in UK-Scotland a justice charter was released at the end of the same year. As a part of this charter standards of court services were introduced, access to justice was improved, special attention were given to specific categories of users (witnesses, jurors, children, victims), information published on the reduction of delays and monitoring mechanisms were implemented in measuring the progress in court proceedings. In 2003 a similar idea was introduced by the French Government under the name ‘Charte Marianne’. This charter is also meant to improve the access to public services, a client friendly approach to citizens, a possibility to file a complaint and a quick response to the request of the citizen for the delivery of public services.
The notion of the importance to take the needs and wishes into account of the users of the courts and not to limit the quality of administration of justice to judicial quality was recognised by other European countries as well. For example in Switzerland (Kanton Genève) a report (1996) was published on the quality of the courts, based on the results of a customer satisfaction survey under the group of lawyers. In the years 1997 and following the survey was extended to other court users (citizens) and notaries.
A more comprehensive approach towards court quality can be found in the Netherlands. As a part of a large reform program of the Dutch Judiciary (Administration of Justice in the 21st Century) a project Quality (1999) was initiated. The main aim of this project was to develop a measurement system for the courts which made it possible to assess court quality and to define areas of improvement. Based on the experiences with the US Trial Court Performance Standards such a system was developed in a three-year period. Comparable with the main elements of TCPS five areas of measurement were defined:
For each area of measurement a list of indicators was developed as well as specific instruments to assess the court quality (court statistics, staff and court user surveys, audits). In contrast with TCPS the team members of the project decided that the measurement system should be ‘light’ i.e. easy to implement and it should not lead to a high demand for personnel and technical resources to collect the relevant data. To facilitate the process of analysis of the data a tailor made software program ‘quality dashboard’ was developed, which included also the four areas of measurement of the balanced score card. The results of the court quality assessment were displayed through the ‘stop sign’ colours red (improvement necessary), yellow(neutral) and green (positive quality outcome). The measurement system court quality was tested in the period 2000 – 2001 in three pilot courts. After the creation of a Dutch Council for the Judiciary in 2002 this system became part of a larger comprehensive and integral court quality system ‘RechtspraaQ’. This system is used by every court in the Netherlands and includes elements of a peer-review of judges, a national complaint proceeding, a court visitation system and (normative) frameworks for the management of the courts as well.
In Finland the measurement of the quality of judicial work has already had the attention for many years (since 1995). What was lacking though was a system of quality benchmarks for adjudication. That is one of the reasons that in 2003 a set of Quality Benchmarks was introduced. The Benchmark makes it possible to analyse the success and failures of the court operations. Similar to the areas of the TCPS system and the Dutch measurement system court quality the benchmarks consist of six aspects that are influencing the quality of the courts positively or negatively, namely: (1) the process, (2) the decision, (3) treatment of the parties and the public, (4) promptness of the proceedings, (5) competence and professional skills and (6) the organization and management of adjudication. In 2005 the Quality Benchmark project of the courts in the jurisdiction of Rovaniemi were awarded with a European prize for the most innovative project in the area of justice (‘Crystal Scales of Justice’).
Not only at the level of individual countries the interest for integral quality systems is growing, but there seems to be a trend at a European level to stimulate countries to introduce quality measures in the courts and to promote a discussion in the judiciary on court quality. Both the European Union and the Council of Europe launched initiatives in this field. For example in 2004 the Committee on Civil Liberties, Justice and Home Affairs of the European Parliament published a working document with the title ‘the quality of criminal justice and the harmonisation of criminal legislation in the Member States’. In this document a quality charter is proposed for (criminal) justice. As a part of the charter a set of criteria for evaluating the functioning of judicial systems should be determined and a mechanism for ‘mutual evaluation’ should be introduced. Elements of this mechanism are: a comparative statistical basis, ‘benchmark’ information, the dissemination of best practices and an evaluation report on compliance with the quality charter. Despite the good initiative to stimulate quality at a cross-country level it has up till now not resulted in material proposals to draft a European charter on the quality of criminal justice. From that point of view more concrete are the documents and reports produced by the CEPEJ (the European Commission for the Efficiency of Justice) of the Council of Europe. Already for three periods (2002, 2004 and 2006 data) detailed information is available on the functioning of judicial systems in the 47 European member states. The reports on evaluation judicial systems give a very good overview how the various systems are working and what the main trends are. At a general level there seems to be an indication that European countries are reducing the number of court locations in favour of increasing efficiency of justice. Also many initiatives are undertaken to improve the judicial proceedings by modifying procedural laws and by introducing several solutions in the area of information and communication technology (e-justice).
More specific on the subject of quality for the judiciary and courts the CEPEJ created in 2007 a working group on quality. One of the aims of this working group is to collect information concerning initiatives taken by member states to promote and increase the quality in the courts. Another task is to develop concrete instruments for the member states in the area of court quality. The first result of the working group concerns the publication of a ‘checklist for quality of the judiciary and courts’. Not surprisingly similar elements as in other court quality systems can be found in this checklist, namely: (1) strategy and policy, (2) operational processes, (3) access to justice, (4) human resources and the status of the judges and the staff and (5) the (financial and material) means of justice. Four each of the five described elements several points of attention are listed. Policy makers, court managers and judges may use the checklist to stimulate a debate in their country or to implement (parts) of a quality system. It is expected that in the near future two other products will be available too: a guideline for preparing a court user satisfaction survey and a document regarding the ‘contractualisation’ of the relationship between the parties and the judge.
Besides the initiatives of the European Union and the Council of Europe the European Network on Councils for the Judiciary (ENCJ) initiated also another European initiative: a working group on Quality management. In 2007 – 2008 this working group, composed of 9 National Councils for the Judiciary or Court Administrations, worked on the drafting of a report with the title ‘Quality Management’. In this report an overview is given of the quality activities that are undertaken by the various countries and the role of the Council for the Judiciary/Court administration in promoting quality management.
5 Australasian initiatives
Two examples can be given concerning the development of quality models for the courts in the Australasian region: (1) the projects of the Subordinate Courts of Singapore and (2) the quality initiatives undertaken by the Australian courts. In 1990 the Singapore government, together with the judiciary, decided to reform the courts, mainly with the intention to reduce the backlog of cases and to set in place a Court charter aiming at timely proceedings and a proper treatment of the parties. Part of the reform was to promote visionary leadership and strategic planning in the courts, strategic use of information, maximising and developing human capital, efficient court proceedings through the use of ICT, mediation and court dispute resolution and a regular measurement of the expectations and level of satisfaction of the users of the courts. At the end of the 1990s the Subordinate courts initiated the Justice Scorecard as a tool for performance measurement. With this scorecard the courts are able to: make a clearer linkage between vision, mission and actions, to establish and maintain pro-active management, to create simple and concise measuring of critical indicators and to develop early warning signals when targets are not met.
In September 2000 the eJustice Scorecard System was launched. In this system all the relevant information on four perspectives is collected (Community, Internal Processes, Learning and Growth and Financial) and is used for management purposes. For the achievements and quality improvements that have been carried out by the Singapore Subordinate Courts they received in 2006 the ‘Singapore Quality Award’.
In Australia a similar approach is followed as New Zealand towards the improvement of the courts. Under the government wide-performance data collection and reporting process the courts are required to deliver reports on an outcomes/output framework. The outcomes/output framework for each of the federal courts is combined in the Attorney General’s Portfolio Budget Statement. During the budgetary year this statement can be compared with the performance results reported in the Annual Reports (send to the Parliament). One of the challenges the Australian courts are facing is that the output of a legal system is not easy to quantify and that it is difficult to balance efficiency against quality.
6 A global initiative: framework for court excellence
At the beginning of 2007 senior district judge Magnus of the Singapore Subordinate Courts took the initiative to initiate a project towards the development of a global Framework for Court Excellence. Inspired by earlier successes in the field of quality achievement the time seems to be right to create a tool which will be applicable for all the courts in the world. As a part of this inspiration American, Australian, European and Asian experts were invited to become member of a project team responsible for the drafting of the framework. To formalize the cooperation between the different partners a Consortium for Court Excellence has been created, represented by the US National Center for State Courts, the US Federal Judicial Center, the Australasian Institute of Judicial Administration and the Singapore Subordinate Courts. The Consortium is assisted by experts from the Council of Europe (CEPEJ) and the World Bank.
The central aim of the framework is to provide courts a resource for assessing their court performance against seven detailed areas of court excellence and it delivers a guidance for courts to improve their performance (in the widest sense of meaning). What the framework makes different from general quality models that are developed for the private sector is that it takes into account the relationship between the core values of a court, the areas of measurement, court performance and quality.
In the framework document it is underlined that for all courts in the world (at least) ten values are important when in comes to the fundamental purposes or ‘reasons of existence’ of a court (see table).
Table 1 Core values of a court (Source: framework of court excellence (2008), p. 8)
equality (before the law)
independence of decision making
These values are incorporated in the framework through the individual areas of measurement that are specified in the framework for court excellence report. In this report an example is given for the measurement area ‘court management and leadership’ where the authors explained that one of the important tasks for the management of a court is to promote shared values, such as independence, integrity and timelines. In addition to this example it is furthermore stated that “a journey towards court excellence is also a journey towards the strengthening of common court values”. Besides describing the relationship between court values and the areas of measurement the framework is based on two concepts that are a key for court excellence: management and leadership. Professional management and strong leadership are of major importance to realize court excellence. The authors of the framework report indicated that for a proper decision making process at the level of the management of courts it is necessary to rely on a sound measurement of key performance areas and reliable data. Measurements, combined with court performance data, indicate what the current state of affairs of a court is and how much progress is made towards targeted goals and to indicate what the areas improvements are.
As has been noted earlier, seven areas of measurement are identified (see table). Concerning court management and leadership the framework showed that proactive and inspiring leadership are crucial for court success and excellence. It showed also the need for clear court policies. For example it is important that a system of court management policies and plans are implemented to rationalize the process of planning and control and that judicial policies may positively contribute to the core values of a court. For example when a court policy is introduced on the ethical behaviour of judges or the treatment of parties during court hearings this may positively influence the court values integrity, accessibility, impartiality and independence. It is clear that fair and efficient court proceedings are contributing factors for court excellence. Inefficient proceedings may lead to a long duration and an increase of the backlog of cases. A well management of court resources (human, personnel and financial resources) are contributing positively to court excellence too, according to the authors of the framework. Concerning the ‘demand’ side the framework identifies two areas: public trust and confidence and user satisfaction. As is the case with other court quality models access to justice, in terms of affordable and accessible court services, is mentioned as an important area of measurement.
Table 2 Areas of measurement in the framework of court excellence
Court management and leadership
Affordable and accessible court services
Public trust and confidence
With respect to the assessment of court quality a list of indicators has been defined for each of the seven areas of measurement. This list can be used by the courts to evaluate the current state of affairs. To facilitate the process of assessment a plan of action (steps to be taken to implement the framework, to conduct the assessment and to use the results for improvement of the court performance and quality) and a self-assessment questionnaire is included in the framework report. The main aim of the questionnaire is to search the strength and weaknesses in the court organization. If needed or requested, the framework contains also a scoring mechanism. Courts can use the scoring method to see how far they are on their journey towards court excellence. Results of different assessments (including the scoring results) may be used in the future to set op a national or international court quality benchmarks and to identify a court that has achieved the highest level of court excellence and made the largest improvements concerning the quality of services delivered in the world.
In addition to the remarks on the scoring mechanism, it must be noted that the authors underlined that there are multiple trajectories towards court excellence. The framework is developed in such a way that it is flexible. Courts can follow different tracks of improvements, sequence reforms differently, select other priorities and choose among the various available tools to achieve their goals. The hypothetical last phase of the trajectory towards excellence shows what an ideal court look like if it fulfills all the quality requirements that are necessary for an optimal court performance and a high level of satisfaction of the users of the court.
7 Concluding remarks
Almost sixty years ago a quality approach was introduced in Japanese companies. At a later stage enterprises in other parts of the world followed this initiative. General quality models were developed and quality awards were given to excellent companies. Nowadays the orientation towards user satisfaction, quality measurement and quality assurances is a common part in the management policy and strategy of companies.
A growing interest for the user became apparent in the public sector when a certain number of countries introduced quality or customer charters at the beginning of the nineties. It was also the same period that in the United States quality was introduced in the courts, as the result of the creation of the trial court performance standards. After national or regional initiatives in other parts in the world (Europe and Australasian region) concerning the development and implementation of quality activities or comprehensive quality systems for the courts, we are now at the beginning of a new phase: the journey towards court excellence. With a framework for court excellence in our hands, it will be possible for courts in all parts of the world to start with quality initiatives, to assess the level of quality in the courts and to improve court performance. It is expected that this framework will contribute to several quality improvements in the judiciary, with concrete benefits for the users of the courts. In the long run it may even lead to, similar to the Malcolm Baldridge Quality award, the creation of a new global quality award: “the Global Court of Excellence Award”. As a tool for promotion and stimulating quality improvements in the courts this award may be given to courts where major achievements have been realized in the area of quality and where courts have followed the trajectory towards court excellence.
 Feigenbaum, A. V. (1951), Quality Control: Principles, Practice, and Administration, McGraw-Hill
 Malcolm Baldrigde National Quality Improvement Act of 1987.
 In 1995 a similar system was developed for the appellate courts: the Appellate Court Performance Standards.
 Ingo Keilitz, Standards and Measures of Court Performance. In: Criminal Justice (2000 vol. No. 4), p. 581.
 Richard Y. Schauffler, Judicial accountability in the US State Courts measuring court performance. In: Utrecht Law review, Volume 3, Issue 1 (June) 2007, p. 120.
 US National Center for State Courts (2008), A unifying framework for court performance measurement, Washington.
 The Scottish Office and the Crown Office, Scottish Courts Administration (1991), Justice Charter.
 French Republic (2003), Charte Marianne: pour un meilleur accueil, Paris.
 Ministry of Justice Finland (2005), Quality projects in the courts of the jurisdiction of the court of Appeal of Rovaniemi. Court of Appeal of Rovaniemi (2006), How to assess quality in the courts.
 European Parliament (2004), Working document on the quality of criminal justice and the harmonisation of criminal legislation in the Member States, Brussels.
 It must be noted that e-justice is high on the agenda of the European Union. Initiated by the Council of the European Union and supported by the European Commission a European e-justice web portal will be build to facilitate access to justice in cross-border disputes for citizens and legal professionals.
 CEPEJ-GT-QUAL (2007)9Rev, Quality of the judiciary and courts (a checklist), Strasbourg.
 The quality activities that are identified in the report are: (1) mission, vision and strategy, (2) total quality systems, (3) leadership and management, (4) complaints procedure, (5) peer review, (6) processins time and working procedures, (7) training, (8) quality assessment and judicial quality, (9) staff evaluation, (10) client evaluation, (11) management information, auditing and reporting and (12) external communication.
 European Network of Councils for the Judiciary (2008), Quality Management.
 Singapore Subordinate Courts (2007), Case Study: the Subordinate Courts of Singapore a journey of excellence, Singapore.
 The methodology of the scorecard is derived from the Balanced Scorecard method.
 The Singapore Quality Award institutionalises the highest standards of business excellence and is based on the standards laid down in the US Malcolm Baldridge National Quality Award, the European Foundation on Quality Management (EFQM) and the Australian Business Excellence Award.
 Framework for Court Excellence (2008), p. 47.
 The author was member of the project team and contributed to the development of the Framework.
 The framework for court excellence was launched at an international “Quality forum” conference in Sydney - Australia (21 –23 September) organized by the Australasian Institute of Judicial Administration. Further information can be found at the website of this institute (http://www.aija.org.au/) or the website of the US National Center for State Courts (http://www.ncsconline.org/).
 Framework of court excellence (2008), p. 10.
 Framework for court excellence (2008), p. 36.