P-PG/Prev (2010) 6
Evaluation of
the drug prevention activities:
theory and practice
Alfred Uhl, Richard Ives
and
Members of the Pompidou Group Prevention Platform
Copyright for the original versions is held by the Council of Europe
© Council of Europe, September 2010
Printed at the Council of Europe
No abbreviations of, additions to or alterations to the original text shall be made without the written permission of the Council of Europe Pompidou Group
Professional and technical coordination of edition:
Tracey Agard, Uche Anigbogu, Sergei Bazarya
Free of charge
Alfred Uhlis a psychologist working in Vienna, Austria. He has been active in addiction research since 1977 in the fields of epidemiology, policy research, prevention, evaluation and research methodology. Presently, he coordinates the section of addiction prevention research and documentation and teaches research methodology at the Institute for Statistics and Mathematics in the University of Economics and Business in Vienna.
Richard Ives founded the company educari, which assists professionals who work with children and young people. He focuses particularly on drugs, alcohol and gambling issues. He has developed a number of educational resources and undertaken project evaluations in the UK and in other countries (working for the UN and other international bodies). He has worked with the Pompidou Group on many activities, including the development and implementation of a Lifeskills Programme, and he is Expert Advisor to PG’s Drug Prevention Prize Young People’s Jury.
The Co-operation Group to Combat Drug Abuse and Illicit Trafficking in Drugs (the Pompidou Group) is an inter-governmental body formed in 1971. Since 1980 it has carried out its activities within the framework of the Council of Europe, and 35 countries are now members of this European forum, which allows policy makers, professionals and experts to exchange information and ideas on a whole range of drug misuse and trafficking problems. Its mission is to contribute to the development of multidisciplinary, innovative, effective and evidence-based drug policies in its member states. It seeks to link policy, practice and science.
By setting up its group of experts in epidemiology of drug problems in 1982, the Pompidou Group was a precursor of the development of drug research and monitoring of drug problems in Europe. The multi-city study, which aimed to assess, interpret and compare drug use trends in Europe, is one of its major achievements. Other significant contributions include the piloting of a range of indicators (Treatment demand indicator) and methodological approaches, such as a methodology for school surveys which gave rise to the ESPAD (European School Survey Project on Alcohol and other Drugs).
BACKGROUND
This publication will explore the limitations for evaluation of drug prevention interventions and address the ways that evaluation can be made more effective. The Prevention Platform of the Pompidou Group (PG) of the Council of Europe has developed a framework of the publication in the context of the work undertaken in recent years in Europe to improve prevention interventions. The results of this work include “Handbook on prevention” by the PG and Jellinek Consultancy, “Prevention and evaluation resources kit” by the European Monitoring Centre for Drugs and Drug Addiction (EMCDDA) as well as its database of good practices developed within the framework of the Exchange on Drug Demand Reduction Action (EDDRA), COST-A6 programme publications and the 2010 Prague Conference of the Pompidou Group “Prevention evaluation: from dogma to useful tool”.
The purpose of this publication is to assist policymakers and their advisors in the decision-making process about the allocation of scarce resources for drug prevention. Evaluation is a key method for assessing the effectiveness of different approaches but it has its limitations. Policymakers often look for quick answers on “what works in prevention”. There is also a certain tendency among practitioners and policymakers to think that “everything must be evaluated, if something is evaluated- it is proved to be good”. Alfred Uhl (2000) created the word “evalopathy” to describe those tendencies. The topic of evaluation is a complex one; there are many different perspectives on evaluation. Defining what is meant by ‘evaluation’ in different contexts is crucial. It is important to point out where evaluation has become ‘a dogma’, and help policymakers and practitioners to ensure that evaluation is ‘a useful tool’.
The Pompidou Group’s Prague conference in May 2010 was dedicated to the topic of drug prevention evaluation. Some of the findings of this conference are:
- different approaches to prevention require different approaches to the evaluation of prevention. Drug prevention takes place in an environment of multiple and interlinked factors as well as in many different settings and it addresses a range of needs. The complexity and diversity of drug prevention defies simplistic assessment summary;
- drug prevention requires a comprehensive and long-term view. Evaluation of drug prevention therefore also needs to take this perspective;
- communicating this complex and multifaceted picture to politicians, policymakers and citizens is a necessary and urgent task;
- more synergy should be looked for in implementing and evaluating prevention – for example, with other social problems and risky behaviour;
- the international transferability of prevention activities is feasible and can be useful. However, there are issues in adoption and adaptation. Evaluation can assist in identifying the essential elements of prevention work that should be retained in any context, and those elements that can be adjusted to suit particular contexts;
- routine project implementation does not necessarily need evaluation; a quality standards approach would often be more appropriate, especially for assessing the value of the implementation of recognised and well-tested approaches to drug prevention;
- setting minimum standards would help to drive up quality. Internal monitoring and process evaluation could help to ensure project implementation fidelity, or identify departures from standard practice and assess their merits or demerits.
I believe this publication will help to achieve the following modest but feasible and practical aims of drug prevention evaluation:
- identifying best practice in drug prevention,
- defining and promulgating quality standards for project and programmes and their implementation, and
- eliminating the programmes and practices that clearly do not work or which make things worse.
I would like to thank the authors of this book: Dr Alfred Uhl (Austria) who provided a theoretical overview of prevention and evaluation fields and Mr Richard Ives (United Kingdom) who has contributed with the illustrative practical examples. I would also like to thank the members of the Pompidou Group Prevention Platform: Mr Raphael Bayer (Austria), Ms Aljona Kurbatova (Estonia), Mr Christoph Lagemann (Austria), Mr Fernando Mendes (Portugal), Ms Christiane Morel-Barnichon (France), for the work they have put into this publication.
Sergei Bazarya
Administrative Officer
Pompidou Group Secretariat
Content
THE AUTHORS 5
THE POMPIDOU GROUP 6
BACKGOUND 7
CONTENT 9
EVALUATION OF THE DRUG PREVENTION ACTIVITIES: THEORY
by Alfred Uhl 12
EVALUATION OF THE DRUG PREVENTION ACTIVITIES: PRACTICE
by Richard Ives 42
1.1........ What goal should SAP aim at?
1.2........ Which measures are legitimate in a modern democratic society?
1.3........ Given limited resources, which measures should be implemented to attain optimal results?
2.1........ Addictive Potential
2.2........ What is Prevention?
2.2.1......... Drug prevention, Substance Abuse Prevention (SAP) vs. Prevention of Addiction
2.2.2......... “Prevention” – a deterministic or a probabilistic concept
2.2.3......... “Prevention” – a relational term
2.2.4......... Three distinct meanings of “prevention”
2.2.5......... “Prevention” in a closer and an extended sense
2.2.6......... The classic “Public Health Classification System” (PH-CS)
2.2.7......... The modern “Mental Health Classification System” (MH-CS)
2.2.8......... “Prevention” within the USA Institute of Medicine (IOM)-Classification Model
3.1........ Attributes to classify protective and risk factors
3.1.1......... Prognostic vs. causal
3.1.2......... Internal vs. external factors
3.1.3......... Passive vs. active factors
3.2........ Different ways to define risk factors and protective factors
3.2.1......... Symmetric risk factors and protective factors (type 1)
3.2.2......... Symmetric causal risk factors and protective factors (type 2)
3.2.3......... Asymmetric causal risk factors (type 3)
3.2.4......... Asymmetric causal risk factors (type 4)
3.3........ Related factors to risk factors and protective factors
3.3.1......... “Deficits” vs. “resources”
3.3.2......... “Weaknesses” vs. “skills” and “vulnerabilities” vs. “resiliencies”
3.4........ The importance of the context
4.1........ The everyday conception vs. professional conception of evaluation
4.2........ State of product (evaluation object) dimension
4.4........ Methodological dimension
4.5........ Data, state-of-product, methodological, evaluator (DSME) classification
4.6........ “Evaluation” according to the guidelines of the German Society for Evaluation
4.7........ Pseudo evaluations
5 Empirical question related to measuring effectiveness
5.1........ Intuition vs. science based enquiry
5.2........ Evidence Based Policy
5.3.1......... Participatory-emancipation perspective vs. paternalistic-controlling perspective
5.3.2......... The impact of external influences
5.4.1......... Non-linear Systems – Generativity
5.5........ Problems in Measuring Treatment Effects
5.5.1......... Programmes vs. Soft Skill
5.5.2......... Proxy Variables – Surrogate Endpoints
5.5.3......... Moderate substance use as inadequate indicator for later problematic substance use
5.5.4......... Interpreting Statements Wrongly As Attitudes – The Parrot Effect
5.5.5......... Vagueness of verbal expression
5.5.6......... Unavoidability of implicit communication
5.5.7......... What we are interested in are long-term effects
5.5.8......... Unreliable self-reports
5.6........ Methodological problems
5.6.1......... Programme Evaluation vs. Prevention Research
5.6.2......... Treatment – Health Promotion – Prevention Dilemma
5.6.3......... Analysis – fishing for significances
5.6.4......... Effect sizes, problem manifestation rate and number needed to treat (NNT)
5.6.5......... Power Problems – Sample-size calculation
5.6.6......... Statistical regression artefacts – pseudo changes
7.1........ Historical development of the classic “Public Health Classification System”
7.1.2......... Classification 2: Caplan (1964)
7.1.3......... Classification 3: (Strasser, 1978)
7.1.4......... Classification 4: (Gordon, 1983)
7.1.5......... Classification 5: Kumpfer & Baxley (1997) or Dadds (2004)
7.1.6......... Classification 6: (Uhl, 1998)
7.1.7......... Classification 7: (Department of Mental Health and Substance Abuse, 2004)
7.2........ Historical development of the modern “Mental Health Classification System”
7.2.1......... Classification: Gordon (1983)
7.2.2......... Classification: Kumpfer & Baxley (1997)
7.2.3......... Classification 10: (EMCDDA, 2007)
7.3........ Table providing an overview over the different classification systems
10 Review of interventions in school settings
11 Review of interventions in non-school settings
12 Review of drug education for the UK’s NICE
13.4...... Communities that Care
14.2...... Lifeskills-based drug education in Sverdlovsk Region, Russia
14.3...... Pompidou Group: capacity-building and dissemination
14.4...... Pompidou Group: testing contrasting approaches in three areas of Russia
16 Evaluation of a theatre-in-education intervention
17 Non-drug-related interventions
18.1...... Limitations and difficulties of evaluation
18.2...... Different views on what drug education and prevention aims to achieve
18.3...... Pointers towards effectiveness
19 Evaluation: future developments
LITERATURE 56
Evaluation of the drug prevention activities: theory
by Alfred Uhl
Scientists might have developed something like the physicians’ Hippocratic Oath, the vow to use their knowledge only for the good of mankind. As things stand now, the best we can hope for is a generation of inventive dwarfs who can be hired for any purpose.
“The Life of Galileo” Berthold Brecht (1948)
We are like dwarfs on the shoulders of giants, so that we can see more than they, and things at a greater distance, not by virtue of any sharpness of sight on our part, or any physical distinction, but because we are carried high and raised up by their giant size.
“Metalogicon” John of Salisbury (McGarry, 1962)
1 Introduction
“It is better to prevent than to heal” is a dogma of common-sense. Due to this truism hardly anybody opposes the idea that societies should invest resources into substance abuse prevention (SAP) targeting legal psychoactive substances like alcohol and tobacco, pharmaceuticals like benzodiazepins and illicit drugs. This consensus easily disintegrates into heated controversial arguments as if specific issues are dealt with. Such critical issues are:
- What specific goal should SAP aim at?
- Which measures are legitimate in a modern democratic society?
- Which measures are effective?
- Given limited resources, which measures should be implemented to attain optimal results?
1.1 What goal should SAP aim at?
A highly controversial issue related to SAP is whether we should focus on reducing the number of psychoactive substance users (abstinence perspective[1], population approach[2]) or on reducing the amount of problems arising due to psychoactive substance use (problem reduction perspective[3]). Some persons are determined in favour of an uncompromising fight to reduce any psychoactive substance use in society, regardless if this fight enhances related problems, and others prefer to reduce substance related problems regardless if some of these approaches mean explicitly tolerating certain forms of unwanted psychoactive substance use. Related to this issue is the question whether all psychoactive substance use is perceived as fundamentally wrong and morally unacceptable (puritan perspective) or if some forms of substance use may be seen as contributing positively to quality of life (hedonistic perspective).
Obviously the aforementioned positions are extremes of a continuum and the attitudes of most individuals lie somewhere in between these extremes. Commonly these attitudes are not universal concerning all psychoactive substances but vary concerning different substances (e.g. a rather tolerant attitude concerning alcohol, nicotine, or cannabis use and a strict prohibitive position concerning heroin or cocaine) and concerning different groups of consumers (e.g. more tolerance towards male consumption and/or adult consumption than towards female consumption and/or consumption of minors or young adults).
Recent scientific discourses attempt to reduce the question of SAP to a purely scientific issue. A trendy buzzword, camouflaging the ethical nature of goals in SAP as factual is “evidence based” in combination with prevention, treatment, policy etc. (Uhl, 2007). The term “evidence based” and how it is abused to mislead thinking will be dealt with later in section 5.2.
The heterogeneity of ethical positions concerning substance use in society constitutes a major threat to persons professionally involved in SAP. Many persons expect that preventionist – particularly those dealing with their kids – share exactly the same values with them concerning substance use and they often get very upset if these expectations are not met. As a result preventionists who do not want to risk their jobs have learned to be very careful about precise wording towards third parties. They developed a skilful form of ambiguity in speech, allowing persons with very different values to believe that they share a common value basis with the prevention expert. This pragmatic way of hiding potential conflicts is understandable from a pragmatic perspective but at the same time counterproductive for rational discourses fostering adequate research about SAP strategies. Developing sensible strategies and evaluating them adequately requires a clear vision and unequivocal terminology.
1.2 Which measures are legitimate in a modern democratic society?
To argue for certain goals, and to try to convince others that those goals do not directly and massively interfere with their interests is one thing. They can sit back and say: “This is your opinion and I have a different one – we can agree to not agree.” Deciding to implement measures which enforce adherence to these goals is a different thing. Such measures directly interfere with the interests of any stakeholders not willing to comply with them. Thus the issue of legitimate measures is even much more controversial than merely defining the goals of SAP. Suddenly we are confronted with the issues of influence and/or coercion. And what is justified in a modern, democratic society adhering to the standards of universal human rights. The question which measures may justly be implemented is an inherently ethical question – and attempts to mask the ethical dimension by presenting certain options as solidly grounded in science by labelling them “evidence based“ is highly misleading and thus rejectable.
This does not mean though that research should not play an important role in the decision process. If research endeavours suggest that certain goals cannot possibly be reached by any means and/or that proposed measures are very likely ineffective, neither maintaining these goals nor implementing these measures makes any sense. Thus, knowing that goals can be achieved and that suggested measures are effective is a necessary but not a sufficient condition to consider them seriously.
The primary question concerning policy measures is: if they are ethically justified. Only if measures are ethically sound, the question “whether these measures can be expected to reach the intended goals?” is relevant along with the practical question of how to assess effectiveness and/or cost-effectiveness in relation to alternative approaches.
If somebody asks an expert: “What strategies are effective in SAP?”, a simple and popular answer is: “Use evidence-based measures!” (see section 5.2). This statement implies that the necessary scientific knowledge to decide which strategies to choose already exists and it somehow suggests that the enquirer is expected to know much about it. Such an answer leaves a shallow taste but only very self-secure enquirers continue to insist “Which specific strategies do you consider to be effective in SAP?”
To demand that the effectiveness of preventive measures must be proven scientifically before any final implementation sounds convincing at first sight, but according to contemporary epistemology proving propositions beyond doubt is impossible. As Kritz et al. (1990) put it: “A specific attribute of science is, that science – in contrast to the everyday conviction that “Something is as it is” – never treats any interpretations as final truth. We are confronted with an abundance of elementary methodological problems and limitations that we cannot simply neglect – even though ignoring them makes a scientist’s life much easier. As Green’s Law of Debate (Bloch, 1985) wittingly formulates: “Anything is possible if you don’t know what you’re talking about!”
Closing the eyes in front of the undeniable is not an option worth to pursue. To demand the impossible produces pseudo solutions and systematically blocks constructive discourses. There are ways to improve our understanding of the world in spite of these problems – and we cannot refrain from acting, just because we do not have final and certain answers. As Hayek’s (1988) provocatively formulated “If we refrained from all actions, just because we don’t know a reason for them or since we cannot justify them, we would probably soon be dead!” Teachers have to teach their students, parents have to educate their children, dietarians have to suggest healthy diets, businessmen have to decide how to invest for optimal returns, and substance abuse preventionists have to choose SAP strategies – and all of them have to decide in the light of uncertainties and ambiguities. There are sensible ways to proceed, but they are not straight forward, there are no precise criteria what steps to take and they force us to live with a lot of uncertainties and ambiguities.
1.3 Given limited resources, which measures should be implemented to attain optimal results?
As long as our theories and practical knowledge concerning SAP is far from being perfect, it makes little sense to develop quantitative models how to achieve maximum returns with given resources. Some models of this kind exist, but due to lack of detailed information, which has to be substituted with more or less arbitrary estimates and inadequate conceptualisation, these models should be approached very sceptically and carefully.
Particularly influential work of this kind concerning alcohol policy was published by Babor at al. (2003) suggesting that traditional prevention and therapy were relatively expensive and ineffective compared to increasing prices and restricting availability of alcoholic beverages. The inherent ethical issues are camouflaged by presenting these conclusions as proven beyond doubt although they are based primarily on correlation data.
2 Definitions
In our everyday use of language the emotional content associated with our words (connotations) dominates over the literal meaning (denotation) even though researchers commonly primarily consider the latter meaning. Particularly when designing questionnaires and interpreting results based on them, there is commonly little consideration that assuming a common understanding of terms is mostly unrealistic[4]. Because of arbitrary elements provided by the context of our memory, we cannot expect that certain terms have a consistent meaning within a certain individual. Another problem related to language use is the fact that interviewees usually try to make sense out of unclear and meaningless questions through reinterpreting them creatively without being aware of this process (Kritz et al., 1990, p 87). In combination with the basic aspiration of the human mind to gain certainty our perception and mind almost automatically and without awareness turns uncertainty into certainty (Gigerenzer, 2002). The latter processes explains why we hardly ever become aware that others use terms with very different meanings and of our own inconsistent use of terminology.
I experienced a very illustrative example for this phenomenon, as a member of an international experts group to design “standards for evaluation”. Practically all group members understood the term “evaluation” differently, but at the same time behaved as if there was just one correct, precise and commonly understood definition. They initially opposed the idea to find a common definition for “evaluation”, being afraid of endless, fruitless discussions on this issue. I had a hard time to convince the group that developing “standards for evaluation” without being able to define the central term “evaluation” was hard to defend to third parties. Finally we managed to find a sensible definition in a reasonable span of time that everybody could go along with.
Even though many persons, having an intuitive, natural relationship to language and being unaware of the immense problems caused by vague and unequivocal terminology are reluctant to invest energy into defining terms or trying to comprehend different definitions, I am convinced that only a precise and common language can serve as basis for a sensible and profitable discourse on issues; or as Socrates said: “The beginning of wisdom is defining the terms.”
2.1 Addictive Potential
The risk assigned to a specific substance (drug) – or in case of a behavioural addiction assigned to a particular behaviour – is commonly labelled “addictive potential of the substance”. The likelihood that a specific individual (set) will be attracted by addictive substances and eventually become addicted to them can be circumscribed with “affinity of the individual to addictive substances”[5]. There is no established term to paraphrase the impact of the environment (setting) on the evolution of addictions – but one could name this influence “addictive potential of the environment”.
It makes little sense to deal with the (1) “addictive potential of a substance”, the (2) “affinity of individuals to addictive substances”, and the (3) “addictive potential of the environment” as independent factors. As Zinberg (1984) formulated: “All three variables – drug, set, and setting – must be included in any valid theory of drug use. It is necessary to understand in every case how the specific characteristics of the drug and the personality of the user interact and are modified by the social setting and its controls.”
2.2 What is Prevention?
2.2.1 Drug prevention, Substance Abuse Prevention (SAP) vs. Prevention of Addiction
When prevention of illicit drug abuse became an issue in the 1960’s the task was initially labelled “drug prevention”. The term “drugs” in this context was almost unanimously interpreted as abbreviation of “illicit drugs”. More and more individuals got involved in “drug prevention” activities. The field gradually turned into a profession. In the course of this professionalization the mandate shifted gradually from the strong focus on “illicit drugs” towards preventing problems related to “any psychoactive substances” and in the last several years to “behavioural addictions” as well.
As a consequence the term “drug prevention” became more and more misleading. Attempts to redefine the term “drugs” in a way to include “illicit drugs”, “legal drugs” plus “prescription drugs” – as proposed by a WHO expert committee already in 1969 – failed totally, since neither the media nor the public went along with this idea. To create an adequate heading for the new profession, the term “prevention of substance abuse and addiction” could be considered to correctly comprise the full scope of activities, namely (1) “problematic substance use in non-addicts”, (2) “substance addiction” and (3) “behavioural addictions”. For practical purposes a term made up of 6 words is not catchy enough and a shorter technical label was needed. Professionals in the English speaking world decided on “substance abuse prevention” (SAP) at the cost of neglecting “behavioural addictions” and professionals in the French and German speaking world went for “addiction prevention” (“Prévention des Toxicomanies” and “Suchtprävention”) at the cost of ignoring “problematic substance use in non-addicts”. Parallel to this the outdated term “drug prevention” remained popular in the public and the media as well[6].
Presently we have to accept that the technical terms “substance abuse prevention” in the English speaking areas and “addiction prevention” in the German and French speaking areas, are to be treated as synonyms, even though the literal meanings are quite different and not self-explanatory at all.
2.2.2 “Prevention” – a deterministic or a probabilistic concept
To prevent something, in the literal sense, means to avert specific negative outcomes that otherwise would have happened. In a professional context prevention is interpreted in a probabilistic sense, meaning that the likelihood for problem manifestation is reduced relative to a counterfactual situation where no preventive intervention took place[7].
2.2.3 “Prevention” – a relational term
The specific activity “prevention” is relational in nature. Intervention could result in a casual chain of undesired conditions, like behavioural problems in childhood that increases the likelihood for substance abuse problems later on, increasing the likelihood for turning to injecting drug use increasing the likelihood to contract an HIV infection, etc., any successful intervention at a certain stage constitutes “treatment” at this problem level and at the same time “prevention” against possible consecutive problems. More specifically: to treat the index problem “substance abuse” at the same time prevents the index problems “substance addiction” and possible consecutive problems. Thus to ask if a certain specific intervention constitutes “treatment” or “prevention” makes little sense – usually any successful intervention is both.
This distinction “treatment vs. prevention” is highly important though, when considering professional group perspectives. The treatment profession tends to name whatever they do “treatment” and the prevention profession tends to label whatever they do “prevention”. The hidden agenda behind the discussion is not to solve a linguistic problem but to claim that certain activities belong into the domain of the own professional group.
2.2.4 Three distinct meanings of “prevention”
In the last chapter “prevention” was defined as a relational term describing specific activities to avoid or reduce a certain undesired outcome (index problem)[8]. This is not the only possible understanding though. “Prevention” can also serve as umbrella term including all activities belonging to the scope of a certain professional group[9] or as label for a profession – the profession of preventionist[10].
2.2.5 “Prevention” in a closer and an extended sense
As Büechi (2003) stated, more and more countries inside and outside of Europe rely on the so called “four pillar concept” in their drug policies. The four pillars represent four professions characterised by the four approaches “prevention”, “therapy”[11], “harm reduction[12]” and “repression/control”.
Reading through the literature, we find that the Public Health Classification System (PH-CS, see section 2.2.6) divides the pillar “prevention” into “primary prevention” and “secondary prevention” and defines “tertiary prevention” as intervention after index problem manifestation, covering the two pillars “therapy” and “harm reduction”. The pillar “repression” in the legal literature is commonly divided into “general prevention[13]” and “special prevention[14]” (Smits, 2006, see Fig. 1).
Fig. 1: Classification of “Prevention in an Extended Sense“
As previously suggested (see section 2.2.3) any intervention to treat a problem is at the same time prevention against consecutive knock-on problems. Thus there is no basic semantic objection against calling all four pillars prevention. The fact that all four pillars constitute “prevention” and that at the same a specific pillar is labelled prevention produces some confusion though.
A possible solution is to refer to the pillar “prevention” as “prevention in the narrower sense” as comprising all non-repressive interventions, where the index problem is not yet manifest, and where therefore neither therapy nor harm-reduction is indicated. (If the term “prevention” is used without an additional attribute it is usually spontaneously interpreted as “prevention in narrower sense”.) and to coin the term “prevention in an extended sense” as an umbrella term comprising all four pillars, i.e. “prevention”, “therapy”, “harm reduction” and “repression/control”. For the purpose of this publication we will use mostly the practical examples of prevention interventions which fall under the term “prevention” in its “narrower sense”.
2.2.6 The classic “Public Health Classification System” (PH-CS)
For several years the Public Health Classification System (e.g. Kumpfer & Baxley, 1997), dividing into the three categories “primary” vs. “secondary” vs. “tertiary prevention” dominated the SAP field. These categories are:
- “primary prevention” = interventions before signs of the index problem exist,
- “secondary prevention” = interventions aiming at persons with elevated risks to develop an index problem,
- “tertiary prevention” = treatment and relapse prevention after index problem manifestation.
Kumpfer & Baxley referred to several previous publications using these terms, without mentioning that the authors had defined the terms very differently. In previous definitions the term “primary prevention” was reserved for activities before problem manifestation and the terms “secondary” respectively “tertiary prevention” for activities after problem manifestation. For an overview covering different definitions of the PH-CS see Appendix Section 6.1
2.2.7 The modern “Mental Health Classification System” (MH-CS)
Gordon (1983) defined the Mental Health Classification System in three categories:
- “universal prevention” = aims at the whole population,
- “selective prevention” = aims at groups with an elevated risk to develop the index problem,
- “indicated prevention” = aims at individuals at risk before index problem manifestation.
The term “universal prevention” is synonymous to “primary prevention” as proposed by Kumpfer & Baxley (1997), “selective prevention” is largely similar to “secondary prevention”, only “indicated prevention”, targeting individuals before problem manifestation, while “tertiary prevention” is targeting individuals after problem manifestation” makes the last one of the category in both concepts different.
For an overview covering different definitions of the MH-CS see Appendix Section 6.2.
2.2.8 “Prevention” within the USA Institute of Medicine (IOM)-Classification Model
Mrazek & Haggerty (1994) coined the label “Mental Health Classification System” for what Gordon (1983) had proposed, and added two terms for treatment (“case identification” and “standard treatment for known diseases”) as well as two terms for maintenance (“compliance with long term treatment with the goal: reduction in relapse and reoccurrence” and “after-care including rehabilitation” (Fig. 2).
Fig. 2: The IOM-Classification Model
2.2.9 Public Health Classification System vs. Mental Health Classification System and IOM-Classification System
A great disadvantage of the Public Health Classification System (primary, secondary, and tertiary prevention) is that these terms are not a bit self-explanatory and that the concept is extremely ambiguous due to many very different interpretations in literature. The Mental Health Classification System is undoubtedly superior since the terms “universal”, “selective”, and “indicated” support their interpretation through their literal meaning. Since both classification systems play an important role in literature and are still widely used, we cannot simply abandon Public Health Classification System though we should be able to deal with the variety of definitions we find in literature.
3 Protective vs. Risk Factors
Prevention deals with intervening in causal chains of undesired conditions. If we decide to define a certain problem as index problem, any causally linked previous factors may be referred to as “risk factors”. If we focus on substance addiction, substance abuse is a risk factor and substance addiction is the index problem.
Protective factors and risk factors play an important role in the discourse on prevention, but the terms “risk factor” and “protective factor” have several different meanings, which commonly causes confusion. Important attributes to classify these factors are the dichotomies “prognostic vs. causal factors”, “internal vs. external factors” and “passive vs. active factors”.
3.1 Attributes to classify protective and risk factors
3.1.1 Prognostic vs. causal
“Prognostic factors” allow predicting that a certain index problem will happen, but changing them has no impact on the index problem, while altering “causal factors” automatically impacts on the index problem. To give an example: turning off an alarm clock (prognostic factor) set to warn that potatoes are cooked, does not prevent the potatoes from getting overcooked but turning off the oven (causal factor) does.
3.1.2 Internal vs. external factors
“Internal factors” (also called “personal factors”) are located inside the target individual (e.g. “frustration tolerance”) and “external factors” (also called “environmental factors”) are located outside the target individual (e.g. “positive work place atmosphere”).
3.1.3 Passive vs. active factors
“Passive factors” impact on individuals without their involvement (e.g. immunisation against a certain virus due to vaccination) and “active factors” support individuals to cope with arising problems (e.g. life skills).
3.2 Different ways to define risk factors and protective factors
A summarizing table of the definitions can be found in Tab. 1.
3.2.1 Symmetric risk factors and protective factors (type 1)
The most global definition of “risk factor” is factors positively associated with the index problem. Based merely on association this concept includes prognostic factors as well as causal factors. According to this concept “protective factor” is the exact opposite (antonym) of “risk factor”. Since these factors represent two poles of the same quantitative dimension, the concept is symmetric and can be referred to as “quantitative concept”.
3.2.2 Symmetric causal risk factors and protective factors (type 2)
A more specific approach (e.g. Kumpfer & Baxley, 1997) defines “risk factors” as factors positively and causally linked to the index problem. Causal relationships provide information how to reduce the index problem through influencing risk factors and protective. Here again risk factors and protective factors are antonyms, the concept is symmetric and quantitative like the previous concept. The problem with this concept is that causality is usually assumed based on correlational data only.
3.2.3 Asymmetric causal risk factors (type 3)
A very different way to define causal risk factors and protective factors does not perceive risk factors and protective factors as antonyms but as different qualities (e.g. Antonovsky, 1987). According to this concept “risk factors” are external threats increasing the risk for problem manifestation (e.g. influenza viruses when the index problem is influenza) and protective factors as internal factors (e.g. antibodies due to anti-influenza vaccination). Within this concept lacking a protective factor must not be called “risk factor” and the non-existence of a risk-factor must not be labelled “protective factor”. Risk factors and protective factors according to this conception are located on two different qualitative dimensions; the concept thus is asymmetric and qualitative in nature (Tab. 1).
3.2.4 Asymmetric causal risk factors (type 4)
There is also a hybrid model between type 2 and type 3 existing in literature, where risk factors may be external as well as internal, while protective factors are always internal to the individual (Berman et al., 2006, cit. in Kumpfer & Baxley, 1997, p12) (Tab. 1).
Tab. 1: Terms describing positive and negative factors related to the index problem
|
Prognostic |
causal |
|||||
|
Internal |
external |
internal |
external |
|||
|
passive |
active |
– |
passive |
active |
– |
|
|
risk vs. protective factors (type 1) |
|
|
|
|
|
|
|
risk vs. protective factors (type 2) |
|
|
|
|||
|
risk factors (type 3) |
|
|||||
|
protective factor (type 3) |
|
|
||||
|
risk factors (type 4) |
|
|
|
|||
|
protective factor (type 4) |
|
|
||||
|
deficits vs. resources |
|
|
|
|||
|
weaknesses vs. skills |
|
|||||
|
Vulnerabilities vs. resiliencies |
|
|||||
3.3 Related factors to risk factors and protective factors
A summarizing table of the following definitions can be found in Tab. 1.
3.3.1 “Deficits” vs. “resources”
Another popular dichotomy closely related to “risk factors” vs. “protective factors” in a symmetric, causal sense (type 2) is “deficits” vs. “resources”. There are external deficits (e.g. good parental support) and internal ones (e.g. high intelligence). The opposite of any deficit is a resource (Tab. 1).
3.3.2 “Weaknesses” vs. “skills” and “vulnerabilities” vs. “resiliencies”
The terms “skills” vs. “weaknesses” as well as “resiliencies” vs. “vulnerabilities” refer exclusively to internal factors. While “skills” and “weaknesses” are the basis for active behaviours, the concepts “resiliencies” vs. “vulnerabilities” addresses the capacity to cope passively with challenges. To give an example: the ability to defend oneself against an aggressor is a skill, while to be immune against influenza viruses is a resiliency (Tab. 1).
Vulnerability and resiliency in everyday language are terms describing a state of susceptibility and lack of resistance to external threats. If taken literally these terms cannot take a plural form. In the process of analysing factors contributing to vulnerability and resiliency in the prevention field, authors abbreviated “factors contributing to vulnerability” into “vulnerabilities” and “factors contributing to resilience” into “resiliencies”. These neologisms may hurt linguists, but we have to accept them by now as an integrated part in the technical language of preventionists.
3.4 The importance of the context
It makes little sense to analyse factors predicting or causally explaining the incidence of various index problems without considering the context. To give an example: intelligence, usually functions as protective factor in relationship to adverse outcomes but this may be the other way round in specific situations as well; e.g. if a highly intelligent person – dramatically overqualified for a simple job at a production line – is unable to find a more challenging alternative.
4 What is evaluation?
The classical categorisation into “basic research” and “applied research” defines that the former is driven by the curiosity of scientists for knowledge's sake while the latter is explicitly designed to solve practical problems in the world. In this sense any research aiming at improving prevention is applied research.
4.1 The everyday conception vs. professional conception of evaluation
A popular classification differentiates between “prevention research” and “evaluation”. According to this conception all approaches necessary to develop the evaluation objects (prevention materials, techniques, strategies etc.) are subsumed under “prevention research” and “evaluation” is the “process of determining whether given evaluation objects is of practical value”. This classification is not in line with the definition of “evaluation” in the scientific and professional world, as a COST-A6-Expert Group (Uhl, 1998) pointed out. If we understand “evaluation” in a professional or scientific sense, all applied research endeavours to plan, develop and judge preventive efforts can be subsumed under prevention as the following sections show.
4.2 State of product (evaluation object) dimension
Scriven (1967, 1991), one of the most renowned international evaluation experts, developed a classification based on three states of the evaluation object. This three-step-concept was slightly amended by the COST-A6-Expert Group dividing the third phase into a “testing phase” and a “routine phase” arriving at:
“Preformative evaluation” circumscribes activities to conceptualise an evaluation object, while a first preliminary draft of a product is being developed, as a purely reflective process without any practical, empirically oriented steps. This concept phase may also be labelled “preformative phase”.
“Formative evaluation” circumscribes activities to form (develop and improve) the evaluation object as well as its background theories while the product is still being developed and not yet final. This developmental phase may also be labelled “formative phase”.
“Summative evaluation” in the testing phase circumscribes activities after the evaluation object is perceived as finished and not worked upon any more to sum up expected or unexpected effects of the evaluation object. This testing phase may also be labelled “first summative phase”.
“Summative evaluation” in the routine phase circumscribes activities after the evaluation object is considered final and useful and is implemented on a routine base. This approach is commonly referred to as “quality assurance” or “quality control”; a necessary step to prevent that application fidelity [15] is wearing off in the routine situation. This testing phase may also be labelled “second summative phase”.
4.3 Data dimension
Another important classification of evaluation relates to the type of data used for the evaluation. The COST-A6-Expert Group defined five data types relating to the classification “process evaluationvs. “outcome evaluationvs. “impact evaluation (e.g. Clayton and Cattarello; 1991) adding "structural data and “context data”:
This results in the classification:
“Structural evaluation” based on structural data; i.e. data describing the structural context, e.g. the place of intervention, the qualification of the persons executing the programme, characteristics of target persons, etc. The aim is to give an idea of the scope of measures set, costs produced, number of individuals targeted etc.
“Process evaluation” based on process data; i.e. all interventions by the programme staff (input), all reactions of the target population (output) and all relevant conditions (context) that might have an influence on the relationship between input and output. The aim is to explain hypothetically why certain interventions under certain context conditions produce certain results.
“Outcome evaluation” based on outcome data; i.e. an assessment of explicitly expected effects. The aim is to assess if the measures were able to achieve the intended goals.
“Impact evaluation” based on impact data; i.e. an assessment of unexpected desired as well as undesired effects. The aim is to assess if the measures produced any intended effects.
“Context evaluation” based on context data; i.e. various specific conditions that might moderate what results specific preventive measures yield. A good context analysis permits to assess the transferability of evaluation results to other contexts.
4.4 Methodological dimension
A third way to classify is between descriptive, exploratory and confirmatory evaluation. This perspective is related to the kinds of conclusions researchers may legitimately draw on epistemological (methodological) grounds.
“Descriptive evaluation” is a synonym for collecting and recording data, documenting and categorising phenomena and summarising the findings without direct attempts to formulate new hypotheses and theories.
“Exploratory evaluation” goes beyond mere description. Exploratory research ranges from collecting basic information in rather unexplored scientific areas to the hypothesis-driven development of new models and theories. There are no strict rules concerning the procedures in exploratory studies. Anything with a chance of providing a greater insight into relevant phenomena is possible and legitimate – as long as it is explicitly clear that the results of the exploratory phase are not final in any sense.
“Confirmatory evaluation” is not concerned with discovering new phenomena and/or formulating new hypotheses, but with proving existing hypotheses experimentally.
Controlled, randomised trials (RCTs) are considered to be the “gold standard” in empirical research to substantiate causality – but RCTs are commonly not feasible in SAP evaluation due to practical, ethical and/or economic reasons – and therefore different options to deal with effectiveness issues are inevitable Evaluator dimension
In any phases of evaluation it naturally is important who organises and directs the evaluation. These tasks can be carried out by programme developers and/or programme staff (internal evaluation), or by external experts (external evaluation). Advantages and disadvantages of both approaches will be dealt with in chapter 4.8.
4.5 Data, state-of-product, methodological, evaluator (DSME) classification
The COST A-6 WG II suggested integrating all the classification concepts mentioned above into the comprehensive, four-dimensional DSME-Classification System. DSME is composed of the four first letters of the below four dimensions.
Data dimension:
structural data
process data
outcome data (explicitly expected effects)
impact data (effects not explicitly expected)
context data
State-of-programme dimension:
concept phase (pre-formative phase)
development phase (formative phase)
testing phase (first summative phase)
routine phase (second summative phase)
Methodological dimension:
descriptive approach
exploratory approach
confirmatory approach
Evaluator dimension:
internal evaluation;
external evaluation.
The DSME classification is quite useful for describing evaluation but, despite integrating most established concepts to classify evaluation, it is not complete or sufficient. Several important forms of evaluation — from “needs assessment” to “efficiency evaluation” are needed to complete the picture (see COST-A6-Expert Group, Uhl, 1998).
4.6 “Evaluation” according to the guidelines of the German Society for Evaluation
In 2002 the German Society for Evaluation instituted a commission of seven evaluators and two client representatives to draft evaluation guidelines (Beywl, 2003). In this context a definition of “evaluation” was proposed as well and defined so broadly that even basic research is included. involving many very different evaluation objects .
According to this definition “professional evaluation” :
- should provide the information needed to judge the value of an evaluation object – but not necessarily needs to formulate a direct judgement on the evaluation object,
- needs a clarified purpose but does not necessarily need precise evaluation goals to start with[16],
- should be systematic and empirically based but not necessarily relying on standardised instruments,
- should document all steps precisely enough to make the results verifiable by others,
- should be performed by well trained evaluators,
- does not necessarily have to be planned before or parallel to the implementation of the evaluation object[17].
4.7 Pseudo evaluations
Which approach to chose for maximum returns depends on the circumstances.
If a competent and motivated team is interested in optimising performance and outcomes of their work, the ideal evaluation approach is an “formative, internal evaluation” supervised in close cooperation by an external evaluator. The team members profit from the experience of the external evaluator and his ability to provide new perspectives and ideas. The evaluator has access to the team members’ existing experiences and permanent involvement with the evaluation object. The evaluation cost are rather small since the involvement of the external evaluator is not extensive and the transfer of the evaluation findings into practise is guaranteed through the fact that the team in the course of the evaluation systematically implements improvements into their routine and in the end understands the final results good enough to profit even more.
If funders suspect that certain programmes or institutions produce unsatisfactory outputs, and if they hire external evaluators to check if their suspicion is justified, naturally only systematic external evaluations make sense. Since the project teams have little motivation to support such evaluations the evaluation steps must account for the reluctance of the staff to cooperate openly. Commonly this type of “external control evaluations” are commenced after decisions to close or change the structures have already been made by the relevant decision makers and the whole endeavour, with an external evaluator being aware what is expected from him, mutates into a pseudo ritual to present the previously taken decisions to third parties as “evidence based”.
A very special case are “routine evaluations” more and more expected as final step to all publicly funded projects. Once these projects have been started the funders who want to justify their expenditures, the project managers interested in good reputation and continued funding, and the evaluators, hired by either the funders or the project managers, have to please the party that involved them, in order to be awarded with similar contracts in the future. In all other words all three parties share a common interest in positive outcomes. These evaluations profit from the popular idea that external evaluators are more objective in their judgments than internal evaluators and commonly produce pseudo evaluations, useless to learn anything from them and never-the-less drawing from the scarce resources for project execution.
As just shown with some examples, which evaluation makes sense and how to organise it sensibly depends on the specific circumstances. If there is insufficient money available to do the job adequately, if the relevant partners are not truly interested in serious outcomes but need certain results to back up predecided decisions and/or if evaluation is only inconvenient, unavoidable ritual required for any publicly funded project, we cannot expect any truly useful and reliable results.
4.8 Evalopathy
After “substance abuse prevention” had been established solidly as profession in most developed societies the demand to evaluate standard preventive activities to scientifically prove their effectiveness steadily increased. Since public resources got scarcer and scarcer all the time the idea to allocate scarce resources in a way to maximise tradeoffs became more and more popular. The public and decision makers – confused by contradicting prevention concepts and expert opinions – expect that properly conducted evaluation of existing programmes will yield reliable results, helping them to choose the most promising approaches and to reject ineffective and counterproductive strategies. Due to this expectation they demand more evaluation in the field of prevention meaning a "proof of effectiveness". As a consequence of this demand the evaluation trade boomed and is still booming. Since professional evaluators have a very wide conception of “evaluation”, as shown above, they have no problem to evaluate virtually anything. As long as one side expects “proof of effectiveness” and the other side provides any assessment and documentation related to the evaluation object, and the first party does not realize that it does not get what it expects, both sides are happy. Those in between – the preventionists depending on public funds – play quietly along, being afraid that their funding will otherwise be discontinued. As long as “This has been evaluated!” is accepted as seal of quality without asking “How was the approach evaluated and what came out?” ridiculous rituals of pseudo-evaluation flourish and hardly anybody puts the finger on weak spots.
The idea that virtually everything has to be permanently evaluated, ignoring well known methodological problems and limitations as well as systematically misinterpreting the results has spread like a new form of disease through the scientific community. I previously labelled this disease “evalopathy” as a menace constantly producing output not good enough to learn anything from, but nevertheless drawing on scarce resources for the work that is being evaluated and that way indirectly diminishing the quality of the object being evaluated.
Not to be misunderstood: We are not at all against evaluation - on the contrary. We think that all our practical work should be routinely analysed and improved by ourselves and/or third parties. All these activities can be subsumed under the term “evaluation” in the professional sense. But we should consider that the workload for the evaluated and the additional resources invested into evaluation are balanced against the expected benefits and that insurmountable epistemological and economic research limitations are not simply ignored.
A sensible evaluation practise beyond what we called “evalopathy” could evolve if, for example, the feasibility standards of the already mentioned Evaluation Guidelines of the German Society for Evaluation (Beywl, 2003) were understood and taken serious in the professional world: “The (feasibility) standards are intended to ensure that an evaluation is planned and conducted in a realistic, thoughtful, diplomatic and cost-effective manner. …(Appropriate) evaluation procedures, including information collection procedures, shall be chosen so that the burden placed on the evaluand or the stakeholders is appropriate in comparison to the expected benefits of the evaluation. … Evaluation processes [in order to be efficient] shall meet scientific merit criteria while not unnecessarily burdening or imposing on the evaluand or stakeholders. The most relevant methods from a scientific point of view are often unsuitable because they are too time-consuming or costly or ethically unacceptable for the situation concerned. The evaluation team shall clarify advantages and disadvantages and justify the relevance of the chosen procedure…The decision on whether or not to conduct an evaluation shall certainly involve a cost-benefit assessment.”
5 Empirical question related to measuring effectiveness
Whenever the effects of interventions lie in the far future, whenever it is hard to measure effects objectively, whenever competing influences cannot be adequately controlled experimentally or statistically, and whenever the available sample sizes in practical applications are way to small to expect any statistically significant results, we have to accept research strategies that are miles away from the ideal approach for investigating causal relationships, i.e. randomized, controlled trials (RCTs).
5.1 Intuition vs. science based enquiry
There is no doubt that humans do not plan most of their behaviour conscientiously regardless if we believe in Adam Smith’s (1759) “invisible hand” [18], who claimed that humans react rational and egoistic in the sense of optimizing their gains (“homo economicus”) and that way indirectly advance the interest of the society and the individuals, or if go with Stiglitz (2002), who rejects this concept as inadequate[19]. The most influential schools of psychotherapy – psychoanalysis and behaviour therapy – claim that most of our behaviour is directed by subconscious processes (psychoanalysis) or are learned without explicit awareness of the process (behaviour therapy). If we are asked why we do certain things, according to Nisbett & Wilson (1977), we come up with instant explanations, but these are commonly not the results of true introspection, but we resort to “a priori, implicit causal theories, or judgements about the extent to which a particular stimulus is a plausible cause of a given response.” This cognitive process is not only relevant for explaining our own behaviour but equally important for explaining any phenomena we observe. As Fischhoff (1980) suggested, the automatic conviction that we understand phenomena and that our explanatory hypotheses are true, prevents us from spending much effort into checking them and thus commonly prevents us from learning anything about it.” To phrase it more directly: we should not perceive man as a rational being, but rather as a rationalising being.
All this should lead us to the conviction that we should not trust in individuals’ intuitions and experiences and that we should check all hypotheses with scientific methods. Around the second half of the 20th century important researchers and philosophers, who had analysed the logical structure and empirical feasibility of science came to the conviction that certainty in knowledge is impossible to achieve and that, particularly in human and social sciences, we are commonly confronted with insurmountable research problems. A popular statement depicting this situation very directly has been attributed to Albert Einstein who said: “For Nature, … is an exorable and not very friendly judge of [researchers’] … work. It never says "yes" to a theory. In the most favourable cases it says "maybe," and in the great majority of cases simply "No" (cited in Dukas & Hoffmann, 1989).
Some researchers on one extreme side, like Cochrane (1972)[20], supported the idea that Randomised Controlled Trials (RCTs) as well as Meta-Analyses are “gold standards” in research. At the same time in the next few lines he admitted that RCTs are not possible in many applied areas and then again instantly turned back to conclude that only RCTs can provide us with adequate knowledge[21]. The method, to admit postmodern problem awareness and at the same time spread positive optimisms about research and existing research findings is quite popular today – but inherently schizophrenic.
The other extreme pole was chosen by Gigerenzer (2007), who after systematically dealing with many research problems and limitations, turned into a vigorous supporter of intuition, arguing that evolutionary process provided humans with simple and efficient implicit rules (heuristics), hard-coded by the evolutionary processes or learned; rules that are commonly more efficient than complex strategies based on much information. He labelled this quality “gut feelings” or “unconscious intelligence” and gave examples showing that too much information is counterproductive in hard to predict situations.
Gigerenzer’s position relying on experience and intuition is positive in the sense, that it relieves us from the necessity to justify our behaviours empirically and/or rationally – but a “carte blanche” to rely on experience and intuition alone is insufficient. History is full with highly inadequate decisions – ranging from unsatisfactory to catastrophic – that brought enormous suffering to the public and to responsible decision makers themselves.
In between Cochrane and Gigerenzer is Dörner (1996), who in his book “The Logic of Failure: Recognizing and Avoiding Error in Complex Situations” divided problems requiring decisions into “Immediate ad hoc problems” and “long-term strategies”, thereby arguing that the evolutionary process works quite well to solve immediate ad hoc problems for individuals, but is hardly suited to shape long-term strategies to deal with complex situations. Dörner showed experimentally that many individuals relying on intuition alone react highly inadequate if confronted with having to plan long-term strategies.
Other experts dealing with the existing problems in research aim at a compromise, on one side openly accepting and considering the research limitations and on the other side suggesting that a critical, open and methodologically informed attitude can help to increase our knowledge basis to base practical decisions on, an approach requiring much ambiguity tolerance. Diederich (1974) formulated the necessary balancing act like this: “The insight, that virtually all scientific theories are false, makes it necessary to conceive scientific progress as a process allowing to decide between better and inferior theories – at the same time being aware that they all are false.”[22]”
5.2 Evidence Based Policy
At first sight the buzzword “evidence-based” in combination with medicine, prevention or policy seems to make sense, if we accept the definition of Sackett et al. (1996) meaning “conscientious, explicit, and judicious use of current best evidence” – but is this approach really new. Most scientists in the past thought that they had made best use of the existing evidence and their brain. To say: “I was able to draw the best conclusions” sounded preposterous, but if the same idea is expressed using the term “evidence based” it suddenly sounds unsuspicious like a neutral commitment to a superior methodology. At second sight we start to realize that the term “evidence-based” transports much more than the literal meaning proposed by Sackett et al. If individuals manage to attach the term “evidence-based” onto their conclusions – it acts like a seal of quality – conveying the implicit message that these conclusions are 'independent of subjective values' and 'proven beyond doubt' – which is absolutely misleading (Uhl, 2007).
5.3 Ethics in SAP
In section 1.2 was argued that mere effectiveness of measures in terms of reaching certain goals is not sufficient to decide to implement these measures – as the term “evidence based policy” misleadingly implies –, but that ethical aspects must be considered as well.
5.3.1 Participatory-emancipation perspective vs. paternalistic-controlling perspective
The central ethical issue in SAP is whether to focus on information, participation and emancipation or paternalistically on control and coercion. The former approach is in line with the health promotion concept defined by the World Health Organization Ottawa Charter (WHO, 1986) and with contemporary pedagogics. The latter approach was the main stream concept how to raise children several decades ago. Putting the focus in education and prevention on either approach naturally does not imply to totally reject the other one. Persons opting radically for “participatory-emancipatory strategies” very likely will be confronted with situations where the degree of self-destruction in the target person or the risk to endanger third parties warrants strict limitations and sanctions as a last resort. Persons in favour of an uncompromising “controlling and coercive strategies” have to provide a certain amount of tolerance, non-manipulative information and autonomy, or their educatees will turn against them, revolt and/or develop psychological problems.
For several decades in most Western countries SAP primarily focused on participatory-emancipatory strategies – but since the public movement against tobacco consumption and alcohol abuse gained impetus in the last years, paternalism and control-oriented ideas regained impact. Particularly the popular book “Alcohol No Ordinary Commodity” by Babor et al. (2003) reinforced this development through concluding that therapy and prevention are expensive and ineffective, while restrictions and sanctions are cheap and effective.
5.3.2 The impact of external influences
A common concern in research and evaluation is that external influences jeopardize the validity and reliability of research. This issue has been addressed in publication principles, like the “Farmington Consensus Statement” (Davis, 1997) drafted by the editors of 20 journals in the field of addictions, which made conflict of interest statements in scientific journals more and more indispensable. Even though a conflicts of interest can be financial, personal, political, or academic (Babor & McGovern, 2008), the focus is almost exclusively put on the impact of industry funding (alcohol industry, tobacco industry, pharmaceutical industry, etc.) and other important sources of influence are widely neglected. NGOs, emerging around certain goals, government agencies and any other research funders commonly expect certain outcomes as well and exercise their influence more or less subtly on research process and outcome. To focus exclusively on industry influences is a situation which provoked Rothman (1993) into labelling the one-sided and intensive fight against industry-funded research “New McCarthyism in Science”.
5.4 Complexity
Everybody is permanently subjected to many different influences that we neither can assess nor control. The only straight forward way to control these influences is to make sure, through randomization, that these influences affect an experimental and a control group equally. As previously mentioned such randomised controlled trials (RCTs) are commonly not feasible due to economic, practical or ethical reasons. This is particularly true if the expected effects are relatively small compared to the total of uncontrolled effects, since then enormous sample sizes are necessary to demonstrate treatment effects statistically.
5.4.1 Non-linear Systems – Generativity
The implicit assumption behind experimental trials to demonstrate programme effectiveness is that the implemented measures effect every individual homogeneously. The idea is that the effect can be expressed by a certain factor plus random variation around the parameter. This model is very appropriate for pharmacological trials. Taking a pill influences the pill taker directly and has little indirect carry-over effect on other individuals.
The situation is quite different though regarding social interventions. Small interventions usually having almost no direct impact on individuals may generate unpredictable effects in some individuals; effects that evoke further unpredictable effects and so on, until finally an abundance of unpredictable and non-reproducible consequences have arisen. Such outcomes may be labelled "generativity" (Uhl, 1998). According to Chaos Theory (e.g. Steward, 1989) such unpredictable cause-effect relationships are a common phenomenon in nature. One of the central contentions of Chaos theory is that the wing stroke of a butterfly in South America may cause a typhoon in Indonesia ("butterfly effect"). Those who consider the butterfly effect in the original version to be rather implausible, will probably have no problem to accept that the same butterfly can cause even greater catastrophes if it diverts the attention of a nuclear power plant technician in a crucial moment. “Generativity” in the above defined sense causes severe problems in evaluation designs. In some situations the magnitude of generativity outweighs systematic prevention effects by far. Generativity should not be mixed up with initially unanticipated but predictable systematic effects that may turn out to be expected effects in future evaluations (e.g. purposely stimulating public discussion in a certain way, changing public opinion, generating structural changes, etc.). The assessment of originally not explicitly expected but never-the-less basically predictable and systematic effects is called impact evaluation.
5.5 Problems in Measuring Treatment Effects
5.5.1 Programmes vs. Soft Skill
Prevention programme manual may be quite helpful for preventionists, but what really counts is what the preventionist makes out of these rough materials and how they actually interact with the target group. Whether a certain programme is implemented can easily be recorded, but all the important soft skills necessary to achieve preventive success are hard to measure and therefore commonly ignored. If only factors of minor importance are considered and the important factors omitted – to detect relevant changes attributable to the programme is hardly possible. The whole thing reminds about the drunkard, who loses his keys where it is dark and then looks for it near a lamp, since it is brighter there.
5.5.2 Proxy Variables – Surrogate Endpoints
Whenever we cannot sensibly assess the efficacy criterion in which we are interested in directly (ultimate outcome variable, endpoint) we often choose surrogate endpoints (proxy variables, presumptive mediating variables) supposedly causally linked to the ultimate outcome. The use of surrogate variables to indirectly assess programme effectiveness is perfectly justified as long as the causal relationships between intermediate variable and efficacy variable are well established through experimental or quasi-experimental empirical research in comparable contexts. The procedure is highly questionable though if causality assumptions are derived from correlations only and generalized to different interventions in different contexts. Association does not imply causality.
5.5.3 Moderate substance use as inadequate indicator for later problematic substance use
Popular proxy variables for “problematic substance use” in prevention research are e.g. “moderate substance use”, “attitudes towards substance use”, etc. The implicit logic of this approach is at first sight quite appealing. A person who never starts using alcohol at all cannot possibly end up using alcohol problematically, and individuals heavily rejecting alcohol can be expected not to use alcohol at all. The first statement is obviously a tautology needing no empirical verification and additionally is supported by findings that “substance use” correlates with “problematic substance use later on”. The second statement is supported by many correlational studies showing that persons using substances have a more positive attitude towards these substances than abstainers.
In spite of the obviousness of the above reasoning, it can be easily demonstrated to be flawed. It can easily be demonstrated in theory that “non-problematic substance use” and “problematic substance use” can develop into opposite directions. We need to consider heterogeneity in the population as well as time dynamics and think causally rather than in terms of association.
Concerning “substance use” as proxy variable to assess “future problematic substance use”: knowing that individuals with few social and psychic problems are less likely to totally abstain from alcohol and at the same time less likely to start abusing alcohol, we may expect that very successful health promotion in a population will decrease the number of future abstainers and at the same time decrease the number of future problematic alcohol users [23]. Interpreting “any alcohol use” as proxy variable to indirectly assess “future problematic alcohol use” later on would consequently lead to conclusions diametrically opposed to reality. A highly successful prevention project could be evaluated as producing a boomerang effect. A particular problem evolves along these lines, if youth are trained specifically to develop adequate ways to consume alcohol moderately in order to prevent future abusive consumption patterns. The central idea behind this prevention concept is that moderate substance use is not necessarily a risk factor for later substance abuse.
5.5.4 Interpreting Statements Wrongly As Attitudes – The Parrot Effect
Concerning “attitudes towards substance use” as proxy variable to assess “future problematic substance use”: for long time prevention against illicit drug focused on information concerning the short-term and long term effects of using these drugs. If children having no experience with illicit drugs yet are confronted with exaggerated risk descriptions they simply reproduce them like parrots (“parrot effect“). If they later on observe drug use among their friend and realize that their induced judgements are totally unrealistic, they may realize that they have been put on by adults and from then on rely on peer information primarily; often underestimating the actual risks involved. Exaggerated risk judgements thus may turn out as risk factor for future problematic use – in other words produce a “boomerang effect”.
To believe that a drug-inexperienced child repeating the sentence “drugs are dangerous and I will never touch them” will stay away from drugs is as naive as believing that a parrot taught to repeat “I hate fruit” will not touch the fruit bowl. Such statements do not reflect attitudes, but they are repetitions of previously heard phrases without emotional relevance. True attitudes naturally do determine behaviours very much – but they are hard to assess.
Manipulating intermediate variables does not necessarily result in corresponding changes in the ultimate target dimension and observed changes in intermediate. Reducing the temperature with antipyretic drugs will not hinder pneumonia from developing in the infected persons, just as raising the body temperature - e.g. in a sauna or in a hot bath - will not cause pneumonia in healthy persons. Similarly a reduction of fever (it could be caused by antipyretic drugs) is not a reliable indicator that the disease is vanishing and fever (it could be an indicator for different types of infections) is not a reliable indicator that a person will develop pneumonia
Not to be misunderstood, we are not claiming that preventing substance use or inducing negative attitudes towards substances is always and automatically counterproductive in terms of the criterion “future problematic substance use” – but we should be aware, that a “boomerang effect” could easily happen with certain interventions in certain context situations. We should be very careful to infer from changes in proxy variables on changes in the ultimate outcome variable.
5.5.5 Vagueness of verbal expression
Most verbal expressions are unavoidably vague and equivocal. Each person involved has to construct the other partner’s intended meaning from the context in order to understand – and this process commonly fails more or less severe. As Luhmann (2000) put it, “It is unlikely that communication happens.” We are hardly ever aware if we systematically misunderstand others[24] which is advantageous for our personal wellbeing but disadvantageous for understanding each other. This problem is particularly severe if specific communication to clarify the intended meaning is impossible, like in surveys, where professional interviewers usually are not aware of the intention behind many questions, or in self-administered questionnaires, where no support is provided at all. To give an illustrative example: In a qualitative evaluation of the Austrian ESPAD study a sample of 100 randomly selected participants was drawn who had already filled in the questionnaire. 16 of them had gravely misunderstood a simple question concerning their alcohol consumption at their last drinking day, in 4 cases it is likely that they misunderstood the question, 3 did not even try to understand the question and gave non-sense answers, 38 understood the question correctly but were unable to produce correct answers and only 39 understood the question as intended and answered correctly (Schmutterer et al., 2008).
5.5.6 Unavoidability of implicit communication
Since verbal communication is unavoidably vague and equivocal, as stated above, implicit communication based on non-verbal behaviour and context between the lines play an important role in communication. If we e.g. ask a person in he ever has used a certain substance and offer the categories “never”, “once or twice”, “three through five times”, “six through ten times” “eleven times or more”, we implicitly communicate that “eleven times” is extremely much”. If we in contrast provide the categories “never”, “once through twenty times”, “twenty-one through hundred times”, “hundred times or more” we implicitly communicate that “eleven times” is quite little. A particularly illustrative example of implicit communication happened to me when asking an interviewee several years ago if he was favouring depenalising Cannabis use. He did not answer as expected, but responded by saying “This is very interesting!”. Asked to explain his reaction he said “If a researcher from a renowned institute asks me such a question I conclude that my son, who claims that Cannabis is rather harmless, must be right. – and this is very interesting news to me.”
5.5.7 What we are interested in are long-term effects
Professional prevention, just like parenting and educating in schools aims at influencing individuals in a way to improve their prospects to lead a successful and joyful life – clearly a long-term perspective. Since limited interventions in a complex concert of uncontrollable competing influences can naturally only have limited effects, and since the scope of evaluation projects is usually restricted in time, demonstrating relevant long-term-effects on the intended target variables with statistical means is mostly impossible. To prove that a preventive intervention produces short-time effect only, as commonly pursued, is of little practical relevance but nevertheless commonly accepted as positive evaluation outcome.
5.5.8 Unreliable self-reports
Since assessing substance use and substance related problems through observation or objective tests is mostly infeasible, a common remedy is to ask persons to self rate their substance use behaviour. Since it is undoubtedly much easier to influence verbal behaviour than to impact on consumption behaviour, we have to be careful when interpreting self-reports. A good example is an experience made in the Austrian Household Surveys 2004 and 2008, covering the 15 through 99 year old population (Uhl et al., 2009). The measured lifetime prevalence of Cannabis use[25] in this 4 year period dropped from 18% to 12% – which is impossible. Current cannabis users can stop using cannabis but cannabis experienced persons cannot possibly regain their innocence[26]. We speculated that the reduced willingness to admit cannabis experiences could be a result of media reports covering experts who claimed that cannabis is less harmless than previously proclaimed.
5.6 Methodological problems
5.6.1 Programme Evaluation vs. Prevention Research
It is possible to gain certain experiences in limited prevention projects, and it makes sense to document them as well as to control if certain processes are performed as planned, but the evaluation of limited projects must not to be confused with systematic prevention research. Only the latter approach is capable to assess changes in variables related to programme effectiveness.
5.6.2 Treatment – Health Promotion – Prevention Dilemma
If a therapist treats an acutely ill patient the most effects happen in a close period of time and there is a wide range of possible symptoms to be included – any improvement in health counts as success. If we do health promotion the perspective is long term, but the scope of possible health improvements is large. If we do SAP the perspective is long term and the scope of relevant improvements is small – obviously the most problematic situation when effectiveness is to be demonstrated.
5.6.3 Analysis – fishing for significances
It is a widely accepted standard that using statistical significance tests is only sensible in decision studies based on one or a small number of primary endpoints. If more than one primary endpoint is chosen, the significance level has to be adjusted accordingly. In practice significance tests are commonly used in exploratory study phases and numerous significance tests simultaneously – an absolutely unacceptable procedure referred to as “fishing for significances” in the methodological literature.
5.6.4 Effect sizes, problem manifestation rate and number needed to treat (NNT)
Preventive interventions are usually just one tone in an orchestra of different concurrent and competing influences. Therefore a limited prevention programme inducing a 20% relative risk reduction (RRR) in problem incidence could be considered to be sensationally successful. If we e.g. consider “problematic use of illicit drugs” as index problem to prevent, we may estimate that roughly 1% of an average a population will develop such a problem in the course of their lifetime (total lifetime prevalence). Since the beginning of “problematic use of illicit drugs” can happen anytime over lifespan of individuals, with some particularly vulnerable age phases characterised by an elevated incidence, it is realistic to assume that the annual manifestation in any age group will hardly exceed 10% of total lifetime prevalence. Phrased differently, we may assume that no more than 1 out of 1000 (0,1%) individuals will show a problem manifestation annually if untreated (annual control event risk = CER). A relative risk reduction of 20% means that we may expect 0,8 problem manifestations in 1000 cases (0,08% = annual experimental event risk = EER) and translates into an absolute risk reduction (ARR) of 0,2 in 1000 cases (0,02%). Phrased differently, we need to treat 5000 subjects (the number needed to treat [NNT] is the reciprocal number of the absolute risk reduction [ARR], Doll (2008) in order to prevent one case of problematic use of illicit drugs per year.
5.6.5 Power Problems – Sample-size calculation
If we calculate the minimum sample size is needed in order to statistically demonstrate the effectiveness of any programme based on the above assumptions (20% relative risk reduction and 0,1% incidence of use of illicit drugs per year), if we decide to reach a statistical power of 80%, accept an Alpha error of 5% we arrive at a minimum sample size of 287 877 subjects in the control group and 287 877 subjects in the experimental group, adding to a total sample size of 575 754 subjects. If we prefer a statistical power of 90% and an Alpha error of 1% the minimum sample size per group increases to a total sample size of 1 190 382 subjects. If we furthermore consider that we usually do not deal with random samples but with cluster samples like school classes, that the criterion “problematic use of illicit drugs” cannot be assessed perfectly valid and reliable, and that we cannot avoid that our subjects are exposed to relevant competing influences our programme the minimum sample size to achieve a significant result has to be increased even further. Needless to say that such a large scale experiments to establish that a certain prevention programme is effective is unfeasible for practical and financial reasons.
5.6.6 Statistical regression artefacts – pseudo changes
Commonly we are not interested in specific states at a certain point in time but in a criterion defined as a generalised condition over a certain period of time. If problems and/or symptoms vary from day to day we usually do not interpret this as permanent process of improvement and aggravation, but we generalize over a longer period of time[27]. To assess this criterion we usually resort to indicators like the situation at the very point of assessment or within a limited time window of a few days. Due to this fact we are not only confronted with the simple measurement error measured with standard reliability indices but additionally with a generalisation error caused by the fact that the indicator does not perfectly represent the criterion. Both errors jointly produce statistical regression artefacts, well known and explained since Galton’s (1886) famous observation that small parents on average have larger siblings and large parents on average have smaller siblings without the variance of height decreasing from generation to generation.
In samples randomly drawn from the population and measured them at two points in time regression artefacts do not distort the change between two points of measurement. The same is true if random samples are drawn out subgroups defined by criteria that are not stochastically dependent on the dependent variable. If we select subpopulations based on criteria though that are identical or related to the dependent variable – e.g. if we select persons with alcohol problems to observe how their alcohol problems develop – or if self-selection process cause such a selection – normally persons seek treatment when their present problem state is relatively bad – regression artefacts can cause enormous pseudo changes[28]. If there was no intervention between assessment one and two these pseudo-changes are commonly misinterpreted as “spontaneous remission”. If there were interventions these pseudo-changes are commonly misinterpreted as “treatment effects”. A straight forward way to protect against the later error is to plan randomised controlled trials.
6 Conclusion
The situation concerning SAP and its evaluation is my no means easy, but we cannot give up SAP, just like we cannot give up parenting, school education, deciding an economic policy and many more activities, just because we have no sound basis to justify our decisions beyond doubt.
What is needed in such a context is willingness to define basic terminology, to accept insurmountable limitations, to creatively construct competing hypotheses and[29] theories based on common-sense, intuition, experience, specific research and rational thinking. As Popper (1972) suggested, we should defend our new hypotheses and theories for some time until we understand them thoroughly and then and then uncompromisingly try to challenge them in order to move on to a higher level of understanding.
Essential for this endeavour is ambiguity tolerance; to be able to openly admit the things we do not know at all and where we still have serious doubts. This is not easy in a world competing with experts ready to provide information on any topic – commonly so often that they have no time to inform themselves on the issues. Another essential prerequisite is to provide adequate working conditions for practitioners, evaluators and researchers permitting them to stay on a certain topic long enough to understand it sufficiently and have a chance to develop the issue further – with sufficient time to discuss and exchange with colleges working on similar issues, and with sufficient time to read and think. Unfortunately the global trend in research funding goes away from adequate and basic funding for centres of excellence and think-tanks to narrowly calculated project oriented funding.
According to my experience, most evaluators and researchers are very interested in increasing their knowledge and gaining true insight, but there is a three-stepped hierarchy to consider. Of primary importance is to be awarded sufficient projects to survive economically, since otherwise any further steps need not even be considered. Of secondary importance is to build up reputation, since only a good reputation guarantees surviving in the field under acceptable economic conditions. Of tertiary importance is gaining knowledge and true insight. Because of this hierarchy gaining knowledge and true insight for many evaluators turned into a precious jewel, they would really like to possess but unfortunately cannot afford, due to the way evaluation and research are organised and funded.
7 Appendices
7.1 Historical development of the classic “Public Health Classification System”
7.1.1 Classification 1: (Commission on Chronic Illness, 1957, and Joint Commission on Mental Illness and Health, 1961)
The classification “primary prevention” vs. “secondary prevention” was coined by the Commission on Chronic Illness (1957): “The Commission adopted definitions to distinguish between two major types of prevention - primary and secondary. Primary prevention means averting the occurrence of disease, for example, averting lung cancer by preventing human exposure to certain chromate ore operations. Secondary prevention means halting the progression of a disease from its early unrecognized stage to a more severe one and preventing complications or sequelae of disease.”
For years later, we find the almost identical position by the Joint Commission on Mental Illness and Health (1961): “More precisely, we construe primary prevention of mental illness to mean the elimination of causes, either by eliminating exposure to them or by building resistance … various levels of service beginning with secondary prevention - early treatment of beginning disturbances to ward of more serious illness, if possible – and continuing through intensive and protracted treatment of the acutely and chronically ill.”
According to both Commissions:
- “primary prevention” = interventions before the index problem is manifest
- “secondary prevention” = treatment after the index problem became manifest
7.1.2 Classification 2: Caplan (1964)
Caplan, G. (1964) added the term “tertiary prevention” to the concept of the Commission on Chronic Illness and redefined “primary prevention” as well as “secondary prevention”: “The term "preventive psychiatry" refers to the body of professional knowledge, both theoretical and practical, which may be utilized to plan and carry out programmes for reducing (1) the incidence of mental disorders of all types in a community ("primary prevention"), (2) the duration of a significant number of those disorders which do occur ("secondary prevention"), and (3) the impairment which may result from those disorders ("tertiary prevention").” Caplan does not consider “relapse prevention”.
According to Caplan (1964):
- “primary prevention” = interventions before the index problem is manifest
- “secondary prevention” = curative treatment after the index problem became manifest
- “tertiary prevention” = palliative treatment after the index problem became manifest
7.1.3 Classification 3: (Strasser, 1978)
Strasser (1978), who focused on preventing coronary heart disease, divided what Caplan had called “primary prevention” into “proto-prophylaxis” or “primordial prevention” in the sense of structural prevention targeting the whole society and formulated a reduced concept of “primary prevention” in the sense of behavioural prevention. He also added “relapse prevention” to the concept of “secondary prevention”: “Called also tertiary prevention, rehabilitation, particularly following myocardial infarction, has become a standard component of good medical practice. Endeavours to prevent recurrences or deterioration of coronary heart disease … called secondary prevention, such endeavours were soon followed by the concept of primary prevention, meaning averting inception of the disease. … Real grass root prevention should start by preserving entire risk-factor-free societies from the penetration of risk factor epidemics. Here lies the possibility of averting one of tomorrow's world health problems. For expressing this important concept, I wish to propose the term of proto-prophylaxis or primordial prevention.
The concept of Strasser (1978) defined:
- “primordial prevention” = structural prevention aiming at the whole society intending to influence persons without signs of the index problem
- “primary prevention” = any interventions aside from primordial prevention before signs of the index problem exist
- “secondary prevention” = harm reduction (stabilisation) and relapse prevention after index problem manifestation
- “tertiary prevention” = treatment after index problem manifestation
7.1.4 Classification 4: (Gordon, 1983)
Gordon (1983) referred to the two categories of the Commission on Chronic Illness (1957), mentioned that a third category “tertiary prevention” had been added by Caplan, and explained how the three categories were to be understood, but he did not mention, that his suggestions were different from all the previous approaches. Her defined: “These classes are summarily defined as primary – practiced prior to the biologic origin of disease; secondary—practiced after the disease can be recognized, but before it has caused suffering and disability; and tertiary - practiced after suffering or disability have been experienced, in order to prevent further deterioration.
Gordon (1978) defined:
- “primary prevention” = interventions before the index problem has begun to evolve
- “secondary prevention” = treatment after the index problem began to evolve but while no relevant negative consequences are present yet
- “tertiary prevention” = treatment after the index problem is manifestation and negative consequences have evolved
It should be mentioned, that Gordon just explained the “Public Health Classification System”, but was not at all in favour of this system and suggested the “Mental Health Classification System” instead (see section 2.2.7).
7.1.5 Classification 5: Kumpfer & Baxley (1997) or Dadds (2004)
Kumpfer & Baxley (1997) redefined the public health classification system once more within a NIDA publication. They formulated the concept which became more and more popular in the substance abuse field, and they also referred to previous authors, who had defined these terms differently – without mentioning these deviations. They did not explicitly include harm reduction in their considerations. The Kumpfer & Baxley concept concerning is still the most commonly used interpretation of primary vs. secondary vs. tertiary prevention in the substance abuse field.
They state: “Within the public health classification of prevention, antidrug efforts have been organized along a continuum of primary, secondary, and tertiary prevention (Commission on Chronic Illness 1957, …) The goal of primary prevention is to protect individuals who have not begun to use substances, thereby decreasing the incidence of new users. The goal of secondary prevention (also called early intervention) is to intervene with persons in the early stages of substance abuse or exhibiting problem behaviours associated with substance abuse to reduce and/or eliminate substance use. The goal of tertiary prevention is to end substance dependency and addiction and/or ameliorate the negative effects of substance abuse through treatment and rehabilitation. In this model, tertiary prevention is most often referred to as treatment, but also includes rehabilitation and relapse prevention.” From the way the previous authors are mentioned without stating any differences, the suspicion arises, that the authors have not actually read the original papers.
A similar definition was given by Dadds (2004): “The traditional model examines prevention from the perspective of onset of disorder (Caplan, 1964). In this model, prevention can be implemented at three levels. The first level, primary prevention, intercedes before the onset of disorder to reduce the likelihood of the person developing psychopathology. Secondary prevention is implemented once problems have been identified, but before these problems become severe. Finally, tertiary prevention involves treatment of current disorders with the aim of shortening the duration of the disorder and also preventing relapse.”
According to Kumpfer & Baxley (1997) and Dadds (2004)
“primary prevention” = interventions before signs of the index problem exist
“secondary prevention” = interventions aiming at persons with elevated risks to develop the index problem
“tertiary prevention” = treatment and relapse prevention after index problem manifestation
7.1.6 Classification 6: (Uhl, 1998)
An international COST-A6-Group of European Prevention Experts (Uhl, 1998) suggested to go along with the Kumpfer & Baxley concept but suggested to split “tertiary prevention” into “treatment” (“tertiary prevention type A or “tertiary prevention”) and “relapse prevention” (“tertiary prevention type B” or “quaternary prevention”). Harm reduction was not considered there as area either: “Within the scientific understanding of "prevention" we can differentiate four distinct areas of preventive action”.
Primary prevention is to prevent the onset of a substance related problem.
Secondary prevention is to intervene if a problem is likely to occur (prevention in high-risk-groups) and/or if a problem exists but is not yet fully manifested. In both cases the aim is prevention of problem manifestation.
Tertiary prevention - type A is to deal with a problem once it is fully manifested (prevention of further harm in addicts).
Tertiary prevention - type B is to prevent a problem from reoccurring again once it has been successfully treated (relapse prevention).
“Obviously the four distinct areas of preventive actions defined above cannot be unambiguously described with three categories. Therefore it has been suggested by members of the working group to introduce the term "quaternary prevention" as unique new entity for relapse prevention. This suggestion certainly makes sense, since the concept of relapse prevention is of increasing importance to the field.”
According to the COST-A6-Expert Group:
- “primary prevention” = interventions before signs of the index problem exist
- “secondary prevention” = interventions aiming at persons with elevated risks to develop the index problem
- “tertiary prevention” = treatment after index problem manifestation
- “quaternary prevention” = relapse prevention after successful treatment of the index problem
7.1.7 Classification 7: (Department of Mental Health and Substance Abuse, 2004)
A very deviating interpretation similar to the old conception of Caplan (1964) was formulated by the WHO Department of Mental Health and Substance Abuse (2004) in cooperation with the Prevention Research Centre of the Universities of Nijmegen and Maastricht. “Primary prevention” was defined as intervention before manifestation of the index problem, “secondary prevention” as treatment after problem manifestation (early intervention and treatment) and “tertiary prevention” as interventions after treatment success (support, rehabilitation, and relapse prevention).
According to the Department of Mental Health and Substance Abuse (2004):
- “primary prevention” = interventions before the index problem becomes manifest
- “secondary prevention” = curative treatment after the index problem became manifest
- “tertiary prevention” = further support and relapse prevention after the index problem manifestation respectively after successful treatment
7.2 Historical development of the modern “Mental Health Classification System”
7.2.1 Classification: Gordon (1983)
Gordon (1983) argued against the Public Health Classification System as mentioned in section 2.2.6 and coined a different system (universal, selective, and indicated prevention) – an approach later referred to as Mental Health Classification System by Mrazek & Haggerty (1994). In this concept prevention is explicitly restricted to interventions set before the index problem becomes manifest: “…we propose would restrict the use of the term "preventive" to measures, actions, or interventions that are practiced by or on persons who are not, at the time, suffering from any discomfort or disability due to the disease or condition being prevented.… universal, is a measure that is desirable for everybody. … member of a subgroup of the population distinguished by age, sex, occupation, or other obvious characteristic whose risk of becoming ill is above average. These measures we shall call selective … we propose to term indicated, encompasses those that are advisable only for persons who, on examination, are found to manifest a risk factor.”
According to Gordon (1983):
- “universal prevention” = aims at the whole population
- “selective prevention” = aims at groups with an elevated risk to develop the index problem
- “indicated prevention” = aims at individuals at risk before index problem manifestation
The classification is inexact respectively incomplete in the sense that focusing at groups with average risk (i.e. school based programmes) is not explicitly considered as category at all. Average school based programmes neither focus at the whole population (like media campaigns) nor do they address groups with an elevated risk. Such interventions could be classified “universal prevention” or “selective prevention” within Gordon’s concept.
7.2.2 Classification: Kumpfer & Baxley (1997)
Kumpfer & Baxley (1997), like Gordon, described the Public Health Classification System (see chapter 2.2.6) as well as the Mental Health Classification System. Kumpfer & Baxley were more precise than Gordon and defined explicitly, that working with average school classes is to be included under “universal prevention” and that “indicated prevention” targets individuals with elevated risk – not necessarily with fully developed risk factors.
“A universal preventive intervention is one that is desirable for all members of a given population. For example, it would include the general population and subgroups such as pregnant women, children, adolescents, and the elderly. … Selective prevention interventions target specific subgroups that are believed to be at greater risk than others. .. Selective prevention targets the entire subgroup regardless of the degree of risk of any individual within the group. … The mission of indicated prevention is to identify individuals who are exhibiting early signs of substance abuse and other problem behaviours associated with substance abuse and to target them with special programmes.”
According to Kumpfer & Baxley (1997):
- “universal prevention” = aims at the whole population or groups with average risk,
- “selective prevention” = aims at groups with an elevated risk to develop the index problem,
- “indicated prevention” = aims at individuals at risk before index problem manifestation
7.2.3 Classification 10: (EMCDDA, 2007)
EMCDDA (2007) added the term “environmental strategies” – in line with Strasser’s “primordial prevention concept” to the concept of Kumpfer & Baxley meaning structural prevention efforts aiming at the whole population. Consequently this leaves for the category “universal prevention” any behavioural oriented interventions aiming at the whole population. More detailed explanations make evident, that the category “environmental strategies” and “universal prevention” include measures addressed at group with average risk (i.e. school based programmes). The categories “selective prevention” and “indicated prevention” are defined similarly to Gordon (1983) and Kumpfer & Baxley (1997).
“In this line, environmental approaches are prevention measures that operate on the level of these social, formal and cultural norms about alcohol, tobacco and also cannabis. While universal prevention intervenes on population level, selective prevention at (vulnerable) group level, and indicated prevention on individual level …”
According to EMCDDA (2007):
- “environmental prevention” aims at the whole population or groups with average risk through structural changes
- “universal prevention” = aims at the whole population or groups with average risk through behavioural oriented measures,
- “selective prevention” = aims at groups with an elevated risk to develop the index problem,
- “indicated prevention” = aims at individuals at risk before index problem manifestation
It is worth mentioning, that the explicit distinction between “environmental prevention” and “universal prevention” was dropped again in the EMCDDA “Prevention Manual” (Burkhart, 2010).
7.3 Table providing an overview over the different classification systems
The following table presents an illustrative overview over the different concepts mentioned in the two previous sections.
Tab. 2: Different Classification Systems
|
factors addressed |
structural factors |
behavioural factors |
||||
|
Type of intervention |
prevention |
prevention |
prevention |
treatment |
relapse prevention |
harm reduction |
|
Stage of index problem |
average |
average |
elevated risk |
problem manifest |
problem overcome |
problem manifest |
|
Commission on Chronic Illness, (1957) |
1.p |
2.P |
||||
|
Caplan, G. (1964) |
2.p |
– |
3 p |
|||
|
Strasser (1978) |
pm.p |
1.p |
1.p |
3.p |
2.p |
2.p |
|
Gordon (1983) |
1.p |
2.p or 3.p depending whether suffering or secondary problems have evolved or not |
||||
|
Kumpfer & Baxley (1997) |
1p |
2p |
3p |
|||
|
Uhl (1998) |
3p |
4p |
– |
|||
|
Department of Mental Health and Substance Abuse (2004) |
1.p |
2.p |
3.p |
|||
|
Gordon (1983) |
u.p. |
s.p. if risk groups; |
– |
|||
|
Kumpfer & Baxley (1997) |
u.p. |
– |
||||
|
EMCDDA (2007) |
e.p |
u.p. |
– |
|||
|
Burkhart (2010) |
u.p. |
|||||
Explanation:
1.p. = primary prevention, 2.p = secondary prevention, 3.p = tertiary prevention, 4.p = quaternary prevention, pm.p = primordial prevention, u.p. = universal prevention, s.p. = selective prevention, i.p. = indicated prevention, e.p. = environmental prevention
Evaluation of drug prevention activities:praCtice
by Richard Ives
8 Introduction
This chapter considers some examples of the evaluation of drug prevention activities and critically analyses the results and approaches taken in the light of the theory and logic presented in previous chapters. It first considers two systematic reviews of drug education; then looks at four major drug prevention initiatives and their evaluations; and then at some smaller pieces of work that have relevance in this study of evaluation approaches. Finally, some conclusions are drawn, and some recommendations made for future drug education and future drug education evaluations.
The reviews are:
- the Cochrane evaluation of drug education by Faggiano and colleagues
- Gates and colleagues review of drug education in non-school settings
- the review of drug education for the UK’s NICE (National Institute of Health and Clinical Excellence) by Liverpool John Moore’s University
The major evaluations are:
- EU-DAP: an evaluation of the ‘Unplugged’ lifeskills-based drug education programme implemented in several EU countries
- Blueprint: an evaluation of a multi-component drug education programme in the UK
- Project ALERT, created by the RAND Corporation, a two-year, classroom-based intervention for 12- to 15 year-olds, with a parent component aiming to motivate students to be against drug use and give them the skills to turn the motivation into behaviour
- CTC, Communities that Care, a community development approach to drug prevention within a wider preventive context.
Other evaluations are:
- Wirral (UK) Life Education Centres[30]
- Lifeskills-based drug education in the Sverdlovsk Region of Russia implemented by the Pompidou Group
- The Pompidou Groups’ multi-dimensional drug education capacity-building and training programme in the Region
- educari‘s evaluation of a theatre-in-education intervention in a North London school
9 Cochrane systematic reviews
Cochrane Reviews are systematic reviews of primary research in human health care and health policy. Each systematic review addresses a clearly formulated question.All the existing primary research on a topic that meets certain criteria is searched for and collated, and then assessed using stringent guidelines, to establish whether or not there is conclusive evidence about a specific treatment.
10 Review of interventions in school settings
A systematic review of ‘School-based prevention for illicit drugs use’ was carried out by [31]Its objective was to evaluate the effectiveness of school-based interventions in improving knowledge, developing skills, promoting change, and preventing or reducing drug use versus usual curricular activities or a different school-based intervention.
The authors asserted that schools offer the most systematic and efficient way of reaching young people. They suggested that school programmes can be designed to:
‘provide knowledge about the effects of drugs on the body and psychological effects, as a way of building negative attitudes toward drugs; to build individual self-esteem and self-awareness, working on psychological factors that may place people at risk of use; to teach refusal and social life skills; and to encourage alternative activities to drug use, which instil control abilities.’
The Cochrane Registers and other databases were searched and work of researchers and reference lists of articles were explored. Randomised controlled trials (RCT), case controlled trials (CCT) and controlled prospective studies (CPS) evaluating school-based interventions designed to prevent substance use were included in the review.
The authors found 32 controlled studies (29 RCTs and 3 CPSs), comparing school-based programmes aimed at prevention of substance use with the usual curriculum. The 46,539 students involved were mainly in sixth or seventh grade in the USA. Programmes that focused on knowledge improved drug knowledge somewhat in six randomised trials. Social skills programmes (more widely used – in 25 randomised trials) effectively increased drug knowledge, decision-making skills, self-esteem, resistance to peer pressure, and drug use. In 20 randomised trials, the programmes were mainly interactive and involved external educators. Effects of the interventions on assertiveness, attitudes towards drugs, and intention to use drugs were not clearly different in any of the trials.
Most trials were conducted in the USA and, as a nation's social context and drug policies have a significant influence on the effectiveness of the programmes, these results may not be relevant to other countries. Measures of change were often made immediately after the intervention with very little long-term follow up or investigation of peer influence, social context, and involvement of parents.
The authors concluded that ‘skills based programs appear to be effective in deterring early-stage drug use. The replication of results with well designed, long term randomised trials, and the evaluation of single components of intervention (peer, parents, booster sessions) are the priorities for research.’
11 Review of interventions in non-school settings
Gates and colleagues[32] reviewed interventions for prevention of drug use by young people delivered in non-school settings such as youth clubs, primary care centres, colleges, with families and in the community. The authors identified 17 controlled studies, 9 cluster randomised studies with 253 clusters and 8 individually randomised studies with 1230 participants. All but two of the studies were conducted in the USA. There were too few – and very different – studies to draw any firm conclusions on whether non-school based interventions prevent or reduce drug use by young people. The interventions with suggested benefits need further evaluation before it can be firmly established that they are effective.
12 Review of drug education for the UK’s NICE
NICE, the National Institute of Health and Clinical Excellence (which provides guidance, sets quality standards and manages a national database to improve people’s health and prevent and treat ill health), based its intervention guidance on how to reduce substance misuse among vulnerable young people on a systematic review of the evidence by researchers at Liverpool John Moore’s University.
Since vulnerable groups of young people report higher levels of drug use than their non-vulnerable peers and account for a disproportionate percentage of drug users, NICE developed public health intervention guidance on the most effective and cost-effective community-based interventions for the reduction of substance misuse in vulnerable and disadvantaged young people.
The review looked at the evidence of effectiveness of community-based interventions at reducing substance misuse among vulnerable and disadvantaged young people, and at reducing the risk factors among vulnerable and disadvantaged young people that may affect their propensity to misuse substances.
A total of 222 studies met the NICE inclusion criteria; 14 were systematic reviews (SRs); 104 were randomised controlled trials (RCTs); 54 were controlled non-randomised trials (CNRTs), 15 were controlled before and after studies (CBAs) and 35 were before and after studies (BAs).
Based on the evidence from the review, NICE’s recommendations include:
- develop a local strategy
- work with parents and carers and other organisations involved with children and young people to provide support and, where necessary, to refer them to other services.
- offer motivational interviews to those who are misusing substances.
- offer group-based behavioural therapy to children aged 10–12 years who are persistently aggressive or disruptive – and deemed at high risk of misusing substances. Offer their parents or carers group-based parent skills training.
- offer a family-based programme of structured support to children aged 11 to 16 years old who are disadvantaged and thought to be at high risk of substance misuse.
13 Major Studies
13.1 EU-Dap
The ‘European Drug Addiction Prevention trial’ (EU-Dap), was a multi-centre study implemented by nine partners from seven different European countries. EU-Dap developed a theory-based school programme for the prevention of use of tobacco, drugs and alcohol, and, using an experimental design, assessed its effectiveness.
The Unplugged programme was thus developed. It is a 12-hour class-based curriculum based on a comprehensive social-influence approach. The lessons are given by school teachers who are trained in the programme over two-and-a-half days. A student’s workbook contains activities that students work through during the Unplugged units. The twelve lessons cover:
- Opening Unplugged
- To be or not to be in a group
- Choices – Alcohol, Risk and Protection
- Your beliefs, norms and information – do they reflect reality?
- Smoking the cigarette drug – Inform yourself
- Express yourself
- Get up, stand up
- Party tiger
- Drugs - Get informed
- Coping competences
- Problem solving and decision making
- Goal setting
The programme was evaluated during the 2004-05 school year among 7,079 12 to14-year-old students. The countries involved in the first phase were Italy, Belgium, Austria, Sweden, Spain, Greece and Germany. A self-completion anonymous questionnaire was administered before, and at several points after, the students participated in the programme in 170 participating schools. Three combinations were implemented in three groups of intervention schools:
- basic curriculum: Unplugged programme led by teachers
- parent component: Unplugged programme plus the involvement of parents
- class-peer component: Unplugged programme plus the involvement of peers.
On a short-term follow-up, the programme was found to reduce the use of tobacco, other drugs, and drinking to intoxication by 25 to 30 per cent compared to the expected trends. The authors concluded that: ‘School curricula based on a comprehensive social-influence model may delay progression to daily smoking and episodes of drunkenness.’[33]
Further details, and the Unplugged materials in 11 languages, are available from www.eudap.net
The Unplugged Manual[34] suggests that two ‘traditional’ approaches to prevention: ‘the knowledge model’ and ‘the affective model’ have had limited success. The Manual’s authors write that:
The knowledge model is based on the assumption that providing adolescents with factual information about potentially destructive behaviour, like smoking or drug use, will prevent them from initiating that behaviour. Research has shown that this method of prevention, with foundations on fear arousal, has limited or no effect. … The affective model… addresses factors related to self-esteem, decision-making and goal setting and often exclude detailed information concerning health consequences of the behaviour it self.
They go on to describe a third model, ‘the social influence model’ (lifeskills is an example) which looks at drug use as social behaviour – this approach has been more successful. What increases the effectiveness of school-based programmes? They suggest the following:
- teacher training, plus support from programme developers or prevention experts
- appropriate information about drugs, including information on the short-term effects and the potential long-term consequences
- an element of normative education: reinforcing awareness that most adolescents do not use drugs (many young people overestimate the frequency of drug use among their peers, normative education helps them to interpret correctly the occurrence and acceptability of the behaviour in their environment)
- focus on personal, social and resistance skills to help to identify internal (e.g. anxiety and stress), as well as external (e.g. peer and advertising) pressures
- structured broad-based skills training such as goal setting, stress management, communication skills, general social skills and assertiveness skills
- active family and community involvement
- cultural sensitivity – for example by including activities that require teacher and student input and which can be tailored to the cultural experience of the classroom.
In addition: ‘independently of the underlying theoretical model, an interactive curriculum has shown to be more effective than a non-interactive one in preventing both illicit and legal drug use among adolescents.’ The authors describe ‘non-interactive programmes’ as ones where the teacher conveys knowledge and the learner is the recipient of information. By contrast, interactive programs focus on active learning which engages the students in a dynamic process of learning information, and developing skills: feedback and constructive criticisms in a safe environment facilitate the practice of skills being taught in the intervention..
As a general principle, ‘prevention programs should indeed be designed to enhance "protective factors" i.e. factors that are associated with protective effects and reduced potential for drug use, while reversing or minimising known "risk factors”.’
And, they suggest, ‘prevention programs should also be long-term (throughout the school years), with repeated interventions to reinforce the original prevention goals. For example, programs carried out in the elementary and middle school should include booster sessions to help with the critical transition from middle to high school.’ It is also important that the whole programme is implemented – too often programmes are only partially implemented.
For school-based prevention, teachers are the key to effective implementation; their motivation and enthusiasm can increase the effectiveness of the intervention. Curriculum implementation and overall programme effectiveness and sustainability are increased when trained teachers, familiar with the theory and conceptual framework of the programme and with its content, teach the programme.
13.2 Blueprint
The Blueprint Programme tested out an evidence-based, well-structured and well-resourced drug education programme; drawing on evidence of effectiveness from the USA. It was piloted in 23 schools in 2004 and 2005; 11- and 12-year-olds received 15 lessons from their usual teachers who had had six days' training in the Programme. Interactive teaching methods were used, and skills development and correcting the normative fallacy[35] were important elements. Classroom work was reinforced by a whole-school approach plus work with parents and the wider community. Other local agencies took action to reduce underage sales of alcohol, tobacco and inhalants.
Originally it was intended to recruit a comparison sample of schools against which the impact of Blueprint could be assessed. However, this turned out to be beyond the scope of the evaluation. The evaluation therefore focused on how well the programme was implemented in the 23 Blueprint schools. An additional six schools not assigned to the programme could not act as a comparison group, but provided some context. Surveys before, and around three years after, the intervention assessed pupils' drug use, attitudes, beliefs and reactions to the lessons; parents and carers were also surveyed.
Pupils enjoyed the lessons (especially the active teaching methods such as role-play), they had remembered the drugs knowledge, and had experience in handling offers of drugs. Parents were pleased that their children had been taught about drugs, liked the materials, and said the programme had increased their knowledge and helped them communicate with their children about drugs. However, pupils from the six schools which did not implement Blueprint were also positive about their drug education, and their parents were pleased that their children had drug education at school. And, of course, despite the Blueprint intervention, as pupils grew older, more of them reported smoking, drinking and drug use; drug-taking was associated truancy and exclusion from school. Many pupils overestimated how many of their peers smoked and drank, although fewer overestimated drug use.
Because of the methodological limitations, no firm conclusions can be made. But since few differences were found between the intervention and non-intervention schools, it reinforces the view that drug education in secondary schools does not contribute to preventing or reducing drinking and illegal drug use (though the evidence regarding smoking prevention is stronger). This disappointing result is probably accounted for by noting that, however well structured, school-based drug education can only have small, if any, prevention impacts. Another possibility is that Blueprint's drugs lessons replaced lessons timetabled for personal, social and health education, which were themselves an effective intervention. Some teachers found the Blueprint approach too prescriptive and not adapted to the needs of their pupils. Blueprint relied on 'normative education' but this component, was not adequately implemented and the validity of the research presented was sometimes questioned by pupils. The 23 schools were selected from a much larger group; they were willing and able to take part and probably had relatively well-advanced and well-organised health education in place; it cannot be assumed that the majority of schools would be able to implement this programme.
A fuller report of the evaluation is available on the Drug and Alcohol Findings website.[36]
13.3 Project ALERT
Project ALERT, created by the RAND Corporation, is a USA universal drug prevention programme also suited for some ‘at risk’ groups. It is two-year, classroom-based intervention for 12- to 15 year-olds, with a parent component. It aims to motivate students to be against drug use and give them the skills to turn the motivation into behaviour.[37] Teachers and others who teach the programmes must complete online training
Project ALERT, uses the social influence model of prevention. Three theories of behavioural change underpin it:
- the health belief model: cognitive factors motivate healthy choices
- the social learning model: social norms are key determinants of behaviour
- self-efficacy theory: a person’s self-belief enables them to act (e.g. in resisting peer pressure).
The programme’s underlying assumptions include:
- young people start using drugs due to social influences
- drug prevention programmes should develop motivation to resist drug use
- drug prevention programmes should target substances first used by young people
- students should be actively involved in learning.
Evaluations by the RAND Corporation since the mid-1980s, found that it did not affect those already smoking cigarettes and that the effects on drinking alcohol were small and short-lived. So, since most pupils had already tried alcohol, it was decided to focus more on harm reduction (i.e. on harmful drinking), and a smoking cessation lesson was added. A more recent evaluation found that, compared to controls, there was significant reduction in risky or harmful drinking, and smoking was reduced even among current smokers. The lessons reduced pupils’ susceptibility to pro-substance use peer influences, improved the accuracy of their estimates of their peers’ substance use, and increased awareness of potential harmful consequences.[38]
13.4 Communities that Care
Communities that Care (CTC) is a prevention approach developed in the USA, which has been applied to communities in the UK. It is a process of community development based on evidence and led by a coalition within the community. First, the coalition is constructed, and a need assessment identifies risk and protective factors around substance use and youth delinquency. An implemented action plan used a variety of interventions known to be effective in other contexts.
A randomised study by Hawkins[39] compared 12 small-town communities which implemented CTC with 12 that did not. CTC communities saw more improvement in youth behaviours, including substance-misuse related behaviours, than the comparison group. Before the activities started, 4,407 10 to 11-year-old pupils were questioned, and then followed annually over three years. By age 12 to 13, delinquency increases had already been significantly curbed in CTC communities (and this trend continued), but only over the following year was there a significant impact on rates of trying alcohol, binge drinking and cigarette smoking, and there was no impact on trying cannabis or inhaling solvents. Combining all substances, the changes were statistically significant compared to the comparison communities. This confirms that, with promising towns and careful implementation, CTC can reduce young people’s smoking and drinking.
There are many attractions of the CTC approach. It draws on the powers of community action; it overcomes what is often the major problem of ‘implementation fidelity’ by having communities develop their own needs assessment and implementation. It has a broad strategy aiming at multiple problems; it does not only focus on young people; it looks at, and attempts to deal with, multiple causes of problems, and does not only address issues at the individual level; this helps to avoid stigmatisation of individuals and is also a more realistic, more effective (and evidence- informed) approach.
14 Minor studies
14.1 Wirral LEC
Life Education Centres (LEC) is an approach to drug prevention that targets younger children with activities that emphasise the importance of a healthy organism. Originating in Australia, the LEC approach has been widely adopted in the UK. Wirral Life Education (in the North-West on England) was evaluated by the Centre for Public Health, Liverpool John Moore’s University. The evaluation focused on the impact of LEC on pupils’ knowledge, it did not measure skills development, changes in attitudes, or behaviour. In four test schools and one control schools the pupils completed two tests (before and after the LEC session) assessing change in their knowledge (such as the effects of smoking, and the implications of alcohol use). An audit of teacher’s and parent's opinions about LEC was also carried out.
Significant improvements in knowledge were found between pre- and post-LEC sessions for five key questions in the test groups. The greatest improvements were seen for knowledge about the physiological harms of alcohol, amount of daily exercise, amount of sleep needed, assertive behaviour, effects of smoking, and social effects of alcohol. The teacher’s response to the LEC programme was overwhelmingly positive. Parents also responded positively, and found the information on drugs and medications particularly useful.
The evaluators conclude: ‘these findings suggest Wirral Life Education Centre can contribute towards government initiatives for 21st-Century Schools, providing an opportunity for school pupils to increase their knowledge on important health issues, which potentially could be translated into real life improvements in health and wellbeing of this population.’ (p39)
14.2 Lifeskills-based drug education in Sverdlovsk Region, Russia
This project was implemented by the Pompidou Group. The project, commencing in the 1990s, aimed to move drug education and prevention from being solely information-based towards a life-skills-based approach, which emerging evidence at the time was showing to be the most effective approach.
However, the Pompidou Group seeks to transfer knowledge between countries and across cultures in a sensitive way, gaining the support of partners through adapting approaches to the needs of the locality. Thus the drug prevention expert employed as the Consultant on this project did not impose the skills-based approach, but with the support of the Pompidou Group and Council of Europe colleagues, worked to convince the local partners of the validity of this approach and the value of adopting it in their situation. This was achieved partly by extended working with a mixed group of local experts, from schools, children’s homes, drugs services and other relevant agencies. This expert group shared their knowledge about drugs issues in the local context and was informed about different approaches to drug education. Then, in a co-operative process, a drug education manual for school was developed,
This Manual was based on the life-skills approach, but included elements that the local partners felt were important – such as recommendations for school medical practitioners. The Manual offered over 30 hours of lessons, intended to be taught during the ‘class-teachers’ hours’; time when the students were in their form-rooms with their most familiar teacher. Few of the lessons were drug-specific but were concerned with developing skills and exploring attitudes. Many of the lessons involved games and activities – since an active approach to learning had been shown to be more effective. This approach was enthusiastically received by most teachers. Lessons took on local relevance by, for example, using local folk tales to illustrate relevant points.
This project was not formally evaluated – like many Pompidou Group projects, its value lay in its innovation and exploratory nature, testing out approaches in different cultural environments. Its development was piecemeal; expanding as more funding became available and as local interest grew. The Manual was adapted by some of the local partners for use in the children’s homes environment, where the young people reached by the programme were potentially more vulnerable to drug-related problems.
The Manual took on a life of its own, some of its content being discovered in drug education manuals from other parts of Russia. This was something both to celebrate and to be concerned about – to celebrate, because ‘organic dissemination’ was occurring; and to be concerned, not because the Pompidou Group wanted to control or copyright the Manual’s contents, but because the Manual was a ‘package’ for drug education and prevention, intended to function as a whole programme.
Week-long training courses for teachers were an important part of this programme. These aimed to allay teachers’ fears about ‘teaching drugs’ and to explore the nature of substance misuse in their part of Russia – after this part of the training everyone was clear that the focus needed to be on alcohol and tobacco – by far the most widely used psychoactive substances. The training also aimed to give teachers confidence to lead their classes using active learning techniques – which although unfamiliar to most teachers – were enthusiastically practiced on the training courses,
14.3 Pompidou Group: capacity-building and dissemination
Some members of the Expert Group were involved in the training and dissemination activities, and after several activities over a number of years, these local experts became very familiar with the Manual and the associated training. When the opportunity came to undertake a similar drug education project in Zaporozhye Region of Ukraine, members of the Group where involved as Pompidou Group experts. As Russian-speakers, their involvement made many practicalities easier (such as reducing the need for interpretation). More importantly, their experience of using the Manual in a comparable cultural context gave the Ukrainian partners more confidence in using and promoting it as an appropriate response to the need for drug education in schools. As in Sverdlovsk Region, local partners included both school-based and non-school-based partners, and a local NGO was given the resources to act as an effective local partner.
In partnership with the Education Ministry in Lithuania and the National Drug Control Department, some work was done to adjust the Manual for use in another cultural context. Here, the main benefit of the work was not the adoption of a version of the Manual used in Sverdlovsk Region, but a training course that helped local partners to define what they wanted to do and to identify key people and local resources to progress drug education in schools.
One very important aspect of the work was the involvement of young people. This is central to the Pompidou Group’s approach to drug education and prevention – and is based on evidence that young people’s involvement in issues that concern them increased effectiveness. As the PG training courses developed, the role of young people became more important. In Ukraine, for example, three young people from the same school as the teachers being trained participated fully in the week’s training course, and made contributions on the role of young people in implementing the programme. In Lithuania, a member of the PG’s Young People’s Drug Prevention Prize Jury contributed to the training.
14.4 Pompidou Group: testing contrasting approaches in three areas of Russia
Wanting to engage a wide range of professionals in prevention, the PG initiated a project ‘Drug Prevention Support Network for Parents and Professionals’ in Sverdlovsk Region as well as in the cities of Kurgan, Chelyabinsk and Hanti-Mansiysk. The idea was to set up support networks (involving both parents and professionals) which would help to advance drug prevention. The towns were chosen partly because they were very different to each other and therefore provided an opportunity to test out different approaches to meet the project aims. For example, Hanti-Mansiysk is in an oil-producing area, the town is relatively well-supplied with public services and many people are connected to the internet. Here, the PG attempted a partly internet-focused strategy to engage parents and professionals, which would have been unrealistic in poorer Kurgan, where few professionals and even fewer parents had regular internet access.
Part of the project involved the identification of key professionals in the three areas. These professionals were helped to organise events locally and were themselves offered training and support, locally and at a Workshop in Helsinki, where the three groups of professionals worked together.
An evaluation of the work was conducted by two independent professionals from Ekaterinburg with support from the PG’s Consultant. The evaluation examined different aspects of the project, including its management and the work in the three pilot areas. Characteristics of the key participants were collected, and outcomes of the work measured. After the initial training events in the three pilot areas, a post-course evaluation telephone interview was conducted. At the joint training event in Helsinki in April 2005 the two Russian-speaking evaluators participated as observers and conducted semi-structured interviews with the participants and unstructured interviews with the organisers and experts. At the end of the project there was a further round of telephone interviews. There was evidence that the Project stimulated concrete activities, such as new or improved websites, and local training events. Formally assessing the impact of the work was beyond the scope of the evaluation.
15 Youth work interventions
Youth work with at-risk youth shows promise – but to be effective it must be interactive and well-structured, and staffed by engaged workers who are supported. This was the finding of an analysis of 48 USA Government-sponsored projects for at-risk 9- to 18-year-olds. [40] Key measures were changes in cigarette, alcohol, and cannabis use compared to control groups. On these measures, the most effective projects focused on substance-free recreational activities which were used to develop personal and social lifeskills. But the lifeskills approach had to be combined with interactive or experiential learning (rather than passive, lecture-style approaches) for it to be effective. These interactive methods were particularly effective when fostered ‘connectedness’ between young people and adults (through collective activities, mentoring, etc) and when they helped young people to look at their own attitudes and behaviours. School projects were less effective than after-school activities, probably because classroom settings limited the scope for including the positive features. Longer and more intense (at least 3.3 hours per week) projects were more effective, but these elements were less important than projects having conceptual coherence and which supported staff in delivering planned activities – such projects were consistently more effective.
In other research, interactive youth work projects which respond to young people’s priorities and which, rather than focusing on drugs, address broader vocational, lifeskills and health issues, have been found to be most attractive to high-risk teenagers.[41] Other effective work with high-risk youngsters has involved interactive family skills training for both parents and children, bringing them together to practise more constructive interactions.
16 Evaluation of a theatre-in-education intervention
A familiar technique for drug prevention in many countries is the use of educational drama. A UNODC booklet gives some examples.[42] This often involves a theatrical presentation followed by workshops for young people where they can explore through drama the implications of what they have seen on stage. If it does not involve an interactive element and is simply a passively-watched performance, or if the theatrical input is a ‘one-off’ and not integrated into the school’s curriculum, it is unlikely to be effective.
A multi-dimension evaluation, in North London schools, of a theatre-in-education intervention which included a performance and workshops, demonstrated (short-term) changes in pupils’ knowledge about drugs, drugs-related skills, and attitudes towards drugs. The evaluation methods included: script analysis, participation observation, questionnaires to students (including a modified ‘draw-and-write’ technique); focus groups, and interviews with teachers and with the actors. Compared to the pre-intervention data, students post-intervention showed that students had: more accurate drug-related knowledge, more developed drug-related skills, and some changed drug-related attitudes.
17 Non-drug-related interventions
Prevention and education initiatives might work better if they were implemented earlier in a child’s schooling; most of those who use drugs problematically start to experiment at a young age, so starting substance education early (with follow-ups and developments in the later school years) could be more effective.
There is some evidence that general attempts to create schools conducive to healthy development will affect substance use along with other behaviours. A long-term randomised trial of the ‘Good Behaviour Game’ reported impacts on substance-related problems at age 19 to 21 years, around 14 years later.[43] This long-term follow-up is a strength of the study, as its measurement of substance misuse problems – experimentation is relatively normal behaviour but is sometimes defined as a problem in evaluation studies. Developed and trialled in the USA, the method has also been successfully implemented in the Netherlands.[44]
The ‘Good Behaviour Game’ is a technique for class management, aiming to reduce aggression or disruptive behaviour and improve children’s acceptance of the role of ‘pupil’. Pupils are awarded prizes for the good behaviour of the team to which they belong. Over the school year, the game time increases from short periods to three hours a week, timing becomes more varied and less predictable, rewards are awarded less immediately and change from tangible things to school-related activities valued by pupils, such as extra time for reading. The evaluation found a range of beneficial effects; young men, especially those who earlier had been rated as aggressive or disruptive, had not developed so many anti-social behaviours; and substance misuse was reduced.
The ‘Good Behaviour Game’ does not occupy scare curriculum time; the usual curriculum is followed and teachers can teach less disruptive classes more effectively. Given that it is difficult to find curriculum time for drug prevention, this is a valuable aspect of the programme. This approach also makes sense: badly-behaved children often have multiple underlying and interacting problems. A positive school environment, (which in other studies[45] has been related to substance use) can help.
The principles of the ‘Good Behaviour Game’ (including setting achievable objectives, rewarding good behaviour, harnessing positive peer pressure, internalising the reward structure, helping children to set their own rules, etc.) will be recognised as important in Social Education. They are included in SEAL (Social and Emotional Aspects of Learning); a programme in British primary schools,[46] but SEAL focuses more on learning social and emotional skills – it is less ‘behaviourist’. However, the ‘Good Behaviour Game’, being practical and easy to describe step-by-step in a manual, may be easier to apply with consistency.
18 Conclusions
A number of conclusions can be drawn from the rather limited evidence base which has been described. These conclusions are now listed under several subheadings.
18.1 Limitations and difficulties of evaluation
Evaluation is as much an art as a science. It is an underdeveloped field, and there are is agreement about what counts as appropriate and effective evaluation. There are many reasons for this; for example:
- Outcomes and results are hard to assess: the deceptively simple question, ‘Does it work?’ is never easy to answer.
- Multiple intended outcomes across different domains are particularly difficult to assess.
- Projects that anticipate long-term impacts generally cannot be measured over such an extended timescale.
- The timescales of evaluation are different to those of policy-makers, who typically need quick answers to complex, multi-layered questions.
Evaluation results (even those carried out successfully) can be hard to translate into practice.
Evaluation is resisted by some: some practitioners (who might find evaluation intrusive); some advocates of a particular prevention approach (who might feel threatened); some policymakers (who might not be able effectively to assess empirical evidence).
18.2 Different views on what drug education and prevention aims to achieve
Similarly, there is no consensus on the purposes of drug education. Different perspectives include:
- prevention of psychoactive substance use – this is generally unachievable, especially given that alcohol is so ubiquitous; although total prevention of tobacco use is an increasing possibility in some contexts
- delaying the onset of psychoactive substance use – this seems to be possible, and it is desirable given the evidence that substance use may adversely affect the developing brain, and that those who start substance use early are more likely to develop problems
- prevention of more dangerous substance use – some substances, and some ways of using them, are more dangerous than others; drug education can inform about such risks and reduce the risks that young people take.
18.3 Pointers towards effectiveness
It is difficult to be conclusive about the results of drug prevention activities. Rather than speak of certainties, we must talk about the ‘balance of probabilities’, or the ‘direction of travel’. With this caveat, where does the evaluation of drug prevention point? Here are some travel directions:
- drug prevention in schools can improve students’ knowledge about drugs, but the intervention has to be fairly lengthy and it has to be refreshed or ‘boosted’ (for example, in follow-up lessons in later years of schooling) if the knowledge is to be maintained
- skills-based drug prevention in schools can develop students’ drug-related skills – such as the ability to say ‘no’ to an offer of drugs, but the application of these skills in real-life situations is limited
- school students can be helped to explore their attitudes to drugs, but the outcomes of such explorations might not be the development of unfavourable attitudes to drug use
- a school drugs curriculum needs to be sufficiently substantial (more than 12 sessions, plus follow-up ‘booster’ sessions)
- interactive methods than involve students are more effective than ‘traditional’ classroom teaching
- additional elements such as peer-led activities can add value to conventional lessons
- changing behaviour through universal, school-based prevention is difficult
- involving parents is difficult but can improve effectiveness – as well as information about substances, parents need help in communicating with their children about drugs issues
- a whole-school approach to drug prevention has a better chance of making a difference than drugs lessons alone – this would include such initiatives as ‘healthy schools’
- general behavioural approaches (such as particular classroom management techniques that create a structured learning environment) may also be effective in reducing drugs-related behaviour
- outside of school, community-based approaches to drug prevention show promise in mobilising communities to take action on some of the roots of substance use and delinquency
- while universal drug education for all young people might be seen as a right, a focus on at-risk groups (‘targeted prevention’) is likely to more cost-effective, at least in the short term.
19 Evaluation: future developments
While the importance of evaluation is more widely recognised than ever before, there are still debates about the underlying philosophy, about the ethics of evaluation and about different approaches.
Some projects (for example, those with multiple intended outcomes across different domains) are hard to assess, and projects that anticipate impacts in the long-term generally cannot be measured over such an extended timescale. The results of evaluations (even those carried out successfully) are sometimes hard to translate into practice. The timescales of evaluation might be different to those of policy-makers, who typically need quick answers to complex, multi-layered questions. The deceptively simple question, ‘Does it work?’ is never easy to answer. There is resistances to evaluation: resistance from practitioners (who might find evaluation intrusive); from advocates of a particular prevention approach (who might feel threatened); and from policymakers (who might not be able effectively to assess empirical evidence).
The Society for Prevention Research's (SPR)[47] Standards of Evidence: Criteria for Efficacy, Effectiveness and Dissemination sets out the expectations of quality evaluation.[48] However, these Standards are demanding and require high standards of evidence and rigorous methodology which are not appropriate – or possible (being too lengthy and expensive) – for many prevention projects.
These Standards come from a particular methodological perspective. There is disagreement and debate about appropriate approaches to evaluation. Some argue that approaches such as randomised controlled trials (RCTs) require a believe in attribution – that is, there is a logical chain of argument which says that if we make this intervention, then this will or should, happen. An alternative approach is to think about the contribution of an intervention. This approach allows the question, ‘what is the added value of the intervention?’ Instead of working from the intervention’s input to impute outcomes, one works from observed outcomes and imputes backwards: this is the result, what might be the contribution of the intervention?
Given the exploratory nature of prevention activities and how difficult it is to impose consistent implementation, ‘programmatic’ interventions may not be appropriate and more open-ended interventions, responsive to the developing situation are required.[49] One might have a separate formative stage of the intervention where the methods can be varied which then moves to a summative stage where the methodology becomes fixed. This approach accepts that there will be changes during implementation – as it moves from the exploratory to the explanatory – but it allows the change to be tracked, and not ignored.
If this logic is accepted, a purely ‘outcomes approach’ to evaluation becomes hard to sustain. What has sometimes been neglected in the ‘outcomes approach’ is the theoretical basis underlying the intervention – the ‘theory of change’. Logic models can help to explicate the connection. A well-constructed logic model may also make evaluation easier and simpler. A logic model will reveal causal links that will demonstrate that the intervention is well-founded in theory – if this is the case, there may be no need to attempt almost impossible evaluation, for example, it is well established that there are risk factors for substance misuse and protective factors that help to prevent it. If an intervention addresses these factors then it is likely to contribute to drug prevention – the evaluation question then is not the difficult (and often long-term) one of ‘does this intervention “work’”, but the question, has the intervention successfully addressed the risk and protective factors that it proposed to do. The question – an easier and more appropriate one – then becomes one not of outcomes achieved but of successful processes.
Liverpool John Moore’s University is currently undertaking a project on quality standards in drug prevention, aiming to‘… improve European drug prevention practice by creating an empirically derived reference framework to bridge the gaps between science, policy and practice. The overall objective of the project is to compile, review and analyse drug prevention standards in EU Member States.’
The outcomes of this project will be‘… a set of commonly agreed evidence-based drug prevention standards for use in the EU. Adoption of these standards will improve drug prevention practice and efficiency of funding, and reduce the likelihood of implementation of ineffective and iatrogenic interventions. Developing a common methodology and locally relevant guidance will provide an impetus for partners and other EU Member States to review and update standards to reflect the evidence base.’ [50]
Antonovsky, A. (1987): Unravelling the Mystery of Health. How People Manage Stress and Stay Well. San Francisco: Jossey-Bass Publ.NIDA (1997): Preventing Drug Use Among Children and Adolescents A Research Based Guide. National Institute on Drug Abuse, Rockville, MD
Babor, T.; Caetano R.; Casswell, S.; Edwards, G.; Giesbrecht, N.; Graham, K.; Grube, J.; Gruenewald, P.; Hill, L.; Holder, H.; Homel, R.; Österberg, E.; Rehm, J.; Room, R.; Rossow, I. (2003): Alcohol: No Ordinary Commodity - Research and Public Policy. Oxford University Press, New York
Babor,T.F.; McGovern, T. (2008): Dante's inferno: Seven deadly sins in scientific publishing and how to avoid them. In: Babor,T.F.; Stenius, K.; Savva,S.; O'Reilly, J. (eds.): Publishing addiction science: a guide for the perplexed (2nd edition). International Society of Addiction Journal Editors, London
Bloch, A. (1985): Murphy’s Law Complete. Methuen, London
Brecht, B. (1948): Life of Galileo - translated by John Willett. Penguin Classic, 2008, New York
Bruun, K.; Edwards, G.; Lumio, M.; Mäkelä, K.; Pan, L.; Popham, R.E.; Room, R.; Schmidt, W.; Skog, O.J.; Sulkunen, P:, Österberg, E. (1975): Alcohol Control Policies in Public Health Perspectives. Finish Foundation for Alcohol Studies, Volume 25
Büechi, M.L. (2003): Public Health Policy in Evidence-Based Management of Drug Addiction - Discussion paper. Paper presented at the Pompidou Group ministerial conference. 16-17 October 2003, Dublin
Bühringer, G.; Batra, A. (2004): Industriesponsoring: Teufelszeug, akzeptable, sinnvolle oder notwendige Finanzierungsquelle? Sucht, 50, 2, 99-101
Burkhart, G. (2010): Prevention and Evaluation Resources Kit (PERK) - A manual for prevention professionals. EMCDDA Manuals, Lisbon
Campbell, D.T. (1988): Evolutionary Epistemology. In: Campbell, D. T.; Overman, E. S. (Ed.): Methodology and epistemology for social sciences: Selected papers. University of Chicago Press, Chicago; Original: Campbell, D. T. (1966): Evolutionary Epistemology. In: Schilpp. P.A. (Ed.): The Philosophy of Karl R. Popper. LaSalle, New York
Caplan, G. (1964): Principles of Preventive Psychiatry. Basic books, New York
Cochrane, A.L. (1972): Effectiveness & Efficiency – Random Reflections on Health Services. The Royal Society of Medicine Press Limited, London (reprinted 2004)
Commission on Chronic Illness (1957): Chronic Illness in the United States: Prevention of Chronic Illness (Volume 1) . Harvard University Press, Cambridge
Cronbach, L.J.; Gleser,G.C.; Nanda, H.; Rajaratnam, N. (1972): The Dependability of Behavioral Measurements: Theory of Generalizibility for Scores and Profiles, Chapter 1, The Multifacet Concept of Observational Procedures. John Wiley & Sons, USA
Dadds, M.R. (2004): Potential for innovative prevention strategies. In: Teesson, M.; Byrnes, L. (eds.): National Drug Strategy and National Mental Health Strategy National Comorbidity Project. Commonwealth Department of Health and Aged Care Canberra, Australia
Davis,R.M. (1997): The Farmington Consensus Statement on editorial guidelines for addiction Journals. Tobacco Control, 6,167-168
Department of Mental Health (1999): Partners in Life Skills Education. WHO, Geneva
Department of Mental Health and Substance Abuse (2004): Prevention of Mental Disorders - effective Interventions and Policy Options. WHO, Geneva
Diederich (Hrsg.) (1974): Theorien der Wissenschaftsgeschichte – Beiträge zur diachronischen Wissenschaftstheorie. Suhrkamp, Frankfurt a.M.
Doll, H.A. (2009): Making sense of the numbers: CER, EER, ARR, RR, RRR, NNT, CIs. The Foundation Years, Volume 5, Supplement 1, 2009, Pages 1-7
Dörner, D. (1996): The Logic of Failure: Recognizing and Avoiding Error in Complex Situations, Perseus Books, Cambridge
Dukas, H.; Hoffmann, B. (1989): Albert Einstein – The Human Side. Princeton University Press, Princeton
EMCDDA (2007): The overall framework of prevention strategies. <www.emcdda.europa.eu/html.cfm/index1568EN.html > (31.12.2007)
Faul, F.; Erdfelder, E.; Lang, A.G.; & Buchner, A. (2007): G-Power 3: A Flexible Statistical Power Analysis for the Social, Behavioral, and Biomedical Sciences. Behavior Research Methods, 39, 175-191
Galton, F. (1886): Regression Towards Mediocrity in Hereditary Stature. The Journal of the Anthropological Institute of Great Britain and Ireland, 15, 246-263 Gerber, J.; Jensen, E. (eds.) (2001): Drug War American Style. Garland Publishing, London
Gigerenzer, G. (2002): Calculating Risks: How to Know When Numbers Deceive You. Simon & Schuster, New York
Gigerenzer, G. (2007): Gut Feelings - The Intelligence of the Unconscious. Viking, New York (Bauchgefühle)
Gordon, R. (1983): An Operational Classification of Disease Prevention. Public Health Report 98, 2, 107-109
Greene, J.A. (1999): Zero Tolerance: A Case Study of Police Policies and Practices in New York City. Crime & Delinquency, 45, 2, 171-187
Hartnoll, R. (2004): Drugs and Drug Dependence: Lessons learned, challenges ahead. Council of Europe Publishing, Strasbourg
Hartnoll, R. (2004): Drugs and Drug Dependence: Lessons learned, challenges ahead. Council of Europe Publishing, Strasbourg
Hayek, F.A. (1988): The Fatal Conceit. Routledge , Chicago
Holden, C. (2001): Behavioral Addictions: Do They Exist? Science, 294, 980-982
Hume, D. (1951): A Treatise of Human Nature, Book 3 of Morals, Part 1, Of Virtue and Vice in General. In: Watkins, F. (Ed.): Hume Theory of Politics. Nelson and Sons, Edinburgh; Original: First Edition (1740) Thomas Longman, London
Inciardi, J.A; Harrison, L.D. (Eds.) (2000): Harm Reduction: National and International Perspectives. Sage, Thousand Oaks
Isaacs, D.; Fitzgerald, D. (1999): Seven Alternatives to Evidence Based Medicine. British Medical Journal, 319
ISAJE Ethics Committee (2009): Ethical Practice Guidelines in Addiction Publishing: A Model for Authors, Journal Editors and other Partners. <http://www.parint.org/isajewebsite/ethics.htm> (27.12.2009)
Joint Commission on Mental Illness and Health (1961): Action for Mental Health. Final Report of the Joint Commission on Mental Illness and Health. John Wiley & Sons, New York
Joint Commission on Mental Illness and Health (1961): Action for Mental Health; Final Report. Basic Books, New York
Kant, I. (1784): Answer to the question: What is Enlightenment?. Berlinische Monatsschrift, 4, 481-494
Kelling G.L.; Coles C.M. (1998): Fixing Broken Windows: Restoring Order and Reducing Crime in Our Communities. Touchstone, New York
Kritz, J.; Lück, H. E.; Heidbrink, H. (1990): Erkenntnis- und Wissenschaftstheorie. Leske, Opladen
Kuhn, T. S. (1976): The Structure of Scientific Revolutions. University of Chicago Press, Chicago
Kumpfer, K.L.; Baxley, G.B. (1997): Drug Abuse Prevention: What Works. National Institute on Drug Abuse, Rockville
Liessmann, K. P. (2009): Theorie der Unbildung: Die Irrtümer der Wissensgesellschaft (“Theory of Bad Education: Errors in a Society Based on Knowledge”) (2nd edition). Piper, Munich
Liessmann, K.P. (2005): Evaluation: Wie viel wiegt Wissen?. Spectrum, 29.1.2005
Lorenz, K. (1977): Behind the Mirror. Methuen, London
Luhmann, N. (2000): Die Unwahrscheinlichkeit der Kommunikation. In: Pias, C.; Vogl, J.; Engell, L.; Fahle, O.; Neitzel, B.: Kursbuch Medienkultur. Die maßgeblichen Theorien von Brecht bis Baudrillard. Deutsche Verlags-Anstalt, Stuttgart
McGarry, D.D. (1962): The Metalogicon of John Salisbury. University of California Press, Berkeley
Moore, G.E. (1903): Principia Ethica. Cambridge: Cambridge University Press (second paperback edition, 1960).
Mrazek, P.J.; Haggerty, R.J. (eds) (1994): Reducing Risks for Mental Disorders. Frontiers for Preventive Intervention Research - report of the Committee on Prevention of Mental Disorders Division of Biobehavioral Sciences and Mental Disorders Institute of Medicine. National Academy Press, Washington DC
Nisbett,RE; Wilson, TD. (1977): Telling More Than We Can Know: Verbal Reports on Mental Processes. Psychological Review, 84, 3, 231-259
O’Boyle, E.J. (2007): Requiem for Homo Economicus. Journal of Markets & Morality, 2, 321–337
Oerter, R. (1999): Klinische Entwicklungspsychologie: zur notwendigen Integration zweier Fächer. In: Oerter, R., von Hagen, C., Röper, G., Noam, G. (Hrsg.): Klinische Entwicklungspsychologie. Ein Lehrbuch, 1-10. Weinheim: Psychologie Verlags Union
Paglia, A.; Room, F. (1998): Preventing Substance Use Problems Among Youth: A Literature Review & Recommendations - ARF Research Document No. 142. Addiction Research Foundation Division Centre for Addiction and Mental Health, Toronto
Pirie M. (1985): The book of the fallacy. London: Routledge & Kegan Paul.
Popper, K. (1959): The Logic of Scientific Discovery, Hutchinson & Co, London
Popper, K. (1976): Original Unended Quest. An Intellectual Autobiography, Fontana, London (Ausgangspunkte)
Popper, K.R. (1972): Objective Knowledge - An Evolutionary Approach. Oxford University Press, New York
Rothman, K.J.; Greenland, S. (1998): Modern Epidemiology Second Edition. Lippincott Williams & Wilkins, Philadelphia
Rothman, K.J. (1993): Conflict of Interest. The new McCarthyism in Science. JAMA, 269, 2782 – 2784
Sackett, D.L.; Rosenberg, W. M. C.; Gray, M.; Haynes, B.; Richardson, S. (1996): Editorial: Evidence Based Medicine: What it is and what it isn't. British Medical Journal, 312, 71-72
Scriven, M.: Evaluation Thesaurus, 4th Edition. Sage, Newbury Park, 1991
Scriven, M.: The Methodology of Evaluation. In. Tyler, R. W., Gagne, R. M., Scriven, M. (ed.): Perspectives of Curriculum Evaluation. Rand McNally, Chicago, 1967
Smith, A. (1759) Theory of Moral Sentiments.
Smits, J.M. (2006): Elgar encyclopedia of comparative law. Edward Elgar Publishing, Cheltenham
Schmutterer, I.; Uhl, A.; Strizek, J.; Bachmayer, S.; Puhm, A.; Kobrna, U.; Grimm, G.; Springer, A. (2008): ESPAD Austria 2007: Europäische SchülerInnenstudie zu Alkohol und anderen Drogen - Band 2: Validierungsstudie. Bundesministerium für Gesundheit, Wien
Stampfer, M.J; Ridker P.M.; Dzau, V.J. (2004): Risk Factor Criteria. Circulation, 109, IV3-IV5
Steward, I. (1989): Does God Play Dice? The Mathematics of Chaos. Penguin, London
Stiglitz, J. (2002) There is no invisible hand. guardian.co.uk, Friday 20th December
Strasser, T. (1978): Reflections on Cardiovascular Diseases. Interdisciplinary Science Reviews, 3, 3, 225-230
Uhl, A. (1998): Evaluation of Primary Prevention in the Field of Illicit Drugs: Definitions - Concepts - Problems. In: Springer, A.; Uhl, A. (eds.): Evaluation Research in Regard to Primary Prevention of Drug Abuse. A COST-A6 Publication. European Commission Social Sciences, Brussels (www.api.or.at/lbi/download.htm)
Uhl, A. (2000): The Limits of Evaluation. In: Neaman, R.; Nilson, M.; Solberg, U.: Evaluation - A Key Tool for Improving Drug Prevention. EMCDDA Scientific Monograph Series, No 5, Lisbon
Uhl, A. (2002): Schutzfaktoren und Risikofaktoren in der Suchtprophylaxe. In: Röhrle, B. (Hrsg.): Prävention und Gesundheitsförderung Bd.II. DGVT, Tübingen
Uhl, A. (2007): How to Camouflage Ethical Questions in Addiction Research. In: Fountain, J.; Korf, D.J. (eds.): Drugs in Society European Perspectives. Radcliffe, Oxford
Uhl, A.; Strizek, J.; Puhm, A.; Kobrna, U.; Springer, A. (2009): Österreichweite Repräsentativerhebung zu Substanzgebrauch 2008, Band 1: Forschungsbericht. Bundesministerium für Gesundheit, Wien
von Weizsäcker C. F. (1978): Deutlichkeit - Beiträge zu politischen und religiösen Gegenwartsfragen. Deutscher Taschenbuchverlag, München
WHO (1969): Expert Committee on Drug Dependence, Sixteenth Report. Technical Report Series 407, Geneva
WHO (1986): Ottawa Charter. World Health Organisation, Geneva
WHO (2009): Lexicon of alcohol and drug terms. <http://www.who.int/substance_abuse/terminology/who lexicon/en/print.html> (15.8.2009)
Pompidou Group publications and documents
20 Publications
Risk factors in adolescent drug use: evidence from school surveys and application in policy, by Richard MUSCAT, Thóroddur BJARNASSON, François BECK and Patrick PERETTI-WATEL, [ISBN978-92-871-6196-3], February 2007
Drug treatment demand data – Influence on policy and practice, by Hamish SINCLAIR [ISBN 10:92-871-6086-4/ISBN 13:978-92-871-6086-7], October 2006
Psychological drug research: current themes and future developments, by Jorge NEGREIROS [ISBN-10:92-871-6032-5/ISBN-13:978-6032-4], September 2006
Biomedical research in the drugs field, by Richard MUSCAT [ISBN-10 :92-871-6017-1/ISBN-13 :978-92-871-6017-1], July 2006
Research on drugs and drug policy from a European perspective - Selected readings from the 15th International Conference of the European Society for Social Drug research (ESDD), [ISBN 3-89967-270-4 5 (Europe) / ISBN 1-59326-078-4 (USA)], 2005 (to order from: Pabst Science Publishers, http://www.pabst.publishers.de)
« Drug addiction », Ethical Eye Series, Council of Europe Publishing, [ISBN 92-871-5639-5], July 2005 (to order from the Council of Europe Publishing: http://book.coe.int)
The ESPAD Report 2003: Alcohol and Other Drug Use Among Students in 35 European Countries, November 2004 – Joint publication Pompidou Group / CAN [ISBN 91-7278-103-3], to order from CAN - The Swedish Council for Information on Alcohol and other Drugs, fax : +46 8 10 46 41 or e-mail : [email protected]
Connecting research, policy and practice – Lessons learned, challenges ahead – Proceedings, Strategic conference, Strasbourg, 6 – 7 April 2004 [ISBN 92-871-5535-6]
Drugs and drug dependence: linking research, policy and practice – Lessons learned, challenges ahead – Background paper by Richard HARTNOLL, Strategic conference, Strasbourg, 6-7 April 2004 [ISBN 92-871-5490-2]
Road traffic and psychoactive substances – Proceedings, Seminar, Strasbourg, 18-20 June 2003 [ISBN 92-871-5503-8] July 2004
Risk reduction linked to substances other than by injection – Proceedings, Seminar, Strasbourg, February 2002 [ISBN 92-871-5329-9]
Prisons, Drugs and Society – Proceedings, Seminar, Bern (Switzerland), September 2001 [ISBN 92-871-5090-7]
Development and improvement of substitution programmes – Proceedings, Seminar, Strasbourg, October 2001 [ISBN 92-871-4807-4]
Calculating of the social cost of illicit drugs: Methods and tools for estimating the social cost of the use of psychotropic substances by Pierre KOPP [ISBN 92-871-4734-5] November 2001 (available in Russian, December 2003)
Contribution to the sensible use of benzodiazepines – Proceedings, Seminar, Strasbourg, January 2001 [ISBN 92-871- 4751-5]
3rd Multi-city study: drug use trends in European cities in the 1990s by Ruud BLESS, co-ordinator [ISBN 92-871-4459-1] December 2000
Drug use in prison - Project of the Group of experts in epidemiology of drug problems: final report by Richard MUSCAT, co-ordinator [ISBN 92-871-4521-0] December 2000
Development and testing of an exit from treatment form for clients in drug abuse treatment - Project of the Group of experts in epidemiology of drug problems: final report by Anna KOKKEVI, co-ordinator [ISBN 92-871- 4523-7] December 2000
Pregnancy and drug misuse: Up-date 2000 – Proceedings, Seminar, Strasbourg, May 2000 [ISBN 92-871-4503-2]
Vocational rehabilitation for drug users in Europe – Proceedings, Seminar, Bratislava, January 2000 [ISBN 92-871- 4406-0]
Attention deficit / hyperkinetic disorders: diagnosis and treatment with stimulants – Proceedings, Seminar, Strasbourg, December 1999 [ISBN 92-871-4240-8]
Drug-misusing offenders in prison and after release – Proceedings, Seminar, Strasbourg, October 1999 [ISBN 92-871- 4242-4]
Road traffic and drugs – Proceedings, Seminar, Strasbourg, April 1999 [ISBN 92-871-4145-2]
Treated drug users in 23 European cities - Data 1997 - Trends 1996-97 - Pompidou Group Project on Treatment Demand : Final report by Michael STAUFFACHER, co-ordinator [ISBN 92-871-4007-3] November 1999
Drug-misusing offenders and the criminal justice system: The period from the first contact with the police to and including sentencing – Proceedings, Seminar, Strasbourg, October 1998 [ISBN 91-871-3790-0]
Multi-city Network Eastern Europe - Joint Pompidou Group / UNDCP project: extension of the multi-city network to Central and Eastern Europe. First city reports from: Bratislava, Budapest, Gdansk, Ljubljana, Prague, Sofia, Szeged, Varna, Warsaw [ISBN 92-871-3509-6] 1997
Pregnancy and drug misuse – Proceedings, Symposium, Strasbourg, March 1997 [ISBN 92-871-3784-6]
Special needs of children of drug misusers - Consultant’s final report by Beate LEOPOLD & Elfriede STEFFAN [ISBN 92-871-3489-8] 1997
Volatile substance abuse among young people in Poland - Consultant’s final report by Richard IVES [ISBN 92-871-3184-8] 1996
Outreach work with drug users : principles et practice - Consultant’s final report by Tim RHODES [ISBN 92-871-3110-4] 1996
Women and drugs / Focus on prevention – Proceedings, Symposium, Bonn, October 1995 [ISBN 92-871-3508-8]
Multi-city study : Drug misuse trends in thirteen European cities [ISBN 92-871-2392-6] 1994
Volatile substance abuse – Proceedings, Seminar, Bratislava, November 1993 [ISBN 92-871-2570-8]
Women and drugs – Proceedings, Symposium, Prague, November 1993 [ISBN 92-871-2838-3]
Outreach – Proceedings, Symposium, Bergen, February 1993 [ISBN 92-871-2601-1]
21 Documents
Prevention Interventions in Recreational Settings by Amador CALAFAT [P-PG/Prev(2010)7] October 2010
Drug testing at school and in the workplace, by the Experts committee on ethical issues and professional standards [P-PG/Ethics(2008)5] September 2008
« Le rôle de la recherche scientifique dans l’élaboration des politiques de drogue », Séminaire international en coopération avec l’Office National de Lutte contre la Drogue et la Toxicomanie d’Algérie, Alger 3-6 décembre 2006, Actes, mai 2008 (only french / arabic)
Enquêtes scolaires sur l’alcool, le tabac et les autres drogues dans la région méditerranée, L’enquête MedSPAD à Alger : de la conception aux résultats, Soumaya Mansouri-Zeghnoun, [P-PG/RES-Med (2007) 1], juin 2007 (french only)
Ministerial Conference “New signals for drug policies across Europe”, Strasbourg, 27-28 November 2006 [P-PG/MinConf (2007)1], Proceedings, June 2007.
Final report – Drug prevention Support Network for Parents and Profesionnals with support of the Nordic Council of Ministers and the Swiss Government [P-PG/Prev-SN (2003) 8], Strasbourg, December 2006.
Young People and Drugs – Care and Treatment, by Janusz SIEROSLAWSKI et al. [P-PG/TT(2006)5], November 2006
“Why shouldn’t we take drugs?” – 2nd European Forum on Drug Prevention, Proceedings, Vilnius, Lithuania, 18-19 May 2006 [P-PG/Prev-CF(2006)6], November 2006
Drugs and alcohol: violence and insecurity? Guide – Integrated project 2 – Responses to violence in everyday life in a democratic society, by Dirk J. KORF et al. [P-PG/CJ (2004)7] September 2005
Polydrug use: mixing different drugs – new trends in youth culture – 1st European Drug Prevention Forum, Proceedings, Ekaterinburg, Russian Federation, 6-7 October 2004 [P-PG/Prev-CF (2005)1]
Validity and reliability of school surveys based on the European ESPAD methodology in Algeria, Libya and Morocco (MedSPAD pilot School survey project), by Ruud BLESS and Richard MUSCAT, [P-PG/Res-Med(2004)2], 2004
Follow-up project on treatment demand: tracking long-term trends – Final Report by Michael STAUFFACHER et al. [P-PG/Epid (2003)37]
Ethics and drug use – Seminar on Ethics, professional standards and drug addiction, Strasbourg, 6-7 February 2003 [P-PG/Ethics (2003) 4] November 2003
International Drug court developments: models and effectiveness, by Prof. Paul MOYLE [P-PG/DrugCourts (2003) 3] September 2003
Outreach work with young people, young drug users and young people at risk – Emphasis on secondary prevention by Njål Petter SVENSSON [P-PG/Prev (2003) 6] September 2003
The general Potential of Police Prevention in the area of illicit drugs, by Prof. Dr. Lorenz BÖLLINGER [P-PG/Prev (2003) 2] September 2003
The Impact of the ESPAD Project – The opinion of ESPAD Researchers by Björn HIBELL and Barbro ANDERSSON [P-PG/Epid (2003)31]
Mediterranean Network: Mediterranean School Survey Project on Alcohol and other Drugs (MedSPAD) – Pilot Survey 1 – Morocco by Richard MUSCAT [P-PG/Med(2003)12]
Estimating the social cost of illicit drugs in Poland [P-PG/Cost (2003) 2]
Targeted Drug Prevention – How to reach young people in the community ? Report of the Conference in Helsinki, November 2002
Pompidou Group Multi-citystudy update report 1999-2000 by Ruud BLESS, coordinator, May 2002 [P-PG/Epid (2002)11]
Prisons, Drugs and Society: A Consensus Statement on Principles, Policies and Practices - Published by WHO (Regional Office for Europe) in partnership with the Pompidou Group, September 2002
Benzodiazepine Use: A report of a survey of benzodiazepine consumption in the member countries of the Pompidou Group by Gary STILLWELL and Jane FOUNTAIN [P-PG/Benzo (2002) 1] February 2002
Missing Pieces: Developing drug information systems in Central and Eastern Europe / Technical reports by Michael STAUFFACHER, co-ordinator (joint PG / UNDCP Project: extension of the Multi-city network to Central and Eastern European Europe) September 2001
Vocational rehabilitation of drug users and drug dependent persons (EUREHA Project) - Report on the state of the art and on the results of a survey in all member states of the Pompidou Group - by Ambros UCHTENHAGEN, Susanne SCHAAF & Christa BERGER (Addiction Research Institute at Zurich University) [P-PG/ Rehab (2000)1]
Joint Pompidou Group - EMCDDA Scientific Report 2000 - Treatment demand indicator : Standard protocol 2.0 and Technical annex - 2 parts available for download at : http://www.emcdda.org
Problem drug use by women – Focus on community-based interventions - by Dagmar HEDRICH [P-PG/Treatment (2000)3]
European Handbook on Prevention: Alcohol, Drugs and Tobacco (1998)
Working group on “Minorities and drug misuse” - Consultant’s final report by G.F.van de WIJNGAART and F. LEENDERS [P-PG/Minorities(98)1] 1998
Postal address :
[1] The term “abstinence orientation” as understood here, particularly in relationship to alcohol and nicotine, does not mean that supporters are in favour of a legal prohibition – even though abstinence orientation includes this position as a radical option – but that they are in favour a society where these psychoactive substances are consumed as little as possible or not consumed at all.
[2] In the Northern European and Anglo-Saxon alcohol policy discussion, the term “population approach” refers to the conviction, that only measures which reduce the alcohol consumption in all alcohol users – of those drinking moderately as well as those drinking heavily – can successfully reduce the number of heavy alcohol users.
[3] “Problem reduction” as understood here includes all measures expected to reduce problems related to substance use – including “harm reduction measures”.
[4] As Luhmann (2000) put it: “Communication is unlikely, even though we experience and practise it daily and could not live without it. … First of all it is unlikely that anyone understands what the other one says, given the separation and individualisation of their consciousness. Meaning can only be understood in relationship to context, and to start with whatever memory provides serves as context to everyone”.
[5] In modern SAP this affinity is commonly related to vulnerability in a non-pathological context and to psychiatric and/or psychosocial comorbidity in a pathological context.
[6] As aforementioned both terms “substance abuse prevention” and “addiction prevention” are suboptimal because they do not cover the whole range of activities and because the constituting expressions “abuse” and “addiction” were considered obsolete by important institutions like the World Health Organisation (WHO) or the Center for Substance Abuse Prevention in the USA (CSAP). The “Lexicon of alcohol and drug terms” (WHO, 2009) suggested to substitute “substance abuse” with “harmful substance use” and to substitute “addiction” with “substance dependence”.
[7]According to this concept, prevention is the activity to reduce the probability for manifestation of a specific index problem relative to the counterfactual situation where no preventive activities takes place.
[8] For example, one goal of substitution treatment is to prevent opiate users from continuing intravenous use and thus from contracting HIV or other blood borne diseases.
[9] Prevention as understood by local prevention centres includes activities to avoid or reduce substance abuse, substance addiction and behavioural addictions.
[10] Fore example, to claim that prevention in a certain country is emancipatory oriented and opposes paternalistic tendencies.
[11] Therapy can be divided into „curative (Treatment to support healing) Therapy“ and „palliative Therapy (Treatment given to relieve symptoms and reduce suffering)“.
[12] Alternative terms are “Harm Minimization” and „Support to Survive“.
[13] “General prevention” aims at influencing the general public against committing crimes. More specifically “negative general prevention” aims at deterring the population through confronting them with others being punished and “positive general prevention” aims at stabilizing legal norms in the public and trust in the legal system.
[14] “Special prevention” aims at achieving that single tortfeasors will in future avoid the sanctions imposed on them. “Negative special prevention” aims at deterring the tortfeasor and “positive special prevention” aims at re-socialising the tortfeasor.
[15] Application fidelity” has to do with how accurately or faithfully a technique, intervention or programme is reproduced from a manual, protocol or theory.
[16] Commonly realistic and useful precise goals are being developed in the evaluation process. An evaluator drafting the guidelines stated that setting precise goals is commonly neither feasible nor sensible in the evaluation planning phase – the former since the necessary knowledge to decide on adequate target criteria emerges in the course of the evaluation and the latter since precise pre-existing goals seduce project staff into manipulating these criteria rather than focusing on the true project purposes. E.g. if a project staff in prevention centres realizes that “number of persons contacted annually” is perceived as indicator for activity in the evaluation design, they may initiate a few lectures to all students and staff in large schools and drop some activities with few participants they manage to multiply this indicator without any increasing the centre’s actual performance.
[17] The popular demand to always plan evaluations before or parallel to the implementation of the evaluation object was discussed and unanimously dismissed by the commission, since evaluations commonly start after evaluands have been already implemented or are even used on a routine basis for some time. This lead to the omission of contradicting formulations but the idea was not explicitly mentioned in the draft of the guidelines.
[18] In spite of their natural selfishness ... they are led by an invisible hand to make nearly the same distribution of the necessaries of life, which would have been made, had the earth been divided into equal portions among all its inhabitants, and ... advance the interest of the society (Smith "Theory of Moral Sentiments")
[19] Adam Smith's “invisible hand” - the idea that free markets lead to efficiency as if guided by unseen forces – is invisible, at least in part, because it is not there (Stiglitz, 2002)
[20] Professor Archie Cochrane was a Scottish epidemiologist and author of the book “Effectiveness and Efficiency: Random Reflections on Health Services” (1972). His name was chosen by the famous Cochrane Collaboration and he is one of the fathers of what is today labelled “evidence based medicine”
[21] In writing this section in praise of the RCT I do not want to give the impression that it is the only technique of any value in medical research. This would, of course, be entirely untrue. I believe, however, that the problem of evaluation is the first priority of the National Health System and that for this purpose the RCT is much the most satisfactory in spite of its snags. The main job of medical administrators is to make choices between alternatives. To enable them to make the correct choices they must have accurate comparable data about the benefit and cost of the alternatives. These can really only be obtained by an adequately costed RCT.
[22] The similar argument is provided by Rothman and Green (2005): “Optimism without specific criteria: Although there are no absolute criteria for assessing the validity of scientific evidence, it is still possible to assess the validity of a study. What is required is much more than the application of a list of criteria. Instead, one must apply thorough criticism, with the goal of obtaining a quantified evaluation of the total error that afflicts the study. This type of assessment is not one that can be done easily by someone who lacks the skills and training of a scientist familiar with the subject matter and the scientific methods that were employed. Neither can it be applied readily by judges in court, nor by scientists who either lack the requisite knowledge or who do not take the time to penetrate the work.” Hartnoll (2004) suggests a similar procedure explicitly deviating from a “modern” epistemological perspective: “The implicit understanding of research is a process where relevant questions evolve, where existing evidence is put together as in a puzzle, where missing pieces are temporarily added based on common-sense and logic and eventually clarified through further research. A researcher, according to this conception, is like a detective who systematically collects and assembles evidence until the case is solved.”
[23] The empirical literature supporting the hypothesis that highly problematic persons are much more likely to either totally abstain from substance use or on the other hand misuse substances can be found in the scientific literature under the terms “u-shaped curve” or “j-shaped curve”.
[24] The nature of this unawareness was nicely expressed in the Film “Everything Is Illuminated” directed by Liev Schreiber (2005). The main character Jonathan asks his guide Alex : “What do you mean?” and the guide answers “Exactly what I said! If I had wanted to express something different I would have said something different!”.
[25] “Lifetime prevalence of Cannabis use” circumscribes the percentage of persons who in their lifetime have used Cannabis at least once.
[26] If nobody had started to use Cannabis in this 4 year period a minimal reduction in lifetime prevalence could have happened, but the results showed that this was not the case. Considering that mortality is highest in those individuals who belong to “Pre-Cannabis-generations”, we can definitely rule out that Cannabis lifetime prevalence could have dropped measurably in this 4 year period.
[27] Cronbach et al.(1972) in their famous book on the “Theory of Generalizibility” claimed that the ideal datum on which to base decisions on is “something like the person's mean score over all acceptable observations”, which they called “universe score."
[28] To give an illustrative example. If a group of individuals suffering from migraine have headache on average three times a month for two days, their probability to have a headache on any randomly chosen day is 20% (6 out of 30 days). If we test them twice in a row, the expected number of subjects with headaches at both assessment days (t1 and t2) is 20%. In other words we will correctly not observe a change from t1 to t2. If we chose only those individuals having a headache at day t1 and decide to follow them up, we have an average score of 100% persons with headache at t1 and only 20% headaches at t2 if the status of the persons remains actually unchanged. Naïve persons will interpret this as an 80% reduction in migraine. In case successful treatment had reduced the actual migraine incidence by half, we could expect a rate of 10% at t2 and thus observe a 90% reduction in headaches. In other words, a very substantial true change halving the incidence would alters the observed change from 80% to 90%.
[29] Let it be your ambition to refute and replace your own theories: this is better than defending them, and leaving it to others to refute them. But remember also that a good defence of a theory against criticism is a necessary part of any fruitful discussion, since only by defending it can we find out its strength, and the strength of the criticism directed against it (Popper, 1972).
[30] Life Education, a charity in the UK with international links, has 43 local operating groups in the UK reaching around a million children and aiming to help children make healthy choices through programmes for children and parents in partnership with primary schools and in the community.
[31] Faggiano F, Vigna-Taglianti F, Versino E, Zambon A, Borraccino A, Lemma P. School-based prevention for illicit drugs' use. Cochrane Database of Systematic Reviews 2005, Issue 2. Art. No.: CD003020. DOI: 10.1002/14651858.CD003020.pub2.
[32] Gates S, McCambridge J, Smith LA, Foxcroft D. 2006 ‘Interventions for prevention of drug use by young people delivered in non-school settings’ Cochrane Database of Systematic Reviews Issue 1. Art. No.: CD005030. DOI: 10.1002/14651858.CD005030.pub2
[33] Burkhart G and others 2008 ‘The effectiveness of a school-based substance abuse prevention program: EU-Dap cluster randomised controlled trial’ Preventive Medicine 47, 5, 537-543
[34] The EU-Dap Study Group (nd) Unplugged: An effective school-based program for the prevention of substance use among adolescentsEudap (Eudap Final Technical Report n.1)
[35] the normative fallacy is the idea that ‘everyone is doing it’ – a common belief among young people in relation to, for example, sex and drugs. Some research has shown that correcting such erroneous beliefs can be protective.
[36] www.findings.org.uk (accessed 15-12-09). See also SteadM et al 2007 ‘Implementation evaluation of the Blueprint multi-component drug prevention programme: fidelity of school component delivery’ Drug and Alcohol Review, 26, 6, pp 653 – 64; and Baker P 2006 ‘Developing a Blueprint for evidence-based drug prevention in England’ Drugs: Education, Prevention, and Policy 13, 1, pp 17-32
[37] for a more detailed description, see http://preventionaction.org/print/970 (accessed 16-12-09)
[38] Ghosh-Dastidar B. et al. 2004 ‘Modifying pro-drug risk factors in adolescents: results from Project ALERT’ Health Education & Behavior:, 31,3, pp 318-34
[39] Hawkins J. et al. 2009 ‘Results of a type 2 translational research trial to prevent adolescent drug use and delinquency: a test of Communities That Care.’ Archives of Pediatric and Adolescent Medicine, 163, 9, pp 789–98.
[40] Quality youth work diverts progression of high-risk youngsters’ ‘Analyses of 48 US government-funded after-school and youth work projects for 9- to18-year-olds at high risk of drug problems’ http://findings.org.uk/count/downloads/download.php?file=nug_13_9.pdf (accessed 25-09-10)
[41] ‘In drug-related youth work de-focus from drugs’ Findings 7.10
(http://www.findings.org.uk/count/downloads/download.php?file=nug_7_10.pdf) (accessed 27-09-10)
[42] Global Youth Network 2003 Performance: using performance for substance abuse prevention UNODC (www.unodc.org/pdf/youthnet/handbook_performance_english.pdf)
[43] Kellam S et al2008 ‘Effects of a universal classroom behavior management program in first and second grades on young adult behavioral, psychiatric, and social outcomes’ Drug and Alcohol Dependence, 95 (suppl. 1), pp. S5–S28‘
[44] Pol A et al 2009 ‘Impact of a preventive intervention targeting childhood disruptive behavior problems on tobacco and alcohol initiation from age 10 to 13 years’ Drug and Alcohol Dependence
100, 3, 1 pp 228-233
[45] for example, Henry K et al 2009 ‘Individual and contextual effects of school adjustment on adolescent alcohol use’ Prevention Science 10, 3, pp 236-47: ‘Students who attended schools where the overall level of school adjustment was higher reported lower levels of alcohol use even after taking their own school adjustment into account. The results demonstrate the importance of both a student's own level of school adjustment and the normative level of school adjustment among students in the school on an adolescent's use of alcohol.’
[46] ‘The underlying causes of difficult behaviour or persistent absence are often emotional or social, and focusing on these – rather than on behavioural outcomes – enables staff to respond more effectively. They can then take action to understand and prevent difficult behaviour, as well as using rewards and sanctions. Social and emotional skills enable the learner to make informed choices about their behaviour. They enhance the learner's self-awareness and self-understanding, developing empathy which allows them to predict the outcomes of their behaviour on others, manage their feelings more effectively and develop a range of responses.’ UK Department of Children, Schools and Families http://nationalstrategies.standards.dcsf.gov.uk/node/66372?uc=force_uj (accessed 17-12-09)
[49] Unlike treatment activities, where there is more consensus, and much intervention is one-to-one. The difference is similar to that between a new and emerging industry and a mature one.
[50] http://www.cph.org.uk/drugprevention/ (accessed 28-09-10)