The Wartegg Info Blog

On the eve before the Finn & Crisi seminar on the 11th of April we invite you to a Crash Course in the Crisi Wartegg System with Alessandro Crisi and Malin Holm

Time: Thu 10th of April 5-7.30 PM, Place: SAHRA Kungsholmstorg 10, Price: 200 SKR including coffee/tea and a snack

  Wartegg Drawing Completion Test (WCFT) is a simple performance based personality test where the client makes a series of drawings in the eight boxes, each containing a small figure. Warteggs careful selection of figures that are simple but have a strong and universal “pull”, consists the fundamental potential of the test. Like all performance-based personality tests underlying dynamics of personality as well as patterns of perception and motivation can be detected based on that person's behavior in response to the relatively unstructured task, through rigorous scoring and analysis according to strict criteria.

  The WCFT has been virtually unknown in the US, but is now gaining ground through the Crisi Wartegg System and  its usefulness in the Therapeutic Assessment. The simplicity of the test makes it quick to administer, score and interpret. With Alessandro Crisis new and structured scoring system it has proved reliable and valid in several studies conducted on clinical populations in Italy and within the Italian Navy and Army. It is much less provocative than the Rorschach which proves useful with highly defensive subjects, and is often more suited to organizational settings. Its rigorous scoring, much like that of the Rorschach, gives information about both personality structure and state, including indices such as suicide index. The actual drawings are a very good starting point for an extended enquiry to further examine the client’s existing story.   

Here is information in Swedish:

----------------------

Work Shop i Crisi Wartegg System (CWS)

En ny metod för tillämpningen av Wartegg Drawing Completion Test

  Med sammanlagt sex heldagar, med genomgång av kodning, bedömning och träning, får deltagarna lära sig Crisi Wartegg System av psykolog och universitetslärare Alessandro Crisi och leg psykolog Malin Holm.

Utbildningen passar dig som vill... lära dig en strukturerad, beforskad och enkel performancebaserad testmetod för bedömning av personlighet, grundad i psykodynamisk personlighetsteori

Utbildningen är uppbyggd på 6 dagar indelade i två moduler.

Modul 1: kodning – Malin Holm administrering, kodning och praktiska kodningsövningar i samt uträkningar (Structural Summary)

Modul 2: bedömning – Alessandro Crisi bedömning/diagnostik utifrån Structural Summary, analys av ordningsföljd, utlåtande utifrån kodningsresultat och praktiska övningar och diskussioner utifrån kliniska fall.

Mellan modulerna finns möjlighet för kursdeltagarna att ta upp och koda egna protokoll för att samla erfarenhet och frågor till modul två, där egna fall sedan kan diskuteras

Datum Modul 1: 28-30 augusti

Modul 2: 9-11 oktober

Plats: Sandahl Partners, Stockholm modul 1; modul 2 meddelas senare; centralt i Stockholm

Pris: 13.000:- exklusive moms, 12.500:- för medlemmar i Svenska Rorschachföreningen

I deltagandet ingår: Kursmaterial, förmiddagskaffe, lunch och eftermiddagskaffe

Fortsättning:

Möjlighet finns att fortsatt delta i kodningsseminarier både fysiskt och via Skype för att hålla sig uppdaterad i metoden

  Läs mer: på www.wartegg.sahra.se och www.wartegg.com

  Anmäl dig: på telefon 070 - 321 30 90 eller via e-post till [email protected]

    SAHRA erbjuder sedan 2009 tjänster inom psykologisk personbedömning och andra psykologtjänster inom urval, second opinion, och företagsutveckling genom personlig utveckling. Vi gör skillnad i urvalsbedömningar där en fördjupad förståelse av individens potential och drivkrafter, medvetna och omedvetna, är viktigt

Jarna Soilevuo Grønnerød

In one of the few, recent publications concerning the Wartegg, which are written in English and which are easily available, Carl-Erik Mattlar (2005) refers to the study by Maija Vehviläinen as follows:

”Vehviläinen (1988) has, for instance, investigated the validity of the selection process employed when selecting students for a professional nursing college. She compared the recommendation based on an integrated multi-method psychological assessment (comprising cognitive tests, a group situation, personality assessment methods [WZT, SCT, group Rorschach, autobiography], and an interview) to the high school graduation certificate. According to the results, the correlation between the integrated psychological assessment, and the grades when graduating from professional nursing college was r = 0.42, while the starting points (based on the high school grades and work practice in order to get practical experience) correlated r = -0.12. Vehviläinen (1998) has crossvalidated this study 10 years later with similar results. Mattlar (1999b) has written a review paper on 59 studies which, taken together, highlight with very satisfactory results the validity of a number of integrated personnel and/or student selection recommendations, based on personality assessment methods, primarily founded on the Wartegg and the Rorschach.” (Mattlar 2005, 7.)

Mattlar refers to his own unpublished manuscript (1999 in this text, 1999b in the citation above) where he calls Vehviläinen's work as an important study on the validity of the Wartegg.

Since the study by Vehviläinen (1998) is only available in Finnish while the text by Mattlar (2005) is easily available for a much larger audience, I would like to provide a more detailed review of Vehviläinen's study in English. Vehviläinen (1998) is a follow-up study of Vehviläinen (1988). Unfortunately. I was unable to retrieve the latter, a Master's thesis, despite several efforts. Thus. my review is based on the follow-up study, a Licentiate's thesis.

The purpose of Vehviläinen's (1998) study is to evaluate the process of selecting students to a nursing college. In total, 358 persons applied and 45 were accepted to the study program in 1993-94. The college employed a psychological recruitment agency to carry out the process of evaluating the applicants according to guidelines agreed on with the leaders of the college. Several personality factors as well as cognitive and communication skills were evaluated by the use of psychological personality evaluation methods, interview and an observed group work situation. The main purpose of the evaluation process was to select out applicants who the college leaders defined as unsuitable (persons with acute mental health disorders, addictions, deficits in social integration and lack of study motivation). This evaluation, including testing the applicants, was done by the psychologists from the recruitment agency. Based on all this information, each candidate received a final evaluation score, which was used as a basis for selection. These final scores were used by Vehviläinen as study data. The test results, the interviews or the observation of the group situation. are thus not Vehviläinen's study material.

In addition to the final scores, Vehviläinen's data consists of questionnaires filled in by the selected 45 students at the end of their studies. The students were asked about their views and experiences on the selection process and their studies. Another pool of data Vehviläinen used was the whole record of the students academic achievements and study characters.

The Wartegg was one of the personality evaluation methods that the recruitment agency used. No details of the Wartegg results are presented, neither does Vehviläinen report which part of the overall psychological aptitude of each candidate stems from the Wartegg results. However, Vehviläinen reports as a result that 77,8% of the selected students responded that in their view, the personality evaluation methods used in the selection process were ”very useful” (p. 172). In a nutshell, Vehviläinen's main result is that the 45 students who succeeded in passing the exhaustive selection process and completing their studies, reported in the questionnaire that they were pleased with the selection process.

Vehviläinen's report includes a comprehensive review of earlier Finnish studies, dissertations and Master's theses on selection processes employing psychological personality evaluation methods, the WZT included, in different types of occupations and educational institutions. Even though the review is merely a listing of earlier results without an analysis, it is valuable as a source of literature.

Vehviläinen's study provides important information and reflection of different aspects of the process of selecting students to the educational institution which she herself is the leader of. There is no doubt of the administrative value of the study. As for the Wartegg, the general problem with Vehviläinen's study design is that the measures used in selecting the applicants were also used to predict the outcome. This deficiency in the design and the fact that no specific Wartegg scores are analysed, do not make the study useful in evaluating the validity of the Wartegg as a selection instrument.

References:

Mattlar, C-E. (1999) I Finland på finska publicerad/utförd forskning rörande Wartegg Zeichen Test (WZT). Unpublished manuscript.

Mattlar, C-E (2005) The Utility of the Rorschach Comprehensive System: An Increasing body of supportive research. South African Rorschach Journal 2(1), 3-31.

Vehviläinen, M. (1988) Yhteisvalinta ja sairaanhoitajakoulutuksessa menestyminen. Sairaanhoidon vuosikirja XXIV. Sairaanhoitajien koulutussäätiö, Helsinki.

This is a translation of the text in Finnish posted earlier to this blog. The same translation is also available at http://wartegg.sahra.se/page6.html

-----------------------------

INCREASE IN WARTEGG EDUCATION AND NETWORKING

    Since 2011, 28 psychologists in Sweden have been educated in the use of the CWS (Crisi Wartegg system), the new Wartegg scoring system created by Alessandro Crisi from Italy. Two 7-day courses have been organized by Alessandro Crisi and Malin Holm, the founder of the recruiting and assessment company SAHRA. Holm says that the CWS has met an exciting interest in Sweden among exerienced Wartegg-users trained by Manfred Gardziella and Carl-Erik Mattlar, as well as younger psychologists. ”The CWS appears to fill a void of a performance based personality assessment instrument with a structured scoring method, which is cost-effective and possible to use on a larger scale,” Holm says. The course provides the participants with detailed guidelines to scoring and interpretation, and to the use of the scoring software developed by Crisi.

    Crisi Wartegg System is spreading

    «I was dissatisfied with the old scoring systems and wanted to develop a more simple, user-friendly and reliable system,» Crisi explains. He has been working on the CWS at the Italian Wartegg Institute in Rome since early 1980's.  Unfortunately, most of his work is available only in Italian, and very little of this work has been known outside Italy until recently. «I started co-operation with Stephen Finn at The Center for Therapeutic Assessment in Texas in 2010» says Crisi. «The CWS is at the moment regularly used at the Centre and the CWS is one of the methods qualifying for certification in therapeutic assessment.” A handbook in English is completed and will be published in 2013. The personnel at the center, including Stephen Finn, who completed the advanced training, have been so excited about the Wartegg that they are now assisting Dr Crisi in collecting US norm data for the WZT, to date 750 protocols.

  Crisi tells that in South America, where they have a vital Wartegg tradition, translations of the CWS into Spanish and Portugese are completed. In spring 2012 Crisi held a two-day workshop on CWS in Buenos Aires, and in Brazil the rights to distribute the CWS are currently being negotiated.

  Different from the Gardziella tradition

  Jarna Soilevuo Grønnerød, a doctor in psychology from the University of Joensuu, but emigrated to Norway, recently participated at the course in Stockholm. She says that the Crisi system differs from the Finnish Gardziella tradition of interpretation and administration. For example, the test is presented to the client only once. The CWS is strongly influenced by the Rorschach method, and the clients' verbalizations play a major role to the aim of the scoring. ”At the first sight, this may look like the non-verbal element of the method is not fully employed. However, the risk in the Gardziella tradition is that psychologists may lean on their own projections of what their clients have drawn,” Soilevuo Grønnerød says.

  «In my scoring system psychologists must score what their clients see in the signs and in their drawings,» Crisi explains. «In this way, the scoring system becomes sensitive to detecting thinking disorders. Psychologists should evaluate their clients, not themselves,» he concludes.

    Integrating the systems of interpretation

  Kristofer Wikstad, who works in assessment, counseling and psychotherapy and uses the Wartegg weekly, found professor Crisi's combination of a clinic knowledge based on many years of experience with the well-structured method of interpretation of the Wartegg test especially inspiring. Wikstad has been using the Wartegg for years but was still surprised by the richness of the possibilities for structured interpretations presented by Crisi. Wikstad thinks the method has the potential to become an important tool in educational contexts, social work and psychiatry. “But, I do think it will take time, not because of the test, but because tests are linked to the questions we are asking. What we want to know about our client and what information is useful for the development of the client, for example. I do think that for the moment it would be easiest to arouse school psychologists’ interest in the Wartegg test. That´s because its usefulness would be much more apparent to them then for others,” says Wikstad.

  Introducing the CWS in Sweden with a strong Gardziella-Mattlar tradition poses a special challenge. “Some colleagues have told me that they integrate the systems, others say that they changed the system of interpretation, “ Wikstad explains. “There are very useful interpretations in the Gardziella-Mattlar system that are not included in the Crisi system, for example giving Wartegg two with a limited time. I do hope that there will be more research done so we will have one system where the useful parts with a high validity will be combined. All of us would benefit from that in our work.”

    Mythbusting

  «You can still hear academic psychologists claiming that there is no empirical research on the Wartegg and that the method is not valid,» says Soilevuo Grønnerød. She and her colleague Cato Grønnerød, Associate professor of personality psychology at the University of Oslo, published recently a meta-analytic study on the reliability and validity of the WZT in Psychological Assessment, a leading journal in the field. «There has been research activities on the method ever since the publication of Ehrig Wartegg's first books, but they have been isolated in small, local traditions with little contact between them,» Soilevuo Grønnerød describes. «There has never been formed a global Wartegg community which would have been able to produce cumulative knowledge on the method.»

  The meta-analysis shows that the inter-rater reliability of the method is high. «As for validity, our work shows that it is premature to demand the method should be buried. At the same time, there is no reason to be satisfied with the amount and quality of existing research either,» Soilevuo Grønnerød sums up. As Taina Kuuskorpi's recent dissertation shows, The Wartegg is the 3rd most used method among psychologists in Finland. «There is a confusing contradiction between the psychologists' interest in the method and the lack of interest by the institutions which educate these psychologists,» says Soilevuo Grønnerød.

  Networking is easy

  The networking of academic and practicing psychologists interested in the development of the method is made easy these days. Alessando Crisi has his website at www.wartegg.com which focuses on the CWS while Soilevuo Grønnerød has established a research based web site www.wartegg.info as a main site where visitors can find links to a blog, to several social media groups and to an online bibliography, which aims at encouraging and facilitating future research by providing one place to find references to scholarly Wartegg literature. Currently the number of references is over 530.

This blog uses English in order to reach a widest possible audience. However, the text below is written in Finnish. Its target audience is practicing psychologists in Finland, where the WZT is the 3rd most used test method. The text was submitted for publication in the psychologists' trade union magazine called Psykologi. During the last 10 years, Psykologi has published several texts on the Wartegg, written by academic psychologists who have dismissed the method as not valid and practicing psychologists who have defended the method because they find it useful in their daily work.

Psykologi refused to publish this text because it is "too much like an advertisement", because "neither the author nor the persons interviewed are members of our union" and because it contains information on results of a scientific study, thus better suited to a scientific journal. The editor of Psykologi did not express any wish to consider publication once again had the text been modified to be less like an advertisement.

The text contains information which corrects claims presented earlier in Psykologi. It also informs Finnish psychologists on the newest developments concerning education and systems of interpretation.

Warteggin koulutustarjonta laajenee ja verkostoituminen tehostuu

Italialaisen Alessandro Crisin kehittämää uutta Wartegg-pisteytysjärjestelmää Crisi Wartegg System (CWS) on opetettu ruotsalaisille psykologeille vuodesta 2011 saakka. Crisi sekä rekrytointi- ja henkilöstöarviointiyritys SAHRAn perustaja Malin Holm ovat järjestäneet kaksi seitsemänpäiväistä kurssia Tukholmassa. Holmin mukaan CWS on saanut innostuneen vastaanoton sekä nuoremman psykologikunnan että kokeneempien, Manfred Garzdiellan ja Carl-Erik Mattlarin kouluttamien Wartegg-käyttäjien keskuudessa. ”CWS täyttää menetelmällisen aukon tarjoamalla itsearviointimenetelmien rinnalle piirrosilmaisuun perustuvan metodin ja strukturoidun pisteytysjärjestelmän, joka on kustannustehokas ja soveltuu käytettäväksi suurienkin ryhmien testaamiseen, ” Holm toteaa. Kurssi perehdyttää pisteytysjärjestelmän käyttöön ja tulkintaan sekä opastaa tulkinnan nopeuttamiseksi suunnitellun tietokoneohjelman käyttöön.

CWS-tulkintajärjestelmä leviää

«Olin tyytymätön aiempiin pisteytysjärjestelmiin ja halusin kehittää yksinkertaisemman, käyttäjäystävällisemmän ja luotettavamman menetelmän,» Crisi kertoo. Hän on työskennellyt CWS:n parissa Roomassa sijaitsevassa Wartegg-instituutissa 1980-luvun alkuvuosista lähtien. Suurin osa kehitystyön tuloksista on julkaistu vain italiaksi ja on siksi jäänyt lähes tuntemattomaksi Italian ulkopuolella aivan viime vuosiin saakka.  «Aloitin yhteistyön teksasilaisenThe Center for Therapeutic Assessment -keskuksen Stephen Finnin kanssa muutama vuosi sitten,» sanoo Crisi.  «Tällä hetkellä Warteggia ja CWS-tulkintajärjestelmää käytetään keskuksessa säännöllisesti ja se on otettu yhdeksi niistä menetelmistä, joiden osaaminen on edellytys keskuksen sertifikaatin saamiseksi,» Crisi jatkaa. Englanninkielinen CWS-käsikirja on valmistunut ja ilmestyy tänä vuonna.  Keskuksen henkilökunta ja Crisi ovat yhdessä keränneet yhdysvaltalaista normiaineistoa, jota on jo koossa huomattava määrä.

Crisin mukaan elinvoimaisen Wartegg-tradition Etelä-Amerikassa tullaan julkaisemaan CWS-käännökset sekä portugaliksi että espanjaksi. «Buenos Airesissa pidettiin kahden päivän työpaja toukokuussa 2012 ja se herätti suurta kiinnostusta,» sanoo Crisi.

CWS eroaa gardziellalaisesta tulkintaperinteestä

Joensuun yliopistossa psykologian tohtorin tutkinnon suorittanut ja sittemmin Norjaan muuttanut Jarna Soilevuo Grønnerød suoritti CWS-kurssin Tukholmassa viime syksynä. Hänen mukaansa CWS eroaa Suomessa käytetystä Gardziellan työhön pohjaavasta tulkintatavasta jo siinä, että testilomake esitetään vain yhden kerran. CWS:ssa Rorschach-pisteytysjärjestelmien vaikutus on suuri ja asiakkaan omat sanalliset kuvaukset piirroksistaan ovat pisteytyksen kannalta keskeisiä.  ”Ensivaikutelma on, että metodin ei-kielellinen elementti jää sivuun eikä tule täysin hyödynnetyksi,” Soilevuo Grønnerød arvioi. «Toisaalta gardziellalaisen metodin ongelma on, että psykologi saattaa lipsua tulkitseman omia projektioitaan, joita asiakkaan piirrokset herättävät,» hän lisää.

«Pisteytysjärjestelmässäni psykologin tulee keskittyä nimenomaan siihen, mitä asiakas näkee alkumerkissä ja omissa piirroksissaan ja miten hän ne ymmärtää,» Crisi tarkentaa. Näin tulkintajärjestelmä pystyy tavoittamaan esimerkiksi ajattelutoimintojen häiriöitä. «Psykologin tulee arvioida asiakkaitaan, ei itseään,» kiteyttää Crisi.

Tulkintajärjestelmien integraatio haasteena

Henkilöstöarvioinnin, neuvonnan ja terapian parissa työskentelevä ja viikoittain Warteggia käyttävä Kristofer Wikstad piti kurssilla erityisen inspiroivana sitä tapaa, jolla Crisi yhdisti Warteggin pisteytykset ja kliinisen kokemuksen. Vuosia Warteggia käyttäneelle Wikstadillekin Crisin tulkintajärjestelmän rikkaus ja tulkintojen monipuolisuus oli yllätys. «Uskon, että Warteggista voi tulla keskeinen työväline koulupsykologeille sekä sosiaalityön ja psykiatrian aloille,» Wikstad arvioi. «Se vie kuitenkin aikaa, sillä metodin hyödyllisyys punnitaan siinä, antaako se juuri sitä tietoa mitä kullakin alalla tarvitaan. Tällä hetkellä juuri koulupsykologit lienee se ryhmä, jonka päivittäin kohtaamiin kysymyksiin Warteggin antama tieto olisi kaikkein ilmeisimmin sovellettavissa.»  

Suomessa ja Ruotsissa CWS-järjestelmä kohtaa vahvan Garzdiella – Mattlar -tradition.  “Osa kollegoista kertoo yhdistelevänsä tulkintajärjestelmiä, osa kertoo vaihtaneensa järjestelmästä toiseen,” Wikstad kertoo. Omassa työssä toimivaksi koetusta vanhan tulkintajärjestelmän osasta onkin vaikea luopua, jos se ei sisällykään uuteen järjestelmään. Wikstad toivoo, että tulevaisuudessa tutkimus toisi vastauksen tähän ja että ennen pitkää psykologeilla olisi käytössään nimenomaan ne tulkintajärjestelmän osat, joiden validiteetti osoittautuu korkeimmaksi.

Tutkimus murtaa aiempia käsityksiä

 «Yliopistoissa kuulee vieläkin väitettävän, ettei Warteggia koskevaa empiiristä tutkimusta ole olemassa ja ettei metodi ole validi,» sanoo Soilevuo Grønnerød.  Hän on julkaissut yhdessä Oslon yliopiston psykologian laitoksen persoonallisuuspsykologian apulaisprofessori Cato Grønnerødin kanssa meta-analyyttisen tutkimuksen Warteggin reliabiliteetista ja validiteetista Psychological Assessment-lehdessä, joka on persoonallisuusarvioinnin keskeisiä julkaisuja. «Tutkimusta on kuitenkin tehty ja julkaistu aina Ehrig Warteggin ensimmäisistä kirjoista saakka, mutta tutkimusperinteet ovat jääneet eristyneiksi saarekkeiksi, joiden välillä ei ole juurikaan ollut yhteyksiä. Warteggia käyttävissä empiirisissä tutkimuksissa tyypillisesti ei viitata lainkaan aiempiin tutkimuksiin, mahdollisesti siksi ettei niistä ole tiedetty,»  Soilevuo Grønnerød arvioi.  «Näin Warteggin ympärille ei ole muodostunut maailmanlaajuista tutkimusyhteisöä, joka olisi pystynyt tuottamaan kumulatiivista tietoa.»

 Meta-analyysi osoittaa, että Warteggin arvioitsijareliabiliteetti on korkea. «Mitä validiteettiin tulee, tutkimuksemme perusteella on ennenaikaista vaatia Warteggia haudattavaksi. Samalla ei kuitenkaan ole syytä olla tyytyväinen olemassa olevan tutkimuksen määrään eikä laatuun,» Soilevuo Grønnerød toteaa. Taina Kuuskorven tuoreen väitöstutkimuksen mukaan Wartegg on suomalaisten psykologien kolmanneksi eniten käyttämä psykologinen testi. «Ristiriita käytännön psykologien Wartegg-kiinnostuksen ja näitä psykologeja kouluttavien yliopistojen kiinnostuksen puutteen välillä on suorastaan hämmentävä,» sanoo Soilevuo Grønnerød.

Verkostoituminen on helppoa

 Warteggista kiinnostuneiden psykologien ja tutkijoiden välinen verkostoituminen on nykyään helpompaa kuin metodin historian alkuvuosikymmeninä. Alessandro Crisi pitää yllä tulkintamenetelmäänsä keskittyvää sivustoa soitteessa  www.wartegg.com. Jarna Soilevuo Grønnerød on avannut tutkimuksellisen sivuston www.wartegg.info, jonka kautta pääsee blogiin, useisiin sosiaalisen median ryhmiin sekä bibliografiaan. Kirjallisuuskokoelman tarkoitus on edistää ja helpottaa metodia koskevaa tutkimusta kokoamalla kaikki Warteggia koskevat kirjallisuusviitteet yhteen paikkaan. Tällä hetkellä viitteitä on yli 530.

The Wartegg Bibliography has been updated. 

If you are having trouble posting comments to this blog, it may be the automatic spam filter which reacts to, for example, space between paragraphs. Please e-mail us at [email protected] in case this happens. 

Jarna Soilevuo Grønnerød and Cato Grønnerød

  We received a comment on our meta-analysis (Soilevuo Grønnerød & Grønnerød, 2012) in a text dated 7 January 2012 on Psychometrics.se, a Swedish web page designed to give, in their own words, ”science based information” on psychometrics (http://psychometrics.se/?p=100 ). Since the text is in Swedish we present a short resumé below and add some qualifying comments.

  The author argues that the use of the WZT in selection should be questioned based on the available studies included in our meta-analysis. Out of the 37 studies we included, only two were selection studies (Bokslag, 1960; Gardziella, 1969) and the both of them have been reviewed earlier in this blog (Soilevuo Grønnerød, 2012a, 2012b). The Bokslag study presented high correlations between the WZT scores and employees' evaluations, while the Gardziella study presented high correlations between the WZT results and school grades. However, because of the less than favourable study design in Gardziella’s research the correlations may be artificially high (Soilevuo Grønnerød, 2012b).

  Based on these results, Psychometrics.se recommends cautiousness in the use of the WZT in selection. The author questions the criteria used in these studies, notifies that the studies are rather old, and points out that the number of subjects is not large enough. We agree that these studies do not provide an adequate base for the use of the WZT as a selection instrument. We also agree that the number of subjects is rather low, but the other critical remarks deserve some comments. The age of a study alone does not indicate poor quality. Obviously, there is a lack of newer studies supporting the conclusions. We also consider school grades and employers' evaluations as relevant criteria in selection studies. Although employers’ evaluations may not always be accurate, they have high ecological validity. Still, we do support the author’s main conclusion that we need to be careful when using the WZT in selection because sufficient data to support its use in critical decision making is lacking.

  In conclusion, we reiterate our main points from the meta-analysis. First, the inter-scorer reliability of the Wartegg is adequate and in line with other performance-based methods (we would like to encourage the author to use ”performance-based method” rather than ”projective test”, see Meyer & Kurtz, 2006). In other words, it is possible to create scoring definitions that, with adequate training, can be scored reliably.

  Second, we do not regard our results as final proof that the method is valid, but rather that the WZT can be valid when used appropriately. Importantly, we consider our meta-analysis as strong evidence against harsh critique of the method and demands that it should be abandoned altogether. It has not been shown that the method is useless. However, more research is needed before we can conclude with any degree of certainty that the method is valid.

  As we note in the meta-analysis, many WZT studies are methodologically weak because they lack theoretically based hypotheses, scorer blinding and inter-scorer reliability checks. Still, our work shows that there is no reason to abandon the method, since it may possess a potential to become a highly useful method. More research needs to be done and the theoretical background of the method needs updating. We strongly encourage further studies as well as continuous, careful and responsible use of the method in psychological practice.

      Bokslag, J. G. H. (1960). De predictieve waarde van de tekentest van Wartegg (W.Z.T.) en van een verrichtingstest (Block Design uit Wechsler-Bellevue-Scale) bij de selectie van leerjongens voor drie bedrijven. [The predictive value of the Wartegg drawing test and of a performance]. Nederlandsch Tijdschrift voor Psychologie, 15, 321–33.

Gardziella, M. (1969). Wartegg-testin validiteetti. In M. Jääskeläinen & A. Valpola (Eds.), (pp. 59–66). Helsinki: Työterveyslaitoksen tutkimuksia 48.

Meyer, G. J., & Kurtz, J. E. (2006). Advancing personality assessment terminology: time to retire “objective” and “projective” as personality test descriptors. Journal of Personality Assessment, 87(3), 223–225. doi:10.1207/s15327752jpa8703_01

Soilevuo Grønnerød, J. (2012a). A Review of the Bokslag (1960) study on the validity of the Wartegg in a selection setting. http://wartegg.tumblr.com/post/16462268873/a-review-of-bokslag-1960.

  Soilevuo Grønnerød, J. (2012b). The Wartegg in predicting educational performance in vocational school. A Review of Gardziella (1969). http://wartegg.tumblr.com/post/19343778253/a-review-of-gardziella-1969.

New evidence of high inter-rater reliability of the Wartegg Zeichen Test

  Jarna Soilevuo Grønnerød

  The literature search for our meta-analysis (Soilevuo Grønnerød & Grønnerød, 2012) was completed in 2009. As we have not conducted as thorough searches since, we are not aware of all the relevant literature published after 2009. We are grateful for all information on WZT works which are not included in our on-line Wartegg Bibliography at http://www.citeulike.org/profile/wartegginfo.

One such article published after our literature search was completed is the inter-rater reliability study by Alves, Dias, Sardinha and Conti (2010). A total of 191 WZT protocols were independently scored by two trained raters. The scoring instructions were adopted from Berlinck (2000) and consisted of the following eight variables. Each WZT square were scored separately.

1.     Pencil pressure: strong, medium, weak or mixed

2.     Line type: straight, curved or mixed

3.     Continuity of the line: continuous or interrupted

4.     Line quality: increasing or blurred

5.     Drawing size: large, medium or small

6.     Shading: present or absent

7.     The impression of movement in the drawing: present or absent

8.     Transparency (inclusion of elements in the drawing which actually should not be visible): present or absent

Kappa coefficient was calculated for each variable and each square. The coefficients varied from 0,66 to 1,00. In the article, all the coefficients are presented in two tables and just a quick glance can tell that there are only a few coefficients ranging from 0,66 to 0,79, most of them are higher. The authors present mean Kappa coefficients for each of the eight variables, but not a total mean Kappa. According to my calculation based on the information given in the article, it is as high as 0,87. The authors conclude that it is possible to create scoring systems for the Wartegg which show high reliability between scorers. However, the next question is whether different practitioners make reliable interpretations as well.

The mean inter-rater reliability of Alves and colleagues (2010) is in line with the result of our meta-analysis, were we found an inter-rater reliability of rw = .79, based on 15 results from 12 samples and 771 subjects (Soilevuo Grønnerød & Grønnerød, 2012).

    Alves, I.C.B.; Dias, A.R.; Sardinha, L.S., & Conti, F.D. (2010) Precisão entre juízes na avaliação dos aspectos formais do teste de Wartegg. Aletheia, 31, 54-65. 

Berlinck, V.R. (2000) O Teste de Completamento de Desenhos de Wartegg em universitários de São Paulo. Dissertação de Mestrado. Instituto de Psicologia. Universidade de São Paulo, São Paulo.

The Wartegg meta-analysis by Soilevuo Grønnerød & Grønnerød has been published in print in the newest issue of Psychological Assessment.

The Wartegg in predicting educational performance in vocational school. A Review of Gardziella (1969).

Jarna Soilevuo Grønnerød

Gardziella, Manfred (1969) Wartegg-testin validiteetti [The Validity of the Wartegg]. In Manu Jääskeläinen & Anneli Valpola (eds.) Ammattikoulumenestyksen ennustaminen [Predicting educational success in vocational school], pp. 59–66. Työterveyslaitoksen tutkimuksia no. 48.

Gardziella's study was a part of a larger research on 67 pupils between 15 and 19 years of age at a vocational school owned by a wood processing factory in Finland. All the pupils were presumably male since gender is not explicitly mentioned. The results of several aptitude tests and personality measures, the WZT included, were compared to school grades at graduation, two years after the testing.

The study group was selected from a larger group of 132 vocational school applicants, which were tested and evaluated by researchers and psychologists from the department of vocational health (Työterveyslaitos). The selection was based on an overall evaluation rather than specific test scores. Unfortunately, a study design like this is compromised since the measures used in the selection were also used to predict the outcome, which was school grades, i.e., a case of criterion contamination.

Out of the total amount of 67 subjects, 26 were evaluated with the WZT as well as with two other free response methods, the Rorschach and the Zulliger. The details of how the group of 26 was selected is not presented. The Wartegg forms were scored blindly by Gardziella, using the following scores. General level (related to intellectual potential and an inner ability to elaborate experiences and conflicts in Gardziella's scoring system) and vitality (motivational reserve) were scored from -3 to +3. Overall WZT evaluation was scored from -3 to +2 (a table on p. 65; the asymmetry of the scale is not explained). A description of what the overall WZT evaluation comprises of is not presented in the study report.

General level correlates r = 0,26 with the mean school grade, which is a medium effect size although the direction is logical. Vitality correlates the same amount but negatively (r = -0,25) with the mean grade. Gardziella suggests that individuals with low vitality appear kind, inoffensive and adaptable, which explains their higher school grades.

Overall WZT evaluation correlates r = 0,42 with the mean school grade and r = 0,42 with the combined overall evaluation based on the Rorschach and the Zulliger.

In addition, the pupils were divided into two extreme groups according to their grades (good, N = 8 vs. poor, N = 10), and tendentious group differences for several traits, e.g., aggression, neuroticism, disturbed self-esteem and egoism, were reported. Because of the small number of cases, the results are tentative. For example, disturbed self-esteem was identified in two out of eight pupils in the high grades group but in eight out of ten in the low grades group.

The WZT results are not compared to the aptitude test results. In general, the aptitude tests correlate between r = 0,35 and r = 0,45 with school grades (p. 45).

The less than favorable study design possibly generates artificially high correlations. Still, the results have certain suggestive values and could be used when designing further studies. The tentative connection between low vitality and good grades in male pupils appears interesting and reflects common critiques against educational institutions favoring adaptable behavior. It is also worth noticing that the testing was done two years prior to graduation and that the pupils were living a period of rapid changes (mean age was 17 years). Still, the measures seem to predict educational performance, although considering the weakness of the study design.

As mentioned in the meta-analysis by Soilevuo Grønnerød and Grønnerød (2011), the Wartegg studies far too often fail to meet the requirements of scientific quality. Scorer blinding was not reported, inter-scorer reliability was not calculated, and hypotheses were not clearly presented. The Gardziella study was included in the overall meta-analysis which yielded a fair effect size of rw = .19. However, the study was excluded from the main meta-analysis which yielded a much higher effect size of rw = .33.

Reference: