Notes on: Sparkes, A (2018) Creating Criteria for Evaluating Autoethnography and the Pedagogical Potential of Lists. In L. Turner, N. Short , A. Grant, & T. Adams (Eds) International Perspectives on Autoethnographic Research and Practice. London: Routledge. Dave Harris [I am very grateful to Prof Sparkes for making this available to me] Pelias offers a list of contrasts between 'a flat piece and an engaging piece' at work in his own evaluations. Tracy has proposed eight universal criteria for judging excellence in qualitative research [they include worthy topic, rigour, sincerity, credibility, resonance, significant contribution, ethical, meaningful coherence (257)]. Richardson offers her criteria — 'substantive contribution, aesthetic merit, reflexivity, impacts, expression of reality' [earlier version then] Barone and Eisner propose: 'incisiveness, conservation, coherence, generativity, social significance, revocation and illumination'. Lots more might be possible. Overall, 'scholars tend to create news lists according to their specific needs and purposes'. Ethnography itself is blurred, but Holman Jones, Adams, and Ellis say there are key characteristics — 'purposefully commenting on/critiquing culture practices, making contributions to existing research, embracing vulnerability with a purpose, creating a reciprocal relationship with audiences in order to compel a response' (258). However how these are 'played out in practice… [Is]… Up for grabs'. Holman Jones, for example says that she still wants to create her own responses, and she has her own list — 'participation as reciprocity, partiality, reflexivity and citation analysis as strategies for dialogue (and not mastery), dialogue is a space of debate and negotiation, personal narrative and storytelling as an obligation to critique, evocation and emotion as incitements to action, engaged embodiment as a condition for change'. Denzin on performance autoethnography: whether or not texts 'unsettle, criticise and challenge taken for granted repressed meanings, invites moral and ethical dialogue while reflexively clarifying their own moral position, engender resistance and offer utopian thoughts about how things can be made different, demonstrate that they care, that they are kind, show, instead of tell, while using the rule that less is more, exhibit interpretive sufficiency, representational adequacy, and authentic adequacy, are political, functional collective and committed. Bochner and Ellis refer to evocative autoethnography and their criteria 'include looking for abundant concrete details, wanting to feel the flesh and blood emotions, people coping with life's contingencies, and being offered structurally complex narratives that are told in a temporal framework representing the curve of time'.  Bochner adds that he wants autoethnographers to examine their actions including those ' underneath them, displaying the self on the page… A demanding standard of ethical self-consciousness… A story that moves me, my heart and belly as well as my head… Acts out [subjective life] in ways that show me what life feels like now and what it can mean' (259). These and other lists can be 'foundational, prescriptive, and normative', used to 'police the boundaries'. Lists can quickly become quality appraisal checklists used in quality control. They can be 'exclusionary and punitive'. None of the scholars want this, of course — Pelias wants to say what he likes but not impose his own evaluative stance and leave open the possibility of other schemes. He does not insist that other readers must adopt this standpoint [but what about his students?]. His criteria are optional. The same goes for Barone and Eisner who see criteria as 'cues for perception… Starting points' (260), not to lead to standardisation. They invite readers to use their own judgement in applying these criteria. Tracy suggest that models can be adapted and that rules and guidelines should not be grasped too strongly, not seen as fixed and inflexible. So there is an expectation that these lists will be used 'with the openness with which they were intended', especially if we are going to be non-foundational or relativistic — Smith and Hodgkinson admit that criteria are socially constructed,'" in part unarticulated… Invariably rooted in our standpoints and… Elaborated through social interactions"'. This contrasts with the universality claimed by Tracy. Others stress the open-endedness of judgement and our capacity to use it. Criteria can change 'depending upon the context and the purposes' [still referring to published articles]. Something new might come up. The practical use of lists is important rather than theoretical labour. It is important to see criteria as being implemented, 'in the doing and engagement with actual enquiries rather than via the distillation of some abstracted epistemology''s Holman Jones says that criteria are generated in the course of writing, others acknowledge that things change and grow. Even Tracy says that we should demonstrate the criteria in our own work, 'as apprentices' so that we can better understand them. Gordon and Patterson explore each of Tracy's criteria and applied them to work they had done 'within a womanist caring framework' (262). They concluded that these criteria did work but the way of achieving them was different in each study: they were a useful guide to evaluate their own work, they were universal but not fixed, specify ends but not means. However, they should be grounded in an ethical framework — womanist caring — rather than having ethics as an item in the list. This shows a useful modification and that lists are not owned or controlled once they enter the public domain. Students can be baffled by 'the vast array of criteria that are available for judging their own work and that of others' Tracy intends her list to provide them with a common language of excellence, and other lists can be used in this way as well, to provide a common language for discussion. This can 'provide a sense of security and direction for novices' when they take risks (263). Autoethnographers might be particularly in need of security, [and the fictional example implies that they also expect their tutors to defend them if criticised]. We all need somebody to say that we are finished, that the work is completed. He has played this role himself — 'it is a worthy role to be celebrated'. Tracy's guide was useful, and might be seen as a tool for scholars to monitor their own quality.  Any of the other lists might work as well. There is a worry that this might become far too 'mechanistic linear and functional', to focus on process rather than product. He sometimes ask about his own reference points, and he says that he has none, although he quotes Winterson on trying to write in a way that dangerous and pushes you where you don't want to go, or Leonard Cohen expressing his unease about getting an award for poetry which makes him feel like a charlatan. For Sparkes, autoethnography is 'an activity I do not command, my own autoethnographic stories have always written me far more than I have ever written them'. Autoethnography is a bodily dimension, something 'pre-objective, enfleshed, multisensory and carnal' not ready for language. Crafting a story is 'somatic work'. As a result he asks his students to think about and with various lists of criteria. He knows that they are 'often contested, overlapping and contradictory'. Students are invited to think about how they feel 'in their guts and in their flesh' to make '"embodied judgements" there are practical, emotional and corporeal as well as discursive'. Students are asked to construct their own lists as well, and this awakens them to their own histories and prejudices [but who decides which ones to adopt informal assessment?]. Judgement always means we must risk our prejudices, open ourselves to questions, be willing to accept challenges to prejudices. We should be prepared to change criteria. However, 'to be open does not mean to accept automatically' (265). There is no method to avoid this risk. There is an admission that evaluation of autoethnography '"is simply another story from a highly situated, privileged, empowered subject about something he or she experienced"'[quoting Gingrich-Philbrook]. As a result [academic?] readers have to accept risks that they may have their entitlements challenged [what about students?]. We have to 'listen carefully and respectfully', attempting to grasp what is being expressed 'emotionally, viscerally, and discursively' so we can make judgements 'in an ethical, fair, and caring manner'. This requires 'connoisseurship', the ability to 'make fine-grained discriminations among complex and subtle qualities… The art of appreciation', which is far more than just liking something. This has been seen as 'a romanticised "intellectual flight from power"', so we should be aware of politics and power at various levels operate, including faculties, and how they are used to sort out good from bad. Thus any delineation of criteria is a political act. Smith and Hodkinson say that researchers are free to use whatever resources they have to support and strengthen their own rules, or rules that are in their interests . [Balkanization beckons]. As pedagogical devices, lists of criteria can help students explore issues of power and politics, legitimacy, and how some voices are silenced. He shares with his students his own experience of how he spoke truth to power and got hostile consequences back — 'managerial power was enacted in its most raw and questionable form' (266). As a result he helps students develop strategies 'for defending and promoting their interests in various contexts'. One of them might be a PhD viva: here it may well be 'correct, acceptable and in the student's interest… To express the view that passing judgement… Is a matter of embodied interpretation, with lists of criteria being fluid and changing, open-ended and context specific, leaving us with only multiple standards and temporary criteria'. But this would not do a job interview 'where the majority of the selection panel is composed of positivists or post positivists' who probably are not connoisseurs. In those situations, 'it may be advantageous to call upon' things like Tracy's universal criteria. 'The tactics suggested earlier might be frowned upon by many as being unethical and dishonest'. However it is equally questionable to send students into situations where power and politics are a play without judgement criteria. They might need to be prepared 'in the dark arts of conceptual self defence and strategies of self-preservation', 'learning to play the criteria game', as a strategy 'to respond and act within, rather than being "worked over" in hostile situations'. Overall, this is not an innocent question of epistemology but actually grounded more in the sort of reasoning done by '"finite practical and moral beings. The Sparkes referred to is Sparkes 2007 — his piece on qualitative research in the audit culture --  and his 2018 piece in a handbook of ethnography of education. Back to Education Studies -- assessment