Notes on: Hammersley, M  (2009) Challenging Relativism: The Problem of Assessment Criteria. Qualitative Inquiry, 15 (1): 3 – 29. DOI: 10.1177/1077800408325325 [This is a response to a piece in one of the Denzin Handbooks, edition three I think, by John K Smith and colleagues]. ‘”Neorealist” or “quasi-foundationalist views”’ have not solved the problems of realism, for Smith (3). Assessment criteria for qualitative research raises important issues, especially when contrasted with ‘a new scientism’ which tries to reinstate older positivist or empiricist views, with implications for review procedures. Smith et al, and Denzin and Lincoln, suggest we are now ‘living in “the era of relativism”’. There is no reliable epistemological foundation for research. All evidence is theory laden. Knowledge claims are relative to socio-contexts. Theory choice in practice really depends on the purposes for which knowledge is being pursued and on subsequent political struggles. As a result we need ‘non-epistemic criteria: in other words… Practical, ethical, and political terms’. For Denzin and Lincoln in particular these criteria must ‘”flow from a feminist, communitarian moral ethic of empowerment, community and moral solidarity”’ (4). The old empiricist paradigms insist that there are real phenomena, that they can be accessed according to scientific method or explicit procedures, and the only task of enquiry is to produce knowledge. The critics reject all three. Neorealists try to preserve empiricism by modifying their methods and retaining the other two assumptions, accepting that methods must always be fallible. For Smith, this is trying to join realist ontology and constructivist epistemology, and this is contradictory, at best a halfway house to relativism. Instead we need to recognise that humans are finite, in other words that any attempt to make sense of the social world reflects our position in it and the cultural resources on which we draw and is a necessarily practical and moral matter. Specifically: 1 we cannot assume that social phenomena exist independently of the way in which they are portrayed; 2 assessing validity and value is always practical, involving judgement, not an automatic consequence of following methods; 3 we should therefore abandon epistemic in favour of moral criteria. These three are assumed to be ‘mutually implicative’. Criticising each assumption in turn, ‘realism’ means many different things, but the basic idea is that phenomena exist and have characteristics independent researchers accounts of them. Neorealism gives so much ground, that ‘realism “does no serious work”, except in helping us resist the idea that anything goes. Much depends on how constructivist epistemology is defined too. It is close to fallibilism in Smith, but that asserts really that we can never know ‘with absolute certainty’ whether claims are true or false. They cite Hindess’s challenge to Popper. Popper searches for logical means by which we can make validity, and this clearly means we can only make inferences about falsity — but the evidence used in falsification is also assumption laden and could be false itself, so that practical conclusions might not follow. Smith and others want to join Popper to broader notions of foundationalism, again assuming that certainty means ‘beyond all possible doubt’ (6). We can have no access to a reality beyond our interpretations, perceptions, language and culture. Humans vary and so it is hard to argue that particular interpretations correspond to how the world really is. We would have to be able somehow to compare ‘different perceptual systems’ against some idea of ‘the world itself’, based on independent access to it. Recent variants suggest that language, discourse and culture are the things that mediate experience. This argument only undermines foundationalism, and ‘by no means are all forms of realism foundationalist’ (7) [I wouldn’t think Popper was]. The specific neorealists critiqued are not foundationalists, and are not claiming that they can have direct and infallible access to reality. Smith’s criticism ‘inherits the very conception of knowledge’, as something absolutely certain, from foundationalism. But there is no need to insist on no possible doubt, and many philosophers have argued against [including Peirce and Wittgenstein]. All knowledge claims might be fallible, but they are not all equally doubtful or uncertain, and fallibilism is really about justified certainty not absolute, and the openness that we might be wrong, ‘until further notice’ (8). For that matter, ‘reasonable doubt’ varies. So we can see knowledge as something which we ‘currently believe corresponds to reality', while admitting we might be wrong, we might accept different degrees of likely truth following comparisons, even while assuming those can also be incorrect. Rationally justifiable knowledge is not the same as truth. Fallibilism breaks with both empiricism and relativism. There might be a confusion about whether correspondence with reality is a definition of truth or ‘a means of assessing the likely truth of any knowledge claim’. We cannot use correspondence with reality to assess our work because we have no direct access to reality, but we might still take it as an ultimate definition of truth and validity. To assume that progress cannot be made in gaining knowledge unless we have some early foundation, is unuwarranted because knowledge is routinely produced without those foundations, not least in our own everyday lives. Our whole experience could still be an illusion, but this broadens the doubt to encompass everything that we currently think we know. Relativists might be relying on some notion of truth as correspondence themselves [the truth is that everything is relative]. Thorough scepticism denies that we have made sense of what’s already happened in some fields and can often act with reasonable success. If we look at language, the relativist view is that children would never be able to learn language because there is no certain link between words and objects, but they do develop natural language ‘without any such foundation’. The infinite regression toward relativism is not insuperable in practice, and we can argue that it is not in enquiry either. At least some of our knowledge can be taken as more or less accurate, unless we are to be continually frustrated, and we should do better learning which assumptions were wrong. In practice is not possible to question all knowledge, and there would be no ground to stand on to do so. Questions involve presuppositions as much as answers, taking some things to be true. Does realism do serious work? Providing an infallible criterion to judge knowledge claims is not the only function for realism. It has other functions in practical contexts, helping us distinguish between knowledge and belief, or make progress in defining knowledge. We can even use it to describe what is currently taken to be knowledge [without offering philosophical endorsement]. We necessarily use terms like knowledge and belief, true and false, even Smith et al. Rorty is used to reject knowledge and truth with capital letters, while ordinary use accepts that we are referring to what passes for knowledge or what is true for people like me. This is normally made explicit unless we are trying to deceive people. We can of course always insist on the truth of knowledge claims even if they are rejected by our own community, without ending in meaninglessness, because this recognises that not everyone actually believes the same things. Again this implies recognising that we could be wrong, fallibilism. We are familiar with various versions of explanations of things like going to war with Iraq. No one thinks contradictory versions are equally true, even if we cannot be certain which one is true. This shows that reasoning like this ‘is fundamental to the ways in which human beings operate in the world’ (11) [so naturalism — just as tricky?]. We still need notions like realism if we are to pursue any enquiry, any activity showing a ‘recurrent concern with finding out the truth about particular matters of fact’. Smith responds to standard arguments to counter relativism — that it is self-contradictory or self refuting, for example [truth is relative]. First they quote Gadamer and Rorty. Gadamer is used to say that we should ‘simply shrug our shoulders at this contradiction’, although he can be read as equally devoted to the discovery of answers to questions, truths, which are not just limited to what is true within a particular sociocultural frame. There may be no single body of true knowledge, but rejecting that does not automatically lead to relativism. His objection is to abstract and formal argument in favour of realism — but you can find opposing but corresponding arguments in Smith et al too, such that all observation is theory laden. Rorty actually rejects relativism because this refers to a theory of knowledge, and Rorty claims to not be discussing that. For Smith, relativism is not a theory of knowledge. But they do deny the possibility of knowledge in empiricism, so they cannot claim not to be engaging in epistemology. They admit that allowing anything to go would also be embarrassing. We must reject a wholly relative point of view, but this confuses absolute and practical argument again [?] (13) [they are accusing realists and committed relativists of absolute judgements, but use arguments in practice only to deny the former any validity?]. Thus we do not normally accept that anything goes, but Smith et al do not clarify the grounds on which we might appeal – so anything really does go. The problem is that relativists do not accept that anything goes in their own culture, but at the same time believe there are multiple, maybe infinite, other cultures that are equally legitimate in their own terms. However, other cultures can leak back into local cultures and affect practical decision-making. [As with multiculturalism]. We have no account of how conflicting arguments can be evaluated, where people disagree, unlike the fallibility approach. We should distinguish between debates about values or policies in the public sphere, and ‘arguments about factual matters, which in my view should be the exclusive substantive concern within research communities’ (14) ‘Methodism’ suggests reliance on standard methods or procedures characteristic of science. There is now ‘a general consensus that none is available’. Smith et al suggest that we should abandon any notion of fixed and explicit criteria to judge knowledge claims as well. In practice, there might be a continuum between extreme methodism at one end and anything goes at the other, with Smith et al trying to locate themselves on it. Criteria are to be open-ended but never explicitly formulated and always revisable [sounds like fallibility]. However there is a difference between the standards by which claims are assessed and the means of coming to judgement. For realists, the central standard is epistemic — whether the knowledge claim is likely to be true. There can indeed be no algorithmic way of deciding this. We do require judgements, but that does not mean that criteria must always be open-ended and subject to revision. Appropriate guidelines can be listed. Knowledge claims can be assessed differently according to whether they are offering descriptions, explanations or theories. The source of the data is important. Much will depend on who produce the documents, what the topic of the interview was, how the participant observer acted and so on. This is a practical process, collective in character. It can provide us with confidence about judgements but can never guarantee them. Research can be judged according to its plausibility [its relation to ‘what is already taken to be known’] (16) and sources of error can be identified. If the new knowledge claim looks implausible this will require us to examine the evidence even more carefully, involving judgements of credibility, whether evidence is ‘open to reasonable doubt because of the way it was produced’[but it never will be with things that we disagree with! Methodological controversy is tactical]. Knowledge can be accepted only until further notice. We might require further evidence proceeding until there is no reasonable doubt, or we run out of available evidence. It is not a matter of accepting knowledge that matches your own existing knowledge: material that does is likely to appear trivial and repetitive. Newsworthy knowledge is what needs to be discussed. Nothing need be rejected, in favour of ‘demands for further evidence’ (16), and we might be forced to revise some prior assumptions judging plausibility. There is no end to this process but it is not a matter of infinite regress, which implies a logical means of validation — these are practical matters, practical judgements need to be evaluated as more or less reasonable, and judgements about reasonableness are crucial: relativists run the risk of insisting that what is reasonable is always relative, and this helps them refute criticism. So relying on judgement does not mean that we need abandon realism or Methodism. The strong sense of both terms is rejected. It is a matter of finding out those methods which are ‘more likely to be successful in producing knowledge’, identifying threats to validity.  Smith et al say very little about what would count as a reasonable list of criteria, and implied constant adaptation as paradigms change — but we would not be allowed to change back to empiricism, although that might be justified in the new climate of scientism [that is they refused to specify an alternative, and this is a weakness]. They should be able to explain why we should adopt particular lists in particular ways, but no explanation is offered — it would infringe relativism. The alternative seems to be either abstract criteria or assessments which are ‘ad hoc’ (18). Must we turn only to moral criteria? A change of metaphor from discoverers to constructors?. Yet the first one might be a useful metaphor, as long as we do not assume we just go out and collect data that is already there: successful research can produce ‘something we did not know or realise before’, especially if we experience resistance to finding answers (19). The metaphor of construction makes us realise that work is involved and that sense is to be made, but it does not sufficiently clarify the distinction between truth and falsity: nor is it always a good thing that researchers control the outcome of the process of enquiry and the effects of publishing the results. The ethical point is that discovery metaphors implied passivity and lack of responsibility. The labelling of children as learning disabled is one of their examples. For them it is just a way of choosing to categorise children in a way that is both practical and moral, and socially contingent. ‘Much of this is true’. However, it would be wrong to imply that we always make up answers to questions. Some of those categories are not given by the world, but must be tested for their practical value, whether they correspond to real differences. This might be one way to check the label. Nor do we have complete control over what we construct or complete responsibility. They assume that nature is responsible for the world in the old conception, but once we have replaced that we must become responsible — but no group fully controls the social. These are equally useful and complementary metaphors. We do construct knowledge but only ‘within quite restricted limits’ (20) [the 11th thesis on Feuerbach does not imply that we can change the world however and whenever we wish]. Relativism does not recognise constraints that operate on us. It’s not a binary choice between discovery and construction metaphors, and problems with the first one need not drive us to the second. Fallibility preserves some commitment to realism as we saw. There is no logical reason that rejecting the first implies the second. Indeed, relativism still implies ‘a particular research community’. They are adopting a consensus or coherence theory of knowledge instead of a correspondence one — but there is still an element of concern for truth not just moral criteria. Perhaps they take epistemic to mean correspondence theory. What about moral judgements — are these seen as universalisable or not? Moral judgements are open to the same relativist doubts as epistemic ones. However, values or moral criteria normally assume a force independent of what any particular group thinks, as seen from ‘ordinary usage’ [!]. This actually ‘plays a crucial role in a commitment to toleration’ (22). Universalism is implicit in morality, and found in Smith et al’s suggestion that none of us should hold to a correspondence theory any more — they are committed to values based on the era of relativism. They praise pluralism but their support for that is more obscure. They are worried that anything goes and believe that post-modern doubt can be taken too far. Their example of dilemmas in refereeing a racist article shows the problems — they think even post-modernists would reject the paper, but how can it be justified except by reference to a universal proscription of racism? Relativism requires us to respect racist cultures and treat them as valid, and accept that there are different conceptions of racism. This goes in the other direction to their support for persuasion and openness. It is not just a matter of developing thick description. A cultural pluralist [Shweder] discusses female circumcision and whether we should suspend our evaluations of it. This is equally a problem for relativism, especially if they also think that research should contribute to make people lives better. Other values claim this too. How could we decide what counted as social justice? There is a reliance on the universal, and a shift of emphasis towards discussion of how we could construct ethical practices [via Utopia in Denzin] The problems multiply. One advantage of limiting research to the goal of producing knowledge avoids problems introduced by other goals. There is a better chance that we can reach agreement about the facts of a particular case compared to agreement on value issues. Political judgements show similar problems, with no agreement unless both sides accept that some conclusions are better than others. ‘We would always have to rely, instead, on negotiation, manipulation, or violence’ (24) [likely if we see it all as a struggle for liberation etc] and these might indeed be necessary in politics, although science attempts to solve disputes differently. Luckily there is no absolute foundation, but the absence of such a foundation ‘does not undercut’ the process. Qualitative research also needs some underlying commitment to investigation if it is to have practical outcomes. Of course researchers might hope that their work has a good effect, they might choose topics on this basis. However, there can also be ‘institutionalised assessment of knowledge claims by researchers’, in refereeing. These should be about whether claims are likely to be true and whether or not they have some minimal relevance. Ethical decisions should be confined to professional ethics, which will avoid controversy in discussing more abstract ethics. This would mark research off as something distinctive. Of course power operates in research communities and is used to resolve issues. Sometimes it is both inevitable and useful [he cites Polanyi] So this is not an effective critique of neo-realism, nor a convincing defence of their own position against the well-known problems of relativism. The things being criticised are still necessarily involved ‘in making any kind of knowledge claim, including those the authors make themselves’ (25). They criticise only extreme forms. They underplay value and stability of existing efforts. Their own insistence that we must move to moral criteria is not strictly related to the rejection of realism but rather to a commitment to ‘epistemological constructivism’ (26). This would lead to [Balkanisation], and membership of communities would be crucial. Ethical commitments arise from the difficulties produced by relativism and anything going. As it is, we are still left with ‘no nonarbitrary grounds for judging competing accounts’ which looks like anything goes. Qualitative researchers probably need a better justification especially in the currently hostile climate. back to Hammersley page