Notes on: Tikley, L. (2015). What works, for whom, and what circumstances? Towards a critical realist understanding of learning in international and comparative education. International Journal of Educational Development 40:237 – 249. Http://dx.doi.org/10.1016/j.ijedudev.2014.11.008 Dave Harris [Useful for my purpose in spelling out his interest in critical realism which he cites as a resource for debates on racism in UK education -- ICE is a field of which I know nothing. It is an interesting argument but it shows the problems of using high-powered theory/philosophy to critique one specific approach you don't like -- empiricism -- only to find to your horror that it also undermines one you do like -- interpretativism. Critical realist would be just as unhappy with the epistemic fallacies of testimony-type research in black activism, and its focus on one surface feature of interaction for political purposes. Tikley et al are critical of the 'garbage can' approach of approaches like Timpson, but his list of relevant factors here is massive as well. I doubt if his programme would ever get started. In other words, it's a resource for critique, but a mirror image of the one he develops in his work on anti-racism: that one says work is not focused enough, this one says we should broaden our enquiries] This particularly focuses on emerging post-2015 education and development in an international education context, and criticises empiricist and interpretativist approaches to research there — both are ontologically reductionist. Critical realism has the potential to build on the strengths but avoid the pitfalls of both. It should proceed from 'an ontologically inclusive and laminated view of learning'. (237), looking at natural and social structures and causal mechanisms that give rise to and inhibit learning at different scales and in different contexts. It should embrace epistemological pluralism, drawing upon 'cross-cultural, interdisciplinary and mixed methods enquiry' making use of 'abductive and retrodeductive forms of inference' and thus moving beyond the 'what works' agenda . Debate rages within international development about the best way to develop and implement different sorts of learning, and millions of dollars are at stake in research programs to find out what works. Yet there are philosophical and methodological assumptions which need to be made explicit because of their profound implications for policy and practice. These assumptions need to be examined, in particular, empiricism and interpretativism. There is the need for a third meta-theoretical approach in education and in the social sciences more generally, informed by critical realism to steer a course between these two. Critical realism assumes that there is 'an external reality outside of what can be perceived by the senses and [yet] amenable to empirical investigation', thus standing between naïve realism and relativism (238). However, this is a large topic beginning with Bhaskar, going through Archer and others. The most relevant for this topic is Archer 1984, Stockfelt 2003 and Tao 2013. Even so there will be necessary oversimplification and we are discussing empiricism and interpretativism as ideal types, although they are really 'broad discursive functions' with a range of approaches, and many actual researchers will not be easily located. Others might embrace a critical tradition including neo-Marxist, Freirian, feminist or postcolonial positions, which might combine a realist ontological position together with interpretativist methods, and others will adopt pragmatic mixtures. Our grouping is therefore an heuristic device. Empiricism at the most general level is based on the assumption that 'positive social research, modelled on the natural sciences and medical research… can "discover" through processes of induction and deduction generalisable laws, replicable findings and reliable predictions on which to base policy and practice' [not how I would have defined it] (238). It claims a robust methodology, often with randomised controlled trials, econometric analysis or school effectiveness studies as best. Constructs and premises are often assumed. Qualitative and participative methodologies are placed lower in the pecking order which limits engagement with processes and context. Empirical data can appear as 'self-evident "facts"'. Causality can be implied from statistical correlation [incorrectly he tells us]. Models can be constructed that are assumed to reflect reality, although claims are often made that findings are theory-free and empirical research is impartial or objective. Empiricism is influential, and claims to be generalisable and predictive. It has a 'hegemonic status' (239) in some social science disciplines and multilateral agencies such as the World Bank, and other donor and government agencies interested in learning. It is often implicated in a quest for [single, it seems in ICE] learning metrics that can evaluate progress and has influenced the '"what works"' agenda where there is empirical testing of the effectiveness of various interventions. It can be useful, for example in measuring the impacts of biological factors and cognitive development, or some econometric or school effectiveness research which can lead to a holistic understanding of learning, but it risks determinism. It does not lead to any particular explicit theory of learning, although it is associated with scientific laws of stimulus response or conditioning, behaviourism, or cognitive neuroscience, even early social constructivist theories. Critical realists have made many criticisms, principally that it 'conflates an understanding of reality with what can be empirically observed'— an 'epistemic fallacy'. Ontological questions about deep structures and mechanisms are displaced by issues of measuring events or phenomena and the surface features of what can be observed and measured can be reductionist when it comes to learning. Critical realism sees fallibility as central because observed phenomena are multi-causal. Closed systems are rare and usually only found in experimental conditions, but only then can we talk about determinism and single causes. Social systems are classically open systems, making prediction and individual causal mechanisms unlikely. Education systems similarly are 'subject to the continuous interplay of structure and agency' making it difficult to reproduce experimental conditions, although empiricism assumes just that. This makes it ultimately fallible [oh, not what I thought – I thought he was saying that fallibility is a good thing, meaning testability, corrigibilty -- I think he means that no system is perfect despite its claims to be infallible as in scientism?]. Far from being a gold standard in research, randomised controlled trials can be problematic. There is the placebo effect, for example, and it is difficult to set up controls. Interventions may lack sound pre-existing scientific evidence. At best, they might show surface level patterns of what works, but not reveal why they work for different groups of learners and what the conditions for their working might be. They have a place in educational research but only as part of a battery of evidence that includes other approaches. Combinations of deduction and induction, as in the hypothetico-deductive model is emphasised, but combining these two does not remove the problems. Deductive approaches only tell us what is already in the premises, and we can learn nothing about any '"abstract structures and mechanisms that make these phenomena possible"' [quoting Danermark] (240). Inductive inference means we can still make incorrect inferences or selective ones, although we can still use it if we focus on not accidental regularities but from supposed 'internal relations structures and ways of acting of things themselves' [ie theoretically significant factors]. This is where abduction and retroduction can also be used. In particular, we must not imply causality on the basis of statistical correlations [but can we test supposed causes by correlations?]. Of course empiricist research has identified patterns that have been overlooked in the past especially by interpretativists, especially biological and cognitive dimensions, — 'evidence of evolutionary psychology and cognitive neuroscience', the development of the brain [Jesus, that all? Not stuff on empirical patterns of social class or race for that matter? He likes these because they seem to match to the causal structures and mechanisms emphasised by critical realism]. We must not treat them deterministically, though, partly because cognitive science is still in its infancy and is not yet the key to unlock learning. It is still contested and it is hard to derive simple policy prescriptions. [Dangerous territory to flirt with for an antiracist I would have thought].  Biology itself is multicausal and complex [genetics and epigenetics]. Social interactions are important in the development of the mind. Emergence is important for critical realists too. We must preserve the insights of other disciplines. Cognitive neuroscience has been influential and controversial with things like early childhood literacy and the use of phonics, with controversy about whether this is to be given sole emphasis [discussed at greater length 241]. School effectiveness studies also show the influence of empiricism and it can be valuable for critical realists to identifiy patterns of variance and associations between variables 'that can then be explored in more depth by qualitative means' especially if we use 'more sophisticated and context sensitive methodologies… Multilevel modelling [for example]' and avoid determinism. For example the importance of class size has long been seen as a determinant, although the results are actually inconclusive comparatively, especially in low income countries, and class size seems to be mediated by a range of other factors like teacher quality and different sorts of pedagogy and these vary with different cultures. The effects of textbooks similarly [comparative African work is cited] [there is also discussion of a notion of a common international learning metric]. The objectivity of empirical research has come under question for its notion of cognition which assumes minds objectively process and reflect external reality, leaving little room for values, language, culture or emotion and self reflexivity. But values are central, even in statistical enquiry, where data has to be sorted into categories, and value judgements or the choice of methods are crucial, and evidence has to be evaluated. There are obvious normative assumptions and underlying theories, for example rational choice theory underpinning research on incentives to improve teacher performance, or neoliberal economic theory about the way markets work to regulate competition between schools: neoliberal discourse and its views of the individual may stand in contrast to 'the more collectivist orientation in many non-Western cultures' (242). [but relativism beckons?] interpretativism — a broad understanding with a range of positions including social constructivism, ethnomethodology, phenomenology, and symbolic interaction. He also wants to include anti-foundational perspectives including post-modernism in poststructuralism [blimey]. All knowledge is produced by social or discursive practices with human perceptions, values and negotiated interactions [big split between ethno and poststructuralism straightaway]. Therefore they 'place emphasis on the relative, context -dependent and subjectivist nature of reality' [bollocks] (242). He also wants to include within interpretativism in ICE anyway various critical emancipatory perspectives including neo-Marxism, Freire, feminism and post-colonialism — and antiracism. These have their own ontological positions like historical materialism or patriarchy, they are often critical of mainstream interpretativism or anti-foundationalism for being ahistorical and ignoring power, but they belong here because they give voice to interpretations and experiences of different oppressed groups and do textual deconstruction, related to oppressive power relationships. They often do '"critical interpretativism"' as in Lincoln and Guba, focusing on dialogue and praxis and social change as a goal. interpretativism has been an important critique of empiricism including the World Bank's Learning for All agenda. It is focused on ideology in that discourse, which is  seen as reductionist and ignoring capitalist social and material relations of production. There has been a contribution to learner centred theories of learning. There is a 'clear affinity'between interpretative based meta-theoretical outlooks and constructivism, especially the role of language, pedagogy and other cultural artefacts, and the need for more contextually relevant learner centred pedagogies. There are however several problems. There is a reluctance to acknowledge reality out there beyond different versions of reality as constructed. There is another epistemic fallacy reducing the realm of the real to how reality is interpreted. Issues of process are highlighted, but there is an overreliance on inductive forms of reasoning, for example in the use of grounded theory, with the same strengths and limitations — 'the risks involved in assuming that the data being analysed represent all aspects of the reality of the phenomenon'. There is an overemphasis on social and cultural dimensions of learning and suspicion and mistrust of any account 'that smacks of biological or social determinism', and a 'wariness towards recognising the real "natural necessity" underlying learning, and thus the quest for any biologically causal mechanisms' (243). There is also 'an implicit normative bias', for example a tacit belief in learner centredness, or an 'ex post facto [sic] interpretation/rationalisation of the perspectives of subjugated individuals and groups', and a 'bias towards qualitative, participative and deconstructive methodologies… And prioritisation of interpretation and critique… Over causality… [Which]… Means that interpretative work within the ICE canon has more limited traction with policymakers'. Critical realism can be a middle way, 'a more sophisticated realist ontology'. Bhaskar and others have attracted growing interest as launching an ontological turn for educational research especially in international and comparative education and in other social sciences. Bhaskar sees critical realism as an under labourer in providing a philosophical basis for scientific and social scientific enquiry. It critiques both empiricism and interpretativism and identifies an epistemic fallacy in both (emphasising epistemology over ontology, and observable associations rather than real causal mechanisms and structure; what one can know as opposed to what is; anthropocentrism). There is a transitive domain, causal mechanisms that are relatively enduring, and an intransitive domain which relates to constructions of that deeper reality:  in ICE the transitive domain means characteristics and current policy priorities, while intransitive refers to 'deeper causal mechanisms and structures including the impacts are relatively enduring and intransigent inequalities in society and/or the structure of the education system itself, including the degree of centralisation/decentralisation'. There are three levels of reality — the empirical (experiences and perceptions of knowing subjects); the actual level (that may exist outside of our perceptions); the real level (deeper lying structures and causal mechanisms that produce the other two). In debates about learning the empirical might relate to measured differences in the performance of learners, the actual might relate to the various factors that impact on learner outcomes, but the real relates to questions of causality, theoretical explanations of how differences between individuals and groups are impacted by different sorts of inequality, and the mechanisms through which interventions have an impact on learning outcomes [still looks pretty superficial to me]. Empiricists are too concerned with flat actualism and not enough with the deeper level of structures and causal mechanisms. Critical realists take structure to mean 'an ensemble of various objects and relations which create a single overall object which has a structure' [circular], just as subatomic particles make up atoms. The education system is a structure, and we also have social structures. These are emergent and irreducible to composite parts. The subsystems are curricular, assessment teacher training and the different institutions. Structures 'give rise to mechanisms that have causal powers'. [Shift of analogy] 'the basic structure of the brain in interaction with the environment give rise to mind' [very naïve 243]. Differences in achievement between girls and boys are 'caused' by mechanisms arising from the structure of the family or school, the gendered nature of social relationships or institutional responses that gendered violence. These in turn can be affected by mechanisms arising from structures such as a gender biased curriculum, all policies emanating from priorities higher up. There are interrelated scales and levels, 'a necessarily laminated view of learning' [looks like functionalism to me]. Social systems are different from natural systems because they have human agency as Archer argues. Social structures are pre-existing and relatively enduring or intransigent, and they constrain or enable. Human agents have attributes relevant for agency, 'self-consciousness, reflexivity, intentionality, emotionality and cognition'(244) so they can formulate projects pursue interests and learn, reflect upon and change social arrangements. There is a dialectic relation rather than a determinist or voluntarist one, hence complexity and emergence. Emergent properties are irreducible, and show material transactions with nature, social interactions between agents, interactions between agents and social structure, and the stratification of agency [my close paraphrase of him citing what Bhaskar says]. Learning emerges from causal mechanisms at biological, psychological and social levels and it is irreducible to any of them. It is enabled and constrained by structures at different levels, including the organisational and social. There is a laminated view of learning systems. There are implications for research. Researchers must accept their fallibility. Abduction is a form of inference makes testing possible, as does critical self reflexivity about the values underpinning research, a 'diatopical hermeneutic [wha?]  to establish an ethical basis… linked to the idea of a critical planetary humanism… Reflection on the ontology of "being human"… And of sharing the planet with other species' [all of it linked to remarks in Archer and Baskhar apparently]. There is an emancipatory moment, linking to emancipatory ideas in ICE. Emancipatory-projects may be articulated around the different issues and levels, although critical realism itself is meta-theoretical and therefore not prescriptive. However it is 'maximally inclusive at an ontological level to the idea of multiple forms of inequality/oppression' and how they might intersect, helping us to resist any arbitrary focus like the current one on inequality in the global South [apparently]. Again there is a focus on fallibility and self reflexivity, not neutrality or absoluteness. [or an arbitrary focus on 'race' or black African Caribbeans?] It's possible to develop a suitable model of a laminated learning system, an ecological model of human development operating at different levels [a diagram of interlinked circles ranging from the macro system of the political economy down to the individual level of genetic predispositions, with all sorts of mediating linkages and processes involving homes and schools, national policies, informal learning environments and so on — functionalism again really]. This is non-reductionist and laminated. Each subsystem points to causal mechanisms and intransitive structures. It identifies interactions between subsystems. There are [of course] double arrows to represent the two-way nature of interactions and be nondeterministic. It allows for variations between different contexts and is consistent with emergence. It is not empiricist but stresses instead dynamic dialectical natures of relationships. It develops another model [by Bronfenbrenner] [it is really a list of factors organised into different levels allowing for the interrelation of everything with everything else]. We need different theoretical models, interdisciplinarity, including transdisciplinarity, multilevel modelling, the vertical case study as in Vavrus and Bartlett, international collaboration, communities of practice, mixed methods, both empiricist and interpretative [bit of a garbage can], used pragmatically, although there is a danger of reducing all this to empiricist notions of truth and reality. We need instead a 'pragmatic idea of "work ability"… Consistent with a critical realist approach although the difference is that in the use of the abductive method qualitative and quantitative findings can be used in combination and as part of an iterative process to advance the development of theoretical understanding' (246). Methods are rarely capable of grasping this complexity on their own. [especially if they are already politicised] Examples: statistical associations need to be expanded by qualitative understandings, for example of the actual theories of aspiration of Jamaican boys in one case based on interviews and focus groups to round out statistical associations. These could then be refined to be more contextually appropriate before being subjected factor analysis [details 246] Abduction is 'a formalised approach to inference… Starts from a theory and then considers the extent to which it fits a case' (247), as in Peirce: 'A surprising fact [NB!], C is observed, but if A were true, C would be a matter of course, hence there is reason to expect that A is true'. What we do is postulate the existence of causal mechanisms, A in this case, theories of learning say, while looking for evidence under which particular events, C, occur [pretty good this] this is how critical realism would argue for 'the existence of reality beyond our perceptions of that reality'. The best theory is the one that provides the best explanation [which opens a can of worms of course]. We don't need a grand theory explaining everything but should pursue an ongoing cumulative approach as more cases are considered. Abduction necessarily involves abstraction as a creative act on the part of the researchers, and creative judgement, 'judgemental rationality' [this is good and it looks like he got it from Pawson R  2013 — The Science of Evaluation: a Realist Manifesto, London, Sage]. Abduction describes how researchers and practitioners actually go about solving problems, and is a more accurate account than empiricism and hypothetico-deductivism. It also describes evidence-based practice in medicine or psychology. Retroduction is closely related but is concerned more with the theories and causes — why things are as they seem to be – 'why do evidence and data appear to follow the patterns they do? Why are theories about the world sometimes wrong and what kinds of bodies of evidence are used to substantiate and underpin each theory? And finally how do we explain the phenomena that we are currently interested in? The third of the sub elements is the expansionary task which most realists take up in a very direct way. They offer causal explanations without having a deterministic approach to cause. The second of the sub elements as an attempt to re-examine existing theories' (247). We can see hints of this sort of approach in some of their programs that have developed in ICE — they use different methodologies, different ways of processing data, mixed methods, they used inductive reasoning, they operated different levels and they develop their framework across different country contexts. Finally they applied retroductive reasoning. Overall a case has been made to consider critical realism, or certainly to begin with making clear ontological assumptions and to be explicit about them although this is not usually been done. Empiricism has been dominant so that learning has often become equated with standardised test scores. interpretativism has also been implicit in several discourses about learner centredness. The ontology of learning needs to be focused on using critical realism to avoid reductionism and this should include a wide range of evidence from empirically focused disciplines including medicine psychology, neuroscience and economics and from school effectiveness research as well as sociology and anthropology using a variety of methodological techniques. Determinism should be avoided as short ontological relativism. The main potential is to bring together different kinds of evidence and to do transdisciplinary research. Abduction and retroduction needs to be developed to develop theories of learning. We should 'adjudicate rationally between competing theories' (248) rather than pretending to be neutral, and this is particularly relevant for ICE, because it allows for a critical engagement and encourages self reflexivity. We still need far more research however especially if we are to 'challenge the hegemony of empiricism'.