Richard Procter, University of Bedfordshire

Abstract

Recent research has shown that improving education processes has become a priority of all governments (OECD, 2010; Barber and Mourshed, 2007). There have also been recent calls for the knowledge that is already in existence to be used more effectively to improve these education systems both internationally (OCED, 2010) and nationally (Pollard, 2008).

This study aims to evaluate an approach to teachers' use of research knowledge to help inform their practice. It will provide a web-based knowledge management system for teachers that will support their professional development. Within this broader evaluation this study is interested in what research practices are used by teachers at present and what value if any, teachers ascribe to these practices?

A questionnaire focusing on the use of research practice by teachers adapted from Levin et al. (2010) shows the importance of asking about practices rather than attitudes when questioning practitioners. The questionnaire is designed using a dual scale format (Pedder et al., 2010) that allows teachers two responses for each questionnaire item; their perception of the extent to which a practice is being used by them and their value of that practice.

This research highlights the value-practice gaps, between the extent that a research practice is being used by a teacher and the value that teachers ascribe to that practice. The study shows a consistent gap between how much teachers value the use of research and how much they use research in their daily practices. This study gives some useful insights into the debate surrounding practitioners use of research in schools (Thomas and Pring, 2004).

Keywords: evidence‑based practice, professional development, teacher education, questionnaires

Introduction

This paper reports on a survey into teachers' use and value of research evidence. It is part of a broader study that will evaluate an online approach for providing research evidence to teachers and how this fits with their current practices. Thus the two questions that are posed in this paper are: what research practices are currently used by teachers and what value do teachers place on these practices?

In recent years there has been an increasing use of online technologies for both the improvement of teaching and learning in the classroom and for the development of teachers' practice. This study will use and adapt an online approach, used in the training of medical doctors[1]. This approach uses graphical pathways or flowcharts, henceforth called online pathways which are used as a structured way of presenting complex knowledge. Each node in an online pathway provides links to the display of more in-depth knowledge. This knowledge will be in the form of written explanations with references to original research evidence and may also include links to video and audio resources. The knowledge presented in online pathways will be reviewed regularly so that it provides an up to date picture of the research knowledge within a field. Online pathways will provide a way for practitioners to engage with research knowledge and for them to use these to develop their classroom practice.

In the 1990s there were a number of major critiques of educational research in the UK (Hargreaves, 1996; Hillage et al., 1998; McIntyre and McIntyre, 1999; Tooley and Darby, 1998). There were also calls for evidence based practices to be adopted at a policy level. In David Blunkett's, the then Education secretary, 2001 lecture to the Economic and Social Research Council (ESRC) he called for a 'revolution in the relationship between government and the research community' (2001, p.21), this was 'coupled with an emphasis on research that demonstrates what types of policy initiatives are likely to be most effective' (Whitty, 2007, p.5). These agendas were also being pursued in a number of other fields such as medicine, public policy and management (Nutley and Davies, 2000).

Evidence-based practice is the idea that within the field of education the practice of teachers should be based on evidence from research. As Hammersley points out, there is already a certain rhetorical effect in the title to discredit opposition to it: 'after all, who would argue that practice should not be based on evidence' (Sharhar, 1997. p.110, quoted in Hammersley, 2001, p.1). This rhetorical effect means that it is difficult for critics to challenge this assertion. There have also have been a number of critiques of the evidence-based approach, Hammersley (1997, 2001) highlights the problems and dangers of privileging evidence over experience and of how research knowledge is refined into summary statements for use by practitioners.

There have been more recent calls for the knowledge that is already in existence to be used more effectively to improve these education systems (OCED, 2010). More specifically in the UK, there has been a call to increase access to existing knowledge about education (Pollard and Oancea, 2010). Hence an important area of research is the use of knowledge that is already in existence and how it that can be transformed (Foley and Hargeaves, 2003; Pollard, 2008), so that it can be leveraged. Although Hammersley, (2001) asserts that this process is not straight forward and can be problematic.

Teachers' use of evidence

Beyond academic arguments of what evidence is there are a number of difficulties with teachers using evidence on which to base their practice. Levin and his team have reviewed the research (see Cooper, Levin and Campbell, 2009; Levin, 2008) and have summarised some of the difficulties; two are highlighted here:

• Concerns about the quality, relevance and accessibility of research in education to practitioners and policy makers (e.g. the use of language or the publication outlets in which research tends to appear).
• Educators report a high level of receptivity to research but a relatively low level of active engagement with research in the sense of spending time reading or discussing it.'

(Levin et al. 2010, p.4)

If research is to be used by teachers it needs to be both of relevance and accessible to teachers. Yet research papers are written for an academic audience and published in research journals, this one included, rather than in a language and in publications that are readily accessible to teachers. Researchers have been criticised for this but also teachers have been criticised for their lack of engagement with available research (Cooper and Levin, 2010).

Other researchers have highlighted the value that teachers ascribe to their interpersonal relationships. They noted that colleagues' personal recommendations affected what research they considered (Cordingley, 2004). Levin et al. (2010) in their review state 'that personal experience usually has more impact than research... [and] that the primary determinates of professional behaviour is related to what colleagues and superiors do and value' (Cordingley, 2008; Levin, 2004, Nutley et al, 2007).

Hence what teachers believe and value and what their colleagues believe and value has an effect on their professional behaviour. Levin et al. goes further to state that it is 'probably more typical that research first acts on people ideas and beliefs, with those changes later translating into changes in policy or behaviour' (2010, p.4)

Teachers values have been looked at in other areas of educational research. James and Pedder looked at teachers' values in their work on assessment for learning stating that 'the importance of this values dimension is underplayed' (2006, p.111). They go on to elaborate their argument:

'Webb et al. (2004) argue that changing practices can lead to change in opinions and values, so it is worth focusing directly on efforts to change practice. However, older, but highly regarded, psychological research, for example by Ajzen and Fishbein (1980), advances the contrary view that changing behaviour depends on first changing beliefs. Beliefs are not sufficient to guarantee behavioural change but they provide the necessary reasons to act.' (James and Pedder, 2006, p.111)

Whether changes in practice lead to changes in values or changes in values lead to changes in practice, it will be useful to have some insight into both teachers' values and practice. Consequently it is important not just to look at teachers' use of research practices but also their value of those practice as these may be a determinate of their proclivity to the use of evidence.

Method

A questionnaire was designed (Procter, 2011) to find which research practices teachers and their schools use and what value teachers ascribe to those research practices. The questionnaire items were adapted from the work of Levin et al. (2010) who shows the importance of asking about practices rather than attitudes when questioning practitioners about research practices. The questionnaire employed uses a dual Likert (1932) scale format, developed by Pedder et al. (2010) who built on the work of the Improving School Effectiveness Project (Robertson et al. 2001; MacBeath and Mortimore, 2001). The format allows respondents' two responses for each item (see Figure 1). Firstly their perception of the extent to which a research practice is being carry out by themselves or their school and secondly their value of that practice irrespective of whether they or their school are carrying out that practice. Their value would indicate how important they felt a practice was for enhancing the use of research.

Figure 1 - Dual scale questionnaire format

The questionnaire was divided into three sections: section A contained 11 items based around 'You and research', section B contained 18 items based around 'Your school and research' and section C, background information, contained 7 item including, gender, years of teaching experience, years at current school, educational phase, post and highest level of education.

Three approaches to competing the questionnaire were used. These were paper-based, email with the questionnaire as an attached document, and an online questionnaire. The paper-based version was given to primary and secondary teacher at a maths conference and to a number of colleagues who distributed it in their schools. These approaches returned forty five questionnaires. The questionnaire was also emailed as an attachment to a number of colleagues and this returned nine questionnaires. An online questionnaire was then developed and the link to this was posted to the Times Education Supplement (TES) teachers online forum. This returned a hundred and two questionnaires. Of the three approaches the online questionnaire provided the most responses followed by the paper-based version. The version that was emailed as an attached document provided the lowest number of responses. The use of multiple ways of delivering the questionnaire meant it was difficult to try to calculate an overall response rate for the questionnaire.

The questionnaire was designed so that it could be completed by school staff of any phase, any role and of any subject specialism. This allowed for comparison between groups such as primary and secondary, teacher and department heads and by highest level of education. Two pilots were completed before the main data gathering and these were both completed with postgraduate students (Procter, 2012). The pilots highlighted that the questionnaire was initially too long with too many items. The questionnaire was completed by 156 respondents and the data were loaded into SPSS for analysis. Although a response rate could not be meaningfully calculated a margin of error based on the size of the sample and size of the population could be calculated. The teacher population in England in 2010 was 448,100 (DfE, 2011) thus with a sample size of n=156, the margin of error is 7.84% at 95% confidence.

Results

Background information is reported initially to give some insight into the nature of the sample before the reporting of section A 'You and research'. The distribution of respondents showed that 77.3% were female and 22.7% male, this compares favourably with the Department for Education's latest school workforce census for England where 74.6% of the head count number of full and part-time regular teachers were female and 25.4% were male (DfE, 2011, p.2).

The respondents reported that 54.6% worked in secondary, 42.3% worked in primary and 3.1% worked in middle schools. The majority of respondents were class teachers 58.0%, 22.9% were head of departments, 6.9% deputy heads and 2.3% head teachers.

Highest level of education was reported as 28.2% Post Graduate Certificate in Education (PGCE), 19.8% were studying for a Masters degree (in progress), 16.2% Masters degree (completed), 14.5% Bachelor of Education, 10.7% Bachelors degree and 6.1% had either completed a PhD or it was in progress. The high level of Masters degree (in progress) may be related to the fact that there was central government funding for teachers to study for Masters degrees starting 2009 and this was subsequently withdrawn in November 2010 (DfE, 2010).

You and research

Section A is about the respondent and their use of research practices. For ease of analysis positive scores were grouped together to provide an overall percentage for each item, the scores for 'often true' and 'mostly true' were added on the practices scale and the scores for 'crucial' and 'important' were added on the values scale. This enables values and practices data to be more easily compared.

This type of analysis not only shows the gaps between teachers' practices and their values but also between their current practices and their aspirations (James and Pedder, 2006). Table 1 below shows the percentage of teachers that placed a high value on statements about research practices. The research practices are listed in descending order of percentage of teachers who valued them as either important or crucial.

Table 1: Top value scores for 'You and research'

Eight out of the eleven items or 72.7% were highly valued. Respondents scored them between 80.5% and 52%, as either important or crucial for enhancing the use of research. The two items 'you engage in research related reading' and 'you engage in research focussed discussion with colleagues' were the most highly valued statements at 80.5% and 74.6% respectively. These two items may be related and thus research that is read about by teachers is then discussed between teachers, to clarify and reflect on their ideas as to how they may be able to use research in their practice. These results are consistent with the finding from Levin et al. who reported that educators had a 'high level of receptivity to research' (2010, p.4).

Although research related events are highly valued by 68.7% of teachers, this then breaks down into what these events are. Thus professional conferences, ones organised by professional associations are more highly valued (59.2%) than Local Authority events (52%), academic conferences (44.9%), and outside organisation sponsored events (36%). This highlights that teachers place more value on their professional conferences over other types of research and is consistent with the critique that research needs to be relevant and accessible to practitioners (Levin et al., 2010, p.4).

Engagement with postgraduate studies was highly valued by 59.2% of teachers and had a practice score of 47.7%. Thus just under 50% of the sample are engaged in postgraduate studies and just under 60% of these highly value this. This may be because of the two years of government funding for the Masters and Teaching and Learning course that teachers have been encouraged to study for. This funding has since been withdrawn for new applicants.

Value-Practice Gaps

This section compares the value scores and the practice scores. Comparisons are made between the percentage of teachers who indicted important and crucial on the value scale and those who indicated often and mostly in the practice scale. In Table 2, practice scores are taken away from value scores to show if there were any value-practice gaps in these data. Items are listed in descending order of percentage of the value-practice gap score.

Table 2: Value-practice gaps for 'You and research'

Table 2 shows that the largest value-practice gap is for the item 'you have received funds from schools to carry out research' which is 35.7%. Although this is the largest gap between value and practice it can be seen that teachers place a low value on this item of only 46.7% and there is an even lower practice score of 11%. Thus teachers do not receive funds to carry out research but equally it seems that they don't value this either.

The second largest gap is for the item 'you engage in research focussed discussion with colleagues' at 34.9%. Teachers place a high value on this item 74.6% but the item has a poor practice score of 39.7%. Again, this reinforces that teachers have a high receptivity to research but a 'low level of active engagement with research in the sense of [...] discussing it' (Levin et al., 2010, p.4).

The next six items with gaps from 31.3% to 20.5% are related to research focused events. These results show that teachers have difficulty in spending time out of their classrooms to engage in events that have a research focus.

The smallest value-practice gaps were for the item ' you engage in research related reading' with a value score of 80.5% and a practice score of 70.3%. This result is at odds with other researchers who have found that teachers have a 'low level of active engagement with research in the sense of spending time reading' (Levin et al., 2010, p.4). This may be due to the fact that 47.7% of this sample do engage in post graduate studies.

Overall, it can be seen that there is a consistent value-practice gap ranging from 35.7% to 10.2% across all eleven items in this section. Teachers highly value research practices and research knowledge again consistent with the finding from Levin et al. who reported that educators had a 'high level of receptivity to research' (2010, p.4).

Discussion

This research shows that teachers place a high value on the use of research practices. Eight out of the eleven items had value scores above 50% whereas only two practice scores out of the eleven items were below 50%. This backs up educators' high receptivity to research, which has been noted by Levin et al. (2010). This result also shows the importance of asking teachers not just about their practices but also about their value of those practices.

A number of value-practice gaps have been reported and these show that teachers are unable to use research practices and yet they still place a high value on these practices. Teachers have the aspiration to use research practices and hence use research in their practice but are unable to do so or are constrained in this aspiration. Thus for the item 'you engage in research focussed discussion with colleagues' the value-practice gap is 34.9% meaning teachers value this highly but are constrained to do this in their daily work James and Pedder state that teachers 'need to find ways of resolving tensions between external constraint and their professional beliefs in a way that does not comprise their educational values' (2006, p.131). There were no negative results for a value-practice gap score across any the items. There are no research practices that teachers carry out that are they value less than they are able to practice. When it comes to the use of research practices, teachers are not being coerced into doing things that they do not value.

In the sample there are 19.8% of teachers, studying for a Masters degree and 16.2% reported that they had already completed a Masters degree. Also nearly 50% reported that they were engaged in post graduate study of some kind. These figures show that these teachers may be more engaged in research through their postgraduate studies and this may explain the high value that these teachers place on the use of research practices.

Overall this study shows that teachers are interested in research and research practices and value these things even if they are unable to engage with them in their work. Teachers are interested in research evidence but are constrained as to how they can base their practice on research evidence.

If teachers are allowed the time and space to engage with research evidence, to be critical of it and reflect on it, then there is a better chance that they can base their practice on research evidence. This challenge has been identified by James and McCormick:

'The key challenge for [school] leadership is, therefore, to create the space and climate for school staff to reflect on and share aspects of their practice. This includes encouraging and stimulating dialogue and risk taking. In this way, innovations can be tested, embedded and sustained. Without it, they remain surface changes that decay and disappear when the next initiative comes along.' (2009, p.982).

Acknowledgements

I would like to thank Marilyn Leask and Andrea Raiker for their help in this study.

References

• Ajzen, I. & Fishbein, M. (1980). Understanding attitudes and predicting social behaviour. Englewood Cliffs, NJ: Prentice-Hall.
• Barber, M. & Mourshed, M. (2007). How the world's best-performing school systems come out on top. London: McKinsey and Company.
• Bunkett, D. (2001). Influence or irrelevance: can social science improve government? London: Department for Education and Employment.
• Cooper, A. & Levin, B. (2010). Some Canadian contributions understanding knowledge mobilisation. Evidence and Policy, 6(3), pp.351-369.
• Cooper, A., Levin, B. & Campbell, C. (2009). The growing (but still limited) importance of evidence in education policy and practice. Journal of Educational Change, 10(2-3), pp.159-171.
• Cordingley, P. (2004). Teachers using evidence: what we know about teaching and learning to Practice in Education. Maidenhead: Open University Press.
• Cordingley, P. (2008). Research and evidence-informed practice: focusing on practice and practitioners. Cambridge Journal of Education, 38(1), pp.37-52.
• Department for Education, (2010). The Importance of Teaching. London: Department for Education. (Available online at https://www.education.gov.uk/publications/standard/publicationdetail/page1/CM%207980)
• Department for Education (2011). School workforce in England 2010 (Provisional November, 2010). London: Department for Education. (Available online at: http://www.education.gov.uk/rsgateway/DB/SFR/s000997/sfr06-2011v5.pdf )
• Foley, D. & Hargreaves, D. (2003). The Production of Knowledge in Different Sectors: A model and some hypotheses. London Review of Education, 1(1), pp.7-19.
• Hammersley, M. (1997). Educational research and teaching: A response to David Hargreaves' TTA lecture. British Educational Research Journal, 23(2), pp.141-161.
• Hammersley, M. (2001). Some questions about evidence-based practice in education. Paper presented at the symposium on 'Evidence-based practices in education' at the British Educational Research Association (BERA) Conference, University of Leeds, UK, September 13-15.
• Hargreaves, D. (1996). Teaching as a research based profession. London: Teachers Training Agency Annual Lecture.
• Hillage, J., Pearson, R., Anderson, A., and Tamkin, P. (1998). Excellence in Research in Schools. London: Department for Education and Employment/Institute of Employment Studies.
• James, M. & Pedder, D. (2006). Beyond method: assessment and learning practices and values. Curriculum Journal, 17(2), pp.109-138.
• James, M. & McCormick, R. (2009). Teachers learning how to learn. Teaching and Teacher Education, 25(7), pp.973-982.
• Levin, B. (2004). Making research matter more. Education Policy Analysis Archives, 12(56), pp.1-22
• Levin, B. (2008). Thinking about knowledge mobilization. Paper prepared for an invitational symposium sponsored by the Canadian Council on Learning and the Social Sciences and Humanities research Council of Canada, Vancouver, May.
• Levin, B., Cooper, A., Mascarenhas, S. and Thompson, K. (2010). Using interventions to increase knowledge mobilization in Canadian secondary schools. Paper presented at the American Educational Research Association Conference, Denver, April.
• Likert, R. (1932). A technique for the measurement of attitudes. Archives of Psychology, 140, pp.1-55.
• MacBeath, J. & Mortimer, P. (Eds) (2001). Improving school effectiveness. Buckingham: Open University Press.
• McIntyre, D & McIntyre, A. (1999). Capacity for Research into Teaching and Learning. Final report to ESRC Teaching and Learning Programme. Cambridge: School of Education, University of Cambridge.
• Nutley , S. M. & Davies, H. T. O. (2000). Making a reality of evidence-based practice: some lessons from the diffusion of innovations, Public Money and Management, 20(4), pp.35-42.
• Nutley, S., Walter, I., & Davies, H. (2007). Using evidence: How research can inform public services. Bristol: Policy Press.
• OCED. (2010). Education at a Glance 2010: OECD Indicators. Paris: Organization for Economic Co-Operation and Development. Available at http://www.oecd.org/edu/eag2010.
• Pedder, D., Opfer, V., McCormick, R., & Storey, A. (2010). Schools and Continuing Professional Development in England – State of the Nation research study: policy context, aims and design. Curriculum Journal 21(4), pp.365-394.
• Pollard, A. (2008). Knowledge transformation and impact: aspirations and experiences from TLRP. Cambridge Journal of Education, 38(1), pp.5-22.
• Pollard, A. and Oancea, A. (2010). Unlocking Learning? Towards Evidence-informed Policy and Practice in Education. Report of the Strategic Forum for Research in Education, 2008-2010. London: SFRE.
• Procter, R. (2011). Knowledge Mobilisation: a dual scale questionnaire to determine teachers use and value of research knowledge. Poster presented at Outcomes and Impacts, University of Bedfordshire, 5-6 July 2011, Luton.
• Procter, R. (2012). Teachers and research: the gaps between their values and their practices. Poster presented at the Going for Gold Conference, University of Bedfordshire, 3-4 July 2012, Luton.
• Robertson, P., Sammons, P., Thomas, S. & Mortimore, P. (2001). The research design and methods, in: J. MacBeath & P. Mortimore (Eds) Improving school effectiveness. Buckingham: Open University Press.
• Shahar, E. (1997). A Popperian view of 'evidence-based medicine. Journal of Evaluation in Clinical Practice 3(2), pp.109-16.
• Thomas, G. & Pring, R. (Eds.) (2004). Evidence-based Practice in Education. Maidenhead: Open University Press.
• Tooley, J. & Darby, D. (1998). Educational research: a critique. A survey of educational research. London: Office of Standards in Education.
• Webb, R., Vulliamy, G., Hamalainen, S., Sarja, A., Kimonen, E. & Nevalainen, R. (2004). A comparative analysis of primary teacher professionalism in England and Finland, Comparative Education, 40(1), 83–107.
• Whitty, G. (2007). Education(al) research and policy-making. In L. Saunders (Ed) Educational Research and Policy Making. London: Routledge.

[1] www.mapofmedicine.com