Paul Henderson: Thinking aloud on Pedagogical Research

I really look forward to contact from Paul. He is a peripatetic music teacher and home educator. He knows and understands manistream schooling and the elective home education sectors.  He has contributed a wide range of throughtful and challenging articles and blog posts over the years. Well he’s currently ‘on a roll’ and has penned a number pieces which we post in a series here. Our many thanks Paul.

Virtually all Pedagogical Research is Based on Junk Science

 Introduction

Pedagogy in its purest theoretical sense is certainly not junk science but the educational research informing what commonly passes for pedagogy in school classrooms raises some serious doubts. It would appear that the only kind of pedagogy that governments are interested in and consequently the only type of pedagogical research that attracts funding is the type that makes cosmetic improvements in league table results. All teaching techniques taught to classroom teachers are designed for and developed out of classroom practice with a view to improving prescribed educational outcomes, yet practically everything that is known about human learning tells us that the classroom is far from an optimal learning environment. One serious flaw in advocating improved classroom pedagogy as a way of increasing the effectiveness of learning in a general sense is that improved classroom pedagogy can only be shown to improve students’ learning in situations where it is not compared to any kind of control group. Science of this type, which does not include a control group, is commonly referred to as junk science.

The research that would validate classroom pedagogy as a true science by functioning as a control group is relatively scarce, attracts little funding and is associated with an uncommon form of learning, yet it is the only accurate and reliable benchmark indicator of the true extent to which any learning institution that utilises classroom teaching affects its student’s learning. Without considering the evidence from a control group, no scientifically reliable conclusions can be drawn from any form of research that looks into ways of improving classroom learning because it cannot determine whether any changes in students’ learning can be attributed to classroom pedagogy or some other variable. Put simply, the only way you can determine what effect classroom pedagogy has on learning is to compare the learning of two similar groups of students, one who have learned as a result of classroom pedagogy, and one who have not. Research into education by means other than school can provide the scientifically essential control group missing from most of research relating to pedagogy. It is deeply concerning that all research into improving the educational outcomes of schooling in developed countries is not properly controlled by comparing its findings to the findings of research into learning by means other than school. Surely there would be huge benefits to be gained from global educational research that is not so scientifically unreliable, especially in light of the tantalising glimpses of ‘what is and what might be’ that research into alternative learning reveals. In this piece I will compare the findings of school classroom based research (with particular regards to formative assessment) with the findings of research on those educated by means other than schooling, and postulate that a possible explanation for the significant educational gains found in alternative learning environments are due to the type of formative assessment utilised by practitioners of alternative learning (such as home educating parents), irrespective of their educational philosophy, and probably without even realising it.

 Educational expectations 

Every educational provision, whether it is mainstream, independent or alternative, has three clear and strongly desired expectations placed upon it by every parent. Different parents may rank them in different orders of importance but the big three highly desirable attributes that all parents expect their children to benefit from through expanding their educational horizons are;

1. Emotional and psychological well being (happiness).
2. Good social skills.
3. High achievement.

All research studies have shown statistically that, on average, home educated children’s results from a range of globally recognised benchmark tests are significantly above the average results of their school educated peers in all three of these essential key aspects. There is a great range of educational philosophies and approaches within home education, but it has one common characteristic which differs from standard classroom pedagogy, and that is the type of formative assessment that home educating parents utilise, probably without even realising it.

 Formative Assessment

Educationalists have been extolling the virtues of the theory of formative assessment for decades and in recent years this has intensified further with classroom teachers being trained extensively in putting the theory into practice while being closely monitored by their superiors to ensure that it becomes an integral part of their teaching. The basic lesson to be learned is that the more formative assessment teachers do, the better, more efficient and more effective is their teaching. All of the formative assessment teaching techniques taught to classroom teachers are designed for and developed out of classroom practice. The classroom is sometimes referred to as ‘the black box,’ taken from the title of a (now quite renowned and available as a free download) booklet on formative assessment entitled ‘Inside the Black Box.’

The basic principle of formative assessment is that teachers should elicit, using a variety of techniques, accurate evidence of learning from pupils and use that evidence to guide and shape the course of further learning. It might be suggested that anyone making a living by advocating such an idea should perhaps do very well as a contestant in the television program ‘Mastermind’ with their special subject being ‘stating bleeding obvious’ as the ‘Fawlty Towers’ character Basil Fawlty once commented about his wife Cybil when she kept repeatedly stating the obvious.  An immense amount of research has been done on formative assessment in the classroom and it all points to the fact that it does indeed lead to vastly improved outcomes. This is all very well and good, but all of this research, in its desperate search for improved school based learning outcomes, completely misses the point if optimal learning is its true goal. It doesn’t even begin to address the most potent kind of formative assessment which is clearly not so ‘bleeding obvious’ to mainstream educational researchers, but has been alluded to by a small number of educational research academics who have had a far deeper insight into the true nature of learning.

In order to uncover the key differences between the formative assessment techniques commonly used in the classroom and the far more potent variety of formative assessment rarely seen inside the ‘black box,’ we need to take a close look at classroom based learning systems and compare them to one-to-one and small group learning systems.

 Classroom Based Learning Systems

The interactions between classroom teachers and their students may be regarded as continual in that teachers continually attempt to interact with as many students as possible during the course of a lesson. It is not possible to continuously interact with any one pupil or small group of pupils throughout the whole duration of the lesson without entirely neglecting the rest of the class, therefore a teacher’s time must be continually managed between individual or small groups of pupils to achieve optimal overall interaction. When it comes to the accurate elicitation of evidence of learning from individual pupils, teachers must take ‘snapshots’ using various classroom formative assessment techniques to get an accurate as possible overall assessment of the class with a view to using that information to shape future learning. The more ‘snapshots’ a teacher can take the more up-to-date, accurate and reliable the information regarding the instantaneous states of pupil learning. This process may be called ‘static formative assessment’ since it involves sampling static information at various time intervals. The difference between a static process and a dynamic process is that static processes take place at various instances in time whereas dynamic processes take place continuously over time. This is a bit like the difference between a snapshot and a video. If teachers were to take snapshots of their pupil’s learning at the rate of 20 frames per second then this may approximate to a dynamic process in a similar way that film does. If more formative assessment means better teaching and learning then, with reference to the film analogy, it might be said that dynamic formative assessment is at the very least 20 times more potent than even the very best forms of static formative assessment that only the very best of classroom teachers are able to utilise.

 One-to-one or Small Group Learning Systems

In one-to-one or small group learning environments such as home education, instrumental music instruction, sports coaching, private tutoring and the like, the interaction between learning guides and learners may occasionally be continual but, far more commonly, it is continuous; and therefore the kind of formative assessment that is utilised is dynamic – meaning that learning guides (coaches, instructors, private tutors etc) alter the course of learning in immediate and real time response to learners’ constantly changing needs precisely as they are perceived to change. By comparison it is clear that there are key differences in approach between the teaching and learning techniques used in the classroom and those used in one-to-one or small group learning, and these key differences manifest themselves in the differing types of formative assessment most commonly found in each of the respective learning environments; namely static and dynamic.

To illustrate the key differences between static and dynamic formative assessment let’s take the example of the differing approaches found between classroom music teachers and instrumental one-to-one or small group peripatetic school music instructors (who travel to various schools, take learners out of their classrooms, and teach them to play specific instruments using specialist knowledge and skill that classroom teachers may not have). The surest way for music instructors to find out what learners have really learned is to observe learners putting into practice what they have learned by listening to them playing their instruments. All the information that music instructors impart to learners is designed to be used by the learner in order to play a musical instrument therefore the ultimate and only true test of the extent to which this information has been assimilated will be displayed in the way the students play their instruments. This is really the only suitable, efficient, reliable and accurate way for instructors to assess what learners have really learned. Learners may think they can accurately articulate the extent of their learning but this may be unreliable and inaccurate since, while learners may understand that they may be having difficulty playing a particular section of music, they will not know if this difficulty stems from a lack of intellectual comprehension of the piece or from a physical technical inadequacy. Conversely they may report that they can play a piece brilliantly but when they actually play the piece their performance is far from brilliant.   Sometimes learner self evaluation may conflate musical quality with personal musical taste but every student is entitled to have their musical taste, which is and should be subjective. Clearly the most reliable and accurate form of assessment comes from the instructor listening to and observing how the instrument is being played.  An experienced instructor will be able to tell in an instant whether a problem stems from an intellectual or physical source or where students’ egos are bigger than their abilities; ‘let the instrument do the talking’ as some instructors say, ‘the proof of the pudding is in the eating.’.

Many of the formative assessment techniques that instructors use are dynamic. They change in real time as a pupil is performing and require continuous multifarious (VAK etc) purposive interaction with students – a luxury rarely afforded to classroom teachers. Dynamic formative assessment in instrumental instruction is entirely dependent on how the student performs the piece and will change from student to student. It is an entirely personalised process. Classroom teaching can never achieve such levels of personalisation. For example an instructor may give a student continuous feedback on elements such as pitch, rhythm and tone by singing along with the student’s performance (to allow the student to adjust his or her pitch in real time as the piece is being performed to that which the instructor is singing), clapping along with the student’s performance (to make the student more aware that the rhythm is drifting slightly or the syncopation is out, in order for the student to use the teachers feedback in real time to adjust their playing accordingly) or by the teacher playing the student’s part in unison (so that the student can directly compare tone and other musical elements in order to make real time adjustments for the better to their own playing). Asking a student to stop playing and to verbally articulate their understanding of their learning is a bit like pulling a plant up by the roots to check if it is still growing. It is disruptive to learning and unnecessary. The irony is that music instructors, sports coaches, private tutors and the like utilise the vastly superior form of formative assessment leading to far deeper and personally meaningful learning quite naturally without ‘benefiting’ from the decades of educational research that has gone into irrelevant classroom teaching techniques. Most one-to-one and small group instructors and coaches are not qualified teachers; they are just those with a recognised ability who just do what comes naturally when it comes to teaching – and what comes naturally to them turns out to be infinitely more suitable, efficient and relevant than all of the extensive myriad of pedagogical techniques derived from decades of classroom based research. The irrelevance, unsuitability and inefficiency of classroom teaching techniques is clearly demonstrated even further by how quickly qualified classroom teachers drop them in favour of purposive interaction when they do private tutoring in the evenings. It readily becomes abundantly clear to them that all of the classroom teaching and management techniques taught to them at teacher training college and in-service days are entirely inappropriate when teaching in a one-to-one or small group setting.

Home Education

In home education, as in one-to-one or small group music instruction, it has been noted by practitioners and researchers alike that it is a regular occurrence for learners to be around two years ahead of classroom age-stage expectations by the time learners are around the age of sixteen irrespective of what educational philosophy was adopted. Mainstream educational philosophy tends to fall into either the traditional or the progressive camps and this is mirrored in home education by ‘homeschoolers’ (more authoritarian, pedagogical and didactic in style) and ‘unschoolers’ (more anti-authoritarian, andragogical and autodidactic in style) but research shows that the improvement in achievement appears to be the same no matter what educational philosophy is adopted. The one difference that may account for the correlation in improvement in achievement is that all home educators, irrespective of their educational philosophy, utilise far higher levels of dynamic formative assessment. Just like in music instruction, there are minimal classroom management and behavioural issues and learning happens through a process of natural continuous multifarious purposive interaction. While the unsuitability and inappropriateness of contemporary classroom teaching techniques may not be so obvious to mainstream academic educational researchers, I can imagine many home educators queuing up to award me with a PhD in stating the bleeding obvious for the mere mention of it, but please bear with me while I look at this from one more angle.

 Evolutionary Development

Most home educators don’t know or care that what they do may be referred to as dynamic formative assessment; all they know and care about is that if they do what comes naturally according to their varying personal attributes (in terms of the personal level of authority with which they are happy, along with all their other personal attributes) it appears to work very well indeed no matter what you call it. This is because humans have naturally evolved to learn in one-to-one or small group situations over millions of years and, given half a chance, will do so quite naturally, suitably and efficiently in today’s information rich society. It is a behaviour which is written into our DNA and is as natural as breathing. Classroom teaching and learning and its associated pedagogy is an extremely recent phenomenon in comparison to the timescale of evolution. It is simply not the optimal way we have evolved to learn and goes entirely against the ‘grain of the brain,’ which is why it is so unsuitable and inefficient for almost all learners when compared to the suitability and efficiency of one-to-one and small group learning environments. Methods of educating other than schooling are often referred to as ‘alternative’ but, ironically, when mass schooling is considered from an evolutionary perspective it is a singular misfit. In the grand scheme of things it is schooling that is a radically alternative educational experiment which has clearly shown through decades of persistently recurring and predictably reproducible poor results that it is a significantly inferior learning environment compared to the historical norm of one-to-one and small group learning environments. Improvements in schooling’s pedagogy, technology, curricula and budgets appear to have made no improvement in the competence of its ‘learners’ (in his book ‘Weapons of Mass Instruction,’ John Taylor Gatto provides ample evidence to prove that they have actually made things worse).

 Conclusion

Virtually all academic pedagogical researchers are classroom teaching specialists and consequently the teaching techniques they develop are not optimised for use in teaching and learning outside of the classroom e.g. in instrumental instruction or home education. Virtually all academic pedagogical researchers have no formal teaching experience, have done no research, and have published no papers or articles or books directly relating to teaching and learning approaches outside of the classroom using primarily extracurricular aims. Consequently such research is quite scarce but it does exist and it is interesting to note that it consistently finds that those in developed countries who have been educated outside ‘the black box’ by means other than schooling have significantly higher levels of achievement (usually around two years ahead of classroom age-stage expectations) – exactly in line with what music instructors (specialists in teaching outside of the black box) regularly experience. This strong correlation suggests that perhaps the ‘black box’ is not the place to look if raising educational achievement is your true aim.

Most parents understand that, as a day care facility, schooling provides an equal opportunity for all to experience bullying and anti-social behaviour but, judging by the startling amount of parents who employ the services private tutors, private after school activities and the like, and by the fact that many of our very best selective schools openly admit that their successful applicants are routinely privately tutored towards their entrance exams, schooling, as a sole educational provision, falls well short of meeting the needs of a great many of its learners, and consequently does not lead to equal opportunity for all in a broad educational or future career sense. The parents who can afford to do something about the educational short fallings of schooling by employing private tutors and the like clearly do not hesitate to do so, thus magnifying the divisive socioeconomic inequality that schooling perpetuates. Parents of less financial means may have good cause to conclude that they only have one of the following three choices; 1) handicap their children by using school as a sole educational provision which may expose them to the significant risk of having to live the rest of their life in the virtual caste into which they were born (see the shocking Sutton Trust reports on how schooling fails to increase social mobility), 2) limit the damage schooling does to their children by employing a more home educational approach outside school hours and perhaps explore the local availability of flexi-schooling (if living in Scotland – it is now illegal in England and Wales) or, 3) fully exploit the well proven social, academic, well-being and social mobility benefits of home education. Interestingly, all research into home education has shown that it does not require parents to have any specialist knowledge, teaching qualifications or prior teaching experience in order for it to be completely successful, and offers the greatest gains to those families who are less well off.

 Further Reading

An instrumental music service (IMS) is a learning system within a learning system and what I’ve done a lot of in this piece is to compare the classroom based learning system with the non-classroom based IMS learning system. For further reading on the comparison of learning systems read (former special professor of education) Roland Meighan’s book ‘Comparing Learning Systems.’ His ideas on the matter are summarised here. http://www.rolandmeighan.co.uk/resources/Learning+Systems.pdf

Articles collating research on teaching and learning outside of the ‘black box’ can be found here.

http://www.hslda.org/docs/news/200908100.asp

http://www.giftedness.org/Giftedness/results_home_education.html

http://www.home-education.org.uk/research/review.htm

Notes. The terms ‘static formative assessment’ and ‘dynamic formative assessment’ are new and were first coined by the author in this piece to illustrate differences in learning and teaching approaches in different learning environments. Of course dynamic formative assessment is only one minor facet of the great many highly prized attributes, privileges and freedoms regularly and uniquely experienced by home educators.

The word ‘education’ is a general umbrella term involving every area of human activity. It has no singular specific and definitive meaning or context that can be optimally applicable or best suited to any particular individual except for that defined by that particular individual, therefore any set of criteria acting as a prescription for a ‘successful education,’ whether it is formulated by state authorities or otherwise, or where it has been shown to be statistically optimal according to a prescribed criterion, is purely arbitrary. Probably the greatest strength of home education is that it offers the most flexibility to achieve a successful education according to the definition of those who seek it.

Some people, particularly experienced home educators, may feel that this piece has been a somewhat convoluted exercise in ‘stating the bleeding obvious,’ so as a bit of light relief the original aforementioned ‘Basil Fawlty’ quote can be viewed here.

http://www.youtube.com/watch?v=-1Y7VKYWdUo                                              

 Paul Henderson, March, 2013