Et tu, Brute: A TESOL Tragedy

Et tu, Brute: A TESOL Tragedy

Last month I took four weeks to complete one of the more reputed TESOL qualification courses available (no, not that one, the other one…).The main purpose it served me was to open my eyes to the truth about the quality of these courses.

My negativity is perhaps best justified by the irony that emerged from one of the particularly disappointing elements of the programme. Towards the end of the final week, a session was scheduled to talk about assessment in the ESOL classroom. I was especially looking forward to this session, because I believe it to be one of the most important topics of discussion in education. As such, I was hopeful for some new insights into assessment philosophies and techniques.

Disappointment

Throughout the four-week course, each day was divided into three ninety-minute sessions in the morning and teaching practise in the afternoon. In the session on assessment, of the allotted ninety minutes, approximately twenty were spent on the trainer telling us about the various tests that they sell at their institution, and even with that the session finished over 30 minutes early. That meant that less than half of the scheduled time was actually used to talk about how to think about, design and administer assessment in a TESOL setting.

Once the marketing spiel was out of the way, the trainer made sure to preface the pending tutorial with the caveat that assessment was not an important thing for us to talk about because, “basically, whatever institution you end up working at, the management will handle it and all you will have to do is monitor the students while they take the test.” Thus began the rise of my ire. Especially when this was reinforced with the reminder that we did not need to demonstrate an understanding of assessment in order to pass the course anyway—my main response to that is, why put it on the schedule in the first place just to say that it isn’t important?

It is also worth noting that, based on my almost-a-decade of experience teaching and assessing in various institutions, it is simply not true that teachers do not have any responsibility for testing their students. I feel that should be obvious, but apparently it isn’t, so now I have said it.

Once we had been fully warned of how unimportant assessments are, the trainer went on to give us handouts with an exercise to match terms with definitions—terms like summative, formative, diagnostic… The participants, through a combination of guesswork and checking against my answers (I should point out here that all other participants were brand new to teaching) were able to mostly match the definitions to the terms. After a few minutes to do so, the trainer encouraged us to compare our answers together. By the time we had been through this collaborative process, anybody who had incorrect answers had been able to check them—largely against mine—and correct them.

The trainer then applauded everybody on getting the right answers and quickly went through some of the terms to check for understanding. Most amusingly, the term that the most time was devoted to was ‘backwash’. For those of you that don’t know, ‘backwash’ is the term for teaching course content that is tailored specifically to ensure that the students have the answers for the test they will be taking at the end. This is of course considered a negative effect, and the trainer was sure to make that clear. Of course, I agree with the sentiment, but for them, all it was was sentiment.

A Widespread Problem

The session on assessment was just too sweet an irony to pass over. However, on countless occasions throughout the course, the trainers would cut short a discussion amongst the participants or brush off questions by pointing out that it was not important for passing the course. As the programme progressed, it became increasingly clear that the trainers were interested in instructing their trainees how to get their the certificate far more than how to actually be good teachers once the course was over.

Another particularly frustrating example of this was an exchange between the trainer and myself regarding lesson planning. Again, a handout was distributed (this was the main mode of delivery throughout the course after reading out bullet lists from Powerpoint slides) listing the reasons for why it is important to plan lessons. Once the sheet had been read aloud, and the trainer was ready to move on, I raised my hand to add the point, just as something to think about, that lesson plans were also valuable in the case of covering; if a teacher is sick, etc. and needs cover, it is very helpful if there is a lesson plan to work from. The trainer responded by saying that that wasn’t really important and that often teachers didn’t make lesson plans anyway. I pointed out that, since this was a session on the importance of planning, perhaps it would be germane to encourage best practice rather than resign to the fact that teachers are often lazy. He responded again saying that most teachers develop their own shorthand when planning that can’t be understood by other teachers, so not to bother with it. He reasserted that the reasons that mattered were those on the handout, and that to discuss anything else was not necessary for the certificate.

This experience, apart from significantly degrading my valuation of the qualification, got me thinking more about the problem of backwash. Even on a course where the sentiment was that backwash is a bad thing, the programme itself fell victim to the problem in quite a severe manner. In their case, it seems it was mostly an issue of licensing. The institution holds a licence to offer this course under the auspices of the course’s creator, and for the sake of quality control there are a number of standards and criteria in place that must be met. As such, the delivering institution needs to make sure that it covers certain ground in just such a way in order that they do not have their licence revoked.

In other cases, backwash might be generated by league tables, teacher accountability and statistics to name but a few. It seems that, given long enough for a programme to establish itself and lay out its standards, backwash is an almost unavoidable phenomenon. In fact, perhaps the only way to avoid it is to abolish standards altogether.

Now, there’s an idea! Let’s think about that for a moment; what would that look like?

Where Backwash Comes From

In most parts of the world, for as long as education has been formalised, it has been standardised. This is to say that when education moves out of the home into the schoolhouse and the responsibility shifts from the parents to the teacher, some kind of control is needed to ensure that the education being provided is good education. How are these controls implemented? Largely by testing to see how well the student is learning and then adjusting the teaching accordingly. Sounds good so far.

The problem though arises at two points within that procedure. First, the testing and secondly the adjusting. This cycle of testing, adjusting, testing again, adjusting some more and so on creates a feedback loop that ends up operating on the wrong subject. That is to say that, the more reliance we place on the test results, the more we end up valuing teaching that can be tested in such a way.

This usually means that somebody (or some body) decides upon a set of things to be achieved by the end of a programme and then writes a test to see how well those things have been achieved. Teachers are then expected to teach the students everything on the list, and their success at doing so will be confirmed by the tests. Again, this doesn’t sound too bad in theory. But what if the things that we are testing are not really the things that we should be focusing on? And what if the things that we should be focusing on aren’t that easily assessed in a formal examination setting with standardised results?

For example, since this article started talking about ESOL, let’s say that our primary goal at the moment of setting up our programme is to teach our students English. We first have to decide what we want the students to achieve by the end of that programme, and then we have to write a test to make sure that those things have been achieved. Ideally, with regard to intended outcomes, we would say that, for one, we want our students to be able to use English effectively to communicate their needs and understand the needs of others. That sounds good, but how do we test it? As it stands, it seems quite vague, so what elements can we draw out to focus the testing on? Well, what makes for effective communication? Accurate expression, perhaps? That is determined by correct use of language systems and lexis, so we can test for that. And what makes for successful understanding? Perhaps an ability to accurately identify the meaning of something that has been expressed, e.g. the meaning of the words we hear or read, which is also something fairly easy to test for.

So, now we write a grammar test and a basic comprehension test, perhaps both in multiple choice format as that is the easiest and most efficient way to extract results, and we tell our teachers that we want them to teach their students to be able to communicate using English, and that they will be tested afterwards to check whether the teaching has been effective. Perhaps this still sounds okay to you so far. But it shouldn’t take too much scrutiny to see the embryo of a problem developing…

The teacher in this scenario might look at the intended outcome—for students to be able to communicate using English—and ask what that means. What exactly do my students need to be able to do in order to achieve that? Where better to look than at the test, which has already been written in order specifically to highlight the elements required to achieve these outcomes. Whereby, the teacher will see that the elements to focus on are accurate use of structure, accurate use of lexis and accurate identification of meaning.

Now the teacher knows exactly what to teach. And if the students score badly in their test at the end of the programme, then the teacher needs to find out how better to teach these areas of accuracy and make sure that students learn them and remember them. Some focus on form and some memorisation activities such as drilling should do the trick. That will ensure that the students get good scores, and good scores mean that teaching and learning has been successful.

This simulation that I have outlined completely discards with any issues such as cheating and intentionally lazy teaching and taking shortcuts and instead indicates how the problem of backwash can arise by purely natural processes with all the best will in the world. We can see how, in just a few short steps, the real intended outcomes that we began with have been forgotten in favour of the minute elements thereof that are most easily tested. The road to Hell, etc., etc.

Reimagining Standards in Education

What needs to be upheld is the importance of the intended outcomes identified at the very beginning. Somewhere in the earliest moments of designing an education system, I am sure these intended outcomes existed. But in many cases, they are now all but unidentifiable; all that is left is the push for higher scores on tests that focus on the wrong things. So, I suppose the question is, is it possible to focus on the right things?

The ideological concept of an education system—and the only model of education I can fully support—is one designed to prepare the learners for life after graduation. Syllabus design should be embarked upon after assessing the political, social, economic and professional landscape of the time, as well as forecasts for the future of these fields, and should be implemented in such a way as to give the learners the skills they will need to survive and thrive in the given environment.

This implies two important points: first, that an effective system of education should not be static—the world that we live in as professional adults is incredibly dynamic, with technological and scientific developments sculpting the landscape and social and cultural norms shifting constantly; as such, syllabuses should be reassessed regularly in the light of the environment that learners are likely to graduate into—and secondly, that many of these things will not be easy to assess against a standardised metric—progress and ability in the post-graduation world is demonstrated by achieving tasks, reaching milestones and overcoming challenges; it is therefore these things that learners need to be prepared for, but they cannot necessarily be effectively tested with a series of multiple choice questions.

Can Do Statements: An Alternative Standard

This is where “can do” statements come in. “Can do” statements are an entirely different way of looking at learner progress and are gaining traction widely in education settings both formal and non-formal, although in my experience they are often poorly implemented. The ideology of “can do” statements is to measure learner progress by simply looking at what they can do. The syllabus is structured primarily around skills and practically applicable abilities, and when a student demonstrates that they can do the things on the syllabus, they are ready to move on to the next unit/module.

An example of a can do statement in my field, English Language Teaching, is, “The student can ask for and give directions”. That seems very practical, perhaps; however, if you’ve been paying attention, it might also seem familiar. I have already said above that such an intended outcome is often the starting point, but it still somehow declines into the murky depths of standardised testing. As we saw earlier, the way this happens is usually by looking at the outcome and looking for easily measurable metrics, and in a number of the education settings where I have seen “can do” statements introduced, the same mistake persists.

To truly use “can do” statements as they were intended, this step needs to be somewhat elided. Instead of looking for metrics, the assessment should be based on one simple question: can they? Can my students ask for and give directions using English? How then can we answer this question without a standardised test and marks out of a hundred? Quite simply by letting the students demonstrate their ability. If we give the student the chance to use the new target language, then we can see how effectively they are doing so. If a student is able to receive and follow directions and end up where they want to be and is able to give directions and get the recipient to the correct place with nobody getting lost, then the answer is, yes: the student can ask for and give directions.

If the student cannot, then some more time is needed practising, and probably a different learning approach is needed to achieve the intended outcomes. This is quite daunting for many teachers, because it means that they have to take charge of the assessment process much more directly; the decision to move on is not based on calculated scores but on a genuine assessment of the students’ ability to survive and thrive in the world outside of the classroom. It means that each student is assessed on his or her merits, and  the feedback from this, unlike grades and percentages, is specific to the strengths and weaknesses of each individual.

A Network of Teachers

The backlash against this idea generally comes from the lack of standards implied. If each teacher is free to decide whether a student can or cannot “do” based on his or her own observations, then how do we maintain any sense of quality? If there are no standards, then how can we talk about aiming for high standards? Surely high standards of education is something we want to achieve?

There are a number of ways of responding to these questions, and I might return to them in detail in a future post. For now though, the primary response I want to focus on is not to think about this as a model implemented arbitrarily by individual and isolated teachers. While there will be no standardised tests delivering nationwide statistics and graphs of high and low scores, it does not mean that each class must exist in a bubble. Teachers might not be handed down pass/fail boundaries and quotas by the education board, but they also shouldn’t be capricious in their assessment and should avoid subjectivity at all costs. To prevent these problems from arising, such a model would rely heavily on mutual support and collaboration amongst teachers, at the smallest scale within an institution, and ideally across not only states or nations but the world.

Teachers and syllabus designers should interact with their counterparts from all over the planet to identify valuable learning objectives that will best prepare students for success after they graduate. Teachers should then share with each other ideas of how they develop these skills in their classrooms and how they assess their students for progress. This way, teachers continue to be held accountable for the quality of the education they are providing, but in a much more meaningful way than by ranking grades. Teachers can spend more time focussing on improving their methods of teaching and assessing—and work together to do so—rather than spending all of their free hours on administration and box-ticking.

Standardised testing enforces unrepresentative homogenisation across all geographical, cultural and economic borders and remains mostly stagnant year after year. By removing it, we stand to see learners developing the skills they will need relevant to the lives they are actually going to lead outside of the class. This is far more valuable than memorising lists of facts and figures that will help them pass their tests but not be much help at all once the tests have been and gone.

The TESOL course that I have just completed, and that I am sure many of my readers will have too, fell foul of the most dangerous error in education, and that is to be more concerned with how to ensure students pass the course itself than to create individuals who will be successful after they have graduated. This on a course which supposedly exists to promote the very ideas that I have talked about in this article. So maybe backwash is unavoidable after all. But perhaps the concept of backwash is not really the problem; perhaps the problem is more to do with what is being washed back. Instead of “teaching to the test”, as it were, perhaps we should be teaching to the challenges of life that our students are likely to face.

Have you any ideas for how to implement these kinds of teaching and assessing? Have you had any experience doing so in your own career? Have you faced challenges from senior members and standards-setting bodies when attempting to implement such an approach? Share your stories and opinions in the comments below.