I entered into the field of education at a particular time in it’s history. Institutions have been long established that package learning into title, diploma, and degree programs. These institutions are governed, implicitly and explicitly, by dozens of other institutions such as grant providers, civil governments and accreditation boards, each with their own agendas. Everyone is accountable to one or more parties in this system. I am accountable to a design budget.
My dilemma is really the same as the others’ in this system: What can I do with the resources I have to produce the best educational outcome? But what exactly is the “best outcome?” I’ve used many versions of “best” in the past. I want the “deepest understanding” or the “richest learning experience.” How about some “good grades” or “higher performance than other instructional methods?” One colleague of mine said it simply, “It is all about efficiency.” We want the largest return for our investment based on how we define return.
So let’s define our return in terms of the kind of student we want to produce. If we can define the outcome then we can design the treatment, right? If I’m not mistaken, this is the moment when numbers will begin to poke their little heads up. At first, we are able to use qualitative statement to describe this student. Here are a few:
“The student will be able to look at a list of metals and make an informed guess at which will best suit a particular engineering design challenge”
“The student will be able to navigate in a french speaking environment”
“The student will have a positive experience with the course material”
What happens next is that we start grouping and counting. How often did the student choose the right material? What kind of french environments? What percentage of students had how good of an experience? Two things just happened here according to Desrosieres and Popkewitz. First, we just created systems of equalities that allowed us to group individual things into categories. Second, we began to record abstractions of those categories by translating them into numbers, which have no meaning in of themselves.
If we are going to group things together, one way is to define the attributes of a group member and look for similarities that exist in in the individual cases. If enough attributes are present to a high enough degree, that particular item is considered part of the category. Thank you Descartes for giving us the tools of deconstruction.
This process has the side effect of creating an equality between members of the group. The fact that they have been combined makes them the same. By following this pattern, the wonders of sociology were created (Desrosieres 1994). We can measure things like poverty, unemployment, working class people, good teachers, bad students, etc. Along the way we have actually fabricated types of people (Popkewitz 2009). We have constructed a notion of a thing that does not exist, an abstract definition of a combination of attributes to stand in for a human being.
Is there anything wrong with this? Morally, I’m not sure. However, it sure is helpful. Galton’s wood board with a falling steel ball converts a random series of coin flips into the famous normal curve abstraction. This abstraction then allows us to see things in a new way and find patterns were we didn’t see them before. Abstraction is the basis of logic. It is the tool in which we can make meaning out of new experiences, record music with marks on paper, create computers, make strategies on how to get out of a fight with you wife and many other wonderful things. Once we have something as complex as a student converted into something as simple as a catagory, we can count, average, find means and calculate predictions. Like the normal curve, these tools give us a new grids of intelligibility (Popkewitz 2009). On the other hand, abstractions are not mearly the subjects of our reason. Once they have been given a name they become objects themselves and push back on us.
For example, when governments write number based rules to establish wealth and taxes, these categories may actually change the way citizens make decisions and do things. I saw a clear example of this when wealthy houses in Nicaragua and India would leave a few bricks out of their building so the house would not be “complete” and therefore exempt from tax. Numbers and categories are also pushing back when students enroll in a particular university based on it’s rankings or a student is not allowed to graduate based on a standardized test score. The stories are countless and the space of many contested critical analysis. According to Heran in L’assi se statistique de la sociologie, as abstractions are linked to hard social facts such as institutions, laws and customs, they take on a substantial existence of their own.
So to come full circle, I believe that statistics become the vehicle for defining the best educational outcome due to the system of accountability and expectations of efficiency that surround the designer. In instructional design we often start by creating an abstract description of a desired outcome. We then design a treatment and asses whether it met the outcome, iterating along the way. The danger in this outcomes based approach is that it will always tend toward a success description in terms that can be numerically or categorically measured. The critical designer must now determine what assumptions are used to create the required categorical equalities. They must also determine how the definition of the outcomes may become actors themselves, changing the system by their very existence.
This last point is quite revenant to my current design investigations. Students will always be tempted to game the system, like the homeowners in Nicaragua, to use the systems of assessment to their own advantage. It may be that if we continue to represent their learning progress only in terms of categorical, statistical abstractions, the actual outcomes may be very different that those predicted, or even within the vision of our mathematics. Could it be that if educational designers are aware of this fact, they could create a design where “playing the game” will actually embody the desired learning experience? Either way, for better or worse, education research has placed much of its faith in numerical studies for its objective research. If such an educational design does ever exist, it had better prove its value numerically.
AERA. Standards for Reporting on Empirical Social Science Research in AERA Publications. (2006) pp. 1-15
Baker. Risk, Insurance and the Social Construction of Responsibility. Embracing Risk: The changing culture of insurance and responsibility (2002) pp. 33-51
Desrosieres. (1994): How to Make Things Which Hold Together: Social Science, Statistics and the State. pp. 15
Popkewitz. (2009): Numbers in Grids of Intelligibility: Making Sense of How Educational Truth is Told. pp. 1-24
Heran, F. (1984): ‘ L’ assi se statistique de la sociologie’ in Economie e t Statistique, 168 (juillet-aout), 23-26