While reading a bit of commentary on education policy recently, I was directed by the author to view what was referred to as a "study" by a major think tank, which he cited to support his argument. It occurred to me that the study was flawed in a number of ways, and that reason behind this was simple: the "study" was really just a polemical, ideological argument dressed up in academic clothes. It had researchers who went out and collected real data, analyzed it, and then presented conclusions. By coincidence(!), it fit perfectly into the ideological narrative of the think tank that funded it. I wondered whether if the study had reached different conclusions, it would have ever been published. Of course not. That would be at cross-purposes to the mission of the think-tank, which is generally to advocate for a particular political philosophy.
I then became interested in how common this practice was among other think tanks. We're all familiar with them as their representatives are frequently used by the media, generally to provide quotes for news articles or television pieces. Frequently, reporters will even cite these so-called studies the tanks put out. To the extent that they are thought of as anything more than propagandists, there is a serious ethical concern. These "studies" are designed to present entirely one-sided arguments that fit perfectly with the tank's pre-conceived ideology.
I took a brief look at four major think tanks to see how prevalent this practice of publishing biased "research" papers is. The Heritage Foundation, The Center for American Progress (CAP) , The Hoover Institute, and The Cato Institute, each had sections devoted to "publications". Both Hoover and Heritage offer what they call "research", while Cato presents "studies" and CAP presents "reports". Each publication presented an entirely predictable argument strictly in line with the organization's politics. Want an article in favor of stricter regulation? The liberal Center for American Progress has you covered. Opposition? The libertarian Cato Institute has everything you might need.
Though they may resemble academic papers in appearance, they have an obvious agenda and a close examination bears this out. No good scholar would cite any of this research. For example, the original cited article I spoke of was a report issued by the Center for American Progress titled, Supporting Effective Teaching Through Teacher Evaluation: A Study of Teacher Evaluation in Five Charter Schools. The article, supportive of stricter teacher evaluations and firing practices, a type of reform popularly touted as key to closing the achievement gap among schools, cited the "report" as evidence of the efficacy of this approach in fundamentally changing education.
Politically, the neoliberal Center for American Progress (along with Obama's education secretary Arne Duncan) has embraced the notion that accountability, standards, charters and testing are the key reforms needed to close the achievement gap - which is universally agreed as being profound. The implicit assumption behind this view is that what is standing in the way of progress is poor teachers and the unions that protect them. One would expect then, that any CAP study would seek to provide evidence that reinforces this narrative. If evidence is found that does not support the preconception, it is not allowed. This is evident in the fact that 100% of think tank publications fall strictly in line with the organization's philosophy. This means either that data is thrown out, or that so-called research is designed in such a way that unwanted results are never found to being with.
This was certainly the case in the report I looked at. The goal of the study was to show how charter schools performed better effectively used teacher evaluation data to improve student performance, often by circumventing union protections. But the study began by choosing charter schools that were already effective. While the reported differences in evaluation procedures were assumed to be different than in public schools which, it was again assumed, were limited by union protections, there was no evidence offered to support or deny this claim. It could very well be the case that most charter schools have ineffective evaluation systems, or that most public schools have just as effective evaluation systems. I doubt this is the case, but you wouldn't know either way from the report. It could also be the case that what these schools were doing to increase student achievement had little to do with their evaluations. But again, you could not tell from the report.
In a truly academic study, these variables would need to be be addressed for the study to be taken seriously. But the purpose of academic study is to find objective, peer-reviewed truth. The purpose of think tank studies is to support the organization's stated philosophical agenda. The word for this is propaganda. When representatives of think tanks are cited by the media, they need to be understood in these terms. Just as studies from an oil company should not be cited in news reports, as their intent is entirely dubious, or industry spokespeople should be understood not as objective authorities but as in service to a particular agenda, so too must think tank resources.
While their contribution to debate should be welcome - after all, their purpose is to make as good a case for their side as possible, they must under no circumstances be treated as objective, or necessarily intellectually honest. When journalists are presenting them in this light, or citing their "research" as authoritative, they are not doing their job. In these politically-charged, partisan and highly polarized times, it is more important than ever to get quality, objective information. While words are relative, the truth is not.
A bastard's take on human behavior, politics, religion, social justice, family, race, pain, free will, and trees
Showing posts with label academia. Show all posts
Showing posts with label academia. Show all posts
Saturday, June 12, 2010
Monday, October 6, 2008
Institutional Chaos
Modern society depends, one could say, on a few powerful institutional entities. Without these, it would surely lose its course and fall prey to any manner of insidious and destructive forces, resulting ultimately in physical and emotional suffering of its people, to say nothing of the failure in living up to the promise of progress, so far defined as general happiness, personal liberties, respect of conscience, etc.
Democratic government is the most significant, but a free press and academia are close seconds and thirds. Imagining the absence of one cannot help but reflect new, poorer light upon the others. And they are interdependent. Arguably, none could really exist without the others, at least to their fullest potential. Without democracy, what good is the expression of journalism, or academic ideas? Without a free press, are democratic principles even possible? And without academia, well, would time not come to a sort of standstill?
But what brings me to this topic a the development in modern conservatism, expressed with vigor by the Republican vice-presidential nominee in the last debate, of a heightened form of ideological fervor that in many ways has essentially obliterated two of those institutions. By defining them in such a way as to make them irrelevant, the end result has been that they have been taken out of the social order.
I have no idea how long this movement has been going on, or how precisely it came to be what it is today. It certainly has roots in many political traditions, some of them more uniquely American than others. Populism is a large part of it: we the people are not our institutions. We create them to serve us, not the other way around. But populism alone is nothing but anarchy, and thus self-defeating as a governmental design.
There seems to be a subtle nihilism at its core - an end-of-days sort of reductionism that, not getting its way, wants to throw everything on the fire and watch it all burn. Sour grapes. In so doing, two of civilization's greatest institutions are spurned.
The first dismissal is the media, often more specifically referred to as the "mainstream media". Derided as hopelessly liberal, biased, partisan, it becomes altogether dismissible. The reporters can no longer be trusted, and so neither can their reporting. Suddenly there is a vacuum. Imagine for a second what life would be like with no journalism. No TV. No radio. No newspapers. No magazines. How would we get our information of the world and current events? Yet, by dismissing all established media as partisan out of hand, this is effectively what one does. In such a cynical situation, there may be hope that a discerning consumer might be able to take their pick from a variety of sources, partisan though each may be in its own right.
But this movement is not skeptical of all media, including that it may agree with - just that of which it does not approve. News is reduced to what comes through one's own approved filter. And what this inevitably means, is that there is news, and then there is news. Certain media outlets, defined by their partisan stances, are to be trusted, while everyone else is not to be. A hallmark of contemporary journalism is the charter claim to attempt objectivity. Yet by defining those one purports to disagree with as wholly biased, and therefore inconsequential, the only alternative is partisan sources. Given that this claim is leveled at almost every major national and international news organization, there isn't much left. One is left receiving only the news which with one preemptively agrees. Inevitably, facts conform to view, instead of view conforming to facts.
The second dismissal is academia, and in many ways is the much scarier position. Journalistic bias is generally not terribly difficult to parse. The assumptions behind its attack may be false, but the requisite facts themselves are usually pretty straight forward. The trust we place in journalists is not so much that we depend on their authority in every issue, but that that do their job well, reporting honestly, accurately and in sufficient detail.
Academia is a much more complex and difficult beast. By its very nature, the contribution it makes to society must be taken on good faith. No ordinary citizen can be expected to be well-versed enough in every area of study to make judgments on the relative merit of each issue. Even the college educated citizen has not been exposed to more than a very brief introduction to most relevant fields. Most research is read by only a very select few. Yet it's effects are obviously enormous. Pertinent information from academic study winds its way throughout the halls of policy-specific think tanks, down into mass media journalism, through the minds of newly graduated job applicants, into courtroom deposition, across government cabinet councils, etc.
And so, faced with its impact on so many issues in so many areas of society, we are forced to choose what makes sense to us and what does not, even while being so little-informed of the context from which each bit of new information arose. What were the parameters of the study involved? What were the prior assumptions involved in examining the issue? What sources were used? We can ask all of these questions. But it would be foolish to pretend that a layman could ever digest the totality of academic output with authority. And so we do not. We trust academia. We may not want to. But what is the alternative?
Looking back over the past 25, 50 or 100 years, the progress of human intellectual development has been staggering. Imagine if 100 years ago, one decided to refuse to accept any new thought coming out of academia for fear of embracing "biased" or distasteful conclusions. Well, we all know what that looked like because there have always been those that have fought every new idea. From evolution to psychoanalysis, racial and gender equality to cigarette smoke, there have always been those uncomfortable with trusting the authority of those more familiar with an issue.
And this is not to say that this forced blind-embrace of academic authority always leads to human progress. Eugenics was maybe the most notorious example of the misappropriation of academic authority. One can only hope for a continued social emphasis on political & social reflection, with an unfailing investment in public education.
But here may lie the greatest irony. Those who would wave away academic thought, especially thought that challenges traditional assumptions, as biased and therefore irrelevant, are themselves contributing to just the sort of environment in which ill-conceived social developments arise. A public who is uncomfortable with the rigors of academic thought, incapable of holding competing positions in their head at once, evaluating issues from many angles, or waiting patiently for more evidence to arise, is exactly the type of public most vulnerable to political & social manipulation.
Democratic government is the most significant, but a free press and academia are close seconds and thirds. Imagining the absence of one cannot help but reflect new, poorer light upon the others. And they are interdependent. Arguably, none could really exist without the others, at least to their fullest potential. Without democracy, what good is the expression of journalism, or academic ideas? Without a free press, are democratic principles even possible? And without academia, well, would time not come to a sort of standstill?
But what brings me to this topic a the development in modern conservatism, expressed with vigor by the Republican vice-presidential nominee in the last debate, of a heightened form of ideological fervor that in many ways has essentially obliterated two of those institutions. By defining them in such a way as to make them irrelevant, the end result has been that they have been taken out of the social order.
I have no idea how long this movement has been going on, or how precisely it came to be what it is today. It certainly has roots in many political traditions, some of them more uniquely American than others. Populism is a large part of it: we the people are not our institutions. We create them to serve us, not the other way around. But populism alone is nothing but anarchy, and thus self-defeating as a governmental design.
There seems to be a subtle nihilism at its core - an end-of-days sort of reductionism that, not getting its way, wants to throw everything on the fire and watch it all burn. Sour grapes. In so doing, two of civilization's greatest institutions are spurned.
The first dismissal is the media, often more specifically referred to as the "mainstream media". Derided as hopelessly liberal, biased, partisan, it becomes altogether dismissible. The reporters can no longer be trusted, and so neither can their reporting. Suddenly there is a vacuum. Imagine for a second what life would be like with no journalism. No TV. No radio. No newspapers. No magazines. How would we get our information of the world and current events? Yet, by dismissing all established media as partisan out of hand, this is effectively what one does. In such a cynical situation, there may be hope that a discerning consumer might be able to take their pick from a variety of sources, partisan though each may be in its own right.
But this movement is not skeptical of all media, including that it may agree with - just that of which it does not approve. News is reduced to what comes through one's own approved filter. And what this inevitably means, is that there is news, and then there is news. Certain media outlets, defined by their partisan stances, are to be trusted, while everyone else is not to be. A hallmark of contemporary journalism is the charter claim to attempt objectivity. Yet by defining those one purports to disagree with as wholly biased, and therefore inconsequential, the only alternative is partisan sources. Given that this claim is leveled at almost every major national and international news organization, there isn't much left. One is left receiving only the news which with one preemptively agrees. Inevitably, facts conform to view, instead of view conforming to facts.
The second dismissal is academia, and in many ways is the much scarier position. Journalistic bias is generally not terribly difficult to parse. The assumptions behind its attack may be false, but the requisite facts themselves are usually pretty straight forward. The trust we place in journalists is not so much that we depend on their authority in every issue, but that that do their job well, reporting honestly, accurately and in sufficient detail.
Academia is a much more complex and difficult beast. By its very nature, the contribution it makes to society must be taken on good faith. No ordinary citizen can be expected to be well-versed enough in every area of study to make judgments on the relative merit of each issue. Even the college educated citizen has not been exposed to more than a very brief introduction to most relevant fields. Most research is read by only a very select few. Yet it's effects are obviously enormous. Pertinent information from academic study winds its way throughout the halls of policy-specific think tanks, down into mass media journalism, through the minds of newly graduated job applicants, into courtroom deposition, across government cabinet councils, etc.
And so, faced with its impact on so many issues in so many areas of society, we are forced to choose what makes sense to us and what does not, even while being so little-informed of the context from which each bit of new information arose. What were the parameters of the study involved? What were the prior assumptions involved in examining the issue? What sources were used? We can ask all of these questions. But it would be foolish to pretend that a layman could ever digest the totality of academic output with authority. And so we do not. We trust academia. We may not want to. But what is the alternative?
Looking back over the past 25, 50 or 100 years, the progress of human intellectual development has been staggering. Imagine if 100 years ago, one decided to refuse to accept any new thought coming out of academia for fear of embracing "biased" or distasteful conclusions. Well, we all know what that looked like because there have always been those that have fought every new idea. From evolution to psychoanalysis, racial and gender equality to cigarette smoke, there have always been those uncomfortable with trusting the authority of those more familiar with an issue.
And this is not to say that this forced blind-embrace of academic authority always leads to human progress. Eugenics was maybe the most notorious example of the misappropriation of academic authority. One can only hope for a continued social emphasis on political & social reflection, with an unfailing investment in public education.
But here may lie the greatest irony. Those who would wave away academic thought, especially thought that challenges traditional assumptions, as biased and therefore irrelevant, are themselves contributing to just the sort of environment in which ill-conceived social developments arise. A public who is uncomfortable with the rigors of academic thought, incapable of holding competing positions in their head at once, evaluating issues from many angles, or waiting patiently for more evidence to arise, is exactly the type of public most vulnerable to political & social manipulation.
Labels:
academia,
conservatism,
government,
journalism,
populism
Subscribe to:
Posts (Atom)
