Re the quality of research in education

     In this article from 2002, the authors respond to the challenge to educational research created by a demand for more and "scientific" research coming from the public/politicians/legislators.
     Though sympathetic to the educational research community, the authors state, "We accept the diagnosis that a self-regulating professional community does not exist in education."
     The authors argue that the educational research community must act quickly and strongly to nurture a "scientific culture" within its ranks. (The assumption, it seems, is that it has failed to do so.)
     One reason for the failure concerns the unusual heterogeneous quality of educational researchers--they are found in many fields beyond "education."

Scientific Culture and Educational Research

by Michael J. Feuer, Lisa Towne, and Richard J. Shavelson

     In this article we make the following arguments:

1. Nurturing and reinforcing a scientific culture of educational research is a critical task for promoting better research. 

2. Scientific culture is a set of norms and practices and an ethos of honesty, openness, and continuous reflection, including how research quality4 is judged. 

3. Individual researchers and research institutions have the responsibility for developing a scientific culture. 

4. A federal educational research agency and the American Educational Research Association (AERA) can and must play crucial leadership roles in fostering, supporting, and protecting a scientific culture among their grantees and members.

. . .
     The federal government’s involvement of the National Academies reveals a number of underlying assumptions: First, educational research can or should be at least in part scientific. Second, the federal government specifically seeks scientific research for policy and practice decisions. Third, the quality of educational research is wanting. And fourth, consideration of the scientific basis of educational research is itself worthy of scientific attention and should be at least partly shielded from political influence.
. . .
     In crude terms, a culture typically grows naturally within a fairly homogeneous group with shared values, goals, and customs. Yet as we have described, researchers in education are quite heterogeneous, engaging in their craft from different disciplinary backgrounds, viewing the enterprise through divergent epistemological lenses, employing various methods, and even holding competing objectives.
. . .
     …It is vital to encourage stronger collective attention to ensuring rigor and objectivity and promoting consensus building, particularly at this unprecedented historical moment. Taking proactive steps to cultivate a “community of practice” (Wenger, 1998) in the profession can focus needed attention on guarding “against the dangers of compartmentalization . . . ” and developing “ . . . a sense of the big picture and how things fit together” (Schoenfeld, 1999, p. 170). It is in this sense that we argue for understanding and appreciating the multiple perspectives in education in the service of developing a strong, self-regulating culture.
     And we believe that a key part of this multifaceted task must include a focus on developing rigorous norms that ensure scientific principles are properly applied to the educational problems and questions that are the grist for the educational researcher’s mill. In short, researchers must have a clear, commonly held understanding of how scientific claims are warranted.
     We believe it is the failure of the field to develop such a community and to forge consensus on such matters as research quality and coordination of perspectives that has contributed to an environment in which members of Congress are compelled to impose them. And we are certainly not the first to suggest that attention to building a community is an essential task of the future for educational researchers (see, e.g., Pallas, 2001; Shulman, 1999). In stark terms, we believe that if the field is to argue convincingly that it is inappropriate for science to be defined by political forces—which we believe is true—then it is incumbent upon the field to cultivate its own form of life including, however difficult this may be, attention to bolstering research quality.
. . .
     A scientific culture begins and ends with people. Attracting and retaining qualified leaders, staff, board members, and peer reviewers is therefore critical to a healthy federal educational research agency. Unfortunately, however, the current federal educational research agency, OERI, suffers from a poor reputation, and meager resources have resulted in drastic reductions in staff in the last few decades (Vinovskis, 2001).
. . .
     Investing in the community is a long-term undertaking. Current scholarship in education is generated by investigators trained in schools of education as well as in, for example, psychology, history, economics, sociology, mathematics, biology, and public policy departments. In schools of education, students often pursue non-research-oriented goals (e.g., school administration) and may therefore reach the graduate level without research training. Beyond graduate training of educational researchers, publication standards and peer review also vary considerably in education journals as in other fields. These complex structural issues will require careful study and innovative approaches to address them effectively. These efforts on the part of a federal agency cannot be accomplished without the active cooperation of individual investigators in pursing these common goals.
. . .
     To its credit, AERA, an international professional organization with the primary goal of advancing educational research and its application, has taken important steps to improve the educational research infrastructure in recent years
. . .
     This is a unique time of possibility—and peril—for the field. We accept the diagnosis that a self-regulating professional community does not exist in education (Lagemann, 2000), but we believe the future holds great promise. The potential exists for developing a stronger sense of community among educational researchers. The current demand for scientific understanding of educational phenomena is unmatched in history. Now is the time for the field to move beyond particularized views and focus on building a shared core of norms and practices that emphasize scientific principles. We also encourage the field, and especially AERA, to engage in studies similar to that conducted by the NRC (2002) into the warrants for other forms of educational research.
     We hope that this article and the NRC report will provide a springboard for meeting the challenge.
. . .
     [Footnote]9 There is an exquisite irony in the way lawmakers and many education leaders seem to want more reliance on research even as they denigrate its quality and rigor. Heralding its promise in one breath, policymakers disparage its quality in the next. As one of the champions of an improved federal presence in educational research put it, “Education research is broken in our country . . . and Congress must work to make it more useful.
. . . Research needs to be conducted on a more scientific basis. Educators and policymakers need objective, reliable research. . . .” (Michael Castle (R-DE), as quoted in Viadero, 2000).

STUDENTS SUE DISTRICT OVER 1ST AMENDMENT

From Dissent 63(?), April (15?), 2002

Originally entitled:

BOOBERY AND BASTARDRY: The board confronts Collins’ decision

By Big Bill [Roy Bauer]

“Laura and I really don’t realize how bright our children is sometimes until we get an objective analysis.”

—George W. Bush, Meet the Press, April 15, 2000

Beijingpolitik:

The other day, I happened to view that footage of student protesters staring down tanks in Tiannamen Square in 1989. Amazing! At the time, people hoped that the protesters’ getting away with such stunts heralded a new era in China. What else can Chinese leaders do now, people said, but start to give these brave students the freedoms they deserve!

What else? Kill ‘em, that’s what. The students were massacred. After Tiannamen, Chinese leaders just hosed off the blood.

Our trustees here in the SOCCCD are something like those Beijing bastards. Thanks to the union Old Guard’s “Board Majority,” the last five years have been marked by lawsuits, accrediting warnings, state fiscal alerts, adverse judicial rulings, recall attempts, lunatic seminar fiascos, poinsettia theft, and more. With each fiasco, dissidents celebrated, dreaming of change.

But nothin’ changes.

The Board Majority do pay a price, of course: everyone in education—and not just in California!—sees our Board for what it is: a crew of ruthless and self-seeking reactionaries.

So what? In case you haven’t noticed, to the Board Majority, “people in education” are precisely the enemy!

Another fine mess:

[We should] create a TQM paradigm where…systems thinking is a model, …quality and continuous improvement are guiding principles, and…benchmarking and best practices are norm…[We should] establish ongoing feedback program for both internal and external customers….

—Chancellor Mathur’s “needs” survey

Well, here we go again. In March, surprising no one, U.S. District Judge Audrey Collins ruled the district’s speech and advocacy policy to be unconstitutional. That led to the usual bad press (Smack!) and harsh editorials (Thwack!). And so, as before, we dissidents now cheer and celebrate. The sound of spankage dinning in our ears, we espy the promised land.

But no. Even now, Raghu Mathur spins his daffy “needs” survey, which, you’ll recall, included such desiderata as: increased “monitoring” of faculty “performance”; introducing a “dress code”; offering “public relations” classes, and—my personal fave—the colleges’ becoming “more customer focused.” (Apparently, this involves setting up “Customer Care Call Centers” at the colleges and distributing “comment cards.”)

At IVC, the deans hired by Mathur are literally screaming at employees (this has yielded yet another lawsuit) or selectively enforcing absurd rules on faculty.

Meanwhile, in an effort to weaken—or simply taunt—the Academic Senates, the Board Majority unilaterally changed the policy requiring “delegation of authority to Academic Senates.”

To top it all off, Larry Oldewurtel has been named IVC’s “Teacher of the Year.”

Inevitably, Chancellor Mathur, his crew of administrative protégés, and the Board itself will generate further embarrassing media attention. There will be spankage. Nancy Padberg will complain about press bias. Once again, the trustees will take big PR hits, and dissidents will again cheer.

But it won’t change a thing. It’s the election, stupid. None o’ this will matter come November.

Mathur and free speech:

“A new President of the United States, in 1988, gained in popularity when he was found halting in speech and loose in grammar.”

—Jacques Barzun, From Dawn to Decadence

The latest court victory by students has a long history. It all started with Mathur’s imposition of arbitrary restrictions on peaceful student protesters back in the Spring of 1998:

…In a rare show of Orange County activism, students Delilah Snell and Diep Burbridge gathered nearly 100 of their colleagues for a series of campus demonstrations, the first in the college’s near-twenty-year history. They denounced the hiring of [Raghu] Mathur, demanded the recall of [Trustee Steven] Frogue and called attention to the possible loss of the college’s accreditation. The rallies attracted major media coverage. In response, the board, Mathur, and their cronies claimed the students were “misled” by a handful of “disgruntled employees” and “leftist” faculty. Even freedom of speech took a nosedive. Snell and Burbridge were initially told to give twenty-four-hour notice before each demonstration and to submit to college officials for review everything they would be passing out. After meetings with the president in which they were accused of “misleading” others and hostile encounters with board supporters, the students were at first permitted one hour a week to hold their demonstrations. Soon it was reduced to thirty minutes.

Now the students, represented by the ACLU, are suing Mathur and the board for violating their First Amendment rights. According to the lawsuit, filed this past summer, the demonstrations were relocated from the center of campus to an isolated area where students were told to keep their noise level down. When the limits were questioned, students were told it was not in the “best interest of the college” to hold a longer protest in a more visible part of campus, given the “political climate.” (Sanaz Mozafarian; The Nation, October 5, 1998)

The suit ultimately resulted in the Board Majority’s adoption of Board Policy 5406 (the 5000 series concerns specifically students) in 1999. The policy was authored by clueless district lawyer Spencer Covert. Students (this time, Burbridge, Stephansky, and Caruso) sued again.

Then, in August of ’99, new VP of Student Services, Armando “Beyondo” Ruiz, at the behest of Mathur and citing BP5406, ordered faculty to remove all signs and posters from their windows and doors, an apparent ploy to eliminate signage critical of the President, a notoriously thin-skinned fellow. This produced howls of protest and embarrassing TV coverage. Reluctantly, the board rescinded Ruiz’s order.

In September of ’99, the court (U.S. Judge Audrey Collins) enjoined enforcement of substantial portions of the policy, citing their unconstitutionality.

That judgment led to the adoption, in May of 2000, of yet another policy, the 32-page BP8000, which was even more restrictive than 5406! (The Board Majority’s Don Wagner, a lawyer, did not support it.)

Consequently, in February of 2001, three students (this time, Burbridge, Irvine, and Khademi) sued again, charging that the policy violates the First and Fourteenth Amendments of the U.S. Constitution; Article 1, sections 2 and 3 of the California Constitution; and California Education Code S 66301.

On February 11, 2002, the students filed a “Motion for Summary Adjudication” (i.e., a motion to secure a judgment without a full trial). The hearing was scheduled for March 11—later “continued” to March 18, 2002 (i.e., last month).

Collins’ ruling:

On the 18th, Collins at long last issued her “Tentative” ruling, which, after brief arguments from the attorneys (Carol Sobel/Wendy Phillips and Allan Wilion), she declared to be her final ruling. In its “conclusion,” she wrote:

…[T]he Court finds that a number of provisions of Board Policy 8000 violate Plaintiff’s First Amendment rights. Because many of the unconstitutional provisions are not…severable from the rest of the policy, all provisions in BP 8000 applicable to students must be struck down in their entirety. (42)

She closed with this:

With [some exceptions]…, the provisions applicable to [not students, but] the general public…remain in effect. (CTR—Collins’ Tentative Ruling)

This last part concerned the “standing” of the plaintiffs—the three students. Some provisions of BP8000 applied, not to students, but to “members of the public.” The question was: did the students have standing as members of the public—who can, as such, rightly challenge the latter provisions?

Collins judged that, since the students “have not alleged that they personally have been or could be affected by these provisions of BP 8000,” they “do not have standing to bring a facial challenge to those sections as being impermissible time, place, or manner restrictions” (CTR, p. 9).

Prior restraint:

One issue raised by the suit concerned whether BP8000 involved impermissible “prior restraint,” i.e., “…‘when the enjoyment of protected expression is contingent upon the approval of government officials’” (CTR, p. 15). Put more simply, prior restraint is banning expression as opposed to punishing it, when offensive, after it has occurred.

Concerning that question, Collins ruled as follows:

…The Court…finds that four individual provisions [of BP8000] are unconstitutional. [One section] provides that “[u]se of any portion of the GROUNDS must be reserved. The decision whether to allow use of the GROUNDS lies solely within the discretion of the President.”…[Another section] provides that on the campus of IVC, “[n]o amplification of any type shall be permitted within Quad A or B or the Grounds without approval of the President.”…[Another section] provides that “[a]ny portion of interior SOCCCD property must be reserved. The decision to allow use of the interior of any SOCCCD property lies solely under the discretion of the President.”…Lastly, interior amplification must be authorized by the President.

These sections are prior restraints because they condition expression in certain areas of the District’s campuses upon approval of the administration. Accordingly, they are subject to strict scrutiny…[and, as such,] they “may not delegate overly broad licensing discretion to a government official.”

…The four sections identified here delegate completely unfettered discretion to the campus presidents to permit or prohibit expression. When a permit scheme is “completely discretionary, there is a danger that protected speech will be suppressed impermissibly because of the government official’s…distaste for the content of the speech.”…Because these provisions provide the presidents with absolutely no standards to guide their decisions, they are unconstitutional and must be stricken. (CTR, pp. 16-17.)

Content of speech:

Another issue raised by the suit concerned the “content” of speech. The “state” can pursue “content-based” exclusion of speech, but only if it can show “…‘that its regulation is necessary to serve a compelling state interest and that it is narrowly drawn to achieve that end’” (CTR, p. 18).

Collins here noted, peevishly, that “BP 8000 purports to be content-neutral, but, in the same sentence, acknowledges that it is not….” (CTR, p. 19).

Further,

Plaintiffs identify fourteen sections and subsections [of BP8000] as content-based…For the reasons noted above, Plaintiffs do not have standing to challenge…provisions in [some of those sections]….With regard to the remaining…sections, the Court agrees that all but one are content-based…

[Several sections] require District administrators to examine the content of expression to determine if it violates section 76120 and then allow or require administrators to take punitive, disciplinary action based on that content…[The] District’s enforcement of that code section infringes on students’ First Amendment rights in that it requires District administrators to examine the content of the students’ expression. Defendant must proffer compelling reasons to justify each prohibition in section 76120…The Court can conceive of a number of legitimate state interests…But Defendant [i.e., the district] has not identified any such interests…More importantly, Defendant has not demonstrated that these interests are compelling…and has not demonstrated that the content-based restrictions are narrowly tailored to these ends…

[T]he Court finds that the District has a compelling state interest in preventing “the commission of unlawful acts on community college premises” and “the substantial disruption of the orderly operation of the community college”…But the Court cannot find that the District has a compelling interest in preventing the other types of expression that are prohibited by section 76120

…Given the interest of the faculty and student body in intellectual freedom, the Court cannot find that the District has a compelling interest in, for example, prohibiting obscenity that justifies these content-based provisions…Because the Court does not find a compelling interest justifying the examination of the content of student expression to root out all speech prohibited by section 76120, the blanket enforcement of section 76120 is not narrowly tailored to those interests that the Court finds compelling…Accordingly, the blanket imposition of the terms of section 76120 must be stricken. (CTR, pp. 19-24)

Whew!

Written materials, reservations, banners:

BP8000 also bans certain types of written material. Collins judged that “This provision, too, must be stricken from the policy. [This section]…singles out certain kinds of commercial advertising based on its content. Defendant has offered no state interest in defense of the advertising ban.”

BP8000 provides for the denial of reservations for use of campus grounds—e.g., for concerts. Said Collins: “The provision as written gives the campus presidents unbridled discretion in choosing which music concerts to prohibit. Accordingly, this basis for denying reservations must be stricken…[The relevant section] also gives the campus presidents unlimited discretion to identify other types of expressive activities that create an unreasonable risk of harm. This unbridled discretion is impermissible and must be stricken.” (28; my emphasis.)

The district excepts itself!

Wrote Judge Collins, BP8000 “excepts the District and ‘all persons or groups who enter into contracts with SOCCCD’ from ‘the provisions pertaining to banners, posters, and/or distribution of material….’…Defendant asserts that it ‘has the right to exempt itself from the scope of the Policy since it is the enforcing entity’…Defendant cites no legal authority in support of this proposition…” (29; my emphasis.)

Overbroad?

An important part of Collins’ ruling concerned the question of whether BP8000 was “Unconstitutionally Overbroad.” A regulation is “overbroad” (relative to the Constitution) if it is drafted so broadly that it manages also to forbid constitutionally protected expression. Collins opined:

…Plaintiffs object to the sections of BP 8000 that prohibit students from distributing writings “inside all buildings including classrooms” and in “parking lots” as overbroad…[and] The Court agrees. These sections ban all writings…regardless of whether or not the school has any legitimate interest in banning their distribution. Like Plaintiffs, the Court conceives of a broad array of protected expression barred by these provisions, ranging from class notes to the campus newspaper to campaign materials. These sections are patently overbroad…[T]he provisions are so sweeping, Defendant will be unable to show that they are narrowly tailored to anything…. (CTR, pp. 37; emphasis in original.)

Well, that’s about it. In court, Judge Collins recommended to the district that it get “a good Constitutional lawyer” for advice.

Reportedly, steam could be seen streaming out of Allan Wilion’s ears.

The bad press bears:

The next day (3/19), the Times reported that

An Orange County community college district’s policy prohibiting students from engaging in certain free-speech activities without permission from administrators is unconstitutional, a federal judge ruled Monday. The judge found that South Orange County Community College District was exercising prior restraint in requiring approval for distributing leaflets in certain areas, posting information on a bulletin board, putting up a banner and using a loudspeaker at a rally.

It was the second time U.S. District Judge Audrey Collins found the district’s free-speech policy wanting…Collins suggested Monday that district officials consult a constitutional law expert when redrafting the policy. Allan Wilion, the district’s attorney, said the college district’s governing board would probably appeal the ruling.

“This case is not over,” he said. “We disagree with the court’s ruling vociferously.”

But Carol Sobel, who represented the three student plaintiffs, said the district’s track record on its free-speech policy indicates that it “is either getting bad advice [from its lawyers] or else the advice they want.”

….Sobel said the 32-page policy, adopted in May 2000, required written permission for students to hand out the student paper or to pass class notes to each other. The policy also allows anyone who objects to a bulletin board posting to remove it.

….The board passed the policy, 4 to 3. Board President Donald Wagner, a lawyer, voted against it.

“I thought it was trouble, so the [judge’s] decision makes a lot of sense,” Wagner said Monday.

But Wilion said the judge used the wrong legal standard. “There are certain legal rulings that are made,” he said. “It doesn’t mean they are correct.”…..

The OC Register offered a similar story, though, there, lawyer Wilion sounded relatively subdued.

By the time of the Irvine World News’ story on the 21st, Wilion had seriously cooled his jets:

Allan Wilion, the college district’s attorney, said the entire policy was struck down because the parts of the policy deemed unconstitutional could not be separated.

“The dispute comes down to a couple of areas,” he said. “We think the policy is reasonable, very moderate and student friendly. But the court made its decision and we respect the decision and will act accordingly.”

….Lawsuits from students and faculty members began in 1997 when Raghu Mathur began his tenure as Irvine Valley College president and established the administrative requirements for campus speech.

Mathur, who is now the district chancellor, did not return phone calls and referred questions to the district’s lawyer.
…..
Donald Wagner, district board president…said that the board has yet to discuss the judge’s ruling and no decisions will likely be made until the end of April.
“But the district should get some advice from a good first amendment lawyer before deciding what to do next,” he said….

Ouch!

In that same issue, the editors slammed the board with an editorial:

Free speech is everyone’s fight

It’s about time administrators and trustees for the South Orange County Community College District take the lead in advocating and protecting free speech rights on campus.

Students and faculty members for more than four years have had to fight chancellors, a campus president and members of the board of trustees simply to exercise their basic right to free speech.

Seeking to muffle dissent, policies have been passed down in those years attempting to clamp down on when and where students can gather on campus to protest and even how loud they can be.

Students aren’t the only targets. Irvine Valley College administrators once threatened to remove posters and messages placed on walls expressing teachers’ disagreement with actions of the board and Mathur, and one professor was unlawfully disciplined when he refused to stop publishing newsletters that lampooned district and college officials.

A federal judge…overturned portions of the policy that prohibit students from distributing written material inside buildings, classrooms and in parking lots. The onerous policy attempts to handle every campus speech situation, including when students can gather on the grass and how many inches a table can be away from a wall. The policy was criticized from many quarters, including the academic senates of both of the district’s colleges.

And Monday wasn’t the first time a court has ruled the district acted unconstitutionally. It was the fourth time since 1998.

Attempts to control free speech at Irvine Valley College began in 1997 when Raghu Mathur was appointed president of the college. Mathur recently was named chancellor of the district. A 1999 Superior Court ruling, incidentally, found that the board of trustees demonstrated a “persistent and defiant disregard” for the state’s open meeting law by staying behind closed doors to discuss Mathur’s appointment to president.

Administrators and trustees in the South Orange County Community College District have a history of acting too quickly to chill free speech and conduct the public’s business behind closed doors.

…In October 1999, 90 percent of 84 Irvine Valley faculty members responding to a survey said they could not “express their opinions about issues at the college without fear of retribution or retaliation.”

Those charged with governing institutions of higher education ought to be the first to champion free speech and open government.

Ten days later, the editors of the Times weighed in:

Meet the 1st Amendment

Free speech is a constitutional right. But the South Orange County Community College District has an unfortunate history of trying to limit how, where and when students can speak.

The district that operates Irvine Valley College and Saddleback College in effect wants students to raise their hands and ask to be recognized before speaking in settings outside the classroom on campus. A court this month struck down four provisions in the recently modified district policy governing when and where students can gather, the use of public address systems and access to buildings on campus.

The district knows it is heading down the wrong path because it now has lost two court challenges. The most recent rejection came March 18 when U.S. District Judge Audrey B. Collins sided with students who went to court to challenge district restrictions. Collins in 1999 sank another version of the policy because it wrongly limited the “time, place and manner of speech.” The district also withdrew its policy on two other occasions before a judge could rule. The district also has had its knuckles rapped for trying to rein in free speech by faculty members. Irvine Valley College professor Roy Bauer in August won another round in a long-running court fight sparked by a faculty newsletter that takes district officials to task. The court ruled that district officials acted unconstitutionally by reprimanding Bauer for the content in his newsletters.

Pourya Khademi, one of the students who filed the most recent lawsuit, knows firsthand the dangers of heavy-handed regulation. As a child in Iran, Khademi saw his parents jailed for reading a banned newspaper. Khademi’s attorney complains that the board keeps banging its head against the constitutional right to free speech because it is “either getting bad advice [from its lawyers] or else the advice they want.” But the board should understand instinctively the value of spirited debate in an academic community.

The district maintains it simply wants to protect students from unnecessary noise, reduce litter and ensure that college lawns and parking lots are well-maintained. Those are logical goals for administrators, but universities have an obligation to balance important real-world concerns against the overriding right to free speech.

At least, that’s what students are being taught during introductory political science classes at Irvine Valley College. “Politics in America,” the approved textbook sold at the college bookstore, underscores that “universities have a very special responsibility to protect freedom of expression.... A free and unfettered exchange of views is essential to the advancement of knowledge.”

The district should go back to school and practice what it teaches. (Times editorial, 3/31/02)

WHAT’S NEXT?

No word yet on whether the Board will appeal the decision.

—Big Bill