Though sympathetic to the educational research community, the authors state, "We accept the diagnosis that a self-regulating professional community does not exist in education."
The authors argue that the educational research community must act quickly and strongly to nurture a "scientific culture" within its ranks. (The assumption, it seems, is that it has failed to do so.)
One reason for the failure concerns the unusual heterogeneous quality of educational researchers--they are found in many fields beyond "education."
Scientific Culture and Educational Research
by Michael J. Feuer, Lisa Towne, and Richard J. Shavelson
In this article we make the following arguments:
1. Nurturing and reinforcing a scientific culture of educational research is a critical task for promoting better research.
2. Scientific culture is a set of norms and practices and an ethos of honesty, openness, and continuous reflection, including how research quality4 is judged.
3. Individual researchers and research institutions have the responsibility for developing a scientific culture.
4. A federal educational research agency and the American Educational Research Association (AERA) can and must play crucial leadership roles in fostering, supporting, and protecting a scientific culture among their grantees and members.. . .
The federal government’s involvement of the National Academies reveals a number of underlying assumptions: First, educational research can or should be at least in part scientific. Second, the federal government specifically seeks scientific research for policy and practice decisions. Third, the quality of educational research is wanting. And fourth, consideration of the scientific basis of educational research is itself worthy of scientific attention and should be at least partly shielded from political influence.
. . .
In crude terms, a culture typically grows naturally within a fairly homogeneous group with shared values, goals, and customs. Yet as we have described, researchers in education are quite heterogeneous, engaging in their craft from different disciplinary backgrounds, viewing the enterprise through divergent epistemological lenses, employing various methods, and even holding competing objectives.
. . .
…It is vital to encourage stronger collective attention to ensuring rigor and objectivity and promoting consensus building, particularly at this unprecedented historical moment. Taking proactive steps to cultivate a “community of practice” (Wenger, 1998) in the profession can focus needed attention on guarding “against the dangers of compartmentalization . . . ” and developing “ . . . a sense of the big picture and how things fit together” (Schoenfeld, 1999, p. 170). It is in this sense that we argue for understanding and appreciating the multiple perspectives in education in the service of developing a strong, self-regulating culture.
And we believe that a key part of this multifaceted task must include a focus on developing rigorous norms that ensure scientific principles are properly applied to the educational problems and questions that are the grist for the educational researcher’s mill. In short, researchers must have a clear, commonly held understanding of how scientific claims are warranted.
We believe it is the failure of the field to develop such a community and to forge consensus on such matters as research quality and coordination of perspectives that has contributed to an environment in which members of Congress are compelled to impose them. And we are certainly not the first to suggest that attention to building a community is an essential task of the future for educational researchers (see, e.g., Pallas, 2001; Shulman, 1999). In stark terms, we believe that if the field is to argue convincingly that it is inappropriate for science to be defined by political forces—which we believe is true—then it is incumbent upon the field to cultivate its own form of life including, however difficult this may be, attention to bolstering research quality.
. . .
A scientific culture begins and ends with people. Attracting and retaining qualified leaders, staff, board members, and peer reviewers is therefore critical to a healthy federal educational research agency. Unfortunately, however, the current federal educational research agency, OERI, suffers from a poor reputation, and meager resources have resulted in drastic reductions in staff in the last few decades (Vinovskis, 2001).
. . .
Investing in the community is a long-term undertaking. Current scholarship in education is generated by investigators trained in schools of education as well as in, for example, psychology, history, economics, sociology, mathematics, biology, and public policy departments. In schools of education, students often pursue non-research-oriented goals (e.g., school administration) and may therefore reach the graduate level without research training. Beyond graduate training of educational researchers, publication standards and peer review also vary considerably in education journals as in other fields. These complex structural issues will require careful study and innovative approaches to address them effectively. These efforts on the part of a federal agency cannot be accomplished without the active cooperation of individual investigators in pursing these common goals.
. . .
To its credit, AERA, an international professional organization with the primary goal of advancing educational research and its application, has taken important steps to improve the educational research infrastructure in recent years
. . .
This is a unique time of possibility—and peril—for the field. We accept the diagnosis that a self-regulating professional community does not exist in education (Lagemann, 2000), but we believe the future holds great promise. The potential exists for developing a stronger sense of community among educational researchers. The current demand for scientific understanding of educational phenomena is unmatched in history. Now is the time for the field to move beyond particularized views and focus on building a shared core of norms and practices that emphasize scientific principles. We also encourage the field, and especially AERA, to engage in studies similar to that conducted by the NRC (2002) into the warrants for other forms of educational research.
We hope that this article and the NRC report will provide a springboard for meeting the challenge.
. . .
[Footnote]9 There is an exquisite irony in the way lawmakers and many education leaders seem to want more reliance on research even as they denigrate its quality and rigor. Heralding its promise in one breath, policymakers disparage its quality in the next. As one of the champions of an improved federal presence in educational research put it, “Education research is broken in our country . . . and Congress must work to make it more useful.
. . . Research needs to be conducted on a more scientific basis. Educators and policymakers need objective, reliable research. . . .” (Michael Castle (R-DE), as quoted in Viadero, 2000).
