Making sense of the RG Group's curious report

"We believe it is the failure of the field [of educational research] to develop … a [scientific culture] and to forge consensus on such matters as research quality and coordination of perspectives that has contributed to an environment in which members of Congress are compelled to impose them."

—Feuer, Towne, and Shavelson

1  Not as advertised
     I’ve been thinking about yesterday’s post, which examined—and ridiculed—a recent report, by the RP Group, entitled What students say they need to succeed – Key themes from a study of student support.
     Evidently, it is being passed around as an important tool (data!) within the CC system for improving student success (completion, persistence, etc.).
     Much verbiage in and about the report, including its title, implies that it reveals, in particular, “what students say they need”—that it’s essentially a survey of student opinion.

     (Naturally, one wonders why we would focus on student opinion about their needs. It isn't obvious that they are the best judges of that. This focus seems to suggest that the road to success for the CCs is "giving students what they want." If that's the road we're on, we're in big trouble.)

     One might suppose, therefore, that the RP people found some students and tried to derive from them their views about what helps them (or what would help them) succeed in completing their studies in college.
     But that’s not quite what happened in this study. The students weren’t provided a blank slate and asked to fill it. No, they were given six success factors—“directed,” “focused,” “nurtured,” “engaged,” “connected,” and “valued” (DFNEC&V for short)—and were asked essentially to rank these six (or otherwise opine about them) as “contributors” to “achievement.”
     These “factors” were evidently gleaned from a “review” of the relevant research literature, such as it is.*
     And so this “study” is more theory-laden than one might at first think. It’s not quite the survey of student opinion that it is advertised to be. (Is that a good thing? Not necessarily.)

Those odd bromidular emissions

2 What causes what?
     Today, I made a brief effort to understand this theory, but that only puzzled me further. As we’ve seen, the RP Group seems to say that research reveals DFNEC&V to cause success. But in a shaded section of the report entitled “Defining the “Six Success Factors” (page 6), we’re told that

     A growing body of evidence indicates that strategic supports … can increase students’ abilities to achieve completion and transfer. … The RPGroup’s review of leading studies on student support found that effective support … helps students become [directed, focused, nurtured, engaged, connected, & valued].

     Did you get that last part? “Effective support…helps students become” directed, focused, etc. That is, “effective support” yields (causes) students achieving the six “success factors.”
     (Or perhaps they mean to say that effective support is support that makes students DFNEC&V. Is that what they mean? If so, then they're saying that DFNEC&V causes success.)
     But aren’t these “factors” supposed to cause success? But, here, they are effects, not causes.
     So maybe it’s like this: the “Effective support” brings about DFNEC&V, and DFNEC&V brings about success. Sheesh, I dunno. I’m lost.

Minority report

     Elsewhere, the RP Group, discussing the “research,” assert that it “demonstrates that students are more likely to succeed when they are directed, focused, nurtured, engaged, connected and valued.” The causal claim being made here seems to be: let’s make sure that students are “directed” (etc.) because that will help them become successful. That is, “the factors” cause the success.
     So now I’m confused. Are we supposed to embrace the aforementioned “effective support” in order to have students that are DFNEC&V? Or are we supposed to bring about DFNEC&V in order to cause student success? Is it both?
      Evidently, the “effective support” is something other than (because it causes) DFNEC&V. Just what is it then?
     *And, again, one big question here is: what is the quality of this research regarding "effective support"? (See Re the quality of research in education.) Studies, I suppose. Did these studies involve large or small numbers of students? Were results replicated? Inquiring minds want to know.
     As you may know, in recent years, the issue of study/research quality (low quality and fraud) has roiled the scientific (let alone the education/pseudoscientific) community.
     SEE False positives: fraud and misconduct are threatening scientific research (Guardian UK) and Negative results are disappearing from most disciplines and countries (Scientometrics) and Scientific fraud is rife: it's time to stand up for good science (Guardian UK)

SEE ALSO EdDreck: the so-called experts