Letter to the Editor: Homeland Security Education — Response to Michael W. Collier

By William V. Pelfrey Sr. and William D. Kelley Jr.

Response to Michael W. Collier’s Letter to the Editor

Mr. Collier’s criticisms of our article, “Homeland Security Education: A Way Forward,” seem to revolve around five issues:

1. Our coverage of homeland security education was partial, not “overall”:
2. Our research methods produced no generalizability and ignored the private sector;
3. We ignored the latest efforts in curriculum development such as the lack of a definition of homeland security and failing to mention the model curriculum developed by HSDECA and perhaps EKU;
4. We were dismissive of undergraduate education; and
5. We understated the demand for security specialists.

Our objective in authoring the article to which Mr. Collier responds was to extend, not limit, the debate around homeland security education. Thus our comment “A formative, rather than summative, assessment perspective underlies this research.” It would be hubris to think we held the answers to homeland security education or could formulate the curriculum others should accept. So, we appreciate and enjoy the reactions from Mr. Collier and others but we never intended for this to be a conclusive and comprehensive statement on homeland security education.

We had hoped there would be challenges of our research by others presenting their own research. Our manifest objective was to move beyond opinion and enter the realm of evidence. Our evidence was accumulated over an eight-year process of assessing, not championing, the homeland security curricula (of which there were at least four iterations) at the Naval Postgraduate School. Assessing the knowledge gained by each individual course and by the program as a whole, as well as the relevance and value of curriculum elements and capabilities, were at the heart of the assessment process that produced the data.

The critique of our research methods is a useful discussion, and almost always the first critique of social science research. We were careful to state that the response rate of 89.5 percent (N=382) allowed us to generalize to the population of graduates of this program, but perhaps that was overlooked. Additionally, endnote 2 stated rather clearly:

The reliance on respondents associated with the graduate program at the Naval Postgraduate School is likely to be a methodological concern. The disciplinary and geographic diversity make these graduates a particularly relevant population and the fact that they were exposed to the same curriculum improves inter-rater reliability. Additionally, the high importance ratings of items they were not exposed to in the curriculum increases the criticality of those assessments and neutralizes the argument of favorable bias. Absent data to the contrary, these results should serve as the basis for hypothesis testing in other populations.

So, the data at the predicate of our findings were the eighteen iterative assessments of relevance and value, by graduates of the NPS program, but the elements included more than the elements of the courses forming that curriculum. Indeed, the elements not included in the courses were more valuable than the cognates of most of the courses. The faculty surveys and the surveys from homeland security leaders (N= 35) across multiple disciplines yield results consistent with and validating the data from the graduate surveys. Taken as a totality, we found the results compelling. We find it encouraging that Mr. Collier, HSDECA, and EKU seem to validate our findings by insisting that the same competencies be part of curricula.

The private sector is important to homeland security. The fifty-one disciplines mentioned in the article included the private sector, as did the elements of the surveys, from both the Office for Domestic Preparedness Prevention Guidelines and the Office for Domestic Preparedness Training Strategy. Again, we are confident that still does not address the issue of private sector involvement in homeland security in any summative fashion. Strategic Collaboration and Critical Thinking are certainly not exclusively bounded by the public sector and we believe that by high ratings of importance of elements including terms such as “private sector” and “homeland security community” (see page 7), respondents desired to go beyond the boundaries of the public sector for some aspects of homeland security preparedness.

We sought research related to curriculum development in homeland security and were surprised by few efforts to gather evidence to guide curriculum decisions. Kiltz, as well as Rollins and Rowan, provided evidence that helps guide the debate. Articles describing the research methods used to gather the data forming the predicate for the “model” curricula to which Mr. Collier refers would be welcome additions to the body of knowledge so like-minded researchers can compare the robust nature of the research and findings. We hope the debate and dialog surrounding homeland security education produces the kind of paradigm revolution Kuhn described and, as we stated “The way forward suggests a dependence on evidence rather than opinion, and reality rather than belief.” We have not seen such evidence sufficient to merit standards of accreditation and moving into such credentialing in an amorphous environment where, as Mr. Collier states, there is lack of consensus on even a definition of homeland security, or convincing evidence countering that which we have produced as to the curricular elements. However, we are not suggesting that lack is a drawback because we are reminded that Hurricane Katrina broadened the perspective of homeland security for many and, had a concrete definition been formulated prior to 2005, it would likely have been challenged. The variance in the focus of homeland security by professionals causes us to believe it premature for academe to formulate a definition at this point or to establish standards for curricula in the midst of such uncertainty. Uncertainty should, as with other fields of study, be addressed with evidence.

Regarding demand for those with undergraduate degrees in homeland security, we would be interested in the results of impact analyses. The graduate program in homeland security at the Naval Postgraduate School measures knowledge gained at the end of courses as well as the end of the program using a retrospective pretest posttest. Critical thinking assessments are also performed for each course and across all courses. As the data from multiple universities coalesce, curriculum planners will be able to better assess the efficacy of the courses and curriculum. So the data from EKU will be a good addition to the body of knowledge.

Finally, we hold undergraduate education in high esteem. Both of us have taught at the undergraduate level. We find the research points to differences in undergraduate education and graduate education, with graduate education most appropriate for advanced study in a professional or academic field. We also mentioned the aspirational statement about the Bologna Process being adopted to identify “with some particularity – the roles and objectives of undergraduate and graduate education, using homeland security education as the example since it is in the germinal stages of development.”

William V. Pelfrey Sr.

William D. Kelley Jr.

---

This letter was originally published at the URL and https://www.hsaj.org/?fullarticle=9.1.9.