GALLUP.ORG concluded:
"A majority of Americans favor having Arabs, even those who are U.S.
citizens, being subjected to separate, more intensive security
procedures at airports. About half of Americans favor requiring
Arabs,even those who are citizens of the United States, to carry special
ID."
The Gallup results represent a short-term pulse in the context of a
dramatic and tragic event. However, their value in terms of long-term
policy development is certainly arguable. Also, one might argue that the
short-term value of some of this data is questionable. The Gallup
organization is not immune to poor question construction and to drawing
questionable conclusions. Consider the following issue of apparent
interest:
But what precisely did they ask and find? Take a look:
Please tell me if you would favor or oppose each of the following as a
means of preventing terrorist attacks in the United States.
-- Requiring Arabs, including those who are U.S. citizens, to undergo
special, more intensive security checks before boarding airplanes in the
U.S.
Support: 58%
Oppose: 41%
--Requiring Arabs, including those who are U.S. citizens, to carry a
special ID
Support: 49%
Oppose: 49%
Look at the survey items carefully. There are two problems. First, the
initial stem preceding the items violates the principal that survey
researchers should avoid questions that impose unwarranted assumptions. In
other words, all of the items below the stem (I did not list them all)
require the respondent to assume or buy into the notion that each of the
items reflect a potentially effective means to reduce terrorist attacks in
the United States.
Second, the two specific items listed above are "double-barreled," which
violates a basic tenet of survey construction. Specifically, each item
requires respondents to answer with respect to Arabs who are citizens AND
those who are not citizens. What if respondents' support or opposition
depends on citizenship status? Not an unreasonable expectation. How
then does an individual respond to these questions--in terms of Arabs who
are citizens, Arabs who are not citizens, a weighted average of the two,
some other strategy? Will every respondent use the same strategy to
answer the question?
Can we interpret these results in any meaningful way? I doubt it. The
"general" conclusion by Gallop quoted above fails to take these issues
into account and frankly is not defensible given the way the questions
were asked.
This seems a basic lesson that the ultimate value of much "opinion data"
depends on methodological rigor and the immediate social context in which
it is collected.
Mike Frone