See Think OK? Go
A Quantitative Analysis of the
Consumer Decision Making Workshop Data
Developed for
COLE Vocational Services
Under a PDF Contract with
Developmental Disabilities Center of Orange County
and
Area XI Board on Developmental Disabilities
by
William T. Allen, Ph.D.
Allen, Shea & Associates
June, 1996
Introduction
A series of recommendations were made prior to the field test of the curriculum
regarding both formative and summative evaluation strategies. This report
deals primarily with summative data and is a quantitative analysis of participant
demographics as well as the pre and post class hypotheticals.
Formative Feedback from Participants Regarding Class Format
While that information was not available for this report, it is known that
it was collected and used to guide changes in both class format and content.
In fact, participant and staff feedback led to an early alteration in the
descriptive imagery of the problem solving strategy. That is, project teachers
changed the acronym from SLOG (Stop, Look, Okay, Go) to STOG ((Stop, Think,
Okay, Go) because they found that participants were not able to distinguish
well between Stop and Look as both images seem represent the eye or eyesight
Summative Evaluation Data
The following summative evaluation strategies were implemented during the
course of this project:
- Pre and Post Class Survey. A survey of each participant was
completed prior to the first and then again at the last class in the workshop
series. The purpose of this survey was to: (1) get an idea of each participant's
interaction style; (2) obtain some general information about current problem-solving
strategies; and, (3) develop a means of comparison. Participants were presented
with (through a survey or on index cards) eight hypothetical situations
which called for making a decision. Project teachers were requested not
to use the situations during the course of the workshop. They were also
asked to rate each participant on each of the eight hypotheticals using
an established criteria (N= No response or could not understand response;
D= Defense avoidance (e.g., refer the decision to someone else, accept things
as they are, put off the decision, no action at all, "don't know"
answers; H =Hypervigilance, an impulsive response without any consideration
of possible consequences; and, V= Vigilance, a recognition of possible choices
and consequences, seeking more information, and making a decision that maximizes
possible benefits).
- The demographics of all participants who completed the workshop
series (e.g., living situation, age, gender).
- A survey of support needs in everyday and major life decisions.
(e.g., who makes your decisions about how to spend money).
- A follow-along survey to be administered after the conclusion
of the project and reported on at that time.
Methodology for Analyzing Summative Information
Demographic information as well as pre and post test scores were collated
into individual data records for all participants who completed the workshop
series. This information was then imported into JMP (Statistics for the
Apple Macintosh from SAS Institute Inc.) for analysis. Some additional visual
analysis of participant responses was completed with the use of Excel.
Our Quantitative Findings.
Demographics. In regards the demographic information collected,
we know the following about the workshop participants:
Number of participants who attended one or more classes: 77
Number of participants who completed the workshop series: 49
Age
Average Age: 34
Age Range: 22-78
Ages 22-30: 23 (47%)
Ages 31-78 26 (53%)
Gender
Men: 29 (59%)
Women: 20 (41%)
Living Arrangement
Group Home: 19 (39%)
Family: 22 (45%)
Own Home: 6 (12%)
Other: 2 (4%)
Current Support Needs in Making Choices*
Need a lot of support: 21 (43%)
Don't need much support: 28 (57%)
* Making choices includes: who takes care of you?; who chooses what
you do during the week?; who chooses what you do on the weekend?; who chooses
how you spend your money?; who choose where you live right now?; who choose
where you work right now?; who makes most of your choices?; what support
do you need in communicating, reading and writing?
Pre and Post Class Hypotheticals. Here is what we learned from an
analysis of pre and post class presentations of hypothetical situations
which required a decision from participants:
- Across all participants, the mean score on the post test (27.1) was
1.9 points higher (a higher score here means that participants moved closer
to a systematic way of making a decision or solving a problem and away from
avoiding a decision or solving a problem) than the mean score on the pre
test (25.2).
- The standard deviation decreased from 5.44 on the pre test to 4.3 on
the post test, indicating more consistent responses (a more focused response
approach) across all participants.
- Twenty seven individuals (55% of total participants) scored an average
of 4.66 points higher on the post test.
- Using a paired t-test to look at differences in pre and post test mean
scores, we found that participants who benefited most from the workshop
are in the following rank order -
1. Individuals with the greatest support needs in the area
of choice
2. Women
3. Individuals who are between the ages of 22 and 30
4. Individuals who live with families
5. Individuals who live in group homes
- Using a paired t-test to look at differences in pre and post
test mean scores, we found that participants who benefited least or not
at all are in the following rank order -
1. Individuals who live on their own
2. Individuals with the fewest support needs in the area of choice
3. Men
- Using a visual analysis of pre and post response patterns across all
participants, we found that defense avoidance and hypervigilant response
patterns generally decreased while vigilant response patterns generally
increased. This is also corroborated by pretest and posttest response pattern
trend lines.
- With one exception, participants moved towards more vigilant and away
from hypervigilant or defense avoidance strategies when presented with hypothetical
situations requiring a decision. The one exception was Story 5 which read
You come home form work and you can tell that someone has been looking
through your personal things. What would you do? While only conjecture,
it may be that this situation produces an emotional response that overrides
the use of a vigilant response pattern. Also, that as participants felt
more secure and assertive in the workshop setting, they were able to express
their stress and concern with this hypothetical through a hypervigilant
or defense avoidance approach.
Conclusions
While we can't draw large, statistically significant conclusions from the
quantitative analysis, we can conclude that it supports the qualitative
analysis of the workshop series. That is, most participants gained something
from the class and report that it has helped them be better problem solvers
and decision makers.
For a copy of the complete report, please contact Debbie Blaine at:
Cole Vocational Services
1174 Nevada St.
Redlands, CA 92374
(909) 307-6584
GO BACK TO THE ASA MAIN MENU.