Research & Evaluation
Research & Evaluation
The One Voice Youth Empowerment Model has been systematically evaluated and is endorsed by the NH Expert Panel as an Evidence-Based Program.
Our program was accepted in 2010 as a Service to Science site to receive a year of technical assistance through SAMSHA. In 2011 we applied for and received a $30,000 capacity building grant awarded to some of the Service to Science sites receiving TA.The One Voice Youth to Youth Model is recognized by the NH Registry of Interventions for Evidence-Based Practices in August of 2013.
The purpose of this grant was to help Dover Youth to Youth develop the systems, tools and processes that will lead to establishing the program as evidence based. Through the Service to Science process the evaluation approach was designed and actual data collection began. This included pre- and post-survey of participants, establishing a control group of non-member students, focus groups, measurement of key advocacy skills and other evaluation activities.
The evaluation is based on our Youth Empowerment Model of Knowledge > Skills > Action. These three elements are critical to achieve true youth empowerment. A carefully crafted Logic Model depicts the program’s theory of change and shows how the program activities lead to expected outcomes.
The logic is IF One Voice is implemented with fidelity THEN youth advocates will increase their knowledge about the consequences of use and knowledge of industry practices; increase presentation skills; increase skills around media; increase knowledge of the legislative process aimed at changing the social norms around AOD use; and develop a network of pro-social peers.
Second, IF the above happens THEN the youth advocates will increase their own perception of harm; increase feelings of self-efficacy and empowerment (perception that they can influence change, competency in skills); increase knowledge of the influence of media and industry in the targeted area; increase feeling accepted by non-using peers; and increase engagement in pro-social activities.
Third, IF the above happens THEN youth advocates will influence social norms and policies at the community level; delay their own initiation in AOD use; increase perception of harm of AOD use among the community; and decrease AOD use at the community level.
The Evaluation of the One Voice Model took place over the course of two study periods (2010-2011 and 2011-2012) in five community- or school-based settings. The Dover implementation site has been running for over 20 years. The four replication sites are all new programs that are just in their first years of implementation. All implementation sites were engaged in the program evaluation and were expected to fully utilize and administer the One Voice evaluation tools which are part of the Toolkit.
A total of 67 middle school participants completed a baseline (pre-test) and follow-up (post-test) survey. The baseline survey was completed at the beginning of the school year or on a rolling-basis within the first two weeks of enrolling in the program. The follow-up survey was administered at the end of the school year to all participants regardless of if they were still active in the program. The average length of time from baseline to follow-up is 8 months (range: 3-9 months).
Using a quasi-experimental research design, the evaluators selected two 6th grade classes in the Dover Middle School as a comparison group. The 6th grade was chosen as this is the grade in which One Voice participants can first enroll in the program providing a true baseline. Independent-Samples T-Tests across 21 variables – including demographic and substance abuse knowledge, volunteer activities, alcohol and other drug use, screen- and study-time hours outside of school, and self-efficacy – confirmed that there were no differences between the participants and comparison groups at baseline.
The table below shows the percent improvement from baseline to follow-up for each of the outcomes measured. Participants demonstrated statistically significant change in all seven short-term outcomes. There was significant change in three of the four intermediate outcomes. Although there was a 28% improvement in participant’s awareness of alcohol industry practice, this change was not statistically significant. Long-term outcome evaluation is forthcoming.
| ALL MIDDLE SCHOOL PARTICIPANTS OUTCOMES (N=67) | ||
| SHORT-TERM OUTCOMES: | PERCENT IMPROVED* | SIGNIFICANT CHANGE (P<.05)** |
| Knowledge of consequences of use | 13.5% | Yes |
| Knowledge of Industry practices | 26.1% | Yes |
| Media Literacy | 42.1% | Yes |
| Presentation Skills | 89.3% | Yes |
| Knowledge of Legislative process | 48.1% | Yes |
| Media development skills | 28.4% | Yes |
| Engaging with pro-social peers (groups) | 28.0% | Yes |
| INTERMEDIATE OUTCOMES: | ||
| Perceived risk of harm | 6.7% | Yes |
| Self-Efficacy (beliefs and behaviors) | 10.5% | Yes |
| Awareness of manipulative industry practices | 28.1% | No change |
| Engagement of pro-social activities | 33.3% | Yes |
| LONG-TERM OUTCOMES: | ||
| Delayed age of onset among participants | *** | *** |
| Community-level perception of harm | *** | *** |
| Decrease AOD use among the community/peers | *** | *** |
| *Average percent change from Baseline to follow-up across one or more constructs for each outcome. | ||
| **P-value derived from Paired-Samples T-Test (pre- to post-test). | ||
| ***The forthcoming evaluation results for the community-level effect will be based on the 2013 Youth Risk Behavior Survey (YRBS) results | ||
When looking at the 6th grade participants there was statistically significant change in ten of the twelve outcomes listed in the table. Among the comparison group there was change in only two of the twelve outcomes. The two outcomes that showed change in the comparison group, consequences of binge drinking and media literacy, were the two that did not change significantly for the participants.
Based on these results it is clear that One Voice provides an effective and highly structured approach to youth empowerment. One Voice is successful because it engages and empowers youth to make changes in their community that lead to a reduction in substance abuse and the harm that accompanies it. This is done by recruiting youth, starting in grade 6, to act as leaders and change agents in the community, providing them with the information and knowledge needed to develop a level of expertise and understanding of the targeted problems, and helping them to acquire very highly developed skills needed to take action. The One Voice participants are youth leaders who are empowered to influence change among their peers and their community.
If you are interested in additional information about the evaluation please check out our 2012 Evaluation Report, 2013 Evaluation Report or 2016 Evaluation Report. If you have questions about our evaluation please Contact Us.
Donate
Toolkit
Thank You!
