The article aims to provide a detailed illustration of the research approach employed by the initiative called HYSQ to establish and outline smoking cessation programs for youth without a clear definition of sample frame. The challenge of little or no information on the existence of the programs of interest makes it difficult for one to carry out a survey of community-based programs (Emery et al., 2010). It was necessary for the HYSQ to get information on the availability of replicable programs to evaluate prior to performing a national evaluation of programs. The authors experienced problems when identifying youth cessation programs since the programs are not subject to regulations. The evaluation was necessary since the authors wanted to understand the types of services the youth smoking cessation program offered.
The authors used key informants and snowball sampling strategy to identify tobacco cessation programs for youth. After defining the target sample, the authors determined the most productive elements and their contribution to the sample. The authors used a two-stage sampling design to communicate to the whole population regarding information on cessation programs. Results of the evaluation indicated that out of the 408 sampled counties in the US, the total number of Tier 1 key informants were 5,978 (Emery et al., 2010). The number of unique and traceable contacts from the snowball process was 9,983 (Emery et al., 2010). Regarding county stratification and identification of contact, counties with high expenditures on tobacco controls had 46% of distinct and traceable contacts (Emery et al., 2010). Counties with medium expenditure had 30%, while those with low expenditures had 23% (Emery et al., 2010). The analysis indicated that the education sector produced the most informants. The number of screened program administrator was higher in health sector compared to voluntary and education sectors.
First, the program’s goal was to find a particular type of health organizations that are community-based. The authors ensured they started by illustrating the snowball sampling strategy and key informants to help in identifying cessation programs. Second, the program ensured it explored the local level and identified required contacts who had information on the existence of youth cessation programs in their areas. The two-stage approach was essential in contacting the whole population in the county on information about the programs (Emery et al., 2010). The third goal was to ensure a successful construction of a national sampling frame of the tobacco cessation program for the youth. It is hard to perform a survey when there is little information about the numbers or location of programs; therefore, the construction of the national sampling frame was essential in searching and identifying youth smoking cessation programs within the county.
The authors proposed to evaluate the program in terms of its goals in the following way. The authors created a sampling design called a two-stage, and it consisted of counties and snowballs as the first and second stage sampling units respectively. Authors profiled cessation programs as they were identified. Moreover, with a snowball sampling, the authors selected a sample of 408 counties randomly. The snowball sampling process was useful in gathering contact information for people with knowledge on youth tobacco cessation (Emery et al., 2010). The lack of information on the number or locations of programs was critical in evaluating the program in terms of its goals. It was necessary for the authors to construct a national sampling frame to aid in contacting information at the local level. Therefore, the information on the availability of replicable programs helped in the evaluation of youth cessation program and determined the services offered.
The theory that is significant in this context is the program theory. It plays the role of connecting the program to the goals. The program theory explains why the authors use the initiative called HYSQ to establish and profile cessation programs, even when there is no precise definition of sample frame. The availability administrators and people with knowledge of the locations and numbers of the programs of interests imply that it is easy to carry out a survey on community-based youth smoking cessation program to determine their services.
The explanations that were ruled out stated that the program theory did not provide attribution at all. The theory provided reports on the implementation of the program and achievements of intended outcomes. Second, since the program theory guides the evaluation of the program, it is during this time when the evaluation process falters. The program theory guided the authors in the evaluation process by ensuring they make right decisions, identify and measure the impact of program outcome.
When performing the evaluation, the principal research methods that the authors used were a two-stage sampling design, survey, and interview. Sampling units of the first-stage were counties, while the second stage consisted of snowball sampling. The authors used the snowball sampling strategy to locate local individuals who had knowledge of youth smoking cessation program (Emery et al., 2010). The multivariate analysis of the findings displayed the relationship between stratification criteria and economic sectors, eligibility, and program screening.
The unintended consequence that arose was managing cases of duplicate referrals. The strategy that the authors used entailed sampling to saturation whereby participants were asked to name additional informants until there were no unique names collected. However, participants nominated a single person more than once, causing a high number of duplicate names.
The conclusion by the authors was that the education sector produced the majority of informants in the United States. Additionally, from the multivariate analysis, the health and other sector recorded the highest proportion of eligible contact since the category consisted of trained providers. Therefore, trained providers have knowledge about the cessation curriculum. The critical elements of the strategy played a significant role in contributing to the author’s search and description of youth smoking cessation program. Without describing the cessation programs, authors would not have defined the other elements involved.
The program achieved its intended goals in whole. The authors ensured they found a particular type of health organizations that are community-based. By identifying individuals at the local level with information about the programs in their communities, it was easy for the authors to find youth smoking cessation programs. Also, the authors ensured they had a successful construction of a national sampling frame of the tobacco cessation program for the youth. Evaluation results were useful in illustrating the importance of having a persistent interviewing team. With such a team it is easy to address logistical challenges and others that may be encountered. The results were useful in understanding the data with stakeholder. The results allowed the evaluators to discuss the opportunity and future efforts.
The authors made the following principal recommendations. First, the snowball sampling strategy could be useful in creating a sample frame that non-profit organizations could use. These other organizations are hard to locate since current directories are unavailable. Second, in circumstances of large-scale snowball sampling, the authors recommend that organization should invest in specialized duplicate checking software.
Evaluation entails a systematic assessment of a program to determine its achievements or failure. Key concepts of evaluation include the following. First, outcome refers to the ultimate effect as a result of the program. This represents what the project anticipates to attain at the beneficiary level. However, the outcome of the program can be intended or unintended. Besides, the outcome may include the environmental, social, and economic effect on people, institutions, or communities. Regarding the article, the outcome of the program from the multivariate analyses showed that initially used Tier 1 informants had a positive association with the programs screened.
Second, output refers to the immediate measure as a result of the implementation of the program. These measures are deliverable. In the article, counties with high expenditures on tobacco controls had 46% of distinct and traceable contacts. Counties with medium expenditure had 30%, while those with low expenditures had 23% (Emery et al., 2010). Similarly, counties with high smoking prevalence recorded 41.7% of snowball sample referrals, while low-prevalence counties recorded 47.4% (Emery et al., 2010).
Third, input refers to the local, financial, and human resources used in a program evaluation. Regarding the article, the authors used the human resource to perform surveys, interviews, and as informants to identify individuals with knowledge on programs of interest. Fourth, the objective of a program refers to the long-term change that is expected. In the article, the objective was to a successfully create a sampling frame of youth programs.
Emery, S., Lee, J., Curry, S. J., Johnson, T., Sporer, A. K., Mermelstein, R. & Warnecke, R. (2010). Finding needles in a haystack: A methodology for identifying and sampling community-based youth smoking cessation programs. Evaluation Review, 34(1), 35-51. https://doi.org/10.1177/0193841X09355258