Draft of Evaluation Plan

Topic Selected: Improving Postpartum Mental Health Support for Black Mothers in South

Queens: A Responsive Health Intervention

To successfully complete this assignment Students must:

1. Create an evaluation plan for your public health program. This might
include a plan to engage stakeholders, utilizing surveys, and/or
planning an RCT.

The evaluation of the Queens Maternal Wellness Circles will be designed as a

mixed-method participatory process, ensuring that both statistical and
experiential outcomes are captured. The plan’s foundation draws from the
methodological strengths observed in Howell et al.’s randomized behavioral
trial, Van Horne et al.’s home visitation model, and Robinson et al.’s quasi-
experimental group care study. These studies collectively emphasize that
postpartum mental health programs are most effective when their evaluation
involves stakeholder inclusion, context-specific metrics, and longitudinal follow-
up.
The primary evaluator will be an independent evaluation specialist supported
by the program coordinator and perinatal counselors. Their joint oversight
guarantees objectivity while maintaining a deep understanding of the
community context. The evaluation will be divided into three main phases:
preparatory, implementation, and post-program review.
The preparatory phase will occur in the first month before program launch. It
will include stakeholder meetings with participating mothers, peer mentors,
and representatives from partner organizations such as Jamaica Hospital and
local faith-based centers. During this stage, focus group discussions will help
refine evaluation questions, ensuring that the assessment tools are culturally
appropriate and resonate with participants’ lived experiences.
The implementation phase will extend throughout the 12-week intervention.
Feedback will be gathered after every session through brief reflective prompts,
similar to those used by Robinson et al., who demonstrated that immediate
feedback captures nuanced emotional changes more effectively than post-hoc
surveys. Peer mentors will note participant engagement, attendance, and
expressions of trust or hesitation during sessions. This documentation will help
identify barriers to participation, such as transportation or stigma, allowing for
real-time adjustments.
The post-program review will begin immediately after the final session and
continue for six months. The evaluation will focus on both short-term and
sustained outcomes, drawing inspiration from Howell et al.’s multi-month
follow-up approach. A combination of repeated Edinburgh Postnatal
Depression Scale (EPDS) screenings and in-depth interviews will allow for
triangulation between quantitative and qualitative data. Participants showing
persistent depressive symptoms will be referred for additional counseling
support.
The evaluation will assess five main dimensions: (1) reduction in depressive
symptoms, (2) perceived emotional resilience, (3) social connectedness, (4)
satisfaction with care, and (5) perceived cultural relevance of the program. The
inclusion of cultural relevance as an explicit domain mirrors Van Horne et al.’s
emphasis on relational trust as a determinant of success in home-based care
models.
Timeline-wise, data collection will occur at four key points: baseline (week 0),
midpoint (week 6), post-intervention (week 12), and follow-up (month 6).
Analysis will be ongoing, with interim reports produced after each cycle to
guide continuous improvement.
By engaging participants and community partners as co-evaluators rather than
passive subjects, the plan aligns with the principle demonstrated in Robinson
et al.’s study that culturally situated maternal programs thrive when
participants perceive the process as a mutual exchange rather than external
scrutiny. This participatory and cyclical evaluation design ensures that the
program’s effectiveness is understood not only in clinical terms but also in
relation to lived maternal experience within South Queens.

2. The evaluation plan should discuss how the program will measure
success, and thoroughly discus measurement tools/instruments that
will be utilized, the type of data that will be collected, and how data
will be analyzed.
Success will be measured through a combination of standardized mental health
indicators, process tracking, and experiential narratives. The Edinburgh
Postnatal Depression Scale (EPDS) will serve as the primary quantitative
instrument. It was used effectively across all three referenced studies and is
sensitive to subtle shifts in postpartum emotional states. Baseline EPDS scores
will be compared with those collected at six weeks, twelve weeks, and six
months post-intervention. A minimum 30% reduction in mean EPDS scores will
be considered clinically meaningful improvement.
Complementary to the EPDS, the program will use a Maternal Social Support
Index adapted from the group care model evaluated by Robinson et al. This
tool measures perceived social connection, trust, and emotional availability
within the participant’s environment. Since isolation and mistrust emerged as
recurrent barriers in Van Horne et al.’s study, monitoring the growth of
perceived social support will help quantify one of the program’s central goals:
creating communal safety nets.
To capture satisfaction and engagement, brief post-session check-ins will be
administered biweekly. Participants will rate, on a five-point scale, how
supported, understood, and connected they felt during each session. These
short assessments replicate the ongoing feedback model that sustained
engagement in Howell et al.’s intervention. Attendance logs, punctuality
records, and qualitative notes by facilitators will supplement these ratings to
give a multidimensional view of participation trends.
Qualitative data will be obtained through semi-structured focus group
discussions at the program’s midpoint and conclusion. The discussions will
explore perceived changes in self-perception, coping strategies, and attitudes
toward help-seeking. Van Horne et al. showed that such relational interviews
reveal emotional shifts not always reflected in numeric scales. To prevent social
desirability bias, discussions will be moderated by trained peer mentors rather
than clinicians, allowing mothers to speak freely in familiar social contexts.
Data analysis will follow a convergent mixed-method approach. Quantitative
data from EPDS and social support indices will be analyzed using paired-sample
t-tests to determine the statistical significance of changes across time points.
Attendance and engagement rates will be summarized descriptively to identify
patterns related to demographic or logistical variables.
Qualitative data will undergo thematic analysis using Braun and Clarke’s
framework. Transcripts from interviews will be coded inductively to identify
recurring themes related to trust, cultural relevance, and emotional
transformation. The triangulation of qualitative and quantitative findings will
ensure that the evaluation captures both symptom reduction and experiential
growth, echoing Howell et al.’s conclusion that success in postpartum
interventions is best understood holistically.
Success will thus be defined not only by reduced depressive symptoms but by
improved maternal agency, renewed self-efficacy, and stronger community
integration. This layered definition mirrors Robinson et al.’s interpretation of
success as emotional restoration within a social matrix rather than isolated
clinical improvement.

3. Include a proposed evaluation tool. This can be a survey instrument,

semi-structured interview guide, or focus group guide.

A semi-structured focus group guide was selected as the primary evaluation

tool for this program because it allows for a deeper understanding of
participants’ lived experiences and the contextual factors influencing the
program’s impact among Black mothers in South Queens. While quantitative
surveys can capture surface-level satisfaction or behavioral change, they often
fail to reveal the cultural, emotional, and relational dynamics that shape
engagement and outcomes in postpartum support programs. The semi-
structured format provides a consistent set of core questions ensuring
comparability across groups, while allowing facilitators the flexibility to probe
further into emerging issues such as perceived respect, cultural safety,
accessibility of services, and the sense of community developed through
participation. This approach is particularly appropriate given the program’s
emphasis on trust, shared identity, and emotional wellbeing, all of which are
best captured through open dialogue. Insights from these discussions will
directly inform the refinement of intervention content, delivery methods, and
staff training, ensuring the program remains responsive to the needs and
priorities expressed by mothers themselves.

Focus Group Guide:

Section 1: Opening and Context
1. Can you briefly share what motivated you to participate in this
postpartum support program?
2. How did you first hear about the program, and what were your initial
expectations before joining?
Section 2: Program Experience
3. How would you describe your overall experience with the program
sessions or activities?
4. Were there specific parts of the program that felt especially helpful or
meaningful to you? Why?
5. Were there any moments or aspects that felt less helpful,
uncomfortable, or difficult to relate to?
Section 3: Cultural Relevance and Sense of Belonging
6. In what ways did the program reflect or fail to reflect your cultural
values, beliefs, and experiences as a Black mother?
7. How comfortable did you feel sharing your personal experiences within
the group setting? What contributed to that comfort or discomfort?
8. How did the cultural background or understanding of the facilitators
affect your engagement and trust in the program?
Section 4: Perceived Impact
9. Since participating, have you noticed any changes in how you feel
emotionally, socially, or in your ability to care for yourself and your baby?
[Link] kinds of support (emotional, informational, or practical) did you
gain through this program that you did not have before?
Section 5: Recommendations and Improvement
[Link] could be changed or added to make the program more useful or
accessible for mothers like you?
[Link] advice would you give to future facilitators to better support Black
mothers during the postpartum period?
Section 6: Closing Reflection
[Link] you could describe this program in one word or phrase, what would it
be and why?