Evaluation of the Arthur Project: Evidence-Based Mentoring in a Social Work Framework with a Social Justice Approach

Abstract

Low-income students from underrepresented communities experience significant educational inequalities, including chronic absenteeism, mental health crises, trauma exposure, and social injustices. These conditions escalate risks of dropping out and justice system involvement. Middle-school youth must navigate intense social and emotional change. To address such challenges, The Arthur Project (TAP) provides intensive, in-school, therapeutic mentoring and family support. The program teaches students essential life skills, such as problem-solving, communication, and academic engagement. Students are empowered to strengthen their resilience, perseverance, self-actualization, and confidence. TAP provides up to 500 h of therapeutic mentoring yearly over three years, delivered individually and in small groups by social work interns. The research question was as follows: Does structured, intensive in-school therapeutic mentoring with social work interns increase the socioemotional, cognitive, noncognitive, and academic skills and engagement of underserved middle-school students? This evaluation for the school year 2022–2023 is based on the program of 200 students and 30 mentors, with quantitative mentee, mentor, teacher, and caregiver surveys and qualitative interview data. The program and evaluation methodology are described, with results and conclusions. Students showed significant improvement in all domains; mentors reported student increases in confidence, perseverance, problem-solving, and communication; and teachers and caregivers reported students’ increased academic engagement. The Arthur Project program can become a national middle-school mentoring model to address widespread student inequalities.

1. Introduction

The mission of The Arthur Project (TAP) is to create a replicable model that professionalizes traditional mentoring with clinically focused mentors who work intensively with underserved youth throughout middle school. TAP seeks to cultivate a profound sense of “mattering” for students that gives them confidence in their “voice” and addresses their critical life skills development, community engagement, and awareness of dismantling oppressive systems, thereby supporting their academic, economic, social, and personal potentials.

Much research has been conducted on mentoring programs similar to TAP that emphasize a strong, trusting mentor–mentee relationship, supported by instrumental activities, such as goal setting [1,2,3]. Mentors help the mentees meet short-term goals, contributing counsel and suggestions about instrumental skills, and simultaneously nurturing longer-term relationships and emotional and social skills [2].

Both emotion-focused and task-focused activities are engaged in. For example, the development and sustaining of trust and empathy are combined with goal setting [4]. Research has shown positive correlations of youths’ developmental and interpersonal processes that increase positive academic, behavioral, and social-emotional outcomes and enable them to reach their full potential [5].

1.1. Need for the Arthur Project

1.1.1. General Need

Low-income students from underrepresented communities are especially affected by the increased income and education gap in the United States [6]. School districts with high numbers of students of color receive 15% less state and local funding per student than the Whitest districts. Black students in New York City have a 68.1% on-time graduation rate [7] compared to 87% nationally [8].

Many factors contribute to these students’ situations. Chronic absenteeism: Chronically absent students are at serious risk of low command of material, low grades and test scores, and negative challenging behaviors. These youth have significantly higher risks of dropping out that, in turn, decreases their economic opportunities and increases possibilities of justice involvement and arrest. The many reasons for students’ chronic absenteeism include parents’ and guardians’ poor health, their nonstandard work schedules, changes in household composition, students’ extensive family responsibilities, and inadequate educational supports [9].

Mental health disorders: Results from the National Comorbidity Survey found that 48.3% of Latinos and 46.8% of Blacks from ages 13 to 17 suffer from mental health disorders [10]. Youth in underserved communities have higher exposure to adversity, including trauma exposure, than similarly situated youth in higher socioeconomic communities [11].

Insufficient social workers: A 2022 New York state audit found that of New York City public schools, only 66% had at least one social worker, and most did not meet the recommended counselor-to-student ratio of 1:250. Additionally, almost 30% of the schools had no social worker [12].

1.1.2. Middle School Students

Middle school presents intense social and emotional change. Many studies have documented the benefits of mentor programs during this transitional time. With the right guidance, goals such as increased resilience, perseverance, and self-actualization may be optimized during middle school years [13,14].

The middle schools The Athur Project serves are located in The Bronx and Brooklyn, New York City. According to 2018 data, 87% of all Bronx students were eligible for free or reduced priced school lunches, indicating household incomes at or below 130% of the federal poverty line. The North Bronx School of Empowerment (NBSE), which the majority of TAP students attend, is 45% Black and 48% Hispanic or Latinx, with 16% English language learners. A total of 27% have Individualized Education Plans compared with 18.3% district-wide [10]. Chronic absenteeism is significantly higher than average. At NBSE, 39% of students are chronically absent, with the city average 20% [15]. At the Brooklyn school, Meyer Levin School for the Performing Arts, 92% of the students are Black and 5% Hispanic or Latinx, with 3% English language learners. A total of 26% have Individualized Education Plans, and 39.3% are chronically absent. A total of 83% are economically disadvantaged and 6% are homeless [16].

Prior to the present evaluation, comprehensive mental health assessments indicated that 100% of TAP 8th graders witnessed violence in their family and/or community. A meaningful number reported feelings of anxiety, depression, and/or post-traumatic stress [17].

No social workers are listed at either school. With 370 students, NBSE employs one counselor. With 552 students, Meyer Levin employs one counselor.

Mentoring programs like TAP have been found to decrease the impacts of adverse life circumstances [18]. Programs that utilize mentors trained in the helping professions lead to greater positive effects, as do programs, like TAP, that require frequent mentor–mentee interactions [19]. A trusting and caring adult can increase a student’s self-efficacy and resilience if the adult encourages the youth’s agency and power and promotes self-confidence and productive decision making, among other skills [19,20].

1.1.3. The Arthur Project Program Goals

TAP’s seven goals encompass the range of academic, mental, emotional, and social factors that warrant development in most underserved middle-school youth [13,20].

• Cultivate youths’ social-emotional wellness and mental health.
• Increase academic engagement.
• Provide youth with leadership, goal achievement, problem-solving, knowledge, self-advocacy, and communication skills to reach their full potential.
• Provide the skills to successfully transition to high school and beyond.
• Foster family engagement.
• Encourage culturally responsive community and civic engagement.
• Apply an antiracist, social justice lens to mentoring in furtherance of awareness and attunement to wider social impacts and social change.

1.1.4. The Arthur Project Focus

TAP’s inputs include financial, in-kind, school facilities, and multiple university partnerships that provide social work graduate and undergraduate student-mentors, as well as research and consultation. Financial inputs include donations and small grants. In-kind donations include consultation and evaluation services. School facilities are donated for all mentoring activities, with the exception of space rented during the COVID pandemic. TAP was founded in 2017 and originally served 24 students. In the evaluation year, 2022–2023, TAP partnered with the New York University Wagner School of Public Service to provide mentoring services by social work students.

TAP implemented the mentor program with 200 students in 6th through 8th grades in two schools in The Bronx and Brooklyn in New York City. Each of the 30 mentors (social work interns) was matched with 4–10 mentees. Mentors and mentees can spend up to 500 h in focused programming, far exceeding most mentoring programs [21]. Mentors provide services to students through individual school-based counseling sessions, small afterschool group work, and Saturday community-based activities. As part of the program, TAP also supports a group of parents and guardians.

Consistent with TAP’s focus on healing and liberation, multiple program components, events, and goals are based on the framework of French et al. [22] for radical healing in communities of color. These include collectivism, critical consciousness, racial hope, strength and resistance, and cultural authenticity and self-knowledge. In 2023, TAP also introduced healing-centered engagement as an anti-oppression alternative to trauma-informed care [23].

2. Methods

Until the present evaluation, the TAP methodology was implementation of the program. Mentors and mentees meet individually during the school day 1–2 h weekly. They also meet in small mentor “families” 2 to 4 h weekly. Mentors participate too with the students in the Saturday activities that further a sense of community and provide experiential opportunities.

For the evaluation, the research question was as follows: Does structured, intensive in-school therapeutic mentoring with social work interns increase the socioemotional, cognitive, noncognitive, and academic skills and engagement of underserved middle-school students? A mixed method approach, with quantitative surveys and qualitative focus groups, was deemed best to capture the TAP outcomes as comprehensively as possible and provide increased validity. Quantitative surveys elicited participants’ self-assessments in many mentored areas. Focus groups prompted subjective, in-depth responses from participants and other key stakeholders [24]. Section 2.8 further describes the frequency of evaluation activities.

2.1. Participating Students

Within the qualifying schools, 6th-, 7th-, and 8th-grade students are generally referred to TAP by the schools based upon a pattern of absences, poor grades, negative behavioral challenges, chronic absenteeism, and social-emotional needs. Referrals may come from the youth, the youth’s family, or school staff. All youth voluntarily participate in the program. Approximately 95% of the mentees were people of color.

2.2. Participating Mentors

The mentors are social work clinicians-in-training who have chosen to apply their internship clinical training hours to service in the TAP program and are supervised by university and external supervisors and the TAP Executive Director. Screening and interviewing of the mentors takes place for their knowledge of therapeutic mentoring and social justice. Approximately 95% of the mentors are of minority ethnicities.

2.3. Therapeutic Mentoring

The Arthur Project has adopted many of its philosophies and structured programming elements from therapeutic mentoring. Creating individualized growth plans and in recognition of the youths’ stated goals, the mentors help them develop their communication skills, conflict resolution abilities, and confidence-building [25]. Mentees and mentors are “matched” during group matching social and networking events [26].

Responding to the mentees’ requests, TAP also instituted group mentoring. The groups can decrease mentees’ feelings of isolation, increase their confidence and commitment, and provide networks of multiple role models [27].

Mentors are encouraged to advocate for their mentees in, for example, maintaining contact with caregivers, ensuring mentees’ access to services, introducing them to contacts and role models, and vouching for them in court proceedings. Advocacy can help mentees’ development and self-confidence and widen social networks that may lead to promising relationships [28].

2.4. Mentor Training and Activities

TAP mentors are trained in healing-centered engagement; this is an alternative approach to trauma-informed care. Recognizing the current mental health crisis, especially among youth in under-resourced communities, TAP added mentee pre- and post-mental health screenings. Applying strength-based practices from social work theory, TAP mentors emphasize mentees’ self-determination, resilience, courage, strengths, abilities, talents, and resources [29].

Consistent with best practices, all mentors receive initial and ongoing weekly training related to mentoring. The training includes relevant research studies to bridge theory and relate it to therapeutic mentoring [26,30].

Furthermore, aligned with TAP’s value of working toward equality and equal justice for all, mentors participate in social justice training modules aligned with “critical mentoring”, the understanding of social justice issues that impact the mentees’ lives [31] (p. 5). Vital to this training is the recognition of a possible sociocultural gap between the lived experiences of the majority of mentors and mentees. National data indicate that approximately 76% of mentees served are Black and Indigenous People of Color and 53% of mentors are White [32]. However, this was not the case with TAP. In the recruiting and screening, mentors were discovered who were of similar ethnicities as the mentees and shared similar lived experiences. Whatever their backgrounds, as the National Association of Social Workers Code of Ethics [33] (n.p.) states, “Social workers pursue social change, particularly with and on behalf of vulnerable and oppressed individuals and groups of people. Social workers’ social change efforts are focused primarily on issues of poverty, unemployment, discrimination, and other forms of social injustice”.

Following Finn’s [34] social justice approach to social work, TAP mentors utilize an antiracist framework through a seven-step process implemented in narrative therapy, art, writing, and other methods. The steps are (1) affirmation and acknowledgement of importance of race, (2) stories of racial experiences, (3) validation of world view and worth, (4) naming of racial oppression experiences, (5) recognition of devaluation, (6) counteracting of devaluation, and (7) rechanneling of rage [34] (pp. 276–277).

For every mentee, mentors are required to submit quarterly reports that include individualized growth plans, biopsychosocial assessments, and progress notes. At the end of each year, mentors also write closure letters to their mentees and letters to the next mentor class. Finally, compatible with best practices, TAP provides mentors crisis training and support throughout the entire period [35].

2.5. Academic Engagement

TAP mentors seek to increase student academic engagement through collaborative goal setting focused on the youths’ individual school and career goals. Engagement is a strong predictor of academic achievement, behavior, grades, test scores, attendance, graduation, and pushout/dropout. If support and opportunities continue, engagement generally builds over time [36]. Noncognitive and cognitive skills often intersect. Thus, as TAP mentors seek to increase student academic engagement, they are aware that social-emotional skills, such as persistence, perseverance, and confidence, are important to develop to increase engagement.

Although TAP does not directly serve mentees in academic subjects, school-based mentoring can increase academic achievement through an increase in students’ beliefs in their own academic capabilities, including organizational, study, and test-taking skills. A strong mentoring relationship can predict improved academic outcomes [37].

2.6. Mattering

Mattering involves individuals’ perceptions that they are important and valued by other people [38]. The three dimensions of mattering are these: Do I matter to myself? Do I matter to the world? Do I matter to my mentor? Many youth served by mentoring programs may face relationship challenges. Mentoring programs such as TAP build upon and create trusting relationships and supportive communities that can help children develop their full potential [9]. Developing a sense of mattering focuses on the power of relationships in a youth’s life [38].

2.7. Purposeful and Positive Relationship Ending

In accord with social work practices [39], the closure processes are carefully considered and implemented to provide mentees with positive experiences. A large number of volunteer mentoring relationships suffer from negative closure experiences for the mentee [40]. To assure positive closure experiences focused on the mentee, TAP educates mentors on relationship closure and a formal “graduation” or “Moving Up” ceremony is held at the end of every school year [41]. The mentors’ closure letters to their mentees celebrate their achievements.

2.8. Documentation of TAP: Surveys and Forums

The evaluation took place during the 2022–2023 school year, with administration of surveys and forum meetings throughout. Aligned with studies conducted with youth participants, TAP did not utilize negatively worded questions [42]. The surveys were kept to about 30 min each, in accordance with best practices for youth and related assessments. Student retention was 100%.

2.8.1. Quantitative Surveys

Several surveys measured TAP’s impacts and the strength of the mentee–mentor relationship. These evaluations were further triangulated with teacher and caregiver surveys. The surveys were short-answer, with Likert-type scales, and frequencies and percentages were calculated, with the results displayed in tables and figures. The software Bonterra (https://www.bonterratech.com/, accessed on 15 January 2023) by Social Solutions was used for descriptive data analysis.

Because middle school is a time of intense development and TAP is a 3-year program, the survey administration for mentees was divided into two groups: 6th graders and then 7th and 8th graders combined. Approximately 31 6th graders and 62 7th and 8th graders completed the surveys (N = 93). Twenty-four mentors, 19 teachers, and 7 caregivers also completed the surveys. All quantitative surveys were administered in April and May 2023.

• Mentee surveys were retrospective pre- then post-tests.
• Mentor surveys were post-tests.
• Teacher surveys were post-tests.
• Family/caregiver surveys were post-tests.

2.8.2. Qualitative Forum Groups

The focus group participants were invited by the staff to participate, with no incentives offered. The evaluator guided groups of 5 to 25 through predetermined topics based upon the evaluation questions and domains. Three focus groups were held with youth, four with mentors (each volunteered for one group), and one with TAP staff and caregivers. All sessions were approximately 30–60 min and held virtually between February and April.

At the start of each group, the evaluator provided a brief overview of the purposes and questions asked. She also reassured participants of the voluntary, private, and anonymous nature of the responses. She explained too that the sessions would not be recorded, participants could withdraw at any time, and no names would be included.

A total of 25 mentees and 19 mentors discussed their lived experiences, with prompts from the evaluator. For example, mentees were asked if they felt safe with their mentors. Mentors were asked about the strength of their relationships, if and how they built trust, and changes they observed in their mentees.

2.8.3. Measurement Tools and Data Collection

Each TAP program goal had associated measurement tools that included surveys, focus group discussions, and school data (e.g., attendance, grades, and state test scores as indicators of academic achievement). Social-emotional outcomes were measured with a proprietary survey, including perceptions of mattering, life satisfaction, and self-worth, as well as through youth and family qualitative forums, mentor surveys and forums, and by retention in the program. Outcomes related to mental health were measured through a proprietary tool developed from validated tools assessing experiences of anxiety, depression, and PTSD. The youth’s sense of mattering can also impact other TAP outcomes, such as an increase in help-seeking behaviors, confidence, self-advocacy, and liberation. Thus, a measure of noncognitive outcomes aligned with TAP’s activities and events was administered.

TAP staff administered the surveys and assured participants that there were no right or wrong answers and that all responses would be anonymous and confidential. The surveys were adapted from professional organizations, research councils, and universities, including the National Mentoring Resource Center, Columbia University Teachers College, and the Institute for Education.

Mentee response bias was controlled by the pre- then post-tests, administered at the beginning and conclusion of the year, and focus groups [43]. Mentor surveys were administered three times a year to monitor the mentor–mentee relationship. At the end of the school year, a teacher survey and a family/caregiver survey were administered.

Table 1. The Arthur Project goals and measurements.

The Author Project Goals	Measurements
1. Cultivate youth’s sense of mattering (social-emotional wellness and mental health) through therapeutic mentoring.	Mattering Index (survey questions; overlap with other domains, including academic engagement) Youth Qualitative Forums Family Qualitative Forums
2. Increase academic engagement.	Attendance, grades, state test scores, graduation pushout/dropout Teacher Survey Family Survey
3. Provide youth with leadership, goal achievement, problem-solving, knowledge, self-advocacy, and communication skills to reach their full potential.	Noncognitive survey domains regarding self-advocacy, leadership, and persistence Qualitative focus groups with students and families Mentor quarterly reports Mentor Survey
4. Provide the skills to successfully transition to high school and beyond.	Student survey and forum questions regarding goal attainment and additional noncognitive and cognitive domains, such as help-seeking skills
5. Foster family engagement through the Education and Family Advocacy Programs.	Family survey questions and focus groups
6. Encourage culturally responsive community and civic engagement.	Student event outcomes, surveys, and qualitative forums
7. Apply an antiracist, social justice lens to mentoring in furtherance of awareness and attunement to wider social impacts and social change.	Mentor survey, mentee forum groups

shows the TAP goals and related measurement tools.

2.9. Data Analysis

All data analyses took place immediately following administration of each component. In the quantitative component, Bonferra software programs were used for the descriptive statistics. Results were tabulated with frequencies and percentages and displayed in tables and figures.

In the qualitative component, focus group analysis was used. This type of analysis is nonlinear and iterative [44]. The evaluator took detailed notes on participants’ responses, reviewed her notes and added additional observations and reflections, with data reduction, collapsing similar ideas into themes [44]. Themes were further categorized by the frequency of repetition and presented in the findings.

In areas where the focus group participants’ responses intersected with one or more domains, the evaluator combined them holistically for a more authentic analysis. For example, strengths identification overlapped with goal achievement and problem-solving. Mentees’ discussions of building trust, being heard, and reaching goals were related to confidence and persistence.

3. Results

For each of the seven TAP goals, results include illustrative quotes, statistics, and emergent themes. With responses to approximately 119 survey questions, the evaluator has presented particularly significant items, illustrated by highly selective and representative figures. For simplicity, the narratives combine “Strongly agree” and “Agree” statistics, although the figures show the differentiations.

3.1. Goal 1: Cultivate Youths’ Social-Emotional Wellness and Mental Health

My mentor makes me feel like I matter. (6th-Grade Mentee)

I have a mentee who was so quiet. She started speaking up and now helps others.

(Mentor)

In the quantitative findings, data that considered social-emotional skills were divided by domains. However, as noted, noncognitive skills often overlap with cognitive skills. For instance, a sense of mattering often gives rise to greater confidence. Social-emotional strengths are also related to cognitive skills such as communication—when youths feel more confident, they are more likely to communicate. Both the survey data and the qualitative themes indicated mentee increases in trust, mattering, and confidence. Teacher surveys indicated that youth increased help-seeking behaviors, skills development, and social and academic engagement.

3.1.1. Persistence, Perseverance, and Confidence

Table 2. Student results before and after TAP in persistence, perseverance, and confidence.

Statement	before TAP	after TAP
I finish what I begin.
6th Graders	24%	29%
7th and 8th Graders	34%	49%
When I have a goal I stick with it.
6th Graders	25%	36%
7th and 8th Graders	55%	78%
I am more likely to stick with my homework. 6th Graders 7th and 8th Graders	57% 59%	75% 83%

displays student results for the first social-emotional domain, Persistence, Perseverance, and Confidence.

Of the 6th graders, after TAP, 25% strongly agreed that they had changed their views positively of finishing what they began. Of the 7th and 8th graders, regarding sticking to a goal, 23% more strongly agreed that they stuck with their goal after TAP. Youth who believe in themselves are more likely to try harder and persist. Likewise, sticking with homework until completed suggests that mentees had persistence and perseverance. After TAP, 18% more 6th graders agreed that they stick with their homework, and 24% more 7th and 8th graders agreed. Especially relevant for the 7th and 8th graders was doing well in high school. Before TAP, 32% agreed and after TAP 46% agreed that this was a concern.

3.1.2. Self-Advocacy and Self-Help

In the second social-emotional domain, Self-Advocacy and Self Help, several survey questions indicated the extent to which mentees’ self-advocacy and self-help skills increased during the year. For 7th and 8th graders, an observable increase took place in self-help seeking skills. Figure 1 illustrates these results (blue indicates after TAP). Prior to the school year, 28.81% strongly agreed that they asked for help if they needed it. After the year in TAP, 44.07% strongly agreed.

Similarly, 47% of the teachers agreed and 6% strongly agreed that their students increased help-seeking behaviors. A total of 29% somewhat agreed (Figure 2).

3.1.3. Mattering

Questions that measure the mentees’ trust in their mentors act as a foundational relationship indicator for youth to gain a sense of mattering. Trust is built through listening, support, and reciprocity even when mistakes are made. For the 6th graders, 100% agreed that they trusted their mentor. Additionally, 97% agreed that their mentor made them feel safe. Sixth graders also felt that their mentors listened to and supported them, as Figure 3 shows.

Per Figure 3, 100% agreed that their mentor listened to them. A total of 93% agreed that their mentor supported them, even with mistakes. They also felt they mattered to their mentors, as seen in Figure 4.

Figure 4 illustrates the 6th-grade mentees’ sense of mattering. An amount of 36% agreed and 54% strongly agreed they mattered to their mentors. This was almost all mentees, at 96%.

Similarly, in Figure 5, 7th- and 8th-grade surveys revealed that a substantial majority of the youth had a great deal of trust in their mentors and felt safe with them. An amount of 96% agreed that they felt safe with their mentors, and 85% agreed that their mentor supported them, even with mistakes.

On mattering to their mentors, the 7th and 8th graders showed similar results to the 6th graders. A total of 90% said they mattered to their mentors. When asked whether they thought they mattered to others, 74% of the 7th and 8th graders agreed that they did.

3.1.4. Mental Health

For the mentees, no major difference was found in anxiety levels between the pre- and post-tests. Before the program, responses indicated a very high level of anxiety. Depression scores were higher after the program than before. The pre-program test yielded a score of 5.11 (minimally considered); the post-test yielded 8.27, a score indicating moderate levels of depression symptoms. Prior to the program, 38 mentees reported an event that could be considered a traumatic stressor. Of those, 13 reported symptoms matching mild to moderate PTSD. In the post-program survey, 31 mentees reported an event that could be considered a traumatic stressor and 7 reported mild to moderate PTSD symptoms.

3.1.5. Qualitative Themes

My kids have grown in different ways. One mentee was so quiet. She broke out of her shell and now comes to the office to ask if we can have a session! (Mentor)

For the mentees, several strong themes emerged during their qualitative forums that indicated the youth felt safe with their mentors and TAP staff and that their mentors listened to them, attesting trust. Additional themes about trust were strong, as were the youths’ sense of mattering.

6th-Grade Mentees’ Illustrative Comments: Trust

My mentor supports my opinions.

They trust you and you trust them.

7th- and 8th-Grade Mentees’ Illustrative Comments: Trust

They listen.

I trust her because she knows how I feel. She understands.

She keeps my stuff to herself and doesn’t judge.

I feel comfortable and safe.

I can say whatever I want.

6th-Grade Mentees’ Illustrative Comments: Mattering

You feel more important.

My mentor makes me feel like I matter.

7th- and 8th- Grade Mentees’ Illustrative Comments: Mattering

If I am upset, she always asks if I am ok and makes me feel comfortable.

If I feel down, she makes me have faith in my mind.

We can tell our mentor anything and they listen to us and give to us.

Concurring with the youths’ themes, the mentors overwhelmingly indicated that they built strong relationships with their mentees and that mentees became more open, communicative, and trusting during the school year.

Mentors’ Illustrative Comments: Building Trust and a Sense of Mattering

We have to be our authentic self to build trust.

I opened myself more to my mentees. Once I revealed myself to them, they began to reveal themselves to me.

I recall what they told me, so they know I am listening.

They share more and have increased self-awareness.

She is not as shy and started asking questions in class.

They have more self-awareness.

These themes indicate that TAP increased mentees’ social-emotional skills. Goal 1 was met based upon multiple types of data, including mentor, mentee, teacher and caregiver surveys, and qualitative themes.

3.2. Goal 2: Increase Academic Engagement

I was going to fail. My mentor helped me. (Mentee)

His mentor sees the potential in him, and she lets him know it. (Caregiver)

TAP is a tremendous help with teachers and school staff. (Caregiver)

Many social-emotional skills, such as help-seeking, confidence, and perseverance, are related to academic engagement. Quantitative findings for the mentees (6th graders) indicated that 89% agreed their mentors helped them with schoolwork (not asked of 7th and 8th graders). In corroboration, approximately 38% of the teachers agreed and 50% somewhat agreed that TAP students improved academically, and 41% strongly agreed, 12% agreed, and 29% somewhat agreed that TAP students increased their skill development. Among caregivers, 57% strongly agreed that TAP improved their child’s academic engagement and 15% agreed and 72% agreed that their children had increased academic engagement.

Furthermore, as Figure 6 shows, 56% of the caregivers agreed on finding greater academic support themselves in the school, although this percentage was lower than some other statements.

Other TAP goals, such as communication, goal achievement, and problem-solving, are also related to academic engagement. Yet, as one TAP staff member explained, these domains can be difficult to fully integrate into the complex school systems. TAP has made specific recommendations to partner middle schools to increase collaboration.

3.3. Goal 3: Provide Youth with Leadership, Goal Achievement, Problem-Solving and Communication Skills to Find Their Strengths and Reach Their Full Potential

We looked up the word “charming”. We found her strengths. (Mentor)

We reminded them of their awesomeness. (Mentor)

Many of these skills overlap, such as problem-solving and goal attainment. The survey data supported mentees’ growth in problem-solving, goal achievement, and leadership capabilities. Problem-solving includes the ability to resolve difficult experiences; 96% of the 6th graders agreed that their mentors helped them solve problems.

For the 7th and 8th graders, prior to the school year, 25.42% strongly agreed that they tried many ways to solve a problem; at year’s end, 47.46% agreed they used many ways. The mentors corroborated these results: approximately 84% agreed that they helped their mentees solve difficult problems, and 86% agreed that they solved problems with their mentees.

Regarding goal achievement, the survey data illustrated that 6th-graders’ mentors helped them set goals (Figure 7), with 89% agreeing. And 82% agreed their mentors helped them reach their goals (Figure 8). Of the 7th and 8th graders, 82% agreed that they “accomplished a lot of things together” with their mentors (Figure 9).

The mentors supported these results. A total of 40% agreed that their mentees achieved the goals they set, and 71% agreed that they helped their mentees complete a task. Furthermore, 76% agreed that they solved problems together.

In the leadership domains, the results also showed progress. Leadership includes others looking up to one and speaking out even if friends disagree. Based upon survey data, 7th and 8th graders increased both capabilities during the year.

In the qualitative forums for Goal 3, mentees reported that they worked on accomplishing goals and could talk freely with their mentors. Mentors said they often problem-solved with their mentees. A stronger theme emerged around communication; most mentors explained that their mentees were more communicative as the year went on than at the beginning.

Mentees (All Grades)

We worked on a goal for me to stay quieter when the teacher is talking.

I got my grades up.

I tell her I want to be a dancer. Sometimes I want to give up and she tells me to keep my dream.

There are people I don’t like and things that are happening at school. She gives me coping strategies.

I have anger issues and it used to bottle up. She helped me express myself.

Mentors

We don’t offer solutions. There is a lot of room for problem solving.

We support them in resolving difficult situations.

At first, my mentee was always late. He is not late anymore and goes to bed earlier.

I see them opening up in class and communicating more.

In summary, a large majority of mentees reported strong indicators of growth in problem-solving, goal achievement, leadership, and communication. Mentors corroborated this progress through their observations and qualitative responses. Teachers in surveys and qualitative responses also reported students’ improvement in greater academic engagement, implying improvements in cognitive areas [45].

3.4. Goal 4: Provide the Skills to Successfully Transition to High School

In addition to academic engagement, this goal encompasses several TAP goals related to mental health, social-emotional skills, problem-solving, goal achievement, and communication. All of these are related to successful high school transition. TAP also offers 8th graders supplementary programming in the spring and summer prior to high school to support successful transition.

Recently, TAP began to support 8th graders’ high school application preparation. Qualitative findings revealed that many mentees and their families did not feel equipped to complete these complex and often competitive applications. As Figure 10 shows, 48% of mentees, of whom about half were 8th graders, agreed that their mentors helped them apply for high school.

3.5. Goal 5: Foster Family Engagement through the Family Educational Advocacy Program

The Arthur Project recognizes that true impact can only be made when a child’s family is also supported and supportive. Generally, students with parents or guardians who are engaged with their learning attend school more regularly, have higher grades and graduation rates, and are more academically engaged than students without engaged caregivers [45]. Positive mentoring relationships are more likely when mentees’ parents are acquainted with the mentors and their expectations coalesce [46].

The TAP Education Advocate works closely with families to support them in whole- family and mentee academic engagement and student transition to high school. The Advocate trains mentors in supporting caregivers and families and collaborating with school and community-based organization partners in developing and implementing workshops, training, events, and gatherings for caregivers and families. Mentors are also encouraged to contact mentee families regularly; mentors who know the families have better outcomes with mentees than those who do not [46].

The caregiver sample was small, seven, but the reports were positive about TAP’s impacts. A total of 72% also agreed that TAP helped their children attend school more regularly, as Figure 11 shows. Most of the caregivers also noticed positive behavior changes in their children, with 72% agreeing.

The caregivers also recognized help for themselves. A total of 56% agreed that TAP helped them find academic support in school. A total of 72% said TAP helped them with support of staff and teachers (Figure 12).

Regarding family engagement, the mentors’ qualitative responses were mixed. Some mentors reported talking with caregivers on a regular basis, and others reported occasional difficulty in communication. This feedback accords with TAP’s efforts to cultivate more ongoing family engagement. For additional caregiver outreach, new training of mentors has been implemented.

3.6. Goal 6: Encourage Culturally Responsive Community and Civic Engagement

This goal had two primary features: engaging those in the TAP community in authentic and meaningful ways, and providing youth opportunities for civic participation and culturally responsive community trips. The mentees acted on this goal after the evaluation: they volunteered at a local aging facility, created gardens at their school, and participated in local park cleanups.

Research confirms the importance of shifting the influence and power to the people in the communities, so that community voices are heard [47]. TAP invites community constituents to contribute suggestions. For example, the TAP Educational Advocate focused on supporting whole-family goal setting. However, parents and caregivers requested support primarily around their children’s education, so TAP adapted to meet that need.

Youth who participate in community engagement have greater academic performance and social-emotional skills than youth who do not participate, especially in low-income communities [48]. To this end, TAP sought feedback from the mentees and the families about preferred activities and programming and implemented the suggestions.

TAP also supports youth in culturally responsive community engagement. Through Saturday activities, youths visit museums, parks, and other places of interest in their communities (e.g., Botanical Gardens, Black Comic Book Exposition, Museum of Natural History). Such trips have been shown to increase academic engagement and increase learning and cultural awareness [49]. During TAP qualitative forums, mentees exclaimed, “We need more Saturday field trips!”.

3.7. Goal 7: Apply an Antiracist, Social Justice Lens to Mentoring in Furtherance of Awareness and Attunement to Wider Social Impacts and Social Change

In accord with a social justice approach, affirmation and acknowledgement are important to healing wounds of racial trauma. Validation of young people’s world views and worth and providing them with resources builds their strengths and counteracts devaluation [35]. All mentees agreed that they spent more time finding their strengths since joining TAP.

The quantitative results also indicated that mentors utilized their social work and TAP training to apply an antiracist and social justice lens to mentoring. Survey data found that 55% of mentors used emotional self-regulation with unjust systems, trauma, and grief and loss training, and 50% reported using young people’s power. Several mentors requested more time to discuss antiracism and anti-oppression with their mentees. This theme was integrated into the evaluation recommendations.

My mentor helped with how I could be better. (6th-Grade Mentee)

Mentees’ qualitative data indicated they felt they could talk about anything with their mentors, that their mentors made them feel important and comfortable, and that they could express themselves—indicators of empowerment. Qualitative themes for mentors illustrated that they discussed unjust systems and provided safe spaces for mentees.

I understand my mentees’ racial identities and I am able to perceive social, political and community cultures. (Mentor)

We talk about unjust systems, how they are perceived, and how they perceive. (Mentor)

I revealed myself to them and they began to reveal themselves to me. (Mentor)

3.8. Limitations

Despite its successful results, this evaluation had several limitations. First, it was conducted during the school year 2022–2023 and therefore represents a single moment in time. Changes for all stakeholders undoubtedly took place over the year. Second, a control group was not used. Given the age of the participants, TAP’s size and budget, the need for parental consents, school cooperation, and the practicability of data acquisition, a control group did not seem feasible. Results may have been interpreted differently with such a group. Third, in the focus groups, some participants may have been hesitant to share their thoughts, with more outspoken participants controlling the conversations. Fourth, some students might have felt pressured to agree with others, despite contrasting experiences [24]. Nevertheless, the results reported reflect best practices in research.

4. Discussion: Recommendations and Conclusions

Carefully designed and implemented mentoring programs can increase the social-emotional and cognitive skills of youths from historically oppressed communities; decrease historical, personal, and familial trauma; and honor the youths’ lived experiences and voices [50]. As previously discussed, programs that promote youths’ trusting relationships with skilled social workers and specific skill development often provide the most benefits for the youths.

4.1. Evaluation and TAP Goals

The evaluation successfully met its purposes. The first was determination of the extent to which the youth suffered from depression or post-traumatic stress. The second was mentees’ growth in social-emotional and cognitive skills as perceived by both youths and mentors. The third was caregivers’ perceptions of mentees’ greater academic engagement.

With regard to anxiety levels, pre- and post-test results indicated no large differences, although the scores in both cases indicated high anxiety. Depression scores were higher post-program than pre-program, as were traumatic stressors. The scores point to the need for ongoing counseling for these youths. In addition, several mentors reported that their mentees seemed more comfortable and responsive with the assessment at the end of the year. These observations may indicate that the mentees had greater confidence in themselves and this was communicated more openly after TAP.

With regard to social-emotional growth, student surveys in all three grades showed growth in persistence, perseverance, and confidence (Table 2), and 7th- and 8th-graders’ self-advocacy and self-help (Figure 1). For cognitive skills growth, teachers observed students’ improvement in help-seeking (Figure 2) and 6th-grade students reported increased mentor help in setting goals (Figure 7) and in reaching goals (Figure 8). Seventh and eighth graders recognized that their mentors helped them accomplish many things (Figure 9). It would appear that skilled one-to-one mentoring greatly aids these youths gain confidence and persistence in succeeding in school and preparing for the future.

With regard to greater academic engagement, the results were similar. Mentees’, teachers’, and caregivers’ surveys and qualitative results indicated TAP increased mentees’ engagement. The increases are evident in Figure 10 for mentees, Figure 2 for teachers, and Figure 6 for caregivers.

4.2. Similarities with Other Programs

To put TAP into perspective, similarities are evident between TAP program and the International Coaching Federation (ICF) programs and Philosophy for Children (P4C). These similarities are important with coaching becoming more popular in recent years for individuals’ control of their life journeys [51]. The ICF values correlate with the competencies and guidelines for social work supervision [52,53], including ethical guidelines for practice, sharing power, honoring self-determination with clients, active listening, and motivational interviewing to assess and develop goals. TAP and P4C have similar goals as well, especially with regard to students’ challenges today of increasing emphasis on the sciences, creation of learning environments, critical thinking, fostering of social and emotional development, and improvement of resilience [54,55].

4.3. Program Recommendations

The evaluation highlighted several additions that would make TAP more effective and fulfill its wider purposes. Some of these have been initiated.

• To address the youths’ mental health symptoms, especially regarding PTSD and ongoing depression, TAP is instituting a program in which students with the greatest needs are matched with mentors with the highest levels of experience. TAP has hired an additional supervisor to oversee these matches and will delay matching pending thorough assessments.
• Based on staff identification of further needs and students’ and families’ feedback, additional programming has been added. A new Young Men’s Group discusses life skills, health education, hygiene, and healthy relationships and is open to the school community for greater social networking opportunities and peer support.
• Based on previous mentor input, new training, supervision, support, and professional development will focus on several areas, including attention to youth with highly complex needs [25] and mentees’ political, social, and economic challenges. Beginning with the 2023–2024 school year, these youths were matched with the most experienced interns.
• Data collection and analysis software will be upgraded to serve more students and calculate results for larger numbers of students. During the school year, TAP will continue to track retention, attendance, grades, and state test scores and monitor disciplinary infractions.
• Several new tools will be employed. For example, instead of the Classroom Behavior form, the widely accepted and validated Functional Behavioral Assessment tool will be used [56]. This tool allows mentors to observe over many points in time students’ behavior in the classroom. A tool will be utilized that captures students’ perceptions of “mattering” regarding family, friends, school, and the community. Another new tool measures quality of the mentor–mentee relationship [57]
• Resource Library: In response to mentors’ requests, TAP increased the online resources and developed a new retrieval system. The resources are accessible to all mentors and staff.
• Youth Advocacy: TAP engaged the NYU Wagner School of Public Service for design of a youth advocacy plan to disrupt the school-to-prison pipeline. Youth advocacy serves youth and communities, including healing, empowerment, social engagement, and civic pride [58].
• Caregiver Engagement: The caregiver responses were minimal, likely for many reasons outlined in the literature [46]. With family engagement crucial to a child’s academic success [45], through the Education Advocacy Program, TAP will increase efforts to engage them with a series of open houses, orientations, and individual staff and mentor visits.
• Alumni Program Plan: With the first cohort of mentees transitioning to high school, TAP developed a means for maintaining support and engagement. The intention is that students remain in The Arthur Project for life, ultimately becoming mentors themselves. Based on research that indicates longer participation results in longer-term positive outcomes [37], TAP asks program alumni to facilitate the 8th Grade Emerging Leadership Institute. To this end also, TAP alumni will be contacted for surveys and interviews on high school outcomes.
• Emerging Leaders Institute: Following their 8th grade year, mentees may participate in TAP workshops and other opportunities that further prepare them for high school. These alumni may also become peer mentors, building on their social and interpersonal relationships, learn how to manage conflict, and take part in social justice advocacy in support of the important high school years [59].
• Town Halls: The Town Hall is a monthly meeting of stakeholders facilitated by the Student Leadership Council (TAP students) and held monthly for open communication with the students.
• Community: Research confirms the importance of shifting the influence and power to community residents [47], especially true in low-income communities and communities of color. Recently, at families’ requests, TAP hosted broader community events. Additional feedback will be sought from community members and stakeholders, for example, requesting input on program improvements and greater student engagement.

4.4. Conclusions

Based on the quantitative and qualitative results, the TAP program of 2022–2023 was highly successful. Goals 1, 2, 3, and 4—social-emotional wellness; academic engagement; leadership, problem-solving, self-advocacy, communication skills; skills for high school transition—were met very successfully. However, mental health was not measurably improved as a result of TAP participation. More mentees reported depression and PTSD symptoms after the program. Based on their greater sharing after the program, it can be assumed that their bonds and trust had grown and become solidified. Perhaps the end-of-year assessment was more accurate than that at the start. Nevertheless, the depression and PTSD symptoms warrant closer examination. With the assumption that the low-income community environment was stressful [60], certain elements in the program may have contributed to this result, and they should be identified. Furthermore, the mentees may have felt undue pressure to succeed, and perhaps additional counseling is called for. In summary, a major consideration may be how the results can be improved.

Goals 5, 6, and 7—family engagement; culturally responsive community engagement; antiracist, social justice lens to social impacts and change—were met successfully. Caregivers gained greater trust in the school; mentees enjoyed the community activities and outings. Mentors reported meaningful discussions on social justice, racial identities, and “the system”. However, no measurements were implemented for mentees on this goal. Possible changes and greater understanding pre- and post-TAP might have shed more light on their greater awareness, as well as additional focus groups on steps that can be taken to combat injustices. In response to these findings, TAP increased the intern training on social justice and antiracism and refined the mentee selection criteria to include more alignment of the students’ lived experiences with their mentors’ lived experiences.

The research question was as follows: Does structured, intensive in-school therapeutic mentoring with social work interns increase the socioemotional, cognitive, noncognitive, and academic skills and engagement of underserved middle-school students? The question was answered in the affirmative. From TAP participation, students gained enlarged experiences, opportunities, and world views. They benefited across a spectrum of socioemotional, cognitive, and noncognitive domains (e.g., self-development, social-emotional skills, trust, confidence, academic development).

Additionally, mentors gained experience in the real-life application of social work theory and practice and saw more vividly than textbook examples into underserved students’ lives. They also better understood underserved students and helped them increase empathy, leadership, confidence, cognitive skills, and many other qualities. Caregivers gained trust in and respect for school staff and realized the importance of their academic support of their children. Teachers saw how the mentoring relationship and caregiver support helped the students in academic engagement. Although the caregiver sample was admittedly low, possible greater outreach from teachers may help more caregivers feel greater comfort with them and in the school environment.

Several implications can be drawn from this evaluation. Students would benefit from additional time with mentors. Mentors would benefit from additional training, especially in mental health symptoms. Caregivers would benefit from more individual attention from teachers. Teachers could benefit from greater acquaintance with TAP and align class lessons with field trips and guest lecturers, as planned for 2024–2025. Teacher workshops are also planned for development of deeper relationships with the teachers.

The TAP programs will continue to promote an environment of continuous improvement, accountability, and impactful outcomes with evaluations and valid measurement tools. This evaluation has demonstrated its value. As TAP enters in its fourth year of operation, the staff looks forward to wider impacts of the program in schools and communities and to supporting students transitioning into high school. TAP remains committed to expansion in the delivery of high-quality services to underserved middle-school students and families to help prepare the youth for high school, reach their potentials, develop their skills and talents, counter racism and oppression, and advance social justice.