IGNITE Goals and Outcomes

The UNC Asheville QEP is designed to impact three overarching goals: Belongingness, Foundations for Learning, and Persistence. Positively impacting Belongingness and Foundations for Learning are short-term (proximal) goals, and increasing Persistence is a long-term (distal) goal. The IGNITE program is likely to have an immediate and significant impact on Belongingness and Foundations for Learning because the IGNITE sessions have been designed to support these goals, affording students opportunities to get to know themselves, each other, and the University. Our theory of change assumes that change in Persistence will take more time to observe, and therefore we plan to begin measuring change in this goal during Year 2 of the QEP. Associated with each of these goals are discrete, measurable outcomes.  See figure below — short-term goals in orange; long-term goals in yellow.

Each of the IGNITE sessions has been designed to include outcomes and objectives supporting these three main goals. IGNITE students will increase self-awareness through the exploration of values, beliefs, and identity. They will also develop greater understanding and appreciation of others through topics related to diversity, equity, and inclusion. The sessions are designed to strengthen interpersonal skills and help students develop the confidence to interact with new people, especially those who are different from themselves. The highest level student outcome for IGNITE is to understand one’s role and responsibilities within the larger community to become a change agent in the future. Additionally, students will learn about UNC Asheville’s academic resources, campus traditions, and opportunities to engage in high impact practices.

Measurements

Belongingness. Students’ sense of Belonging will be measured using three self-report surveys: the Continuing Student Survey (CSS) Belongingness Scale; items from the UNC Asheville Chemistry Scholars survey, an instrument that has been used in the Chemistry department for several years; and two subscales from the Hello Insight measure of social emotional learning (SEL).

Continuing Student Survey, Belongingness Scale

The CSS was developed to provide a comprehensive look at why students were leaving UNC Asheville and to better understand the characteristics, behaviors, and attitudes of those students who do persist. As described in Section II, survey items were based on a literature review of best practices and examination of instruments in use at other COPLAC institutions. Belongingness emerged as a significant discriminator between students who stayed vs. left UNC Asheville. The six items on this scale (see Table 1; see Appendix A for full measure) are rated according to students’ level of agreement vs. disagreement (1 = strongly agree, 4 = strongly disagree). 

Chemistry Department Survey Questions

Emotional engagement measures of the Chemistry Scholars Program include student perception surveys validated in the literature linked to motivation in the academic major (the MUSIC Model of Academic Motivation Inventory; Jones 2009) and students’ level of agreement for questions related to sense of belonging to the academic major and to the university (Goodenow 1993). Students are asked to rank the degree to which they agree with the following survey questions using a 6-point likert scale ranging from 1=Strongly Disagree to 6=Strongly Agree for both measures:

University items:

Sometimes I feel like I don’t belong here.

I wish I were at a different college.

Academic major items:

I feel like a real part of the XX major at UNC Asheville.

I feel proud of belonging to the XX major at UNC Asheville.

Hello Insight, Promoting Peer Bonding and Engaging Authentically Scales

Hello Insight is an online platform that evaluates SEL outcomes in youth development organizations. Two factor scales from the Hello Insight post test will be utilized in the QEP assessment: Promoting Peer Bonding and Engaging Authentically. Both of these factors have been shown to have good reliability (α=0.79 and α=0.86, respectively). According to Hello Insight (2023), Promoting Peer Bonding is “a young person’s experience with an adult who helps them work with peers in teams and groups, creating a safe space to learn with and from one another, exploring similarities and differences, and developing deep bonds and relationships” (Hello Insight, 2023). Engaging Authentically is defined as “a young person’s experience with an adult who takes the time to get to know them by listening and understanding who they are — their cultural and lived experiences, their interests, and their passions.”  Both of these factors support students’ sense of community and connectedness.  See Appendices D and E for the full Hello Insight pre and post surveys.

Foundations for learning. Foundations for learning will be measured using one self-report instrument (from Hello Insight), and two observational measures (AAC&U VALUE Global Learning Rubric and AAC&U VALUE Foundations for Learning Rubric).

Hello Insight, Academic Self-Efficacy Scale

Social emotional learning has been shown to promote several long-term outcomes, including academic self-efficacy, which in turn is associated with academic success (e.g., Kanopka, Claro, Loeb, West, & Fricke, 2020).  We will assess Academic Self Efficacy, defined by Hello Insight as “a young person’s motivation and perceived mastery over their own learning, school performance, and potential to attain academic success.” This measure has good reliability, α=0.78. Students will complete both pre and post tests of Hello Insight.

Personal and Social Responsibility

According to the Global Learning VALUE Rubric (AAC&U, 2014), personal and social responsibility is defined as “the ability to recognize one’s responsibilities to society—locally, nationally, and globally—and to develop a perspective on ethical and power relations both across the globe and within individual societies. This requires developing competence in ethical and moral reasoning and action.” The Institutional Effectiveness Committee of the faculty at UNC Asheville will use this rubric to evaluate student artifacts from their ePortfolio.

Reflection

According to the Foundations for Learning VALUE Rubric (AAC&U, 2009), “reflections tell what students think or feel or perceive. Reflection provides the evaluator with a much better understanding of who students are because, through reflection, students share how they feel about or make sense of their learning experiences. Reflection allows analysis and interpretation of the work samples or collections of work for the reader. Reflection also allows exploration of alternatives, the consideration of future plans, and provides evidence related to student growth and development. Perhaps the best fit for this rubric are those assignments that prompt the integration of experience beyond the classroom.” The Institutional Effectiveness Committee of the faculty at UNC Asheville will use this rubric to evaluate the PAL reflection journals.

Persistence

Persistence will be measured using standard metrics provided annually to IPEDS and SACSCOC: Fall to Spring retention, Fall to Fall retention, and 150% graduation rate.