SOWk 460w Spring 2025 Week 02 - Steps in Program Evaluation

SOWk 460w Spring 2025 Week 02 - Steps in Program Evaluation

title: SOWk 460w Spring 2025 Week 02 - Steps in Program Evaluation date: 2025-01-26 19:53:49 location: Heritage University tags:

• Heritage University
• BASW Program
• SOWK 460w presentation_video: > “” description: >

Program evaluation is an ethical obligation to us as social workers, but also beneficial to staff, agencies, and clients. During week two of this course, we will focus on the why of program evaluation and an overview of the evaluative process. We will also pick our learning teams for the program evaluation you will conduct this semester. The following is the agenda for the class session:

• Why program evaluation as a process
• The steps involved in program evaluation
• Getting settled with the program evaluation assignment

SOWK 460w Week 02 Plan: What we will be covering today

• Why program evaluation as a process
• The steps involved in program evaluation
• Getting settled with the program evaluation assignment

General Questions About the Class

[Whole Class Activity] Open up for general questions about the class.

… We will talk more practically about the program evaluation later in class.

Single Case Design

You all had readings about SCD. It is good stuff. I wanted you to read it, but we aren’t really going to talk about it today.

Social Workers Ethical Responsibility to the Social Work Profession

5.02 Evaluation and Research has 17 specific points…

[Small Group Activity] Think, Pair, Share

1. I want you to go to https://www.socialworkers.org/About/Ethics/Code-of-Ethics/Code-of-Ethics-English/Social-Workers-Ethical-Responsibilities-to-the-Social-Work-Profession
2. Read through it
3. Discuss the following questions some people around you.

What stands stands out to you from this section? What are ways that you imagine fulfilling this in your career?

[Whole Class Activity] Solicit ideas from group.

Reference National Association of Social Workers. (2021). NASW code of ethics. https://www.socialworkers.org/About/Ethics/Code-of-Ethics/Code-of-Ethics-English

Why Program Evaluation: What it can mean for us and our agencies

Program evaluation important task for social workers.

[Whole Class Activity] Why should we do program evaluation?

• Ethical Obligation: Social workers have an ethical obligation to research
• Improve skills: Improve skills and understanding of practice behaviors (consider PEER EBD)
• Identify Needs: Evaluation can help to effectively identify the service needs of client populations (consider being effective the most effective)
• Improve Impact: Evaluation can improve program’s potential impact
• Clarify Program: Evaluation can make participants perspectives about the program more understood (sharing ideas together. Fleshing out what the program does and looks like)
• Support Funding: It is important in receiving funding
• Advocate for Program: It provides an avenue to share about the program and the work they are doing. This can help with program advocacy and development, increasing resources

(Kapp & Anderson, 2010)

Characteristics of Strong Evaluation: Some of the Things We Should Consider

I think it is useful that Royse (2022) framed the rationale for engaging in program evaluation related to being able to process and understand research and use critical thinking skills. He provides a number of aspects we should consider in accepting if something a strong evaluation (which also is what we should build into our own)

• Enough people in the sample size
• Objective data reported (facts, not some testimonials)
• No unfounded claims
• Evaluation of intervention across different types of client
• Use of a valid/reliable standardized instrument for pre/post-test
• Sharing all data (both good and bad)
• Reporting all limitations and challenges, along with the positive
• Provide details connecting the claims with what happened in the intervention

Looking for Evidence: In Social Worker Interventions

Social work is broad field with a lot of potential interventions, and even some things that sound odd or weird (consider EMDR). The practices we engage in should be grounded in strong research and evidence based practice.

I don’t think this means we can’t go new places. Share story of supervisee who wants to do therapy and personal training.

[Small Group Activity] Working in groups of three or four do the following:

Royse (2022) puts forth an activity using Holden and Barker (2018) and their article to consider the evidence. Examine the Holden and Barker (2018) article “Should Social Workers Be Engaged in These Practices?” Try to find one of the URLs still active to review the intervention or practice presented on the website. Then respond to questions below.

• What is the intervention, practice, or product that you examined
• Explore for any objective evidence did you find to support the intervention?
• Were any professional journal articles cited?
• Was any comparison made to a comparison or control group with the same problem?
• Estimate the benefit, in your opinion, clients would receive from this intervention or product. Explain your response.

Steps in an Program Evaluation: A Flow Chart

(Kapp & Anderson, 2010)

The following are the eight steps that we think about taking as a part of program evaluation.

Steps 1: Identify Evaluation Question

The first thing we must do is develop the question we are going to address in our evaluation. This questions guides all aspects of the process.

The general focus of the project is stated in the form of a research question.

Step 2: Conceptualize Question

The second step we take is conceptualize the question.

When we have properly done this step, we will be able to say that…

• The key concepts in the evaluation question are clarified
• A clear conceptual framework is chosen after considering different approaches

Step 3: Operationalize Question

After we know know what our question is asking and what it means, we need to define the terms and concepts in the question.

Specific concepts of the evaluation question are defined.

This means defining the terms within that question. For example, if word family who is meant by term.

Step 4: Pick and Define Method

The fourth step is to pick and define the method being used.

You will notice that the authors choose to use various mon and bi-directional arrows. I have kept these in my rendition of the graphic. That is because there is a flow back and forth and are fluid.

Some of the methods for program evaluation we will be talking about during this course include:

• Reviewing archival data
• Group designs and methods
• Qualitative designs and applications
• Consumer satisfaction

Step 5: Select Sample

Then we select the sample

Evaluation participants/data collected.

We could talk about types of sampling (both probability and non-probability)

Step 6: Data Collection

Step six we gather the data we are going to collect

Step 7: Data Management & Analysis

Step seven is managing the data we have collected securely and our analysis of that data.

Step 8: Utilization of Information

The final step is that the agency needs to use the information. The authors even made the graphic to include an oversized box around this topic to designate this tasks importance.

The use of information for service improvement is primary.

All Steps of Program Evaluation

Practicums and Evaluation Ideas: Thinking About Research Projects

[Whole Class Activity] Restorative Justice style circle.

Review norms:

• Respect the talking piece
• Speak from the heart
• Listen from the heart
• Trust that you know what to say
• Say just enough

Ask Questions:

• Where is your practicum at? (Tell about agency and activities)
• What are some of the things that you are doing and learning at your practicums?
• What are ideas of potential program evaluations that you could imagine at your practicum placement

Reference: Clifford, A. (2013). Teaching restorative practices with classroom circles. http://restorativejustice.org/am-site/media/teaching-restorative-practices-with-classroom-circles.pdf

Developing a Learning Team

It takes different pieces

[Whole Class Activity]

• What kind of qualities would we be looking for or consider to have an effective learning team
• What is important for us all to do as a part of our team

[Whole Class Activity] Take time for people to go and find group mates.

Program Evaluation as an Assignment

• Starting week 5 there are 10 Individual Weekly Journal Entries
• Group Work Plan for the Program Evaluation: Includes evaluation design and data collection with tasks and deadlines assigned to group members
• Agency Logic Model: The logic model looks at how a program is implemented through understanding the program’s resources, staff activities, program processes, and outcomes (often framed as immediate, intermediate, and long-range)
• Executive Summary: Includes introduction, logic model, ethical framework, methods, data/findings, narrative/discussion, and references
• Group Presentation: Inviting agency staff and university faculty

For Next Week (Week 03)

Next Week is Asynchronous What to do for week three…

Tasks:

• CITI Research Ethics and Compliance Training
• Forum Questions