Spring 2026 SOWK 460w Week 07 - Building the Foundation of a Program Evaluation

Spring 2026 SOWK 460w Week 07 - Building the Foundation of a Program Evaluation

title: Spring 2026 SOWK 460w Week 07 - Building the Foundation of a Program Evaluation date: 2026-03-02 08:45:39 location: Heritage University tags:

• Heritage University
• BASW Program
• SOWK 460w presentation_video: > “” description: >

Students will be submitting their logic models this week and should be considering their agency as a whole. The reading from Royse (2023) looks at formative and process evaluations, and we will start class by looking at some of the considerations we should have for process evaluations and then moving on to consider your research question. This will help students to narrow down exactly what part of the program you are evaluating is being examined. Student will complete their logic models this week. During class, we will discuss how to use our logic models to develop a program description. There will also be a discussion about presentations, and students will be able to complete midterm feedback.

The agenda for the class session includes:

• Developing a research question
• Presentation planning
• Developing a program description
• Midterm feedback

Learning objectives this week include

• Critically reflect on contributions to the program evaluation project.
• Identify and describe the seven foci of process evaluations.
• Develop a clear and focused research question for a program evaluation.
• Use a program logic model to identify key information about an agency.

Week 07 Plan

Agenda

• Developing a research question
• Presentation planning
• Developing a program description
• Midterm feedback

Learning Objectives

• Identify and describe the seven foci of process evaluations.
• Develop a clear and focused research question for a program evaluation.
• Use a program logic model to identify key information about an agency.

Foci for Process Evaluations

I want to start our conversation by exploring one final method for evaluation, that of process evaluations. There are seven foci that might be addressed when considering this methodology:

• Context (political, economic & other influences on the program)
• Recruitment (methods used, successes, and failures)
• Reach (% of target population participating, sufficient resources?)
• Dose Delivered (how much intervention is typically delivered?)
• Dose Received (clients’ engagement & compliance with treatment)
• Fidelity (was the intervention applied as designed?)
• Implementation (a composite program implementation score can be created by averaging reach, dose delivered, dose received, and fidelity)

Developing a Research Question (1 of 7) Topic Introduction

I think that most of you already have a general idea what you want to do for your project. I want to make sure we are starting from the place of what do you want to know.

We are going to go through a process to help identify this.

Developing a Research Question (2 of 7) Frame In Steps of Evaluation

Before we start to look at what method will be used and how you will collect the data, analyze it, or use that information, please spend some time in your groups considering your research questions.

This diagram shows the first five steps in a program evaluation, which we discussed in week two of this class.

1. Identify Evaluation Question
2. Conceptualize Question
3. Operationalize Question

This should all happen before we can really pick a method. Today, my goal is for you to define your overarching research question. In your groups, you will have to come up with 3 to 5 subquestions that you are trying to determine with the components of your research project.

Developing a Research Question (3 of 7) Step 1

Step one is helping draw out your research question and understanding some potential questions you might be trying to answer. The following are some activities you could consider:

• What is your topic? i.e., burnout, how transitions happen, gaps in services, etc.
• What is the context or location of your research? i.e., DSHS, Domestic Violence, etc.
• What do you want to achieve? i.e., to discover, to describe, to change, to explore, to explain, etc.
• What is the nature of your question? i.e., a what, where, how, when, or why question?
• Are there potential relationships you want to explore? i.e., impacts, increases, decreases, relationships, correlations, causes, etc.

[Small Group Activity] Answer each of these questions using one or two word answers.

(O’Leary, 2018)

Developing a Research Question (4 of 7) Step 2 Examples

I’m drawing this activity from a neat little book, a part of little quick fix series and it is about research questions.

The second step is about starting with the nature of the question (e.g., who, what, why, where, how, when) put the pieces together.

The following are examples the author used. You will see that all three examples are about the same topic and context, but the other elements are all different.

Topic: pornography Context: high school Goal: to explore prevalence of watching porn Nature of your question: how much/ often Relationship: N/A

Question: How prevalent is watching pornography among high school students?

Topic: pornography Context: high school Goal: to understand how porn changes sexual expectations Nature of your question: how Relationship: watching porn and expectations

Question: How does watching pornography change sexual expectations among high school students?

Topic: pornography Context: high school Goal: to understand education programmes in high school that address porn viewing Nature of your question: what Relationship: N/A.

Question: What education programmes have been shown to have a positive impact on the pornography viewing habits of high school students?

(O’Leary, 2018, pp. 72-73)

Developing a Research Question (5 of 7) Step 2

You will write it out as follows:

Potential Question 1
Topic:
Context:
Goal:
Nature of Question:
Relationships:
Question:

Potential Question 2
Topic:
Context:
Goal:
Nature of Question:
Relationships:
Question:

etc.

[Small Group Activity] Work in your team to start to develop a few potential questions

(O’Leary, 2018)

Developing a Research Question (6 of 7) Step 3

Now we need to actually write out a question. Include the details identified information to make a well defined question.

(O’Leary, 2018)

Developing a Research Question (7 of 7) Step 4

1. Rewrite your question and circle terms that could be ambiguous.
2. Go through and clarify those terms.
3. Then, redraft your question, bringing more clarity and description

Presentation Planning

I wanted to spend some time tonight talking about our presentations that will take place on May 4th.

Discuss location

• Who do we invite
• What do we call it
• Do we have food

Show example of flyer from last year:

2025-05-05, Student Program Evaluation Symposium .pages

Evaluation Resources

To get us warmed up and thinking about logic models again, the following link (also in MyHeritage)

Click link: Evaluation for Leaders complete one as a group

• Parenting and infant health
• Youth smoking cessation
• Senior fall prevention
• Neighborhood crime prevention

Evaluation Resources or go directly to the page Logic Model Puzzles.

Using Logic Models to Identify Key Information Needs

So, we talked about logic models last week and some of the format. Yours are due this week. How many of you have yours started or partially finished?

What do You See in a Logic Model

If you have your logic model or one of the examples shared or that you can find. Review it an ask what information do we need to evaluate the program’s processes and outcomes effectively?”

Process: A process is the implementation of a key program component. In the Kellogg logic model, processes are described as activities and quantified as outputs.

Outcomes: Outcomes are results that occur that are directly linked to program processes.

Residential Treatment Program

[Whole Class Activity] Discuss this as an example logical model. Review how it would be created, the details within it, and how it helps in determining the evaluative process

Resources/Inputs

• Funding
• Staff
• Clients
• Facilities

Activities

• Group therapy
• Family therapy
• Residential care

Outputs

• 4 weekly group therapy sessions
• Weekly family therapy sessions for 30 families
• 30 youth in residential care

Outcomes

• Youth maintain participation in employment or education program
• Youth maintain legal lifestyle

Impact

• Families functioning productively
• Families satisfied with functioning

Residential Treatment Program Components

Conceptualization for program and component and the indicators that we can track.

• Weekly group therapy sessions # of sessions
• Weekly family therapy sessions # of sessions

In our logic model these are conceptulizied as the outputs

Residential Treatment Program

Adding to the conceptualization

We could add the following two items. They are some of the impacts that… have defined them as more specific indicators.

• Youth maintain participation in employment or education program – % of youth employed or in education program 3 months, 6 months, and 12 months out
• Youth maintain legal lifestyle – % of youth with no encounters with the legal system with first 12 months

Using Logic Models to Identify Key Information Needs

so we come back to:

“What information do you need to effectively evaluate the processes and outcomes of the program?”

Writing About Your Description of the Program

• Need: What need or needs is this program designed to meet?
• Context: What is the program’s context? Be sure to discuss the contextual or cultural factors that may affect the program’s effectiveness?
• Population Addressed: Who is included in the population for whom activities are intended?
• Stage of Development: How long has the program been in place?
• Resources: What resources (e.g., staff, money, space, faculty time, partnerships, technology, etc.) are available to support the program?
• Activities: What specific activities are conducted to achieve the program’s outcomes? (Note: Here is where you can provide additional information that you may have been asked to eliminate from the logic model, such as information on hours and availability of services, length of tutoring sessions, etc.)
• Outputs: What do the activities produce? (Remember, these are goal numbers, in this case measuring services delivered.)
• Outcomes: What are the program’s intended outcomes? (Remember that outcomes are changes in attitudes, skills, or behaviors of participants. You may want to divide outcomes into short-term, intermediate, and long-term categories–it’s up to you.)
• Impact: What is the program’s intended impact on the larger community?

Updated Template:

Description of the Program

Need {Explain what the problem or need the program is designed to address, including the conditions or gaps that prompted the program’s development. The need should be grounded in your agency’s context and clearly linked to the program’s purpose.}

Context {Describe the organizational, community, and cultural context in which the program operates. Discuss contextual or cultural factors that may influence how the program is implemented or its effectiveness.}

Population Addressed {Identify the population the program is intended to serve. Describe who is included and note any characteristics relevant to the program’s design or goals. If you did not include a discussion of the population in your literature review, include some general facts about them connected to the literature in this section.}

Stage of Development {Summarize how long the program has been in place and describe its stage of development (e.g., new, established, expanding, or evolving).}

Resources {Identify the key resources available to support the program, including staffing, funding, space, partnerships, technology, and other inputs necessary for implementation. Many of these are likely included in your logic model, but if there wasn’t space in the logic model and there are additional resources, it would be appropriate to include them in your discussion here.}

Activities {Describe the core activities conducted to achieve the program’s intended outcomes. Include relevant details that clarify how services are delivered, such as frequency, duration, or format, as appropriate. You may include more activities here than what was connected to your study and the created logic model.}

Outputs {Identify the direct products of program activities, typically expressed as service counts or participation levels. Focus on what the program delivers rather than the changes it produces.}

Outcomes {Describe the intended changes in program participants’ knowledge, attitudes, skills, or behaviors. Outcomes may be organized into short-, intermediate-, and long-term categories, with further subheadings if desired.}

Impact {Explain the program’s intended long-term impact on the broader community, system, or population beyond individual participants.}

Complete you Midterm Feedback

Feedback can be completed at the course evaluation form.

Time to Work on your Logic Models

[Small Group Activity] Students have time to work on their project… submitting draft version.