Spring 2025 SOWK 460w Week 06 - Quality Assurance Designs and Understanding an Agency Through a Logic Model

Spring 2025 SOWK 460w Week 06 - Quality Assurance Designs and Understanding an Agency Through a Logic Model

title: Spring 2025 SOWK 460w Week 06 - Quality Assurance Designs and Understanding an Agency Through a Logic Model date: 2025-02-24 17:30:00 location: Heritage University tags:

• Heritage University
• BASW Program
• SOWK 460w presentation_video: > “” description: >

In week six, we are continuing to explore methodologies for program evaluations. Many of your program evaluations will gather staff perspectives on the agency, similar to quality assurance design methodologies. We will work on how to develop a survey and create an example version during class. We will also examine your logic models and how to develop them. Students will continue to work on their general evaluations and reflect on the experience in their journals.

The agenda for the class is as follows:

• Quality assurance design strategies
• Developing questions used on surveys
• Practice developing a quality assurance design study
• Logic models

Agenda

• Quality assurance design strategies
• Developing questions used on surveys
• Practice developing a quality assurance design study
• Logic models

What is Getting Evaluated in Quality Assurance

Royse (2023) summarizes the work of Fraser and Wu (2016) and the World Health Organization (2000) in describing some aspects that get evaluated as a part of quality assurance. I have group these into the following:

• Client Satisfaction and Experience
  • Acceptability, approval, and appeal
  • Liking or feeling of being personally involved with the service
  • Positive attitudes toward a service
  • Contentment with the outcome of the service or derived benefits
• Service Effectiveness and Outcomes
  • The effectiveness of services in reducing or eradicating problems
  • Met performance expectations
  • Correspondence between clients’ expectations and their appraisals of service performance
• Provider Competence and Professionalism
  • The competence of counselors
  • The courtesy of providers
  • The helpfulness of staff
• Service Reliability and Accessibility
  • The reliability of services (assurance that services are provided in a consistent and dependable manner)
  • Waiting times for service
  • Amount of time spent with the provider
  • The cost of services
• Relevance and Appropriateness of Services
  • The relevance of services to consumers’ needs
  • The “humanness” of services

Collection Strategies

Royse (2023) provides examples of a number of collection strategies that we might use to gather quality assurance data.

• Personal Interviews: Very labor intensive and consider if you are forming dual relationships
• Telephone Surveys: Still very labor intensive, probably needs to be pre-planned
• Mailed Survey: Less labor intensive, but often low response rates (consider incentives, multiple mailings)
• Electronic Survey: Increasingly common, can even be done in person
• Ballot Box: Short response, overcomes many challenges of mailed/emailed surveys
• Focus Groups: Qualitative methodology. Consider selection methods and having multiple groups.
• Faces Scale: Can be quick and easy to collect a majority of data. Example - Loves bathrooms. Author described not a lot of examples in the literature.

Bridges Program Parent Questionnaire

To show an example of a pretty long form version of a quality assurance survey, We used one a few of the years I was invovled with the Pasco School District Bridges Program:

Folder on my Mac can be found at Parent Survey

Consider showing the actual Survey Bridges Parent Survey.pdf and my results 00 - Bridges Program Parent Questionnaire Report.pdf

Not including an adequate range of response categories may require respondents to choose answers that do not accurately reflect their experiences or may cause respondents to be frustrated and skip the question.

Developing Survey Questions (1 of 2) Avoiding Pitfalls

The CDC (2011) provides a resource sharing some potential pitfalls for questions we might include on a survey.

Doublebarreled questions

Description: Double-barreled items ask multiple things in one question but only allow for a single answer.
Example: How satisfied are you with the caseworker’s support and the timeliness of services provided? Revision: How satisfied are you with the support provided by your caseworker? And How satisfied are you with the timeliness of services provided?

Leading Questions

Description: Leading items introduce bias and may influence how a respondent answers.
Example: How helpful was our highly trained and compassionate staff in addressing your needs?
Revision: How would you rate the staff’s ability to address your needs?

Inadequate Response Options

Description: Not including a sufficient range of response categories may force inaccurate answers or frustrate respondents.
Example:

How many years have you been a client? [ ] 1 year [ ] More than 10 years

Revision:

How many years have you been a client? [ ] 1 year [ ] 2-3 years [ ] 4-7 years [ ] 7-10 years [ ] More than 10 years

Negative Wording

Description: Negative questions can confuse respondents and lead to inaccurate responses.
Example: Does the program not meet your needs?

Revision:
How well does the program meet your needs?

Developing Survey Questions (2 of 2) Types of Questions

We can think about the question types and some other suggestions for our surveys that they offer.

• Close-Ended with Ordered Choices
• Close-Ended with Unordered Choices
• Partially Close-Ended
• Open-Ended Questions

They also provide some other considerations we might think of:

• Words and terms are clear.
• Flow of questions is logical.
• Response categories are adequate.
• Skip patterns make sense.
• Format and layout are easy to follow.
• Mechanics of survey administration are feasible.
• Language is culturally appropriate.
• Length of time to complete is reasonable.

Adapted from

Developing a Quality Assurance Survey: for Heritage BASW Program

We are going to go through a three step process for creating a survey to evaluate the BASW Program.

Make sure you have somebody who is taking notes during each step.

[Small Group Activity] Planning questions for the survey

Step 1: Working in small groups, discuss what research questions you might have in developing a quality assurance survey.

Step 2: Divide the class into two groups and merge your research questions. Then, start developing the questions you would include in your survey.

[Whole Group Activity] Developing a joined final survey

Step 3: Combine the whole class. Somebody will facilitate the discussion by merging and combining the questions the two groups developed. Create a completed survey.

Link to survey for somebody to develop actual questions: 2025 Spring SOWK 460w BASW Quality Assurance Survey

Logic Model

A program logic model shows how an organization works, including the theory and assumptions underlying the program. It links outcomes (both short- and long-term) with program activities/processes and the program’s theoretical principles/assumptions.

(Kapp & Anderson, 2010)

Purpose of Logic Models: Why We Do It

There are a number of reasons why it is useful to have a logic model for a program.

• Stakeholders are allowed to articulate their views about the program
• Stakeholders are allowed to hear and appreciate the views of other stakeholders
• Divergent views about the program are synthesized in a collaborative process.
• An integrated model is developed with stakeholder ownership
• The integrated model is scrutinized publicly for feasibility
• The model serves as a solid reference for program management decisions
• The model is used as an organizing tool for evaluation
• The model can be used as a promotional tool for the program
• The model can be employed to support program proposals for grant activities

(Kapp & Anderson, 2010)

Potential Interview Questions

There are many different questions that you might consider asking as a part of building your logic model

• How is the program staffed and organized?
• What components are involved with the program other than staff?
• Can you describe the major activities of the program?
• What resources are devoted to these activities?
• What are the main objectives of the program? What is the program trying to accomplish?
• What accomplishments is the program likely to achieve in the next 2 and 3 years? What would you expect?
• How will these activities undertaken by the program accomplish the results?
• What kinds of information do you have on the program?
• How do you use this information?
• What kinds of information do you need to assess program performance?
• How would you use this information?
• What measures or indicators are relevant to the program?
• What problems face the program?
• What factors are likely to influence the program over the next 2 to 5 years?

(Kapp & Anderson, 2010)

Book Template for Logic Model

The following is a general idea of what can go into your logic model. You don’t have to have the same categories, but should generally follow the format.

• Resources: Resources available to the program that allow and support service delivery, including money, staff, volunteers, clients, materials, or equipment
• Staff activities: The methods of service delivery carried out by staff
• Program processes: The product delivered or unit of service provided, usually described numerically, such as number of people served or number of hours of service delivered
• Immediate outcomes: The first changes that occur for the individuals, families, organizations, or community as a result of the program
• Intermediate outcomes: The subsequent benefit for people during or after their involvement in a program
• Long-range outcomes: The eventual impact on individuals, families, organizations, or community for which the program is accountable

(Kapp & Anderson, 2010)

Example from the textbook

(Royse, 2023, p. 114)

They call out

• Outputs are a counting or measurement of efforts made.
• Outcomes are the benefits to the clientele or community; this is another way of saying what the impact of the activities (program) has been.
• Resources can be partners or collaborators; they can supply funding or in-kind staff or even host educational or self-help material on their web sites.
• Need, of course, would likely be determined from a needs assessment.
• Antecedent conditions may or may not be necessary to list in all logic models

Family Assessment Response Pathway

1 of many examples in this report of a logic model. They call them out by components.

Example Logic Model From Wikimedia

Resources/Inputs: The human, financial, organizational, and community resources a program has available to direct toward doing the work

Activities: What the program does with its resources (processes, tools, events, technology, actions)

Outputs: The direct product of program activities; described in terms of size and/or scope of services/products delivered or produced by the program

Outcomes: Specific changes in program participants’ behavior, knowledge, skills, level of functioning, etc.

Impact: Organizational, community, or systems-level changes expected as a result of the program

Logic Model Example - PDC

Different type of logic model, but similar in many ways.

Developing a Logic Model Practice

Ask questions about the Children’s Day Program to work in small groups to create an example of a logic model

[Whole Group Activity] Be in the hot seat to ask questions

Make a Plan for Developing Your Logic Model

• Who are the stakeholders for you agency
• What would be some of your questions
• What would the categories look like