Week 04 - Program Evaluation Process and Logic Models

Slide 1
Location: CBC Campus - SWL 208 Time: Mondays from 5:30-8:15 Week 04: 2/3/20 Topic and Content Area: Cultural Competency Logic Models Reading Assignment: Kapp and Anderson chapters 4, 7 Assignments Due: A-02 Reading Quiz 02/03/20 A-04a: Weekly Journal 01 02/09/20 A-04b: Group Logic Model 02/09/20 Other Important Information: N/A

Location: CBC Campus - SWL 208
Time: Mondays from 5:30-8:15
Week 04: 2/3/20
Topic and Content Area:

  • Cultural Competency
  • Logic Models

Reading Assignment: Kapp and Anderson chapters 4, 7
Assignments Due:

  • A-02 Reading Quiz 02/03/20
  • A-04a: Weekly Journal 01 02/09/20
  • A-04b: Group Logic Model 02/09/20

Other Important Information: N/A

Slide 2
Overview of program evaluation process Logic models
  • Overview of program evaluation process
  • Logic models
Slide 3
There are number of parts that we must consider as we move into doing program evaluation. Engage stakeholder in the evaluation process Assess and address agency political factors Choose an evaluation design Data collection Reporting and using data Addressing culturally competent program evaluation

There are number of parts that we must consider as we move into doing program evaluation.

  • Engage stakeholder in the evaluation process
  • Assess and address agency political factors
  • Choose an evaluation design
  • Data collection
  • Reporting and using data
  • Addressing culturally competent program evaluation
Slide 4
In preparing for a program evaluation, the following would be some of the steps to take. Identify evaluation question Conceptualize question Operationalize question Pick and define method Select sample Data collection Data management and analysis Utilization of information

In preparing for a program evaluation, the following would be some of the steps to take.

  1. Identify evaluation question
  2. Conceptualize question
  3. Operationalize question
  4. Pick and define method
  5. Select sample
  6. Data collection
  7. Data management and analysis
  8. Utilization of information
Slide 5
A program logic model is a picture of how an organization does its work — the theory and assumptions underlying the program. A program logic model links outcomes (both short- and long-term) with program activities/processes and the theoretical principles/assumptions of the program.

A program logic model is a picture of how an organization does its work — the theory and assumptions underlying the program. A program logic model links outcomes (both short- and long-term) with program activities/processes and the theoretical principles/assumptions of the program.

Slide 6
A program logic model is a picture of how an organization does its work — the theory and assumptions underlying the program. A program logic model links outcomes (both short- and long-term) with program activities/processes and the theoretical principles/assumptions of the program.

A program logic model is a picture of how an organization does its work — the theory and assumptions underlying the program. A program logic model links outcomes (both short- and long-term) with program activities/processes and the theoretical principles/assumptions of the program.

Slide 7
There are a number of reasons why it is useful to have a logic model for a program. Stakeholders are allowed to articulate their views about the program Stakeholders are allowed to hear and appreciate the views of other stakeholders Divergent views about the program are synthesized in a collaborative process. An integrated model is developed with stakeholder ownership The integrated model is scrutinized publicly for feasibility The model serves as a solid reference for program management decisions The model is used as an organizing tool for evaluation The model can be used as a promotional tool for the program The model can be employed to support program proposals for grant activities

There are a number of reasons why it is useful to have a logic model for a program.

  • Stakeholders are allowed to articulate their views about the program
  • Stakeholders are allowed to hear and appreciate the views of other stakeholders
  • Divergent views about the program are synthesized in a collaborative process.
  • An integrated model is developed with stakeholder ownership
  • The integrated model is scrutinized publicly for feasibility
  • The model serves as a solid reference for program management decisions
  • The model is used as an organizing tool for evaluation
  • The model can be used as a promotional tool for the program
  • The model can be employed to support program proposals for grant activities
Slide 8
Engaging all of the stakeholders will get a more complete picture Clients Direct service staff Supervisory / management staff Executive staff Board of directors Funders Community partners

Engaging all of the stakeholders will get a more complete picture

  • Clients
  • Direct service staff
  • Supervisory / management staff
  • Executive staff
  • Board of directors
  • Funders
  • Community partners
Slide 9
The following would be some potential interview questions to help develop a logic model. How is the program staffed and organized? What components are involved with the program other than staff? Can you describe the major activities of the program? What resources are devoted to these activities What are the main objectives of the program? What is the program trying to accomplish What accomplishments is the program likely to achieve in the next 2 and 3 years? What would you expect? How will these activities undertaken by the program accomplish the results? What kinds of information do you have on the program? How do you use this information? What kinds of information do you need to assess program performance? How would you use this information? What measures or indicators are relevant to the program? What problems face the program? What factors are likely to influence the program over the next 2 to 5 years?

The following would be some potential interview questions to help develop a logic model.

  • How is the program staffed and organized?
  • What components are involved with the program other than staff?
  • Can you describe the major activities of the program?
  • What resources are devoted to these activities
  • What are the main objectives of the program? What is the program trying to accomplish
  • What accomplishments is the program likely to achieve in the next 2 and 3 years? What would you expect?
  • How will these activities undertaken by the program accomplish the results?
  • What kinds of information do you have on the program?
  • How do you use this information?
  • What kinds of information do you need to assess program performance?
  • How would you use this information?
  • What measures or indicators are relevant to the program?
  • What problems face the program?
  • What factors are likely to influence the program over the next 2 to 5 years?
Slide 10
Resources Staff activities Program processes Immediate outcomes Intermediate outcomes Long-range outcomes
  • Resources
  • Staff activities
  • Program processes
  • Immediate outcomes
  • Intermediate outcomes
  • Long-range outcomes
Slide 11
Different type of logic model, but similar in many ways.

Different type of logic model, but similar in many ways.

Slide 12
Resources/Inputs: The human, financial, organizational, and community resources a program has available to direct toward doing the work Activities: What the program does with its resources (processes, tools, events, technology, actions) Outputs: The direct product of program activities; described in terms of size and/or scope of services/products delivered or produced by the program Outcomes: Specific changes in program participants’ behavior, knowledge, skills, level of functioning, etc. Impact: Organizational, community, or systems-level changes expected as a result of the program

Resources/Inputs: The human, financial, organizational, and community resources a program has available to direct toward doing the work

Activities: What the program does with its resources (processes, tools, events, technology, actions)

Outputs: The direct product of program activities; described in terms of size and/or scope of services/products delivered or produced by the program

Outcomes: Specific changes in program participants’ behavior, knowledge, skills, level of functioning, etc.

Impact: Organizational, community, or systems-level changes expected as a result of the program

Slide 13
Ask questions about the Children’s Day Program to work in small groups to create an example of a logic model [Whole Class Activity] Be in the hot seat to ask questions

Ask questions about the Children’s Day Program to work in small groups to create an example of a logic model

[Whole Class Activity] Be in the hot seat to ask questions

Slide 14
Who are the stakeholders for you agency What would be some of your questions What would the categories look like
  • Who are the stakeholders for you agency
  • What would be some of your questions
  • What would the categories look like