Location: Online - Zoom Time: Monday’s from 5:30-8:15 Week 14: 11/23/20 Topic and Content Area: Evaluation and Termination Reading Assignment: Hepworth et al. (2017) chapter 19 Assignments Due: A–02: Asynchronous Class Engagement What worked and did not work for the class 11/29/20 at 11:55 PM via flipgrid A–03: Reading Quiz for chapter 19 is due at 5:30 PM before class via My Heritage Other Important Information: N/A
Evaluation The evaluation process Terminationa Follow up Self Care
We do evaluations because we must continually try to improve our professional skills and knowledge. To increase your own effectiveness as a worker requires feedback from others. We want to know how client systems experienced their work with us which gives them a chance to be heard (empowerment). Helps build the knowledge base of the profession.
During the last 25 years the funding for social programs have become scarce and the demands for evaluation has increased. This increases accountability from social programs. Accountability in social work involves being responsible for and answerable to others for the quality and effectiveness of one’s efforts. There are different things that could get in the way of evaluation: Time consuming Clinician vulnerability (judgments) Lack of a routine system (not something done frequently) Insufficient effort focused on evaluation from agencies Deficient skills and training regarding use of evaluation approaches
When a social worker is completing an evaluation there are some major thrusts and important terminology… We can think of it as the Four “E’s” of Evaluation Effort (Monitor ongoing operation) Effects (Assess the outcomes) Effectiveness (Succeeding at what you set out to accomplish) Efficiency (Doing it at a reasonable cost)
Evaluation of practice follows the same planned change approach used in social work practice itself. Engagement: Define the problem to be evaluated Planning: Consider the type of research approach that might be appropriate Planning: Choose the best approach Implementation: Carried out Evaluation: Examine and evaluate the results of the research effort.
There are a number of types of evaluations… Formative Evaluations: assess the adequacy or amount of effort directed at solving a client systems problem and gathering data during the actual intervention (i.e. teacher evaluations). Summarative Evaluations: an evaluation that takes place after completing the planned change process (i.e. end of semester examinations). Baseline: is a measure of the frequency, intensity, or duration of a behavior.
There are a number of terms that are associated with evaluation. Validity-the extent to which you are measuring what you think you are measuring. Face validity-professional judgment about whether the measure actually measure what it is supposed to. Concurrent validity exists when scores on one instrument correlate well with scores on another instrument that is already considered valid. Predictive validity-when it can be used to predict future events. Reliability-is the extent to which an instrument measures the same phenomenon in the same way each time the measure is used. Reliable instruments produce consistent results over time. Data: Gathering Methods-commonly used methods include surveys, scores of instruments, interviews with significant others, collected data, surveys or interviews, self reports, products which is achievement of a specific task or change in behavior, or observational measures these rely on others to observe a change in the clients behavior. Independent Variable: is the factor we think is responsible for causing certain behaviors, reactions or events. Dependent Variable: is the outcome or end product of the helping process. Generalizability: is the ability of a set of results in one situation to fit another circumstance or instance. Generalization across the globe have resulted in numerous failures of practice and policy both in the US and across the globe.
Single-Subject Designs are research methods aimed at determining whether or not an intervention was successful. Often used with a single case or client. Is a method used when achievement of the goal is sufficiently important to be used as the primary outcome criterion. This focus on the desired state and not the problem. A / B data points Prior to intervention and after intervention
There are other forms of single system design studies that we can do. Goal Attainment Scaling is a method used when achievement of the goal is sufficiently important to be used as the primary outcome criterion. This focus on the desired state and not the problem. Task achievement scaling is a method of evaluating the degree to which an identified set of tasks has been accomplished. Client satisfaction questionnaires are a series of questions designed to learn the reactions of the customer to various aspects of the products. Target problem scaling is a process where a problem is identified , a plan is implemented, and changes in target problems are measured to determine if the problem has changed in severity.
Program evaluation can help determine whether or not one program is more effective or efficient than another. Includes periodic as well as ongoing evaluation of the process and the outcomes. Program evaluation tools include: Needs Assessment Evaluability Assessment Process Analysis Program Outcome Analysis Continuous Quality Assurance Evaluations Program Monitoring
PEER-EBD Participatory Evaluation and Expert Review for Classrooms Serving Students with EBD The PEER-EBD is a program evaluation that our school district participates in. It is an interesting means for evaluating a program. It is researched based, reviewing 18 different areas which are indicators of evidence-based practices in four categories (Systems & Philosophy,Structure, Individual Programming: Builds Academic and Behavioral Competence, Climate & Group Process). The questionnaire is quite lengthy to complete. There are three parts: Expert review (completed through observation, interviews, reviewing artifacts) Individual surveys Facilitated team assessment
Lack of generalizability: One successful program may not necessarily be general to implement with other groups. Interventions that are effective for one client may not work as well for others. Wrong choice of evaluation tools. Ethical considerations: It is important to always involve clients in the evaluation process. Doing research on clients without informing them is ethically wrong. No Buy in: Staff distrust of evaluation, they may feel they are being criticized or condemned. Difficulty: Clients may find the process overly time-consuming or intrusive.
There are a few tasks that me must complete in the termination phase. Decide when to terminate the professional worker-client relationship. Evaluate achievement of the objectives. Maintain and continuing progress. Resolving emotional reactions of the worker and client. Make appropriate referrals. . Ultimately the goal at termination is to empower the client system Think about AMA, early termination… etc
Whenever possible the worker should plan for termination and involve the client in the process. There may be different reactions and feelings when it comes time for termination on behalf the client and the worker. A way to help a client at termination is planning for it, address feelings about it, and summarize progress. Stabilization of change examples There are factors that will change the intensity of reactions. Factor | More Intense | Less Intense —- | —- | —- Time | Open ended relationships | Time-limited relationships Contact | Frequent contact | Infrequent Problem focus | personal problem focus | Environmental problem focus Outside supports | Limited supports | strong supports Level of intervention | Individual or family system | Organizational or community Emotional content | High level of emotional content | Low level of emotional content Type of group | treatment group | task group
There is sufficient evidence that changes that occur in the context of therapeutic relationships do not necessarily carry over into other areas of client examples… we need to look for ways to make it more generalizable. Helping clients select relevant and appropriate situations to work on (client driven, addressing reason for coming in…) Helping clients build confidence in their own abilities (Strengths perspective, reviewing progress) Using multiple situations and settings when helping members learn new behaviors (addressing at home, work, school, practice examples) Using naturally occurring consequences rather than artificial ones (going out and doing it in real life vs. just with clinician) Extending treatment through use of follow up (Looking past the end of the services to monitor and follow up) Reducing setbacks in other environments (Getting outside supports involved, making environmental changes) Helping members confront future problems by teaching problem solving process (teach a man to fish… etc)
Fink-Samnick, E. (2009) The professional resilience paradigm: Defining the next dimension of professional self-care. Professional Case Management 14(6):330-2. DOI: 10.1097/NCM.0b013e3181c3d483 Fink-Samnick describes 20 strategies for self care in her The Professional Resilience Paradigm: The Next Dimension of Professional Self-Care Value verses devalue your professional self (don’t undersell yourself) Have positive contacts with colleagues and peers (work with people who motivate you) Take that break (vacations, lunch… etc) Pace yourself (Working faster does not always yield increased output) Achieve validation Identify goals Engage with positive contacts Get a mentor Use the power of professional networking (Engage with professional associations & networking sites to - Keep up with new trends and expand horizons & opportunities)
Present with a presence How do others see you? How do you dress? How on edge do you feel? Laugh at least once a day Stop to take that long deep breath Develop a grounding list (favorite song, picture, aroma… etc) Stop and take 10 10 seconds: process, breathe 10 minutes: unsure what to say or do? 10 hours: time for a day off? 10 days: time for vacation? 10 months: time for a new job?) Take control & shift activities (walk away and shift gears)
Use creative visualization De-connect to Re-connect (Give distance to work get closer to life) Release frustration with a silent meow (They tense their body, open their mouth & let loose!) Exercise turn off your professional switch Make it a ritual Have peers who are NOT in the biz Set limits for yourself plus others Think of teflon Stay attuned to boundaries Don’t let the tough stuff stick! Anticipate & let things roll off!
Revision honestly and regularly Revise previously defined life goals Define an individualized schedule Identify realistic obstacles to the schedule’s implementation Progress with a plan to reflect your current perspective. Share professional resilience with health and human services professionals everywhere