This week, you will begin by selecting an agency, interviewing key personnel at the agency regarding the programs they provide, and developing a logic m

All of the discussions and written assignments in this course will focus on a program evaluation of one specific non-profit agency.  Each week’s discussion thread will focus on the issues that will be relevant to that week’s written assignment.  Use the discussion as a means to clarify your understanding of the issues related to the topics each week.  This week, you will begin by selecting an agency, interviewing key personnel at the agency regarding the programs they provide, and developing a logic model that provides a graphic overview of the program’s purpose, services, inputs, activities, outputs, and outcomes that will be evaluated. 

Please note that, due to the very short duration of this course, it will likely not be possible to actually collect and analyze data from that agency.  You will have the option to collect agency data (with the permission of the agency, of course) or to create data files using hypothetical data in future weeks.

Review the PowerPoint presentation of a program evaluation conducted at Program Evaluation for Broward Partnership for the Homeless.pptx

In this discussion thread, please identify the following. Items 1, 2, and 3 are due by Wednesday; item 4 is due by Sunday.

  1. The name of the agency that you will be evaluating in this course NOTE: If the agency prefers, you may use a pseudonym for confidentiality purposes. The agency should provide psychological and/or social services and be a not-for-profit agency. 
  2. Discuss the ethical issues involved with being a consultant for a social services agency.  How will the confidentiality of the clients be maintained throughout your program evaluation project?  Are there other ethical or legal issues that need to be considered?
  3. By the end of the week, you will need to interview at least one key personnel at the agency.  When deciding who to interview, consider the following:
    • Who can provide the best information about the services delivered by the agency?
    • Who can provide the best information about how the agency currently measures outcomes related to effectiveness of their programs?
    • Review the components that are required for your logic model.  The person(s) interviewed should be able to explain the inputs, activities, outputs, and desired short-term, intermediate-term, and long-term outcomes.

Broward Partnership for the Homeless, Inc. (BPHI)

Program Evaluation Study

Single-site program evaluation

Homeless shelter outcomes

1 ½ year duration

2001 – 2003

BPHI

Homeless shelter in Broward County, FL

Approximately 1700 residents per year

Up to one-year stay

Average six months

200 beds

10 family units

BPHI – Services

Case Management

Psychiatric Treatment

Substance abuse, mental health counseling

Job training

Resume Writing

Education

GED

Referrals for educational services

Employment Services

BPHI – Services

Three meals a day

Medical evaluation and treatment

Dental office

Child care

Library

Clothing

Hair salon

Message service

BPHI – Need for Evaluation

Funding agencies require evidence of treatment effectiveness

Donors often require evidence

No data to support effectiveness of services

BPHI – Program Evaluation

Grant from Health Foundation of South Florida

Goals

Increase behavioral health services

Increase engagement in center residents

Improve behavioral stability in center residents

Reduce symptom severity

Increase functionality

Provide evidence of results achieved

Research Design Considerations

Pre-Post design with control group

What is the main purpose of assessment?

Who will do the assessments?

What type of assessment?

Is diagnostic interview necessary?

Assessment versus screening

BPHI – Measures

Public Domain = Free!

CAGE for drug/alcohol screening

Mental Health Screening Form (MHSF)

SOFAS

GAF

BPHI – CAGE

1. Have you felt you ought to cut down on your drinking or drug use?

2. Have people annoyed you by criticizing your drinking or drug use?

3. Have you felt bad or guilty about your drinking or drug use?

4. Have you ever had a drink or used drugs first thing in the morning to steady your nerves or to get rid of a hangover (eye-opener)?

BPHI – MHSF

Mental Health Screening Form (MHSF; Carroll & McGinley, 2000).

Screen for mental health symptoms

Designed for use with co-occurring disorders

17 items

Instrument available here

BPHI – SOFAS

Social and Occupational Functioning Scale (SOFAS; American Psychiatric Association, 2000)

Focus on social, occupational functioning

Impairment not exclusively due symptom severity

Includes general medical conditions

Case Manager rated at intake and discharge

BPHI – GAF

Global Assessment of Functioning (GAF; American Psychiatric Association, 2000)

Impairment due to symptom severity

Impairment not due to general medical conditions

Rated by therapist/psychiatrist each session

BPHI – Issues

Training

Oversight

Supervision

Data Collection

Data Entry

Results

BPHI
Compared those treated to not treated
↓ Substance abuse symptoms
↓ Psychiatric symptoms
↑ SOFAS scores
↑ GAF scores
↑ attained housing

References

American Psychiatric Association. (2000).  Diagnostic and statistical manual of mental disorders (4th ed., text revision).  Washington, DC:  Author.

Carroll, J.F.X., & McGinley, J.J. (2000). Mental Health Screening Form III. Tampa, FL: Project Return Foundation. Retrieved from http://smchealth.org/sites/default/files/docs/1309587941MentalHealthScreeningFormIII.pdf

media1.m4a

image2.png

image3.png

media2.m4a

media3.m4a

media4.m4a

media5.m4a

media6.m4a

media7.m4a

media8.m4a

media9.m4a

media10.m4a

media11.m4a

media12.m4a

media13.m4a

media14.m4a

media15.m4a

media16.m4a

Last Completed Projects

topic title academic level Writer delivered