Skip Navigation
 

PADM615 - Program Appraisal

Course Details

Course Code: PADM615 Course ID: 2675 Credit Hours: 3 Level: Graduate

This course is designed to develop fundamental skills essential for students to evaluate public programs. Knowledge of the policy process and research methods is brought together in the ethical assessment of program needs, processes, and outcomes.





Course Schedule

Registration Dates Course Dates Session Weeks
05/27/19 - 11/01/19 11/04/19 - 12/29/19 Fall 2019 Session I 8 Week session
07/29/19 - 01/03/20 01/06/20 - 03/01/20 Winter 2020 Session B 8 Week session
09/30/19 - 02/28/20 03/02/20 - 04/26/20 Winter 2020 Session D 8 Week session

Current Syllabi

After successfully completing this course, you will be able to:

  1. Evaluate the role of program appraisal and evaluation in public administration.
  2. Analyze the basic models of program evaluation.
  3. Evaluate appraisal and evaluation methods as they relate to a given problem.
  4. Appraise the different project cycle phases.
  5. Analyze the professional standards and ethical issues involved in conducting program evaluations.
  6. Assess how data is collected and analyzed.
  7. Compare and contrast the use of quantitative and qualitative research methods in evaluating public programs.
  8. Design a survey and evaluation plan for a real world example.
  9. Produce an evaluation presentation that would serve public program administrators.

Forum Questions

Post your initial response to each forum by 11:55 pm ET on Thursday. This course is geared toward a practical application of the concepts we are studying so the forums may or may not be supplemented with scholarly sources. It is more important they are original and documented with interview contacts and detailed responses to the question. Just remember to give attribution!

Reply to at least three of your classmates in each forum by 11:55 pm ET on Sunday. Responses to classmates are informative and contribute to advancing knowledge of the topic and improving their evaluation product.

*The Forum Rubric is listed under the first substantive forum.

Assignments

Assignment #1 Survey Draft

This week we start by designing a survey. For the purposes of this class you have already chosen the National Park that you would like to evaluate and have reviewed the lesson that contains an example of the visitor survey the National Parks actually use. I have used that as my model for my examples but you do not need to ask those same questions. Design your survey to measure what you want to measure about the Park. If it is visitor count that would be fine, if it is something else like the use of maintenance vehicles per Park employee, the appropriate number of campsites per acre, etc. please feel free. If the results are not available, you are welcome to make up hypothetical results for the purpose of this class. For the first part of this assignment you are to create a word document survey using the knowledge gained from the readings and lessons so far. You may find more help at https://www.surveymonkey.com/mp/how-to-create-surveys/ . In reality you would use a site like Survey Monkey to create and disseminate the survey but for our purposes you will do a very rough draft in Word. It should contain the basic format and questions. Please save that document. Please submit your survey to the students identified in the assignment. Please also submit it to the assignment tab for my comment. (I am evaluating the survey in this assignment.)

Assignment #2 Survey Peer Review

Once you have the draft from your classmate, please comment on the survey instrument. Did the student ask pertinent questions? Do they get to the result intended by the survey? Is the information useful? How would you feel as a consumer taking the survey? Too long? Not worth your time? Makes you feel like a contributing member of the organization? Are there questions that should be there that are not? Once you have commented please send this back to your classmate. Please upload the survey with your comments to the assignment (I am evaluating the comments in this assignment)

Assignment #4 – Evaluation Presentation

For this assignment you will create a presentation of your findings. Consider that you were a consultant asked to do this evaluation and now you are presenting your results to the Park staff and local community. If the stats are available for your Park on the Visitor Use stats site https://irma.nps.gov/Stats/ incorporate those if they are relevant. If they are not you will need to generate hypothetical results for your survey in order to complete this assignment. You can choose to do this in many different ways. Attached is an example of my presentation of the Dayton Aviation Heritage National Park done in Adobe Premier Pro. You may use Adobe Apps, Knovio, iMovie or any other application that allows for a multimedia presentation. It has to contain at least voice and at least one image. You can use multiple images like those I took and uploaded to Adobe Premier Pro in the example or it can be simply an image of you sitting in front of your webcam giving the report as if you were in front of a crowd. It could be an animated image if you are extremely good with these apps but I will be looking for something that the class can view so it can't be in program we can’t view. If you create the report in Knovio, you will provide a link for me to view it. If you create it in something that creates a movie, you need to export it in a file that can be uploaded to Kaltura under the video option so we can view it. Remember, as a public administrator you should be concerned not only with your presentation (between 5 - 8 minutes generally) and its content, but also that it is accessible and ADA compliant. In order to insure that it is, you should have a transcript along with your presentation.

Assignment #5 - Program Evaluation Poster

It is important that the results of an evaluation are clearly and simply portrayed to the funder/requesting organization and your target audience. Develop a poster for the Park with which you have been working. Presume you have been asked to present your evaluation at the American Evaluation Association annual conference. If you are not familiar with poster sessions, see their guidelines at http://www.eval.org/p/cm/ld/fid=172 . A poster is just a visual display of your evaluation, highlighting the important points so that someone walking by can visually understand your findings. Examples of posters and the process of a poster competition can be found here. More information can be found at http://www.uhd.edu/academic/colleges/sciences/scholars/files/workshop-poster.pdf and http://colinpurrington.com/tips/academic/posterdesign . I am not looking for a professional graphic design but some of the tips in these websites will help you create a clear, concise presentation of your findings. The “poster” should be on one page. You can create it in word, power point, or any of the adobe creative cloud applications (inDesign and Illustrator would work well for this project). If you create it using other programs, please save it as a pdf before submitting.

  • The assignment rubric is attached to each assignment in the assignment section of the classroom.
NameGrade %
Forum Questions 30.00 %
Forum Question 1 0.37 %
Forum 1.2 3.70 %
Forum Question 2 3.70 %
Forum Question 3 3.70 %
Forum Question 4 3.70 %
Forum Question 5 3.70 %
Forum Question 6 3.70 %
Forum Question 7 3.70 %
Forum Question 8 3.70 %
Assignments 40.00 %
Evaluation Presentation 20.00 %
Evaluation Poster 20.00 %
Survey Peer Review 10.00 %
Survey Peer Review 10.00 %
Survey Draft 20.00 %
Survey Draft 20.00 %

Required Readings

Week One – Program Evaluation Overview

Benchmarking Foundation Evaluation Practices. Center for Effective Philanthropy.

Retrieved from

http://www.evaluationinnovation.org/sites/default/files/Benchmarking-Foundation-Evaluation-Practices.pdf

State of Evaluation 2016 Evaluation Practice and Capacity in the nonprofit sector. Innovation Network. Retrieved from https://www.innonet.org/media/2016-State_of_Evaluation.pdf

American Evaluation Association. AEA 365. A Tip A Day for Evaluators. Retrieved from http://aea365.org/blog/

Review several times a week throughout the class.

USAID Evaluation Toolkit. Learning Lab. Retrieved from https://usaidlearninglab.org/evaluation?tab=1

Review all tabs

Free Management Library. Basic Guide to Program Evaluation. Retrieved from http://managementhelp.org/evaluation/program-evaluation-guide.htm

Review all tabs

Li, W., & Mauerhofer, V. (2016). Behavioral patterns of environmental performance evaluation programs. Journal of Environmental Management, 182, 429-435. doi:10.1016/j.jenvman.2016.07.085


Gargani, J., Miller, R. L., & Spiegelman, D. (2016). WHAT IS PROGRAM EVALUATION?/SPIEGELMAN RESPONDS. American Journal of Public Health, 106(6), E13-E14. doi:http://dx.doi.org.ezproxy2.apus.edu/10.2105/AJPH.2016.303159

Huckel Schneider, C., Milat, A. J., & Moore, G. (2016). Barriers and facilitators to evaluation of health policies and programs: Policymaker and researcher perspectives. Evaluation and Program Planning, 58, 208-215. doi:10.1016/j.evalprogplan.2016.06.011

Hall, J. L. (2017). PERFORMANCE MANAGEMENT: CONFRONTING THE CHALLENGES FOR LOCAL GOVERNMENT. Public Administration Quarterly, 41(1), 43-66. Retrieved from https://search-proquest-com.ezproxy2.apus.edu/docview/1871486263?accountid=8289

Week Two – Assessing Program Need

Altschuld, J. W., & Watkins, R. (2014). A primer on needs assessment: More than 40 years of research and practice. New Directions for Evaluation, 2014(144), 5-18. doi:10.1002/ev.20099

Engle, M., & Altschuld, J. W. (2014). Needs Assessment: The Perspective From the Public Sector. New Directions For Evaluation, 2014(144), 33-45. doi:10.1002/ev.20101


Lubejko, B. G. (2016). Developing a program evaluation plan: Options and opportunities. Journal of Continuing Education in Nursing, 47(9), 388 - 389. doi:10.3928/00220124-20160817-02

Watkins, R., & Altschuld, J. W. (2014). A Final Note About Improving Needs Assessment Research and Practice. New Directions For Evaluation, 2014(144), 105-114. doi:10.1002/ev.20106

Wedman, J. (2014). Needs Assessments in the Private Sector. New Directions For Evaluation, 2014(144), 47-60. doi:10.1002/ev.20102

Energy.gov. Program Evaluation. Why, What and When to Evaluate. Retrieved from https://energy.gov/eere/analysis/program-evaluation-why-what-and-when-evaluate

Please review all links.

Center for Disease Controls and Prevention. Program Performance and Evaluation Office. Retrieved from

https://www.cdc.gov/eval/index.htm

Review all tabs.

Week Three - Data Source and Analysis/Sampling and Surveys

Carr, K. M., & Bradley-Levine, J. S. (2016). Translational research design: Collaborating with stakeholders for program evaluation. The Qualitative Report, 21(1), 44 - 58.

Mojtahed, R., Nunes, M. B., Martins, J. T., & Peng, A. (2014). Equipping the constructivist researcher: The combined use of semi-structured interviews and decision-making maps. Electronic Journal of Business Research Methods, 12(2), 87 - 95.

Wageman, R., Hackman, J. R., & Lehman, E. (2005). Team diagnostic survey: Development of an instrument. The Journal of Applied Behavioral Science, 41(4), 373-398. doi:10.1177/0021886305281984

McIver, D., & Wang, X. ". (2016). Measuring knowledge in organizations: A knowledge-in-practice approach. Journal of Knowledge Management, 20(4), 637-652. Retrieved from https://search-proquest-com.ezproxy2.apus.edu/docview/1826809475?accountid=8289

Couper, M. P., & Peterson, G. J. (2017). Why do web surveys take longer on smartphones?Social Science Computer Review, 35(3), 357-377. doi:10.1177/0894439316629932

Conrad, F., Tourangeau, R., Couper, M., & Zhang, C. (2017). Reducing speeding in web surveys by providing immediate feedback. Survey Research Methods, 11(1), 45-61. doi:10.18148/srm/2017.v11i1.6304

Vaughn, P., & Turner, C. (2016). Decoding via coding: Analyzing qualitative text data through thematic coding and survey methodologies. Journal of Library Administration, 56(1), 41-51. doi:10.1080/01930826.2015.1105035

Willits, F. K., Theodori, G. L., & Luloff, A. (2016). another look at likert scales. Journal of Rural Social Sciences, 31(3), 126 - 139.

Week Four – Reporting the Results

Schmitt, C. L., Glasgow, L., Lavinghouze, S. R., Rieker, P. P., Fulmer, E., McAleer, K., & Rogers, T. (2016). Measuring infrastructure: A key step in program evaluation and planning. Evaluation and Program Planning, 56, 50-55. doi:10.1016/j.evalprogplan.2016.03.007

Ebener, Patricia et al. (2017) Rand Corporation. Getting to Outcomes Guide for Community Emergency Preparedness. Retrieved from https://www.rand.org/pubs/tools/TL259.html

Download or read the Guide online.

Emery, Ann K. (2014) How to Present Data When you are Presenting: Storyboarding your Data…Retrieved from http://annkemery.com/storyboarding/ . Please read the short article and watch the 2 videos.

Emery, Ann K. (2015) When Pie Charts are OK. Retrieved from http://annkemery.com/pie-chart-guidelines/ .

Week Five – Logic Models

Drayton, V. L., Caleb, Walker, D. K., Ball, S. W., Donahue, S. M., A., & Fink, R. V. (2015). Selected findings from the cross-site evaluation of the federal healthy start program. Maternal and Child Health Journal, 19(6), 1292-1305. doi:http://dx.doi.org.ezproxy1.apus.edu/10.1007/s10995-014-1635-4

Claes, C., Ferket, N., Vandevelde, S., Verlet, D., & De Maeyer, J. (2017). Disability policy evaluation: Combining logic models and systems thinking. Intellectual and Developmental Disabilities, 55(4), 247-257,276,278. doi:http://dx.doi.org.ezproxy2.apus.edu/10.1352/1934-9556-55.4.247

Developing a Logic Model to Guide Evaluation: SAMHSA’s Center for the Application of Prevention Technologies. Retrieved from

https://www.samhsa.gov/capt/sites/default/files/resources/developing-logic-model-guide.pdf

Innovation Network. Logic Model Workbook. Retrieved from

http://www.pointk.org/client_docs/File/logic_model_workbook.pdf

Week Six – Writing the Report and Presenting Your Findings

Program Evaluation: Some Agencies Reported that Networking, Hiring and Involving Program Staff Help Build Capacity. (2014)

Retrieved from https://www.gao.gov/assets/670/666893.pdf

Assessment and Evaluation. OPM. Retrieved from https://www.opm.gov/services-for-agencies/assessment-evaluation/

Review all links under each tab on the left under the topic.

Week Seven – Student Projects

Please review readings on how to conduct an evaluation and how to report on it to better inform your project this week.

Week Eight – Student Projects (analyze professional standards/ethics in conducting evaluations)

Place, K. R. (2015). Exploring the role of ethics in public relations program evaluation. Journal of Public Relations Research, 27(2), 118 - 135. doi:10.1080/1062726X.2014.976825

Morris, M. (2015). Research on Evaluation Ethics: Reflections and an Agenda. New Directions For Evaluation, 2015(148), 31-42. doi:10.1002/ev.20155

American Evaluation Association. American Evaluation Association Guiding Principles for Evaluators. Retrieved from http://www.eval.org/p/cm/ld/fid=51

Additional Resources

Web Sites

The APUS library offers a fabulous resource, the Library Course Guide! You can find the Guide for this course at http://apus.campusguides.com/content.php?pid=313913 .

Please explore all the tabs. The Articles Tab contains many of the Public Administration Journal links and the web resources are extensive.

In addition to the required course texts, the following public domain web sites are useful. Please abide by the university’s academic honesty policy when using Internet sources as well. Note web site addresses are subject to change.

http://www.cdc.gov/EVAL/resources/index.htm

http://www.eval.org

http://managementhelp.org/evaluation/program-evaluation-guide.htm

http://web.undp.org/evaluation/

http://www.innonet.org

http://www.data.gov

http://www.cdc.gov/tb/programs/evaluation/TBEvaluationHandbook_tagged.pdf

Book Title:Various resources from the APUS Library & the Open Web are used. Links provided inside the classroom in the Lessons section.
Author: No Author Specified

Previous Syllabi

Not current for future courses.