Course Code: PADM615 Course ID: 2675 Credit Hours: 3 Level: Graduate
This course is designed to develop fundamental skills essential for students to evaluate public programs. Knowledge of the policy process and research methods is brought together in the ethical assessment of program needs, processes, and outcomes.
|Registration Dates||Course Dates||Session||Weeks|
|11/30/20 - 04/30/21||05/03/21 - 06/27/21||Spring 2021 Session I||8 Week session|
|01/25/21 - 07/02/21||07/05/21 - 08/29/21||Summer 2021 Session B||8 Week session|
|03/29/21 - 09/03/21||09/06/21 - 10/31/21||Summer 2021 Session D||8 Week session|
After successfully completing this course, you will be able to:
- Evaluate the role of program appraisal and evaluation in public administration.
- Analyze the basic models of program evaluation.
- Evaluate appraisal and evaluation methods as they relate to a given problem.
- Appraise the different project cycle phases.
- Analyze the professional standards and ethical issues involved in conducting program evaluations.
- Assess how data is collected and analyzed.
- Compare and contrast the use of quantitative and qualitative research methods in evaluating public programs.
- Design a survey and evaluation plan for a real world example.
- Produce an evaluation presentation that would serve public program administrators.
Post your initial response to each forum by 11:55 pm ET on Thursday. This course is geared toward a practical application of the concepts we are studying so the forums may or may not be supplemented with scholarly sources. It is more important they are original and documented with interview contacts and detailed responses to the question. Just remember to give attribution!
Questions and topics posed in the Forums are designed to promote thought and insight. Students must provide a critical review of the questions, topics and issues posed and substantively reply to the contributions of at least three peers. Individual postings should include a full discussion of the content of the question posed and explain how it relates to the concepts in the weekly text readings and other resources. The postings should be analytic in nature and include comparisons/contrasts, and examples that can bolster your point. The Forum is for your benefit and it is important to respond to the discussion topic and to engage others in a running dialogue.
Your initial post should be made by Thursday each week. You should then respond to 3 or more posts. Note that at least one response to your classmates must be made before Sunday. If you make all of your responsive posts on Sunday, you will not earn full points for timeliness.
This can be accomplished by
· Validating with additional evidence from the literature.
· Posing a thoughtful question with commentary which generates further discussion.
· Providing an alternative point-of-view, with evidence and examples.
· Offering additional insight into how the concept might be understood, with evidence provided with real world examples.
You should be active in the classroom throughout the week and actively engaged in the back-and-forth discussion between your colleagues and the professor.
The forum grading rubric can be found in gradebook by clicking on the forum entry.
Assignment #1 Survey Draft
This week we start by designing a survey. For the purposes of this class you have already chosen the National Park that you would like to evaluate and have reviewed the lesson that contains an example of the visitor survey the National Parks actually use. I have used that as my model for my examples but you do not need to ask those same questions. Design your survey to measure what you want to measure about the Park. If it is visitor count that would be fine, if it is something else like the use of maintenance vehicles per Park employee, the appropriate number of campsites per acre, etc. please feel free. If the results are not available, you are welcome to make up hypothetical results for the purpose of this class. For the first part of this assignment you are to create a word document survey using the knowledge gained from the readings and lessons so far. You may find more help at https://www.surveymonkey.com/mp/how-to-create-surveys/ . In reality you would use a site like Survey Monkey to create and disseminate the survey but for our purposes you will do a very rough draft in Word. It should contain the basic format and questions. Please save that document. Please submit your survey to the students identified in the assignment. Please also submit it to the assignment tab for my comment. (I am evaluating the survey in this assignment.)
Assignment #2 Survey Peer Review
Once you have the draft from your classmate, please comment on the survey instrument. Did the student ask pertinent questions? Do they get to the result intended by the survey? Is the information useful? How would you feel as a consumer taking the survey? Too long? Not worth your time? Makes you feel like a contributing member of the organization? Are there questions that should be there that are not? Once you have commented please send this back to your classmate. Please upload the survey with your comments to the assignment (I am evaluating the comments in this assignment)
Assignment #4 – Evaluation Presentation
For this assignment you will create a presentation of your findings. Consider that you were a consultant asked to do this evaluation and now you are presenting your results to the Park staff and local community. If the stats are available for your Park on the Visitor Use stats site https://irma.nps.gov/Stats/ incorporate those if they are relevant. If they are not you will need to generate hypothetical results for your survey in order to complete this assignment. You can choose to do this in many different ways. Attached is an example of my presentation of the Dayton Aviation Heritage National Park done in Adobe Premier Pro. You may use Adobe Apps, Knovio, iMovie or any other application that allows for a multimedia presentation. It has to contain at least voice and at least one image. You can use multiple images like those I took and uploaded to Adobe Premier Pro in the example or it can be simply an image of you sitting in front of your webcam giving the report as if you were in front of a crowd. It could be an animated image if you are extremely good with these apps but I will be looking for something that the class can view so it can't be in program we can’t view. If you create the report in Knovio, you will provide a link for me to view it. If you create it in something that creates a movie, you need to export it in a file that can be uploaded to Kaltura under the video option so we can view it. Remember, as a public administrator you should be concerned not only with your presentation (between 5 - 8 minutes generally) and its content, but also that it is accessible and ADA compliant. In order to insure that it is, you should have a transcript along with your presentation.
Assignment #5 - Program Evaluation Poster
It is important that the results of an evaluation are clearly and simply portrayed to the funder/requesting organization and your target audience. Develop a poster for the Park with which you have been working. Presume you have been asked to present your evaluation at the American Evaluation Association annual conference. If you are not familiar with poster sessions, see their guidelines at http://www.eval.org/p/cm/ld/fid=172 . A poster is just a visual display of your evaluation, highlighting the important points so that someone walking by can visually understand your findings. Examples of posters and the process of a poster competition can be found here. More information can be found at http://www.uhd.edu/academic/colleges/sciences/scholars/files/workshop-poster.pdf and http://colinpurrington.com/tips/academic/posterdesign . I am not looking for a professional graphic design but some of the tips in these websites will help you create a clear, concise presentation of your findings. The “poster” should be on one page. You can create it in word, power point, or any of the adobe creative cloud applications (inDesign and Illustrator would work well for this project). If you create it using other programs, please save it as a pdf before submitting.
- The assignment rubric is attached to each assignment in the assignment section of the classroom.
|Forum Questions||30.00 %|
|Forum Question 1||0.37 %|
|Forum 1.2||3.70 %|
|Forum Question 2||3.70 %|
|Forum Question 3||3.70 %|
|Forum Question 4||3.70 %|
|Forum Question 5||3.70 %|
|Forum Question 6||3.70 %|
|Forum Question 7||3.70 %|
|Forum Question 8||3.70 %|
|Survey Draft||20.00 %|
|Survey Draft 1 week 3||20.00 %|
|Evaluation Presentation week 6||20.00 %|
|Evaluation Poster week 7||20.00 %|
|Survey Peer Review||10.00 %|
|Survey Peer Review week 4||10.00 %|
Week One – Program Evaluation Overview
Benchmarking Foundation Evaluation Practices. Center for Effective Philanthropy.
State of Evaluation 2016 Evaluation Practice and Capacity in the nonprofit sector. Innovation Network. Retrieved from https://www.innonet.org/media/2016-State_of_Evaluation.pdf
American Evaluation Association. AEA 365. A Tip A Day for Evaluators. Retrieved from http://aea365.org/blog/
Review several times a week throughout the class.
USAID Evaluation Toolkit. Learning Lab. Retrieved from https://usaidlearninglab.org/evaluation?tab=1
Review all tabs
Free Management Library. Basic Guide to Program Evaluation. Retrieved from http://managementhelp.org/evaluation/program-evaluation-guide.htm
Review all tabs
Li, W., & Mauerhofer, V. (2016). Behavioral patterns of environmental performance evaluation programs. Journal of Environmental Management, 182, 429-435. doi:10.1016/j.jenvman.2016.07.085
Gargani, J., Miller, R. L., & Spiegelman, D. (2016). WHAT IS PROGRAM EVALUATION?/SPIEGELMAN RESPONDS. American Journal of Public Health, 106(6), E13-E14. doi:http://dx.doi.org.ezproxy2.apus.edu/10.2105/AJPH.2016.303159
Huckel Schneider, C., Milat, A. J., & Moore, G. (2016). Barriers and facilitators to evaluation of health policies and programs: Policymaker and researcher perspectives. Evaluation and Program Planning, 58, 208-215. doi:10.1016/j.evalprogplan.2016.06.011
Hall, J. L. (2017). PERFORMANCE MANAGEMENT: CONFRONTING THE CHALLENGES FOR LOCAL GOVERNMENT. Public Administration Quarterly, 41(1), 43-66. Retrieved from https://search-proquest-com.ezproxy2.apus.edu/docview/1871486263?accountid=8289
Week Two – Assessing Program Need
Altschuld, J. W., & Watkins, R. (2014). A primer on needs assessment: More than 40 years of research and practice. New Directions for Evaluation, 2014(144), 5-18. doi:10.1002/ev.20099
Engle, M., & Altschuld, J. W. (2014). Needs Assessment: The Perspective From the Public Sector. New Directions For Evaluation, 2014(144), 33-45. doi:10.1002/ev.20101
Lubejko, B. G. (2016). Developing a program evaluation plan: Options and opportunities. Journal of Continuing Education in Nursing, 47(9), 388 - 389. doi:10.3928/00220124-20160817-02
Watkins, R., & Altschuld, J. W. (2014). A Final Note About Improving Needs Assessment Research and Practice. New Directions For Evaluation, 2014(144), 105-114. doi:10.1002/ev.20106
Wedman, J. (2014). Needs Assessments in the Private Sector. New Directions For Evaluation, 2014(144), 47-60. doi:10.1002/ev.20102
Energy.gov. Program Evaluation. Why, What and When to Evaluate. Retrieved from https://energy.gov/eere/analysis/program-evaluation-why-what-and-when-evaluate
Please review all links.
Center for Disease Controls and Prevention. Program Performance and Evaluation Office. Retrieved from
Review all tabs.
Week Three - Data Source and Analysis/Sampling and Surveys
Carr, K. M., & Bradley-Levine, J. S. (2016). Translational research design: Collaborating with stakeholders for program evaluation. The Qualitative Report, 21(1), 44 - 58.
Mojtahed, R., Nunes, M. B., Martins, J. T., & Peng, A. (2014). Equipping the constructivist researcher: The combined use of semi-structured interviews and decision-making maps. Electronic Journal of Business Research Methods, 12(2), 87 - 95.
Wageman, R., Hackman, J. R., & Lehman, E. (2005). Team diagnostic survey: Development of an instrument. The Journal of Applied Behavioral Science, 41(4), 373-398. doi:10.1177/0021886305281984
McIver, D., & Wang, X. ". (2016). Measuring knowledge in organizations: A knowledge-in-practice approach. Journal of Knowledge Management, 20(4), 637-652. Retrieved from https://search-proquest-com.ezproxy2.apus.edu/docview/1826809475?accountid=8289
Couper, M. P., & Peterson, G. J. (2017). Why do web surveys take longer on smartphones?Social Science Computer Review, 35(3), 357-377. doi:10.1177/0894439316629932
Conrad, F., Tourangeau, R., Couper, M., & Zhang, C. (2017). Reducing speeding in web surveys by providing immediate feedback. Survey Research Methods, 11(1), 45-61. doi:10.18148/srm/2017.v11i1.6304
Vaughn, P., & Turner, C. (2016). Decoding via coding: Analyzing qualitative text data through thematic coding and survey methodologies. Journal of Library Administration, 56(1), 41-51. doi:10.1080/01930826.2015.1105035
Willits, F. K., Theodori, G. L., & Luloff, A. (2016). another look at likert scales. Journal of Rural Social Sciences, 31(3), 126 - 139.
Week Four – Reporting the Results
Schmitt, C. L., Glasgow, L., Lavinghouze, S. R., Rieker, P. P., Fulmer, E., McAleer, K., & Rogers, T. (2016). Measuring infrastructure: A key step in program evaluation and planning. Evaluation and Program Planning, 56, 50-55. doi:10.1016/j.evalprogplan.2016.03.007
Ebener, Patricia et al. (2017) Rand Corporation. Getting to Outcomes Guide for Community Emergency Preparedness. Retrieved from https://www.rand.org/pubs/tools/TL259.html
Download or read the Guide online.
Emery, Ann K. (2014) How to Present Data When you are Presenting: Storyboarding your Data…Retrieved from http://annkemery.com/storyboarding/ . Please read the short article and watch the 2 videos.
Emery, Ann K. (2015) When Pie Charts are OK. Retrieved from http://annkemery.com/pie-chart-guidelines/ .
Week Five – Logic Models
Drayton, V. L., Caleb, Walker, D. K., Ball, S. W., Donahue, S. M., A., & Fink, R. V. (2015). Selected findings from the cross-site evaluation of the federal healthy start program. Maternal and Child Health Journal, 19(6), 1292-1305. doi:http://dx.doi.org.ezproxy1.apus.edu/10.1007/s10995-014-1635-4
Claes, C., Ferket, N., Vandevelde, S., Verlet, D., & De Maeyer, J. (2017). Disability policy evaluation: Combining logic models and systems thinking. Intellectual and Developmental Disabilities, 55(4), 247-257,276,278. doi:http://dx.doi.org.ezproxy2.apus.edu/10.1352/1934-9556-55.4.247
Developing a Logic Model to Guide Evaluation: SAMHSA’s Center for the Application of Prevention Technologies. Retrieved from
Innovation Network. Logic Model Workbook. Retrieved from
Week Six – Writing the Report and Presenting Your Findings
Program Evaluation: Some Agencies Reported that Networking, Hiring and Involving Program Staff Help Build Capacity. (2014)
Retrieved from https://www.gao.gov/assets/670/666893.pdf
Assessment and Evaluation. OPM. Retrieved from https://www.opm.gov/services-for-agencies/assessment-evaluation/
Review all links under each tab on the left under the topic.
Week Seven – Student Projects
Please review readings on how to conduct an evaluation and how to report on it to better inform your project this week.
Week Eight – Student Projects (analyze professional standards/ethics in conducting evaluations)
Place, K. R. (2015). Exploring the role of ethics in public relations program evaluation. Journal of Public Relations Research, 27(2), 118 - 135. doi:10.1080/1062726X.2014.976825
Morris, M. (2015). Research on Evaluation Ethics: Reflections and an Agenda. New Directions For Evaluation, 2015(148), 31-42. doi:10.1002/ev.20155
American Evaluation Association. American Evaluation Association Guiding Principles for Evaluators. Retrieved from http://www.eval.org/p/cm/ld/fid=51
The APUS library offers a fabulous resource, the Library Course Guide! You can find the Guide for this course at http://apus.campusguides.com/content.php?pid=313913 .
Please explore all the tabs. The Articles Tab contains many of the Public Administration Journal links and the web resources are extensive.
In addition to the required course texts, the following public domain web sites are useful. Please abide by the university’s academic honesty policy when using Internet sources as well. Note web site addresses are subject to change.
|Book Title:||Various resources from the APUS Library & the Open Web are used. Please visit http://apus.libguides.com/er.php to locate the course eReserve.*|
Not current for future courses.