Announced On Site Evaluation Visits
Prior to engaging in this process, a thorough review of available documentation is completed by the Program Evaluation Unit, to include a review of the approved grant application and amendments.
Evaluation Interview
As with previous years, the evaluation visit will consist of an interview, site visits, and a debriefing. In addition to other topics, the interview content will include questions regarding the following:
- Program Changes: Changes during this program year to goals, objectives, activities, operations, or personnel for which your program has not submitted a formal grant amendment.
- Principal Goals of the 21st CCLC Grant: Successes to date in helping 21st CCLC students meet or exceed state and local standards in core academic subjects, enrichment activities offered to complement the regular school day, and services offered to families/guardians of students in your program.
- Data / Overall Program Evaluation: Staff responsibility for data collection, submission, and analysis at program sites and for the overall program, data storage and security, frequency of data analysis and with whom the analyses are shared, use of data for program improvement (including adjustments made when not on target to meet an objective), and (if applicable) services provided by external evaluator.
- Evaluation of Individual Objectives: Review of data collected, and plans for future collection, for each objective (including baseline/pre-test, ongoing assessment of progress, and post-test), and current progress towards meeting each objective. Evaluators will ask to see the actual data collected for each objective, so please have data available. NOTE: You do not need to make copies for the evaluators to take with them, but it is important that you are prepared to show them data for each objective in whatever form it is available (e.g., spreadsheets of test scores, family night attendance sheets).
- Training in Afterschool Instruction: Trainings or orientations your program provides for site coordinators, teachers, and any other staff who may work directly with students.
- Program Management: Project director's level of involvement with program sites (e.g., how often they meet with site coordinators and staff, how often they visit sites).
- Service Learning: Service learning activities implemented or plan to implement.
- Program Dissemination: Program information dissemination for student recruitment, communication with the families of eligible and participating students, and communication with program partners or potential partners. Please be prepared to provide evaluators with examples of informational materials (e.g., flyers, brochures).
- Program Success / Improvement: The most significant benefit of your program for the community served and areas most in need of improvement.
- Grant Administration Feedback: Greatest overall challenges in administering the 21st CCLC grant and help or services needed to overcome the challenges. You will also have an opportunity for questions, comments, or suggestions for the Program Evaluation Unit.
Student Attendance
As part of the 2009-2010 evaluation process, all 21st CCLC sub-grantees must provide evaluators copies the last 10 days of actual attendance rosters for EACH SITE at the beginning of the evaluation visit. This is not simply numbers or counts of students, but is the full attendance rosters with student names for all sites in the program. This must be provided as a hard copy of the rosters.
Site Visits
Information regarding expectations of the site visits is included in the Site Worksheet. While some sites may not be visited, the program must provide a completed Site Worksheet for each site in the program by the end of the evaluation visit. Please note that site visits will last for the entire duration of program operation (i.e., student arrival through student dismissal) on the day on which they occur.
Debriefing
The debriefing will take place following the interview and site visits. During the debriefing, the program evaluators will provide immediate feedback regarding program strengths and challenges.
In addition to the debriefing, the program will be provided a program-wide written report. Programs are strongly encouraged to take detailed notes during the debriefing process, as the written report may not include the same level of detail presented during debriefing sessions. The report will, however, outline the evaluators' findings with the intent of assisting you and your staff in sharing successes and addressing concerns. Reports will be distributed to the program director and grant administrators via email. It is anticipated that reports will be released within two weeks following the evaluation visit.