Program Review Process
The program review is a participatory process where state staff and local project teams for the Oregon Outreach demonstration sites meet collaboratively for an in-depth look at the local program. It is intended to be a congenial, informative session where local staff can showcase the local program, while at the same time state staff can become more knowledgeable about program activities and more aware of how they can best support local efforts. (For more information on the program review process, see Evaluation Techniques.)
Listed below is a list of the program review questions clustered into the categories of educational programming, collaborative efforts, involvement of volunteers, connectivity, sustainability, and general management. The introductory paragraph is directed to the local program staff who will be hosting the review.
Program Review Questions
Program review questions are listed below, grouped by category. Knowing the questions ahead of time will give you time to collect your thoughts, as well as any supporting materials which may help illustrate or support your response. These materials might include photos of project activities, promotional brochures, lesson plans, activity sheets, examples of participant activities, and so on. This is an opportunity for you to showcase your project.
Questions will not necessarily be asked in the order they appear below, and some questions may be added as they spontaneously occur. You may ask questions of us at any time it seems appropriate to the discussion. The intent is to have a congenial, free-flowing, and productive conversation. If the overall process is not clear to you, please let us know as soon as possible so we can clarify any issues before the review.
Educational Program(s)
(If more than one program is being offered, the following questions apply to each program.)
- Has the program been implemented according to plan? What changes have been made since last year, why, and with what results?
- What have been major successes of the program so far? Major disappointments?
- How many youth are participating in the program? What is your target number? How does this compare with the number of potential participants? What is the pattern of participation? (regulars? drop-ins? drop-outs? ) How do program participants differ from non-participants? Is the following information being documented on an ongoing basis?
- Attendance records with the names of participants, volunteers, and program staff present at each session;
- The gender and ethnicity of participants and volunteers;
- The time, date, location, and length of each session offered;
- Notes, outlines, or materials identifying the content and process of each session and program staff in attendance.
- Do participants pay for any of the program costs?
- What is the approximate cost per program participant? (total program income divided by the number of participants)
- How are participants recruited?
- How is the program marketed locally?
- Is the list of educational goals and objectives in the program description up-to-date?
- What teaching strategies (educational methodologies) are being used?
- What learning activities are being used to specifically address each of the learner objectives? (Go through each learner objective individually.)
- How are youth involved in helping to design the learning experience?
- How are participants reacting to the program?
- How are school or community groups involved in planning and/or implementing the program?
- How is cultural sensitivity/appropriateness/appreciation/diversity being addressed?
- Is the program interrelated with the rest of the 4-H youth development program? If so, how?
- What is the status of program evaluation plans?
- What evidence have you documented the past year relating to the primary outcome(s) targeted by your project? Secondary outcomes?
- Has the program had any unexpected or unintended outcomes (positive or negative)?
- In what ways can state staff be more supportive of your educational program?
Collaborative Efforts
- Have your proposed activities been implemented according to plan? What changes have been made, why, and with what results?
- What are the major successes so far? The major disappointments?
- What progress is being made in achieving your outcome objectives?
- What collaborations have been initiated or supported?
- What is the nature of these collaborations? (networking, cooperation or alliance, coordination or partnership, coalition, collaboration)
- What missions/mandates/purposes do these collaborations serve?
- What are the results of these collaborations? (real people impacts, policy development, systems development, resource development)
- How have youth been involved in these collaborative efforts?
- What capacity-building activities have been implemented (or supported) by the project?
- How are cultural sensitivity/appropriateness/appreciation/diversity being addressed?
- Have there been any secondary outcomes? Unexpected or unintended outcomes? (positive or negative)
- Is there any particular assistance and/or resources state staff might provide in this area?
Involvement of Volunteers
- How are volunteers involved in the overall project?
- How many are involved and in what roles? How many are adults? Youth?
Latino? Non-Latino?
- Are there written position descriptions for the various volunteer roles?
- How are the volunteers selected/recruited/screened?
- How are volunteers oriented? What training do they receive?
- How are volunteers recognized for their efforts?
- How are cultural sensitivity/appropriateness/appreciation/diversity addressed?
- What types of support from the state staff would be helpful in activities relating to volunteers?
Connectivity
- How are computers being used in the project?
- How are youth and adults being trained in computer skills? Who does the training?
- What opportunities do youth and adults have to practice their computer skills?
- In what ways do project staff use computers in support of project activities?
- What types of support would be helpful in connectivity efforts?
Sustainability
- Which parts of the project do you see continuing beyond the grant?
- How do you see these being sustained when grant money is no longer available?
- Is there a plan or timeline relating to sustainability?
- What strategies are you using (will you be using) to bring about sustainability? What progress has been made so far?
- How can state staff be helpful as you deal with this issue?
General Management
- Who is involved in making decisions about the project?
- Is there an advisory group?
- What contributions have individuals or groups in the community made to the program? (Funds, supplies and materials, facilities, transportation, etc.)
- What activities take up most of the staff's time?
- Are state timelines, reporting expectations, etc. realistic?
- Considering the overall project, what is working? Not working? How are you addressing the parts of the project that need improvement?
- What are the most important things you've learned so far?
- Is there anything you wish you had done differently?
- Are there additional ways state project staff can be more supportive of your efforts?
Prepared by Barbara Sawer, Ed.D., Oregon Outreach project evaluator. |