Program review

Purpose (download Program Review Overview Memo)

In 2015, Syracuse University had 502 registered academic programs. While recent “clean up” efforts have helped with retiring some long-dormant programs, we have almost double the number of academic programs as our peer universities of similar size. That is a huge workload – in curriculum maintenance, program assessment, advising, compliance and record keeping – and a diversion from engagement in our missions of teaching and scholarship. In a 2015 report, the Educational Advisory Board commented, “… the proliferation of courses, specializations, and programs spreads resources more thinly across a broader array of activities, reducing quality. . . while at the same time producing a level of complexity that creates barriers to student success.” By giving better attention to our existing academic offerings we can teach the topics we love in the context of academic programs that are better structured to meet the learning goals of our students.

Our regional accreditor, the Middle States Commission on Higher Education, comments that, “program review [is] used to change and improve educational programs, consistent with institutional values, purpose, and goals.” Middle States expects us to review every academic program we offer on a four-year cycle, to use our feedback from program assessment to improve existing programs, and to merge or sunset programs that have reached the natural juncture for such changes. The purpose of program review is to craft and maintain a set of high-quality academic programs that support our educational objectives for students while making effective use of our institutional resources. If we do this well, we will maintain a list of high-quality programs that are consistent with our mission, sought by students, and sustainable.

Academic programs are reviewed for their quality, demand, cost-effectiveness, and centrality to mission (emphasis added).” Defined below, these four characteristics comprise the basis of Syracuse University’s program review:

  • Quality
    • The quality of the program is demonstrable by the extent of student learning, student persistence, employment outcomes, or other markers appropriate to the discipline.
  • Demand
    • here is sufficient student demand, in the form of student enrollments and/or student majors, and sustenance or growth potential to warrant maintaining the program.
  • Cost Effectiveness
    • The value of the program to students and to the university warrants the resources required to maintain the program.
  • Centrality to Mission
    • The program is deeply connected to successful execution of our mission as a pre-eminent and inclusive student-focused research university as well as the specific mission of the school/college.

2017 Timeline

Syracuse University Program Review Timeline
Syracuse University Program Review 2017 Timeline

Process

Each school and college at Syracuse University will engage in a process that reviews each program on a four-year cycle. Syracuse University’s program review process incorporates all of the following features:

  • A fair and equitable, faculty-driven procedure for evaluating each program for which the school/college is responsible, using assessment outcomes, institutional data, and disciplinary norms to make recommendations;
  • Collecting evidence for evaluation that address the four characteristics of quality, demand, cost- effectiveness, and centrality to mission;
  • A schedule for program review that allows for each program in the school or college to be evaluated at least once every four years; for schools/colleges or programs with specialized accreditation, the specialized accreditation timelines should be factored into the program review calendar to minimize redundant work;
  • A process of cross-college consultation on joint programs and other programs where modifications, mergers, or closures would affect the work of another school/college;
  • An annual, school-wide or college-wide review of the full portfolio of programs to act on recent recommendations and ensure fit with the mission of the school/college and the university. This review should consider program-specific assessment plans and annual assessment progress reports as one element of the evaluation process.
  • Evaluation conclusions and recommended improvements identified in a review process should be included as success criteria in the next review cycle.

Steps to Undertake Systematic Program Review

  1. Appoint a Program Review Chair: Each dean can appoint a faculty member or staff member to take responsibility for structuring the review process. This can be the same person who leads in the area of assessment or curriculum. Program review is an every-year activity, so this responsibility would ideally be for a multi-year period. For schools that have specialized/disciplinary accreditation, these processes may mesh with the existing assessment and compliance duties of a staff or faculty member who runs the specialized accreditation process. The Program Review chair will set the annual schedule for evaluation and provide the reporting deadlines for program representatives and others involved in the program evaluation process.
  2. Appoint Program Representatives: For each unit (e.g., an academic department) that “owns” a program or set of related programs, the dean appoints one or more faculty representatives who can gather data (see next item) about the programs under review in a given year. Depending upon local culture, this could be a faculty member, program director, a department chair, or an associate chair.
  3. Collect Data: Data about a program should always contain the common data elements (below) plus any additional indicators the program representatives consider relevant in each of the four areas. The Office of Institutional Research and Assessment (OIRA) will provide the common data elements, and will also be helpful assembling additional data.
  4. Submit Program Reports to Curriculum Committee: Each school and college has at least one committee dedicated to curriculum management. This committee can obtain program reports from program representatives and evaluate them using a uniform set of judgment criteria that apply to all programs in the school or college. For each reviewed program, the Curriculum Committee should make an evaluative judgment and a recommendation: update the program with suggested improvements, maintain the program as is, merge the program with another related program, or close the program.
  5. Provide Mechanisms for Faculty Appeal of Recommendations: Program representatives should have an opportunity to consult with program faculty and, if necessary, appeal program recommendations by presenting additional program data to the curriculum committee. Acting through the program review chair, the dean of the school/college can apply a set of deadlines and adjudication procedures to ensure a fair and equitable final decision about the program(s) in question.
  6. Conduct Consultations and Program Actions: Substantive modifications to a program’s academic content, a decision to merge, or a decision to close should be undertaken in consultation with other schools/colleges that may be affected by program changes. Most program and course changes require the approval of the Senate Committee on Curricula, and some program changes also involve communications with the New York State Department of Education and our regional accreditator, the Middle States Commission on Higher Education. The Office of the Provost and Vice Chancellor can provide guidance and support on all of these administrative steps.

Evidence for Program Evaluation

Evidence evaluated for program review should comprise a core data set common across all program reviews, plus any additional data the school/college may deem appropriate for individual programs. A concise narrative should accompany the presentation of the data. The table below lists the core data set, follow by examples of additional data schools and colleges may find valuable when examining individual programs. Additional data should be chosen with the aim of providing multiple types of evidence to provide a holistic view of the program.

Within broad disciplinary areas (such as STEM) it will be valuable to use consistent criteria for programs in each area. Academic program leaders should not pick and choose what evidence to present, rather, the school/college curriculum committee should provide guidance and set expectations as to what evidence should be included in each program’s case file.


Data Elements Common to All Program Reviews

Quality

  • Student learning outcomes assessment results
  • Retention and graduation rates
  • Certification or licensing exam pass rates (if applicable)
  • Post-graduate outcomes (employment, graduate school)

Centrality to Mission

  • Demonstration of tie to mission and strategic plan at institution and school/college levels

Demand

  • Five-year trend of student majors
  • Five-year trend of degrees awarded
  • Five-year trend of applications to program

Cost-effectiveness

  • Total cost of salary and benefits for faculty and staff supporting program
  • Faculty FTE per semester credit hour of instruction
  • Instructional cost per semester credit hours of instruction
  • Class size