Venia ~ Veritas ~ Sanctitas - University Motto

Foreword

The Capacity and Preparatory Review for Point Loma Nazarene University (PLNU) is the result of many months of planning and work by a wide variety of personnel in the University community.  This report, prepared for the Western Association of Schools and Colleges (WASC), is the evidence for PLNU’s case that it meets the WASC Core Commitment to Institutional Capacity.  Under the direction of our Steering Committee, we carefully followed the outline of tasks as found in the approved Institutional Proposal.  Early in the planning phase, we chose to address the process of reaffirming accreditation standards in the following ways.  First, we established an institutional Data Portfolio following WASC guidelines both for our use in the process at hand and for future decision making at the University.  Second, we created and focused on indicators of PLNU’s educational effectiveness.  Third, we analyzed the University’s record on selected WASC Standards.  Fourth, we completed a study of concerns previously mentioned by WASC:  planning, assessment, and diversity.  Finally, we concluded with a synopsis of the overall review.  There were no post-proposal changes in our approach to this work.

The reflective essays and supporting documentation in this Capacity and Preparatory Review (CPR) report are the work of seven study groups involving Board of Trustees, administration, staff, and faculty members.  The process of writing, finalized by the Steering Committee, is the culmination of data gathering, focus groups, surveys, discussions, and analyses.  We also posted group reports for comment from the broader University community.  With the Steering Committee as a review panel, we considered suggestions and comments, and then created a final draft that flowed directly out of that work.  We made available to the entire University community the original studies done by various groups as well as the final CPR report in its entirety, and we encouraged everyone to review and read the Institutional Presentation as posted on the PLNU Web site.  The reflective essays in this CPR report discuss the development of a data set and educational effectiveness indicators, key WASC Standards, planning, assessment, and diversity.

Data Portfolio.  The assembling of an institutional Data Portfolio was a major undertaking for the University.  While much data was available in disparate units of the University, there was no central repository as there is today.  The insights and evidence gleaned from this common site, essential for the writing of the CPR report, already are having an impact on quality improvements and strategic planning.

Educational Effectiveness Indicators.  We developed widely shared general educational goals over the course of many months and in several settings, suggested specific predictors of student success in achieving these goals, and designed a system for measuring those predictors.  As a result of this process, we reviewed the results and made recommendations focusing primarily on the commonalities of undergraduates in general education.

Standards One and Three.  These Standards focus on purposes and educational objectives as well as on the resources and structures that ensure sustainability in the institution.  While we are confident that we meet or exceed the Standards in these categories, in this section we focused on issues not addressed precisely in other parts of the CPR report as outlined in the Institutional Proposal.  These include the assessment of educational objectives, academic freedom, institutional autonomy, faculty commitment and qualifications, information technology and resources, and decision making processes.

Standards Two and Four.  The Standards in this section of the CPR report underscore the core functions required to achieve educational objectives and the fundamentally essential factor of creating and improving ourselves as a learning community.  We assert our capacity to meet these Standards and address specifically the issues related to graduate programs and curricula, retention and graduation rates, the improvement of teaching and learning, transfer student support, quality assurance review, institutional research, and assessing teaching and learning.

Planning.  After a series of documented efforts, planning has come to a new level of maturity at the University.  This reflective essay, together with supporting documentation, recounts this journey and demonstrates a viable system that flows from Mission and Vision, involves key personnel, invites broad PLNU community input, is thorough yet uncomplicated, and is tied to the budget process.  Along with this self-evaluation of planning, we created an Academic Plan (as requested by WASC) that transforms the past model of adequate yet informal notions of needed improvements into a dynamic approach to planning that is flexible and also includes specific annual initiatives.

Assessment.  One of the significant benefits of the WASC process has been the identification of multiple efforts taken by the University to build formal structures for assessment activities, practices, oversight, and support.  We now have in place an assessment program in which a culture shift has occurred in academic and co-curricular units.  The members of these units are involved in a continual learning process with the purpose of improving unity, coherence, and outputs within a democratized governance structure.  Rather than being forced on reluctant units uncertain about the purpose and benefits of assessment, the process has embraced faculty and co-curricular units in a community-wide effort for common, on-going learning.

Diversity.  PLNU is committed to our Core Value of Diversity, and we have taken deliberate steps to put in place a variety of strategies, both curricular and co-curricular.  Our reflection for this report covers efforts to support gender equality, cultural diversity, and racial/ethnic diversity.  There are indications of steady improvements in the data.  Our evaluation of the current status of this Core Value has produced not only a catalog of initiatives and encouraging news but also several recommendations for future improvements.

Reflective Essay One:  Data Portfolio

Introduction.  As stated in the Institutional Proposal, an important beginning point for the entire WASC process was the creation of a centralized data set that provides evidence of effective development and deployment of resources, structures, and systems.  Since PLNU had limited sources of evidence in disparate locations, such as the home page of the Office of Institutional Research, we first undertook the task of expanding dramatically the institutional data set.  We also completed an analysis of data collection and use, with suggestions for how these processes could be improved.

Development of the University Data Set.  For the last 15-20 years, many offices of the University gathered data for planning and assessment purposes, yet without the intent of or the concern for maintaining a centralized institutional data set.  Although PLNU established the Office of Institutional Research in 1984, since that time the office has served exclusively to satisfy the reporting requirements of various agencies and organizations and to conduct routine surveys.  We recognized the need of a comprehensive data set and began by utilizing WASC’s Required Data Elements to Support the Capacity and Preparatory Review.

The process of creating the data set began in October of 2004 and ended in May of 2005.  We carefully documented and footnoted the sources of the data in the tables so that there would be consistency across the years as we update these tables.  This detailed work not only made an important contribution to the University’s commitment to establishing a culture of evidence, but it also uncovered difficulties in the use and storage of data.

Following provisions in the Institutional Proposal, we then focused on the analysis of the Data Portfolio including a discussion of issues, implications for the institution, and recommendations for action.  We focused on five categories:  1) verification that the Data Portfolio meets the expectations of WASC, 2) an analysis of the type of data being gathered for decision making, 3) issues arising from the data gathered, 4) additional pertinent questions related to the appropriate use of data for decision making and strategic planning, and 5) recommendations to the University regarding these issues.

Verification that the Data Portfolio Meets the Expectations of WASC.  The PLNU tables meeting the basic data requirements of WASC are located on the University’s Web site under University Data Tables.  The Educational Effectiveness Indicators (EEI’s) also are posted since they are a vital part of PLNU’s determination to establish indicators and utilize them in self-evaluation, planning, and improvement.  When it became apparent that there was a weakness in data collection for graduate programs, we captured data beginning with the 2004-2005 academic year.  We now post data on the Web, store it in a central location, and code it in a uniform way.

Analysis of the Type of Data Being Gathered for Decision Making.  The implementation of an institutional Data Portfolio has resolved to a large extent the problem of universal data collection and access.  The WASC Required Elements are a good beginning to the data set that will be tracked annually for institutional decision making.  The type of data being gathered also aligns with recently updated and adopted Mission and Vision statements (2004).  The Vision Statement establishes the expectation that “Point Loma Nazarene University will be a nationally prominent Christian university and a leading Wesleyan voice in higher education and the church—known for excellence in academic preparation, wholeness in personal development, and faithfulness to Mission.”  Since our current institutional planning efforts are rooted in the Vision, we organized the Strategic Planning Data Set around the five elements of that Vision:  Excellence in Academic Preparation; Wholeness in Personal Development; Faithfulness to Mission; Leading Wesleyan Voice in Higher Education; and Leading Wesleyan Voice in the Church.  We added Support of Vision as a sixth data category to capture financial and management goals.  Essential elements from the WASC data tables now have an association with these categories as well as the University’s Key Performance Indicators (KPI’s) and EEI’s.  The Strategic Planning Committee (SPC) continues to work with individual units of the University to determine what additional data is necessary for planning and what features will be added to the central data set of the Vision planning categories listed above.

Issues Arising from the Data Gathered.  We looked at the institutional data set represented by the University Data Tables 1.1.1 through 6.3 and made several observations.  These include admissions and student preparation; student enrollments; degrees awarded; faculty and staff composition; information, physical and fiscal resources; and institutional and operating efficiency.

Admissions and Student Preparation
One of our discoveries in building an expanded data set was the serious absence of consistent graduate admissions data (including the California teaching credential track). This was due to a lack of uniformity in admissions processes across programs and campuses.  There was also a lack of uniformity in record keeping and data coding.  These two issues now have been resolved with the introduction of a single graduate application and a centralized admissions process.  There are on-going University discussions about how to track and quantify graduate admission interviews and thus track selectivity.

Other observations include a decrease in the number of undergraduate readmissions (Table 1.1.1), good progress in the aggregate GPA and SAT scores of our incoming undergraduate classes (Table 1.2.1), a continuing imbalance in the ratio between women (60 percent) and men (40 percent) among undergraduate students—though this matches the demographics in universities of our type (Table 1.3.1), the surprisingly high percentage of men in the transfer student cohorts (Table 1.3.1), and the excellent record of growth in ethnic diversity in the undergraduate population (Table 1.4.1).  While the percentages of diverse students who apply have remained at 20 percent for the last few years, the percentage of diverse students that enrolled has increased from 15 percent to 23 percent.  The University also has detailed ethnic composition information about enrolled freshmen (Table 1.4.2).

Further study needs to be done to determine the reasons for a downturn in the University’s readmissions, whether the institution is satisfied with the ratio of women to men on the undergraduate level, whether it is significant that many more men enter the University as transfer students than as first-time freshmen, how our ethnic patterns compare to the composition of the University’s recruitment area, and which of the University’s admissions strategies are the most effective for building student diversity.  Also, it would be helpful to gather disaggregated ethnic data about applicants and admissions in order to track our success among individual ethnic groups.

Student Enrollments
In the past, much of the information reported to Integrated Postsecondary Education Data Systems (IPEDS) relative to student population was based on an annual “snapshot” taken in mid-October.  However, many of the University graduate programs did not begin instruction before that date.  Though trends in the data as it existed were correct, the actual numbers for a given year did not include all students in the snapshot.  To resolve this problem, we moved the data collection time for graduate students to early December beginning in the fall of 2005.

With the main campus enrollment cap, required by the City of San Diego, and the trend toward maximizing that limit beginning in 1999, the University eventually moved graduate-level instruction for San Diego to a new facility in nearby Mission Valley.  With this action and a more deliberate enrollment management strategy, undergraduate enrollments on the main campus have remained stable, while graduate program enrollments outside of San Diego have continued to grow (Table 2.1).  Also, there was a significant increase in the number of part-time undergraduate students (those below 12 units) in AY 2003-2004.

Finally, the gender mix in the undergraduate student population has remained stable (Table 2.2), the diversity of both undergraduate and graduate student populations has increased (Table 2.3), and the number of students receiving Pell Grants has decreased significantly (Table 2.4).  There may be value in comparing the diversity in our student population to that of our recruitment area, and studying the decrease in Pell Grants (reportedly a national phenomenon), whether due to changing government policy or a change in the affluence of our undergraduate population, and the consequences of this decrease relative to the number of hours students are working while enrolled at PLNU.

Degrees Awarded
At least three issues surfaced from an analysis of the data regarding the awarding of degrees.  1) The work of compiling Table 3.1.1 Bachelor’s Degrees Granted was a complex task because codes used for majors differed across the years.  A list linking PLNU degree titles and IPEDS codes need to be established.  2) Freshman retention and freshman graduation rates show significant improvement over the last five years (Tables 3.2.1 and 3.2.2).  Freshman six-year graduation rates are expected to increase in the next few years, as AY 2004-2005 was the graduation year of the first class under the enrollment cap for the main campus, and selectivity has increased in the intervening years.  Transfer student retention and graduation data should be tracked.  3) Currently no data on graduate student retention and graduation rates is available, though collection of this data began with the fall 2005 graduate-level snapshot.
 
Faculty and Staff Composition
Faculty and staff data has been gathered every other year in response to the demands of IPEDS reporting.  Beginning in the 2005-2006 academic year, we changed this process to an annual data collection.  The gender mix in the full-time faculty, adjunct faculty, and staff has remained consistent over time; ethnic diversity has increased (Table 4.1 and Table 4.3).  The turnover in full-time faculty and staff has been fairly low (Table 4.4).  It would be helpful, however, to track the turnover in part-time faculty, and it would also be helpful to know the average age of the full-time faculty.  The University has done well in making strategic use of adjunct faculty while maintaining its commitment to a high percentage of classes taught by full-time faculty (Table 4.2).

Information, Physical and Fiscal Resources
The University is on sound financial footing.  The percentage of income derived from tuition, however, is high and should be monitored carefully.  Based on work done through the Office of Institutional Research over the last several years, there appears to be sufficient classroom space (Table 5.2.1).  While the University’s physical facilities are well maintained and reflect the growing importance of technology, plans should be made to provide a variety of classrooms that allow for various pedagogical methods, given the increasing sophistication of incoming students and the hiring of younger faculty.

Institutional and Operating Efficiency
The University has done well in holding back grade inflation.  The average undergraduate cumulative grade-point average (GPA) has risen by approximately 0.1 over the past five years, consistent with the rise in the GPA of incoming students over the same period.  Though significant progress has been made in class-size reductions, further study should be given to trends in the number of both large and small undergraduate classes.

Additional Questions.  It is important to address how the University has reflected on the data, made meaning of it, and decided how to act on it; where the data actually enters planning and decision-making processes; and what can be said about where and how such data is considered.

While the University is collecting a very large body of data, much of it appears to be used in only a limited way by individual units.  The SPC is making decisions about the essential data to be collected, analyzed, and displayed as part of the core data set on the University Web site.  The Assessment Committee is also part of the work and is responsible for supporting and guiding PLNU academic departments and schools in the use of data to assess and improve learning outcomes.   The General Education Task Force is gathering data and assesses general learning outcomes.

Finally, it is clear from the reports of individual departments, schools, and units of the University that data is being used for planning.  This process could be helped if assessment data, department and school reviews, and unit planning documents were on display.  However, the lack of standardization among these documents makes gathering and sharing data difficult.  Data gathering would be much easier if annual reports and reviews across the University featured a consistent set of information.

See WASC CPR Group 1: Data Portfolio for the full report.

Recommendations:   (For a complete list see Recommendations to the President.)

1.1	Improve the collection and reporting of graduate program data;
1.2	Collect faculty and staff data on an annual basis, particularly for the purpose of projecting retirement trends over the coming years and financial implications;
1.3	Make available a glossary of terms for the data set;
1.4	Provide a “non-expert” financial report, such as the summary provided for the Board of Trustees, in the institutional Data Portfolio;
1.5	Study data on class size and current and future classroom needs of the University as part of the academic planning process;
1.6	Continue to grow the Data Portfolio as units of the University identify essential data needed for decision making;
1.7	Increase staffing and operational support for the Office of Institutional Research in order to assist departments and schools with their data needs;
1.8	Establish procedures for gathering standardized data, including collection dates and identifiable individuals responsible for gathering the data;
1.9	Provide a central location for the Data Portfolio, building and maintaining these tables automatically by means of the institutional database (CARS); and
1.10	Document the use of data in official reports as part of the University’s effort to have a culture of evidence and verify its usefulness in decision making and improvement.

Reflective Essay Two:  Educational Effectiveness Indicators (EEI’s)

Introduction.  As a part of the University’s drive to create a sustainable body of evidence for strategic planning and assessment, the Institutional Proposal calls for the development, review, and formalizing of EEI's.  To accomplish this, we focused on four questions: 1) What specific outcomes indicate that PLNU has successfully educated its students?  2) What items would predict student success in these areas?  3) How can PLNU measure those particular items? and 4) What are the results of those measures?

Because the work on EEI's was strongly tied to general education (GE), we worked with the faculty as a whole and with the General Education Task Force in particular. In addition, since EEI's need to be tested, we also worked in conjunction with the University’s Assessment Committee.  The involvement in this work of the faculty in the initial stages and of both these groups subsequently played an important role in the general institutional engagement in this work (see Description and Summary of Faculty Participation).

Selection of Critical Student Outcomes.  During the fall faculty convocation in August of 2005, we created a list of the learning outcomes that all students receiving a PLNU degree ideally should demonstrate.  We organized the learning outcomes thematically and presented them to faculty for critique (see General Learning Outcomes August 17, 2004).

For assessment purposes, we made a distinction between goals (what faculty can reasonable control) and desires (what is beyond faculty control) (General Learning Outcomes September 22, 2004).  After examining the statements of comparator institutions, we determined that the outcomes selected by PLNU faculty were very similar to those adhered to by a majority of institutions.  These outcomes also were central to the thinking found in most of the current literature consulted about general university-level education (Research Tools).

Discussion of Educational Effectiveness.  We understood effectiveness indicators to be specific, measurable facts whose presence is a solid predictor of success in a particular area that might have many more aspects than just the effectiveness indicator itself.  We thus avoided focusing on coverage of all the detailed areas of general education and instead worked to identify specific, measurable items that are excellent predictors of success in the knowledge, competencies, and attitudes that the faculty identified as expectations of all our graduates.

This approach aligned our work well with the suggestions present in the Association of American Colleges & Universities (AAC&U) document, Our Students’ Best Work:  A Framework for Accountability Worthy of Our Mission (2004).  We wanted to avoid the easy solution of common tests currently available that do not assess students’ highest skill level.  We were interested in critical thinking, evidence-based reasoning, integrative thinking, and problem-solving skills.  Finally, we also wanted to examine what knowledge, skills, and attitudes the students had appropriated as their own. We recognize that there is a vast difference between what one knows immediately after a course and what one has actually absorbed and integrated into one’s own set of skills, one’s compilation of knowledge, and one’s personal attitudes.  We have tried to assess these integrated elements, as demonstrated in the following EEI’s.

This indicator relates to the ability to use and manipulate knowledge, to connect it to new contexts, and to integrate it with other knowledge.  Rather than test a series of biblical or theological facts, we chose to focus on the student’s ability to recognize theological or biblical concepts when these concepts were present in a very different context than the one in which the student originally learned them.  We believe that students can only engage in this level of thought with concepts they have appropriated as their own.  We also felt that this measured higher intellectual skills.

Given the fact that PLNU is a faith-based institution, we felt it was important that students graduating from the University be able to articulate their own personal belief system and understand how that position related to a spectrum of religious beliefs, especially within Christianity.  We are concerned that students understand what their personal spiritual choices are and that they have reasons for making those choices.

We rely on the discipline-specific summative assessment measures to determine whether students have developed competence in a major field. The task of measuring students’ quantitative skills has been substantially facilitated by the nature of the GE problem-solving requirement.  That course is less concerned with arithmetical manipulations than with the ability to apply quantitative methods to solve real-life problems.

We identified three good predictors of critical thinking.  First, evidence of writing skills that are needed to construct a well-reasoned argument—the ability to formulate sub-points in support of a position and the ability to perceive logical objections to one’s own position; second, the ability to apply quantitative thinking to the solution of real-life problems; and third, the ability to interrogate an issue from different disciplinary perspectives.  We also felt that this measure was a good indicator of general knowledge of different disciplines.

The major attitudes that we wish to examine in our assessment process, a basic social literacy requirement, are a willingness to work with others as part of a team, an attitude of service to others, and openness to diversity.  Although we understand that these items are difficult to measure with accuracy, we also believe that these goals are critical enough that it is important the try to assess our success in achieving them.  In the case of an attitude of service and the ability to be part of a team, we felt that repeated exposure to opportunities to exercise these skills would be a good indicator of a positive attitude.  We are aware, of course, that exposure can yield negative results, but we also believe that a poor experience is more likely to have a negative effect if there are few other experiences with which to balance it.  In the case of diversity, we felt that good predictors of openness to others who are radically different from us are the ability to see things from the other’s perspective and the comprehension that the culture we take for granted can be very foreign to others.

Finally, as an institution that values the education of whole individuals, we want to determine whether our students are capable of perceiving themselves as whole persons and are committed to working to lead balanced lives.

Discussion of Assessment Means for EEI’s.  After constructing the EEI’s, we turned them over to the General Education Task Force and the Assessment Committee.  The current plan, approved by both groups, is to test general education via three different home-constructed exams, all of which have been written and given to graduating seniors, the first in April 2004, the second in April 2005, and the most recent in April 2006. (see EEI Table for test results)

The first exam was a holistic writing exam in which students defended a position in a well-reasoned, evidence-based argument.  The exam also asks the student to indicate what might be a Christian response to the particular issue.  We included this aspect to determine how individualized or how “borrowed” a student’s belief system might be.  A group of 43 senior volunteers took this 60-minute exam in April 2004, and a team of faculty members graded the exams in early June, using a grading rubric and the two-person consensus method that is a standard practice among composition faculty.  The results of that exam were less than hoped for, though not unforeseen. Although our students can formulate a clear thesis about a significant topic, they are much weaker in the tasks of identifying basic sub-points needed to prove an argument, of bringing pertinent evidence to bear, and of dealing with a logical objection to their argument.  The General Education Task Force does not believe that the problem is the Freshman Composition class, but rather the quality and quantity of writing that students are asked to do during the balance of their education at PLNU.  In the current process of revising the GE program, we will include measures to ensure that formative writing continues past the freshman year.

The second instrument was a multi-subject exam administered in April 2005 to a group of 97 students; it consisted of three questions.  One examined students’ comprehension of theological concepts, the second looked at their quantitative reasoning skills, and the third assessed critical thinking and flexibility of perspective.  The Department of Mathematical, Information, and Computer Sciences graded the second question, and we were pleased to see that 76 percent of students demonstrated an ability to use quantitative methods for solving a real-life problem.  With regard to the first question, 84 percent of students identified passages of political rhetoric in which theological concepts were embedded, and 72 percent were able to explain what theological concept was being referenced.  Only 47 percent, however, were able to indicate how the concept was functioning as a vehicle for separate political purposes.  This lower percentage may have been the result of the question’s higher level of sophistication, but instructions for this question were not adequately clear, and we needed a more appropriate grading rubric.  Although the last question has been graded, we are still in the process of analyzing the results.

The third exam, administered in April of 2006 to approximately 250 students, was a critical thinking and attitudinal exam with four questions.  We created one to determine analytical reading skill, one to assess if-then thinking, one to measure the ability to think holistically; and a final question to assess students’ understanding of diversity.

In summary, we have identified predictors of student success in meeting broad educational goals, we have designed ways to measure these predictors, and we have implemented those measures with steadily increasing effectiveness.  Results of this assessment are guiding the revision of our general education.  With that goal in mind, we recommend that PLNU do the following:

See WASC CPR Group 2: Educational Effectiveness Indicators for the full report.

Recommendations:  (For a complete list see Recommendations to the President.)

2.1	Provide more ongoing formative writing experiences for undergraduates, building on the work done in Freshman Composition;
2.2	Inform faculty about steps in writing development and about what kinds of progress they ought to expect of students;
2.3	Provide writing support for all classes that engage in formative writing;
2.4	Encourage faculty in GE courses to make more explicit the meta-cognitive levels of their disciplinary work;
2.5	Inform faculty about overarching themes, common goals, and meta-cognitive skills in GE courses so that faculty in subsequent courses can interact with that previous material;
2.6	Consider a core of eight courses in the pending GE revision that permits programmatic work on development of skills and conversations around themes, referencing past courses or anticipating future courses;
2.7	Study the value of additional features in GE revision, such as a one-unit post-international-experience course, a statistical literacy requirement, and reading skills in either new or existing courses;
2.8	Encourage departments and schools, through the GE Committee, to identify portions of multi-section courses (theme, method, text, experience) that are shared across all sections;
2.9	Create a mechanism for the ongoing assessment and revision of GE; and
2.10	Pursue continued integration of the curricular and co-curricular through the General Education Committee, the First-Year Experience Committee, the Assessment Committee, and through shared programs such as service learning, learning/living communities, and the Preface Reading Program associated with New Student Orientation (NSO).

Reflective Essay Three:  Standards One and Three

Introduction.  The Institutional Proposal calls for the University to select key Criteria for Review (CFR) and to reframe them as questions for investigation.  The goal is to open deliberate discussion of the Standards, gather evidence about them, and analyze the institution’s case that it meets the Standards.  Our investigation yielded specific concerns and pointed out observable gaps in some areas.  We communicated the resulting recommendations for improvement to the President for action.  WASC Standards One and Three are the focus of Reflective Essay Three, considering CFR’s:  1.2, 1.4, 1.6, and 3.2, 3.6, 3.7, and 3.8.

STANDARD ONE: Defining Institutional Purposes and Ensuring Educational Objectives

CFR 1.2 - Assessment of Educational Objectives

In 2000, the University established an Assessment Committee.  This group verifies that educational objectives and learning outcomes across the institution, documents assessment efforts, and examines their connection to Mission and Core Values.  Each academic department and school has implemented an assessment plan.  Most areas utilize a combination of assessment tools, frequently a portfolio of student work, an exit exam, or an alumni survey.  PLNU has established mechanisms for the assessment of general education and co-curricular objectives as well (see Reflective Essay Six on Assessment).  These areas, as other non-academic units, report to the Assessment Committee.

CFR 1.4 - Academic Freedom

The Faculty Handbook states clearly the University’s affirmation of academic freedom (Section IV.A).  In a faculty survey administered in the spring of 2005, a majority of faculty members (68.3 percent) agreed or strongly agreed that they have “the freedom to pursue [their] chosen field of scholarship even when it is potentially controversial.”  Only 1.3 percent of faculty who responded to this question disagreed.  Some departments and schools have used discretion in anticipation of potential problems, even though they cannot point to any specific incidents or action limiting academic freedom.  Although the vast majority of faculty at the University feel that academic freedom is strongly supported, ironically most were unaware of PLNU’s formal policy in the Faculty Handbook.  A few faculty members, upon reading the policy, were unsure that it would be adequate if academic freedom actually were threatened at PLNU.  However, a strong majority of the faculty were comfortable with the Faculty Handbook statement.

CFR 1.6 – Institutional Autonomy

It is common for church-related universities to face the challenge of balancing desired spiritual instruction and a full examination of issues, including those not embraced by the sponsoring religious body.  This balance depends on, in part, the presence of a Board of Trustees that both guards the University’s ties to the denomination and yet also allows autonomy to the administration and faculty.  While the denominational affiliations of the faculty are more diverse, all Board members, with two exceptions, must be members of the Church of the Nazarene, and they are elected to their positions in local district assemblies.  This process has been subject to continued analysis and discussion by administration and the Board, even though we know of no incidents in which the sponsoring denomination or outside agencies have interfered in substantive decisions or educational functions.  A number of social events have been offered in recent years to bring faculty and trustees together, including dinner, receptions, and a question-and-answer session that included testimonials from faculty involved in model projects.

STANDARD THREE: Developing and Applying Resources and Organizational Structures to Ensure Sustainability

CFR 3.2 - Faculty Commitment and Qualifications

Faculty commitment to the institution can be indirectly measured by the length of service, the number of faculty with tenure, and the ratio of full-time to part-time faculty. Currently, the mean length of faculty service at PLNU is eight years.  As a norm, most faculty members will have qualified for tenure at the end of eight years and will either be ranked as professors or within one or two years of promotion.  In 1996, the WASC report noted that many faculty members were uncertain and unclear about the promotion and tenure process.  Since then, the University has further standardized these processes and has offered several informational sessions for interested faculty.  As a result, most younger faculty apply for promotion and tenure as a matter of course, and many longer-term faculty who had previously found this process “too much trouble” have now received tenure.  As of the 2005-2006 academic year, 37 percent of faculty are tenured.  In 2002-2003, 75 percent of the undergraduate schools and departments had at least 70 percent of their faculty on full-time contracts (see Key Performance Indicators).  Within five years, the University hopes to have all schools and departments with at least 70 percent of faculty with full-time status.  As recommended in the 1996 WASC report, we produced an Adjunct Faculty Handbook, distributed it widely, and have begun providing orientations for recently hired adjunct faculty.

PLNU has continued to recruit a strong faculty well qualified to pursue both teaching and scholarship, though the high cost of living in the San Diego has created real challenges for faculty housing .  In 2005-2006, 79 percent of full-time faculty held terminal degrees in their field (see Key Performance Indicators).  For all faculty members, the University offers several programs to promote professional and pedagogical development, including seminars and discussion groups.  PLNU also annually provides professional development funds for each faculty member (see Faculty Handbook IV. R).. Capturing the growth of faculty scholarship and research would be greatly helped if the University were to adopt an electronic portfolio of their accomplishments.   The Center for Teaching and Learning has sponsored several programs that help faculty learn, observe, and discuss new pedagogical techniques.  Each faculty member is also required to administer student evaluations in three courses each year.  Faculty also must regularly have their teaching evaluated by a peer (see Faculty Handbook IV. Q).  Since the 1996 WASC report, the University has seen continued growth in academic programs and enrollment.  The Cabinet has approved additional faculty and staff positions to support this growth.  Staff satisfaction in all categories is higher than it was at the last WASC report, and morale is generally high (80 percent agree).

CFR 3.6 - Information Resources

The responsibility for assessing the continuing adequacy of information resources at the University and planning for the future resides with the director of Ryan Library and Learning Services.  Currently, the director, professional librarians, and staff are working with the Vice Provost for Academic Administration on a self-study that addresses these issues.  Of particular importance is the goal of renewing these resources with the possibility of making alterations in their size and character (see Ryan Library for information about available resources).  During AY 2005-2006, we conducted a thorough self-evaluation through the LibQual+ software, held regular meetings on a phased renovation of facilities, and made plans for the summer 2006 visit of a consultant-evaluator from an institution of our type.

Collection development at Ryan Library is a cooperative process involving both library and school/academic department faculty.  Although the library offers an extensive electronic database, faculty and professional librarians would profit from a second large aggregate database for electronic journal access.  In the narrative portion of a spring 2005 survey, faculty respondents supported other improvements such as the renovation of the library building, the addition of more specialized resources such as digitized music listening library and discipline-specific electronic journals, and the building of on-site library collections at the Mission Valley and Arcadia campuses.

New program proposals on the main campus are expected to address the additional information resources needed to support these programs.  At the undergraduate level, this assessment must occur before the program can be approved by the Academic Policies Committee (APC).  Recently, the template for proposed changes in the curriculum added the requirement of indicating the impact on library holdings and services.  This is an important quality improvement process step.  Currently, the library faculty and staff regularly travel to Arcadia, Bakersfield, and Mission Valley campuses, and to district teaching sites to discuss library resources and student needs.  As off-campus sites have continued to grow and as technological advances have occurred, the Cabinet approved for the 2006-2007 academic year the addition of an Electronic Services Librarian position.

PHINEAS, Ryan Library’s catalogue, is accessible via Internet to all students and faculty at any location.  In order to check out materials from the main library, students and faculty at other locations must either visit Ryan Library or request that library books be sent to them.  All electronic databases are available to faculty and students (both undergraduate and graduate).  Ryan Library subscribes to an e-book collection, and students and faculty have InterLibrary Loan service, though this service is not available in Bakersfield, an area that currently lacks a large university library facility.

CFR 3.7 - Information Technology Resources

Perhaps no area of the University has undergone as many improvements in the past ten years as information technology resources.  New developments in technology have produced improved processes in order to integrate this investment into the changing environment of teaching and learning.  The University supports the use of technology to assist in fulfilling its educational purposes through computer labs, technological improvements in the classroom, residence hall networking, and both academic and administrative services and training.

There are nine computer labs (Macintosh and PC work stations) on the Point Loma main campus that are available both for formal instruction and for independent student work.  The Arcadia and Mission Valley campuses each have a computer lab with both Macintosh and PC work stations.  In Bakersfield, students have access to a nearby lab.

All classrooms have network connections.  Point Loma, Mission Valley, Arcadia, and Bakersfield also have both wired and wireless network connectivity throughout the facilities.  All classrooms on the main Point Loma campus also have access to the University-owned and operated closed-circuit television (CCTV) system, and all have permanent monitors installed.  In addition to the PLNU Information Channel, PLNU manages its own cable television channel over which educational and /or student-driven content can be presented.  Some classrooms and labs have Smart Boards, Smart Sympodiums, document cameras, High Definition LCD and Plasma screens, audio equipment, and Internet Protocol cameras with recording capabilities.  In residence halls, all rooms are connected to the campus network (see Table 5.1).

Full-time faculty members have laptops and some adjuncts as well; most adjuncts have access to shared desktops.  The University has installed a wireless grid at all Point Loma facilities for both student and staff/faculty use.  All faculty members have access to Blackboard, an on-line Course Management System.  In order to sustain support for faculty, a Director of Academic Computing serves in assisting faculty with computing needs.  In the fall of 2005, Information Technology Services (ITS) completed a document entitled “Model for Supporting the Technology Needs of Faculty and Students in a Multi-Site Environment.”  In the spring of 2006, the ITS department hired an Instructional Technologist to work with the director in making recommendations to faculty related to methods of using technology for enhancing teaching and learning as well as for promoting technological literacy.

Specific training in the use of technology is also an important concern for the University.  The Center for Teaching and Learning provides some training for using computer technology, and ITS has an extensive menu of year-round computer classes for faculty and staff.  ITS employs a Technology Training Coordinator, and this person makes periodic visits to the other campuses for training customized to meet their needs.  Element K is available upon request to all faculty and staff for access to e-learning courses in a wide variety of applications.

3.8 – Organizational structures and decision processes

The University has a clearly defined and illustrated organizational structure (see PLNU Web site).  Organizational structure has grown and changed over the past 100 years, and the University is studying ways to support its changing needs, particularly at the graduate level.

The University’s governance and decision making process is consistent with and functions in accordance to the organizational structure (see Faculty Handbook II).  Committees and task force groups bring issues, concerns, and policies to the forefront.  Membership on faculty committees is assigned by the Nominating Committee, and task force groups are developed by the members of the Administrative Cabinet.  Academic decision making is generally made on four levels:  1) Administrative Cabinet (budget), 2) Provost’s Council, 3) Academic Council (vice provosts, department chairs, school deans, and site directors), and 4) general faculty.  It may also be profitable for faculty to expand interaction with the offices of Student Development and Spiritual Development in order to ensure the institution’s purpose of meeting the needs of the whole student—academically, developmentally, and spiritually.  Some committees have begun to utilize teleconferencing as a way to include more full-time faculty from Bakersfield and Arcadia campuses in discussions and decision making.

See WASC CPR Group 3: Standards 1 and 3 for the full report.

Recommendations:  (For a complete list see Recommendations to the President.)

3.1	Initiate a University-wide discussion on academic freedom in order to foster a clearer understanding of the official policy in the Faculty Handbook;
3.2	Continue to study the composition of the Board of Trustees;
3.3	Include the School of Theology and Christian Ministry in future strategies vis-à-vis the sponsoring denominational church;
3.4	Increase opportunities for members of the Board of Trustees to meet faculty on a personal level;
3.5	Continue development of creative solutions to recruit and retain faculty members in light of the high cost of living in the San Diego area;
3.6	Document faculty accomplishments annually by electronic means;
3.7	Develop library resources addressing specific needs of disciplines and departments and schools, such as a digitized music listening library, discipline-specific electronic journals, and on-site library collections at other campuses;
3.8	Give increased attention to the impact of proposals relative to the need for information and technology resources, especially at the graduate level;
3.9	Expand interaction and input of faculty with Student Development and Spiritual Development; and
3.10	Increase the use of teleconferencing to allow faculty at other PLNU campuses to become involved in committees and task force groups.

Reflective Essay Four:  Standards Two and Four

Introduction.  In addressing WASC Standards Two and Four, the Institutional Proposal calls for a review of selected CFR’s that we consider to be key and not addressed in other portions of the CPR report.  We reframed these CFR’s into questions and looked at the Standards, gathering evidence and analyzing them vis-à-vis the institution.  In this essay, we have cited any concerns and made specific recommendations for improvement to the President for action.  WASC Standards Two and Four key CFR’s include 2.2, 2.4, 2.9, 2.14, and 4.4, 4.5, and 4.7.

STANDARD TWO: Achieving Educational Objectives through Core Functions

CFR 2.2 – Graduate Programs and Curricula

A survey of 281 graduate and undergraduate faculty members was very encouraging overall, but provided a variety of opportunities for further discussion, examination, and program improvement.  The responses revealed a strong fit between graduate programs and institutional Mission, but weaker commitment by undergraduate faculty and by the institution to support the long-term success of these programs.  The lack of support from the institution might be explained in part due to the fact that many of the graduate programs are new, and it takes time to build the infrastructure and graduate culture.  Undergraduate faculty expressed concern about the perception of academic rigor in graduate-level scholarship and noted the need for improved access to academic resources to support graduate-level work.  We summarize the results of our survey in the following paragraphs, including the questions of Mission fit, preventing institutional drift, long-term commitment of faculty teaching in graduate areas, and the currency of student research.

Mission Fit.  Graduate faculty overwhelmingly feel that the institutional Mission, “To Teach, To Shape, To Send,” is integrated into the classroom through personal commitment, curricular design, pedagogy, and resources.  Commitment to Mission is high among adjuncts as well (Report 4 Table 3), strengthened, no doubt, by a fall 2005 new adjunct faculty orientation session centered on the Mission.  A strong majority of graduate faculty and a somewhat smaller majority of graduate students feel that their classes stimulate moral thinking and critical reflection on faith and learning (Report 4 Table 6).  Students even felt that the University’s Mission was evident in the admissions process.  However, less than half of full-time and adjunct faculty thinks that the Mission is integrated with course goals and classroom assessment (Report 4 Table 4), and we would like to see a stronger awareness of the Mission in the initial contacts with admissions as well as a smaller discrepancy between faculty and student perceptions about graduate courses.

Preventing Institutional Drift.  Faculty integration of the Mission into key areas of course and program development as well as administrative oversight of all graduate programs help prevent institutional drift away from our Mission and Core Values.  A solid majority of the graduate faculty report that PLNU’s Mission influences syllabi, curriculum, and teaching methodology (Report 4 Table 10) as well as classroom assessment and instructional standards (Report 4 Table 11).  The Graduate Studies Committee (GSC) reviews new and on-going programs to ensure they support the Mission and the University’s purpose and character.  However, the integrity of the review system depends on the determination of each academic department or school in maintaining and continuously assessing commitment to the Mission and goals of the institution.  In addition, the faculty is not strongly convinced that adequate systems are in place for the assessment of graduate faculty fit with the Mission (Report 4 Tables 12 and 13).

Long-term Commitment.   In order to be healthy, graduate programs also need commitments from the institution itself, as reflected in support services, resources, and support for professional training and development.  Although a relatively equal majority of graduate and undergraduate faculty feel that the University provides adequate support for professional training, conferences, and continuing education (Report 4 Table 14), fewer graduate faculty feel that the University provides adequate support for professional development (Report 4 Table 15).  Although full-time graduate faculty have professional development resources identical to undergraduate faculty, many graduate faculty are adjuncts and are not eligible for internal grants or for professional development funds.

The perception among graduate faculty that institutional support is weak may also reflect the awareness that some undergraduate faculty regard the current growth of graduate programs with concern (Report 4 Tables 16 and 17).  As undergraduate faculty better understand the role of graduate programs and that growth in these programs will not drain resources from undergraduate programs, undergraduate faculty support of graduate-level growth should increase.   The fact that more graduate students and faculty see PLNU as providing continuous education and life-long learning opportunities than undergraduate faculty alone (Report 4 Tables 18, 19 and 20) again suggests that undergraduate faculty have not yet fully grasped the goals of graduate education.  It also indicates the need for a continued conversation about the goals of graduate education and its place in the PLNU Mission.  Meanwhile, PLNU is working to create supportive links between the main, undergraduate campus and graduate units at the various locations.  Recent visits to Mission Valley, Arcadia, and Bakersfield campuses by individuals involved in faculty development and in academic planning have been positive experiences for graduate faculty.

Although a majority of graduate students and faculty believe that graduate students have state-of-the-art resources (library, media, information technology, etc.), in both cases the majority is too small and suggests the need for greater capitalization of graduate programs (Report 4 Tables 21 and 22).  Concerted efforts being made to provide better paper and electronic resources need to continue and should be monitored carefully.  Graduate students may also need better orientation to these services so that they know what resources are available.

Graduate students were pleased with the support level from frequently used services such as Academic Advising, Admissions, and Student Financial Services, although there is still some room to improve.  Graduate students’ inability to rate satisfaction levels for support provided by the Provost’s Office or by Conference Services (who maintains and schedules the Mission Valley campus) probably indicates a lack of understanding or knowledge rather than a negative judgment.

Currency and Student Research.  We believe that each graduate program should have internal standards of excellence that can be monitored and maintained, including a research component, faculty professional development, and access to leading-edge research and/or professional publications.  Maintaining standards of academic excellence cannot be jeopardized in order to meet market demand.  A slight majority of graduate students believe they have state-of-the-art library, information technology, and media resources (Report 4 Table 24) or that they have the leading-edge journals for their research (Report 4 Table 25).  Many graduate students believe that leading academic and professional literature is used by graduate students for research (Report 4 Table 26).

Graduate students perceive their project/thesis work as encouraging academic research and high-level professional experience (Report 4 Tables 27 and 28).  Fewer students, however, feel that they have “easy access to leading-edge curricula, textbooks, journals, and conference proceedings” (Report 4 Table 25) or that they have “sufficient academic resources and preparation for projects and theses” (Report 4 Table 29).  Graduate student perception seems to be consistent with the views of adjunct faculty, full-time faculty, and undergraduate faculty (Report 4 Tables 30, 31, 32 and 33) and points to an important issue that we may need address in graduate programs (Report 4 Tables 34, 35, 36 and 37).

CFR 2.4 – Retention and Graduation Rates

In comparison with similar institutions (by Carnegie classification or in size and Christian mission), PLNU’s graduation rate compares favorably to similar institutions and has improved dramatically over time, from 44.9 percent in the 1993 cohort to 68 percent in the 1997 cohort (Report 4 Table 38).  The freshman retention rate for the University has been low but recently shows a steady rate of improvement (Report 4 Table 39), and we have taken steps to bolster this change.  We instituted the First-Year Experience Committee which has begun to implement retention-effective ideas from national First-Year research.  For example, the Committee began a freshman summer reading program in the fall of 2005, and is continuing this program with further follow-up activities during the academic year.  We designed the project both to help prepare entering freshmen mentally in the transition to college and to give them immediate personal contact with a faculty member.  Additionally, the recently created position of Associate Vice President for Student Development focuses on Student Engagement and Retention through intentional programming related to these issues.  Further, the Student Care Group, staffed by Student Development professionals, responds to struggling students who are at-risk for attrition.  Finally, improvements in NSO have helped the perception of incoming students concerning the type of institution PLNU is, as well as helping students get acquainted with each other and with faculty.  At this point, however, it is difficult to obtain consistent data collection from all levels of instruction due to the inherent inconsistencies among academic departments and schools and graduate-level programs at PLNU.  Steps are being taken to improve data collection on retention and graduation rates at the graduate level (see Reflective Essay on Data Portfolio).

CFR 2.9 – Linking scholarship, teaching, student learning, and service

Following Ernest Boyer’s expanded definition of scholarship, we view as scholarship many significant activities that have a strategic impact on student learning.  These activities include classical research (including the creative work of faculty in the literary, visual, and performing arts), integration (of scholarly work across disciplines), applied scholarship or outreach, teaching (curriculum development as well as instructional and classroom research), and service to the institution or to the profession (citizenship) (see Report 4 Table 40).

Relative to research and creative activities, full-time faculty engage at major to moderate levels primarily in classical research and secondarily in collaborative integration projects.  As might be expected, graduate faculty, many of whom are in the content area of education, mentor more students in research or in-depth study than faculty who teach only undergraduate students.  They also use research-based innovations in pedagogy more than those who teach undergraduates (Report 4 Tables 40, 41 and 42).

Almost all faculty report setting challenges and are clear about how students can meet course expectations.  Eighty percent of the faculty report placing an explicit emphasis on insightful and creative student work (Report 4 Table 41).  However, faculty do not always engage students in active and collaborative learning projects (Report 4 Table 42).  Faculty do, nonetheless, attempt explicitly to integrate and apply their learning beyond the classroom (Report 4 Table 44).  Equally important, almost all faculty report trying to get to know students as individuals in the classroom, and an even higher percentage of students believe that faculty want to get to know them and support their learning (Report 4 Table 43).

CFR 2.14 – Transfer Student Support

Based on transfer student transcript evaluations through the admissions process and again more thoroughly in the offices of Academic Advising and the Vice Provost for Academic Administration, each transfer student receives an initial course schedule.  Also, during NSO, we offer a session designed especially for transfer students in their transition.

An on-going evaluation of student transcripts each year provides necessary information about student success.  Two-thirds of transfer students report that all their academic needs are being met.  For students in academic difficulty, an early alert system provides information from faculty members on their progress.  This system depends, however, on faculty compliance, and this varies by individual.  Additionally, the Academic Support Center offers on-going tutorial services and academic advising.  Two-thirds of students state that their academic support needs have been well assessed by the University.

The Office of Records and the Office of Institutional Research track retention trends of all students, including transfer students, and conduct on-going evaluation of transcripts to ensure that students are making progress toward meeting graduation requirements.

STANDARD FOUR: Creating an Organization Committed to Learning and Improvement

CFR 4.4 – Quality Assurance Review

Although most programs have some type of internal review process, not all have one that features an outside review (Report 4 Table 45).  Both internal and external review is necessary in order to maintain and improve programs, and we are offering two recommendations to the President in this regard.  This CFR is the focus of two reflective essays in the Educational Effectiveness (EE) review that deal with quality assurance, one on Program Review and another on New Program Development.

CFR 4.5 – Institutional Research

The Office of Institutional Research opened in 1984 and since that time has served to satisfy the reporting requirements of various agencies and organizations, respond to faculty and administrative needs, and conduct routine surveys.  Currently the office is under the supervision of the Vice Provost for Academic Administration, who works closely with the Director of Institutional Research.  In response to the overwhelming need for data, the office recently requested in the 2006-2007 budget cycle the addition of a research assistant; the request was approved.  As a consequence of the work of preparing the CPR report, we have made recommendations relative to the effective functioning of this office, especially as it assumes the important tasks of maintaining the University Data Tables and participating in the efforts of the Strategic Planning and Assessment committees.  (See the Reflective Essay on Data Portfolio.)

CFR 4.7 – Assessing Teaching and Learning

The University attempts to identify effective approaches to assessing teaching and learning.  PLNU offers its full-time and part-time instructors, at both graduate and undergraduate levels, the possibility of using either of two student evaluation systems. The two systems, SIR II and IDEA, are two of the three available nationally-normed and thoroughly tested course evaluation systems.  Faculty have a choice so that they can select an instrument that best matches the type of classes they teach and the sorts of objectives they have established.

The current student evaluation system has been in place at PLNU for at least 20 years.  Faculty surveyed considered the evaluations adequate and understood as tools for the improvement of teaching (Report 4 Table 46).  However, it is a concern that so few (17.1 percent) of full-time faculty feel strongly that academic leadership identifies and explains available assessment options.  PLNU provides all new faculty with a semester-long New Faculty Seminar designed to give orientation to a variety of tasks that they will need to assume, including the task of teaching well and evaluating teaching using one of these instruments.  During this time, new faculty learn how to use the teaching evaluation tool and how to interpret it.  Additionally, we offered an orientation session for first-time adjunct faculty in 2005, and we recently revised the Adjunct Faculty Handbook in order to provide better information on issues such as the evaluation of teaching.  However, it is still necessary to take steps to show faculty how the available national instruments can be interpreted so that student evaluations can be used for formative purposes.

Much of the support for teaching and learning is available through the Center for Teaching and Learning and through several other faculty-run centers, especially the Margaret Stevenson Center for Women’s Studies, the Center for Justice and Reconciliation, and the Wesleyan Center for 21st Century Studies.  We designed programs in the Center for Teaching and Learning after examining over 100 Teaching and Learning sites in the U.S. and after a wide discussion with faculty leaders (primarily department chairs and school deans).

Every full-time professor at the University is on a schedule of teaching assessment that involves a full profile for the first two years followed by a year of student evaluations only. Thereafter the faculty member does one year of a full evaluation cycle followed by two years of student evaluations only.  This continues until a tenure decision has been made.  Subsequently, the faculty member continues on a schedule that involves one year of a full evaluation profile followed by three years with student evaluations only.  A full profile includes a self/chair evaluation form and a peer evaluation form.

Assessing student learning is of concern to every academic department and school at the University and is monitored by the Assessment Committee.  In addition to bringing a team of experts to the University in 2001 to train faculty in doing student outcomes-based assessment (see the Reflective Essay on Assessment), PLNU has supported the respective administrators and a number of faculty in attending conferences on the assessment of learning.  The University as a whole is now on an annual schedule of assessment, with academic departments and schools on a five-year cycle of program review.

Although PLNU strongly supports faculty efforts at excellent teaching, there is much that we still hope to accomplish.  In particular, we want to make the current faculty evaluation system more geared toward formative evaluation and less focused on purely summative measures.  We hope to make the self/chair form and the peer evaluation process more formative.  The director of the Center for Teaching and Learning has examined several other models such as student-based formative evaluations, teaching portfolios, and more interactive mentoring relationships.  The Faculty Resources Committee will be examining options and working to bring a proposal to the faculty for their approval.

See WASC CPR Group 4: Standards 2 and 4 for the full report.

Recommendations:  (For a complete list see Recommendations to the President.)

4.1	Create more formal policies and procedures to protect the University as it grows through off-main campus programs, including the professional development of graduate faculty, careful review of course syllabi, and examination of department/school expectations for support of the Mission of the University;
4.2	Continue the orientation and preparation of adjunct faculty with sessions at the beginning of each academic year;
4.3	Inquire further into the weaker item response of graduate faculty as compared to undergraduate faculty regarding the institution’s commitment to long-term success in professional development;
4.4	Inquire further as to why there may be an information gap between adjunct and full-time faculty, as well as graduate and undergraduate faculty, regarding commitment to graduate programs;
4.5	Take steps to foster an institutional commitment to support the growth of graduate education and professional development;
4.6	Explore a policy and procedural structure designed to prevent institutional drift as the institution continues to grow off the main San Diego campus;
4.7	Improve the level of access to academic resources for graduate students with the goal of improving the level of academic rigor in research and scholarly activity;
4.8	Improve the orientation of graduate students to University support services;
4.9	Utilize benchmarks to measure standards of excellence such as professional accreditation, national exams, publications, and significant research, in order to gage improvement and growth in each field; and
4.10	Improve academic support in the following ways:  1) document the use of literature in the field of study and its availability to graduate students and faculty for their research agenda, 2) assure the commitment of library and media services in developing graduate-level facilities and resources, 3) review curricula, faculty involvement, academic resources and technology for currency in scholarly literature, and 4) conduct a “best practices” survey of graduate curricula, textbooks, professional/academic journals, and conference proceedings available to faculty and students at comparable graduate institutions.

Reflective Essay Five:  Planning

Introduction.  In addition to responding to framing questions referenced to key CFR’s with specific recommendations for action, the Institutional Proposal calls for a review of concerns expressed in the WASC team visit of 1996.  While response to recommendations from a previous team visit is a regular feature of the CPR report (see Response to the 1996 WASC Visit), we judged their concerns to be of such importance that we decided to analyze them more thoroughly.  One of these concerns was planning, and particularly, academic planning.

While the WASC team commended the University in the 1996 report for its commitment to the necessity and value of planning (Ralph Wolff Letter), it also indicated a desire for documented outcomes of a planning process and evidence that these outcomes inform the process in a systematic way.  Specific issues raised by the team’s report included the need for a systematic and comprehensive planning effort of a highly participatory nature.  In addition, our efforts needed to value the analysis of institutional data that contributes to planning and decision making and to include an academic plan that considers the broad scope of academics, as well as systematic documentation.

To address the concern for planning, we focused on documenting past planning efforts and on describing how the concerns expressed by WASC in 1996 indeed have been addressed by the University in the intervening ten years.  Finally, we wrote a description and an evaluation of the current planning cycle (2004-2006), providing specific recommendations for improving this process in the future.

Past Planning Efforts.  The planning efforts of the University over the past several years have worked to clarify the appropriate balance between responsiveness and precision in an institution of our size and character.  We have refined its process to be one that is thorough, participatory, data-driven, evidence-based, flexible, and practical.  In an attempt to move toward a more effective planning process, we analyzed the work of each successive planning group during this period of time and summarized the lessons learned.  A detailed description of the history of PLNU’s planning efforts can be found in the Planning History. 

Planning and the Office of Institutional Research (1984-present)
For many years, the institutional research function of the University was part of the work of the Records Office.  An Office of Institutional Research was created in 1984 to carry out tasks such as the preparation of statistical reports (denominational, controlling board, federal, faculty-related, etc.), oversight of survey processes, and maintenance of limited data on the Web.  Clearly, the University understands the need for an office that is responsible for data collection and reporting, but also one that is capable of conducting research for planning purposes rather than simply responding to reporting needs.  Likewise, such an office needs to make use of the latest technology to build a central data set for the University and take its essential supporting role in planning (see the WASC CPR Group 1: Data Portfolio Report).

Recent Planning Models (1990-2002)
Since 1990, the University has moved through a series of planning models.  These models have yielded results in terms of actual planning as well as new understandings about what model might work best in the institution’s culture.  These models included a Strategic Planning Group (SPG) with a director of planning, a Planning and Budgeting Committee (PBC), and an independent Planning Officer linked to the President’s Office.  During the institution’s Centenary celebration in 2002, a Century II Commission gathered information for planning efforts, but it did not function as a formal planning model.

These approaches to planning made it clear to University leadership that, whatever the label, there are certain characteristics that help to improve planning processes.  For example, we found that planning must emanate from a broadly embraced University Mission while involving University-wide input with both implementation steps and individuals who are accountable for their execution.  We learned as well that planning must be centralized and involve key decision makers.  Likewise, planning is best achieved when the process is embedded in routine functions of the University and directly linked to the process of setting budget priorities.

Strategic Planning Committee (2002-present)
In 2002, the University returned to a committee model for planning, but one with the involvement of the President and the Cabinet (see the Strategic Planning Committee Composition).  The SPC created the 2002-04 Principles, Assumptions and Goals for planning.  The Committee then began to work on developing an updated system for planning.  The 2002-2004 goals informed the Cabinet unit goals and plans (see the Cabinet Area Reports).  For approximately 18 months, the SPC focused on rewriting the Mission and creating a Vision statement along with specific goals arising out of them.

Through a process of Board of Trustees review and University community engagement, we finalized new Mission and Vision statements in 2004, created a set of 2004-2006 goals (see 2004-2006 Mission, Vision and Goals), and named goal managers.   The goal teams developed action plans and carried them out (see the Strategic Goal Reports).  We also tracked progress on KPI’s (see Key Performance Indicators).  The SPC completed an analysis of the existing process, made modifications, and now is positioned to carry out the new process in 2006-2009.

Having settled into the current planning model, we realize that it was helpful to have deep involvement of the Board of Trustees in reviewing and affirming the University Mission.  We know that planning work now being done by each unit of the University needs to be carefully defined so that it does not become too time consuming or cumbersome.  We see, as well, that adequate support needs to be provided for units of the University that request assistance in developing their unit plans, that there need to be clear links between planning and the budgetary process with a system of accountability, that the data used for planning should be linked to Mission and Vision to aid in the setting of goals, and that a multi-year planning and budgeting cycle will allow for greater flexibility in planning.

Concerns Expressed by WASC in 1996.  The University has made significant progress in addressing the concerns relative to planning expressed by WASC, including a systematic and comprehensive planning effort that is highly participatory, analysis of institutional data that informs planning and decision making, and a formal Academic Plan that considers the broad scope of academics, with an improved level of documentation.

The current Strategic Planning Committee includes representatives from each major area of the University.  In addition, the SPC utilizes a broad collection of information and data so that planning is holistic.  We are also emphasizing a “big picture” approach in unit planning.  For example, we ask academic departments and schools to consider technology needs when proposing changes to their major programs.  All new academic program proposals now must discuss the impact on technology, library holdings and functions, facilities, and student support services (records, student financial aid, etc).

As a direct response to WASC concerns, the University has launched a formal Academic Plan.  The Plan discusses institutional academic values and carefully links to PLNU’s Mission and Vision statements.  It focuses on faculty, students, curriculum (general education, academic majors, and graduate programs), support for instruction (pedagogy, information technology, library, and physical resources), the co-curriculum, educational effectiveness and learning outcomes, quality assurance, and academic leadership.  We developed specific initiatives via the work of various WASC study groups and institutional dialog sessions (see the Academic Planning Schedule).  After each conversation, we posted ideas and initiatives (see the Academic Plan Initiatives).  The Provost, in consultation with the Academic Council and the SPC, prioritized these initiatives in the spring of 2006, and we incorporated them into the strategic planning process.  Finally, we presented the final list of initiatives to the entire faculty in the spring of 2006.

We structured the SPC to include members from a wide variety of areas in our academic community.  In addition, the Committee makes use of focus groups, forums, and scheduled conversations (either in person or electronically) in order to gather input.   We used information assembled from the larger University community in the development of the Vision and the goals for 2004-2006.  We used a similar system of broad institutional community participation in the spring of 2006 to obtain information to aid in the setting of the 2006-2009 goals.  The SPC is developing an e-mail and a Web-based system of providing the entire PLNU community with regular updates on the planning process and inviting comment.

With regard to data and its use, the current planning process has used KPI’s (institutionally developed benchmarks), consolidated financial reports, survey data, institutional research data (such as the information assembled for the University Data Tables), and topic-specific data gathered for particular decisions (see Cabinet Area Reports and Strategic Goal Reports).  We have modified the next round of the planning process to organize goals and data around the five key themes in our Vision statement:  “A nationally prominent Christian university and a leading Wesleyan voice.”  They are described in these terms:  1) Excellence in Academic Preparation, 2) Wholeness in Personal Development, 3) Faithfulness to Mission, 4) Leading Voice in Higher Education, and 5) Leading Voice in the Church.

We are gathering additional data to build a larger longitudinal core data set to assist in decision making.  This core data set will include items from the University Data Tables, curricular and co-curricular assessment data, and selected information identified by the SPC as critical measures of progress (see the Strategic Planning Data Set).

Current Planning Process (2004-2006 Planning Cycle).  For the most recent cycle of planning (2004-2006), we identified six strategic goals with a goal manager and a team to work on each goal.  We have made progress on five of the six goals, setting aside Goal Five with the intention of incorporating it into the 2006-2009 planning cycle.  Preliminary work on that goal began late in the fall of 2005.

The characteristics of the current planning cycle include a cycle length of two years; the centrality of Mission and Vision; broad community input for building goals; day-to-day management of the process by the President, Cabinet, SPC, and the goal managers; and the linking of annual goal planning to the institutional budgeting cycle.  Progress on the six goals and the 13 KPI’s has been the focus of the strategic efforts, and assessment has been a growing part of the culture of PLNU, especially as a vital part of planning.  Detailed steps in the planning process as it was originally designed (for the 2002-2004 cycle) are located at the end of the Planning History.  The key modifications made in the 2004-2006 round of planning include a reduced emphasis on formal unit plans and a more structured approach for progress on goals and KPI’s.  Goal and KPI reports demonstrate the results of this change (Strategic Goal Reports).

We developed the Academic Plan from the spring of 2005 to the spring of 2006.  The Plan is intended to be a model for unit plans from the larger divisions of the University, those seven areas linked with Cabinet members (for a more detailed see the Cabinet Organizational Chart).  The Plan describes philosophy and practice that will be updated on a longer cycle (coinciding with the re-examination of University Vision).  Associated with the Plan is a list of initiatives that we are incorporating into the strategic planning process.  We will update these initiatives annually to reflect progress that is made, tasks that are completed, and any shifts in priorities that occur in the context of a dynamically changing university.  (For a diagram of the 2004-2006 see the Current Planning Process Diagram.)

Evaluation of the 2004-2006 Planning Cycle.  In 2004-2005 and in the fall of 2005, the SPC conducted an evaluation of the current planning cycle.  During this review, we noticed three key issues that needed to be addressed as the current planning cycle is fine tuned for the next interval beginning in 2006:  1) There is a one-year time lag between the setting of strategic goals and the implementation of the strategic plan, 2) Assessment is happening too early; in particular, assessment for the purpose of setting goals for the next round of planning occurs before we complete the first full year of the existing plan, and 3) We are setting goals for the next round of planning when the current round has more than a year remaining.

A review of the history of planning at PLNU and the many lessons we learned also led us to build a set of core principles of planning, including simplicity, linkage to Vision, the necessity of periodic evaluation, three-year cycling, the importance of data and information, accountability for tasks, budget linkage, involvement of key decision makers, transparency, constituency voice, progress reporting, and assessment and continuance of initiatives.

Looking Ahead to the 2006-2009 Planning Cycle.  There are a few issues that will be resolved as the University progresses through the 2006-2009 planning cycle.  The SPC is committed to answering these questions as it develops the details of each step in the current cycle.  These issues include funding, goal managers, and information flow.  The University has defined the funding streams for strategic initiatives.  The source of the funding depends on the type of initiative, including start-up funds for new programs and annual budget allocations for strategic initiatives.  Start-up funds, held separate from the annual operating budget, can be used for start-up costs for new revenue-generating programs.  We expect that the net income from a new program will repay its start-up costs and grow the fund to keep up with inflation.  Annual budget allocation for strategic initiatives is earmarked to fund initiatives that surface in unit plans that advance the University Vision and that are not part of the strategic goals.

In summary, the University has taken the concerns of the last WASC team visit seriously, developed and reworked various planning processes, written a formal Academic Plan, and developed a consistent, data-driven planning system, greatly strengthening the institution’s ability to plan effectively.

See WASC CPR Group 5: Planning for the full report.

Recommendations:  (For a complete list see Recommendations to the President.)

5.1	Establish a three-year planning cycle;
5.2	Create a University-wide communication strategy to keep the PLNU community informed about the progress in planning;
5.3	Expand the existing planning data set to include a broader range of Key Performance Indicators for the institution; and
5.4	Study and modify the structure of the SPC with the goal of optimizing the communication between planning and budgeting and ensuring direct information access to key University areas.

Reflective Essay Six:  Assessment

Introduction.  In addition to WASC’s concern for planning, the 1996 visiting team report also pointed to the issue of assessment.  The Institutional Proposal calls for the posing of key questions related to the University’s current assessment program, considering its embryonic state at the time of that visit.  In addition, we proposed to analyze the program and to provide insights learned from our review.  While complimentary of PLNU’s initial steps in assessing student learning, the team report and the subsequent letter from the executive director urged continued monitoring of the assessment model.  They called for the application of more energy to the process with the goal of establishing a data-integrated, coherent assessment of institutional effectiveness.  In focusing careful attention on our assessment program and after an in-depth analysis of processes, we can report success in energizing and sustaining a multi-year process whereby the assessment of student learning has become a common project of key units of the University community.

A review of the current University assessment program reveals a good level of understanding and significant compliance across academic programs and co-curricular units, with defined learning objectives and measurement tools, and data being collected and used in thoughtful ways both to improve program quality and to have an impact on student learning.  In addition, major advances have been made in the assessment of broad educational outcomes in general education and the crafting of EEI’s that provide a link between the University’s Core Values, educational outcomes, and measurement tools allowing evidence-based decision making at an institutional level.

Major Steps Taken Since the Last WASC Visit.  Until recently, academic programs at PLNU, as at many American universities, functioned through individual schools and academic departments that designed and structured offerings as they saw best.  They were isolated units, contributing courses to academic major programs and general education.  In this system, the focus was on input (teaching), not output (actual learning).  There were few systematic mechanisms for rethinking programs such as general education, for linking courses to a common concept of general education, or for making outcomes beyond individual course grades a priority.  The rise of the assessment movement in higher education called into question the single focus on input.  At PLNU, we have shifted that focus and now have an assessment program designed to accomplish three purposes:  1) to improve student learning, 2) to help academic and co-curricular units become more aware of this common project and of the value of assessment, and 3) to motivate faculty to align individual courses to the Mission.

Earlier in the last decade, our initial efforts toward reaching these goals involved solely a system of academic Program Review, a five-year rotation of self-evaluations and outside reviewers.  As a result of policies put in place at that time, no school or academic department can submit proposed curricular changes for faculty action without having completed a well-researched, evidence-based review that grounds the curricular recommendation.  To this date, all schools and academic departments have completed at least one cycle of Program Review, and most have done two cycles.  An ad hoc Assessment Committee, created just two years before the 1996 WASC team visit, agreed upon a definition of assessment, initiated an inventory, and joined the Council for Christian Colleges and Universities (CCCU)/Fund for the Improvement of Postsecondary Education (FIPSE) project.  In preparation for the 1996 WASC visit, the Assessment Committee prepared a report which contained an assessment plan, expected uses of assessment data, an assessment inventory and resulting changes, assessment instruments in use, and future directions.  The lack of convincing implementation of and significant progress on a viable assessment program, however, were the foundations of the visiting team’s call for significant improvement.

Creation of Permanent Assessment Structures:  2000-2005.  In order to accomplish our further goals, the Provost appointed a Vice Provost for Educational Effectiveness with responsibility for institution-wide assessment.  The Vice Provost established a standing Assessment Committee and an assessment model for University-wide educational effectiveness, and put in place assessment support structures.  The Provost also named a Dean of General Education with responsibility for overseeing general education.

In March 2000, during a two-day workshop, Jim Nichols—a nationally recognized leader in assessment—initiated PLNU academic and co-curricular leaders into a flexible and workable assessment process, the Nichols model.  The University adopted this five-column model of assessment and began training leaders in the selection of student outcomes relevant to specific programs and University Mission, the designation of means of assessment, the establishment of criteria for success, and the connection of subsequent results to program change and development.  At this same point, the University adopted a timeline that would assist us as we prepared ultimately to submit our assessment program to WASC.  The President also issued a strong statement of support for assessment to the University community.

Adoption of the Nichols model helped define and focus the responsibility of the Assessment Committee as one of support to department chairs, school deans, educational support units, and the Office of Institutional Research in the assessment process, in program reviews, and in linking assessment to programmatic change.  At the same time, the Committee became the central repository for summary documentation and University assessment reports and began working to ensure the link between assessment and the University’s strategic and academic planning processes.

Recognizing the need for more generalized faculty learning about assessment, the Assessment Committee organized a series of workshops, experimented with rubrics as a means of quantifying and measuring educational outcomes, and met with specific schools, departments, and co-curricular units to provide focused tutorials on assessment.  Concurrently, other parts of the institution were developing programs that would complement and slowly transform the overall institutional understanding of learning.  The SPC developed KPI’s for the institution such as retention rates, class sizes, and the percentage of full-time faculty. The Office of Institutional Research, organized earlier to collect institutional and survey data, could now link that data to the institutional learning process. 

As we have established structures and focused their roles, PLNU has been able to move forward through several important steps.  The Committee created the draft of a planning and goals guide for the fall of 2002, indicating that assessment was to become a continuing process, not simply a pre-WASC self-study “snapshot” of the institution.  During the 2003-2004 academic year, most departmental plans were approved, and the Committee worked on developing a summative assessment tool and rubric for general education, administered in the spring of 2004.  By May 1, 2004, the Committee completed and submitted an Assessment Inventory for the WASC Institutional Proposal.  During the opening fall faculty convocation in the fall of 2004, the faculty began the process of developing EEI’s in order for the University to assess whether general educational goals were being met.

The General Education Committee, now taking a programmatic rather than an individual course view of GE, developed first principles and common objectives for general education.  The Committee, in conjunction with the WASC EEI study group and the Assessment Committee, designed two more summative general education exams that would complete the assessment of all the newly designed EEI’s.  Finally, the Assessment Committee set the further goal of “closing the loop” by clarifying through written guidelines what should be submitted annually in the department assessment report to monitor program changes taking place from the assessment process.  We articulated Assessment Committee progress, goals, and expectations for schools and departments to the entire faculty in a subsequent faculty meeting (see Assessment Committee Report November 17, 2004).

Assessment is now the common project of all units of the organization.  Academic and co-curricular units are involved in a continual learning process whose purpose is to improve unity, coherence, and outputs within a democratized governance structure.   Rather than being forced on a poorly informed faculty and co-curricular units, groups embrace the process in a community-wide effort for a common on-going learning process.  While the process has been slow, it has been a necessary passage to achieve significant change in institutional understanding of and approach to learning.
 
Current State of Assessment and Its Effectiveness .  Through sustained efforts over the last decade, we have made good progress in the establishment of an effective and efficient assessment system.  Our current assessment system includes these features:

• a uniform, yet flexible structure for university-wide assessment (the Nichols model);
• a clearly designated structure for overseeing and enabling assessment efforts (an Assessment Committee headed by the Vice Provost for Educational Effectiveness);
• an assessment cycle that can connect to meaningful programmatic change;
• a Dean of General Education personally responsible for creating and administering general education assessment;
• on-going general education assessment aligned with GE principles and goals, with EEI’s, and with a GE philosophy statement (itself aligned to the Academic Plan and University Mission);
• nearly complete compliance with the assessment effort from all academic units;
• growing compliance with the assessment effort from co-curricular units;
• established mechanisms for the reporting of assessment efforts and results;
• an assessment inventory that provides the Committee with a broad view of PLNU assessment and the ability to spot quickly areas in need of development;
• a start to developing mechanisms for data storage;
• a start to developing mechanisms to allow for data availability to appropriate units;
• a beginning on reporting mechanisms to faculty at large; and
• on-going, though not yet regularized, efforts at training and teaching faculty and other assessment leaders.

The assessment structure in place for all academic majors, general education, and many co-curricular programs at PLNU follows the Nichols model (referenced in the Institutional Proposal, Appendix B).  In this model, units specify, for each of their programs, three to five student-learning outcomes that are linked to the University Mission.  For each outcome, units specify measurement tools; collect, analyze, and interpret data; and use the results to inform program improvement.  All schools and academic departments have provided verbal assessment update reports to the Assessment Committee (see Summary of Reports to the Assessment Committee).

An updated Educational Effectiveness Inventory has been completed.  The current inventory reveals that approximately 90 to 95 percent of academic units are on target with an assessment plan, measurement tools, and operating data gathering activities.  Approximately 75 percent of the units have been collecting and using assessment data for three to fo