Committee Members

Maria Zack, (chair) – Professor of Mathematics; Chair, Department of Mathematical, Information, and Computer Sciences, Keith Bell (liaison) –Vice Provost for Academic Administration, Greg Crow - Professor of Mathematics, Patricia Leslie – Associate Professor of Sociology and Social Work , Ken Martin – Professor of Chemistry; Chair, Department of Chemistry, Dee Puntenney – Professor of Physics, Kim Schaeffer – Professor of Psychology and Harry Watkins – Associate Professor of Marketing

Committee Charge

CPR Group One was presented with three interrelated tasks:

The development of the PLNU Institutional Data Set;
A high level analysis of data collection and use at PLNU and suggestions for how these processes could be improved;
A high level analysis of the data set.

Construction of the PLNU Institutional Data Set

For the last 15-20 years many divisions of the University have gathered data for planning and assessment purposes however very little of this data has been centralized. In 1990, PLNU established the Office of Institutional Research whose main function was to satisfy the reporting requirements of various agencies and organizations (Integrated Postsecondary Education Data System (IPEDS), US News and World Report, etc.) and oversee the University’s participation in various surveys (Higher Education Research Institute (HERI) surveys from UCLA, Council of Christian Colleges and Universities (CCCU) surveys, etc.). A limited amount of this data was posted on the website, and since 2002 the site has been updated only sporadically.

It was an eight month process to construct the present University data tables as required by WASC. From October 2004 - May 2005, institutional data from a variety of sources was gathered, processed and arranged in standard data tables. The sources of the data were carefully documented and footnoted in the tables so that there can be consistency across the years as the tables are updated. This detailed work uncovered some difficulties in the use and storage of university data. The general recommendations to the University resulting from both data gathering and data analysis can be found summarized on page 5 with a detailed list in Data Issues.

The team charge for WASC Group One: Data was divided into five areas. What follows is a reflective response to each task in the charge.

Verification That the Data Portfolio Meets the Expectations of WASC

The PLNU WASC Institutional Proposal states “Our Portfolio will provide evidence of effective development and deployment of resources, structures, and systems. We will develop a Web-based data and policy portfolio including all data elements required by WASC, including admissions and student preparation, student enrollments, degrees awarded, faculty and staff composition, physical and fiscal resources, institutional and operational efficiency, and Educational Effectiveness Indicators (EEI’s)” (Institutional Proposal, page 6). The core data set was built using the guidelines found in the WASC Evidence Guide and the WASC Required Data Elements to Support the Preparatory Review and meets the requirements set by WASC.

The PLNU tables meeting these basic data requirements can be found in the University Data Tables on the University’s website. The EEI’s have yet to be posted and all data tables will be updated annually on a rolling schedule linked to IPEDS reporting. As the data set was being built, it became apparent that PLNU had a weakness in the area of data collection for graduate programs. This problem was a product of the rapid expansion of our graduate programs in multiple sites. All current graduate data is now stored in a central location and coded in a uniform way. Thus the institution was be able to capture complete and reliable graduate data beginning with the 2004-05 academic year. This data is now posted on the web site. There will be further modifications to the system for collecting graduate data in the 2005-06 academic year.

The University website has become the institution’s “central filing cabinet” and the posted data will grow to include survey data and other data elements that are found useful in the process of analysis and decision making by each WASC Group and PLNU’s Strategic Planning Committee. It will also include key policy and planning documents.

Analysis of the Type of Data Being Gathered for Decision Making

Various units of the University have been gathering significant amounts of data and using it for their own planning purposes for a several years. However the difficulty in making wide-spread use of this data has arisen from the fact that very little of it has been kept in a central location or in a standard format. The implementation of a PLNU Data Portfolio has resolved this problem. The WASC required elements are a good beginning to creating an “essential minimum” of data that will be tracked annually for institutional decision making and has enhanced the creation of a culture of evidence at PLNU.

Large-scale planning and the implementation of the University vision is the responsibility of the Strategic Planning Committee. This committee consists of the President, representatives from each university “unit” (the divisions associated with each Cabinet member), faculty and staff members, and a student representative. In the last ten years, as PLNU has grown, the University has gone through an evolution of its centralized planning systems. From 2002-2004 the University mission was re-examined and updated and based on the mission statement a new vision statement was crafted. For further details of the development of the planning process and the current system see the WASC CPR Group Five: Planning Report.

Current institutional planning (both large scale and in individual units) is rooted in the institutional vision. Thus, the growing central data set is organized around the five elements of the vision.

Our vision states:
Point Loma Nazarene University will be a nationally prominent Christian university and a leading Wesleyan voice in higher education and the church – known for excellence in academic preparation, wholeness in personal development, and faithfulness to mission.

This is broken into the following six planning categories:

Excellence in Academic Preparation
Wholeness in Personal Development
Faithfulness to Mission
Leading Wesleyan Voice in Higher Education
Leading Wesleyan Voice in the Church
Support of Vision (this is for financial and management goals)

Essential elements from the WASC prescribed university data tables are associated with these categories as well as the University’s Key Performance Indicators (KPI’s) and Educational Effectiveness Indicators (EEI’s). The Strategic Planning Committee is working with individual units of the University to determine what additional data is necessary for planning and thus should be added to the central data set in each of the six vision planning categories.

Issues Arising From the Data Gathered

The committee looked at the institutional data set represented by Tables 1.1.1 through 6.3 and made the following observations some of which should be investigated by other WASC or institutional committees, or offices.

Admissions and Student Preparation

There are key issues related to graduate (both credential and degree granting programs) admissions data that have been identified:

The absence of consistent graduate admissions data for past years. This is due to a lack of uniformity in admissions processes and paperwork requirements across programs and sites. There is also a lack of uniformity in record keeping and data coding. These two issues have been resolved with the introduction of a single graduate application and a centralized admissions process and 04-05 graduate admissions data is now posted.
Difficulties with tracking selectivity in graduate programs. The majority of our graduate programs are aimed at working professionals who are seeking advanced degrees, there is a certain amount of self-selection that occurs before a student applies to our programs. In addition, a personalized admissions process that is conversation based is causing the number of applications to be roughly equal to the number of students admitted to graduate programs. Before most graduate students submit an application they often have an in-depth consultation with a member of the graduate program faculty or staff. It is in this process that the portfolio of admission indicators (pre-requisites, GPA, standardized test results, work experience, career goals, etc.) is discussed. If it is expected that the student will not succeed in the program, the student is counseled not to apply and guided toward other educational opportunities. There are current University discussions about how to track and quantify these interviews and thus track our student selectivity; a system must be developed.

The numbers of undergraduate readmissions is down (Table 1.1.1). What is the reason for this?
There is good progress in the average GPA and SAT scores of our incoming undergraduate students (Table 1.2.1).
The balance between male (40%) and female (60%) undergraduate students has remained constant and is consistent with national trends (Table 1.3.1). Are we satisfied with the existing ratio?
Roughly half the transfer students are male (Table 1.3.1). Is there something significant in that?
There has been excellent progress in increasing ethnic diversity among the undergraduate population (Table 1.4.1). The percentage of diverse undergraduate students who apply and enroll has increased from 15% to 20% over the last five years. The Admissions Office of PLNU has worked hard for this outcome. Further study need to be done to determine which of their many strategies are the most effective. It would be helpful to gather more detailed ethnic data about applicants and admissions to track our success among individual ethnic groups.
We have detailed ethnic composition information about our freshmen once they enroll (Table 1.4.2). How does this compare to the ethnic composition of our catchment area?

Student Enrollments

This section has further problems related to graduate data. Much of the information reported to IPEDS about our student population is based on an annual “snapshot” taken in mid-October. Many of our graduate programs are cohort based and do not begin instruction before that date. So though trends in the data are correct, the actual numbers for a given year do not include all students. To resolve this problem, starting in the Fall of 2005 the graduate student snapshot will be taken in early December.
The enrollment in our undergraduate programs has remained stable since 1999 (due to the City of San Diego imposed enrollment cap) and there has been significant growth in the enrollment in our graduate programs (all at sites other than Point Loma) (Table 2.1). There was a significant increase in the number of part-time undergraduate students (those below 12 units) in 2003-04. Whether this is an anomaly or the beginning of a pattern should be investigated.
The gender mix in the student population has remained stable (Table 2.2). Are we satisfied with the 60/40 split female/male split?
The diversity of both undergraduate and graduate student populations has increased (Table 2.3). As with the admissions data, it would be useful to compare the diversity in our student population to that of our catchment area.
The number of students receiving Pell Grants has decreased significantly (Table 2.4). This should be examined. Is it because of changing government policy or because of a change in the affluence level of our undergraduate population? Does this have a impact on the number of hours students are working while in school?

Degrees Awarded

Compiling Table 3.1.1 Bachelors Degrees Granted was a complex task because the code numbers used for majors has differed across the years. A list linking PLNU degree title and IPEDS code number needs to be established.
Freshmen retention (Table 3.2.1) and freshman graduation rates (Table 3.2.2) have shown significant improvement over the last five years. Freshmen six-year graduation rates are expected to increase dramatically in the next few years since 04-05 was the graduation year of the first class under the university enrollment cap (selectivity has increased since 1999). Transfer student retention and graduation data should be tracked.
There is currently no data on graduate student retention and graduation rates being collected. This needs to be gathered. The process will begin with the Fall 2005 graduate snapshot.
One problem with the faculty and staff data is that much of it has been gathered every other year in response to the demands of IPEDS reporting. Starting in the 2005-06 academic year, data will be gathered annually.
The gender mix in the full-time faculty, adjunct faculty and staff has remained consistent over time and the ethnic diversity has increased (Table 4.1 and Table 4.3).
The turn over in full-time faculty and staff has been fairly low (Table 4.4). It would be helpful to track the turn over in part-time faculty. It would also be helpful to know the average age of the full-time faculty. The committee expressed concern about the number of retirements in the sciences anticipated in the next 4-5 years. For planning purposes, the University needs to know the coming retirement trends and budget for the associated costs (searches, housing co-investment, etc).
The University has done a good job of making strategic use of adjunct faculty while maintaining its commitment to have a high percentage of classes taught by full-time faculty (Table 4.2).

Information, Physical and Fiscal Resources

Based on work done through the Office of Institutional Research over the last several years, there appears to be sufficient classroom space (Table 5.2.1). The issue that needs to be studied is the type of classrooms. Do we have the correct number of rooms needed to teach using various pedagogical methods?
The University appears to be on sound financial footing. It was observed that some definitions for various financial items in a glossary would be helpful. Is the endowment adequate for future needs and are plans being made to grow this fund?

Institutional and Operating Efficiency

It appears that the university has done well in holding back grade inflation. The average undergraduate GPA has risen by approximately 0.1 over the past five years which is consistent with the rise in the GPA of incoming students over the same period (see Table 6.1.1 and Table 6.1.2).
The committee recommends that some further study be given to the trends in the number of both large and small undergraduate classes (see Table 6.1.1).
Further financial definitions would be helpful for these tables as well.

General Observations

In addition, the process of gathering data in order to build the Data Portfolio has pointed out some larger institutional issues relative to data:

The institutional Research function of PLNU needs to be strengthened.
Key institutional data needs to be kept in a centralized location.
There are several processes related to data gathering that need to be adjusted.

Recommendations

Specific recommendations that arose in the creation of Tables 1.1.1 through 6.3 and in the analysis detailed above are listed in Data Issues. They fall into several major categories:

The University needs to continue to study institutional diversity. Great strides have been made and some analysis is called for to identify our most effective strategies. In addition, a comparison needs to be made between the university’s faculty, staff and student population and that of our catchment area.
There needs to be improvement in the collection and reporting of graduate program data. The University is in the process of correcting the problems, but must be diligent in seeing the task through over the next few years.
Faculty and staff data needs to be collected on an annual basis. Particular attention needs to be given to projecting trends in retirement in the coming years and making institutional financial plans accordingly. The HERI faculty survey data indicates that roughly 20% of our full-time faculty are 60 or older, 10.9% of them plan on retiring in the next three years, and 20% of the faculty have considered early retirement.
That the University continues its emphasis on having a significant portion of its courses (both graduate and undergraduate) taught by full-time faculty.
A glossary of terms for the data set should be available. It is recommended that the IPEDS definitions be a link on our webpage and that a short list of financial definitions be created.
A “non-expert” financial report would be a helpful addition to the data set. Such a report is prepared annually for the Board of Trustees. Consideration should be given to adding that to the web portfolio.
As part of the academic planning process, some consideration should be given to the data on class size and the current and future classroom needs at all sites.
The data portfolio needs to continue to grow as various WASC teams and divisions of the University identify essential data needed for decision making.
PLNU needs to increase the staffing and support for Institutional Research including assisting individual departments with their data needs. For specific recommendations, see Institutional Research Survey.
Procedures for gathering standardized data need to be written down. These should include dates and identify individuals responsible for gathering the data.
The Data Portfolio provides a much-needed central location for data; however, a great deal of manual work currently underlies the creation of the tables. Ultimately, PLNU needs to look at making better use of its central data base in CARS to build and maintain these tables automatically.
All raw survey data needs to be stored electronically with appropriate privacy protections. This includes standardized surveys in which PLNU participates. When practical, the additional fee to obtain raw data should be paid.
As part of our effort to create a “culture of evidence” at PLNU, the use of data needs to be documented in reports. If data is used, it should be cited; this should be standard practice for all annual reports. In addition, the institution needs to verify that data collected is useful. Are we truly using the information gathered as part of decision making or research processes?

Additional Questions

How will we demonstrate that the University has reflected on the data, made meaning of it, and decided how to act on it?
Where does the data actually enter our planning and decision making processes?
What can we say in the Capacity and Preparatory Review Report about where and how such data is considered?

The university is collecting a very large body of data and much of it appears to be used in only a limited way by individual units. Decisions need to be made about the essential data to be collected, analyzed and displayed as part of the core data set on the website. This is part of the work that will be done by the Strategic Planning Committee.

The Assessment Committee under the leadership of the Vice Provost for Educational Effectiveness is the group responsible for supporting and guiding PLNU departments in the use of data to assess and improve learning outcomes. The General Education Task force under the leadership of the Dean of General Education gathers data and assesses general learning outcomes. See the report of WASC CPR Group Two: Educational Effectiveness Indicators and WASC CPR Group Six: Assessment for further details of their work.

It is clear from the reports of individual departments, schools and units of the University that data is being used for planning. Such items as assessment data, department reviews and unit planning documents should be on display. In those documents the what, where and how of data use in individual areas of the university is discussed. There is, however, a lack of standardization among these documents. Data gathering would be made easier if annual reports and department reviews had a more consistent set of information across University units. The documents currently are kept in a variety of locations:

Department Reviews (in the office of the relevant area dean);
Annual Assessment Reports (with the Vice Provost for Educational Effectiveness);
Strategic Planning Action Plans and Progress Reports (with the Chairs of the Strategic Planning Committee);
Unit Goals and Plans (with the individual units, Spiritual Development, Student Development, etc.).

The Strategic Planning Committee is attempting to gather some of this information for inclusion as part of the University’s “central filing cabinet” on the web, but a large scale effort will be needed.

The use of data for global institutional decision making is the responsibility of the Strategic Planning Committee (SPC). The SPC has used Key Performance Indicators (KPI’s) as well as institutional financial data in its work in previous years. The SPC must finish expanding the list of data for planning (the data gathered around the vision) and make use of it in the planning cycle in the 2005-06 academic year. This new expanded use of data in planning is key to our developing culture of evidence and must be in process and posted on the website by the time the Capacity & Preparatory Review (CPR) document is written in the summer of 2006.

Appendix Items:
Institutional Research Survey
Data Issues