Venia ~ Veritas ~ Sanctitas - University Motto

Preface

The Educational Effectiveness Review, like the Capacity and Preparatory Review before it, has been a significant educational vehicle for the Point Loma Nazarene University community. This report represents a significant step along the outcomes road. It reports on the results of assessment across the academic programs, explores the impacts of new program development and ongoing program review on program quality and student learning, and evaluates the ways the University’s goals of To Teach, To Shape, To Send actually have an impact on students’ lives. While there clearly is more remaining to be done for this outcomes approach to be fully institutionalized, we acknowledge that we are now asking ourselves the right questions.

Development of the Educational Effectiveness Review Report. The Educational Effectiveness Review (EER) report for Point Loma Nazarene University (PLNU) is the result many months of work by a substantial cross-section of senior administrators, professional staff, and faculty. This EER report for the Western Association of Schools and Colleges (WASC) provides evidence for PLNU’s assertion that it meets the accrediting agency’s Core Commitment to Educational Effectiveness. That commitment is summarized in the following sentences, taken from the WASC 2001 Handbook of Accreditation:

The institution evidences clear and appropriate educational objectives and design at the institutional and program level. The institution employs processes of review, including the collection and use of data, that assure delivery of programs and learner accomplishments at a level of performance appropriate for the degree or certificate awarded.

Under the direction of a Steering Committee, we carefully followed the outline of tasks as found in the Institutional Proposal approved by WASC in 2004. We took the research and commentary that six assigned sub-groups completed and reworked materials focusing on outcomes, so as to stay with the spirit of the Institutional Proposal. The following pages contain the various stipulated elements, including updates on the recommendations put forward in the Capacity and Preparatory Review (CPR) report (October 4-6, 2006), our response to the CPR visiting team’s suggestions and the Commission’s letter of February 27, 2007, a description of PLNU’s Educational Effectiveness approach, several analytical essays following the Special Themes Model (as described in the WASC 2001 Handbook of Accreditation), and a concluding integrative component. In addition, we have provided additional evidence for the analysis of Educational Effectiveness in updates to the Institutional Portfolio and new data exhibits.

The Institutional Proposal outlines the context of our EER report and prescribes five principle elements. Having opted for the themes model, we begin the report by providing updates on institutional actions and documented results since the CPR visit. Second, since our Educational Effectiveness Indicators (EEI’s) are embedded in the University’s assessment program, we revisit the EEI inventory as part of a comprehensive review. Third, understanding that vital quality assurance processes are critical in sustaining educational effectiveness, we identify assessment, new program approval, and program review processes as the focus of three essays. We understand that these are particularly important factors in developing new graduate programs while sustaining quality in a more traditional undergraduate setting. Fourth, in order to assist us in significant engagement with the thematic statements that have evolved as helpful shorthand for the institution’s Mission Statement: To Teach, To Shape, To Send , we devote reflective essays to each of these three major themes for the University. This aspect of the report is in direct response to one of our intended outcomes from the reaffirmation of accreditation process, namely, to “ensure that the new mission statement is embraced, shared, and embodied throughout the institution” (Institutional Proposal, page 3). Finally, we summarize our experience with the entire accreditation review process, pointing to fresh understandings, improvements and implemented recommendations, identifying shortcomings, and establishing an appropriate agenda for future action.

Capacity and Preparatory Review Results. The CPR report produced an initial array of 83 recommendations to the President. Subsequent to these findings, the institution established a small group of professional staff and faculty members to function as a Recommendations Sub-committee for the purpose of coming to terms with these findings and recommendations, in cooperation with the WASC Steering Committee and the President. The recommendations were codified by the Office of Institutional Research according to the “responsible persons or group” classifications indicated in the CPR report. Subsequently, individuals named to the group were assigned to on-going conversations with contact persons with the goal of determining progress made. The Sub-committee also made appropriate decisions relative to strategy and, in cooperation with the Steering Committee and the President, established a list of priority recommendations.

First, we determined that all recommendations labeled as “on-going” were fundamental commitments that the University would retain beyond the accreditation review cycle. Second, we noted that a small group of recommendations had been accomplished already, either during the days and weeks between the filing of the CPR report and the actual visit of the CPR team or immediately thereafter. Third, the Sub-committee recognized early in its review the need to prioritize the recommendations to the President for more efficient benefit to the institution, a point also clearly made in the CPR visiting team report.

The findings of the Recommendations Sub-committee and the Steering Committee, of which the President is a member, identified the following issues as appropriate priorities emerging from the CPR cycle:

• Incorporation of the recommendations gleaned throughout the entire review process into University planning;
• Continued development of a sound infrastructure and organizational model for both undergraduate and graduate studies that includes effective assessment protocols, together with specialty accreditations where appropriate;
• Attention to a deepened and extended approach to the assessment of all aspects of learning outcomes at the University, including general education, major programs at both undergraduate and graduate levels, and co-curricular programs;
• Continuation of efforts by the University to increase retention and graduation rates for all programs, especially analyzing disaggregated data by various categories; and
• Clarification of the University’s policy on academic freedom with the opportunity for extended dialog on its implications for teaching and scholarship in the context of our mission and vision.

The President has referred these priorities to the Strategic Planning Committee for their attention, as well as to the Administrative Cabinet who also serve as the University’s budget committee. This will help to ensure the possibility of developing specific strategies for these priority issues as well as appropriate funding in the annual budget cycle. Beyond this, we have created a spreadsheet that summarizes the current status of all recommendations made to the President in the CPR report.  (See Responses to the CPR Visiting Team and the Commission.)

Context of the Educational Effectiveness Review Report. Moving from the CPR process to the EER process has proven to be an important source of learning for Point Loma Nazarene University. As we moved from thinking about what is in place to evaluating what difference does this make in student learning , we found that we needed a new set of questions and a new mindset. We know that this shift in mindset will serve the institution well for years to come. In addressing the requirements for the EER report, we recognize the central importance of focusing on inviting sustained engagement by the institution on the extent to which PLNU fulfills its educational objectives. In the context of this report, we attempt to review our efforts to assess educational effectiveness, examine practices for evaluating student learning in order to improve the process of teaching and learning, test the alignment of resources with educational activities, and maintain a sustained engagement that will be on-going and not merely an effort to respond to subsequent reaffirmation of accreditation processes.

Reflecting on the three quality assurance processes of assessment, new program development procedures, and program review has been particularly beneficial to the University. We know that vital quality assurance processes are critical in sustaining educational effectiveness. We identified these three areas as we continue to focus more on student learning, develop new graduate programs and teaching locations while also sustaining quality at the undergraduate level in an enrollment-capped environment, and utilize an improved process of program review.

The Themes Model (as described in the WASC 2001 Handbook of Accreditation) has helped to provide evidence that we have a system of quality assurance for student learning and organizational effectiveness that demonstrates “an institutional and leadership focus; appropriate educational objectives; learning outcomes that are widely shared and reflected in academic programs and policies, faculty who take collective responsibility for demonstrating and reviewing attainment of those expectations; a growing culture of evidence that is well established and used regularly for improvement; and on-going, regular collection and use of evidence to assure program delivery and learner accomplishments” (WASC presentation, January 8, 2004).

Approach to Educational Effectiveness

Markers of Educational Effectiveness. In contemplating our approach to an analysis of educational effectiveness, we recognized the appropriateness of mentioning both historical and current issues in this context. For example, it is clear that there already exist several markers of effectiveness embedded in the University’s culture, since its founding in 1902 in the Los Angeles area and especially within the last 34 years in the San Diego community. As pointed out in the Institutional Proposal, the vision of our founder, Dr. Phineas F. Bresee, was for a substantial university of 3,000 students with various professional schools and an undergraduate residential liberal arts college. The institution that has emerged since the turn of the twentieth century has, in fact, become the realization of Dr. Bresee’s grand expectations.

One hundred and five years later, PLNU is a prominent Christian university that has Mission and Vision statements and stated Core Values that are congruent with our historical roots. We can point to increasingly selective admissions patterns along with dramatic improvements in persistence and graduation rates, strength in leadership development, robust connections to its sponsoring denomination (the Church of the Nazarene), and a commitment to the liberal arts and professional programs. In addition, the University can claim financial health and fiscal responsibility; a respected pre-medical program; a successful School of Nursing prominent in the greater San Diego area; a long-standing undergraduate research program in the sciences; an excellent teaching faculty committed to professional improvement, scholarship, research, and service; a series of specialized accreditations (allied heath education, business, dietetics, education, music, nursing); and a functioning strategic planning design. Moreover, team members from the recent CPR visit recognized and applauded many of these characteristics. Aside from these markers, however, PLNU understands that educational effectiveness cannot be fully measured in traditional ways, and we are determined to cultivate and sustain a “culture of evidence” with increased emphasis on demonstrating student learning and enhanced educational vitality. Consequently, PLNU has seen a dramatic shift from the use of descriptive portrayals of educational effectiveness to more analytical and evidentiary representations.

This important transformation is producing results that began during the early timeframe for creation of the CPR report, that continue in the current EER cycle, and that will endure in the years to come. With the establishment of an institutional Data Portfolio , the monitoring of trends has become easier and more efficient, and staffing in the Office of Institutional Research has increased in addition to the establishment of a routine for collection protocols (CIRP, YFCY, CSS, NSSE, alumni surveys). The assessment of general education learning outcomes continues as first outlined in the CPR report, and academic departments and schools are adopting new approaches to monitoring educational effectiveness including standardized tests, senior capstone courses, and multiple measures. Not only have the inventory of Educational Effectiveness Indicators tables been updated in connection with the EER cycle, but the President, Provost and Strategic Planning Committee are designing updated structures (including staffing) to support PLNU's growing culture of evidence. Plans are also underway to make more systematic and effective the utilization of alumni surveys, in an attempt to capture data on the acquisition of graduate education, professional growth, and other value-centered measures of effectiveness after students are well into post-baccalaureate experiences. With the naming of two new Vice Presidents in the co-curricular units of Spiritual Development and Student Development, assessment efforts will continue and grow in sophistication. All of these markers resonate with the two focal points of our EER report.

Two Focal Points. Our approach to educational effectiveness, as approved by WASC in our Institutional Proposal, includes two broad areas of attention: 1) three quality assurance processes and 2) three institutional themes. To accomplish this, we established six study groups of professional staff, faculty, and administrators and began collecting data, analyzing our findings, and drawing conclusions that resulted in recommendations to the President.

Quality Assurance Processes

Introduction. In preparation for opening the focus on quality assurance addressing assessment, new program development, and program review, we charged each study group with specific target questions. In general, we wanted to determine and verify the impact of assessment in enhancing institutional quality and student learning and establish to what extent quality assurance processes actually are improving program quality and student learning. We present these reflective essays in the following section, each with an opening statement regarding the institutional culture, evidence and analysis, a summary of our conclusions, and final recommendations to the President for action.

Assessment

Perhaps one of the most important aspects of verifying quality at an institution is the establishment of efficient and effective assessment protocols that provide data to be interpreted in meaningful ways that can point to necessary improvements. The following reflective essay looks at a variety of issues, including the University’s EEI’s, an evaluation of the PLNU Assessment Program, improvements in student learning, institutional engagement, evidentiary support for conclusions, and finally assessments in academic majors, general education, and the co-curriculum. This reflective essay addresses CFR’s 1.2, 2.4, 2.6, 4.4, 4.5, 4.6, 4.7, and 4.8.

The Institutional Culture

At Point Loma Nazarene University there is a strong commitment to improvement. It is a culture and a practice steeped in who we are and is even rooted in our theology. John Wesley, forefather of our Christian faith tradition, believed that our gifts and talents belong to God. As a community of faith in God that believes our work is an act of worship, we are compelled to strive for excellence. We genuinely care about our students and want the best experience for them.

The University, therefore, has a core philosophy of improvement. This is evidenced by the work spear-headed by the University’s standing Assessment Committee that assists units in reviewing programs and finding ways to improve them. The formal assessment program using a Nichols Model ( Institutional Proposal p. 48) began in 2001, although periodic academic program review was systematized in 1994. In the latter model, academic units used information from assessing student learning outcomes as part of the second round of program review. Units then instituted program changes following the review and the assessment of student learning outcomes.

While a cornerstone of our identity, until recently the practice of improvement work has happened to a large extent by relying on the goodwill of individuals more than on structures and common processes. The WASC review process has helped us add structure and form to our commitment to improvement. In the last seven years, we have made progress in identifying learning outcomes befitting our Mission, measuring them, reflecting on the meaning of data, and making improvements based on the evidence. To this point, evidence shows that while our educational programs are generally effective, there are improvements to be made. Having a system that forces us to reflect about what improvements could be made, based on assessing identified learning outcomes that we care about, has made us better.

The philosophy of the University Assessment Committee is to build structures that are flexible enough to allow academic departments and schools to set their own learning outcome goals, but that also provide guidelines requiring those goals to be set within the context of the University’s mission.  Our philosophy mandates that they are measured using meaningful tools with external validity and that the data are used for program improvement. In addition, the Committee’s purpose is to build breadth and depth across the University and to gain 100 percent compliance by using a carrot rather than a stick—by providing positive support to do the work of assessment, and by helping academic and co-curricular units see the value of assessment. The Committee encourages units to set a few goals at a time, measure them creatively and reliably, set a target that they truly aspire to reach, and learn from the process. Setting targets that are realistic yet ambitious allows room for program improvement.

At PLNU we expect academic and co-curricular assessment leaders to spend time doing the creative planning of assessment goals and measurements, and to make decisions about needed program changes. Ideally, the Committee would like to provide the staff support to assist leaders in data collection, data analysis, and report generation.

Evidence and Analysis

The Educational Effectiveness Inventory . The Educational Effectiveness Inventory, currently contains an assessment plan for general education (12 Educational Effectiveness Indicators, called EEI’s), 64 plans for academic programs (one EEI, 57 undergraduate majors and/or concentrations, eight graduate programs, and five teaching credentials in 17 academic departments/schools), and plans for co-curricular programs (four in Student Development and five in Spiritual Development). Plans are under construction but still need to be posted for two undergraduate academic majors in Exercise Science (Department of Kinesiology), a new major in International Studies (Department of History and Political Science), and a new graduate program (master’s degree in General Biology).

We have continued to work throughout the WASC Institutional Proposal stage, Capacity and Preparatory Review report, and the Educational Effectiveness Review to update these plans and to close the loop by adding findings data and resulting program changes. To date, data exists for all of the general education outcomes, 73 percent of the academic program outcomes, all of the Student Development program objectives, and 80 percent of the Spiritual Development program objectives (see Data on Co-Curricular Outcomes). Assessment data has been used to begin discussions about improvements in the general education curriculum and to make program changes in all of the academic units for which data has been collected. Findings from assessment activities have had an impact on the co-curricular programs in Student Development and Spiritual Development. (For details see program-specific data tables.)

Evaluating the University Assessment Program. The University completed an extensive evaluation of the assessment program in the CPR Group Six: Assessment reflective essay, focusing specifically on capacity. Findings at that time included recommendations for increased staff support to do the work of assessment, and better support mechanisms to help those responsible for assessment activities at the unit level to develop in their expertise and do their work. We continue to consider these recommendations in the budget process at the Administrative Cabinet level, and the Assessment Committee is instituting some improved support mechanisms based on these recommendations (web page, mentorship, development opportunities). A culture shift continues as movement toward full compliance occurs and assessment activities become routine.

Analyzing the assessment plans currently posted to draw conclusions about whether the University is an effective educational institution also requires another broad look at the effectiveness of the assessment program itself. Further recommendations for the University assessment program become evident as we analyze the findings from academic plans. Gaps in the assessment program also become evident as academic plans are analyzed as a whole, as we consider co-curricular program assessment broadly, and as we review linkages between the assessment program and other University processes. Recommendations will emerge at the systems level as we analyze the effectiveness of the entire assessment program.

Improvements in Student Learning. Implementation of a systematic assessment program at PLNU has given focus and intentionality to our stated philosophy of improvement. It has forced us to think about and identify student learning outcomes for programs, measure them, reflect on the data gathered, and consider ways to change our programs as a result. In some cases, the data served to inform leaders that the measurement methods were inadequate, and they altered the tools. In others, the data informed leaders that students were performing well. In yet others, student performance did not meet targets, and the assessment activities helped identify where the deficiencies occurred so that changes could be made. All of these circumstances represent an important learning process and reflect the culture of the institution as a learning organization.

Overall, assessment activities have informed the University that students are learning (meeting knowledge targets), growing (being shaped professionally and ethically), and that they are adequately prepared to enter the world of professions and/or continue formal studies at an advanced-degree level. These results align with the University Mission summarized in the phrases: To Teach, To Shape, To Send (see reflective essays below).

University-wide Engagement and Improvement . Analysis of the general education program outcomes discovered that students are skilled at lower levels of Bloom’s Taxonomy, but student performance in higher level skills (critical thinking) are less impressive (see EER Task Force I: To Teach report as well as the reflective essay on To Teach). The General Education Task Force is currently deliberating changes in the general education program in order to address these findings.

An assessment of major programs finds students have attained an appropriate level of knowledge, skills, research, recognition/appreciation, maturity/growth, faith integration, moral/ethical behavior, preparation for future study, influence (in field) and life-long learning (more than half of the outcomes cited met targets). Performance on production/performance, career preparation and engagement in the field needs attention (less than half of the outcomes cited met the targets).

Co-curricular results for the Office of Spiritual Development indicate generally that PLNU is moving from input-focused to output-driven data collection. Results indicate that PLNU student participation in volunteer service exceeds that of Comparator Institutions, and student growth in Christian disciplines between freshman and senior years meets targets, though there are outcome areas for which data is not yet available (see Spiritual Development Learning Outcomes).

The Office of Student Development collects and reviews extensive data annually on retention; engagement; social, emotional, and physical development; and leadership. The last five years reveal an increase in student retention, a steady growth in student participation in co-curricular activities, acceptable growth targets in wellness measures, and an expansion in leadership opportunities (see Student Development Learning Outcomes). The data identified diversity as an area needing improvement. We have instituted changes, as reflected in the CPR report (see CPR Group Seven: Diversity), based on the findings, including the creation of new opportunities for student engagement, wellness program modifications, and additional leadership opportunities.

Centers serve a variety of purposes to enrich students’ educational experiences and have an indirect impact on teaching and learning by supporting faculty performance. All Centers have specific goals (see Centers by Unit and Mission and the home pages of the individual  Centers). There is anecdotal evidence that the Centers are making an impact on student learning, but we need to develop a systematic way to measure learning outcomes. Some have collected data  and made changes to programs based on the findings, and generally we have found Centers to have an outcome-driven focus and conduct periodic review, even if a uniform system is not used across all Centers (see Centers by Output and Program Impact).

Evidentiary Support for Conclusions. While the documentation of assessment activities that exist is an on-going challenge, the Assessment Committee recently updated assessment plans and reviewed them in their current form by developing analytic tools showing common outcomes, instruments, findings, and use of results across the 64 academic assessment plans. Further, the Committee developed a spreadsheet to identify 22 common outcomes across 64 programs, which of the 22 outcomes was selected by each academic program, whether data was collected for each of the cited outcomes, and whether the targets were met, not met, or unclear. A second spreadsheet tracks the common tools used by academic programs to measure outcomes, and a third summarizes the common types of changes that resulted from assessment activities.

Assessment of Academic Majors. We have  Assessment plans for 64 out of 67 (96 percent) academic programs on the PLNU Assessment website, revealing measurement of specific student learning outcomes (100 percent), collection and reflection on the meaning of the data (73 percent), outcomes met (67 percent of outcomes for which data exists), and changes made to programs as a result of data collection (100 percent). (For details see Data on Common Learning Outcomes by Department and Program Changes by Department.) In assessment of academic majors it is helpful to analyze more in depth the commonalities from the standpoint of outcomes, methods of measurement, results, and the program changes made as a result of assessment.

Relative to outcomes, we can see that common themes identified in assessment plans mirror the general student outcomes of the University’s mission: learning (teaching), growing (shaping), and serving (sending). There are 22 distinct learning outcome categories cited by at least one academic program. These are listed by mission, category, and frequency in Common Learning Outcomes by Program. Knowledge is the most often cited student learning outcome (cited by 80 percent of programs) and career preparation is the second most often cited outcome (45 percent). Clinical skills in practice (39 percent) and technical skills (33 percent), are the third and fourth ranked outcomes respectively.

There are six common outcome categories cited that pertain to the teaching mission of the University: knowledge (80 percent), technical skills (ranked second), production of work or performance, writing skills, critical thinking skills, and dispositions or attitudes. There are eight common outcome categories cited by programs that relate to the shaping mission of the University, involving student growth or maturation processes: clinical or practical skills (39 percent), faith integration (20 percent), analysis or research skills, recognition or appreciation, maturity or growth, writing or reflection, communication, and lifestyle commitments. In the sending mission group, eight common outcome categories emerge: career preparation (45 percent), engagement (27 percent), further academic study (22 percent), moral or ethical behavior, analysis or evaluation in practice, influence in the field, life-long learning, and service. Outcomes that might be considered distinctives of a Christian university—faith integration, moral or ethical behavior, and service—were cited by 31 percent of the programs. Faith integration was identified by 20 percent; moral or ethical behavior by 9 percent; and service by 2 percent. These seem relatively low, and yet since academic programs were asked to identify three to five learning outcomes, it is not surprising that most programs identified professional or discipline-specific outcomes rather than more generic mission or faith-based outcomes.

An analysis of subgroups by mission and the number of outcome citations that occur in each mission area, referenced in Common Learning Outcomes by Citation, reveals that 43 percent of the outcomes cited are in the teach subgroup, 28 percent are in the shape subgroup, and 30 percent are in the send subgroup. These results reflect a fairly balanced set of outcome citations across programs, but a higher proportion of outcomes appear in the teach area, again possibly due to the fact that learning outcomes may be easier to quantify.

We also provide a breakdown of common outcomes by graduate and undergraduate programs in Common Learning Outcomes for Graduate and Undergraduate Programs. The highest ranked outcomes for graduate programs are knowledge (88 percent), practical skills (75 percent), and career preparation (38 percent). This pattern mirrors undergraduate programs in ranking, although for graduate programs practical skills are cited more often (75 percent for graduate and 34 percent for undergraduate), and career preparation is cited less often (38 percent for graduate and 46 percent for undergraduate), results that may reflect the professional nature of most of the graduate programs, taken by working adults, and with a greater focus on practical skills and less emphasis on career preparation.

In connection with common methods of measurement, each program cites a variety of methods to measure student learning outcomes (see  Methods Used by Academic Programs to Measure Outcomes). The frequency represents the number of times a particular method is cited, or how many of the 64 programs use each of the methods. The method most often used for assessment of academic programs is a senior exam (61 percent), and the second most popular is an alumni survey (55 percent).

It is important to know something about the reliability of the evidence collected to assess student learning outcomes. Each of the methods used by academic programs appears as either direct or indirect (self-reported) evidence, and each was identified as externally evaluated, internally evaluated through jury review, or internally evaluated. The validity of the evidence is relatively higher if direct evidence is used and if the data is externally evaluated. Having evidence that is internally evaluated by a jury provides the next best validity. Validity of Common Instruments identifies how many of the 252 instruments cited by academic programs are direct evidence (74 percent) or self-reported evidence (26 percent), and how many have external validity (68 percent), internal validity with juried review (28 percent), or internal validity (4 percent). The greatest reliability in data collection is for instruments having direct evidence with external validity (42 percent). However, a significant proportion of the outcomes have direct evidence with either external validity or internal validity with juried review (70 percent), and nearly all outcomes are measured with either external validity or internal validity with juried review (96 percent). In sum, the instruments used to assess Point Loma’s academic programs are largely direct evidence with a high degree of external validity.

In terms of common results, we have collected data on 73 percent of the 254 learning outcomes cited by academic programs. The percentage of outcomes for which data has been collected by outcome and mission category appear in Data on Common Learning Outcomes. Outcomes that might be harder to measure seem to be ones where data collected is lower, such as faith integration (38 percent) or shaping goals in general (69 percent for the subgroup). Outcomes cited by relatively few programs have a great degree of variability with respect to percentages of data collected, due to the low number of outcomes cited.

Data on Common Learning Outcomes for Graduate and Undergraduate Programs relays data gathered on outcomes by graduate and undergraduate programs. Ninety-two percent of the outcomes cited for graduate programs have data findings, compared to 71 percent for undergraduate programs. This shows comparatively greater compliance but for significantly fewer outcomes (25 in graduate programs compared to 229 in undergraduate programs).

The existence of data on outcomes by department or school appears in Data on Common Learning Outcomes by Department, which reveals the unevenness of the number of outcomes and the progress on measuring them across units. Ten out of the 19 listed have collected data on all of their outcomes. Four of the 19 have collected data on over half of their outcomes, three have data on nearly half, and two have collected data on approximately 30 to 40 percent of their program outcomes. There remain three programs without plans, not included here. The Department of Music has the largest number of programs and by far the most outcomes to measure. The School of Theology and Christian Ministry and the Department of Literature, Journalism, and Modern Languages have the next most outcomes to measure. While a majority of outcomes cited have been measured, there remain 27 percent of the outcomes without data. However, 100 percent of departments and schools have measured at least some of their outcomes. These results underscore the need for additional support to assist academic units in measuring their outcomes and a need for more guidance in streamlining outcomes and measurement.

Findings for the outcomes, for which data exists, appear in Analysis of Common Findings by Outcome. These findings represent data on 186 outcomes, or 73 percent of the outcomes, and they show whether the targets for outcomes have been met, not met, or if it is unclear whether the target has been met. Of the 186 learning outcomes that have reported findings, 124 (67 percent) met their targets, 12 (6 percent) did not, and 50 (27 percent) either reported findings that were too ambiguous or data that was inadequate to determine whether targets were met.

Findings by department/school and by graduate-undergraduate subgroups for outcomes for which data exists appear in Analysis of Common Findings by Department. For eight academic units and two joint programs, or 10 out of 19 (just over half), all of the outcome targets were achieved. Only two programs did not meet any of the outcome targets for which they report findings. If considered by individual outcomes, 67 percent of the outcomes for which data exists met their targets. Forty-eight percent of graduate program outcome targets were met, and 69 percent of undergraduate program targets were achieved. Only 6 percent of all outcomes were not met, 7 percent of undergraduate outcomes, and none of the graduate outcomes fell short of targets. Only four of the 19 academic units targeted outcomes that fell short of their desired levels, ranging between 6 to 50 percent of the outcomes they targeted.

With respect to program changes, in a few cases program changes led to documented improvement in student learning. Most programs have either used results to change curriculum and/or operational elements of the program, or revised data collection efforts. While all of these efforts toward change show that Point Loma is becoming a learning organization, they also show that the primary work ahead centers around more effective data collection efforts.

Common threads across academic programs with respect to the use of results from assessment activities appear in Program Changes by Department, which also shows the national standards by area and program review cycles. These results also appear according to the frequency of changes by category in Program Changes by Category.

Curriculum and data collection are the most often cited changes resulting from assessment activities in undergraduate units (76 percent). Operations and curriculum are the most often cited changes in graduate units (75 percent). There are six undergraduate academic units (35 percent) and two graduate programs (50 percent) that met targets and did not find a reason to make changes in their programs but continue collecting data and monitoring the findings (see Program Changes by Department and Program Changes by Category).

Assessment of General Education. An analysis of general education program outcomes discovered that students are skilled at lower levels of Bloom’s taxonomy, but are weak in critical thinking skills and diversity (see the 2007 EEI GE Summary, the  EER Task Force I: To Teach, and the reflective essay on To Teach). The General Education Task Force is currently working on using assessment results to improve the program (see the CPR Group Two: Educational Effectiveness Indicators reflective essay).

Assessment of Co-Curricular Programs. Assessment results for Office of Spiritual Development indicate that PLNU is strong in service outcomes relative to Comparator Institutions but that there is a need for improved data collection (see Spiritual Development Learning Outcomes) . The Office of Student Development assessment reveals good results in growth, wellness, and leadership, but also the need for improvement in diversity (see Student Development Learning Outcomes). Changes have been instituted as a result of the findings. A review of assessment activities in co-curricular areas reveals the need for improvement in data collection methods.

There are ten Centers at the University that enrich the education of students through a variety of academic and co-curricular activities, listed in Centers by Unit and Mission. As mentioned previously, Centers serve a variety of purposes to enrich students’ educational experiences and to have an impact on teaching and learning indirectly by supporting faculty performance. All have specific goals and ways to measure effectiveness (see Centers by Unit and Mission). Some have collected data and made changes to programs based on the findings (see Centers by Outputs and Program Impact). Generally, centers have an outcome-driven focus and conduct periodic review, even if a uniform system is not used across all. The pertinent question is: what work has been done by centers to set goals, review their programs, and discern their impact on students’ lives?

While there is no systematic, uniform review that is required for PLNU Centers, evidence reveals that the centers set appropriate goals, regularly evaluate their activities, and adjust them to meet objectives. A general review of Centers’ methods of measuring effectiveness and findings from data collection appears in Centers by Outputs and Program Impact and reveals that centers are to a degree output-focused and that a few have collected data on outcomes. However, we believe there is significant enrichment activity going on in the Centers and that a rich opportunity to collect data relative to their impact on students is being missed.

An initial assessment of the study abroad program was conducted in 2004, revealing a significant impact on student learning. Students reported experiencing increased growth, enhanced learning about cultural and cultural differences, and gaining a new perspective on the United States. Students who studied abroad also experienced greater college satisfaction and higher retention rates. An updated assessment of the study abroad program is currently underway (2007).

Summary of Conclusions

In this reflective essay, we have attempted to show evidence of a link between program improvement and the systematic assessment process. We have demonstrated common outcomes in keeping with the University mission, appropriate tools of measurement, and resulting program changes. The analysis of 64 academic assessment plans shows that data has been collected on 73 percent of the outcomes selected by academic programs, and that 67 percent of the outcomes for which data exist have met their criteria for success. We have made program changes due to assessment activities; while we can only identify one case of double loop closure, through significant dialog with departments and schools and the huge efforts to document assessment work at the University, we believe that our challenge is one of documentation.  We remain committed to a culture of improvement and rigorous assessment activity.

Point Loma’s strengths are that we care about student learning; we have a good structure, good compliance, common outcomes that fit our mission, and a good start on data collection. We have findings on most outcomes. We have a culture of improvement, and we are making good progress on becoming a learning organization. Systematic measurement of general education learning outcomes has occurred, and work is underway to analyze the findings and use the results to improve the general education program. Assessment of the co-curricular is underway, particularly in Spiritual Development, Student Development, and Centers. Finding ways to document effectively the assessment activities in the co-curricular arena and to deepen these activities across the University is one of the current challenges.

Our weakness is that we are missing data on some outcomes, and we have some unclear findings on outcomes for which we have data. As pointed out in the CPR Group Six: Assessment reflective essay, we are missing one central common structure to collect, analyze, and report data so it can be used effectively and efficiently by the community, and we lack sufficient staff support to do the work of assessment.

Directions for Improvement. The University has identified several issues that provide direction for improvements in efficiency, effectiveness, interpretation, and analysis. In terms of efficiency and effectiveness, these include the creation of a broad-based documentation system, the identification of ways to measure faith-based (shaping) outcomes, the commitment of staff support, and the streamlining of outcomes and measurement methods. In addition, we need to bridge the processes already in place (program review, assessment, and professional accreditations), establish mechanisms for assessment of the co-curriculum, and organize an alumni survey that is centralized in the University with data results disaggregated by identified units. Finally, we need to discover ways to document program improvements to determine whether assessment processes have an impact on learning outcomes, and capture the impact of Centers on the enrichment of student learning.

Relative to improving the interpretation and analysis of data, PLNU needs to share common outcomes among academic program leaders for the purpose of refining outcomes, and discuss the relationship between assessment results in general education and academic majors. Likewise, we need to stratify outcomes and findings by area to see if insights emerge, and provide developmental activities for leaders of assessment efforts to build expertise and skills, such as workshops on methods of measurement, writing a good survey instrument, and thematic analysis.

An effective assessment system of the future will intertwine micro-level and macro-level assessments with strategic planning. Micro-level assessment needs to intertwine program review and annual assessment, where program review (five-year cycle) answers the question, “Are we doing the right thing?” Annual assessment activities seek to answer the question, “Are we doing what we claim relative to student learning outcomes?” In the latter, unit directors and chairs/deans design outcome goals for programs and ultimately use findings to make improvements; there is institutional support for the design of instruments, implementation and deployment of instruments, and the collection and reporting of findings in a coherent, usable fashion. Macro-level assessment entails the design of broad outcome goals at the Administrative Cabinet and Strategic Planning Committee level, using findings to make broad University improvements.

As a result of our work in preparing this reflective essay, we have developed the following recommendations to the President. Upon receipt of these recommendations, the President will develop cooperatively a prioritized list of all EER recommendations, assign each to a responsible person or group, and establish a timeline for action. As appropriate, these recommendations will be the focus of review by the Strategic Planning Committee for funding consideration, as needed.

See EER Group One: Assessment.

Recommendations

(See Recommendations to the President)

1.1 Improve data collection systems, both in efficiency and effectiveness.

1.2 Improve data interpretation and analysis.

New Program Development

A second aspect of verifying quality at an institution is the establishment of appropriate mechanisms for addressing new program approvals. This is especially important as the University attends to program quality improvements at the undergraduate level (while limited in a capped environment on the main campus) and improves guarantees of quality to new programs at the graduate level, with the challenges of infrastructure, assessment of learning, and quality controls. The following reflective essay looks at new program protocols and then examines two new program approval cases-in-point: the undergraduate major in International Development Studies, and the graduate program in General Biology. This reflective essay  directly addresses CFR’s 2.2, 3.8, 4.2, and 4.4. In addition, it indirectly speaks to issues in CFR's 2.3, 2.4, 2.7, 2.13, 3.1, 3.3, 3.5, 3.7, and 3.11 .

The Institutional Culture

As any educational institution evaluates its opportunities, there is occasion to add new programs to the curriculum. This process must be done with care to ensure alignment with institutional mission, appropriate financial and staff support, and with the intent of enhancing the quality and depth of student learning in the institution. How does a new program move from an idea to an institutionally adopted curriculum? And how is the institution guaranteeing the integrity, appropriateness, and mission-fit of new programs as they enter the stream of curriculum offerings at the institution? Clearly, the answer to these questions resides in a number of procedures and policies both formal and informal.

Point Loma Nazarene University has been in a season of robust health at the undergraduate level since reaching the City of San Diego-imposed undergraduate enrollment cap at the turn of the Twenty-First Century. In addition, the past several years have seen rapid expansion at the graduate level both on the main campus in San Diego and in the Regional Centers as far north as Bakersfield. As proposed in the Institutional Proposal, in such a robust environment, it is appropriate to examine and analyze current protocols for the development of new programs. While the beginning ideas and plans for a new program are usually generated as a part of the University’s program review activities (see EER reflective essay on Program Review), and ongoing quality assurance is a part of the PLNU’s assessment efforts (see CPR and EER reflective essays on Assessment), a key element in evaluating the effectiveness of the process for developing new programs centers on the adoption phase of these programs. This reflective essay proposes to determine answers to the following questions: What are the protocols for the development of new programs? Are quality assurance processes built into the process? and, How are the programs assessed and modified based on those evaluative findings?

New Program Protocols. In the Point Loma governance structure, proposals for new programs are evaluated through the Academic Policies Committee (APC) for undergraduate programs or the Graduate Studies Committee (GSC) for graduate programs. In each case, there are two different processes involved. First, each program must be evaluated against a set of established criteria to ensure that quality issues have been thoroughly addressed. Second, a separate set of protocols evaluate the necessary steps for adopting the new program into planning and organization at the University.

An initial review of these two important processes revealed a divergence between the undergraduate and graduate structures. The undergraduate process had a clearly defined adoption process but no stated criteria for proposing a new program. Ironically, the graduate process had clearly defined criteria through which a new program would be added but lacked a clearly stated adoption process. This governance issue was resolved in the fall of 2006 by the careful work on both the APC and the GSC to adopt parallel processes on both fronts (see Criteria for New Undergraduate Programs, Adoption Procedure for New Undergraduate Programs, Criteria for New Graduate Programs, and Adoption Procedure for New Graduate Programs). The faculty approved these changes in its October meeting (see Faculty Meeting Minutes 10/18/06).

In spite of the fact that these two important components of new program addition did not exist as formal documents, review of recent additions, at both the undergraduate and graduate level, demonstrate that there was an intuitive understanding of both quality criteria and adoption into institutional planning initiatives. To conclude this reflection, these new programs are then evaluated in terms of quality assurance and positive impact on student learning within the context of the institution’s mission and values.

Evidence and Analysis

To assess how the two processes described above contribute to quality assurance and student learning, we conducted a careful review of two recent additions to the curriculum: the undergraduate major in International Development Studies in the Fermanian School of Business and a master’s degree in General Biology.

Baccalaureate Degree in International Development Studies (IDS). The faculty of the Fermanian School of Business began in 2000 to revise the Economics major, a program that had been characterized by very low enrollments. First, the major was converted into an unsuccessful Economic Development major, and finally into a successful International Development Studies major and minor (see Enrollment Patterns in Three Business Majors). In the late 1990’s, the declining enrollment of students in the Economics major required that the School of Business either eliminate the major, offer under-prescribed courses at a loss to the University, or redesign the major in a way that retained the basic core of the Economics major and yet attracted a wider range of the undergraduate student population.

The initial attempt to redesign the major was to take the core components of the Business major and add a small set of Economic Development courses. This approach left the School with a degree program characterized by an oversized number of required courses that did not attract students desiring to major in Business or Economics.

Following this unsuccessful revision, the faculty set out to identify a market niche that would be a strong mission fit as well as complement the Business major. The faculty conducted extensive research into a variety of programs across the nation, considered options with a faith and service components, visited other campuses (including Chalmers Institute, Covenant College and School of International Leadership and Development, and Eastern University), and gathered informal perceptions from then-enrolled business students. The conclusion was that a strong International Development major and minor would require adding Economics courses and eliminating the series of Business courses that did not directly contribute to desired student learning and service outcomes in International Development. The School also believed that the new IDS minor would complement other majors including, Sociology, Political Science, Education, and Nursing.

The International Development Studies major and minor show a potential for success and have led to a new dimension within the Fermanian School of Business that enhances its mission fit and service goals. While the major is only beginning to generate graduates, some early anecdotal indications suggest that it has potential for significant impact on student learning. For example, in the spring of 2007, Economics professor Senyo Abjibolosoo spent a sabbatical in Ghana, working with two students majoring in International Development. The impact of this field learning is long-lasting (see the Ghana Newsletters). Similarly, the work of the Armenian Center for International Development has put students in learning situations that shape their activities beyond Point Loma (see the 2005-2006 PLNU Annual Report featuring the IDS major).

Master’s Degree in General Biology. The master’s degree in General Biology (M.A. and M.S.) was suggested by the Department of Biology as part of their ongoing commitment to educational outreach. In 1997, Point Loma launched the University Now program, designed to introduce college-level biology to high school students in underprivileged schools in local San Diego public schools. The program includes an invitation to students to spend time during the summer on Point Loma’s main campus working in the laboratories of the Department of Biology. This program was initially supported by a grant from the Howard Hughes Medical Institute. Between 1997 and 2003, 81 students participated in University Now. The program was evaluated by tracking the number of participants who went on to attend college. Summary data for University Now demonstrates that a remarkable 80 of 81 students went on to post-secondary education, supporting claims that student learning and future aspirations were both impacted by the program.

As the University Now program got underway, it became clear that support of science teachers in the public schools would be the next logical step. In 1998, the University launched the Perspectives on Science program (POS), a series of lectures that connect San Diego science teachers with high-level researchers. Feedback from the POS participants pointed out that there were no appropriate master’s-level programs in the field of biology in the San Diego area. Existing evening programs were only in Education (not Biology), and Biology programs were only offered in daytime formats. A survey of POS participants in 2003 showed interest in a master’s degree in Biology for public school science educators offered by Point Loma.

Faculty created a steering committee that included experts within the University’s Department of Biology and the School of Education, together with representatives from the public school community, to design a conceptual framework for a master’s degree. In 2003 the faculty then presented this initial structure to the Provost and Chief Academic Officer. Following a dinner with a POS focus group (see focus group notes), faculty reshaped the conceptual structure and presented it to key individuals within the science education community in San Diego. This resulted in a model that provided more flexible entry points in place of a more rigid cohort model.

In order to test this initial model in a market research study, faculty leaders enlisted the assistance of a team headed by a faculty member of the Fermanian School of Business who specializes in marketing. The test included the mailing of surveys to members of the San Diego Science Alliance. Given general support from the surveys and the critical mass of PLNU science faculty with background in science education, the faculty developed a formal proposal and submitted it through regular channels to the GSC. This was followed by a series of informational meetings between the GSC and Biology faculty, addressing general concerns about the launch of a new program. The faculty gave specific attention to the criteria for adding graduate programs discussed above. At the same time, faculty and the Provost presented the general concept to the Administrative Cabinet. Discussions with the Provost proceeded periodically with the assumption that full plans were being shared with the Cabinet and that any concerns they had were being incorporated into planning.

The GSC approved the proposal for a master’s degree in General Biology on November 19, 2004 (see GSC minutes). This action went to the full faculty for approval on December 8, 2004. At this point, it became clear that while the Cabinet had general knowledge of the proposal, a full review of financial requirements and enrollment models had not taken place. Working closely with the Vice President for Finance and Administrative Services, we determined that what was needed included a two-year model of enrollment that yielded sufficient contributions to overhead costs, following a start-up period.

The program began in the summer of 2006 with an enrollment of 13 students. In the initial modeling of the program, faculty leadership assumed that most students would opt for a non-thesis option. As students entered the first year of the program and looked toward completion in the second year, however, it became clear that this assumption was in error; nearly three of four students opted for a thesis. This had a positive impact on the revenue picture for the program but also demanded more faculty oversight. This oversight called for the hiring of an additional faculty member in Biology with a background in science education to assist with this demand. Revenue projections resulting from increased enrollment and the more popular thesis option allowed this additional hire without placing undue burden on the financial strength of the new program. As the program looks to its second recruiting class and moves toward long-term sustainability, outcomes assessment plans will allow for further improvements of this program, improvements designed to strengthen public school biology teachers in the San Diego market and beyond (see the process of approval narrative for further details).

Impact on Program Quality and Enhancement of Student Learning. The PLNU Institutional Proposal (page 10) attempted to make a connection between new program addition and program review, quality assurance, and improved student learning. It is for this purpose that we conducted a careful examination of new programs.

The specific review of the International Development Studies major and the master’s degree in General Biology demonstrates that both of these new programs were in some embryonic form at the time the Institutional Proposal was written. When these programs were in development, not the new program documents, nor the conversations at the APC or GSC levels, nor any Administrative Cabinet level conversations required a new program proposal to make explicit assumptions about enhanced quality or enhanced student learning.

For example, in the new International Development Studies major, it was far more likely that potential impact was stated in terms of strengthening a weak major. We note that building enrollment does not immediately correlate with quality concerns. In terms of ensuring program currency in light of institutional mission and core values, the revision in a new major does infer improved student learning, but the focus of the curriculum revision was one built out of best practices as known by the faculty.

By contrast, the master’s degree in General Biology can trace its origins to deep concerns about student learning. Beginning with underprivileged high school science students in local public schools, the concerns moved on to the learning outcomes of their classroom teachers. It does not appear that these intended outcomes were ever quantified and made part of the ongoing assessment processes of the University, in large measure because the program has not yet had any students complete the program.

Because the faculty members overseeing both the International Development Studies major and the master’s program in General Biology are deeply committed to the mission, values, and ethos of Point Loma Nazarene University, there is a great deal of confidence that awareness of program quality issues and student learning outcomes runs deep. As the University continues on its path toward embracing continuous quality improvement, these issues should become more explicit.

Summary of Conclusions

In our review of new program development, we have learned several lessons in our analysis of the protocols for the development of new programs at the University. We have identified four examples of institutional learning in this analysis. These include: the necessity of aligning parallel process for the undergraduate and graduate levels, the value of establishing specific measures to be addressed in new program development, the need to articulate procedures particularly for programs housed in Regional Centers, and the challenges of addressing new program development in the context of a fully built undergraduate environment.

Parallel Processes. The first lesson learned in the preparation of this reflective essay is that PLNU needs to ensure that the parallel processes of new program development for undergraduate and graduate programs are sufficiently aligned with one another. While the addition of a graduate program in a Regional Center may face very different dynamics than the restructuring of a program on the main campus, we must make certain that we ask the same critical questions and sufficiently answer them before a program is added.

Outcome Specificity. A second lesson is that we can only evaluate what we specifically ask new programs to address in their planning. If we focus on improved quality and enhanced student learning for new programs in the future, we must find ways of making these expectations more explicit at the earliest stages of program planning. In many ways, the language of intended outcomes is a new vocabulary to many faculty members, and we will need to provide training in how to ask and answer the critical questions. In a related fashion, we need to make sure we build evaluative processes along the way around the measures of program quality and student learning.

Review and Approval Process. A third lesson is that we need to be able to articulate the review and approval process for our Regional Centers in ways that are within the spirit of off-campus programming. Both of the programs added recently were housed on the main campus in Point Loma. The practice for the Regional Centers may follow a similar course but reflect some uniqueness. For example, we have recently learned that adding an existing program offering (e.g., a reading certificate) to a Regional Center when it has been modeled at another location may not constitute a new program but simply a new implementation. On the other hand, when a new program is added at a Regional Center, it must fit within the long-range planning for that location and in no way detract from its overall commitment to program quality.

Change in a Capped Environment. A fourth lesson speaks specifically to the challenges of new program development on the main campus in Point Loma. Because the University operates under a City of San Diego-imposed enrollment cap of 2,000 FTE students, we can no longer suggest new programs with the promise of enrollment growth. It may be more helpful for the process to involve explicit assumptions about program modification that speak to program quality and student learning. In the world of limited resources that is higher education in the 21^st century, it may be necessary to think about ideal program balance that involves moving resources to maximize quality and student learning across the curriculum.

As a result of our work in preparing this reflective essay, we have developed the following recommendations to the President. Upon receipt of these recommendations, the President will develop cooperatively a prioritized list of all recommendations, assign each to a responsible person or group, and establish a timeline for action. As appropriate, these recommendations will be the focus of review by the Strategic Planning Committee for funding consideration, as needed.

See EER Group Two: New Programs.

Recommendations

(See Recommendations to the President)

2.1 Develop explicit statements concerning expectations of enhanced quality and improved student learning as part of new program proposal processes.

2.2 Review new program development and adoption processes to articulate the relationship to program review protocols.

2.3 Evaluate the impact of the main-campus enrollment cap on the nature of new program development.

Program Review

A third factor in Point Loma’s efforts with regard to quality assurance is a system of academic program review, a valuable and productive exercise that rigorously tests our commitments to procedures, policies, and standards. The purpose of this reflective essay, as stated in the Institutional Proposal , is to rehearse briefly the historical development of program review efforts at the University, demonstrate the value of these protocols, and research the assertion that program review assists the units to improve program quality and student learning. This reflective essay directly or indirectly addresses CFR's 1.2, 1.3, 2.2, 2.4, 2.6, 2.7, 4.2, 4.4, 4.5, 4.6, 4.7, and 4.8.

The Institutional Culture

Establishing a Formal System. Plans for a formal system of academic program review began at PLNU in 1990 when the University established the position of Dean of Liberal Arts, now called Dean of Arts and Sciences. The appointment of an experienced and respected faculty member to this position prompted subsequent visits by the newly appointed dean to Calvin College and Bethel University (MN) for an examination of program review practices at these institutions. From those visits, the dean determined that the concept of program review offered the possibility of a significant and formalized quality assurance process characterized by a systematized cyclical schedule, helpful indicators of program effectiveness, and information that aided in monitoring growth by academic unit.

Following this field research during the 1992-1993 academic year, the dean drafted a program review document for consideration at Point Loma Nazarene University. The plan, submitted to unit chairs for comment, outlined directives for PLNU’s proposed program review processes. Finally, a formal proposal for a system of review program received approval in 1994 from the academic leadership team, made up of deans and department chairs. In addition, the University established an ad hoc committee to develop review strategies. Following that, program review cycles became an integral part of our efforts to evaluate academic programs at the University.

Subsequent Developments in Program Review . It was the expectation at the inception of the program review design that each academic unit would complete a program review every ten years (a first round) with a modified review five years later (second round). Four academic areas at PLNU were the first to conduct the initial program reviews beginning in 1995-1996: Biology, Business, Chemistry, and History/Political Science. The final department completed its first round review in 2001-2002. According to established policy, no curricular or program changes could be made until first-round program reviews had been completed, providing a significant motivation to all concerned.

The first round in the University’s program review model provides specific content and directives for the academic units (see Program Review: Cycle 1). Additionally, essential questions invigorate the reviews such as the health of unit functions, identification of standards of excellence, status of learning outcomes, future directions, mission fit, and the results of assessment.

The University introduced the second round of review (Cycle 2) in 2001. This second round, slightly modified from the first round, functions as a quality assurance assessment conducted by academic units five years after the first round. In some cases, second-round reviews have become the means of fine-tuning changes made after the first round. In other programs, units have made significant changes following the second reviews (see 2006 Program Review Survey Respondent Comments). In the interim, departments and schools began to engage in more informal assessment strategies (see 2006 Program Review Survey Respondent Comments). By 2002, the University took steps to formalize these strategies using the Nichols Assessment Model (see Institutional Proposal Appendix B and Assessment Committee Report to the Faculty, November, 30, 2005).

In preparation for this reflective essay, we assembled data on PLNU program review and quality assurance processes from three key sources: interviews with University administrators involved with program review development and oversight, program reviews and self-study documents, and a 2006 Program Review Survey: Part I and Part II (see also the Instructional Letter).

Evidence and Analysis

Though informal approaches to addressing program quality certainly have existed at the University for decades, our reflection during the Educational Effectiveness Review on processes utilized across the years affirms a thoroughly tested yet evolving approach to program review, especially in more recent years. Since the initial action of academic leadership 12 years ago in designing protocols for evaluating academic units and their effectiveness, program review has matured and developed into an efficient and profitable discipline. To discover more literally the patterns of use, benefits, and concerns relative to these processes, we recently surveyed all academic units at the University.

Improvements in Program Quality. Our survey data indicates that program review at PLNU functions reasonably well, involving a majority of academic units. Indeed, most understand the expectations of program review processes, recognize the value of systematic reviews, and can claim concrete outcomes as a result. For example, 82 percent of academic units report updated curricula and restructured majors, while 76 percent added new courses with 65 percent eliminating obsolete courses. Eighty-six percent of respondents agree that program reviews are a necessary aspect of maintaining quality; 78 percent of respondents recognize the inherent value of program review “as a means to promote academic quality at PLNU” (2006 Program Review Survey Respondent Comments). Most review experiences involved at least one and sometimes two external evaluators from respected institutions, and they resulted in significant changes to programs including, at times, important new hires in specific areas. In sum, program review as a quality assurance process has produced improvements in personnel, in curricular offerings, in increased attention to state and national standards, in the expansion of community partnerships, in the job opportunities for students, and in improved acceptance rates at the graduate school level.

Two Examples of Program Review . In an effort to analyze more closely the process of program review at Point Loma, we looked at recent studies conducted by two academic units: the Department of Family and Consumer Sciences (College of Social Sciences and Professional Studies) and the Department of Art and Design (College of Arts and Sciences).

Program review in the Department of Family and Consumer Sciences (FCS) led to student learning improvement through better knowledge and better preparation for careers. The first program review in 1999 generated extensive curriculum overhaul, producing a curriculum with greater depth and breadth. Subsequent outcomes assessments of student knowledge revealed a general upward trend in scores.

The first program review also began a trend of on-going review. In 2003, the Department also examined the Early Childhood Learning Center academic program, resulting in significant changes. In that same year, FCS wrote a proposal for accreditation by the American Dietetic Association (ADA), and submitted an application for eligibility in 2004. Subsequent to the self-study required for candidacy, Point Loma was approved for a Didactic Program in Dietetics (DPD) in 2005.

A combination of the program review process, on-going review, and student outcomes assessment contributed to changes leading to improvements in student learning. The FCS senior seminar contains an outcomes assessment component to solicit feedback on the program by graduating seniors. FCS has used this student feedback to improve program experiences which will better prepare students for professional practice, including more field trips and guest speakers. These focus groups also revealed the need to introduce the philosophy of the discipline of FCS earlier in the students’ program. The Department will be submitting a proposal to the Academic Policies Committee (APC) to make this change in the 2007-2008 academic year.

Program review in Department of Art and Design has led to better quality student artwork and an improvement in students’ ability to articulate intellectual thought, or reflection on their work. The first program review in 2001 laid out a 5-year plan with recommendations to expand variety in the curriculum and in the diversity of coursework. The Department proposed restructuring in the major, the addition of upper-division coursework, and the embedding in the curriculum of an increased focus on the conceptual content of artwork. As an example of the impact on student learning, a course in 20^th century art history was added to the curriculum and within a year following the change, students were able to reflect intelligently on their own artwork within the deeper context of recognizable 20^th century artists.

Outcomes assessment data showed a significant quality improvement in student artwork production. As all students’ senior exhibits are evaluated by a panel of professional external reviewers, there was a noticeable improvement in the comments of reviewers over the time since the Department instituted changes due to program review, including recognition that, in many cases, Point Loma senior art exhibit work is as good as graduate students’ artwork elsewhere.

It should also be mentioned that the data suggested there are a variety of ways that analyses of quality are accomplished at the University outside normal program review or the Nichols assessment model. Some units utilize other forms of quality inventory, review, and control for review and assessment purposes. Some academic units (e.g., the School of Nursing) use external accreditation self-studies in place of a traditional PLNU program review. Some centers, whose programs are markedly distinct from traditional aspects of the academic department or school, have approached program review in the context of rigorous assessment efforts, without adapting the measures typical of reviews at the present time (see EER reflective essay on Assessment).

Concerns Related to Program Review. While we can boast of many things concerning the utilization of program review in assuring ourselves of quality, some concerns have arisen during our writing of this reflective essay. These concerns include the definitions, participation levels, and coordination of efforts with respect to external accreditations and the University’s assessment program.

Some respondents to our survey indicated that the meanings and applications of some terminologies have blurred with the development of a robust assessment program at the University. The definition of program review may need clarification as to the academic units to which the cycles apply, such as University centers. Furthermore, to some survey respondents a better definition of the distinctions between first and second rounds may be needed. This may call for a general updating or design revision of the PLNU program review cycles.

Relative to participation concerns, the survey of academic units revealed some gaps in both reporting and participating in standard program review activities. While the majority of units did participate in significant ways, there were some areas in graduate studies that apparently did not. In some cases, this could be due to their use of external standards such as those required by the California Commission Teacher on Credentialing (CCTC). Likewise, some University academic centers conducted modified reviews, such as the Wesleyan Center, the Institute of Politics and Public Service, and the Early Childhood Learning Center.

There may also be some work to be done in understanding the coordination of program reviews and our continuing annual efforts in assessment. As new assessment expectations filtered back to academic departments and schools, faculty members sometimes confused these quality assurance/educational effectiveness efforts with simply another version of “program review” (see 2006 Program Review Survey Respondent Comments). In time, because academic units were scheduled to conduct program reviews in groups of four to five per year over a five-year period, some had not completed reviews while others had already moved on to specific assessment efforts, followed by the second round of program reviews. We faced the challenge of coordinating our efforts, particularly for units that had also written self-studies for external professional accrediting agencies within the same five-year time periods. The timing of program reviews could be better coordinated with the reaffirmation of program accreditations and/or the complications of synchronizing with other units that shared courses. Also, since some academic units avail themselves exclusively of external accreditation procedures, it is possible that certain expectations of PLNU program reviews even could go unaddressed.

While the College deans suggested that the second round of program reviews seem not to be as effective as first round ten-year reviews, due perhaps to the fact that units are not required to complete their five-year reviews before curricular changes can be made, 50 percent of academic units felt second rounds were of equal value to their first reviews. Some indicated that the second round continued the culture of evidence in their respective units. In the interim years, some units have conducted extensive reviews for external accreditations, prior to making curricular changes (Department of Music, the Fermanian School of Business, Department of Family and Consumer Sciences).

Pending Issues. We gathered findings on the effectiveness and quality assurances of program review through the 2006 WASC PLNU Program Review Survey. These responses are itemized and examined in Program Review Committee Study, Summative Findings, Program Review Chart and 2006 Program Review Survey Respondent Comments. Survey questions and resulting data addressed the charge, “To what extent are PLNU quality assurance processes actually improving program quality?” (Institutional Proposal, page 8). The weakness of this particular survey instrument is that it focused on the faculty’s informal conclusions and assumptions concerning the perceived effects of program review. As such, their assessment is very limited and provides only an input in this regard. We know that, while there are positive programmatic comparisons with respected institutions, we must engage multiple measures of assessing student learning that not only complement the capacity concerns of program review but also point to the educational effectiveness toward which all review benefits must point. Each round of program review was followed in interim years by an increasing number of assessment strategies to measure program effectiveness and student learning outcomes (see Institutional Proposal Appendix B and Assessment Committee Report to the Faculty, November, 30, 2005). These assessment efforts were directed through the PLNU Office of Educational Effectiveness, which collected data to address the Institutional Proposal question, “What is the evidence that student learning outcomes are improved as a result?”

Summary of Conclusions

The University’s system for academic program review has been in place since 1994, providing a valuable quality assurance function. We have demonstrated that this system (ten-year reviews followed by a five-year review) is firmly institutionalized in academic departments and schools and is, in fact, an expectation in on-going efforts for improvement. This process has brought qualitative changes to many programs since its beginning.

We can assert that Point Loma Nazarene University has developed a robust climate of program review and assessment since the last WASC visit in 1997. Administrators and academic leaders are committed to quality assurance processes and are actively involved in fostering these efforts at PLNU. Whether they utilize external accreditation standards or choose to compare themselves with respected institutions nationwide, all PLNU academic departments and schools are able to reach specific conclusions from their program review efforts, citing program strengths and weaknesses. Each program review document demonstrates serious self-study activities ending with recommendations for future development, including specific program changes (see Mathematical, Information and Computer Sciences Program Review for a sample program review and further documents in the Team Room evidence display).

As a result of our work in preparing this reflective essay, we have developed the following recommendations to the President. Upon receipt of these recommendations, the President will develop cooperatively a prioritized list of all EER recommendations, assign each to a responsible person or group, and establish a timeline for action. As appropriate, these recommendations will be the focus of review by the Strategic Planning Committee for funding consideration, as needed.

See EER Group Three: Program Review.

Recommendations

(See Recommendations to the President)

3.1 Study the current program review plan, addressing primarily the clarification of definitions, participation, and coordination with other quality assurance efforts.

3.2 Redesign program review processes to create a focus on the improvement of student learning, including external review mechanisms and robust assessment efforts.

3.3 Consider establishing exemplary program review formats in order to provide a selection of models accommodating differing profiles of departments, schools (including interdepartmental majors), academic centers, and reviews that flow out of external accreditations.

Institutional Themes

Introduction. After reviewing the four recommended approaches to the Educational Effectiveness Review, we chose the Special Themes Approach (as described in the WASC 2001 Handbook of Accreditation) believing that it held the most promise to assist us in accomplishing our intended institutional outcomes for this review. At the outset of our new strategic planning process several years ago, we took more than an entire year to develop a new mission statement. It is as follows:

Point Loma Nazarene University exists to provide higher education in a vital Christian community where minds are engaged and challenged, character is modeled and formed, and service becomes an expression of faith. Being of Wesleyan heritage, we aspire to be a learning community where grace is foundational, truth is pursued, and holiness is a way of life.

A short version of this statement emerged over time in the phrases “To Teach, To Shape, To Send.”   Since one of our intended outcomes from the review process is to “ensure that the new mission statement is embraced, shared, and embodied throughout the institution,” we decided to use these categories as themes to frame a large portion of the Educational Effectiveness Review.  We were careful to ensure that the analytical essays for each theme went beyond a description of activities to include analysis and reflection on the evidence that the institution has collected about teaching students (learning), shaping students (growth), and sending (service)—both while enrolled as actively engaged men and women and as alumni.

Since these themes encompass such a broad spectrum of activities and outcomes, the Steering Committee drafted several guiding questions to focus the investigation and subsequent discussions (see Institutional Proposal). These guiding questions identified specific areas of investigation, questions for engagement, and initial sources of evidence to guide the writing of the reflective essays. While these questions for engagement provided some general guidance, as we moved to the actual construction of the report we made two important discoveries. First, a review of the questions shows a much clearer statement of intended outcomes in the To Teach arena than in the others. Second, our interest in focusing on outcomes rather than process, or simple capacity, highlighted the reality that we have not operationalized in a systematic way all the questions in ways that would allow a compendium of data sources to inform our evaluation. Addressing each of these discoveries will be an important component of the post-EER work at Point Loma.

Theme One: To Teach

The first institutional theme that emerges from our Mission is TO TEACH. This aspect of our institutional purposes establishes for us the fundamental focus of teaching excellence and scholarship among the faculty but also the mirror of student learning and intellectual growth. In this reflective essay, we consider the institutional context, evidence and analysis, a summary of conclusions, and recommendations for action and improvement. This reflective essay addresses CFR’s 1.2, 1.6, 2.3, 2.4, and 2.6.

The Institutional Culture

Point Loma Nazarene University seeks to be a teaching/learning community " where minds are engaged and challenged " and " truth is pursued ." These key phrases from the institution’s Mission Statement  plainly express our commitment to teaching as central to the mission, the structures, and the daily life of the University. This essay, in keeping with our purposes as stated in the Institutional Proposal, seeks to elaborate on the University's teaching mission and the student learning that results from our efforts.

We can cite several descriptive facts concerning the value of teaching at the University. Teaching is the first-mentioned of PLNU’s eight Core Values. In addition, teaching is embedded into the evaluation and reward structure of the University. Student evaluation of teaching is required of all faculty members yearly and peer evaluation of teaching is a part of the full evaluation profile that faculty regularly submit. Moreover, an important factor in the granting of sabbaticals is the applicability of the project to the candidate’s teaching. Effective teaching is also required of faculty members for  promotion and tenure.

The University supports teaching through a variety of internal centers, grants, and programs. New faculty members attend an intensive orientation to PLNU through a day-long New Faculty Orientation at the beginning of the academic year and through the New Faculty Seminar, a faculty course that is offered in a 2½ hour-a-week format to faculty in the fall. The seminar covers a variety of topics in its 15 weekly meetings, but fully one-third of these deal with teaching issues. Additionally, an Adjunct Faculty Orientation also attempts to provide new part-time faculty with a portion of what is experienced in the New Faculty Seminar. The University also provides internal grants from the Wesleyan Center  for Twenty-First Century Studies, the Alumni Office and the Center for Teaching and Learning (CTL) for faculty members engaged in certain kinds of course development.

The CTL has sponsored ongoing programs to improve faculty teaching since the 2001-2002 academic year. These programs are accessible to all faculty members, full-time or adjunct, new or experienced. The CTL provides teaching tips, offers support for attendance at pedagogical conferences, directs regular reading and discussion groups, arranges for sessions on the pedagogical use of technology, and supports initiatives such as service learning, writing in the disciplines, and active learning.

Although support systems and structures are critical to the encouragement of excellent teaching, the actual effectiveness of teaching cannot be demonstrated by the existence of these systems. There are, however, some measures separate from support systems and structures that can be utilized to determine the general effectiveness of teaching at PLNU. The IDEA evaluation service, that provides the student evaluation form for the majority of PLNU faculty, recently provided student evaluations for all PLNU courses during each of five consecutive academic years, from 2000-2001 to 2005-2006. Looking particularly at objectives, progress toward objectives, excellence of teacher and excellence of the course, we can see in this Combined IDEA Data a general trend of increased effectiveness of teaching.

This composite data indicates that faculty are making application of course material or analysis and critical evaluation an increasingly important factor among their course goals (faculty using the IDEA form are asked to select three to five of their top objectives).  We cannot assume that the absence of particular objectives means that they are of no importance to the professor since the instructor selects from a range of possible objectives. More meaningful, however, is the strong student self-report of significant gains toward intellectual objectives. PLNU students indicate a stronger sense of intellectual progress than do the students in the IDEA system at large, while only in the area of development of creative capacities do PLNU students score below the average. Most important, PLNU students have shown increasing gains in higher intellectual skills such as application, analysis, and critical evaluation of knowledge.

The only solid measure of teaching effectiveness, however, remains the student learning that results. We therefore re-conceptualized the question about the effectiveness of faculty teaching as a question about the adequacy of student learning, especially student learning in central areas (general education and majors), in areas about whose value there is consensus among faculty (writing, quantitative literacy, critical thinking), and in areas where we have a high investment because of mission (religious values, service, teamwork, and diversity, both domestic and international).  In addition, we wanted to examine the effectiveness of our teaching in view of the purpose of preparing students to enter the world of working professionals.

We also wanted to examine faculty growth in the areas of teaching, research, and faculty/student collaboration. As experienced practitioners, we understand that teachers are made, not born. Faculty members who constantly work at improving their teaching often grow into an excellence that may not have originally characterized their work with students. Likewise, faculty actively engaged in research, especially collaborative research with students, are productively mentoring students’ independent and creative academic work.

Evidence and Analysis

Teaching in General Education . We assessed student learning in general education through a series of three summative exams administered one per year over three academic years (2004-2006). These exams, carefully described in the University’s CPR Group Two reflective essay, focused on skills such as writing, quantitative problem solving, and critical thinking as well as on informational and attitudinal areas like acceptance of diversity, whole-person thinking, development of a personal set of religious values, and understanding of the basic tenets of Christianity.

The assessment indicated that we are having mixed success on our desired goals. On the positive side, students learn to do quantitative reasoning, to understand the concepts of Christianity, to work cooperatively in teams, to do service learning projects, to think about their lives holistically, and to interrogate an issue from at least three different perspectives. However, it appears that our students may need to be pushed harder to develop critical thinking skills.

In general, students met expectations on easier tasks that demanded information or recognition skills, but did less well on tasks that demanded application, analysis, synthesis or reasoned evaluation. Nowhere was this more apparent than on a holistic writing exam administered in April, 2004, to 43 seniors. The  results show that students did relatively well at simple tasks such as selecting a topic of social significance (88 percent pass), stating a clear and coherent thesis (51 percent pass), and bringing the thesis into the paper in a timely manner (84 percent pass). On measures of logical, critical thinking, however, the results were more disappointing. Few of the students tested could divide a main idea into logical sub-points (28 percent pass), bring appropriate evidence to bear (26 percent pass), or deal with a logical objection (33 percent pass).

The gap noted above between student success at easy tasks and struggles with more difficult ones resurfaced on the 2005 general education exam, as reported in the reflective essay for CPR Group Two, in questions assessing students’ grasp of religious concepts and their ability to think from multiple perspectives (see exam results). On the religious concepts question, 84 percent of students tested could do the simplest task of identifying three appropriate passages in a political speech where the speaker was casting America or Americans in the role of Christ and 72 percent could properly identify the attribute of Christ being referenced; only 47 percent could explain the application to America. In an earlier exam, only 44 percent of students had been able to articulate how Christian beliefs could be applied to a specific issue (see exam results). Similarly, on the 2005 exam, 70 percent of students could interrogate a complex issue from at least three different disciplinary perspectives, but only 51 percent of students could explain why different specific questions were appropriate to different points of view (see exam results). While our assessment data does indicate that students are learning basic Christian concepts adequately and leave PLNU with a reasonable capacity to think from different viewpoints, the more detailed data consistently suggests that we need to provide students with more opportunities to practice higher levels of thinking.

There is some evidence that suggests that approximately 30 percent of our students have substantial contact with another culture or sub-culture  (see cross cultural participation data and CSS cultural contact survey results). Indeed, there is anecdotal and survey evidence that some of our students who study and/or do a mission trips abroad make important strides in developing tolerance toward other national cultures. The results of the general education assessment exam of 2006, as noted in CPR Group Two, administered to 256 students, however, suggest that these lessons may not be sufficiently transferred and applied to tolerance toward domestic diversity. Indeed, only 28.8 percent of students were able to understand a given problem from a minority racial position and only 25.3 percent of students showed empathy toward that position. The results of the general education exam coincide with self-report survey data culled from the College Senior Survey (CSS) and the National Survey of Student Engagement (NSSE).

The diversity results do not indicate the absence of diversity issues in the curriculum. Diversity issues are a topic in the Psychology 101 convocation required of all PLNU first-time freshmen, and in that same class all freshmen participate in small groups purposefully created to be embodiments of diverse cultures. But an open attitude toward diversity, like skill in writing, is probably not acquired through one course alone. Both probably necessitate numerous exposures and repeated opportunities to learn.

Teaching within the Academic Major . If student intellectual engagement in general education purports to develop broadly-based thinking skills, study in the academic major has a more focused goal of preparing the student for the world of working professionals in specific fields. We thus examined how successfully we were providing students with the knowledge and the skills needed to enter the world of professionals in their field—whether that world was constituted by the world of work or the world of advanced professional study. Finally, since we feel that inclusion of the spiritual and moral aspects of life constitutes part of the Point Loma Nazarene University distinctive, we wanted to discover whether we do empower students to bring an awareness of moral and ethical issues to their work world.

The common learning outcomes table for PLNU majors reveals that knowledge is a primary goal for our academic majors. Fully 80 percent of academic major that have managed to gather data found that their majors were indeed learning to the level of competence they deemed appropriate and another 17 percent were unsure about the meaning of the data they had (see the analysis of findings). Results were equally positive for student mastery of technical skills and for student demonstration of knowledge through writing. 

Results also seem generally positive for professional preparation, selected as a goal for 45 percent of major programs (see common learning outcomes table). Although many programs are still working at collecting and interpreting data, in the overwhelming majority of programs with usable data students were well prepared for entry into a career or into a professional program. In only about four percent of major programs did we discover that students were not well prepared for the world they encountered after graduation.

Data provided by the Assessment Committee show that academic majors cite six common teaching outcomes (common outcomes): knowledge, technical skills, production of professional work or performance, writing skills, critical thinking skills, and dispositions or attitudes. Of 254 outcomes cited across 64 academic programs, 108 (43 percent) are in the teaching category. Of the 108 cited teaching outcomes, data exists (Data on Common Learning Outcomes by Department) on 76 (70 percent), and of the 76 teaching outcomes for which we have data, 50 (66 percent) have met outcome targets (Analysis of Common Findings by Outcome).  If the 254 outcomes are broken down according to graduate and undergraduate (Data on Common Learning Outcomes for Graduate and Undergraduate Programs), the teaching outcomes cited by graduate areas is a slightly lower frequency (10 out of 25, or 40 percent) relative to undergraduate (98 out of 229, or 43 percent).

This data reflects a significant emphasis on the teaching mission of the University by academic majors (43 percent), good data collection efforts (70 percent), and good success in achieving our teaching outcome targets (66 percent).

Far more mixed results emerged from our study of students’ ability to engage with the ethical and theological issues of their discipline. These issues are handled differently in different schools and academic departments. In some disciplines, these issues are woven into a variety of courses; in others, individual courses have been developed with a primary ethical and theological focus (see list of courses). In addition, schools and departments offering internships for their students involve those students in very practical experiences where ethical and theological issues are addressed. Those majors that did include faith and learning issues among their goals and that did have data to be analyzed were either positive or uncertain as to the meaning of their data. Sixty percent of those majors found that their students were able to see faith and knowledge in appropriate ways (see the analysis of findings). Although this result is good, the large number of majors that found testing of this aspect challenging speaks to the need for some conversations in the University community about how to test the very things we care most about.

Off-Campus Learning Experiences.  As a supplement to its general education program and its schedule of academic majors, PLNU has encouraged students to study off-campus in approved international programs. To this end, the University has created the Office of International Studies, which maintains information about programs, evaluates potential programs, and assists students in the process of applying for these programs. Of the changes that students have reported as resulting from their off-campus studies, the most statistically significant were in culture and personal growth. In addition, students stated that their study abroad experience helped them to gain a new perspective of the United States (see Assessing Off-Campus Programs Study Programs).

Faculty Growth. Since faculty need to continue to grow in order to maintain the kind of vibrant intellectual energy that yields inspired teaching and energized learning, it is crucial to determine whether faculty are sustaining such vitality by continued growth in research, by involvement in faculty/student research, and by ongoing work on their teaching craft.

The professional engagement of PLNU faculty in traditional scholarship of discovery (see Boyer's Definition of Scholarship), as noted in the 2005 Survey Data Tables, is generally quite strong with nearly 70 percent attending a conference in the last year and over one third (36 percent) publishing an article, chapter, or book in the same time period. Taking into account the multitude of activities that constitute professional engagement, a significant number of PLNU faculty are highly productive, especially given the strong teaching, advising, and service loads that faculty routinely assume. In fact, many members of the faculty show this level of activity on an ongoing basis. There could be great value in exploring ways to encourage broader participation by the entire faculty at all levels and locations of PLNU. 

It is important to note that, of all the professors responding to the 2005 survey of faculty, only nine taught exclusively at the graduate level. As a result, for the purposes of this report, “graduate faculty” means those who teach at least some graduate courses; “undergraduate faculty” denotes those who teach exclusively at the undergraduate level. Looking at the disaggregated faculty data, it is clear that both graduate and undergraduate faculty have productive scholarly lives.

As noted in the 2005 Survey Data Tables, graduate faculty certainly believe in the relevance of scholarly activities. They are, indeed, very interested in bringing research opportunities into graduate work. There currently exists no standardized mechanism by which PLNU graduate faculty can do collaborative research with their graduate students, although many of the master’s degrees that PLNU offers include the option of a research thesis completed under the guidance of a faculty member. Indeed more than a third of all graduate professors have overseen a master’s thesis (see the disaggregated faculty data).

The scholarship of integration (see Boyer's Definition of Scholarship), is somewhat harder to document because this work has mostly resulted in curricular programming rather than in easily counted articles. Nevertheless, four programs have been developed by varied teams of individuals from different academic and administrative unit of the University that serve as good examples of this form of scholarly activity. PQS (Program Quick Start), LEAP (Learning Experiences for Academic Progress), ISF (Integrated Semester for Freshmen) and University Now are integrative and collaborative ventures designed to serve different student populations creatively. In a less programmatic way, other collaborative and integrative curricular ventures have been supported by Alumni Grants or Center for Teaching and Learning grants. The Wesleyan Center has also served to promote faculty research, both traditional research and collaborative research as have other forms of institutional support, such as the Point Loma Press, Research and Special Project (RASP) grants (see list of RASP recipients), sabbatical awards (see list of recent sabbatical recipients), and Reading and Discussion Groups.

We recognize that the scholarship of teaching and learning (see Boyer's Definition of Scholarship), is, for the most part, privatized, if not entirely hidden, even from the view of colleagues. Many PLNU faculty members routinely work at improving classes, even those they have taught for many years. Some areas, such as Mathematics, Literature, Journalism, Modern Languages, Psychology, Sociology, and Biology, work together on improving teaching in their major programs and in general education (see samples of course collaboration and cross departmental collaboration). For the most part, however, this scholarship of teaching and learning is intuitive rather than experimental, private rather than public, and informal rather than methodical; its results, therefore, are personal guesses rather than measurements to be replicated. This is true as well for the scholarship of application (see Boyer's Definition of Scholarship), though there are specific individuals at the University who do this in the controlled and methodical manner characteristic of rigorous scholarship.

Individual faculty research has been well supported at the University through a professional development fund that offers every full-time faculty member $1,000 annual support for professional development activities and through Wesleyan Center grants, CTL grants, Alumni grants, RASP grants, and sabbaticals. Recipients of RASP grants, Wesleyan Center grants and sabbaticals annually present results of their study to the faculty in one of several  Faculty Resources Committee-led events during the subsequent academic year. In addition, the Point Loma Press, the academic publishing arm of Point Loma Nazarene University, “supports the scholarly work of the faculty and provides articulation of Wesleyan themes and trajectories, and may include coverage of important University-sponsored conferences, symposia, and events.”

Point Loma Nazarene University faculty are, therefore, busy in these various forms of scholarship. Approximately half of all faculty have seen their work published in the past two years. Most faculty have published at some point in their careers, including 70 percent who have published scholarly articles, 33 percent who have published chapters in books, and 28 percent who have written books or monographs. Several faculty members have published multiple articles and books. Finally, beyond print publications, the activity of faculty members in music and the fine arts is reflected in the 25 percent of all faculty who have regularly given public performances, recitals, and exhibitions, as noted in the 2005 Survey Data Tables.

Collaboration between students and faculty through undergraduate research has become a powerful link between good research and excellent student learning. Student/faculty research has not, of course, been limited to projects resulting in senior-year distinction projects.  A strong model of collaborative work, mainly in traditional research areas, exists at PLNU and should be replicated across the University. The strongest consistent work is in the natural sciences and social sciences. For example, the departments of Biology and Chemistry have had a tradition of student/faculty research since 1964 because they have shared the benefits of financial gifts and grants that compensated both faculty and students for summer time spent doing collaborative laboratory research. Summer research has resulted, for Biology and Chemistry students, in a steady flow of collaborative articles and in an equally steady flow of Biology and Chemistry students accepted into graduate, medical or dental school.

Some disciplines, particularly natural science or social science disciplines, have used their research methods course as a way to lead students into collaborative or independent research projects. In order to provide the kind of support needed to spread these collaborative efforts to other disciplines, the administration convened a task force to create a uniformly delineated program called, the Honors Program, that could define and support the undergraduate research process. At the same time, the Provost designated financial support to compensate, however slightly, faculty time and to create a low-risk, high-yield environment to help motivate students to take on a demanding senior project. In addition, the faculty recently approved a course, Honors 399, whose purpose is to provide course credit for doing the kind of extensive reading in both primary and secondary sources or the prolonged studio time that often forms the foundation on which an honors project in the humanities or arts is built. As a consequence of these initiatives, the production of Honors Projects has mushroomed in recent years.

The final major element in faculty vitality is their involvement in ongoing growth in teaching craft. The University requires annual student course evaluations to help provide professors with feedback that is directly relevant to course effectiveness, a feedback that approximately half of the faculty use as a guide in making course adjustments (see 2005 Survey Data Tables). An even larger number of faculty use informal student feedback to help guide course changes. In addition, faculty use a variety of other methods to improve their pedagogy: collaboration with faculty in their own disciplines, collaboration with colleagues in other disciplines, interaction with colleagues off-campus, reading of books and articles on pedagogy, attendance at conferences on pedagogy, and attendance at teaching workshops.

Summary of Conclusions

If good teachers are unwilling to ever consider the job perfect and are determined to continue to look for ways to improve, the same is true of a good academic institution. Although teaching at PLNU is generally effective and students are learning, there are numerous areas where more could be done. Individual professors, specific majors, departments, schools, and even broad areas are all engaged in ongoing assessment of their varied goals. Through various WASC committees as well as standing PLNU committees, a broad-based university conversation has been on-going. The primary focal points of this conversation and of efforts toward improvement are the need to embed missional and broadly consensual academic goals more deeply into the general education curriculum, the major curricula, and into co-curricular areas; the need to continue encouraging faculty scholarship at both undergraduate and graduate teaching levels; and the need to support the growth of faculty as teachers and professionals.

As a result of our work in preparing this reflective essay, we have developed the following recommendations to the President. Upon receipt of these recommendations, the President will develop cooperatively a prioritized list of all EER recommendations, assign each to a responsible person or group, and establish a timeline for action. As appropriate, these recommendations will be the focus of review by the Strategic Planning Committee for funding consideration, as needed.

See EER Task Force I: To Teach.

Recommendations

(See Recommendations to the President)

4.1 Explore ways, in both curricular and co-curricular arenas, for all students to make greater progress on the key learning objectives of diversity and skill in written communication and critical thinking.

4.2 Encourage academic departments and schools to be intentional in making moral, ethical, and faith connections within their majors and to be creative in finding ways to assess that goal.

4.3 Explore further ways to increase the number of faculty engaged in research.

4.4 Explore ways to increase the visibility of the Scholarship of Teaching and Learning, the Scholarship of Integration, and the Scholarship of Application.

4.5 Rewrite the Faculty Handbook to make explicit the promotion and tenure expectations for teaching excellence, scholarship productivity, and engagement in service as well as the general range of the desired balance between these three.

Theme Two: To Shape

The second institutional theme that emerges from our Mission is TO SHAPE. This aspect of our institutional purposes establishes for us the concomitant intention of helping students in their growth and development in a variety of areas that we believe constitute the whole person: spiritual, intellectual, social, emotional, interpersonal, and physical. In this reflective essay, we consider the institutional context, evidence and analysis, a summary of conclusions, and recommendations for action and improvement. This reflective essay addresses CFR’s 1.1, 1.6, 2.4 and 2.6.

The Institutional Culture

Point Loma Nazarene University seeks to be a shaping university, one of the three points specifically mentioned in the opening lines of the institution’s Mission Statement: “Point Loma Nazarene University exists to provide higher education in a vital Christian community where minds are engaged and challenged, character is modeled and formed , and service becomes an expression of faith.” While faculty, staff, and administration nurture relationships, engage in multiple activities, and design enriching environments aimed at teaching, shaping, and sending, it is clear that the end purpose of these activities is the positive development of the student. We are teaching, shaping, and sending effectively when students are, in fact, learning, growing, and serving . While strict distinctions between teaching, shaping, and sending are somewhat artificial (e.g. when one learns, one also grows; when one serves, one is also taught), it is perhaps useful as we consider the nature and character of the institution. In addition, we speak specifically to shaping and growing as essential to our character as an institution in the elaboration of four of the University’s seven Core Value statements, including the development of students as whole persons, a global perspective and experience, ethnic and cultural diversity, and the stewardship of resources. This reflective essay, in keeping with our purposes as stated in the Institutional Proposal, seeks to elaborate on the University’s shaping of students and their growth and development.

The Wesleyan tradition, as we understand it, calls us to holiness, to right living. One manifestation of holiness is consistency of belief and action. Further, holiness may also be understood as “wholeness.” We affirm that education of the mind is essential to our Mission, but not sufficient, and therefore we also aim to enable each student to grow in other ways to the fullest extent possible during time spent at the University. We invite each student to traverse the intellectual and developmental challenges inherent to higher education within a community of respect, care, responsiveness, and support.

Shaping is not a top-down process. We do not see only staff, faculty, and administrators as the exclusive “shapers,” and students as the “growers.” Rather, the entire community shapes; the entire community is privileged to grow. Staff, faculty, and administrators accept responsibility for initiating relationships and creating and maintaining an environment conducive to growth. Shaping and growing occur in a deeply relational context, within an environment that is challenging enough yet supportive enough for growth to occur. Our goal is for each student to understand and experience the University as a whole rather than in disjointed segments. While organizational and administrative distinctions are internally useful, they may be meaningless to the student; we seek opportunities to integrate the curriculum and co-curriculum in meaningful ways in order to facilitate an experience of wholeness. To the degree that PLNU is a more seamless experience, then, we facilitate the student’s opportunity to grow as a whole person.

For the purposes of this reflective essay, we envision that shaping occurs as a student is engaged and equipped , resulting in the embodiment of personal growth. Opportunities for intentional shaping and growth are knit into the fabric of the University. In this essay, we use a whole-person model of wellness to discuss selected examples of intentional shaping and growth typical at Point Loma. We present the available evidence of student growth, both quantitative and qualitative, seeking to give examples that are curricular, co-curricular, or collaborative in nature in keeping with the engaging, equipping, or embodying nature of the shaping process.

Evidence and Analysis

Though the University holds strongly to the idea of wholeness in individual development, an analysis of our efforts in this regard of necessity requires a deconstruction of the elements that we utilize to work to this end. At all points, the University understands the importance of connecting student experiences in meaningful ways, of seeing over-arching themes and perspectives. We recognize that this requires a deliberate effort to achieve connectedness at every level of leadership and participation. Point Loma began its participation in the National Survey of Student Engagement (NSSE) in 2000, followed by a second administration in 2005. Some themes and trends that relate to student shaping and growth follow. First-year students and seniors reported in 2005 that attendance at University events was emphasized “quite a bit;” concurrently, both groups reported increased participation in co-curricular activities between 2000 and 2005. Seniors reported a dramatic increase in their study abroad participation (see International Study Statistics in the CPR report). Freshmen reported in 2005 that the PLNU experience helped their self-understanding at a rate significantly higher than freshmen in comparable institutions. Freshmen and seniors in 2005 rated their entire experience at 3.5 (on a 4.0 scale where 4.0 is “excellent”), an increase for both groups over the 2000 survey administration and significantly higher than ratings for NSSE participants nationwide.

These selected results from the NSSE indicate a positive level of student engagement at PLNU. Engagement, in turn, predicts numerous positive outcomes, including depth of learning and graduation rates. Indeed, the University’s six-year undergraduate graduation rate has increased from 44 percent of the 1994 cohort to 65 percent for the 2000 cohort, an improvement of nearly 50 percent, indicative in part of successful efforts to engage students and allow them the opportunity to experience personal growth.

The following sections address PLNU’s efforts to shape students in six areas of endeavor: spiritual considerations, intellectual life, social environments, emotions, student responsibility, and the physical realm. Evidence referencing the 2006 College Senior Survey is based on a rather small sample from that graduating class; we expect larger samples in the future with the new option of both paper and web responses.

Shaping and Growth in the Spiritual Dimension. Point Loma Nazarene University is a faith-based institution, and students select the institution in large measure because of an appreciation of our religious heritage. The Cooperative Institutional Research Program (CIRP) survey of incoming freshmen, an instrument that catches responses every fall from approximately 95 percent of PLNU new students, shows a five-year aggregate of 73 percent of students (2002-2006) who cite the importance of religious affiliation as a reason for selecting this institution. At the same time, the University seeks to interact with students at the spiritual level in efforts that challenge growth in this regard. Though precise data is not plentiful, there are indicators of the spiritual impact of the institution on graduating seniors. In the 2006 administration of the CSS, for example, 83 percent of respondents strongly agreed or agreed that they had “a better basis for making ethical decisions in personal relationships;” 89 percent believed they had “a better basis for making ethical decisions” in the workplace; 79 percent asserted “a better understanding of what it means to be a Christian;” and 75 percent said, “I have grown in my Christian faith.” As in the case of intellectual life, we address the spiritual dimension at PLNU in a variety of ways.

Although we have indirect measures related to the theme of shaping , these measures demonstrate the beginning of a surveying process and provides us with a snapshot of student attitudes. The data gives us some indication of shaping outcomes, and we recognize the need for analysis over time. It would also be helpful to be able to compare this kind of data to national norms.

Some of the data related to shaping among undergraduate students in the spiritual dimension includes the following : 64.5 percent of students agree or strongly agree that they have "a better basis for making ethical decisions" in the workplace; 69.3 percent agree or strongly agree they have "a better understanding of what it means to be a Christian;” 71.0 percent agree or strongly agree they have "grown" in their Christian faith. At the graduate-student level,  data shows that 54.2 percent agree or strongly agree they have "a better basis for making ethical decisions in personal relationships;" 67.6 percent agree or strongly agree they have "a better basis for making ethical decisions" in the workplace; 73.3 percent agree or strongly agree they "think critically about the integration of faith and profession" in classes; and 77.5 percent agree or strongly agree their "moral and ethical thinking is stimulated in classes."

The undergraduate community gathers three times per week for chapel. Students actively participate in this experience, not only through corporate worship but in areas of leadership such as chapel musical bands, public reading of Scripture, leading in prayer, and speaking. Chapel is central to spiritual life at the University, offering a time of worship, devotion, and biblical teaching shared by faculty and students alike. In the 2006 College Senior Survey (CSS), 74 percent of students agreed with the statements “I have a stronger personal relationship with God” and “I have grown in my Christian faith.”

The office of Discipleship Ministries serves the PLNU community by offering various opportunities for Christian formation. This office trains 60 junior and senior students who live in residence with freshmen and sophomores, lead weekly Bible studies, and serve as spiritual mentors. A total of 351 students participated regularly in these groups during the 2004-2005 academic year. In the 2006 College Senior Survey (CSS), nearly 85 percent of students indicated that they had participated voluntarily in either a Covenant or Discipleship group.

Student Ministries at PLNU is made up of 27 ministry groups composed of approximately 385 students reaching out to meet community needs in a variety of ways. Groups serve in locations ranging from orphanages in Mexico, to elderly homes and homeless shelters, to churches and children’s hospitals. Student participation in 2005-2006 increased 57 percent from 2003-2004 (see the Student Ministries data in cross cultural participation data). Administration of this program is vested in a full-time staff person who is assisted by the Associated Student Body  (ASB) Director of Student Ministries.

Approximately 150 students participate annually in a three-week domestic or international mission experience through LoveWorks. These students receive 30 hours of advance cross-cultural training. Sixty-one percent of graduating seniors agreed that leadership skills were enhanced through missions and ministry at PLNU (see CSS 2006).

Voices in Praise (VIP) Gospel Choir, with 75 student, alumni, faculty, and staff members, seeks to provide spiritual ministry within the PLNU community and beyond. VIP seeks to foster an environment that accepts and values diversity by providing a variety of opportunities for multi-ethnic expressions of worship and praise. Performances in local churches and community events, University activities (i.e., chapel and MOSAIC events) and a spring concert are examples of VIP service opportunities.

The Center for Justice and Reconciliation (CJR), based on John Wesley’s belief that "the gospel of Christ knows of no religion but social; no holiness but social holiness," sponsors curricular experiences such as Urban Term, an intensive cross-cultural immersion sociological and theological educational curriculum designed to combine praxis and academic reflection on the complexities of urban life while living and serving in City Heights, a diverse, low-income community in San Diego. The Center offers a lecture and discussion series, “ Brewed Awakening,” on a monthly basis as a means of stirring social consciousness and teaching Christian means of social engagement.

Shaping and Growth of the Intellect. In addition to the obvious impacts of the teaching/learning environment in the classroom (see EER reflective essay To Teach), there exist many rich opportunities for intellectual growth outside that arena. While specific data on intellectual growth is difficult to amass apart from traditional measures, there are indicators of important connections for students in this regard.

In 2005, the University inaugurated the Preface Reading Program, designed for all entering new students in August of each year. In this program, we ask new students to read a common text and to participate in a New Student Orientation (NSO) discussion group as co-facilitated by two of the following individuals: a member of the faculty or Administrative Cabinet and a professional staff member from a co-curricular unit. The goals of this program are multiple: to choose an engaging text with developmentally relevant themes, to model collaboration of professionals from curricular and co-curricular areas, to demonstrate the importance of reading and reflection early in a student’s University experience, and to provide a common topic of discussion for students as they are integrating socially. An evaluation of Preface 2006 found that 78 percent of students read most or all of the assigned text, 76 percent found the book extremely interesting, and 90 percent of students found the book discussion extremely stimulating (see Preface 2006 survey data).

Discipleship Ministries and Nicholson Commons co-sponsor a film forum series each semester to view and discuss culturally important films. Over 800 students attend each semester and participate in faculty-led discussion and analysis of each film.

Approximately 20 percent of first-time freshmen enter the University with an undeclared academic major. The Director of Academic Advising supports and coaches these students toward exploration of strengths and interests. Since 2003, we have encouraged these students to declare an academic major in the spring semester of their freshman year rather than waiting a year longer, which had been customary. In 2004-2005 and 2005-2006, 60 percent of undeclared students met the goal of declaring a major before the end of the freshman year, thus connecting earlier with a faculty advisor and academic department or school (see academic advising data from the Student Development Annual Report for 2005-06).

Tutorial Services in the Academic Support Center provides peer tutoring, review sessions, paper editing, and tutorial computer software to all students at no cost. We provide tutoring in support of every academic major in approximately 85 courses. This office recommends tutors designated by faculty members in the academic areas, rigorously screened, and trained in effective tutorial techniques.

Some of the data from the WASC 2005 survey related to shaping among graduate students in intellectual growth shows that 73.2 percent agree or strongly agree "the project and thesis requirements engage graduate students in high-level professional experience."

Shaping and Growth in Social Environments . Multiple, rich social opportunities exist at the University. We desire that students learn skills that serve them well in the collegiate environment and in the years beyond. Some of these opportunities for growth follow.

In 2005, the Associated Student Body (ASB) recognized the need for increased attention to traditions as means of building and maintaining student loyalty to the University. Since that time, ASB has resurrected the tradition of the Spirit Bell, a tradition beginning in the 1960’s. New Student Orientation begins with the ringing of the bell, and the annual Commencement ceremonies end in the same manner. The Spirit Bell appears at key athletic events where residence halls compete for the privilege of “owning” the Bell. The Bell was the focal point of Homecoming 2006 with an amazing display of student spirit. Many PLNU alumni and guests of the opposing team recently commented on the high level of student participation and spirit at this occasion.

Loma Learning Communities (LLC) represent an opportunity for students to enhance their residential experience by joining a community that has common purposes, goals, and activities. Each LLC chooses a faculty or staff advisor who participates in the life of the community for a full academic year. In 2006-2007, the second full year of this program at Point Loma, 152 students participated in 18 groups (see LLC Data). In an end of the year evaluation, a male student commented, “[We] met on a weekly basis and held an accountability and small group Bible study… [We] desired to be an active example within the community of Point Loma…We met our goals…and were able to bond together through a few outdoor trips. …The overall idea of this LLC is great…“

As part of the Psychology 101 course (Psychology of Personal Development), each first-year student participates in a semester-long discussion group led by upper-division students. These groups create opportunities to learn the culture of PLNU and interact with other students. This interaction promotes a deep reflection of one’s personal views and assumptions toward others that is the essential aspect of tolerance. The results of these interactions are logged in personal reflections kept inviolate to the student, though reviewed by instructors.

Over 30 clubs and organizations operate under the auspices of ASB, and involve a large number of students. Each club represents unique interests and includes an aspect of service in its charter.

The director of Diversity Student Services advises MOSAIC (Multi-Cultural Opportunities for Students Actively Involved in Community), composed of Brothers and Sisters United (BSU), Hui O Hawaii, Asian Student Union, Association of Latino/Latina Students (ALAS), and Team Barnabas . Team Barnabas is a group of junior and senior students who live in residence with freshmen and sophomores and mentor diverse students and first generation college students. In responding to the 2006 College Senior Survey, 71 percent of students agreed with the statements “I have a greater respect and appreciation for cultural differences” and “I am more tolerant of people from other cultures.”

Some of the data related to shaping among undergraduate students in social environments includes the following: 90.4 percent agree or strongly agree they "feel comfortable at Point Loma;" and 84.5 percent agree or strongly agree "most others at Point Loma accept me."

Finally, as a result of the beginning of a surveying process that provides us with a snapshot of student attitudes, we have some indication of outcomes in the area of the University’s social environment; clearly, it will need to be evaluated over time. In our 2005 survey of undergraduate students, for example, we learned that 66 percent of respondents believe people of color are taken seriously; 85.4 percent believe women are taken seriously; and 39.7 percent believe they have become more sensitive to gender issues as well as to race and class issues (41.7 percent) while at the University. At the graduate studies level, respondents felt that people of color are taken seriously 74.9 percent), women are taken seriously (91.6 percent), and they are more sensitive to gender issue (33.8 percent) and race and class issues (48.6 percent).

Shaping and Growth in Emotional Well-being. Emotional health is essential to mature adult functioning. Opportunities to grow in this arena present themselves spontaneously in the classroom, the residence hall, and in daily personal interactions. The Wellness Center provides guidance and leadership in this area. As a means of responding to the whole student, the University combined the Counseling Center and Student Health Services in 1996 into the Wellness Center. In 2001, the University expanded services to include Wellness Outreach, offering prevention activities to the broader University community. This combination of services has been well received by students and has appeared to be effective. In 2006-2007, Wellness Outreach made 11,873 individual contacts with students (see outreach data from the Student Development Annual Report for 2005-06). Counseling Services provided 647 hours of clinical therapy to 142 students in 2005-2006 (see mental health data from Student Development Annual Report for 2005-06).

The general education assessment administered to graduating seniors in spring 2006 included a question designed to assess students’ understanding of the whole person and his/her needs. The question provided a hypothetical description of a person’s life and asked the student to determine the likely causes of unhappiness. The question sought to assess how holistically students think about real-life situations (see 2006 exam results).

Shaping and Growth in Responsible Roles. Three areas of engagement at PLNU offer students opportunities for shaping and growth. These include leadership roles in connection with the Office of Student Development (especially during the opening days and weeks of the fall semester), on-campus employment, and interactions with the Office of Career Services.

Approximately 12 percent of undergraduate students serve in positions of leadership at the University (see leadership data). They come to the main campus two weeks prior to the beginning of the fall semester for LEAD (Leadership Enrichment and Development) week, a time of intensive leadership training and equipping. Their time of preparation culminates as each student leader fills a specific, meaningful role during New Student Orientation (NSO). This model offers them specific opportunities to utilize the principles of servant leadership in a timely and practical way.

Several hundred part-time employment positions are available to be filled by students on the main campus (see student employment data) . Many students have their first formal occupational experience in the context of student employment. Each year, the University invites supervisors to submit nominations for Student Employee of the Year. Nominees respond to the question, “What Has Student Employment Meant to You?” The response of a senior 2007 nominee noted growth in time management, increased understandings of personal strengths and abilities, and “proper respect for others” in the performance of on-campus employment responsibilities. The student goes on to conclude that “I wouldn’t have understood how to work as efficiently in my current environment if not for the building blocks of previous work experience at PLNU.”

The office of Office of Career Services sponsors a popular Etiquette Dinner each semester, which attracts 120 students annually. The University provides a multi-course dinner in a business-type setting. The experience features coaching appropriate manners in the business setting. The experience features the coaching of appropriate manners in the business setting. Narrative comments reflected student feedback that the skills learned would enhance their readiness to enter the professional world in a social environment. 

Shaping and Growth in the Physical Realm. The University highlights the value of one’s physical being in the general education curriculum with two classes in physical fitness and nutrition, including the course Physical Education 300 Optimal Health. Also, approximately 500 students participate annually in intramural athletic competitions (see Intramural data).

The Outdoor Leadership (ODL) program trains students to guide their peers in weekend rock climbing, camping, kayaking, hiking, skiing, and biking in what are called “ Great Escapes ” trips. Twenty students serve as guides and approximately 130 students participate in Great Escapes trips each semester (see Great Escapes data). As part of a qualitative assessment of the ODL program, conducted in 2003, one student responded to the question, “How has your involvement in the Outdoor Leadership program impacted your college experience?” The student noted reaching a new level of understanding and appreciation for cultural diversity while participating in the program, saying, “I suppose that I could describe my reaction to San Diego over the last two years as culture shock. I came to this school not knowing anyone, and the people who I did meet were very different than those I had known before. . . . Weekend trips were a good way to meet other students...My memories of the Great Escapes programs will be some of the best ones when I reflect on my college career.” (Student reflection, 2003).

Data provided by the Assessment Committee shows that academic majors cite eight common shaping outcomes (common outcomes): clinical skill development, faith integration, analysis or research skill development, recognition and appreciation of the discipline, personal or professional maturity and growth, reflection, communication development, and lifestyle. Of 254 outcomes cited across 64 academic programs, 70 (28 percent) are in the shaping category. Of the 70 cited shaping outcomes, data exists (Data on Common Learning Outcomes by Department  on 48 (69 percent), and of the 48 shaping outcomes for which we have data, 37 (77 percent) have met outcome targets (Analysis of Common Findings by Outcome). If the 254 outcomes are broken down according to graduate and undergraduate (Data on Common Learning Outcomes for Graduate and Undergraduate Programs), the shaping outcomes cited by graduate areas is a slightly higher frequency (9 out of 25, or 36 percent) relative to undergraduate (61 out of 229, or 27 percent).

This data reflects a significant emphasis on the shaping mission of the University by academic majors (28 percent) - representing one-third of 3 mission categories, a good start on data collection (69 percent), and significant success in achieving our shaping outcomes (77 percent).

Summary of Conclusions

In a consideration of the University’s goal to shape students, a holistic way of thinking seeks to encompass multiple layers of meaning and experience. We recognize that each relationship that a student develops within the University community serves to increase depth of learning. Curricular, co-curricular, and collaborative efforts provide each student with multiple opportunities for shaping and growth. In this reflective essay, we have addressed the areas of spiritual, intellectual, social, emotional, responsible roles, and physical growth and highlighted selected examples of these efforts with supporting data as a means of elucidating Point Loma Nazarene University’s commitment to the whole student.

Part of wholeness is recognizing that the boundaries of our concerns for shaping students do not end at the graduation line. Effective shaping is helping students to develop a sense of vocation or calling, an ability to articulate the significant activities that help give form, purpose, and meaning to fulfilled lives.

The data presented in this essay is often in a simple numerical form or percentage of participation. This fact reflects the challenge of assessing student outcomes on a programmatic or micro level. We are successful in demonstrating impact on the institutional or macro level, with measures such as retention and graduation rates. We continue to seek means of demonstrating effectively the micro outcomes, knowing that the questions change from input and output information to more meaningful ones that focus on shaping, growing, and changing that occurs in students.

Given the array of various profiles and surveys related to students (Cooperative Institutional Research Program (CIRP), National Survey on Student Engagement (NSSE), Student and Spiritual Development Annual assessments, College Student Survey [CSS]), a composite document or “State of the PLNU Student” report could be released annually and distributed to the entire University community. Such a profile report might inform pedagogy, policy, and practice with regard to students.

In summary, we recognize the positive impact of collaborative and integrative experiences, such as the Preface Reading Program, the Film Forum series, Student Ministries, Loma Learning Communities, student employment opportunities, and Outdoor Leadership and Recreation activities. These experiences serve as powerful antidotes to a kind of silo culture that students may face at increasing intensity as they move further into their major programs. Continued development of such experiences should be an on-going priority for the institution.  As a means of validating the student's entire experience, we have made available to undergraduate students, beginning in the summer of 2007, a Co-Curricular Transcript (CCT). The CCT parallels the academic transcript and documents co-curricular engagement. In addition, PLNU now recognizes the value of capturing indicators of students’ growth in as many areas as possible, as we develop a culture of evidence across all units of the University.

As a result of our work in preparing this reflective essay, we have developed the following recommendations to the President. Upon receipt of these recommendations, the President will develop cooperatively a prioritized list of all EER recommendations, assign each to a responsible person or group, and establish a timeline for action. As appropriate, these recommendations will be the focus of review by the Strategic Planning Committee for funding consideration, as needed.

See EER Task Force II: To Shape.

Recommendations

(See Recommendations to the President)

5.1 Study ways to connect the student’s curricular and co-curricular experiences.

5.2 Evaluate and improve the University's new Co-Curricular Transcript.

5.3 Release an annual “State of the PLNU Student” report that includes data from multiple sources to inform decision making and practice relative to students. 

5.4 Develop and measure outcomes appropriate to the shaping and growth of students.

Theme Three: To Send

The third institutional theme that emerges from our Mission is TO SEND. This aspect of our institutional purposes establishes for us the resultant focus following one’s ultimate departure from the institution of finding a place of meaningful life and service. This entails not only the possibility of continuing study at the graduate level common among all college and university alumni, but also, as a faith-based institution, a determination to embrace service as a value and establish life-long connections with a faith community. In this reflective essay, we consider the institutional context, evidence and analysis, a summary of conclusions, and recommendations for action and improvement. This reflective essay addresses CFR’s 1.1, 1.2, 2.3, 2.4, 2.6, 2.8, 2.13, and 4.8.

The Institutional Culture

The University seeks to be a sending community characterized by a vital commitment to a lifestyle of service. As with the previous two Special Themes, “To Send” flows directly from, and is inextricably tied to, the University’s Mission Statement. The fact that this statement recently has been revisited and re-crafted reflects the truism that it is not a mere showcase created for positive public relations, but that it is a dynamic, realistic, and accurate description of the University's work with students. Students leave the University, prepared by the community in which they have learned and been shaped. This essay, in keeping with our purposes as stated in the Institutional Proposal, seeks to elaborate on the University's sending mission and the student learning that results from our efforts.

For our purposes in this reflective essay, we determined that the phrase “To Send” speaks to our desire to prepare students for vocation, understood broadly though measured specifically as preparation for graduate school or the professions, service to the community, and participation in a local faith community. While the nature of vocation is, of course, far broader in scope than can be addressed in this essay, we use the term here to refer to the overall unfolding of our students’ lives, as they participate in this learning community, and take key aspects of that experience with them when they leave and develop a direction or calling in their lives. While we recognize that vocation does not measure all aspects of the University's sending students into their world, we believe it is strongly predictive of our general success in our mission of sending. As a means of examining the nature of “To Send” at PLNU, therefore, we focused on these three facets of vocation: career, community, and church. We contend that these factors can serve as representative elements of a life well-lived.

In the preparation for writing this reflective essay, we utilized a number of sources, many of which were already in place and widely implemented by particular academic departments and schools and administrative units for their own on-going assessment purposes. These sources included Graduate Record Examination (GRE) scores, professional licensure examinations, graduate school admission records, data from the Office of Spiritual Development, the 2005 Alumni Survey, focus groups, Office of Career Services data, senior exit interviews, senior surveys, and registration surveys.

Evidence and Analysis

Preparation for Graduate Work and a Profession.  The importance of academic preparation flows directly from the University’s primary mission and vision of academic excellence. Indeed, the Mission Statement begins with these words: “Point Loma Nazarene University exists to provide higher education in a vital Christian community where minds are engaged and challenged ,” and "excellence in teaching and learning" is the first item in the list of the University’s Core Values.

In determining whether we are, in fact, successfully preparing our students for graduate school and for their chosen professions, we surveyed 1300 alumni who graduated from PLNU between 1994 and 2004. We believe the results of this 2005 Alumni Survey demonstrate that we are indeed successfully preparing our students for graduate study. Of those alumni who obtained at least a master’s degree, approximately 90 percent felt they had been well prepared. Moreover, 44 percent of respondents indicated they had completed some type of professional certification, a master’s degree, a doctoral degree, or some other post-baccalaureate program at another institution. It appears that nearly half of our graduates complete an advanced degree of some type. In addition, 78.3 percent of the alumni who pursued graduate work indicated they were well prepared for the social and psychological challenges of post-baccalaureate study.

While our alumni may well be the best source as to how successful we are in preparing our students for graduate work, additional considerations verify the positive results suggested by our alumni. If doctoral production is an accurate measure of our academic success, then we can claim that our students are well prepared for graduate education when they leave the University. PLNU, for example, places in the 85 ^th percentile when compared with the Ph.D. production of all California institutions; in the 79 ^th percentile when compared nationwide with master’s degree-granting institutions; in the 73 ^rd percentile when compared nationwide with bachelor’s degree-granting institutions; and in the 59 ^th percentile when compared with member institutions of the Council for Christian Colleges and Universities (CCCU).

Not only do we believe that alumni are one of the most reliable indicators as to how well we are preparing our students for graduate study, but we also believe our current students have much to tell us. In a recent survey among graduating seniors (College Senior Survey [CSS]), 49 percent indicated they were frequently encouraged while at PLNU to pursue graduate or professional study. This compares favorably with 39 percent at other faith-based institutions and with 39.1 percent at private colleges nationwide. The perception among our students that they are actively encouraged to pursue some sort of graduate-level education is, moreover, borne out by the efforts the University itself undertakes in this regard. While we have not yet compiled an exhaustive list of University-directed endeavors, their number and impact is significant. They include the Center for Teaching and Learning, an active Pre-Medical Committee, Pre-Law Advisors, the Research Associates (a group of some 290 alumni who work in the field of science and provide approximately $30,000 a year for our students to engage in summer research programs), and the Careers Option Program sponsored by the Department of Biology. Furthermore, the Department of Mathematical, Information and Computer Sciences has a similar program, as does the School of Education .

Although PLNU does not have hard data as to how many students university-wide are, and are not, accepted into the various graduate and professional programs, the Pre-Med Committee reports that in the past five years, 78 percent of those who applied to medical schools were accepted, and 66 percent of those who applied to dental school were accepted. Moreover, over the past ten years, the medical school acceptance rate has been 72 percent, while the dental school acceptance rate has been 88 percent. Likewise, the pre-law advisors report that our law students are accepted into some of the most prestigious law schools in America and that virtually all our applicants are accepted into an American Bar Association-accredited law program. In the past ten years, five of our students have been selected as Fulbright Scholarship recipients. The University has a standing Fulbright Committee which actively seeks out qualifying students for this prestigious program and prepares them for the process as well, two of those committee members having been Fulbright Scholars themselves, as have two other faculty members (see Fulbright data).

Since not all students pursue graduate study upon leaving PLNU, we also inquired in the alumni survey as to how well they believed themselves to have been prepared for the professional world. Fully 90 percent of those participating in the 2005 Alumni Survey believe they were well prepared with the skills necessary for the world of work. Also, 91.5 percent of the responding alumni feel that their experience at PLNU had a major to moderate impact upon the development of critical thinking and analytical skills; 84.5 percent believe PLNU’s impact upon the development of leadership skills was major, or at least moderate; and 87.6 percent judge their verbal communication skills were significantly improved at the University.

Service to the Community.  As with the goal of academic preparation, PLNU’s Mission Statement also includes the words, “…where service becomes an expression of faith.” Equally important to its quest for academic excellence, the University intends to be known for wholeness in the personal development of students. This objective entails creating 1) a collegial community characterized most notably by civility and respect, where all members are valued and encouraged to fulfill their individual potential, and 2) an exemplary model of student engagement for service to a community. The University strives to create an intentionally Christian community, one that includes global perspectives and experiences, fosters the notion that ethnic and cultural diversity are inviolate priorities, and promotes service to others as a vital expression of our personal faith. The data seem to corroborate our claim to be providing students with this kind of community.

With regard to our commitment to sustaining a community where all members are valued, we have made substantial progress in recent years. The University has institutionalized support networks and services for diverse groups (see CPR Group Seven: Diversity). Since most of the programs created to support our minority students have been recent additions to PLNU, it is too early to measure the overall impact that these programs are having.  These programs will be part of our assessment work as we enrich our current protocols.

We have been able to measure, however, the degree to which students are actively engaged in some sort of service program that embraces the larger, more diverse community during their time at the University. According to the Senior Survey (CSS) given to our graduating students, approximately 59 percent of our recent graduates indicate that they frequently, or at least occasionally, performed community service as part of a class while a student at PLNU. This compares with 57.3 percent of students at other religious-affiliated institutions, and with 54.1 percent with students at private colleges in general. Moreover, 80.4 percent of respondents indicated they had taken part in some sort of volunteer work during the past year. This compares with 77.3 percent at religious-affiliated institutions, and with 72.2 percent at private colleges. While we are pleased with these results, we are confident more can be done to improve.

Since service is one of PLNU's Core Values, it is in our interest to foster in students a commitment to community service that will extend beyond their time at the University. In this regard, according to our 2005 Alumni Survey, 81 percent of respondents indicated they had volunteered their time to a service organization at least once a semester or more. Additionally, the proportion of PLNU alumni who took part in some sort of voluntary work during the past year is nearly twice the national average of 42.1 percent, according to a study conducted by independent researcher C. Toppe (see Faith and Philanthropy). A full 89 percent of PLNU alumni responding indicated they contributed financially to a service or charitable organization at least once a semester, 22.6 percent indicated they had volunteered their time for a community or civic organization, 36.1 percent indicated they had volunteered their time in schools or in other educational organizations, and 13.7 percent indicated they had volunteered time in social organizations.

Participation in a Local Faith Community.  Similar to the community component, this commitment flows from the University’s statement that “Point Loma Nazarene University exists to provide higher education in a vital Christian community .” It is the University’s vision that Point Loma Nazarene University will be known for its commitment to developing the whole person (see the EER reflective essay on To Shape) and for faithfulness to its mission, demonstrated by a spiritual vitality that is centered on the Lordship of Jesus Christ. When we speak of creating an intentionally Christian community, we mean encouraging students to embody the life example and spiritual authority of Jesus Christ in whatever community they find themselves. A presupposition to this ideal is that this vitality be observed and experienced while a student at Point Loma Nazarene University, and that it be modeled by professors, administrators, and staff alike. The data demonstrates that we are succeeding in preparing our students for involvement in a local faith community.

The 2005 Alumni Survey provides a compelling picture of just how involved our graduates become in a faith community when they leave PLNU. Of those responding, for example, 79.6 percent indicated they regularly attend a church service; 72.9 percent indicated they financially support a church-related organization; 32.6 percent report making financial contributions to various mission organizations; and 11 percent volunteer time in various mission organizations. In addition, the data suggests a high correlation between church attendance and volunteer work. Over the last year, for example, 70.4 percent of respondents indicated they attend church on a weekly basis, and of this number, 51.9 percent perform some sort of volunteer work for a church-related organization.

Summary of Conclusions

In this reflective essay, we analyzed the academic preparation of students, the importance of updating contact mechanisms with alumni, and issues related both to inculcating students with the institution’s value of service to others and our graduates’ connections to a faith community.

Academic Preparation.  PLNU’s 2005 Alumni Survey indicates that we are preparing students well for graduate study and the world of work. Students also perceive that we are actively encouraging them to pursue additional training after graduating from PLNU. They are earning doctoral degrees, show high acceptance rates into medical schools, and are studying law in significant numbers at highly esteemed law schools. In short, the data indicates we are “sending” our graduates well prepared for their various life goals.

Several entities exist at PLNU that work actively to promote specific vocational awareness among respective groups of students, more specifically, to provide students with information, resources, tools, and the requisite preparation for taking the next step toward their professional careers. Although we recognize the success of the Office of Career Services in bringing employers to the University to engage with students, we maintain that the process of vocational thinking should begin in the student’s freshman year and continue through one’s academic preparation with far more depth and intentionality than is currently the case. It is also essential that we learn to what degree the Career Services office effectuates this goal.

Finally, a growing number of departments and schools are administering undergraduate Educational Testing Service (ETS) examinations to their graduating seniors. These examinations provide nationally normed data. Such a testing process enables the respective departments and schools—and the University at large—to evaluate more reliably how the academic preparation taking place at PLNU compares to that of other institutions and simultaneously to assess what percentage of students are performing at or above a given benchmark measure in their discipline of choice. Comparing students’ levels of learning and understanding with other institutions gives us the ability to assess more accurately the success of our programs and may well be a better indicator than the feelings of our graduating seniors themselves. This information also might assist us in on-going efforts to measure more accurately the success of our academic programs and make changes and improvements to programs as necessary .

Alumni Data.  In light of recent technological advances that render data collecting and assessing matters relatively easy, it may be possible for the University to implement a comprehensive system of tracking data on alumni.  One facet of the current system is the portal. Communication is ultimately achieved, but it can be slow, tedious, and often inadequate. Many of the addresses we have for recent alumni are, in fact, the addresses of their parents. Currently less than half of the alumni from the classes of 1995, 2000, and 2005 have put any occupation data into the AlumNews data gathering system. What is needed, therefore, is a 21 ^st century approach to communication whereby the University can communicate with graduates, at their option, from the day the degree is posted and potentially continuing throughout their entire lives.

Improvements to our system will not only benefit the University at large, but will also provide the graduate of PLNU a critical means of staying in touch with the University, not to mention a world-wide web of alumni. Briefly stated, we need to know where our students go after graduation, where they live, work, may perform volunteer service, attend graduate programs, and sustain involvement in a faith community. To this end, it is essential that we improve the current web portal, rendering archaic mail surveys obsolete and allowing communication to be far more personal and reliable.  The University should consider providing life-time e-mail addresses to alumni.

Service as a Value.   Service to the community is a long-standing mainstay of the mission of the sponsoring denomination (Church of the Nazarene), and service to the community therefore is one of the chief mission objectives of PLNU. The data show that the University is sending service-minded men and women into the wider global community. But while taking considerable satisfaction in our students' values as they leave the University, we remain committed to seeking even more meaningful ways to create community at the University and to prepare students for a life of service beyond.

The 2005 Alumni Survey indicates that our graduates volunteer their time with service organizations at a level that is twice as high as that which is undertaken by the rest of the U.S. population. Moreover, the overwhelming majority of our graduates contribute financially to charitable organizations, consistent with the sponsoring denomination’s strong commitment to social action. At the same time, we need to have better information as to the actual extent of service engaged in by our alumni. Valuable research might include, for example, specific details such as where our alumni perform their community service, the nature of the service, and what groups and types of individuals are, in fact, affected by the contributions of our alumni.

Given the University’s mission to send service-minded men and women into the global community, it is paramount that service be viewed as related to the academic program in general and not merely as a discrete entity unto itself. To the degree feasible, and without taking the form of a mandated service-training program, the University should encourage as many academic disciplines as possible to develop service-related experiences and/or training for their students, however these endeavors might inherently relate to a given discipline.

Connection to a Faith Community.   In terms of students’ involvement in a community of Christian believers, we continue to encourage graduates to be active participants in local faith communities. According to our 2005 Alumni Survey, the vast majority of our graduates are, in fact, involved in their local churches. The evidence shows they support the work of churches both at home and around the world. They are, moreover, decidedly dedicated to their faith communities.

Once again, however, there are things we desire to know about our alumni, but which we have not yet developed the capacity to learn. We do not know, for example, why and how our alumni make the choice to worship in a given church community. Since it is important to our mission that our students join a church family upon completion of their educational experience at PLNU, it is essential that our students adopt this behavior as a life-style during their time at the University, and that they do so on their own initiative and without compulsion.

While we have a fairly good understanding of the degree to which our alumni are part of a church community, we do not know how often our students themselves are attending a church service. Opportunities exist for careful research on past and current students. For example, it would be good to know about why and how our alumni make the choice to worship and research the impact of required chapel attendance for undergraduate students on later decisions to join a church community.  Similarly, research on current students' involvement in a church community while attending PLNU would help to craft a broader understanding of the role of chapel in spiritual development. And finally, given the denominational diversity present in the student body, research that disaggregates these results by denominational affiliation would provide a fuller picture of a student's decision to join a church community subsequent to leaving the University.

Data provided by the Assessment Committee shows that academic majors cite eight common sending outcomes (common outcomes): career preparation, engagement in the discipline, further study, moral or ethical behavior, analysis or evaluation in practice, being influential in the field, life-long learning, and service. Of 254 outcomes cited across 64 academic programs, 76 (30 percent) are in the sending category. Of the 76 cited sending outcomes, data exists (Data on Common Learning Outcomes by Department) on 62 (82 percent), and of the 62 sending outcomes for which we have data, 37 (60 percent) have met outcome targets (Analysis of Common Findings by Outcome). If the 254 outcomes are broken down according to graduate and undergraduate (Data on Common Learning Outcomes for Graduate and Undergraduate Programs), the sending outcomes cited by graduate areas is a slightly lower frequency (6 out of 25, or 24 percent), relative to undergraduate (70 out of 229, or 31 percent).

This data reflects a significant emphasis on the sending mission of the University by academic majors (30 percent) – representing about one-third of three mission categories, excellent compliance on data collection (82 percent), and good success in achieving our sending outcome targets (60 percent).

As a result of our work in preparing this reflective essay, we have developed the following recommendations to the President. Upon receipt of these recommendations, the President will develop cooperatively a prioritized list of all EER recommendations, assign each to a responsible person or group, and establish a timeline for action. As appropriate, these recommendations will be the focus of review by the Strategic Planning Committee for funding consideration, as needed.

See EER Task Force III: To Send.

Recommendations

(See Recommendations to the President)

6.1 Develop expanded opportunities for helping students to explore their vocations.

6.2 Study the feasibility of developing a program that provides for an initial contact with PLNU graduates by telephone, mail, or via web technology, followed by a tracking system via web technology on a five and ten-year cycle.

6.3 Strengthen mechanisms to facilitate connections to local churches for both Nazarene and non-Nazarene students.

Integrative Essay

Reflecting on the Review Process

The Institutional Proposal Phase . When PLNU’s first draft of an Institutional Proposal began to emerge in January of 2004, we were only beginning to grasp the implications of the process described in the WASC 2001 Handbook of Accreditation. We feel fortunate that the proposal document, approved in the early summer of that same year, had both the resilience and the flexibility to tolerate the growth in our understanding, and in that of WASC, about exactly what the expectations of the reaffirmation of accreditation process were. As a result, no changes in our proposal were necessary during the subsequent stages.

The Institutional Proposal identified focal points relative to capacity in the CPR and key issues to quality assurance processes and to institutional mission in the context of the EER. Without the impetus of the CPR, the institution likely would still be struggling to establish the wealth of data tables that we can appreciate today, perhaps the single best result of our “preparatory” phase in the entire process. Likewise, the document also cast a net for the substantial involvement of personnel across the University. In addition to this, the Criteria for Review (CFR’s) helped delineate the models for good practice both in the context of the CPR report and the EER report.

The Capacity and Preparatory Review . Beyond meeting CFR expectations in the CPR, the University was able to concentrate in this phase on areas of concern that WASC had helped us to identify ten years prior. By the conclusion of our work on this report for the visiting team’s arrival in October, 2006, the University could claim a data set that had become institutionalized; the establishment of educational effectiveness indicators by faculty consensus; in-depth consideration of key CFR’s; reflective essays on assessment, strategic planning, and diversity (three issues identified in the 1996-1997 cycle); the release of an Academic Plan (also called for in the previous WASC visit); and significant recommendations to the President related to capacity.

The University has now moved beyond the helpful report of the CPR visiting team and the Commission letter of February 27, 2007, and on through the completion of the EER report as we anticipate the team visit set for this fall. It is our strong feeling that the institution has begun to make substantial progress from the old mindset of enumerating programs and effectuating intuitive improvements to a healthier balance of meeting both capacity and effectiveness concerns. This progression became clearly apparent in the completion of the EER report cycle.

The Educational Effectiveness Review . Our work in the EER report reveals the institution’s commitment to a system that demonstrates educational infrastructure, educational outcomes at all levels, and a culture of inquiry and evidence . In this report, we focused on two broad areas: three specific quality assurance processes and the mission-related themes of teaching, shaping, and sending . For each of these six topics, we prepared a reflective essay.

ASSESSMENT (the first reflective essay) – The firm establishment of the Nichols Model through the intensive work of the Assessment Committee, linked soundly to academic and co-curricular units, has assisted us in evidencing the link between systematic assessment and program improvement. Both curricular and co-curricular units are now actively engaging in these efforts.

NEW PROGRAM DEVELOPMENT (the second reflective essay) – We are learning that program development is accomplished best in our circumstances when characterized by parallel processes for undergraduate and graduate levels, the clarification of specific measures to be addressed in that development process, attention to the dynamic of main campus and regional operations, and the unique challenges of program modifications in a capped environment on the main campus.

PROGRAM REVIEW (the third reflective essay) – Though program review is nothing new at the University, it has proven consistently to be the vehicle through which many improvements have been made by nearly all academic units. Its growing reliance on rigorous assessment efforts, in addition to capacity issues, is an indication of our next step in developing a revision that represents a fully mature system across all units.

THEME ONE: TO TEACH (the fourth reflective essay) – A broad-based conversation has been on-going concerning the improvement of teaching and learning. To the extent that we can embed missional and widely consensual goals more deeply into the general education curriculum, academic majors, and co-curricular units, together with the continuing of faculty scholarship at undergraduate and graduate levels, we can move more closely to the purposes of the University as a teaching-learning community.

THEME TWO: TO SHAPE (the fifth reflective essay) – While the University has a vast array of co-curricular programs, we are only in the beginning stages of more thorough analyses of outcomes. Many students at the undergraduate level find satisfying involvement in these programs, involvements that yield only anecdotal indications of growth and shaping. We intend to move on to rigorous assessments in the arenas we identify in this reflective essay: the spiritual, intellectual, social, emotional, leadership, and physical dimensions. The development and measurement of outcomes will allow us to ascertain the adequacy of program quality and assist in the creation a more natural connect between the curricular and co-curricular aspects of the student’s experience.

THEME THREE: TO SEND (the sixth reflective essay) – The final theme of sending reflects the University’s conviction that essential education includes the positive actions of our graduates in their immediate spheres of influence. This essay describes the University’s efforts to send our graduates into the three-fold areas of career, community, and church.

Accomplishments and Shortcomings . In a uniquely matter-of fact way, the 2004 Institutional Proposal included a list of anticipated outcomes as a result of the entire review process. The following section lists each of the eight expectations from that document and comments briefly on the results of the entire process three years later.

1. Extend our maturing institutional culture of evidence, incorporating critical data into strategic, operational, and academic planning. Since the last WASC reaffirmation of accreditation review process, we have made strides in bringing critical data elements into the planning process. As a foundation for establishing our culture of evidence, we have designed an impressive data set. To give guidance to our planning process, the Strategic Planning Committee also has developed a set of "input" performance indicators and much of its work in the summer of 2007 and the academic year 2007-08 will be devoted to refining this list and developing further "output" indicators. The Office of Institutional Research (IR) has developed a series of data reports to assist in academic planning regarding faculty resource allocation, class size analysis, enrollment trends, student retention, and academic success. The on-going and intensive work of the Assessment Committee and an improving program review process is assisting us in this development. However, we recognize the need to expand these efforts and continue developing a more mature institutional culture of evidence. We also need to find more effective ways to disseminate information and to document the changes that result from our assessments.

2. Ensure that the new mission statement is embraced, shared, and embodied throughout the institution. The first step in revamping our planning process was to restate our institutional mission, and this process took nearly a year to complete. The new mission identifies three primary institutional purposes that can be summarized in the phrases—To Teach, To Shape, To Send. During the Fall Faculty Convocation in 2003, members of the faculty spent an entire day together thinking about these core functions. Likewise, Student Development staff addressed these themes in workshop experiences at the beginning of the academic year. There was substantial agreement that these functions expressed the essential nature of the University and that more dialogue would be necessary to be sure that the functions are fully embraced and embodied. The thematic approach adopted for this Educational Effectiveness Review report provided the ideal opportunity for a significant cross-section of professional staff, faculty, and administrators to advance this goal. Indeed, it is common today, not merely to see them in print but more importantly, to hear these phrases in the everyday conversations that occur in a variety of contexts all across the institution.

3. Formalize the link between mission, vision, strategic planning, and the annual budgeting process. Now that we have a new mission statement and a vision statement to guide a revamped strategic planning process, we are developing more effective ways to link planning and budgeting. Both the CPR and the EER reports speak to this outcome. Recent adjustments in the membership of the Strategic Planning Committee to include key policy makers, and the President’s personal commitment to formalizing this link, give us confidence that previous informal, intuitive processes are maturing into formal, data-driven processes designed for the improving the attainment of our mission.

4. Develop a more specific and reliable set of outcomes measures and means of assessment for general education and the co-curricular. We address this goal head-on in the EER report by the very nature of the focus points that we chose. Through the work of the Assessment Committee and the program review process, we have made significant progress in the assessment of student learning outcomes, particularly as it relates to academic majors. With regard to general education, however, it has been difficult to develop an assessment strategy because our current goals are couched in general and aspirational language; specific, measurable outcomes cannot be identified easily. The on-going work of the General Education Task Force will assist us in identifying learning outcomes more clearly and gathering assessment data from graduating seniors. We also embrace this goal in the context of the themes of teaching, shaping, and sending students, our concluding reflective essays in the EER report.

5. Review and improve our quality assurance processes, focusing particularly on academic program review and the improvement of instruction. While we can claim cycles of program review in all 17 academic departments and schools for over a decade, our review confirms that it is time to assess this program along with other key quality assurance processes. The establishment of an active Assessment Committee, specialty accreditations, and new program development processes are already having an impact on the enrichment of program review and instruction. We have recognized again that student learning and the improvement of instruction are critical to our success.

6. Sustain discourse about what educational effectiveness means for Point Loma Nazarene University and sharpen our assessment of student learning outcomes. The reflective essays addressing institutional themes in the Educational Effectiveness Review report provided the structure for a serious, ongoing discussion about educational effectiveness and attention to student learning outcomes. A careful, prolonged process of embracing and implementing increased attention to learning outcomes across the University is critical as we work to develop an institution-wide culture of evidence. The institutionalization of assessment efforts is a strong ally in this cultural change in our community, including academic majors, centers, and co-curricular units.

7. Examine the impact of growth in graduate programs (new programs, new sites) with an eye to quality assurance and faculty governance processes. Since our main campus must operate within the constraints of an enrollment cap, we look for continued growth in off-campus programs, particularly at the graduate level, to reduce upward pressure on tuition. Both the CPR and the EER assisted the University in addressing this goal. As a result, we have made significant progress in infrastructural improvements and in the clarification of faculty governance issues. The hiring of a Vice Provost for Graduate Studies has produced impressive advances in procedures, and graduate programs are making intentional progress in assuring quality through both internal and external means.

8. Evaluate current strategies to enhance diversity and, in the process, renew our commitment to building a diverse and respectful University environment. John Wesley, an 18th Century English clergyman and reformer (from whom we trace our theological tradition) proclaimed in his theology of holy living that the world was his parish. He equated piety with love of God and neighbor, and he connected learning with practical acts of love and service. As an institution in the Wesleyan tradition, PLNU’s goal is to cultivate compassion, cultural sensitivity, and a passion for social outreach in our students. Because diversity is one of our Core Values, in addition to the fact that it was an issue highlighted by the last visit team and in our own institutional self-review, we made it the theme of one of the reflective essays in the CPR report. Beyond this, new data is emerging as a result of the EER report that will assist us in continuing our attention to building a respectful environment.

In sum, we have gained fresh understandings, identified shortcomings for which we made improvements, implemented the vast majority of CPR recommendations, and embraced the implications of the EER report at hand. We can now turn our attention to establishing an appropriate agenda for future action.

Looking to the Future

An accreditation review always provides a snapshot of what is happening in an institution’s journey. Point Loma Nazarene University is making strides toward incorporating the best practices in CPR and EER review and using those to move us along our pathway. Some of the initiatives we can identify at this point include the following:

Capacity and Preparatory Review:

• Fully embrace the strategic planning and budgeting protocols now in place;
• Develop strategies for continuing the increase in retention and graduation rates with special attention to disaggregated data; and
• Engage the academic community in further dialog on academic freedom in our institutional context as a Christian liberal arts institution.

Educational Effectiveness Review:

• Improve assessment data collection, systematic interpretation, and strategic use for institutional improvement;
• Address the implications of new program development in a capped (main campus) and growing environment (Regional Centers);
• Redesign program review processes to reflect both capacity and learning outcomes focuses; and
• Operationalize the desired outcomes flowing from the institution’s mission.

In short, Point Loma is continuing on its path of being a learning organization. Future planning will be focused on outcomes and evidence, not process and assertions. At the center of our institutional decision making will be the evaluation of how a given policy, program, or initiative enhances the quality of student learning achieved by those who invest their time, dollars, and lives on these impressive bluffs overlooking the Pacific. At the end of the day, it is the impact on students and not the beautiful views that defines life at Point Loma Nazarene University.