WASC Home > CPR Reports
Back

Appendix 13
Assessment Structure Evaluation: Executive Summary of Survey Results

April 8, 2005

Attached for review:
Grid of responses to scaled questions
Comments and responses to open ended questions
Commentary on results of scaled questions
Highlights of responses to open ended questions and comments

  1. Results from the PLNU WASC Prep-Review Assessment Structure Evaluation have been collected.
    • 50 Total number of surveys distributed
    • 31 returned
    • 62% return
    • Listing of returns by category provided on grid
  1. The following scaled questions received responses indicating cause for concern:
    • #2 – planning is a team effort in my unit
      Total of 1 or 2 as response= 5 or 16%
      Total of 1, 2 or 3 as response= 8 or 25%
    • #9 – Assessment is productive in guiding program improvement
      Total of 1 or 2 as response = 6 or 19%
      Total of 1, 2 or 3 as response = 15 or 48%
    • #10 – Assessment is productive in guiding curricular change
      Total of 1 or 2 as response = 8 or25_%
      Total of 1, 2 or 3 as response = 15 or 48%
    • #11 – Individuals in unit are supportive of assessment system
      Total of 1 or 2 as response = 4 or 13%
      Total of 1, 2 or 3 as response = 14 or 45%
  1. Overall Perceptions
    • Effective support is not being provided within the unit and from the assessment committee
    • Assessment efforts are not producing effective program or curriculum results, participants do not feel their efforts have result

  2. Recommendation to use a 4 point scale with no neutral or undecided column.

  3. Comments and open ended responses vary but some themes become evident:
    • The committee is well received and most understand its function
    • Administration needs to make clear the importance of the work to foster buy-in
    • Mentoring by the committee is critical
    • Assessment details, calendar, etc. need to be accessible to units

alt


Point Loma Nazarene University
WASC Preparatory Review
Assessment Structure Evaluation

Comments and Open Ended Questions

Comments:

#3 Our office is a service organization, so most of our goals relate to service, not to student learning.

#8 We are still in process toward finalizing assessment.  We talk a lot about and collect data but to date have not used the data much except in conversation.

#12 I am not sufficiently tied into the work of the Institutional Assessment Committee.  This is my fault and I need to work to see that change.

#14 It is difficult to answer these questions for 8 departments.  The results are mixed, thus far not much change has taken place as the result of assessment, some are enthused, some are put-off.

#15 A Team effort and an atmosphere of collegial support exist in my committee.

#25 For the most part the process is useful, but the precise step-by-step tasks, submittals and time frame are not completely clear.  Is this information in place somewhere?

#16 Use our own assessment system, not the Nichols—Please don’t enforce Nichols for the school of Nursing.  We have already been doing internal assessment for over 20 years.  The systems identified by both the BRN (Board of Registered Nursing) and CCNE (Commission on Collegiate Nursing Education) provide a framework for CQI.  Adding the PLNU component of Nichols model seems highly burdensome.

#25 We could use some advice on creating an effective exit exam for studio art majors.

#29 I am not familiar with it enough to make comments or answer the open-ended questions.

#36 Department Chair provides little leadership.

#38 Some of this is hard for me to answer since I received extensive support from a NSF grant on program assessment.  I am not sure that I could have gotten our program set up just based on the support I received from the PLNU process.  I found the instructions sometimes confusing and the example system I was shown (Lit etc) excessively complicated. 

Our department does have a system with which we are happy and can implement with a minimum amount of pain.

#41 I applaud the efforts of the Assessment Committee.  The university is beginning to develop a true culture of evidence. PA

#46 The School of Theology and Christian Ministry has not done a good job at analyzing the data collected.  As a result, assessment results have not been significantly used for program improvement and curricular change.

#47 We are in the process of setting up our assessment system in the School of Education—the Nichols model fits well or vice versa.

#48 There is good structure and support.

Open Ended Questions: In this section, please comment on the effectiveness of the assessment planning structure, and your suggestions for improvements.

1. What is working well about the current assessment planning structure?

#1. We are always being reviewed by outside agencies so we are aware of process.

#3 Great Committee.

#2 Mentoring by committee, comments on documentation and assistance with defining goals.

#6 The process is well defined, systematic, and common across the University.  The shared resources and knowledge of the process provides useful data for action and improvement.

#7 Getting people to think in terms of data-driven program improvement.

#9 The fact that expectations are clear and meetings are regular.

#10 Strong support and offers of assistance to schools/departments/ co-curricular areas from the Assessment Committee.

#12 I am afraid that I am so separated from the institution-wide efforts that I wouldn’t know what is and isn’t working well.  We in Education have been so overwhelmed with so many other important tasks that we’ve fallen behind on this one as it relates to the institution as a whole.  Having said that I need to emphasize, however, that we are held accountable in regular ongoing reviews of our programs and objectives by the State---but this is not enough.  Keeping up with State requirements has preoccupied us for several years.

#14 Assessment directions and plan are easy to understand.

#15 Distance is somewhat of a challenge, but I don’t think there is a solution for it.

#16 Our evaluation plan for accreditation purposes is helpful to evaluate student learning and success in meeting outcomes.  Certainly our NCLEX pass rate, graduate employment, portfolio completion and graduate survey data show effectiveness.

#24 Chair person is doing a wonderful job of leading the process.

#30 It is a team effort in our department.  Department meetings are used for review of assessment tool and analysis.

#33 I am not sure.  This is due to my lack of basic interest and belief in assessment rather than weaknesses in the planning structure itself.

#37 It’s very clear—good to allow time to develop plans, then to implement, then to figure out necessary changes.

#40 The Faculty and Administration are all aware of key areas of assessment. The WASC Committee structures seem to be functioning and productive. 

Each department has flexibility to develop measures that are customized.

#41 Broad involvement
Good overall assessment design
Faculty owned

Documented departmental changes

#46 Well, we’re doing an OK job collecting data.

#47 Great framework—to implement the School of Education’s model

2. How would you revise the current assessment system to make it more effective to meet your needs?

#2 I think it works well.

#3 Don’t know.

#7 It needs ore emphasis from the university leadership to motivate everyone to get on board.

#9 There is a lot of on-going assessment that occurs in Student Development that is not captured via the Nichols Model. It is a function of time that I (Caye) have not captured this data in another manner.

#12 The needs of Education are highly specific.  To hold us accountable on a long term basis, I have  been pushing for NCATE accreditation.  Assessment, of course, is central to getting and maintaining this accreditation.   NCATE accreditation takes us to a much deeper level of accountability than does the State accrediting agency.  With the growth in our off-the-main-campus programs, we are setting ourselves up for major failure, if we don’t have a first class assessment system in place.  I consider this the top priority for the School of Education.

#14 Assessment is “on top” of all of the other activities. At this point I don’t think we have seen it as incorporated into the general flow of things.  It is a great deal of work for the departments

#16 I would allow already established assessment policies that are working to suffice as the recommended data.  Nursing is already over-regulated!

#33 I wish I could gather the energy and interest to care deeply enough to ‘see’ ways to improve and upgrade assessment.

#35 It needs to be built into the departmental budgets because it is a cost that has been added without being budgeted for.

#36 Make sure chairs are “on board” with assessment.

#37 More concrete examples of exactly how to operationalize assessment—help in creating opportunities for assessment (like developing senior seminar courses).

#38 It appears that the committee was set up to get the program running and to offer support where needed.  Now that we have a workable system in our department, it seems that the function of the Assessment Committee is more one of monitoring that assessment is being done campus wide.

#40 Increased understanding of the strategic plan, more direct feedback re: expectations / guidelines.  (This suggestion is made with recognition that it is hard to balance the benefit of flexibility with greater clarity and directions.)

It would be helpful to have a calendar of “milestones” from the committee. 

#41 Have departments more carefully and precisely define and measure intended outcomes
Permit GE assessment to guide curricular change – on an incremental basis.

#46 It would be nice if it were better integrated into other assessment events—IDEA/SIR forms, 5 year program review, WASC accreditation review and so on.  As it is, we’re fighting on multiple fronts.

#47 NA until we actually do something!

#48 No, real revision it just seems overwhelming at times the amount of “extra” duties that need to be taken care of.  I am not very good in the area of program assessment but have been encouraged and helped, what more could I ask for?

#49 Spell things out a bit more clearly.  Timelines are helpful.  Please send things numerous times via different modes, e.g., Email, snail mail, etc.

3. In your opinion, what should be the role of the Assessment Committee?

#2 Guide, assist in establishing department goals and carrying out assessment, coordinator of documentation.

#3 Support and advise others.

#6 The Assessment Committee should provide continuing assessment of the process and effectiveness of the process as a means to keep quality improvement occurring across the University.  Opportunities for training, enhancement of expertise, and use of assessment are important if we are to keep an effective system in operation.

#7 Watchdog to ensure all units are meeting requirements, to goad to spur compliance and reporting of best practices and guidance.

#8 To provide ideas, materials, and models and to demand the school use the material provided or develop their own.

#9 To define assessment expectations for the campus and to centralize the resulting data.

#10 Overseeing the support and instruction to various groups involved in the assessment process.

#12 To help to ensure that our unit (Education) is working within and is compatible with the framework of the university as a whole and to help monitor and hold our unit accountable to university-wide objectives.

#14 Someone has to make sure this is done, encourages the work, and sets the standards.  I see those things as the committee’s role.

#15 To support the overall efforts of the institution in accomplishing the assessment goals for WASC as well as for on-going program assessment and program improvement and to effect change for on-going program improvement efforts.

#16 Be supportive to existing methods and act as advisory when assistance is warranted.  (When something is not broken, don’t try to fix it!)

#24 Guidance and establishing criteria to be used.

#33 I am not sure.

#35 Review assessment data and keep reporting units accountable for follow through on assessment commitments.

#36 Information dissemination about assessment, data collection and interpretation as it is received from various units.

#40 Feedback form the initial submissions to the assessment committee were not very helpful. Later, the examples and context provided by committee members was more beneficial. 

The role of the committee should be to foster awareness of the expectations, provide guidance & support to the departments and offer technical assistance or capacity-building resources to the process, and provide feedback on where we ‘stand’ in the process.

#41 To oversee the ongoing efforts institution-wide and to be a cheerleader for the institution.

#46 I don’t know.  I mainly need just a single person whom I can contact and get help.

#47 Provide collegial support for units to understand how to implement their system.

#48 Structure and Support.

#49 Resource

Ending Comments:

#16 Thank you for your continuing support and for doing your due diligence.

#25 Overall, assessment is very useful.  Our program has benefited from the feedback and data we have collected.

#27 I am not really familiar with the Nichols model (and am embarrassed to admit it).  My response to this is probably not accurate as I am only four months into this new position.  I have forwarded it to a couple of colleagues who are probably better prepared to respond.

#33 I am truly sorry I am not an encourager in this area.

#48 I think the committee is doing their job, I am not doing my part adequately.

 


Responses to Highlighted Comments and
Answers to Open Ended Questions

Possible actions or recommendations resulting from comments and answers to open ended questions follow.  The numbers refer to those assigned in the document titled: Assessment Structure Evaluation Comments and Open Ended Questions.

Comments Section:

Number:  Action or recommendation:

3

Assessment committee should include a broad enough scope for both curricular and co-curricular depts. to feel and relate to the value of assessment

12

Demonstrate Connectivity to assessment efforts – importance of assessment to service and learning efforts

25

Insure access to, and awareness of, available information.  Website, etc.

16

Provide connectivity and supportive/accommodative ways for assessment efforts to be incorporated within the framework of the Nichols model

Open Ended Questions Section:

Q and #: Action or recommendation:

1, #2

AC should consider being more proactive in increasing awareness of assessment committee purposes, resources, and procedures.  Help others to see the AC as a resource and support rather than as a critical or authoritarian body.

1, #30

Assist in developing and fostering cooperative team efforts in departments.

2, #7

Keep the administrative leadership support visible and encouraging towards assessment efforts and the efforts of the committee.  Admin should provide continual reinforcement that assessment efforts are not just for institutional or program review, such as WASC, but that assessment is on going and vital to overall program effectiveness.

2, #35

AC needs to work with Admin to insure appropriate budgetary support for the assessment efforts.

2, #36

AC should consider regular updates and communication with chairs.

2, #40

AC should post a calendar of events, deadlines, etc.

3, #6 & #8

Make sure individuals see AC as a resource and that it functions in a facilitative role. 

3, #33 & #46

Stress effective communication to insure that all are aware and see the importance of institutional assessment efforts.