Questions & Answers


Questions and Answers about Assessment

Click a question to jump directly to the answer or read the document in the usual way.

 

 

1.   What is "an assessment"?
2.   What is the purpose of assessment?
3.   In answering the previous question, you said the chief purpose is improvement. What are its other purposes?
4.   Who and what gets assessed?
5.   How is an assessment performed?
6.   "Mission" was not mentioned in the assessment cycle. How does it figure into things?
7.   The relation between mission and assessment seems straightforward, but then there is discussion of "goals" and "outcomes." How do these relate?
8.   How do measures of success differ from outcomes?
9.   How often is an assessment performed?
10.   Who reviews and uses the results of an assessment?
11.   Does the University have an assessment plan?
12.   What does Opportunitas in the 21st Century: Seizing the Moment for 2010-2015, have to say about assessment?
13.   I am a faculty member, and I am concerned that more closely prescribed learning objectives will detract from freedom for curricular spontaneity. Being blunt about it, I am concerned about micromanagement and loosing control of my classroom.
14.   It looks like assessment will require significant effort from faculty members. Where am I expected to find the time and energy for these additional chores?
15.   Assessment interests me. How can I become more involved?
16.   Outcomes assessment is fine for accounting, nursing, and computing where learning objectives may be defined with respect to concrete competencies. However, does it not seem to trivialize the higher purposes of the liberal arts?

~~~~~~~~~~

 

 

 

Questions and Answers

1.   What is "an assessment"?
    An assessment is an empirical inquiry into the effectiveness of any aspect of University activity or life: the Core Curriculum, a major, an academic support unit, or an administrative unit.
    Assessment's sight is on gauging how successfully Pace is developing talent and identifying aspects of programs or students' experience that make important contributions. Illustrative of assessment's application are the following:
   
  • With respect to the Core Curriculum, an assessment
    could be a study on how well it is achieving any
    one of its objectives of fostering communication skills,
    quantitative skills, critical thinking, ethical maturity,
    and civic responsibility.
     
  • For a major holding a professional objective, the
    caliber of abilities and skills delivered might be
    inferred through comments from CO-OP supervisors or
    through the success of students on licensure
    examinations.
     
  • The caliber of an academic major might be evidenced
    by student portfolios or course portfolios.
    An assessment is conducted with a specific objective and in accordance with the same methodology required for rigor in education, psychology, and the social sciences. It is neither cavalier nor capricious. [top]



 

2.   What is the purpose of assessment?
    To improve student learning is always the chief objective. This may occur through studies to strengthen a curriculum and to strengthen its delivery, to upgrade campus services, to enhance the interpersonal climate, and, in general, to enrich "the Pace Experience." [top]



 

3.   In answering the previous question, you said the chief purpose is improvement. What are its other purposes?
    Other purposes include garnering data to fulfill accreditation requirements of the Middle States Commission on Higher Education as well as those of professional accrediting organizations including the Association to Advance Collegiate Schools of Business, the Commission on Collegiate Nursing Education, the National Council for Accreditation of Teacher Education, the Accreditation Board for Engineering and Technology (for computer science and Information systems), and more.
    The federal government in reauthorizing the Higher Education Act, and other sources of grants and loans for student tuition, may require statistics on retention, graduation rates, and employment success. There are definite movements afoot to hold institutions of higher education accountable for performing effectively and efficiently. There is no doubt that we will be called upon to provide ever more data that convincingly documents efficacy.
    Maintaining fiscal health requires decisions about the allocation of resources. Assessments may be made of efficiencies and cost effectiveness.
    The bottom line is always University betterment, from one perspective or another. One faculty member in computer science remarked that professional integrity was her motivation to undertake a curriculum assessment. She wanted to know, not merely to suppose, that the courses were pitched to equip students with the right skills. All assessment activities, in the final analysis, aim at helping us to do our best for those who are depending on us. [top]



 

4.   Who and what gets assessed?
    Programs, majors, services, and the University as a whole are assessed. The Core Curriculum may be assessed; the Bachelor of Science in Computer Science may be assessed; the quality of service provided by the Mortola Library may be assessed; the ability of Student Life to foster student development may be assessed; Pace's ability to engender student engagement may be assessed; how well we do with “at risk” students may be assessed; and the list goes on.
    Individuals are not assessed, they are subject to performance reviews. The performance review process is separate and independent of assessment, and it should not be changed by assessment activities. The data collected for assessments should not be used for performance reviews (unless one's job is to perform assessment). Assessment's view is from thirty thousand feet, where aggregated effectiveness is clear but not the contributions of individual members of the faculty and staff. [top]


5.   How is an assessment performed?
    There is no one way to perform an assessment. Its conduct is determined by its goal and ranges over the gamut of methods for social research, from in-depth open-ended interviewing in focus groups to the analysis of institutional statistics. The only requirement is to strive for trustworthy and compelling data, be they qualitative or quantitative.
    When it comes to assessing learning and the personal development of students, indicators rooted directly in their performance, products, attitudes, and opinions are more forceful than arms length correlates such as number of books in the library or average class size. "Process" may be a useful source of assessment information, such the means through which collaborative skills are developed within a program or how tutoring is made available.
    The best non-specific guidance on performing assessment is to follow what is known as "the assessment cycle." It involves:
   
  • identifying "an outcome" to assess
     
  • determining the measure(s) indicative of success
     
  • gathering the data
     
  • evaluating the data with an eye toward formulating improvements
     
  • making the improvement
     
  • gathering data again to confirm the gains
    Improving a weakness should not unintentionally detract from a strength. [top]



 

6.   "Mission" was not mentioned in the assessment cycle. How does it figure into things?
    “Mission” is what a department, program, major, or support unit is out to accomplish; its raison d’etre. Assessment is nothing more than establishing that a body fulfills its mission, which is to say that it does what it purports to do. [top]



 

7.   The relation between mission and assessment seems straightforward, but then there is discussion of "goals" and "outcomes." How do these relate?
    A mission is an expression of purpose, but it is usually cast as a “high purpose,” in terms of ideals. Goals are the elements of a more detailed description – the components identified in answering “what does this mission really mean?” Outcomes are the demonstrable masteries or abilities following from each goal. Going down the abstraction ladder, mission expands into goals, and goals are realized as outcomes.
   
To exemplify, the mission of the computer science major may be to offer a depth of understanding in computer science.
Goals could be the programming proficiency, ability to analyze the time and space requirements of algorithms; knowledge of contemporary techniques for sorting and searching; familiarity with the practice of large-scale system analysis, design, coding,testing, and maintenance; an understanding of data communications networks, including the use of tools for simulating their performance.
Outcomes could be the ability to solve problems of types x, y, and z; explain concepts a and b; give examples of i, j, and k; diagram processes m and n; and so forth.
If a goal is to equip students for meaningful careers in software development, outcomes could be reflected in the employment profile of each class compiled six months after graduation and again three, six, and nine years out.
[top]


8.   How do measures of success differ from outcomes?
    Outcomes are demonstrable abilities. Measures are particular indicators, or the particular test items, that are used for an assessment. Better readings are obtained when there is more than one measure per outcome.
    If the goals of a course are what appears in its description in the catalog, outcomes might be what the instructor tells the students will be on the final. The measures are the specific test items.
    When it comes to measuring students' development and experiences, a variety of instruments have been used:
   
  • departmental tests (guided by what is taught)
     
  • national achievement tests (guided by academic consensus, such as the Educational Testing Service's Major Field Test)
     
  • licensure tests (guided by professional relevance)
     
  • certification tests
     
  • placement tests
     
  • surveys such as the National Survey on Student Engagement (NSSE)
     
  • psychological instruments as described in the Mental Measurements Yearbook
     
  • class portfolios (samples of excellent, good, and weak exams and projects)
     
  • essays (each essay might be evaluated on a set of explicit criteria by three independent judges, and their judgments pooled into an overall rating)
     
  • student portfolios
     
  • students' self reflections
     
  • focus groups
     
  • alumni achievements
     
  • alumni comments
     
  • ratings from internships supervisors
     
  • employment experiences of new graduates
   [top]

 

9.   How often is an assessment performed?
    This is like asking, “How often need we reflect on the quality of our services and goods?”  Assessment, like quality control, must be on-going.  Those who are sensitized to the excellence that builds from assessment speak of “a culture of assessment.” 
    Just as individuals complete forms pertinent to their performance review on an annual basis, it is likely that OPAIR will expect specified units to submit an assessment report each year.
    [OPAIR, formerly OPARAB, is the Office of Planning, Assessment, and Institutional Research.]
    [top]

  

10.   Who reviews and uses the results of an assessment?
    While preserving the privacy of students, faculty, and staff is a absolute requirement; the findings will be seen first by the individual(s) in charge of an ad hoc unit, school, department, or program.  They engineered the study, and they are the ones responsible for interpreting and using the results. 
    OPAIR will ultimately serve as a repository for outcomes studies and reports (as it likewise serves as a source of assessment resources and assistance).  This will allow methodologies and findings to be shared internally and enable the University to comply with external requests for information. [top]

 

 

11.   Does the University have an assessment plan?
   

OPAIR, with a University-wide assessment committee, has been working on a document titled the University Assessment Plan since the summer of 2002.  The plan itself, in the section titled “Assessment Plan Procedures,” is a set of general requirements.  Individual units are responsible for designing plans of their own that satisfy these.
   

Essentially, each Unit, School, Department, or Program earmarked for assessment will need to identify an individual to manage assessment activities.  A mission statement will need to be drawn-up along with derivative goals and outcomes.  Measurements for outcomes will need to be selected and a timeframe for gathering and reporting data will need to be formulated.  These assessment designs were due in the Provost’s Office beginning in the summer of 2004. 
   

OPAIR is eager to help in the development of these assessment designs, and constructive feedback will be forthcoming on submissions to the Provost’s Office. 
   

OPAIR has been active in assessment for some time, and before OPAIR the Office of Institutional Statistics. High profile global assessment activities include the annual Pace Scorecard:  Some Comparative Results for Benchmark Institutions and the annual National Survey of Student Engagement.  Assessment is an exigency of the times, and all of us need to pull together and get on with it.  Anyone who doubts this should read Student Learning Assessment:  Options and Resources published by the Middle States Commission on Higher Education or simply keep up to date with the Chronicle of Higher Education (a recent piece is the “Point of View,” “Legislation to Improve Graduation Rates Could Have the Opposite Effect” by Watson Scott Swail). . [top]

 

 

12.   What does Opportunitas in the 21st Century: Seizing the Moment for 2010-2015, have to say about assessment? 
   

The current Strategic Plan has directed that assessment of all areas of the University will be given top priority. 
   

GOAL I: Advance Pace University’s Academic Programs begins “Seizing the opportunity presented by this moment of high stress in the American economy and in higher education requires a relentless focus on making Pace education better and more effective every year. The expectation of constant improvement is demanding. It requires continuous renewal and development of the faculty; stringent institutional self-examination, assessment and benchmarking…”   One of the goals within this section, Ib4, states “Articulate clear program evaluation criteria the University will use in deciding which academic programs to encourage and support financially.” 
   

Goal VI. Strengthen and Reinforce a Culture of Accountability highlights assessment again.  The two major sections within this goal are:
VIa: Establish and consistently use clear metrics to evaluate and strengthen the University’s programs, activities, and services.
VIb: Annually measure progress in achieving Pace’s strategic goals and objectives
   

The Annual Reports of the Implementation of the Strategic Plan can be found on the Strategic Plan website at http://strategicplan.blogs.pace.edu/

[top]

 

 

 

13.  

I am a faculty member, and I am concerned that more closely prescribed learning objectives will detract from freedom for curricular spontaneity.  Being blunt about it, I am concerned about micromanagement and loosing control of my classroom.
   

You will not lose control of your classroom.  Lower level classes have always had a specified curriculum, especially courses that laid the foundation for later courses (e.g.  Calculus I, American Civilization to 1877).  Instructors taught them using the methods they deemed best and embellished them in accordance with their interests.  Assessment will not change this.
   

At the other end of the continuum, special topics courses, seminars, learning communities, and the like remain welcome. You may develop these yourself or in collaboration with colleagues, whatever the tradition in your Department or School.  Still and all, assessment canons call for a course description and a syllabus with goals and objectives. 

[top]

 


 

14.   It looks like assessment will require significant effort from faculty members.  Where am I expected to find the time and energy for these additional chores?
    Burdensome expectations are bundled into the role of the faculty member: teaching; publishing; mentoring students and junior faculty members; performing departmental, school, and University-wide service; demonstrating civic engagement; and attending to the emotional labors of listening to students with problems.  Is this reasonable?  Assessment provides the perfect venue for investigating the stresses, conflicts, and dysfunctions within the professorate and seeking mechanisms for reducing these.
 [top]

 

 

15.   Assessment interests me.  How can I become more involved?
   

The two things to do are to read-up on it and to volunteer to work on it with whomever is at the helm in your domain. There is certainly so much work to be done that your  creativity and labors will be valued.


The following resources may be a good place to start your reading: 
    
by  ASTIN, Alexander W.
           Assessment for Excellence
           Oryx Press (Phoenix, AZ; 1991)
           ISBN:  0-89774-805-0


by  the Middle States Commission on Higher Education:

           Framework for Outcomes Assessment – Second Edition
           (1996)

           Student Learning Assessment:  Options and Resources
           (2003)

 

Astin’s book is filled with fascinating insights on learning and testing and casts assessment at its best, which is as rigorously designed and implemented social research.  The handbooks from Middle States are down-to-earth manuals on what we need to be doing to remain in business.

Other resources are cited on our Website, and some are just a mouse-click away.

[top]

 

 

16.   Outcomes assessment is fine for accounting, nursing, and computing where learning objectives may be defined with respect to concrete competencies.  However, does it not seem to trivialize the higher purposes of the liberal arts?
    Measures of mastery are, by their nature, less lofty than the high-minded goals they represent.  They are not purported to have breadth, depth, or anything profound; merely a degree of representative validity.
    Teaching to think like an historian or a mathematician may be your aim, not the laundry list of outcomes enumerated for an assessment.  Yet an educated perspective consisting of the ability to think, to absorb information, to communicate, and to persevere is a gradually forming internal synthesis.  It is like wisdom.  When it arises, it is a majestic metamorphosis derived from a mix of pedestrian pieces.
 [top]