Bush Faculty Development Committee
Anne Oppegard and Jacquelyn Howell attended the November 20-21 Collaboration Conference in Minneapolis. The conference theme was Coming Full Circle: Using Assessment to Transform Teaching and Learning. Following are reflections from Anne and Jackie on some of the sessions they attended.
Assessment's Power To Transform Teaching
by Anne Oppegard
Dr. Trudy Banta presented the opening keynote address at the fall conference of the Collaboration for the Advancement of College Teaching & Learning held at the Radisson South in Bloomington, MN, on November 20 & 21.
The Council for Higher Education Accreditation (CHEA) has promulgated the national goals assessment which the regional accrediting bodies (including our North Central) now employ in their campus/program reviews. The impact which CHEA and the regional bodies have had on higher education is apparent when one considers the statistic that 94% of U.S. institutions are engaged in assessment at some level. (1995 American Council on Education "Campus Trends" survey of chief academic officers). Yet Dr. Banta characterized this involvement in assessment as a "thin veneer." Two national surveys have gathered data which indicate that accreditation spurs on episodic engagement and that some institutions literally have to begin again with assessment as the site visit of the regional accrediting body looms on the horizon. Furthermore, the results of these surveys indicate that use of the results is rare! This observation provided strong support for the purpose of the conference, as indicted by its title: "Coming Full Circle: Using Assessment to Transform Teaching and Learning."
So how does an institution engender an on-going, non-episodic assessment effort? Banta maintains that while collaboration is the key, it is also a potential stumbling block to a successful assessment program. Faculty need to collaborate to set program outcomes, to choose measures, to interpret the findings, and to make responsive changes. Yet faculty function in an environment which, through its tradition, raises many barriers to the requisite collaboration: faculty typically train, accept positions, undertake scholarly pursuits, and present themselves for promotion and tenure as individuals. Institutions must provide strong leadership to overcome these real or perceived barriers. Augustana's Bush "Mini-Grants" is a wonderful example of support for cross-discipline collaborative efforts.
In addition to collaboration, Dr. Banta cited "success factors" including committed leadership, a supportive campus climate, assessment which is designed by the faculty rather than imposed, and the effective communication of results followed by conscientious follow-up. Only when the circle is closed through follow-up will assessment transform teaching.
Developing a Sense of Urgency for Change and for Using Assessment
by Jacquelyn Howell
Lion Gardiner, Rutgers University Professor, began with the announcement that a web-based university (Western Governors' University) recently applied for regional accreditation (fall, 1998) and may be able to deliver a quality program that is accessible and reduces barriers of time, distance, and scheduling and is responsive to outcomes desired by employers. He anticipates this institution will be accredited. Thus, he believes, web-based institutions have the potential for being real competition to traditional undergraduate education. Gardiner identified sources of complacency in higher education and encouraged faculty to become more informed on the research on outcomes and best practices related to education.
Gardiner challenged us to focus daily/weekly (every class session) on achieving overall outcomes of the program and on student outcomes. Areas that he highlighted include critical thinking and the ability to make moral judgments. He noted that "lots more is taught than learned in every course."
Gardiner raised the following questions for group dialogue: What is your students' potential for learning? How do you know? What is an acceptable minimal level of education? What are the implications for students and society? What is the urgency for change on your campus? What are the implications for students and society?
We did an exercise on evaluating student responses
to a one-paragraph essay on "What specific characteristics distinguish
science from all other human endeavors? Be specific." We were supplied
criteria for comprehension of content that defined specifically what was
meant by (4) thorough comprehension, (3) good comprehension, (2) unsatisfactory
level of comprehension, and (1) little comprehension. Evaluations by participating
faculty spanned two or three categories for most student responses. A personal
reflection: Is this same variation in evaluation of student responses evident
on our campus? Is this the basis for student focus on "what do you
want?" rather than on the content to be mastered?
"And Now, A Word From Our Sponsor . . ."
by Anne Oppegard
Two back-to-back sessions were conducted by Dr. Cecilia L. Lopez, Associate Director, Commission on Institutions of Higher Education, North Central Association of Colleges and Schools. The first was entitled "Transforming How We Look at Assessing Student Learning;" the second was billed as a question and answer session. Dr. Lopez answered the audience's fundamental question within moments of convening her first session: Question: "Is North Central's assessment of student learning a passing fad?"Answer: [a resounding] "No!!!" Having set the tone of the session, Dr. Lopez moved into the body of her presentation.
Baccalaureate institutions face three main challenges, according to Lopez. First, the public institutions are confronted with an ever-increasing portion of their operating budget from performance-based funding. While Augustana may not directly feel the impact of this issue, the same concept clearly applies when we consider that our catalog makes certain promises which society then (reasonably) expects a graduate of Augustana to ably demonstrate. So, even though we are not held answerable to a state legislature or a public oversight board such as a state board of higher education, we can expect to be held to certain standards by our constituent groups. Secondly, the drive for outcomes assessment did not originate with the regional accrediting bodies but rather the Council for Higher Education Accreditation (CHEA). CHEA has now mandated that all "regionals" focus on institutional credibility and consider how an institution's issues of quality assurance and quality improvement are operationalized.
The third (and most significant) challenge Lopez presented can be summarized with the phrase "Market-Driven Providers." Lopez maintains that this challenge must be the compelling and overarching reason that an institution takes outcomes assessment and program review to heart. She cited a long list of dizzying scenarios which illustrate the dramatic paradigm shift that is occurring in higher education. A recent survey revealed that 83% of 35 state governors surveyed believed that anytime-anyplace education was "important" to "very important" while 3% of the same 35 considered traditional education and tenure "important" to "very important." Lopez spoke at length about the emergence of what she called mega-universities: eleven institutions which have an enrollment totaling 3.8 million students. The British Open University (current enrollment is approximately 500,000) is one such institution; it has recently applied for incorporation in Delaware and is seeking accreditation by the Middle States Association. It is also partnering with Western Governors' University and Florida State. Colorado Electronic College brokers courses and degrees in much the same way as Western Governors does. Such an institution provides one-stop shopping in the catalogs of all participating institutions. In addition to these mega-universities, corporate education is making great strides. Recently Motorola University began a drive to seek accreditation and Kaplan has put a law degree on-line and intends to seek accreditation for that program. (A non-surprising aside: the ABA has vowed a vigorous fight against the Kaplan efforts.) As a final example of avant-garde educations, Lopez cited New York University which is forming a for-profit subsidiary to deliver certain programs and degrees.
What is the implication of these changes for "traditional" institutions? Predictions have been made that one-third of current institutions will not survive the sea change in education. Lopez believes that only those institutions that are well-financed and that fill a distinctive niche will survive. She believes that those with heavy investments in on-site adult education will be the first to collapse since that is the very market that the 'virtual universities' will go after. She believes that the "threat" that traditional education is facing can be first turned into a challenge, and then further reconstituted as an opportunity. That opportunity is that, now and in the future, more than ever before, our assessment of outcomes and our response to the results of that assessment will serve as hallmarks of truly excellent and sustainable institutions.
She is adamant that administrations must provide "enthusiastic and active support," both with time and budget. Assessment must become a line item in the E&G budget; thus providing on-going resources to sustain the assessment efforts. Furthermore, she cautioned against the use of an ad hoc committee to oversee assessment efforts since the very notion of an ad hoc group sends a message of temporary efforts. Lopez assured her listeners that visiting teams search for evidence that the infrastructure which supports assessment is in place. In words which echoed portions of Dr. Trudy Banta's keynote address, Lopez stressed that episodic assessment is virtually worthless as an institutional enhancement.
Assessment must be done by the faculty, for the faculty, for the students, and for the institution, but not for North Central! Ultimately, assessment results should be incorporated into program review. While outcomes assessment is on-going and systematic, it feeds information to the periodic program review. Together, on-going assessment and periodic program review enhance the institution's effectiveness in planning and budgeting.
If outcomes assessment becomes institutionalized and is an evident part of the campus culture, Lopez believes that an institution can only reap rewards from its efforts. She encourages faculty to become mobilized and energized rather than frightened by the prospect of outcomes assessment. We (institutions and programs) must identify our mission -- purpose -- niche. If these elements cannot be defined then they cannot be operationalized, and if they cannot be operationalized, they cannot be assessed.
WEB SITES TO VISIT:
The American Open University, based on the British Open University, which has been enrolling students since 1983
A vast site sponsored by the Annenberg/CPB Project (the Corporation for Public Broadcasting with funding from the Annenberg School of Communications). Prepare to spend some time here! The sub-site http://www.learner.org/edtech/distlearn focuses on distance education.
Assessing Skill Development: A University of Minnesota/
Inver Hills Community College Pilot Project
by Jacquelyn Howell
The team from Inver Hills Community College and the University of Minnesota presented a model they have developed for the development of ten essential skills for the 20th century through a liberal arts education. These skills are:
These skills are paired in the model to depict relationships that are two aspects of the same process. The goal is to document specific skill development that is measured as it is intentionally developed. The long term plan will be for students to develop a profile that documents accomplishments in the above areas. Students apply to participate in this program. Courses in which development and measurement of one or more skills is documented in a computer data base are designated LS/PS (liberal studies, professional skills) courses. Currently, 29 sections involving 21 courses and 16 faculty are in place. The process is very complex to differentiate personal abilities and to identify a person's strengths and weaknesses. The emphasis is on student control in broad development of skills. It is anticipated that students will be able to present this evidence of skill performance to prospective employers. It looks like a serious attempt to foster development of the broad intellectual and interpersonal skills that we all know are important to lifelong learning and productivity in the workplace.
This team cautioned that faculty need to begin small and tackle serious work with students on these skills 1 or 2 skills at a time in 1 or 2 courses at a time so that faculty don't drown in their own efforts. The total profile includes 50 skills distributed among the ten skill areas. This project represents a very substantial effort on outcome assessment tailored to the individual student.
Assessment Strategies and Institutional Realities:
U of Wisconsin Platteville
by Jacquelyn Howell
Three faculty at the U of Wisconsin-Platteville presented their experiences with assessment at the campus-wide level. Serious efforts at assessment began in 1991 following a mandate by the Board of Regents that outcomes assessment on the 13 system campuses occur every five years and the NCA criteria which also requires outcome assessment.
George Smith identified some of the challenges/barriers to assessment as well as factors that contributed to success on their campus. He highlighted ways to achieve a broad base of faculty involvement in assessment activities and a sense of faculty ownership in the assessment process. He provided rationale for using a combination of national standardized tests (ACT CAPP tests or ETS tests) and home-grown tests that match particular program outcomes.
John Simonson, Professor of Economics, identified the greatest challenge in the assessment process as utilization of assessment test results to impact curricular improvements.
Program review of all programs now happens on a three-year rotational basis and includes four assessment-type questions: What are your program's goals and objectives? What output measures are currently being used? How are results of your output measures being used to improve the program? What noteworthy problems is the department having in meeting its goals and objectives?
Terry Liska, Professor of Economics, identified costs and benefits of their efforts at assessment. He stated, "The biggest advantage of assessment is forcing departments, colleges, and universities to ask basic questions about their goals and objectives." Costs include costs of testing materials and staff to score and analyze data as well as time and efforts of faculty and students who participate in the assessment process. Benefits include attention to course objectives and pedagogy, and heightened interest in scholarly activities involving education in the discipline.
Closing Plenary Session:
Using Research to Produce Dramatic Gains in Student Learning
Lion Gardiner, Rutgers University, highlighted research findings from his book Redesigning Higher Education: Producing Dramatic Gains in Student Learning (1997). His passion is for faculty to read, discuss, reflect on, debate, and use the research on student learning.
Lion highlighted research which identifies key competencies required of our graduates for their success as citizens, employees, and leaders and the research that provides guidance for how to develop these key competencies. He cited research that showed a high correlation between student involvement and student learning. He noted that students have only a vague idea of why they are taking general education courses and that most do not read the information in the college catalog. He cited evidence that discussion is more powerful than lecture for developing higher learning skills. The attention span of college students is 10-20 minutes, and thus faculty need to change pace that often. He estimates that 50% of students are on task at any given time.
Lion encouraged use of Bloom's taxonomy both in analyzing the questions that students ask (most are recall questions), and classroom tests - which also tend to focus on recall. Student quality of effort is often a mismatch with faculty desires. In the studies he considered, students spent 0.3-1 hour of study for each hour of class whereas faculty hope for 2 hours outside class for each hour in class.
Lion cited Chickering's work on academic advising. Chickering believes that academic advising should be the hub for every student and that effective faculty advising focuses on student values and goals and is intrusive. He classified much academic advising as primarily a clerical procedure and missing the opportunity for a developmental process to enhance learning. His urgent challenge is that faculty use the existing research on teaching to improve their own teaching and more actively involve students in all components of the educational process.