How Should Colleges Assess and Improve Student Learning?

First, a few caveats.

1. I don’t believe that all we’re about in education is preparing future employees.


2. Just because Business says something is good doesn’t necessarily make it so.


3. I don’t know enough about the methodology of this survey to judge bias.

Okay, now that we’ve got that out of the way, I found How Should Colleges Assess and Improve Student Learning? (pdf) rather interesting:

From November 8 to December 12, 2007, Peter D. Hart Research Associates, Inc., interviewed 301 employers whose companies have at least 25 employees and report that 25% or more of their new hires hold at least a bachelor’s degree from a four-year college. Respondents are executives at their companies, including owners, CEOs, presidents, C-suite level executives, and vice presidents.

So they polled these executives about the quality of college graduates and what they’d like to change, particularly in the area of assessment. Here are a few excerpts:

Employers are satisfied that the majority of college graduates who apply for positions at their companies possess a range of skills that prepare them for success in entry-level positions, but they are notably less confident that graduates are prepared for advancement or promotion. While recent graduates are seen to demonstrate solid skills in the areas of teamwork, ethical judgment, and intercultural skills, employers are less convinced of their preparedness in terms of global knowledge, self-direction, and writing.

When it comes to the assessment practices that employers trust to indicate a graduate’s level of knowledge and potential to succeed in the job world, employers dismiss tests of general content knowledge in favor of assessments of real-world and applied-learning approaches. Multiple-choice tests specifically are seen as ineffective. On the other hand, assessments that employers hold in high regard include evaluations of supervised internships, community-based projects, and comprehensive senior projects.

Employers’ emphasis on integrative, applied learning is reflected in their recommendations to colleges and universities about how to assess student learning in college. Again, multiple-choice testing ranks lowest among the options presented, just below an institutional score that shows how a college compares to other colleges in advancing critical thinking skills. Faculty evaluated internships and community-learning experiences emerge on top. Employers also endorse individual student essay tests, electronic portfolios of student work, and comprehensive senior projects as valuable tools both for students to enhance their knowledge and develop important real-world skills, as well as for employers to evaluate graduates’ readiness for the workplace.

And, from the “Key Findings”:

. . . Most employers indicate that college transcripts are not particularly useful in helping evaluate job applicants’ potential to succeed at their company.

. . . Few employers believe that multiple-choice tests of general content knowledge are very effective in ensuring student achievement. Instead, employers have the most confidence in assessments that demonstrate graduates’ ability to apply their college learning to complex, real-world challenges, as well as projects or tests that integrate problem-solving, writing, and analytical reasoning skills.

. . . Employers deem both multiple-choice tests of general content knowledge and institutional assessments that show how colleges compare in advancing critical-thinking skills of limited value for evaluating applicants’ potential for success in the workplace. They anticipate that faculty-assessed internships, community-based projects, and senior projects would be the most useful in gauging graduates’ readiness for the workplace.

. . . When asked to advise colleges on how to develop their methods for assessing students’ learning, employers rank multiple-choice tests of students’ general content knowledge and institutional scores for colleges as conspicuously low priorities.

So, let me summarize (bias alert! bias alert!) via a single multiple choice question:

1. According to this report:

a. Grades are pretty much a non-factor in the hiring process.

b. Multiple choice tests are an unreliable predictor of success.

c. Employers are pretty much satisfied with the content knowledge of their employees and think assessments that cover content are relatively meaningless.

d. Employers want their employees to be more globally oriented, to take charge of their own job, and they must be able to communicate effectively through writing.

e. Employers prefer meaningful, relevant, experiential learning over an isolated, content-focused-only approach.

f. All of the above.

So, keeping those caveats I started with in mind, what does this mean for what we’re doing here at Arapahoe? Regular readers of this blog can probably predict what I would say about the ramifications of this in terms of the way we grade, what and how we teach, coverage of the curriculum, and how much of the responsibility for learning we put in the hands of students. But, just in case there was any doubt (and that the above wasn’t biased enough), let me add on to the choices above.

a. Since the primary way that many folks use grades appears to be superfluous, perhaps we need to take a hard look at the efficacy of grading in the first place, and perhaps switch to a focus on formative assessment versus “grades.”

b. Other than standardized test companies, politicians, and teachers who have a tight deadline to turn in final grades (see previous item), who’s really in favor of multiple choice tests? (Yes, I’ve ranted about this before somewhere, but I can’t find it at the moment.)

c. Content is necessary, but not sufficient, to be successful in the 21st century. So perhaps we should stop trying to “cover” the content, and instead focus on understanding the essential concepts and applying them in real world settings.

d. Constructivism. Blogs. RSS. Read/Write Web. Personal Learning Networks.

e. We need to take a hard look at our current system.

I’d love to hear your thoughts (especially AHS folks, but of course everyone is welcome to chime in) on what specific changes we might think about making. If you do comment, please try to not focus on the rant portion of the above, but on the results of the survey and on meaningful changes that will benefit our students.