Evaluating Human Potential

One of societies great unsolved problems is how to evaluate human potential. This post will mostly deal with college admissions, but job searches, electoral politics, and credit scores all have similar problems.

This post was inspired by an article in the on the NBC News website about changes in the SAT test. The SATs were originally conceived of as a way to have a universal measure of how successful a college applicant was likely to be. The value of grades and the quality of education in different districts varied greatly, and there was a desire on the part of University admissions boards to have a universal test that would level the playing field for all applicants.

As recently as the early 90’s when I took the SATs (760 math, 480 verbal) the common wisdom was that one could not study for the SATs. First Kaplan and soon a raft of other companies were in the process of putting the lie to that assumption. For those with the money, test prep could raise scores by 10 percentiles. It became clear to everyone from the College Board who administers the SATs to admission directors that an SAT score was as much a measure of the quality of a student test prep as it was their likelihood to succeed at a college or university.

Many will argue that any test is more a measure of ones ability to take tests then of any intangible quality being tested for. What then should we use to decide who gets into our colleges and universities. No, we cannot just let everyone in. There is not room. As just one example, UC Berkeley in 2013 had 67,701 applicants but could admit only 14,158 of them. (source) There is no question that a choice must be made, the only question is how to most fairly decide.

Personal Contact: The Historic Model #

Tests and grades were not always the ultimate arbiter of worthiness. Historically, opportunity was granted or withheld based on personal contact. At some ancient universities in Europe, this is still the standard. Professors take on students based on what ever criteria they feel most important, and are then expected to take responsibility for overseeing their students education. PhD programs in the US still follow this model. On a different front modern employment search still generally involves at least one face-to-face interview.

While sitting down across a table from a real person seems so much more humane then using a #2 pencil to fill in bubbles, this history of the 20th century is an object lesson in just how unfair this process can be.

Humans are bias. Recent social and psychological experiments show that even when the evaluator does not intend bias, it is impossible to remove. Women, minorities, the disabled, the poor, immigrants, their children and the LGBT community have all felt and continue to feel the sting of this bias.

Landmark pieces of legislation in this country (as well as others) have sought to outlaw both overt discrimination and more subtle institutional bias present our society. The goal was to remove the subjective bias and replace it with objective measures; measures like the SATs. But as we have seen, privileged families are able to subvert these so-called objective measures rather effectively.

Shocked by the achievement gap in the SATs many universities including UC Berkeley have started adding admissions essays and crediting extra-curricular activities. The very predictable response has been a boom in community service placement organizations and admissions essay consultants. College admissions have turned into an arms race between parents and admissions staff. Every time a university comes up with a new marker of an outstanding student, wealthy parents fill the pockets of any company that can endow their children with that marker.

Just Give Up #

Not all schools are as selective as Cal. In California we have an entire system of post high school education that is targeted at those students who for reasons of financial or academic poverty cannot attend competitive admission Universities. The community college system requires of its students only a high school diploma or the equivalent and is prohibited from selecting students using any form of academic achievement beyond that diploma.

What happens when more student apply to a community college program then it can handle? Just such a situation occurs every year at the Sacramento City College in the Nursing and Dental Hygiene programs. The problem of selection has not gone away. Instead of ranking students based on some measure of academic merit, all students as placed into a lottery potential system.

I’m sure that the administrators and politicians who brought about this system thought that they had side-stepped the problem of unfair tests and unequal opportunities. Everyone had the same chance.

What actually happens is not so equitable. IN order to have the best chance at being admitted students apply to every nursing program in the state (or a large region of the state.) The then move or commute to wherever the win the lottery. This leads to the odd occurrence of one student from Modesto (two hours South of Sacramento) driving here five days a week while another who lives within blocks of Sac. City drives the same two hours to the Community College in Modesto for the same program. The way this program is set-up, those who cannot travel because of poverty, family obligations or disability have a much smaller chance of admission.

Another interesting effect shows up when the Community College system is placed along side more expensive and selective schools. Many students who could afford admission at a private nursing school have the academic credentials to attend a CSU apply in the Community College system because it is cheaper and faster then the alternatives. If their name does not come up in the lottery they simply move on to the next best alternative, while if they get in, they displace someone who may have no alternative open to them.

Both these problems could be partially remedied by a more complex form of lottery. What cannot be solved so easily is the issue of drop-outs. Many impacted Community College programs suffer from a higher then necessary rate of student failure. By attempting to asses the chance of student success, these programs admit at a higher rate student who are not able to complete the program. Each of these students has also displaced a student who could have actually benefited from the opportunity.

I see this same problem when our Church sells parking permits. We can accommodate about 34 students each semester, but many more want to buy then we have space for. Our solution has been to sell on a first-come-first-served basis. This has lead to the rather silly situation of selling parking passes to those willing to wait ever longer time in the cold early hours of the morning.

Why should sitting in an empty parking lot at 4:00am make one more worthy of parking in our lot then the person who only arrived at 7:00? One option might be to raise the price until only the allotted number of passes sold. We could even auction them off to the highest bidder on e-bay. but, as a christian church we find that selecting people based on economic privilege is not a suitable criteria either. Yet, with more demand then supply we must choose some how.

Weighting the Scales #

We have seen that subjective evaluation leads to evaluation bias, while attempts at an objective scoring system are subject to being distorted by wealth and privilege. We have also observed that trying to replace evaluation with a non-evaluative system like a lottery does not eliminate the advantage of privilege, it merely alters the strategy. Bias cannot be avoided or ignored.

Knowing this to be the case, many have suggested weighting the scales in favor of the poor and oppressed. If we know, for example, that people of a certain income level can buy test prep that increases SAT scores by x amount, then simply increase the scores of everybody else by that amount. Alternatively, do what the College Board has done and try to provide everyone with an equal level of test prep to erase the advantage. Doing so may help to move the test closer to “fair” but it does not solve the problem that the SATs no longer are testing Scholastic Aptitude or Ability but the willingness and ability to do test prep – not a good indicator of academic success.

Another issue raise when weighting the scales is the perceived unfairness to those being selected against. At least to this point in history, it is my observation that attempts to make up for discrimination with numeric adjustments have been clumsy, overly broad and ill targeted. Fair or not this history has placed affirmative action in an poor light. I have yet to see a system that accounts for even a good range or factors effecting privilege and oppression much less a comprehensive one. At least the versions we read about in the general media are focused on only race, or only income, or only immigration status. This has not endeared such programs to those who believe they are just as oppressed (or nearly so) yet not covered by a similar numeric adjustment.

Such resentment leads inevitably to passage of laws like California Proposition 209, a 1996 ballot initiative that prohibited the use of race, sex, or ethnicity in a number of public institutions including public post-secondary schools for any purpose (including admissions).

SCA-5 which is now working its way through the California legislature would exempt public post-secondary schools from prop 209 with regard to admissions. SCA-5 could be on the California ballot this November, and its success or failure may well hinge on showing the voters of California that there is no unbiased objective criteria by which student can be admitted to universities and then by conversing voters that where the to allow this change in the law, that the UC and CSU systems would use these factors well and not in the rather naive ways that fueled Prop 209 to begin with.

A Look Ahead #

At the present time I see this as a rather intractable dilemma in society, and as I have said, one that we cannot simply ignore. Systems of privilege and oppression are not evaporating into a harmonious utopia. Indeed, I think one might be able to make a solid argument that just the opposite is occurring.

At the same time, the legitimate need to select skilled and talented people from among a large pool of applicants is growing rapidly. The kind of employers able to pay the best wages cannot settle for hiring someone who is merely good enough. In a hyper-competitive global economy in which technology is crushing the historic barriers to entry, if you do not hire the smartest and most talented prospective employee your competition will.

This kind of labor market will in turn put pressure on educational institutions who train and education such employees. Private Colleges and trade schools are springing up to provide such training and education to those who have the means to pay for it. It is the responsibility of Public University systems to provide a fair shot to those who cannot.

It is therefore critical that a way be found to fairly and honestly assess the abilities and talents of individuals in an ever growing multiplicity of fields and abilities. I am pleased to see the College Board begin making changes that acknowledge this need and I hope society at large and academia in particular will continue to wrestle with this problem.