At the end of August, President Obama outlined new proposals to make college more affordable and to develop a new rating system before the fall of 2015 that will appear in the existing College Scorecard. With the start of the semester, you might have missed the details, but the plan is essential reading—note that congress does not have to approve: this will happen. I applaud the effort. Students and parents need better ways to think about the cost/benefit of college, but if these new metrics become a singular way to judge the value of college, the conversation will be far too narrow. We will, however, only have ourselves to blame.

Despite our collective and constant pleas for prospective students to think more about fit than rankings, the US News rankings continue their stranglehold on how the public, and even many of us, think about higher education. We are not going to dissuade politicians or parents to give up looking for ways measure and rank something that is so important and expensive. The real question, and the place we should put our energy, is in creating better measures for what we do.

We have a choice. We can either develop our own measures of the learning that happens on our campuses, or prepare for the White House and others to measure less meaningful things. We need to do this quickly. Note that financial aid may eventually be tied to how we do in the new ratings: “In the upcoming reauthorization of the Higher Education Act, the president will seek legislation allocating financial aid based on these college ratings by 2018, once the ratings system is well established. Students can continue to choose whichever college they want, but taxpayer dollars will be steered toward high-performing colleges that provide the best value.” Ouch!

Let’s be fair. The objectives and the general thrust on outcomes are good. Parents and students easily confuse prestige with quality and college costs too much. It is hard to argue with the political pressure for a “a new ratings system to help students compare the value offered by colleges and encourage colleges to improve.” The US News rankings rely too heavily on input measures, and if we are going to be judged, it should not be on the students we attract, but on what we do for the students we have. The U.S. Department of Education says “These ratings will compare colleges with similar missions and identify colleges that do the most to help students from disadvantaged backgrounds as well as colleges that are improving their performance.” Faculty often say to me, “just bring us better students,” but surely the real measure of success is how different colleges improve the lives of similar students. But whatever we think, there will be a new ranking system soon.

Most of what are currently being considered as outcome indicators, are quantitative, but secondary measures like graduation rates or graduate earnings.  Walter M. Kimbrough has correctly pointed out that these need to be modified with a “degree of difficulty” measure (as in judging diving).   The National Institute for Learning Outcomes Assessment and others  have developed other measures, but we need to do better. We must start measuring the change we hope to produce.

Prestige, better pay, better job opportunities and fun in fraternities are often seen as benefits of college. Indeed, networking and alumni connections are a valuable part of the college experience, but if we sell jobs or salaries as the only or even primary outcome measure, then we all become network facilitators.

We need instead to talk about how thinking and more complex mental models are the real engines of future success, a robust democracy and a better life. We will then, need to focus our attention on getting serious about measuring this. Are we actually changing our students and improving their ability to think?  Would they develop as quickly if we het left them alone or they got a job?  The answers may make us nervous, but we can’t improve without data.

If anyone from the White House is listening—we CAN measure learning! Please give us a chance to suggest, create and develop some better measures.