Do (Should) Rankings of Ph.D. Programs in Political Science Matter?

Not long into the process of working toward a terminal master's degree in political science, I realized I couldn't not pursue a Ph.D. -- the field was too interesting, and (strange as it is) I had come to the realization that I wanted to teach and do research. Once I made this decision and began the process of applying to various programs, my naiveté soon caught up to me. Through discussions with my professors I discovered that all doctoral degrees in political science were not equal, ceteris paribus. To the contrary, the rule of thumb iterated to me was this (more or less): you can only get a job at a university of equal or lesser rank than the school where you earned your Ph.D.

Though some might argue this rule of thumb is unfair, it makes some sense; though, it nevertheless alarmed me. While getting a faculty position at an especially prestigious school didn't necessarily concern me, the idea that pedigree could either help or hurt my chances of finding a job did. 

Affiliated honor (in this context, the perceived excellence that an individual who earns a Ph.D. receives simply by virtue of having graduated from a prestigious program) has been shown in several studies to impact the hiring practices of universities. Though the effect of affiliated honor has not (as far as I'm aware) been disentangled from the impact of individual merit on job placement (do people who are equally qualified place equally well, regardless of pedigree?), the possibility that the rank of the institution from which I would receive my Ph.D. could either maximize or minimize my chances in the job market was enough incentive for me to be selective in my application process.

Of course, I couldn't be unduly selective and apply to only Harvard, Yale, and Princeton. I had to find a school that both maximized my future prospects and had a good chance of admitting me (my pedigree hereto was not especially prestigious in its own right).

In the end, I think I landed pretty well, but the experience of applying to and visiting several programs taught me some important lessons that I never would have learned had I chosen a school on the basis of on-paper metrics alone.

How to Measure Prestige?

One of the challenges I encountered during the application process was how to best measure institutional prestige. There are several sources out there that offer a measure of "rank" (U.S. News and World Rankings, the National Research Council, etc.). While some overlap exists between these various sources, differences in methodology and definitions and measures of quality account for a great deal of variation in assessments of rank. To illustrate, I collected data from the National Research Council's 2010 rankings for Ph.D. programs in political science to show how various methods of gaging program quality can lead to notably different results.

The below two figures display rank based on S-Rank and R-Rank, respectively. The former ranks programs based on "the criteria that scholars say are most important." The latter ranks programs based on whether "they have similar features to programs viewed by faculty as top-notch."

Inconsistencies between these ranking methods are apparent, and both S- and R-Rank are only two of the metrics the National Research Council offers. Below are figures that show rank based on faculty research, student outcomes, and student diversity.

It should be clear that a single measure of program quality is hard to pin down. It is, however, possible to create a rank index that incorporates several of the above ranking systems into a single measure of institutional rank.

However, before creating such an index, I wanted to be sure that there was some congruence between the various measures of rank offered by the National Research Council. The below figure displays the correlations between each of the above ranking systems. Red values signify negative correlation, blue values signify positive correlation, and a "X" means that the correlation fails to reach statistical significance (p < 0.05).

Diversity, as it turns out, isn't significantly correlated with the ranking system for student outcomes, faculty research, or the S- and R-Rank systems. Outside of diversity, student outcomes are positively associated with S- and R-Rank and with faculty research; however, these latter three have the strongest correlation with one another.

Based on these correlations I developed the following rank index. Because diversity had a very weak association with the other factors, I excluded it from my index. This is not to say that diversity isn't important; it simply doesn't seem strongly associated with the other ranking systems included here. Also of note, student outcomes is given less weight relative to the other factors since its correlation with them, though statistically significant, was not as strong.

The results for institutional rank are shown below.

The above ranking system, of course, yields results different from those already discussed, and I won't even dare to purport that it's more reliable. If anything, the purpose of this exercise is simply to show that rank, despite the fact that it's given such weight by the political science community at large, is a bit ineffable. Most professors with whom I've talked have a general sense for where certain programs fall, but putting a hard number on it is, though far from an exercise in futility, hardly straightforward.

Should Rank Matter?

Even though it is hard to arrive at a single, infallible ranking system, rank neverthless does matter. But should it?

To offer the most indecisive answer to this question: yes and no.

Yes, because the most prestigious programs, in theory, are also highly selective, in part because they can afford to be since highly talented candidates are probably more likely to apply to these programs thereby offering admissions committees a larger pool of qualified candidates from which to draw, and in part because they typically hire from more prestigious and selective programs, meaning that their faculty are (probably) pretty good at what they do. Though I'm basing my "yes" answer on more conjecture than fact, this explanation has intuitive appeal.

No, because excellent pedigree is not a necessary (or for that matter, sufficient) condition for merit. That is, graduation from a prestigious program is not a requisite for, nor does it guarantee, a high degree of future potential and a past record of excellent achievements. It is possible that an individual with both great potential and an excellent resume can come from a less prestigious program. Again, I'm basing this "no" on conjecture more than hard evidence that there are exceptional candidates who graduate from "lesser" programs, but it's a plausible argument.

Also, no, because an institution with a lower reputation is not necessarily bad. One of the things that I learned during the process of applying to and considering admission offers from various schools is that reputation is "sticky." This means that a program with low prestige could make some radical, positive changes to the quality and rigor of its program, yet this change will hardly bolster said institution's standing due to the inertia reputation produces.

To bring some synthesis to these antitheses, my yes-no answer, in short, can be collapsed into the following thesis: it is more probable than not that graduates from prestigious Ph.D. programs have great talent (both latent and realized), but because institutional prestige is not a requirement for and does not guarantee individual merit, hiring institutions should not pass over job applicants from Southeast Northwest International State University without having considered said applicants' latent and realized ability.

Of course, I'm saying nothing new here, and, besides, it's possible that affiliated honor is not the insidious problem I've made it out to be. Research (and I'm at loss for where I've read this) has shown that when you account for pedigree, individuals of relatively equal merit typically do just as well as one another in terms of employment and income. Of course, there are more important things in life than simply having a job and making lots of money, and this research to which I have referred did not account for the types of jobs these individuals with comparable merit obtained, but nevertheless, based on anecdotal evidence I've been told that extremely qualified candidates from less prestigious programs do land well provided they put in the work. However, the battle for these individuals is decidedly uphill.

*To obtain the data and R code for this post, see my GitHub repository for this analysis.

UPDATE: The below figure displays U.S. News and World Rankings for 2017, color coded by quantile. According to U.S. News and World Rankings methodology, a quality score of 5 means "excellent," 4 means "strong," 3 means "good," 2 means "adequate," and 1 means "marginal." Although schools in the marginal category are listed as NA in the original data, I coded each as 1. A rank of NA means that the school either had a score less than 2 or there were not enough survey responses for the school.