Business Schools Say MBA Rankings Can’t Be Trusted

Another concern is that some rankings, including that of the Financial Times, convert salaries to US dollars using the IMF’s purchasing power parity (PPP) rates, which equalize purchasing power across countries by taking into account the cost of living and inflation differences. This may distort salaries, seemingly inflating the pay of those who work in developing countries.

A senior business school administrator said: “The majority of rankings are determined by financial data, and that depends upon the location of the respondents. A great program in a country that has a low purchasing power parity value is disadvantaged.

“Also, they [rankings] don't recognize programs that have a focus on areas such as innovation management or entrepreneurship, where managers are neither likely to, nor inclined to, [want] executive positions and thus won’t see the huge salary increases.”

READ MORE: Can You Trust MBA Rankings?

© grinvalds via iStock

A significant number of survey respondents are also unconvinced about the evidence-collection techniques employed by rankings producers. They said that methodologies and questions are sometimes ambiguous, making it difficult to know whether schools are providing comparable answers.

One respondent said that “a survey may not be asking the right questions to get to the true areas of excellence of a school, and schools may have to manipulate their data to fit the needs of the surveying organization”.

Will Dawes, research and insight manager at AMBA and BGA, said: “Much of the feedback generated in the survey points to low trust in the way responses are collected, with stakeholders suggesting that self-reporting of evidence could potentially lead to distorted results.”

He added: “Others highlight methodological issues, such as the quality of questionnaire design, skewed sampling of survey respondents, and inherent biases towards larger programs, which have a better opportunity to reach minimum response requirements.

“Other concerns relate to data verification processes, including the implementation of feedback back-checking and the opportunity for schools to manipulate data (for example, to retrospectively take advantage of currency fluctuations).”

Check Out Our Compare The Rankings Table

© monkeybusinessimages via iStock

Such concerns were likely magnified following the rankings scandal at Temple University’s Fox School of Business last year, which admitted to, for years, having reported inflated admissions exam figures and other student data to media outlets for its online MBA and six other degrees.

The majority (59%) of those polled by AMBA and BGA have views on changes they would make to MBA rankings (just 12% of stakeholders do not think any changes are needed while 29% do not know what changes should be made).

The strongest preference was for a high mean weighting to be given to ‘the quality of management faculty’ such as qualifications and research (25%), followed by ‘the overall student experience’ (24%).

The next highest weighting suggested was for ‘the outcomes of MBA graduates’ such as career progression and salary (21%) and ‘the diversity, breadth and quality of the MBA cohort’ (18%). Meanwhile, ‘alumni engagement’ or the frequency and quality of the network is, on average, given a suggested weighting of 13% among this range of criteria.

For their part, rankings providers have announced changes and reviews to their methodologies. This is the first year that the FT has assessed business schools on their CSR activities, for example, and the newspaper has said it is reviewing its MBA ranking methodology more widely.

Bloomberg Businessweek also revamped its MBA ranking following extensive consultations with business school administrators. The ranking now focuses on four “indexes”: compensation, learning, networking, and entrepreneurship.

Meanwhile, the Wall Street Journal last year released a new ranking of MBA programs, a joint effort with Times Higher Education of the UK, that was heavily criticized for its complicated methodology and the fact that many top schools pulled out of the ranking.

Will said: “The findings from this study suggest that MBA rankings are largely seen to be out of touch with the delivery of MBAs on the ground.

“One cause might be that the measurements used to assess the quality of MBAs are outdated – with the recent developments in program design, student compositions and the evolution of what an ‘MBA’ means to students and schools—and that further consideration needs to be given to rankings modernization.”