Why the Solomon Index is Garbage | RefinerLink

RL Blogs

Why the Solomon Index is Garbage

By Ralph Laurel

Mar 03, 2012

Many refiners spend an extraordinary amount of effort submitting input to and digesting results from Solomon surveys. Few companies truly know how leverage data, while others appear to be extracting little value.                             


So I want you to think like someone in upper management for a second. Now take a look at the Solomon data for an unnamed organization below. Can you assess their performance from just this chart?



If your managers are anything like my managers, your first reaction would be that this is unacceptable. Even though you don’t even know what the categories are, you’ve already focused in on the stars in the 3Q and that terrible looking one at the end of the 4Q. After all, that’s what managers are supposed to do right? Focus on the gaps and try to improve them.


Now what if moving out of the third and fourth quartile in those given categories meant taking a step back in some of the categories where you’re in the leading 1Q? Would you be willing to make that trade off? Again, if your managers are like mine, you’d rather have a whole bunch of stars in the second quartile than have to explain why you’re at the bottom of the 4Q in that one category. 


So which organization can lay claim to this Solomon chart? It’s the 2011 data for the New England Patriots.  One of the most dominant professional football franchises in the last decade. Here’s a chart with the categories displayed.



For years the Patriots have consistently been in the running for the best team in football. They’ve achieved this by having one of the most dominant offenses, lead by their passing game. Most people would recognize that the most important star on this chart is the one indicating leading 1Q for overall position. They were the second best team in 2011.


It would be a mistake to focus in on stars in the 3Q and 4Q and assume that this is a mediocre organization. It would also be a mistake to take drastic steps to try to improve the position of the 3Q and 4Q stars without understanding how that will impact the rest of the categories and the overall position of the team.


The better thing to do with Solomon data is to try to understand it. Has the passing defense always been terrible or it just this year? Was it caused by injuries? Maybe it was a result of the fact that the stellar offense constantly forced opposing offenses to pass the ball since they were always playing from behind. The answers to these types of questions are what should trigger the proper actions.


Too many times managers will base major decisions on Solomon indicators without fully understanding the data. I realize the football comparison doesn’t translate over completely to the energy industry but it does show that a few stars out of the first or second quartile don’t mean your organization is destined for disaster. And sometimes, a star in the 4Q may actually be ok if it’s allowing better performance in more important areas.


If anything, comparing football teams to each other is a lot easier than comparing refineries. Football teams play on a level playing field while refineries differ in their market place, feedstocks, hardware, and labor costs among other things.  Sure there are normalizing factors such as kEDC, RSCs, and PADDs, but that only goes so far.


When it comes to refineries, many of the categories are very intertwined so making major moves in one area can drastically affect another. For instance, if a refinery makes its profits from challenged crudes it will drive up operating expenses.   Cutting expenses in order to improve your Solomon position will reduce margin and most likely hurt overall profitability. And profitability in refining is the equivalent of overall position in football. That star should outweigh all others. 


Let’s be clear that I value the importance of Solomon surveys; however, I feel that many do not understand how to really assess the results.  Solomon data should be used as a tool to generate questions and prompt further analysis. Good Solomon results shouldn’t be the end objective, but a means to an end. Until that is fully understood throughout an organization, Solomon Indices are garbage.

Enjoy this content? Join our Free Newsletter    
  • :   very amusing comparison above - well done!

    Apr 24, 2012

  • :   Yes, very entertaining post, but you cannot fairly compare an oil business with a sports organization.

    Apr 27, 2012

  • :   Isn't a sports organization a business? They've got different areas of the organization they're constantly assessing versus others as well.

    May 16, 2012

Do NOT follow this link or you will be banned from the site!