Comparing Klout competitors and alternatives: PeerIndex and Twittergrader

Posted: June 8th, 2011 | Author: | Filed under: Klout, PeerIndex | 2 Comments »

After spending two blog posts on the shortcomings of Klout, it only seems fair that I look into the quality of its competitors PeerIndex and Twitter Grader.

Like Klout, both consume your Twitter stream and provide a summary score of your “influence” between 0 and 100. PeerIndex also allows you to connect your Facebook, LinkedIn and Quora accounts, while Twitter Grader is limited to Twitter. In my previous post I claimed that any good measure of influence should have the following properties:

  1. Ordering should make sense in the real world – the score should roughly represent the degree to which one is influential or has clout
  2. The score should not be easy to game – people should not be able to hack their score in a few days by getting bots to RT, squatting on hashtags, or simply connecting a Facebook account
  3. The score should be monotonic – if another member has higher stats than me in ALL categories, then he/she should have a higher score

The primary competitor is PeerIndex, so we will start by seeing if their score satisfies the above conditions more than Klout. Here is my PeerIndex summary:

Though overall PeerIndex better satisfies the three rules better, the first thing I noticed is a bug: my PeerIndex is reported as 50 at the top of this screenshot and 44 at the bottom. PeerIndex focuses on three areas: Activity, Authority, and Audience (as compared to Network Influence, Amplification Probability, and True Reach on Klout). These individual numbers are also different at the top and bottom of the screenshot. Though I don’t have a great idea of how these scores are calculated, Activity and Audience (followers) sound much more easily gamed to me. Authority is in theory a great measure, but in practice it falls short. I encountered several other bugs. Mouseover elements in the graph don’t always disappear, and some accounts just aren’t able to be added to the comparision. @Williamsonwines (my favorite winery) and @uberjon a product marketing executive at Facebook and X-Googler are two examples. Several other issues and bugs are highlighted in the comparison screenshot below:

PeerIndex claims that @feltron should have an audience score of 0, however, the well known data analyst and designer had 11,219 Followers and was listed 907 times at the time of this post. @coachella, the epic music festival in Indio, CA, has an activity score of 0, though the account has tweeted 436 times and several immediately preceding this blog post. Finally, @MummNapaWinery has 0’s across the board even though the account has 312 tweets, 3,122 followers, and is one of my favorite wineries in Napa. I also find it ironic and mildly awkward that PeerIndex gives itself a low authority score and Klout a slightly higher one.

For due diligence purposes, I performed the same group comparisons as in my original Klout post, but I didn’t find the same monotonicity issues. As a result, they are harder to poke fun at, so I placed them at the bottom of this post. You will notice a few individuals are left out relative to the original Klout comparisons. The UI requires 6 or less people, so I took a few out randomly. If you stare at the authority scores in these examples long enough, I think you’ll agree that they are kind of wonky (check out Vic Gundotra and Carla Borsoi, VPs at Google and AOL, respectively). Put more succinctly:

PeerIndex Advantages:

  • No or fewer monotonicity issues
  • Facebook, LinkedIn, and Quora already integrated (Klout is just adding LinkedIn)
  • Twitter Elite Lists are VERY interesting: Top Users, Top Women, Top Brands, Top Cities

PeerIndex Disadvantages:

  • Can’t see or track your score over time
  • Fewer details in summary
  • VERY slow to update (~7 days initially and updates only every couple of days)
  • Authority score needs more explanation

In summary, PeerIndex is a legitimate competitor, but they need to fix the user facing bugs I highlighted and really speed up their scoring cycle. If a user shows up to the site and can’t immediately access their score they will have HUGE retention issues. One of the most under appreciated features of Klout, regardless of your feelings concerning the legitimacy of its score, is its infrastructure. The fact that they can ingest the Twitter and Facebook streams, process the data, and update every day is an incredible engineering feat, especially for a startup of its size. Perhaps there is an unseen tradeoff between speed and quality of score within Klout and PeerIndex.

Next we consider Twitter Grader.

Their approach is totally opaque. The summary simply lists stats that I can get from my own twitter account, a number from 0 to 100, and a relative rank. I have no idea where this rank comes from, especially because there are way more than 9 million people on Twitter. Perhaps this is the number of accounts ever scored on Twitter Grader? I certainly hope not, because that would be incredibly biased. They do provide an article in their help center: How does Twitter Grader Calculate Twitter Rankings. Honestly their score may be great, but I just don’t have enough information and their site is lacking many of the features found in PeerIndex and Klout.

Twitter Grader Advantages:

  • Pulls your data and calculates score instantly
  • Associated with Hubspot (which has a great reputation)
  • It only uses twitter (see also disadvantages)

Twitter Grader Disadvantages:

  • It only uses twitter
  • Can’t see or track your score over time
  • Lack features

In my next post on the subject, I’ll have more followup from the ranking folks at Klout and the CEO of PeerIndex

As previously mentioned, the PeerIndex score comparisons mirroring those from the original Klout post can be found below:

Comparison 1

Alex Braunstein (me), @alexbraunstein – Statistician, Research Scientist at Chomp, X-Googler
Binh Tran, @binhtran – the co-founder and CTO of Klout,
Chomp, @chomp – app search engine
Vic Gundotra, @vicgundotra – SVP and head of social at Google
Carla Borsoi, @u_m – VP of Consumer Insights at AOL

Comparison 2

Paul Graham, @paulg – the fearless leader of Y Combinator.
Y Combinator, @ycombinator – startup incubator
500 Startups, @500startups – startup incubator
Adria Richards, @adriarichards – a tech consultant and popular blogger (also my roommate)

Comparison 3

Tim Ferriss, @tferriss – author of the 4 Hour Workweek and 4 Hour Body
Matt Cutts, @mattcutts – head of web spam team at Google
MG Siegler, @parislemon – my favorite writer for Techcrunch
Klout, @klout – the service I’m trashing in this post
Jeffrey Zeldman, @zeldman – designer, writer, and publisher

Comparison 4

Robert Scoble, @scobleizer – blogger, tech evangelist, and author
Perez Hilton, @perezhilton – master of celebrity gossip
Charlie Sheen, @charliesheen – #winning
Guy Kawasaki, @guykawasaki – entrepreneur and former Chief Evangelist at Apple
Justin Bieber, @justinbieber – never saying never

2 Comments on “Comparing Klout competitors and alternatives: PeerIndex and Twittergrader”

  1. 1 Azeem said at 5:56 am on June 9th, 2011:

    Great piece Alex!

    Thanks for taking the time to do some proper analysis on this.

    Let me apologise for some of the user facing bugs you’ve discovered. We’ve had to implement some very heavy caching on our consumer facing website in the past weeks as we deal with growth in usage of our API. (6000% growth in the past 4 months).

    The result is that there are some unacceptable delays in data updating on the consumer UI that you have tested us on (though not on our API). We’re working on it.

    One other issue you’ve spotted is the quirkiness of some data points -like Vic Gundotra. Again we’re aware of this – and have various fixes on the way to deal with it.

    I promise to get back to you with answers to your other questions in the near future.

    best wishes
    Azeem Azhar
    CEO, PeerIndex

  2. 2 Vortiz said at 7:53 am on June 22nd, 2011:

    Have you looked at mpact and compared to the above?

Leave a Reply