Klout reacts

Posted: June 2nd, 2011 | Author: | Filed under: Klout | 5 Comments »

Seems like I struck a nerve with my earlier post about Klout. Binh Tran, the CTO of Klout, Megan Berry, marketing manager at Klout, and Klout itself are all now following me on Twitter. In addition, I received a lengthy response from Ash Rust, the Director of Ranking at Klout, which I have included in full at the end of the post.

Ash’s three main points were:

  • Klout is just beginning and has flaws
  • Your Klout Score is about quality not quantity
  • Adding additional networks should increase your Klout

I appreciate Ash’s candid response. On the first point, he’s right. It’s unrealistic for me or anyone to expect perfection of Klout or any of the competing metrics/companies, especially at this relatively early stage of Klout. Think about Google 2 years afters its launched and how far its come since then. Still, companies need to know what users find wrong with their products to iterate and improve. I didn’t receive hundreds of RTs because my writing was so exceptional or witty; I received them because I articulated a set of issues seen by others in Klout.

Ash’s second and third points seem to contradict each other. Categorically stating that adding another network will always increase your score, seems to be a victory for quantity, not quality. In the Klout Chat yesterday, CEO Joe Fernandez announced that Foursquare and LinkedIn will soon be added. Without the proper Twitter/Facebook balance in the current system, I worry that adding two additional networks to the mix, will exacerbate existing issues. Additionally, I wonder how Klout will deal with identical individuals across multiple networks. My Foursquare friends are a strict subset of my Facebook and Twitter friends. Do they double count? Should I get any credit at all for adding friends already accounted for elsewhere in the system?

As I have time over the next few days, I’ll gather a few unanswered questions from the #Kloutchat in addition to others I have. I’ll send Ash a list that they will hopefully answer. Let me know if you have a few to add to the list.

As promised, here is Ash’s full response:

Hi Alex,
I’m the Director of Ranking here at Klout and wanted to respond directly to some of the points you raised here.
1) Thanks a lot for writing this.
It’s great feedback on the understandability of our score and mirrors a lot of the (intense) debates we have internally around how the score works and what data to deliver to our users.
2) Klout is just beginning.
We believe we’re at the very first stage of development for this paradigm, much like online document search was in 1998 when Google was founded, so we can expect some growing pains especially given the volume of data we process. That said, we know we need to do better and we’re working hard to improve, we have a team of excellent scientists working on improving the score.
3) Your Klout score is about quality not quantity.
While some users may have amassed many thousands of friends and followers, those people may not be listening or may not even be real people at all; this is why we use our own audience metric: True Reach. We also assess the influence of each person in your audience, so if someone you interact with is very influential that can have a much larger impact on your score than a group of people with lower levels of influence; for example if @BarackObama retweets you, it’ll increase your score more than if I do.
4) Adding additional networks to your Klout should increase your score.
We can only measure the data we have. If you add a network, like Facebook, and are influencing people on that network, then it should increase your score; assuming you’re influencing people on that network. If I influence 10 people on Facebook and then add my Twitter account, where I influence 3 people, Klout can now see me influencing 13 people, hence the score increase.
I hope this answers some of your questions and please feel free to follow up with me directly.
Ash Rust
Director of Ranking | Klout
ash [at] klout dot com

5 Comments on “Klout reacts”

  1. 1 Chris Westin said at 1:14 pm on June 2nd, 2011:

    I’d never heard of this outfit, but I tried it after I saw your post. They do pick up some weird stuff. Apparently I’m influential about St. Louis, even though I’ve never been there, and don’t ever say anything about it. Is there such a thing as influence-by-omission?

    Peer Index seems to have done a better job of recognizing this kind of thing: http://www.peerindex.net/dashboard . Try that.

  2. 2 Lemi4 said at 10:36 pm on June 7th, 2011:

    I don’t know, is it just me or does Chris Westin sound spammy up there on comment no.1?
    I mean, Alex has already mentioned peer index before this, so its pretty obvious that Alex already knows about Peer Index…

  3. 3 Alex Braunstein said at 10:41 pm on June 7th, 2011:

    Actually I met Chris Westin at MongoSF a few weeks ago. He gave a great presentation on the new aggregation features in mongodb. More on Peer Index coming soon though!

  4. 4 Observer said at 10:59 am on June 29th, 2011:

    There’s an unexplored tension here between Klout attracting customers and Klout serving clients.

    Klout clearly believes that there’s a limited window here for a dominant influence-ranking site to emerge. It’s prioritizing things that will attract – and retain – a user base. And it wants access to as much data as it can get.

    That’s apparently driven some interesting decisions. For example, every user is going to have different levels of influence on different networks. If a user adds Facebook or LinkedIn, and their score drops, they’re going to remove them. And then, Klout loses the data. So instead, it rewards users – at least in the short term – for adding accounts. And the corollary, which I’ve actually tested, is that it punishes users for removing accounts. Both effects seem to fade beyond the 30-day window. But they’re about behavioral modification, not accuracy.

    In a similar vein, Klout updates daily. That drives users back to check their scores. It rewards constancy and consistency of activity. If you were the Justin Bieber of March, and haven’t tweeted since, your score will have plummeted – even if your actual influence hasn’t. (It’s likely, of course, that when someone enormously influential takes a Twitter hiatus, their long-anticipated tweets when they resume may be even more influential than before they left.) There’s a reason for that, too. To be most interesting to users, Klout wants to be extremely sensitive to small signals. If it’s not changing every day, users won’t check in. So it has a graph which shows dips and surges even below the level of whole integers. It overvalues recent developments, and undervalues long-term trends. And it deliberately doesn’t smooth its data.

    The result is addictive and rewarding for users – a score that encourages them to use social media, and then rewards them for their participation.

    In its present form, though, it’s somewhat less useful for commercial clients. It does offer a raw gauge. Its user database is more comprehensive, and more timely, than that of any of its erstwhile rivals. But as you point out, the actual results are generally head-scratchers. Klout is clearly betting that if it captures a large enough user base, it can refine its algorithms down the road to appeal to commercial users. We’ll see if it’s correct about that.

    One other note. In your analyses, you’re using the scores that Klout itself provides. My back-of-the-envelope calcs, though, suggest that the most important stats are ones that Klout does not explicitly list: ratios. Tweeting a thousand times and get 200 RTs is good; tweeting a hundred times and getting 200 RTs is spectacular. And Klout knows this. Try dividing each of the categories by the number of tweets that the users have authored, and you’ll see what I mean.

  5. 5 Impact VS Influence another Dichotomy – Muskblog said at 9:48 pm on July 20th, 2013:

    […] out, the author Alex Braunstein is “a Ph D Statistician and search quality engineer”.  His post even got an official response from @Klout or somebody at Klout which are two different things. 😉  I myself am running out of […]

Leave a Reply