More fun with mongodb, map/reduce, and sorting records by value using pymongo

Posted: May 11th, 2011 | Author: | Filed under: mongodb, pymongo | 2 Comments »

A couple of weeks ago I promised a fun application in mongodb. That time has arrived. Suppose you have a collection of records and you want to group by some id (canonicalized url, user id, checkin venue) and see how many user actions (perhaps clicks, status updates, or checkins for the three examples listed) are associated with each. Those familiar with mongodb would ask, “Why not do this with the group() function?” Its limited to 20,000 unique ids. Many applications have more than this, so map/reduce is the way to go. Below I provide code to do this map reduce and then sort by value. From the shell sort is done as:

result.find().sort({u’value': -1}):

but if you run this from within python using the pymongo driver you will receive the error:

TypeError: if no direction is specified, key_or_list must be an instance of list

If this occurs make sure you sort with:

result.find().sort(u’value’, -1):

Some more hardcore/sophisticated mongodb examples/applications will be coming soon!

#!/usr/bin/env python

from bson.code import Code
from pymongo import Connection

# code for example map/reduce
db = Connection().map_reduce_example
map = Code(“function () {”
“emit(this.id, 1)”
“}”)
reduce = Code(“function (key, values) {”
” var total = 0;”
” for (var i = 0; i < values.length; i++) {" " total += values[i];" " }" " return total;" "}") result = db.things.map_reduce(map, reduce, "myresults") for doc in result.find().sort(u'value', -1): print doc


2 Comments on “More fun with mongodb, map/reduce, and sorting records by value using pymongo”

  1. 1 Alex Braunstein's Blog » Blog Archive » Pymongo: distinct items and an example map reduce on subset of db said at 4:45 pm on June 8th, 2011:

    […] the example from my previous post, you simply add query = {} and add out = for the collection in which you want your results to end […]

  2. 2 Sean Scott said at 8:59 am on August 22nd, 2013:

    thanks this post saved me from a lot of additional hair pulling


Leave a Reply