MongoDB add sharding -- Just a note

1. Configure Configuration Server.

1.1. Create a directory: e.g. C:\data\dbs\config 

1.2. Start config server

       mongod --dbpath c:\data\dbs\config --port 20000

2. Start mongos connecting to config server

      mongos --port 30000 --configdb  localhost:20000 

3. Create two shards 

3.1 Create shard 1

3.1.1 Create folder - C:\data\dbs\shard1
3.1.2 mongod --dbpath c:\data\dbs\shard1 --port 10000

3.2 Create shard 2

3.2.1 Create folder - C:\data\dbs\shard2 
3.2.2 mongod --dbpath c:\data\dbs\shard2 --port 10001

4. Add shard to mongos

4.1 Connect to mongos 

mongo  localhost:30000/admin

MongoDB shell version: 2.4.4
connecting to: localhost:30000/admin
mongos>

4.2. Add shard

mongos> db.runCommand({addShard:"localhost:10000", allowLocal:true})
{ "shardAdded" : "shard0000", "ok" : 1 }

mongos> db.runCommand({addShard:"localhost:10001", allowLocal:true})
{ "shardAdded" : "shard0001", "ok" : 1 }

5. Enable sharding on database (e.g. test)

mongos> db.runCommand({"enablesharding":"test"})
{ "ok" : 1 }

6. Enable sharding on collection 

mongos> db.runCommand({"shardcollection":"test.gem_deal_data", "key":{"_id":1}})
{ "collectionsharded" : "test.gem_deal_data", "ok" : 1 }

 

Then can use sharding in the application code. For example, 

import pymongo

class MongoDb(object):
    def __init__(self, host='localhost', port=27017):
        self.conn = pymongo.MongoClient(host, port)
        self.db = self.conn.test

    def close(self):
        self.conn.close()


def open_connection():
    # Specify the port 30000 to connect to mongos
    conn = pymongo.MongoClient('localhost', 30000)
    return conn.db

 

7. (Updated on 2014-11-10)

If you want to shard the GridFS collection, please refer to  http://docs.mongodb.org/manual/tutorial/shard-gridfs-data/
Quick Notes:

* files collection is usually small in size, thus no need to shard this collection in general

* chunks collection

   {files_id:1, n:1} and {files_id:1} are the only supported shard keys for the chunk collections of a GridFS store.

   To shard the chunks collection by {files_id:1, n:1}, issue commands similar to the following:

   ** db.fs.chunks.ensureIndex({files_id:1, n:1})   <--- Maybe not need to create this manually, some driver like Pymongo would create this index automatically.

   ** db.runCommand({shardCollection:"test.fs.chunks", key:{files_id:1, n:1}})

   You may also want to shard using just the file_id field, as in the following operation:

   db.runCommand({shardCollection:"test.fs.chunks", key:{files_id:1}})