MongoDB.pdf

MongoDB.pdf

• 1.
  Technical Overview Learnizo GlobalLLP 1
• 2.
  Agenda • Welcome andIntroductions • The World We Live In • MongoDB Technical Overview • Use Case Discussion • Demo 2
• 3.
  MongoDB – It’sAbout the Data 3
• 4.
  MongoDB, It’s forthe Developers 4
• 5.
  MongoDB Brings ItAll Together 5 Volume of Data Agile Development • Cloud Computing • Commodity servers • Trillions of records • 100’s of millions of queries per second • Iterative • Continuous Hardware Architectures
• 6.
  MongoDB Use Cases UserData Management High Volume Data Feeds Content Management Operational Intelligence Product Data Mgt 6
• 7.
  10gen: The Creatorsof MongoDB Set the direction & contribute code to MongoDB Foster community & ecosystem Provide MongoDB management services Provide commercial services • Founded in 2007 – Dwight Merriman, Eliot Horowitz – Doubleclick, Oracle, Marklogic, HP • $31M+ in funding – Flybridge, Sequoia, Union Square • Worldwide Expanding Team – 150+ employees – NY, CA and UK 7
• 8.
  Agenda • Welcome andIntroductions • The World We Live In • MongoDB Technical Overview • Use Case Discussion • Demo 8
• 9.
  Why was MongoDBbuilt? 9
• 10.
  1 0 NoSQL • Key-value • Graphdatabase • Document-oriented • Column family
• 11.
  Traditional Architecture • Relational –Hard to map to the way we code • Complex ORM frameworks – Hard to evolve quickly • Rigid schema is hard to change, necessitates migrations – Hard to scale horizontally • Joins, transactions make scaling by adding servers hard 11
• 12.
  RDBMS Limitations 12 Productivity Cost
• 13.
  MongoDB • Built fromthe start to solve the scaling problem • Consistency, Availability, Partitioning - (can’t have it all) • Configurable to fit requirements 13
• 14.
  1 4 Theory of noSQL:CAP CAP Theorem: satisfying all three at the same time is impossible A P • Many nodes • Nodes contain replicas of partitions of data • Consistency – all replicas contain the same version of data • Availability – system remains operational on failing nodes • Partition tolarence – multiple entry points – system remains operational on system split C
• 15.
  Supported languages 15
• 16.
  1 6 ACID - BASE •Atomicity • Consistency • Isolation • Durability • Basically • Available (CP) • Soft-state • Eventually consistent (AP)
• 17.
  MongoDB is easyto use 17 START TRANSACTION; INSERT INTO contacts VALUES (NULL, ‘joeblow’); INSERT INTO contact_emails VALUES ( NULL, ”joe@blow.com”, LAST_INSERT_ID() ), ( NULL, “joseph@blow.com”, LAST_INSERT_ID() ); COMMIT; db.contacts.save( { userName: “joeblow”, emailAddresses: [ “joe@blow.com”, “joseph@blow.com” ] } ); MongoDB MySQL
• 18.
  As simple aspossible, but no simpler Depth of functionality Scalability & Performance Memcached Key / Value RDBMS 18
• 19.
  Representing & QueryingData 19
• 20.
  Schema design  RDBMS:join 20
• 21.
  Schema design  MongoDB:embed and link  Embedding is the nesting of objects and arrays inside a BSON document(prejoined). Links are references between documents(client-side follow-up query).  "contains" relationships, one to many; duplication of data, many to many 21
• 22.
  Schema design 22
• 23.
  Tables to Collections ofJSON Documents { title: ‘MongoDB’, contributors: [ { name: ‘Eliot Horowitz’, email: ‘eliot@10gen.com’ }, { name: ‘Dwight Merriman’, email: ‘dwight@10gen.com’ } ], model: { relational: false, awesome: true } } 23
• 24.
  Terminology RDBMS MongoDB Table Collection Row(s)JSON Document Index Index Join Embedding & Linking 24
• 25.
  Documents Collections contain documents Documentscan contain other documents and/or Documents can reference other documents Flexible/powerful ability to relate data Schemaless Flexible Schema 25
• 26.
  2 6 CRUD • Create – db.collection.insert(<document> ) – db.collection.save( <document> ) – db.collection.update( <query>, <update>, { upsert: true } ) • Read – db.collection.find( <query>, <projection> ) – db.collection.findOne( <query>, <projection> ) • Update – db.collection.update( <query>, <update>, <options> ) • Delete – db.collection.remove( <query>, <justOne> )
• 27.
  Documents var p ={ author: “roger”, date: new Date(), title: “Spirited Away”, avgRating: 9.834, tags: [“Tezuka”, “Manga”]} > db.posts.save(p) 27
• 28.
  Linked vs EmbeddedDocuments { _id : ObjectId("4c4ba5c0672c685e5e8aabf3"), author : "roger", date : "Sat Jul 24 2010 …”, text : "Spirited Away", tags : [ "Tezuka", "Manga" ], comments : [ { author : "Fred", date : "Sat Jul 26 2010…”, text : "Best Movie Ever" } ], avgRating: 9.834 } { _id : ObjectId("4c4ba5c0672c685e5e8aabf3"), author : "roger", date : "Sat Jul 24 2010 19:47:11 GMT-0700 (PDT text : "Spirited Away", tags : [ "Tezuka", "Manga" ], comments : [ 6, 274, 1135, 1298, 2245, 5623], avg_rating: 9.834 } comments { _id : 274, movie_id : ObjectId(“4c4ba5c0672c6 author : "Fred", date : "Sat Jul 24 2010 20:51:0 text : "Best Movie Ever”} { _id : 275, movie_id : ObjectId(“3d5ffc88 author : "Fred", 28
• 29.
  Querying >db.posts.find() { _id :ObjectId("4c4ba5c0672c685e5e8aabf3"), author : "roger", date : "Sat Jul 24 2010 19:47:11 GMT-0700 (PDT)", text : "Spirited Away", tags : [ "Tezuka", "Manga" ] } Note: - _id is unique, but can be anything you’d like 29
• 30.
  Query Operators • ConditionalOperators – $all, $exists, $mod, $ne, $in, $nin, $nor, $or, $size, $type – $lt, $lte, $gt, $gte // find posts with any tags > db.posts.find( {tags: {$exists: true }} ) // find posts matching a regular expression > db.posts.find( {author: /^rog*/i } ) // count posts by author > db.posts.find( {author: ‘roger’} ).count() 30
• 31.
  Atomic Operations • $set,$unset, $inc, $push, $pushAll, $pull, $pullAll, $bit > comment = { author: “fred”, date: new Date(), text: “Best Movie Ever”} > db.posts.update( { _id: “...” }, $push: {comments: comment} ); 31
• 32.
  Arrays • $push -append • $pushAll – append array • $addToSet and $each – add if not contained, add list • $pop – remove last • $pull – remove all occurrences/criteria • { $pull : { field : {$gt: 3} } } • $pullAll - removes all occurrences of each value 32
• 33.
  Indexes // Index nesteddocuments > db.posts.ensureIndex( “comments.author”:1 ) > db.posts.find({‘comments.author’:’Fred’}) // Index on tags (array values) > db.posts.ensureIndex( tags: 1) > db.posts.find( { tags: ’Manga’ } ) // geospatial index > db.posts.ensureIndex({ “author.location”: “2d” ) > db.posts.find( “author.location” : { $near : [22,42] } ) Create index on any Field in Document >db.posts.ensureIndex({author: 1}) 33
• 34.
  Aggregation/Batch Data Processing •Map/Reduce can be used for batch data processing – Currently being used for totaling, averaging, etc – Map/Reduce is a big hammer • Simple aggregate functions available • (2.2) Aggregation Framework: Simple, Fast – No Javascript Needed, runs natively on server – Filter or Select Only Matching Sub-documents or Arrays via new operators • MongoDB Hadoop Connector – Useful for Hadoop Integration – Massive Batch Processing Jobs 34
• 35.
  Deployment & Scaling 35
• 36.
  • Data Redundancy •Automatic Failover / High Availability • Distribution of read load • Disaster recovery Why Replicate? 36
• 37.
  Replica Sets Asynchronous Replication 37
• 38.
  Replica Sets • Oneprimary, many secondaries – Automatic replication to all secondaries • Different delays may be configured – Automatic election of new primary on failure – Writes to primaries, reads can go to secondaries • Priority of secondary can be set – Hidden for administration/back-ups – Lower score for less powerful machines • Election of new primary is automatic – Majority of replica set must be available – Arbiters can be used • Many configurations possible (based on use case) 38
• 39.
  Replica Sets Asynchronous Replication 39
• 40.
  Replica Sets 40
• 41.
  Replica Sets Automatic Leader Election 41
• 42.
  Replica Sets 42
• 43.
  Sharding 43
• 44.
  Sharding mongod Write Scalability Key Range 0..100 mongodmongod Key Range 0..30 Key Range 31..100 44
• 45.
  Sharding mongod mongod Write Scalability KeyRange 0..30 Key Range 31..100 Write Scalability 45
• 46.
  Sharding mongod mongod mongod mongod KeyRange 0..30 Key Range 31..60 Key Range 61..90 Key Range 91.. 100 Write Scalability Key Range 31..100 46
• 47.
  • Splitting datainto chunks – Automatic – Existing data can be manually “pre-split” • Migration of chunks/balancing between servers – Automatic – Can be turned off/chunks can be manually moved • Shard key – Must be selected by you – Very important for performance! • Each shard is really a replica set Sharding Administration 47
• 48.
  Full Deployment mongod mongod mongodmongod Key Range 0..30 Key Range 31..60 Key Range 61..90 Key Range 91.. 100 Write Scalability MongoS MongoS MongoS Primary Secondary Secondary Key Range 0..30 Primary Secondary Secondary Key Range 31..60 Primary Secondary Secondary Key Range 61..90 Primary Secondary Secondary Key Range 91.. 100 48
• 49.
  Primary Secondary Secondary Primary Secondary Secondary Primary Secondary Secondary Primary Secondary Secondary Key Range 0..30 Key Range 31..60 KeyRange 61..90 Key Range 91.. 100 MongoS MongoS Queries MongoS Config Config Config 49
• 50.
  MMS: MongoDB MonitoringService • SaaS solution providing instrumentation and visibility into your MongoDB systems • 3,500+ customers signed up and using service 50
• 51.
  Agenda • Welcome andIntroductions • MongoDB and the New Frontier • MongoDB Technical Overview • Use Case Discussion • Demo 51
• 52.
  Agenda • Welcome andIntroductions • MongoDB and the New Frontier • MongoDB Technical Overview • Use Case Discussion • Demo 52
• 53.
  Queries • Importing Datainto Mongodb – mongoimport --db test --collection restaurants -- file dataset.json • Exporting Data from MongoDB – mongoexport -db test -collection newcolln -file myexport.json 53
• 54.
  Queries • MongoDB queryoperation: • Query in SQL 54
• 55.
  Create/Insert Queries • Db.collection.insert() db.inventory.insert( {item: "ABC1", details: { model: "14Q3", manufacturer: "XYZ Company" }, stock: [ { size: "S", qty: 25 }, { size: "M", qty: 50 } ], category: "clothing" } ) 55
• 56.
  Find Queries • db.collection.find() •db.inventory.find( {} ) • db.inventory.find( { type: "snacks" } ) • db.inventory.find( { type: { $in: [ 'food', 'snacks' ] } } ) • db.inventory.find( { type: 'food', price: { $lt: 9.95 } } ) • db.inventory.find( { $or: [ { qty: { $gt: 100 } }, { price: { $lt: 9.95 } } ] } ) 56
• 57.
  Update Queries • Touse operators on fields of a subdoc use $ db.inventory.update( { item: "MNO2" }, { $set: { category: "apparel", details: { model: "14Q3", manufacturer: "XYZ Company" } }, $currentDate: { lastModified: true } }, false, true ) • False: Update by replacement • True: update all matching documents 57
• 58.
  Queries • Aggregation – SQLQuery – SELECT state, SUM(pop) AS totalPop FROM zipcodes GROUP BY state HAVING totalPop >= (10*1000*1000) – MongoDB – db.zipcodes.aggregate( [ { $group: { _id: "$state", totalPop: { $sum: "$pop" } } }, { $match: { totalPop: { $gte: 10*1000*1000 } } } ] ) 58
• 59.
  Indexes • db.collection.find({field:’value’}).explain() • db.collection.ensureIndex({title: 1 }); • db.collection.dropIndex("index_name"); • db.mycoll.ensureIndex({'address.coord': ‘2d’}) • db.mycoll.find({"address.coord": { $near: [70, 40], $minDistance: 0.05 }}) 59
• 60.
  Indexed and NonIndexed Search >db.mycoll.find({"name" : "Tov Kosher Kitchen"}).pretty().explain() { "cursor" : "BtreeCursor name_1","isMultiKey" : false, "n" : 1, "nscannedObjects" : 1, "nscanned" : 1, "nscannedObjectsAllPlans" : 1,… } > db.mycoll.find({"cuisine" : "Jewish/Kosher"}).pretty().explain() { "cursor" : "BasicCursor", "isMultiKey" : false, "n" : 316, "nscannedObjects" : 25359, "nscanned" : 25359,… } > 60
• 61.
  Thank You 61