HCI NoSQL Databases 2. PyMongo

NoSQL Databases CPDD Computer Education Unit Version: Feb 2019 1 Name: __________________________ ( ) Class: _________ Date: _________ Lesson 2: Using PyMongo Instructional Objectives: By the end of this task, you should be able to:  Use PyMongo to connect to MongoDB server  Create MongoDB databases using PyMongo  Access MongoDB databases using PyMongo  Obtain and modify MongoDB documents with PyMongo  Use query operators in PyMongo What is PyMongo? MongoDB databases can be accessed using different programming languages like C, Java and Python. To access MongoDB databases using Python, we use the Python driver for MongoDB, PyMongo. To use PyMongo, start your Python program by importing the pymongo package. Try typing and running program 1 below. (Remember to start the MongoDB server before you run the program.) The program connects to the MongoDB server and outputs the databases currently in the MongoDB server. Program 1: access.py 1 2 3 4 5 6 import pymongo client = pymongo.MongoClient("127.0.0.1", 27017) databases = client.database_names() print("The databases in the MongoDB server are:") print(databases) client.close() Line 2 connects to the local MongoDB database which is usually at port 27017. You can see the port number when you start the MongoDB server. Line 6 closes the connection to the server. The MongoDB server window should remain open while you want to access the MongoDB database.

NoSQL Databases CPDD Computer Education Unit Version: Feb 2019 2 Line 3 of the code retrieves the names of the databases, stored as a Python list. As an example, let’s create a database to store details on movie information. Please note that MongoDB waits until you have inserted at least one document before it actually creates the database and collection. Program 2: insert.py 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 import pymongo client = pymongo.MongoClient("127.0.0.1", 27017) db = client.get_database("entertainment") coll = db.get_collection("movies") coll.insert_one({"_id":1, "title":"Johnny Maths", "genre":"comedy"}) coll.insert_one({"title":"Star Walls", "genre":"science fiction"}) coll.insert_one({"title":"Detection"}) #no genre list_to_add = [] list_to_add.append({"title":"Badman", "genre":"adventure", "year":2015}) list_to_add.append({"title":"Averages", "genre":["science fiction","adventure"], "year":2017}) list_to_add.append({"title":"Octopus Man", "genre":"adventure", "year":2017}) list_to_add.append({"title":"Fantastic Bees", "genre":"adventure", "year":2018}) list_to_add.append({"title":"Underground", "genre":"horror", "year":2014}) coll.insert_many(list_to_add) c = db.collection_names("entertainment") print ("Collections in entertainment database: ",c) client.close() Program 2 demonstrates two ways of inserting documents into collection entertainment. To insert one document, you can use the insert_one() method shown in lines 5 and 6. Notice that all not fields are required for insertion, as shown

NoSQL Databases CPDD Computer Education Unit Version: Feb 2019 3 in lines 5 to 7. To insert multiple documents, you can use the insert_many() method to insert a list of documents as shown in line 13. MongoDB will assign a unique _id to each document. You can customise the _id by stating it during the insertion process, as shown on line 5. However, this means that you cannot run program 2 again until you remove this document, otherwise the program will produce an error. You can try to run the program again with line 5 commented out. Duplicates of the other documents will be created. Line 15 gathered the list of collections while line 16 prints it as a list. 1. Write a Python program to ask for one movie title and the year of movie, then insert the document into the movie collection. Assume no genre is given. Q1 Program: q1.py import pymongo title = input("Enter movie title") year = input("Enter year of movie") client = pymongo.MongoClient("127.0.0.1", 27017) db = client.get_database("entertainment") coll = db.get_collection("movie") coll.insert_one({"title":title, "year":year}) client.close() Go further! Can you extend the program to include genres (where movies can have none or multiple genres)? Of course, for large amount of data, it is more efficient to import from a file. 2. The program below reads from a delimited text file and insert the documents into the database. Parts of the input file and the program are given below. Fill in the blanks. Input File: input.txt Amanda,45 Bala,28 Charlie,33 Devi,29 ... Q2 Program: q2.py

NoSQL Databases CPDD Computer Education Unit Version: Feb 2019 4 import pymongo, csv client = pymongo.MongoClient("127.0.0.1", 27017) db = client.get_database("entertainment") coll = db.get_collection("users") with open('input.txt') as csv_file: csv_reader = csv.reader(csv_file, delimiter=',') for row in csv_reader: coll.insert_one({"name":row[0], "age":row[1]}) client.close() If the file is in JSON (JavaScript Object Notation), the data can also be imported using the load() function. A sample JSON file and program is shown below. JSON file: input.json [ { "name": "Amanda", "age": "45" }, { "name": "Bala", "age": "28" }, { "name": "Charlie", "age": "33" }, { "name": "Devi", "age": "29" } ] Program 3: loadjson.py 1 2 3 4 5 6 import pymongo, json client = pymongo.MongoClient("127.0.0.1", 27017) with open('data.json') as file: data = json.load(file) client['entertainment']['moreusers'].insert_many(data) client.close() Let’s now try to get the data from the database. Program 4: view.py

NoSQL Databases CPDD Computer Education Unit Version: Feb 2019 5 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 import pymongo client = pymongo.MongoClient("127.0.0.1", 27017) db = client.get_database("entertainment") coll = db.get_collection("movies") result = coll.find() print("All documents in movie collection:") for document in result: print(document) print("Number of items in movie collection:", coll.count()) result = coll.find({'genre': 'adventure'}) print("All movies with adventure genre:") for document in result: print(document) query2 = {'genre': 'adventure', 'year': {'$gt': 2016}} result = coll.find(query2) print("All titles of movies with adventure genre after 2016:") for document in result: print(" - " + document.get('title')) print("There are",result.count(),"movies in the list above.") client.close() The method find() in line 5 returns a Cursor of all the documents in the movie collection. The results can be printed with a loop. The count() method gives the number of documents in the movie collection. Line 11 onwards demonstrates the searching of specific documents in MongoDB. The query can be formed directly as shown in line 11, or built with variables (see lines 16 and 17). Each document is just a Python dict, so you can use the usual built-in methods for dict. For example, line 20 uses the get() method to retrieve the value of title. This allows you to extract the value for a particular field in the document. Line 16 of the code creates the query to find the documents with adventure genre and year greater than 2016. It can be rewritten using the $and operator: query2 = {'$and':[{'genre': 'adventure'}, {'year': {'$gt': 2016}}]} Line 21 shows how to obtain the number of documents in the search results. Using the count() method, it gives the number of titles of movies with adventure genre after 2016. The following is a list of commonly used query operators.

NoSQL Databases CPDD Computer Education Unit Version: Feb 2019 6 $eq Equals to $gt Greater than $gte Greater than or equal to $lt Less than $lte Les