mysql,mongodb,mongo-java , Why mongoDB takes less time for Select than Fetch time?


Why mongoDB takes less time for Select than Fetch time?

Question:

Tag: mysql,mongodb,mongo-java

I have collection with 10mill rows without any index.In this case system should read whole table ?

When i use eplain statement then it shows db.employees.find({hundreds2:{$lt:1}},{}).explain(); "nscannedObjects" : 10000000, "n" : 105 millis" : 6027

It works fine.

But i am using java to process query . Here is code

                  whereQuery = new BasicDBObject();
                  whereQuery.put("hundreds2",new BasicDBObject("$lt", rangeQuery));
                  timer.start();   
                        setupMongoDBReadQuery(arrForRange[posOfArr]);
                        cursor = coll.find(whereQuery);
                  timer.stop();
                  this.selectTime= timer.elapsed();

                    timer.start();
                        while (cursor.hasNext())
                        {
                          numberOfRows++;
                      cursor.next();
                    }
                timer.stop();
               this.fetchTime= timer.elapsed();
               this.totalOfSelAndFetch=this.selectTime+this.fetchTime; 

But after test result .I got this information

selTime=2 fetchTime=6350 numRows105 TotalTime6352
 selTime=0 fetchTime=6290 numRows471 TotalTime6290
 selTime=0 fetchTime=6365 numRows922 TotalTime6365

Why fetch time is more than select .As per my knowledge ,while loop is just printing data . Why it taking so much time to print and how mongoDB select number of rows with 0 or 2 millSec?

Same experiment i did in MySQL with similiar code and results are

selTime=6302 fetchTime=1 numRows105 TotalTime6303
selTime=6318 fetchTime=1 numRows471 TotalTime6319
selTime=6387 fetchTime=2 numRows922 TotalTime6389


Answer:

MongoDB uses lazy evaluation with cursors. That means in many cases when you start a MongoDB query which returns a cursor, the query doesn't get executed yet.

The actual selection happens when you start requesting data from the cursor.

The main reason is that this allows you to call methods like sort(by), limit(n) or skip(n) on the cursor which can often be processed much more efficiently on the database before selecting any data.

So what you measure with the "fetch time" is actually also part of the selection.

When you want to force the query to execute without fetching any data yet, you could call explain() on the cursor. The database can't measure the execution time without actually performing the query. However, in actual real-world use, I would recommend you to not do this and use cursors the way they were intended.


Related:


MySQL Query returning strange values


php,mysql
The query is supposed to do the following: Obtain the question and associated answers based on the identification number of the question. (In this case the identification number is called account_id. Order them so that each question (and it's appropriate answers) are lined up in order. The query: SELECT *...

How do I display my mysql table column headers in my php/html output?


php,html,mysql,table,data
2 Questions... Scenario: I would like to query my database table via a form and then display the results that occur(if there are results) and my current situation is that it does work but it clears the form completely and leaves my to an empty page with just the results...

Purging Database - Count purged/not-purged tables


mysql,sql,sql-server,database,stored-procedures
So I am working with a database where I will be purging various tables that contain rows that are older than 30days. I have fairly limited knowledge of SQL and wanted to know if there was a certain way to select the row that will be purged and the rows...

C# - Can't connect to remote MySQL server


c#,mysql
My problem is that I can't connect to my website remote MySQL server. I have read all answers in stackoverflow.com, but I can't find right answer. Here's my C# code: using System; using System.Collections.Generic; using System.Data; using System.Data.SqlClient; namespace ConsoleApplication3 { class Program { static void Main(string[] args) { SqlConnection...

mysql detect field value change


mysql
I have found a very similar question with exact same title here: Mysql, Check field value change?, but it is not exactly what I need. DB: MySql, Table: tbl_geodata I have query: SELECT id,timestamp,ver_fw FROM tbl_geodata WHERE imei LIKE '353227026533507' Result is: --Full data +------+---------------------+--------+ | id | timestamp |...

how will i order this query by DoctorName


php,mysql
I use this query but I don't know how will I order this by DoctorName? $q=mysqli_query($link,"Select * from doctor where status='". $stat ."'"); ...

Using VLOOKUP formula or other function to compare two columns


mysql,excel,vba,date
I have one table like this: SHORT TERM BORROWING 1/6/2009 94304 12/31/2010 177823 6/30/2011 84188 12/31/2011 232144 6/30/2012 94467 9/30/2012 91445 12/31/2012 128523 3/31/2013 83731 6/30/2013 78330 9/30/2013 70936 12/31/2013 104020 3/31/2014 62345 6/30/2014 62167 9/30/2014 63494 12/31/2014 104239 3/31/2015 69056 I have another column which lists each date from...

Trying to rewrite mysql_* to pdo


php,mysql,pdo
First of all I should say that I started to learn PDO and trying to rewrite some old mine codes. Here is one that I found on tutorial for a jquery/ajax/pdo/mysql comment system which is working. This is the mysql_* part which I'm trying to rewrite -> file submit.php which...

MySQL substring match using regular expression; substring contain 'man' not 'woman'


mysql,regex
I have an issue while I fetch data from database using regular expression. While I search for 'man' in tags it returns tags contains 'woman' too; because its substring. SELECT '#hellowomanclothing' REGEXP '^(.)*[^wo]man(.)*$'; # returns 0 correct, it contains 'woman' SELECT '#helloowmanclothing' REGEXP '^(.)*[^wo]man(.)*$'; # returns 0 incorrect, it can...

Can Rails deal with DB uniqueness without index?


mysql,ruby-on-rails,rdbms
I see add_index ~ unique: true statement in schema.rband think uniqueness is constraint for table, not for index.Using index is one of way realizing uniqueness, Programmer should not designate to the RDBMS "how" and index quicken searching but take costs inserting. In fact, is there another way to keep uniqueness...

Symfony2 creating and persisting entity relationships


php,mysql,symfony2,doctrine2
I have two entities Skin and Email. I want Email to be a part of the Skin entity, however I can't use the console to update schema automatically right now, and that is probably why I can't get the relationship to work. So I want to store all the Emails...

How to call MySQL view in Struts2 or Hibernate


java,mysql,hibernate,java-ee,struts2
I have a view in MySQL database CREATE VIEW CustInfo AS SELECT a.custName, a.custMobile, b.profession, b.companyName, b.annualIncome FROM customer a INNER JOIN cust_proffessional_info b ON a.cust_id=b.cust_id Is there any way that i can call this view using Struts2 or in Hibernate. I have tried to search it but could not...

PHP: While loop not working after adjusting SELECT for SQL injection prevention


php,mysql,select,sql-injection,associative-array
I am trying to set up PHP queries for MySQL in a way to prevent SQL injection (standard website). I had a couple of INSERT queries where changing this worked well but on the following SELECT I keep getting an error since the update and it looks like the while...

Temporarily Number Rows to Group by in MYSQL


mysql
I am trying to create a temporary column that gives each set of five results a number so that I can group by.. I am having a lot of trouble trying to work out how to do this. I don't want to take any criteria into account, I want the...

MongoDB Java Driver 3.0 MapReduce


mongodb,mongodb-java
This is the code I'm using to run map reduce on sourceCollectionName and to get the output to targetCollectionName. But the targetCollectionName is never created. new MongoClient("localhost").getDatabase(dbName).getCollection(sourceCollectionName) .mapReduce(map, reduce) .action(MapReduceAction.REPLACE) .databaseName(dbName) .collectionName(targetCollectionName) .sharded(false); Although I'm able to get output as MapReduceIterable and when I iterate this the result is dumped...

Use JSON file to insert data in database


javascript,json,mongodb,meteor,data
I'm using my JSON file like this to insert data in my collection : var content = JSON.parse(Assets.getText('test.json')); console.log('inserting...'); Profiles.insert({ user: id, data:content }; But I would like to have a "data's tree" like that : [ user: "rtegert23423131", firstname:"test", surname:"test2", // ... ] Not like that : [ user:...

I Want to fetch SQL Records in MySQL of current Year


mysql
I am using the query SELECT count(enq.`enquiryId`), Month(enq.`date`), Year(enq.`date`) FROM enquiry enq GROUP BY MONTH(enq.`date`) But I get all year records. I just want to get Current or any input year(At Runtime) records. Please help....

PHP / MySQLi: How to prevent SQL injection on INSERT (code partially working)


php,mysql,mysqli,sql-injection,sql-insert
I am new to PHP and hope someone can help me with this. I would like to store two values (an email and a password) in a MySQL db using PHP. The input is passed to the PHP page via Ajax in jQuery (through an onclick event on a website)....

How to overwrite object Id's in Mongo db while creating an App in Sails


mongodb,sails-mongo
I am new to Sails and Mongo Db. Currently I am trying to implement a CRUD Function using Sails where I want to save user details in Mongo db.In the model I have the following attributes "id":{ type:'Integer', min:100, autoincrement:true }, attributes: { name:{ type:'String', required:true, unique:true }, email_id:{ type:'EMAIL',...

What are some patterns I can look at for database implementations in JavaScript?


javascript,node.js,mongodb
I'm fairly new to JavaScript, and I'm busy playing around with a node test app and MongoDB. I'm at a point where I'd like to start with the db side of the app, but I'm not sure what patterns are most commonly used in such a stack, and more importantly,...

Select most recent entry where timestamp is from the past hour


mysql
I've tried multiple ways, all of them resulting in incorrect rows returned, so far I've tried SELECT * FROM table WHERE id=1 AND timestamp >= NOW() - INTERVAL 1 HOUR ORDER BY timestamp DESC LIMIT 1 .... AND timestamp > DATE_SUB(NOW(), INTERVAL 1 HOUR) ORDER BY timestamp DESC LIMIT 1...

SQL Group By multiple categories


php,mysql,sql,mysqli
This is the query I'm using to count how many events are booked for each category of bookings. SELECT Categories.name, count(case when locations.name ='loc 1' then 1 end) as Location1, count(case when locations.name ='loc 2' then 1 end) as Location2, count(case when locations.name ='loc 3' then 1 end) as Location3,...

If I export my database with phpmyadmin will it lock my tables or take my database down?


mysql,database,phpmyadmin
I want to create a backup of my database using the phpmyadmin export function. the database can't have any down time so I need to know if running a database export will take the database down? I've looked on-line but all I get is instructions on how to export. No...

Fatal error in if stament [duplicate]


php,mysql,mysqli
This question already has an answer here: Compile Error: Cannot use isset() on the result of an expression 2 answers Error - Cannot use isset() on the result of an expression (you can use "null !== expression" instead) I'm getting the error above the start of the if statement...

Inner Join 3 tables pl/sql


mysql,sql
Basically i want to inner join 3 tables and the case is understood by the query itself as follows. Tables: A has 2 columns column 1 and column 2 B has 2 columns column 3 and column 4 C has 3 columns column 5,column 6 and column 7 Query: select...

IBM Cognos _days_between function not working


mysql,database,date,cognos
I'm having an issue with Cognos 10. I'm trying to calculate the number of days between to dates so I use the _days_between( date1, date2 ) function. _days_between ([Derniere Date Changement diaphragme] , [Premiere Date Changement diaphragme]) I'm quite sure the two Dates are Date objects ( I set them...

How do I find records in one table that do exist in another but based on a date?


mysql,sql
I want to find all the dealers who haven't had an order in 2015 yet. I know this query doesn't work but I thought it might be helpful to understand what I want to do. In this example I want to get just "Bob" as a result. He is the...

Having two arrays in variable php


php,mysql,arrays,variables,multidimensional-array
i've this code, i'm trying to explode my date e.g "04 June 2015" to have the day and the month in other 2 variables, but i have all in a method, with an array, is it possible, my code doesn't work, it writes "Array[0]" why? while ($i < $number) {...

How to program a recurring billing/invoice system using PHP and MySQL


php,mysql
I have already programmed a basic invoicing system using PHP/MySQL which has a table structure as follows; Invoice table; invoice_id, invoice_date, customer_id, etc Invoice line table; invoice_line_id, invoice_id, quantity, price, description, etc I need the system to automatically generate future invoices at set intervals (for example every 1 week or...

MySQL trigger help, capturing a variable from php insert into


php,mysql,triggers
I am writing a MySQL trigger to encrypt a column of sensitive data using RSA plugin. There is an insert statement from the website which is PHP based $sql = "insert into redcap_data (project_id, event_id, record, field_name, value) values ($project_id, " . getEventId() . ", '" . prep($_GET['id']) . "',...

How to search images by name inside a folder?


php,mysql,image
I have a MySQL table with a column "rounds" and each "rounds" has his own photos. Exemple round1 has photos from start=380 end=385. This means it has 6 photos and the name of the photos contains 380,381,382,383,384 or 385 inside. I use this PHP code to display the photos from...

Select all none null values from all columns in a table


mysql,sql
I am having an issue going through column by column and getting a count of all the non-null values in each column. I think issue is how I am getting the column names. In the last select statement where I am trying to select all of the values in each...

How to post time value in database in PHP


php,mysql
I was trying to post date value inside my database but every time I try, the date is not coming, but the auto row is working. Here is my PHP code: <?PHP mysql_connect("localhost","root","") or die("Could not connect: " . mysql_error()); mysql_select_db("test") or die("Couldn't connect db!"); $times = $_POST['times']; $id_time =...

Ignore Group if LIMIT is not reached in MySQL


mysql
I am working on a rather tricky SQL for my level of knowledge. I have searched and searched for an answer but haven't came across anything. Hopefully someone can shed some light on this. How can you stop SQL from outputting group of rows if the limit set is not...

How to create multiple jquery form fields and insert in mysql database, without using mysql_real_escape_string


javascript,php,jquery,mysql
I need to create dynamic text fields with a button (which simply adds a new row of dynamic text fields). Then I need to INSERT the data in a in the database. I borrowed an example of what I need done from someone else's question, the problem is, it uses...

timestamp SQL to Excel


php,mysql,sql,excel
If this is a duplicate, please let me know, I haven't found anything. I have written a php file that can read content from a database table and write it into a excel .xls file. Everything works fine except by that timestamps. In my generated .xls file every timestamp is...

MySQL - How can I know my query is tuned?


mysql,performance,explain
I'm creating a DB from scratch and I'm trying to create queries with performance in mind. For test purpose I filled my tables with test data. My query needs to join 3 tables: 2 with millions of rows and the third with hundred thousands rows. How can I know my...

Query how often an event occurred at a given time


mysql,sql
[Aim] We would like to find out how often an event "A" ocurred before time "X". More concretely, given the dataset below we want to find out the count of the prior purchases. [Context] DMBS: MySQL 5.6 We have following dataset: user | date 1 | 2015-06-01 17:00:00 2 |...

Query with filter builder on nested array using MongoDB C# driver


c#,mongodb,mongodb-query,mongodb-csharp,mongodb-csharp-2.0
Consider the following object structure stored as documents: public class Foo { public string Id { get; set; } public ICollection<FooBar> Bars { get; set; } // ... } public class FooBar { public string BarId { get; set; } // ... } Using a LINQ-style query with the driver...

concatenate field names in a mysql update with inner join


php,mysql
Concatenating two field names from different tables using INNER JOIN during UPDATE statement. I am trying to concatenate two field names in a mysql update statement. This is what I have so far, needless to say it isn't working, any suggestions would be much appreciated. UPDATE products INNER JOIN sub_categories...

MySQL: Select several rows based on several keys on a given column


mysql,sql,database
I have this mysql table structure: ------------------------------------ | item_id | meta_key | meta_value | ------------------------------------ 159 category Bungalow 159 location Lagos 159 price 45000 160 category Bungalow 160 location Abuja 160 price 53500 ... 350 category Bungalow 350 location Lagos 350 price 32000 What I'd like to do is select...

Inserting a variable in MongoDB specifying _id field


python,mongodb,pymongo
I want to insert a variable, say, a = {1:2,3:4} into my database with a particular id "56". It is very clear from the docs that I can do the following: db.testcol.insert({"_id": "56", 1:2, 3:4}) However, I cannot figure out any way to insert "a" itself, specifying an id. In...

compare today's date with unix timestamp value in database


php,mysql
In database I am storing date value unix timestamp value for e.g.'1434952110' using time() now I am trying to compare todays value with this value as below $jobpostdate = date("Y-m-d");//today's date passing in database to compare query $sql = "SELECT jsp_title, jsp_subtitle, jsp_desc, jsp_location, jsp_date "; $sql .= "FROM js_projects...

Retrieve Values As Column


mysql,sql
I've the following table in my PHPMYADMIN The desired output is headline                              impressions              clicks Buy A new Iphone                 ...

Notice: Array to string conversion in “path of php file” on line 64


php,mysql,arrays,oracle
PHP CODE -: <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <title>Untitled Document</title> </head> <body> <?php $rows =0; $fp = fopen("leave1.csv","r"); if($fp){ while(!feof($fp)){ $content = fgets($fp); if($content) $rows++; } } fclose($fp); //echo $rows; $_SESSION['rows'] = $rows; ?>...

Php Mysql Query not working properly


php,mysql
I have a table name tblnetworkstatus and I have 11 columns Id issue_name affected_server affected_service issue_type priority duration status start_date end_date description I am getting id in affected_server and affected_service which I am storing in my DB, now I have three situations Either both affected_server and affected_service has been selected...

MySQLdb Python - Still getting error when using CREATE TABLE IF NOT EXISTS


python,mysql
I'm using this code to create a database with tables in Python: def CreateDatabase(): global DB_CNX global DB_NAME cursor = DB_CNX.cursor() cursor.execute("""CREATE DATABASE IF NOT EXISTS {} DEFAULT CHARACTER SET 'utf8'""".format(DB_NAME)) cursor.execute("""CREATE TABLE IF NOT EXISTS NAMES(NAME VARCHAR(50) PRIMARY KEY NOT NULL)""") DB_CNX.close() But even if I use the syntax...

C# MySQL Parameters.AddWithValue


c#,mysql
Hello I found this code snippet for Adding Values to MySQL Commands in C# MySqlCommand command = connection.CreateCommand(); command.CommandText = "INSERT INTO tb_mitarbeiter (Vorname) VALUES (?name)"; command.Parameters.AddWithValue("?name", mitarbeiter); connection.Open(); command.ExecuteNonQuery(); Now I want to add data to more than one coloumn, but if i try it like this: command.Parameters.AddWithValue("?id", Projektid,...

mongodb populate method not working


node.js,mongodb,model,populate,auto-populate
Here is my code for models var postSchema = new mongoose.Schema({ created_by: {type: Schema.ObjectId, ref:'User', autopopulate: true }, //should be changed to ObjectId, ref "User" created_at: {type: Date, default: Date.now}, text: String }); var userSchema = new mongoose.Schema({ username: String, password: String, //hash created from password created_at: {type: Date, default:...

Using Sum in If in Mysql


mysql,sql,select,sum
I want to select the sum of values of my_field. The unit of my_field is in seconds. I don't want to exceed 24 hours, so when it exceeds 24 hours, I want it to select 24*60*60 = 86400. SELECT IF(SUM(my_field) > 86400, 86400, SUM(my_field)) ... This solution doesn't seem to...