mongodb,group-by,nested,aggregate,multiple-columns , Display mongodb multiple group by query results in a nested format


Display mongodb multiple group by query results in a nested format

Question:

Tag: mongodb,group-by,nested,aggregate,multiple-columns

Given the following dataset:

[
  {
    "account_id" : "1111"
    "task_id" : "aaaa",
    "workweek" : "20",
    "hours": "18"
  },
  {
    "account_id" : "1111"
    "task_id" : "aaaa",
    "workweek" : "20",
    "hours": "12"
  },
  {
    "account_id" : "1111"
    "task_id" : "aaaa",
    "workweek" : "21",
    "hours": "10"
  },
  {
    "account_id" : "1111"
    "task_id" : "bbbb",
    "workweek" : "21",
    "hours": "5"
  },
  {
    "account_id" : "2222"
    "task_id" : "cccc",
    "workweek" : "21",
    "hours": "15"
  }
]

I'd like to group the documents and have the results in the following format:

[
  {
    "account_id": "1111",
    "tasks": [
      {
        "task_id": "aaaa",
        "workweeks": [
          {
            "workweek": "20",
            "total_hours": "30" 
          },
          {
            "workweek": "21",
            "total_hours": "10" 
          }
        ]
      },
      {
        "task_id": "bbbb",
        "workweeks": [
          {
            "workweek": "21",
            "total_hours": "5" 
          }
        ]
      }
    ]
  },
  {
    "account_id": "2222",
    "tasks": [
      {
        "task_id": "cccc",
        "workweeks": [
          {
            "workweek": "21",
            "total_hours": "15" 
          }
        ]
      },
    ]
  }
]

My broken aggregation code is the following:

db.tasks.aggregate([
  {"$group": { 
    "_id": {
      "account_id": "$account_id", 
      "task_id": "$task_id",
      "workweek": "$workweek",
    },
    "total_hours": {$sum: "$hours"}
  }},
  {"$group": { 
    "_id": "$_id.account_id",
    "tasks": { 
      "$push": {
        "task_id": "$_id.task_id",
        "workweeks": {
          "$push": {
            "workweek": "$_id.workweek",
            "total_hours": "$total_hours"  
          }
        }
      }
    }
  }}
]);

I receive this error: "errmsg" : "exception: invalid operator '$push'", And I assume it is because I can only $push once per $group block.

Any ideas on how to achieve the desired result?


Answer:

Please try with one additional (middleware) "group-by" step:

db.tasks.aggregate([
  {"$group": { 
    "_id": {
      "account_id": "$account_id", 
      "task_id": "$task_id",
      "workweek": "$workweek",
    },
    "total_hours": {$sum: "$hours"}
  }},
  {"$group": { 
    "_id": { 
        "$_id.account_id",
        "$_id.task_id"
    },
    "workweeks": {
      "$push": {
        "workweek": "$_id.workweek",
        "total_hours": "$total_hours"  
      }
    }
  }},
  {"$group": { 
    "_id": "$_id.account_id",
    "tasks": { 
      "$push": {
        "task_id": "$_id.task_id",
        "workweeks": "$workweeks"
      }
    }
  }}
]);

Related:


mongodb populate method not working


node.js,mongodb,model,populate,auto-populate
Here is my code for models var postSchema = new mongoose.Schema({ created_by: {type: Schema.ObjectId, ref:'User', autopopulate: true }, //should be changed to ObjectId, ref "User" created_at: {type: Date, default: Date.now}, text: String }); var userSchema = new mongoose.Schema({ username: String, password: String, //hash created from password created_at: {type: Date, default:...

Query not working - MongoDB


mongodb
Given this kind of documents: "_id" : ObjectId("5461c8f0426f727f16010000"), "f" : [ { "t" : "ry", "v" : 1972 }, { "t" : "g", "v" : [ "Crime", "Drama" ] }, { "t" : "ml", "v" : "English" }, { "t" : "k", "v" : "movie" }, { "t" : "ai",...

MongoDB explain() cursor field


mongodb
I am starting to investigate into MongoDB and was looking into query stats and optimising through indexes. The examples I am reading use the explain function but their output is different from mine. Theirs looks like: { "cursor" : "BtreeCursor username_1", "nscanned" : 1, "nscannedObjects" : 1, "n" : 1,...

Query with filter builder on nested array using MongoDB C# driver


c#,mongodb,mongodb-query,mongodb-csharp,mongodb-csharp-2.0
Consider the following object structure stored as documents: public class Foo { public string Id { get; set; } public ICollection<FooBar> Bars { get; set; } // ... } public class FooBar { public string BarId { get; set; } // ... } Using a LINQ-style query with the driver...

using MongoDB aggregate count subdata


mongodb,aggregate
here is my data in mongodb: { "data": { "order_goods": [{ "category": 235 }, { "category": 666 }] } }, { "data": { order_goods: [{ "category": 235 }] } } here is my expected output: {"category":235, "total":2} {"category":666, "total":1} I have try many ways about aggregate such as $group, but...

Meteor/MongoDB limiting the result


mongodb,meteor
I am trying to find all documents and publish at most 5 from the results. Following this section of the MongoDB doc, I am trying to do this: Meteor.publish('teams', function () { return Teams.find().limit(5); }); Yet, in the server console, I get an exception: Exception from sub teams id Pm6jKL8Sv3FSDSTfM...

How to add a subdocument in a mongoose schema


javascript,node.js,mongodb,mongoose
I am trying to create a subdocument in a mongoose schema from node.js/Express. I have two schemas: Member and Address Member.js // app/models/member.js // load mongoose since we need it to define a model var mongoose = require('mongoose'), Schema = mongoose.Schema var Address = require('./address'); var MemberSchema = Schema({ FName...

How to add new items to an array in MongoDB


arrays,node.js,mongodb
I'm trying to add a new item to whichever name that was passed in under whichever id. My first problem is that it seems like its not grabbing the values from any of my variables (name, item, id), instead just using them as object keys. My next issue is that...

c# mongodb driver groupby


c#,mongodb,mongodb-csharp
I try to execute this query: MongoCollection<AnalyticsClicks> dbCollection = DetermineCollectionName<AnalyticsClicks>(); var query = from c in dbCollection.AsQueryable() where c.UserId == userId && c.CampaignId == campaignId select new { c.Email, c.Link }; var res = query.GroupBy(x => x.Email, b => b.Link).Count(); but I have exception: The GroupBy query operator is not...

filtering a dataframe after groupby in pandas


python,pandas,group-by
I have the following dataframe: In [4]: df Out[4]: Symbol Date Strike C/P Bid Ask 0 GS 6/15/2015 200 c 5 72 1 GS 6/15/2015 200 p 5 72 2 GS 6/15/2015 210 c 15 0 3 GS 6/15/2015 210 p 15 54 4 GS 7/15/2015 200 c 20 50...

Inserting a variable in MongoDB specifying _id field


python,mongodb,pymongo
I want to insert a variable, say, a = {1:2,3:4} into my database with a particular id "56". It is very clear from the docs that I can do the following: db.testcol.insert({"_id": "56", 1:2, 3:4}) However, I cannot figure out any way to insert "a" itself, specifying an id. In...

How to get node.js to connect to mongolab using mongoose


database,node.js,mongodb,mongoose,mongolab
I've been trying to use mongoose (module for node.js and mongodb). And tried to get a connection with mongolab up and running. I tried the following at the top of my app.js file, but I couldn't seem to enter the db.on function. global.mongoose = require('mongoose'); var uri = 'mongodb://username:[email protected]:#####/db'; global.db...

Use JSON file to insert data in database


javascript,json,mongodb,meteor,data
I'm using my JSON file like this to insert data in my collection : var content = JSON.parse(Assets.getText('test.json')); console.log('inserting...'); Profiles.insert({ user: id, data:content }; But I would like to have a "data's tree" like that : [ user: "rtegert23423131", firstname:"test", surname:"test2", // ... ] Not like that : [ user:...

selecting a column value based on a different column value after applying Groupby


python,group-by,dataframes
i am able to get this to work, but not after I apply a groupby. in this example I simply want to have the last column contain the lowest value from column x. I have popuymated the df with a column called yminx with is what I would like my...

How to use a variable as an Object Key [MongoDB] [duplicate]


node.js,mongodb
This question already has an answer here: How to use a variable as a field name in mongodb-native findOne()? 1 answer I'm trying to use variable name given to me by my function, but its actually setting the object key equal to 'name' function addlist(name, item, id){ // Add...

MongoDB Java Driver 3.0 MapReduce


mongodb,mongodb-java
This is the code I'm using to run map reduce on sourceCollectionName and to get the output to targetCollectionName. But the targetCollectionName is never created. new MongoClient("localhost").getDatabase(dbName).getCollection(sourceCollectionName) .mapReduce(map, reduce) .action(MapReduceAction.REPLACE) .databaseName(dbName) .collectionName(targetCollectionName) .sharded(false); Although I'm able to get output as MapReduceIterable and when I iterate this the result is dumped...

Do you get the same performance using index prefixes?


performance,mongodb,indexing
Say I have a collection containing documents like the one below: { _id: ObjectId(), myValue: 123, otherValue: 456 } I then create like below: {myValue: 1, otherValue: 1} If I execute the following query: db.myCollection.find({myValue: 123}) will I get the same performance with my index as I would if I...

How to check if multiple documents exist


mongodb,mongodb-query
Is there such a query that gets multiple fields, and returns which of these exists in the collection? For example, if the collection has only: {id : 1} {id : 2} And I want to know which of [{id : 1} , {id : 3}] exists in it, then the...

elastic search sort in aggs by column


sorting,elasticsearch,group-by,order
I am trying to sort in elastic search in aggs, equivalent in mysql "ORDER BY Title ASC/DESC". Here is the index structure: 'body' => array( 'mappings' => array( 'test_type' => array( '_source' => array( 'enabled' => true ), 'properties' => array( 'ProductId' => array( 'type' => 'integer', 'index' => 'not_analyzed'...

Creating index while updating the documents


mongodb,indexing
I have a collection I am updating adding a new field. The document looks like: {"A": "P145", "B":"adf", "C":[{"df":"14", "color":"blue"},{"df":17}], "_id":ObjectID(....), "Synonyms":{"Synonym1": "value1", "Synonym2": ["value1", "value2"]}} In the update I am adding new elements to C I want to create a index on the field A and B. A and...

Cassandra data model to store embedded documents


mongodb,database-design,cassandra
In mongodb we can able to store embedded documents into a collection.Then, How do we store embedded documents into cassandra??? For this sample JSON representation??? UserProfile = { name: "user profile", Dave Jones: { email: {name: "email", value: "[email protected]", timestamp: 125555555}, userName: {name: "userName", value: "Dave", timestamp: 125555555} }, Paul...

How could I get the matched nested items in array


mongodb
How could I get the matched nested items in array I want to return the matched items in nested array. For example, I want to filter out the records that contain "A428 ","A429 " in their items How could I get it ? Query pipeline_work = [ { '$match': 'records.items':...

Get json values from key-value pair without passing keys in meteor


javascript,json,mongodb,meteor
i am using keys to get the values from json in meteor. My json is in this format. { "_id" : "SXTJBs7QLXoyMFGpK", "Brand" : { "value" : "Nike" }, "Material" : { "value" : "Cooton" }, "Price" : { "value" : "67484" }, "ComboId" : { "value" : "y23" },...

Mongoose : update collection with previous data


node.js,mongodb,mongoose
I am learning NodeJs and for a demo project i need to use MonGoDB with Mongoose. I created a collection for a photo album. The first row is : albumName and the second row is : pictures. I don't undersand how i can save all my pictures in the row...

Get matching elements in mongodb array


php,mongodb,aggregation,pipeline
I want to use aggregation to get this array only with those tickets, which have start field after 2015-06-16. Can someone help me with the pipeline? { "name" : "array", "tickets" : [ { "id" : 1, "sort" : true, "start" : ISODate("2015-06-15T22:00:00.000Z") }, { "id" : 2, "sort" :...

Query an array of embedded documents in mongodb


mongodb,embedded-documents
I'm having a little trouble writing a query that needs to compare a given value against a certain field in all embedded documents within an array. I will give an example to make the issue less abstract. Let's say I want to use MongoDB to store the last queries that...

Find Mongoid geospacial circles that contain a point


mongodb,mongoid,geospatial
I have Mongoid documents that represent services offered by local merchants. Each has one or more locations (lat/lng point) with a service area (radius). Given a customer location (lat/lng point), how can I find all the documents where the customer location falls within the service area? class Service include Mongoid::Document...

Why the query scans 8x times the documents it finds - MongoDB


mongodb
I've got 50 documents (for trial purposes) more or less like this one: "_id" : ObjectId("5461c8f0426f727f16010000"), "f" : [ { "t" : "ry", "v" : 1972 }, { "t" : "g", "v" : [ "Crime", "Drama" ] }, { "t" : "ml", "v" : "English" }, { "t" : "k",...

Is there a built-in function to get all unique values in an array field, across all records?


arrays,node.js,mongodb,mongoose,schema
My schema looks like this: var ArticleSchema = new Schema({ ... category: [{ type: String, default: ['general'] }], ... }); I want to parse through all records and find all unique values for this field across all records. This will be sent to the front-end via being called by service...

group by is not working in postgreSQL


sql,postgresql,group-by
I want group my data by createdAt but it is not working I don't know why... Only group by id which is useless for me. This is working: SELECT "id", "createdAt", "updatedAt" FROM "tables" AS "Table" WHERE "Table"."createdAt" BETWEEN '2014-04-21 20:46:25.938-04' AND '2015-04-21 20:46:25.938-04' GROUP BY "id"; This is what...

How to overwrite object Id's in Mongo db while creating an App in Sails


mongodb,sails-mongo
I am new to Sails and Mongo Db. Currently I am trying to implement a CRUD Function using Sails where I want to save user details in Mongo db.In the model I have the following attributes "id":{ type:'Integer', min:100, autoincrement:true }, attributes: { name:{ type:'String', required:true, unique:true }, email_id:{ type:'EMAIL',...

Getting failed to load c++ bson extension error using Mongodb and Node.js


javascript,node.js,mongodb
I am getting the following error while trying to run server using Node.ja with Mongodb. Error: { [Error: Cannot find module '../build/Release/bson'] code: 'MODULE_NOT_FOUND' } js-bson: Failed to load c++ bson extension, using pure JS version { [Error: Cannot find module '../build/Release/bson'] code: 'MODULE_NOT_FOUND' } js-bson: Failed to load c++...

Capping values after a trigger level in a different variable _after GroupBy


pandas,triggers,group-by
There was an elegant answer to a question almost like this provided by EdChum. The difference between that question and this is that now the capping needs to be applied to data that had had "GroupBy" performed. Original Data: Symbol DTE Spot Strike Vol AAPL 30.00 100.00 80.00 14.58 AAPL...

Convert string to ISODate in MongoDB


javascript,python,mongodb,date
I am new to MongoDB and I am stuck on the String to Date conversion. In the db the date item is stored in String type as "date":"2015-06-16T17:50:30.081Z" I want to group the docs by date and calculate the sum of each day so I have to extract year, month...

I'd like to count the documents with the matching “name” property AND group them by “name” at the same time


node.js,mongoose,group-by
Let's say I have a User collection: schema = mongoose.Schema({ firstName: { type: String, required: true }, ... ... }); module.exports = mongoose.model("User", schema); I would like to write a mongoose query that would count how many users go by the name Mike, Andy, Jerry... In other words, I would...

Are the changes done by db command in MongoDB permanent and how?


mongodb,mongodb-query
Are the changes done using the shell in mongodb permanent ? And how MongoDb detects the config file? I am stuck in setting Slow query logs in MongoDb. We have a command db.setProfilingLevel() to set profiling level. But how can we set profiling for all databases and how to set...

What are some patterns I can look at for database implementations in JavaScript?


javascript,node.js,mongodb
I'm fairly new to JavaScript, and I'm busy playing around with a node test app and MongoDB. I'm at a point where I'd like to start with the db side of the app, but I'm not sure what patterns are most commonly used in such a stack, and more importantly,...

Data not visible in mongoDB when inserted from PyMongo


python,mongodb,pymongo
I am trying to insert data in a mongodb collection from python but the data is not being logged. This is how I am doing it: from pymongo import MongoClient import time class data2db: def __init__(self): pass def enter_data(self,data): client = MongoClient('127.0.0.1', 27017) db = client.db coll=db.Temperature1 post = {"auth":...

DB relationship: implementing a conversation


mongodb,meteor
I want to implement a simple conversation feature, where each conversation has a set of messages between two users. My question is, if I have a reference from a message to a conversation, whether I should have a reference the other way as well. Right now, each message has conversationId....

SUM case returns value without GROUP BY


mysql,sql,group-by,sum,case
When I add SUM around my case select, it returns the summed value without the GROUP BY. The query I am using, without the SUM, is the following SELECT CASE WHEN subscription_types.type = 'Succes lidmaatschap' THEN 7 ELSE 8 END FROM subscription_used INNER JOIN training_sessions ON training_sessions.id = subscription_used.training_session_id INNER...

AngularJS factory dependencies


javascript,html,angularjs,mongodb
I'm using AngularJS to extract information stored in mongodb. I'm trying to use a factory to retrieve that information using $http . I read so much information about how to do it, and no one works for me. Also I'm using node + express, the routes works fine. The problem...

Async await usage for MongoDB repository


c#,mongodb,asynchronous,parallel-processing,async-await
I have a MongoDB repository class as you see below: public class MongoDbRepository<TEntity> : IRepository<TEntity> where TEntity : EntityBase { private IMongoClient client; private IMongoDatabase database; private IMongoCollection<TEntity> collection; public MongoDbRepository() { client = new MongoClient(); database = client.GetDatabase("Test"); collection = database.GetCollection<TEntity>(typeof(TEntity).Name); } public async Task Insert(TEntity entity) { if...

Compare Array with Collection-Array containing Objects


node.js,mongodb,mongoose,schema
This is my collection schema: var objectSchema = new Schema({ members: [{ user_id: ObjectId, settings: { type: Boolean } }], title: String }); And now I'm trying to search for objects with specific members (identified by their "user_id", for example ["asdf123lkd", "asdf1223"]). Is there any way to search for these...

Mongoose population in instance methods


node.js,mongodb,mongoose,mongoose-populate
I have a model with reference to other documents. I would like to have a method in that model that can process data used in the referenced models. 'use strict'; var mongoose = require('mongoose') , Schema = mongoose.Schema , deepPopulate = require('mongoose-deep-populate'); var MainSchema = new Schema({ childs: [{type:Schema.ObjectId, ref:...

MongoJS - No Error On Unique Index


mongodb,mongojs
So I'm messing around with MongoDB using MongoJS and can't seem to get an error to be returned when a unique ID exists. So this is the code I am using: server.post('/register', function (req, res, next) { var details = req.params; details.password = md5(details.password).toString(); db.users.insert(details, function (err, test1, test2) {...

mongodb aggregate unwind array and no array


arrays,mongodb,aggregation-framework
I'm trying to write an aggregate query using $unwind no matter the element is an array or not. I know $unwind does not work on no array elements, but I wonder if there is a way to make it work, like converting this element into an array. I have a...

push item in sub document


mongodb,mongodb-query
I have simple collection. I want to push item in question, but it gives an exception.How can I resolve it ? /* 1 */ my collection data { "_id" : ObjectId("557e8c93a6df1a22041e0879"), "QuestionCount" : 2.0000000000000000, "Questions" : [ { "_id" : ObjectId("557e8c9ba6df1a22041e087a"), "DataSource" : [], "DataSourceItemCount" : NumberLong(0) }, { "_id"...

Use $or operator in @Query annotation in spring data mongodb repository


java,spring,mongodb,spring-data-mongodb
I am using spring-data-mongodb. I want to use $or operator in my repository. This is my query: @Query("{'type':?0}") List<Doc> findByType(String type, Pageable pageable); How do use $or in @Query so as it can match either type or name and fetch me a document. Please help....

Incorrect response to mapReduce query in mongo-db


mongodb,mapreduce
I have 1000 user records in collecton, in which 459 document has gender male and remaining as female //document structure > db.user_details.find().pretty() { "_id" : ObjectId("557e610d626754910f0974a4"), "id" : 0, "name" : "Leanne Flinn", "email" : "[email protected]", "work" : "Unilogic", "dob" : "Fri Jun 11 1965 20:50:58 GMT+0530 (IST)", "age" :...

paging subdocument in mongodb subdocument


mongodb,mongodb-query
I want to paging my data in Mongodb. I use slice operator but can not paging my data. I wish to bring my row but can not paging in this row. I want to return only 2 rows of data source. How can resolve it My Query : db.getCollection('forms').find({ "_id":...