FAQ Database Discussion Community


How to carry out this aggregation in python

python,dictionary,data-structures,aggregation
I have the following list of dictionaries which contains the country and the values for corresponding servers. [ {'country': 'KR', 'values': ['Server1']}, {'country': 'IE', 'values': ['Server1', 'Server3', 'Server2']}, {'country': 'IE', 'values': ['Server1', 'Server3']}, {'country': 'DE', 'values': ['Server1']}, {'country': 'DE', 'values': ['Server2']}, ] Now I need to calculate the percentage of...

Elasticsearch aggregation on part of string, not full string

node.js,mongodb,elasticsearch,aggregation
Basically, what I'm trying to do here is get the second-level-down categories from a hierarchically stored string. The problem is that the level of hierarchy vary and one product category could have six levels and another only four, otherwise I would have just implemented predefined levels. I have some products...

aggregate by distance in R

r,aggregation
I have a data set of road traffic data for certain roads, and I have a few columns in my data set. One of them is "Distance from start" (starts from 0 until the end of the road, in unit meters.), and another column is: "Speed". I want to find...

elastic search aggregation on more than one field

elasticsearch,aggregation
I am new to elastic search and I want to implement a specific use case on it. I want to have a multi field sum aggregation. I will try to explain on example: I have following objects inserted into es index: {"a":"aval", "b":"bval", "c":"cval", "aggcount":100} where a,b and are strings...

Calculate the average of fields in MongoDB

mongodb,average,aggregation
I want to calculate the rating_average field of this object with the rating fields inside the array ratings. Can you help me to understand how to use aggregation with $avg? { "title": "The Hobbit", "rating_average": "???", "ratings": [ { "title": "best book ever", "rating": 5 }, { "title": "good book",...

Elasticsearch - Lsit whole object in terms aggregation

elasticsearch,nested,aggregation
I need to make a query that lists the whole objects in terms aggregation. The mapping is like this: { "travelers": { "properties": { "traveler": "string", "cars": { "type":"nested", "properties": { "type": { "type":"string" }, "color": { "type":"string" } } } } } } And the query I can make...

Fastest way to aggregate two typed arrays in javascript [duplicate]

javascript,arrays,aggregation,typed-arrays
This question already has an answer here: How can I merge TypedArrays in JavaScript? 2 answers Hello everyone I'm trying to figure out the fastest way to aggregate two typed arrays into a single array. Imagine data looks like this [Int8Array, Int8Array] and I want to end up with...

UML: packages within packages

package,uml,aggregation,composition
I am working on a UML package diagram, and was wondering: a lot of my packages have a related topic in common, so i put them into a more general package. When drawing the UML diagram, do I literally draw the packages within a larger package? Or, do I use...

user-group aggregation relationship in UML

uml,aggregation
if a user can be part of many groups and a group can contain many users , in UML this relationship has to be represented as an Aggregation relationship? or we can just use a simple association ?

ElasticSearch Aggregations - sum_other_doc_count in a (Term?) Query?

elasticsearch,aggregation
By default, a Terms aggregation gives me the top-10 most-popular terms, and their counts, and then a sum_other_doc_count field representing the "Other" items. I can display these to the user: first (150) second (122) third(111) ... other(19) ...The user can then filter their results by choosing one of the terms....

How to express a condition in a Mongo Pipeline with multiple $and's on the same level

php,mongodb,aggregation,boolean-expression
My problem is that I need to express a condition in a Mongo Aggregation pipeline in the format ((A&B)||(C&D)) which when you write it out in the format that Mongo writes it's boolean expressions in yields {$or: {$and: {A, B}, $and: {C ,D}}. The problem with this is that you...

Elastic Search multi-value field aggregation

elasticsearch,aggregation,multivalue
My indexed documents have a schema: { ... 'authors': [{'first name': 'John', 'last name': 'Smith'}, {'first name': 'Mark', 'last name': 'Spencer'}] ... } I would like to search them and aggregate by the individual authors, so get a list with top authors which occurred in my hits. Terms aggregation seems...

R Geospatial Aggregation: Collapsing Directionality without Using Forloop (Remove Duplicates)

r,combinations,aggregation
I am trying to perform aggregation (basic sum) on geospatial data that contains directionality information. I am trying to collapse the directionality dimension. So if A -> B and B -> A, then count both. I have the following data: u <- data.frame(str_statn=c(42, 106, 3, 6, 4), end_statn=c(106, 42, 6,...

How to do a rolling aggregation of data week wise in python?

python,pandas,aggregation,itertools
I have a dataset whose structure is : Date Profit A sample of the dataset is: Date Profit 2013-06-21 14 2013-06-22 19 2013-06-23 11 2013-06-24 13 2013-06-25 6 2013-06-26 22 2013-06-27 22 2013-06-28 3 2013-06-29 5 2013-06-30 10 2013-07-01 17 2013-07-02 14 2013-07-03 9 2013-07-04 7 Sample input is :...

Get result from mongo aggregation using pymongo 3.0

python,mongodb,mongodb-query,aggregation
I know it may be a very simple question but I cant figure it out, im getting the same error again and again: I amb using python to query a mongo collection and retrive a value from it, here it is: subquery = db.partsupp.aggregate([ {"$match": {"r_name": region }}, {"$group": {...

Elasticsearch which is the better aggregation

elasticsearch,aggregation
I need to run a an aggregation which is to get the unique ids for a day where the ids in the specific set. So the SQL query will be like SELECT count (DISTICT my_field) FROM my_table WHERE time BETWEEN '2015-02-13 00:00:00' AND '2015-02-13 23:59:59' AND my_field IN (value1,value2,value3....value n)...

Entity Framework - How to get Standard Deviation using GroupBy?

.net,entity-framework,aggregation,standard-deviation
I'm using entity framework 5, code first fluent API. I've built a simple query with a GroupBy for aggregation: var query = context.Table1 .GroupBy(g => new { Prop1= g.Prop1, Prop2 = g.Prop2 }) .Select(s => new { Prop1= s.Key.Prop1, Prop2= s.Key.Prop2, Avg = s.Average(m => m.Minutes), Min = s.Min(m =>...

UML - association or aggregation (simple code snippets)

uml,associations,aggregation,class-diagram,diagramming
I drives me crazy how many books contradicts themselves. Class A {} class B {void UseA(A a)} //some say this is an association, no reference is held but communication is possible Class A {} class B {A a;} //some say this is aggregration, a reference is held But many say...

Aggregation with Group By date in Spark SQL

sql,group-by,apache-spark,aggregation
I have an RDD containing a timestamp named time of type long: root |-- id: string (nullable = true) |-- value1: string (nullable = true) |-- value2: string (nullable = true) |-- time: long (nullable = true) |-- type: string (nullable = true) I am trying to group by value1,...

How to populate a referenced document from other collection and query?

node.js,mongodb,mongoose,aggregation-framework,aggregation
Here is the sample collections for the query, I want the result like given below. How can I show the details of device and user's details in issuedDevices collection's document using mongo/nodejs query, please help: 1. collection user /* 0 */ { "_id" : 1, "name" : "Pradeep", "email" :...

How to compare grouped data between two dates using the mongodb aggregate framework

javascript,mongodb,aggregation-framework,aggregation
Consider a data set as follows Sector Value Date a 2 1/1/2015 b 5 1/1/2015 c 8 1/1/2015 a 3 1/1/2015 a 6 1/1/2015 a 1 1/1/2015 c 1 1/1/2015 Sector Value Date a 2 2/1/2015 b 10 2/1/2015 b 5 2/1/2015 c 8 2/1/2015 Is there a way to...

mongodb aggregrate timestamp by 10 minute intervals

mongodb,aggregation
I'm trying to group a large data set that matches a query into 10 minute "time slices". I am wondering if there is any way to do this efficiently with mongodb's aggregation functions. I have a document that looks like: { _id: ObjectID(""), groupID: '1234', name: 'dataPointName', timestamp: 1432765200000, value:...

Elasticsearch cardinality sorted wrong

sorting,elasticsearch,aggregation,cardinality
When I make a request like this curl -XGET "http://localhost:9200/log/travels/_search?pretty" -d ' { "aggs":{ "travelers":{ "terms":{ "field":"traveler", "shard_size":0, "size":5, "order":{ "cities":"desc" } }, "aggs":{ "cities":{ "nested":{ "path":"cities" }, "aggs":{ "city_count":{ "cardinality":{ "field":"cities.name" } } } } } } } }' I get a response that's ordered wrong, like this "aggregations"...

Django admin aggregation

python,django,admin,aggregation
I'm using Django to manage deposit,but I found a problem, there is a medel looks like: class Deposit(models.Model): user = models.ForeignKey(User) money_re = models.DecimalField(max_digits=11, decimal_places=2) ... I want to add a sum of money_re in the model to the admin change_list page in the right-bottom of the page. Is there...

Get matching elements in mongodb array

php,mongodb,aggregation,pipeline
I want to use aggregation to get this array only with those tickets, which have start field after 2015-06-16. Can someone help me with the pipeline? { "name" : "array", "tickets" : [ { "id" : 1, "sort" : true, "start" : ISODate("2015-06-15T22:00:00.000Z") }, { "id" : 2, "sort" :...

MongoDB Aggregate Time Series

mongodb,mongodb-query,aggregation
I'm using MongoDB to store time series data using a similar structure to "The Document-Oriented Design" explained here: http://blog.mongodb.org/post/65517193370/schema-design-for-time-series-data-in-mongodb The objective is to query for the top 10 busiest minutes of the day on the whole system. Each document stores 1 hour of data using 60 sub-documents (1 for each...

ElasticSearch - Aggregations with document details

elasticsearch,aggregation
I need to aggregate the following documents: { "title": "American Psycho", "releaseDate": "7/06/2000", "imdbRate": "7.6", "casting": [ { "name": "Christian Bale", "category": "Actor" }, { "name": "Justin Theroux", "category": "Actor" } ] } { "title": "The Dark Knight", "releaseDate": "13/08/2008", "imdbRate": "9.0", "casting": [ { "name": "Christian Bale", "category": "Actor"...

Sorting after aggregation in Elasticsearch

sorting,elasticsearch,aggregation
I have docs with this structure: { FIELD1:string, FIELD2: [ {SUBFIELD:number}, {SUBFIELD:number}...] } I want to sort on the result of the sum of numbers in FIELD2.SUBFIELDs: GET myindex/_search { "size":0, "aggs": { "a1": { "terms": { "field": "FIELD1", "size":0 }, "aggs":{ "a2":{ "sum":{ "field":"FIELD2.SUBFIELD" } } } } }...

XQuery - Doing math on elements within a sequence and aggregating results

function,operators,xquery,sequence,aggregation
I'm trying to execute a XQuery sum function in a multiplication of two XML elements, but it has been difficult to avoid the iteration in a sequence of elements. For example, consider this case: sample data: <Orders> <Order> <OrderKey>1</OrderKey> <LineItem> <LineNumber>1</LineNumber> <Quantity>41</Quantity> <ExtendedPrice>70848.0000</ExtendedPrice> <Discount>0.0913</Discount> <Tax>0.0663</Tax> <ReturnFlag>A</ReturnFlag>...

event type to convert as integer and insert to rethinkdb

jquery,aggregation,rethinkdb
this is my message which is coming for every event. How can i collect only one type data (which is string) and give it a value of 1, and also at the same time insert to rethinkdb as a value 1, and also update it's total value sample data ----...

MongoDB how do I use $set and $aggregate to set field of nested documents? [duplicate]

mongodb,aggregation
This question already has an answer here: Update Elements in Array 1 answer I want to write an expression that allows me to set the field checked of my items nested within an other document. How can I achieve this in MongoDB? { "_id" : ObjectId("55081de2162072120758fc53")[ { "_id": "55081de2162072120758fc53",...

MongoDB Aggregation $group needs an array for _id but $out won't handle it

mongodb,group,aggregation
I need to sum the number of occurrences of an array. I need to output this to a collection but when I try and use the $out keyword, it fails with "can't use an array for _id\" Is there any way to project the value of the _id field from...

Aggregate example sets in Rapidminer based on a specific attribute

union,aggregation,rapidminer
I am a bit stuck with a simple task of aggregating two example sets using rapidminer and need an advise on a process and possible operators. I have two CSVs like: key A B C D E first 2 2 2 2 2 second 3 3 3 3 3 and...

How Camel 2.11 batch aggregation works with separate route?

java,apache-camel,aggregation
First there is a similar unanswered question Joining routes into single aggregator We have some consumer routes (ftp, file, smb) reading files from remote systems. Simplified for test with direct route, but similar behavior with batch consumers: from("direct:"+routeId).id(routeId) .setProperty(AGGREGATION_PROPERTY, constant(routeId)) .log(String.format("Sending (${body}) to %s", "direct:start1")) .to("direct:aggregate"); After transformation all results...

ElasticSearch aggregation - get the exact time of a max histogram value in a timeseries

java,elasticsearch,max,histogram,aggregation
I am quite new to elasticsearch, so apologies if this is a trivial question. I have a time series with irregular updates every n number of seconds that I want to plot historically. The data contains a long variable called 'score', and a long variable with the epoch called 'time'...

Aggregations Size makes diffrent results

elasticsearch,aggregation,elastic
Hi I have simples aggregation like "aggs": { "firm_aggregation": { "terms": { "field": "experience.company_name.slug", "size": 10 } } } and this gives me result like "aggregations": { "firm_aggregation": { "buckets": [ ... (some others) { "key": "freelancer", "doc_count": 33 }, but when I increase aggregation size to 2000 i get...

Group By to return multiple different grouped results SQL Server

sql-server,group-by,aggregation
I don't think this is going to be too rough, though I am not seeing it in the asked questions and answered questions. I am looking for a way to group by sales person for two different types of sales. So something like Name PendingORders CompletedOrders TotalOrders Grouped by the...

How to get mongoDB aggregated month wise data using java [closed]

java,mongodb,grouping,aggregation
Here is my data in db. "Accounts" : [ { "Total_Credits" : 4000, "Total_Debits" : 0, "Date" : "25-05-2015" }, { "Total_Credits" : 1000, "Total_Debits" : 0, "Date" : "26-05-2015" }, { "Total_Credits" : 1000, "Total_Debits" : 0, "Date" : "10-07-2015" }] I want to extract sum of total credits...

iOS : How to fetch objects with maximum count in core data?

ios,core-data,nspredicate,aggregation,nsfetchrequest
I've two classes in my core data model, and its one to many relationship. Lets take Class Album <--->> Class Songs for the explanation. Now I want to fetch (top) five albums from Album which have the maximum no.of Songs. Class Album { albumID (Integer) name (String) songs (class B)...

SQL Conditional aggregation

sql,oracle,conditional,aggregation
I'm using Oracle. Having a table as: Year Type Value 2011 1 500 2011 2 550 2011 3 600 ... ... 2012 1 600 2012 2 750 2012 3 930 I needed to subtract all the values from different types, grouped by year. The operation would be: For 2011 ->...

Mongodb: aggregate array of integers for each array position without map/reduce

arrays,mongodb,aggregation
I am new to MongoDB. I am wondering if it is possible to aggregate each "column" in array. Assume we have below documents: db.test.insert([{"player": "A", "year": 2010, "value": [1, 1 ]}, {"player": "A", "year": 2011, "value": [2, 1 ]}, {"player": "A", "year": 2012, "value": [2, 1 ]}, {"player": "B", "year":...

using pandas to create new columns based on intra-group rank-order

python,algorithm,numpy,pandas,aggregation
I have a pandas DataFrame with a large number of rows. I am trying to create new columns for the frame based on the intra-group rank-order of the members. Here is some fake data illustrating what I have: Num_members = int(1.e7) Num_groups = int(1.e5) members = pd.DataFrame({ 'ID': np.arange(Num_members), 'groupID':...

Assigning a template-generated class to a C struct with the same layout

c++,c,data-structures,aggregation
If I understand correctly, the object ’A’ defined thus: typedef struct { int n; float *p; } myStruct; myStruct A; is an aggregate with exactly the same layout in memory as the object ‘B’ defined as: template <typename T> class myTemplateClass { public: int n; T* p; }; myTemplateClass<float> B;...

Mongodb Group by count Based on Array fields

mongodb,group,aggregation
I am having following data . I want to group by fields in array . { "_id" : ObjectId("54e73137758646882f445383"), "items" : [ { "name" : "one", "quality" : "high" }, { "name" : "one", "quality" : "low" }, { "name" : "two", "quality" : "high" } ] } /* 1...

Is there a way to use @Query to return a MongoDB $sum result?

spring,mongodb,annotations,repository,aggregation
I'm pretty new to both MongoDB and StackOverflow (as a questioner). I've seen a great response for how to return a MongoDB "count" projection from a Spring repository using the @Query annotation (Count in Spring data mongodb repository). Basically the count example looks like: @Query(value = "{'productDetails.productType': {$regex: ?0, $options:...

Grouping over multiple fields in MongoDb

mongodb,mapreduce,grouping,aggregation
How would I go about grouping over multiple fields? I need to get a unique count for case insensitive true over multiple independent documents. I've looked at both map/reduce and aggregation and I don't quite know what would be the best approach. Lets say I have the following data in...