sorting,elasticsearch,aggregation , Sorting after aggregation in Elasticsearch


Sorting after aggregation in Elasticsearch

Question:

Tag: sorting,elasticsearch,aggregation

I have docs with this structure:

{
    FIELD1:string,
    FIELD2:
        [ {SUBFIELD:number}, {SUBFIELD:number}...]
}

I want to sort on the result of the sum of numbers in FIELD2.SUBFIELDs:

GET myindex/_search
{
  "size":0,
  "aggs": {
    "a1": {
      "terms": { 
        "field": "FIELD1",
        "size":0
      },
      "aggs":{
        "a2":{
          "sum":{
            "field":"FIELD2.SUBFIELD"
          }
        }
      }
    }
  }
}

If I do this I obtain buckets not sorted, but I want buckets sorted by "a2" value. How I can do this? Thank you!


Answer:

You almost had it. You just need to add an order property to your a1 terms aggregations, like this:

GET myindex/_search
{
  "size":0,
  "aggs": {
    "a1": {
      "terms": { 
        "field": "FIELD1",
        "size":0,
        "order": {"a2": "desc"}      <--- add this
      },
      "aggs":{
        "a2":{
          "sum":{
            "field":"FIELD2.SUBFIELD"
          }
        }
      }
    }
  }
}

Related:


Sort array by keys in custom order


php,arrays,sorting
I have the following multidimensional array Array ( [June 2015] => Array ( [LOW] => Array ( [0] => 160.50 ) [MEDIUM] => Array ( [0] => 0.00 ) [HIGH] => Array ( [0] => 60.80 ) ) [July 2015] => Array ( [MEDIUM] => Array ( [0] => 226.00...

ElasticSearch asynchronous post


database,post,asynchronous,elasticsearch,get
I'm posting data on my ElasticSearch database. I've noticed that data is not immediately available, it requires some milliseconds to show up in a GET request. I can live with that (after all, the calls are asynchronous so this behavior is expected) but in my test code I need to...

Re-index object with new fields


elasticsearch,nest
It seems like as long as the id field is maintained, its super easy to re-index a document by simply calling Index(), but is there a way to given an object was updated and new fields were added, to have it include these new fields in the index? I'm not...

How to compute the scores based on field data in elasticsearch


elasticsearch
I have the following fields in documents { name: "Pearl", age : 43, weight: 54, bodyWeight : 103, height : 1.8 } Now i want to get scores for the documents based on the bodyWeight to height ratio of the documents. How to do that?...

Elasticsearch geospatial search, problems with index setup


elasticsearch,geospatial
I'm trying to search for documents previously added to an index, which has been configured to allow geospatial queries (or so I think). My elasticsearch instance is hosted on qbox.io. This is the code I wrote to create an index from the command line curl -XPOST username:[email protected]/events -d '{ "settings"...

Elasticsearch standard analyser stopwords


elasticsearch
I am trying to guess what is the default stopwords list in standard analyzer in elasticsearch. I run version 1.3.1, and it seems to me that the English list is used, because running a wildcard query like this { "wildcard" : { "name" : { "wildcard" : "*in*" } }...

How to read data in logs using logstash?


elasticsearch,logstash
I have just started log stash, i have log files in that log file whole object is printed in the logs, Since my object is huge i cant write the grok patterns to the whole object and also i expecting only two values out of those object. Can you please...

Sorting vector of Pointers of Custom Class


c++,sorting,c++11,vector
I have vector<FPGA*> current_generation_, which I'd like to sort by FPGA member fitness_ using the sort_members function. Applicable code follows: bool sort_members (FPGA* fpga_first, FPGA* fpga_second) { return (fpga_first->fitness() < fpga_second->fitness()); }; fpga.hpp #include <vector> class FPGA { public: explicit FPGA(int input_gates, int output_gates, int normal_gates); const int fitness(); protected:...

ElasticSearch - Configuration to Analyse a document on Indexing


elasticsearch
In a single request, I want to retrieve documents from a SOR, store them in ElasticSearch, and then search those documents using the ES search API. There seems to be some lag from the time the document is indexed and the time it is analyzed and ready to be searched....

How to write search queries in kibana using Query DSL for Elasticsearch aggregation


elasticsearch,querydsl,kibana-4
I am working on ELK stack to process Apache access logs. Spent a lot of time understanding Query DSL format so that more complex queries can be written. Currently am facing issues with running the queries in kibana interface but the same queries work just fine when posted using curl...

Formatting a Pivot Table in Python


python,sorting,pandas,format,dataframes
I am trying to reformat a table based on counts in different columns. df = pd.DataFrame({'Number': [1, 2, 3, 4, 5], 'X' : ['X1', 'X2', 'X3', 'X3', 'X3'], 'Y' : ['Y2','Y1','Y1','Y1', 'Y2'], 'Z' : ['Z3','Z1','Z1','Z2','Z1']}) Number X Y Z 0 1 X1 Y2 Z3 1 2 X2 Y1 Z1 2...

Given an array/object of datetimes, how can I return an array/object of the times sorted by hour and times sorted by days of the week


php,arrays,sorting,datetime,laravel
PHP/Laravel, I'm getting an array of objects that includes date times for each record. I need to generate analytics on objects on an hourly basis of a day and daily basis of a week. So for example: For a date range of 1/1/2015 - 1/10/2015 I return 100 records all...

EXC_BAD_ACCESS error occurring when running recursive merge sort method in a thread


c++,multithreading,sorting,recursion
I'm having trouble with my C++ code in xCode. I've tried debugging and I can't fix it. I know it's roughly when my mergeSort() method calls my mergeNumbers() method. I know it's not the method itself because I've run the method without threading and it works just fine. It's when...

elastic search sort in aggs by column


sorting,elasticsearch,group-by,order
I am trying to sort in elastic search in aggs, equivalent in mysql "ORDER BY Title ASC/DESC". Here is the index structure: 'body' => array( 'mappings' => array( 'test_type' => array( '_source' => array( 'enabled' => true ), 'properties' => array( 'ProductId' => array( 'type' => 'integer', 'index' => 'not_analyzed'...

Sort multiple columns of Excel in VBA given the top-left and lowest-right cell


excel,vba,excel-vba,sorting
I am trying to sort these three columns (Sort By Col-2) in excel using VBA. Top-left (Row number and Column number e.g. 1,1) and lowest-right cell (Row number and Column number e.g. 9,3) are known. Every cell contains the values of String type. Input: Col-1 Col-2 Col-3 P1 I1 XYZ...

NEST ElasticSearch.NET Escape Special Characters


c#,elasticsearch,nest
I have been experimenting with the use of the NEST client for Elastic Search, but seem to have hit a barrier when filtering on a term which contains special/reserved characters such as '/' Below is a JSON representation of my model.. "categories": { "count": 1, "default": "root/Hello/World/Category", } When submitting...

Sum NA values in r


r,sorting,na
I am using a dataframe that has multiple NA values so I was thinking about sorting the attributes based on their NA values. I was trying to use a for loop and this is what I have so far: > data <- read.csv("C:/Users/Nikita/Desktop/first1k.csv") > for (i in 1:length(data) ) {...

Sorting in Ruby on rails


ruby-on-rails,json,sorting
I'm very new to Ruby on rails. I try to edit the following api. I want to sorting "can_go" which is true are shown at the top of list. I added this row before sending data, but the result is still order by "user_id". user_infos.sort { |a, b| - (a['can_go']<=>b['can_go'])...

Javascript sort array of objects in reverse chronological order


javascript,arrays,sorting
I have an array of objects which holds a list of jobs and I would like to sort them in reverse chronological order as they would appear on a resume for example. I came up with the below solution which 'seems' to work but I was wondering if there is...

MultiMatch query with Nest and Field Suffix


c#,elasticsearch,nest
Using Elasticsearch I have a field with a suffix - string field with a .english suffix with an english analyser on it as shown in the following mapping ... "valueString": { "type": "string", "fields": { "english": { "type": "string", "analyzer": "english" } } } ... The following query snippet won't...

Ranking with time weighting


python,algorithm,sorting,math
I am looking for a basic algorithm that gives more weigh to the recent reviews. So, the output value of the algorithm is mutable. For example, two reviews with exactly the same score, will have a different ranking based on the timestamp of the creation. Review_1 Score 10 creation 10/5/2014...

How do I remove duplicate objects in an array by field in Javascript?


javascript,arrays,sorting,object
I have an array of objects: [{ id: 1, name: 'kitten' }, { id: 2, name: 'kitten' },{ id: 3, name: 'cat }] How do I remove the second kitten? Sorting into an array of names doesn't work, because I can't know if I am deleting id 1 or or...

Merge multiple lists based on List priority


java,sorting,collections,priority
I have couple of lists where each list has different priority. When i merge these lists i should get with height priority on top. Assume i have 3 lists and different objects value common in 3 lists should come first and then the top priority list should be appended vice...

Find column with unique values move to the first column and sort worksheets


excel,vba,excel-vba,sorting
I have 2 worksheets with the same headers in different orders. Headers are I.D, Name, Department, Sales, Start date, End Date and a few others. What I am aiming to do is search through the workbooks in which the headers may be in different orders, find the column which has...

NEST - Using GET instead of POST/PUT for searching


c#,elasticsearch,nest
Is there a way to tell NEST to use GET instead of POST when performing searches? Similar to how the ElasticSearch documentation shows CURL using GET I'd like to use GET when using NEST instead of using POST as it currently does.

How to use arrays in lambda expressions?


c#,elasticsearch,nest
I am writing a program with NEST library of ElasticSearch. I want to write a lambda expression for a function with this argument: HighlighDescriptor<parentdocument> HighlighDescriptor.onFields (param Action<HighlightFieldDescriptor<ParentDocument>>[] fieldHighlighters) I don't know what is the array in the function argument?...

Sort function giving floating point exception for a large input of 0's


c++,sorting,radix-sort,floating-point-exceptions
I have written a code for this problem: Given a list of non negative integers, arrange them such that they form the largest number. For example, given [3, 30, 34, 5, 9], the largest formed number is 9534330. Note: The result may be very large, so you need to return...

C# sorting arrays in ascending and descending order


c#,arrays,sorting
I'm having trouble writing a method that returns true if the elements of an array (numbers) are in sorted order, ascending or descending, and false, if they are not in any sorted order. I can return a correct boolean value if the array is ascending but i do not know...

ElasticSearch REST - insert JSON string without using class


elasticsearch,elastic,elasticsearch-net
I am looking for an example where we can push below sample JSON string to ElasticSearch without using classes in REST api. { "UserID":1, "Username": "Test", "EmailID": "[email protected]" } We get the input as xml and we convert it to JSON string using NewtonSoft.JSON dll. I know REST api is...

Sorting a HTML structure


javascript,html,sorting
Im trying to sort a div structure based on a paramter using a small javscript i found. It seems to not perform exactly as expected. I understand the sorting function is not parsing the values perfectly... This is the sorting logic is use... <script type="text/javascript"> // Sorting Logic $(function() {...

Sort arrayList based on both date and ID


java,sorting
i want to sort the list of task,first by date and then by taskID below is the code ArrayList<fullist> taskdet = new ArrayList<fullist>(); public static class fullist { public int date; public int id; public fullist(int id, int date) { this.date = date; this.id = id; } } i used...

Sort List of Numbers according to Custom Number Sequence


list,python-2.7,sorting
Question :A set of numbers will be passed as input. Also the redefined relationship of the digits 0-9 in ascending order will be passed as input. Based on the redefined relationship, the set of numbers must be listed in ascending order. Input Format: The first line will contain the the...

Bad scoring due to different maxDocs of IDF


elasticsearch
I have two documents with a field title of: News New Website If I search for the term new website the score for the News document is much higher than the other one which is obviously not what I want. I wrapped an explain around it and got: 'hits': [{'_explanation':...

Freeing array of dynamic strings / lines in C


c,string,sorting,malloc,free
I am writing a program that is sorting the lines from the input text file. It does its job, however I get memory leaks using valgrind. #include <stdio.h> #include <stdlib.h> #include <string.h> char* getline(FILE * infile) { int size = 1024; char * line = (char*)malloc(size); int temp; int i=0;...

sorting vector of pointers by two parameters


c++,sorting,vector
I am trying to sort a vector of pointers to a class A using std::sort, but am having some difficulties. Imagine class A beings just a point, containing x and y coordinates. I want to sort the vector by y coordinates from biggest to lowest using some fixed offset value....

Docker container http requests limit


http,elasticsearch,docker
I'm new to Docker so, most likely, I'm missing something. I'm running a container with Elasticsearch, using this image. I'm able to setup everyhing correctly. After that I was a using a script developed by a collegue in order to insert some data, basically querying a MySQL database and making...

Associative array not storing first result from SQL


php,mysql,arrays,sorting
I have a database in MySQL and I'm using this query to select certain rows from it using PHP: $q = "SELECT Number, Body FROM boxes WHERE Number BETWEEN '1' AND '4' ORDER BY Number ASC"; Then calling the query and initiating arrays: $r = $mysqli->query($q); $row = mysqli_fetch_array($r, MYSQLI_ASSOC);...

Elasticsearch - Query document missing an array value


elasticsearch
I would like to query my elasticsearch index in order to retrieve the documents that don't contain a specific value in an array. For instance, if my query is : { "query": { "bool": { "must": [ { "match_all": {} } ], "must_not": [], "should": [] } }, "from": 0,...

Should checking loop conditions be counted towards total number of comparisons?


c++,algorithm,sorting,c++11
I have implemented three different sorting algorithms and now I want to confirm that my approach of counting the total number of comparisons is correct. In my mind, the number of comparisons shouldn't be tied to the conditional branches because if the condition isn't met, the comparison was still made...

Not able to access Kibana running in a Docker container on port 5601


elasticsearch,docker,dockerfile,kibana-4
I have built a docker image with the following Docker file. # gunicorn-flask FROM devdb/kibana MAINTAINER John Doe <[email protected]> ENV DEBIAN_FRONTEND noninteractive RUN apt-get update RUN apt-get install -y python python-pip python-virtualenv gunicorn # Setup flask application RUN mkdir -p /deploy/app COPY gunicorn_config.py /deploy/gunicorn_config.py COPY app /deploy/app RUN pip install...

Sorting jQuery dataTables by class name when there is no type or value


javascript,jquery,sorting,jquery-datatables
I'm using DataTables: https://www.datatables.net/ Heres my HTML: <div class="dataTable_wrapper"> <table class="table table-striped table-bordered table-hover center" id="dataTables-example"> <thead> <tr> <th>ID</th> <th>Title</th> <th>Actions</th> </tr> </thead> <tbody> <tr> <td>1</td> <td>title 1</td> <td><a href="./action-1"><span class="on"></span></a></td> </tr> <tr>...

WPF Listbox Collection custom sort


wpf,sorting,listbox,compare,collectionview
I have a listbox DropPrice MyPrice Price1 Price2 I want to sort it like this Price1 Price2 DropPrice MyPrice I mean, if there's an item that starts with the sequence "price", it gets priority, else the smallest string should get the priority. My source code: var lcv = (ListCollectionView)(CollectionViewSource.GetDefaultView(_itemsSource)); var...

Javascript Sorting Array of Objects [duplicate]


javascript,arrays,sorting,object
This question already has an answer here: Sorting an array of JavaScript objects 14 answers (Please excuse any errors - this is my first post and I am also relatively new to Javascript) I'm trying to sort an array of objects by a specific property value in Javascript. I...

Algoritm to sort object by attribute value without allowing gaps or duplicates


python,sql,django,sorting,django-orm
I have a agenda with multiple dates, each date can contain 0 > ... items. Items can be sorted by position, positions should be Integer values without gaps and duplicates. class Item(models.Model): date = models.DateField() position = models.IntegerField() def move_to(position): qs = self.__class__.objects.filter(date=self.date) # if the position is taken, move...

Parsing Google Custom Search API for Elasticsearch Documents


json,python-2.7,elasticsearch,google-search-api
After retrieving results from the Google Custom Search API and writing it to JSON, I want to parse that JSON to make valid Elasticsearch documents. You can configure a parent - child relationship for nested results. However, this relationship seems to not be inferred by the data structure itself. I've...

Get document on some condition in elastic search java API


java,elasticsearch,elasticsearch-plugin
As I know we can parse document in elastic search, And when we search for a keyword, It will return the document using this code of java API:- org.elasticsearch.action.search.SearchResponse searchHits = node.client() .prepareSearch() .setIndices("indices") .setQuery(qb) .setFrom(0).setSize(1000) .addHighlightedField("file.filename") .addHighlightedField("content") .addHighlightedField("meta.title") .setHighlighterPreTags("<span class='badge badge-info'>") .setHighlighterPostTags("</span>") .addFields("*", "_source")...

Elasticsearch NumberFormatException when running two consecutive java tests


java,date,elasticsearch,numberformatexception,spring-data-elasticsearch
I have two test in a class, each of them containing the following query: SearchQuery searchQuery = new NativeSearchQueryBuilder().withQuery(matchAllQuery()).withFilter(rangeFilter("publishDate").lt(date)).build(); In one of the tests, the number of the results elasticsearchTemplate.count(searchQuery, Article.class), in the other one the returned values are verified elasticsearchTemplate.queryForPage(searchQuery,Article.class) If I run any of these two tests separately,...

Operator '??' cannot be applied to operands of type IQueryContainer and lambda expression


c#,elasticsearch,nest
I am trying to create a method to process a certain query. I follow an example posted on the Nest repository (line 60), but still the MatchAll is not recognized by the compiler and if I try to build the solution, the error that shows is: Operator '??' cannot be...

Sort when values are None or empty strings python


python,list,sorting,null
I have a list with dictionaries in which I sort them on different values. I'm doing it with these lines of code: def orderBy(self, col, dir, objlist): if dir == 'asc': sorted_objects = sorted(objlist, key=lambda k: k[col]) else: sorted_objects = sorted(objlist, key=lambda k: k[col], reverse=True) return sorted_objects Now the problem...