FAQ Database Discussion Community


Modify an Array Function Parameter Bash

arrays,bash,parameters,jq
I am trying modify an array that I have passed as a parameter to a function. So far, I have an empty array outside of the function: buckets=() Then I have the function which takes in 2 arguments. The first argument is the empty array that I want to fill....

Using jq to extract certain info from JSON file

json,linux,bash,jq
I am extracting some info from my json files that are formatted like this: { "name": "value", "website": "https://google.com", "type" : "money", "some": "0", "something_else": "0", "something_new": "0", "test": [ {"Web" : "target1.com", "type" : "2" }, {"Web" : "target2.com", "type" : "3" }, {"Web" : "target3.com", "type" : "3"...

How to convert flat array of objects with “categories” and “subcategories” to tree with JQ?

json,category,data-manipulation,jq
I have a JSON array of objects with groups and subgroups that looks like this: [ { "Group name": "Elevation", "Subgroup": "Contours", "name": "Contours - Labels" }, { "Group name": "Elevation", "Subgroup": "Contours", "name": "Contours" }, { "Group name": "Elevation", "Subgroup": "Cuttings", "name": "Cuttings" }, { "Group name": "Framework", "Subgroup":...

Pretty print of nested stringified json in logs

json,bash,jq
I'm tailing some logs, and to be able to read them easier, I use jq (http://stedolan.github.io/jq/), but either it's missing something, or I don't know how to do what I need to do. So all lines are json, and currently I'm doing: tail -f /path/to/log | jq . Issue is,...

Passing JSON arguments (not simple strings) to jq

bash,jq
How do I pass in bash variables into a jq --arg parameter? All I can get to work is this: FINAL_JSON= #some JSON PAYLOAD=$(echo $FINAL_JSON | jq ' { "attachments": . } ') What's bothering me is that echo $FINAL_JSON thing. I should be able to pass in variables in...

Multiple filters using JQ

json,filter,jq
My boss wants our team to use JQ to parse JSON files. An app I use produces JSON that I need to transform. I have a file that looks like this: { "Collections": [ { "OptionGroups": [ { "OptionGroupName": "Test1", "Status": "in-sync" } ], "Version": "3.4.25", "InstanceIdentifier": "Dev1" }, {...

Pretty print xml encased in JSON

xml,jq
Log messages from an application conform to LogStash expectations (JSON), but we haven't got LogStash up and running in every case, so I've been trying to learn to use jq to read output when LogStash isn't available. The problem becomes tricky when the output is XML within the JSON. I...

JSON JQ if without else

json,jq
I use the following JQ command to filter out the JSON. My requirement is to filter out the JSON message if the expected node is present. Or else, do nothing. Hence, I use if, elif, .... sed -n "s/.*Service - //p" $1/response.log* | jq "if (.requests | length) != 0...

JQ: Enumerate Object Stream

object,stream,enumerate,jq
I've got a pipe full of objects and I am trying to add a accumulating count string to each object in a stream with jq to get the following output {"count":"Num_000"} {"count":"Num_001"} {"count":"Num_002"} {"count":"Num_003"} {"count":"Num_004"} {"count":"Num_005"} {"count":"Num_006"} {"count":"Num_007"} {"count":"Num_008"} {"count":"Num_009"} Something like the following but I'm sure I don't need...

Merging multiple json arrays in the same file

bash,jq
I'm currently downloading a ton of jira issues to generate a report. Currently the 'full data' file has a ton of individual records like this: { "key": "645", "type": "Bug", "typeid": "1", "status": "Closed", "summary": "Crash when saving document", "closedDate": "2014-10-03T09:01:23.000+0200", "flag": null, "fixVersionID": "123", "fixVersionName": "2.7" } However, because...

Error message when trying to flatten JSON file to CSV

json,unix,csv,flatten,jq
Trying to flatten a JSON file to CSV in UNIX but getting an error message. Any help here to get the contents copied to CSV will be appreciated: Command: cat Filename.json | jq --raw-output '._data.data[] | [.Date, .Object, .ID, .Name ] | @csv' > /home/Filename.csv Error: jq: error: Cannot iterate...

jq in shellscript: parse argument with '-'

json,linux,bash,shell,jq
Here's the json I need to parse: {"protocol-config":[ {"protocol":"p1","config-var1":"vc1", "config-var2":"vc2"}, {"protocol":"p2","config-var1":"vc3", "config-var2":"vc4"} ]} And my code to parse is like the following: declare -a vals=("var1" "var2") for val in "${vals[@]}"; then eval "result=($(cat $INPUT | jq -r --arg i "$i" --arg var "config-$val" '.protocol-config[$i | tonumber][$var]'))" done The problem is...

filtering data using parameters

json,bash,jq
I have this command that is working.. cat ~/Desktop/results.json | jq '.[] | .environmentStatuses[].deploymentResult | select(.key.entityKey.key=="39583746-39747586") | .lifeCycleState ' I want to pass the entity key as variable , tried the below ones ,but none seems to work- enkey="39583746-39747586" cat ~/Desktop/results.json | jq '.[] | .environmentStatuses[].deploymentResult | select(.key.entityKey.key=="""${enkey}""") | .lifeCycleState...

JQ Aggregations and Crosstabs

jq
I have some data thus: [ { "count": 4, "trial_status": "Follow up", "date": "2015-06-06" }, { "count": 3, "trial_status": "Hold", "date": "2015-06-06" }, { "count": 2, "trial_status": "Trial", "date": "2015-06-06" }, { "count": 1, "trial_status": "Trial + Confirm", "date": "2015-06-06" }.... which I would like to transform to {"2015-06-06": {"Trial...

jq - apply a filter on multiple level

jq
I have a json file like this { "log": { "version": { "$t": "1.1" }, "creator": { "name": { "$t": "Internet Explorer Network Inspector" }, "version": { "$t": "10.0.9200.17229" } } } And I need to flatten the file so it should read { "log": { "version": "1.1", "creator": {...

How to add a header to CSV export in jq?

json,csv,jq
I'm taking a modified command from the jq tutorial: curl 'https://api.github.com/repos/stedolan/jq/commits?per_page=5' \ | jq -r -c '.[] | {message: .commit.message, name: .commit.committer.name} | [.[]] | @csv' Which does csv export well, but missing the headers as the top: "Fix README","Nicolas Williams" "README: send questions to SO and Freenode","Nicolas Williams" "usage()...

Extracting matched child json from a nested JSON object with jq

json,jq
Given a JSON array of objects thus: { "header" : { "user" : "baskar" }, "requests" : [ { "first_name" : "mike", "last_name" : "mat" }, { "first_name" : "mike1", "last_name" : "mat" } ], "check" : [ "Y" ] } I'd like to extract a subset of the key/values...

Split a JSON file into separate files

json,jq
I have a large JSON file that is an object of objects, which I would like to split into separate files name after object keys. Is it possible to achieve this using jq or any other off-the-shelf tools? The original JSON is in the following format { "item1": {...}, "item2":...

How to extract keys from multiple, nested arrays using jq

json,amazon-web-services,jq
SETUP I'm trying to figure out how jq filters work and having trouble figuring out nested arrays. Using the data below I cannot make a flat 5 key output. I can get 1 key and 4 nulls or 4 keys and 1 null, but not all 5 keys. 1 key,...

jQ: transformin json to tsv on Windows

json,transformation,tsv,jq
I am using jQ on a windows machine and I have the following data: {"a":"Person","birthDay":"12","deathDay":"15", ...} {"a":"Person","birthDay":"13","deathDay":"16", ...} ... And I would like to have a tab separated file format: Person 12 15 Person 13 16 I can get the values with ".a" or ".birhtDay". But how can I get...

Processing JSON of arrays in array with jq

json,jq
Environment: JQ 1.5, Windows 64-bit. I have the following JSON: { "unique": 1924, "coordinates": [ { "time": "2015-01-25T00:00:01.683", "xyz": [ { "z": 4, "y": 2, "x": 1, "id": 99 }, { "z": 9, "y": 9, "x": 8, "id": 100 }, { "z": 9, "y": 6, "x": 10, "id": 101 }...

How can I extract multiple items and keep them as one json structure

jq
I have an example JSON of { "reason": "TCP FINs", "bytes": "1759", "duration": "0:00:00", "dest": { "interface": "inside", "ip": "192.168.2.2", "port": "88" }, "source": { "interface": "outside", "ip": "10.1.5.1", "port": "60470" }, "connection-id": "400508768", "proto": "TCP", "hostname": "192.168.1.1", "timestamp": "Apr 28 13:00:01", "event.tags": [ "cisco", "ASA-6-302014" ] } and I...

Using jq to extract specific property values and output on a single line

json,jq
I've this ElasticSearch snapshot output and would like to reduce it to print, on a single line, from each snapshot the values of the end_time_in_millis and snapshot property separate by a space: 1429609790767 snapshot_1 1429681169896 snapshot_2 Basically the output of cat data | jq '.snapshots[].end_time_in_millis' and cat data | jq...

JQ JSON select with dynamic shell param

json,shell,jq
I use JQ for JSON formatting and filtering the data. sed "s/.*Service - //p" tes.log | jq 'if (.requests | length) != 0 then .requests |= map(select(.orderId == "2260")) else "NO" end' > ~/result.log Here, the orderid is been hardcoded to 2260. But my requirement is to make it parameter...

How can I filter an array of json objects using jq?

json,jq
I have been able to get this far. json input, 'data.json': [ { "Selected": null, "Family Name": "Jones", "Couple Name": "Jones, Adam & Rachael Margaret", "Family Phone": "404-4477", "Family Email": "[email protected]", "Family Address": "777 Aggies Court Kindly, California 95388 ", "Head Of House Name": "Jones, Adam", "Head Of House Phone":...

How to print outer JSON attribute on each line with the inner ones?

json,jq
I have JSON like this: { "files": [ { "size": 3227238, "name": "file1.mp3" }, { "size": 3737995, "name": "file2.mp3" } ], "id": "music" } And I want to print each file on its own line, but including the id from the outer element. file1.mp3 music file2.mp3 music I can get...

Store value to variable in Shell script [duplicate]

bash,shell,jq
This question already has an answer here: How to set a BASH variable equal to the output from a command? 8 answers "$emails" has the array of values, i want to parse the values from it, To do it, i am using the jq. if i do below command...

How to select values of keys that are provided as variables in jq?

json,variables,dictionary,key,jq
If this was the input, { "a_key": 2, "another_key": 100, "one_more_key": -4.2 } what would be the best way to select the value of any of these keys by providing the name of the key as a variable? Ideally, I was looking for something like: "a_key" as $key | .$key...

extracting json using jq

json,jq
I have sample Json file like this.How to extract data from it? I have tried few things.But could not get the desired result { "adults":{ "car":{"length":[20,25],"width":[225,40]}, "tractor":{"length":[20,23],"width":[223,40]} }, "children":{ "cycle":{"length":[20,21],"width":[221,40]}, } } I want to insert this data to the database with the following columns: age_group | vehicle | measurements...

How do I sum the values in an array of maps in jq?

jq
Given a JSON stream of the following form: { "a": 10, "b": 11 } { "a": 20, "b": 21 } { "a": 30, "b": 31 } How do I sum the values in each of the objects and output a single object? i.e. { "a": 60, "b": 63 } I'm...