Bodybuilder
At VIMANA, we have an Elasticsearch backend for storing most of our Big Data. While Elasticsearch is an amazing search engine which can do a variety of queries/aggregations, the Elasticsearch Query DSL is difficult to construct, especialy for newcomers. I have spent countless hours trying to get the brackets right and the searching for the specific terms. While the new SQL options allows the user to query using SQL, that still doesn’t have all the capabilities supported by the full DSL.
We NodeJS in the backend to query ES. However, construct the query in JS is quite simple thanks to the amazing Elasticsearch Bodybuilder Package on NPM. This allow the user to make a query like this -
bodybuilder()
.query('match', 'message', 'this is a test')
.filter('term', 'user', 'kimchy')
.orFilter('term', 'user', 'johnny')
.notFilter('term', 'user', 'cassie')
.aggregation('terms', 'user')
.build()
which is converted into the admittedly complex query below:
{
"query": {
"bool": {
"filter": {
"bool": {
"must": {
"term": {
"user": "kimchy"
}
},
"should": [
{
"term": {
"user": "johnny"
}
}
],
"must_not": [
{
"term": {
"user": "cassie"
}
}
]
}
},
"must": {
"match": {
"message": "this is a test"
}
}
}
},
"aggs": {
"agg_terms_user": {
"terms": {
"field": "user"
}
}
}
}
When we wanted to build some of our apps in Python, however, there was no readymade package available to be able to construct queries in a similar manner as above. Hence bodybuilder(python) was born!
Bodybuilder is envisioned as an (almost!) drop in replacement in python of the original Node Package. The API has been designed to be as close to the original package. Continuing the example above.
from bodybuilder import BodyBuilder as bodyBuilder
import json
output = bodybuilder() \
.query('match', 'message', 'this is a test') \
.filter('term', 'user', 'kimchy') \
.orFilter('term', 'user', 'johnny') \
.notFilter('term', 'user', 'cassie') \
.aggregation('terms', 'user') \
.build()
print(json.dumps(output, indent=4, sort_keys=True))
Gives the result as follows:
{
"aggs": {
"agg_terms_user": {
"terms": {
"field": "user"
}
}
},
"query": {
"bool": {
"filter": {
"term": {
"user": "kimchy"
}
},
"must": {
"match": {
"message": "this is a test"
}
},
"must_not": [
{
"term": {
"user": "cassie"
}
}
],
"should": [
{
"term": {
"user": "johnny"
}
}
]
}
}
}
The two queries are semantically equivalent.
Test it out!
Installation instructions and variations from the original package can be found on the project home page
You can use https://bodybuilder.js.org/ to test your constructions
Credits
Thanks to Danpaz and contributors of the original package for the original package from which I have liberally copied (with his permission!)