In general, we use es to build an index by creating an index, defining the fields in the index and the mapped types, and then importing data into the index. Dynamic mapping is a very important concept in ES. You can directly import a piece of data into the document. At the same time, index, field and field type will be created automatically without any other operation. This is the magic of dynamic mapping.
Dynamic field mapping
The dynamic mapping of ES is on by default, and the default rules of dynamic mapping are as follows:
| Data type of JSON | Data types in ES |
|---|---|
| null | Fields will not be mapped |
| true or false | boolean type |
| Floating point number | float |
| Integer number | long |
| JSON object | Object |
| array | First non null value type |
| String | 1. If the format of date type is satisfied, map to date type |
| 2. If the format of number type is satisfied, map to long or float | |
| 3. If it is a string, it will be mapped to a text type and a keyword type |
Next, let's look at an example of dynamic mapping. We store a piece of data directly into the dynamic index index. Note that we haven't created the dynamic index. If we store the data directly, the index will be created automatically. Next, let's look at the specific request:
PUT /dynamic-index/_doc/1 { "my_null": null, "my_boolean": false, "my_float": 1.56, "my_long": 3, "my_object": { "my_first": "first value", "my_second": "second_value" }, "my_array": [1,2,3], "my_date_1": "2020-05-01", "my_date_2": "2020/05/01 12:03:03", "my_date_3": "05/01/2020", "my_string_long": "1", "my_string_float": "4.6", "my_string": "The People's Republic of China" }
After the request is executed successfully, let's first look at the type of index:
GET /dynamic-index/_mapping
The results returned are as follows:
{ "dynamic-index": { "mappings": { "properties": { "my_array": { "type": "long" }, "my_boolean": { "type": "boolean" }, "my_date_1": { "type": "date" }, "my_date_2": { "type": "date", "format": "yyyy/MM/dd HH:mm:ss||yyyy/MM/dd||epoch_millis" }, "my_date_3": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "my_float": { "type": "float" }, "my_long": { "type": "long" }, "my_object": { "properties": { "my_first": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "my_second": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } } } }, "my_string": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "my_string_float": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "my_string_long": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } } } } } }
The returned result is relatively long. Let's take a look at each field to see if the dynamically mapped field meets our expectation:
| field | Mapping results | Whether it meets the expectation | Reason |
|---|---|---|---|
| my_null | No mapping | yes | null value does not map |
| my_boolean | boolean | yes | |
| my_float | float | yes | |
| my_long | long | yes | |
| my_object | object | yes | The mapping of two fields is automatically generated in my object |
| my_array | long | yes | The numbers in the array are long |
| my_date_1 | date | yes | |
| my_date_2 | date | yes | |
| my_date_3 | text | no | This date format is not specified, so it is mapped to text |
| my_string_long | text | no | Digital detection is off by default, not on |
| my_string_float | text | no | Digital detection is off by default, not on |
| my_string | text | yes | Normal string, mapped to text |
Next, we turn on the digital detection and execute the following request:
PUT /dynamic-index { "mappings": { "numeric_detection": true } }
Since there is a mapping relationship in our index dynamic index, setting it again will result in an error. Therefore, we need to delete the index and execute the following request:
DELETE /dynamic-index
After the index is deleted successfully, the previous settings will be executed. The execution is successful and the digital probe has been opened. Then add a date format MM/dd/yyyy. The request is as follows:
PUT /dynamic-index { "mappings": { "dynamic_date_formats": ["MM/dd/yyyy"] } }
The error message is the same as before. It seems that the setting of the date and the number probe should be together. Then we delete the index and send the request. The two settings are together:
PUT /dynamic-index { "mappings": { "numeric_detection": true, "dynamic_date_formats": ["MM/dd/yyyy"] } }
Execution successful, we will send the request to create index data before
PUT /dynamic-index/_doc/1 { "my_null": null, "my_boolean": false, "my_float": 1.56, "my_long": 3, "my_object": { "my_first": "first value", "my_second": "second_value" }, "my_array": [1,2,3], "my_date_1": "2020-05-01", "my_date_2": "2020/05/01 12:03:03", "my_date_3": "05/01/2020", "my_string_long": "1", "my_string_float": "4.6", "my_string": "The People's Republic of China" }
The execution is successful. Let's look at the index mapping again,
"my_string_float": { "type": "float" }, "my_string_long": { "type": "long" } "my_date_1": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "my_date_2": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "my_date_3": { "type": "date", "format": "MM/dd/yyyy" },
Let's take a look at the above fields. My string? Float and my string? Long map to the type we want, because we have turned on number detection. Let's take a look at the three date types we map, eh? Only my date 3 maps the date type, and the other two map to text type. This is because we only specify one format when setting dynamic date formats. We just need to add the other two types of date formats.
{ "mappings": { "numeric_detection": true, "dynamic_date_formats": ["MM/dd/yyyy","yyyy/MM/dd HH:mm:ss","yyyy-MM-dd"] } }
I won't give you a specific demonstration here. I'd like to have a try.
Dynamic field is a very important function in ES, which brings us great convenience and saves us the time of creating index field in development. It's a multiplier with half the effort, so we need to master it well~~