`_
:param requests: A set of typical search requests, together with their provided
ratings.
@@ -4053,113 +3870,10 @@ class AsyncElasticsearch(BaseClient):
In this case, the response includes a count of the version conflicts that were encountered.
Note that the handling of other error types is unaffected by the conflicts property.
Additionally, if you opt to count version conflicts, the operation could attempt to reindex more documents from the source than max_docs until it has successfully indexed max_docs documents into the target or it has gone through every document in the source query.
- NOTE: The reindex API makes no effort to handle ID collisions.
- The last document written will "win" but the order isn't usually predictable so it is not a good idea to rely on this behavior.
- Instead, make sure that IDs are unique by using a script.
- Running reindex asynchronously
- If the request contains wait_for_completion=false, Elasticsearch performs some preflight checks, launches the request, and returns a task you can use to cancel or get the status of the task.
- Elasticsearch creates a record of this task as a document at _tasks/<task_id>.
- Reindex from multiple sources
- If you have many sources to reindex it is generally better to reindex them one at a time rather than using a glob pattern to pick up multiple sources.
- That way you can resume the process if there are any errors by removing the partially completed source and starting over.
- It also makes parallelizing the process fairly simple: split the list of sources to reindex and run each list in parallel.
- For example, you can use a bash script like this:
- for index in i1 i2 i3 i4 i5; do
- curl -HContent-Type:application/json -XPOST localhost:9200/_reindex?pretty -d'{
- "source": {
- "index": "'$index'"
- },
- "dest": {
- "index": "'$index'-reindexed"
- }
- }'
- done
-
- ** Throttling**
- Set requests_per_second to any positive decimal number (1.4, 6, 1000, for example) to throttle the rate at which reindex issues batches of index operations.
- Requests are throttled by padding each batch with a wait time.
- To turn off throttling, set requests_per_second to -1.
- The throttling is done by waiting between batches so that the scroll that reindex uses internally can be given a timeout that takes into account the padding.
- The padding time is the difference between the batch size divided by the requests_per_second and the time spent writing.
- By default the batch size is 1000, so if requests_per_second is set to 500:
- target_time = 1000 / 500 per second = 2 seconds
- wait_time = target_time - write_time = 2 seconds - .5 seconds = 1.5 seconds
-
- Since the batch is issued as a single bulk request, large batch sizes cause Elasticsearch to create many requests and then wait for a while before starting the next set.
- This is "bursty" instead of "smooth".
- Slicing
- Reindex supports sliced scroll to parallelize the reindexing process.
- This parallelization can improve efficiency and provide a convenient way to break the request down into smaller parts.
- NOTE: Reindexing from remote clusters does not support manual or automatic slicing.
- You can slice a reindex request manually by providing a slice ID and total number of slices to each request.
- You can also let reindex automatically parallelize by using sliced scroll to slice on _id.
- The slices parameter specifies the number of slices to use.
- Adding slices to the reindex request just automates the manual process, creating sub-requests which means it has some quirks:
-
- - You can see these requests in the tasks API. These sub-requests are "child" tasks of the task for the request with slices.
- - Fetching the status of the task for the request with
slices only contains the status of completed slices.
- - These sub-requests are individually addressable for things like cancellation and rethrottling.
- - Rethrottling the request with
slices will rethrottle the unfinished sub-request proportionally.
- - Canceling the request with
slices will cancel each sub-request.
- - Due to the nature of
slices, each sub-request won't get a perfectly even portion of the documents. All documents will be addressed, but some slices may be larger than others. Expect larger slices to have a more even distribution.
- - Parameters like
requests_per_second and max_docs on a request with slices are distributed proportionally to each sub-request. Combine that with the previous point about distribution being uneven and you should conclude that using max_docs with slices might not result in exactly max_docs documents being reindexed.
- - Each sub-request gets a slightly different snapshot of the source, though these are all taken at approximately the same time.
-
- If slicing automatically, setting slices to auto will choose a reasonable number for most indices.
- If slicing manually or otherwise tuning automatic slicing, use the following guidelines.
- Query performance is most efficient when the number of slices is equal to the number of shards in the index.
- If that number is large (for example, 500), choose a lower number as too many slices will hurt performance.
- Setting slices higher than the number of shards generally does not improve efficiency and adds overhead.
- Indexing performance scales linearly across available resources with the number of slices.
- Whether query or indexing performance dominates the runtime depends on the documents being reindexed and cluster resources.
- Modify documents during reindexing
- Like _update_by_query, reindex operations support a script that modifies the document.
- Unlike _update_by_query, the script is allowed to modify the document's metadata.
- Just as in _update_by_query, you can set ctx.op to change the operation that is run on the destination.
- For example, set ctx.op to noop if your script decides that the document doesn’t have to be indexed in the destination. This "no operation" will be reported in the noop counter in the response body.
- Set ctx.op to delete if your script decides that the document must be deleted from the destination.
- The deletion will be reported in the deleted counter in the response body.
- Setting ctx.op to anything else will return an error, as will setting any other field in ctx.
- Think of the possibilities! Just be careful; you are able to change:
-
- _id_index_version_routing
- Setting _version to null or clearing it from the ctx map is just like not sending the version in an indexing request.
- It will cause the document to be overwritten in the destination regardless of the version on the target or the version type you use in the reindex API.
- Reindex from remote
- Reindex supports reindexing from a remote Elasticsearch cluster.
- The host parameter must contain a scheme, host, port, and optional path.
- The username and password parameters are optional and when they are present the reindex operation will connect to the remote Elasticsearch node using basic authentication.
- Be sure to use HTTPS when using basic authentication or the password will be sent in plain text.
- There are a range of settings available to configure the behavior of the HTTPS connection.
- When using Elastic Cloud, it is also possible to authenticate against the remote cluster through the use of a valid API key.
- Remote hosts must be explicitly allowed with the reindex.remote.whitelist setting.
- It can be set to a comma delimited list of allowed remote host and port combinations.
- Scheme is ignored; only the host and port are used.
- For example:
- reindex.remote.whitelist: [otherhost:9200, another:9200, 127.0.10.*:9200, localhost:*"]
-
- The list of allowed hosts must be configured on any nodes that will coordinate the reindex.
- This feature should work with remote clusters of any version of Elasticsearch.
- This should enable you to upgrade from any version of Elasticsearch to the current version by reindexing from a cluster of the old version.
- WARNING: Elasticsearch does not support forward compatibility across major versions.
- For example, you cannot reindex from a 7.x cluster into a 6.x cluster.
- To enable queries sent to older versions of Elasticsearch, the query parameter is sent directly to the remote host without validation or modification.
- NOTE: Reindexing from remote clusters does not support manual or automatic slicing.
- Reindexing from a remote server uses an on-heap buffer that defaults to a maximum size of 100mb.
- If the remote index includes very large documents you'll need to use a smaller batch size.
- It is also possible to set the socket read timeout on the remote connection with the socket_timeout field and the connection timeout with the connect_timeout field.
- Both default to 30 seconds.
- Configuring SSL parameters
- Reindex from remote supports configurable SSL settings.
- These must be specified in the elasticsearch.yml file, with the exception of the secure settings, which you add in the Elasticsearch keystore.
- It is not possible to configure SSL in the body of the reindex request.
+ Refer to the linked documentation for examples of how to reindex documents.
- ``_
+ ``_
:param dest: The destination you are copying to.
:param source: The source you are copying from.
@@ -4283,7 +3997,7 @@ class AsyncElasticsearch(BaseClient):
This behavior prevents scroll timeouts.
- ``_
+ ``_
:param task_id: The task identifier, which can be found by using the tasks API.
:param requests_per_second: The throttle for this request in sub-requests per
@@ -4329,7 +4043,7 @@ class AsyncElasticsearch(BaseClient):
human: t.Optional[bool] = None,
params: t.Optional[t.Mapping[str, t.Any]] = None,
pretty: t.Optional[bool] = None,
- source: t.Optional[str] = None,
+ source: t.Optional[t.Union[str, t.Mapping[str, t.Any]]] = None,
body: t.Optional[t.Dict[str, t.Any]] = None,
) -> ObjectApiResponse[t.Any]:
"""
@@ -4339,7 +4053,7 @@ class AsyncElasticsearch(BaseClient):
Render a search template as a search request body.
- ``_
+ ``_
:param id: The ID of the search template to render. If no `source` is specified,
this or the `id` request body parameter is required.
@@ -4396,7 +4110,24 @@ class AsyncElasticsearch(BaseClient):
async def scripts_painless_execute(
self,
*,
- context: t.Optional[str] = None,
+ context: t.Optional[
+ t.Union[
+ str,
+ t.Literal[
+ "boolean_field",
+ "composite_field",
+ "date_field",
+ "double_field",
+ "filter",
+ "geo_point_field",
+ "ip_field",
+ "keyword_field",
+ "long_field",
+ "painless_test",
+ "score",
+ ],
+ ]
+ ] = None,
context_setup: t.Optional[t.Mapping[str, t.Any]] = None,
error_trace: t.Optional[bool] = None,
filter_path: t.Optional[t.Union[str, t.Sequence[str]]] = None,
@@ -4408,15 +4139,22 @@ class AsyncElasticsearch(BaseClient):
"""
.. raw:: html
- Run a script.
- Runs a script and returns a result.
-
-
- ``_
-
- :param context: The context that the script should run in.
- :param context_setup: Additional parameters for the `context`.
- :param script: The Painless script to execute.
+ Run a script.
+ Runs a script and returns a result.
+ Use this API to build and test scripts, such as when defining a script for a runtime field.
+ This API requires very few dependencies and is especially useful if you don't have permissions to write documents on a cluster.
+ The API uses several contexts, which control how scripts are run, what variables are available at runtime, and what the return type is.
+ Each context requires a script, but additional parameters depend on the context you're using for that script.
+
+
+ ``_
+
+ :param context: The context that the script should run in. NOTE: Result ordering
+ in the field contexts is not guaranteed.
+ :param context_setup: Additional parameters for the `context`. NOTE: This parameter
+ is required for all contexts except `painless_test`, which is the default
+ if no value is provided for `context`.
+ :param script: The Painless script to run.
"""
__path_parts: t.Dict[str, str] = {}
__path = "/_scripts/painless/_execute"
@@ -4482,7 +4220,7 @@ class AsyncElasticsearch(BaseClient):
IMPORTANT: Results from a scrolling search reflect the state of the index at the time of the initial search request. Subsequent indexing or document changes only affect later search and scroll requests.
- ``_
+ ``_
:param scroll_id: The scroll ID of the search.
:param rest_total_hits_as_int: If true, the API response’s hit.total property
@@ -4613,7 +4351,6 @@ class AsyncElasticsearch(BaseClient):
] = None,
lenient: t.Optional[bool] = None,
max_concurrent_shard_requests: t.Optional[int] = None,
- min_compatible_shard_node: t.Optional[str] = None,
min_score: t.Optional[float] = None,
pit: t.Optional[t.Mapping[str, t.Any]] = None,
post_filter: t.Optional[t.Mapping[str, t.Any]] = None,
@@ -4635,7 +4372,7 @@ class AsyncElasticsearch(BaseClient):
script_fields: t.Optional[t.Mapping[str, t.Mapping[str, t.Any]]] = None,
scroll: t.Optional[t.Union[str, t.Literal[-1], t.Literal[0]]] = None,
search_after: t.Optional[
- t.Sequence[t.Union[None, bool, float, int, str, t.Any]]
+ t.Sequence[t.Union[None, bool, float, int, str]]
] = None,
search_type: t.Optional[
t.Union[str, t.Literal["dfs_query_then_fetch", "query_then_fetch"]]
@@ -4688,7 +4425,7 @@ class AsyncElasticsearch(BaseClient):
This situation can occur because the splitting criterion is based on Lucene document IDs, which are not stable across changes to the index.
- ``_
+ ``_
:param index: A comma-separated list of data streams, indices, and aliases to
search. It supports wildcards (`*`). To search all data streams and indices,
@@ -4766,10 +4503,9 @@ class AsyncElasticsearch(BaseClient):
per node that the search runs concurrently. This value should be used to
limit the impact of the search on the cluster in order to limit the number
of concurrent shard requests.
- :param min_compatible_shard_node: The minimum version of the node that can handle
- the request. Any handling node with a lower version will fail the request.
:param min_score: The minimum `_score` for matching documents. Documents with
- a lower `_score` are not included in the search results.
+ a lower `_score` are not included in search results and results collected
+ by aggregations.
:param pit: Limit the search to a point in time (PIT). If you provide a PIT,
you cannot specify an `` in the request path.
:param post_filter: Use the `post_filter` parameter to filter search results.
@@ -4788,18 +4524,19 @@ class AsyncElasticsearch(BaseClient):
:param preference: The nodes and shards used for the search. By default, Elasticsearch
selects from eligible nodes and shards using adaptive replica selection,
accounting for allocation awareness. Valid values are: * `_only_local` to
- run the search only on shards on the local node; * `_local` to, if possible,
+ run the search only on shards on the local node. * `_local` to, if possible,
run the search on shards on the local node, or if not, select shards using
- the default method; * `_only_nodes:,` to run the search
- on only the specified nodes IDs, where, if suitable shards exist on more
- than one selected node, use shards on those nodes using the default method,
- or if none of the specified nodes are available, select shards from any available
- node using the default method; * `_prefer_nodes:,` to if
- possible, run the search on the specified nodes IDs, or if not, select shards
- using the default method; * `_shards:,` to run the search only
- on the specified shards; * `` (any string that does not start
- with `_`) to route searches with the same `` to the same shards
- in the same order.
+ the default method. * `_only_nodes:,` to run the search
+ on only the specified nodes IDs. If suitable shards exist on more than one
+ selected node, use shards on those nodes using the default method. If none
+ of the specified nodes are available, select shards from any available node
+ using the default method. * `_prefer_nodes:,` to if possible,
+ run the search on the specified nodes IDs. If not, select shards using the
+ default method. * `_shards:,` to run the search only on the
+ specified shards. You can combine this value with other `preference` values.
+ However, the `_shards` value must come first. For example: `_shards:2,3|_local`.
+ * `` (any string that does not start with `_`) to route searches
+ with the same `` to the same shards in the same order.
:param profile: Set to `true` to return detailed timing information about the
execution of individual components in a search request. NOTE: This is a debugging
tool and adds significant overhead to search execution.
@@ -4950,8 +4687,6 @@ class AsyncElasticsearch(BaseClient):
__query["lenient"] = lenient
if max_concurrent_shard_requests is not None:
__query["max_concurrent_shard_requests"] = max_concurrent_shard_requests
- if min_compatible_shard_node is not None:
- __query["min_compatible_shard_node"] = min_compatible_shard_node
if pre_filter_shard_size is not None:
__query["pre_filter_shard_size"] = pre_filter_shard_size
if preference is not None:
@@ -5137,51 +4872,6 @@ class AsyncElasticsearch(BaseClient):
Optionally, a geo_bounds aggregation on the <field>. The search only includes this aggregation if the exact_bounds parameter is true.
If the optional parameter with_labels is true, the internal search will include a dynamic runtime field that calls the getLabelPosition function of the geometry doc value. This enables the generation of new point features containing suggested geometry labels, so that, for example, multi-polygons will have only one label.
- For example, Elasticsearch may translate a vector tile search API request with a grid_agg argument of geotile and an exact_bounds argument of true into the following search
- GET my-index/_search
- {
- "size": 10000,
- "query": {
- "geo_bounding_box": {
- "my-geo-field": {
- "top_left": {
- "lat": -40.979898069620134,
- "lon": -45
- },
- "bottom_right": {
- "lat": -66.51326044311186,
- "lon": 0
- }
- }
- }
- },
- "aggregations": {
- "grid": {
- "geotile_grid": {
- "field": "my-geo-field",
- "precision": 11,
- "size": 65536,
- "bounds": {
- "top_left": {
- "lat": -40.979898069620134,
- "lon": -45
- },
- "bottom_right": {
- "lat": -66.51326044311186,
- "lon": 0
- }
- }
- }
- },
- "bounds": {
- "geo_bounds": {
- "field": "my-geo-field",
- "wrap_longitude": false
- }
- }
- }
- }
-
The API returns results as a binary Mapbox vector tile.
Mapbox vector tiles are encoded as Google Protobufs (PBF). By default, the tile contains three layers:
@@ -5436,9 +5126,10 @@ class AsyncElasticsearch(BaseClient):
Some cells may intersect more than one vector tile.
To compute the H3 resolution for each precision, Elasticsearch compares the average density of hexagonal bins at each resolution with the average density of tile bins at each zoom level.
Elasticsearch uses the H3 resolution that is closest to the corresponding geotile density.
+ Learn how to use the vector tile search API with practical examples in the Vector tile search examples guide.
- ``_
+ ``_
:param index: Comma-separated list of data streams, indices, or aliases to search
:param field: Field containing geospatial data to return
@@ -5597,6 +5288,7 @@ class AsyncElasticsearch(BaseClient):
human: t.Optional[bool] = None,
ignore_unavailable: t.Optional[bool] = None,
local: t.Optional[bool] = None,
+ master_timeout: t.Optional[t.Union[str, t.Literal[-1], t.Literal[0]]] = None,
preference: t.Optional[str] = None,
pretty: t.Optional[bool] = None,
routing: t.Optional[str] = None,
@@ -5611,7 +5303,7 @@ class AsyncElasticsearch(BaseClient):
If the Elasticsearch security features are enabled, you must have the view_index_metadata or manage index privilege for the target data stream, index, or alias.
- ``_
+ ``_
:param index: A comma-separated list of data streams, indices, and aliases to
search. It supports wildcards (`*`). To search all data streams and indices,
@@ -5624,11 +5316,15 @@ class AsyncElasticsearch(BaseClient):
:param expand_wildcards: Type of index that wildcard patterns can match. If the
request can target data streams, this argument determines whether wildcard
expressions match hidden data streams. Supports comma-separated values, such
- as `open,hidden`. Valid values are: `all`, `open`, `closed`, `hidden`, `none`.
+ as `open,hidden`.
:param ignore_unavailable: If `false`, the request returns an error if it targets
a missing or closed index.
:param local: If `true`, the request retrieves information from the local node
only.
+ :param master_timeout: The period to wait for a connection to the master node.
+ If the master node is not available before the timeout expires, the request
+ fails and returns an error. IT can also be set to `-1` to indicate that the
+ request should never timeout.
:param preference: The node or shard the operation should be performed on. It
is random by default.
:param routing: A custom value used to route operations to a specific shard.
@@ -5655,6 +5351,8 @@ class AsyncElasticsearch(BaseClient):
__query["ignore_unavailable"] = ignore_unavailable
if local is not None:
__query["local"] = local
+ if master_timeout is not None:
+ __query["master_timeout"] = master_timeout
if preference is not None:
__query["preference"] = preference
if pretty is not None:
@@ -5706,7 +5404,7 @@ class AsyncElasticsearch(BaseClient):
search_type: t.Optional[
t.Union[str, t.Literal["dfs_query_then_fetch", "query_then_fetch"]]
] = None,
- source: t.Optional[str] = None,
+ source: t.Optional[t.Union[str, t.Mapping[str, t.Any]]] = None,
typed_keys: t.Optional[bool] = None,
body: t.Optional[t.Dict[str, t.Any]] = None,
) -> ObjectApiResponse[t.Any]:
@@ -5716,7 +5414,7 @@ class AsyncElasticsearch(BaseClient):
Run a search with a search template.
- ``_
+ ``_
:param index: A comma-separated list of data streams, indices, and aliases to
search. It supports wildcards (`*`).
@@ -5730,8 +5428,7 @@ class AsyncElasticsearch(BaseClient):
:param expand_wildcards: The type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether
wildcard expressions match hidden data streams. Supports comma-separated
- values, such as `open,hidden`. Valid values are: `all`, `open`, `closed`,
- `hidden`, `none`.
+ values, such as `open,hidden`.
:param explain: If `true`, returns detailed information about score calculation
as part of each hit. If you specify both this and the `explain` query parameter,
the API uses only the query parameter.
@@ -5859,7 +5556,7 @@ class AsyncElasticsearch(BaseClient):
- ``_
+ ``_
:param index: A comma-separated list of data streams, indices, and index aliases
to search. Wildcard (`*`) expressions are supported. To search all data streams
@@ -5929,7 +5626,20 @@ class AsyncElasticsearch(BaseClient):
)
@_rewrite_parameters(
- body_fields=("doc", "filter", "per_field_analyzer"),
+ body_fields=(
+ "doc",
+ "field_statistics",
+ "fields",
+ "filter",
+ "offsets",
+ "payloads",
+ "per_field_analyzer",
+ "positions",
+ "routing",
+ "term_statistics",
+ "version",
+ "version_type",
+ ),
)
async def termvectors(
self,
@@ -5992,10 +5702,11 @@ class AsyncElasticsearch(BaseClient):
The information is only retrieved for the shard the requested document resides in.
The term and field statistics are therefore only useful as relative measures whereas the absolute numbers have no meaning in this context.
By default, when requesting term vectors of artificial documents, a shard to get the statistics from is randomly selected.
- Use routing only to hit a particular shard.
+ Use routing only to hit a particular shard.
+ Refer to the linked documentation for detailed examples of how to use this API.
- ``_
+ ``_
:param index: The name of the index that contains the document.
:param id: A unique identifier for the document.
@@ -6006,9 +5717,9 @@ class AsyncElasticsearch(BaseClient):
(the sum of document frequencies for all terms in this field). * The sum
of total term frequencies (the sum of total term frequencies of each term
in this field).
- :param fields: A comma-separated list or wildcard expressions of fields to include
- in the statistics. It is used as the default list unless a specific field
- list is provided in the `completion_fields` or `fielddata_fields` parameters.
+ :param fields: A list of fields to include in the statistics. It is used as the
+ default list unless a specific field list is provided in the `completion_fields`
+ or `fielddata_fields` parameters.
:param filter: Filter terms based on their tf-idf scores. This could be useful
in order find out a good characteristic vector of a document. This feature
works in a similar manner to the second phase of the More Like This Query.
@@ -6046,41 +5757,41 @@ class AsyncElasticsearch(BaseClient):
__body: t.Dict[str, t.Any] = body if body is not None else {}
if error_trace is not None:
__query["error_trace"] = error_trace
- if field_statistics is not None:
- __query["field_statistics"] = field_statistics
- if fields is not None:
- __query["fields"] = fields
if filter_path is not None:
__query["filter_path"] = filter_path
if human is not None:
__query["human"] = human
- if offsets is not None:
- __query["offsets"] = offsets
- if payloads is not None:
- __query["payloads"] = payloads
- if positions is not None:
- __query["positions"] = positions
if preference is not None:
__query["preference"] = preference
if pretty is not None:
__query["pretty"] = pretty
if realtime is not None:
__query["realtime"] = realtime
- if routing is not None:
- __query["routing"] = routing
- if term_statistics is not None:
- __query["term_statistics"] = term_statistics
- if version is not None:
- __query["version"] = version
- if version_type is not None:
- __query["version_type"] = version_type
if not __body:
if doc is not None:
__body["doc"] = doc
+ if field_statistics is not None:
+ __body["field_statistics"] = field_statistics
+ if fields is not None:
+ __body["fields"] = fields
if filter is not None:
__body["filter"] = filter
+ if offsets is not None:
+ __body["offsets"] = offsets
+ if payloads is not None:
+ __body["payloads"] = payloads
if per_field_analyzer is not None:
__body["per_field_analyzer"] = per_field_analyzer
+ if positions is not None:
+ __body["positions"] = positions
+ if routing is not None:
+ __body["routing"] = routing
+ if term_statistics is not None:
+ __body["term_statistics"] = term_statistics
+ if version is not None:
+ __body["version"] = version
+ if version_type is not None:
+ __body["version_type"] = version_type
if not __body:
__body = None # type: ignore[assignment]
__headers = {"accept": "application/json"}
@@ -6125,6 +5836,7 @@ class AsyncElasticsearch(BaseClient):
human: t.Optional[bool] = None,
if_primary_term: t.Optional[int] = None,
if_seq_no: t.Optional[int] = None,
+ include_source_on_error: t.Optional[bool] = None,
lang: t.Optional[str] = None,
pretty: t.Optional[bool] = None,
refresh: t.Optional[
@@ -6162,10 +5874,11 @@ class AsyncElasticsearch(BaseClient):
The document must still be reindexed, but using this API removes some network roundtrips and reduces chances of version conflicts between the GET and the index operation.
The _source field must be enabled to use this API.
- In addition to _source, you can access the following variables through the ctx map: _index, _type, _id, _version, _routing, and _now (the current timestamp).
+ In addition to _source, you can access the following variables through the ctx map: _index, _type, _id, _version, _routing, and _now (the current timestamp).
+ For usage examples such as partial updates, upserts, and scripted updates, see the External documentation.
- ``_
+ ``_
:param index: The name of the target index. By default, the index is created
automatically if it doesn't exist.
@@ -6180,6 +5893,8 @@ class AsyncElasticsearch(BaseClient):
term.
:param if_seq_no: Only perform the operation if the document has this sequence
number.
+ :param include_source_on_error: True or false if to include the document source
+ in the error message in case of parsing errors.
:param lang: The script language.
:param refresh: If 'true', Elasticsearch refreshes the affected shards to make
this operation visible to search. If 'wait_for', it waits for a refresh to
@@ -6224,6 +5939,8 @@ class AsyncElasticsearch(BaseClient):
__query["if_primary_term"] = if_primary_term
if if_seq_no is not None:
__query["if_seq_no"] = if_seq_no
+ if include_source_on_error is not None:
+ __query["include_source_on_error"] = include_source_on_error
if lang is not None:
__query["lang"] = lang
if pretty is not None:
@@ -6351,6 +6068,24 @@ class AsyncElasticsearch(BaseClient):
A bulk update request is performed for each batch of matching documents.
Any query or update failures cause the update by query request to fail and the failures are shown in the response.
Any update requests that completed successfully still stick, they are not rolled back.
+ Refreshing shards
+ Specifying the refresh parameter refreshes all shards once the request completes.
+ This is different to the update API's refresh parameter, which causes only the shard
+ that received the request to be refreshed. Unlike the update API, it does not support
+ wait_for.
+ Running update by query asynchronously
+ If the request contains wait_for_completion=false, Elasticsearch
+ performs some preflight checks, launches the request, and returns a
+ task you can use to cancel or get the status of the task.
+ Elasticsearch creates a record of this task as a document at .tasks/task/${taskId}.
+ Waiting for active shards
+ wait_for_active_shards controls how many copies of a shard must be active
+ before proceeding with the request. See wait_for_active_shards
+ for details. timeout controls how long each write request waits for unavailable
+ shards to become available. Both work exactly the way they work in the
+ Bulk API. Update by query uses scrolled searches, so you can also
+ specify the scroll parameter to control how long it keeps the search context
+ alive, for example ?scroll=10m. The default is 5 minutes.
Throttling update requests
To control the rate at which update by query issues batches of update operations, you can set requests_per_second to any positive decimal number.
This pads each batch with a wait time to throttle the rate.
@@ -6385,21 +6120,11 @@ class AsyncElasticsearch(BaseClient):
Query performance is most efficient when the number of slices is equal to the number of shards in the index or backing index. If that number is large (for example, 500), choose a lower number as too many slices hurts performance. Setting slices higher than the number of shards generally does not improve efficiency and adds overhead.
Update performance scales linearly across available resources with the number of slices.
- Whether query or update performance dominates the runtime depends on the documents being reindexed and cluster resources.
- Update the document source
- Update by query supports scripts to update the document source.
- As with the update API, you can set ctx.op to change the operation that is performed.
- Set ctx.op = "noop" if your script decides that it doesn't have to make any changes.
- The update by query operation skips updating the document and increments the noop counter.
- Set ctx.op = "delete" if your script decides that the document should be deleted.
- The update by query operation deletes the document and increments the deleted counter.
- Update by query supports only index, noop, and delete.
- Setting ctx.op to anything else is an error.
- Setting any other field in ctx is an error.
- This API enables you to only modify the source of matching documents; you cannot move them.
+ Whether query or update performance dominates the runtime depends on the documents being reindexed and cluster resources.
+ Refer to the linked documentation for examples of how to update documents using the _update_by_query API:
- ``_
+ ``_
:param index: A comma-separated list of data streams, indices, and aliases to
search. It supports wildcards (`*`). To search all data streams or indices,
@@ -6424,9 +6149,8 @@ class AsyncElasticsearch(BaseClient):
:param expand_wildcards: The type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether
wildcard expressions match hidden data streams. It supports comma-separated
- values, such as `open,hidden`. Valid values are: `all`, `open`, `closed`,
- `hidden`, `none`.
- :param from_: Starting offset (default: 0)
+ values, such as `open,hidden`.
+ :param from_: Skips the specified number of documents.
:param ignore_unavailable: If `false`, the request returns an error if it targets
a missing or closed index.
:param lenient: If `true`, format-based query failures (such as providing text
@@ -6619,7 +6343,7 @@ class AsyncElasticsearch(BaseClient):
Rethrottling that speeds up the query takes effect immediately but rethrotting that slows down the query takes effect after completing the current batch to prevent scroll timeouts.
- ``_
+ ``_
:param task_id: The ID for the task.
:param requests_per_second: The throttle for this request in sub-requests per
diff -pruN 8.17.2-2/elasticsearch/_async/client/_base.py 9.1.0-2/elasticsearch/_async/client/_base.py
--- 8.17.2-2/elasticsearch/_async/client/_base.py 2025-03-19 06:35:35.000000000 +0000
+++ 9.1.0-2/elasticsearch/_async/client/_base.py 2025-07-30 08:51:18.000000000 +0000
@@ -174,7 +174,7 @@ def create_sniff_callback(
"GET",
"/_nodes/_all/http",
headers={
- "accept": "application/vnd.elasticsearch+json; compatible-with=8"
+ "accept": "application/vnd.elasticsearch+json; compatible-with=9"
},
request_timeout=(
sniff_options.sniff_timeout
@@ -298,7 +298,6 @@ class BaseClient:
def mimetype_header_to_compat(header: str) -> None:
# Converts all parts of a Accept/Content-Type headers
# from application/X -> application/vnd.elasticsearch+X
- nonlocal request_headers
mimetype = request_headers.get(header, None)
if mimetype:
request_headers[header] = _COMPAT_MIMETYPE_RE.sub(
diff -pruN 8.17.2-2/elasticsearch/_async/client/async_search.py 9.1.0-2/elasticsearch/_async/client/async_search.py
--- 8.17.2-2/elasticsearch/_async/client/async_search.py 2025-03-19 06:35:35.000000000 +0000
+++ 9.1.0-2/elasticsearch/_async/client/async_search.py 2025-07-30 08:51:18.000000000 +0000
@@ -44,7 +44,7 @@ class AsyncSearchClient(NamespacedClient
If the Elasticsearch security features are enabled, the deletion of a specific async search is restricted to: the authenticated user that submitted the original search request; users that have the cancel_task cluster privilege.
- ``_
+ ``_
:param id: A unique identifier for the async search.
"""
@@ -94,11 +94,11 @@ class AsyncSearchClient(NamespacedClient
If the Elasticsearch security features are enabled, access to the results of a specific async search is restricted to the user or API key that submitted it.
- ``_
+ ``_
:param id: A unique identifier for the async search.
- :param keep_alive: Specifies how long the async search should be available in
- the cluster. When not specified, the `keep_alive` set with the corresponding
+ :param keep_alive: The length of time that the async search should be available
+ in the cluster. When not specified, the `keep_alive` set with the corresponding
submit async request will be used. Otherwise, it is possible to override
the value and extend the validity of the request. When this period expires,
the search, if still running, is cancelled. If the search is completed, its
@@ -157,13 +157,17 @@ class AsyncSearchClient(NamespacedClient
Get the async search status.
Get the status of a previously submitted async search request given its identifier, without retrieving search results.
- If the Elasticsearch security features are enabled, use of this API is restricted to the monitoring_user role.
+ If the Elasticsearch security features are enabled, the access to the status of a specific async search is restricted to:
+
+ - The user or API key that submitted the original async search request.
+ - Users that have the
monitor cluster privilege or greater privileges.
+
- ``_
+ ``_
:param id: A unique identifier for the async search.
- :param keep_alive: Specifies how long the async search needs to be available.
+ :param keep_alive: The length of time that the async search needs to be available.
Ongoing async searches and any saved search results are deleted after this
period.
"""
@@ -277,7 +281,6 @@ class AsyncSearchClient(NamespacedClient
] = None,
lenient: t.Optional[bool] = None,
max_concurrent_shard_requests: t.Optional[int] = None,
- min_compatible_shard_node: t.Optional[str] = None,
min_score: t.Optional[float] = None,
pit: t.Optional[t.Mapping[str, t.Any]] = None,
post_filter: t.Optional[t.Mapping[str, t.Any]] = None,
@@ -295,7 +298,7 @@ class AsyncSearchClient(NamespacedClient
runtime_mappings: t.Optional[t.Mapping[str, t.Mapping[str, t.Any]]] = None,
script_fields: t.Optional[t.Mapping[str, t.Mapping[str, t.Any]]] = None,
search_after: t.Optional[
- t.Sequence[t.Union[None, bool, float, int, str, t.Any]]
+ t.Sequence[t.Union[None, bool, float, int, str]]
] = None,
search_type: t.Optional[
t.Union[str, t.Literal["dfs_query_then_fetch", "query_then_fetch"]]
@@ -342,7 +345,7 @@ class AsyncSearchClient(NamespacedClient
The maximum allowed size for a stored async search response can be set by changing the search.max_async_search_response_size cluster level setting.
- ``_
+ ``_
:param index: A comma-separated list of index names to search; use `_all` or
empty string to perform the operation on all indices
@@ -397,9 +400,8 @@ class AsyncSearchClient(NamespacedClient
per node this search executes concurrently. This value should be used to
limit the impact of the search on the cluster in order to limit the number
of concurrent shard requests
- :param min_compatible_shard_node:
:param min_score: Minimum _score for matching documents. Documents with a lower
- _score are not included in the search results.
+ _score are not included in search results and results collected by aggregations.
:param pit: Limits the search to a point in time (PIT). If you provide a PIT,
you cannot specify an in the request path.
:param post_filter:
@@ -522,8 +524,6 @@ class AsyncSearchClient(NamespacedClient
__query["lenient"] = lenient
if max_concurrent_shard_requests is not None:
__query["max_concurrent_shard_requests"] = max_concurrent_shard_requests
- if min_compatible_shard_node is not None:
- __query["min_compatible_shard_node"] = min_compatible_shard_node
if preference is not None:
__query["preference"] = preference
if pretty is not None:
diff -pruN 8.17.2-2/elasticsearch/_async/client/autoscaling.py 9.1.0-2/elasticsearch/_async/client/autoscaling.py
--- 8.17.2-2/elasticsearch/_async/client/autoscaling.py 2025-03-19 06:35:35.000000000 +0000
+++ 9.1.0-2/elasticsearch/_async/client/autoscaling.py 2025-07-30 08:51:18.000000000 +0000
@@ -44,7 +44,7 @@ class AutoscalingClient(NamespacedClient
NOTE: This feature is designed for indirect use by Elasticsearch Service, Elastic Cloud Enterprise, and Elastic Cloud on Kubernetes. Direct use is not supported.
- ``_
+ ``_
:param name: the name of the autoscaling policy
:param master_timeout: Period to wait for a connection to the master node. If
@@ -104,7 +104,7 @@ class AutoscalingClient(NamespacedClient
Do not use this information to make autoscaling decisions.
- ``_
+ ``_
:param master_timeout: Period to wait for a connection to the master node. If
no response is received before the timeout expires, the request fails and
@@ -151,7 +151,7 @@ class AutoscalingClient(NamespacedClient
NOTE: This feature is designed for indirect use by Elasticsearch Service, Elastic Cloud Enterprise, and Elastic Cloud on Kubernetes. Direct use is not supported.
- ``_
+ ``_
:param name: the name of the autoscaling policy
:param master_timeout: Period to wait for a connection to the master node. If
@@ -206,7 +206,7 @@ class AutoscalingClient(NamespacedClient
NOTE: This feature is designed for indirect use by Elasticsearch Service, Elastic Cloud Enterprise, and Elastic Cloud on Kubernetes. Direct use is not supported.
- ``_
+ ``_
:param name: the name of the autoscaling policy
:param policy:
diff -pruN 8.17.2-2/elasticsearch/_async/client/cat.py 9.1.0-2/elasticsearch/_async/client/cat.py
--- 8.17.2-2/elasticsearch/_async/client/cat.py 2025-03-19 06:35:35.000000000 +0000
+++ 9.1.0-2/elasticsearch/_async/client/cat.py 2025-07-30 08:51:18.000000000 +0000
@@ -50,7 +50,6 @@ class CatClient(NamespacedClient):
h: t.Optional[t.Union[str, t.Sequence[str]]] = None,
help: t.Optional[bool] = None,
human: t.Optional[bool] = None,
- local: t.Optional[bool] = None,
master_timeout: t.Optional[t.Union[str, t.Literal[-1], t.Literal[0]]] = None,
pretty: t.Optional[bool] = None,
s: t.Optional[t.Union[str, t.Sequence[str]]] = None,
@@ -65,7 +64,7 @@ class CatClient(NamespacedClient):
IMPORTANT: CAT APIs are only intended for human consumption using the command line or the Kibana console. They are not intended for use by applications. For application consumption, use the aliases API.
- ``_
+ ``_
:param name: A comma-separated list of aliases to retrieve. Supports wildcards
(`*`). To retrieve all aliases, omit this parameter or use `*` or `_all`.
@@ -78,10 +77,6 @@ class CatClient(NamespacedClient):
:param h: List of columns to appear in the response. Supports simple wildcards.
:param help: When set to `true` will output available columns. This option can't
be combined with any other query string option.
- :param local: If `true`, the request computes the list of selected nodes from
- the local cluster state. If `false` the list of selected nodes are computed
- from the cluster state of the master node. In both cases the coordinating
- node will send requests for further information to each selected node.
:param master_timeout: The period to wait for a connection to the master node.
If the master node is not available before the timeout expires, the request
fails and returns an error. To indicated that the request should never timeout,
@@ -113,8 +108,6 @@ class CatClient(NamespacedClient):
__query["help"] = help
if human is not None:
__query["human"] = human
- if local is not None:
- __query["local"] = local
if master_timeout is not None:
__query["master_timeout"] = master_timeout
if pretty is not None:
@@ -161,7 +154,7 @@ class CatClient(NamespacedClient):
IMPORTANT: CAT APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications.
- ``_
+ ``_
:param node_id: A comma-separated list of node identifiers or names used to limit
the returned information.
@@ -250,7 +243,7 @@ class CatClient(NamespacedClient):
They are not intended for use by applications. For application consumption, use the get component template API.
- ``_
+ ``_
:param name: The name of the component template. It accepts wildcard expressions.
If it is omitted, all component templates are returned.
@@ -334,7 +327,7 @@ class CatClient(NamespacedClient):
They are not intended for use by applications. For application consumption, use the count API.
- ``_
+ ``_
:param index: A comma-separated list of data streams, indices, and aliases used
to limit the request. It supports wildcards (`*`). To target all data streams
@@ -412,7 +405,7 @@ class CatClient(NamespacedClient):
They are not intended for use by applications. For application consumption, use the nodes stats API.
- ``_
+ ``_
:param fields: Comma-separated list of fields used to limit returned information.
To retrieve all fields, omit this parameter.
@@ -498,7 +491,7 @@ class CatClient(NamespacedClient):
You also can use the API to track the recovery of a large cluster over a longer period of time.
- ``_
+ ``_
:param format: Specifies the format to return the columnar data in, can be set
to `text`, `json`, `cbor`, `yaml`, or `smile`.
@@ -556,7 +549,7 @@ class CatClient(NamespacedClient):
Get help for the CAT APIs.
- ``_
+ ``_
"""
__path_parts: t.Dict[str, str] = {}
__path = "/_cat"
@@ -591,7 +584,9 @@ class CatClient(NamespacedClient):
filter_path: t.Optional[t.Union[str, t.Sequence[str]]] = None,
format: t.Optional[str] = None,
h: t.Optional[t.Union[str, t.Sequence[str]]] = None,
- health: t.Optional[t.Union[str, t.Literal["green", "red", "yellow"]]] = None,
+ health: t.Optional[
+ t.Union[str, t.Literal["green", "red", "unavailable", "unknown", "yellow"]]
+ ] = None,
help: t.Optional[bool] = None,
human: t.Optional[bool] = None,
include_unloaded_segments: t.Optional[bool] = None,
@@ -623,7 +618,7 @@ class CatClient(NamespacedClient):
They are not intended for use by applications. For application consumption, use an index endpoint.
- ``_
+ ``_
:param index: Comma-separated list of data streams, indices, and aliases used
to limit the request. Supports wildcards (`*`). To target all data streams
@@ -721,7 +716,7 @@ class CatClient(NamespacedClient):
IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the nodes info API.
- ``_
+ ``_
:param format: Specifies the format to return the columnar data in, can be set
to `text`, `json`, `cbor`, `yaml`, or `smile`.
@@ -899,7 +894,7 @@ class CatClient(NamespacedClient):
application consumption, use the get data frame analytics jobs statistics API.
- ``_
+ `
+ 