java - Updating elasticsearch entities with bulk

Question

I have this database data as below (ES 7.xx) version

    {
   "id":"1234",
   "expirationDate":"17343234234",
   "paths":"http:localhost:9090",
   "work":"software dev",
   "family":{
      "baba":"jams",
      "mother":"ela"
   }
},
{
   "id":"00021",
   "expirationDate":"0123234",
   "paths":"http:localhost:8080",
   "work":"software engi",
   "family":{
      "baba":"stev",
      "mother":"hela"
   }
}

how can i update the entity which its expirationDate smaller than current Time? to be the current time for example:

the id 00021 is expired because its expiration date is smaller than today then it should updated to current time.

something like void updateExpiredEntity(List<ids> ids,Long currentTime) suing void bulkUpdate(List<UpdateQuery> queries, BulkOptions bulkOptions, IndexCoordinates index);

Please provide me some code implementation

is it correct like this?

    public void update(UUID id,Long currentDate) {
        UpdateQuery updateQuery = UpdateQuery.builder(id.toString()).withRouting("expirationDate=currentDate")
            .build();
        elasticsearchTemplate.bulkUpdate(List.of(updateQuery), IndexCoordinates.of("index"));
    }
}

Answer 1

If you are using Elasticsearch 7.xx, I will assume that you have use Spring Data Elasticsearch version 4.0.x that comes with Spring boot 2.3.x. Since it's the version that support Elasticsearch 7.xx.

There're many things that have change in this Spring Data Elasticsearch version. Update document by query is one of them. Unlike before that we autowired ElasticsearchTemplate, we now have to use ElasticsearchRestTemplate and RestHighLevelClient instead.

In your case if you might want to use RestHighLevelClient to update document by query. Assume that you stored expirationDate as number mapping type in seconds unit then the code that you have asked for should look like this.

public class ElasticsearchService {

    @Autowired
    private ElasticsearchRestTemplate elasticsearchRestTemplate;

    @Autowired
    private RestHighLevelClient highLevelClient;

    public void updateExpireDateDemo() throws IOException {
        String indexName = "test";
        Date currentDate = new Date();
        Long seconds = (Long) (currentDate.getTime() / 1000);
        UpdateByQueryRequest request = new UpdateByQueryRequest(indexName);
        request.setQuery(new RangeQueryBuilder("expirationDate").lte(seconds));
        Script updateScript = new Script(
                ScriptType.INLINE, "painless",
                "ctx._source.expirationDate=" + seconds + ";",
                Collections.emptyMap());
        request.setScript(updateScript);
        highLevelClient.updateByQuery(request, RequestOptions.DEFAULT);
    }
}

I'm not quite get why you really need to use the bulkUpdate but if that's the case then. You have to query the record that need to be update from Elasticsearch to get and id of each document first. Then you can update with list of UpdateQuery. So your code will look like this.

@Service
public class ElasticsearchService {
    @Autowired
    private ElasticsearchRestTemplate elasticsearchRestTemplate;

    public void updateExpireDateByBulkDemo() throws IOException {
        String indexName = "test";
        Date currentDate = new Date();
        Long seconds = (Long) (currentDate.getTime() / 1000);
        List<UpdateQuery> updateList = new ArrayList();
        RangeQueryBuilder expireQuery = new RangeQueryBuilder("expirationDate").lte(seconds);
        NativeSearchQuery query = new NativeSearchQueryBuilder().withQuery(expireQuery).build();
        SearchHits<Data> searchResult = elasticsearchRestTemplate.search(query, Data.class, IndexCoordinates.of(indexName));
        for (SearchHit<Data> hit : searchResult.getSearchHits()) {
            String elasticsearchDocumentId = hit.getId();
            updateList.add(UpdateQuery.builder(elasticsearchDocumentId).withScript("ctx._source.expirationDate=" + seconds + ";").build());
        }
        if (updateList.size() > 0) {
            elasticsearchRestTemplate.bulkUpdate(updateList, IndexCoordinates.of(indexName));
        }
    }
}

However, this only update first page of the search result. If you require to update every record matched your query then you have to use searchScroll method in oder to get every document id instead.