rollback - Spring Batch: Commit-Interval not honored after roll back during write

Question

Say my commit interval is 1000.

And during writing I get a error at 990th record which is skippable as per skip policy.

So a rollback will occur and the writer will start again writing the same records from record 1.

However, this time, It is commiting on each record. It does not honour commit interval. This is making the job dead slow.

Why the behavior is like that ?? Am I missing something in my configuration ??

Thanks.

Answer 1

that bevaviour is mandatory for spring batch to isolate the bad item(s), basically it rollbacks the chunk and processes/writes each item with commit-rate=1 to find the bad one (in either processor or writer)

see spring batch forum comment to similar problem

the relevant part

--> 5 items read, processing starts
<processor called:15>
<processor called:16>
<processor called:17> will throw an error on write
<processor called:18>
<processor called:19>
<before write:[15, 16, 17, 18, 19]>
<on write error>
--> error on item 17, but it was in the list, lets find it
--> rollback
<before chunk>
--> spring batch goes through all items of the chunk again to find the bad item
--> basically it runs now with commit-rate="1" (only for this chunk)
<processor called:15>
<after write:[15]>
<after chunk>
<before chunk>
<processor called:16>
<after write:[16]>
<after chunk>
<before chunk>
<processor called:17> called again for the bad item, because it's still unknown to spring batch, that this is the bad one
--> no write, because itemWriter.write() was called with the bad item only and did throw an exception (again)
--> but now spring batch knows the bad item
<before chunk>