Spring Batch 'RunIdIncrementer' not generating next value -

i have several spring batch (2.1.9.release) jobs running in production use org.springframework.batch.core.launch.support.runidincrementer.

sporadically, following error:

org.springframework.batch.core.repository.jobinstancealreadycompleteexception: job instance exists , complete parameters={run.id=23, tenant.code=xxx}.  if want run job again, change parameters.     @ org.springframework.batch.core.repository.support.simplejobrepository.createjobexecution(simplejobrepository.java:122) ~[spring-batch-core-2.1.9.release.jar:na]     @ sun.reflect.nativemethodaccessorimpl.invoke0(native method) ~[na:1.6.0_39]     @ sun.reflect.nativemethodaccessorimpl.invoke(nativemethodaccessorimpl.java:39) ~[na:1.6.0_39]     @ sun.reflect.delegatingmethodaccessorimpl.invoke(delegatingmethodaccessorimpl.java:25) ~[na:1.6.0_39]     @ java.lang.reflect.method.invoke(method.java:597) ~[na:1.6.0_39]     @ org.springframework.aop.support.aoputils.invokejoinpointusingreflection(aoputils.java:318) ~[spring-aop-3.1.1.release.jar:3.1.1.release]     @ org.springframework.aop.framework.reflectivemethodinvocation.invokejoinpoint(reflectivemethodinvocation.java:183) ~[spring-aop-3.1.1.release.jar:3.1.1.release]     @ org.springframework.aop.framework.reflectivemethodinvocation.proceed(reflectivemethodinvocation.java:150) ~[spring-aop-3.1.1.release.jar:3.1.1.release]     @ org.springframework.transaction.interceptor.transactioninterceptor.invoke(transactioninterceptor.java:110) ~[spring-tx-3.1.1.release.jar:3.1.1.release]     @ org.springframework.aop.framework.reflectivemethodinvocation.proceed(reflectivemethodinvocation.java:172) ~[spring-aop-3.1.1.release.jar:3.1.1.release]     @ org.springframework.batch.core.repository.support.abstractjobrepositoryfactorybean$1.invoke(abstractjobrepositoryfactorybean.java:168) ~[spring-batch-core-2.1.9.release.jar:na]     @ org.springframework.aop.framework.reflectivemethodinvocation.proceed(reflectivemethodinvocation.java:172) ~[spring-aop-3.1.1.release.jar:3.1.1.release]     @ org.springframework.aop.framework.jdkdynamicaopproxy.invoke(jdkdynamicaopproxy.java:202) ~[spring-aop-3.1.1.release.jar:3.1.1.release]     @ sun.proxy.$proxy64.createjobexecution(unknown source) ~[na:na]     @ org.springframework.batch.core.launch.support.simplejoblauncher.run(simplejoblauncher.java:111) ~[spring-batch-core-2.1.9.release.jar:na]     @ org.springframework.batch.core.launch.support.commandlinejobrunner.start(commandlinejobrunner.java:349) [spring-batch-core-2.1.9.release.jar:na]     @ org.springframework.batch.core.launch.support.commandlinejobrunner.main(commandlinejobrunner.java:574) [spring-batch-core-2.1.9.release.jar:na]     @ (omitted brevity) 

a sampling various xml contexts:

<bean     id="jobparametersincrementer"     class="org.springframework.batch.core.launch.support.runidincrementer" />  <batch:job id="rootjob"     abstract="true"     restartable="true">     <batch:validator>         <bean class="org.springframework.batch.core.job.defaultjobparametersvalidator">             <property name="requiredkeys" value="tenant.code"/>         </bean>     </batch:validator> </batch:job>  <batch:job id="rootjobwithincrementer"     abstract="true"     parent="rootjob"     incrementer="jobparametersincrementer" /> 

i use org.springframework.batch.core.launch.support.commandlinejobrunner execute job:

java org.springframework.batch.core.launch.support.commandlinejobrunner /com/xxx/job123/job123-context.xml job123 tenant.code=xxx -next  

all of jobs (that use incrementer) have rootjobwithincrementer parent.

i did quite bit of research , found got error had success changing isolation level of transaction manager. fiddled several levels, arriving @ read_commited.

<batch:job-repository     id="jobrepository"     data-source="oracle_hmp"     transaction-manager="datasourcetransactionmanager"     isolation-level-for-create="read_committed"/> 

based on understanding, type of error should happen if same job executed @ same time multiple processes -- there might contention incrementer. in instance, not case, yet see error.

any ideas might causing problem? should try different isolation level? else?

thanks!

there similar question here, not documented (and lacks , answer).

this might long shot took me long time figure out because symptom sporadically getting jobinstancealreadycompleteexception describe figured i'd suggest it.

the database using oracle , batch_job_seq , batch_job_execution_seq had created both had cache_size of 10.

this had effect of causing job_instance_id , job_execution_id not ordered correctly. spring batch makes assumption recent job_instance 1 max(job_instance_id) (see org.springframework.batch.core.repository.dao.jdbcjobinstancedao.find_last_jobs_by_name). since sequence thrown off, assumption did not hold true.

i fixed setting sequences no_cache.

an easy way tell if might problem check if sequences set cache @ and/or make sure job_instance_id , job_execution_id ascending each new run.