Caused by: java.lang.IllegalArgumentException: Unable to deserialize the execution context in Spring Batch

Question

I am developing Spring Boot + Batch XML based approach. In this example, I have created following classes. When I simply load or class the Spring Batch Job. I get the below error.

I web search links like : Migration to Spring Boot 2 and using Spring Batch 4 , but it did not solve my problem.

Could anyone please guide what exact solution needs to be applied here ?

Error:

Caused by: java.lang.IllegalArgumentException: Unable to deserialize the execution context
    at org.springframework.batch.core.repository.dao.JdbcExecutionContextDao$ExecutionContextRowMapper.mapRow(JdbcExecutionContextDao.java:325) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
    at org.springframework.batch.core.repository.dao.JdbcExecutionContextDao$ExecutionContextRowMapper.mapRow(JdbcExecutionContextDao.java:309) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
    at org.springframework.jdbc.core.RowMapperResultSetExtractor.extractData(RowMapperResultSetExtractor.java:93) ~[spring-jdbc-5.0.8.RELEASE.jar:5.0.8.RELEASE]
    at org.springframework.jdbc.core.RowMapperResultSetExtractor.extractData(RowMapperResultSetExtractor.java:60) ~[spring-jdbc-5.0.8.RELEASE.jar:5.0.8.RELEASE]
    at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:667) ~[spring-jdbc-5.0.8.RELEASE.jar:5.0.8.RELEASE]
    at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:605) ~[spring-jdbc-5.0.8.RELEASE.jar:5.0.8.RELEASE]
    at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:657) ~[spring-jdbc-5.0.8.RELEASE.jar:5.0.8.RELEASE]
    at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:688) ~[spring-jdbc-5.0.8.RELEASE.jar:5.0.8.RELEASE]
    at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:700) ~[spring-jdbc-5.0.8.RELEASE.jar:5.0.8.RELEASE]
    at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:756) ~[spring-jdbc-5.0.8.RELEASE.jar:5.0.8.RELEASE]
    at org.springframework.batch.core.repository.dao.JdbcExecutionContextDao.getExecutionContext(JdbcExecutionContextDao.java:112) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
    at org.springframework.batch.core.explore.support.SimpleJobExplorer.getJobExecutionDependencies(SimpleJobExplorer.java:202) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
    at org.springframework.batch.core.explore.support.SimpleJobExplorer.getJobExecutions(SimpleJobExplorer.java:83) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_162]
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_162]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_162]
    at java.lang.reflect.Method.invoke(Unknown Source) ~[na:1.8.0_162]
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343) ~[spring-aop-5.0.8.RELEASE.jar:5.0.8.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:197) ~[spring-aop-5.0.8.RELEASE.jar:5.0.8.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.0.8.RELEASE.jar:5.0.8.RELEASE]
    at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) ~[spring-aop-5.0.8.RELEASE.jar:5.0.8.RELEASE]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.0.8.RELEASE.jar:5.0.8.RELEASE]
    at com.sun.proxy.$Proxy70.getJobExecutions(Unknown Source) ~[na:na]
    at org.springframework.batch.core.JobParametersBuilder.getNextJobParameters(JobParametersBuilder.java:264) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
    at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.execute(JobLauncherCommandLineRunner.java:162) ~[spring-boot-autoconfigure-2.0.4.RELEASE.jar:2.0.4.RELEASE]
    at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.executeLocalJobs(JobLauncherCommandLineRunner.java:179) ~[spring-boot-autoconfigure-2.0.4.RELEASE.jar:2.0.4.RELEASE]
    at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.launchJobFromProperties(JobLauncherCommandLineRunner.java:134) ~[spring-boot-autoconfigure-2.0.4.RELEASE.jar:2.0.4.RELEASE]
    at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.run(JobLauncherCommandLineRunner.java:128) ~[spring-boot-autoconfigure-2.0.4.RELEASE.jar:2.0.4.RELEASE]
    at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:800) [spring-boot-2.0.4.RELEASE.jar:2.0.4.RELEASE]
    ... 5 common frames omitted
Caused by: com.fasterxml.jackson.databind.exc.MismatchedInputException: Unexpected token (START_OBJECT), expected VALUE_STRING: need JSON String that contains type id (for subtype of java.lang.Object)
 at [Source: (ByteArrayInputStream); line: 1, column: 9] (through reference chain: java.util.HashMap["map"])
    at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:59) ~[jackson-databind-2.9.6.jar:2.9.6]
    at com.fasterxml.jackson.databind.DeserializationContext.wrongTokenException(DeserializationContext.java:1498) ~[jackson-databind-2.9.6.jar:2.9.6]
    at com.fasterxml.jackson.databind.DeserializationContext.reportWrongTokenException(DeserializationContext.java:1273) ~[jackson-databind-2.9.6.jar:2.9.6]
    at com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer._locateTypeId(AsArrayTypeDeserializer.java:151) ~[jackson-databind-2.9.6.jar:2.9.6]
    at com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer._deserialize(AsArrayTypeDeserializer.java:96) ~[jackson-databind-2.9.6.jar:2.9.6]
    at com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer.deserializeTypedFromAny(AsArrayTypeDeserializer.java:71) ~[jackson-databind-2.9.6.jar:2.9.6]
    at com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializer$Vanilla.deserializeWithType(UntypedObjectDeserializer.java:712) ~[jackson-databind-2.9.6.jar:2.9.6]
    at com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringKeyMap(MapDeserializer.java:529) ~[jackson-databind-2.9.6.jar:2.9.6]
    at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:364) ~[jackson-databind-2.9.6.jar:2.9.6]
    at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:29) ~[jackson-databind-2.9.6.jar:2.9.6]
    at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4013) ~[jackson-databind-2.9.6.jar:2.9.6]
    at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3077) ~[jackson-databind-2.9.6.jar:2.9.6]
    at org.springframework.batch.core.repository.dao.Jackson2ExecutionContextStringSerializer.deserialize(Jackson2ExecutionContextStringSerializer.java:70) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
    at org.springframework.batch.core.repository.dao.Jackson2ExecutionContextStringSerializer.deserialize(Jackson2ExecutionContextStringSerializer.java:50) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
    at org.springframework.batch.core.repository.dao.JdbcExecutionContextDao$ExecutionContextRowMapper.mapRow(JdbcExecutionContextDao.java:322) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
    ... 34 common frames omitted

CommonConfig.java

@Configuration
@ComponentScan("com.XXXX")
@EnableBatchProcessing
@EnableScheduling
@PropertySource("classpath:database.properties")
@ImportResource({ "classpath:jobs/XYZ.xml"})
public class CommonConfig {
    @Bean
    BatchConfigurer configurer(@Qualifier("dataSource") DataSource dataSource) {
        return new DefaultBatchConfigurer(dataSource);
    }
}

XYZ.xml

<bean id="databaseConfig" class="com.XX.config.DatabaseConfig" />

    <bean id="transactionManager" class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />

    <bean id="job_Repository" class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean">
        <property name="transactionManager" ref="transactionManager" />
    </bean>

    <bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
        <property name="jobRepository" ref="job_Repository" />
    </bean>

    <bean id="stepScope" class="org.springframework.batch.core.scope.StepScope">
        <property name="autoProxy" value="true"/>
    </bean>

    <batch:job id="myXYZBatchJob">
        <batch:step id="XYZContextStep" >
            <batch:tasklet ref=XYZContextTasklet" />
        </batch:step>
    </batch:job>
    ......
    ........
</bean>

DatabaseConfig.java

@Configuration
@EnableTransactionManagement
@ComponentScan({ "com.XXX" })
@EnableJpaRepositories(basePackages = {"com.XX.repository", "com.XX.custom.repository"}, 
                        entityManagerFactoryRef = "entityManagerFactory", 
                        transactionManagerRef = "transactionManager")
public class DatabaseConfig {
    @Bean
    @Primary
    @ConfigurationProperties("abc.datasource")
    public DataSourceProperties dataSourceProperties() {
        return new DataSourceProperties();
    }

    @Primary
    @Bean(name = "dataSource")
    @ConfigurationProperties(prefix = "abc.datasource")
    public DataSource dataSource() {
        return dataSourceProperties().initializeDataSourceBuilder().build();
    }

    @Primary
    @Bean(name = "entityManagerFactory")
    public LocalContainerEntityManagerFactoryBean entityManagerFactory(@Qualifier("dataSource") DataSource dataSource) {

        LocalContainerEntityManagerFactoryBean em = new LocalContainerEntityManagerFactoryBean();
        em.setDataSource(dataSource);
        em.setPackagesToScan(new String[] { "com.XX.Entity" });
        em.setPersistenceUnitName("devcloud");

        JpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
        em.setJpaVendorAdapter(vendorAdapter);
        return em;
    }

    @Primary
    @Bean(name = "transactionManager")
    public PlatformTransactionManager transactionManager(@Qualifier("entityManagerFactory") EntityManagerFactory entityManagerFactory) {
        return new JpaTransactionManager(entityManagerFactory);
    }
}

EDIT-1: I am using Spring Boot Parent version 2.0.4.RELEASE, which is latest. It pulls all the Spring versions including Context version of 5.0.8.RELEASE.

Request you to post code snippet and detailed description to support your answer.

Edit-2:

public class BillingConfig implements BatchConfigurer{

    private PlatformTransactionManager transactionManager;
    private JobRepository jobRepository;
    private JobLauncher jobLauncher;
    private JobExplorer jobExplorer;

    @Override
    public JobRepository getJobRepository() {
        return jobRepository;
    }

    @Override
    public PlatformTransactionManager getTransactionManager() {
        return transactionManager;
    }

    @Override
    public JobLauncher getJobLauncher() throws JobExecutionAlreadyRunningException, JobRestartException, JobInstanceAlreadyCompleteException, JobParametersInvalidException {
        return jobLauncher;
    }

    @Override
    public JobExplorer getJobExplorer() {
        return jobExplorer;
    }

    @PostConstruct
    void initialize() throws Exception {
        if (this.transactionManager == null) {
            this.transactionManager = new ResourcelessTransactionManager();
        }

        // A FactoryBean that automates the creation of a SimpleJobRepository using non-persistent in-memory DAO implementations. 
        MapJobRepositoryFactoryBean jobRepositoryFactory = new MapJobRepositoryFactoryBean(this.transactionManager);
        jobRepositoryFactory.afterPropertiesSet();
        this.jobRepository = jobRepositoryFactory.getObject();

        // A FactoryBean that automates the creation of a SimpleJobRepository using non-persistent in-memory DAO implementations. 
        MapJobExplorerFactoryBean jobExplorerFactory = new MapJobExplorerFactoryBean(jobRepositoryFactory);
        jobExplorerFactory.afterPropertiesSet();
        this.jobExplorer = jobExplorerFactory.getObject();
        this.jobLauncher = createJobLauncher();
    }

    private JobLauncher createJobLauncher() throws Exception {
        SimpleJobLauncher simpleJobLauncher = new SimpleJobLauncher();
        simpleJobLauncher.setJobRepository(jobRepository);
        simpleJobLauncher.afterPropertiesSet();
        return simpleJobLauncher;
    }
}

Solution 1

This error happens when the execution context of your job is serialized with version 3 (using XStream by default) and then deserialized with version 4 (using Jackson by default). So either downgrade Spring Batch to version 3 or configure your job repository to use the XStreamExecutionContextStringSerializer.

In your case, you have already defined a bean of type BatchConfigurer, so you can override the createJobRepository method and configure the XStream serializer. For example:

@Bean
BatchConfigurer configurer(@Qualifier("dataSource") DataSource dataSource, PlatformTransactionManager transactionManager) {
    return new DefaultBatchConfigurer(dataSource) {
        @Override
        protected JobRepository createJobRepository() throws Exception {
            JobRepositoryFactoryBean factory = new JobRepositoryFactoryBean();
            factory.setDataSource(dataSource);
            factory.setTransactionManager(transactionManager);
            factory.setSerializer(new XStreamExecutionContextStringSerializer());
            factory.afterPropertiesSet();
            return factory.getObject();
        }
    };
}

Solution 2

Try cleaning up the spring batch tables. Use following link : --cleaning spring batch tables ---- http://forum.spring.io/forum/spring-projects/batch/122103-clean-spring-batch-metadata-tables

Comments

[Mahmoud Ben Hassine]

This error happens when the execution context of your job is serialized with version 3 (using XStream by default) and then deserialized with version 4 (using Jackson by default). So either downgrade Spring Batch to version 3 or configure your job repository to use the XStreamExecutionContextStringSerializer (See jobRepositoryFactoryBean.setSerializer(new XStreamExecutionContextStringSerializer());)

[Jeff Cook]

@Hassine - One more thing XStreamExecutionContextStringSerializer is deprecated in 4.0.1.RELEASE

[Jeff Cook]

@Hassine - Thanks so much !. Please refer Edit-2 and guide me where to adjust code ? –

[Devendra Singh]

While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes.

[Jeff Cook]

On Prod, you cant clean-up the data. The accepted answer look correct approach

[Mahmoud Ben Hassine]

The MapJobRepositoryFactoryBean does not expose a method to customize the serializer, because this job repository does not serialize and persist data in a persistent store. So I'm confused with your EDIT-2 about how you can have this exception with an in-memory job repository.

[Nirmal Kumar]

Adding to this, you might have to clear all previous batch execution context metadata which got added because of the non-serializable object.

[gzp___]

@MahmoudBenHassine I added the code you suggested to fix a localDateTime deserialization issue and it worked, but I am getting an exception deserializing a map with an object value as ExecutionContext (added by a paritioner), is there a way to fix this ?. Caused by: com.fasterxml.jackson.databind.exc.InvalidTypeIdException: Missing type id when trying to resolve subtype of [map type; class java.util.HashMap, [simple type, class java.lang.String] -> [simple type, class java.lang.Object]]: missing type id property '@class' at [Source: (ByteArrayInputStream); line: 1, column: 1274]

[Mahmoud Ben Hassine]

@gzp___ That's another (not related) issue, see stackoverflow.com/questions/62718203/…

[gzp___]

@MahmoudBenHassine thanks for pointing me to the thread, I found out that if using jobExplorer as in my case the same serializer (using javaTimeModule) must be added also to the jobExplorer, hence the customer configurer class must override the createJobExplorer method and set the same serializer there