Spring Boot + Spring Batch + Spring JPA

Question

I am working on Spring Batch job that moves data from Sql Server to Cassandra. I am using Spring Data JPA to read and write the data. I have created entities and JPA Repositories for both databases.

Now I can not figure out how to use my JpaRepostorty with Spring Batch ItemReader. I have searched on internet and found few refs where they mentioned to use JpaPageItemReader. But that requires specifying query and configuring other details. But I cannot figure out how to use existing JpaRepository that I have. Below is snippet of relevant code-

My JpaRepostory for SQL Server -

public interface ScanJpaRepository extends JpaRepository<Scan, Integer> 
{

   @Transactional(readOnly = true)
   @Query("select s from Scan s left join fetch s.projectVersion")
   Stream<Scan> findAllScan();
}

My Spring Batch Job -

@Configuration
@EnableBatchProcessing
public class SSCBatchConfigurationCassandra {

    @Autowired
    public JobBuilderFactory jobBuilderFactory;

    @Autowired
    public StepBuilderFactory stepBuilderFactory;


    @Bean
    public PlatformTransactionManager transactionManager() {
        return new ResourcelessTransactionManager();
    }


    @Bean
    public JobExplorer jobExplorer() throws Exception {
        MapJobExplorerFactoryBean jobExplorerFactory = new MapJobExplorerFactoryBean(mapJobRepositoryFactoryBean());
        jobExplorerFactory.afterPropertiesSet();
        return jobExplorerFactory.getObject();
    }

    @Bean
    public MapJobRepositoryFactoryBean mapJobRepositoryFactoryBean() {
        MapJobRepositoryFactoryBean mapJobRepositoryFactoryBean = new MapJobRepositoryFactoryBean();
        mapJobRepositoryFactoryBean.setTransactionManager(transactionManager());
        return mapJobRepositoryFactoryBean;
    }

    @Bean
    public JobRepository jobRepository() throws Exception {
        return mapJobRepositoryFactoryBean().getObject();
    }

    @Bean
    public JobLauncher jobLauncher() throws Exception {
        SimpleJobLauncher simpleJobLauncher = new SimpleJobLauncher();
        simpleJobLauncher.setJobRepository(jobRepository());
        return simpleJobLauncher;
    }


    @Bean
    public ItemReader<Project> reader() {
         **// how to read from ScanJpaRepository ??**
    }

    @Bean
    public CassandraItemProcessor processor() {
        return new CassandraItemProcessor();
    }

    @Bean
    public ItemWriter<CassandraProject> cqlWriter() {
         final CassandraBatchItemWriter writer = new CassandraBatchItemWriter();
         return writer;
    }



    // tag::jobstep[]
    @Bean
    public Job importUserJob(JobCompletionNotificationListener listener, Step step1) {
        return jobBuilderFactory.get("importUserJob")
                .incrementer(new RunIdIncrementer())
                .listener(listener)
                .flow(step1)
                .end()
                .build();
    }

    @Bean
    public Step step1() {
        return stepBuilderFactory.get("step1")
                .<Project, CassandraProject> chunk(100)
                .reader(reader())
                .processor(processor())
                .writer()
                .build();
    }


    // end::jobstep[]
}

Update #1: I added item reader to my batch job as mentioned.

@Configuration
@EnableBatchProcessing
public class FortifySSCBatchConfigurationCassandra {
   ....
   @Autowired
   public ScanItemReader itemReader;
   .....

   @Bean
   public Step step1() {
    return stepBuilderFactory.get("step1")
            .<Scan, CScan> chunk(100)
            .reader(itemReader)
            .processor(processor())
            .writer(cqlWriter())
            .build();
     }


}

My IDE complains about this -

The method reader(ItemReader<? extends Scan>) in the type SimpleStepBuilder<Scan,CScan> is not applicable for the arguments (ScanItemReader)

Update #2:

public class CassandraItemProcessor implements ItemProcessor<Scan, CScan> {


    @Override
    public CScan process(Scan s) throws Exception {


        .. 

        return new CScan();
    }

}


public class CassandraBatchItemWriter implements ItemWriter<CScan> {




    @Override
    public void write(List<? extends CScan> arg0) throws Exception {
        // TODO Auto-generated method stub

    }


}

Solution

You can declare your Reader like this

@Component
@JobScope
public class ScanItemReader extends RepositoryItemReader<Scan> {

  private final ScanJpaRepository repository;

  @Autowired
  public ScanItemReader(final ScanJpaRepository repository) {
    super();
    this.repository = repository;
  }

  @PostConstruct
  protected void init() {
    final Map<String, Sort.Direction> sorts = new HashMap<>();
    sorts.put("Your sort parameter"), Direction.ASC);// This could be any field name of your Entity class
    this.setRepository(this.repository);
    this.setSort(sorts);
    this.setMethodName(""); // You should sepcify the method which  
               //spring batch should call in your repository to fetch 
               // data and the arguments it needs needs to be  
               //specified with the below method.  
               // And this method  must return Page<T>
    this.setArguments();
  }
}

Autowire this reader bean and use it in your StepBuilder.

Comments

[indusBull]

Thanks for response. I am not able to import package for QAbstractOrder class with my current pom. Do I need to import any other dependency?

[pvpkiran]

sorry that was a mistake, you need to give your sort parameter for sorting of results, if you need sorting. You can specify field name of any of your Entity class

[indusBull]

I updated my question on next error. StepBuilder doesn't accept ScanItemReader.

[pvpkiran]

how is your processor class and writer class. Show the class declaration atleast. Your processor should take Scan as input and CScan as output. Because you have declared as <Scan, CScan>

[indusBull]

Sorry about that. Added both classes as requested.

[indusBull]

Please ignore that error. I had CScan and Scan swtiched.

[indusBull]

Let us continue this discussion in chat.

[sk555]

did you find any solution. I have this the same issue. Thanks in advance