Spring Kafka SSL setup in Spring boot application.yml

Question

I am trying to setup a Spring Boot Application with a Kafka Client to use SSL. I have my keystore.jks and truststore.jks stored on a filesystem(on a docker container) because of this: https://github.com/spring-projects/spring-kafka/issues/710

Here is my application.yml:

spring:
  kafka:
      ssl:
        key-password: pass
        keystore-location: /tmp/kafka.client.keystore.jks
        keystore-password: pass
        truststore-location: /tmp/kafka.client.truststore.jks
        truststore-password: pass

But when I start the application ( in a docker container) it says:

Caused by: java.lang.IllegalStateException: Resource 'class path resource [tmp/kafka.client.keystore.jks]' must be on a file system
[..]
Caused by: java.io.FileNotFoundException: class path resource [tmp/kafka.client.keystore.jks] cannot be resolved to URL because it does not exist

I checked on the container and the .jks are there in /tmp .

I cannot understand how to pass .jks to spring boot.

UPDATE 06/07/2018

This is my dockerfile

FROM openjdk:8-jdk-alpine
VOLUME /tmp
COPY ssl/kafka.client.keystore.jks /tmp
COPY ssl/kafka.client.truststore.jks /tmp
ARG JAR_FILE
ADD ${JAR_FILE} app.jar
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar"]

Solution 1

According to discussion and to enable kafka ssl configuration, first need to enable and set ssl properties in consumerFactory

@Bean
public ConsumerFactory<String, ReportingTask> consumerFactory() {
    Map<String, Object> props = new HashMap<>();
    props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServers);
    props.put(ConsumerConfig.GROUP_ID_CONFIG, groupId);
    props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
    props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, JsonSerializable.class);
    props.put(ConsumerConfig.CLIENT_ID_CONFIG, clientId);
    props.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, enableAutoCommit);
    props.put(ConsumerConfig.AUTO_COMMIT_INTERVAL_MS_CONFIG, autoCommitInterval);
    props.put(ConsumerConfig.SESSION_TIMEOUT_MS_CONFIG, sessionTimeout);
    props.put(ConsumerConfig.MAX_POLL_RECORDS_CONFIG, maxRecords);
    props.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, offSet);
    if (sslEnabled) {
        props.put("security.protocol", "SSL");
        props.put("ssl.truststore.location", trustStoreLocation);
        props.put("ssl.truststore.password", trustStorePassword);

        props.put("ssl.key.password", keyStorePassword);
        props.put("ssl.keystore.password", keyStorePassword);
        props.put("ssl.keystore.location", keyStoreLocation);
    }
    return new DefaultKafkaConsumerFactory<>(props, new StringDeserializer(), new JsonDeserializer<>(Task.class));
}

And copy the certificates into docker container

COPY ssl/stage/* /var/lib/kafka/stage/

Solution 2

If anyone is still looking at this, try prepending file:// to the file path:

truststorelocation: "file:///tmp/kafka.client.keystore.jks"

The error is complaining about the lack of a URL - adding a protocol (file://) makes the path a URL (speaking very basically)

Comments

[Justin]

It doesn't work. I got Caused by: java.io.FileNotFoundException: class path resource [tmp/kafka.client.keystore.jks] cannot be resolved to URL because it does not exist

[Justin]

I removed these two lines and changed my application.yml like you said but I am keep getting the same exception. @Deadpool

[nitinsridar]

What is sslEnabled in if condition here?

[Anupreet]

You can have your own custom property if you want to enable ssl for kafka or not.

[Michael Rountree]

This is the correct answer. If the properties are coming from a file, then it needs to prepend file.

[obe6]

You can configure Kafka only configuring application.yaml, there's no need to write code normally

[saganas]

This worked with properties passed into DefaultKafkaConsumerFactory