How to have KafkaProducer to use a mock Schema Registry for testing?

Question

I'm using KafkaProducer in my test cases and my producer uses the schemaRegistryUrl which points a my local instance of Schema Registry. Is there a way to mock how KafkaProducer connects with the Schema Registry? That is, to have KafkaProducer/Consumer in my tests to work without a running instance of Schema Registry.

Solution 1

https://github.com/confluentinc/schema-registry/blob/master/avro-serializer/src/main/java/io/confluent/kafka/serializers/AbstractKafkaAvroSerDe.java

In 5.3.x, you can find a MOCK_URL_PREFIX = "mock://", so just set the test schemaRegistryUrl with the prefix "mock://", like: "mock://testurl".

Solution 2

There absolutely is. The KafkaAvroSerializer and KafkaAvroDeserializer both have a constructor that takes in a SchemaRegistryClient. You can use a MockSchemaRegistryClient as the SchemaRegistryClient. Here's a code snippet showing how to do that:

private MockSchemaRegistryClient mockSchemaRegistryClient = new MockSchemaRegistryClient();
private String registryUrl = "unused";

public <T> Serde<T> getAvroSerde(boolean isKey) {
    return Serdes.serdeFrom(getSerializer(isKey), getDeserializer(isKey));
}

private <T> Serializer<T> getSerializer(boolean isKey) {
    Map<String, Object> map = new HashMap<>();
    map.put(KafkaAvroDeserializerConfig.AUTO_REGISTER_SCHEMAS, true);
    map.put(KafkaAvroDeserializerConfig.SCHEMA_REGISTRY_URL_CONFIG, registryUrl);
    Serializer<T> serializer = (Serializer) new KafkaAvroSerializer(mockSchemaRegistryClient);
    serializer.configure(map, isKey);
    return serializer;
}

private <T> Deserializer<T> getDeserializer(boolean key) {
    Map<String, Object> map = new HashMap<>();
    map.put(KafkaAvroDeserializerConfig.SPECIFIC_AVRO_READER_CONFIG, "true");
    map.put(KafkaAvroDeserializerConfig.SCHEMA_REGISTRY_URL_CONFIG, registryUrl);
    Deserializer<T> deserializer = (Deserializer) new KafkaAvroDeserializer(mockSchemaRegistryClient);
    deserializer.configure(map, key);
    return deserializer;
}

Comments

[dawsaw]

Probably helpful if you are more clear what you are testing here. Can you clarify if you need the schema registry function? If you do, then you can't mock test its function really. If you don't need to test its function then what are you testing?

[Vivek Sethi]

The github url has moved. Can you update that. Btw, just adding mock://testurl for my schemaRegistryUrl property worked like a charm!

[Eoin]

I second @VivekSethi request. Is there any documentation saying to use this? I couldn't find anything anywhere. I came across this solution purely by chance. It works like a charm

[Yohan]

Work great also for me. It's can be great if you can add any link that can explain what exactly happen when it was request mock:// url or add your explanation.. Is it possible to make additional use of it for tests or is it unique for a mocking schema?

[Tommy Schmidt]

New links: github.com/confluentinc/schema-registry/blob/… and github.com/confluentinc/schema-registry/blob/…