Spring Cloud Stream Kafka — Atypical Configuration Strategy
Spring Cloud Stream is undoubtedly a go-to framework for any spring developer to get started with building a highly scalable event-driven system. As it gives a flexible programming model which enables the developer to build the system with efficiency and high productivity. The framework provides multiple binder implementations such as Kafka, RabbitMQ and various others.
Though the Spring Team has done a very good job of easing the way how the binders and bindings can be configured, there is always a new requirement that arises which leaves you astounded.
I came across this aberrant use case where all the configuration related to binders and bindings are supposed to be fetched from a cloud-config server. Nothing unusual in this isn't it. But here comes the trouble, we shouldn’t be specifying any configuration in application.properties or as an environment variable or as command-line arguments. For which we don’t have any doc support out of the box from Spring.
So how to do this anyway!!!
We’ll be discussing only the way how to configure the binders and bindings using a configuration class. And this blog is completely based on Kafka as the binder.
I have got this configuration class SpringCloudStreamConfig.java which holds all the destination binders and bindings properties defined at one place. Let’s get started with configuring bindings.
Destination Bindings: Bridge between the external messaging systems and application provided Producers and Consumers of messages (created by the Destination Binders).
We have to define a Bean of type BindingServiceProperties. Since there is already a spring defined bean, we should make sure we have made our custom defined BindingServiceProperties Bean as Primary. For this article, we’ll use two topics test1 and test2 as the bindings. A typical binding will be requiring destination — topic name, group — consumer-group name, consumer — all the consumer-related properties, and the binder — name of the binder which we are binding in our application. Since we are using a single binder, we can ignore the binder property. It’s pretty much straight forward implementation here. Let's move on to configuring our binder now.
Destination Binders: Components responsible to provide integration with the external messaging systems.
Similar to BindingConfiguration we have to create a Bean of type KafkaBinderConfigurationProperties and annotate with @Primary to override the default spring implementation. There are a lot of configuration associated with Kafka Binders, but for our example, we can just stick with the two most important properties — brokers and defaultBrokerPort. And that's the end of it. Boot up your application with this configuration class in place and everything should fall in place.
That’s it for this post. The complete example can be found here. Feel free to comment on the content.
