Skip to main content

Changelog

Dec 17, 2024​

Kafka Client (0.37.0)​

✨ responsive-pub/403: Apache Kafka 3.8 and Kafka Spring 3.3 Support: this commit makes the necessary changes to run Responsive with kafka 3.8.1 and kafka-spring 3.3. Note that we recommend you use Kafka Streams as a transitive dependency of Responsive kafka-client to avoid class loading issues.

🐛 responsive-pub/396: TTD Async Support: this commit fixes a bug in the ResponsiveTopologyTestDriver that prevented async processing from properly working.

Nov 19, 2024​

Kafka Client (0.34.0)​

❗responsive-pub/387 Responsive Business Source License is the project license for the Responsive Kafka Client. You can obtain a copy of the license here. Version 0.33.0 was the last version shipped under the Apache 2.0 License.

✨ responsive-pub/394: MongoDB Window Range Queries are now supported in the default Window Store implemenetation in MongoDB.

Nov 1, 2024​

Kafka Client (0.32.0)​

✨ responsive-pub/370: Row-level ttl is now available for Responsive key-value stores. You can now define a default ttl as well as a function that computes a row-level override for an individual records from the key, value, or both. See the "Row-level ttl" section of the State Stores documentation for more details on this feature and instructions on how to use it.

Aug 20, 2024​

Kafka Client (0.27.0)​

✨ responsive-pub/331: Async Processing now supports all PAPI topologies with any number of async processors. The very first version of the Responsive Async Processor limited applications to only one async processor per topology. This limitation has now been lifted. See the async processing documentation for the latest state of the feature and how to use it.

Jun 10, 2024​

Kafka Client (0.25.0)​

✨ responsive-pub/268: Async Processing is now available in ResponsiveKafkaStreams. Async processing allows Kafka Streams applications to concurrently process records within a single task, thus changing the upper limit of parallelization from the level of a Kafka partition to the level of a record key. See the documentation for more information.

🐛 responsive-pub/292: Adds back a previously truncated metric (last rebalance seconds ago).

❗responsive-pub/301: As of 0.25.0, changelog truncation is no longer supported.

Operator (0.15.0)​

✨ responsive-pub/274 This is the first in a chain of pull requests that allows for actions to be triggered remotely from the Responsive Controller (such as restarting a pod).

Apr 5, 2024​

Kafka Client (0.23.0)​

❗responsive-pub/269 This is a backwards incompatible change to the way we name configurations. See the PR description for a migration guide.

Mar 6, 2024​

Kafka Client (0.22.0)​

✨ responsive-pub/248: Responsive now supports all window stores, including Session Stores!

Operator (0.13.0)​

✨ responsive-pub/257 The cooldown for the policy execution is now configurable, so you can schedule more aggressive policies if your application is sensitive to throughput changes.

Feb 26, 2024​

Kafka Client (0.20.0)​

🐛 responsive-pub/249: Fixes issues with restoration when restoring from an early point in the changelog topic. This is usually only relevant if you are running the bootstrap tool introduced in 0.17.0.

Feb 20, 2024​

Kafka Client (0.17.0)​

✨responsive-pub/233: Introduce tooling to bootstrap a new storage cluster using an existing changelog topic. This can be used to migrate your stores from RocksDB to Responsive, or to migrate between Responsive storage solutions.

🐛 responsive-pub/230: Changes the default Cassandra consistency levels to QUORUM/QUORUM and allows configuration via config.

Feb 13, 2024​

Kafka Client (0.16.0)​

✨GA for MongoDB Stores: MongoDB is now considered generally available and supports nearly all of the Kafka Streams operations, including IQ and windowed stores.

✨responsive-pub/221: We made various improvements to performance, including this super cool PR to maintain local per-window bloom filters to avoid looking data up in remote storage if it's known not to exist during a window.

Dec 18, 2023​

Kafka Client (0.15.0)​

✨responsive-pub/163: This change supports backing MongoDB storage! With the release, MongoDB is still in Beta but supports EOS and KV Tables.

✨responsive-pub/181: Cassandra implementation now supports Window stores!

✨responsive-pub/191: You no longer need to configure OTEL agent in order to integrate with Responsive. Simply set responsive.metrics.enabled=true in your configuration file along with the API key/secret provided.

✨responsive-pub/167: This is one of many PRs that have improved out metric suite to include a bunch of new metrics. Some of my personal favorites:

  • Metrics around the current kafka streams state (RUNNING, REBALANCING, etc...)
  • Some metrics around the Responsive Cache effectiveness
  • Additional metrics around restoration/thread errors

October 16, 2023​

Breaking Changes (Code Incompatibility)​

Kakfa Client (0.14.0)​

❗responsive-pub/134 and responsive-pub/135 changed the way a new instance of ResponsiveKafkaStreams is created. These changes apply to dev.responsive.kafka-client versions 0.14.0 and above.

Please modify your code to use the constructor in place of the static factory method:

-  KafkaStreams streams = ResponsiveKafkaStreams.create(...);
+  KafkaStreams streams = new ResponsiveKafkaStreams(...);

It also removed the timestamped versions of the StoreSuppliers since all Responsive store implementations are compatible with timestamps. To create a timestamped store, you now only need to specify it in one place: the StoreBuilder. For a timestamped KeyValueStore you will still be using the ResponsiveStores#timestampedKeyValueStoreBuilder API, but can now pass in any kind of KeyValueBytesStoreSupplier provided by ResponsiveStores. You can apply all of these changes to your application code with the following sed commands:

FILE="<application-code-file-path>"
sed -ie "s|ResponsiveKafkaStreams.create|new ResponsiveKafkaStreams|g" ${FILE}
sed -ie "s|ResponsiveKeyValueParams.timestamped|ResponsiveKeyValueParams.keyValue|g" ${FILE}
sed -ie "s|ResponsiveKeyValueParams.timestampedFact|ResponsiveKeyValueParams.fact|g" ${FILE}
sed -ie "s|ResponsiveStores.timestampedKeyValueStore|ResponsiveStores.keyValueStore|g" ${FILE}

❗responsive-pub/148: This change restructured code to distinguish between public apis and internal implementation classes. Some classes, therefore, may have slightly shifted package names. If your application fails to compile after Responsive client versions, remove the imports that are not found and re-import them with the same class name.

Kafka Client (0.14.0)​

✨responsive-pub/133: Support configuring changelog truncation. This feature will allow you to optionally truncate changelogs when using Responsive stores to reduce storage utilization on your Kafka cluster. We recommend leaving the changelog enabled as an additional source of data recovery. Responsive guarantees durability of your records independently from this features.

✨responsive-pub/144: Responsive now supports GlobalKTables! You can now create GlobalKTables backed by Responsive's remote storage. Plugging them into your Streams application is just as easy (and in fact exactly the same) as configuring a regular KTable or KeyValueStore using ResponsiveStores:

    final GlobalKTable<Long, Long> globalTable = builder.globalTable(
        GLOBAL_TOPIC,
        ResponsiveStores.materialized(ResponsiveKeyValueParams.keyValue(STORE_NAME))
    );

✨ responsive-pub/150: Responsive platform is now compatible with all topology optimizations, including the previously restricted config option reuse.ktable.source.topics. This optimization enables source KTables to reuse their input topic as a changelog, avoiding the need to duplicate the source topic contents. Note that this optimization is incompatible with the new changelog truncation feature, however, and applying it will automatically disable changelog truncation for all source KTables.

🐛 responsive-pub/119: Fixes a bug in the handling of writes with a null value so that they are correctly identified as a tombstone.

🐛 responsive-pub/146: Fixes an (internal) bug where the internal property grace_gc_seconds was not correctly configured for state stores with TTL enabled.

Operator (0.14.0)​

✨responsive-pub/120: The Expected Latency Diagnoser was introduced, which allows you to specify a goal for the expected latency of a record being processed by a subtopology. The "expected latency" is defined here as the amount of time between when the record is enqueued on the source topic and the point at which it has finished being processed through the subtopology.

✨responsive-pub/131: Introduced the Thread Saturation Diagnoser, which will scale down the number of nodes in your application if threads are not being fully utilized.