Cluster Configuration Properties

Capacity (in number of txns) of an abort index segment.

Each partition tracks the aborted transaction offset ranges to help service client requests. If the number of transactions increases beyond this threshold, they are flushed to disk to ease memory pressure. Then they’re loaded on demand. This configuration controls the maximum number of aborted transactions before they are flushed to disk.

Interval, in milliseconds, at which Redpanda looks for inactive transactions and aborts them.

Whether Admin API clients must provide HTTP basic authentication headers.

Enable aggregation of metrics returned by the /metrics endpoint. Aggregation can simplify monitoring by providing summarized data instead of raw, per-instance metrics. Metric aggregation is performed by summing the values of samples by labels and is done when it makes sense by the shard and/or partition labels.

The amount of time since the last broker status heartbeat. After this time, a broker is considered offline and not alive.

The duration, in milliseconds, that Redpanda waits for the replication of entries in the controller log when executing a request to alter topic configurations. This timeout ensures that configuration changes are replicated across the cluster before the alteration request is considered complete.

Size of direct write operations to disk in bytes. A larger chunk size can improve performance for write-heavy workloads, but increase latency for these writes as more data is collected before each write operation. A smaller chunk size can decrease write latency, but potentially increase the number of disk I/O operations.

Defines the number of bytes allocated by the internal audit client for audit messages. When changing this, you must disable audit logging and then re-enable it for the change to take effect. Consider increasing this if your system generates a very large number of audit records in a short amount of time.

Enables or disables audit logging. When you set this to true, Redpanda checks for an existing topic named _redpanda.audit_log. If none is found, Redpanda automatically creates one for you.

List of strings in JSON style identifying the event types to include in the audit log. This may include any of the following: management, produce, consume, describe, heartbeat, authenticate, schema_registry, admin.

List of user principals to exclude from auditing.

List of topics to exclude from auditing.

Defines the policy for rejecting audit log messages when the audit log queue is full. If set to 'permit', then new audit messages are dropped and the operation is permitted. If set to 'reject', then the operation is rejected.

Defines the number of partitions used by a newly-created audit topic. This configuration applies only to the audit log topic and may be different from the cluster or other topic configurations. This cannot be altered for existing audit log topics.

Defines the replication factor for a newly-created audit log topic. This configuration applies only to the audit log topic and may be different from the cluster or other topic configurations. This cannot be altered for existing audit log topics. Setting this value is optional. If a value is not provided, Redpanda will use the value specified for internal_topic_replication_factor.

Interval, in milliseconds, at which Redpanda flushes the queued audit log messages to the audit log topic. Longer intervals may help prevent duplicate messages, especially in high throughput scenarios, but they also increase the risk of data loss during shutdowns where the queue is lost.

Defines the maximum amount of memory in bytes used by the audit buffer in each shard. Once this size is reached, requests to log additional audit messages will return a non-retryable error. Limiting the buffer size per shard helps prevent any single shard from consuming excessive memory due to audit log messages.

Use Redpanda’s internal communication system to write audit logs. When disabled, Redpanda uses a Kafka client to write audit logs instead.

Allow automatic topic creation. To prevent excess topics, this property is not supported on Redpanda Cloud BYOC and Dedicated clusters. You should explicitly manage topic creation for these Redpanda Cloud clusters.

If you produce to a topic that doesn’t exist, the topic will be created with defaults if this property is enabled.

Cluster identifier.

Size (in bytes) for each compacted log segment.

Target backlog size for compaction controller. If not set the max backlog size is configured to 80% of total disk space available.

Derivative coefficient for compaction PID controller.

Integral coefficient for compaction PID controller.

Maximum number of I/O and CPU shares that compaction process can use.

Minimum number of I/O and CPU shares that compaction process can use.

Proportional coefficient for compaction PID controller. This must be negative, because the compaction backlog should decrease when the number of compaction shares increases.

The interval (in milliseconds) for updating the controller responsible for compaction tasks. The controller uses this interval to decide how to prioritize background compaction work, which is essential for maintaining efficient storage use.

This is an internal-only configuration and should be enabled only after consulting with Redpanda support.

How often Redpanda runs the collection loop when enable_consumer_group_metrics is set to consumer_lag. Updates will not be more frequent than health_monitor_max_metadata_age.

How often to run the collection loop when enable_consumer_group_metrics contains consumer_lag.

Reducing the value of consumer_group_lag_collection_interval_sec increases the metric collection frequency, which may raise resource utilization. In most environments, this impact is minimal, but it’s best practice to monitor broker resource usage in high-scale settings.

This property lets you enable the batch cache for the consumer offsets topic. By default, the cache for consumer offsets topic is disabled. Changing this property is not recommended in production systems, as it may affect performance. The change is applied only after the restart.

Interval between iterations of controller backend housekeeping loop.

The maximum number of cluster updates the controller can process at the same time. Higher values speed up cluster changes but use more resources.

Maximum capacity of rate limit accumulation in controller ACLs and users operations limit.

Maximum capacity of rate limit accumulation in controller configuration operations limit.

Maximum capacity of rate limit accumulation in controller move operations limit.

Maximum capacity of rate limit accumulation in controller node management operations limit.

Maximum capacity of rate limit accumulation in controller topic operations limit.

Maximum amount of time before Redpanda attempts to create a controller snapshot after a new controller command appears.

If set to true, move partitions between cores in runtime to maintain balanced partition distribution.

Interval, in milliseconds, between trigger and invocation of core balancing.

Unit: milliseconds

If set to true, and if after a restart the number of cores changes, Redpanda will move partitions between cores to maintain balanced partition distribution.

Enables CPU profiling for Redpanda.

The sample period for the CPU profiler.

Timeout, in milliseconds, to wait for new topic creation.

The maximum size for a deployable WebAssembly binary that the broker can store.

The commit interval at which data transforms progress.

Enables WebAssembly-powered data transforms directly in the broker. When data_transforms_enabled is set to true, Redpanda reserves memory for data transforms, even if no transform functions are currently deployed. This memory reservation ensures that adequate resources are available for transform functions when they are needed, but it also means that some memory is allocated regardless of usage.

Buffer capacity for transform logs, per shard. Buffer occupancy is calculated as the total size of buffered log messages; that is, logs emitted but not yet produced.

Flush interval for transform logs. When a timer expires, pending logs are collected and published to the transform_logs topic.

Transform log lines truncate to this length. Truncation occurs after any character escaping.

The amount of memory to reserve per core for data transform (Wasm) virtual machines. Memory is reserved on boot. The maximum number of functions that can be deployed to a cluster is equal to data_transforms_per_core_memory_reservation / data_transforms_per_function_memory_limit.

The amount of memory to give an instance of a data transform (Wasm) virtual machine. The maximum number of functions that can be deployed to a cluster is equal to data_transforms_per_core_memory_reservation / data_transforms_per_function_memory_limit.

The percentage of available memory in the transform subsystem to use for read buffers.

The maximum amount of runtime to start up a data transform, and the time it takes for a single record to be transformed.

The percentage of available memory in the transform subsystem to use for write buffers.

Maximum amount of time the coordinator waits to snapshot after a command appears in the log.

Option to explicitly disable enforcement of datalake disk space usage.

The datalake disk usage monitor reclaims the overage multiplied by this this coefficient to compensate for data that is written during the idle period between control loop invocations.

Size, in bytes, of each memory block reserved for record translation, as tracked by the datalake scheduler.

The size, in bytes, of the block of disk reservation that the datalake manager will assign to each datalake scheduler when it runs out of local reservation.

The maximum number of translations that the datalake scheduler will allow to run at a given time. If a translation is requested, but the number of running translations exceeds this value, the request will be put to sleep temporarily, polling until capacity becomes available.

Time, in milliseconds, for a datalake translation as scheduled by the datalake scheduler. After a translation is scheduled, it will run until either the time specified has elapsed or all pending records on its source partition have been translated.

Size, in bytes, of the amount of scratch space datalake should use.

Size of the scratch space datalake soft limit expressed as a percentage of the datalake_scratch_space_size_bytes configuration value.

Size, in bytes, of the amount of per translator data that may be flushed to disk before the translator will upload and remove its current on disk data.

If set, how long debug bundles are kept in the debug bundle storage directory after they are created. If not set, debug bundles are kept indefinitely.

Path to the debug bundle storage directory. Note: Changing this path does not clean up existing debug bundles. If not set, the debug bundle is stored in the Redpanda data directory specified in the redpanda.yaml broker configuration file.

The recursion depth after which debug logging is enabled automatically for the log reader.

Default settings for preferred location of topic partition leaders. It can be either "none" (no preference), or "racks:<rack1>,<rack2>,…​" (prefer brokers with rack ID from the list).

The list can contain one or more rack IDs. If you specify multiple IDs, Redpanda tries to distribute the partition leader locations equally across brokers in these racks.

If enable_rack_awareness is set to false, leader pinning is disabled across the cluster.

Default number of quota tracking windows.

Default number of partitions per topic.

Default replication factor for new topics.

Default quota tracking window size in milliseconds.

Disable batch cache in log manager.

Disables the cluster recovery loop. This property is used to simplify testing and should not be set in production.

Disable registering the metrics exposed on the internal /metrics endpoint.

Disable registering the metrics exposed on the /public_metrics endpoint.

The percentage of total disk capacity that Redpanda will avoid using. This applies both when cloud cache and log data share a disk, as well as when cloud cache uses a dedicated disk.

It is recommended to not run disks near capacity to avoid blocking I/O due to low disk space, as well as avoiding performance issues associated with SSD garbage collection.

Raft election timeout expressed in milliseconds.

Enables cluster metadata uploads. Required for whole cluster restore.

List of enabled consumer group metrics. Accepted values include:

Limits the write rate for the controller log.

Enable exporting of some host metrics like /proc/diskstats, /proc/snmp and /proc/net/netstat.

Host metrics are prefixed with vectorized_host and are available on the /metrics endpoint.

Enable idempotent producers.

Enable automatic leadership rebalancing.

Enable the cluster metrics reporter. If true, the metrics reporter collects and exports to Redpanda Data a set of customer usage metrics at the interval set by metrics_reporter_report_interval.

Enable Redpanda extensions for MPX.

Enable PID file. You should not need to change.

Enable rack-aware replica assignment.

Enable SASL authentication for Kafka connections. Authorization is required to modify this property. See also kafka_enable_authorization.

Controls whether Redpanda validates schema IDs in records and which topic properties are enforced.

Values:

Enable creating shadow links from this cluster to a remote source cluster for data replication.

Enable transactions (atomic writes).

Enables the usage tracking mechanism, storing windowed history of kafka/cloud_storage metrics over time.

Whether new feature flags auto-activate after upgrades (true) or must wait for manual activation via the Admin API (false).

Maximum number of bytes returned in a fetch request.

Derivative coefficient for fetch PID controller.

Integral coefficient for fetch PID controller.

The maximum debounce time the fetch PID controller will apply, in milliseconds.

Proportional coefficient for fetch PID controller.

A fraction, between 0 and 1, for the target reactor utilization of the fetch scheduling group.

The strategy used to fulfill fetch requests.

Time to wait for the next read in fetch requests when the requested minimum bytes was not reached.

Time duration after which the inactive fetch session is removed from the fetch session cache. Fetch sessions are used to implement the incremental fetch requests where a consumer does not send all requested partitions to the server but the server tracks them for the consumer.

Delay added to the rebalance phase to wait for new members.

The maximum allowed session timeout for registered consumers. Longer timeouts give consumers more time to process messages in between heartbeats at the cost of a longer time to detect failures.