You are managing a cluster with a large number of topics, and each topic has a lot of partitions. A team wants to significantly increase the number of partitions for some topics. Which parameters…

The correct answer is C. Check the max open file count on brokers. Each Kafka partition maps to multiple log segment files, and each segment results in open file descriptors on the broker. When the number of partitions increases significantly, it can exceed the OS-level limit for open files per broker process, leading to failures or degraded…

Managing Topics, Partitions, and Replicas

Question

You are managing a cluster with a large number of topics, and each topic has a lot of partitions. A team wants to significantly increase the number of partitions for some topics. Which parameters should you check before increasing the partitions?

Options

ACheck the producer batch size and buffer size.
BCheck if compression is being used.
CCheck the max open file count on brokers.
DCheck if acks=all is being used.

How the community answered

(59 responses)

A
2% (1)
B
5% (3)
C
85% (50)
D
8% (5)

Explanation

Each Kafka partition maps to multiple log segment files, and each segment results in open file descriptors on the broker. When the number of partitions increases significantly, it can exceed the OS-level limit for open files per broker process, leading to failures or degraded performance. Therefore, it is essential to check and possibly increase the ulimit -n (max open files) setting on the broker machines.

Topics

#partition scaling#open file descriptors#broker capacity#file handle limits

Community Discussion

No community discussion yet for this question.

Full CCAAK Practice