nerdexam
Confluent

CCAAK · Question #65

Kafka Connect is running on a two node cluster in distributed mode. The connector is a source connector that pulls data from Postgres tables (users/payment/orders), writes to topics with two…

The correct answer is B. Increase the number of Connect Tasks (tasks max value). The connector is currently configured with "tasks.max": "1", which means only one task is handling all tables (users, payment, orders). This can create a bottleneck and lead to lag. Increasing tasks.max allows Kafka Connect to parallelize work across multiple tasks, which can…

Kafka Connect

Question

Kafka Connect is running on a two node cluster in distributed mode. The connector is a source connector that pulls data from Postgres tables (users/payment/orders), writes to topics with two partitions, and with replication factor two. The development team notices that the data is lagging behind. What should be done to reduce the data lag*? The Connector definition is listed below:

{ "name": "confluent-postgresql-source", "connector class": "PostgresSource", "topic.prefix": "postgresql_", & nbsp;& nbsp;& nbsp;… "db.name": "postgres", "table.whitelist": "users.payment.orders”, "timestamp.column.name": "created_at", "output.data format": "JSON", "db.timezone": "UTC", "tasks.max": "1" }

Options

  • AIncrease the number of Connect Nodes.
  • BIncrease the number of Connect Tasks (tasks max value).
  • CIncrease the number of partitions.
  • DIncrease the replication factor and increase the number of Connect Tasks.

How the community answered

(18 responses)
  • B
    83% (15)
  • C
    11% (2)
  • D
    6% (1)

Explanation

The connector is currently configured with "tasks.max": "1", which means only one task is handling all tables (users, payment, orders). This can create a bottleneck and lead to lag. Increasing tasks.max allows Kafka Connect to parallelize work across multiple tasks, which can pull data from different tables concurrently and reduce lag.

Topics

#tasks.max#source connector#distributed mode#data lag

Community Discussion

No community discussion yet for this question.

Full CCAAK Practice