nerdexam
Amazon

DEA-C01 · Question #62

When processing large datasets using distributed computing frameworks, uneven distribution of data can lead to processing delays. What is this phenomenon commonly known as?

The correct answer is A. Data skew. The phenomenon where uneven distribution of data in distributed computing frameworks leads to processing delays because some tasks process significantly more data than others is commonly known as data skew.

Data Operations and Support

Question

When processing large datasets using distributed computing frameworks, uneven distribution of data can lead to processing delays. What is this phenomenon commonly known as?

Options

  • AData skew
  • BData partitioning
  • CData shuffling
  • DData fragmentation

How the community answered

(35 responses)
  • A
    94% (33)
  • B
    3% (1)
  • C
    3% (1)

Why each option

The phenomenon where uneven distribution of data in distributed computing frameworks leads to processing delays because some tasks process significantly more data than others is commonly known as data skew.

AData skewCorrect

Data skew occurs when data is unevenly distributed across partitions or processing nodes in a distributed computing system, leading to some nodes processing a disproportionately larger amount of data and causing bottlenecks and delays.

BData partitioning

Data partitioning is a technique used to divide a large dataset into smaller, more manageable parts for distributed processing, and it is a solution or a step, not the problem of uneven distribution itself.

CData shuffling

Data shuffling refers to the process of redistributing data across nodes in a distributed computing cluster, often occurring during operations like joins or aggregations, and it is a mechanism, not the problem of uneven data distribution causing delays.

DData fragmentation

Data fragmentation typically refers to data being stored in non-contiguous locations on a storage device, which can affect I/O performance, but it is not the term for uneven data distribution in the context of distributed processing delays.

Concept tested: Data skew in distributed computing

Source: https://docs.aws.amazon.com/glue/latest/dg/improve-performance-etl-jobs.html

Topics

#Distributed computing#Data skew#Performance optimization#Data processing

Community Discussion

No community discussion yet for this question.

Full DEA-C01 Practice