CERTIFIED-DATA-ENGINEER-PROFESSIONAL · Question #96
A Data engineer wants to run unit's tests using common Python testing frameworks on python functions defined across several Databricks notebooks currently used in production. How can the data…
The correct answer is A. Run unit tests against non-production data that closely mirrors production. Running unit tests directly against production data risks unintended modifications, performance degradation, and violation of data governance policies. The correct practice is to test against non-production data that closely mirrors production in structure, types, and…
Question
A Data engineer wants to run unit's tests using common Python testing frameworks on python functions defined across several Databricks notebooks currently used in production. How can the data engineer run unit tests against function that work with data in production?
Options
- ARun unit tests against non-production data that closely mirrors production
- BDefine and unit test functions using Files in Repos
- CDefine units test and functions within the same notebook
- DDefine and import unit test functions from a separate Databricks notebook
How the community answered
(29 responses)- A90% (26)
- C7% (2)
- D3% (1)
Explanation
Running unit tests directly against production data risks unintended modifications, performance degradation, and violation of data governance policies. The correct practice is to test against non-production data that closely mirrors production in structure, types, and representative values. This provides realistic test coverage without touching live systems. Option B (Files in Repos) is about code organization, not data safety. Option C (same notebook) creates maintainability issues. Option D (importing from separate notebooks) is a structural choice but does not address the core concern of testing against production data safely.
Topics
Community Discussion
No community discussion yet for this question.