You recently developed a wide and deep model in TensorFlow. You generated training datasets using a SQL script that preprocessed raw data in BigQuery by performing instance-level transformations of th

Since the training dataset preprocessing is already handled by a SQL script in BigQuery, using the Kubeflow Pipelines SDK with a `BigQueryJobOp` (or similar BigQuery component) allows directly executing this existing SQL preprocessing within the pipeline. This approach minimizes model development time by avoiding rewriting or adapting the preprocessing logic in another framework.

Using `DataflowPythonJobOp` would require rewriting the existing SQL preprocessing logic into Python for Dataflow, which increases development time instead of minimizing it.

While TensorFlow Extended (TFX) is a robust framework, using its `ExampleGen` component alone might not be sufficient to directly incorporate complex BigQuery SQL preprocessing, and adapting the SQL logic into TFX's data transformation components would typically involve more development effort.

Implementing preprocessing within the TensorFlow Extended (TFX) SDK would necessitate rewriting the existing BigQuery SQL preprocessing logic using `tf.Transform` or similar TFX components, which goes against the goal of minimizing development and training time.