Hey,
I work in ad tech, and we are massively scaling up our data operations. Instead of letting a vendor handle campaign performance data across clients and ad platforms, we are going to start handling it ourselves. In the past i was just doing analytics engineering and business intelligence, but now the core of the business will be running on my pipelines.
I know i need to move beyond what i have now which is cloud run jobs running on schedules + dbt core, for better task dependency management and observability primarily. I've considered a few options, but the ones I'm deciding between are:
fivetran and its integrated dbt
airflow to orchestrate imports + exports dbt core
I have experimented a bit with google managed airflow in the past but have never used fivetran. Am i correct that the main benefit for choosing fivetran in this situation (which will cost 3-4x as much) is the pre-built connectors? Or is there more to fivetran than I understand? Also, any drawbacks to using fivetran I should be considering?