r/dataengineeringjobs

Hello everyone,

I am a backend SWE with 3+ yoe who has worked on ETL teams. I am interviewing to make a switch to a full fledge DE role, I’ve worked more on the upstream data side, my experience has been preparing and validating the data that downstream teams use for reporting and analytics. I have an upcoming round where I will be presented with a business case/case study and provided with a data set, this is all the information I got.

Best resources to prepare for this kind of round? What to expect?

Career

Hi everyone, I’m an Azure Data Engineer with over 3 years of experience specializing in building end-to-end scalable data pipelines and modern Medallion architectures. I have a strong track record of optimizing data systems and integrating emerging technologies like GenAI to drive business value.

My core stack includes:

• Cloud & Big Data: Azure (ADF, Databricks, Synapse, ADLS Gen2), Delta Lake
• Languages: Python (PySpark, Pandas), SQL (T-SQL, Spark SQL)
• BI & Visualization: Power BI (DAX, Dashboard Design)
• Specialized Expertise: ETL/ELT design, Performance Tuning (Z-Ordering, Partitioning), and GenAI-integrated automation

Key Highlights of my work:

• Advanced Architectures: Designed and implemented Medallion Architecture (Bronze, Silver, Gold) for scalable processing and real-time ingestion using Event Hubs.
• GenAI Integration: Built automated pipelines using LLMs for data enrichment, metadata generation, and classification, reducing manual effort by ~50%.
• Performance & Cost: Consistently reduced query latency by 30-40% through rigorous optimization and implemented cost-reduction strategies like resource scheduling and auto-shutdown.
• Full Lifecycle Ownership: Experienced in managing the entire data lifecycle, from ingestion and validation to CI/CD and production monitoring.

I am currently based in Hyderabad and have a 30-day notice period.

If you’re hiring or know of any relevant opportunities where I could help scale your data infrastructure, please feel free to DM me. I’d love to share my full resume and discuss how I can contribute to your team!

Thanks! 🙌

I've been working in Data field for four years now, touching a little bit of everything in the Microsoft stack (Azure, Fabric, Power BI, Databricks...). I'm thinking about starting a YT channel talking about a little bit of everything in the field: tutorials, news...

It would help me to learn new things and strengthen my current knowledge. Also, it would force me to speak English, which is not my first language. However, I'm not sure what kind of content can be more interesting to the audience.

What are your thoughts? What kind of videos you would find useful / interesting? Do you think it would have an audience or It would be something just my friends would watch?

Hi Everyone, Currently I am working in the frontend domain with 1.8 year experience and I am not good

in frontend skills

I want to go into Data Engineer skills but because of no interest in this field what should I do ?

I want to become a Data Engineer, I was thinking that only understanding business problems and performing the ETL process on data is not a cool thing

please help to become a Data Engineer

Job Title: Data Modeler (GCP & Analytics) FHIR Datasets

Job Location: United States (Remote)

Job duration: 2 years (C2C or 1099)

Experience required: 6 years+

Max Pay: $60/Hr on C2C

Role Overview:

The Data Modeler will be the primary architect of our flattened FHIR dataset. You will be responsible for transforming highly nested, hierarchical data into high-performance, analytic-friendly schemas within Google Cloud Platform (GCP). Your goal is to create a reusable framework that aligns with downstream analytics and specific clinical use cases.

Key Responsibilities:

Schema Design: Own the end-to-end design of flattened

schemas optimized for Big Query and downstream "serve views."

Flattening Strategy: Develop mapping patterns that

transform nested FHIR structures into relational or semi-relational models without losing data integrity.

Framework Development: Establish data modeling

standards and reusable patterns to ensure scalability across multiple clinical domains.

GCP Alignment: Partner with engineering teams to ensure the data model supports high-concurrency analytics and reporting tools.

Qualifications:

Expertise in data modeling for GCP (Big Query, Dataflow, etc.).

Experience transforming No-SQL/Nested structures (JSON, FHIR) into flattened schemas.

Strong understanding of healthcare data architectures and normalization strategies.

Education

Bachelors or Masters in Information Technology, Computer Science or relevant field.

reach me out:

https://www.linkedin.com/posts/kartheek-jegan\\\_hi-connections-hope-you-are-doing-good-please-share-7445494030387548160-Q8Ql?utm\\\_source=share&utm\\\_medium=member\\\_android&rcm=ACoAAE9bZMYB4Bov4sDO7L9VW1ZmYDOfwmBpQYU