Principal Data Engineer - Databricks

4w4 weeks ago

Bede Gaming

Sofia, BG · Full-time · €75,000 – €95,000

About this role

This is a hands-on technical leadership role for tackling the hardest data engineering challenges, shaping technical direction, and raising the bar for reliability, scalability, and engineering quality across our data estate. The Principal Data Engineer combines deep technical delivery with cross-squad influence, working directly on complex platform and pipeline problems while improving engineering standards.

Day to day, you will lead complex implementations across both our legacy Pentaho ETL estate and modern Databricks lakehouse platform, solving difficult production incidents and performance bottlenecks. You will design and build scalable batch and streaming pipelines, contribute production-grade Python and SQL code, and drive root-cause analysis for systemic weaknesses.

You will collaborate closely with Data Engineers, Platform Engineering, SRE, Architecture, Governance, Product teams, and engineering leadership. A key part of the role is mentoring other engineers, improving developer experience through reusable frameworks and CI/CD patterns, and promoting safe adoption of AI-assisted engineering approaches.

This role is ideal for someone who enjoys staying close to the code while making a visible impact across platform evolution and day-to-day delivery. You will shape engineering patterns that make delivery safer, faster, and more scalable, and help raise the quality bar for an entire tribe.

Requirements

Strong hands-on experience designing, building, operating, and optimizing enterprise-scale data platforms and pipelines
Experience with Pentaho ETL or similar technologies in large-scale production environments
Strong hands-on experience with Databricks lakehouse platform
Extensive Python and SQL coding skills for production-grade pipelines
Experience with batch and streaming data pipeline architectures
Proven ability to troubleshoot complex performance bottlenecks and data integrity issues
Experience in observability, monitoring, and operational tooling for data estates

Responsibilities

Lead and implement complex engineering work across critical data pipelines and platform capabilities
Lead optimization and stabilization efforts across Pentaho ETL and Databricks lakehouse ecosystems
Improve Spark workload efficiency, cluster utilization, query performance, storage optimization, orchestration reliability, and platform scalability
Design, build, and operate production services, APIs, and integration layers that expose new data capabilities into products and workflows
Drive root-cause analysis and remediation for systemic engineering and operational weaknesses
Define and implement observability standards for critical pipelines and datasets
Support data-heavy and AI-enabled product initiatives including recommendation engines, behavioral scoring, propensity modeling, and intelligent automation
Develop squad engineering leads through mentorship, coaching, technical guidance, and hands-on support