Irth Solutions Careers: Data Engineer (Multi-Cloud Lakehouse)

Irth Solutions is a software product company building cutting-edge technology platforms that continuously set industry benchmarks. We thrive on a strong product culture, a collaborative spirit, and high growth potential.

Currently, we are embarking on a major initiative: building a modern, multi-cloud, enterprise-grade data estate. We are using Databricks to centralise and govern data across AWS, Azure, and GCP. We are looking for a hands-on Data Engineer (3–5 years experience) to help us implement and operationalise this unified Lakehouse platform.

The Role: Bridging Clouds with Databricks

This is an ideal role for an engineer looking to expand their expertise in Delta Lake, Spark, and modern lakehouse architecture. Working closely with our Senior Data Architect, you will play a critical role in building reliable, secure, and high-performance data pipelines.

Key Responsibilities:

• Data Pipeline Development: Build batch and streaming ingestion pipelines across three clouds using PySpark, SQL, Delta Live Tables, and Lakeflow.

• Medallion Architecture: Implement structured data transformation patterns, CDC, SCD Type 1/2, and schema evolution.

• Performance Optimisation: Manage Delta Lake storage and optimise table performance using Z-ORDER, OPTIMIZE, and VACUUM.

• Unified Governance: Support metadata management, lineage tracking, and cataloguing through Unity Catalogue.

• Automation & DevOps: Manage workflows via Databricks Workflows or ADF, and support CI/CD pipelines for deployment automation.

• Security & Compliance: Apply RBAC, encryption, and data classification to ensure the estate remains secure and compliant.

Your Profile: The Ambitious Data Engineer

We are looking for someone who has a solid foundation in ETL and is ready to tackle the challenges of a multi-cloud environment.

• Experience: 3–5 years in Data Engineering, ETL, or Cloud Data Platforms.

• Core Skills: * Strong proficiency in PySpark, SQL, and Spark.

  • Experience with at least one cloud platform (Azure preferred; AWS/GCP acceptable).

  • Understanding of security practices (RBAC, encryption).

• Technical Mindset: Knowledge of data modelling, schema evolution, and pipeline troubleshooting.

• Preferred Assets: Familiarity with Unity Catalogue, medallion architecture, or Databricks certifications (DP-203 / Data Engineer Associate).

 Why Join Irth Solutions?

• Cutting-Edge Tech: Work with the latest Databricks features like Delta Live Tables and Lakeflow.

• Impact & Scale: Be an integral part of a dynamic, growing company that is an industry benchmark setter.

• Mentorship: Collaborate directly with a Senior Data Architect to implement sophisticated architecture designs.

• Competitive Rewards: Receive competitive pay based on your experience and a chance to grow as the platform evolves.

 How to Apply

Ready to help us build a unified data estate across the major cloud providers?

1. Highlight Your Spark Experience: Share how you’ve optimised pipelines in the past.

2. Detail Your Cloud Familiarity: Whether you are an Azure expert or an AWS enthusiast, let us know how you handle cloud storage.

[Apply Now: Data Engineer at Irth Solutions]