Senior Data Engineer

Design, build, maintain, and document scalable data pipelines and ingestion frameworks. Develop and optimize batch and real-time data processing solutions. Implement and support modern data lakehouse architectures using Delta Lake and cloud-native technologies. Build and manage data streaming solutions using Kafka, Spark Streaming, and Change Data Capture (CDC) workflows. Ensure high standards of data quality, governance, security, and reliability across data platforms. Collaborate closely with Data Scientists, Machine Learning Engineers, BI teams, and Platform Engineering teams to deliver data solutions. Optimize data workflows for performance, scalability, and cost efficiency. Lead technical design discussions and contribute to architectural decisions. Mentor junior and mid-level engineers, providing technical guidance and fostering team growth. Promote data engineering best practices, coding standards, and operational excellence. Requirements 5–8+ years of experience in Data Engineering or a related field. Strong expertise in cloud-based data platforms, particularly Databricks. Deep understanding of Data Lake and Lakehouse architectures, including Delta Lake. Hands-on experience with real-time data streaming technologies such as Kafka, Apache Spark, and CDC frameworks. Strong programming skills in Python, PySpark, and PL/SQL.Experience with containerization and orchestration technologies including Docker and Kubernetes. Hands-on experience developing and managing Apache Airflow DAGs. Proficiency with Git and modern software development practices. Strong analytical, problem-solving, and troubleshooting skills. Excellent communication and stakeholder management abilities.