Senior Data Engineer
We are looking for an experienced Senior Data Engineer to join our team. The candidate will be a key member of our data engineering team, responsible for developing and optimizing scalable data pipelines, ensuring data integrity, and providing a robust infrastructure that serves as the foundation for all data-driven activities.
We are expecting from Senior Data Engineer:
Design, develop, and maintain data pipelines to extract, transform, and load data from various sources into a centralized data store.
Create and maintain scalable data infrastructure, including databases, data warehouses, and other storage and processing systems.
Ensure data pipelines and infrastructure are optimized for efficiency and can handle large volumes of data without compromising performance.
Enforce data validation, verification processes, and security measures like encryption and access controls to ensure the accuracy, completeness, and security of data.
Work closely with data scientists, analysts, and business stakeholders to understand data needs and develop solutions to support their analytical and operational needs.
Provide technical guidance and mentorship to junior data engineers, fostering a collaborative learning environment.
Create comprehensive documentation for data infrastructure and processes, ensuring they are understandable and maintainable by other team members.
Continuously explore emerging technologies and best practices in data engineering to keep the team and the organization at the cutting edge.
Technical Skills:
Strong hands-on experience with Azure Databricks.
Proficiency in SQL for data extraction and manipulation.
Programming experience in Python.
Strong problem-solving and analytical skills with a proven ability to optimize data architecture.
Experience in developing solutions that support high-volume data processing and real-time analytics.
Excellent communication and collaboration skills, capable of working with cross-functional teams in a dynamic environment.
Ability to translate complex data engineering concepts into actionable insights for stakeholders.
Education:
BE/BTech in Computer Science, Data Engineering, or a related field, or equivalent experience.
