Role Value Proposition:
The position sits within the newly consolidated Data and Analytics (D&A) organization supporting the U.S. Business of MetLife. U.S. D&A assists all business lines of MetLife's U.S. business (about 2/3 of MetLife Global by earnings) with everything related to data, analytics and data science, from data infrastructure, data governance, data engineering, data modeling, data analysis to business intelligence, data science, and AI.
The Lead Data Engineer is crucial to DnA USB's Engagement Strategy team for creating and optimizing data architecture, solutions, and operations, and for ensuring alignment with the data management and governance framework. The Lead Data Engineer serves as a big data development expert within the DnA Engagement Strategy organization.
This position is responsible for leading, architecting, and building ETL, data warehousing, and reusable components using cutting-edge big data and cloud technologies. The resource will collaborate with the architect, business systems analyst, technical leads, project managers, and business/operations teams in building data enablement solutions across different LOBs and use cases.
Key Responsibilities:
Design and solution end-to-end data architecture for data hubs/data products, all the way from source systems to consumption
Oversee the design and management of data solutions to ensure data is stored, processed, curated, and utilized effectively
Own and build a reusable data pipeline utilizing Azure/Databricks for Engagement Program data products.
Ingesting huge volumes of data from various platforms for Analytics needs and writing high-performance, reliable, and maintainable ETL code.
Leadership: Lead and mentor a team of data engineers, ensuring the efficient flow of data within the organization with the defined processes and tools.
Collect, store, process, and analyze large datasets to build and implement extract, transfer, load (ETL) processes
Develop reusable frameworks to reduce the development effort involved, thereby ensuring cost savings for the projects.
Utilizing CI/CD Pipelines: Utilize and enhance CI/CD practices to automate the delivery of data solutions, ensuring reliability and scalability based on the defined tools.
Utilize Cloud technologies (Azure Databricks) to enable data product solutions.
Develop quality code through performance optimizations in place right at the development stage.
Appetite to learn new technologies and be ready to work on new cutting-edge cloud technologies.
Partner with Marketing, Marketing Tech, Business, BI, and Data Science teams to create reusable data products
Work with a team spread across the globe in driving the delivery of projects and recommend development and performance improvements.
Track and report on KPIs for solution delivery and data quality.
Communicate and present use cases, solutions, and impact to business stakeholders and mid/senior management.
Optimize reusable frameworks, Spark jobs for performance and cost efficiency in large-scale environments.
Ability to interact with business analysts and functional analysts in getting the requirements and implementing ETL solutions.
Essential Business Experience and Technical Skills:
Required:
8+ years of Data solutions, development, and delivery experience with 4+ years of recent experience in Azure/Databricks environments.
Proficiency and extensive experience with SQL, Spark &/or Scala/Python and performance tuning
Hands-on expertise in: Azure SQL Synapse, Azure Databricks, and/or Data Factory experience is a MUST.
Strong experience in building/designing Data warehouses, data stores for analytics consumption on Cloud (real-time as well as batch use cases)
Building and implementing a data ingestion and curation process developed using Cloud data tools such as Azure SQL Synapse, Data Factory, Spark (Scala/Python), Data Bricks, Delta Lake, etc.
Good scripting experience, primarily on shell/bash/ PowerShell.
Strong SQL knowledge and data analysis skills for data anomaly detection and data quality assurance.
Experience implementing data governance and data quality using the enterprise toolset.
Databricks certifications and/ or Microsoft Azure Certifications
Very good problem solver and excellent communication skills - both written and verbal
Preferred:
Expertise in Python and experience writing Azure functions using Python/Node.js.
Experience using Event Hub for data integrations.
Eagerness to learn new technologies on the fly and ship to production.
Hive database management and Performance tuning - Partitioning / Bucketing.
Very good problem solver and excellent communication skills - both written and verbal
At MetLife, we're leading the global transformation of an industry we've long defined. United in purpose, diverse in perspective, we're dedicated to making a difference in the lives of our customers.
Equal Employment Opportunity/Disability/Veterans
If you need an accommodation due to a disability, please email us at accommodations@metlife.com. This information will be held in confidence and used only to determine an appropriate accommodation for the application process.
MetLife maintains a drug-free workplace.
