Public Partnerships

Data Engineer

Click Here to Apply

Job Location

Carson Meadows, NV, United States

Job Description

Key Responsibilities:

Design, build, and manage Azure Data Factory pipelines for data ingestion.

Handle both batch and near real-time data ingestion using Azure Service Bus.

Integrate structured, semi-structured, and unstructured data sources.

Utilize formats such as Parquet, Avro, JSON, etc., and manage loading into the Bronze layer.

Build and manage data engineering pipelines using Databricks (PySpark, SparkSQL).

Implement Spark-based data transformation jobs from Bronze to Silver and Gold layers.

Establish and manage Data Quality Framework using Databricks and Data Factory.

Set up data security using RBAC and ACLs to manage permissions on sensitive datasets.

Implement automation frameworks for pipeline execution using Databricks Workflows.

Monitor pipelines using Azure Monitor, set up alerts and ensure SLAs are met.

Optimize storage in ADLS Gen2 and use Azure Key Vault to secure credentials.

Optimize costs using Azure Cost Management.

Technical Skills:

Azure Data Factory for ETL/ELT operations.

Databricks (PySpark, SparkSQL) for data transformations.

Azure Data Lake Storage (ADLS Gen2).

SQL Server, PostgreSQL, Cosmos DB, CRM, and ERP systems.

Data Governance, RBAC, and ACLs for managing permissions.

CI/CD tools: Azure DevOps, GitHub Actions, Terraform.

Azure Monitoring, Alerting, and Logging tools.

Public Partnerships is an Equal Opportunity Employer dedicated to celebrating diversity and intentionally creating a culture of inclusion. We believe that we work best when our employees feel empowered and accepted, and that starts by honoring each of our unique life experiences. At PPL, all aspects of employment regarding recruitment, hiring, training, promotion, compensation, benefits, transfers, layoffs, return from layoff, company-sponsored training, education, and social and recreational programs are based on merit, business needs, job requirements, and individual qualifications. We do not discriminate on the basis of race, color, religion or belief, national, social, or ethnic origin, sex, gender identity and/or expression, age, physical, mental, or sensory disability, sexual orientation, marital, civil union, or domestic partnership status, past or present military service, citizenship status, family medical history or genetic information, family or parental status, or any other status protected under federal, state, or local law. PPL will not tolerate discrimination or harassment based on any of these characteristics.

PPL does not discriminate based on race, color, religion, or belief, national, social, or ethnic origin, sex, gender identity and/or expression, age, physical, mental, or sensory disability, sexual orientation, marital, civil union, or domestic partnership status, protected veteran status, citizenship status, family medical history or genetic information, family or parental status, or any other status protected under federal, state, or local law.



Location: Carson Meadows, NV, US

Posted Date: 10/31/2024
Click Here to Apply
View More Public Partnerships Jobs

Contact Information

Contact Human Resources
Public Partnerships

Posted

October 31, 2024
UID: 4918898087

AboutJobs.com does not guarantee the validity or accuracy of the job information posted in this database. It is the job seeker's responsibility to independently review all posting companies, contracts and job offers.