Azure Data Engineer

Azure Data Engineer

Buchanan Ingersoll & Rooney is a national law firm with a proven reputation for providing progressive, industry-leading legal, business, regulatory, and government relations advice to our regional, national, and international clients.

We are currently seeking an Azure Data Engineer with strong experience in Databricks to support our IT Department in Pittsburgh. The Azure Data Engineer will be responsible for designing, developing, and implementing data solutions on the Firm's Azure Cloud platform, with a particular emphasis on leveraging Databricks for data processing and analytics. This role is crucial for architecting and building scalable, high-performance data pipelines that ensure efficient extraction, transformation, and loading of data from various sources into the Firm's Azure enterprise cloud data storage solutions.

The ideal candidate will possess robust data engineering and database architecture skills, proficiency in programming languages, and a deep understanding of Azure data services, particularly Databricks, to design, develop, and implement scalable data solutions that drive actionable insights and support informed decision-making within the organization.

Duties and Responsibilities:

• Automate data pipeline monitoring, error handling, and performance tuning to ensure data integrity and availability, collaborating with the Digital Data Solutions Team or external consultants as needed.
• Design, develop, and implement scalable and high-performance data solutions on the Azure platform, specifically architecting data pipelines and ETL/ELT processes utilizing Azure Data Factory and Azure Databricks to extract, transform, and load data from various sources into Azure data storage solutions.
• Optimize data storage and retrieval mechanisms to ensure efficient processing of large datasets, particularly focusing on the capabilities of Databricks.
• Develop and maintain data models and schemas to support data analysis and reporting requirements.
• Implement data security and privacy measures to protect sensitive data.
• Develop and maintain data ingestion processes to acquire data from various sources, including APIs, databases, and streaming platforms.
• Implement data quality checks and validation procedures to ensure the accuracy and integrity of data.
• Design and implement data transformation and enrichment processes using Azure Databricks, Azure Data Factory, and other relevant tools.
• Document technical design, deployment processes, and best practices for data engineering solutions.
• Implement data partitioning, indexing, and optimization techniques to improve query performance.
• Collaborate with data scientists and analysts to operationalize and deploy machine learning models on Azure, utilizing Databricks for model training and deployment.
• Configure and manage data storage options, including Azure Blob Storage, Azure Data Lake Storage, and Databricks LakeHouse.
• Collaborate with DevOps teams to deploy and manage data engineering solutions in Azure.

Required Education/Experience:

• Bachelor's degree in Computer Science, Engineering, or a related field.
• Azure certifications, such as Microsoft Certified: Enterprise Data Engineer Associate, are a plus.
• Preferred hands-on experience with Azure data services, including Azure Data Lake Storage, Azure SQL Database, Azure Data Factory, and particularly Azure Databricks and Azure Synapse Data Warehouse.
• Strong experience in data engineering and architecture, with a focus on Azure cloud technologies.
• Proficiency in programming languages such as Python, SQL, and PowerShell.
• Knowledge of data integration techniques and tools, such as ETL/ELT processes, data pipelines, and data warehousing concepts.
• Familiarity with data modeling and schema design principles.
• Understanding of data security and privacy regulations, including GDPR and CCPA.
• Excellent problem-solving and analytical skills.
• Strong communication and collaboration abilities.

Why Work at Buchanan?

Our Firm offers outstanding benefits that include:

• Hybrid work schedules
• Generous Paid Time Off
• Paid Holidays, including a floating holiday
• WorkWell wellness program, including free use of the Calm App
• Free use of building gym
• Caregiving assistance with Bright Horizons (child, elder, and pet care!)
• Access to our Firm-wide emergency assistance fund
• Free full access to LinkedIn Learning
• Insurance – Medical, Dental, Vision
• 401K Program
• Retirement Savings Program

We are an Equal Opportunity Employer.

Equal Opportunity Employer/Protected Veterans/Individuals with Disabilities
This employer is required to notify all applicants of their rights pursuant to federal employment laws. For further information, please review the Know Your Rights notice from the Department of Labor.