Databricks Engineer Job at Programmers.io, Remote

  • Programmers.io
  • Remote

Job Description

Technical/Functional Skills 5 6 years of total experience in data engineering or big data development. 2 3 years hands-on experience with Databricks and Apache Spark. Proficient in AWS cloud services (S3, Glue, Lambda, EMR, Redshift, CloudWatch, IAM). Strong programming skills in PySpark, Python, and optionally Scala. Solid understanding of data lakes, lakehouses, and Delta Lake concepts. Experience in SQL development and performance tuning. Familiarity with Airflow, dbt, or similar orchestration tools is a plus. Experience in CI/CD tools like Jenkins, GitHub Actions, or Code Pipeline. Knowledge of data security, governance, and compliance frameworks.

Experience Required 5-6 years+

Roles & Responsibilities Develop and maintain scalable data pipelines using Apache Spark on Databricks. Build end-to-end ETL/ELT pipelines on AWS using services like S3, Glue, Lambda, EMR, and Step Functions. Collaborate with data scientists, analysts, and business stakeholders to deliver high-quality data solutions. Design and implement data models, schemas, and Lakehouse architecture in Databricks. Optimize and tune Spark jobs for performance and cost-efficiency. Integrate data from multiple structured and unstructured data sources. Monitor and manage data workflows, ensuring data quality, consistency, and security. Follow best practices in CI/CD, code versioning (Git), and DevOps practices for data applications. Write clean, reusable, well-documented code using Python / PySpark / Scala.

Job Tags

Similar Jobs

Cornell University

Diagnostic Technologist - Virology Job at Cornell University

 ...Diagnostic Technologist I The Cornell University College of Veterinary Medicine seeks a Diagnostic Technologist I to join the Animal Health Diagnostic Center's Virology Laboratory. In this role, the selected candidate will perform specialized laboratory testing to... 

Quantum Education Professionals

Sign Language Interpreter Job at Quantum Education Professionals

 ...Registered with the South Carolina Department of Labor, Licensing and Regulation (LLR). # Fluency in English and American Sign Language (ASL) Adherence to the Registry of Interpreters for the Deaf and the Educational Interpreter Performance Assessment Codes of Professional... 

Christus Health

Registered Nurse - Labor and Delivery Job at Christus Health

 ...Description Summary: The competent Nurse, in the same or similar clinical setting, practices independently and demonstrates an awareness of all relevant aspects of a situation. Provides routine and complex care, with the ability to on long-range goals or plans. Continues... 

ALO

Internal Investigator Job at ALO

 ...inventory/RFID systems. ~ Experience supporting civil recovery, restitution, or criminal prosecution workflows. ~ Professional certifications such as CFI, LPC, CFE, or related investigative credentials. For CA residents, Job Applicant Privacy Policy HERE.... 

Cardinal Health

Delivery Driver - Pharmacy Services Job at Cardinal Health

 ...Cardinal Health - JobID: 273f1047-c81e-45e4-adf7-279f293ebee5 [Package Delivery Driver / Flex Driver] As a Delivery Driver at Cardinal Health, you'll: Prepare and make deliveries to customer accounts on assigned route using a company vehicle (non-cdl); Package radiopharmaceutical...