Middle Big Data Engineer

We are seeking a proactive Middle Big Data Engineer to join our team. In this role, you will be responsible for designing, developing, and maintaining scalable data pipelines and reporting-ready data assets within Palantir Foundry. The ideal candidate will possess a robust background in cloud technologies, data architecture, and a passion for solving complex data challenges.

You will collaborate with a cross-functional team and stakeholders to translate business requirements into scalable data solutions, enabling reporting, improving data quality, and driving data transformation initiatives.

About the Client:

Our client is a leading global healthcare organization, driving innovation in pharmaceuticals, biotechnology, and patient-centric solutions. They develop advanced data and analytics solutions that help global organizations turn complex data into intuitive digital tools, enabling teams to make informed decisions in their daily operations and long-term strategy. Their projects span modern operational applications, interactive dashboards, and advanced analytics platforms designed to deliver actionable insights across multiple business domains.

Key Responsibilities:

• Collaborate with cross-functional teams to gather business requirements and design, implement, and maintain scalable data pipelines in Palantir Foundry, ensuring end-to-end data integrity and optimized workflows
• Design and maintain data ingestion, transformation, and orchestration pipelines using Palantir Foundry, Python, and PySpark
• Develop reporting-ready datasets and data models to support downstream analytics and Power BI reporting
• Develop, optimize, and maintain efficient ETL/ELT processes to collect, process, and integrate data from multiple sources, ensuring timely and accurate data delivery
• Implement data quality validation, monitoring, and health checks to ensure reliability and consistency of data assets
• Monitor pipeline performance, identify bottlenecks, troubleshoot production issues, and continuously improve scalability and processing efficiency
• Plan and prioritize work independently, communicate progress transparently, and proactively manage changing priorities while proposing practical solutions when requirements evolve
• Maintain clear technical documentation and promote automation and continuous improvement across data engineering processes
• Stay current with emerging technologies and industry best practices, incorporating innovative approaches into data engineering solutions

Required Qualifications:

• 3+ years of experience in data engineering, preferably within the pharmaceutical or life sciences industry
• Strong proficiency in Python and PySpark
• Proficiency with big data technologies (e.g., Apache Hadoop, Spark, Kafka, BigQuery, etc.)
• Hands-on experience with cloud services (e.g., AWS Glue, Azure Data Factory, Google Cloud Dataflow)
• Expertise in data modeling, data warehousing, and ETL/ELT concepts
• Hands-on experience with database systems (e.g., PostgreSQL, MySQL, NoSQL, etc.)
• Proficiency in containerization technologies (e.g., Docker, Kubernetes)
• Effective problem-solving and analytical skills, coupled with excellent communication and collaboration abilities

Nice to have:

• Experience with market research data
• Familiarity with Veeva CRM, Reltio, SAP, and/or Palantir Foundry
• Background in pharma or healthcare
• Experience with automation / pipeline optimization