Senior Data Engineer

Milano
Tempo indeterminato
Full time

20 giorni fa

About the RoleWe are looking for a Senior Data Engineer with extensive of hands-on experience in data engineering, big data processing, and cloud native pipelines. You will design, build, optimize, and maintain data pipelines on Databricks, leveraging Apache Spark, Python/PySpark, SQL, and modern cloud services.Your role1. Build and Optimize Data Pipelines

Design, develop, and maintain scalable ETL/ELT pipelines using Databricks and Apache Spark.
Ingest, transform, and process structured and unstructured data across batch and streaming workflows.
Implement data processing workflows leveraging Delta Lake for ACID reliability, schema enforcement, and incremental ingestion.

2. Platform Engineering & Performance

Optimize pipeline performance, troubleshoot and tune Spark jobs for scalability and reliability.
Monitor, maintain, and evolve existing pipelines and data models, ensuring performance and operational health.

Apply data quality techniques, validation, transformation logic, and lineage management to ensure trusted data delivery.

3. Collaboration & Cross‑functional Enablement

Partner with data scientists, analysts, architects and business stakeholders to translate business requirements into technical pipelines.

4. Automation, CI/CD, Cloud Services

Implement and maintain CI/CD pipelines for Databricks notebooks, jobs, and workflows.
Work with cloud‑native services (AWS/Azure depending on environment) for orchestration, storage, ingestion and monitoring.
Automate processes where possible to improve reliability and reduce manual operations.

Your experience

Extensive experience in data engineering, with at least 2 years working hands-on with Databricks or Spark based platforms.
Strong programming experience in Python/PySpark and solid SQL for transformations and data modeling
Experience with Delta Lake, data lake/lakehouse architectures, and modern cloud data engineering
Knowledge of CI/CD tools, Git, workflow orchestration, and DataOps best practices.
Experience with ETL/ELT design and cloud pipelines (Azure Data Factory, AWS Glue, GCP Dataflow, etc.).
Strong understanding of data warehousing, data modeling, and pipeline reliability patterns.
Experience with Unity Catalog, lineage, and governance in Databricks environments.
Exposure to streaming technologies (Kafka, Event Hubs, Kinesis).
Familiarity with DevOps tools (Docker, Kubernetes) for data engineering workflows.
Databricks certifications (Data Engineer Associate/Professional, Spark Developer).
Experience integrating machine learning workloads or MLflow.

Prysmian , as an Equal Opportunity Employer, aims to attract and recruit individuals with diverse backgrounds, skills, and abilities. We strongly believe that diversity brings significant value at all levels of the organization, increasing the possibility of capturing market opportunities and maximizing value for our customers and stakeholders. With Diversity, Equity, and Inclusion (DE&I) as part of our Social Ambition 2030 and a strategic pillar of our Company culture, Prysmian is committed to the development of an organization that prioritizes talent, where people feel respected, included, and free to fully express their potential just as they are.All Managers and HRs in Prysmian are responsible for ensuring DE&I policies are respected during the recruiting process, as well as recognizing and mitigating unconscious biases that must not influence our selection processes. All persons will be considered for employment without regard to their race, ethnicity, religion, nationality, origin, citizenship status, socio-economic status, age, sex, gender identity or expression, sexual orientation, marital status, disability, military service or veteran status, pregnancy, parental leave, medical conditions, or any other characteristic protected by applicable federal, state or local laws. Prysmian will endeavor to make a reasonable accommodation for any disclosed physical or neurological condition or disability of a qualified applicant unless the accommodation would impose an undue hardship on the operation of our business.Visit our to learn more about Prysmian's commitments.Your application data will be treated according to our Data Protection Policy. If you believe you require assistance to complete this form or to participate in an interview, please let us know.

Prysmian

Candidati