SKM Group
Lead Data Engineer with Crypto
SKM Group
€52k - €120k
Europe (Remote)
Apply Now ->
Python
Spark
SQL
Data

Lead Data Engineer with Crypto

Overview

We are a software house with a 18-year history, a rich portfolio, projects all over the world and an appetite for more. We have built our brand on professionalism and flexibility in delivering software solutions. We are not afraid of unconventional ideas and value innovation and imaginative change.

Job Description

A large, global data infrastructure platform needs to be comprehensively migrated to Databricks Unity Catalog, including a transition to DBXaaS. This migration involves the transfer of tables, pipelines, permissions, and related components across all relevant environments and assets. Our goal is to ensure data security, manageability, and scalability while maintaining consistency and governance across the platform.

Responsibilities

  • - Migrate data infrastructure to Databricks Unity Catalog and DBXaaS
  • - Develop, maintain, and optimize data pipelines using Spark and SQL
  • - Design and implement robust data pipelines for blockchain data processing
  • - Ensure data governance, security, and compliance across environments
  • - Optimize data architecture for scalability and performance
  • - Collaborate with cross-functional teams to meet business requirements
  • - Lead a team / pod of 3 Data Engineers, 1 DevOps Engineer and 1.5 QA

Required Skills

  • - Advanced proficiency in Python and strong software engineering fundamentals
  • - Proven experience with Spark and SQL in data engineering and analysis
  • - Hands-on experience with Databricks for data ingestion and transformation
  • - Proficiency with data orchestration tools (Airflow or similar)
  • - Experience with one or more cloud services (Azure preferred, AWS, GCP)
  • - Solid understanding of RDBMS/NoSQL data stores
  • - Experience with Docker (required) and Kubernetes (preferred)
  • - Familiarity with Delta Lake and data warehousing concepts
  • - Understanding of blockchain data structures and cryptography (preferred)
  • - Experience with blockchain indexing and data extraction (a plus)
  • - Excellent communication, interpersonal, and presentation skills
  • - Experience leading teams

Benefits

  • - Attractive salary
  • - Large freedom and real influence
  • - No unhealthy competition, team approach to meeting challenges
  • - Remote or hybrid work model
  • - Company apartments in cool cities across Europe: work and enjoy a memorable getaway
Apply Now ->

About the company

SKM Group

Experience the transformative power of Industry 4.0: automation, connectivity, and data-driven insights. We offer software solutions that meet the challenges of your business.


All Job Openings at SKM Group