The Data Integration Engineer is responsible for designing, developing, and maintaining data integration
solutions that enable the seamless flow of information across the organization. This role focuses on
extracting, transforming, and loading (ETL/ELT) data from multiple source systems into centralised data
platforms while ensuring data quality, consistency, and reliability.
Key Responsibilities
- Integration Development: Design, build, and deploy end-to-end data pipelines that move data
bi-directionally between various customer ERPs and the John Galt software package. - Requirement Gathering: Act as a technical liaison by meeting with customers to understand
their data structures, ERP capabilities, and specific integration needs. - Standardization: Develop repeatable, scalable integration patterns that fit into our standard
system architecture while remaining flexible enough to handle diverse customer requirements. - Cloud-Native Execution: Deploy and manage data processing jobs and integration workflows
within a Kubernetes-orchestrated infrastructure. - Collaboration: Work closely with internal implementation consultants and software engineers
to ensure data integrity and alignment with our core product roadmap. - Problem Solving: Troubleshoot and resolve complex data mapping and synchronization issues
arising from different data formats and source systems.
Technical Requirements
- SQL Expertise: Advanced proficiency in SQL Server (writing complex queries, stored
procedures, and optimizing data transformations). - Python Programming: Strong experience using Python for data manipulation, automation,
and building integration scripts. - API Integration: Proven experience working with RESTful APIs to extract and push data.
- Data Formats: Deep understanding of semi-structured data formats, specifically JSON and
XML. - Containerization: Experience running and managing jobs/workloads within Kubernetes (K8s)
environments. - ETL/ELT Logic: Solid understanding of staging database concepts and the lifecycle of data
moving from source to target.
Preferred Qualifications (The "Plus" List)
- Experience with Talend or similar ETL integration tools.
- Familiarity with Supply Chain management processes or ERP data structures (e.g., SAP, Oracle,
Microsoft Dynamics). - Experience in a customer-facing technical role.