Data Engineer (Agribusiness Software Solutions) Mexico Company Background Our client is a company that delivers integrated software and tools for the agricultural business. As part of a leading global software group, the company supports the diversified grain industry and agricultural co-ops with best‑in‑class solutions covering business management, commodity management, agronomy, trading, patronage, and analytics. Project Description Own the end‑to‑end data flow for Data Analytics Platform (DAP). You will ingest XML data from AGRIS Web Services into Azure Blob Storage, orchestrate event‑driven transformation pipelines via Azure Functions and Databricks Jobs, model data using dbt on Databricks (medallion architecture: bronze → silver → gold), and deliver analytics‑ready datasets to Azure SQL Database and Luzmo. You'll also maintain multi‑tenant data isolation, automate deployments, and keep the platform performant and cost‑efficient. Technologies Databricks (Delta Lake, PySpark, SQL Warehouses) dbt (dbt-databricks + dbt-sqlserver) Azure Functions (Python) What You’ll Do Maintain and support Azure Functions pipelines (Event Grid → Databricks Jobs) Build and optimize Databricks notebooks for XML parsing and Parquet landing in DBFS/ADLS Manage Auto Loader and incremental data ingestion processes Design and maintain dbt models across medallion layers (bronze COPY INTO, silver current‑changes, gold incremental merge, staging/CDC to Azure SQL) Write custom macros and ensue data quality with dbt tests Optimize schemas, indexes, and CDC merge pipelines in Azure SQL Database Manage analytical views, RBAC roles, and Luzmo integration for BI consumers Maintain tenant‑scoped DBFS mounts and data isolation Configure backup, storage tiering, and define RPO/RTO targets, monitor pipeline health Implement monitoring, alerting, and optimize infrastructure and compute costs in Azure and Databricks Job Requirements 5+ years designing and building data solutions on Microsoft Azure 2+ years with Databricks (Delta Lake, PySpark, SQL Warehouses) Strong experience with dbt (data build tool): incremental models, custom macros, multi‑adapter setups (Databricks + SQL Server) Expert‑level SQL skills: Databricks SQL for Delta Lake transformationsand T‑SQL for Azure SQL Database performance tuning Hands‑on experience with Azure Functions (Python) and event‑driven architectures (Azure Event Grid, Blob Storage triggers) Familiarity with medallion architecture patterns (bronze/silver/gold) and CDC (Change Data Feed) pipelines Working knowledge of XML data parsing/ingestion at scale (PySpark XML processing) Strong understanding of Azure security fundamentals (Key Vault, RBAC, managed identities) Experience with multi‑tenant data platform design and tenant‑scoped data isolation English level: B1+ (Intermediate) or higher Nice to Have Experience with IaC (Bicep or Terraform) and CI/CD (GitHub Actions) Familiarity with BI platforms, particularly Luzmo, for provisioning datasets and self‑service reporting Knowledge of Python for Databricks notebooks, Azure Functions, and utility scripts Experience with cost‑governance / FinOps in Azure and Databricks Exposure to SQLFluff or similar SQL linting/quality tools What Do We Offer The global benefits package includes: Technical and non‑technical training for professional and personal growth Internal conferences and meetups to learn from industry experts Support and mentorship from an experienced employee to help you professional grow and development Health insurance Sports activities to promote a healthy lifestyle Flexible work options, including remote and hybrid opportunities Referral program for bringing in new talent Work anniversary program and additional vacation days #J-18808-Ljbffr