Responsibilities: • Lead the architecture, design, and implementation of Databricks-based data platforms, pipelines, and AI/ML workloads. • Define scalable lakehouse patterns using core Databricks capabilities such as Apache Spark, Delta Lake, Databricks SQL, MLflow, and governance components. • Design robust batch and streaming data pipelines for high-volume, business-critical workloads. • Establish architecture standards for performance optimization, workload orchestration, reliability, observability, and cost control. • Translate business and technical requirements into solution blueprints, reference architectures, and implementation roadmaps. • Collaborate with data engineers, analysts, data scientists, DevOps teams, and business stakeholders to deliver end-to-end solutions. • Ensure strong data governance, security, and compliance practices across the Databricks environment. • Support AI and advanced analytics use cases by enabling reliable feature engineering, model lifecycle practices, and production-ready data foundations. • Provide technical leadership during solution delivery, troubleshooting, and optimization of existing workloads. • Act as a trusted advisor to client stakeholders, offering recommendations on architecture decisions, risks, trade-offs, and delivery priorities. Mandatory Skills Description: • Deep hands-on expertise with the Databricks platform, including architecture, workspace design, cluster strategy, jobs orchestration, and platform optimization. • Strong command of Apache Spark and distributed data processing concepts, including performance tuning and optimization for complex data engineering workloads. • Proven experience designing and delivering enterprise data solutions using Delta Lake and lakehouse architecture principles. • Strong proficiency in Python and SQL; Scala is an advantage. • Demonstrated capability in building and optimizing ETL/ELT pipelines, data models, and large-scale ingestion frameworks. • Experience supporting AI/ML workloads on Databricks, including MLflow, model lifecycle considerations, and production-ready data preparation. • Solid knowledge of cloud-native architecture patterns on AWS, Azure, or GCP, including storage, networking, identity, and security integration. • Experience with data governance, access control, lineage, and compliance frameworks in enterprise environments. • Familiarity with CI/CD, infrastructure-as-code, monitoring, and operational best practices for data platforms. • Ability to engage in deep technical problem-solving and contribute immediately to complex delivery scenarios with minimal ramp-up time. Nice-to-Have Skills Description: • Databricks certifications relevant to data engineering, machine learning, or platform architecture. • Experience in large enterprise or global delivery environments with complex stakeholder landscapes. • Background in consumer goods, manufacturing, supply chain, or similarly data-intensive industries. • Experience with real-time processing, orchestration frameworks, and integration with enterprise data ecosystems. • Strong communication skills with the ability to explain architecture decisions to both technical and non-technical stakeholders.