PROFILE SUMMARY
Cloud and Databricks professional with 9+ years of experience in cloud infrastructure, data engineering, and DevOps. Experienced in designing and deploying secure Databricks workspaces, implementing Unity Catalog for data governance, and managing Azure resources using Terraform and CI/CD pipelines. Proven ability to optimize performance, cost, and availability in large-scale enterprise cloud environments.
PROFESSIONAL EXPERIENCE
Lead II, Cloud Infrastructure Services
UST | May 2021 - Present- Designing, provision, configuration of Databricks workspace and clusters.
- Implemented remediation steps to secure Databricks as per Infosec/Governance standard and implemented workspace level policy.
- Creating and maintaining Azure Databricks Interactive and Job clusters and maintaining the ACL.
- Experience in provisioning Azure resources and Databricks components using Terraform, Azure DevOps CI/CD pipeline.
- Configuring access control for workspace objects, such as notebooks and folders, clusters and jobs.
- Creating secret scope, maintaining ACLs and linking Databricks to ADLS using Service Principal and storing credentials in Key Vault.
- Experience working on Databricks CLI and REST API to perform activities related to clusters, jobs, etc.
- Integration of Databricks with Azure DevOps or GitHub to move the notebooks to different environments through CI/CD pipelines.
- Designed and deployed Databricks workspaces across multiple environments (Dev, QA, Prod) with secure network architecture (VNET injection).
- Set up Unity Catalog and managed fine-grained access to data and tables across multiple catalogs.
- Configured CI/CD pipelines (Azure DevOps) for deploying notebooks in Databricks.
- Migrated legacy Hive Metastore assets to Unity Catalog with minimal downtime and ensured data lineage tracking.
- Configured and managed external locations and storage credentials to isolate environments and manage secure access.
Associate Hadoop Administrator
Clairvoyant India Pvt. Ltd | Sep 2019 - Apr 2021- Configured and managed clusters, pools, jobs, and libraries for performance and cost efficiency.
- Configured secure connections with ADLS Gen2, ensuring compliance with security standards.
- Enabled monitoring and logging with Azure Monitor & Log Analytics for proactive incident management.
- Collaborated with data engineering teams to troubleshoot job failures, optimize queries, and manage dependencies.
- Handled user provisioning, workspace governance, and RBAC implementation across environments.
- Azure Cloud Administrator Expertise in Azure Cloud Compute, Storage, Networking, Management Tools, Security, Identity & Compliance.
- Creation of ADF pipelines, interactive cluster and linking ADF with Databricks.
- Configure and monitor VNets, subnets, NSGs.
- Collaborated with DevOps teams to integrate Azure services with CI/CD pipelines.
- Configured Service Principals and Managed Identities for secure application authentication.
Software Engineer
Sunniva Encon | Jun 2015 - Dec 2018- Managed Azure resources: VMs, storage accounts, VNets, and security groups.
- Implemented centralized monitoring with Azure Monitor and Log Analytics.
- Configured RBAC, SAS tokens, Azure AD integration for secure access.
- Collaborated with data engineering teams to optimize data pipelines with Blob and ADLS Gen2.
- Designed and implemented a geo-redundant storage strategy for disaster recovery.
- Deployed Load Balancers for high-availability workloads across multiple regions.
- Applied Azure Policies to enforce governance at subscription and resource group levels.
Ankit Dagadu
Databricks DevOps Engineer (Azure)
Experience: 9 Years
TECHNICAL SKILLS
Azure Databricks Unity Catalog Azure Data Factory (ADF) Terraform Azure DevOps CI/CD Git Databricks CLI REST API Azure Monitor Log Analytics System Table Utilization VNet Subnets NSGs Private/Service Endpoints RBAC Key VaultEDUCATION
B.E (Fr. Conceicao Rodrigues College of Engineering) - 0
CERTIFICATIONS
- Databricks Cloud Architecture & System Integration Fundamentals
- Databricks Accredited Platform Administrator
- Microsoft Azure Administrator (AZ-104)
- Designing and Implementing Microsoft DevOps Solutions (AZ-400)
- Microsoft Azure Fundamentals (AZ-900)
