Big Data Platform Engineer

Experience Requirements

● 8+ years of overall IT experience ● 5+ years of experience in Cloud Infrastructure Engineering ● 4+ years of experience working with Kubernetes platforms ● 3+ years of experience supporting enterprise-scale Data Platforms ● 3+ years of experience in DevOps/SRE practices ● 2+ years of experience working with Azure Data Services

Mandatory Skills

Candidates must possess strong hands-on experience in the following areas: ● Apache NiFi ● Azure Event Hub ● Azure Functions ● Enterprise-scale Data Platform Administration These are core and non-negotiable requirements. Profiles without hands-on experience in Apache NiFi and Azure Event Hub will not be considered.

Requirements

1. Azure Kubernetes Service – AKS

Strong Expertise

● AKS cluster deployment and administration ● Cluster upgrades and patch management ● Node pool management ● Ingress Controllers, including NGINX and Application Gateway ● RBAC implementation ● AKS security hardening ● Autoscaling using HPA, VPA, and Cluster Autoscaler ● Persistent storage management ● Troubleshooting cluster issues ● Network policies

Experience

● Production support of AKS environments ● Multi-node and multi-environment AKS setups ● High availability and disaster recovery implementation

2. Kubernetes and Azure Networking

Kubernetes Networking ● Kubernetes networking concepts ● Services, Ingress, and Load Balancers ● DNS resolution ● Pod-to-Pod communication ● Pod-to-Service communication ● Network Policies ● Service Mesh concepts, preferably Istio ● CNI Architecture

Azure Networking

● Virtual Networks ● Subnets ● Private Endpoints ● Peering ● Network Security Groups ● User Defined Routes ● Azure Firewall ● VPN connectivity

3. Docker and Containerization

● Docker image creation and optimization ● Container lifecycle management ● Registry management ● Security scanning ● Multi-stage builds ● Container troubleshooting ● Azure Container Registry ● Image versioning and governance

4. Azure DevOps and CI/CD

CI/CD Expertise ● Azure DevOps Pipelines ● YAML pipelines ● Build and Release pipelines ● Git repositories ● Branching strategies

Infrastructure Deployment

● Infrastructure as Code ● ARM Templates ● Bicep ● Terraform

Automation

● Deployment automation ● Environment promotion ● Release management

5. Monitoring and Observability

Azure Monitor

● Monitoring strategy implementation ● Alert configuration ● Dashboard creation

Log Analytics

● KQL query development ● Performance analysis ● Troubleshooting