Tech Lead/Manager, K8s

Description

• Lead a team of SREs responsible for Kubernetes infrastructure to achieve their personal and shared goals, thrive in their roles, as well as spearheading the engineering efforts by your own example.  
• Ensure maximum availability, reliability, and scalability of our multi-datacenter hybrid Linux environments (10 clusters, 1000+ nodes, 20k+ pods, 1mil+ qps).  
• Performance and resilience testing. This may include reviewing configuration, software choices/versions, hardware specs, etc.  
• Advance our technology stack with innovative ideas and new creative solutions.  
• Participate in capacity management of core systems and services, application analysis, performance, and security tuning.  
• Create strategies for long-term permanent fixes to critical production incidents.  
• Maintain documentation, build tooling, and create alerts to both identify and address infrastructure reliability.  
• Proactively identify system anomalies.  
• Conducting post-mortems and communicating impact and remediation strategies with service owners and C-level staff.  

• East Coast U.S. hours 9am-6pm EST preferred, but we can be flexible as long as you can work until 12pm/1pm EST; you can work fully remotely  

• Location: Remote in U.S. (preferred) or anywhere in the world 
• Minimum 10 years of relevant experience  
• Minimum 3 years of management experience  

• Thorough understanding of Linux (we use CentOS/Rocky).  
• Advanced knowledge of K8s and its ecosystem:  
• Extensive hands-on experience deploying and managing Kubernetes clusters on bare metal servers in production environments.  
• Immaculate knowledge of best practices for architecting cross-datacenter Kubernetes clusters running on-premise with automated management using kubeadm.  
• Bare metal server configuration optimization know-how for Kubernetes workloads, including networking, storage and security considerations.  
• Kubelet and CRI tuning in accordance with best practices, including but not limited to NUMA and GPU optimization.  
• Deep knowledge of Kubernetes internals, including etcd, container network interfaces and container runtimes.   
• Thorough understanding of PKI certificates for all components (ability to manually troubleshoot and solve client, server, and control-plane certificate issues within Kubernetes with zero downtime).  
• Vast experience in the development of custom Kubernetes operators and autoscalers, as well as tailored ingress/egress controllers, custom resource definitions.  
• Fluency in GitOps automation tools (Flux v1/v2), comprehensive knowledge of Helm and Kustomize controllers.  
• Ability to manage BGP configuration, mastery in kube-router and GoBGP, as well as MetalLB.  
• Understanding of the most intricate details in rook/ceph implementation for Kubernetes.  
• Deep understanding of Puppet configuration management toolset (experience with Chef, CFEngine or Salt also works).  
• Experience administering NoSQL databases (Redis, ES).  
• Experience with scalable infrastructure monitoring solutions such as Icinga, Prometheus, ELK.  
• Strong scripting and automation skills using languages like Python, Ruby, Java, or Go.  
• Advanced understanding of networking concepts (TCP/IP stack, BGP, DNS, CDN, load balancing).  

• Close familiarity with Cassandra at scale.  
• Understanding of Kafka architecture.  
• Experience in AdTech or High-Frequency Trading is a plus.  
• Experience with Security-related best practices is a plus.