AI Architecture & Governance Leader Enterprise AI Platforms

Job Posting Date

2025-09-15

General Summary:

Drive the design, governance, and responsible adoption of AI across the enterprise. In this role, you’ll establish a collaborative governance ecosystem spanning models, datasets, fine‑tuned adapters, prompts, agents, AI products, and implementations—while partnering closely with Enterprise Architecture to define high‑level patterns and reference architectures. You’ll guide the prioritization of use cases, shape the enterprise AI platform strategy (cloud and on‑prem), and ensure AI solutions are secure, scalable, cost‑efficient, and compliant. If you thrive at the intersection of architecture, governance, and product leadership—and you’re excited to unlock value from agentic automation—this is for you.

Key Responsibilities

Architecture & Platform

• Define end‑to‑end AI solution architectures (cloud & on‑prem) including model serving, RAG/LLM patterns, vector indexing, data integration, and observability.
• Establish reference architectures, “golden paths,” and reusable templates that integrate with the enterprise AI platform.
• Lead evaluations and POCs of AI capabilities (LLM serving engines, vector DBs, orchestration frameworks, evaluation toolchains, guardrails).
• Partner with Enterprise Architecture to align AI patterns with enterprise standards, security, and roadmaps.
• Guide the design of scalable inference topologies (GPU/CPU, autoscaling, caching, batching, token optimization) and performance tuning.

AI Governance & Risk Management

• Stand up and run a federated, collaborative AI governance council with clear RACI across business, security, legal, compliance, and data teams.
• Define and enforce policies across the AI lifecycle: model/data catalogs, lineage, approvals, evaluations, bias/fairness testing, usage controls, and retention.
• Implement model/data registries, adapter/prompt catalogs, and change control with traceability from use case → model → dataset → deployment.
• Operationalize Responsible AI: safety guardrails, prompt/response policies, red‑teaming, monitoring for drift/toxicity, and human‑in‑the‑loop controls.
• Ensure AI supply‑chain security (licenses, provenance, SBOMs, model signing), privacy, and regulatory compliance.

Use Case Portfolio & Technical Product Leadership

• Run intake, triage, and prioritization of AI and agentic automation use cases; align with business OKRs and platform strategy.
• Shape success metrics and delivery roadmaps in partnership with product, data, security, and engineering teams.
• Drive build/partner/buy analyses and vendor selections; negotiate guardrail requirements and SLAs.
• Provide hands‑on guidance to product squads on decomposition, MVP scoping, and path‑to‑production.

Agentic Automation & RPA

• Define architecture and governance for agentic automation (LLM‑based agents, tools, skills) and RPA integrations.
• Establish patterns for secure tool invocation, approvals, auditability, and exception handling across business processes.

Operations, Observability & Cost

• Define SLOs/SLIs for AI services; implement robust logging, tracing, and evaluation pipelines (quality, latency, cost).
• Build cost governance and FinOps practices for AI workloads (token usage, GPU utilization, autoscaling policies).
• Lead incident response and post‑incident reviews for AI systems; drive continuous improvement.

Leadership & Influence

• Evangelize best practices, create enablement materials, and mentor architects/engineers and product managers.
• Drive alignment across security, data, platform, and enterprise architecture; foster a culture of responsible innovation.

Required Qualifications

• 10+ years in software/AI/ML engineering, platform or enterprise architecture, with 5+ years in a leadership role managing cross‑functional initiatives.
• Engineering degree (Computer Science, Electrical/Computer Engineering, or related).
• Proven experience defining AI solutions architectures (cloud & on‑prem), including LLM/RAG patterns and model lifecycle.
• Strong understanding of AI inference—throughput/latency trade‑offs, batching/caching, GPU/CPU sizing, quantization, token optimization.
• Demonstrated Enterprise AI Governance experience (policies, approvals, model/data lineage, risk/compliance, Responsible AI).
• Hands‑on with Kubernetes (Helm/Kustomize, autoscaling, service mesh, GPU operators) and LLM serving engines (e.g., vLLM, TensorRT‑LLM, Triton, KServe/Seldon, Ray Serve).
• Experience with agentic automation frameworks (e.g., LangGraph, Semantic Kernel, AutoGen) and RPA (e.g., Microsoft Power Automate, UiPath, Automation Anywhere).
• Excellent full‑stack web & mobile architecture knowledge (APIs, eventing, microservices, identity/authorization, mobile backends).
• Experience as a Technical Product Manager or close TPM partnership—portfolio planning, vendor evaluation, and stakeholder management.
• Working knowledge of the enterprise IT ecosystem (identity, networking, security, data platforms, DevSecOps, compliance).
• Strong communication and executive‑level storytelling; ability to influence and drive consensus across diverse stakeholders.

Preferred Qualifications

• Familiarity with Enterprise Architecture frameworks and tools (e.g., TOGAF, Zachman; LeanIX/Ardoq/Sparx EA).
• Experience operating AI platforms at scale (multi‑tenant, multi‑cloud/on‑prem), including GPU scheduling (NVIDIA GPU Operator/MIG) and edge/hybrid scenarios.
• Knowledge of MLOps/LLMOps toolchains (MLflow, Databricks/Mosaic AI, Vertex AI, Azure AI/ML, SageMaker; model/data catalogs and evaluators).
• Experience with vector databases and RAG components (e.g., Azure AI Search, Pinecone, Weaviate, Milvus), and feature stores (e.g., Feast).
• Observability expertise (OpenTelemetry, Prometheus/Grafana) and AI quality monitoring (e.g., human feedback, eval pipelines, drift detection).
• Security, privacy, and compliance background (policy‑as‑code with OPA/Kyverno, model/content safety, data masking, DLP, encryption).
• Certifications: TOGAF, CKA/CKS, major cloud AI certifications (Azure/AWS/GCP), or Responsible AI training.
• Experience establishing governance councils and federated operating models across business units.
• Track record delivering agentic automations that integrate with enterprise systems (ERP/CRM/ITSM) with measurable ROI.