AI Platform and Harness Engineer

Location: United States – Remote
Clearance: Ability to obtain and maintain a Public Trust

LTS is seeking a Senior AI Platform and Harness Engineer to develop and maintain the infrastructure, tooling, and evaluation frameworks that power enterprise AI solutions. This role is responsible for building the AI platform and reusable "AI harnesses" that enable Large Language Models (LLMs), AI agents, Retrieval-Augmented Generation (RAG), and Generative AI applications to be securely developed, tested, evaluated, monitored, and deployed at scale.

The ideal candidate has experience with AI platforms, LLMOps, software engineering, cloud-native technologies, and backend systems, along with a passion for building reliable, observable, and production-ready AI solutions. You will work closely with AI architects, software engineers, data scientists, and product teams to ensure AI solutions are scalable, secure, cost-effective, and continuously improving.

What You'll Do:

• Design, build, and maintain enterprise AI platform capabilities supporting Large Language Models (LLMs), AI agents, RAG, and Generative AI applications.
• Lead data preparation, training and fine-tuning of LLMs
• Develop reusable AI harnesses to automate testing, prompt evaluation, model benchmarking, regression testing, and quality assurance.
• Build AI evaluation frameworks to measure model accuracy, retrieval quality, hallucination detection, latency, throughput, cost, and overall application performance.
• Implement observability and monitoring solutions for AI applications, including telemetry, tracing, logging, dashboards, and operational metrics.
• Build and maintain LLMOps pipelines supporting model deployment, versioning, evaluation, experimentation, rollback, and continuous improvement.
• Design automated workflows for prompt testing, retrieval evaluation, AI system validation, and performance benchmarking.
• Develop internal tools for prompt management, model experimentation, AI performance optimization, and developer productivity.
• Build scalable backend services and APIs supporting AI platforms and enterprise AI integrations.
• Collaborate with AI architects and engineering teams to integrate LLMs, RAG pipelines, vector databases, and agentic AI solutions into enterprise applications.
• Support deployment of AI services across AWS, Azure, or Google Cloud using containerized and cloud-native architectures.
• Implement CI/CD pipelines and infrastructure automation supporting enterprise AI development and deployment.