Test Analyst

Overview The AI Test Analyst is responsible for ensuring the quality, reliability, and effectiveness of AI-powered solutions, including generative AI applications, intelligent agents, and AI-enabled business processes. The role focuses on validating AI models and outputs, assessing data quality, monitoring performance, and ensuring adherence to Responsible AI principles, regulatory requirements, and organizational standards. The role supports the adoption of AI-driven testing practices by leveraging automation, innovative testing techniques, and AI-enabled tools to improve testing efficiency and coverage. It also contributes to the development of testing capabilities across the organization by promoting best practices and supporting the upskilling of testing teams in AI and automation technologies. As MUFG Retirement Solutions continues to integrate AI across its products, services, and testing functions, the AI Test Analyst plays a key role in embedding AI into testing processes, identifying cost-effective quality assurance solutions, and ensuring that testing strategies align with business objectives, quality standards, and the organization's AI transformation agenda.
Key Accountabilities and main responsibilities Strategic Focus 

• Develop and execute testing strategies for AI, Machine Learning (ML), and Generative AI solutions to ensure accuracy, reliability, performance, and business value.
• Establish and maintain test automation frameworks and scripts that support various types of testing, such as functional testing, integration testing, API testing, system testing, model validation, data validation, API testing, system testing, and end-to-end AI solution testing. 
• Validate AI model outputs against defined acceptance criteria, business requirements, and responsible AI standards.
• Ensure comprehensive testing coverage across the AI lifecycle, including data ingestion, preprocessing, model training, inference, output validation, and reporting.
• Assess AI solution performance across dimensions such as accuracy, relevance, consistency, explainability, fairness, robustness, and reliability.
• Integrate AI testing activities and test automation framework & scripts with continuous integration, CI/CD pipelines and tools to support continuous testing and deployment of AI solutions.
• Collaborate with developers, business analysts, product owners, and project managers to define testing requirements, success criteria, and quality measures.
• Manage the execution of automated test suites, monitor the test results and collaborate with teams to troubleshoot and resolve the issues promptly.
• Identify opportunities to leverage AI-driven testing tools and automation techniques to improve testing efficiency and effectiveness.