Skip to content
  • Start any day: Day 1 opens when you enrol
  • 1–2 hrs a day, in your own time
  • Reviewed work and interview prep
  • Individual feedback every week
KLYMB.AI
Log in
Career track 01This job is changing

QA Engineer, becoming AI Test Architect

Testing, automation, defect analysis, API testing and AI-assisted QA.

Choose this trackSee the 30 days
AI skillsRAGAI agentsLLMOpsEvalsAI safetyGuardrails
Two people reviewing code together at their monitors
days
30days
checkpoints
4checkpoints
interview-prep days
2interview-prep days

What's changing in this role

Agents now draft and maintain test suites from tickets. Teams need fewer people executing tests and more people directing, reviewing and designing quality.

What you carry over

  • + Product and edge-case intuition
  • + Risk thinking: what must never break
  • + Release discipline

Who it is for

Built for people like you.

  • Manual testers moving to automation
  • Graduates with testing basics
  • Developers moving into quality engineering

Skills covered

What you will practise.

  • Test design
  • Playwright automation
  • Defect analysis
  • API testing
  • CI pipelines
  • AI-assisted testing
  • Testing AI features
A team gathered around a laptop, talking through a piece of work

30-day overview

Every day, planned.

Four weekly phases of daily problems, an assessment after each week, and interview prep on Days 29–30. Open a week to see every day.

Enroll now
Week 1Days 1–7FoundationsThe new test stackWhere agentic testing sits in a modern pipeline, and which parts of the job are already automated.
  1. Day 1Test design from a user story: Write boundary, negative and edge cases for a checkout story.
  2. Day 2Writing a defect report that gets fixed: Turn a vague bug complaint into a reproducible report.
  3. Day 3Prompting for test cases: Use AI to draft test cases and correct what it gets wrong.
  4. Day 4First Playwright test: Automate a login flow with stable locators.
  5. Day 5API testing basics: Validate status codes, schemas and error responses for a REST API.
  6. Day 6What should an agent own?: Mark which tests in a suite should be automated, agent-generated or kept manual.
  7. Day 7Checkpoint: Suite auditAudit a real regression suite and mark what an agent should own, with reasoning.Practical task + short written reasoning, scored against a rubric
Week 2Days 8–14Practical executionDirecting test agentsSpecs as prompts and acceptance criteria as contracts: generating the suites you used to hand-write.
  1. Day 8Acceptance criteria as contracts: Rewrite ambiguous criteria so an agent can generate tests from them.
  2. Day 9Generating a suite from a ticket: Generate Playwright tests from a raw ticket and fix the failures.
  3. Day 10Reviewing AI-written tests: Find flaky locators, missing assertions and false passes.
  4. Day 11Test data on demand: Generate realistic test data sets for edge cases.
  5. Day 12API contract tests: Automate contract checks for a payments API.
  6. Day 13Flaky test triage: Diagnose three flaky tests and fix the root cause.
  7. Day 14Checkpoint: Ticket to suiteGenerate a test suite from a raw ticket and defend its coverage.Practical task + short written reasoning, scored against a rubric
Week 3Days 15–21Real-world problem solvingTesting AI systemsEvaluating non-deterministic output: eval sets, regressions in model behaviour and guardrail tests.
  1. Day 15Why AI features break normal tests: Identify what cannot be asserted with exact matches.
  2. Day 16Building an eval set: Create 30 test prompts with expected behaviour for a chatbot.
  3. Day 17Prompt injection checks: Try to break a support bot's instructions and document findings.
  4. Day 18Scoring with rubrics: Score model outputs consistently using a rubric.
  5. Day 19Regression on model changes: Compare two model versions and report what changed.
  6. Day 20Reporting AI quality to stakeholders: Summarise eval results for a product manager.
  7. Day 21Checkpoint: AI feature evalBuild an eval set for a non-deterministic feature and report pass/fail criteria.Practical task + short written reasoning, scored against a rubric
Week 4Days 22–28Portfolio and interview readinessPipelines, gates and portfolioWiring agent-generated coverage into CI with review gates that hold up under audit.
  1. Day 22Tests in CI: Run your suite on every pull request with GitHub Actions.
  2. Day 23Quality gates: Define what blocks a release and why.
  3. Day 24Test strategy document: Write a one-page test strategy for a new feature.
  4. Day 25Portfolio: architecture: Document your agentic test architecture.
  5. Day 26Portfolio: walkthrough: Record a 5-minute walkthrough of your project.
  6. Day 27Interview drill: test design: Answer live test-design questions under time pressure.
  7. Day 28Checkpoint: Panel reviewPresent your agentic test architecture to a review panel.Practical task + short written reasoning, scored against a rubric

Weekly checkpoints

Four checkpoints. Real feedback.

Each one is marked on your reasoning, not just the output: and you defend it.

  1. 7After Day 7Suite auditAudit a real regression suite and mark what an agent should own, with reasoning.
  2. 14After Day 14Ticket to suiteGenerate a test suite from a raw ticket and defend its coverage.
  3. 21After Day 21AI feature evalBuild an eval set for a non-deterministic feature and report pass/fail criteria.
  4. 28After Day 28Panel reviewPresent your agentic test architecture to a review panel.
A woman presenting her work to colleagues in a meeting room

Expected outcomes

What you will be able to do.

  • Design test cases from requirements
  • Automate UI and API tests
  • Review and correct AI-generated tests
  • Test AI features with eval sets
  • Explain a test strategy in an interview

Role-specific projects

Evidence you can show.

  1. 01Playwright + API test suite for a demo e-commerce app, running in CI
  2. 02Eval set and quality report for an AI chatbot feature

Interview preparation

Practise the questions QA Engineer interviews actually ask.

Preparation runs through Week 4, then the most-asked questions on Days 29–30. Here is what you will be asked, and what a strong answer looks like.

  • Days 29–30Interview prep after Week 4
  • Three kinds of questionTechnical, scenario, behavioural
  • Model answersWhat a strong answer shows
  1. 01Technical

    “How would you test a login page?”

    A strong answer shows

    Covers functional, negative, security, accessibility and performance angles, and prioritises by risk.

  2. 02Technical

    “How do you deal with flaky tests?”

    A strong answer shows

    Finds root causes (timing, data, environment) instead of adding retries.

  3. 03Scenario

    “Release is tomorrow and 20 tests are failing. What do you do?”

    A strong answer shows

    Triages by risk, communicates clearly, and makes a recommendation.

  4. 04Scenario

    “How would you test a chatbot whose answers change every time?”

    A strong answer shows

    Uses eval sets, rubrics and behaviour checks rather than exact matches.

  5. 05Behavioural

    “Tell me about a bug you found that others missed.”

    A strong answer shows

    Clear situation, the thinking behind it, and the impact.

  6. Topics covered

    • Test design techniques
    • Automation framework design
    • API testing
    • Defect lifecycle
    • Testing AI features
    • CI/CD and quality gates
    Enroll now