Quality
CI regression gates for AI
Run evaluation suites in continuous integration and block regressions.
Target proficiency: Applied
Course outcomes
- AIN 401 CO4
Evaluate AI systems with golden sets, calibrated LLM-as-judge, cost/latency metrics and CI regression gates, and justify engineering decisions with evidence.
- MST 501 CO5
Demonstrate Evaluation, Observability & Security competence: complete the Day 61–75 builds, explain the underlying theory, and defend the phase artefact under live questioning.
- MST 501 CO6
Demonstrate Production, Operations & Mastery competence: complete the Day 76–90 builds, explain the underlying theory, and defend the phase artefact under live questioning.
- MST 502 CO4
Demonstrate Evaluation & Observability competence: complete the Day 46–60 builds, explain the underlying theory, and defend the phase artefact under live questioning.
- MST 505 CO5
Demonstrate Evaluation, Caching, Multimodal & Observability competence: complete the Day 61–75 builds, explain the underlying theory, and defend the phase artefact under live questioning.
- MST 506 CO4
Demonstrate Evaluation, Observability & Retrieval Quality competence: complete the Day 46–60 builds, explain the underlying theory, and defend the phase artefact under live questioning.
- MST 508 CO4
Demonstrate Evaluation & Observability competence: complete the Day 46–60 builds, explain the underlying theory, and defend the phase artefact under live questioning.
Taught in weeks & phases
- AIN 401 · Week 9 — Evaluation Engineering: Golden Sets, LLM-as-Judge & Regression Evals
- AIN 401 · Week 11 — AI-DLC Construction: Mob Construction, AI Code Review & Testing
- MST 501 · Phase 5 — Evaluation, Observability & Security
- MST 501 · Phase 6 — Production, Operations & Mastery
- MST 502 · Phase 4 — Evaluation & Observability
- MST 505 · Phase 5 — Evaluation, Caching, Multimodal & Observability
- MST 506 · Phase 4 — Evaluation, Observability & Retrieval Quality
- MST 508 · Phase 4 — Evaluation & Observability