Testing AI-Powered Features
Testing the product built around a model, not the model itself: acceptance criteria for variable output, suite seams and repeats, and the ship call. Interviewers probe where that line falls.
on this pageshowhide
explore
- Case Design12 questions
- Acceptance for Variable Output4 questions
- The Deterministic Surface4 questions
- Fallback & Degraded Paths4 questions
- Suite Mechanics12 questions
- Choosing the Seam4 questions
- Living with Variance4 questions
- Capturing the Run Context4 questions
- Release Judgement8 questions
- Sign-Off Under Uncertainty4 questions
- Defect or Model Limit4 questions
- AI & Data Scientistrole
- AI Engineerrole
- Backend Developerrole
- Data Engineerrole
- Frontend Developerrole
- Full Stack Developerrole
- Game Developerrole
- Java Backend Developerrole
- Java SDETrole
- Kotlin Backend Developerrole
- MLOps Engineerrole
- Machine Learning Engineerrole
- QA Engineerrole
- Software Architectrole
- iOS Developerrole
questions
page 2 of 2When a feature's generative step is swapped and its wording shifts, which test cases should fail and which must not?
level: seniorimportance: nice to knowfreq 30%
basics
~20 sNothing on the exact-assertion surface should move: dispatch, response fields, citation rendering, states and permissions keep passing. Only the wording-reading checks may fail, and their output must name the text as what changed rather than the surrounding behaviour.
A generative product feature's test case met its criteria in four of five repeats - what does a single pass mark hide?
level: seniorimportance: nice to knowfreq 32%
basics
~20 sA single pass mark collapses a rate into a certainty. It hides which repeat missed and on which criterion, how badly it missed, whether four-in-five is normal for this case, and whether the misses cluster on one input.
showing 31–32 of 32