Good example or bad example?

You are building the classic bagel or puppy classifier, and you have ten photos ready to add to the training pile. Some are fine. Some will teach the model the wrong thing without ever being wrong themselves. Some are broken before training starts.

Sort each into Keep it, Shortcut risk, or Label problem.

A shortcut risk is a photo that is correctly labeled and still dangerous, because something incidental in it lines up with the label across the whole pile. The model will happily learn the incidental thing, since it is cheaper to learn than the actual subject.

A label problem is a photo where the label is wrong, or where you have not decided what the label rule is yet, which is the same thing with extra steps.

You need 7 out of 10 to pass.