What happens in cases where the agent knows that given her evidence the prospectively best option (i.e., the option that would maximize expectable value given the agent's evidence) will either be to give drug A or drug C, the prospectively worst option will either be to give drug A or drug C, and the second best option will be to give drug B? If she’s certain that drug B is second best and the effects of giving the wrong drug to the patient are dire enough, can the conscientious but innumerate moral agent say that she should just give the patient drug B rather than take the risk of responding improperly to the evidence she has?
To make this just a bit concrete, suppose our agent had been given drugs A and C by her professor. One kills, he said, but one cures. She’s given a quick quiz to test her knowledge of Bayes’ Theorem. If she gets the right answer, she gets both drugs and she’s told which one kills and which one cures. If she gets the wrong answer, she gets both drugs and she’s told a lie about which one kills and which one cures. She’s pretty sure she knows what Bayes’ Theorem is and she’s used it with some success in the past to determine the probability of some event. She also knows that she barely passed probability and statistics. She takes the quiz and now she faces a three-option case. The professor tells her that drug A cures and drug C kills. Given her evidence, I think that she could say quite reasonably that she ought to give drug B even if she knows that this is not the option that is prospectively best (the option that is prospectively best involves giving the drug that the teacher says cures if she got the right answer or the teacher says kills if she got the wrong answer. Alas, she's not at all certain that she got the right answer to the quiz, but she knows what evidence she had and that this evidence settles what the right answer is and thereby settles the question as to which option is prospectively best).
[Here’s the question on her quiz. 3% of the sprockets from Acme have been defective. 6% of the sprockets from Bloggs have been defective. 40% of the sprockets came from Acme. The remaining 60% of the sprockets in your factory came from Bloggs. A sprocket is randomly selected from a box of sprockets and found to be defective. What is the probability that it came from Bloggs?]