Brian is correct, I had same problem, only with an Appliance Module controlling the fan. If the fan turns ON and OFF correctly when the fan is set to the high speed, it is noise or back EMF causing the WS13 to be triggered back ON.
Either replace the WS13A with a XPDI3 inductive dimmer switch which will give you remote speed control (Be warned, some fans will buzz or hum on this switch) OR
Wire a 33K ohm, 3 watt resistor across the fan motor. Make sure the resistor is well insulated and properly supported. I do not know if a resistor, up in the fan shroud or in the switch wall box, meets code, probably not. So if you don't know how to make it safe, go for the XPDI3. My 3 watt resistor barely gets warm, but putting resistors across the line is at your risk.