There is another issue going on.
If you mash your keys and get to auto attack right before you start trying to cast, the spell gets canceled to get the full attack animation + correlated dmg.
Not wearing concentration gear in pve against 1 mob is not gonna let you stop casts so often (unless you aggro 5 of them
).