FoF build not giving room for other casts

I am no expert but from what I have been told and simmed you should almost always cast frostbolt immediately before casting flurry (single target). This person explains it:

It feels like a loss of dps because of the cast time but I actually did gain more dps

When in doubt, just remember:
Frostbolt, flurry, ice lance, ice lance