I’d say subterfuge and shadow dance share stealth because their form is like you are in stealth, while in combat. Vanish just gets you out of combat and to stealth. Macro will probably detect it still like stealth.
The only way to distinguish between different forms is to check your buff. I guess you should have different icons depending on which form you use. But to do so, you would need to run a lua script and check e.g. what “UnitBuff()” returns. Based on that, you would cast a spell. 255 chars may not be enough to build your macro.
EDIT.
Run this script in chat window when you in different forms:
/run for i=1,40 do local B=UnitBuff("player",i); if B then print(i.."="..B) end end
and check if you see distinct buffs in a list, e.g. you may see Vanish, Stealth, Subterfuge, Shadow Dance, etc. If you do, then you can think of building a short script macro using “if else if else end” etc. and cast a spell you are interested in. Hope it helps.