- Talent swapping was part of the original MoP.
- It is not needed to beat the gold timer easily with any comp which means no one should feel forced to do it.
- It is easy to do quickly thanks to the Blizzard API. But even without addons/macro it can be done quickly because all talent choices are visible at once (unlike glyph changes where we have to scroll through a potentially long list. But glyph changes appear to remain possible in CM).
- It is fun because of the possibilities it opens up in speedruns. People on the challenge mode discord seem to agree with that.
In short, why remove gameplay mechanics that were fun for some and irrelevant for most when the implementation cost has already been payed.
For another voice on this matter see this post from the US forums: https://us.forums.blizzard.com/en/wow/t/a-plea-keep-spectalent-swapping-for-challenge-modes/2130842