Mouse Affecting FPS?

Is anyone else on PC having an issue where their mouse is affecting their performance? I’ve seen a lot of people also having this issue on reddit, does anyone have a fix?

If I play with my mouse on 1000hz Polling rate my game is completely unplayable, 500hz, also unplayable, 250hz the mouse drags a lot but I can at least look around, so I’m playing on 125hz and I can play fine but it’s drastically affected my game performance for some odd reason, I’ve got the game set to normal in task manager also…