This is ofc maybe a bit late, but someone might look for this info regardless so i’m posting this as a suppliment.
I have a physical authenticator that was on showing the “low batt” error. As i didnt want to go over to using an app i deciced to open the thing and sure enough there were a regular battery that i had some spare of, so i changed it and it still works as earlier.
The battery was a flat 3V Sony CR2032, but i recon any of that type will do (mine was not Sony but the same type, CR2032)