One of mine was off as well. You can adjust it with a small, specialized 3.5 mm wrench (available on amazon) but the threads are easily stripped on the OEM. I replaced it with a post from Blitzkrieg components.
https://www.blitzkriegcomponents.com/benelli-shotgun-sight-posts.html
The threads are not so delicate on the replacement.
An option would be a wave spring washer. I have quite a few of them actually from a prior product I did. PM me if you want one, just pay shipping and it’s yours.