I recently needed to do this.
git tag -d 12345
But i realized, that these tags where just deletet locally. Even a push --tags didn't help. But luckily I found this blog post. You need to do this push to force a deletion.
git push origin :refs/tags/12345