What command should I use to fully remove the package (and cache if any) so that I can retry the pip install with a new version of the tar.gz file ?
You should be testing installation of your package inside a virtualenv. That way, when you need to try again, you can just delete the virtualenv and create a new, fresh one.
If you need to delete a cached wheel built from your .tar.gz file (not sure if it matters when installing from a .tar.gz on disk), run pip cache remove $PROJECT_NAME. To delete all cached wheels, run pip cache purge.