In my opinion, the right is a rule that resulta in violence when broken.
That violence deosn't have to be from you, since other societal members or even policce/army can enforce it for you, but the moment you remove the violent consequences from the rule being broken, that right ceases to exist.