If you want you can disable a keypair. All LMS content referring to this key pair will be unaccessible immediately.
Disabled keypairs will temporally be kept in the orphaned assets recycle bin.
You can search for deleted keypairs when you check Show deleted links
