You have the User - Contact Creator plugin enabled. That plugin comes with Joomla but is disabled by default. When a new user is registered a "Contact" will be made in the Contact Mangager component (also comes with Joomla). However, if you delete the user from Joomla, the contact is not deleted. If you try to register again with the same account (common while you're testing), you'll get that error since the contact already exists.
Either disable the User - Contact Creator plugin or delete the user manually from the Contact Manager when you delete them from Joomla for testing purposes.
I hope that helps,
Alex