If regular login takes a long time, the issue isn't related to JFBConnect. Our functionality only affects the social login process. During a normal Joomla credentialed login, our plugins don't have any effect.
To diagnose the cause further, I'd recommend disabling any User and Authentication plugins that you may not be using or that aren't necessary for Joomla login and then test again. If still slow, start looking at plugins in the system group that may have some effect on the login process and disable those. Ideally, you'll find the plugin that is causing the slowness.
I hope that helps. If you do find something that points toward JFBConnect being the culprit, just let us know and we'll gladly help investigate further.