I have the same problem. Followed all instructions, the problem with the site loading in a pop up seems to be intermittent. If I am already logged into facebook, at first it seemed to be OK. I then tried to log out of my site and it would not log out. I did get a pop up from facebook saying "you will now be logged out of both facebook and this site"
I tried it with www and no www. If you access my site without the www, it switches to www in the browser after the site loads. I don't know if this helps, but here is the URL in the popup
<!-- m --><a class="postlink" href="http://www.dirt.com/component/option,com_jfbconnect/format,raw/view,xdreceiver/?fb_login&fname=_opener&guid=0.37432259344495833&session=%7B%22session_key%22%3A%223.miI7m80EZRajCqBxB2LDgw__.3600.1266652800-631957567%22%2C%22uid%22%3A631957567%2C%22expires%22%3A1266652800%2C%22secret%22%3A%22GSbBQGP2wfWMWcDIssxHFg__%22%2C%22sig%22%3A%22f589b951dc5be9a3277f3f35328afdfe%22%7D">http://www.dirt.com/component/option,co ... fdfe%22%7D</a><!-- m -->
I do have the trailing slash in my settings in facebook