parr,
Yes, that makes sense. Your server will definitely need to be able to connect to Facebook to fetch the avatar. JFBConnect uses fsockopen to do so, which can be restricted. In case you want to open up just one URL for communication (which I think you can do, but I honestly don't know how), the URL we talk with is "profile.ak.fbcdn.net".
Hope that helps, and good luck!
Alex