Topic-icon Blocked resources in Robots.txt

Active Subscriptions:

None
9 years 11 months ago #58683 by micfre
Hi,

after install the JFB Component i take a look and see 52 blocked files over the google webmaster tools.
I fixed that entry with Allow: /components/com_jfbconnect/includes/jfbconnect.js?v=6
but the v6 is a daily changeable variable or not ?

We dont find in the documentation a helping guide to fix Robots.txt entries.

Is our robots.txt entry correct for JFB

Best
Michael
The topic has been locked.
Support Specialist
9 years 11 months ago #58685 by alzander
Your robots.txt file really shouldn't matter. It can be used to block automatic bots from scanning your site. While the webmaster tools shows the jfbconnect.js file blocked, that's ok... the google bot doesn't need to execute any code in that file. You shouldn't need to make any changes to your robots.txt file for normal JFBConnect operation.

I hope that helps explain, but if you need any other assistance, please let us know.

Thanks,
Alex
The topic has been locked.
Active Subscriptions:

None
9 years 11 months ago #58725 by micfre
Hi Alex,

i look into the google webmaster tools and see 52 blocked resources. Your Component blocked my pages or not. The google crawler can not crawl this pages.
ANd all this pages are our product pages. if the google bot don't find the pages he will delete this out of the index or the page-ranking goes down. Here it goes not for normal the operation of the JFB component here it goes the index of our site.

If i dont understand it, please correct me.

Best
Michael
The topic has been locked.
Support Specialist
9 years 11 months ago #58750 by alzander
There's a few answers to your questions:
* JFBConnect isn't blocking anything. We don't modify the robots.txt file at all.
* The JFBConnect.js file does not need to be scanned by robots at all. Your free to unblock it in the robots.txt file, but it shouldn't make any difference since Google or other automated tools shouldn't need to read that file
* I'm not sure about the 52 files that are being blocked, you only mention one file from JFBConnect.

I hope that helps explain. In general, I don't think the blocking is a big deal since users and their browsers can still see those files. The robots.txt file simply blocks automated scanners from using resources on your site or seeing files you don't want them to see.

Thanks,
Alex
The topic has been locked.