- Posts: 29
COMMUNITY FORUM
Should the following be in robots.txt
- boodoo
-
Topic Author
- Offline
- Junior Member
Less
More
12 years 3 weeks ago #112478
by boodoo
Should the following be in robots.txt was created by boodoo
Hello again,
I notice all my images related to my K2 module are stored here:
www.mysite.com/media/k2/items/cache/
Therefore, if I want these crawled by googlebot etc, should I have the following line in my robots.txt:
Allow: /media/k2/items/cache/
Thanks!!
I notice all my images related to my K2 module are stored here:
www.mysite.com/media/k2/items/cache/
Therefore, if I want these crawled by googlebot etc, should I have the following line in my robots.txt:
Allow: /media/k2/items/cache/
Thanks!!
Please Log in or Create an account to join the conversation.
- Krikor Boghossian
-
- Offline
- Platinum Member
Less
More
- Posts: 15920
12 years 3 weeks ago #112479
by Krikor Boghossian
Replied by Krikor Boghossian on topic Re: Should the following be in robots.txt
Hello boodoo,
By default Joomla! (not K2) blocks the media and images folders from robots.
You can comment out these two lines in your robots.txt with a hashtag.
By default Joomla! (not K2) blocks the media and images folders from robots.
You can comment out these two lines in your robots.txt with a hashtag.
Please Log in or Create an account to join the conversation.
- boodoo
-
Topic Author
- Offline
- Junior Member
Less
More
- Posts: 29
12 years 3 weeks ago #112480
by boodoo
Replied by boodoo on topic Re: Should the following be in robots.txt
Thanks for the reply.
If I do not allow bots to crawl the /media/ folder (via robots.txt), will the K2 comments on my website still be crawled if I have links to that page?
If I do not allow bots to crawl the /media/ folder (via robots.txt), will the K2 comments on my website still be crawled if I have links to that page?
Please Log in or Create an account to join the conversation.
- Krikor Boghossian
-
- Offline
- Platinum Member
Less
More
- Posts: 15920
12 years 3 weeks ago #112481
by Krikor Boghossian
Replied by Krikor Boghossian on topic Re: Should the following be in robots.txt
This is a template specific issue. Some templates have rel="nofollow" attributes in the comments sections.
If they don't have something like that then it's up to the search engine to honour the no-index request.
If they don't have something like that then it's up to the search engine to honour the no-index request.
Please Log in or Create an account to join the conversation.
- boodoo
-
Topic Author
- Offline
- Junior Member
Less
More
- Posts: 29
12 years 3 weeks ago #112482
by boodoo
Replied by boodoo on topic Re: Should the following be in robots.txt
Thanks Krikor,
Last question. If I want my K2 images to be crawled by google image bot for example, which line should I add to my robots.txt (one with trailing backslash and one without).
Allow: /media/k2/items/cache/
or
Allow: /media/k2/items/cache
Last question. If I want my K2 images to be crawled by google image bot for example, which line should I add to my robots.txt (one with trailing backslash and one without).
Allow: /media/k2/items/cache/
or
Allow: /media/k2/items/cache
Please Log in or Create an account to join the conversation.
- Krikor Boghossian
-
- Offline
- Platinum Member
Less
More
- Posts: 15920
12 years 3 weeks ago #112483
by Krikor Boghossian
Replied by Krikor Boghossian on topic Re: Should the following be in robots.txt
As I said I would comment out the entire Disallow: media and Disallow: images rules in your position
Please Log in or Create an account to join the conversation.