Spider Session Remover This feature look for specific spiders and bots, and remove the session (osCsid) from the URL. The file catalog/includes/spiders.txt is used in conjunction with the "Prevent Spider Sessions" feature in Admin->Sessions. When that feature is enabled, osC downcases the "user agent" from the HTTP request and looks to see if any of the strings in spiders.txt are found in it. This is to identify search engine spiders (also called robots). If there is a match, the spider is not assigned a PHP session, but can still access the page. The benefits of this are:
- The search engines do not include session IDs in their index.
- The spiders are not able to add items to a "cart", thus filling up your database with worthless carts
This feature does NOT prevent spiders from indexing your site. Rather, it improves the quality of the indexing.
List of known spiders and bots stored in spiders.txt file. |
