JobBot
Overview
RealCaliforniaJobs.com has a job search robot called "JobBot" which searches job
listings of government agency websites in California. The basic job information
(job title, salary, application deadline, url etc.) is used to allow job seekers
on www.RealCaliforniaJobs.com to search for public sector jobs throughout California
from one website. The search results page sends job seekers directly to the job
posting information on your website.
Robot behavior
The JobBot robot has the following HTTP User-Agent: Mozilla/5.0 (compatible; JobBot/1.0;
+http://www.realcaliforniajobs.com/bot.aspx). The JobBot searches websites during
off hours (usually late at night) and should not impact performance on your web-server.
If your jobs are listed on multiple pages a delay of 3 seconds is observed between
calls to successive pages. JobBot will not usually visit your site more than once
or twice a week.
If you wish to exclude your agencies' jobs from this website simply
Contact Us or exclude the JobBot using a robots.txt file placed in the
root directory of the website which hosts your job postings (as described below).
Robots.txt Protocol
RealCaliforniaJobs.com will obey the first entry in the robots.txt file with a User-agent
containing "JobBot". If there is no such record, it will obey the first entry with
a User-agent of "*". If it is not able to retrieve a robots.txt file, it will assume
there are no restrictions for JobBot. It will keep trying to retrieve the file,
and will obey it if becomes available.
Example of robots.txt:
User-agent: JobBot
Disallow: /
