CHAPTER 19. FIRST WEBSITE 214
http://www.robotstxt.org/ has more information, or you can Google
“robots.txt”.
19.8.3 .htaccess
The.htaccessfile provides you the opportunity to customize things related
to your website or individual webpages.
Passwords can be used to restrict access to authorized users.
URLs can be rewritten to change what the user typed into what you wish
they had typed.
You can put Options -Indexes in your .htaccess file to explicitly turn
off index generation.
Custom Error Pages
Add this line to your .htaccess file.
ErrorDocument 404 http://whatever/404.html
Then create a webpage named 404.html with the message you want to pro-
vide.
Some websites automatically specify404.htmlas the default error docu-
ment.
You can add this line to your .htaccess file to handle type-500 errors (cgi
program crashes).
ErrorDocument 500 http://whatever/whatever.html
http://en.wikipedia.org/wiki/List_of_HTTP_status_codeshas a list
of HTTP status codes. Some are usable for error documents. But 404 is
really the only one that comes up very often.
Password Protection
One popular thing to customize is password protection. This can be used
to restrict access to authorized users.
You can password-protect a webpage or group of pages. Tutorials are avail-
able online. Google search for “apache htaccess password” for some of them.