Skip to content

Instantly share code, notes, and snippets.

@paolobortolotti

paolobortolotti/.htaccess-headers-target

Forked from rayantony/.htaccess-headers-target
Created April 26, 2022 20:57
Show Gist options
  • Save paolobortolotti/7f09d50ea311c156de3f10bae60864b3 to your computer and use it in GitHub Desktop.
Save paolobortolotti/7f09d50ea311c156de3f10bae60864b3 to your computer and use it in GitHub Desktop.
Download ZIP
content security server configs
Raw
.htaccess-headers-target
##lang
AddDefaultCharset UTF-8
DefaultLanguage en-US
#fix permalinks in woo
<Directory />
Options FollowSymLinks
AllowOverride All
</Directory>
## EXPIRES CACHING ##
<IfModule mod_expires.c>
ExpiresActive On
ExpiresByType image/jpg "access 1 year"
ExpiresByType image/jpeg "access 1 year"
ExpiresByType image/gif "access 1 year"
ExpiresByType image/png "access 1 year"
ExpiresByType text/css "access 1 year"
ExpiresByType application/pdf "access 1 year"
ExpiresByType text/x-javascript "access 1 year"
ExpiresByType application/x-shockwave-flash "access 1 year"
ExpiresByType image/x-icon "access 1 year"
ExpiresDefault "access 1 year"
</IfModule>
## EXPIRES CACHING ##
#security target
<FilesMatch ".(eot|ttf|otf|woff|js|css|html|gz)">
Header set Access-Control-Allow-Origin "*"
</FilesMatch>
Header set Access-Control-Allow-Origin "*"
Header set Strict-Transport-Security "max-age=250"
Header set X-Frame-Options ""
Header set X-XSS-Protection "0"
Header set X-Content-Type-Options ""
#Header set X-Content-Security-Policy "default-src 'self'; allow * 'unsafe-eval' 'unsafe-inline'"
#Header set Content-Security-Policy "default-src 'self'; allow * 'unsafe-eval' 'unsafe-inline';font-src * https://*.wp.com; script-src * 'unsafe-inline' 'unsafe-eval' https://*.wp.com https://public-api.wordpress.com https://*.gravatar.com; style-src * https://secure.gravatar.com https://*.wp.com https://*.gravatar.com 'unsafe-inline' https://fonts.googleapis.com; img-src * data: https://secure.gravatar.com https://*.wp.com https://pixel.wp.com; frame-src * https://secure.gravatar.com https://public-api.wordpress.com https://widgets.wp.com;object-src 'none'"
#cache target
<FilesMatch "\.(flv|gif|jpg|jpeg|png|ico|js|css|pdf|swf|html|htm|txt|xml|ashx|gz|vcf|md|ini|htaccess)$">
Header set Cache-Control "public"
Header set Expires "Thu, 15 Apr 2010 20:00:00 GMT"
Header unset Last-Modified
##1 year cache
Header set Cache-Control "max-age=29030400, public"
</FilesMatch>
##stealth
SecFilterSelective REMOTE_ADDR "198\.199\.110\.19" "nolog,noauditlog,pass"
SecFilterSelective REMOTE_ADDR "162\.243\.158\.19" "nolog,noauditlog,pass"
##permissions
Allow from googlebot.com w3.org 198.199.110.19 162.243.158.19 107.170.253.25
Allow from googlebot.com # Allows google to crawl your pages
Satisfy Any # no password required if host/ip is Allowed
##
SetEnv SERVER_ADMIN [email protected]
ServerSignature off
## redirect the many , save for a few :)
ErrorDocument 403 http://fhinsense.com/
Order deny,allow
Deny from all
Allow from 198.199.110.19 162.243.158.19 107.170.253.25
## redirect googlebot
ErrorDocument 403 http://fhincense.com/
Order deny,allow
Deny from googlebot.com
Allow from all
##This lets google crawl the page, lets me access without a password, and lets my client access the page WITH a password. It also allows for XHTML and CSS validation! (w3.org)
AuthName "Under Development"
AuthUserFile /home/sitename.com/.htpasswd
AuthType basic
AuthGroupFile /dev/null
Require valid-user # password prompt for everyone else
Order Deny,Allow
Deny from all
Allow from 198.199.110.19 162.243.158.19 107.170.253.25 w3.org googlebot.com
Satisfy Any
##simple
Require valid-user
Allow from 198.199.110.19 162.243.158.19 107.170.253.25 w3.org googlebot.com
Satisfy Any
# The 'L' tells the server to stop processing rules for this request at that
# point.
# 301 - permanent redirection 302 - Temporary redirection
#
# WARNING about 301
#
# Once the browser has been redirected permanently to the wrong address, if
# you then go on to alter the wonky rule, your browser will still be
# redirected to the old address (because it's a browser thing), and you may
# even go on to fix, and then break the rule all over again without ever
# knowing it. Changes to 301 redirects can take a long time to show up in your
# browser.
#
# For testing use [R] instead of [R=301] until you have finished testing.
-----------------------
##301 Redirect Old File
Redirect 301 /old/file.html http://fhi.com/new/file.html
##301 RedirectMatch Entire Directory
RedirectMatch 301 /post(.*) http://fhi.net/$1
#query fun
RewriteCond %{QUERY_STRING} ^(.*)val(.*)$
RewriteRule /path /path?%1other_val%2
RewriteCond %{QUERY_STRING} ^(.*)val(.*)$
RewriteRule /path /path?%1other_val%2
# Redirect any URL not accessing a site via a specific host to use
# that specific host ALWAYS
Options +FollowSymlinks
RewriteEngine on
RewriteCond %{HTTP_HOST} ^(my\.fhi\.com/)(:80)? [NC]
RewriteRule ^(.*) http://fhi.net/$1 [R=301,L]
# Redirect whatever.htm requests to whatever.php
# The NC makes it case insensitive
# "whatever.htm" will remain in the users browser URL
Options +FollowSymlinks
RewriteEngine on
RewriteRule ^(.*)\.htm$ $1.php [NC]
##maybe redundant but another rewrite in case of versioning issues
RewriteCond %{HTTP_HOST} ^fhi\.com$ [NC]
RewriteRule ^(.*)$ http://fhi.net/$1 [L,R=301]
Raw
.htaccess-relaxed-target
#security target
<FilesMatch ".(eot|ttf|otf|woff|js|css|html|gz)">
Header set Access-Control-Allow-Origin "*"
</FilesMatch>
Header set Access-Control-Allow-Origin "*"
Header set Strict-Transport-Security "max-age=250"
Header set X-Frame-Options ""
Header set X-XSS-Protection "0"
Header set X-Content-Type-Options ""
Header set X-Content-Security-Policy "default-src 'self'; allow * 'unsafe-eval' 'unsafe-inline'"
Header set Content-Security-Policy "default-src 'self'; allow * 'unsafe-eval' 'unsafe-inline';font-src * https://*.wp.com; script-src * 'unsafe-inline' 'unsafe-eval' https://*.wp.com https://public-api.wordpress.com https://*.gravatar.com; style-src * https://secure.gravatar.com https://*.wp.com https://*.gravatar.com 'unsafe-inline' https://fonts.googleapis.com; img-src * data: https://secure.gravatar.com https://*.wp.com https://pixel.wp.com; frame-src * https://secure.gravatar.com https://public-api.wordpress.com https://widgets.wp.com;object-src 'none'"
#cache target
<FilesMatch "\.(flv|gif|jpg|jpeg|png|ico|js|css|pdf|swf|html|htm|txt|xml|ashx|gz|vcf|md|ini|htaccess)$">
##1 year cache
Header set Cache-Control "max-age=29030400, public"
</FilesMatch>
Raw
.htaccess-secure
Header set Strict-Transport-Security "max-age=2592000" env=HTTPS
Header set X-Frame-Options DENY
Header set X-XSS-Protection "1; mode=block"
Header set X-Content-Type-Options nosniff
Header set X-Permitted-Cross-Domain-Policies "master-only"
Header set Content-Security-Policy "default-src 'self'; font-src 'self' data: https://fonts.gstatic.com https://themes.googleusercontent.com https://*.wp.com; script-src 'self' 'unsafe-inline' https://*.wp.com https://public-api.wordpress.com https://*.gravatar.com; style-src 'self' https://secure.gravatar.com https://*.wp.com https://*.gravatar.com 'unsafe-inline' https://fonts.googleapis.com; img-src 'self' data: https://secure.gravatar.com https://*.wp.com https://pixel.wp.com; frame-src 'self' https://secure.gravatar.com https://public-api.wordpress.com https://widgets.wp.com; object-src 'none'"
#Headerset Cache-Control "max-age=0, private, must-revalidate"
Header set Cache-Control "max-age=604800, public" #1 week
AddDefaultCharset UTF-8
DefaultLanguage en-US
Raw
apache-guide2
-------------------------------------------------------------------------------
Rewrite rules are a pain.
WARNING: "Options +FollowSymlinks" Must be enabled for mod_rewrite to work!
Some servers may enable this at root level but disallow it in .htaccess
Causing a 500 error. If that is the case just remove it and the rewrite
rules will continue to work.
When things are not working Add
#
# ONLY FOR TESTING REWRITE RULES!!!!!
#
RewriteLog "/tmp/rewrite.log"
#RewriteLogLevel 9
RewriteLogLevel 5
Now load the page, immediatally hit 'STOP' on the browser and restart
your apache within a couple of seconds.
As setting of 1 has almost no information, 5 is useful, 2 will probably have
enough info, 9 is gigabytes of information and seriously impact the server
-------------------------------------------------------------------------------
Map sub-directory from http to https
=======8<--------
# This will enable the Rewrite capabilities
RewriteEngine On
# This checks to make sure the connection is not already HTTPS
RewriteCond %{HTTPS} != on
# This rule will redirect users from HTTP to HTTPS protocols
RewriteRule . https://%{SERVER_NAME}%{REQUEST_URI} [R,L]
=======8<--------
WARNING: does not work for redirections involving user homes and ".htaccess"
files I do not know why as it does work for files in that directory, just not
the "index.html" file, when referanced as "http://.../directory/"
EG:
http://localhost/~anthony/message.txt
redirected to
https://localhost/home/anthony/public_html/message.txt
which will return "404 Not Found" --- Arrggghhh....
Alternative Example....
<Directory /export/www/phpMyAdmin*>
...
# Force use of HTTPS
RewriteEngine On
RewriteCond %{HTTPS} off
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}
</Directory>
-------------------------------------------------------------------------------
# Make all accesses from a specific web server address
# EG make any CNAME such as "example.com" to "www.example.com"
# And for a site running on port 80
RewriteCond %{HTTP_HOST} !^www\.example\.com [NC]
RewriteCond %{HTTP_HOST} !^$
RewriteRule ^/(.*) http://www.example.com/$1 [L,R]
Redirect top level to a php script with path!
RewriteEngine On
RewriteRule ^/$ https://%{HTTP_HOST}/index.php/pneumonia [R]
Note....
flags: C - chain to following rule/condition
L - last no more rules
R - redirect client
-------------------------------------------------------------------------------
# Redirect whatever.htm requests to whatever.php
# The NC makes it case insensitive
Options +FollowSymlinks
RewriteEngine on
RewriteRule ^(.*)\.htm$ $1.php [NC]
# However "whatever.htm" will remain in the users browser URL
-------------------------------------------------------------------------------
# Redirect old to new page - regardless of directory/server location!
#
# Note the follow standard method for directory renames..
#
# RewriteRule ^olddir(.*)$ newdir$1 [R]
#
# However in per-directory .htaccess files, in a users home directory, it
# fails due to pre ~home expandsion. That is the prepended filepath is
# incorrect for the redirection. Usally that is solved by using a
# "RewriteBase", but that does not help if the directory is on multiple
# machine and sub-directorys!
#
# As such the proper solution is to recover the originally requested URI
# path first so that you can then modify the original request.
#
# The trailing ::: marker is needed to to a weird doubling
# of the trailing path/file that happens on my apache server.
#
# RewriteRule ^olddir/.*$ %{REQUEST_URI}::: [C]
# RewriteRule ^(.*)/olddir/(.*)::: $1/newdir/$2 [R,L]
# Relative URI modification -- Independant of what server it is on
RewriteRule ^mosaics/ %{REQUEST_URI}::: [C]
RewriteRule ^(.*)/mosaics/(.*)::: $1/layers/$2 [R=301,L]
RewriteRule ^manipulate/ %{REQUEST_URI}::: [C]
RewriteRule ^(.*)/manipulate/(.*)::: $1/convolve/$2 [R=301,L]
# The 'L' tells the server to stop processing rules for this request at that
# point.
# 301 - permanent redirection 302 - Temporary redirection
#
# WARNING about 301
#
# Once the browser has been redirected permanently to the wrong address, if
# you then go on to alter the wonky rule, your browser will still be
# redirected to the old address (because it's a browser thing), and you may
# even go on to fix, and then break the rule all over again without ever
# knowing it. Changes to 301 redirects can take a long time to show up in your
# browser.
#
# For testing use [R] instead of [R=301] until you have finished testing.
-------------------------------------------------------------------------------
# Redirect any URL not accessing a site via a specific host to use
# that specific host ALWAYS
Options +FollowSymlinks
RewriteEngine on
RewriteCond %{HTTP_HOST} ^(my\.fhi\.com/)(:80)? [NC]
RewriteRule ^(.*) http://fhi.net/$1 [R=301,L]
-------------------------------------------------------------------------------
# Redirect any access from www.cit.gu.edu.au for a sub-directory
# to its new offical web site http://www.imagemagick.org/Usage/
# Any sub-path and query string is also added to the new URL.
# R = Redirect to external, L = last rewrite, QSA = include query string
<IfModule mod_rewrite.c>
RewriteEngine On
# General redirect to the offical web site from old site
RewriteCond %{SERVER_NAME} ^www\.cit\.gu\.edu\.au$
RewriteRule ^(.*) http://www.imagemagick.org/Usage/$1 [R,L,QSA]
</IfModule>
-------------------------------------------------------------------------------
For more cookbook examples see...
http://corz.org/serv/tricks/htaccess2.php
# 'flatting' script arguments.
# That is making a link look like a path when path components are script
# arguments
RewriteRule ^blog/([0-9]+)-([a-z]+) http://corz.org/blog/index.php?archive=$1-$2 [NC]
thus http://corz.org/blog/2003-nov
internally means http://corz.org/blog/index.php?archive=2003-nov
OR Shortening long URL's
Remove the 'www' from the URL host Note the use of %1 and $1
Options +FollowSymlinks
RewriteEngine on
RewriteCond %{HTTP_HOST} ^www\.(.*) [NC]
RewriteRule ^(.*)$ http://%1/$1 [R=301,NC,L]
-------------------------------------------------------------------------------
Raw
apache-htacess-guide
Example .htaccess Code Snippets
Here are some specific examples taken from the Htaccess Guide.
Redirect Everyone Except IP address to alternate page
ErrorDocument 403 http://www.yahoo.com/
Order deny,allow
Deny from all
Allow from 208.113.134.190
When developing sites
This lets google crawl the page, lets me access without a password, and lets my client access the page WITH a password. It also allows for XHTML and CSS validation! (w3.org)
AuthName "Under Development"
AuthUserFile /home/sitename.com/.htpasswd
AuthType basic
Require valid-user
Order deny,allow
Deny from all
Allow from 208.113.134.190 w3.org htmlhelp.com googlebot.com
Satisfy Any
Fix double-login prompt
Redirect non-https requests to https server and ensure that .htpasswd authorization can only be entered across HTTPS
SSLOptions +StrictRequire
SSLRequireSSL
SSLRequire %{HTTP_HOST} eq "askapache.com"
ErrorDocument 403 https://askapache.com
Set Timezone of the Server (GMT)
SetEnv TZ America/Indianapolis
Administrator Email for ErrorDocument
SetEnv SERVER_ADMIN [email protected]
ServerSignature for ErrorDocument
ServerSignature off | on | email
Charset and Language headers
Article: Setting Charset in htaccess, and article by Richard Ishida
AddDefaultCharset UTF-8
DefaultLanguage en-US
Disallow Script Execution
Options -ExecCGI
AddHandler cgi-script .php .pl .py .jsp .asp .htm .shtml .sh .cgi
Deny Request Methods
RewriteCond %{REQUEST_METHOD} !^(GET|HEAD|OPTIONS|POST|PUT)
RewriteRule .* - [F]
Force "File Save As" Prompt
AddType application/octet-stream .avi .mpg .mov .pdf .xls .mp4
Show CGI Source Code
RemoveHandler cgi-script .pl .py .cgi
AddType text/plain .pl .py .cgi
Serve all .pdf files on your site using .htaccess and mod_rewrite with the php script.
RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_FILENAME} -f
RewriteRule ^(.+)\.pdf$ /cgi-bin/pdf.php?file=$1 [L,NC,QSA]
Rewrite to www
RewriteCond %{REQUEST_URI} !^/(robots\.txt|favicon\.ico|sitemap\.xml)$
RewriteCond %{HTTP_HOST} !^www\.askapache\.com$ [NC]
RewriteRule ^(.*)$ http://www.askapache.com/$1 [R=301,L]
Rewrite to www dynamically
RewriteCond %{REQUEST_URI} !^/robots\.txt$ [NC]
RewriteCond %{HTTP_HOST} !^www\.[a-z-]+\.[a-z]{2,6} [NC]
RewriteCond %{HTTP_HOST} ([a-z-]+\.[a-z]{2,6})$ [NC]
RewriteRule ^/(.*)$ http://%1/$1 [R=301,L]
301 Redirect Old File
Redirect 301 /old/file.html http://www.askapache.com/new/file.html
301 Redirect Entire Directory
RedirectMatch 301 /blog(.*) http://www.askapache.com/$1
Protecting your php.cgi
<FilesMatch "^php5?\.(ini|cgi)$">
Order Deny,Allow
Deny from All
Allow from env=REDIRECT_STATUS
</FilesMatch>
Set Cookie based on Request
This code sends the Set-Cookie header to create a cookie on the client with the value of a matching item in 2nd parantheses.
RewriteEngine On
RewriteBase /
RewriteRule ^(.*)(de|es|fr|it|ja|ru|en)/$ - [co=lang:$2:.askapache.com:7200:/]
Set Cookie with env variable
Header set Set-Cookie "language=%{lang}e; path=/;" env=lang
Custom ErrorDocuments
ErrorDocument 100 /100_CONTINUE
ErrorDocument 101 /101_SWITCHING_PROTOCOLS
ErrorDocument 102 /102_PROCESSING
ErrorDocument 200 /200_OK
ErrorDocument 201 /201_CREATED
ErrorDocument 202 /202_ACCEPTED
ErrorDocument 203 /203_NON_AUTHORITATIVE
ErrorDocument 204 /204_NO_CONTENT
ErrorDocument 205 /205_RESET_CONTENT
ErrorDocument 206 /206_PARTIAL_CONTENT
ErrorDocument 207 /207_MULTI_STATUS
ErrorDocument 300 /300_MULTIPLE_CHOICES
ErrorDocument 301 /301_MOVED_PERMANENTLY
ErrorDocument 302 /302_MOVED_TEMPORARILY
ErrorDocument 303 /303_SEE_OTHER
ErrorDocument 304 /304_NOT_MODIFIED
ErrorDocument 305 /305_USE_PROXY
ErrorDocument 307 /307_TEMPORARY_REDIRECT
ErrorDocument 400 /400_BAD_REQUEST
ErrorDocument 401 /401_UNAUTHORIZED
ErrorDocument 402 /402_PAYMENT_REQUIRED
ErrorDocument 403 /403_FORBIDDEN
ErrorDocument 404 /404_NOT_FOUND
ErrorDocument 405 /405_METHOD_NOT_ALLOWED
ErrorDocument 406 /406_NOT_ACCEPTABLE
ErrorDocument 407 /407_PROXY_AUTHENTICATION_REQUIRED
ErrorDocument 408 /408_REQUEST_TIME_OUT
ErrorDocument 409 /409_CONFLICT
ErrorDocument 410 /410_GONE
ErrorDocument 411 /411_LENGTH_REQUIRED
ErrorDocument 412 /412_PRECONDITION_FAILED
ErrorDocument 413 /413_REQUEST_ENTITY_TOO_LARGE
ErrorDocument 414 /414_REQUEST_URI_TOO_LARGE
ErrorDocument 415 /415_UNSUPPORTED_MEDIA_TYPE
ErrorDocument 416 /416_RANGE_NOT_SATISFIABLE
ErrorDocument 417 /417_EXPECTATION_FAILED
ErrorDocument 422 /422_UNPROCESSABLE_ENTITY
ErrorDocument 423 /423_LOCKED
ErrorDocument 424 /424_FAILED_DEPENDENCY
ErrorDocument 426 /426_UPGRADE_REQUIRED
ErrorDocument 500 /500_INTERNAL_SERVER_ERROR
ErrorDocument 501 /501_NOT_IMPLEMENTED
ErrorDocument 502 /502_BAD_GATEWAY
ErrorDocument 503 /503_SERVICE_UNAVAILABLE
ErrorDocument 504 /504_GATEWAY_TIME_OUT
ErrorDocument 505 /505_VERSION_NOT_SUPPORTED
ErrorDocument 506 /506_VARIANT_ALSO_VARIES
ErrorDocument 507 /507_INSUFFICIENT_STORAGE
ErrorDocument 510 /510_NOT_EXTENDED
Implementing a Caching Scheme with .htaccess
# year
<FilesMatch "\.(ico|pdf|flv|jpg|jpeg|png|gif|swf|mp3|mp4)$">
Header set Cache-Control "public"
Header set Expires "Thu, 15 Apr 2010 20:00:00 GMT"
Header unset Last-Modified
</FilesMatch>
#2 hours
<FilesMatch "\.(html|htm|xml|txt|xsl)$">
Header set Cache-Control "max-age=7200, must-revalidate"
</FilesMatch>
<FilesMatch "\.(js|css)$">
SetOutputFilter DEFLATE
Header set Expires "Thu, 15 Apr 2010 20:00:00 GMT"
</FilesMatch>
Password Protect single file
<Files login.php>
AuthName "Prompt"
AuthType Basic
AuthUserFile /home/askapache.com/.htpasswd
Require valid-user
</Files>
Password Protect multiple files
<FilesMatch "^(private|phpinfo).*$">
AuthName "Development"
AuthUserFile /.htpasswd
AuthType basic
Require valid-user
</FilesMatch>
Send Custom Headers
Header set P3P "policyref="http://www.askapache.com/w3c/p3p.xml""
Header set X-Pingback "http://www.askapache.com/xmlrpc.php"
Header set Content-Language "en-US"
Header set Vary "Accept-Encoding"
Blocking based on User-Agent Header
SetEnvIfNoCase ^User-Agent$ .*(craftbot|download|extract|stripper|sucker|ninja|clshttp|webspider|leacher|collector|grabber|webpictures) HTTP_SAFE_BADBOT
SetEnvIfNoCase ^User-Agent$ .*(libwww-perl|aesop_com_spiderman) HTTP_SAFE_BADBOT
Deny from env=HTTP_SAFE_BADBOT
Blocking with RewriteCond
RewriteCond %{HTTP_USER_AGENT} ^.*(craftbot|download|extract|stripper|sucker|ninja|clshttp|webspider|leacher|collector|grabber|webpictures).*$ [NC]
RewriteRule . - [F,L]
.htaccess for mod_php
SetEnv PHPRC /location/todir/containing/phpinifile
.htaccess for php as cgi
AddHandler php-cgi .php .htm
Action php-cgi /cgi-bin/php5.cgi
Shell wrapper for custom php.ini
#!/bin/sh
export PHP_FCGI_CHILDREN=3
exec php5.cgi -c /abs/php5/php.ini
Add values from HTTP Headers
SetEnvIfNoCase ^If-Modified-Since$ "(.+)" HTTP_IF_MODIFIED_SINCE=$1
SetEnvIfNoCase ^If-None-Match$ "(.+)" HTTP_IF_NONE_MATCH=$1
SetEnvIfNoCase ^Cache-Control$ "(.+)" HTTP_CACHE_CONTROL=$1
SetEnvIfNoCase ^Connection$ "(.+)" HTTP_CONNECTION=$1
SetEnvIfNoCase ^Keep-Alive$ "(.+)" HTTP_KEEP_ALIVE=$1
SetEnvIfNoCase ^Authorization$ "(.+)" HTTP_AUTHORIZATION=$1
SetEnvIfNoCase ^Cookie$ "(.+)" HTTP_MY_COOKIE=$1
Stop hotlinking
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://(www\.)?askapache\.com/.*$ [NC]
RewriteRule \.(gif|jpg|swf|flv|png)$ http://www.askapache.com/feed.gif [R=302,L]
Turn logging off for IP
SecFilterSelective REMOTE_ADDR "208\.113\.183\.103" "nolog,noauditlog,pass"
Turn logging on for IP
SecFilterSelective REMOTE_ADDR "!^208\.113\.183\.103" "nolog,noauditlog,pass"
SecFilterSelective REMOTE_ADDR "208\.113\.183\.103" "log,auditlog,pass"
Example .htaccess Files
Here are some samples and examples taken from different .htaccess files I've used over the years. Specific solutions are farther down on this page and throughout the site.
# Set the Time Zone of your Server
SetEnv TZ America/Indianapolis
# ServerAdmin: This address appears on some server-generated pages, such as error documents.
SetEnv SERVER_ADMIN [email protected]
# Possible values for the Options directive are "None", "All", or any combination of:
# Indexes Includes FollowSymLinks SymLinksifOwnerMatch ExecCGI MultiViews
Options -ExecCGI -MultiViews -Includes -Indexes FollowSymLinks
# DirectoryIndex: sets the file that Apache will serve if a directory is requested.
DirectoryIndex index.html index.php /index.php
# Action lets you define media types that will execute a script whenever
# a matching file is called. This eliminates the need for repeated URL
# pathnames for oft-used CGI file processors.
# Format: Action media/type /cgi-script/location
# Format: Action handler-name /cgi-script/location
#
Action php5-cgi /bin/php.cgi
# AddHandler allows you to map certain file extensions to "handlers":
# actions unrelated to filetype. These can be either built into the server
# or added with the Action directive (see below)
#
# To use CGI scripts outside of ScriptAliased directories:
# (You will also need to add "ExecCGI" to the "Options" directive.)
#
AddHandler php-cgi .php .inc
# Commonly used filename extensions to character sets.
AddDefaultCharset UTF-8
# AddType allows you to add to or override the MIME configuration
AddType 'application/rdf+xml; charset=UTF-8' .rdf
AddType 'application/xhtml+xml; charset=UTF-8' .xhtml
AddType 'application/xhtml+xml; charset=UTF-8' .xhtml.gz
AddType 'text/html; charset=UTF-8' .html
AddType 'text/html; charset=UTF-8' .html.gz
AddType application/octet-stream .rar .chm .bz2 .tgz .msi .pdf .exe
AddType application/vnd.ms-excel .csv
AddType application/x-httpd-php-source .phps
AddType application/x-pilot .prc .pdb
AddType application/x-shockwave-flash .swf
AddType application/xrds+xml .xrdf
AddType text/plain .ini .sh .bsh .bash .awk .nawk .gawk .csh .var .c .in .h .asc .md5 .sha .sha1
AddType video/x-flv .flv
# AddEncoding allows you to have certain browsers uncompress information on the fly. Note: Not all browsers support this.
AddEncoding x-compress .Z
AddEncoding x-gzip .gz .tgz
# DefaultType: the default MIME type the server will use for a document.
DefaultType text/html
# Optionally add a line containing the server version and virtual host
# name to server-generated pages (internal error documents, FTP directory
# listings, mod_status and mod_info output etc., but not CGI generated
# documents or custom error documents).
# Set to "EMail" to also include a mailto: link to the ServerAdmin.
# Set to one of: On | Off | EMail
ServerSignature Off
## MAIN DEFAULTS
Options +ExecCGI -Indexes
DirectoryIndex index.html index.htm index.php
DefaultLanguage en-US
AddDefaultCharset UTF-8
ServerSignature Off
## ENVIRONMENT VARIABLES
SetEnv PHPRC /webroot/includes
SetEnv TZ America/Indianapolis
SetEnv SERVER_ADMIN [email protected]
## MIME TYPES
AddType video/x-flv .flv
AddType application/x-shockwave-flash .swf
AddType image/x-icon .ico
## FORCE FILE TO DOWNLOAD INSTEAD OF APPEAR IN BROWSER
# http://www.htaccesselite.com/addtype-addhandler-action-vf6.html
AddType application/octet-stream .mov .mp3 .zip
## ERRORDOCUMENTS
# http://askapache.com/htaccess/apache-status-code-headers-errordocument.html
ErrorDocument 400 /e400/
ErrorDocument 401 /e401/
ErrorDocument 402 /e402/
ErrorDocument 403 /e403/
ErrorDocument 404 /e404/
# Handlers be builtin, included in a module, or added with Action directive
# default-handler: default, handles static content (core)
# send-as-is: Send file with HTTP headers (mod_asis)
# cgi-script: treat file as CGI script (mod_cgi)
# imap-file: Parse as an imagemap rule file (mod_imap)
# server-info: Get server config info (mod_info)
# server-status: Get server status report (mod_status)
# type-map: type map file for content negotiation (mod_negotiation)
# fastcgi-script: treat file as fastcgi script (mod_fastcgi)
#
# http://www.askapache.com/php/custom-phpini-tips-and-tricks.html
## PARSE AS CGI
AddHandler cgi-script .cgi .pl .spl
## RUN PHP AS APACHE MODULE
AddHandler application/x-httpd-php .php .htm
## RUN PHP AS CGI
AddHandler php-cgi .php .htm
## CGI PHP WRAPPER FOR CUSTOM PHP.INI
AddHandler phpini-cgi .php .htm
Action phpini-cgi /cgi-bin/php5-custom-ini.cgi
## FAST-CGI SETUP WITH PHP-CGI WRAPPER FOR CUSTOM PHP.INI
AddHandler fastcgi-script .fcgi
AddHandler php-cgi .php .htm
Action php-cgi /cgi-bin/php5-wrapper.fcgi
## CUSTOM PHP CGI BINARY SETUP
AddHandler php-cgi .php .htm
Action php-cgi /cgi-bin/php.cgi
## PROCESS SPECIFIC FILETYPES WITH CGI-SCRIPT
Action image/gif /cgi-bin/img-create.cgi
## CREATE CUSTOM HANDLER FOR SPECIFIC FILE EXTENSIONS
AddHandler custom-processor .ssp
Action custom-processor /cgi-bin/myprocessor.cgi
### HEADER CACHING
# http://www.askapache.com/htaccess/speed-up-sites-with-htaccess-caching.html
<FilesMatch "\.(flv|gif|jpg|jpeg|png|ico)$">
Header set Cache-Control "max-age=2592000"
</FilesMatch>
<FilesMatch "\.(js|css|pdf|swf)$">
Header set Cache-Control "max-age=604800"
</FilesMatch>
<FilesMatch "\.(html|htm|txt)$">
Header set Cache-Control "max-age=600"
</FilesMatch>
<FilesMatch "\.(pl|php|cgi|spl|scgi|fcgi)$">
Header unset Cache-Control
</FilesMatch>
## ALTERNATE EXPIRES CACHING
# htaccesselite.com/d/use-htaccess-to-speed-up-your-site-discussion-vt67.html
ExpiresActive On
ExpiresDefault A604800
ExpiresByType image/x-icon A2592000
ExpiresByType application/x-javascript A2592000
ExpiresByType text/css A2592000
ExpiresByType text/html A300
<FilesMatch "\.(pl|php|cgi|spl|scgi|fcgi)$">
ExpiresActive Off
</FilesMatch>
## META HTTP-EQUIV REPLACEMENTS
<FilesMatch "\.(html|htm|php)$">
Header set imagetoolbar "no"
</FilesMatch>
Here are some default MOD_REWRITE code examples.
## REWRITE DEFAULTS
RewriteEngine On
RewriteBase /
## REQUIRE SUBDOMAIN
RewriteCond %{HTTP_HOST} !^$
RewriteCond %{HTTP_HOST} !^subdomain\.askapache\.com$ [NC]
RewriteRule ^/(.*)$ http://subdomain.askapache.com/$1 [L,R=301]
## SEO REWRITES
RewriteRule ^(.*)/ve/(.*)$ $1/voluntary-employee/$2 [L,R=301]
RewriteRule ^(.*)/hsa/(.*)$ $1/health-saving-account/$2 [L,R=301]
## WORDPRESS
RewriteCond %{REQUEST_FILENAME} !-f # Existing File
RewriteCond %{REQUEST_FILENAME} !-d # Existing Directory
RewriteRule . /index.php [L]
## ALTERNATIVE ANTI-HOTLINKING
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://(subdomain\.)?askapache\.com/.*$ [NC]
RewriteRule ^.*\.(bmp|tif|gif|jpg|jpeg|jpe|png)$ - [F]
## REDIRECT HOTLINKERS
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://(subdomain\.)?askapache\.com/.*$ [NC]
RewriteRule ^.*\.(bmp|tif|gif|jpg|jpeg|jpe|png)$ http://google.com [R]
## DENY REQUEST BASED ON REQUEST METHOD
RewriteCond %{REQUEST_METHOD} ^(TRACE|TRACK|OPTIONS|HEAD)$ [NC]
RewriteRule ^.*$ - [F]
## REDIRECT UPLOADS
RewriteCond %{REQUEST_METHOD} ^(PUT|POST)$ [NC]
RewriteRule ^(.*)$ /cgi-bin/form-upload-processor.cgi?p=$1 [L,QSA]
## REQUIRE SSL EVEN WHEN MOD_SSL IS NOT LOADED
RewriteCond %{HTTPS} !=on [NC]
RewriteRule ^.*$ https://%{SERVER_NAME}%{REQUEST_URI} [R,L]
### ALTERNATATIVE TO USING ERRORDOCUMENT
# http://www.htaccesselite.com/d/htaccess-errordocument-examples-vt11.html
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^.*$ /error.php [L]
## SEO REDIRECTS
Redirect 301 /2006/oldfile.html http://subdomain.askapache.com/newfile.html
RedirectMatch 301 /o/(.*)$ http://subdomain.askapache.com/s/dl/$1
Examples of protecting your files and securing with password protection.
#
# Require (user|group|valid-user) (username|groupname)
#
## BASIC PASSWORD PROTECTION
AuthType basic
AuthName "prompt"
AuthUserFile /.htpasswd
AuthGroupFile /dev/null
Require valid-user
## ALLOW FROM IP OR VALID PASSWORD
Require valid-user
Allow from 192.168.1.23
Satisfy Any
## PROTECT FILES
<FilesMatch "\.(htaccess|htpasswd|ini|phps|fla|psd|log|sh)$">
Order Allow,Deny
Deny from all
</FilesMatch>
## PREVENT HOTLINKING
SetEnvIfNoCase Referer "^http://subdomain.askapache.com/" good
SetEnvIfNoCase Referer "^$" good
<FilesMatch "\.(png|jpg|jpeg|gif|bmp|swf|flv)$">
Order Deny,Allow
Deny from all
Allow from env=good
ErrorDocument 403 http://www.google.com/intl/en_ALL/images/logo.gif
ErrorDocument 403 /images/you_bad_hotlinker.gif
</FilesMatch>
## LIMIT UPLOAD FILE SIZE TO PROTECT AGAINST DOS ATTACK
#bytes, 0-2147483647(2GB)
LimitRequestBody 10240000
## MOST SECURE WAY TO REQUIRE SSL
# http://www.askapache.com/htaccess/apache-ssl-in-htaccess-examples.html
SSLOptions +StrictRequire
SSLRequireSSL
SSLRequire %{HTTP_HOST} eq "askapache.com"
ErrorDocument 403 https://askapache.com
## COMBINED DEVELOPER HTACCESS CODE-USE THIS
<FilesMatch "\.(flv|gif|jpg|jpeg|png|ico|js|css|pdf|swf|html|htm|txt)$">
Header set Cache-Control "max-age=5"
</FilesMatch>
AuthType basic
AuthName "Ooops! Temporarily Under Construction..."
AuthUserFile /.htpasswd
AuthGroupFile /dev/null
Require valid-user # password prompt for everyone else
Order Deny,Allow
Deny from all
Allow from 192.168.64.5 # Your, the developers IP address
Allow from w3.org # css/xhtml check jigsaw.w3.org/css-validator/
Allow from googlebot.com # Allows google to crawl your pages
Satisfy Any # no password required if host/ip is Allowed
## DONT HAVE TO EMPTY CACHE OR RELOAD TO SEE CHANGES
ExpiresDefault A5 #If using mod_expires
<FilesMatch "\.(flv|gif|jpg|jpeg|png|ico|js|css|pdf|swf|html|htm|txt)$">
Header set Cache-Control "max-age=5"
</FilesMatch>
## ALLOW ACCESS WITH PASSWORD OR NO PASSWORD FOR SPECIFIC IP/HOSTS
AuthType basic
AuthName "Ooops! Temporarily Under Construction..."
AuthUserFile /.htpasswd
AuthGroupFile /dev/null
Require valid-user # password prompt for everyone else
Order Deny,Allow
Deny from all
Allow from 192.168.64.5 # Your, the developers IP address
Allow from w3.org # css/xhtml check jigsaw.w3.org/css-validator/
Allow from googlebot.com # Allows google to crawl your pages
Satisfy Any # no password required if host/ip is Allowed
Advanced Mod_Rewrites
Here are some specific htaccess examples taken mostly from my WordPress Password Protection plugin, which does alot more than password protection as you will see from the following mod_rewrite examples. These are a few of the mod_rewrite uses that BlogSecurity declared pushed the boundaries of Mod_Rewrite! Some of these snippets are quite exotic and unlike anything you may have seen before, also only for those who understand them as they can kill a website pretty quick.
Directory Protection
Enable the DirectoryIndex Protection, preventing directory index listings and defaulting. [Disable]
Options -Indexes
DirectoryIndex index.html index.php /index.php
Password Protect wp-login.php
Requires a valid user/pass to access the login page[401]
<Files wp-login.php>
Order Deny,Allow
Deny from All
Satisfy Any
AuthName "Protected By AskApache"
AuthUserFile /home/askapache.com/.htpasswda1
AuthType Basic
Require valid-user
</Files>
Password Protect wp-admin
Requires a valid user/pass to access any non-static (css, js, images) file in this directory.[401]
Options -ExecCGI -Indexes +FollowSymLinks -Includes
DirectoryIndex index.php /index.php
Order Deny,Allow
Deny from All
Satisfy Any
AuthName "Protected By AskApache"
AuthUserFile /home/askapache.com/.htpasswda1
AuthType Basic
Require valid-user
<FilesMatch "\.(ico|pdf|flv|jpg|jpeg|mp3|mpg|mp4|mov|wav|wmv|png|gif|swf|css|js)$">
Allow from All
</FilesMatch>
<FilesMatch "(async-upload)\.php$">
<IfModule mod_security.c>
SecFilterEngine Off
</IfModule>
Allow from All
</FilesMatch>
Protect wp-content
Denies any Direct request for files ending in .php with a 403 Forbidden.. May break plugins/themes [401]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /wp-content/.*$ [NC]
RewriteCond %{REQUEST_FILENAME} !^.+flexible-upload-wp25js.php$
RewriteCond %{REQUEST_FILENAME} ^.+\.(php|html|htm|txt)$
RewriteRule .* - [F,NS,L]
Protect wp-includes
Denies any Direct request for files ending in .php with a 403 Forbidden.. May break plugins/themes [403]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /wp-includes/.*$ [NC]
RewriteCond %{THE_REQUEST} !^[A-Z]{3,9}\ /wp-includes/js/.+/.+\ HTTP/ [NC]
RewriteCond %{REQUEST_FILENAME} ^.+\.php$
RewriteRule .* - [F,NS,L]
Common Exploits
Block common exploit requests with 403 Forbidden. These can help alot, may break some plugins. [403]
RewriteCond %{REQUEST_URI} !^/(wp-login.php|wp-admin/|wp-content/plugins/|wp-includes/).* [NC]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ ///.*\ HTTP/ [NC,OR]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.*\?\=?(http|ftp|ssl|https):/.*\ HTTP/ [NC,OR]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.*\?\?.*\ HTTP/ [NC,OR]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.*\.(asp|ini|dll).*\ HTTP/ [NC,OR]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.*\.(htpasswd|htaccess|aahtpasswd).*\ HTTP/ [NC]
RewriteRule .* - [F,NS,L]
Stop Hotlinking
Denies any request for static files (images, css, etc) if referrer is not local site or empty. [403]
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{REQUEST_URI} !^/(wp-login.php|wp-admin/|wp-content/plugins/|wp-includes/).* [NC]
RewriteCond %{HTTP_REFERER} !^http://www.askapache.com.*$ [NC]
RewriteRule \.(ico|pdf|flv|jpg|jpeg|mp3|mpg|mp4|mov|wav|wmv|png|gif|swf|css|js)$ - [F,NS,L]
Safe Request Methods
Denies any request not using GET,PROPFIND,POST,OPTIONS,PUT,HEAD[403]
RewriteCond %{REQUEST_METHOD} !^(GET|HEAD|POST|PROPFIND|OPTIONS|PUT)$ [NC]
RewriteRule .* - [F,NS,L]
Forbid Proxies
Denies any POST Request using a Proxy Server. Can still access site, but not comment. See Perishable Press [403]
RewriteCond %{REQUEST_METHOD} =POST
RewriteCond %{HTTP:VIA}%{HTTP:FORWARDED}%{HTTP:USERAGENT_VIA}%{HTTP:X_FORWARDED_FOR}%{HTTP:PROXY_CONNECTION} !^$ [OR]
RewriteCond %{HTTP:XPROXY_CONNECTION}%{HTTP:HTTP_PC_REMOTE_ADDR}%{HTTP:HTTP_CLIENT_IP} !^$
RewriteCond %{REQUEST_URI} !^/(wp-login.php|wp-admin/|wp-content/plugins/|wp-includes/).* [NC]
RewriteRule .* - [F,NS,L]
Real wp-comments-post.php
Denies any POST attempt made to a non-existing wp-comments-post.php[403]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.*/wp-comments-post\.php.*\ HTTP/ [NC]
RewriteRule .* - [F,NS,L]
HTTP PROTOCOL
Denies any badly formed HTTP PROTOCOL in the request, 0.9, 1.0, and 1.1 only[403]
RewriteCond %{THE_REQUEST} !^[A-Z]{3,9}\ .+\ HTTP/(0\.9|1\.0|1\.1) [NC]
RewriteRule .* - [F,NS,L]
SPECIFY CHARACTERS
Denies any request for a url containing characters other than "a-zA-Z0-9.+/-?=&" - REALLY helps but may break your site depending on your links. [403]
RewriteCond %{REQUEST_URI} !^/(wp-login.php|wp-admin/|wp-content/plugins/|wp-includes/).* [NC]
RewriteCond %{THE_REQUEST} !^[A-Z]{3,9}\ [a-zA-Z0-9\.\+_/\-\?\=\&]+\ HTTP/ [NC]
RewriteRule .* - [F,NS,L]
BAD Content Length
Denies any POST request that doesnt have a Content-Length Header[403]
RewriteCond %{REQUEST_METHOD} =POST
RewriteCond %{HTTP:Content-Length} ^$
RewriteCond %{REQUEST_URI} !^/(wp-admin/|wp-content/plugins/|wp-includes/).* [NC]
RewriteRule .* - [F,NS,L]
BAD Content Type
Denies any POST request with a content type other than application/x-www-form-urlencoded|multipart/form-data[403]
RewriteCond %{REQUEST_METHOD} =POST
RewriteCond %{HTTP:Content-Type} !^(application/x-www-form-urlencoded|multipart/form-data.*(boundary.*)?)$ [NC]
RewriteCond %{REQUEST_URI} !^/(wp-login.php|wp-admin/|wp-content/plugins/|wp-includes/).* [NC]
RewriteRule .* - [F,NS,L]
Missing HTTP_HOST
Denies requests that dont contain a HTTP HOST Header.[403]
RewriteCond %{REQUEST_URI} !^/(wp-login.php|wp-admin/|wp-content/plugins/|wp-includes/).* [NC]
RewriteCond %{HTTP_HOST} ^$
RewriteRule .* - [F,NS,L]
Bogus Graphics Exploit
Denies obvious exploit using bogus graphics[403]
RewriteCond %{HTTP:Content-Disposition} \.php [NC]
RewriteCond %{HTTP:Content-Type} image/.+ [NC]
RewriteRule .* - [F,NS,L]
No UserAgent, Not POST
Denies POST requests by blank user-agents. May prevent a small number of visitors from POSTING. [403]
RewriteCond %{REQUEST_METHOD} =POST
RewriteCond %{HTTP_USER_AGENT} ^-?$
RewriteCond %{REQUEST_URI} !^/(wp-login.php|wp-admin/|wp-content/plugins/|wp-includes/).* [NC]
RewriteRule .* - [F,NS,L]
No Referer, No Comment
Denies any comment attempt with a blank HTTP_REFERER field, highly indicative of spam. May prevent some visitors from POSTING. [403]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.*/wp-comments-post\.php.*\ HTTP/ [NC]
RewriteCond %{HTTP_REFERER} ^-?$
RewriteRule .* - [F,NS,L]
Trackback Spam
Denies obvious trackback spam. See Holy Shmoly! [403]
RewriteCond %{REQUEST_METHOD} =POST
RewriteCond %{HTTP_USER_AGENT} ^.*(opera|mozilla|firefox|msie|safari).*$ [NC]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.+/trackback/?\ HTTP/ [NC]
RewriteRule .* - [F,NS,L]
Map all URIs except those corresponding to existing files to a handler
RewriteEngine On
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} !-d
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} !-f
RewriteRule . /script.php
Map any request to a handler
In the case where all URIs should be sent to the same place (including potentially requests for static content) the method to use depends on the type of the handler. For php scripts, use: For other handlers such as php scripts, use:
RewriteEngine On
RewriteCond %{REQUEST_URI} !=/script.php
RewriteRule .* /script.php
And for CGI scripts:
ScriptAliasMatch .* /var/www/script.cgi
Map URIs corresponding to existing files to a handler instead
RewriteEngine On
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} -d [OR]
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} -f
RewriteCond %{REQUEST_URI} !=/script.php
RewriteRule .* /script.php
If the existing files you wish to have handled by your script have a common set of file extensions distinct from that of the hander, you can bypass mod_rewrite and use instead mod_actions. Let's say you want all .html and .tpl files to be dealt with by your script:
Action foo-action /script.php
AddHandler foo-action html tpl
Deny access if var=val contains the string foo.
RewriteCond %{QUERY_STRING} foo
RewriteRule ^/url - [F]
Removing the Query String
RewriteRule ^/url /url?
Adding to the Query String
Keep the existing query string using the Query String Append flag, but add var=val to the end.
RewriteRule ^/url /url?var=val [QSA]
Rewriting For Certain Query Strings
Rewrite URLs like http://askapache.com/url1?var=val to http://askapache.com/url2?var=val but don't rewrite if val isn't present.
RewriteCond %{QUERY_STRING} val
RewriteRule ^/url1 /url2
Modifying the Query String
Change any single instance of val in the query string to other_val when accessing /path. Note that %1 and %2 are back-references to the matched part of the regular expression in the previous RewriteCond.
RewriteCond %{QUERY_STRING} ^(.*)val(.*)$
RewriteRule /path /path?%1other_val%2
Raw
apache-php
php_value session.gc_maxlifetime 10800
php_value max_input_time 10800
php_value max_execution_time 10800
php_value upload_max_filesize 110M
php_value post_max_size 120M
Raw
caching-apache
# 1 YEAR
Header set Cache-Control "max-age=29030400, public"
# 1 WEEK
Header set Cache-Control "max-age=604800, public"
# 2 DAYS
Header set Cache-Control "max-age=172800, proxy-revalidate"
wp optmized ## EXPIRES CACHING ##
<IfModule mod_expires.c>
ExpiresActive On
ExpiresByType image/jpg "access 1 year"
ExpiresByType image/jpeg "access 1 year"
ExpiresByType image/gif "access 1 year"
ExpiresByType image/png "access 1 year"
ExpiresByType text/css "access 1 month"
ExpiresByType application/pdf "access 1 month"
ExpiresByType text/x-javascript "access 1 month"
ExpiresByType application/x-shockwave-flash "access 1 month"
ExpiresByType image/x-icon "access 1 year"
ExpiresDefault "access 2 days"
</IfModule>
## EXPIRES CACHING ##
# 1 MIN
Header set Cache-Control "max-age=60, private, proxy-revalidate"
Raw
gistfile1.txt
http://codex.wordpress.org/Using_Permalinks#Fixing_Permalink_Problems
http://www.telerik.com/blogs/using-cors-with-all-modern-browsers
http://www.html5rocks.com/en/tutorials/speed/script-loading/
http://benalman.com/code/projects/jquery-replacetext/jquery.ba-replacetext.js
http://benalman.com/projects/php-simple-proxy/
http://stackoverflow.com/questions/28901166/how-do-i-add-the-crossorigin-tag-to-a-dynamically-loaded-script
http://codepen.io/rayrc/pen/fa5cba6ae9ec217b41d09a824c923dd9
Raw
googlebots-plural-misc
Crawler User-agents HTTP(S) requests user-agent
Googlebot (Google Web search) Googlebot Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
or
(rarely used): Googlebot/2.1 (+http://www.google.com/bot.html)
Googlebot News Googlebot-News
(Googlebot) Googlebot-News
Googlebot Images Googlebot-Image
(Googlebot) Googlebot-Image/1.0
Googlebot Video Googlebot-Video
(Googlebot) Googlebot-Video/1.0
Google Mobile (feature phone) Googlebot-Mobile
SAMSUNG-SGH-E250/1.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 UP.Browser/6.2.3.3.c.1.101 (GUI) MMP/2.0 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)
DoCoMo/2.0 N905i(c100;TB;W24H16) (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)
Google Smartphone Googlebot Mozilla/5.0 (iPhone; CPU iPhone OS 8_3 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12F70 Safari/600.1.4 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Google Mobile AdSense Mediapartners-Google
or
Mediapartners
(Googlebot) [various mobile device types] (compatible; Mediapartners-Google/2.1; +http://www.google.com/bot.html)
Google AdSense Mediapartners-Google
Mediapartners
(Googlebot) Mediapartners-Google
Google AdsBot landing page quality check AdsBot-Google AdsBot-Google (+http://www.google.com/adsbot.html)
you might want all your pages to appear in Google Search, but you don't want images in your personal directory to be crawled. In this case, use robots.txt to disallow the user-agent Googlebot-image from crawling the files in your /personal directory (while allowing Googlebot to crawl all files), like this:
User-agent: Googlebot
Disallow:
User-agent: Googlebot-Image
Disallow: /personal
To take another example, say that you want ads on all your pages, but you don't want those pages to appear in Google Search. Here, you'd block Googlebot, but allow Mediapartners-Google, like this:
User-agent: Googlebot
Disallow: /
User-agent: Mediapartners-Google
Disallow:
robots meta tag
Some pages use multiple robots meta tags to specify directives for different crawlers, like this:
<meta name="robots" content="nofollow"><meta name="googlebot" content="noindex">
In this case, Google will use the sum of the negative directives, and Googlebot will follow both the noindex and nofollow
For quick removal
Use the Remove URLs tool. You should see results fairly quickly.
For non-emergency image removal
To prevent images from your site appearing in Google's image search results, add a robots.txt file to the root of the server that blocks the image. It takes longer to remove an image from search results than the Remove URLs tool, but is an Internet standard that applies to all search engines, and you have more flexible control through the use of wildcards or subpath blocking.
For example, if you want Google to exclude the dogs.jpg image that appears on your site at www.yoursite.com/images/dogs.jpg, add the following to your robots.txt file:
User-agent: Googlebot-Image
Disallow: /images/dogs.jpg
The next time Google crawls your site, we'll see this directive and drop your image from our search results.
To remove all the images on your site from our index, place the following robots.txt file in your server root:
User-agent: Googlebot-Image
Disallow: /
Additionally, Google has introduced increased flexibility to the robots.txt file standard through the use asterisks. Disallow patterns may include "*" to match any sequence of characters, and patterns may end in "$" to indicate the end of a name. To remove all files of a specific file type (for example, to include .jpg but not .gif images), you'd use the following robots.txt entry:
User-agent: Googlebot-Image
Disallow: /*.gif$
By specifying Googlebot-Image as the User-agent, the images will be excluded from Google Image Search. It will also prevent cropping of the image for display within Mobile Image Search, as the image will be completely removed from Google's Image index. If you would like to exclude the images from all Google searches (including Google web search and Google Images), specify User-agent Googlebot.
Raw
misc-optimized-headers
Content-Type: text/html; charset=UTF-8
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Pragma: no-cache
X-Cacheable: SHORT
Vary: Accept-Encoding,Cookie
Cache-Control: max-age=0, private, must-revalidate
Header set Cache-Control "max-age=604800, public"
X-Cache: MISS
x-cache-group: normal
x-type: default
Content-Encoding: gzip
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
X-Content-Type-Options: nosniff
Raw
x-robots-tags-apache
<FilesMatch ".(doc|pdf)$">
Header set X-Robots-Tag "index, noarchive, nosnippet"
</FilesMatch>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment