Skip to content

Instantly share code, notes, and snippets.

@k9-dev
Forked from nbourguig/crawler-detect.php
Created May 5, 2014 21:52
Show Gist options
  • Select an option

  • Save k9-dev/a4bb4c5d81e443e3d876 to your computer and use it in GitHub Desktop.

Select an option

Save k9-dev/a4bb4c5d81e443e3d876 to your computer and use it in GitHub Desktop.
<?php
// http://www.cult-f.net/detect-crawlers-with-php/
$crawlers = array(
'Google'=>'Google',
'MSN' => 'msnbot',
'Rambler'=>'Rambler',
'Yahoo'=> 'Yahoo',
'AbachoBOT'=> 'AbachoBOT',
'accoona'=> 'Accoona',
'AcoiRobot'=> 'AcoiRobot',
'ASPSeek'=> 'ASPSeek',
'CrocCrawler'=> 'CrocCrawler',
'Dumbot'=> 'Dumbot',
'FAST-WebCrawler'=> 'FAST-WebCrawler',
'GeonaBot'=> 'GeonaBot',
'Gigabot'=> 'Gigabot',
'Lycos spider'=> 'Lycos',
'MSRBOT'=> 'MSRBOT',
'Altavista robot'=> 'Scooter',
'AltaVista robot'=> 'Altavista',
'ID-Search Bot'=> 'IDBot',
'eStyle Bot'=> 'eStyle',
'Scrubby robot'=> 'Scrubby',
);
function crawlerDetect($USER_AGENT)
{
// to get crawlers string used in function uncomment it
// it is better to save it in string than use implode every time
// global $crawlers
// $crawlers_agents = implode('|',$crawlers);
$crawlers_agents = 'Google|msnbot|Rambler|Yahoo|AbachoBOT|accoona|AcioRobot|ASPSeek|CocoCrawler|Dumbot|FAST-WebCrawler|GeonaBot|Gigabot|Lycos|MSRBOT|Scooter|AltaVista|IDBot|eStyle|Scrubby';
if ( strpos($crawlers_agents , $USER_AGENT) === false )
return false;
// crawler detected
// you can use it to return its name
/*
else {
return array_search($USER_AGENT, $crawlers);
}
*/
}
// example
$crawler = crawlerDetect($_SERVER['HTTP_USER_AGENT']);
if ($crawler )
{
// it is crawler, it's name in $crawler variable
}
else
{
// usual visitor
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment