Skip to content

Instantly share code, notes, and snippets.

@mrLexx

mrLexx/detectSearchBot.md

Forked from worka/detectSearchBot.md
Created December 10, 2019 00:11
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save mrLexx/13799e4fc0a928db38b17c005e362e47 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save mrLexx/13799e4fc0a928db38b17c005e362e47 to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. @worka worka revised this gist Oct 25, 2019. 1 changed file with 6 additions and 1 deletion.
    7 changes: 6 additions & 1 deletion detectSearchBot.md
    View file
    Original file line number Diff line number Diff line change
    @@ -3,13 +3,18 @@
    ```php
    function detectSearchBot($ip, $agent, &$hostname)
    {
    // check HTTP_USER_AGENT not to pull gethostbyaddr in vain
    $hostname = $ip;

    // check HTTP_USER_AGENT what not to touch gethostbyaddr in vain
    if (preg_match('/(?:google|yandex)bot/iu', $agent)) {
    // success - return host, fail - return ip or false
    $hostname = gethostbyaddr($ip);

    // https://support.google.com/webmasters/answer/80553
    if ($hostname !== false && $hostname != $ip) {
    // detect google and yandex search bots
    if (preg_match('/\.((?:google(?:bot)?|yandex)\.(?:com|ru))$/iu', $hostname)) {
    // success - return ip, fail - return hostname
    $ip = gethostbyname($hostname);

    if ($ip != $hostname) {
  2. @worka worka revised this gist Oct 25, 2019. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion detectSearchBot.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,4 +1,4 @@
    ### Detect google and yandex search bots
    ### Detect google and yandex search bots (crawl, spider)

    ```php
    function detectSearchBot($ip, $agent, &$hostname)
  3. @worka worka renamed this gist Oct 24, 2019. 1 changed file with 0 additions and 0 deletions.
    0 gist_13.md → detectSearchBot.md
    View file
    File renamed without changes.
  4. @worka worka revised this gist Oct 24, 2019. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion gist_13.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,4 +1,4 @@
    ### detect google and yandex search bots
    ### Detect google and yandex search bots

    ```php
    function detectSearchBot($ip, $agent, &$hostname)
  5. @worka worka created this gist Oct 24, 2019.
    24 changes: 24 additions & 0 deletions gist_13.md
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,24 @@
    ### detect google and yandex search bots

    ```php
    function detectSearchBot($ip, $agent, &$hostname)
    {
    // check HTTP_USER_AGENT not to pull gethostbyaddr in vain
    if (preg_match('/(?:google|yandex)bot/iu', $agent)) {
    $hostname = gethostbyaddr($ip);

    if ($hostname !== false && $hostname != $ip) {
    // detect google and yandex search bots
    if (preg_match('/\.((?:google(?:bot)?|yandex)\.(?:com|ru))$/iu', $hostname)) {
    $ip = gethostbyname($hostname);

    if ($ip != $hostname) {
    return true;
    }
    }
    }
    }

    return false;
    }
    ```