Skip to content

Instantly share code, notes, and snippets.

@geor-g

geor-g/0 ProPublica Tor hidden service config.md

Forked from mtigas/0 ProPublica Tor hidden service config.md
Created April 10, 2023 16:31
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save geor-g/e1e8aad63b4e91b897dd9b2ae21af7a5 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save geor-g/e1e8aad63b4e91b897dd9b2ae21af7a5 to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. @mtigas mtigas revised this gist Dec 16, 2021. 1 changed file with 5 additions and 1 deletion.
    6 changes: 5 additions & 1 deletion 0 ProPublica Tor hidden service config.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,3 +1,7 @@
    Note (December 16, 2021): These example files haven't been updated since 2016. In either 2019 or 2020, our onion domain was changed to a longer v3 onion address (`p53lf57qovyuvwsc6xnrppyply3vtqm7l6pcobkmyqsiofyeznfu5uqd.onion`). The examples below don't reflect this, but the configuration portions remain accurate regarding how we currently serve the onion site. (Tor Browser dropped support for v2 addresses, such as `propub3r6espa33w.onion`, [in the second half of 2021](https://support.torproject.org/onionservices/v2-deprecation/).)

    ---

    These files contain the base configuration for ProPublica’s Tor hidden service mirror.

    Of note:
    @@ -8,7 +12,7 @@ Of note:

    If you try doing this and have issues using `sudo service nginx restart` due to leftover connections using the socket, you may have to nuke the previous sockets before starting a new nginx process:

    ```sudo service nginx stop && sudo rm /var/run/nginx-pponion-*.sock && sudo service nginx start```
    ```sudo service nginx stop && sudo rm /var/run/nginx-pponion-*.sock && sudo service nginx start```)

    Read our post for more details, including an extended tutorial about running a hidden service:
    https://www.propublica.org/nerds/item/a-more-secure-and-anonymous-propublica-using-tor-hidden-services
  2. @mtigas mtigas revised this gist Jan 19, 2016. 2 changed files with 10 additions and 15 deletions.
    9 changes: 5 additions & 4 deletions 1-torrc
    View file
    Original file line number Diff line number Diff line change
    @@ -7,9 +7,10 @@ Sandbox 1
    # Disable the SOCKS port. Not like anything else on this box is using tor.
    SocksPort 0

    # propub3r6espa33w.onion -> www.propublica.org
    # Set up the hidden service. propub3r6espa33w.onion -> www.propublica.org
    # We're using unix sockets instead of "127.0.0.1:xxxxx". see nginx conf.
    # Docs: https://www.torproject.org/docs/tor-manual.html.en#HiddenServicePort

    HiddenServiceDir /var/run/tor/pp_www_hidserv
    HiddenServicePort 80 unix:/var/run/nginx-pponion-80.sock
    HiddenServicePort 443 unix:/var/run/nginx-pponion-443.sock
    #HiddenServicePort 80 127.0.0.1:33170
    #HiddenServicePort 443 127.0.0.1:33180
    HiddenServicePort 443 unix:/var/run/nginx-pponion-443.sock
    16 changes: 5 additions & 11 deletions 2-nginx
    View file
    Original file line number Diff line number Diff line change
    @@ -1,4 +1,8 @@
    # /etc/nginx/sites-enabled/33170-propubonion.conf
    # /etc/nginx/sites-enabled/propubonion.conf
    #
    # Note that all of our hostnames listen to a unix socket instead
    # of "127.0.0.1:xxxxx".
    # Docs: http://nginx.org/en/docs/http/ngx_http_core_module.html#listen

    map $http_upgrade $connection_upgrade {
    default "upgrade";
    @@ -7,7 +11,6 @@ map $http_upgrade $connection_upgrade {

    # HTTP BARE ONION
    server {
    #listen 127.0.0.1:33170;
    listen unix:/var/run/nginx-pponion-80.sock;
    server_name propub3r6espa33w.onion;
    #allow 127.0.0.1;
    @@ -19,7 +22,6 @@ server {

    # HTTPS BARE ONION
    server {
    #listen 127.0.0.1:33180 ssl spdy;
    listen unix:/var/run/nginx-pponion-443.sock ssl spdy;
    server_name propub3r6espa33w.onion;
    #allow 127.0.0.1;
    @@ -33,8 +35,6 @@ server {

    # WWW ONION
    server {
    #listen 127.0.0.1:33170;
    #listen 127.0.0.1:33180 ssl spdy;
    listen unix:/var/run/nginx-pponion-80.sock;
    listen unix:/var/run/nginx-pponion-443.sock ssl spdy;

    @@ -195,8 +195,6 @@ server {

    # STATIC ONION
    server {
    #listen 127.0.0.1:33170;
    #listen 127.0.0.1:33180 ssl spdy;
    listen unix:/var/run/nginx-pponion-80.sock;
    listen unix:/var/run/nginx-pponion-443.sock ssl spdy;
    server_name static.propub3r6espa33w.onion;
    @@ -296,8 +294,6 @@ server {

    # CDN ONION
    server {
    #listen 127.0.0.1:33170;
    #listen 127.0.0.1:33180 ssl spdy;
    listen unix:/var/run/nginx-pponion-80.sock;
    listen unix:/var/run/nginx-pponion-443.sock ssl spdy;
    server_name cdn.propub3r6espa33w.onion;
    @@ -396,8 +392,6 @@ server {

    # FEEDS ONION
    server {
    #listen 127.0.0.1:33170;
    #listen 127.0.0.1:33180 ssl spdy;
    listen unix:/var/run/nginx-pponion-80.sock;
    listen unix:/var/run/nginx-pponion-443.sock ssl spdy;
    server_name feeds.propub3r6espa33w.onion;
  3. @mtigas mtigas revised this gist Jan 14, 2016. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion 0 ProPublica Tor hidden service config.md
    View file
    Original file line number Diff line number Diff line change
    @@ -4,7 +4,7 @@ Of note:

    * We're using the nginx "subs_filter" and "headers more" modules to allow us to rewrite content and update headers, so that we can convert clearnet links into onion links, where possible.

    * Based on feedback we've received, we're using Unix sockets (instead of a `127.0.0.1:___` TCP port) where nginx listens internally for the inbound connection from Tor.
    * Based on feedback we've received, we're using Unix sockets (instead of a `127.0.0.1:___` TCP port) where nginx listens internally for the inbound connection from Tor. This ensures that a firewall misconfiguration can't expose the site running in nginx, which is likely overkill for an already-public (clearnet) website; this may also slightly improve performance and reduce socket overhead, however.

    If you try doing this and have issues using `sudo service nginx restart` due to leftover connections using the socket, you may have to nuke the previous sockets before starting a new nginx process:

  4. @mtigas mtigas revised this gist Jan 14, 2016. 2 changed files with 150 additions and 17 deletions.
    12 changes: 11 additions & 1 deletion 0 ProPublica Tor hidden service config.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,4 +1,14 @@
    These files contain the base configuration for ProPublica’s Tor hidden service mirror.

    Of note:

    * We're using the nginx "subs_filter" and "headers more" modules to allow us to rewrite content and update headers, so that we can convert clearnet links into onion links, where possible.

    * Based on feedback we've received, we're using Unix sockets (instead of a `127.0.0.1:___` TCP port) where nginx listens internally for the inbound connection from Tor.

    If you try doing this and have issues using `sudo service nginx restart` due to leftover connections using the socket, you may have to nuke the previous sockets before starting a new nginx process:

    ```sudo service nginx stop && sudo rm /var/run/nginx-pponion-*.sock && sudo service nginx start```

    Read our post for more details, including an extended tutorial about running a hidden service:
    https://www.propublica.org/nerds/item/a-more-secure-and-anonymous-propublica-using-tor-hidden-services
    https://www.propublica.org/nerds/item/a-more-secure-and-anonymous-propublica-using-tor-hidden-services
    155 changes: 139 additions & 16 deletions 2-nginx
    View file
    Original file line number Diff line number Diff line change
    @@ -7,19 +7,23 @@ map $http_upgrade $connection_upgrade {

    # HTTP BARE ONION
    server {
    listen 127.0.0.1:33170;
    #listen 127.0.0.1:33170;
    listen unix:/var/run/nginx-pponion-80.sock;
    server_name propub3r6espa33w.onion;
    allow 127.0.0.1;
    #allow 127.0.0.1;
    allow "unix:";
    deny all;
    server_tokens off;
    rewrite ^/(.*) http://www.propub3r6espa33w.onion/$1 permanent;
    }

    # HTTPS BARE ONION
    server {
    listen 127.0.0.1:33180 ssl;
    #listen 127.0.0.1:33180 ssl spdy;
    listen unix:/var/run/nginx-pponion-443.sock ssl spdy;
    server_name propub3r6espa33w.onion;
    allow 127.0.0.1;
    #allow 127.0.0.1;
    allow "unix:";
    deny all;
    server_tokens off;
    ssl_certificate www.propub3r6espa33w.onion.pem;
    @@ -29,15 +33,18 @@ server {

    # WWW ONION
    server {
    listen 127.0.0.1:33170;
    listen 127.0.0.1:33180 ssl;
    #listen 127.0.0.1:33170;
    #listen 127.0.0.1:33180 ssl spdy;
    listen unix:/var/run/nginx-pponion-80.sock;
    listen unix:/var/run/nginx-pponion-443.sock ssl spdy;

    server_name www.propub3r6espa33w.onion;

    ssl_certificate www.propub3r6espa33w.onion.pem;
    ssl_certificate_key www.propub3r6espa33w.onion.key;

    allow 127.0.0.1;
    #allow 127.0.0.1;
    allow "unix:";
    deny all;

    root /usr/local/opt/nginx/html;
    @@ -68,7 +75,7 @@ server {
    subs_filter_types text/css text/xml application/x-javascript application/javascript text/javascript application/json;

    location / {
    #proxy_pass https://192.168.xxx.xxx; # gist note: internal ip address
    #proxy_pass https://192.168.xxx.xxx; # gist note: internal ip address
    proxy_pass https://www.propublica.org;
    proxy_http_version 1.1;
    proxy_set_header Host "www.propublica.org";
    @@ -111,6 +118,9 @@ server {
    subs_filter http://tiles-[abcd].propublica.org/ https://d3i4wq2ul46tvd.cloudfront.net/ ir;
    subs_filter http://tiles-\{s}.propublica.org/ https://d3i4wq2ul46tvd.cloudfront.net/ i;
    subs_filter (http:|https:)//securedrop.propublica.org http://pubdrop4dw6rk3aq.onion i;
    subs_filter http://feeds.propublica.org/ //feeds.propub3r6espa33w.onion/ i;
    subs_filter https://feeds.propublica.org/ //feeds.propub3r6espa33w.onion/ i;
    subs_filter (http:|https:)?//feeds\.propublica\.org/ //feeds.propub3r6espa33w.onion/ gir;
    subs_filter (http:|https:)//propublica.s3.amazonaws.com/ //static.propub3r6espa33w.onion/ i;
    subs_filter //propublica.s3.amazonaws.com/ //static.propub3r6espa33w.onion/ i;
    subs_filter (http:|https:)//cdn.propublica.net.s3.amazonaws.com/ //cdn.propub3r6espa33w.onion/ i;
    @@ -158,7 +168,8 @@ server {
    listen 127.0.0.1:33180 ssl;
    server_name projects.propub3r6espa33w.onion;

    allow 127.0.0.1;
    #allow 127.0.0.1;
    allow "unix:";
    deny all;

    access_log /var/log/nginx/access-projects_hidserv.log;
    @@ -184,11 +195,14 @@ server {

    # STATIC ONION
    server {
    listen 127.0.0.1:33170;
    listen 127.0.0.1:33180 ssl;
    #listen 127.0.0.1:33170;
    #listen 127.0.0.1:33180 ssl spdy;
    listen unix:/var/run/nginx-pponion-80.sock;
    listen unix:/var/run/nginx-pponion-443.sock ssl spdy;
    server_name static.propub3r6espa33w.onion;

    allow 127.0.0.1;
    #allow 127.0.0.1;
    allow "unix:";
    deny all;

    access_log /var/log/nginx/access-assets_hidserv.log;
    @@ -239,6 +253,9 @@ server {
    subs_filter http://tiles-[abcd].propublica.org/ https://d3i4wq2ul46tvd.cloudfront.net/ ir;
    subs_filter http://tiles-\{s}.propublica.org/ https://d3i4wq2ul46tvd.cloudfront.net/ i;
    subs_filter (http:|https:)//securedrop.propublica.org http://pubdrop4dw6rk3aq.onion i;
    subs_filter http://feeds.propublica.org/ //feeds.propub3r6espa33w.onion/ i;
    subs_filter https://feeds.propublica.org/ //feeds.propub3r6espa33w.onion/ i;
    subs_filter (http:|https:)?//feeds\.propublica\.org/ //feeds.propub3r6espa33w.onion/ gir;
    subs_filter (http:|https:)//propublica.s3.amazonaws.com/ //static.propub3r6espa33w.onion/ i;
    subs_filter //propublica.s3.amazonaws.com/ //static.propub3r6espa33w.onion/ i;
    subs_filter (http:|https:)//cdn.propublica.net.s3.amazonaws.com/ //cdn.propub3r6espa33w.onion/ i;
    @@ -279,11 +296,14 @@ server {

    # CDN ONION
    server {
    listen 127.0.0.1:33170;
    listen 127.0.0.1:33180 ssl;
    #listen 127.0.0.1:33170;
    #listen 127.0.0.1:33180 ssl spdy;
    listen unix:/var/run/nginx-pponion-80.sock;
    listen unix:/var/run/nginx-pponion-443.sock ssl spdy;
    server_name cdn.propub3r6espa33w.onion;

    allow 127.0.0.1;
    #allow 127.0.0.1;
    allow "unix:";
    deny all;

    access_log /var/log/nginx/access-cdn_hidserv.log;
    @@ -334,6 +354,9 @@ server {
    subs_filter http://tiles-[abcd].propublica.org/ https://d3i4wq2ul46tvd.cloudfront.net/ ir;
    subs_filter http://tiles-\{s}.propublica.org/ https://d3i4wq2ul46tvd.cloudfront.net/ i;
    subs_filter (http:|https:)//securedrop.propublica.org http://pubdrop4dw6rk3aq.onion i;
    subs_filter http://feeds.propublica.org/ //feeds.propub3r6espa33w.onion/ i;
    subs_filter https://feeds.propublica.org/ //feeds.propub3r6espa33w.onion/ i;
    subs_filter (http:|https:)?//feeds\.propublica\.org/ //feeds.propub3r6espa33w.onion/ gir;
    subs_filter (http:|https:)//propublica.s3.amazonaws.com/ //static.propub3r6espa33w.onion/ i;
    subs_filter //propublica.s3.amazonaws.com/ //static.propub3r6espa33w.onion/ i;
    subs_filter (http:|https:)//cdn.propublica.net.s3.amazonaws.com/ //cdn.propub3r6espa33w.onion/ i;
    @@ -369,4 +392,104 @@ server {
    more_clear_headers "Content-Security-Policy-Report-Only";
    ### /HEADERS ###
    }
    }
    }

    # FEEDS ONION
    server {
    #listen 127.0.0.1:33170;
    #listen 127.0.0.1:33180 ssl spdy;
    listen unix:/var/run/nginx-pponion-80.sock;
    listen unix:/var/run/nginx-pponion-443.sock ssl spdy;
    server_name feeds.propub3r6espa33w.onion;

    #allow 127.0.0.1;
    allow "unix:";
    deny all;

    access_log /var/log/nginx/access-feeds_hidserv.log;

    server_tokens off;

    ssl_certificate www.propub3r6espa33w.onion.pem;
    ssl_certificate_key www.propub3r6espa33w.onion.key;

    rewrite ^/$ http://www.propub3r6espa33w.onion/;

    subs_filter_types text/css text/xml application/x-javascript application/javascript text/javascript application/json;

    location / {
    proxy_pass https://feeds.propublica.org;
    proxy_http_version 1.1;
    proxy_set_header Host "feeds.propublica.org";
    proxy_set_header Connection $connection_upgrade;
    proxy_set_header Upgrade $http_upgrade;
    proxy_ssl_server_name on;
    proxy_read_timeout 30;
    proxy_connect_timeout 30;

    # Don't compress data, since the subs module can't replace
    proxy_set_header Accept-Encoding "";

    ### SUBS https://github.com/yaoweibin/ngx_http_substitutions_filter_module ###
    # We're rewriting links, but we need to preserve rel=canonical for analytics.
    # So rewrite it to a special value and re-rewrite it back to real value later.
    subs_filter "rel=\"canonical\" href=\"http://www.propublica.org" "-----CANONICALHTTPPROPUBLICADOTORG-----" i;
    subs_filter "rel=\"canonical\" href=\"https://www.propublica.org" "-----CANONICALHTTPSPROPUBLICADOTORG-----" i;
    subs_filter "rel=\"canonical\" href=\"http://projects.propublica.org" "-----CANONICALHTTPPROJECTSPROPUBLICA-----" i;
    subs_filter "rel=\"canonical\" href=\"https://projects.propublica.org" "-----CANONICALHTTPSPROJECTSPROPUBLICA-----" i;
    # Keep in .onion
    subs_filter http://www.propublica.org/ //www.propub3r6espa33w.onion/ i;
    subs_filter https://www.propublica.org/ //www.propub3r6espa33w.onion/ i;
    subs_filter "http://www.propublica.org" "//www.propub3r6espa33w.onion" i;
    subs_filter "https://www.propublica.org" "//www.propub3r6espa33w.onion" i;
    subs_filter 'http://www.propublica.org' '//www.propub3r6espa33w.onion' i;
    subs_filter 'https://www.propublica.org' '//www.propub3r6espa33w.onion' i;
    subs_filter http://projects.propublica.org/ //projects.propub3r6espa33w.onion/ i;
    subs_filter https://projects.propublica.org/ //projects.propub3r6espa33w.onion/ i;
    subs_filter http://static.propublica.org/ //static.propub3r6espa33w.onion/ i;
    subs_filter https://static.propublica.org/ //static.propub3r6espa33w.onion/ i;
    subs_filter (http:|https:)?//(www\.)?propublica\.org/ //www.propub3r6espa33w.onion/ gir;
    #subs_filter (http:|https:)?//projects\.propublica\.org/ //projects.propub3r6espa33w.onion/ gir;
    subs_filter (http:|https:)?//static\.propublica\.org/ //static.propub3r6espa33w.onion/ gir;
    subs_filter http://tiles-[abcd].propublica.org/ https://d3i4wq2ul46tvd.cloudfront.net/ ir;
    subs_filter http://tiles-\{s}.propublica.org/ https://d3i4wq2ul46tvd.cloudfront.net/ i;
    subs_filter (http:|https:)//securedrop.propublica.org http://pubdrop4dw6rk3aq.onion i;
    subs_filter http://feeds.propublica.org/ //feeds.propub3r6espa33w.onion/ i;
    subs_filter https://feeds.propublica.org/ //feeds.propub3r6espa33w.onion/ i;
    subs_filter (http:|https:)?//feeds\.propublica\.org/ //feeds.propub3r6espa33w.onion/ gir;
    subs_filter (http:|https:)//propublica.s3.amazonaws.com/ //static.propub3r6espa33w.onion/ i;
    subs_filter //propublica.s3.amazonaws.com/ //static.propub3r6espa33w.onion/ i;
    subs_filter (http:|https:)//cdn.propublica.net.s3.amazonaws.com/ //cdn.propub3r6espa33w.onion/ i;
    subs_filter //cdn.propublica.net.s3.amazonaws.com/ //cdn.propub3r6espa33w.onion/ i;
    subs_filter (http:|https:)//cdn.propublica.net/ //cdn.propub3r6espa33w.onion/ i;
    subs_filter //cdn.propublica.net/ //cdn.propub3r6espa33w.onion/ i;
    subs_filter (http:|https:)//s3.amazonaws.com/cdn.propublica.net/ //cdn.propub3r6espa33w.onion/ i;
    subs_filter //s3.amazonaws.com/cdn.propublica.net/ //cdn.propub3r6espa33w.onion/ i;
    subs_filter (http:|https:)//d1t8xfi7n2gbr1.cloudfront.net/ //cdn.propub3r6espa33w.onion/ i;
    subs_filter //d1t8xfi7n2gbr1.cloudfront.net/ //cdn.propub3r6espa33w.onion/ i;
    subs_filter http://s3.amazonaws.com/ https://s3.amazonaws.com/ i;
    subs_filter \"//s3.amazonaws.com/ \"https://s3.amazonaws.com/ i;
    # Restore the rel="canonical" tag
    subs_filter "-----CANONICALHTTPPROPUBLICADOTORG-----" "rel=\"canonical\" href=\"http://www.propublica.org" i;
    subs_filter "-----CANONICALHTTPSPROPUBLICADOTORG-----" "rel=\"canonical\" href=\"https://www.propublica.org" i;
    subs_filter "-----CANONICALHTTPPROJECTSPROPUBLICA-----" "rel=\"canonical\" href=\"http://projects.propublica.org" i;
    subs_filter "-----CANONICALHTTPSPROJECTSPROPUBLICA-----" "rel=\"canonical\" href=\"https://projects.propublica.org" i;
    ### /SUBS ###

    ### HEADERS http://wiki.nginx.org/HttpHeadersMoreModule ###
    more_set_input_headers "Host: feeds.propublica.org"
    more_set_headers "Access-Control-Allow-Origin: https://www.propublica.org, https://projects.propublica.org, https://static.propublica.org, http://www.propub3r6espa33w.onion, http://projects.propub3r6espa33w.onion, http://static.propub3r6espa33w.onion"
    more_clear_headers "Age";
    more_clear_headers "Server";
    more_clear_headers "Via";
    more_clear_headers "X-From-Nginx";
    more_clear_headers "X-NA";
    more_clear_headers "X-Powered-By";
    more_clear_headers "X-Request-Id";
    more_clear_headers "X-Runtime";
    more_clear_headers "X-Varnish";

    more_clear_headers "Content-Security-Policy-Report-Only";
    ### /HEADERS ###
    }
    }
  5. @mtigas mtigas revised this gist Jan 14, 2016. 1 changed file with 9 additions and 2 deletions.
    11 changes: 9 additions & 2 deletions 1-torrc
    View file
    Original file line number Diff line number Diff line change
    @@ -1,8 +1,15 @@
    # /etc/tor/torrc

    # Try to run Tor more securely via a syscall sandbox.
    # https://www.torproject.org/docs/tor-manual.html.en#Sandbox
    Sandbox 1

    # Disable the SOCKS port. Not like anything else on this box is using tor.
    SocksPort 0

    # propub3r6espa33w.onion -> www.propublica.org
    HiddenServiceDir /var/run/tor/pp_www_hidserv
    HiddenServicePort 80 127.0.0.1:33170
    HiddenServicePort 443 127.0.0.1:33180
    HiddenServicePort 80 unix:/var/run/nginx-pponion-80.sock
    HiddenServicePort 443 unix:/var/run/nginx-pponion-443.sock
    #HiddenServicePort 80 127.0.0.1:33170
    #HiddenServicePort 443 127.0.0.1:33180
  6. @mtigas mtigas revised this gist Jan 13, 2016. No changes.
  7. @mtigas mtigas revised this gist Jan 13, 2016. 1 changed file with 2 additions and 1 deletion.
    3 changes: 2 additions & 1 deletion 0 ProPublica Tor hidden service config.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,3 +1,4 @@
    These files contain the base configuration for ProPublica’s Tor hidden service mirror.

    Read our post for more details, including an extended tutorial about running a hidden service. (link coming soon)
    Read our post for more details, including an extended tutorial about running a hidden service:
    https://www.propublica.org/nerds/item/a-more-secure-and-anonymous-propublica-using-tor-hidden-services
  8. @mtigas mtigas created this gist Jan 13, 2016.
    3 changes: 3 additions & 0 deletions 0 ProPublica Tor hidden service config.md
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,3 @@
    These files contain the base configuration for ProPublica’s Tor hidden service mirror.

    Read our post for more details, including an extended tutorial about running a hidden service. (link coming soon)
    8 changes: 8 additions & 0 deletions 1-torrc
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,8 @@
    # /etc/tor/torrc

    SocksPort 0

    # propub3r6espa33w.onion -> www.propublica.org
    HiddenServiceDir /var/run/tor/pp_www_hidserv
    HiddenServicePort 80 127.0.0.1:33170
    HiddenServicePort 443 127.0.0.1:33180
    372 changes: 372 additions & 0 deletions 2-nginx
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,372 @@
    # /etc/nginx/sites-enabled/33170-propubonion.conf

    map $http_upgrade $connection_upgrade {
    default "upgrade";
    "" "";
    }

    # HTTP BARE ONION
    server {
    listen 127.0.0.1:33170;
    server_name propub3r6espa33w.onion;
    allow 127.0.0.1;
    deny all;
    server_tokens off;
    rewrite ^/(.*) http://www.propub3r6espa33w.onion/$1 permanent;
    }

    # HTTPS BARE ONION
    server {
    listen 127.0.0.1:33180 ssl;
    server_name propub3r6espa33w.onion;
    allow 127.0.0.1;
    deny all;
    server_tokens off;
    ssl_certificate www.propub3r6espa33w.onion.pem;
    ssl_certificate_key www.propub3r6espa33w.onion.key;
    rewrite ^/(.*) https://www.propub3r6espa33w.onion/$1 permanent;
    }

    # WWW ONION
    server {
    listen 127.0.0.1:33170;
    listen 127.0.0.1:33180 ssl;

    server_name www.propub3r6espa33w.onion;

    ssl_certificate www.propub3r6espa33w.onion.pem;
    ssl_certificate_key www.propub3r6espa33w.onion.key;

    allow 127.0.0.1;
    deny all;

    root /usr/local/opt/nginx/html;

    # Set a short cache on this nginx end so that we avoid fetching from
    # the real infrastructure when possible.
    proxy_cache propubonion;
    proxy_cache_valid any 5m;
    proxy_cache_revalidate on;
    proxy_cache_use_stale timeout updating;
    proxy_cache_key $request_uri;
    proxy_ignore_headers expires set-cookie;

    access_log /var/log/nginx/access-propublica_hidserv.log;

    server_tokens off;

    # Blocking access to some possible abuses
    location /donate { rewrite .* https://www.propublica.org/site/donate permanent; }
    location /donate/ { rewrite .* https://www.propublica.org/site/donate permanent; }
    location /site/donate { rewrite .* https://www.propublica.org/site/donate permanent; }
    location /site/donate/ { rewrite .* https://www.propublica.org/site/donate permanent; }
    location /xxxxxxxxxxxxxx { return 403; } # deny CMS
    location /xxxxxxxxxxxxxx/ { return 403; } # deny CMS for now
    location /site/email { return 403; }
    location /site/email/ { return 403; }

    subs_filter_types text/css text/xml application/x-javascript application/javascript text/javascript application/json;

    location / {
    #proxy_pass https://192.168.xxx.xxx; # gist note: internal ip address
    proxy_pass https://www.propublica.org;
    proxy_http_version 1.1;
    proxy_set_header Host "www.propublica.org";
    proxy_set_header Connection $connection_upgrade;
    proxy_set_header Upgrade $http_upgrade;
    proxy_ssl_server_name on;
    proxy_read_timeout 30;
    proxy_connect_timeout 30;

    # Don't compress data, since the subs module can't replace
    proxy_set_header Accept-Encoding "";

    # TODO: currently denying non-GET requests due to some bot-related
    # abuse on some endpoints that poorly handle that.
    limit_except GET {
    deny all;
    }

    ### SUBS https://github.com/yaoweibin/ngx_http_substitutions_filter_module ###
    # We're rewriting links, but we need to preserve rel=canonical for analytics.
    # So rewrite it to a special value and re-rewrite it back to real value later.
    subs_filter "rel=\"canonical\" href=\"http://www.propublica.org" "-----CANONICALHTTPPROPUBLICADOTORG-----" i;
    subs_filter "rel=\"canonical\" href=\"https://www.propublica.org" "-----CANONICALHTTPSPROPUBLICADOTORG-----" i;
    subs_filter "rel=\"canonical\" href=\"http://projects.propublica.org" "-----CANONICALHTTPPROJECTSPROPUBLICA-----" i;
    subs_filter "rel=\"canonical\" href=\"https://projects.propublica.org" "-----CANONICALHTTPSPROJECTSPROPUBLICA-----" i;
    # Keep in .onion
    subs_filter http://www.propublica.org/ //www.propub3r6espa33w.onion/ i;
    subs_filter https://www.propublica.org/ //www.propub3r6espa33w.onion/ i;
    subs_filter "http://www.propublica.org" "//www.propub3r6espa33w.onion" i;
    subs_filter "https://www.propublica.org" "//www.propub3r6espa33w.onion" i;
    subs_filter 'http://www.propublica.org' '//www.propub3r6espa33w.onion' i;
    subs_filter 'https://www.propublica.org' '//www.propub3r6espa33w.onion' i;
    subs_filter http://projects.propublica.org/ //projects.propub3r6espa33w.onion/ i;
    subs_filter https://projects.propublica.org/ //projects.propub3r6espa33w.onion/ i;
    subs_filter http://static.propublica.org/ //static.propub3r6espa33w.onion/ i;
    subs_filter https://static.propublica.org/ //static.propub3r6espa33w.onion/ i;
    subs_filter (http:|https:)?//(www\.)?propublica\.org/ //www.propub3r6espa33w.onion/ gir;
    #subs_filter (http:|https:)?//projects\.propublica\.org/ //projects.propub3r6espa33w.onion/ gir;
    subs_filter (http:|https:)?//static\.propublica\.org/ //static.propub3r6espa33w.onion/ gir;
    subs_filter http://tiles-[abcd].propublica.org/ https://d3i4wq2ul46tvd.cloudfront.net/ ir;
    subs_filter http://tiles-\{s}.propublica.org/ https://d3i4wq2ul46tvd.cloudfront.net/ i;
    subs_filter (http:|https:)//securedrop.propublica.org http://pubdrop4dw6rk3aq.onion i;
    subs_filter (http:|https:)//propublica.s3.amazonaws.com/ //static.propub3r6espa33w.onion/ i;
    subs_filter //propublica.s3.amazonaws.com/ //static.propub3r6espa33w.onion/ i;
    subs_filter (http:|https:)//cdn.propublica.net.s3.amazonaws.com/ //cdn.propub3r6espa33w.onion/ i;
    subs_filter //cdn.propublica.net.s3.amazonaws.com/ //cdn.propub3r6espa33w.onion/ i;
    subs_filter (http:|https:)//cdn.propublica.net/ //cdn.propub3r6espa33w.onion/ i;
    subs_filter //cdn.propublica.net/ //cdn.propub3r6espa33w.onion/ i;
    subs_filter (http:|https:)//s3.amazonaws.com/cdn.propublica.net/ //cdn.propub3r6espa33w.onion/ i;
    subs_filter //s3.amazonaws.com/cdn.propublica.net/ //cdn.propub3r6espa33w.onion/ i;
    subs_filter (http:|https:)//d1t8xfi7n2gbr1.cloudfront.net/ //cdn.propub3r6espa33w.onion/ i;
    subs_filter //d1t8xfi7n2gbr1.cloudfront.net/ //cdn.propub3r6espa33w.onion/ i;
    subs_filter http://s3.amazonaws.com/ https://s3.amazonaws.com/ i;
    subs_filter \"//s3.amazonaws.com/ \"https://s3.amazonaws.com/ i;
    # Restore the rel="canonical" tag
    subs_filter "-----CANONICALHTTPPROPUBLICADOTORG-----" "rel=\"canonical\" href=\"http://www.propublica.org" i;
    subs_filter "-----CANONICALHTTPSPROPUBLICADOTORG-----" "rel=\"canonical\" href=\"https://www.propublica.org" i;
    subs_filter "-----CANONICALHTTPPROJECTSPROPUBLICA-----" "rel=\"canonical\" href=\"http://projects.propublica.org" i;
    subs_filter "-----CANONICALHTTPSPROJECTSPROPUBLICA-----" "rel=\"canonical\" href=\"https://projects.propublica.org" i;
    ### /SUBS ###

    ### HEADERS http://wiki.nginx.org/HttpHeadersMoreModule ###
    more_set_headers "Access-Control-Allow-Origin: https://www.propublica.org, https://projects.propublica.org, https://static.propublica.org, http://www.propub3r6espa33w.onion, http://projects.propub3r6espa33w.onion, http://static.propub3r6espa33w.onion"
    more_clear_headers "Age";
    more_clear_headers "Server";
    more_clear_headers "Via";
    more_clear_headers "X-From-Nginx";
    more_clear_headers "X-NA";
    more_clear_headers "X-Powered-By";
    more_clear_headers "X-Request-Id";
    more_clear_headers "X-Runtime";
    more_clear_headers "X-Varnish";

    more_clear_headers "Content-Security-Policy-Report-Only";
    ### /HEADERS ###
    }
    }






    # PROJECTS ONION
    server {
    listen 127.0.0.1:33170;
    listen 127.0.0.1:33180 ssl;
    server_name projects.propub3r6espa33w.onion;

    allow 127.0.0.1;
    deny all;

    access_log /var/log/nginx/access-projects_hidserv.log;

    server_tokens off;

    ssl_certificate www.propub3r6espa33w.onion.pem;
    ssl_certificate_key www.propub3r6espa33w.onion.key;

    rewrite ^/$ https://www.propub3r6espa33w.onion/data/;

    # temporarily redirecting this to projects.propublica.org (which works)
    # until we can until we can get better routing rules (and figure out
    # anti-abuse)
    rewrite ^/(.*) https://projects.propublica.org/$1 redirect;
    }







    # STATIC ONION
    server {
    listen 127.0.0.1:33170;
    listen 127.0.0.1:33180 ssl;
    server_name static.propub3r6espa33w.onion;

    allow 127.0.0.1;
    deny all;

    access_log /var/log/nginx/access-assets_hidserv.log;

    server_tokens off;

    ssl_certificate www.propub3r6espa33w.onion.pem;
    ssl_certificate_key www.propub3r6espa33w.onion.key;

    rewrite ^/$ http://www.propub3r6espa33w.onion/;

    subs_filter_types text/css text/xml application/x-javascript application/javascript text/javascript application/json;

    location / {
    proxy_pass https://static.propublica.org;
    proxy_http_version 1.1;
    proxy_set_header Host "static.propublica.org";
    proxy_set_header Connection $connection_upgrade;
    proxy_set_header Upgrade $http_upgrade;
    proxy_ssl_server_name on;
    proxy_read_timeout 30;
    proxy_connect_timeout 30;

    # Don't compress data, since the subs module can't replace
    proxy_set_header Accept-Encoding "";

    ### SUBS https://github.com/yaoweibin/ngx_http_substitutions_filter_module ###
    # We're rewriting links, but we need to preserve rel=canonical for analytics.
    # So rewrite it to a special value and re-rewrite it back to real value later.
    subs_filter "rel=\"canonical\" href=\"http://www.propublica.org" "-----CANONICALHTTPPROPUBLICADOTORG-----" i;
    subs_filter "rel=\"canonical\" href=\"https://www.propublica.org" "-----CANONICALHTTPSPROPUBLICADOTORG-----" i;
    subs_filter "rel=\"canonical\" href=\"http://projects.propublica.org" "-----CANONICALHTTPPROJECTSPROPUBLICA-----" i;
    subs_filter "rel=\"canonical\" href=\"https://projects.propublica.org" "-----CANONICALHTTPSPROJECTSPROPUBLICA-----" i;
    # Keep in .onion
    subs_filter http://www.propublica.org/ //www.propub3r6espa33w.onion/ i;
    subs_filter https://www.propublica.org/ //www.propub3r6espa33w.onion/ i;
    subs_filter "http://www.propublica.org" "//www.propub3r6espa33w.onion" i;
    subs_filter "https://www.propublica.org" "//www.propub3r6espa33w.onion" i;
    subs_filter 'http://www.propublica.org' '//www.propub3r6espa33w.onion' i;
    subs_filter 'https://www.propublica.org' '//www.propub3r6espa33w.onion' i;
    subs_filter http://projects.propublica.org/ //projects.propub3r6espa33w.onion/ i;
    subs_filter https://projects.propublica.org/ //projects.propub3r6espa33w.onion/ i;
    subs_filter http://static.propublica.org/ //static.propub3r6espa33w.onion/ i;
    subs_filter https://static.propublica.org/ //static.propub3r6espa33w.onion/ i;
    subs_filter (http:|https:)?//(www\.)?propublica\.org/ //www.propub3r6espa33w.onion/ gir;
    #subs_filter (http:|https:)?//projects\.propublica\.org/ //projects.propub3r6espa33w.onion/ gir;
    subs_filter (http:|https:)?//static\.propublica\.org/ //static.propub3r6espa33w.onion/ gir;
    subs_filter http://tiles-[abcd].propublica.org/ https://d3i4wq2ul46tvd.cloudfront.net/ ir;
    subs_filter http://tiles-\{s}.propublica.org/ https://d3i4wq2ul46tvd.cloudfront.net/ i;
    subs_filter (http:|https:)//securedrop.propublica.org http://pubdrop4dw6rk3aq.onion i;
    subs_filter (http:|https:)//propublica.s3.amazonaws.com/ //static.propub3r6espa33w.onion/ i;
    subs_filter //propublica.s3.amazonaws.com/ //static.propub3r6espa33w.onion/ i;
    subs_filter (http:|https:)//cdn.propublica.net.s3.amazonaws.com/ //cdn.propub3r6espa33w.onion/ i;
    subs_filter //cdn.propublica.net.s3.amazonaws.com/ //cdn.propub3r6espa33w.onion/ i;
    subs_filter (http:|https:)//cdn.propublica.net/ //cdn.propub3r6espa33w.onion/ i;
    subs_filter //cdn.propublica.net/ //cdn.propub3r6espa33w.onion/ i;
    subs_filter (http:|https:)//s3.amazonaws.com/cdn.propublica.net/ //cdn.propub3r6espa33w.onion/ i;
    subs_filter //s3.amazonaws.com/cdn.propublica.net/ //cdn.propub3r6espa33w.onion/ i;
    subs_filter (http:|https:)//d1t8xfi7n2gbr1.cloudfront.net/ //cdn.propub3r6espa33w.onion/ i;
    subs_filter //d1t8xfi7n2gbr1.cloudfront.net/ //cdn.propub3r6espa33w.onion/ i;
    subs_filter http://s3.amazonaws.com/ https://s3.amazonaws.com/ i;
    subs_filter \"//s3.amazonaws.com/ \"https://s3.amazonaws.com/ i;
    # Restore the rel="canonical" tag
    subs_filter "-----CANONICALHTTPPROPUBLICADOTORG-----" "rel=\"canonical\" href=\"http://www.propublica.org" i;
    subs_filter "-----CANONICALHTTPSPROPUBLICADOTORG-----" "rel=\"canonical\" href=\"https://www.propublica.org" i;
    subs_filter "-----CANONICALHTTPPROJECTSPROPUBLICA-----" "rel=\"canonical\" href=\"http://projects.propublica.org" i;
    subs_filter "-----CANONICALHTTPSPROJECTSPROPUBLICA-----" "rel=\"canonical\" href=\"https://projects.propublica.org" i;
    ### /SUBS ###

    ### HEADERS http://wiki.nginx.org/HttpHeadersMoreModule ###
    more_set_input_headers "Host: static.propublica.org"
    more_set_headers "Access-Control-Allow-Origin: https://www.propublica.org, https://projects.propublica.org, https://static.propublica.org, http://www.propub3r6espa33w.onion, http://projects.propub3r6espa33w.onion, http://static.propub3r6espa33w.onion"
    more_clear_headers "Age";
    more_clear_headers "Server";
    more_clear_headers "Via";
    more_clear_headers "X-From-Nginx";
    more_clear_headers "X-NA";
    more_clear_headers "X-Powered-By";
    more_clear_headers "X-Request-Id";
    more_clear_headers "X-Runtime";
    more_clear_headers "X-Varnish";

    more_clear_headers "Content-Security-Policy-Report-Only";
    ### /HEADERS ###
    }
    }


    # CDN ONION
    server {
    listen 127.0.0.1:33170;
    listen 127.0.0.1:33180 ssl;
    server_name cdn.propub3r6espa33w.onion;

    allow 127.0.0.1;
    deny all;

    access_log /var/log/nginx/access-cdn_hidserv.log;

    server_tokens off;

    ssl_certificate www.propub3r6espa33w.onion.pem;
    ssl_certificate_key www.propub3r6espa33w.onion.key;

    rewrite ^/$ http://www.propub3r6espa33w.onion/;

    subs_filter_types text/css text/xml application/x-javascript application/javascript text/javascript application/json;

    location / {
    proxy_pass https://d1t8xfi7n2gbr1.cloudfront.net;
    proxy_http_version 1.1;
    proxy_set_header Host "d1t8xfi7n2gbr1.cloudfront.net";
    proxy_set_header Connection $connection_upgrade;
    proxy_set_header Upgrade $http_upgrade;
    proxy_ssl_server_name on;
    proxy_read_timeout 30;
    proxy_connect_timeout 30;

    # Don't compress data, since the subs module can't replace
    proxy_set_header Accept-Encoding "";

    ### SUBS https://github.com/yaoweibin/ngx_http_substitutions_filter_module ###
    # We're rewriting links, but we need to preserve rel=canonical for analytics.
    # So rewrite it to a special value and re-rewrite it back to real value later.
    subs_filter "rel=\"canonical\" href=\"http://www.propublica.org" "-----CANONICALHTTPPROPUBLICADOTORG-----" i;
    subs_filter "rel=\"canonical\" href=\"https://www.propublica.org" "-----CANONICALHTTPSPROPUBLICADOTORG-----" i;
    subs_filter "rel=\"canonical\" href=\"http://projects.propublica.org" "-----CANONICALHTTPPROJECTSPROPUBLICA-----" i;
    subs_filter "rel=\"canonical\" href=\"https://projects.propublica.org" "-----CANONICALHTTPSPROJECTSPROPUBLICA-----" i;
    # Keep in .onion
    subs_filter http://www.propublica.org/ //www.propub3r6espa33w.onion/ i;
    subs_filter https://www.propublica.org/ //www.propub3r6espa33w.onion/ i;
    subs_filter "http://www.propublica.org" "//www.propub3r6espa33w.onion" i;
    subs_filter "https://www.propublica.org" "//www.propub3r6espa33w.onion" i;
    subs_filter 'http://www.propublica.org' '//www.propub3r6espa33w.onion' i;
    subs_filter 'https://www.propublica.org' '//www.propub3r6espa33w.onion' i;
    subs_filter http://projects.propublica.org/ //projects.propub3r6espa33w.onion/ i;
    subs_filter https://projects.propublica.org/ //projects.propub3r6espa33w.onion/ i;
    subs_filter http://static.propublica.org/ //static.propub3r6espa33w.onion/ i;
    subs_filter https://static.propublica.org/ //static.propub3r6espa33w.onion/ i;
    subs_filter (http:|https:)?//(www\.)?propublica\.org/ //www.propub3r6espa33w.onion/ gir;
    #subs_filter (http:|https:)?//projects\.propublica\.org/ //projects.propub3r6espa33w.onion/ gir;
    subs_filter (http:|https:)?//static\.propublica\.org/ //static.propub3r6espa33w.onion/ gir;
    subs_filter http://tiles-[abcd].propublica.org/ https://d3i4wq2ul46tvd.cloudfront.net/ ir;
    subs_filter http://tiles-\{s}.propublica.org/ https://d3i4wq2ul46tvd.cloudfront.net/ i;
    subs_filter (http:|https:)//securedrop.propublica.org http://pubdrop4dw6rk3aq.onion i;
    subs_filter (http:|https:)//propublica.s3.amazonaws.com/ //static.propub3r6espa33w.onion/ i;
    subs_filter //propublica.s3.amazonaws.com/ //static.propub3r6espa33w.onion/ i;
    subs_filter (http:|https:)//cdn.propublica.net.s3.amazonaws.com/ //cdn.propub3r6espa33w.onion/ i;
    subs_filter //cdn.propublica.net.s3.amazonaws.com/ //cdn.propub3r6espa33w.onion/ i;
    subs_filter (http:|https:)//cdn.propublica.net/ //cdn.propub3r6espa33w.onion/ i;
    subs_filter //cdn.propublica.net/ //cdn.propub3r6espa33w.onion/ i;
    subs_filter (http:|https:)//s3.amazonaws.com/cdn.propublica.net/ //cdn.propub3r6espa33w.onion/ i;
    subs_filter //s3.amazonaws.com/cdn.propublica.net/ //cdn.propub3r6espa33w.onion/ i;
    subs_filter (http:|https:)//d1t8xfi7n2gbr1.cloudfront.net/ //cdn.propub3r6espa33w.onion/ i;
    subs_filter //d1t8xfi7n2gbr1.cloudfront.net/ //cdn.propub3r6espa33w.onion/ i;
    subs_filter http://s3.amazonaws.com/ https://s3.amazonaws.com/ i;
    subs_filter \"//s3.amazonaws.com/ \"https://s3.amazonaws.com/ i;
    # Restore the rel="canonical" tag
    subs_filter "-----CANONICALHTTPPROPUBLICADOTORG-----" "rel=\"canonical\" href=\"http://www.propublica.org" i;
    subs_filter "-----CANONICALHTTPSPROPUBLICADOTORG-----" "rel=\"canonical\" href=\"https://www.propublica.org" i;
    subs_filter "-----CANONICALHTTPPROJECTSPROPUBLICA-----" "rel=\"canonical\" href=\"http://projects.propublica.org" i;
    subs_filter "-----CANONICALHTTPSPROJECTSPROPUBLICA-----" "rel=\"canonical\" href=\"https://projects.propublica.org" i;
    ### /SUBS ###

    ### HEADERS http://wiki.nginx.org/HttpHeadersMoreModule ###
    more_set_input_headers "Host: d1t8xfi7n2gbr1.cloudfront.net"
    more_set_headers "Access-Control-Allow-Origin: https://www.propublica.org, https://projects.propublica.org, https://static.propublica.org, http://www.propub3r6espa33w.onion, http://projects.propub3r6espa33w.onion, http://static.propub3r6espa33w.onion"
    more_clear_headers "Age";
    more_clear_headers "Server";
    more_clear_headers "Via";
    more_clear_headers "X-From-Nginx";
    more_clear_headers "X-NA";
    more_clear_headers "X-Powered-By";
    more_clear_headers "X-Request-Id";
    more_clear_headers "X-Runtime";
    more_clear_headers "X-Varnish";

    more_clear_headers "Content-Security-Policy-Report-Only";
    ### /HEADERS ###
    }
    }