Skip to content

Instantly share code, notes, and snippets.

@stikhun
stikhun / pget.go
Created January 27, 2022 10:38 — forked from montanaflynn/pget.go
Bounded Parallel Get Requests in Golang
package main
import (
"fmt"
"net/http"
"sort"
"time"
)
// a struct to hold the result from each request including an index
@stikhun
stikhun / wget.sh
Created June 4, 2020 18:07 — forked from crittermike/wget.sh
Download an entire website with wget, along with assets.
# One liner
wget --recursive --page-requisites --adjust-extension --span-hosts --convert-links --restrict-file-names=windows --domains yoursite.com --no-parent yoursite.com
# Explained
wget \
--recursive \ # Download the whole site.
--page-requisites \ # Get all assets/elements (CSS/JS/images).
--adjust-extension \ # Save files with .html on the end.
--span-hosts \ # Include necessary assets from offsite as well.
--convert-links \ # Update links to still work in the static version.

Keybase proof

I hereby claim:

  • I am stikhun on github.
  • I am stikhun (https://keybase.io/stikhun) on keybase.
  • I have a public key whose fingerprint is 1BAD 9B14 5528 E33A 66D9 A631 F3D4 AA9C 5500 0CAF

To claim this, I am signing this object:

@stikhun
stikhun / dumprequest.php
Created April 17, 2018 19:08 — forked from magnetikonline/dumprequest.php
PHP script to dump full HTTP request to file (method, HTTP headers and body).
<?php
// https://gist.github.com/magnetikonline/650e30e485c0f91f2f40
class DumpHTTPRequestToFile {
public function execute($targetFile) {
$data = sprintf(
"%s %s %s\n\nHTTP headers:\n",
$_SERVER['REQUEST_METHOD'],