Yefan YefanZhou

Preparation of ImageNet (ILSVRC2012)

The dataset can be found on the official website if you are affiliated with a research organization. It is also available on Academic torrents.

This script extracts all the images and group them so that folders contain images that belong to the same class.

Download the ILSVRC2012_img_train.tar and ILSVRC2012_img_val.tar
Download the script wget https://gist.github.com/antoinebrl/7d00d5cb6c95ef194c737392ef7e476a/raw/dc53ad5fcb69dcde2b3e0b9d6f8f99d000ead696/prepare.sh
Run it ./prepare.sh

To remove a submodule you need to:

	import torch
	from torch import LongTensor
	from torch.nn import Embedding, LSTM
	from torch.autograd import Variable
	from torch.nn.utils.rnn import pack_padded_sequence, pad_packed_sequence

	## We want to run LSTM on a batch of 3 character sequences ['long_str', 'tiny', 'medium']
	#
	# Step 1: Construct Vocabulary
	# Step 2: Load indexed data (list of instances, where each instance is list of character indices)

	import torch
	import torch.nn as nn
	import torch.nn.functional as F

	import torchvision.models as tmodels
	from functools import partial
	import collections

	# dummy data: 10 batches of images with batch size 16
	dataset = [torch.rand(16,3,224,224).cuda() for _ in range(10)]

	apt-get update
	apt install vim
	apt install git
	apt install gpustat
	apt install wget
	wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh

	link: https://www.ellicium.com/python-multiprocessing-pool-process/

	Summary: To summarize this, pool class works better when there are more processes and small IO wait.
	Process class works better when processes are small in number and IO operations are long.

	# https://stackoverflow.com/questions/25010369/wget-curl-large-file-from-google-drive
	pip install gdown
	gdown https://drive.google.com/uc?id={}