Kumar Abhishek kumar-abhishek

Purpose

Bootstrap knowledge of LLMs ASAP. With a bias/focus to GPT.

Avoid being a link dump. Try to provide only valuable well tuned information.

Prelude

Neural network links before starting with transformers.

Useful materials:

Machine Learning cheatsheet: https://stanford.edu/~shervine/teaching/cs-229.html
Pattern Recognition and Machine Learning Book

import boto3

client = boto3.client('glue')

response = client.create_crawler(
    Name='SalesCSVCrawler',
    Role='AWSGlueServiceRoleDefault',
    DatabaseName='sales-cvs',
    Description='Crawler for generated Sales schema',

Create 2 files: main.go and docker-compose.yml. Once both are created use docker-compose up and it will build the necessary files and start Elastic Search.

I found it necessary to add elastic.SetSniff(false) or I could not connect. Also remember the docker containers have security enabled for Elasticsearch with the password set as elastic:changeme. You can test it using curl:

curl http://127.0.0.1:9200/_cat/health -u elastic:changeme

Once it is running you can successfully run the main.go file using go run main.go.

	import pandas as pd
	import time
	import requests
	import time
	import hmac
	from requests import Request
	import sys
	import json
	import os

	## Imports
	from typing import Tuple
	import torch
	from torch import Module, Tensor
	from transformers.models.roberta.modeling_roberta import RobertaPreTrainedModel, RobertaConfig, RobertaModel, RobertaEncoder
	from torch.nn import CrossEntropyLoss, CosineEmbeddingLoss



	## Function

	class HardExampleMiner(tf.keras.utils.Sequence):

	def __init__(self, model, x, y, batch_size, map_fn=None, ratio=0.8):
	self.x = np.array(x, dtype=np.float32)
	self.y = np.array(y, dtype=np.float32)
	self.batch_size = batch_size
	self.model = model
	self.ratio = ratio
	self.num_of_batch = int(math.floor((len(self.x) / self.batch_size)))
	self.hard_idxs = np.arange(self.num_of_batch * self.batch_size)

	import torch as th


	class NLL_OHEM(th.nn.NLLLoss):
	""" Online hard example mining.
	Needs input from nn.LogSotmax() """

	def __init__(self, ratio):
	super(NLL_OHEM, self).__init__(None, True)
	self.ratio = ratio