Skip to content

Instantly share code, notes, and snippets.

View muellerzr's full-sized avatar

Zach Mueller muellerzr

1.4k followers · 15 following
View GitHub Profile
@muellerzr
muellerzr / pr_template.md
Created April 5, 2022 20:10
My personal PR template that I'll be experimenting with for the next few months

{feature_name}

What does this add?

{summary}

Who is it for?

{audience/issues closed}

@muellerzr
muellerzr / source-code-tracking.ipynb
Created June 2, 2021 19:45
Source Code Tracking
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@redknightlois
redknightlois / ralamb.py
Last active August 9, 2023 20:50
Ralamb optimizer (RAdam + LARS trick)
class Ralamb(Optimizer):
def __init__(self, params, lr=1e-3, betas=(0.9, 0.999), eps=1e-8, weight_decay=0):
defaults = dict(lr=lr, betas=betas, eps=eps, weight_decay=weight_decay)
self.buffer = [[None, None, None] for ind in range(10)]
super(Ralamb, self).__init__(params, defaults)
def __setstate__(self, state):
super(Ralamb, self).__setstate__(state)