# Python Cheat Sheet
# Resources
- [Python list implementation](https://www.laurentluce.com/posts/python-list-implementation/)
- [TimeComplexity](https://wiki.python.org/moin/TimeComplexity)
- https://realpython.com/learning-paths/writing-pythonic-code/
- Books:
- [Python Tricks - Dan Bader](https://www.amazon.com/Python-Tricks-Buffet-Awesome-Features/dp/1775093301)
- [Effective Python - Brett Slatkin](https://effectivepython.com/)
# Language
## Python Quirks
- Mod operator:
- `-1 % 10` results in `9` instead of `-1`. Use `math.fmod(-1, 10)` instead of `-1 % 10`
- Rounding integers towards zero
- In most cases rounding down `-0.1` towards zero should give `0`, this is not the case in Python unless you are using `int(x/y)`. The table below shows that using `int(x/y)` always works.
| Operation | Result | Correct |
| --------------------- | ------ | ------- |
| `math.floor(-1 / 10)` | -1 | No |
| `math.floor(1 / 10)` | 0 | Yes |
| `1 // 10` | 0 | Yes |
| `-1 // 10` | -1 | No |
| `int(-1 / 10)` | 0 | Yes |
| `int(1 / 10)` | 0 | Yes |
## Primitives
- Integers in Python3 are unbounded, the maximum integer representable is a function of the available memory
- Unlike integers, floats are not infinite precision, and it's convenient to refer to infinity as `float('inf')` and `float('-inf')`. These values are comparable to integers, and can be used to create psuedo max-int and pseudo min-int.
## For loops
- Index based loop
```python
for i in range(4):
print(i)
```
- Loop through list
```python
x = [1,2,3]
for n in x:
print(n)
```
- Loop through list with index
```python
for i, n in enumerate(nums):
print(i, n)
```
- Loop through dictionary
```python
x = {'x': 1, 'y': 2, 'z': 3}
for k, v in x.items():
print(k, v)
```
- Loop through characters of string
```python
x = "hello"
for c in x:
print(c)
```
## Sorting
- Documentation: https://wiki.python.org/moin/HowTo/Sorting#Sortingbykeys
- Starting with Python 2.2, sorts are guaranteed to be [stable](http://en.wikipedia.org/wiki/Sorting_algorithm#Stability).
- Sort list ASC by lambda
```python
x = [2,3,1]
new_list = sorted(x, key=lambda x: x)
```
- Operator Module Functions: Python provides convenience functions to make accessor functions easier and faster. The [operator module](http://docs.python.org/library/operator.html#module-operator) has `itemgetter`, `attrgetter`, and `methodcaller`
```python
>>> from operator import itemgetter, attrgetter, methodcaller
>>> sorted(student_tuples, key=itemgetter(2))
[('dave', 'B', 10), ('jane', 'B', 12), ('john', 'A', 15)]
>>> sorted(student_objects, key=attrgetter('age'))
[('dave', 'B', 10), ('jane', 'B', 12), ('john', 'A', 15)]
# The operator module functions allow multiple levels of sorting.
# For example, to sort by grade then by age:
>>> sorted(student_tuples, key=itemgetter(1,2))
[('john', 'A', 15), ('dave', 'B', 10), ('jane', 'B', 12)]
>>> sorted(student_objects, key=attrgetter('grade', 'age'))
[('john', 'A', 15), ('dave', 'B', 10), ('jane', 'B', 12)]
```
- Sort list in-place
```python
x = [2,3,1]
x.sort()
print(x) # sorted list
```
- Sort string
```python
x = "zxa"
sorted(x) # ["a", "x", "z"]
''.join(sorted(x)) # "axz"
```
## Data Structures
### Namedtuple
- Usage:
```python
from collections import namedtuple
# Define namedtuple type
Point = namedtuple('Point', 'x y')
# Create namedtuple instances
point1 = Point(1, 2)
point2 = Point(3, 4)
# To access elements, use attribute names instead of indexes:
x_coord = point1.x
y_coord = point2.y
```
- Operations:
- Namedtuples are fully iterable and unpackable like regular tuples.
- They support comparisons `(==, !=, etc.)` based on their elements.
- They are immutable, meaning their elements cannot be changed after creation.
- Additional features:
- Field names: Can be any valid Python identifier (avoid keywords).
- Optional rename argument: Automatically replaces invalid/duplicate names.
- _asdict() method: Converts namedtuple to a dictionary.
- _replace() method: Creates a new namedtuple with modified elements.
- Immutability: Ensures data consistency and simplifies multi-threaded programming.
### Dictionary
- `defaultdict`: https://realpython.com/python-defaultdict/
- Get dictionary keys
```python
x = {'x': 1, 'y': 2, 'z': 3}
keys = list(x) # much better!
keys = list(x.keys())
```
### List
- Instantiation:
```python
[3, 5, 7, 11]
[1] + [0] * 10
list(range(100))
```
- Existence check:
```python
1 in [1,2,3]
```
- Copy list
```python
# Shallow copy
B = list(A) # or copy.copy(B) or A[:]
# Deep copy
B = copy.deepcopy(A)
```
- Binary search sorted list
```python
bisect.bisect(A,6)
bisect.bisect_left(A,6)
bisect.bisect_right(A,6)
```
- Reverse list
```python
A.reverse() # in-place
reversed(A) # returns an iterator, wrap with list()
```
- Sort list
```python
A.sort() # in-place
sorted(A) # returns copy
```
- Delete items from list
```python
del A[i] # delete single item
del A[i:j] # remove slice
```
- Slicing a list
- `A[start:stop:step]` with all of `start`, `stop`, and `step` being optional
```python
# index 0 1 2 3 4 5 6
# reversed -5 -6 -5 -4 -3 -2 -1
A = [1, 6, 3, 4, 5, 2, 7]
A[2:4] # [3,4]
A[2:] # [3,4,5,2,7]
A[:4] # [1,6,3,4]
A[:-1] # [1,6,3,4,5,2] (except last item)
A[-3:] # [5,2,7] (index from end)
A[-3:-1] # [5,2],
A[1:5:2] # [6, 4]
A[5:1:-2] # [2, 4]
A[::-1] # [7, 2, 5, 4, 3, 6, 1] (reverses list)
A[k:] + A[:k] # rotates A by k to the left
```
- List comprehension
- A list comprehension consists of:
1. An input sequence
2. An iterator over the input sequence
3. A logical condition over the iterator (optional)
4. An expression that yields the elements of the derived list.
```python
[x ** 2 for x in range(6)] # [0, 1, 4, 9, 16, 25]
[x ** 2 for x in range(6) if x % 2 == 0] # [0, 4, 16]
```
- Nested
- As a general rule, it is best to avoid more than two nested comprehensions, and use conventional nested for loops
```python
A = [1, 3, 5]
B = ['d', 'b']
[(x, y) for x in A for y in B]
# [(1, 'a'), (1, 'b'), (3, 'a'), (3, 'b'), (5, 'a'), (5, 'b')]
```
- Also supported in `set` and `dict`
### Set
- Lookup operation is O(1)
- Set is implemented as hashmap
### Counter
- docs: https://docs.python.org/3/library/collections.html#collections.Counter
- Create counter
```python
from collections import Counter
freq = Counter([1,1,1,2,2])
print(freq) # freq {1: 3, 2: 2}
```
### Heap and Priority Queue
- Create heap (default = min-heap)
```python
import heapq
h = []
heapq.heappush(h, 5)
heapq.heappush(h, 0)
print([heapq.heappop(h) for i in range(len(h))]) # [0 5]
```
- Create max heap or priority queue
```python
import heapq
h = []
heapq.heappush(h, (-2, 5)) # priority = 2 (highest)
heapq.heappush(h, (-1, 0)) # priority = 1
print([heapq.heappop(h)[1] for i in range(len(h))]) # [5, 0]
```
### String
- Pad string from left
```python
"4".rjust(4, "0") # 0004
```
- String formatting (Python 3.6+)
```python
>>> f'Hello, {name}'
'Hello, Bob'
>>> '{:.2}'.format(0.1234) # Limit number of decimals to show
'0.12'
# Format to Hex
>>> f'{errno:#x}'
'0xbadc0ffee'
```
- Trim / Strip characters
- By default `[l|r]strip` functions remove whitespace when no argument is passed
```python
>>> s1 = '__abc__'
>>> s1.lstrip('__') # Trim from left
abc__
>>> s1.rstrip('__') # Trim from right
__abc
>>> s1.strip('__') # Trim from left and right
abc
```
### Stack
- Using list
```python
s = []
# push O(1)
s.append(1)
s.append(2)
print(s) # [1,2]
# pop O(1)
s.pop() # 2
print(s) # [1]
```
### Deque
- [**collections.deque**](https://docs.python.org/3.7/library/collections.html#collections.deque) (pronounced "deck") - double-ended queue
- Usage:
```python
from collections import deque
d = deque(['a','b','c'])
print(d) # deque(['a','b','c'])
# Append from the right side of list
d.append("f") # O(1)
print(d) # deque(['a','b','c', 'f'])
# Pop from the right side of list
x = d.pop() # O(1)
print(x) # 'f'
print(d) # deque(['a','b','c'])
# Append from the left side of list
d.appendleft("z") # O(1)
print(d) # deque(['z', 'a','b','c'])
# Pop from the left side of list
x = d.popleft() # O(1)
print(x) # 'z'
print(d) # deque(['a','b','c'])
```
### Queue
- Usage:
```python
from collections import deque
queue = deque()
# Append from right side O(1)
queue.append(1)
queue.append(2)
print(queue) # deque([1,2])
# Pop from left side O(1)
queue.popleft() # 1
print(queue) # deque([2])
```
### Linked List
- Using `collections.deque`
```python
from collections import deque
l = deque()
# Append from end
l.append(1)
l.append(2)
# Remove end
l.pop()
# iterate over items
for n in d:
print(n)
# or
while queue:
print(queue.popleft())
```
- Using custom data structure
```python
class Node:
def __init__(self, data):
self.data = data
self.next = None
def __repr__(self):
return self.data
class LinkedList:
def __init__(self):
self.head = None
def __repr__(self):
node = self.head
nodes = []
while node is not None:
nodes.append(node.data)
node = node.next
nodes.append("None")
return " -> ".join(nodes)
```
# Unit testing
- https://docs.python-guide.org/writing/tests/
# Pandas
- [Pandas Illustrated: The Definitive Visual Guide to Pandas](https://scribe.citizen4.eu/pandas-illustrated-the-definitive-visual-guide-to-pandas-c31fa921a43)
# Python Tricks
### 1. Trailing Comma
- Smart formatting and comma placement can make your list, dict, or set constants easier to maintain.
- Python’s string literal concatenation feature can work to your benefit, or introduce hard-to-catch bugs.
- Gotcha:
```python
>>>'hello' 'world'
'helloworld'
```
- Always add trailing comma to container literal (list or dict):
```python
>>> names = [
... 'Alice',
... 'Bob',
... 'Dilbert', # <- this one
... ]
```
### 2. Context managers - Supporting `with` in Your Own Objects
- Context manager in Python is a an interface that your object needs to follow in order to support the `with` statement.
- Basically, all you need to do is add `__enter__` and `__exit__` methods to an object if you want it to function as a context manager. Python will call these two methods at the appropriate times in the resource management cycle.
- Example 1: using class based approach
```python
class ManagedFile:
def __init__(self, name):
self.name = name
def __enter__(self):
self.file = open(self.name, 'w') return self.file
def __exit__(self, exc_type, exc_val, exc_tb): if self.file:
self.file.close()
```
- Example 2: using `contextlib.contextmanager`
```python
from contextlib import contextmanager
@contextmanager
def managed_file(name): try: # generator function!
f = open(name, 'w')
yield f finally:
f.close()
```
- Both examples can be used as:
```python
>>> with ManagedFile('hello.txt') as f:
... f.write('hello, world!')
... f.write('bye now')
```
### 3. Asserts
- Python’s assert statement is a debugging aid that tests a condition as an internal self-check in your program. Example:
```python
assert counter == 10, "counter should be equal to 10"
```
- Why asserts?
> the proper use of assertions is to inform developers about unrecoverable errors in a program. Assertions are not intended to signal expected error conditions, like a File-Not-Found error, where a user can take corrective actions or just try again.
>
> Assertions are meant to be internal self-checks for your program. They work by declaring some conditions as impossible in your code. If one of these conditions doesn’t hold, that means there’s a bug in the pr gram.
>
> If your program is bug-free, these conditions will never occur. But if they do occur, the program will crash with an assertion error telling you exactly which “impossible” condition was triggered. This makes it much easier to track down and fix bugs in your programs. And I like anything that makes life easier—don’t you?
- Caveats
- Caveat #1 – Don’t Use Asserts for Data Validation
- Asserts should only be used to help developers identify bugs. They’re not a mechanism for handling run-time errors.
- Asserts can be globally disabled with an interpreter setting.
- Caveat #2 – Asserts That Never Fail due to syntax mis-interpretation
- This has to do with non-empty tuples always being truthy in Python
```python
assert(1 == 2, 'This should fail')
```
### 4. Underscores
1. Single Leading Underscore `_var`
- Single underscores are a Python naming **convention** that indicates a name is meant for internal use. It is generally not enforced by the Python interpreter and is only meant as a hint to the programmer.
- if you use a wildcard import to import all the names from the module, Python will not import names with a leading underscore (unless the module defines an `__all__` list that overrides this behavior)
2. Single Trailing Underscore: `var_`
- A single trailing underscore (postfix) is used by **convention** to avoid naming conflicts with Python keywords. Example:
```python
def make_object(name, class): # SyntaxError: "invalid syntax"
pass
def make_object(name, class_):
pass
```
3. Double Leading Underscore: `__var`
- Name mangling: the interpreter changes the name of the variable in a way that makes it harder to create collisions when the class is extended later.
```python
class Test:
def __init__(self):
self.foo = 11
self._bar = 23
self.__baz = 23
>>> t = Test()
>>> dir(t)
['_Test__baz', '_bar', 'foo', '__class__', '__dict__', ...]
```
- `__bar` becomes `_Test__baz`
4. Double Leading and Trailing Underscore: `__var__`
- Reserved for special use in the language. This rule covers things like `__init__` for object constructors, or `__call__` to make objects callable.
5. Single underscore `_`
- Per convention, a single stand-alone underscore is sometimes used as a name to indicate that a variable is temporary or insignificant.
```python
for _ in range(32):
print('Hello, World.')
```
- You can also use single underscores in unpacking expressions as a “don’t care” variable to ignore particular values.
- This meaning is per convention only and it doesn’t trigger any special behaviors in the Python parser. The single underscore is simply a valid variable name that’s sometimes used for this purpose.
### 5. String interpolation
- Dan’s Python String Formatting Rule of Thumb:
> If your format strings are user-supplied, use Template Strings to avoid security issues. Otherwise, use Literal String Interpolation if you’re on Python 3.6+, and “New Style” String Formatting if you’re not.
- "New Style" String Formatting
```python
>>> errno = 50159747054
>>> name = 'Bob'
>>> f"Hey {name}, there's a {errno:#x} error!"
"Hey Bob, there's a 0xbadc0ffee error!"
```
### 6. Functions
- Functions are objects
```python
def yell(text):
return text.upper() + '!'
>>> yell('hello')
'HELLO!'
>>> bark = yell
>>> bark('woof')
'WOOF!'
```
- The name of a function is just a pointer to the object where the function is stored, you can have multiple names pointing to same function.
- Python attaches a string identifier to every function at creation time for debugging purposes
```python
>>> bark.__name__
'yell'
```
- Functions Can Be Stored in Data Structures
```python
>>>funcs = [bark, str.lower, str.capitalize]
>>> for f in funcs:
... print(f, f('hey there'))
'HEY THERE!'
'hey there'
'Hey there'
>>> funcs[0]('heyho') # call a function object stored
'HEYHO!'
```
- Functions Can Be Passed to Other Functions
```python
>>> def greet(func):
... greeting = func('Hi, I am a Python program')
... print(greeting)
>>> def whisper(text):
... return text.lower() + '...'
>>> greet(whisper)
'hi, i am a python program...'
```
- Higher-order function
- The ability to pass function objects as arguments to other functions is powerful. It allows you to abstract away and pass around behavior in your programs. In this example, the greet function stays the same but you can influence its output by passing in different greeting behaviors. Functions that can accept other functions as arguments are also called higher-order functions.
- The classical example for higher-order functions in Python is the built-in `map` function:
```python
>>> list(map(bark, ['hello', 'hey', 'hi']))
['HELLO!', 'HEY!', 'HI!']
```
- Functions Can Be Nested
```python
def speak(text):
def whisper(t):
return t.lower() + '...'
return whisper(text)
>>> speak('Hello, World')
'hello, world...'
```
- Functions Can Capture Local State
```python
def get_speak_func(text, volume):
def whisper():
return text.lower() + '...'
def yell():
return text.upper() + '!'
if volume > 0.5:
return yell
else:
return whisper
>>> get_speak_func('Hello, World', 0.7)()
'HELLO, WORLD!'
```
- Functions that do this are called closures. A closure remembers the values from its enclosing lexical scope even when the program flow is no longer in that scope.
- Objects Can Behave Like Functions
- While all functions are objects in Python, the reverse isn’t true
- But objects can be made callable, which allows you to treat them like functions in many cases and invoke them with `()` syntax using the `__call__` method.
```python
class Adder:
def __init__(self, n):
self.n = n
def __call__(self, x):
return self.n + x
>>> plus_3 = Adder(3)
>>> plus_3(4)
7
```
- "calling" an object as a function attempts to execute the object’s `__call__` method.
- Use `callable(