Skip to content

Memory Management

Reference counting

Reference counting is the primary method Python uses to manage memory. Each object in Python maintains a count of references pointing to it. When the reference count drops to zero, the memory occupied by the object is deallocated.

1
2
3
4
5
import gc
import json
import sys
import tracemalloc
from typing import Optional

One reference is coming from a and another from getrefcount, so there are two references.

1
2
3
a = []

print(sys.getrefcount(a))

2

One reference is coming from a, other for b and another from getrefcount, so there are three references.

1
2
3
b = a

print(sys.getrefcount(a))

3

When b is deleted, one value is automatically deallocated.

1
2
3
del b

print(sys.getrefcount(a))

2

Garbage Collection

Python includes a cyclic garbage collector to handle reference cycles. Reference cycles occur when objects reference each other, preventing their reference counts from reaching zero.

1
2
3
4
# Enable gc
gc.enable()
# Disable gc
gc.disable()

1
gc.collect()

6

Get garbage collection stats.

1
print(json.dumps(gc.get_stats(), indent=4))

[

    {

        "collections": 158,

        "collected": 1510,

        "uncollectable": 0

    },

    {

        "collections": 14,

        "collected": 187,

        "uncollectable": 0

    },

    {

        "collections": 2,

        "collected": 31,

        "uncollectable": 0

    }

]

Get unreachable objects

1
print(gc.garbage)

[]

Handling circular references.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
class SomeObject:
    def __init__(self, name):
        self.name = name
        self.ref: Optional[SomeObject] = None
        print(f"Object: {name} created")

    def __del__(self):
        print(f"Object: {self.name} deleted ")


foo = SomeObject("foo")
bar = SomeObject("bar")

# Circular reference
foo.ref = bar
bar.ref = foo

# Due to the circular reference, the objects are not garbage collected
del foo
del bar

Object: foo created

Object: bar created

The garbage collector should be manually triggered.

1
gc.collect()

Object: foo deleted 

Object: bar deleted

35

Generators for memory efficiency

1
2
3
4
5
6
7
8
9
def generate_numbers(number: int):
    for item in range(number):
        yield item


for number in generate_numbers(1000):
    print(number)
    if number > 9:
        break

0

1

2

3

4

5

6

7

8

9

10

Profiling memory usage

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
def create_list():
    return [number for number in range(10000)]


def main():
    tracemalloc.start()

    create_list()

    snapshot = tracemalloc.take_snapshot()
    top_stats = snapshot.statistics("lineno")

    print("[Top 10] memory consumption files")

    for index, stat in enumerate(top_stats[:10]):
        frame = stat.traceback[0]
        print(
            "#%s: %s:%s: %.1f KiB"
            % (index, frame.filename[21:], frame.lineno, stat.size / 1024)
        )


main()

[Top 10] memory consumption files

#0: /.local/share/uv/python/cpython-3.12.6-linux-x86_64-gnu/lib/python3.12/contextlib.py:105: 0.2 KiB

#1: /.cache/uv/archive-v0/tb82FWm1ArcUI7kLRed3k/lib/python3.12/site-packages/zmq/sugar/attrsettr.py:45: 0.1 KiB

#2: /.cache/uv/archive-v0/tb82FWm1ArcUI7kLRed3k/lib/python3.12/site-packages/ipykernel/iostream.py:287: 0.1 KiB

#3: /.cache/uv/archive-v0/tb82FWm1ArcUI7kLRed3k/lib/python3.12/site-packages/ipykernel/iostream.py:276: 0.1 KiB

#4: /.local/share/uv/python/cpython-3.12.6-linux-x86_64-gnu/lib/python3.12/contextlib.py:301: 0.1 KiB

#5: /.cache/uv/archive-v0/tb82FWm1ArcUI7kLRed3k/lib/python3.12/site-packages/IPython/core/history.py:1011: 0.1 KiB

#6: /.local/share/uv/python/cpython-3.12.6-linux-x86_64-gnu/lib/python3.12/asyncio/base_events.py:815: 0.1 KiB

#7: /.cache/uv/archive-v0/tb82FWm1ArcUI7kLRed3k/lib/python3.12/site-packages/ipykernel/iostream.py:722: 0.1 KiB

#8: /.cache/uv/archive-v0/tb82FWm1ArcUI7kLRed3k/lib/python3.12/site-packages/IPython/core/history.py:1030: 0.1 KiB

#9: /.cache/uv/archive-v0/tb82FWm1ArcUI7kLRed3k/lib/python3.12/site-packages/IPython/core/history.py:1009: 0.1 KiB