Python path · Memory and modules · Topic 7 of 12

Reference counting and the cycle collector

Not started

CPython manages memory mostly by reference counting: every object records how many references point at it, and the moment that number reaches zero the object is freed. Reference counting can't see objects that point at each other in a loop, so CPython also runs a cycle garbage collector that occasionally looks for groups of objects nothing outside the group can reach. Between the two, you almost never free anything yourself.

Why it matters: It explains why CPython frees files and sockets promptly, why reference cycles linger, and what gc.collect() is actually for.

  1. 1Every object keeps a count

    CPython stores in each object how many references point to it, and adjusts that count on every assignment, call and del.

  2. 2Zero means freed, immediately

    When the count drops to zero the object is deallocated on the spot, running __del__ and freeing anything it referred to.

  3. 3Cycles need a second mechanism

    Objects that refer to each other never reach zero, so a separate cycle collector periodically finds and frees unreachable groups.

  4. 4An implementation detail

    The language only promises that unreachable objects may be reclaimed. PyPy, for example, has no reference counts, so close files with `with`.

1

How it works

The explanation, with examples you can run.

Reference counting

Each object's header holds a count. Binding a name, storing the object in a list, or passing it to a function increments it; rebinding the name, removing it from the list or returning from the function decrements it. When the count hits zero, CPython deallocates the object right there, which also decrements everything it referred to, so whole structures unravel at once.

import gcimport sys class Node:    def __init__(self, name):        self.name = name        self.peer = None    def __del__(self):        print("freed", self.name) a = Node("a")print(sys.getrefcount(a))   # 2: the name a, plus the argument to getrefcountdel a                       # freed a: the count reached zero

del a deletes the name, not the object. The object is freed because that was its last reference. This immediacy is why, in CPython, a file object you forget to close is usually closed as soon as its last reference disappears. It is still a CPython behaviour, not a language rule, and other implementations such as PyPy use a tracing collector and free objects later. Write with open(...) and don't depend on the timing.

Cycles and the collector

b, c = Node("b"), Node("c")b.peer, c.peer = c, b       # a cycle: each keeps the other's count at 1del b, c                    # nothing printedprint("collecting")gc.collect()                # freed b, freed c

After del, the two nodes are unreachable but each still holds a reference to the other. The cycle collector, the gc module, tracks container objects (lists, dicts, class instances and so on; not ints or strings, which can't form cycles). When it runs, it subtracts the references that come from inside the tracked set; anything left with no outside references is garbage, and it is freed even if it has a __del__ method (since Python 3.4, PEP 442).

On 3.12 and 3.13 the collector is generational. New objects start in generation 0; survivors are promoted to 1 and then 2, because objects that have survived a while tend to keep surviving. A collection of generation 0 starts when allocations minus deallocations exceed the first threshold: gc.get_threshold() returns (700, 10, 10) on 3.12, and 3.13 raised the first value to 2000. Python 3.14.0 to 3.14.4 shipped an incremental collector instead, but 3.14.5 reverted to the 3.13 generational one after reports of memory pressure in production.

You rarely need to call gc.collect(). It helps in tests that check for leaks, and some long-running servers call gc.freeze() before forking workers so the collector doesn't touch, and therefore copy, pages shared with the parent.

Avoiding cycles you don't need

Parent pointers, caches and callbacks are the usual sources of cycles. When one side only needs to find the other, not keep it alive, use weakref.ref, WeakValueDictionary or WeakSet: a weak reference does not increase the count and returns None once the object is gone.

Small details worth knowing

Since 3.12 some objects are immortal (PEP 683): None, True, small ints and other core singletons have their count pinned, so sys.getrefcount(None) returns a huge fixed number (4294967295 on a 64-bit 3.12). Small objects are carved out of CPython's own allocator (pymalloc) in larger blocks, so freeing memory in Python does not always shrink the process as the operating system sees it.

2

Check yourself

3 questions. Get them all right to finish this topic.

  1. 1.Two objects refer to each other and nothing else refers to either. In CPython, when are they freed?
  2. 2.Why does `sys.getrefcount(x)` report one more than you expect?
  3. 3.What does a weak reference (weakref.ref) do to an object's reference count?

0 of 3 right so far. Get all three to finish this topic.