Memory Management and Limits¶
sys includes several functions for understanding and controlling memory usage.
Reference Counts¶
Python uses reference counting and garbage collection for automatic memory management. An object is automatically marked to be collected when its reference count drops to zero. To examine the reference count of an existing object, use getrefcount().
import sys
one = []
print 'At start :', sys.getrefcount(one)
two = one
print 'Second reference :', sys.getrefcount(one)
del two
print 'After del :', sys.getrefcount(one)
The count is actually one higher than expected because there is a temporary reference to the object held by getrefcount() itself.
$ python sys_getrefcount.py
At start : 2
Second reference : 3
After del : 2
Object Size¶
Knowing how many references an object has may help find cycles or a memory leak, but it isn’t enough to determine what objects are consuming the most memory. That requires knowledge about how big objects are.
import sys
class OldStyle:
pass
class NewStyle(object):
pass
for obj in [ [], (), {}, 'c', 'string', 1, 2.3,
OldStyle, OldStyle(), NewStyle, NewStyle(),
]:
print '%10s : %s' % (type(obj).__name__, sys.getsizeof(obj))
getsizeof() reports the size in bytes.
$ python sys_getsizeof.py
list : 72
tuple : 56
dict : 280
str : 38
str : 43
int : 24
float : 24
classobj : 104
instance : 72
type : 904
NewStyle : 64
The reported size for a custom class does not include the size of the attribute values.
import sys
class WithoutAttributes(object):
pass
class WithAttributes(object):
def __init__(self):
self.a = 'a'
self.b = 'b'
return
without_attrs = WithoutAttributes()
print 'WithoutAttributes:', sys.getsizeof(without_attrs)
with_attrs = WithAttributes()
print 'WithAttributes:', sys.getsizeof(with_attrs)
This can give a false impression of the amount of memory being consumed.
$ python sys_getsizeof_object.py
WithoutAttributes: 64
WithAttributes: 64
For a more complete estimate of the space used by a class, provide a __sizeof__() method to compute the value by aggregating the sizes of attributes of an object.
import sys
class WithAttributes(object):
def __init__(self):
self.a = 'a'
self.b = 'b'
return
def __sizeof__(self):
return object.__sizeof__(self) + \
sum(sys.getsizeof(v) for v in self.__dict__.values())
my_inst = WithAttributes()
print sys.getsizeof(my_inst)
This version adds the base size of the object to the sizes of all of the attributes stored in the internal __dict__.
$ python sys_getsizeof_custom.py
140
Recursion¶
Allowing infinite recursion in a Python application may introduce a stack overflow in the interpreter itself, leading to a crash. To eliminate this situation, the interpreter provides a way to control the maximum recursion depth using setrecursionlimit() and getrecursionlimit().
import sys
print 'Initial limit:', sys.getrecursionlimit()
sys.setrecursionlimit(10)
print 'Modified limit:', sys.getrecursionlimit()
def generate_recursion_error(i):
print 'generate_recursion_error(%s)' % i
generate_recursion_error(i+1)
try:
generate_recursion_error(1)
except RuntimeError, err:
print 'Caught exception:', err
Once the recursion limit is reached, the interpreter raises a RuntimeError exception so the program has an opportunity to handle the situation.
$ python sys_recursionlimit.py
Initial limit: 1000
Modified limit: 10
generate_recursion_error(1)
generate_recursion_error(2)
generate_recursion_error(3)
generate_recursion_error(4)
generate_recursion_error(5)
generate_recursion_error(6)
generate_recursion_error(7)
generate_recursion_error(8)
Caught exception: maximum recursion depth exceeded while getting the str of an object
Maximum Values¶
Along with the runtime configurable values, sys includes variables defining the maximum values for types that vary from system to system.
import sys
print 'maxint :', sys.maxint
print 'maxsize :', sys.maxsize
print 'maxunicode:', sys.maxunicode
maxint is the largest representable regular integer. maxsize is the maximum size of a list, dictionary, string, or other data structure dictated by the C interpreter’s size type. maxunicode is the largest integer Unicode point supported by the interpreter as currently configured.
$ python sys_maximums.py
maxint : 9223372036854775807
maxsize : 9223372036854775807
maxunicode: 65535
Floating Point Values¶
The structure float_info contains information about the floating point type representation used by the interpreter, based on the underlying system’s float implementation.
import sys
print 'Smallest difference (epsilon):', sys.float_info.epsilon
print
print 'Digits (dig) :', sys.float_info.dig
print 'Mantissa digits (mant_dig):', sys.float_info.mant_dig
print
print 'Maximum (max):', sys.float_info.max
print 'Minimum (min):', sys.float_info.min
print
print 'Radix of exponents (radix):', sys.float_info.radix
print
print 'Maximum exponent for radix (max_exp):', sys.float_info.max_exp
print 'Minimum exponent for radix (min_exp):', sys.float_info.min_exp
print
print 'Maximum exponent for power of 10 (max_10_exp):', sys.float_info.max_10_exp
print 'Minimum exponent for power of 10 (min_10_exp):', sys.float_info.min_10_exp
print
print 'Rounding for addition (rounds):', sys.float_info.rounds
Note
These values depend on the compiler and underlying system. These examples were produced on OS X 10.6.4.
$ python sys_float_info.py
Smallest difference (epsilon): 2.22044604925e-16
Digits (dig) : 15
Mantissa digits (mant_dig): 53
Maximum (max): 1.79769313486e+308
Minimum (min): 2.22507385851e-308
Radix of exponents (radix): 2
Maximum exponent for radix (max_exp): 1024
Minimum exponent for radix (min_exp): -1021
Maximum exponent for power of 10 (max_10_exp): 308
Minimum exponent for power of 10 (min_10_exp): -307
Rounding for addition (rounds): 1
See also
The float.h C header file for the local compiler contains more details about these settings.
Byte Ordering¶
byteorder is set to the native byte order.
import sys
print sys.byteorder
The value is either big for big-endian or little for little-endian.
$ python sys_byteorder.py
little
See also
- Wikipedia: Endianness
- Description of big and little endian memory systems.
- array, struct
- Other modules that depend on the byte order.