Dense matrices implementation in Python

Types of the elements

We can see the type used (descr in C) with:

a.dtype

which outputs dtype('int64'). It means that the elements are ints stored with 64 bits each. We can change this type, and then see data as bytes with:

a = a.astype('int16')
b = a.data.tobytes()
print(b)

It outputs b'\x01\x00\x02\x00\x03\x00\x04\x00\x05\x00\x06\x00'. Here, \x01 is the hexadecimal string representation of the bits contained in one byte: for instance 0 is represented by \x00 and 255 is represented by \xff (we can store \(256 = 2^8\) values in one byte). Since the values are now stored with 16 bits, i.e. 2 bytes, we can retrieve the first two elements of the array (1 and 2) with:

int.from_bytes(b[:2], 'little')
int.from_bytes(b[2:4], 'little')

Two questions arise:

• What does little mean?
• And why is 2 the second element and not 4?

Little endian or big endian

Here, little corresponds to little-endian. It means that we read the bytes from left to right to get the number represented. If instead we want the data to be stored in big-endian, we can change it and see the data with:

a = a.astype('>i2')
print(a.data.tobytes())

It now outputs b'\x00\x01\x00\x02\x00\x03\x00\x04\x00\x05\x00\x06'. Note here that each pair of bytes (corresponding to one element of the array) is inverted. We can read the array without changing the data but reading it as if it were stored in little-endian:

a.newbyteorder('<')

which outputs:

array([[ 256,  512,  768],
       [1024, 1280, 1536]], dtype=int16)

Let's understand what happened by looking at each element of the array (i.e. two bytes). With big-endian:

In this table, we wrote the least significant bit on the right (i.e. if this bit is 1 then it corresponds to \(2^0\) in the decomposition, the second from the right corresponds to \(2^1\) and so on). In big-endian, the bytes are read from right to left. If instead we read the same bytes in little-endian (i.e. the least important byte is on the left), we have the following numbers:

Note that we inverted the order of the bytes but not all the bits as a whole: for \x00\x01 we get 00000001 00000000 in little-endian and not 10000000 00000000.

We saw how each element is stored individually. Let's see why the second element in the data buffer is 2 (first row and second column) and not 4 (first column and second row).

Column major or row major

The second element is 2 because the rows are considered to be laid out as contiguous blocks: 1, 2, 3, 4, 5, 6 (the first row is 1, 2, 3 which are contiguous and the second row is 4, 5, 6). We name it "row major" or "C-style contiguous". The flags property (a.flags) gives us how the data is actually stored in memory:

C_CONTIGUOUS : True
F_CONTIGUOUS : False
OWNDATA : True
WRITEABLE : True
ALIGNED : True
WRITEBACKIFCOPY : False
UPDATEIFCOPY : False

In this case the data is stored in C-style. We can also store it as contiguous blocks of columns, called "Fortran-style contiguous" or "column major" with:

y = np.asfortranarray(a)

The flags changed:

C_CONTIGUOUS : False
F_CONTIGUOUS : True
OWNDATA : True
WRITEABLE : True
ALIGNED : True
WRITEBACKIFCOPY : False
UPDATEIFCOPY : False

but the data didn't: a.data == y.data returns True. The data array is still indexed as C-style, even if the underlying data is stored in memory as Fortran-style. This unables to manipulate the array the same way independently of how it is stored in memory. Storing it as row major or column major influences on the efficiency of the operations on the array: for C contiguous memory layout, operations such as a.sum(axis=1) (sums the rows) are usually faster than column-wise operations since they access memory addresses which are next to each other. a.strides gives us how far in bytes it has to step to get to the next value, repectively for the next row and the next column. Here it is (6, 2) for C-style and (2, 4) for Fortran-style.

We saw how each data element is stored and how elements are laid out along with the different fields in the C structure and in Python to get this information. Let's move on to references.