Sparse matrices with named rows and columns

Anton Antonov
​Python-packages at GitHub​
​PythonForPrediction at WordsPress​
December 2021
September 2022

Introduction

This Python package, "SSparseMatrix", provides the class SSparseMatrix the objects of which are sparse matrices with named rows and columns.
We can say the package attempts to cover as many as possible of the functionalities for sparse matrix objects that are provided by R's library Matrix . R is a implementation of S. S introduced named data structures for statistical computations, [RB1], hence the name SSparseMatrix .)
The package builds on top of the `scipy` sparse matrices . (The added functionalities though are general -- other sparse matrix implementations could be used.)
Here is a list of functionalities provided for SSparseMatrix :
◼
  • Sub-matrix extraction by row and column names:
    • ◼
  • Single element access
    • ◼
  • Subsets of row names and column names
    • ◼
  • Slices (with integers)
    • ◼
  • Row and column names propagation for dot products with:
    • ◼
  • Lists
    • ◼
  • Dense vectors (`numpy.array`)
    • ◼
  • scipy sparse matrices
    • ◼
  • SSparseMatrix objects
    • ◼
  • Row and column sums
    • ◼
  • Vector form
    • ◼
  • Dictionary form
    • ◼
  • Transposing
    • ◼
  • Representation:
    • ◼
  • Tabular, matrix form ("pretty printing")
    • ◼
  • String and repr forms
    • ◼
  • Row and column binding of SSparseMatrix objects
    • ◼
  • "Export" functions
    • ◼
  • Triplets
    • ◼
  • Row-dictionaries
    • ◼
  • Column-dictionaries
    • ◼
  • Wolfram Language full form representation
    • The full list of features and development status can be found in the org-mode file SSparseMatrix-work-plan.org .
    This package more or less follows the design of the Mathematica package SSparseMatrix.m .
    The usage examples below can be also run through the file "examples.py" .

    Usage in other packages

    The class SSparseMatrix is foundational in the packages SparseMatrixRecommender and LatentSemanticAnalyzer . (The implementation of those packages was one of the primary motivations to develop SSparseMatrix .)
    The package RandomSparseMatrix can be used to generate random sparse matrices SSparseMatrix objects.)

    Wolfram Language interfacing

    The package has a stubs for interfacing with Wolfram Language (WL). See the last section.

    Installation

    Install from GitHub

    〉
    pip install -e git+https://github.com/antononcube/Python-packages.git#egg=SSparseMatrix-antononcube\&subdirectory=SSparseMatrix

    From PyPi

    〉
    pip install SSparseMatrix

    Setup

    Import the package:
    〉
    In[]:=
    from SSparseMatrix import *
    The import command above is equivalent to the import commands:
    〉
    In[]:=
    from SSparseMatrix.SSparseMatrix import SSparseMatrix
    from SSparseMatrix.SSparseMatrix import make_s_sparse_matrix
    from SSparseMatrix.SSparseMatrix import is_s_sparse_matrix
    from SSparseMatrix.SSparseMatrix import column_bind

    Creation

    Create a sparse matrix with named rows and columns (a SSparseMatrix object):
    〉
    In[]:=
    mat = [[1, 0, 0, 3], [4, 0, 0, 5], [0, 3, 0, 5], [0, 0, 1, 0], [0, 0, 0, 5]]
    smat = SSparseMatrix(mat)
    smat.set_row_names(["A", "B", "C", "D", "E"])
    smat.set_column_names(["a", "b", "c", "d"])
    Out[]=
    ExternalObject
    Command: SSparseMatrix
    System: Python
    ​
    Type: PythonObject
    IsModule: False
    
    Print the created sparse matrix:
    〉
    In[]:=
    smat.print_matrix()
    ===================================
    | a b c d
    -----------------------------------
    A | 1 . . 3
    B | 4 . . 5
    C | . 3 . 5
    D | . . 1 .
    E | . . . 5
    ===================================
    Another way to create using the function make_s_sparse_matrix :
    〉
    In[]:=
    ssmat=make_s_sparse_matrix(mat)
    ssmat
    Out[]=
    ExternalObject
    Command: SSparseMatrix
    System: Python
    ​
    Type: PythonObject
    IsModule: False
    

    Structure

    The SSparseMatrix objects have a simple structure. Here are the attributes: - _sparseMatrix - _rowNames - _colNames - _dimNames
    Here are the methods to "query" SSparseMatrix objects: - sparse_matrix() - row_names() and row_names_dict() - column_names() and column_names_dict() - shape() - dimension_names()
    SSparseMatrix over-writes the methods of scipy.sparse.csr_matrix that might require the handling of row names and column names.
    Most of the rest of the scipy.sparse.csr_matrix methods are delegated to the _sparseMatrix attribute.
    For example, for a given SSparseMatrix object smat the dense version of smat 's sparse matrix attribute can be obtained by accessing that attribute first and then using the method todense :
    〉
    In[]:=
    print(smat.sparse_matrix().todense())
    [[1 0 0 3]
    [4 0 0 5]
    [0 3 0 5]
    [0 0 1 0]
    [0 0 0 5]]
    Alternatively, we can use the "delegated" form and directly invoke todense on smat :
    〉
    In[]:=
    print(smat.todense())
    [[1 0 0 3]
    [4 0 0 5]
    [0 3 0 5]
    [0 0 1 0]
    [0 0 0 5]]
    Here is another example showing a direct application of the element-wise operation sin through the scipy.sparse.csr_matrix method sin :
    〉
    In[]:=
    smat.sin().print_matrix(n_digits=20)
    ===================================================================================
    | a b c d
    -----------------------------------------------------------------------------------
    A | 0.8414709848078965 . . 0.1411200080598672
    B | -0.7568024953079283 . . -0.9589242746631385
    C | . 0.1411200080598672 . -0.9589242746631385
    D | . . 0.8414709848078965 .
    E | . . . -0.9589242746631385
    ===================================================================================

    Representation

    Here the function print uses the string representation of SSparseMatrix object:
    〉
    In[]:=
    print(smat)
    ('A', 'a') 1
    ('A', 'd') 3
    ('B', 'a') 4
    ('B', 'd') 5
    ('C', 'b') 3
    ('C', 'd') 5
    ('D', 'c') 1
    Here we print the representation obtained with `repr` :
    〉
    In[]:=
    print(repr(smat))
    Here is the matrix form ("pretty printing" ):
    〉
    In[]:=
    smat.print_matrix()
    The method triplets can be used to obtain a list of `(row, column, value)` triplets:
    〉
    In[]:=
    smat.triplets()
    The method row_dictionaries gives a dictionary with keys that are row-names and values that are column-name-to-matrix-value dictionaries:
    〉
    In[]:=
    smat.row_dictionaries()
    Similarly, the method column_dictionaries gives a dictionary with keys that are column-names and values that are row-name-to-matrix-value dictionaries:
    〉
    In[]:=
    smat.column_dictionaries()

    Multiplication

    Multiply with the transpose and print:
    〉
    In[]:=
    ssmat2 = ssmat.dot(smat.transpose())
    ssmat2.print_matrix()
    Multiply with a list-vector:
    〉
    In[]:=
    smat3 = smat.dot([1, 2, 1, 0])
    smat3.print_matrix()
    Remark: The type of the .dot argument can be: - SSparseMatrix - list - numpy.array - scipy.sparse.csr_matrix

    Slices

    Single element access:
    〉
    In[]:=
    print(smat["A", "d"])
    print(smat[0, 3])
    Get sub-matrix of rows using row names:
    〉
    In[]:=
    smat[["A", "D", "B"], :].print_matrix()
    Get sub-matrix using row indices:
    〉
    In[]:=
    smat[[0, 3, 1], :].print_matrix()
    Get sub-matrix with columns names:
    〉
    In[]:=
    smat[:, ['a', 'c']].print_matrix()
    Get sub-matrix with columns indices:
    〉
    In[]:=
    smat[:, [0, 2]].print_matrix()
    Remark: The current implementation of scipy (1.7.1) does not allow retrieval of sub-matrices by specifying both row and column ranges or slices.
    Remark: "Standard" slices with integers also work.

    Row and column sums

    Row sums and dictionary of row sums:
    〉
    In[]:=
    print(smat.row_sums())
    print(smat.row_sums_dict())
    Column sums and dictionary of column sums:
    〉
    In[]:=
    print(smat.column_sums())
    print(smat.column_sums_dict())

    Column and row binding

    Column binding

    Here we create another SSparseMatrix object:
    〉
    In[]:=
    mat2=smat.sparse_matrix().transpose()
    smat2 = SSparseMatrix(mat2, row_names=list("ABCD"), column_names="c")
    smat2.print_matrix()
    Here we column-bind two SSparseMatrix objects:
    〉
    In[]:=
    smat[list("ABCD"), :].column_bind(smat2).print_matrix()
    Remark: If during column-binding some column names are duplicated then to the column names of both matrices are added suffixes that designate to which matrix each column belongs to.

    Row binding

    Here we rename the column names of smat to be the same as smat2 :
    〉
    In[]:=
    smat3 = smat.copy()
    smat3.set_column_names(smat2.column_names()[0:4])
    smat3 = smat3.impose_column_names(smat2.column_names())
    smat3.print_matrix()
    Here we row-bind smat2 and smat3 :
    〉
    In[]:=
    smat2.row_bind(smat3).print_matrix()
    Remark: If during row-binding some row names are duplicated then to the row names of both matrices are added suffixes that designate to which matrix each row belongs to.

    In place computations

    ◼
  • The methods for setting row- and column-names are "in place" methods -- no new SSparseMatrix objects a created.
    • ◼
  • The dot product, arithmetic, and transposing methods have an optional argument whether to do computations in place or not.
    • ◼
  • The optional argument is copy , which corresponds to argument with the same name and function in scipy.sparse .
    • ◼
  • By default, the computations are not in place: new objects are created.
    • ◼
  • I.e. copy=True default.
    • ◼
  • The class SSparseMatrix has the method copy() that produces deep copies when invoked.

    • Unit tests

    The unit tests (so far) are broken into functionalities; see the folder ./tests . Similar unit tests are given in [AAp3].

    Interfacing with Wolfram Language

    In this section we describe a way to interface the Python package "SSparseMatrix", [AAp1] with WL and the WL package "SSparseMatrix.m", [AAp2].
    Here we import AAp2:
    Provide definition to the deferred method to_wl:
    〉
    In[]:=
    from SSparseMatrix import *
    from wolframclient.language import wl
    from wolframclient.serializers import export, wolfram_encoder
    ​
    # Provide definition of the deferred to_wl() method
    def _my_to_wl(self):
    return wl.SSparseMatrix.ToSSparseMatrix(self.to_dict())
    SSparseMatrix.to_wl = _my_to_wl
    ​
    @wolfram_encoder.dispatch(SSparseMatrix)
    def encode_s_sparse_matrix(serializer, ssmat):
    # encode the class as a symbol called SSparseMatrix
    return serializer.encode(ssmat.to_wl())
    〉
    In[]:=
    smat3.to_dict()
    〉
    In[]:=
    smat3

    References

    Articles

    [AA1] Anton Antonov, "RSparseMatrix for sparse matrices with named rows and columns" , (2015), MathematicaForPrediction at WordPress .
    [RB1] Richard Becker, "A Brief History of S", (2004).

    Packages

    [AAp1] Anton Antonov, SSparseMatrx Python package, (2021), PyPI.org.
    [AAp2] Anton Antonov, SSparseMatrix.m , (2018), MathematicaForPrediction at GitHub .
    [AAp3] Anton Antonov, SSparseMatrix Mathematica unit tests , (2018), MathematicaForPrediction at GitHub .