Python
Introduction
Introduction
This notebook (and other notebooks) were used in the presentation "Random mandalas deconstruction with R, Python, and Mathematica" given at "Boston useR! Meetup" in February 2022. Here is a link to presentation's recording.
Here is presentation's abstract:
In this presentation we discuss the application of different dimension reduction algorithms over collections of random mandalas. We discuss and compare the derived image bases and show how those bases explain the underlying collection structure. The presented techniques and insights (1) are applicable to any collection of images, and (2) can be included in larger, more complicated machine learning workflows. The former is demonstrated with a handwritten digits recognition
application; the latter with the generation of random Bethlehem stars. The (parallel) walk-through of the core demonstration is in all three programming languages: Mathematica, Python, and R.
application; the latter with the generation of random Bethlehem stars. The (parallel) walk-through of the core demonstration is in all three programming languages: Mathematica, Python, and R.
Remark: This is a Mathematica notebook with Python code. There are corresponding notebooks with Mathematica/WL and R/RStudio with Python, R, and WL codes.
Workflow
Workflow
Here is a diagram that shows the overall process followed in all notebooks used in the presentation:
Out[]=
Get Python mandalas collection
Get Python mandalas collection
Setup
Setup
In[]:=
# "Standard" packages
import pandas
import numpy
import random
import io
import time
# SSparseMatrix, SMR, and LSA packages
from SSparseMatrix import *
from SparseMatrixRecommender import *
from LatentSemanticAnalyzer import LatentSemanticAnalyzer
# Plotting packages
import matplotlib
import matplotlib.pyplot as plt
from mpl_toolkits.axes_grid1 import ImageGrid
# Image processing package(s)
from PIL import Image, ImageOps
# Random mandalas packages
from RandomMandala import random_mandala, figure_to_image
import pandas
import numpy
import random
import io
import time
# SSparseMatrix, SMR, and LSA packages
from SSparseMatrix import *
from SparseMatrixRecommender import *
from LatentSemanticAnalyzer import LatentSemanticAnalyzer
# Plotting packages
import matplotlib
import matplotlib.pyplot as plt
from mpl_toolkits.axes_grid1 import ImageGrid
# Image processing package(s)
from PIL import Image, ImageOps
# Random mandalas packages
from RandomMandala import random_mandala, figure_to_image
In[]:=
pythonSession=First@Pick[ExternalSessions[],#["System"]=="Python"&/@ExternalSessions[]]
Out[]=
ExternalSessionObject
In[]:=
from SSparseMatrix import *
from wolframclient.language import wl
from wolframclient.serializers import export, wolfram_encoder
# Provide definition of the deferred to_wl() method
def _my_to_wl(self):
return wl.SSparseMatrix.ToSSparseMatrix(self.to_dict())
SSparseMatrix.to_wl = _my_to_wl
@wolfram_encoder.dispatch(SSparseMatrix)
def encode_s_sparse_matrix(serializer, ssmat):
# encode the class as a symbol called SSparseMatrix
return serializer.encode(ssmat.to_wl())
from wolframclient.language import wl
from wolframclient.serializers import export, wolfram_encoder
# Provide definition of the deferred to_wl() method
def _my_to_wl(self):
return wl.SSparseMatrix.ToSSparseMatrix(self.to_dict())
SSparseMatrix.to_wl = _my_to_wl
@wolfram_encoder.dispatch(SSparseMatrix)
def encode_s_sparse_matrix(serializer, ssmat):
# encode the class as a symbol called SSparseMatrix
return serializer.encode(ssmat.to_wl())
Out[]=
ExternalFunction
Mandala collection
Mandala collection
In[]:=
# A list to accumulate random mandala images
mandala_images = []
# Generation loop
random.seed(443)
tstart = time.time()
for i in range(512):
# Generate one random mandala figure
fig2 = random_mandala(n_rows=None,
n_columns=None,
radius=[8, 6, 3],
rotational_symmetry_order=6,
symmetric_seed=True,
connecting_function='bezier_fill',
face_color="0.",
alpha = 1.0)
fig2.tight_layout()
# Convert the figure into an image and add it to the list
mandala_images = mandala_images + [figure_to_image(fig2)]
# Close figure to save memory
plt.close(fig2)
# Invert image colors
mandala_images2 = [ImageOps.invert(img) for img in mandala_images]
# Binarize images
mandala_images3 = [im.convert('1') for im in mandala_images2]
# Resize images
width, height = mandala_images3[0].size
print([width, height])
ratio = height / width
new_width = 200
mandala_images4 = [img.resize((new_width, round(new_width * ratio)), Image.ANTIALIAS) for img in mandala_images3]
width, height = mandala_images4[0].size
print([width, height])
print("Process time: " + str(time.time()-tstart))
mandala_images = []
# Generation loop
random.seed(443)
tstart = time.time()
for i in range(512):
# Generate one random mandala figure
fig2 = random_mandala(n_rows=None,
n_columns=None,
radius=[8, 6, 3],
rotational_symmetry_order=6,
symmetric_seed=True,
connecting_function='bezier_fill',
face_color="0.",
alpha = 1.0)
fig2.tight_layout()
# Convert the figure into an image and add it to the list
mandala_images = mandala_images + [figure_to_image(fig2)]
# Close figure to save memory
plt.close(fig2)
# Invert image colors
mandala_images2 = [ImageOps.invert(img) for img in mandala_images]
# Binarize images
mandala_images3 = [im.convert('1') for im in mandala_images2]
# Resize images
width, height = mandala_images3[0].size
print([width, height])
ratio = height / width
new_width = 200
mandala_images4 = [img.resize((new_width, round(new_width * ratio)), Image.ANTIALIAS) for img in mandala_images3]
width, height = mandala_images4[0].size
print([width, height])
print("Process time: " + str(time.time()-tstart))
[640, 480]
[200, 150]
Process time: 22.455907106399536
Process random mandalas collection
Process random mandalas collection
Show one of the random mandalas:
In[]:=
mandala_images4[11]
Out[]=
Show an array of inverted random mandalas:
In[]:=
random.sample(mandala_images4, 36)
Out[]=
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
Convert each image into array and flatten that array:
In[]:=
mandala_arrays = [numpy.asarray(x, dtype="int32") for x in mandala_images4]
len(mandala_arrays)
len(mandala_arrays)
Out[]=
512
Make a matrix of flattened mandalas:
In[]:=
mandalaMat = [x.reshape(x.shape[0] * x.shape[1]) for x in mandala_arrays]
mandalaMat = numpy.array(mandalaMat)
mandalaMat.shape
mandalaMat = numpy.array(mandalaMat)
mandalaMat.shape
Out[]=
{512,30000}
Make the corresponding SSparseMatrix object:
In[]:=
mandalaSMat = SSparseMatrix(mandalaMat, row_names="", column_names="")
print(repr(mandalaSMat))
print(repr(mandalaSMat))
<512x30000 SSparseMatrix (sparse matrix with named rows and columns) of type '<class 'numpy.int32'>'
with 3436475 stored elements in Compressed Sparse Row format, and fill-in 0.22372884114583333>
LSAMon object creation
LSAMon object creation
Assign the number of topics for the computations below:
In[]:=
numberOfTopics = 40
Create the LSAMon object:
In[]:=
tStart = time.time()
lsaObj = LatentSemanticAnalyzer().set_document_term_matrix(mandalaSMat).apply_term_weight_functions("None", "None", "None")
tEnd = time.time()
print("\n\t\tCreation time : ", tEnd-tStart)
lsaObj = LatentSemanticAnalyzer().set_document_term_matrix(mandalaSMat).apply_term_weight_functions("None", "None", "None")
tEnd = time.time()
print("\n\t\tCreation time : ", tEnd-tStart)
Creation time : 0.07116007804870605
SVD
SVD
Here we extract image topics using Singular Value Decomposition (SVD):
In[]:=
tStart = time.time()
lsaSVDObj = lsaObj.extract_topics(number_of_topics=numberOfTopics, min_number_of_documents_per_term=0, method="SVD", max_steps=50)
tEnd = time.time()
print("Topic extraction time SVD : ", tEnd-tStart)
lsaSVDObj = lsaObj.extract_topics(number_of_topics=numberOfTopics, min_number_of_documents_per_term=0, method="SVD", max_steps=50)
tEnd = time.time()
print("Topic extraction time SVD : ", tEnd-tStart)
Topic extraction time SVD : 1.5853939056396484
Get the SVD topics matrix:
In[]:=
svdH = lsaSVDObj.normalize_matrix_product(normalize_left=False).take_H().copy()
NNMF
NNMF
Here we extract image topics using Non-Negative Matrix Factorization (NNFM):
In[]:=
tStart = time.time() lsaNNMFObj=(LatentSemanticAnalyzer() .set_document_term_matrix(mandalaSMat) .apply_term_weight_functions("None", "None", "None") .extract_topics(number_of_topics=numberOfTopics, min_number_of_documents_per_term=0, method="NNMF", max_steps=16))tEnd = time.time()print("Topic extraction time NNMF : ", tEnd-tStart)
Get the NNMF topics matrix:
In[]:=
nnmfH = lsaNNMFObj.normalize_matrix_product(normalize_left=False).take_H().copy()
ICA
ICA
Not implemented in Python yet.
Show topics interpretation
Show topics interpretation
SVD
SVD
Show the importance coefficients of the topics (if SVD was used the plot would show the singular values):
Get the topics matrix from the LSA object and convert it into a dense array:
In[]:=
svdTopicsArray = svdH.sparse_matrix().todense()
svdTopicsArray.shape
svdTopicsArray.shape
Show the interpretation of the extracted image topics:
In[]:=
imageSizeX, imageSizeY = mandala_images4[0].size
svdTopicImages = [(svdTopicsArray[i,:]*255).reshape(imageSizeY, imageSizeX) for i in range(svdTopicsArray.shape[0])]
svdTopicImages2 = [Image.fromarray(x) for x in svdTopicImages]
svdTopicImages = [(svdTopicsArray[i,:]*255).reshape(imageSizeY, imageSizeX) for i in range(svdTopicsArray.shape[0])]
svdTopicImages2 = [Image.fromarray(x) for x in svdTopicImages]
In[]:=
svdTopicImages2
"Image adjusted" basis:
NNMF
NNMF
Show the importance coefficients of the topics :
In[]:=
nnmfTopicsArray = nnmfH.sparse_matrix().todense()
nnmfTopicsArray.shape
nnmfTopicsArray.shape
Show the interpretation of the extracted image topics:
In[]:=
imageSizeX, imageSizeY = mandala_images4[0].size
nnmfTopicImages = [(nnmfTopicsArray[i,:]*255).reshape(imageSizeY, imageSizeX) for i in range(nnmfTopicsArray.shape[0])]
nnmfTopicImages2 = [Image.fromarray(x) for x in nnmfTopicImages]
nnmfTopicImages = [(nnmfTopicsArray[i,:]*255).reshape(imageSizeY, imageSizeX) for i in range(nnmfTopicsArray.shape[0])]
nnmfTopicImages2 = [Image.fromarray(x) for x in nnmfTopicImages]
In[]:=
nnmfTopicImages2
"Image adjusted" basis:
In[]:=
nnmfTopicImages3 = [(numpy.clip(a = nnmfTopicsArray[i,:], a_min = 0.01, a_max = 1) * 255).reshape(imageSizeY, imageSizeX) for i in range(nnmfTopicsArray.shape[0])]
nnmfTopicImages4 = [Image.fromarray(x) for x in nnmfTopicImages3]
nnmfTopicImages4 = [Image.fromarray(x) for x in nnmfTopicImages3]
"Image adjusted" basis clipped:
Clip the right, topics factor of NNMF LSA object:
In[]:=
nnmfH.sparse_matrix() > 0.01
ICA
ICA
Not implemented in Python yet.
Approximation
Approximation
Get a new, unseen mandala:
In[]:=
# Consider using radius = [8, 6, 3]
fig = random_mandala(radius=[8, 6, 3],
rotational_symmetry_order=6,
symmetric_seed=True,
connecting_function='bezier_fill',
face_color="0.",
alpha = 1.0)
# Convert the mandala figure to image
testMandala = figure_to_image(fig)
testMandala
fig = random_mandala(radius=[8, 6, 3],
rotational_symmetry_order=6,
symmetric_seed=True,
connecting_function='bezier_fill',
face_color="0.",
alpha = 1.0)
# Convert the mandala figure to image
testMandala = figure_to_image(fig)
testMandala
In[]:=
# Resize image
testMandala1 = testMandala.resize((new_width, round(new_width * ratio)), Image.ANTIALIAS)
# Invert image colors
testMandala2 = ImageOps.invert(testMandala)
# Binarize image
testMandala3 = testMandala2.convert('1')
# Resize image
testMandala4 = testMandala3.resize((new_width, round(new_width * ratio)), Image.ANTIALIAS)
# The image
testMandala4
testMandala1 = testMandala.resize((new_width, round(new_width * ratio)), Image.ANTIALIAS)
# Invert image colors
testMandala2 = ImageOps.invert(testMandala)
# Binarize image
testMandala3 = testMandala2.convert('1')
# Resize image
testMandala4 = testMandala3.resize((new_width, round(new_width * ratio)), Image.ANTIALIAS)
# The image
testMandala4
Get the corresponding vector:
In[]:=
# Flatten array:
testMandalaArray = numpy.asarray(testMandala4, dtype="int32")
testMandalaArray = numpy.asarray(testMandala4, dtype="int32")
Make a query matrix:
In[]:=
testMandalaMat = testMandalaArray.reshape(1, testMandalaArray.shape[0] * testMandalaArray.shape[1])
testMandalaMat = numpy.array(testMandalaMat)
matQuery = SSparseMatrix(testMandalaMat, row_names="", column_names="")
print(repr(matQuery))
testMandalaMat = numpy.array(testMandalaMat)
matQuery = SSparseMatrix(testMandalaMat, row_names="", column_names="")
print(repr(matQuery))
SVD
SVD
Represent by topics:
In[]:=
resMat = lsaSVDObj.represent_by_topics(matQuery, method="recommendation").take_value()
resMat
resMat
In[]:=
approx = resMat.dot(svdH)
approx2 = apply_term_weight_functions(approx, "None", "None", "AbsMax")
approx2.data
approx2 = apply_term_weight_functions(approx, "None", "None", "AbsMax")
approx2.data
In[]:=
approxImg = (255 - approx2.todense().reshape(imageSizeY, imageSizeX)*255)
approxImg2 = Image.fromarray(approxImg)
fig = plt.figure(figsize=(5., 5.))
grid = ImageGrid(fig, 111,
nrows_ncols=(1,2),
axes_pad=0.02,
)
for ax, img in zip(grid, [approxImg2, testMandala1]):
ax.imshow(img)
ax.set(xticks=[], yticks=[])
figure_to_image(fig)
approxImg2 = Image.fromarray(approxImg)
fig = plt.figure(figsize=(5., 5.))
grid = ImageGrid(fig, 111,
nrows_ncols=(1,2),
axes_pad=0.02,
)
for ax, img in zip(grid, [approxImg2, testMandala1]):
ax.imshow(img)
ax.set(xticks=[], yticks=[])
figure_to_image(fig)
Direct manipulation:
NNMF
NNMF
Represent by topics:
In[]:=
resMat = lsaNNMFObj.represent_by_topics(matQuery, method="recommendation").take_value()
resMat
resMat
In[]:=
approx = resMat.dot(nnmfH)
approx2 = apply_term_weight_functions(approx, "None", "None", "AbsMax")
approx2.data
approx2 = apply_term_weight_functions(approx, "None", "None", "AbsMax")
approx2.data
In[]:=
approxImg = (255 - approx2.todense().reshape(imageSizeY, imageSizeX)*255)
approxImg2 = Image.fromarray(approxImg)
fig = plt.figure(figsize=(5., 5.))
grid = ImageGrid(fig, 111,
nrows_ncols=(1,2),
axes_pad=0.02,
)
for ax, img in zip(grid, [approxImg2, testMandala1]):
ax.imshow(img)
ax.set(xticks=[], yticks=[])
figure_to_image(fig)
approxImg2 = Image.fromarray(approxImg)
fig = plt.figure(figsize=(5., 5.))
grid = ImageGrid(fig, 111,
nrows_ncols=(1,2),
axes_pad=0.02,
)
for ax, img in zip(grid, [approxImg2, testMandala1]):
ax.imshow(img)
ax.set(xticks=[], yticks=[])
figure_to_image(fig)
Direct manipulation (sometimes produces better results):
Remark: I have not found Python implementation sparse matrix pseudo inversion. (It is in my ToDo list to implement one.) Hence the approximation results with NNMF are not that good. (Compare with the Mathematica / WL application of NNMF.)
ICA
ICA
Not implemented yet in Python.