Introduction
Introduction
My Wolfram Summer School project is related to my profession - architecture.
The aim of the project was to try to use neural nets and modern computational methods to analyze architectural objects. The use of modern computation methods can allow the analysis of a very large number of buildings in a short time. Perhaps in the future the results of this type of analysis will allow people to discover something interesting and new.
The aim of the project was to try to use neural nets and modern computational methods to analyze architectural objects. The use of modern computation methods can allow the analysis of a very large number of buildings in a short time. Perhaps in the future the results of this type of analysis will allow people to discover something interesting and new.
Project
Project
Database
Database
The database used in the project was prepared by Radim Tyleèek and Radim Šára in 2013. The database included 606 facades from different places and periods divided into two groups: CMP-base (regular, planar and dense facades) and CMP-extended (irregular, non- planar and sparse facades) data sets. Dataset of facade images was assembled at the Center for Machine Perception.
The CMP Facade Database Radim Tyleèek and Radim Šára “Spatial Pattern Templates for Recognition of Objects with Regular Structure” in “Proc. GCPR” 2013, Saarbrucken, Germany
The CMP Facade Database Radim Tyleèek and Radim Šára “Spatial Pattern Templates for Recognition of Objects with Regular Structure” in “Proc. GCPR” 2013, Saarbrucken, Germany
Out[]=
Machine learning
Machine learning
The first stage of the project was to find a neural net that would allow further analysis. Three possible approaches were selected: a classification neural net, a segmentation neural net, and the use of YOLO: Real-Time Object Detection system. Due to the limited period of time, the initial analysis was limited to the first 2 methods.
This project is my first contact with machine learning, which resulted in extending the work time on each of the models. Further work on selected models and trying out other approaches is necessary and will improve the method.
Below are presented the selected neural nets and the results of their training:
This project is my first contact with machine learning, which resulted in extending the work time on each of the models. Further work on selected models and trying out other approaches is necessary and will improve the method.
Below are presented the selected neural nets and the results of their training:
The sgnet1 segmentation network was selected for further work.
The classification networks used are not accurate enough, and their use requires further work on their accuracy in order to use them for architectural research. Segmentation networks provide quantitative data specifying the surface area of architectural elements in the facade picture. The measurement inaccuracy in this case does not prevent the analysis from being attempted. Obtained data, however, do not require accuracy in the spatial location of the elements.
The classification networks used are not accurate enough, and their use requires further work on their accuracy in order to use them for architectural research. Segmentation networks provide quantitative data specifying the surface area of architectural elements in the facade picture. The measurement inaccuracy in this case does not prevent the analysis from being attempted. Obtained data, however, do not require accuracy in the spatial location of the elements.
picture[x_]:=Module[{a,img=ImagePad[ImageResize[x,{388,388}],92,"Reversed"]},segmentation=ImageResize[Colorize[segnet1b[img]],ImageDimensions[x]];a=AssociationMap[0&,Range[12]],Counts@Flatten@segnet1b[img];facadeInformation=Grid[{{Style["windows to facade ratio:","Text"],Style[ToString[PercentForm[N[(a[3]+a[8])/(Total[a]-a[1])],2]],"Text"]},{Style["architectural detail to facade ratio:","Text"],Style[ToString[PercentForm[N[(a[5]+a[6]+a[9]+a[10]+a[11])/(Total[a]-a[1])],2]],"Text"]},{Style["banister to facade ratio:","Text"],Style[ToString[PercentForm[N[(a[7])/(Total[a]-a[1])],2]],"Text"]}},AlignmentLeft];xsegmentationfacadeInformation]
In[]:=
picture
Out[]=
windows to facade ratio: | 20% |
architectural detail to facade ratio: | 23% |
banister to facade ratio: | 11% |
In[]:=
picture
Out[]=
windows to facade ratio: | 6.7% |
architectural detail to facade ratio: | 10% |
banister to facade ratio: | 0.24% |
Analysis
Analysis
In order to reduce measurement errors, larger facade groups divided by region and period of occurrence are analyzed.I decided to divide the facades into three periods:• the Middle Ages - Romanesque and Gothic architecture,• Early Modern Period - the architecture of the Renaissance and Mannerism, Baroque and Rococo, classicism,• 19th and the begging of the 20th century - historicism, orientalism, academism, secessionAnd into 3 regions:• France• Italy• Central EuropeI chose a few interesting (in my opinion) surface relations:• Window surface to the facade surface• The surface of architectural details to the facade surface • Surface of banisters to the facade surface - the database I used contained the specified balconies - examples of buildings selected for analysis rarely contain balconies, however, there is a relationship in the selection of openwork elements - barriers as balconies by the neural net, which results in the change of name to banisters.
facadeSegmentation[y_]:=Module[{a,img=ImagePad[ImageResize[y,{388,388}],92,"Reversed"]},a=AssociationMap[0&,Range[12]],Counts@Flatten@segnet1b[img];"windows to facade ratio"N[(a[3]+a[8])/(Total[a]-a[1])],"architectural detail to facade ratio"N[(a[5]+a[6]+a[9]+a[10]+a[11])/(Total[a]-a[1])],"banister to facade ratio"N[(a[7])/(Total[a]-a[1])]]
In[]:=
allratio="Period"->FileNameTake[#,{-3}],"Region"->FileNameTake[#,{-2}],facadeSegmentation[Import@#]&/@FileNames["C:\\Users\\majaj\\Desktop\\tymczasowe\\wolfram\\Project\\Facades\\Europe\\*\\*\\*.jpg"];
Results
Results
The most interesting results observed include the increase in the window surface in relation to the facade surface and changes in the amount of architectural details. There is a clear increase in the number of details in modern times and then a decrease in the quantity at the turn of the 19th and 20th centuries.
In[]:=
table=Dataset@GroupBy[allratio,{#Period&,#Region&},"n"Length@#,PercentForm/@KeyMap[StringReplace[" ratio"""],Mean[KeyDrop[#,{"Period","Region"}]]]&]
Conclusions
Conclusions
The project presents an introduction to architecture analysis using Wolfram Language. The results show that analyzing building facades using computational methods can lead to interesting results.
I would like to continue working on developing this method by using YOLO to analyze architectural objects and improve the classification and segmentation methods. Improvement of computational methods and training in neural networks includes more accurate analysis and conclusions.
Another element that should be improved is to increase the number of facades to be analyzed. Selected periods are very wide, during which architecture has changed a lot. Such a small test sample may not be allowed to draw an opinion due to a quick test result from the subjective selection of facades for analysis.
I think that improving these two elements will make it possible to try to create a new architecture analysis tool.
I would like to continue working on developing this method by using YOLO to analyze architectural objects and improve the classification and segmentation methods. Improvement of computational methods and training in neural networks includes more accurate analysis and conclusions.
Another element that should be improved is to increase the number of facades to be analyzed. Selected periods are very wide, during which architecture has changed a lot. Such a small test sample may not be allowed to draw an opinion due to a quick test result from the subjective selection of facades for analysis.
I think that improving these two elements will make it possible to try to create a new architecture analysis tool.
Keywords
Keywords
◼
Wolfram Summer School 2020
◼
Machine lerning
◼
Architecture
◼
Neural nets
◼
Facade
Acknowledgment
Acknowledgment
Mentor: Philip Maymin
I would like to thank the people who made this project possible:
My mentor Philip Maymin for many helpful discussions, Markus van Almsick for an additional lecture on neuro nets and creation of the segmenting neural network used in the project and Giulio Alessandrini for explaining the YOLO code.
My mentor Philip Maymin for many helpful discussions, Markus van Almsick for an additional lecture on neuro nets and creation of the segmenting neural network used in the project and Giulio Alessandrini for explaining the YOLO code.
References
References