AN APPROACH TO COMPUTER-AIDED RECONSTRUCTION OF MUSEUM EXHIBITS

The article discusses the possibility of using 3D laser and 3D structured light scanning technology to support the restoration of museum objects and presents the concept of applying techniques of 3D scanning and CAD processing to create a model of the missing fragment of a museum object, based on scans of the damaged surface and fragments of the object with a similar shape. The resulting model can be used as a base element in the restoration of the original shape of the artifact, both virtually and in reality. The paper also presents the proposal of a process of reconstructing a missing fragment model of an actual museum object (an exhibit from the Zamoyski Museum in Kozłówka) using the method in question and mobile 3D scanning equipment.


INTRODUCTION
The use of advanced 3D computer technology in the field of cultural heritage has for many years interested numerous scientists from different disciplines.Archaeologists, conservators and art historians are trying to put into practice 3D scanning devices in order to obtain a digital representation of objects and thus create and analyse their virtual models.Computer scientists want to create more and more powerful software allowing faster and better data processing in order to increase the efficiency of the process of generating photorealistic virtual models of historic buildings.Engineers take actions aimed at a rapid exchange of a virtual object for its real copy.
Computer action in the area of three-dimensional treatment of historic buildings can be divided into three main groups.First comes the aspect of data collection, with their evaluation and optimisation, as well as their processing.Secondly, reconstruction of a small loss or a damage of considerable size by generating mesh surfaces well match-ing the existing elements.Third: creation of solid (water-resistant) models, which can be replicated by using modern rapid prototyping technology.
The first area of activity can be found in numerous works.Pavlidis, Koutsoudis, Arnaoutoglou and Tsioukas in [10] describe a number of digitisation methods, trying to determine their suitability for scanning different types of museum objects by preparing a set of nine criteria for their evaluation.In addition, they show examples of attempts to scan objects by using a laser and a structural light scanner.Wachowiak and Karas [15] in addition to the review of methods for 3D imaging of objects of cultural heritage, present the criteria for the selection of the system and techniques of working with data, identifying the types of objects that are not suitable for digitisation.Vozikis, Haring, Vozikis and Kraus in [14] present issues concerning the practice of obtaining a point cloud of very large archaeological objects, e.g. an ancient theatre or large figural objectsa historic Art Nouveau monument.They analyse the accuracy of the scanning method compared to the photogammetric method and ways to reduce the number of acquired points for their faster processing and obtaining a virtual 3D model.The question of the size of the created objects is also studied by Comes, Buna and Badiu in [4] on the example of 3D scanning of objects of Dacian culture of different sizes, using the laser and photogrammetry methods.The obtained point cloud is then converted into mesh objects, and in the end superimposed with textures.The development of effective tools for matching the virtual surface represented by point clouds is dealt with by Liu, Pottmann and Wang [7].As a practical example they show surfaces reflecting a ceramic pitcher.The developed method does not address objects represented by surface models, e.g.mesh or solid ones suitable for 3D printing.Procedures for conducting scanning and data processing are presented by Sitnik, Mączkowski and Krzesłowski [12].The first algorithm defined conduct in the process of data backup in order to preserve maximum information about the shape of an object.The second concerned the visualisation process, and the third -obtaining copies on a 3D printer.Although presented algorithms include the issue being major part in our approach, the described method involves the use of a stationary scanning system.Such a system is optimal for small overall dimensions of museum objects that can be moved to the scanning platform.Using of such a system for scanning objects in the exhibition site is either difficult or completely impossible.Such a case is a common issue as exhibits are usually not permitted to be moved from its place of exhibition without a complicated and time consuming procedure .The use of handheld scanners allows to scan such objects on site.Scan procedure for handheld scanners, despite some differences, leads to a model of the same type.
The issue of virtual reconstruction of architectural objects is addressed by Patay-Horváth [9].The author has attempted a virtual 3D reconstruction of a mythical scene, the exact interpretation of which is still unexplained by art historians.At the first stage, on the basis of a critical assessment of existing information on the structure itself and its sculptural decorations, a computer model of the temple was prepared.Then, by 3D scanning digital replicas of models of statues were made by making the necessary virtual reconstruction.At the end, a virtual model of the eastern front end of the temple was made in four alternative versions.Li, Luo and Zha in [6] created a virtual model of the statue using the method of reverse engineering, which consisted of 4 stages: 3D scanning, 3D recording, geometric modelling and texture mapping.The model thus constructed is set in a virtual environment modelled in a 3D environment.
The issue of reconstruction of museum objects is addressed by Pereira, Martínez and Lourenço in [11].The authors scanned fragments of a damaged sculpture of the crucified Jesus Christ.Digitised elements were assembled in the virtual world and then non-existent fragments recreated by generating mesh surfaces.This gave a virtual reconstruction of the entire figure, which then allowed to sculpt the missing pieces and complete the restoration of the sculpture in the real world.Castagno [3] presents an experiment carried out in the church of San Martino dall'Argine built by Scipione Gonzaga in 1582, which is the main religious architectural monument of Mantua.Using modern computer 3D technology: scanning, modelling and printing, reconstruction was performed of the altar in a side chapel, which had two cherub heads missing.For this purpose heads of other cherubs in the altar were used.
An important issue that affects virtual representations of museum objects is the possibility of their proper accessibility.This issue is dealt with by Berndt, Buchgraber, Havemann, Settgast and Fellner [2] to create a concept of online accessibility of digital 3D models, which are associated with metadata as additional information about the object.The idea of sharing discussed also allows for the collective search of virtual museum objects characterised by certain common elements.
The aim of paper is the preparation of an effective procedure in the process of reconstruction of damaged elements of museum exhibits.It involves selection of the equipment and the parameters of the 3D scanning process, as well as the tools for processing the obtained point cloud.

IDEA OF RECONSTRUCTION
Considering the above, this article focuses mainly on the second group of interests -aiding the reconstruction of museum objects.Currently damaged objects (especially when some of their parts are missing), are reconstructed manually on the basis of the reconstruction team's knowledge and skills.A common approach is using a pattern from an existing and undamaged part of an object under reconstruction or from other objects of similar characteristics.[6,9,11], the application of 3D technologies in the renovation process can be particularly useful in two cases.The first one considers the situation when the object is physically supplemented with substitutes of the missing parts that are reconstructed using the object's 3D scan.The second case applies when the interference in the object's current state is undesirable or prohibited.The reconstruction can still be conducted on an object's model that can be used for 3D printing or presented virtually.The latter case allows for the presentation of such a reconstructed model in electronic media.

As was indicated on examples
This paper presents an approach to use 3D scanning techniques (supplemented by computer aided construction of 3D models) for the reconstruction and digitization of items belonging to the cultural heritage.The proposed method assumes preparation of a digital model of the missing fragment, based on 3D scans of the damaged part of an object as well as areas of the object that follow the same pattern as the missing part.The selection of such areas is left for a specialist in the domain of object renovation, nevertheless 3D reconstruction of the chosen area will belong to the domain of computer science.The virtually reconstructed part of an object can then be physically recreated using CNC or 3D print methods and used in the process of object reconstruction.Together with physical reconstruction a visualisation of the complete form of the object can be presented for reference.
The issue resented in the article covers the area from the border of two domains: protection of cultural heritage and widely understood computer science.This reveals the need to build an agreement between representatives of the two fields, in particular in the common area, where the understanding of the tasks and the acceptability of the methods of their implementation may be different (Fig. 1).The cooperation of the Institute of Computer Science Lublin University of Technology, Zamoyski Museum in Kozłówka and Alisher Navoi Samarkand State University allowed for the formation of such preliminary agreement and finding solutions acceptable from the point of view of the protection of cultural heritage as well as conducting computer science activities.

SELECTION OF EQUIPMENT AND TECHNIQUES
The problems associated with the selection of scanning devices and establishing the digitisation process for archiving of museum objects has already been discussed in [5,8,13].The main emphasis in these articles is put on model fidelity and precision of the object being scanned.Without denying the findings of that work, it should be considered whether the same criteria apply in the case of the discussed approach.One should agree with the view presented in the above-mentioned articles, but striving for the highest scan resolution and model precision is restrained not only by the maximum precision of the scanning equipment, but also by the available maximum precision of 3D printing, measured by the thickness of the applied layer.
Table 1 shows the average accuracy of the current types of scanners and 3D printers that can be used in museum applications.As can be seen there, the use of high-precision desktop scanners allows for scans of the precision achievable only by expensive stereolitography printers.The de-  sired precision of a printout depends on a fragment surface complication and can be lower in case of simple shapes.When using a less precise printing, scans obtained with the help of handheld structural light scanners can be successfully used.The use of a slightly higher precision handheld laser scanners is not always acceptable because of the emitted radiation.
As mentioned, the highest available precision scanner is not the optimal solution in every case.When scanning an object of a complex shape it is necessary to perform multiple scans from different positions in relation to the object.In the absence of a possibility or permission to move the scanned object from its exposition place the stationary scanner solution turns out to be inapt.The use of handheld scanners makes it possible to scan all the available surfaces of the object in only a few cycles.Allowing for the execution of the scanning process on the exhibition site and without moving the exhibits was also desirable for the heritage preservation institutions, because it avoided the implementation of complicated procedures to protect the exhibits during their movement.To sum up: handheld scanners were chosen for the procedure in spite of their smaller accuracy, due to their better suitability on site consisting in reducing the risk of harming the exhibits.

COMPUTER-AIDED RENOVATION PROCEDURE
The proposal of a procedure for computerassisted restoration of museum artifacts assumes the existence of four stages that require the diverse involvement of professionals from both fields, Cultural heritage and IT (Fig. 2).
The first, scanning stage begins with identifying all the areas to be scanned: damage areas and references.That part greatly involves the heritage domain experts.Then the 3D scanning procedure is performed until all the scans are acquired in satisfactory quality.The separate scans are processed, joined and cleaned up.IT domain experts are mostly involved here although the supervision of heritage domain expert is required in order to avoid actions that could harm the exhibit.
The second, construction stage includes removing any unwanted areas from the scans, leaving only the surfaces needed to compose the missing fragment.It is particularly important to determine the shape of the damage surface, as it will be directly connected to the exhibit.The edges of the damage region are not always clearly distinguished on the object's scan surface (by means of a 3D position).The place of damage can alternatively be distinguished from its surroundings by its colour.Thus additional information, such as the texture of that region, can reveal surface borders by utilising recognition of the edges.The next step in this stage is matching the surfaces to construct the missing fragment.Reference surfaces are fitted around the area of damage.For mutual adjustment of their point clouds, point cloud fitting algorithms can be used, but it should be noted that the clouds from different areas or even objects are matched together, which can lead to false correspondences.So far the most reliable method is to use a simple 3-point-fitting algorithm in combination with a further elimination of mesh parts protruding from the desired shape.Selection of the fit point pairs should be done with care in order to preserve the desired shape continuity.Again the recorded texture can be helpful to match similar colour patterns on the surface.The current stage involves also transition to the mesh model with the utilization of algorithms such as poisson surface reconstruction (Amenta 2014) together with the use of computer aided modelling and design techniques in order Fig. 2. The workflow of the procedure for computer-assisted restoration of museum artifacts to clear the inaccuracies and fill the gaps and receive a waterproof model.It should be noted here that computer aided design (abbreviation CAD) is commonly reserved to applications working on parametric models described mathematically, like AutoCAD, NX, Solid Edge, etc. Applications that operate on triangle meshes directly (like 3DS Max or Blender) are not regarded as CAD software, although these are also utilised to design models of 3D objects.When considering reconstruction of object part from a mesh acquired by 3D scanning, it is more suitable to use the latter software than to conduct parameterisation of complicated surfaces with the former one (classical CAD).
The visualization stage isn't mandatory.It serves as a reference creating possibility.The fragment model is virtually matched to the proper damage area to verify the continuity of its surfaces in relation to the main object and to eliminate any remaining discontinuities.The display of a virtual connection of a newly created fragment to the scan of an exhibit can also serve as a guide for a further physical fragment assembly.
Finally the printing stage covers the activities required to print.The fragment model is examined for potential problems when printing with chosen method.This may include too thin features, fragile connections and so on.Basing on heritage conservation expert decision such fragments will need to be strengthened or deleted.Dependent on chosen print method, supports may need to be added for proper conduction of print process.Supports will need to be removed when printing is finished.

REAL EXHIBIT TESTS
When considering the test, 2 cultural preservation institution were involved: Zamoyski Museum in Kozłówka and Alisher Navoi Samarkand State University.Several types of artifacts for potential reconstruction were chosen and test scans were made.Figure 3 shows objects selected for test scanning with Artec Spider Scanner.
Finally the full procedure has been tested while scanning real museum object acquired in the Zamoyski Museum in Kozłówka.A scan of a decorative picture frame with well seen damage places was selected.At the current stage of testing, only the 3 first stages of the procedure were considered due to the lack of permission for physical interference with the exhibit.
Figure 4 shows a section of the scanned frame containing the damaged spot, which is marked with a different color spot existing on acquired texture.Edges of the scanned point cloud and texture facilitate the location of the defect.The b) part of the figure presents a fragment containing the potential reference element to compose the structure of the missing fragment.
Figure 5 shows the same areas in the form of the scanned model.Similarly, part a) shows the damage place with the texture applied, and part b) the reference fragment.Both parts are acquired and processed in the form of a triangle mesh.As mentioned above, the core CAD software operates on models consisting of parametric elements.Converting a non-regular surface to a parametric model is not a trivial task and is not required at a final stage, since 3D replicators use as input layer models created from waterproof meshes.Thus the mesh processing software is more applicable here, like 3DS Max, Blender, Meshlab and similar.
Although the fragment chosen as the reference one is considered the best match available, it cannot be fitted to the damaged region as is.It represents a mirror image of the desired shape of the missing part.A vertical flip of its mesh can be utilised with basic tools, e.g.Blender.
The flipped part needs to fit into the damage spot, which requires the use of standard 3D mesh modification techniques such as repositioning vertexes, deleting faces, filling holes, etc. Due to the fact that a common basic shape for all potentially missing fragments cannot be defined, the automatising of this stage is not fully developed and so far reduced to techniques aiding with extraction of the damaged surface and its edges.That surface serves as a mold.The reconstructed part's surfaces are adjusted to fit the mold edges.This allows for seamless joining with the main model.The adequate surplus in length of this part of the surface was left for greater freedom of element matching.The final fragment fitting requires aligning to the main model, which can be aided by the 3-point-mesh-matching technique.Any left inconsistencies in both fitted meshes are at this stage resolved manually to create a fully seamless connection (Fig. 6).

CONCLUSIONS
The proposed combination of 3D scanning and computer aided 3D modelling techniques proved to be a significant aid in renovation of cultural heritage objects.Thus acquiring a substitute for a missing part of a historical object has moved from hand-art to the computer science domain.
The four-stage procedure, concerning the designing of a supplemented object model based on a scan of its current appearance was successfully used with a real life object, leading to the creation of a reconstructed model, without the need to redesign the missing parts from scratch.
Using a handheld scanner and the structured light technology allowed for conducting the scanning process without moving the exhibit from its place of exposure.This is particularly important in the case of museum objects, whose displacement is often not allowed or impossible.The use of the structured light movable scanner method allowed to reach all the needed fragments of the complicated shape of the picture frame.The current feature-and-texture based scanner positioning algorithms were sufficient to avoid placing markers on the exhibit, which would be unacceptable due to the protection of its surface.
The requirement for high scanning equipment resolution is less important than the ability to conduct the scanning with a possibly minimal object disturbance.Using the proposed method meets that requirement while still producing scans of a resolution suitable for most 3D printers.
Further work needs to be conducted to implement automatized algorithms of seamless mesh and texture connection.This would allow to simplify the procedure of fitting the parts of the fragment being reconstructed.

Fig. 1 .
Fig. 1.The graphical representation of the overlapping area in cultural Heritage and Information Science domain

Fig. 3 .
Fig. 3. Objects selected for test scans with Artec Spider scanner: a) object from Samarkand, b) object from Kozłówka

Fig. 4 .Fig. 5 .
Fig. 4. Real image of the frame parts, a) the area around the missing fragment, b) fragment containing the potential reference element

Fig. 6 .
Fig. 6.Part of the frame model a) with missing fragment, b) with fragment restored

Table 1 .
The average accuracy of the current types of scanners and 3D printers