Murten Panorama Digital Twin Scanning Project - "the making of"Written by Paul BourkeJune 2024
 Navigable interface to the final result at 1/10th of the available resolution.
The following documents my involvement in the creation of the digital twin to the Murten panorama, namely, managing the scanning process and delivering the final stitched image. With a project of this scale, mine is just one contribution, the others include the staff and students at the eM+ laboratories [1] where the project was conceived and managed, Phase One, the engineers who designed and built the motorised rig [10], and the historians and conservators on the project. [See achowledgements] The painting measures about 10 m high and 100 m long, and exists in three parts, each on a separate roll. The project aimed at scanning it at 1000 DPI (dots per inch) or in metric units, 40 pixels per mm, the approximate width of a human hair. The resulting image, combined, is therefore calculated to be around 400,000 pixels high and 4 million pixels long. It is expected to be the highest resolution digital scan of an artwork. In 2004 a modest resolution (0.5 pixels per mm) digital version was created by Cuno Vollenweider by digitising and manually merging/blending together the photographs taken during the 2001 restoration. For a brief history of the painting, it was painted during 1893-1894 by a team of painters lead by Louis Braun. It depicts the battle between the Swiss Confederates and their allies against the army of the Duke of Burgundy in 1476. It was first exhibited between 1894 and 1909 in custom designed rotunda style buildings in Geneva and Zurich. In 1924 it was put into storage where it stayed until a restoration process was conducted, starting in 1996. The painting was briefly put on display at the 2002 Swiss national exposition, located in a cubic structure (monolith) on Lake Murten. Following the exposition the painting was put back into storage within a Swiss mountain facility where it has stayed until 2022 when this digitisation project started. For more information on the history and subject matter of the painting the reader is referred to documentation supplied by the "Foundation [2] for the Panorama of the Battle of Murten".  Earlier manually stitched image (Cuno Vollenweider) Restoration, 1996 - 2001
The following are some photographs of the panorama showing how it was previously hung in a cylinder. For practical reasons (size and weight) the full 100 m length was cut into three sections for transport and storage. Each section weighs approximately 1/2 tonne.
Photographs from the 1996-2001 restoration process.
Relocation from storage, June 2022
The three scrolls were transported from their mountain based storage facility to the eM+ laboratory. The first part of the project was documentation and a conservation exercise, the second stage was scanning for the digital twin.  Relocation from storage (2022 EPFL eM+) First light, July 2022
An initial scanning exercise was conducted in order to get a feel for the surface properties and ensure sufficient control points could be acquired for successful stitching. This was particularly important to understand for the sky portion of the painting which may have been quite featureless.
For this exercise a Canon 5D MK III camera, the Canon 100mm macro lens and simple ring light was used. The jury rigged platform was manually moved along an exposed portion of scroll 3 in four columns. Due to the sheer scale of the painting the only viable option was parallel multi-viewpoint scanning [3]. The biggest risk to this approach are parallax issues related to any 3D structures, fortunately in this case the 3D nature was at most 2mm for paint brush structure and didn't raise any problems.
The resulting 4 columns of the panorama (sky cropped) is shown below. This small test scan was designed so as to result in the same photograph density expected for the final scan. Since the camera was lower resolution the final result stitched to 15 pixels per mm, less than half the target resolution. This confirmed that the characteristics of the painting supported control (feature) point based stitching, however due to the higher glossiness of the sky a cross polarisation [13] arrangement was deemed necessary.
An important consideration was how flat the painting would lie on a flat surface. The painting on canvas was originally tensioned on a 30m diameter and 10m high cylinder. This results in the canvas stretching in such a way that it was no longer a cylinder but a hyperboloid, the lowest energy configuration. That is, the radius towards the top and bottom edges is larger than the radius at the mid-height. This is illustrated (exaggerated) below.
The cross section of the building from 2002 exhibition shows the hyperbolic nature.
The upshot is that while a cylindrical surface can be laid flat, a hyperboloid cannot be laid flat without folding/crinkling. This is a consequence of it being a parametric surface of 3 variables, rather than only 2 variables for a cylinder. It was determined that 1m could be laid reasonably flat, in particular, the surface needed to deviate in height within the focus depth as focus stacking at this scale was determined to be impractical. This influenced the design of the scanning table and other aspects of the approach taken. For example, if the scrolls could be laid flat one might consider unrolling them in a large warehouse and designing a totally automatic 2D scanning rig. The crinkling/folding of the upper and lower edges is obvious from a previous exercise before the 2002 exposition where the canvas was laid flat.
Second test scan, November 2022
The second test scan used the proposed camera and lens, namely the Phase One [4] iXH and Schneider Kreuznach 120 mm lens. The purpose of the exercise was to gain experience with the PhaseOne iXH, the Capture One (22) CH (Cultural Heritage) software and to decide on the lighting arrangement. The lighting scenarios tested included a ring light and a direct area light both perpendicular to the painting surface, and a raking (45 degrees) area light. Both situations were evaluated with and without cross polarisation.
As it happens, in order to achieve 1000 DPI the 120 mm lens was on the extreme edge of it's focus range. The switch was made to the Phase One 72 mm MKII iX-RS lens [4]. The resulting 4 columns of the panorama was stitched together to the target 40 pixels per mm resolution.
The final decision on lighting was to use a raking light from the "top" (the sky). While the direct lighting along the camera axis might be considered more correct, it was visually much less appealing. The image looked "flat" and the 3D structures of paint thickness and cracking were largely hidden. As such, this was a purely aesthetic decision. Because adjacent photographs have slightly different shading of the 3D structure (paint strokes) it could have degraded control point detection. If it did then the effect was minimal as there were typically hundreds of control points detected in the overlap zone between photographs. It was additionally decided to use cross polarised lighting to reduce specular reflections from the oil paint. The principle being that specular surfaces preserve the polarisation of impinging light, whereas the reflected polarised light from diffuse surfaces will be depolarised. With a polarising filter on the light source and a polarising filter on the lens at 90 degrees, the specular highlights are removed. A couple of sample comparisons with raking lighting with and without cross polarisation are shown below, the choice was obvious.
One disadvantage of using cross polarisation is that for diffuse reflections one only collects about 30% of the impinging light energy. Fortunately for this exercise a relatively bright light is used and modest exposure times can be tolerated. Another disadvantage of cross polarisation occurs if there are metal objects or metallic paint in the painting since they will have largely specular reflections. There were small gold and silver coated pieces, these end up looking rather different than their intended appearance. In the following mobile phone photograph on the left, the golden metallic handle of the sword shows good hilight detail while in the digitized painting (right) it looks more like a bit of mud.
In the following, the silver addition on the end of the sword blends in with the surrounding paint, but gets turned into a dark piece in the cross polarised photograph.
The Capture One (22) Cultural Heritage user interface is shown below.
Each photograph is 14204 x 10652 pixels, the iiq files typically occupy 200MB on disk. Sample photographs at full resolution are provided below.
Final scan, August-September 2023
Between the test scan above and the final scanning, the conservators worked on the painting. The equipment list of the main components for the final scanning process are as follows: Phase One iXH (Medium format, 150 MPixel), Phase One 72 mm MKII iX-RS lens, Ajurat D8 area LED light with known colour temperature and CRI of 97+, linear polariser on the light source and lens, motion/vibration detector, 10 GB network, custom design scanning rig, Mac Mini computer as controller, Windows workstation for stitching. The camera settings were: ISO 50, 1/3 second exposure time and aperture of f10. The area light array was set to maximum brightness. The scanning rig provides a 2D positioning framework approximately 12 m long and 1.5 m wide. Computer control, over ethernet, positions the camera platform to within 0.01 mm. A regular grid was decided on with 59 rows and the number of columns determined by the length of the particular scroll, noting that the length of the scrolls are not identical with scroll 1 being the longest and scroll 3 the shortest. Each photograph was approximately 360 mm by 270 mm, overlapping by 30% vertically and 50% horizontally. The larger horizontal overlap is chosen in order to manage the less precise positioning in the scroll advancing process. The scrolls are scanned 4 columns at a time called strips, the strips are approximately 1 m wide. Each strip is photographed entirely automatically using the two axis camera positioning rig. At the end of each strip the scrolls are manually advanced allowing for suitable overlap with the column of the last strip.
At the start of each four column strip, three additional photographs were taken. They include a Spyder colour chart for calibration, a white sheet for flat field correction arising from the raking light source and a metal dome (door handle) for polarisation angle check. The light variation due to the raking light can be observed in the figure below. The documented white value on the colour chart is also used for exposure setting to avoid white clipping.
The scanning rig is shown below, the truss spans the two scrolls avoiding the need for a 12 m long truss.  CAD drawings of the 2D capture rig (Nelissen Decor en Constructe)
The drop of the scroll on the sides of the scanning table helps to pull the scroll flat. The canvas is rolled in 1 m steps from the roll on the right onto the roll on the left.
Timelapse movies, 15x real-time. 1 strip consisting of 4 columns and 59 rows (236 photographs) takes 30 minutes. The camera is moved in a zig-zag motion in order to minimise the distance travelled.
Automation
Automation consists of Capture One (22) CH (Mac) software, custom software to control 2D scanning robot (Windows), and custom software to manage the overall process (Mac based). Process consists of: Move camera to next position, wait until vibration sensor indicates no movement, capture a photograph, transfer to the Mac over network, name the photograph with grid (x,y) position indices, save as raw .iiq file as well as a 50% size jpeg file. The Capture One automation was achieved using the AppleScript interface which exposes almost every aspect of the software. Both the camera, vibration sensor and 2D robotic platform are controlled over a 10 GB network interface.
At the end of every strip (4 columns), the 50% jpeg images for the current strip and the last strip are stitched together (PTGui) to ensure no photographs are missing and they stitch correctly. When the photographs and stitching is verified the scrolls are manually advanced, this checking was key since it would be very difficult to reverse the scroll positioning accurately enough to rephotograph at a missing position. The following is a typical test stitch of 2 strips, consisting of 8 columns and 472 photographs.
Each photograph has a global file naming scheme consisting of the row and column index. In the case of the strip stitching above revealing a missing photograph, the 2D scanning rig could be instructed to re-photograph the missing position. During the entire process a missing photograph only occurred 6 times out of the total of 27,000 photographs. It was a cedit to the engineering of the 2D scanning rig that it drifted by at most 1 mm during the photography of each scroll. The pseudo code for the digitisation of one strip could be written as follows.
Move rig to the colour chart position
Wait for the vibration sensor to report no motion
Trigger the camera
Save the photograph called "chart"
Move rig to the light field sheet
Wait for the vibration sensor to report no motion
Trigger the camera
Save the photograph call "background"
Move rig to the metal orb
Wait for the vibration sensor to report no motion
Trigger the camera
Save the photograph call "metal"
Move rig to the first position
Wait for the vibration sensor to report no motion
repeat 4 * 59 times
Form the photograph name based upon the current position index
Trigger the camera
Save the photograph with name
Update the camera position moving in a zip-zag order
Move rig to the new position
Wait for the vibration sensor to report no motion
end repeat
In order to minimise the turn-around time the rig is moved to the next position while the image is being copied from the camera and saved on the computer storage, this later process takes about 4 seconds. On average the time for the motion sensor to indicate no movement was between 3 and 4 seconds. The time to capture one strip and verify is approximately 1 hour, 30 minutes for the scanning and 30 minutes for the image and stitching check. Between 5 and 7 strips were digitised per day, 20 days to digitise the whole painting. Stitching
Stitching is performed in PTGui Pro [5] (Graphical User Interface for Panorama Tools). Since the photographs lie on a regular 2D grid, this is supplied to PTGui and control point detector only searches between adjacent neighbours.  PTGui grid settings
Two custom utilities were developed to assist in the process, these utilities act upon the .pts files and are used to detect missing control points between neighbouring photographs and to filter neighbouring control points for an even distribution over the grid. Between 1 and 1.5 million control points were typically recorded for each scroll, noting that the maximum number of control points between any two adjacent images was set to 40.
While the control point assistant in PTGui is useful, a connectivity visualisation tool proved key in both understanding the control point matrix but also knowing which parts to focus on. The following is an example of the grid of photographs. Some common features include: the region in the bottom right corner where the painting canvas was damaged and replaced with a featureless material, regions where the 50% overlap of columns resulted in control point connections between more than the immediate neighbours, gaps arising from the variation of grid spacing between the strips of 4 when the scroll was rolled.
Each of these defects is addressed by either disabling photographs in the case of featureless repair materials, the filtering utility removed the distant neighbour connections, and the gaps filled by either running the control point detection at a different grid overlap percentage or manually adding connections. Noting that the utility that generates this connectivity visualisation also lists exactly what image pairs need to be addressed. In general the alignment phase was only applied when all three defects were addressed and there was a clean connectivity of every grid photograph with each of it's 8 neighbours. For example, the following is the final connectivity matrix for a subset for scroll 1. All photographs are connected to their 4 orthogonal neighbours, and in the vast majority of cases also to their diagonal neighbours, in a few cases in the more plain parts of the sky there are no control points between diagonal neighbours.
PTGui is not really designed for this multi-viewpoint type of stitching, it is more intended for zero parallax nodal photograph stitching. For this type of stitching the software is informed that the lens being used is, say, 10,000 mm. Despite stitching terabyte images, each consisting of around 10 thousand 150 MPixel photographs, the project occupying typically 200GB of RAM, it never crashed once. The hyperboloid, again
The radii of the physical scroll is smallest at the mid-height and increases towards the upper and lower edge. This applies to the stitched image as well. To understand why, if PTGui were used to stitch a few rows across from the lower or upper edge of the painting then the resulting image would be wider than if it stitched a few rows from the centre of the image. Two consequences of this were experienced, they are both a natural consequence of PTGui trying to minimise the control point error. The first is if the lower half of the panorama were stitched, leaving out the sky, the overall shape of the panorama appears as a smiley, that is, curved upwards towards the left and right edge.
The second is revealed when the whole height is stitched the resulting image is bowed in at the left and right edge.
The two cases are illustrated below. The red line is shorter on the scroll than the blue lines and this is resolved in the stitch by warping as shown. 
By measuring the difference in pixels between the centre and the top (or bottom) edge one can estimate the difference in radii of the physical scroll since the pixels per mm is known. For scroll 3 the difference is about 550 mm which is reasonably close to the documented value of 600 mm, the average between the actual radii difference between the top and bottom row as measured in the restoration process of 2002. 
In short, the hyperboloid shape affects the digital twin as much as it does the physical scrolls. For the real scroll the consequence is that it cannot be laid flat. For the digital twin it means it cannot be mapped to a regular rectangle. The digital version can be warped to be a rectangle by applying a global distortion, currently there are only local distortions distributed across the whole painting. There are analogies here to map projections (equirectangular, Mercator, etc) which are also involved in applying various distortions in order to map a 3D surface onto a 2D plane. Final scroll statistics
The final renders are stitched with a vertical pixel count of 440,000. This resulted in horizontal pixel counts that varied from 1 million pixels to 1.4 million pixels depending on the scroll. Typical image file sizes for 8 bit colour ranged from between 1.3 TB and 1.5 TB. This is simply doubled for the 16 bit versions given that the files are saved as uncompressed RGB values. Note that for each scroll the first and last row did not contribute to the final image, so 59 rows were captured but only 57 were required to span the painted area of the scroll.
File formats
In general the standard image formats and/or software have image size limitations that don't come even close to the requirements for this project. For some formats this might be 32K, for others 64K, and software such as Photoshop is limited to only 300,000 pixels on any side. Obviously these limitations are problematic for this project. Even if one had the compute resources required, the grey region below shows the limit of the maximum image size PhotoShop can open for editing.  Largest portion of the image PhotoShop can manage
The main standards based image formats employed are BigTiff (including the pyramidal layout) [7] which is supported as an export format by PTGui, and iiif [8]. Both of these layouts can additionally contain image tiles that span different scales and are therefore suited to viewing and online distribution since they allow just the visible portion of the image at the appropriate resolution to be presented. The VIPS [9] ".v" format has been the basis for the whole imaging pipeline. The ".v" format used is simply the uncompressed RGB raw variant, this consists of a simple header (width, height, depth etc) followed by the binary image data. VIPS is used to convert the bigTIFF files from PtGui into ".v" files, all processing is using locally developed software reading and writing ".v" files, and finally VIPS is used (dzsave, tiffsave) to create the pyramidal files for presentation. Image processing toolsA number of software tools have been developed for the project. All of these require at most 1 row of the image to be housed in RAM, as such they use a negligible amount of RAM. This approach was only viable since SSD's were used as storage for all image processing.
The following shows the complete digital image as a cylindrical panorama, click on the image for the interactive view. This is a 1/10th resolution version, so one pixel in the following is represented by 100 pixels in the original scans. This image is 380534 x 42500 pixels, the full image is 3805340 x 425000 pixels, 1.62 terapixels.  Interactive panorama at 1/10 resolution mapped onto a cylinder
Digital presentation in a physical 360 degree cylindrical display at the eM+ research laboratory.
Some favourites
With a painting of this size and scale there is lots of exploring to be done. Some example small regions presented below, each telling it's own story.
Zoom examples
The following are some "random" examples of zooming into regions of the painting to the full resolution. 
 Recreation
Approximately 7 working days were required per scroll with another day or two to bump out the old scroll and prepare the next. For some relief from the tedium the site on which the painting is based was visited. The Murten (Morat) castle [6] still exists and bears a good relationship with the representation in the painting (scroll 2). One could align the left castle tower with the distant mountain and determine exactly where the sketches were taken from. For example, the sketch was taken from a position much further to the right and slightly lower on the hill than the location of the photograph.
 Alignment between sketch location, castle tower and cliff
With regard to capturing a matching panorama from the current site, the vertical field of view of a 10:1 aspect cylindrical panorama is about 35 degrees. This is accomplished with a 360 panorama using a camera with a 50mm lens.  Panorama from the battle site Amusements
The final processing by PTGui took approximately one week per scroll. Three to four days to determine control points, half a day of manual effort adding control points to missing neighbours, one day for alignment and two days to export the final image. This was performed on a relatively high end machine with dual AMD Ryzen Threadripper Pro with 5955WX processors (32 cores), NVIDIA Quadro P6000 graphics, 256 GB RAM, SSD drives for project photographs and 8TB SSD drive for PTGui scratch space. To relieve some of the boredom while waiting for various processes to complete, Midjourneys view of the battle was explored.
Midjourney can be prompted to create 360 equirectangular panoramas, a navigable example below. 
Digitizing and Augmenting the Panorama of the Battle of Murten (DIAGRAM) is a
project led by the Laboratory of Experimental Museology (EPFL, eM+) [1] in partnership with the
"Foundation [2] for the Panorama of the Battle of Murten". The project team consists of:
Prof. Sarah Kenderdine (lead),
Dr. Daniel Jaquet (project manager),
Paul Bourke (image specialist),
Raphael Chau (PhD student),
eM+ software engineers: Samy Mannane, Loic Sutter and Adriano Viegas Milani and numerous other collaborators.
The first phase of the DIAGRAM project between 2022–2023 has been supported by Loterie Romande, Municipality of Murten, Canton of Fribourg, Federal Office for Culture, the Association of the Friends of the Panorama and the Foundation Etrillard. Phase OneTM is a sponsor for this project. The second phase between 2024-2026 is supported by the Stiftung für Kunst, Kultur und Geschichte, UBS Culture Foundation, Swiss National Science Foundation, Ernst Göhner Stiftung, Foundation Etrillard, and Association Suisse pour l’Histoire et les Sciences Militaires. Other examples of large scans
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
