Note: This is an archived post from my previous blog. It can also be found on WaybackMachine. -John
For my last three weeks at the Faculty of Geo-Information Science and Earth Observation (ITC) at the University of Twente in May 2012, I worked on an individual final assignment to end my program, Post-Graduate Diploma of Geoinformatics. My supervisors were Dr. Kourosh Khoshelham and Sander Oude Elberink.
Following is a presentation of the resulting application. I have published the source code (Processing) along with the applications (Linux, Mac and Windows) on my Github. To simply try the app, download the zip-file for your operating system and the test data.
Linux
Mac OS
Windows
Test data
CloudMerger on GithubBackground
In indoor mapping, the Kinect sensor provides an affordable and highly portable possibility for capturing 3D data point clouds for analysis. For this to work, several point clouds of each room are needed to completely cover the area of interest and capture objects from all angles.
The coordinate system of the range data from the Kinect is relative to the sensor itself. This means that if you capture consecutive point clouds with it having moved the sensor in between captures, they will not display correctly relative to each other.
To amend this, the clouds have to be transformed (rotated and translated in space) into the same coordinate system. For clouds with overlapping areas, there exist several algorithms to do this. The assignment
The assignment was to develop a Java based application that loads two point clouds, allows the user to select corresponding points, perform a transformation algorithm from one cloud to the other and display the result visually.
From the assignment, defining functional requirements were easy to identify:
Load point clouds.
Display point clouds.
Register corresponding points using cursor.
Apply transformation.
Display and print result to file.The input data are two point cloud files that are plain text files with 6 space-delimited numbers for each point. These represent x-, y- and z-coordinates (distance to Kinect sensor in cm) and red, green and blue color value (0-255).
-164.81 309.59 115.65 170 145 165 -164.82 309.59 115.12 174 141 162 -166.31 312.35 115.62 181 155 160
The files from the Kinect consists of about 300000 points each, at about 10 MB each.
Two output files are created. One merged point cloud and one report file stating which points were registered, the resulting rotation matrix and translation parameters and finally, the calculated residuals. The Application
Following is a boring walkthrough of how the application works. Feel free to test it out first and then consult this text when you get stuck. Loading the point clouds
When the application is first opened, the user is met with a menu screen with four buttons. At first, only the ”load cloud” are active. Click the ”load cloud 1” to open a file chooser dialog and select the point cloud file to load it. It should be on the format specified in the input section. Do the same for ”load cloud 2”.
Loading point clouds in CloudMerger
The application may in some cases work on data of different format, but this is the only format it has been tested for. Other formats may yield unexpected results or cause the application to crash. If you test it out, please let me know!
The ”view cloud” buttons are now active. Click one of them to view the point cloud. You can now use the mouse to rotate and zoom around the cloud, press the TAB key on your keyboard to switch between the cloud screens and the menu screen or the BACKSPACE key to go back to the menu screen. Registering corresponding points
To start registering points, click on ”view cloud 1”. Rotate and zoom the cloud until you see the point you want to register (see picture below). Hold the SHIFT key while clicking with your mouse over the point you want to register. If successful, you should see that the window title changes from ”Cloud 1, registered 0 points”, to ”Cloud 1, registered 1 points”. You can then press the TAB key to switch to the other cloud and repeat the process for the other cloud. Viewing cloud and registering points.
Viewing cloud and registering points.
It is also possible to register more points in the first cloud consecutively, before registering the points in the other. Just make sure to register them in the correct sequence. When you have registered at least three points in each cloud (and the same number of points in each), you are ready to perform the transformation. Press the BACKSPACE key to go to the menu screen.
If you make a mistake while registering points, you may press BACKSPACE to go to the menu screen and click the ”reset registration” button to start over. Perform transformation
When at least three points have been registered in each cloud, more buttons appear on the menu screen (picture below). First, press the ”merge clouds” button to perform transformation on cloud 1 to match cloud 2. When ready, more options appear on the menu screen.
When ready, more options appear on the menu screen. View and export merged clouds
When the transformation is performed, two more buttons appear. You are able to view the merged clouds in one screen and rotate and zoom it like with the input clouds (see picture below). The clouds are displayed in different colors, so that it is easy to see if the transformation was successful.
Viewing the merged clouds.
Press BACKSPACE to go back to the menu screen, and click the ”Print merged cloud to file” button to export the merged clouds as a .txt-file. A ”save as” dialog appears (see picture below) and you are able to set the name of the output file. The file is stored at the desired location, along with a report file. Printing merged cloud to file.
Printing merged cloud to file.