IntroductionThis post describes a way to display an OpenCV Mat object within a JavaFX Image.
Java part - Input part
- A Mat object is created in Java and populated with an image file using Highgui.imread().
- A ByteBuffer object is created using ByteBuffer.allocateDirect(). This buffer will contain the image after it has been operated upon by native code. allocateDirect() allows native code to directly access the buffer without needing expensive buffer copies.
- Native code is passed the address of the Mat object along with the ByteBuffer created previously.
Example code is shown below (along with relevant comments) -
C++ Native part - to process Mat object
- The code which processes the Mat image is inside a native C++ method, which is invoked by Java code using JNI (Java Native Interface). This method is passed the address of the Mat object, the ByteBuffer to which it should return the processed image, and the number of channels in the output.
- Default pixel format in a Mat object is BGR while JavaFx Image display format is BGRA. Image can also accept RGB format. So after the native code has finished processing the Mat object, it should convert it either to BGRA (preferred, as that's one less conversion on the Java side) or RGB, using cv::cvtColor(). In case BGR is converted to BGRA, a new destination Mat is required but in case of BGR to RGB conversion, in-place conversion can be done on the source Mat.
- Finally, the data member (which points to image array) of the processed/converted Mat image is copied to the previously passed-in ByteBuffer.
Example code is shown below -
Java part - Image display
- The PixelFormat (BGRA or RGB) of the native-code processed image in the ByteBuffer is set appropriately then the ByteBuffer is loaded into a WritableImage.
- The WritableImage is loaded into an ImageView node and displayed on a JavaFX Stage (window) as usual.
Example code is shown below -
- In Java, load an image file into a Mat object.
- Pass the address of the Mat object and a ByteBuffer to native code.
- Native code processes the image and puts processed image in previously passed ByteBuffer.
- Java takes the processed image from the ByteBuffer and displays it.
The objective of this post has been to show a way to efficiently display an OpenCV Mat object within a JavaFX Image. I haven't yet found a way to pass the Mat.data member from native code to Java without copying, hence the need to pass the direct ByteBuffer. If you have any ideas for more efficient code, please add a comment.
JavaFX Javadocs for Image, WritableImage, PixelReader, PixelWriter, PixelFormat.
OpenCV Javadocs for Mat, Highgui.
Java Javadocs for ByteBuffer.
- The Java bit was coded in NetBeans IDE 8.0.
- Add opencv-248.jar as a dependency.
- Java platform is 1.8.0 64-bit.
- C++ bit was coded in Visual Studio Express 2013.
- Project mode is Release x64.
- Add Include paths for OpenCV includes and Java JNI includes.
- Add Library paths for OpenCV x64 VC12 libs.
- Add Linker additional dependencies - opencv_core248.lib, opencv_highgui248.lib and opencv_imgproc248.lib.
- OpenCV version 2.4.8.
- OS is Windows 7 Home Premium.
Add paths to OpenCV DLLs and OpenCV Java DLL to system PATH environment variable. For my machine and setup they are like -