Cloud Imaging Using Ground-based Sky Cameras

Soumyabrata Dev, Florian M. Savoy, Yee Hui Lee, Stefan Winkler{soumyabr001, eyhlee}@ntu.edu.sg , {f.savoy, stefan.winkler}@adsc.com.sg

http://vintage.winklerbros.net/

Introduction• Fine scale cloud monitoring needed for a variety of applications: weather observation, solar energy generation.

• Whole Sky Imagers (WSIs) are ground-based cameras that captures images at regular intervals [1].

• We use WSIs to detect clouds for automatic cloud coverage computation, recognizing cloud types, small-scaletracking for cloud prediction and 3D cloud volumetric estimation.

WAHRSIS: Our custom-built sky camera• Commercial WSIs are expensive, have low image resolution and limitedflexibility.

• We build our own sky-camera model called WAHRSIS: Wide AngleHigh Resolution Sky Imaging System [2].

Figure 1: Several versions of WAHRSISs have been designed and deployed atvarious rooftops of our university campus.

Cloud Segmentation• Existing algorithms are basedon threshold-based methods.

• We propose a threshold-free,learning-based segmentation al-gorithm [3], and release SWIM-SEG: a large-scale cloud seg-mentation database.

Figure 2: Input image, Binary out-put, 3-level output, probabilistic out-put image [From L to R].

Cloud Classification• We propose a texton-based ap-proach integrating both colorand texture of clouds [4].

• We also release an annotatedsky/cloud image categorizationdatabase called SWIMCATcomprising 784 images.

Figure 3: Various sky/cloud pat-terns in SWIMCAT: clear sky, pat-terned clouds, thick dark cloud, thickwhite clouds and veil clouds [From Lto R].

Cloud Base Height• Fish-eye images are undistortedusing camera calibration.

• Stereo-calibration of imagers areperformed using sun’s positionand trajectory.

• We perform cloud feature pointdetection and matching to gen-erate 3D models of cloud base.

Figure 4: Undistortion of fish-eyelens in a stereo-camera setup.

Motion Prediction• We perform fine-scale, short-term cloud motion prediction.

• Using optical flow techniques ontwo successive captured frames,we predict its future location.

Frame at t+2 Actual Frame at t+2

Frame at t Predicted Frame at t+2

Figure 5: Prediction with lead timeof 2 minutes.

References[1] S. Dev, B. Wen, Y. H. Lee, and S. Winkler. Ground-based image analysis: A tutorial on machine-learning techniques and applications. IEEE Geoscience and

Remote Sensing Magazine, 4(2):79–93, June 2016.

[2] S. Dev, F. M. Savoy, Y. H. Lee, and S. Winkler. WAHRSIS: A low-cost, high-resolution whole sky imager with near-infrared capabilities. In Proc. IS&T/SPIEInfrared Imaging Systems, 2014.

[3] S. Dev, Y. H. Lee, and S. Winkler. Systematic study of color spaces and components for the segmentation of sky/cloud images. In Proc. IEEE InternationalConference on Image Processing, pages 5102–5106, 2014.

[4] S. Dev, Y. H. Lee, and S. Winkler. Categorization of cloud image patches using an improved texton-based approach. In Proc. IEEE International Conferenceon Image Processing, pages 422–426, 2015.