Skip to main content

Roughness measurement of leaf surface based on shape from focus

Abstract

Background

Surface roughness has a significant effect on leaf wettability. Consequently, it influences the efficiency and effectiveness of pesticide application. Therefore, roughness measurement of leaf surface offers support to the relevant research efforts. To characterize surface roughness, the prevailing methods have drawn support from large equipment that often come with high costs and poor portability, which is not suitable for field measurement. Additionally, such equipment may even suffer from inherent drawbacks like the absence of relationship between pixel intensity and corresponding height for scanning electron microscope (SEM).

Results

An imaging system with variable object distance was created to capture images of plant leaves, and a method based on shape from focus (SFF) was proposed. The given space-variantly blurred images were processed with the proposed algorithm to obtain the surface roughness of plant leaves. The algorithm improves the current SFF method through image alignment, focus distortion correction, and the introduction of NaN values that allows it to be applied for precise 3d-reconstruction and small-scale surface roughness measurement.

Conclusion

Compared with methods that rely on optical three-dimensional interference microscope, the method proposed in this paper preserves the overall topography of leaf surface, and achieves superior cost performance at the same time. It is clear from experiments on standard gauge blocks that the RMSE of step was approximately 4.44 µm. Furthermore, according to the Friedman/Nemenyi test, the focus measure operator SML was expected to demonstrate the best performance.

Background

In light of concerns for the environment, resource and costs, the improvement of pesticide efficiency is a classical issue in agricultural engineering. Despite the variance in the definition of leaf wettability [1], in this paper, the term refers to the manifestation of submicron physicochemical interactions between leaf surface and solution droplet [2], i.e. the affinity of leaves to water or medicine. There exists a large number of studies on leaf wettability that aim to enhance the adhesion of droplets on the surface of leaves, thereby averting the off-target deposits (e.g. rebound, roll, slide) that result from the adhesive characteristics of leaf surface. Leaf wettability may differ significantly among different species, varieties, and may even vary during the different stages of a life cycle [3]. Plant leaves, when observed at a high resolution, are rarely absolutely flat. Previous studies found it is the chemical composite and microstructure of epicuticular wax formations on leaves that determine leaf wettability. Increases in the surface roughness of a hydrophobic surface normally lead to increases in the hydrophobic properties [4]. Contact angle is the angle where a liquid–vapor interface meets a solid surface (plant leaf in this case). Though contact angle is a common metric of leaf wettability, its contribution to intensive studies on wettability is very limited. When quantifying the separate factors of leaf wettability, instead of contact angle, surface roughness is used as a viable index, as wettability is a function of roughness.

In the manufacturing industry, surface roughness characterization is standardized, whereas, unfortunately, a mature method for surface roughness characterization of leaves is lacking [5]. Pioneers of the field attempted to account for contact angle as a parameter in roughness coefficient [6, 7] by contrasting the contact angle of rough surfaces with that of smooth surfaces, where the roughness coefficient is merely a relative and indirect assessment of roughness. Scanning electron microscope (SEM) provides high-resolution micrograph of leaf surface, and with the help of computer vision techniques (e.g. fractal dimensional analysis [1, 8, 9], Fourier descriptors [10, 11], visual classification [12, 13]), it is one of the viable options for obtaining leaf surface roughness. However, before inspection using SEM, the preprocessing of samples is cumbersome. Besides, the computer vision techniques based on two-dimensional SEM micrograph require significant correlation between surface texture and roughness. Atomic force microscope (AFM) can be used for highly accurate probing of the surface profile of leaves through a stylus [4, 14]. The adoption of AFM, like most of stylus-enabled methods of characterizing surface roughness in the manufacturing industry, may damage the surface, despite the creation of ameliorated non-contact AFM that aims to alleviate the problem at the cost of accuracy. Besides, AFM can only gain access to one-dimensional profile within an action cycle, and its z-range for leaves of many plant species is unacceptably narrow. Optical profiler, a profiler based on optical principles such as phase shifting interferometry, was used to render the surface of plant leaves [5, 15]. Using interferometry, speckles formed by white light interferer compose optical sections, and then the leaf surface can be reconstructed from these sections. This procedure is often automated and faster, but it may omit the most valuable information concerning the spatial organization of a surface [11].

The prevailing methods for characterizing leaf surface roughness, dependent on expensive instruments, have to deal with a number of shortcomings. To address the issue, this study proposes a new approach to measure the leaf surface roughness based on shape from focus (SFF). As a textured rough surface moves with respect to a fixed imaging system, variation of image sharpness is observed, enabling the recovery of the shape from the textured image. In this study, in order to reconstruct the surface topography where surface roughness is recovered, micrograph sequences of leaves captured by high-resolution digital camera with diverse focusing deviation are processed by the SFF algorithm. Cost of the method proposed in this paper is notably lower compared to the previous methods. Moreover, it utilizes 3-demensional information and features noncontact measurement process. To evaluate performance of the method, results of the optical profiler are specified hereunder as reference.

Materials and methods

Materials

Customized steel gauge blocks that are precisely 1000 µm to 1100 µm ± 0.2 µm thick at intervals of 10 µm function as the reference standard. The pairwise combination of gauge blocks yielded 10 measurement results at different step height: from 10 to 100 µm by 10 µm. Leaves of lhe hybrid indica rice (Oryza sativa L.), grown from seed in an incubator, were sampled at seedling stage. The Rape (Brassica chinensis L.) was Zheda 618, which was bred by Zhejiang University. The cotton (Gossypium hirsutum L.) was upland TM-1. The tobacco (Nicotiana tabacum L.) was benthamiana.

A SFF imaging system (Fig. 1) was built in Agricultural Information Technology Institute of Zhejiang University (Hangzhou, PRC) to capture focused and defocused images. The system comprised an optical experimental platform (Fenghua Technology Co., Ltd, Shenzhen, PRC), an industrial RGB camera (Tuoriweiye Technology Co., Ltd, Shenzhen, PRC), a portable microscope (Meijing Electronics Co., Ltd, Shanghai, PRC), and a micromotion objective table (Hengyang Electronic Technology Co., Ltd, Guangzhou, PRC). The resolution of the RGB camera is 3072 pixels \(\times\) 2048 pixels. The portable microscope, functioning as a camera lens, is capable of magnification of \(\times 10\). The micromotion objective table, installed on the optical experimental platform, shifts leaf vertically in the motion range of 12.5 mm and accuracy of 0.5 µm. Manual adjustment to ratchet knob lifts the table in precision, thus creating defocus. The camera was mounted on the portable microscope, upside down towards the table, and the images were transmitted to computer via Ethernet.

Fig. 1
figure1

The shape from focus imaging system

Wyko NT9100 (Veeco, NY, US) optical profiler was utilized to measure leaf roughness as a comparison. This instrument, suitable for measuring 3d surface topography, has a vertical scanning range of 0.1nm–10 mm. The measurement was made in College of Optical Science and Engineering, Zhejiang University (Hangzhou, PRC), where the optical profiler was located. Upon measurement, the defocused images were immediately captured, and the leaves were moved to the optical profiler for observation. Thanks to the supporting software, the three-dimensional structure was reconstructed and the surface roughness was obtained with ease.

Shape from focus

The notion of shape from focus, also known as depth from focus (DFF), is hardly new. It involves a classic problem in computer vision. SFF is widely studied as a passive method of estimating depth or shape from monocular focal cues [16,17,18,19].

With a fixed setting of focus length, observations along the z-axis are different in blurriness, which is one of the most obvious cues for human observers to understand depth in a two-dimensional image. As to the microscopic level, when projecting to the sensor plane (Fig. 2), the radiation of each point on the object spread onto a fuzzy circle, which is usually expressed by point spread functions (PSFs). In particular, if the focus locates on the sensor plane, the diameter of the fuzzy circle is infinitely close to zero. In this case, since the sensor plane and the focal plane are coincident, the projection of point M is an ideal point. For a constant focal length f and a fixed location of lens and sensor plane, when the objects move away from the lens, the focus of the point \(M^{\prime}\) moves towards the lens, and a fuzzy circle with the diameter of c is synchronously detected on the sensor plane.

Fig. 2
figure2

Convex lens model with objects motion along with z-axis

Methods known as depth from defocus (DFD) or shape from defocus (SFD) aim to compute the distance between the defocused object points and the sensor plane by estimating the diameter of fuzzy circles. These methods are speedy but inaccurate. Furthermore, when adopting them, intrinsic and extrinsic parameters must also be determined in advance. According to SFF theories, a focused image surface (FIS) is defined as the surface formed by a set of points at which the object points are focused by the lens [20]. FIS is determined on the basis of the sharpness at each pixel in a sequence of images with constantly varying focus levels, and a focus measure function or operator is applied to compute the sharpness. Conventionally, FIS comprises pixels where the corresponding frame gives the maximal sharpness among the images, but it is also true that certain applications feature optimized techniques (e.g. quadratic interpolation, Gaussian interpolation) for smooth surfaces. For example, quadratic interpolation method determines a quadratic function by 3 maximal focus measures, and the final maximal sharpness should be ideally located at the axis of symmetry. When FIS is available, all corresponding points on the object surface can be retrieved.

Image registration

Image registration focuses on aligning two or more images of the same scene through geometrical warping and overlapping. Research schemes of most SFF studies ignore the differences in view-of-field during the adjustment of focus level, as image registration comes with considerable costs. However, this study proposes the correction of the pixel offset, while taking into account the trade-off between the accuracy requirement and the computational complexity in roughness measurement of leaf surface. An illustration of the image registration process is given in the following paragraphs.

First, the relevant features are detected using Speed-Up Robust Feature (SURF) [21]. Owing to its rapidity and robustness, SURF has extensive applications in image registration. The strategies in relation to Gaussian pyramid and Harr wavelet make the SURF descriptor scale- and rotation-invariant. In vast image registration tasks, SURF outperformed other prevailing methods. Thus, this paper proposes the use of SURF as the feature detector in this scheme. For each image, SURF algorithm yields dozens of features expressed by 128-deminsional vectors.

Second, image features are matched based on the correspondence between the features within pairs of images. The so-called correspondence is a function, which characterizes the spatial relationship, of two features. As a frequently used distance, Euclidean distance was adopted in this scheme. By calculating the distance of every pair of features between neighbor frames, the brute-force matches can be obtained. The amount of matches indicates the clarity of the images to some extent. Points on images with few matches are largely defocused, which is not appropriate for SFF. Therefore, for the sake of arithmetical simplification in subsequent procedures, the images with insufficient matches were removed from the dataset, and the threshold \(\eta\) was set according to the texture richness of leaf surface.

Then, homography matrices are estimated from the matches. Theoretically, the distortion of images resulting from the adjustment of focal level is nothing but scaling, yet in practice, it is clear that miscellaneous errors may incur other forms of distortion. The homography matrix denotes the mapping relation between two image coordinates, which is a 3 \(\times\) 3 matrix, and is independent of scalar multiplication. Thus, 8 parameters should be estimated for each homography matrix, i.e. at least 4 matches are needed before an image can be transformed. In addition, Random Sample Consensus (RANSAC) works as a filter of matches, considering the high sensitivity of homography to noise.

At last, the homography matrices are applied to image transformation. In this stage, all of the remaining images (meeting the limit of the aforementioned threshold \(\eta\)) are transformed into the same image coordinate (e.g. that of the first image). Linear interpolation is used to deal with non-integer pixel indices that occurred due to the spatial discreteness of digital images. After the transformation, the pixel offsets among the image frames are eliminated so that an overlapping region emerges. For each image frame, the edge outside the overlapping region should be trimmed off, thus ensuring that there is no absent pixel.

Focus measure

Focus measure plays a dominant role in SFF since it produces information that are fundamental for FIS. A great number of focus measure operators have been proposed in previous studies. Pertuz et al. [17] offered a summary of these focus measure operators by grouping them into 6 families. Though it is difficult to determine which operator family performs best under any given imaging conditions, the Laplacian family demonstrates the best performance overall. Some operators working in frequency domain, widely used in autofocus, also showed sound performance results; nevertheless, the time-consuming Fourier transformation offers little support to solving SFF problems since focus measure is determined in local windows.

In this paper, performances of four spatial focus measure operators were compared.

Energy of Laplacian (EL)

Energy of Laplacian considers the second derivative of images. It is calculated by summing the pixel intensity of an image convolved with a Laplacian mask. This step is carried out within a local window for denoising purpose, which is expressed by the following formula:

$$\text {EL}(u,v)= \sum _{(x,y)\in \Omega (u,v)}\mathcal {L}(x,y),$$
(1)
$$\mathcal {L}(x,y)= \frac{\partial ^{2}f(x,y)}{\partial x^{2}}+\frac{\partial ^{2}f(x,y)}{\partial y^{2}}.$$
(2)

wherein, \(\mathcal {L}(\cdot )\) denotes Laplacian transformation; \(f(\cdot )\) denotes the pixel intensity of image; and \(\Omega (\cdot )\) denotes the adjacent region (same below).

Sum-modified Laplacian (SML)

Laplacian operator calculates the second partial derivatives of images with respect to x and y, which can be either positive or negative. Therefore, Energy of Laplacian may show a small response at pixels where the two partial derivatives at orthogonal directions cancel out. Different from Energy of Laplacian, Sum-modified Laplacian sums the energy of a window for an image convolving with a modified Laplacian (ML) operator, which is defined as the sum of absolute value of second partial derivatives:

$$\text {SML}(u,v)= \sum _{(x,y)\in \Omega (u,v)}\mathcal {L}_{m}(x,y),$$
(3)
$$\mathcal {L}_{m}(x,y)= \left| \frac{\partial ^{2}f(x,y)}{\partial x^{2}}\right| +\left| \frac{\partial ^{2}f(x,y)}{\partial y^{2}}\right| .$$
(4)

wherein, \(\mathcal {L}_{m}(\cdot )\) denotes modified Laplacian transformation.

Tenenbaum gradient (TG)

Tenenbaum [22] proposed a focus measure operator based on Sobel operator, which is named Tenenbaum gradient. Sobel operator, known as an edge detector, has two forms which produce the first derivative of images with respect to x and y, respectively. By summing the length of gradient within a window, Tenenbaum gradient is obtained by the following formula:

$$\text {TG}(u,v)= \sum _{(x,y)\in \Omega (u,v)}G(x,y),$$
(5)
$$G(x,y)= \sqrt{G_{x}^{2}(x,y)+G_{y}^{2}(x,y)}.$$
(6)

Gray level variance (GLV)

Gray level variance is a statistic operator based on the assumption that points at FIS come with the maximal gray level variance. Naturally, GLV is also calculated in local windows:

$$\text {GLV}(u,v)=\sum _{(x,y)\in \Omega (u,v)}(f(x,y)-\mu )^2.$$
(7)

wherein, \(\mu\) denotes the mean pixel intensity of the window.

FIS searching

The principal and most straightforward way to determine FIS is to search for the maxima of focus measure at each pixel [23]. Though implementation of such an approach is both fast and simple, the resolution of FIS depends on the movement interval of object with respect to the image detector, i.e. the reconstructed surface looks discrete. A great many of SFF studies inclines to interpolate the points nearby the maximum. In such studies, effective interpolation techniques including quadratic [24] and Gaussian [25] interpolation have been extensively adopted. Interpolation methods aim at smoothing FIS, yet, like the former approach, a maxima caused by noise may lead to its distortion. Besides, though closed form solutions can be provided for the specific situations, interpolation methods are often time-consuming. Some researchers optimized the FIS using tensor voting [26, 27] but the algorithm iterates every token from depth cloud and computes the eigenvalues, which is not favorable to addressing the current problem.

This paper proposed a simple but efficient and robust method for FIS search using weighted average of stereo blurred focus measure.

The values of focus measure for each pixel were obtained by applying the aforementioned focus measure operators according to the specified procedures. Considering the potentially abnormal values (due to noise), the original values of focus measure were blurred with a stereo mean filter.

The blurred focus measure is defined as:

$$\text {BFM}_k(i,j)=\frac{1}{N}\sum _{(x,y,z)\in \Omega '(i,j,k)}\text {FM}_z(x,y).$$
(8)

wherein, N denotes the number of pixels in the window; k denotes the image frame of the focus measure; \(\text {FM}_z\) (x,y) denotes the focus measure at pixel (xy) for image frame z; and \(\Omega^{\prime}(\cdot )\) denotes the adjacent region of the pixel over frames. Here, \(\Omega^{\prime}(\cdot )\) is a \(3\times 3\times 3\) window.

In order to achieve fast and precise FIS search, information of the focus measure should be fully utilized without increasing the difficulty of computation. With respect to pixels in different image frames at the same pixel coordinate, values of focus measure were sorted from largest to smallest. Then, FIS is calculated by the following formula:

$$\text {FIS}(i,j)= {\left\{ \begin{array}{ll} \text {NaN}, & \sigma _\varphi > \lambda m\\ \frac{\delta \sum _{k\in \varphi _m}k\text {BFM}_k(i,j)}{\sum _{k\in \varphi _m}\text {BFM}_k(i,j)}, & \sigma _\varphi \le \lambda m \end{array}\right. },$$
(9)

wherein, \(\varphi _m\) denotes the first m frame numbers of sorted focus measure values; \(\delta\) denotes the interval distance between neibour frames; and \(\sigma _\varphi\) denotes the standard deviation of \(\varphi _m\). The coefficient \(\lambda\) is a manual setting parameter. The experimental value of \(\lambda\) is 0.5. \(\text {NaN}\) (Not a Number) represents an invalid value.

Formula (9) indicates that FIS is determined by the weighted average of \(\varphi _m\), where the blurred focus measure functions as the weight. The method proposed in this paper considers the score of image frame where FIS can possibly be found, and yields the comprehensively predicted location. A large \(\sigma _\varphi\) indicates the unreliability of focus measure. Therefore, fixing the invalid values for FIS on these pixels is an approach to prevent them from getting involved in the subsequent computation. Figure 3 shows the performance of the proposed FIS search method on cotton leaf surface. It is evident that this method is less sensitive to maximum compared with interpolation methods.

Fig. 3
figure3

FIS searching on the section of cotton leaf image frames. Focus measure of the transversal are shown using pseudo color map, which increases as tone turns from cool to warm. Flecked with black points marked as NaN, the curve formed by white points is the FIS to search

In most cases of application, the SFF procedures will end upon finding FIS, and the subsequent measurement of surface feature parameters is taken for granted. In practice, an abnormal bulge emerges from the central region of the reconstructed surface. Drift of FIS is potentially ubiquitous in SFF due to defects in relation to the fabrication of camera, the assembly accuracy of microscope and the sensitivity discrepancies of sensing units. For example, if the lens is not in ideal shape, the lights, which should have intersected on the sensor plane, will intersect forward or backward. It will therefore result in a SFF offset. Although this effect is often negligent, it becomes significant in microscopic applications. Essentially, the distortion can be regarded as the manifestation of barrel distortion in depth. Universality of barrel distortion in short focus lens accounts for the significance of FIS distortion in microscopic applications. To eliminate such distortion, the difference of the primary FIS and a drift field substitute the primary FIS, in which the drift field denotes the strongly blurred FIS of a flat surface. This is not a cumbersome process of duplication, as the measure of the drift field, i.e., system calibration, is completed once for all upon establishment of the system. Figure 4 demonstrates the effective correction of a flat surface with scratches of aluminum alloy. It is clear that an apparently abnormal uplift of FIS is restrained in the undistorted image.

Fig. 4
figure4

Pseudo-color map of FIS comparison before and after correction by a drift field. Pixel height increases as tone turns from cool to warm. a Distorted FIS. b Undistorted FIS

Results and discussion

Comparison of surface rendering

Plant leaves of various species were observed in both optical profiler and the SFF system. Figure 5 compares the surface renderings of these two methods. In this comparison, the height of surface gets larger as the color shifts from cool tone to warm tone, and the SML operator was chosen for focus measure in SFF renderings. It should be noted that it was very challenging to present the identical region of interest in the two methods due to differences in field of view. Thus, a qualitative evaluation had be conducted according to the comparison of overall performances.

Fig. 5
figure5

Computer rendering of rice leaves. a SFF rendering. b Optical profiler rendering

Plant leaves normally present fractal structures. As is shown in Fig. 5a, b, the surface of rice leaf comprises complex microstructures including parallel wavy veins, hairy trichomes, quasi-1D arranged micropapillae and patchy protuberances. An inherent drawback of SFF lies in its mediocre performance for details and abrupt changes due to local windows in focus measure. In the SFF rendering, comparatively speaking, theses details are suppressed; certain detailed information are preserved nonetheless. SFF rendering demonstrates a distinctly higher altitude at areas where the trichomes and protuberances were expected. Relatively speaking, the veins are well defined in SFF reconstruction since larger scale of scenes are less sensitive to the local windows.

It is evident that the field of view of optical profiler is much smaller than that of the SFF system. The optical profiler renders no more than one vein per observation, while the SFF system offers a rendering of several veins thanks to its advantage of a wider field of view. Since the wavy veins on rice leaves are considerably different in size, the accuracy of roughness measurement for optical profiler is largely dependent on the selection of single vein. By contrast, such dependence abates in the SFF system.

Figure 6 shows the computer renderings of rape, tobacco, cotton and rice leaf surface. Dicotyledons are characterized by reticular veins. Leaf surface trends to be flat where the veins are sparse, a fact that has been well demonstrated by the renderings.

Fig. 6
figure6

Computer renderings of leaf surface of different species (rape, tobacco, cotton and rice from top to bottom). The first column is Computer renderings with texture. The second column is Computer renderings with depth in pseudo color map

Table 1 shows the areal roughness of plant leaf surface displayed in Fig. 6. If A is the area where the roughness is measured, then the areal surface roughness based on arithmetical mean height is calculated by the following formula:

$$S_a=\frac{1}{A}\int _{(x,y)\in A}\left| \text {FIS}(x,y)\right| \,\text {d}x\,\text {d}y.$$
(10)
Table 1 Areal surface roughness of the plant leaves

Quantitative analysis on gauge blocks

The measurement of identical micro-surface profile on different instruments is quite a challenge. In order to quantify the performance of the SFF system for each FIS method, experiments were conducted via pairwise combination of the gauge blocks along their edges, creating a step at the shared border. Figure 7 shows the diagram of experimental measure where the microscope aimed at the seam of the blocks. The pairwise combination of gauge blocks yields 10 measurements at different step height: 10–100 µm at intervals of 10 µm. For each measurement, approximately 10,000 samples were selected from the FIS at the two step faces, respectively, and the height of the step was calculated. The number of samples depends on the alignment and NaN values since the sampling was conducted in columns. Root Mean Square Error (RMSE) and Pearson correlation were computed using ground truth and reconstructed depth map. The smaller the RMSE and the more significant the correlation, the better the performance. If f(mn) and h(mn) are the computed value and the actual value of the step height for the n-th sample of the m-th measurement, then the RMSE and the Pearson correlation r are expressed by the following formula:

$$\text {RMSE}= \sqrt{\frac{1}{MN}\sum _{m=1}^{M}\sum _{n=1}^{N}\left( f\left( m,n\right) -h\left( m,n\right) \right) ^2},$$
(11)
$$r= \frac{\sum _{m=1}^{M}\sum _{n=1}^{N}\left( f\left( m,n\right) -\bar{f}\right) \left( h\left( m,n\right) -\bar{h}\right) }{\sqrt{\left( \sum _{m=1}^{M}\sum _{n=1}^{N}\left( f\left( m,n\right) -\bar{f}\right) \right) \left( \sum _{m=1}^{M}\sum _{n=1}^{N}\left( h\left( m,n\right) -\bar{h}\right) \right) }}.$$
(12)

Figure 8 shows the comparison between the proposed SFF method and traditional SFF method concerning RMSE and Pearson correlation with different focus measure operators. It is evident that the proposed SFF method demonstrates superior performance as it features smaller RMSE and more significant correlation. It is believed that the amelioration on distortion contributes to the significant improvement on performance. Moreover, system robustness was improved by the introduction of NaN value and the design of weighted average in focus measure. Regarding the proposed SFF method, the SML operator outperformed the other operators: the RMSE of SML is lower than 4.44 µm. Correlations of the four operators are close to 1, indicating high correlation between the estimated depth and the ground truth, despite the relatively smaller correlation shown by the GLV operator.

Fig. 7
figure7

Diagrammatic sketch of FIS measure on gauge blocks

Fig. 8
figure8

RMSE and Pearson correlation of proposed SFF and traditional SFF using four different focus measure operators

Friedman test (a non-parametric hypothesis test) and post-hoc Nemenyi [28] test were adopted as a further evaluation of the focus measure operators. If \(r_i\) is the average rank of i-th algorithm, and k is the number of algorithms, and N is number of datasets, the statistic

$$\tau _F=\frac{(N-1)\tau _{\chi ^2}}{N\left( k-1\right) -\tau _{\chi ^2}},$$
(13)

is subject to F distribution, wherein

$$\tau _{\chi ^2}=\frac{12N}{k(k+1)}\left( \sum _{i=1}^{k}r_i^2-\frac{k\left( k+1\right) ^2}{4}\right) .$$
(14)

Here, the performance of the four operators for each dataset were ranked, and the average rank of each operator was calculated. Table 2 shows rankings of the four operators, and unsurprisingly, the SML operator ranks first in most of datasets. According to Formulas (13, 14), the statistic can be calculated: \(\tau _F=26.714\). Taking the significance level \(\alpha =0.1\), the critical value of F distribution is \(t_\alpha =2.490<\tau _F\). Here, null-hypothesis is rejected, and it can be said that performance of the four operators differs significantly.

Table 2 Rank table of four operators

The post-hoc Nemenyi test distinguishes the algorithms by pairs. Two algorithms are significantly different if the difference of corresponding average ranks reaches the critical difference

$$\begin{aligned} CD=q_{\alpha }\sqrt{\frac{k\left( k+1\right) }{6N}}. \end{aligned}$$
(15)

If the confidence is \(\alpha =0.1\), the critical value is \(q_\alpha =2.291\) according to Tukey distribution. Then the critical difference is \(CD=1.323\). Figure 9 shows the Friedman/Nemenyi test of the four focus measure operators. Normally, Laplacian family performs better than the other operators, a fact that agrees with results from previous studies [17]. The SML operator showed the best performance in all datasets among the operators, and its performance differ significantly from that of TG and GLV.

Fig. 9
figure9

Friedman/Nemenyi test of four focus measure operators in our proposed method

Discussion on size of local window in focus measure

The recurrent blur techniques in the algorithm specified in this paper were used as denoising modules, but the lost details may result in low resolution and inaccurate measurements. In subsection Focus measure, \(\Omega (\cdot )\) is defined as the adjacent region implemented by a local window. The size of local window is a trade-off between denoising and resolution. To find an appropriate window size, RMSE of height measurement with respect to side length of square local window (Fig. 10) was demonstrated. The window side length was tested from 5 to 95 pixels by 10 pixels, and from 11 to 101 pixels by 10 pixels. In addition, size of 3, 7, 9 pixels were included for distinct revealing.

Fig. 10
figure10

RMSE of height measurements with respect to different length of local window. The red area denotes the standard deviation

In Fig. 10, a reasonably decreasing trend can be observed concerning RMSE and the corresponding standard deviation as the side length of local window grows. While large window size causes information loss, small window size leads to increased noise, which inhibits the number of valid depth of pixels because of NaN. Therefore, in order to reserve as much information as possible for the sake of high accuracy, it is preferred to select a value where RMSE tends to be flat (e.g. 25 pixels in the algorithm proposed herein).

Though the FM operator SML and dataset \(D_5\) are adopted in this subsection as a typical pattern, similar results are also available for other conditions.

Conclusion

This paper summarized the prevailing methods for measuring leaf surface roughness through a statement of merits and demerits. A new method based on shape from focus was proposed, and the SFF system was established. Different from traditional SFF used in macrocosm, the proposed method requires image registration as preprocessing. According to the experiments, the introduction of distortion correction and focus measure optimization succeeded in increasing the accuracy and robustness. The performance of the 4 focus measure operators were quantified by RMSE and Pearson correlation on the step between the two precise gauge blocks. The proposed method with the SML operator reached the best performance on RMSE and Pearson correlation, which are 4.44 µm and 0.988. Freidman test and post-hoc Nemenyi test also indicated that the SML operator performed best among the 4 operators, exhibiting significant superiority, especially when compared with EL. Roughness measurement of leaf surface based on SFF was not been fully explored, but its potential in plant phenotyping was demonstrated. We believe the method has broad research prospect in the field of plant phenotyping. Future studies may aim to optimize the integration of the SFF system, and to improve the time-consuming algorithm for practical adoption.

Availability of data and materials

The datasets used or analysed during the current study are available from the corresponding author on reasonable request.

References

  1. 1.

    Nairn JJ, Forster WA, van Leeuwen RM. Quantification of physical (roughness) and chemical (dielectric constant) leaf surface properties relevant to wettability and adhesion. Pest Manag Sci. 2011;67(12):1562–70. https://doi.org/10.1002/ps.2213.

    CAS  Article  PubMed  Google Scholar 

  2. 2.

    Puente DWM, Baur P. Wettability of soybean (Glycine max L.) leaves by foliar sprays with respect to developmental changes. Pest Manag Sci. 2011;67(7):798–806. https://doi.org/10.1002/ps.2116.

    CAS  Article  PubMed  Google Scholar 

  3. 3.

    Holloway P. Surface factors affecting the wetting of leaves. Pestic Sci. 1970;1(4):156–63. https://doi.org/10.1002/ps.2780010411

    CAS  Article  Google Scholar 

  4. 4.

    Burton Z, Bhushan B. Surface characterization and adhesion and friction properties of hydrophobic leaf surfaces. Ultramicroscopy. 2006;106(8–9):709–19. https://doi.org/10.1016/j.ultramic.2005.10.007.

    CAS  Article  PubMed  Google Scholar 

  5. 5.

    Abbott JR, Zhu HP. 3d optical surface profiler for quantifying leaf surface roughness. Surf Topogr Metrol Prop. 2019;7(4):17. https://doi.org/10.1088/2051-672X/ab4cc6.

    Article  Google Scholar 

  6. 6.

    Wenzel RN. Resistance of solid surfaces to wetting by water. Ind Eng Chem. 1936;28(8):988–94. https://doi.org/10.1021/ie50320a024

    CAS  Article  Google Scholar 

  7. 7.

    Cassie A, Baxter S. Wettability of porous surfaces. Trans Faraday Soc. 1944;40:546–51. https://doi.org/10.1039/tf9444000546.

    CAS  Article  Google Scholar 

  8. 8.

    Sa JJD, Backes AR. A simplified gravitational model to analyze texture roughness. Pattern Recognit. 2012;45(2):732–41. https://doi.org/10.1016/j.patcog.2011.07.023.

    Article  Google Scholar 

  9. 9.

    Nairn JJ, Forster WA. Methods for evaluating leaf surface free energy and polarity having accounted for surface roughness. Pest Manag Sci. 2017;73(9):1854–65. https://doi.org/10.1002/ps.4551.

    CAS  Article  PubMed  Google Scholar 

  10. 10.

    Journaux L, Simon JC, Destain MF, Cointault F, Miteran J, Piron A. Plant leaf roughness analysis by texture classification with generalized Fourier descriptors in a dimensionality reduction context. Precis Agric. 2011;12(3):345–60. https://doi.org/10.1007/s11119-010-9208-z.

    Article  Google Scholar 

  11. 11.

    Bediaf H, Sabre R, Journaux L, Cointault F. Comparison of leaf surface roughness analysis methods by sensitivity to noise analysis. Biosyst Eng. 2015;136:77–86. https://doi.org/10.1016/j.biosystemseng.2015.04.012.

    Article  Google Scholar 

  12. 12.

    Saebo A, Popek R, Nawrot B, Hanslin HM, Gawronska H, Gawronski SW. Plant species differences in particulate matter accumulation on leaf surfaces. Sci Total Environ. 2012;427:347–54. https://doi.org/10.1016/j.scitotenv.2012.03.084.

    CAS  Article  PubMed  Google Scholar 

  13. 13.

    Wang HX, Shi H, Li YY, Wang YH. The effects of leaf roughness, surface free energy and work of adhesion on leaf water drop adhesion. PLoS ONE. 2014;9(9):10. https://doi.org/10.1371/journal.pone.0107062.

    CAS  Article  PubMed Central  Google Scholar 

  14. 14.

    Koch K, Neinhuis C, Ensikat HJ, Barthlott W. Self assembly of epicuticular waxes on living plant surfaces imaged by atomic force microscopy (AFM). J Exp Bot. 2004;55(397):711–8. https://doi.org/10.1093/jxb/erh077.

    CAS  Article  PubMed  Google Scholar 

  15. 15.

    Bhushan B, Jung YC. Micro- and nanoscale characterization of hydrophobic and hydrophilic leaf surfaces. Nanotechnology. 2006;17(11):2758–72. https://doi.org/10.1088/0957-4484/17/11/008.

    CAS  Article  Google Scholar 

  16. 16.

    Moeller M, Benning M, Schonlieb C, Cremers D. Variational depth from focus reconstruction. IEEE Trans Image Process. 2015;24(12):5369–78. https://doi.org/10.1109/tip.2015.2479469.

    Article  PubMed  Google Scholar 

  17. 17.

    Pertuz S, Puig D, Garcia MA. Analysis of focus measure operators for shape-from-focus. Pattern Recognit. 2013;46(5):1415–32. https://doi.org/10.1016/j.patcog.2012.11.011.

    Article  Google Scholar 

  18. 18.

    Liu H, Li HY, Luo J, Xie SR, Sun Y. Construction of all-in-focus images assisted by depth sensing. Sensors. 2019;19(6):15. https://doi.org/10.3390/s19061409.

    Article  Google Scholar 

  19. 19.

    Darell T, Wohn K. Depth from focus using a pyramid architecture. Pattern Recognit Lett. 1990;11(12):787–96. https://doi.org/10.1016/0167-8655(90)90032-w.

    Article  Google Scholar 

  20. 20.

    Shim SO, Malik AS, Choi TS. Accurate shape from focus based on focus adjustment in optical microscopy. Microsc Res Tech. 2009;72(5):362–70. https://doi.org/10.1002/jemt.20662.

    Article  PubMed  Google Scholar 

  21. 21.

    Bay H, Ess A, Tuytelaars T, Van Gool L. Speeded-up robust features (surf). Comput Vis Image Underst. 2008;110(3):346–59. https://doi.org/10.1016/j.cviu.2007.09.014.

    Article  Google Scholar 

  22. 22.

    Tenenbaum JM. Accommodation in computer vision. Stanford University, 1971.

    Google Scholar 

  23. 23.

    Shim SO, Malik AS, Mahmood MT, Choi TS. Estimation of depth map based on focus adjustment. Proc SPIE Int Soc Opt Eng. 2008;7073:707320-1. https://doi.org/10.1117/12.798191.

    Article  Google Scholar 

  24. 24.

    Subbarao M, Tyan J-K. Selecting the optimal focus measure for autofocusing and depth-from-focus. IEEE Trans Pattern Anal Mach Intell. 1998;20(8):864–70. https://doi.org/10.1109/34.709612

    Article  Google Scholar 

  25. 25.

    Nayar SK, Nakagawa Y. Shape from focus. IEEE Trans Pattern Anal Mach Intell. 1994;16(8):824–31. https://doi.org/10.1109/34.308479

    Article  Google Scholar 

  26. 26.

    Hariharan R, Rajagopalan A. Shape-from-focus by tensor voting. IEEE Trans Image Process. 2012;21(7):3323–8. https://doi.org/10.1109/TIP.2012.2190612.

    CAS  Article  PubMed  Google Scholar 

  27. 27.

    Mahmood MT, Lee IH. Shape from focus based on 3d structure tensor using optical microscopy. Microsc Res Tech. 2020;83(1):48–55. https://doi.org/10.1002/jemt.23386.

    Article  PubMed  Google Scholar 

  28. 28.

    Sheldon MR, Fillyaw MJ, Thompson WD. The use and interpretation of the Friedman test in the analysis of ordinal-scale data in repeated measures designs. Physiother Res Int. 1996;1(4):221–8. https://doi.org/10.1002/pri.66

    CAS  Article  PubMed  Google Scholar 

Download references

Acknowledgements

The assistance from professor Tianzhen Zhang is gratefully appreciated for providing experimental material.

Funding

This work is supported by National Key Research and Development Program of China (No: 2016YFD0200701) and Zhejiang Province Public Welfare Technology Application Research Project (No: LGN19F020001).

Author information

Affiliations

Authors

Contributions

ZZ performed the experiment and analyzed the data; FL contributed significantly to the data analysis; ZZ wote the manuscript; YH contributed to the experiment with constructive suggestions; HF contributed to the concept of the study. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Hui Fang.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Written informed consent for publication was obtained from all participants.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Zhang, Z., Liu, F., Zhou, Z. et al. Roughness measurement of leaf surface based on shape from focus. Plant Methods 17, 72 (2021). https://doi.org/10.1186/s13007-021-00773-y

Download citation

Keywords

  • Shape from focus
  • 3d-reconstruction
  • Surface roughness