- Open Access
A low-cost and open-source platform for automated imaging
Plant Methods volume 15, Article number: 6 (2019)
Remote monitoring of plants using hyperspectral imaging has become an important tool for the study of plant growth, development, and physiology. Many applications are oriented towards use in field environments to enable non-destructive analysis of crop responses due to factors such as drought, nutrient deficiency, and disease, e.g., using tram, drone, or airplane mounted instruments. The field setting introduces a wide range of uncontrolled environmental variables that make validation and interpretation of spectral responses challenging, and as such lab- and greenhouse-deployed systems for plant studies and phenotyping are of increasing interest. In this study, we have designed and developed an open-source, hyperspectral reflectance-based imaging system for lab-based plant experiments: the HyperScanner. The reliability and accuracy of HyperScanner were validated using drought and salt stress experiments with Arabidopsis thaliana.
A robust, scalable, and reliable system was created. The system was built using open-sourced parts, and all custom parts, operational methods, and data have been made publicly available in order to maintain the open-source aim of HyperScanner. The gathered reflectance images showed changes in narrowband red and infrared reflectance spectra for each of the stress tests that was evident prior to other visual physiological responses and exhibited congruence with measurements using full-range contact spectrometers.
HyperScanner offers the potential for reliable and inexpensive laboratory hyperspectral imaging systems. HyperScanner was able to quickly collect accurate reflectance curves on a variety of plant stress experiments. The resulting images showed spectral differences in plants shortly after application of a treatment but before visual manifestation. HyperScanner increases the capacity for spectroscopic and imaging-based analytical tools by providing more access to hyperspectral analyses in the laboratory setting.
Remote monitoring of plant physiology and biochemistry holds enormous potential for understanding plant growth and development in settings ranging from the laboratory to the field . Reflectance spectroscopy is rapidly emerging as a highly effective and practical approach for the rapid, non-destructive estimation of a wide variety of chemical, biophysical, and metabolic plant traits in living tissue . The technique uses variations in leaf optical properties that arise from the interaction of light and chemical bonds [3, 4]. For example, measurement of absorbance and reflectance features in the visible spectrum and out into the infrared (~ 400 to 2500 nm) have been used to directly estimate foliar structure, plant chemical composition, water content, and metabolic status [5,6,7,8,9]. Some spectral features are known to be associated with specific chemical or stress responses, such as the detection of plant physiological stress using the photochemical reflectance index [10,11,12,13]. These spectra remain a rich resource information yet to be mined for plant studies and phenotyping, with the potential for many additional features of plant physiology and chemistry to be extracted [14, 15].
While such spectroscopic imaging techniques offer huge potential, they also raise many practical issues that currently limit applications . For example, scheduling collections for multiple spectroscopic measurements across many samples and over many time points is often logistically difficult and even prohibitively time consuming, especially in the field where variable light conditions affect measurements . These become important issues for the analysis of plant responses which tend to change rapidly in response to environmental or biotic stressors (requiring time-course data collection) and also to vary widely between different plant species or genotypes (requiring sampling from many individuals). To alleviate these setbacks, researchers and private companies have developed machines to automate most if not all of the imaging process [17,18,19,20,21,22].
The widest use of hyperspectral imaging is from sensors mounted on drones or small aircraft [23, 24]. Recently, stationary machines have also been created to scan non-moving fields or targets [1, 17, 25, 26]. In these machines, the imaging instrument is typically mounted to an optimal point and a motor-driven axis moves the plants that need to be scanned, or vice versa. The machine is controlled by software that accepts user input such as positional and instrument control commands. Although these machines can produce highly informative data from multiple samplings due to the non-invasive nature of hyperspectral imaging, there are significant drawbacks that have severely limited their accessibility to many plant researchers .
The main constraints with stationary imaging machines are high cost, often complex construction, and relatively large size, which all impact application in the laboratory setting . For example, the LemnaTec Gmbh Lab Scanalyzer has a robust design and features a flexible arsenal of sensors, but its cost is prohibitive to most researchers . Similarly, the Field Scanalyzer is suitable for field-based research, but it is even more expensive, requires a team of people to build, and can only be used in a large crop setting . Custom machines have been built by researchers to provide equivalent non-invasive analysis, but a lack of information and technical details make these platforms relatively inaccessible and difficult to reproduce by most groups. In addition, differences in design for each platform mean that comparisons of results between studies can be harder to make and/or reproduce [6, 30,31,32,33].
However, with the advent of open-sourced, or so-called “maker” electronics and parts, researchers can now build these types of machines more easily. High quality, inexpensive, and well-documented tools are becoming increasingly available. Low-cost electronics and customizable materials (such as 3D printed parts) have given rise to a unique set of novel lab hardware [34, 35]. In addition, the nature of open-source materials makes sharing these new inventions easier as well [36, 37].
In this study, we have created HyperScanner: a non-invasive, lab-based system for hyperspectral imaging (Fig. 1a). Although most commercial spectrometers are not open-sourced, the HyperScanner platform itself is based entirely on other open-sourced systems and products that are also affordable. We combined an already existing open-source Computer Numerical Control (CNC) machine, the X-Carve, with custom software to create HyperScanner . If one already has a preferred imaging instrument, the cost of the HyperScanner platform (not including the imaging spectrometer) totals less than 3000 USD.
HyperScanner’s hardware and software are designed with the flexibility to tailor it to specific experiment protocols with minimal commitment of effort and time. The large scanning area allows for many plant samples, with a current capacity of about ~ 20 standard seed trays, to be studied simultaneously. A scan of two trays each containing 18 Arabidopsis thaliana plants takes approximately 5 min. In addition to HyperScanner’s versatility, the design is fully modular: any part can be reengineered for a different sensor or experiment [39, 40]. For example, the dimensions of the instrument mount can be changed and 3D printed again to house a different instrument (Fig. 1b). Presently, the system is equipped with a Headwall Photonics (Bolton, MA, USA) Nano Hyperspec (Nano) Visible and Near Infrared (VNIR, 400–1000 nm) detector but has the flexibility to integrate other imaging modalities to provide an even deeper set of structural and chemical data to monitor plant performance (Fig. 1b, c). Further, the aim of this project was not only to create a low-cost and lab-based imaging machine, but also to provide documentation on its operation and construction with the goal of making this type of machine more accessible to plant researchers. Details regarding technical information, construction, and software are discussed in the methods section.
We validated HyperScanner for plant research by imaging Arabidopsis plants experiencing drought or saline stress, as both are easily imposed and controlled environmental stresses . Along with the readily available tools to quantify plant health by means of Red–Green–Blue (RGB) photography, Arabidopsis was chosen as the test subject because of the extensive literature characterizing its responses to a wide range of environmental conditions [42, 43]. This broad background of knowledge allows us to place insights from HyperScanner’s spectral data into the broader context of physiological, biochemical, and molecular changes already characterized under those conditions. The HyperScanner was able to identify spectral shifts in the plant before any physiological harm could be detected using RGB photography  (Fig. 1d). If applied in the field, HyperScanner would allow the horticulturist or agronomist to amend the environment or use stress resistant varieties to ensure robust crop yields [45, 46].
HyperScanner proved to be a consistent and reliable tool that is able to collect reflectance data. The construction of HyperScanner consisted of low-cost and open-source materials, which resulted in a modular design. This approach allows the system to be modified and customized to support many different kinds of sensors and experiments. In our case, we optimized the position of the light mounts so that they provided effective lighting for an Arabidopsis experiment (Fig. 1b). Our custom software ensured that the operation of HyperScanner was not only reliable but also intuitive for the user.
We grew the plants for 19 days in preparation for the stress period and hyperspectral analysis and used daily time-lapse photography with an 8MP Raspberry Pi Camera (RGB imaging; Raspberry Pi camera V2; Adafruit, New York, NY, USA). Differences in plant morphology were observed by performing this time-lapse photography of plant growth and applying the Phenotiki analysis package  (Fig. 2a).
On day 19, water was added to the control plants, the salt stress sample was given 2 L of 500 mM NaCl solution as an osmotic and ionic stress, and the drought sample was allowed to dry out by withholding watering from this time point onwards. The plants’ growth environment (weight, moisture level and temperature) was monitored from day 13 until the final scan, and the data is available in Additional file 1.
Analysis of plant morphology is presented in Fig. 2b–d. The investigation of morphological traits showed a reduction in growth rates of the drought samples and a gradual decline of plant size in the saline group (that correlates with the bleaching of the leaves) (Fig. 2a). Visual indicators of stress response represented by trends in reduction in rosette diameter, perimeter, and area were detectable 1–2 days after the stress point (Fig. 2a). Statistical analysis of these parameters showed a significant difference 1–2 days after the saline treatment and 4–6 days after the drought period began (Fig. 2b–d). On day 23, significant effects from saline stress on all traits occurred but only a significant effect on leaf area due to drought could be seen. Drought effects on leaf diameter and perimeter were significant on days 24 and 25, respectively. Thus, using conventional morphometric analysis it was possible to see a difference on day 24 for the drought and day 21 for the saline stress.
Hyperspectral scanning began on day 20 (1 day after the stress point) and continued until day 26. Radiance images were converted to absolute reflectance and vector normalized. Pixels containing plants were extracted and sample pixels (n = 2000) were used for further analysis. Reflectance curves on day 20 revealed significant contrasts between the stress and control groups and provided a rapid indication that the plants were experiencing either drought or saline stress. Reflectance data measured on day 20 are presented as Normalized Difference Spectral Index (NDSI, comparing all wavelength pairs) heatmaps and wavelength by wavelength t-tests in Fig. 3. NDSI correlation (r-value) heatmaps indicate statistical trends in all treatment combinations (Fig. 3a). The NDSI heatmaps show narrowband sensitivity to drought at ~ 700 to 900 nm, and saline addition at ranges ~ 500 to 650 nm, ~ 700 to 720 nm, and ~ 800 to 900 nm. Differences between the drought and salinity are present at ~ 580 to 650 nm and ~ 750 to 800 nm. Although each sample combination exhibited unique trends, significant effects on wavelengths in the red-edge and near-IR ranges are present in all sample groups.
Wavelength by wavelength p values from t-tests on day 20 reflectance data between treatments are presented in Fig. 3b. A log (base 10) plot of the p values illustrates wavelengths in the reflectance spectrum with the greatest significance for differentiating treatments (values below the dotted lines on Fig. 3b). The resulting p values denote the specific wavelengths that changed due to stress. Each sample combination exhibited major significance in the 650–700 nm range and wavelengths past the red edge inflection point (~ 700 nm), and minor significance in the 500–650 nm range. Drought samples exhibited significant effects broadly in the near infrared, related to leaf/plant structure. Salinity samples exhibited narrower features, particularly at red wavelengths (related to effects on chlorophyll), at the red-edge (720 nm) due to stress, and in several locations in the near infrared due to impacts on leaf structure. As well, p values between the two stress treatments reveal differences in the patterns that all plant samples experienced. Notably, the two treatments exhibited significantly different responses in the green and red wavelengths, at the red-edge (greater effects of salinity in green, red and near-infrared) and very significant differences at 770 nm. While each treatment showed significant effects relative to the control at longer near-infrared wavelengths (> 800 nm), the two treatments were less distinguishable from each other at these wavelengths, pointing to the utility of broadband versus narrow spectral data. Significant differences between all sample groups could be observed on day 20, i.e., several days before the morphological RGB analysis was able to do so.
Figure 4 presents trends in the stress plants from day 20 to day 26 using reflectance ratios derived from the hyperspectral imagery, based on significant wavelengths identified in Fig. 3. This enables visualization of changes in plant stress as it progresses by date. Ratios specific to the stress type were calculated with representative wavelengths, 782 nm/544 nm to compare drought stress with the control and 676 nm/743 nm to compare the salinity stress with the control. The calculated ratios were interpolated into each pixel of representative plant images on days 20, 23, 24, 25, and 26. Differences with the control are seen on day 20 in each stress type. Generally, ratios increased over time in stress samples and little to no change was seen in the control. In contrast to the day 20 analysis (Fig. 3), this method allowed us to observe spectral shifts in the spatial domain as well as across a period of time.
To assess the radiometric fidelity of the Nano measurements, we compared reflectance from a flat leaf of a control plant made with an Analytical Spectral Devices full-range contact spectrometer and calibrated light source (ASD; FS3 350-2500; Analytical Spectral Devices, Boulder, CO, USA) with the Nano image of the same plant. The comparison demonstrated the congruence between measurements from the two types of instruments (Additional file 2), indicating that the measurement setup for the HyperScanner with the Nano (i.e., light source, calibration panels) is sufficient to match leaf level reflectance from a higher Signal-to-Noise Ratio (SNR) instrument such as the ASD.
Plants experience a range of abiotic stresses in natural ecosystems, and in the context of an agroecosystem, environmental stresses can lead to reductions in growth rate and altered vegetative and reproductive development, which often plays out as being detrimental to crop yields. We mimicked environmental stresses common to agroecosystems within our controlled environment  to ask how well the HyperScanner could be used to rapidly monitor plant responses to these challenges. As the plants sustained the effects of drought or salinity, visual symptoms of the stress appeared (Fig. 2a). Monitoring plant growth by means of conventional RGB photography coupled to statistical analysis of the morphometric data extractable from these images allowed us to define a point when the physiological effects from the stress were detectable as being statistically significant alterations in leaf growth rates from the control. Thus, analysis of rosette diameter, perimeter, and leaf area indicated that the drought samples’ growth rates were not statistically significant until day 23, i.e., 4 days after imposition of drought by cessation of watering (Fig. 2). The addition of the 500 mM saline solution affected the plants more rapidly than the drought stress. The salinity caused the plants to exhibit chlorosis of the leaves  and statistical significance on leaf expansion was again seen on day 22. In both treatments, statistical significance of plant growth responses to stress application was detected several days after the stress application. This delay is likely due to the limitations of visual analysis of parameters, such as growth, on detecting the earliest responses to stress that are likely to be through alterations in gene expression and plant biochemistry . If a farmer or horticulturist were to be solely relying on analysis of RGB photography to assess stress conditions in the field, significant reduction in plant size and yield would be inevitable as these would be tightly linked to the changes being used to detect stress response from the imaging data.
By using hyperspectral data, the amount of information available to a researcher dramatically increases [50, 51], and so, the RGB analysis in Fig. 2 does not contain the vast amount of information that the hyperspectral imaging can potentially provide. Thus, statistical analysis from the hyperspectral data provided information on plant stress being statistically significant well before alterations in growth were detected from the RGB data. The NDSI analysis in Fig. 3a reveals effects on the plants 1 day after the stress period began. When compared to the control, the drought stress was most significant in red edge to near IR bands (~ 700 to 900 nm). The heatmap of the salt and control revealed more significant trends (both magnitude and quantity), reinforcing that the saline stress was likely affecting the plants to a greater degree (at least in terms of changes in hyperspectral signal) than the drought stress (Fig. 2). The salt and drought NDSI show that the two types of stress have comparable effects on Arabidopsis 1 day after stress. Red-edge and near-IR wavelengths show significance in both treatments when compared to the control, but not in their own comparison, indicating that the two samples were affected in the same range of wavelengths and to similar amounts. On the other hand, correlations in the 580–650 nm and 750–800 nm ranges are present in the third NDSI and are not seen in comparisons with the control, which suggests that the stress samples changed differently in these ranges.
Analyses of p values in Fig. 3b reveal changes along certain wavelengths in treatments and confirm our hypotheses drawn from the NDSI heatmaps in Fig. 3a. Significant p values resulting from the comparison of 580–640 nm between the two stress treatments indicate that these changes are unique to each stress type (Fig. 3b), namely through greater effects on reflectance in the saline treatment potentially due to differences in changes in relative pools of accessory (non-chlorophyll) pigments between the treatments. Similarly, wavelengths between 600 and 650 nm changed in both stresses when compared to the control, but when the stresses are compared against each other, major statistical differences are present, indicating that the effect on the (most likely) chlorophyll absorption in the red was much stronger with the saline treatment. As well, significance in the 700–800 nm range affirms that the NDSI correlations from that range are indeed unique to the stress type, with a greater impact on red-edge reflectance (an indicator of overall plant health) in the saline treatment. T-tests between the stress types allow for powerful conclusions to be made that are not available with comparisons to the control. Not only can change be seen between stresses and control groups, but the change relative to different stresses can be analyzed.
Figure 3 also suggests that certain response mechanisms were employed by the plants to each stress condition. The drought samples experienced a shift in reflectance at the ~ 520 to 530 nm wavelengths compared to the control. This corresponds to the location of the photochemical reflectance index band at 531 nm , which has been shown to relate strongly to plant xanthophyll cycle pigment pools that change in response to stress . In contrast, the ~ 520 to 530 nm band was not significantly changed in the saline stress, which confirms that the associated physiological change was likely only experienced by the drought stress. Similarly, the saline stress saw shifts in the sub-500 nm range that were not present in the drought samples, perhaps due to effects on chlorophyll-b and carotenoids that have strong absorptance features in the blue . Similar observations can be drawn on different wavelengths. In each treatment, the most significant changes are seen after 700 nm and into the near-IR range. In the 750–800 nm range, both stress sample’s reflectance shifted compared to the control, and the drought shifted more than the salinity. Analysis between the treatments reinforce this idea, as t-tests on those wavelengths resulted in the smallest p values. In addition, bands in the near-IR range were significant in both the drought and salinity samples; however, the t test between the two stresses shows only a small degree of significance, demonstrating that the plants’ reflectance changed in a similar manner.
Visualization of the hyperspectral imagery in Fig. 4 offers the capacity to track the progression of change using reflectance-based plant experiments. Because the response wavelength ratios can be visualized on the plant itself, spatial trends can be analyzed over time along with spectral trends. For instance, the effects on the drought ratio were initially noticeable at the base of the plant but spread through the stem and then to the leaves along the vasculature (Fig. 4a). Interestingly, the older salt stressed leaves experienced a reflectance change before the younger leaves (Fig. 4b). Such analyses are not possible if one only considers purely spectral data.
HyperScanner is easy and inexpensive to build and suitable for many varied plants and experiments. Any plant can be scanned as long as they can fit into the scan area and is not taller than the height of the instrument, although, the height of the instrument, scanning speed, and the scanning routes can be changed to accommodate different species of plants. For example, in addition to Arabidopsis, we have successfully used HyperScanner on much larger cotton plants.
Along with being able to support many different plants, HyperScanner can support different sensors and indeed, expansion to incorporate multiple parallel imaging modalities is a core concept for the HyperScanner. Thus, the 80/20 rail system combined with printing custom mounts allows for the rapid integration of new sensors. In this study, only the Nano was mounted as a detector in order to efficiently test the feasibility of HyperScanner. One notable current limitation is that the Nano is operated through the manufacturer’s software. This makes starting the scans slightly more cumbersome. The full integration of the Nano into our software will increase the quality of operation.
We are also currently implementing other sensors, including a laser rangefinder to detect and compensate for different plant heights . Arabidopsis is very flat and not variable enough in height for a rangefinder to have been relevant for this study, but this will be an important addition for larger, more 3-dimensional plants. A parallel thermal imaging system will allow for the assessment of changes in the critical parameters of transpiration and photosynthetic capacity and a laser scanner will further enable the assessment of changes in foliar structure and biomass across future experiments [6, 14, 27, 53]. The integration of these imaging systems will be documented in future work using the HyperScanner.
Perhaps the greatest potential for HyperScanner is in full automation of control and analysis . The control system is currently being transformed from open-loop to closed-loop (i.e., using internal sensors and feedback) control. In addition, an exciting area for future development is to incorporate neural networks for plant classification . The combination of HyperScanner and neural networks will allow for even more rapid acquisition and classification of reflectance data [28, 56]. Automated stress detection via a neural network could allow a researcher to maintain healthy samples with minimal interference. Indeed, the complete automation of the HyperScanner will result in an extremely efficient system .
The HyperScanner system was designed to collect hyperspectral data with minimal human effort while keeping the system accessible to researchers, affordable, and available for the addition of more instruments. The software, data, and other relevant files are publicly available within the article. In multiple experiments, we were able to measure absolute reflectance in Arabidopsis stress experiments. The data from HyperScanner showed spectral differences at an earlier point during the stress than visual observations and identified differences in stress responses between two treatments. HyperScanner can be used for improved detection of plant stress and holds a high potential to be a commonplace method for studying plants in many research settings.
Arabidopsis growth environment
A controlled growth chamber was used to grow plants adjacent to the HyperScanner. Six 1020 seed trays (Greenhouse MegaStore, Danville, IL, USA) each with 18 Arabidopsis Col-0 plants growing in potting soil were maintained at 22 °C with an 18:6 h day/night cycle (100 µmol/m2/s−1, from 4 foot fluorescent lights). Pot weights and RGB pictures of the plants were taken daily. Plant phenotypic response was analyzed from Raspberry Pi camera images using Phenotiki . Hyperspectral scans were taken each day for 7 days starting at 20 days post germination. Two trays were used as control samples and were kept with constant water availability by adding 500 mL every day. The remaining four trays were split into 2 treatments. Two trays were not watered from day 19 onwards to impose drought stress. The two other trays were used for salt stress experiment by adding 2 L of 500 mM NaCl. To mitigate a possible border effect, edge plants should be excluded from data analysis and uniform lighting across the plant tray should be ensured.
Overview of CNC and HyperScanner
HyperScanner is based on CNC technology . CNC is an automation of machine tools that utilizes computer control, smart sensors, and stepper motors. In CNC, the control computer executes sequential commands which are calculated based on user inputs and the machine’s physical properties; smart sensors provide necessary information used to execute the computer’s control algorithms. Stepper motors control the physical position of the tools. The computer-based control of CNC allows processes to be predetermined and also enables the recalibration of computer commands based on external changes. This digitization results in an automated and high-precision system.
HyperScanner can precisely and accurately move to a point (X, Y, Z) with a user-chosen speed. The detector (in our case, the Nano Hyperspec line scanner, see below) is mounted to a central point which can move along the X, Y, and Z axes. The central point moves over the scan area underneath which facilitates the scanning of the plants. Our software gives the user the ability to intuitively control the movement of the machine in real time, create pre-planned scanning routes, and execute those routes.
Nano Hyperspec Imaging Spectrometer
The Nano is a line scanner (also known as pushbroom scanner) designed for the VNIR range (400–1000 nm) . It consists of 640 spatial bands (pixels) and 270 spectral bands. The spectral bands are spaced at 2.2 nm/pixel. The Nano weighs 0.544 kg and has built-in mounting points, making it extremely suitable for the HyperScanner.
Technical specifications of the HyperScanner
HyperScanner was built to achieve a large scan area, variable speed, and precise movement control. HyperScanner features a scan area of 2.1 m2. Due to the constraints of imposed by the operating speed of the Nano, HyperScanner runs at a scanning speed of 1 cm/s; however, this may vary when using other detectors, and so the speed can be adjusted within the HyperScanner interface. Additionally, as the Nano is a line scanner, the length of the scan line is proportional to the height of the Z axis. The height can be optimized to a line that covers the sample dimensions. For instance, a line of ~ 8 cm was used in this experiment to cover the length of each pot. Table 1 lists the relevant technical values.
Construction of the HyperScanner
HyperScanner was built using the 80/20 aluminum rail system, and the construction is based on X-Carve’s existing system . Although detailed building instructions can be found on X-Carve’s website, many adjustments were made for our purposes [40, 59]. 3D models of HyperScanner are presented in Fig. 5a and Additional file 3. Four 1.8 m rails are vertically placed at each corner of the machine to support eight additional 1.8 m rails which are used to create two 1.8 m by 1.8 m horizontal sections. The bottom section supports removable platforms on which plant trays or pots can be scanned. The top section supports the stepper motor movement system. Two stepper motors are mounted to metal side plates that are attached to the top section support rails (Fig. 5a, Additional file 3). The side plates attached with motors are mounted with wheels that allow them to slide along two of the top section support rails (Y axis). The X axis rail is between the side plates. A metal gantry with wheels moves across this rail and also supports the linear actuator (Z axis). The X and Y axes operate with a belt drive and the Z axis operates with a worm drive. The Nano and lights are mounted at the bottom of the linear actuator by using custom-made mounts. Custom made mounts and additional support brackets were designed within SolidWorks, and each mount was specifically designed to match the existing hardware. The mounts were printed using an open-source 3D printer with PLA filament (LulzBot TAZ 5; Aleph Objects, Inc., Loveland, CO, USA). All SolidWorks files are available for download at (https://doi.org/10.7910/DVN/9DLR7S). The bill of materials (Additional file 4) lists all materials necessary to the construction of the HyperScanner. Necessary tools are not listed.
Control and wiring
The control algorithms that most CNC machines implement can be classified into two categories: open-loop control and closed-loop (feedback) control. In open-loop control, the process is linear, meaning that the computer simply accepts inputs from the operator and outputs a signal to control the system. Figure 5b shows the fundamental open-loop control design . In closed-loop control, sensors are placed in the system to feed back information, which enables a higher degree of automation (Fig. 5c). HyperScanner is open-loop controlled. HyperScanner consists of four stepper motors and one linear actuator. The X axis is controlled by two motors; the Y axis is controlled by one motor; the Z axis is controlled by one motor and a linear actuator that allows for precise vertical positioning. Although open-loop control is currently implemented in HyperScanner, we intend to integrate feedback control to achieve a more robust and fully automated control mechanism.
HyperScanner’s basic wiring topology is included in Additional file 5. The computer accepts user input and transcribes it to stepper motor movements, which follows the same control structure in Fig. 5b. User input is translated from the computer to the Arduino and then to the gShield through custom software. The gShield is a stepper motor driver: once the gShield receives correctly notated commands, the appropriate level of power is driven to the motors. The Arduino is powered by the computer through a Universal Serial Bus (USB) connection that also doubles as the serial connection. The stepper motors, gShield, Nano, and lights are powered by a digital-control direct current power supply. A dedicated power source has not been implemented, as the weight and power load on the linear actuator is constantly changing due to the addition of various devices. Thus, the power supply needs to be constantly adjusted.
Custom light mounts were 3D printed along with the Nano mount. The mounts are fitted to 20 W halogen bulbs (MR11; Simba Lighting, Torrance, CA), and the bulbs are powered close to the maximum power rating. The lights are mounted in parallel to the Nano’s scan line so maximum even lighting is achieved. The mounts are connected to the end of the Z axis so that the lighting environment does not change while the scanner is moving. Additionally, the mounts are connected to 80/20 rails that slide along the Z axis linear actuator so that their height from the samples can be adjusted.
Due to the indoor setting, artificial lighting was one of the most important considerations when building HyperScanner. The lighting and light mounts were reiteratively modified until a desirable configuration was achieved, resulting in even and consistent illumination throughout each of the scans. The interchangeability and adjustability of the lights’ power and position made this an easy task.
A set of software tools were developed to support the operation of the HyperScanner, both for direct interfacing with the machine’s hardware and for higher-level user functions. These tools, named Ardupy, have been made publicly available on the University of Wisconsin EnSpec organization’s Github page (https://github.com/EnSpec/Plant_CNC_Controller) as well as on Zenodo (https://doi.org/10.5281/zenodo.1406721) . Ardupy manages the conversion of movement instructions received over the Uno’s USB serial port into appropriate gShield instructions. Direct interfacing with the gShield is handled by the Arduino AccelStepper library . AccelStepper manages the General-Purpose Input/Output (GPIO) outputs that are required to move a stepper motor to an absolute position with a given speed. User control of the AccelStepper library is achieved through a custom instruction set established between a host computer and the Arduino over a Universal Asynchronous Receiver/Transmitter (UART) serial connection. The instructions are encoded as 3-tuples of 32-bit integers that specify AccelStepper values and parameters. A full description of the instruction set is provided in Table 2. To prevent incoming instructions from interrupting the execution of previously received ones, instructions are enqueued in a First-In, First-Out (FIFO) as they are received via UART and dequeued when a previous instruction completes. In the current iteration of the software, this FIFO is implemented with the Arduino QueueList library . A diagram of the data flow between the host computer and CNC machine is provided in Fig. 5d.
The operation of HyperScanner is simple, after creating scanning routes and tuning instrument parameters (Additional file 7). First, the Nano is attached to the mount and connected to the computer. Scan height, speed, and line length are calculated separately, based on the Nano’s field of view and integration time. This experiment used an integration time of 14 ms. Once the power supply and lights are turned on and Ardupy has been launched, the Nano is moved to a white panel and calibrated. A scanning route is chosen, and the plant samples are placed in the correct positions according to the selected route. After the initial setup, the Nano is set to capture, and the route is executed. One scan of two plant trays (18 plants each) takes less than five minutes. After the scan is completed, files can be transferred off of the Nano or different routes and plants can be loaded. Note that a scan of the white panel should be included for image processing. After the all the plants are scanned, the images can be processed.
One important element of operation requiring attention during the equipment setup is that the belt drives must be checked to ensure they do not need to be re-tensioned. Correct belt tension is needed for optimal movement ability of the scanner and therefore this check is important. Although, re-tensioning the belts is needed very seldom.
Hyperspectral scans were processed with ENVI 5.0 (Exelis Visual Information Solutions, Inc., Boulder, CO, USA). Each image scan includes a calibrated 99% reflectance spectralon panel (Labsphere, North Sutton, NH, USA), which was used to calculate reflectance and estimate the noise level of the hyperspectral image. Coefficient of Variation (CV) was used as the criteria for high SNR wavelength selection. After calculating along-track CV of the white panel image, wavelengths ranging from 477.1 to 903.42 nm, which yield low CV, are considered for further analysis. The white reference radiance spectrum was estimated for the whole image from the vertical scan line of the spectralon panel that had the maximum median radiance. Every plant pixel was used for analysis by delineating regions of interest within ENVI. The spatial and spectral edges of the hyperspectral image cube were excluded from analysis because it minimizes smile and keystone effects (e.g., cross-track variation in wavelength centers) . Relative reflectance is calculated as:
where i and j correspond to the row and column of a pixel. k is the wavelength of the pixel. DNi,j is the radiance spectrum of each pixel. DNwhite is the radiance spectrum of the reference white panel. The calculation is done on a wavelength-by-wavelength basis.
Normalized difference spectral indices were calculated for each of the Nano’s 270 spectral bands. The difference in reflectance for a pair of bands (e.g., i and j) is divided by the sum, as in the following:
For each pair, indices were calculated for a sample of n = 2000 pixels. Statistical tests were then done on each NDSI combination and heatmaps were generated with the resulting statistical data.
computer numerical control
Headwall Photonics Nano Hyperspec
visible and near infrared
Normalized Difference Spectral Index
analytical spectral devices
universal serial bus
universal asynchronous receiver/transmitter
graphical user interface
coefficient of variation
Pieruschka R, Poorter H. Phenotyping plants: genes, phenes and machines. Funct Plant Biol. 2012;39:813–20.
Fiorani F, Schurr U. Future scenarios for plant phenotyping. Annu Rev Plant Biol. 2013;64:267–91.
Curran PJ. Remote sensing of foliar chemistry. Remote Sens Environ. 1989;30:271–8.
Ustin SL, Gitelson AA, Jacquemoud S, Schaepman ME, Asner GP, Gamon JA, et al. Retrieval of folair information about plant pigment systems from high resolution spectroscopy. Remote Sens Environ. 2009;113:S67–77.
Matsuda O, Tanaka A, Fujita T, Iba K. Hyperspectral imaging techniques for rapid identification of Arabidopsis mutants with altered leaf pigment status. Plant Cell Physiol. 2012;53:1154–70.
Humplík JF, Lazár D, Husičková A, Spíchal L. Automated phenotyping of plant shoots using imaging methods for analysis of plant stress responses—a review. Plant Methods. 2015;11:1–10.
Serbin SP, Singh A, Desai AR, Dubois SG, Jablonski AD, Kingdon CC, et al. Remotely estimating photosynthetic capacity, and its response to temperature, in vegetation canopies using imaging spectroscopy. Remote Sens Environ. 2015;167:78–87.
Singh A, Serbin SP, McNeil BE, Kingdon CC, Townsend PA. Imaging spectroscopy algorithms for mapping canopy foliar chemical and morphological traits and their uncertainties. Ecol Appl. 2015;25:2180–97.
Asner GP, Martin RE, Anderson CB, Knapp DE. Quantifying forest canopy traits: imaging spectroscopy versus field survey. Remote Sens Environ. 2015;158:15–27.
Gamon JA, Serrano L, Surfus JS. The photochemical reflectance index: an optical indicator of photosynthetic radiation use efficiency across species, functional types, and nutrient levels. Oecologia. 1997;112:492–501.
Garbulsky MF, Peñuelas J, Gamon J, Inoue Y, Filella I. The photochemical reflectance index (PRI) and the remote sensing of leaf, canopy and ecosystem radiation use efficiencies: a review and meta-analysis. Remote Sens Environ. 2011;115:281–97.
Suárez L, Zarco-Tejada PJ, González-Dugo V, Berni JAJ, Fereres E. The photochemical reflectance index (PRI) as a water stress indicator in peach orchards from remote sensing imagery. Acta Hortic. 2012;962:363–70.
Sarlikioti V, Driever SM, Marcelis LFM. Photochemical reflectance index as a mean of monitoring early water stress. Ann Appl Biol. 2010;157:81–9.
Kuska MT, Mahlein AK. Aiming at decision making in plant disease protection and phenotyping by the use of optical sensors. Eur J Plant Pathol. 2018;5:5. https://doi.org/10.1007/s10658-018-1464-1.
Wahabzada M, Mahlein AK, Bauckhage C, Steiner U, Oerke EC, Kersting K. Plant phenotyping using probabilistic topic models: uncovering the hyperspectral language of plants. Sci Rep. 2016;6:1–11.
Kolukisaoglu Ü, Thurow K. Future and frontiers of automated screening in plant sciences. Plant Sci. 2010;178:476–84.
Walter A, Studer B, Kölliker R. Advanced phenotyping offers opportunities for improved breeding of forage and turf species. Ann Bot. 2012;110:1271–9.
Virlet N, Sabermanesh K, Sadeghi-Tehran P, Hawkesford MJ. Field scanalyzer: an automated robotic field phenotyping platform for detailed crop monitoring. Funct Plant Biol. 2017;44:143–53.
Li L, Zhang Q, Huang D. A review of imaging techniques for plant phenotyping. Sensors (Switzerland). 2014;14:20078–111.
Arvidsson S, Pérez-Rodríguez P, Mueller-Roeber B. A growth phenotyping pipeline for Arabidopsis thaliana integrating image analysis and rosette area modeling for robust quantification of genotype effects. New Phytol. 2011;191:895–907.
Fanourakis D, Briese C, Max JFJ, Kleinen S, Putz A, Fiorani F, et al. Rapid determination of leaf area and plant height by using light curtain arrays in four species with contrasting shoot architecture. Plant Methods. 2014;10:1–11.
Tisné S, Serrand Y, Bach L, Gilbault E, Ben Ameur R, Balasse H, et al. Phenoscope: an automated large-scale phenotyping platform offering high spatial homogeneity. Plant J. 2013;74:534–44.
Liebisch F, Kirchgessner N, Schneider D, Walter A, Hund A. Remote, aerial phenotyping of maize traits with a mobile multi-sensor approach. Plant Methods. 2015;11:9.
Berni JAJ, Zarco-Tejada PJ, Suárez L, González-Dugo V, Fereres E. Remote sensing of vegetation from UAV platforms using lightweight multispectral and thermal imaging sensors. Int Arch Photogramm Remote Sens Spat Inform Sci. 2009;38:6.
Atkinson JA, Pound MP, Bennett MJ, Wells DM. Uncovering the hidden half of plants using new advances in root phenotyping. Curr Opin Biotechnol. 2019;55:1–8.
Gamon JA, Cheng Y, Claudio H, MacKinney L, Sims DA. A mobile tram system for systematic sampling of ecosystem optical properties. Remote Sens Environ. 2006;103:246–54.
Araus JL, Cairns JE. Field high-throughput phenotyping: the new crop breeding frontier. Trends Plant Sci. 2014;19:52–61.
Minervini M, Scharr H, Tsaftaris SA. Image analysis: the new bottleneck in plant phenotyping. IEEE Signal Process Mag. 2015;32:126–31.
LemnaTec. https://www.lemnatec.com/. Accessed 20 Oct 2018.
Pereyra-Irujo GA, Gasco ED, Peirone LS, Aguirrezábal LAN. GlyPh: a low-cost platform for phenotyping plant growth and water use. Funct Plant Biol. 2012;39:905–13.
Jansen M, Gilmer F, Biskup B, Nagel KA, Rascher U, Fischbach A, et al. Simultaneous phenotyping of leaf growth and chlorophyll fluorescence via Growscreen Fluoro allows detection of stress tolerance in Arabidopsis thaliana and other rosette plants. Funct Plant Biol. 2009;36:902–14.
Bergsträsser S, Fanourakis D, Schmittgen S, Cendrero-Mateo MP, Jansen M, Scharr H, et al. HyperART: non-invasive quantification of leaf traits using hyperspectral absorption-reflectance-transmittance imaging. Plant Methods. 2015;11:1–17.
Granier C, Aguirrezabal L, Chenu K, Cookson SJ, Dauzat M, Hamard P, et al. PHENOPSIS, an automated platform for reproducible phenotyping of plant responses to soil water deficit in Arabidopsis thaliana permitted the identification of an accession with low sensitivity to soil water deficit. New Phytol. 2005;169:623–35.
Nuñez I, Matute T, Herrera R, Keymer J, Marzullo T, Rudge T, et al. Low cost and open source multi-fluorescence imaging system for teaching and research in biology and bioengineering. PLoS ONE. 2017;12:1–21.
Green JM, Appel H, Rehrig EM, Harnsomburana J, Chang JF, Balint-Kurti P, et al. PhenoPhyte: a flexible affordable method to quantify 2D phenotypes from imagery. Plant Methods. 2012;8:1–12.
Dobrescu A, Scorza LCT, Tsaftaris SA, McCormick AJ. A “Do-It-Yourself” phenotyping system: measuring growth and morphology throughout the diel cycle in rosette shaped plants. Plant Methods. 2017;13:1–12.
Open-source Lab. http://www.appropedia.org/Open-source_Lab. Accessed 20 Oct 2018.
X-Carve. https://www.inventables.com/technologies/x-carve. Accessed 22 Oct 2018.
Katz R. Design principles of reconfigurable machines. Int J Adv Manuf Technol. 2007;34:430–9.
Mehrabi MG, Ulsoy AG, Koren Y, Heytler P. Trends and perspectives in flexible and reconfigurable manufacturing systems. J Intell Manuf. 2002;13:135–46.
Julkowska MM, Klei K, Fokkens L, Haring MA, Schranz ME, Testerink C. Natural variation in rosette size under salt stress conditions corresponds to developmental differences between Arabidopsis accessions and allelic variation in the LRR-KISS gene. J Exp Bot. 2016;67:2127–38.
Minervini M, Giuffrida MV, Perata P, Tsaftaris SA. Phenotiki: an open software and hardware platform for affordable and easy image-based phenotyping of rosette-shaped plants. Plant J. 2017;90:204–16.
Katori T, Ikeda A, Iuchi S, Kobayashi M, Shinozaki K, Maehashi K, et al. Dissecting the genetic control of natural variation in salt tolerance of Arabidopsis thaliana accessions. J Exp Bot. 2010;61:1125–38.
Römer C, Wahabzada M, Ballvora A, Pinto F, Rossini M, Panigada C, et al. Early drought stress detection in cereals: simplex volume maximisation for hyperspectral image analysis. Funct Plant Biol. 2012;39:878–90.
Yao R, Yang J, Wu D, Xie W, Gao P, Jin W. Digital mapping of soil salinity and crop yield across a coastal agricultural landscape using repeated electromagnetic induction (EMI) surveys. PLoS ONE. 2016;11:1–20.
Roy SJ, Negrão S, Tester M. Salt resistant crop plants. Curr Opin Biotechnol. 2014;26:115–24.
Hu Y, Schmidhalter U. Drought and salinity: a comparison of their effects on mineral nutrition of plants. J Plant Nutr Soil Sci. 2005;168:541–9.
Apse MP, Aharon GS, Snedden WA, Blumwald E. Salt tolerance conferred by overexpression of a vacuolar Na+/H+ antiport in Arabidopsis. Science. 1999;285:1256–8.
Chaerle L, Van Der Straeten D. Imaging techniques and the early detection of plant stress. Trends Plant Sci. 2000;5:495–501.
Schimel DS, Asner GP, Moorcroft P. Observing changing ecological diversity in the Anthropocene. Front Ecol Environ. 2013;11:129–37.
Thompson DR, Boardman JW, Eastwood ML, Green RO. A large airborne survey of Earth’s visible-infrared spectral dimensionality. Opt Express. 2017;25:9186.
Parida AK, Das AB. Salt tolerance and salinity effects on plants: a review. Ecotoxicol Environ Saf. 2005;60:324–49.
Chaerle L, Pineda M, Romero-Aranda R, Van Der Straeten D, Barón M. Robotized thermal and chlorophyll fluorescence imaging of pepper mild mottle virus infection in Nicotiana benthamiana. Plant Cell Physiol. 2006;47:1323–36.
Xu XW, Newman ST. Making CNC machine tools more open, interoperable and intelligent—a review of the technologies. Comput Ind. 2006;57:141–52.
Taghavi Namin S, Esmaeilzadeh M, Najafi M, Brown TB, Borevitz JO. Deep phenotyping: deep learning for temporal phenotype/genotype classification. Plant Methods. 2018;14:1–14.
Vijayarangan S, Sodhi P, Kini P, Bourne J, Sun H, Poczos B, et al. High-throughput robotic phenotyping of energy sorghum crops. F Serv Robot. 2017;5:1–14.
Bollinger JG, Duffie NA. computer control of machines and processes. 1st ed. Boston: Addison-Wesley Longman Publishing Co., Inc.; 1988.
Headwall Photonics Hyperspectral Imaging Sensors. http://www.headwallphotonics.com/hyperspectral-sensors. Accessed 20 Oct 2018.
X-Carve Instructions. http://x-carve-instructions.inventables.com/1000mm/. Accessed 24 Oct 2018.
Zenodo Archive of Ardupy. https://zenodo.org/record/1406721. Accessed 30 Aug 2018.
McCauley, M. AccelStepper Library for Arduino. http://www.airspayce.com/mikem/arduino/AccelStepper/. Accessed 20 Oct 2018.
Chatzikyriakidis, E. QueueList Library for Arduino. https://playground.arduino.cc/Code/QueueList. Accessed 20 Oct 2018.
Tomczak, C. CEF Python. https://github.com/cztomczak/cefpython. Accessed 20 Oct 2018.
Greenblatt, M. Chromium embedded framework. https://bitbucket.org/chromiumembedded/cef. Accessed 20 Oct 2018.
The jQuery Foundation. https://jquery.org/. Accessed 20 Oct 2018.
Bostock, M. d3.js Library. https://d3js.org/. Accessed 20 Oct 2018.
Liechiti, C. PySerial module. https://pythonhosted.org/pyserial/. Accessed 20 Oct 2018.
Mouroulis P, Green RO, Chrien TG. Design of pushbroom imaging spectrometers for optimum recovery of spectroscopic and spatial information. Appl Opt. 2000;39:2210.
ML constructed the machine, ran the experiments, and drafted the manuscript. RG designed parts in SolidWorks and created technical drawings. MW wrote the custom software. ZY processed the images and provided the resulting data. RB and SG designed the plant stress experiments. AS conceptualized the machine and helped with initial development of analysis techniques. PT provided instrumentation and technical support for the project, and oversaw the quantitative analyses of spectral data. PT, RB, SG all contributed to the writing and analyses. All authors read and approved the final manuscript.
This work was performed in the Environmental Spectroscopy Lab at the University of Wisconsin. Thanks are due to Katie Gold for helping to create the NDSI heatmaps and Ting Zheng for discussing methods of statistical analysis. Clayton Kingdon provided early guidance on prototyping the system. Thanks also to Amanda Gevens and Paul Bethke for securing preliminary financial support for the effort.
The authors declare that they have no competing interests.
Availability of data and materials
The datasets supporting the conclusions of this article are available in the Cyverse repository (https://de.cyverse.org/de/?type=data&folder=/iplant/home/elytas/experiment_repository). The 3D printable model files are available on the Harvard Dataverse (https://doi.org/10.7910/DVN/9DLR7S). Ardupy is available on our Github (https://github.com/EnSpec/Plant_CNC_Controller) and archived on Zenodo (https://doi.org/10.5281/zenodo.1406721).
Consent for publication
All of the authors consent to the publication of this article.
Ethics approval and consent to participate
No human subjects were involved in this study.
Funding for this research was provided to PT by the Wisconsin Potato Industry Board, USDA-NIFA-SCRI Grant 2011-51181-30629, the College of Agricultural and Life Sciences at UW-Madison, Hatch award WIS01874 and NASA Grant NNX15AN25G. SG and RB were supported by NASA Grants NNX14AT25G and NNX17AD52G, and NSF award IOS-1557899.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.