Design of Cartesian Type Manipulator for Automatically Capturing Plant Images Inside Greenhouse

Article


INTRODUCTION
Digital image is one of data source that is widely used in various scientific research because digital image contains visual information similar to what is seen directly by humans. In the field of agricultural engineering, digital images are used to determine fruit ripeness in sorting machines (Siskandar et al., 2020) as well as for non-destructive measurements based on image color (Saputra et al., 2022). In the field of biology and plant breeding, digital image data is widely used to study the phenotypes of breeding plants (Bayati & Fotouhi, 2018;Gao et al., 2018;Zhang et al., 2016;Subramanian et al., 2012). However, the application of phenomics in plant breeding requires a large set of datasets that are difficult to obtain in the field in a timely manner if done manually, making it difficult to capture important information that influences plant growth and development. The application of a robotic platform that is capable of collecting data on plant phenotypic traits will help plant breeder scientists to begin to uncover genetic mechanisms so as to increase crop production to meet increasing human needs.
In addition to research in the field of plant phenotypes, large numbers of digital images are also needed in artificial intelligence research which is currently growing rapidly. Many researchers in the field of artificial intelligence use secondary data in their research (Yuliany et al., 2022;Wahid et al., 2021) because the amount of digital image data they have is limited, so it is necessary to carry out an image augmentation process to increase the accuracy of the CNN model they are developing. Augmentation process is done by flipping the image horizontally, rotating and enlarging the image randomly. From the above artificial intelligence research activities, it can be seen that researchers tend to use secondary data because capturing images manually requires quite a long time and is tiring. Therefore it is necessary to develop a device capable of capturing images periodically and automatically to obtain sufficient data so that the accuracy of the artificial intelligence model can be increased. One of the devices capable of recording images regularly and automatically is the Robot platform.
The development of robotics technology in agriculture can be said to be quite advanced although for some types of work such as harvesting there are still many obstacles so that commercially viable fruit harvesting robots are still very limited. However, robots with simpler functions and task are already grow up, such as (Bhogavalli et al., 2021) developing gantry-type robots for cultivating vegetable plants independently. Researchers reported that the robot they developed was able to grow vegetables independently and provide irrigation water automatically for several weeks of experiments without human involvement. (Anh et al., 2020) designed a gantry-type manipulator equipped with a machine vision system for harvesting pineapples. Researchers say the success rate of robot manipulators for harvesting pineapples is 95.55% with a harvesting time of 12 seconds per fruit. (Permadi et al., 2021) designed a gantry type robot manipulator to demonstrate the application of robot platform in agriculture. The manipulator design chosen in this study is a cartesian configuration with three linear axes because the gripper control position is more precise and the structure is more robust.
Based on the above descriptions, this research aims to design a Cartesian type robot manipulator with two horizontal translational axes for capturing plant images automatically and continuously for certaint cycles or period of time.

MATERIALS AND METHODS
The equipment used in this study included: Xiaomi Xiaovv HD Webcam with resolution of 1920 ´ 1080 pixels and viewing angle of 150 degree, a computer ASUS Tuf Gaming A15, RAM 8GB, NVIDIA GeForce GTX 1650 Ti along with Python software version 3.10.4, OpenCV version 4.6.0, Visual Studio version 1.70.2, Solid work 2019 and Solid Work 2020. The materials used include manipulator construction materials and materials for plant cultivation. The manipulator construction materials include: 20 mm x 40 mm aluminum V-slot, 3 cm diameter galvanized iron, NEMA 17HS4401 stepper motor with a shaft rotation resolution of 1.8 degrees/step, a supply voltage of 12 volts and a current consumption of 1.7 A, A4988 type stepper motor driver with an output drive capacity of up to 35 V and 2 A, Solid V-Wheel Gantry, 6 mm GT2 timing pulley, 6 mm GT2 timing belt , Raspberry pi 3 with processor: 64-bit quad-core ARM Cortex-A53,

Robot Manipulator Structural Design
The robot manipulator made in this study is a Cartesian type manipulator which is also often called the gantry type with two axes of movement, namely the x-axis and the yaxis as shown in Figure 1.

Figure 1. Construction of Cartesian type manipulator
The rail mechanism for the x and y axes traverse is made using a V-slot aluminum profile with dimensions of 20 mm x 40 mm and is supported using Galvani's iron with a diameter of 3 cm. The dimensions of the manipulator are: length 240 cm, width 130 cm and height 100 cm according to the hydroponic installation that already exists in the greenhouse.

Design Analysis
The manipulator structure that has been drawn is then analyzed statically using SolidWorks 2020 software to determine displacement, safety factor, and von misses stress. The analysis is carried out on the manipulator frame model by providing a continuous force according to the actual loading (Sungkono et al.,

Hardware Control System
The robot manipulator is controlled using a raspberry pi 3 microcontroller to move the camera to the position of capturing plant images, carry out the image capturing process and save the captured plant images. The control system hardware is shown in Figure 3 and the block diagram of the control system is shown in Figure 4.

Control Algorithm
The robot manipulator is designed to capture images of hydroponic plants at 15 different positions in 1 work cycle. The 15 position images were taken to ensure that each image contains more than 6 plant clumps. The image capturing cycle takes place automatically every day at 07.00 AM and 17.00 PM. The applied control algorithm is: The control system first checks whether it is time to perform an image capturing cycle or not. When the time has come, the camera is moved towards position number 1, which is away from the center of the axis coordinates in the direction of the x-axis by 1250 steps, then moved in the direction of the y-axis by 1460 steps. In this case, the stepper motor has a shaft rotation resolution of 1.8 degrees/step. The stepper motor is connected to a timing pulley with a diameter of 12.5 mm so that each step will move the camera translationally as far as 0.197 mm. Because the stepper motor control is done in an open loop so that the rotation of 1250 step of the stepper motor shaft is equivalent to a translational movement of the camera of 1250 x 0.197 mm = 246 mm and a rotation of 1460 steps is equivalent to 288 mm. After the plant image is recorded and stored from that position, the camera is moved to position 2, which is away from the center axis coordinate in the y-axis direction by 2170 steps, followed by capturing and storing the plant image and so on. The complete control algorithm for one image capturing cycle is shown in Figure 5.

Functional Test
Functional tests were carried out to find out whether the robotic manipulator system developed was capable of moving the camera towards the plant image capturing position. The functional test was carried out using three treatments of the stepper motor controller step period, namely 0.002 seconds (500 Hz), 0.003 seconds (333 Hz), and 0.004 seconds (250 Hz). The parameters tested including: pulley slip, theoretical and actual movement distance from the camera position, traveling time, positioning accuracy, stability, system response, system energy consumption. Pulley slip is calculated using equations 1 and 2: (1) where S is pulleys slip (%), Sav is Average pulley slip (%), Xa is actual movement distance (cm), Xt is theoretical movement distance (cm), Spi is pulley slip for the first image capturing position, second, and so on (%), and n is amount of data.
The average movement error for all image capturing positions is calculated using equations 3 and 4.
where K is position errors (cm), K av is average error (cm), and K pi is movement distance error for the first position, second, and so on (cm).
The average speed of camera movement is calculated using equations 5 and 6: where V is speed (cm/s), V av : Average speed (cm/s), V pi is movement speed to the first position, second, and so on (cm/s), X is actual movement distance (cm), and t is actual traveling time (s)

Performance Test
Performance tests were carried out to analyze the performance of the manipulator to capture and store images of hydroponic plants automatically. The robotic manipulator is programmed to capture plant images and is then turned on for 7 consecutive days. Every day the robotic manipulator executes two image capturing cycles, namely at 7 o'clock in the morning and 17 o'clock in the afternoon. In this study, water spinach seeds were used as experimental material.

RESULTS AND DISCUSSION
As stated in the methodology section, before the robot manipulator model is created, an analysis of von Mises stress, displacement and safety factor is carried out using the SolidWorks 2020 application. The material used for the von Mises stress simulation is Aluminum Alloys 6063 T5 with the specifications shown in the table 1. The calculation results of the load acting on the robot manipulator frame are presented in Table 2. Table 2. Load acting on the robot manipulator frame As already mentioned before, the effect of external loading applied to the structure on the stress distribution that occurs in the structure is tested using Von mises stress analysis. Before analysis, some of the data that needs to be input in the simulation includes: technical drawings of the tool, the value and location of the loads acting on the structure, and the materials specification data used in the structure design. After the data is entered in the simulation, the stress distribution that occurs in the structure is displayed by solid work as shown in Figure 6. The von Mises stress simulation results (Figure 6) shown the highest value of 13.783 MPa which is in the safe category because it is far below the yield strength of 145 MPa (Pamungkas et al., 2020).
The green object in Figure 6 shows a fixed position or will not move as long as the robot's manipulator operates. The simulation results on the manipulator frame produce the highest displacement value of 1.081 mm located at the camera position as shown in Figure 7.   The Tresca Failure criterion uses a material's shear strength value which is half of the tensile strength to calculate the safety factor value. Based on these criteria, the value of shear strength is 185 Mpa/ 2 = 92.5 Mpa. The minimum safety factor is calculated as the ratio between the shear strength and the maximum Von Mises, namely 92.5 MPa/ 13,783 MPa. Therefore, the safety factor value obtained from the analysis using Solid work 2020 is a minimum of 6.869 and a maximum of 10 as shown in Figure 8. The higher the safety factor value, the safer the structure can withstand the load. The results of the safety factor analysis show that the frame is able to withstand dynamic loads to impact loads because the minimum safety factor value required for a material capable of withstanding impact loads is in the range of 3 to 5 (Imran & Kadir, 2017;Wibawa & Diharjo, 2019). Islami et al. (2022) conducted a structural analysis of the aluminum V slot material for the 3D printer frame and find out a safety factor value of 15 which was better than the Cartesian manipulator in this study. However, the safety factor of 6,869 is still in the safe category

Robotic Manipulator Prototype
A robotic manipulator for capturing images of plants inside a greenhouse has been successfully made according to the isometric projection size that has been simulated using Solid work 2020. The prototype is shown in Figure 9a. The robot manipulator has a length of 240 cm, a width of 130 cm and a height of 100 cm.

Functional Testing
The pulley slip test results for the three treatments of the stepper motor controller step period are shown in Table 3. From Table 3 it can be seen that the lowest average pulley slip was obtained in the treatment of 0.002 seconds. The slip on the x-axis is greater than the slip on the y-axis due to the loading on the x-axis is greater than that on the y-axis.

Positioning Accuracy
The position error of the camera movement is shown in Table 4. The negative sign in the table indicates that the actual distance is shorter than that of the theoretical once. From the average error in Table 4, the accuracy of the image capturing position can be calculated with the equation: An error on the x-axis of 0.38 cm is for the average translation distance of 32 cm so the error is 1.2%. The error on the y-axis of 0.076 cm is for the average translation distance of 43 cm so the error is 0.2%. Thus the positional accuracy for the x-axis becomes 98.9% while the positional accuracy for the y-axis becomes 99.8%. The average capturing position error for each cycle is measured to ensure that errors do not accumulate during manipulator operation. Fluctuations error for the three tested treatments of the stepper motor controller step period are shown in Figures 10 and 11. The slower speeds of the stepper motor rotation, the back EMF in the winding tend to decrease causes the current to increase in the load, resulting in missed steps. Therefore the positioning error become greater at the step period of 0.004 s compared to the step period of 0.002 s.  Figures 10 and 11 it can be seen that the position error for the step period treatment of 0.002 seconds produces the smallest error fluctuations and it seems that there is no accumulation of errors on both the x and y axes for 14 cycles of plant capturing images. Positional accuracy of agricultural robots is generally lower than industrial robots but is still able to provide good performance. Jiang et al. (2020), used an artificial neural network algorithm to increase the positioning accuracy of industrial robots from 0.8497 mm to 0.0490 mm. Yu et al. (2020), conducted a study on a strawberry harvesting robot with an average position error of 2 mm and a maximum of 4 mm, and they said this error is still acceptable according to the positional accuracy of the fruit harvesting end-effector. Gong et al., (2022), conducted research on fruit harvesting robots with a position accuracy of 8.21 mm but still able to harvest with a success of 73.04%.

System Stability
From Figure 10 it can be seen that the translational movement on the x-axis is quite stable around the set point with an error ranging from -0.1 cm to +0.9 cm for the stepper motor step period of 0.002 seconds. Meanwhile, the stepper motor step period of 0.004 seconds has stability around the set point with an error ranging from -0.25 cm to 2.4 cm. This stability is lower than the stepper motor step period of 0.002 seconds. From Figure 11 it can be seen that the translational movement on the y-axis is also quite stable around the set point with an error ranging from -0.065 cm to +0.15 cm for the stepper motor step period of 0.002 seconds. Meanwhile, the stepper motor step period of 0.004 seconds has stability around the set point with an error ranging from 0.015 cm to 0.40 cm. This stability is lower than the stepper motor step period of 0.002 seconds.

System Response
The system response is measured using a linear potentiometer starting when the motor shaft rotation command is executed with the results as shown in Figure 12. From Figure 12 it can be seen that the stepper motor response to the shaft rotation command is less than 1 ms.

Electric Power Consumption
The electric power consumption of the manipulator is calculated using the formula: The power consumption consists of: Stepper motor adapter: 12 V x 5A = 60 watts. The A4988 stepper motor driver includes load power = 12 V x 2 A x 2 units = 48 watts and power for the logic system = 5.5 V x 0.00002A x 6 lines = 0.00066 watt. Stepper motor power: 12 V x 1.7 A x 2 phase x 2 units = 81.6 watts. Raspberry pi 3 microcontroller power = 5V x 2.5 A = 12.5 Watts. Raspberry pi 3 power adapter = 5V x 3A = 15 watts. Power Xiaomi Xiaovv Camera = 5V x 0.16 A = 0.8 watts. So that the total consumption of electric power was about 218 watts. One cycle of the plant capturing images takes time: 0.074 hours, so energy consumption = 16.132 watt-hours/ cycle.
The average translational motion speed of the camera is calculated based on the traveling distance and traveling time of the camera. The calculation results show that the highest average speed of about 8.96 cm/second is obtained from the step period of 0.002 seconds as shown in Table 5.
From Table 5 it can be seen that the highest translation speed of 8.96 cm/sec was obtained for the 0.002 second step period treatment. Therefore, the 0.002 second treatment was chosen in this study. (Chancharoen et al., 2019) designed a robotic manipulator with a V-slot with a positioning accuracy of 0.45 mm. the robotic (8) manipulator was tested for repeatability at two translational speeds, namely 20 cm/ minute and 240 cm/minute. they concluded that the higher the translation speed, the lower the accuracy of the repeatability. Table 5. Average of translational speed of camera

Manipulator Performance Test
Performance tests were carried out inside the greenhouse to capture images of hydroponic plants using the best value of stepper motor step period according to the results of functional testing, namely 0.002 seconds. The manipulator was turned on continuously during 7 days with two image capturing cycles per day, namely at 7 o'clock am and 17 o'clock pm. In every cycle, capturing images are carried out from 15 different positions as described in the methodology. The time required for one cycle capturing images are 4 minutes 24 seconds. The capturing images for one cycles as an example is then arranged based on the capturing position as shown in Figure 13.  Figure 13 it can be seen that the camera was recorded at least 6 plants for each position, however, because the total number of plants in the hydroponic tub is only 50 clumps, there are overlapping images between adjacent positions, so further processing or rearrangement of the image capturing position is necessary to obtain different plant images for each position. As written in the title of the article, this research focuses on designing a manipulator that is capable of recording images at a determined position and time automatically, without further image processing. Coincidentally the camera used is a convex camera. Of course, the plant images obtained from this study need to be further processed to be used in the digital imagebased analysis.

CONCLUSIONS
The design of a Cartesian-type robotic manipulator with two Cartesian axes has been successfully created for automatic capturing and storing of plant images in greenhouses. The manipulator best performance for the three tested step period treatments of the stepper motor controller was the treatment with a step period of 0.002 seconds. The average pulley slip is 0.57 %, the average position error is 0.380 cm on the x-axis and 0.076 cm on the y-axis, and the average translation speed is 8.96 cm/ sec. Thus, the Cartesian-type robotic manipulator that has been designed is capable of capturing and storing hydroponic plant images automatically and continuously for 7 days of experiment with two capturing cycles per day. The average time per cycle is 4 minutes 24 seconds or about 0.074 hour.