Soil is a compound mixture of organic matter, minerals, gases, liquids, and even living organisms. In addition to supporting the plant life we need for crops, soil also functions as a means of storing, transporting, and purifying water; it helps to modify the atmosphere we all depend on, and even serves as habitat for organisms large and small. Sustainable soil management is as critical for future food production as it is for life on Earth.
Soil health is elemental to sustainable land management and is an important consideration for farms of all sizes. Anything from erosion to contamination, loss of biodiversity, soil compaction, and everything in between, can be detrimental to crop production and the viability of the farm itself.
Numerous studies and technologies are in development for analyzing and managing soil health and Avantes is at the forefront of research and technology development, protecting our future food supply.
Historically, measurements of soil moisture have employed a device called a tensiometer which uses a hollow tube with a porous reservoir of water on top and a gauge. The tube is inserted into the soil and the water is drawn into the soil from the cup, creating a vacuum, until it reaches equilibrium. The gauge allows the user to gather a reading based on that vacuum that correlates to water carrying potential in the soil matrix. This data allows farmers to determine the need for irrigation, but there are drawbacks. The tensiometer is slow, requiring a period of time for water to reach equilibrium which limits the scale of use.
As early as 1978, researchers have been interested in the effects of soil moisture on, first the visible spectrum, but as spectroscopy techniques improves, eventually extending into the near- and mid-infrared.  In 2014, researchers in Hungary were working to calibrate spectral data intended to develop algorithms that would allow fast, field-scale measurements of soil moisture content .
The development of algorithms that will one day allow for field-scale deployment of spectroscopy-based moisture measurements begins first in the laboratory with data collection. Soil samples collected from orchards with varying soil characteristics from around the region were first kiln dried in the lab to a consistent aridity. Water was then reintroduced slowly, with spectra collected each 2.5ml until the samples were fully saturated. They identified the wavelengths 1450-1460 nm and 1920-1930 nm as the most sensitive for quantifying soil moisture. The AvaSpec-NIR256-2.5-HSC-EVO and AvaSpec-NIR256-1.7-EVO allow the range and sensitivity for this type of application in the laboratory, but future developments in compact NIR spectrometers, , could one day lead to replacing the tensiometer with handheld field instruments for rapid assessment of moisture content and integration with irrigation systems and to calibrate and authenticate (ground truth) airborne hyperspectral imaging technology. 
Soil samples for classification
Soil is not a homogenous mixture, differences in minerals, organic matter, and particle size, as just a few examples, can alter any number of soil characteristics.  Soil type will determine everything from irrigation schedules to types of crops likely to prosper. The US Department of Agriculture recognizes 12 types of soil with sand at one end of the spectrum and clay at the other.  Each type has predictable characteristics, including color. In fact, the most common method for determining soil type relies on subjective (and fallible) personal experience to compare against a specially designed color chart from the Munsell Corporation. 
Other methods for soil classification require chemical processes that might have adverse effects on data interpretation.  They are also time-consuming and require more advanced technical skills to perform. Optical spectral sampling, on the other hand, requires little or no sample preparation and no harsh chemicals, and several parameters can be analyzed from the same spectral data.
Researchers in Hamadan, Iran investigated UV/VIS/NIR spectroscopy to analyze a number of soil parameters including color, pH, electrical conductivity, moisture content, available organic carbon, total nitrogen, and exchangeable cations, as well as identifying minerals such as iron, titanium oxides, calcium, magnesium, potassium, and sodium, just to name a few.  Samples were collected, randomized, and kiln-dried then ground and sieved before being divided into calibration and validation sets. A full battery of chemometric data was collected from the set of validation samples.
An average of more than 24 broadband spectral scans were collected per sample in the calibration set using the spectrometer to cover the UV-VIS range from 200-1100 nm at 1 nm resolution. For the 1000-2400 nm range, the allows measurements in the NIR. Spectral analysis performed on the calibration set allowed a series of operations for statistical analysis based on principal component analysis (PCA) and least partial squares regression analysis (LPS-R) to relate spectral reflectance measurements to observable soil properties. Based on their results, these scientists determined that they could sufficiently validate soil classification using a fraction of available variables.
Effects of Soil Compaction
Bulk Density and Soil Compression
Soil compaction is the compression of spaces between soil particles that would normally hold air or water. It leads to poor root development, oxygen deficiency, and other deficiencies. These factors ultimately reduce crop quality and yields. Compaction can be man-made or the result of natural processes, but is a serious environmental and agricultural problem.  Common causes include extensive use of heavy machinery, repeated plowing to uniform depth, and use by large animal populations. Sustainable agricultural systems must work to manage soil compaction beginning with the measurement of key parameters associated with soil compaction.
Bulk soil density (BD), representing the ratio of volumetric moisture (Θv) content to gravimetric moisture (ω) [expressed as BD = Θv/ω], is positively correlated with soil compaction, i.e. the higher the soil density, the greater the degree of compaction.  Current technology for measuring soil compaction is the penetrometer, an instrument that, at its heart, hasn’t advanced a great deal beyond pointy stick. The penetrometer is meant to mimic a growing plant root and involves a pressure gauge atop a graduated driving shaft tipped with a wider 30-degree stainless steel cone.
Research published in 2018 in the journal Computers and Electronics in Agriculture unveiled a prototype measuring system that combines a traditional penetrometer with fiber-coupled NIRS sensor.  The system operated in situ to combine measurement of soil penetration resistance, frequency domain reflectometry to analyze volumetric moisture content, and near-infrared diffuse reflectance for the measurement of gravimetric moisture content, to allow calculation of bulk density (BD) with one easy-to-manage probe system. 
This novel probe design combined a penetrometer for the measurement of soil penetration resistance, with a hollow shaft housing optical fibers coupled to a sapphire window in the shaft body at one end, and on the other, to the 20 watt halogen lamp for input, and to the AvaSpec-NIR256-2.5 spectrometer for NIR reflection data output and capture. Time domain reflectometry measurements quantify volumetric moisture content (Θv) achieved by integrating an electrode in the shaft of the probe in the form of a copper ring insulated from the probe body. Hundreds of measurements were used to train an artificial neural network (ANN) to model the fusion of data and produce encouraging results for the future development of bulk density solutions for rapid results in field applications. 
Avantes Meets the Challenge
NIR Spectroscopyi n the Laboratory
The need for affordable, efficient, and reliable data is not without challenges. Many available systems are not suitable for field deployment. Data collection can also be compromised because of operator errors and inconstancies. Much of the work of designing all-in-one spectroscopy-based solutions appears to focus on eliminating opportunities for user error or inconsistencies in deployment.  Working with an experienced partner in spectroscopy can help the system design engineer to discover solutions to any number of field deployment challenges. Avantes brings 25 years of experience working hand-in-hand with researchers and equipment manufacturers to design systems that meet and exceed measurement requirements. Test drive an Avantes instrument with our exclusive demo program and discover the Avantes advantage today.
- Al-Asadi, Raed A., and Abdul M. Mouazen. "A Prototype Measuring System of Soil Bulk Density with Combined Frequency Domain Reflectometry and Visible and near Infrared Spectroscopy." Computers and Electronics in Agriculture. Elsevier, 30 June 2018. Web. 19 Nov. 2019. https://www.sciencedirect.com/science/article/pii/S0168169917316277
- Han, Pengcheng, et al. “A Smartphone-Based Soil Color Sensor: For Soil Type Classification.” Computers and Electronics in Agriculture, vol. 123, 2016, pp. 232–241., doi:10.1016/j.compag.2016.02.024. https://www.sciencedirect.com/science/article/pii/S0168169916300618
- Monavar, Hosna Mohamdi. “34th International Conference on Food and Agricultural Engineering.” The Ires, Determination of Several Soil Properties Based on Ultra-Violet, Visible, and near-Infrared Reflectance Spectroscopy, 12 Mar. 2016, https://www.researchgate.net/publication/297322303_Determination_of_several_soil_properties_based_on_ultra-violet_visible_and_near-infrared_reflectance_spectroscopy “Soil.” Wikipedia, Wikimedia Foundation, en.wikipedia.org/wiki/Soil.
- “Soil.” Wikipedia, Wikimedia Foundation, https://en.wikipedia.org/wiki/Soil.
- USDA. “Natural Resources Conservation Service.” The Color of Soil | NRCS Soils, US Department of Agriculture, https://www.nrcs.usda.gov/wps/portal/nrcs/detail/soils/edu/?cid=nrcs142p2_054286