This blog covers various data analysis applicable to IoT and Big Data.
A design of experiments (DOE) will reveal the factors that contribute to the performance of a product or process. The result is a ranked listing of factors that influence the product/process performance. A DOE can resolve the importance of one or more factors when interactions exist between them. Whenever possible, conducting a partial / fractional factorial DOE rather than a full factorial DOE will save time.
The frequency content of what is being measured will drive the choice of sensor used for measurement, and the sample rate required. You don't need an exact value, but you should have a rough order of magnitude idea of what the highest frequency is (0.5 Hz, 1 Hz, 10 Hz, etc.). If you don't get this correct, and you select a sample rate that is too low, you will completely miss measuring minimum and maximum extreme values, compromising all subsequent data analysis. The sensor response time must also also be checked to insure it is sufficiently shorter than the period for the maximum frequency of the factor being measured.
The best way to measure the frequency content is to use data acquisition equipment along with a sensor with a very low response time, and measure the factor under a wide scope of conditions while sampling at no less than 10 kHz. Then perform a fast fourier transform (FFT) on the data collected, and then plot the frequency versus peak amplitude results. The fundamental frequencies should be apparent.
The Fast Fourier Transform (FFT) provides you with the frequency content of time domain signal(s) in the frequency domain. Any signal can be thought of as a sum of a series of simple sinusoids of different frequencies, amplitudes, and phases. Fourier analysis or spectrum analysis will deconstruct a signal into its individual sine wave components. The result is the vibration amplitude as a function of frequency. The highest frequency that can be analyzed from a signal is limited by the one half the sample rate of the signal (Nyquist frequency). The duration of the signal analyzed will limit the frequency resolution. For a sample rate of Fs in Hz, and a recording duration of T in seconds, The frequency resolution in Hz will be Fs / T, and the maximum frequency in Hz that can be analyzed will be Fs/2.
The FFT signal analysis can be performed using freely available Python libraries, or I can perform the analysis for you using a professional application from NI called DIAdem.
Characterized by a sudden change in acceleration over a short period of time.
In general, the sample rate for shock testing should be > 10 kHz. A Bessel or Butterworth filter is usually a good choice. Digital low pas filtering a shock event will not influence the total energy or damage potential of the signal.
Peak pseudo velocity (PV) is the primary method for analyzing and quantifying a shock event. It correlates better to damage than peak acceleration. Calculation of pseudo velocity will achieve filtering of the data as a result of the calculation. In general, a pseudo velocity less than 100 inches per second (ips) is unlikley to result in damage (very unlikely at < 50 ips). Sample rate has a minimal effect on pseudo velocity. Peak pseudo velocity approximates the total velocity change when there is little damping.
Motion (random or oscillating) in both directions about a point where the object is at rest (equilibrium). Examples include rotating equipment (motor or pump rotating shaft, bearings, etc.)
The required sample rate required is typically > 5 kHz. A Butterworth filter is typically employed.
The common application is to monitor changes in the vibration of rotating equipment with shafts and bearings such as motors, pumps, etc. These changes are usually caused by worn components such as bearings, u-joints, slip splines, etc., although they may also be caused by installation errors, out of balance components, or even design errors where something goes into resonance under normal operation.
ISO standard 10816 defines vibration severity for machines. It has recommendations on the maximum velocity by machine type.
Measuring the standard deviation of the acceleration in the time domain is the easiest way to obtain a metric that can be used to identify vibration changes in a system over time. It is similar to measuring the RMS (root mean square) of the acceleration in the time domain, but it is not influenced by DC bias like the RMS is. Both the standard deviation and the RMS measure the energy content of the vibration profile (distructive capability). When a DC bias is present and the standard deviation is used as a metric, then the recording duration needs to be sufficient to provide a good sample of the vibration for analysis.
The displacement RMS is useful for identifying an imbalance in a rotating system.
Tracking the frequency of peak pseudo velocity will indicate a change in the structure of the system being monitored if the frequency increases over time.
Cargo on a truck will typically be exposed to +/- 5 g in the vertical direction, along with a lot of lower amplitude random vibration. Cargo on a jet aircraft may see +/- 20 g of vibration. Rail cargo will see +/- 2.5 g (see MIL-STD-810). A seat in a commercial aircraft will see +/- 0.6 g from 1 Hz to 1 kHz.
In the case of vibration data, it is more useful to perform a frequency analysis (FFT) and root mean square (RMS) calculation on the data. The FFT will identify the dominant frequency of vibration, and the RMS tells you about the energy content of the vibration profile. Note that the FFT works well when a finite number of dominant frequency compoents exist. When this is not the case (i.e. the vibration frequency changes with time), then a power spectral density (PSD) analysis (spectrogram) needs to be performed to characterize the random vibration.
Cases where slow changes in position or velocity occur. Examples include gravity, sensor orientation (game controller), lateral acceleration from a vehicle making a turn, vehicular braking and acceleration, bridges, and robotic motion.
A 1 kHz sample rate is typically adequate.
If the motion is very low in frequency (to 0 Hz) then a DC response accelerometer may be required.
Correlation is a mathematical operation to compare two signals and provide a result that represents how similar the two signals are. The correlation is calculated by multiplying two signals together, and then summing the product. The result is a single number that indicates the similarity between the signals. Freely available Python libraries are available to perform this calculation, or I can perform the analysis for you using a professional application from NI called DIAdem.
A Trend Analysis will decompose the data using a moving / rolling average to identify over time if the trend is an uptrend, downtrend, or stationary. The stationary situation can be determined by looking for changes in results for mean, variance, and covariance at regular intervals. Use linear regression to generate a line fitted to the data and then use the slope of that line to measure the uptrand or downtrend. A Mann-Kendall score can also be used to to statistically assess if there is a monotonic upward or downward trend over time. Watch for seasonality and other effects that appear as a cyclical pattern when the data is plotted. Freely available Python libraries are available to perform this calculation, or I can perform the analysis for you using a professional application from NI called DIAdem.
Machine Learning (ML) is typically performed on raw data. ML inference can be conducted with a microcontroller (MCU) to detect complex motion, recognize sounds, classify images, or find anomalies in sensor data.
| Application | Sensor | Sample Rate | Max Event Frequency | Comments |
|---|---|---|---|---|
| Shock / Impact | Accelerometer | 10 kHz | 1 kHz | Bessel or Butterworth filter. Peak pseudo velocity (PV) correlates better to damage than peak acceleration. |
| Vibration | Accelerometer | 5 kHz | 500 Hz | Butterworth filter. Standard deviation is easiest metric for vibration changes over time. Displacement RMS works well for rotating shaft imbalance. FFT works well only when a finite number of dominant frequency components exist. Use PSD for random vibration signals and to compare PSDs between various environments.. |
| Motion | Accelerometer | 1 kHz | 100 Hz | Cases where slow changes in position or velocity occur. Standard Deviation of acceleraiton is the most robust method to get the AC coupled RMS value and avoid DC bias. DC response accelerometer required for very low frequency. |
| IoT | Temperature / Humidity | 1 kHz (1) | 100 Hz (1) |
(1) Based on the typical sensor capability, not the application frequency content.
Do you need help with your project? Send me an email requesting a free phone / web share consultation.
Copyright © 2018-2023 Mechatronic Solutions LLC, All Rights Reserved
.png "click to see larger image")