Is there a Camera Sensor Algorithm that detects "Significant Deviation" ??
Christof Eb.
Posts: 1,234
Hi all,
my actual project knitting machine should work unattended for hours. Unfortunately it deals with soft material (yarn and fabric) which has variations in behaviour. So a method of handling might work very often but fail eventually. Also problems can occur at different places.
So I am dreaming, if there might be a camera sensor (or better algorithm), which can be triggered at certain points in time in a repeating process and will detect differences in pictures. It should learn somehow, what is a normal deviation and what is a non-tolerable deviation and in this case stop the machine.
Perhaps someone can give me some hints?
Thanks Christof
Comments
Well, if it's due to weak yarn then maybe add a tensioner early in the feeder so that the yarn breaks before it is used.
maybe some hints in these:
I would think you'd start around here
Canny Edge Detection
some of these may be too pattern oriented, but may come in more useful detecting quality of the knit
GLCM
GLCM
this might serve well, too
HOG
In your knitting scenario, I expect, you'd take a picture at the consistent points in the process to compare a picture of each succesive image of that same point, and there fore have the variation constrained to small portions of the whole image. I wonder if you need to get that clever at all. but, the HOG link is standing out to me, even for my own projects, with these features:
There are several key properties that make HOG well-suited for feature extraction in medical imaging:
Insensitive to geometric and photometric transformations — Medical images can vary significantly in terms of scale, rotation, illumination etc. HOG is invariant to these transformations, allowing robust feature extraction.
Captures edge/shape information — The histogram of gradients captures important shape and edge information in an image which can be useful for identifying anatomical structures.
Unsupervised feature extraction — HOG does not require any labeled data to extract features. This is important for medical imaging where labeled data is often scarce.
Noise tolerant — Due to the coarse regional sampling and normalization, HOG can handle noise in images quite well. Medical images frequently contain noise.
Simple and efficient to compute — The HOG algorithm is relatively simple to implement and efficient to compute making it very practical.
If you go the camera way, why not take a look at Pixy2 and OpenMV cameras. At least the newer OpenMV camera models are able to also run machine learning models, although your use case might not need ML.
Thank you for your inputs! Very much appreciated! Plenty to read, @refaQtor . @banjo I had had a very short look at these two, but I will look more closely. Sometimes I don't know the right words/terms to see or google.
@evanh The variation of the breaking load of the yarn is not the problem. Breaking of yarn was only a secondary failure. The minimum of the variance here is quite high; it is really hard to break this yarn by hand. Variation of bending stiffness and of friction are making more trouble. They lead to variations of the position of the slings. Which in turn can add up, that the needle does not find the sling.
But your input is good: The yarn should be the weakest point, as it can be easily replaced and also yarn breakage can be relatively easily detected.
Cool.
Any chance that latent heat, after a short radiation, might differ with the different makeup of the yarn? I'm assuming the stiffness and friction is due to variations in material makeup of the yarn.
There's also the possibility of improving the behaviour of the yarn by adjusting treatment. Rasing the ambient knitting temperature and adding moisture for example.
I don't understand this sentence. What do you mean with "short radiation"?
(The last failure which I don't understand at all, occurred around midnight but it was full moon then. Some knitters will be convinced, that the phase of the moon has influence....)
Yes temperature and moisture are factors to consider. Unfortunately my machine is partly built of wood, so some deflection might come from that side also. I have now turned off the automatic lowering of room temperature in the night. - All in all I hope that the process can be made sufficiently resilient to allow "usual" variations.
As in some directed heating, a radiant element, at the yarn as it feeds. The idea here is an IR sensor can provide a fast response simple amplitude of the fading temperature after having been heated. This may well vary with differing stiffness/texture of the yarn. It's a slightly wild idea.
Oh, I had seen another topic about that, it hadn't been something I was reading ... two of them:
https://forums.parallax.com/discussion/174934/p2-controls-circular-knitting-machine-for-2-color-jacquard-pattern/p1
https://forums.parallax.com/discussion/176024/super-liesl-a-p2-controlled-cnc-linear-knitting-loom/p1
I yesterday read about "Histogram of Oriented Gradients". Yes, this seems to be promising here, because in a picture the "deviations" to look for are deviations of direction of the yarn.
It's a pity that I don't have a good camera interface for P2. The HOG method should be applicable to be used with P2, because it reduces the amount of data very early in the processing. It's enough to hold 3 lines in RAM at a time. And I could spare a cog for it. Plenty of time to analyse the picture here. Unfortunately I have used the same pins now to interface the step motors. On the other hand, a HOG camera seems to be a bigger project in itself....
OpenMv seems perhaps to be a possibility to get this running faster. I have an ESP32Cam, but it is not good and it suffers badly from lack of free pins.
Funny, internet seems to become swamped by articles created by AI. So many resources about HOG just reiterate the same content. For example they all refer to 9 direction bins, without telling, why they don't use 8.
I have not yet found an article about reaction to a significant but unknown deviation. Must be a common problem: "Hey there must something wrong here."
there need only be 2 pins for UART between the ESP32 and the P2. cook up a simple message pattern between the two.... was the strategy I had been planning with my ESP32CAM boards (when I get to them)
you've gotten deeper already than me, but I think about it as a 3x3 grid - 8 directions around the edges, and 1 of the no-gradient case in the center. don't take that too literally, it is just how I'm holding these first thoughts in my head that might jog something in yours as you understand/apply it.
also notice, there is another version of the ESP32 setup for camera the WROVER that has an exposed RxTx https://www.amazon.com/FREENOVE-ESP32-WROVER-Compatible-Wireless-Detailed/dp/B0CJJHXD1W/
Yep, ordered such bigger WROVER board with camera and also with SD-card holder and USB-Serial. They seem to have GPIO32+33 free. And the message pattern shall be a Forth system....
This shows a good pattern:
