Broken tool detection with machine vision

ManAtWork · 2025-10-17 12:03

My new automatic tool changer means I can let the machine run unattended with longer jobs and multiple tools. But this also means if something goes wrong in the middle it can get more expensive. Sometimes the cutter breaks which is no big deal if you notice it. You stop the machine, change the cutter and continue.

But if unattended the machine continues to run but without removing material. So let's assume we want to tap pre-drilled holes. If the drill bit breaks the machine changes the tool to the tap drill and ruins it by ramming it into the solid material where the pre-drilled holes should be but aren't.

The usual fix to this is to measure the tool length with a touch probe or an optical laser barrier, once after the tool has been picked up from the tool changer and once again before it is stored back. But this takes time as the machine has to move to the location of the probe and then has to slowly advance the Z axis until the tool tip hits the probe or laser beam.

Inspired by the latest "AI & Vision with the cyber:bot and MakeCode" tutorial a better idea would be to do this with a camera and simply compare the "before" and "after" pictures. They could look something like this:

The point of interest is the distance from the collet to the lowest point of the cutter tip.

This point could be an a different side after the cutting process due to a different rotation of the tool.

But the overall length should stay the same within some small tolerance. If the length differs considerably then there's something broken.

I did some experiments with machine vision a long time ago. At that time this was usually be done with a PC, an industrial camera and a frame grabber card and you could easily spend multiple $1000s on it. Today, there a small gadgets like the HuskyLens for less than $100. But I don't know if this paticular thing is suitable. It looks like a "toy" for students with a very limited set of features like face or color recognition.

Does anybody here have experience with machine vision and can recomment something that can easily be programmed and tuned?

banjo · 2025-10-17 13:08

Well, I have some experience with using the Edge Impulse machine learning platform, have written numerous tutorials for them, e.g. this project write-up.
Machine learning is however not the optimal solution for every use case, in some cases simpler machine vision algorithms might be better. I recommend you look at any of the OpenMV cams as they support both traditional machine vision as well as machine learning methods.
OpenMV has some new camera models coming soonish, here a presentation video.

ManAtWork · 2025-10-17 13:50

Ah thanks. That's a good starting point. I also understand that AI and "machine learning" might not be the optimal solution. At least, training an AI only with several example pictures won't work. It will be required to define as exactly as possible what parameters to focus on and what can be ignored. Otherwise a small change in illumination or dirt/chips can cause false alarms. But that's true for all projects, not just AI. The better you define what you want the better the output and the less time is spent on fixing mistakes.

banjo · 2025-10-17 14:25

Just for fun I asked ChatGPT for its opinion on using any OpenMV camera in this scenario. While I cannot comment on its correctness, it might give some ideas.

Using an OpenMV Camera for CNC Tool Integrity Monitoring

Author: Thomas Vikström (via ChatGPT)
Date: 2025-10-17 17:08 (Europe/Helsinki)

Problem description

An automatic tool changer allows long, unattended CNC operations. However, if a cutter or drill breaks during machining, the machine may continue executing toolpaths, damaging the workpiece or even destroying subsequent tools. For example, if a pre-drill breaks, a following tap might crash into solid material instead of a hole.

Traditional solutions use a touch probe or laser barrier to verify tool length before and after machining. These methods work but are slow, requiring the spindle to move to a specific station and advance slowly along the Z-axis. The goal is to achieve the same verification using a camera that compares “before” and “after” images to detect tool loss or breakage.

Solution concept with OpenMV

An OpenMV camera can quickly capture a silhouette of the tool at a fixed inspection point. By measuring the tip position or visible tool length in pixels and comparing it to a learned reference, the system determines whether the tool is intact, chipped, or missing. This process is nearly instantaneous and requires only a quick move to a fixed inspection coordinate.

Implementation details

Fixed view and lighting: Mount the camera at a stable inspection station near the spindle or tool changer. Use a strong backlight (white or infrared) to create a clear dark silhouette of the tool.
Measurement principle: Define a Region of Interest (ROI) where the tool appears. Binarize the image and find the lowest dark pixel (tool tip). Convert pixels to millimeters using a calibration constant.
Calibration: Use a reference gauge pin or intact cutter to calibrate pixels per millimeter.
Thresholding: A small deviation (e.g., 0.2–0.3 mm) indicates breakage. For taps, even 0.1 mm can be enough.
Communication: Connect the OpenMV to the CNC via USB serial or digital I/O. The CNC sends a CHECK TOOL command and waits for an OK/FAIL signal.
Environmental protection: Use an air blast or protective shroud for the lens. Fast shutter speeds and strong backlight minimize motion blur.

CNC integration example

Method A – USB serial

CNC sends: LEARN,23 → stores Tool 23’s reference length
CNC sends: CHECK,23 → compares current to stored length
OpenMV replies: OK or FAIL Δ=0.3mm

Method B – Digital I/O

CNC output triggers the OpenMV measurement
OpenMV sets an output line HIGH (OK) or LOW (FAIL)
CNC halts program execution on FAIL

Full OpenMV Python program

# Program: OpenMV tool-tip silhouette length check (learn+verify) for CNC tool integrity — 2025-10-17 16:57 — Thomas Vikström
import sensor, image, time, pyb, ujson, math

PIXELS_PER_MM   = 12.0
THRESHOLD       = 0.25
ROI             = (60, 20, 200, 200)
LED_BACKLIGHT   = pyb.LED(1)
OK_PIN          = pyb.Pin('P7', pyb.Pin.OUT_PP)
SER             = pyb.USB_VCP()
REF_PATH        = '/tool_refs.json'

def load_refs():
    try:
        with open(REF_PATH, 'r') as f:
            return ujson.loads(f.read())
    except:
        return {}

def save_refs(refs):
    with open(REF_PATH, 'w') as f:
        f.write(ujson.dumps(refs))

def setup_camera():
    sensor.reset()
    sensor.set_pixformat(sensor.GRAYSCALE)
    sensor.set_framesize(sensor.VGA)
    sensor.skip_frames(time=500)
    sensor.set_auto_gain(False)
    sensor.set_auto_whitebal(False)
    sensor.set_auto_exposure(False, exposure_us=1000)
    return time.clock()

def measure_tip_y(img):
    r = img.copy(roi=ROI)
    r.binary([(0, 80)])
    x0, y0, w, h = ROI
    for j in range(h-1, -1, -1):
        if r.get_statistics(roi=(0, j, w, 1)).l_mean() < 250:
            return y0 + j
    return None

def mm_from_pixels(py):
    _, y0, _, _ = ROI
    return (py - y0) / PIXELS_PER_MM

def capture_length_mm():
    LED_BACKLIGHT.on()
    img = sensor.snapshot()
    LED_BACKLIGHT.off()
    py = measure_tip_y(img)
    if py is None:
        return None
    return mm_from_pixels(py)

def reply(msg):
    try:
        SER.write((msg + '\n').encode())
    except:
        pass

def main():
    clock = setup_camera()
    refs = load_refs()
    OK_PIN.high()
    reply('READY')
    buf = b''
    while True:
        clock.tick()
        if SER.any():
            buf += SER.read()
            if b'\n' in buf:
                line, _, buf = buf.partition(b'\n')
                try:
                    cmd = line.decode().strip()
                except:
                    continue
                parts = cmd.split(',')
                if len(parts) >= 2 and parts[0].upper() in ('LEARN', 'CHECK'):
                    action, tool_id = parts[0].upper(), parts[1].strip()
                    length = capture_length_mm()
                    if length is None:
                        OK_PIN.low()
                        reply('FAIL no_silhouette')
                        continue
                    if action == 'LEARN':
                        refs[tool_id] = round(length, 4)
                        save_refs(refs)
                        OK_PIN.high()
                        reply('OK LEARN ' + tool_id + ' {:.3f}mm'.format(length))
                    else:
                        if tool_id not in refs:
                            OK_PIN.low()
                            reply('FAIL unknown_tool')
                            continue
                        ref = refs[tool_id]
                        delta = abs(length - ref)
                        if delta <= THRESHOLD:
                            OK_PIN.high()
                            reply('OK ' + tool_id + ' Δ={:.3f}mm len={:.3f}mm'.format(delta, length))
                        else:
                            OK_PIN.low()
                            reply('FAIL ' + tool_id + ' Δ={:.3f}mm len={:.3f}mm ref={:.3f}mm'.format(delta, length, ref))
                else:
                    if cmd.upper() == 'PING':
                        reply('PONG')

main()

Why not full before/after image comparison?

While OpenMV supports image.difference() for image comparison, it’s sensitive to lighting and alignment differences. Using silhouette-based tip measurement is faster and more reliable, giving a direct numeric measurement of tool length.

Possible extensions

Add chipped-edge detection by measuring width at multiple heights
Detect bent tools by comparing two rotated frames
Verify workpiece features (holes) after drilling
Use multiple ROIs for tools of different lengths

Conclusion

An OpenMV camera provides a cost-effective, compact, and robust vision-based solution for tool breakage detection. It can prevent expensive crashes during unattended operation and runs far faster than mechanical probing. Once calibrated, it delivers consistent accuracy and integrates easily with standard CNC control systems.

ManAtWork · 2025-10-17 15:08

Nice! Although I intentionally asked the forum instead of ChatGPT directly to hopefully get some answers from real humans I have to admit that this is quite helpful. I don't know if I would trust the quoted code blindly but any flaws can be fixed easily if you have some basic technical understanding.

Calibration is not necessary because the actual length doesn't really matter, only the difference. OK, for the scale of the tolerance it matters, but a rough guess is enough.

BTW, I can take the picture inside the tool changer at the lowest position of the caroussel magazine. The tool stays there for a brief moment so no extra movement of the spindle is necessary and motion blurr is no issue.