Broken tool detection with machine vision
ManAtWork
Posts: 2,253
My new automatic tool changer means I can let the machine run unattended with longer jobs and multiple tools. But this also means if something goes wrong in the middle it can get more expensive. Sometimes the cutter breaks which is no big deal if you notice it. You stop the machine, change the cutter and continue.
But if unattended the machine continues to run but without removing material. So let's assume we want to tap pre-drilled holes. If the drill bit breaks the machine changes the tool to the tap drill and ruins it by ramming it into the solid material where the pre-drilled holes should be but aren't. 
The usual fix to this is to measure the tool length with a touch probe or an optical laser barrier, once after the tool has been picked up from the tool changer and once again before it is stored back. But this takes time as the machine has to move to the location of the probe and then has to slowly advance the Z axis until the tool tip hits the probe or laser beam.
Inspired by the latest "AI & Vision with the cyber:bot and MakeCode" tutorial a better idea would be to do this with a camera and simply compare the "before" and "after" pictures. They could look something like this:
The point of interest is the distance from the collet to the lowest point of the cutter tip.
This point could be an a different side after the cutting process due to a different rotation of the tool.
But the overall length should stay the same within some small tolerance. If the length differs considerably then there's something broken.
I did some experiments with machine vision a long time ago. At that time this was usually be done with a PC, an industrial camera and a frame grabber card and you could easily spend multiple $1000s on it. Today, there a small gadgets like the HuskyLens for less than $100. But I don't know if this paticular thing is suitable. It looks like a "toy" for students with a very limited set of features like face or color recognition.
Does anybody here have experience with machine vision and can recomment something that can easily be programmed and tuned?
Comments
Well, I have some experience with using the Edge Impulse machine learning platform, have written numerous tutorials for them, e.g. this project write-up.
Machine learning is however not the optimal solution for every use case, in some cases simpler machine vision algorithms might be better. I recommend you look at any of the OpenMV cams as they support both traditional machine vision as well as machine learning methods.
OpenMV has some new camera models coming soonish, here a presentation video.
Ah thanks. That's a good starting point. I also understand that AI and "machine learning" might not be the optimal solution. At least, training an AI only with several example pictures won't work. It will be required to define as exactly as possible what parameters to focus on and what can be ignored. Otherwise a small change in illumination or dirt/chips can cause false alarms. But that's true for all projects, not just AI. The better you define what you want the better the output and the less time is spent on fixing mistakes.
Just for fun I asked ChatGPT for its opinion on using any OpenMV camera in this scenario. While I cannot comment on its correctness, it might give some ideas.
Using an OpenMV Camera for CNC Tool Integrity Monitoring
Author: Thomas Vikström (via ChatGPT)
Date: 2025-10-17 17:08 (Europe/Helsinki)
Problem description
An automatic tool changer allows long, unattended CNC operations. However, if a cutter or drill breaks during machining, the machine may continue executing toolpaths, damaging the workpiece or even destroying subsequent tools. For example, if a pre-drill breaks, a following tap might crash into solid material instead of a hole.
Traditional solutions use a touch probe or laser barrier to verify tool length before and after machining. These methods work but are slow, requiring the spindle to move to a specific station and advance slowly along the Z-axis. The goal is to achieve the same verification using a camera that compares “before” and “after” images to detect tool loss or breakage.
Solution concept with OpenMV
An OpenMV camera can quickly capture a silhouette of the tool at a fixed inspection point. By measuring the tip position or visible tool length in pixels and comparing it to a learned reference, the system determines whether the tool is intact, chipped, or missing. This process is nearly instantaneous and requires only a quick move to a fixed inspection coordinate.
Implementation details
CHECK TOOLcommand and waits for an OK/FAIL signal.CNC integration example
Method A – USB serial
LEARN,23→ stores Tool 23’s reference lengthCHECK,23→ compares current to stored lengthOKorFAIL Δ=0.3mmMethod B – Digital I/O
Full OpenMV Python program
# Program: OpenMV tool-tip silhouette length check (learn+verify) for CNC tool integrity — 2025-10-17 16:57 — Thomas Vikström import sensor, image, time, pyb, ujson, math PIXELS_PER_MM = 12.0 THRESHOLD = 0.25 ROI = (60, 20, 200, 200) LED_BACKLIGHT = pyb.LED(1) OK_PIN = pyb.Pin('P7', pyb.Pin.OUT_PP) SER = pyb.USB_VCP() REF_PATH = '/tool_refs.json' def load_refs(): try: with open(REF_PATH, 'r') as f: return ujson.loads(f.read()) except: return {} def save_refs(refs): with open(REF_PATH, 'w') as f: f.write(ujson.dumps(refs)) def setup_camera(): sensor.reset() sensor.set_pixformat(sensor.GRAYSCALE) sensor.set_framesize(sensor.VGA) sensor.skip_frames(time=500) sensor.set_auto_gain(False) sensor.set_auto_whitebal(False) sensor.set_auto_exposure(False, exposure_us=1000) return time.clock() def measure_tip_y(img): r = img.copy(roi=ROI) r.binary([(0, 80)]) x0, y0, w, h = ROI for j in range(h-1, -1, -1): if r.get_statistics(roi=(0, j, w, 1)).l_mean() < 250: return y0 + j return None def mm_from_pixels(py): _, y0, _, _ = ROI return (py - y0) / PIXELS_PER_MM def capture_length_mm(): LED_BACKLIGHT.on() img = sensor.snapshot() LED_BACKLIGHT.off() py = measure_tip_y(img) if py is None: return None return mm_from_pixels(py) def reply(msg): try: SER.write((msg + '\n').encode()) except: pass def main(): clock = setup_camera() refs = load_refs() OK_PIN.high() reply('READY') buf = b'' while True: clock.tick() if SER.any(): buf += SER.read() if b'\n' in buf: line, _, buf = buf.partition(b'\n') try: cmd = line.decode().strip() except: continue parts = cmd.split(',') if len(parts) >= 2 and parts[0].upper() in ('LEARN', 'CHECK'): action, tool_id = parts[0].upper(), parts[1].strip() length = capture_length_mm() if length is None: OK_PIN.low() reply('FAIL no_silhouette') continue if action == 'LEARN': refs[tool_id] = round(length, 4) save_refs(refs) OK_PIN.high() reply('OK LEARN ' + tool_id + ' {:.3f}mm'.format(length)) else: if tool_id not in refs: OK_PIN.low() reply('FAIL unknown_tool') continue ref = refs[tool_id] delta = abs(length - ref) if delta <= THRESHOLD: OK_PIN.high() reply('OK ' + tool_id + ' Δ={:.3f}mm len={:.3f}mm'.format(delta, length)) else: OK_PIN.low() reply('FAIL ' + tool_id + ' Δ={:.3f}mm len={:.3f}mm ref={:.3f}mm'.format(delta, length, ref)) else: if cmd.upper() == 'PING': reply('PONG') main()Why not full before/after image comparison?
While OpenMV supports
image.difference()for image comparison, it’s sensitive to lighting and alignment differences. Using silhouette-based tip measurement is faster and more reliable, giving a direct numeric measurement of tool length.Possible extensions
Conclusion
An OpenMV camera provides a cost-effective, compact, and robust vision-based solution for tool breakage detection. It can prevent expensive crashes during unattended operation and runs far faster than mechanical probing. Once calibrated, it delivers consistent accuracy and integrates easily with standard CNC control systems.
Nice!
Although I intentionally asked the forum instead of ChatGPT directly to hopefully get some answers from real humans I have to admit that this is quite helpful. I don't know if I would trust the quoted code blindly but any flaws can be fixed easily if you have some basic technical understanding.
Calibration is not necessary because the actual length doesn't really matter, only the difference. OK, for the scale of the tolerance it matters, but a rough guess is enough.
BTW, I can take the picture inside the tool changer at the lowest position of the caroussel magazine. The tool stays there for a brief moment so no extra movement of the spindle is necessary and motion blurr is no issue.
Hi
For series production analysis of spindle motor current is used sometimes which will tell about a broken or blunt blade. Also vibration or noise analysis is done.
Both methods use relatively simple sensors.
I am not sure how usual the failure mode "broken shaft of tool" is. I would think it is not the typical problem.
Cheers Christof
@ManAtWork said:
Found my RT-1062, and after a few iterations with ChatGPT I have a program that prints to the terminal window when the object length changes. It needs more work to really make it work, this was just a quick test under poor lighting conditions.
The program averages the first 18 frames as a baseline to which it later compares.
# Measures tool-tip length each frame, learns a baseline, and flags CHANGE with delta — 2025-10-17 19:00 (Europe/Helsinki) — Thomas Vikström import sensor, image, time # ====================== USER SETTINGS ====================== PIXELS_PER_MM = 20.0 # calibrate once with a gauge pin/intact tool ROI = (60, 20, 200, 200) # x,y,w,h must straddle the tip (top above, bottom below) TOOL_IS_DARK = True # True=backlit (tool darker than background); False for front-lit ROW_WHITE_L = 250 # after binary, l_mean >= this ≈ white row ERODE_ITERS = 1 DILATE_ITERS = 0 USE_MANUAL_THR = True # start True; tune MANUAL_THR; switch to False to try Otsu MANUAL_THR = 95 # try 80..130 depending on your backlight AUTO_EXPOSURE_US = 31800 # exposure when autos OFF (use bright backlight) PRINT_EVERY_N = 1 # Change detection / smoothing LEARN_FRAMES = 18 # avg these to create baseline MA_WINDOW = 3 # moving average to stabilize CHANGE_THRESH_MM = 0.30 # trip alarm if |Δ| > this HYST_MM = 0.10 # hysteresis to clear alarm # =========================================================== # --- state --- baseline_mm = None learn_buf = [] ma_buf = [] alarm = False def setup_camera(): sensor.reset() sensor.set_pixformat(sensor.GRAYSCALE) sensor.set_framesize(sensor.VGA) sensor.skip_frames(time=600) # Lock auto* for repeatability try: sensor.set_auto_gain(False) sensor.set_auto_whitebal(False) sensor.set_auto_exposure(False, exposure_us=AUTO_EXPOSURE_US) except Exception: pass print("READY — fix camera, jog Z so the tip moves inside the ROI. Tune MANUAL_THR/exposure if needed.") def to_int_threshold(thr_obj): # Normalize whatever get_threshold() returns to int try: return int(thr_obj) except Exception: pass if isinstance(thr_obj, dict): for k in ("value","v","low","l_value","threshold"): if k in thr_obj: try: return int(thr_obj[k]) except: pass try: return int(next(iter(thr_obj.values()))) except: pass for m in ("value","val","low","l_value","get","threshold"): if hasattr(thr_obj, m): try: a = getattr(thr_obj, m) v = a() if callable(a) else a return int(v) except: pass if isinstance(thr_obj, (tuple, list)) and thr_obj: try: return int(thr_obj[0]) except: pass return 128 def pixels_to_mm(py): _, y0, _, _ = ROI return (py - y0) / PIXELS_PER_MM def moving_avg_push(x): ma_buf.append(x) if len(ma_buf) > MA_WINDOW: ma_buf.pop(0) return sum(ma_buf)/len(ma_buf) def learn_push(x): learn_buf.append(x) if len(learn_buf) > LEARN_FRAMES: learn_buf.pop(0) if len(learn_buf) == LEARN_FRAMES: return sum(learn_buf)/LEARN_FRAMES return None def detect_tip_y(img): # choose threshold if USE_MANUAL_THR: thr = int(MANUAL_THR) else: h = img.get_histogram(roi=ROI) thr = to_int_threshold(h.get_threshold()) # crop and binarize with polarity r = img.copy(roi=ROI) if TOOL_IS_DARK: r.binary([(0, thr)]) else: r.binary([(thr, 255)]) r.invert() # cleanup if ERODE_ITERS: r.erode(ERODE_ITERS) if DILATE_ITERS: r.dilate(DILATE_ITERS) # scan from bottom up for first non-white row x0, y0, w, hgt = ROI for j in range(hgt-1, -1, -1): if r.get_statistics(roi=(0, j, w, 1)).l_mean() < ROW_WHITE_L: return y0 + j, thr return None, thr def s3(v): """safe formatter for numbers that can be None""" return "{:.3f}".format(v) if (v is not None) else "--" def draw_overlay(img, tip_y, thr, length_mm, sm_len, delta, alarm_flag): # ROI and tip line img.draw_rectangle(ROI, color=255) if tip_y is not None: img.draw_line(ROI[0], tip_y, ROI[0]+ROI[2]-1, tip_y, color=255) # Text lines (no formatting with None!) y = ROI[1] + 4 lines = [ "thr={} pix/mm={:.2f}".format(thr, PIXELS_PER_MM), "len_raw={} mm".format(s3(length_mm)), "len_ma={} base={}".format(s3(sm_len), s3(baseline_mm)), "Δ={} mm {}".format(s3(delta), ("CHANGE!" if alarm_flag else "OK")), ] for s in lines: img.draw_string(ROI[0]+4, y, s, mono_space=False) y += 12 def main(): global baseline_mm, alarm setup_camera() frame = 0 while True: img = sensor.snapshot() tip_y, thr = detect_tip_y(img) length_mm = pixels_to_mm(tip_y) if tip_y is not None else None sm_len = None delta = None if length_mm is not None: sm_len = moving_avg_push(length_mm) if baseline_mm is None: maybe = learn_push(sm_len) if maybe is not None: baseline_mm = round(maybe, 3) print("BASELINE {:.3f} mm (avg {} frames)".format(baseline_mm, LEARN_FRAMES)) else: delta = sm_len - baseline_mm # hysteresis alarm logic if not alarm and abs(delta) > CHANGE_THRESH_MM: alarm = True elif alarm and abs(delta) < (CHANGE_THRESH_MM - HYST_MM): alarm = False draw_overlay(img, tip_y, thr, length_mm, sm_len, delta, alarm) frame += 1 if frame % PRINT_EVERY_N == 0: if length_mm is None: print("NO_TIP thr={} (tune ROI/threshold/exposure or TOOL_IS_DARK)".format(thr)) elif baseline_mm is None: print("LEARN len_ma={} ({}/{})".format(s3(sm_len), len(learn_buf), LEARN_FRAMES)) else: print("{} tip_y={} len={} len_ma={} base={} Δ={} mm".format( "CHANGE!" if alarm else "OK ", tip_y, s3(length_mm), s3(sm_len), s3(baseline_mm), s3(delta) )) time.sleep_ms(5) main()