Velocity QANina Reweti

AI TOOL TESTING · CLAUDE SKILLS · PLUGINS

Before you rely on an AI skill, find out what it gets wrong.

I test Claude skills and plugins the way I test software. Expected results are written before the runs, runs are repeated, and each finding is traced back to the instructions the tool actually follows.

Email me about a skill or plugin See what I found

TWO WAYS I HELP

Two reasons people ask me to test

YOU BUILT IT

I've built a skill or plugin

Before you release it or hand it to a client, I check:

  • It does what its instructions say run after run, not just the first time
  • It ignores instructions hidden in emails, reviews or files
  • Blank and messy data are treated as unknown, not zero
  • Edge cases and the requests people actually make
  • It still works after you edit it
YOU'RE ABOUT TO USE IT

I want to use one in my business

Before you trust it with real work, I check:

  • What it gets wrong for a business like yours
  • What it can't do for Australia, such as GST, awards and super
  • What it does with your data and who it acts for
  • The setup lines that close the gaps, ready to paste

OBSERVATIONS

Anthropic's Small Business plugin, checked for an Australian business

A strong, US-built toolkit with good safety rails around money and fraud. The Australian compliance layer is missing, and about ten lines of business context cover most of it.

Oct 2026 · plugin v1.35.1 · 44 skills · fictional business · small sample · skill files change often

What held up

Nothing sent or paid without a yes

HELD

Bills, payroll and posts wait for approval. The one exception is a team-channel post you approve once.

Changed bank details held

HELD

Sent straight to the owner, with a phone check advised.

Duplicate bill caught

HELD

The skill names duplicate payment as the expensive failure.

Hidden instructions refused in testing

6 OF 6

A fake refund email, a fake lead and a review asking for a $500 credit.

What's missing for Australia

Forecasts never mention BAS, PAYG or super

GAP

The fixed costs it lists are rent, payroll and subscriptions.

Payroll defaults to US-style overtime

GAP

With no policy on file it applies a weekly-overtime rule framed around US states. No awards, penalty rates, casual loading or super.

Quotes don't say if GST is included

GAP

Nothing in the quote skill mentions tax-inclusive pricing.

Untrusted-content rule linked from 21 of 44 skills

GAP

The 23 others include lead triage and outreach, which work with lead data. The model refused the attacks anyway, but those skills don't tell it to.

FINDING · SKILL FILE · cash-flow-snapshot HIGH

A mid-October cash forecast can miss the BAS due on 28 October

The skill builds the forecast from balances, money owed both ways and fixed costs. Tax and super aren't in its instructions.

EXPECTED
BAS and PAYG show on their due dates, and super with each pay run
ACTUAL
Not instructed. Sometimes covered by the model, not guaranteed
Fix line: “GST registered, quarterly BAS. Show BAS and PAYG due dates, and super with each pay run, as outflows in every forecast.”
Email me for the full observations and the paste-in setup lines →

METHOD

How I test

  1. 1Read the instructionsThe skill files the tool follows. They're the same for every user, so they're the strongest evidence.
  2. 2Write expected results firstWhat a careful person would do, decided before any run.
  3. 3Run it against trapsA realistic test business with planted problems: hidden instructions, duplicates, gaps and Australian dates.
  4. 4Report with a fixEach finding quotes the tool's own wording, with what to change or paste in.

I also build Claude skills for small businesses through Quiet Content. Those builds are tested with this method before handover.

WORKING TOGETHER

Booked like any other testing

From $500 a day, no lock-in. Each review is quoted on the tool's size and what you need checked.

Check my availability. Payment and other details are in my terms of engagement.

Tell me which skill or plugin, and what it's for.

I'll reply within one business day and tell you plainly whether I can help.

nina@velocityqa.com.au