Why this exists

Nobody is checking whether the answer was true

The gap this register was built to sit in, why it is expensive now, and what would have to happen for the register itself to be worthless.

An agent tool gets checked for two things before anyone trusts it. Is this the package it claims to be, and does it contain a known vulnerability. Both are questions about the code. Neither is a question about the answer it returns.

Two questions are answered. The third is not.

Package registries answer identity. Signing, provenance and lockfiles establish that the artifact you installed is the artifact the maintainer published. That is a real problem and it is largely solved.

Scanners answer contents. They read the code and the dependency graph and report known vulnerabilities and suspicious patterns. Also real, also largely solved, and improving fast.

Between them there is a question nothing answers. When the data underneath the tool is missing, stale, malformed or contradictory, does the tool stop, or does it return something plausible and wrong? Identity and contents are both satisfied while that happens. The package is exactly what it claims to be. The code contains nothing malicious. The answer is simply not true.

An agent cannot tell wrong data from no data

This gap was survivable while a person sat between the tool and the decision. A human reading a stock price four days stale might notice the date. A human handed an empty result where data plainly should exist tends to ask why.

An agent does neither. It receives a well formed response, finds nothing in it that signals a problem, and proceeds. Wrong data and correct data arrive through the same channel in the same shape, and nothing in the response distinguishes them. So the agent acts on both, identically, at machine speed, in a loop nobody is reading.

The failure has a name worth using precisely: silent semantic corruption. A response that is well formed and plausible but materially wrong, stale, mis-scoped, truncated, empty where data existed, or served from a fallback nobody disclosed. It passes every schema check and every scanner, because nothing about it is malformed.

One test governs everything here. Does it fail loud, or does it lie quiet? A tool that stops when it cannot be sure has left you something to route around. A tool that guesses confidently has not.

The tool's own tests cannot close it

Almost every agent tool ships with tests, and almost all of them are written by the person who wrote the tool. That is not a criticism of anyone's diligence. It is a structural limit. A suite written by the maintainer encodes the maintainer's model of what the tool does, and the failures in question are precisely the ones that model does not contain. You cannot write an assertion against a fallback you do not know your dependency performs.

There is a sharper version of the same problem. A test that imports the code it is testing has already agreed to that code's account of what happened. If the tool believes it returned four decades of history, a test built on the tool's own objects will believe it too. The only way to catch the difference is to speak to the thing from outside, the way an agent does, over its own protocol, and compare what came back to a source that was written before anyone went looking.

A status page cannot close it

Uptime monitoring answers whether the service replied. The failure here is a reply. Two hundred, well formed body, correct schema, wrong contents. Every dashboard stays green through the entire event, because every dashboard is measuring the wrong property. Availability is not accuracy, and no amount of the first produces the second.

A score cannot close it

The obvious product here is a number. Rate every tool out of a hundred, sort the list, let buyers glance. That product would sell faster and be worth less than nothing, because the glance is the failure. A buyer who reads a number does not read what was tested, and what was tested is the entire content of the claim.

So a record carries one of five states and never an average. PASS, FAIL_SAFE, FAIL_UNSAFE, INDETERMINATE, OUT_OF_SCOPE. Refusing to answer and answering wrongly are different failures with different costs, and any function that averages them into one figure has destroyed the only distinction that matters.

Which is why it has to sit outside

A finding that costs a maintainer something cannot be produced by a party the maintainer pays. That is not an accusation about anyone's integrity, it is a statement about what a reader is able to verify. A reader cannot see whether a payment shaded a conclusion, so a reader who knows the payment exists is right to discount the conclusion, and the finding was worth nothing before anyone read it.

So the graded party does not pay, is not offered a way to pay, and cannot buy a re-run on friendlier conditions. What a subject is owed is the complete evidence and a fixed window to answer it, and the answer publishes beside the record, unedited. Nothing else here is for sale to them.

What would make this worthless

A register that cannot say how it would fail is asking to be taken on faith, which is the thing it exists to replace. Any one of these ends it:

• The suite never becomes public, so no stranger can re-run a verdict and reach the same answer. Today it is not public, and until it is, every record here rests on one party's word.
• A verdict is issued against a subject that cannot be resolved exactly, so nobody can reconstruct what was actually tested.
• The graded party pays, in money or in anything that behaves like money.
• A record softens between the moment its subject sees it and the moment it publishes. The hash written down before the window opens exists so that this is checkable rather than promised.
• A finding is published against a named party before that party has had the window to answer, or the register hints at who a held finding is about while withholding the evidence for it.

Those are not hypotheticals chosen for effect. Every one of them has been a live risk in this project already, and the rules exist because of specific occasions when the rule was not yet there.

What is here today

The register carries the current state: how many records exist, how many are held, and what any published record found. It is compiled from the records on every build, and no page on this site retypes it, because a second hand kept copy of a register is how a name under embargo eventually gets published by accident.

The honest summary is that this is early, and the register is the only accurate statement of how early, because the register is compiled and this paragraph is typed. What is defensible here regardless of the count: the method is written down, the suite that applies it is readable by anyone, and the rules were applied to Nobulex before they were applied to anyone else. That is a smaller claim than most projects at this stage make. It has the advantage of being checkable.

The suite is at github.com/arian-gogani/nobulex-registry, MIT licensed. If you can construct a corrupted tool response that it grades clean, that is a defect in the grader, and it is the single most useful thing anyone could send here.

Read the register for the current state, or the method for how a verdict is decided.

If something on this page is wrong, say so: nobulex.dev@gmail.com. The argument is meant to be attacked, and a correction that lands changes the page.