A penetration test and a StackAttest validation answer different questions
If you are looking for a penetration testing alternative, the honest answer is that there is not one. A person hunting for the failure nobody anticipated cannot be automated, and this page is not going to pretend otherwise. What follows is what each instrument is for, so you can tell which one your situation calls for, and when the answer is both.
Which one fits your situation
A penetration test may be the better fit when a customer contract, an insurer, an auditor or a regulator asks for one by name, when your product handles something whose loss would be expensive, or when the risk you are worried about is business logic that nobody wrote a rule for. A tester chains two harmless-looking findings into one that is not, and that judgement is the thing you are paying for.
StackAttest may be the better fit when you need to know where the product stands this week rather than next quarter, when the same controls need checking again after every release, and when the result has to be readable by an investor, an accelerator or a customer without them taking your word for it.
The same questions, asked of both
Each row is one question put to both instruments. Nothing here is a score, and neither column is trying to win.
What it is for
Finding the failure nobody anticipated. A tester works to an agreed scope and a documented methodology, and the value is in the judgement they bring to what they find.
Establishing whether a defined set of controls holds right now, and recording how strongly each result was established.
Who does the work
People. The engagement is their time, which is what makes it expensive and what makes it good.
A deterministic engine and bounded runtime probes. No improvisation, and no ability to follow a hunch.
Scope
Whatever the engagement letter defines: an application, an API, a network, a cloud account, sometimes the people who work there.
Thirteen controls covering the application, its dependencies, its running configuration and its operational readiness, mapped to OWASP ASVS 5, NIST SSDF, CWE, OWASP API Security Top 10, WSTG, SLSA. A defined floor, not a complete security review.
Source analysis
It depends on the engagement. A white box test reads the source; a black box test deliberately does not, because the point is what an outsider can reach.
Where a repository is connected, read only: where credential material lives, whether authorisation is enforced on the server rather than in the interface, whether data access is injection safe, and what the dependency graph carries. Findings cite the file they came from.
Runtime testing
Yes, and not confined to a script. A tester follows whatever looks interesting and combines findings that are individually small, which is the part that cannot be automated.
Yes, within bounds. The application is built and run in an isolated environment and probed against fixed checks, including whether the deployed system actually refuses a request for another account's data.
What the evidence looks like
A written report: findings, severity, reproduction steps, and the tester's assessment of what they mean together.
A result for each control with an evidence grade from E0 unknown to E5 independently reproduced, and an explicit list of the controls the run could not reach.
Repeatability
A retest normally follows remediation. Past that, a second engagement is a second engagement, and a different tester will find a different set. That variety is part of the value, and it is also why two reports are hard to compare.
The same control pack every run, so two validations are comparable and a change in the product shows up as a change in the result.
Cadence
Scheduled. Annually, or ahead of a significant release, because it is booked human time.
Whenever you want, including on every release. A check against the public URL needs nothing but the address and no account.
Who asks for it
An enterprise customer, an auditor, an insurer or a regulator, usually by name and usually in writing.
A founder who needs to know where the product stands, and the investor, accelerator or buyer who wants that in a form they can read.
What you have at the end
A report, a remediation list, normally a retest, and a conversation with the people who did the work.
A Passport you control, carrying the control results, the evidence grade behind each one, and what stayed uncovered.
Where it stops
It describes the system as it was during the test window. Code shipped the week after was not tested, and the report ages from the day it is issued.
It is not adversarial. Nothing here hunts for the failure nobody anticipated, and it does not satisfy a requirement that names a penetration test.
Penetration testing is described here in general terms, as the practice is defined by published methodologies rather than by any one firm. Where a specific requirement is quoted, it comes from the standard that carries it.
Where they overlap
Both will tell you about a missing security header, an error response that discloses internals, a dependency carrying a public advisory, and an object your server hands over without checking who asked. If a tester spends the first two days of an engagement on that list, you are paying an expert rate for something a machine can do on every deploy. That is the argument for running the cheap thing continuously, and it is also the whole of the overlap. Everything past it belongs to the person.
This does not replace a penetration test
That is the first thing to say, because it is the thing a page like this is tempted to fudge. A penetration test is an adversarial human engagement: someone with an attacker's instincts spends days inside your product looking for what nobody wrote a rule for, chains findings together, and writes up what it means. StackAttest runs defined checks against defined controls. It is repeatable because it is defined, and it is bounded for the same reason. If somebody has asked you for a penetration test, this is not one.
Where a requirement names a penetration test, nothing here satisfies it
Requirements are written in specific words and are read literally. PCI DSS, for example, requires internal and external penetration testing at least once every twelve months and after any significant change, carried out to a documented methodology. A customer's security addendum can say the same thing in its own words. Where that is your situation, book the test. A Passport is a different artifact and will not be accepted in place of one.
What a defined check is genuinely good at
The failures that recur in fast-built software are not exotic. Authorisation that lives in the interface instead of on the server, a privileged key that shipped inside a browser bundle, a database rule that was never switched on, a dependency with a public advisory and an available fix. These are worth catching mechanically, on every release, because they are the same shape every time and because nobody wants to spend an expert's week finding them.
The order most early teams end up in
An engagement you cannot afford this quarter is not protecting anything this quarter. Running defined checks from the first deploy means the recurring classes of failure are dealt with continuously, and when there is finally something worth an attacker's week, the tester's days go on the work only a person can do rather than on a list a machine already produced. Continuous first, adversarial when the stakes or a contract call for it, is the sequence that wastes the least money.
What you can hand to someone who asked
A penetration test report is usually confidential and is often the wrong document to send anyway: it is long, it is addressed to your engineers, and it describes a moment that has passed. A Passport is founder controlled, shows a result per control with the strength of the evidence behind it, and names what was not covered. It answers a due diligence question. It does not answer a demand for a penetration test report, and it does not try to.
What this page is not claiming
- It is not a penetration test, and running it more often does not make it one. Deterministic checks and bounded runtime probes are not a person hunting for the failure nobody anticipated.
- Where a contract, an insurer, an auditor or a regulator requires a penetration test, this does not satisfy that requirement, no matter how good the result looks.
- It does not test business logic. A rule nobody wrote is a rule nothing here can check, and that is exactly the class of problem an engagement exists to find.
- Thirteen controls is a defined floor, not a complete security review. A clean result means the controls that were tested held at the evidence strength recorded, mapped to OWASP ASVS 5, NIST SSDF, CWE, OWASP API Security Top 10, WSTG, SLSA, and nothing wider than that.
- It is not SOC 2 and does not replace it. SOC 2 attests to organisational controls over a period; this validates technical controls in the software. They answer different questions for different buyers.
- It does not replace human due diligence. It gives a reviewer repeatable evidence to start from, which is a different job from being the reviewer.
- Controls a run could not reach are reported as uncovered rather than passed. A result you did not earn is worse than no result.
Questions
Is there an alternative to a penetration test?
Not a substitute for one. There are cheaper instruments that catch different things: automated validation of defined controls, dependency scanning, static analysis. They are worth running, and worth running far more often than an engagement can be booked. None of them replaces a person trying to break your product, and where a contract asks for a penetration test, only a penetration test answers it.
Do I need a pen test before launch?
If a customer, an insurer, an auditor or a regulator has asked for one, yes, and the date they asked for is the date that matters. If nobody has asked and you are shipping an early product, the money usually goes further on repeatable checks that run on every release, with an engagement booked once there is something worth an attacker's week.
What does automated security testing actually catch?
The failure classes defined in advance: transport security and response headers, credential material reachable from client code or committed to the repository, authorisation enforced only in the interface, injection-unsafe data access, dependencies carrying known critical advisories, and error responses that disclose internals. Where a runtime validation runs, it also establishes whether the deployed system refuses a request for another account's data.
Can I use this to prepare for a penetration test?
Yes, and it is a reasonable way to spend the weeks before one. Everything a defined check finds is something a tester would otherwise write up on the first day at their rate. Clearing that list first buys their remaining days for the work only a person can do.
My customer asked for a pen test report. Will a Passport do?
No. Send them a penetration test report. A Passport reports technical control results with graded evidence, which is a different document answering a different question, and offering it as a substitute would fail on the first read.
How often should a validation run?
As often as you deploy, which is the reason to have something that costs little to run. A check against the public URL needs only the address, and repository or runtime validation is worth re-running whenever the code that could change the answer has changed.
Related
Find out where your application actually stands
Start with the layer you can run today. The result says which layers ran and what they could not reach, so it is honest about its own limits.