Catch PII leaks before they merge
Your gateway sits between your app and the company that runs the model: OpenAI, Anthropic, whoever you call. That company is the provider, and your gateway's job is to strip personal data out of the request before it goes there.
This job checks that it still does, on every pull request, and fails the build when it does not. Setup is one file and three lines. No account, no API key, no paid model.
The address arrives exactly as your user typed it, and lands in the provider's logs. The reply your app gets is correct, which is the whole problem: nothing on your side records that it happened.
The provider sees a placeholder. On the way back the gateway puts the real address in again, so your app gets what it sent and the request works as before.
Nothing leaked, and your app is broken anyway: the caller gets a placeholder where its own data should be. The job calls this CHECK FAILED rather than a leak, because the fix is a different one. If your product masks in one direction on purpose, duty: anonymize drops the two restore checks and an otherwise sound run comes back CLEAN. The summary prints the duty beside the result, so that CLEAN is never mistaken for a scored return leg.
The hard part is that nothing tells you. Redaction is a pattern-matching rule, and rules break quietly. Someone edits a prompt template. Someone narrows a regex to kill a false alarm. The provider adds a field. Your tests still pass, because the reply coming back still looks right, while real customer data leaves your network and nobody knows until it turns up in OpenAI's logs.
So it has to be checked. This checks on the pull request that broke it, names the data type, shows you the value as it left your network, and says what to do about it, while the fix is still one line.
See it work in 30 seconds, with no gateway at all
Nothing to set up. This runs the same check against a direct connection with no gateway in the middle, so every value leaks, on purpose:
pipx run pii-leak-benchmark selfcheck --target-base-url capture://self
It starts a local capture, sends six kinds of fake data through, and reports what arrived:
LEAK
Raw fixture values reached the upstream: AWS_ACCESS_KEY_ID, CREDIT_CARD,
EMAIL, GITHUB_TOKEN, SLACK_TOKEN, SSN.
LEAK EMAIL reached the model provider
you sent: [email protected]
the provider saw: [email protected] <- nothing removed it on the way
That is the failure the job catches. Nothing leaves your machine, the values are fake, and
capture://self is the control: with no gateway in it, it must say LEAK. Anything else
means the instrument itself is broken.
Point --target-base-url at a real gateway instead and it is the same check your CI will run.
How it can tell
Because the capture stands where the provider normally does, it records the request byte for byte and can compare it against what was sent. There is no model: the capture streams the message it received straight back, one character per event, so whatever your gateway sent is exactly what it has to read on the way in. That is the hardest case for restoring a value, and the benchmark scores both directions of the same trip. The values are synthetic and nothing leaves your machine.
The check stands where the provider normally does, so it sees the request your gateway actually sent and compares it against what went in. Anything that comes out the far side unchanged is something OpenAI would have received.
The trip back matters as much. A gateway that masks on the way out and never puts the value
back is not leaking, but it hands your caller a placeholder where its own data should be, which
breaks the app just as effectively. The capture replies with the message it received, one
character at a time, so the value your gateway has to rebuild arrives split across as many
events as it has characters. response_fidelity records whether the caller got its own text
back, fragmentation_safety whether it survived being split, and that is why a run can report
no leak and still fail.
No gateway of your own? Point the check at someone else's
capture://self measures nothing about any product, because there is no product in it. For a
real result you need something in the middle. Whichever gateway you pick, the shape is the same:
- Tell the gateway its provider is the capture. The check listens on
http://127.0.0.1:8765. Whatever setting normally holds your OpenAI base URL gets that address instead. - Start the gateway, however its own documentation says to.
- Point the check at the gateway rather than at a provider.
Only step 1 differs between products, and it is the one setting your gateway already uses to
reach OpenAI: usually api_base, base_url, custom_host or an OPENAI_BASE_URL environment
variable, wherever that gateway keeps its provider configuration.
Their documentation is the authority on that, not this page. Setting names change and a copy here would go stale unnoticed, so what follows links to each project's own instructions instead of repeating them. If a link below is dead or the setting has moved, theirs is right and ours is out of date: please tell us and we will fix it.
LLM-Shield-Proxy (the fastest way to see a real result)
Ours, so this one is exact and kept working. Installing it brings the check along with it, so there is nothing else to fetch.
pip install llm-shield-proxy
UPSTREAM_BASE_URL=http://127.0.0.1:8765 UPSTREAM_API_KEY=unused VALID_VIRTUAL_KEYS=sk-demo llm-shield-proxy --port 4000 &
pii-leak-benchmark selfcheck --target-base-url http://127.0.0.1:4000/v1 --target-api-key sk-demo
On Windows PowerShell, where per-command environment variables and & do not work:
pip install llm-shield-proxy
$env:UPSTREAM_BASE_URL = "http://127.0.0.1:8765"; $env:UPSTREAM_API_KEY = "unused"; $env:VALID_VIRTUAL_KEYS = "sk-demo"
Start-Process -NoNewWindow llm-shield-proxy -ArgumentList "--port 4000"
Remove-Item Env:UPSTREAM_BASE_URL, Env:UPSTREAM_API_KEY, Env:VALID_VIRTUAL_KEYS
pii-leak-benchmark selfcheck --target-base-url http://127.0.0.1:4000/v1 --target-api-key sk-demo
VALID_VIRTUAL_KEYS is the key your client must send, and UPSTREAM_API_KEY is the key the
proxy sends on to the capture, which ignores it. Without the first every request gets a 401;
without the second every request gets a 500.
The started process keeps the settings, and the third line takes them back out of your shell. Without it they last as long as the window, and the next gateway you start in it would be pointed at the capture without saying so. The POSIX form scopes them to the one command already.
That should print CLEAN, against the LEAK you got with nothing in the middle. Full options:
deployment guide.
LiteLLM
Install and run the proxy: LiteLLM proxy quick start.
Point the provider base URL for your model at http://127.0.0.1:8765/v1.
To measure redaction rather than a bare relay, switch a guardrail on first: LiteLLM guardrails. Without one it forwards everything, which the check will correctly report as a leak.
Portkey
Run the gateway: Portkey open-source gateway. It takes
the provider host per request as a header, so set that to http://127.0.0.1:8765/v1.
Portkey applies no guardrail unless the request names one, and it answers 200 either way, so a
run without one measures a plain relay rather than its redaction.
LLM Guard
LLM Guard is a scanner library rather than a gateway, so there is no base URL to change: something has to call its scanners around an HTTP endpoint you provide.
This repository contains a minimal one used for the published measurements, at
benchmarks/llm-guard-v2-profile/gateway.py. Read its docstring before drawing conclusions from
a result: it exposes the one genuine choice a streaming integrator has to make, through
LLMGUARD_MODE=chunk-local or buffered, and that choice rather than LLM Guard decides how the
numbers come out.
Other gateways have already been measured this way. The results wall shows what each one scored and exactly how it was configured, including several that sent every data type straight through with redaction switched on. It is the quickest way to pick one worth trying.
1. Add the workflow
If you want to build and measure the proxy source in its own fork, start with a copyable source-build recipe for LLM-Shield-Proxy, Portkey, LiteLLM, or NeMo Guardrails. The generic Action below assumes you provide the gateway startup command.
Save this as .github/workflows/pii-leak-check.yml:
name: PII leak check
on: [pull_request]
permissions:
contents: read
jobs:
pii-leak:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
# Build or install YOUR gateway here, however your project does it.
- uses: actions/setup-python@v7
with:
python-version: '3.12'
- run: pip install -r requirements.txt
- uses: ninadphalak/LLM-Shield-Proxy@benchmark-v0.6.0
with:
target-base-url: http://127.0.0.1:4000/v1
start-command: ./scripts/start-test-gateway.sh
upstream-env: UPSTREAM_BASE_URL
2. Change three lines
| Line | Put your own |
|---|---|
target-base-url | The /v1 address your gateway listens on |
start-command | A command that starts it in the foreground and keeps running |
upstream-env | The environment variable it reads for its provider URL, such as OPENAI_BASE_URL |
3. Open a pull request
It runs on every pull request from now on and takes about a minute. It shows up as a green or red check in the PR, and the full result is written to the job summary, the page you land on from Details. You do not have to read the logs.
The summary opens with the verdict, one of CLEAN, LEAK, CHECK FAILED or NOT MEASURED,
then one row for each of the six data types every run sends: EMAIL, SSN, CREDIT_CARD,
AWS_ACCESS_KEY_ID, GITHUB_TOKEN and SLACK_TOKEN. You get all six whether you went looking
for them or not, so a gateway written around email addresses finds out in the same minute what
it does with a card number and an AWS key.
| Data type | Baseline | Current | Next step |
|---|---|---|---|
| not run | leak | Enable or repair request redaction for this format. | |
| SSN | not run | contained | No leak observed for this test shape. |
Those six are also the whole list, and the summary says so in a Not tested by this profile section, so a clean run is not mistaken for coverage of names, addresses, clinical data, government IDs or private keys.
Baseline reads not run until you set up a previous version to compare against, which is
optional and covered below.
Under that, for anything that leaked, a section called What leaked, and why it matters puts
the value you sent beside the value the provider received, and says why that data type is worth
caring about. Values there are printed as shapes such as <EMAIL>, because the summary is
visible to everyone who can see the pull request. The real specimens go to the terminal of the
machine that ran the check.
Then the required behaviour checks, pass or fail, and a What to do next line written for the verdict you actually got rather than a generic one.
Red means your gateway sent that value to the provider unmasked, and the job blocks the merge if you require the check. Nobody is emailed and nothing is sent anywhere: the result stays in the pull request.
Publishing the result, from the summary
The summary ends with everything needed to say something checkable about the run:
-
The citation block, in a collapsible section. It names the harness revision, the test data and the configuration the numbers came from, so a reader can rerun your exact run, and it is in the page rather than behind a zip and a second command.
-
Where the reports are, spelled out, because "it is in the artifact" is not directions:
- Open the Actions tab of your repository.
- Click the run, named after your workflow.
- Scroll to the bottom of that page, past the job list, to the Artifacts section.
- Download
pii-leak-benchmark, or whatever you setartifact-nameto.
The zip holds
summary.md(this page, as a file),current.json(the operator result),current.raw.json(everything measured) andpii-leak-badge.json(the README badge). Artifacts expire, 90 days by default, so a link to a run is not a permanent link to its reports. -
A one-click link to the results wall, prefilled with the version, the outcome and a link back to this exact run. You add the gateway name and the licence, which a run cannot know, and your row is published automatically from there.
A workflow that uploads its own evidence bundle, such as the
proxy source-build recipes, sets submission-section: 'false' and writes a
submission link that points at that bundle instead.
Nothing leaves your repository unless you click that link. The link carries the citation, never the report: a report can hold your gateway's base URL and your capture hostname.
The published row's numbers are read from this run's artifact rather than from anything you type, which is why the run has to be public and why artifacts expiring matters. Submit while the run is still there.
That is the whole setup. Everything below is optional.
Optional: the rest
Run the response-split profile as well
profiles: operator is the default and runs the request-path check above. profiles: both
adds the response-split profile from the same job: the action starts its own capture on
response-capture-port (8799 by default), your gateway must send upstream traffic there
while it runs, and the step summary gains a table with FidelityRate, the leak rate with the
value whole and split across two chunks, and DeltaFrag. The raw report is uploaded next to
the operator report and validates against the published schema. profiles: response runs
only that profile. The seed is response-seed, the published one by default. This needs a
harness with the pii-leak-benchmark-v2 command, so source: ./pii-leak-benchmark from a
current checkout, or a release later than 0.4.1.
This profile measures the reply, so the gateway has to be scanning replies. LLM-Shield-Proxy
does not by default: start it with ENABLE_RESPONSE_PII_REDACTION=true for this profile, or
the values the capture injects pass through and the profile reports a leak with the request
path still clean. It is off by default because it changes what callers receive (a value the
model rephrases rather than quotes is redacted and not restored); the published row for the
proxy was measured with it on and says so in its configuration.
Show the result on your README
GitHub publishes a status badge for the workflow. Add it to your README, replacing
OWNER/REPO:
[](https://github.com/OWNER/REPO/actions/workflows/pii-leak-check.yml)
It tracks the default branch and turns red the moment a leak lands there. On a public repository anyone can see it, which is the point: it is a claim a reader can click and check instead of taking your word for it.
That badge reports the job, not the measurement. It is green whenever the workflow exited cleanly, so a gateway that contained everything and one that was never asked to look both show the same tick. To put the result itself on your README, the run writes a second file:
pii-leak-badge.json
It lands in the reports directory beside summary.md, so it is already in the artifact the
workflow uploads. It is a Shields.io endpoint
file. Publish it anywhere your repository serves raw files, then point Shields at it:
[](https://github.com/OWNER/REPO/actions/workflows/pii-leak-check.yml)
Nothing is hosted by this project and there is nowhere to register: the file comes from your run
and is served by you, which is also why it is not evidence to anyone else. A reader who wants to
check it follows the badge to the run and reruns the instrument named in
pii-leak-benchmark cite.
The Action renders it from the operator result, so it names the types rather than a rate. These four are the only badges a CI run can produce, and the one you get is decided by the measurement, not by anything you configure:
| Badge | What earns it |
|---|---|
 | No test value reached the upstream and the required checks passed |
 | Those data types were seen arriving upstream, named in full |
 | Nothing leaked, but a required check did not hold |
 | Setup, attribution, inspection or comparison was incomplete |
The green one is the only one worth putting on a README, and it is the only one you cannot get by accident: it takes a run that measured something and found nothing.
To render one by hand from any finished report:
pii-leak-benchmark badge ./PII_LEAK_BENCHMARK_LATEST.json --out ./pii-leak-badge.json
Handed a research-profile report instead, the same command counts scored cases
(leaked 4 of 16) and reads the redaction claim rather than the verdict. That adds
no leak, profile not met in yellow, no redaction offered in blue for a product that does not
claim to redact, and the greys inconclusive, redaction not enabled and claim unstated.
Only a real pass is ever green. A grey badge is not a soft pass: it means the run did not learn anything about the gateway, and it is worth fixing the configuration and rerunning.
The four possible results
| Result | Meaning | Job status |
|---|---|---|
| CLEAN | Required checks completed for the selected duty | Pass |
| LEAK | Fake values were seen arriving upstream | Fail |
| CHECK FAILED | Response or transport checks failed without an observed leak | Fail |
| NOT MEASURED | Setup, attribution, inspection or comparison was incomplete | Fail |
Leaks you already had also fail the current job. A no-regression result does not excuse them.
Locally, exit 0 means clean, 1 means a measured failure, and 2 means incomplete measurement.
Compare against your previous version
Add a second gateway and the summary separates new failures from ones you already had. The runner builds both versions; the old one needs no permanent deployment.
- uses: actions/checkout@v7
with:
path: candidate
- uses: actions/checkout@v7
with:
ref: ${{ github.event.pull_request.base.sha }}
path: baseline
# Build both versions here in separate environments or container images.
- uses: ninadphalak/LLM-Shield-Proxy@benchmark-v0.6.0
with:
target-base-url: http://127.0.0.1:4000/v1
start-command: cd candidate && ./scripts/start-test-gateway.sh --port 4000
target-version: ${{ github.sha }}
baseline-base-url: http://127.0.0.1:4001/v1
baseline-start-command: cd baseline && ./scripts/start-test-gateway.sh --port 4001
baseline-version: ${{ github.event.pull_request.base.sha }}
upstream-env: UPSTREAM_BASE_URL
The baseline is measured first, then the candidate, with identical fixture values, prompt context, model, profile, seed and expectations. Record the code, policy and dependency versions too: any of them can cause a difference.
A scheduled or push job needs an explicitly selected baseline commit or released image.
The cheaper option is to reuse the last result from your default branch instead of building the old gateway at all. The job fetches that result itself, so there is nothing to do by hand:
permissions:
contents: read
actions: read # needed to read the earlier run
steps:
- name: Fetch the last main-branch result
env:
GH_TOKEN: ${{ github.token }}
run: |
id=$(gh run list --workflow pii-leak-check.yml --branch main --status success --limit 1 --json databaseId --jq '.[0].databaseId')
gh run download "$id" --name pii-leak-benchmark --dir baseline || echo "no baseline yet"
- uses: ninadphalak/LLM-Shield-Proxy@benchmark-v0.6.0
with:
target-base-url: http://127.0.0.1:4000/v1
start-command: ./scripts/start-test-gateway.sh
upstream-env: UPSTREAM_BASE_URL
baseline-report: baseline/current.json
Two things to know before you rely on it. It compares across environments and time, so an
unrelated runner change can look like a gateway change; measuring both versions in one job
does not have that problem. And a baseline is rejected outright, as NOT MEASURED, if the
benchmark version, profile, model, iterations, seed or duty differ, or if either run was
incomplete. That is deliberate, but it means the job fails after you upgrade the benchmark
until your default branch has produced a fresh baseline.
One-way masking, and which data types are tested
duty: restore is the default and requires your gateway to give the caller back the original
text. Set duty: anonymize if your product masks in one direction on purpose; that drops the
restoration checks but still requires upstream containment, valid SSE and completed requests.
The benchmark cannot guess which you meant, so say which.
profile: pii-secrets-v1 is the default: email, a fake US SSN, a published test card, and
fixed AWS, GitHub and Slack credential examples. profile: pii-v1 tests only the three
personal-data formats.
The default seed is gateway-ci-v1. Keep it stable for pull request comparisons and vary it
in a separate scheduled job.
Gateways in Docker, or on another host
On Linux runners a foreground docker run --rm --network host ... can reach the capture.
Pass the upstream setting from BENCHMARK_UPSTREAM_BASE_URL into the container, and build
your image first. A bridge-network container has its own loopback, so its 127.0.0.1 is not
the runner's.
For a gateway on another host, use the CLI's authenticated public-capture options from the hosted gateway guide. The Action example above uses the runner's local capture and cannot inspect arbitrary production traffic from a URL alone.
Run it on your laptop first
One line per command, so there is no continuation to get wrong in any shell. --start-command
takes whatever starts your own gateway, and the .sh path below is a placeholder: on Windows it
would be your own script or a py call.
pip install "pii-leak-benchmark>=0.3.0"
pii-leak-benchmark ci --target-base-url http://127.0.0.1:4000/v1 --start-command ./scripts/start-test-gateway.sh --upstream-env UPSTREAM_BASE_URL --out privacy-check
pii-leak-benchmark ci --help lists every input, with the same meanings as the Action. Use a
fresh --out directory each time.
To pin the Git source instead of the PyPI package:
pip install "pii-leak-benchmark @ git+https://github.com/ninadphalak/[email protected]#subdirectory=pii-leak-benchmark"
Files the job leaves behind
You do not need any of these to read the result. The table above is written straight into the workflow run page, so you read it in the browser and there is nothing to fetch.
The job also uploads the underlying reports as an artifact called pii-leak-benchmark, on
failures too. They are worth opening in two cases: feeding current.json back as a baseline,
and looking at exactly what arrived upstream when a result surprises you.
| File | Purpose |
|---|---|
summary.md | Findings and suggested next steps |
current.json | Operator result and comparison contract |
current.raw.json | Underlying HTTP measurements |
baseline.json | Optional previous operator result |
baseline.raw.json | Raw baseline when measured during this job |
control.raw.json | Negative-control evidence |
current.gateway.log | The tail of what your start-command printed, only with startup-log: 'true' and only when it failed to start (baseline.gateway.log for the baseline) |
pii-leak-badge.json | Shields.io endpoint file for the README badge |
What it does not cover
The Action stands up a fake provider, points your gateway at it through upstream-env, and
records every request that arrives. It runs a control first, so a run that measures nothing
fails instead of passing quietly. It starts your gateway, waits for its port, measures, writes
the summary, and stops what it started. When the command exits before its port opens, the
summary names the exit code. What the command printed is not kept unless you set
startup-log: 'true' (locally, --startup-log): the reports directory is uploaded as an
artifact, and a crashing command may print the secrets it was given. With it set, the tail
goes to current.gateway.log, and only when the gateway failed to start; once it is up, what
it prints is discarded, because by then it may be logging the test requests. A script
committed from Windows usually exits with 126: Git stored it without the executable bit, and
git update-index --chmod=+x scripts/start-test-gateway.sh puts it back.
The fake provider is listening before your start command runs, so a gateway that contacts its provider during startup finds it there. Requests sent during startup are part of the measured record, like any other request to the configured upstream.
Configure the same redaction policy you intend to ship. The benchmark measures your gateway; it
does not turn protection on for you. Pin the Action to an immutable commit SHA if your
organization requires it. For a gateway already running and already pointed at the capture,
omit start-command and upstream-env. Authenticate to your gateway with
CONFORMANCE_TARGET_API_KEY and CONFORMANCE_TARGET_HEADERS.
Limits. The fixture uses a fixed set of formats. The values change every run, the formats do not, so a program written for those formats can pass without being a general detector. Credential examples are fixed shapes, not a representative sample. Only the capture and client boundaries are watched: logs, internal stores and other outbound destinations are outside this check. Run response fragmentation tests separately for injected response data.