Responsible CV: Privacy, Fairness, Security, Governance

The traffic counting system works. It also has eight cameras pointed at a public street, storing footage of everyone who walks past, and a pedestrian detector whose recall nobody has checked across different groups of people. None of that shows up in the mAP.

Prerequisites: Model explainability and bias checks and deploying computer vision models.

Four questions, all measurable

Fairness, computed

The pedestrian detector, evaluated on a held-out set split by apparent skin tone using the Fitzpatrick scale:

Equal opportunity difference — the gap in recall:

$\Delta = 0.8918 - 0.8446 = 0.0472$

Disparate impact ratio — the ratio of the lower to the higher:

$\text{ratio} = \frac{0.8446}{0.8918} = 0.9471$

At 0.947 this passes. But before acting on either number, check whether they are real.

The confidence interval that decides whether the gap exists

Recall is a proportion, so the standard error is:

$SE = \sqrt{\frac{p(1-p)}{n}}$

For the darker-skinned group at $n = 1210$:

$SE = \sqrt{\frac{0.8446 \times 0.1554}{1210}} = \sqrt{\frac{0.13125}{1210}} = \sqrt{0.00010847} = 0.01041$

$\text{95\% CI} = 0.8446 \pm 1.96(0.01041) = 0.8446 \pm 0.0204 = [0.824,\; 0.865]$

The lighter group’s interval is $[0.878,\; 0.906]$. They do not overlap, so the 4.7-point gap is real.

Now suppose the slice had only 120 samples:

$SE = \sqrt{\frac{0.13125}{120}} = \sqrt{0.0010937} = 0.03307$ $\text{95\% CI} = 0.8446 \pm 0.0648 = [0.780,\; 0.909]$

That interval comfortably contains the other group’s recall. The same 4.7-point gap would be indistinguishable from noise.

Where the gap comes from, and what to do

Slicing by two variables at once is usually the revealing step. The gap above was 4.7 points overall — but 1.9 points in daylight and 11.3 points after dark. That is not a fairness problem in the abstract; it is a low-light problem that lands unevenly, and the fix is exposure handling and night-time training data.

Privacy through not collecting things

The traffic system’s purpose is counting vehicles. Compare what that requires with what a default installation stores:

The ratio is the argument. Aggregate counts are 400,000 times smaller than the footage and answer the question completely. Everything above them is stored for reasons nobody wrote down — usually “in case we need it later”, which is exactly the reasoning that fails a data protection review.

flowchart TD
  A["Camera frame"] --> B["Detect + track<br/>on device"]
  B --> C["Aggregate to counts"]
  C --> D["Publish counts<br/>per minute"]
  A --> E{"Retain frame?"}
  E -->|"default"| F["Discard immediately"]
  E -->|"explicit request only"| G["Blur faces + plates"]
  G --> H["Encrypted store<br/>auto-delete after 7 days"]
  H --> I["Access logged<br/>+ approval required"]

The default path never writes a frame anywhere. The retention path exists but requires a specific request, and it expires by itself.

Security

Three distinct threats, often conflated.

Evasion, with numbers

The standard demonstration adds a small perturbation in the direction that most increases the loss:

$x_{\text{adv}} = x + \epsilon \cdot \text{sign}\!\left(\nabla_x \mathcal{L}(x, y)\right)$

With $\epsilon = 8/255 = 0.0314$, every pixel moves by at most 8 of 255 levels — under the threshold at which a person notices anything.

Physical attacks exist too — printed patches that stop a person being detected, and stickers that change a sign’s classification. They are less transferable than the digital versions, but they need no access to anything.

Whether to care depends entirely on the threat model. A traffic counter has no motivated adversary and adversarial training would cost 7 points of clean accuracy for nothing. A system controlling access to a building does, and the trade looks completely different.

Governance you can actually run

The out-of-scope section is where a counting model gets stopped from becoming an enforcement model. Writing “this system is not validated for identifying individuals and must not be used for enforcement” before launch is a two-minute job. Writing it after someone has asked for that feature is a negotiation.

A release checklist

flowchart TD
  A["Model trained"] --> B["Slice metrics + CIs"]
  B --> C{"Any ratio < 0.8?"}
  C -->|yes| D["Diagnose cause<br/>fix or document"]
  D --> A
  C -->|no| E["Explainability spot-check"]
  E --> F{"Shortcut found?"}
  F -->|yes| D
  F -->|no| G["Calibration + thresholds"]
  G --> H["Model card + owner"]
  H --> I["Canary: 5% of sites"]
  I --> J{"Metrics hold for 7 days?"}
  J -->|no| K["Roll back"]
  K --> D
  J -->|yes| L["Full rollout"]
  L --> M["Monitor drift"]
  M --> N{"Drift detected?"}
  N -->|yes| A

Even in the exempt cases, slice the metrics. It costs an hour and it is the check that finds real bugs, not just governance ones.

Practice task

Use any classifier or detector with metadata available.

1. Pick two or more slices. Compute the metric for each, with n.
2. Compute the standard error and 95% CI for each slice. Do the intervals overlap?
3. Compute the disparity ratio. Compare against 0.8.
4. Re-slice on two variables at once — group and lighting, or group and time of day.
5. Write down every piece of data your system stores and how long it keeps it. Cross out what is not needed.
6. Implement FGSM at ε of 2, 4, 8, 16 of 255. Plot accuracy. Look at the ε=8 image.
7. Write a one-page model card, including the out-of-scope section.
8. Run the ten-item release checklist honestly. Note which items you cannot currently pass.

Step 8 is the one that finds things. Most systems fail three or four items on the first pass, and the failures are usually the easy ones — no enforced deletion, no named owner, no tested rollback.

Summary

Fairness is measurable. The pedestrian detector had recall 0.8918 and 0.8446 across two groups: a gap of 0.0472 and a ratio of 0.9471, which passes the four-fifths rule. At $n = 1210$ the confidence intervals were ±0.020 and did not overlap, so the gap was real — at $n = 120$ the margin would have been ±0.065 and the same gap would have been invisible. Re-slicing by time of day showed 1.9 points in daylight and 11.3 after dark, which turns a vague fairness concern into a specific low-light fix.

Privacy is mostly about not collecting. Aggregate counts are 26 MB a year against 10.4 TB of rolling footage, and they answer the question completely.

Security depends on the threat model. FGSM at ε = 8/255 took accuracy from 94.2% to 10.9% with no visible change to the image, and adversarial training recovered it to 64.2% at a cost of 7 points on clean data. Only worth it if someone is actually attacking you.

Governance is a one-page model card and ten checks. The out-of-scope section is the one that matters most and takes the least time.

Where this leaves you

That is the series. You started with a pixel grid and a rule about dark fractions, and worked through image formation, filtering, edges, contours, features, calibration, stereo, motion, classification, detection, segmentation, transformers, explainability, 3D, retrieval, real-time systems, deployment, and now the checks that decide whether any of it should ship.

The series roadmap has the full map if you want to revisit a track. The most useful next step is not another topic — it is one project end to end: collect a small dataset, work the project loop, get something running, and slice its metrics before you tell anyone it works.