Fortmilo

Evidence & Coverage

Source availability, completeness, Coverage and rendered evidence state remain separate.

Source availability, completeness, Coverage, rendered evidence state and licence-assignment capture status answer different questions, and the Observatory keeps them separate. Security Observatory uses evidence-bounded terminology and does not assert that a Security Benchmark for Salesforce (SBS) control is satisfied. The product implementation source supporting source-established statements is privately maintained; those statements remain self-attested unless a public artefact is cited. Validation against the release build and target environment is stated separately rather than implied.

Release-status boundary

Current claims are evidence-classed, not universally release-validated

Selected genuine-zero, unavailable-source and classified scanner-failure paths have automated-test and controlled runtime evidence in a reviewed Salesforce environment. Running-user sharing, CRUD and field-level-security behaviour, complete raw-IP data-shape handling and environment compatibility remain separate validation questions for the release build and target environment. Public package installation is not yet claimed.

Evidence detail levels

One scanner plan, three retention levels

The same 20-family scanner plan runs at Top Issues, Balanced and Everything. The selected evidence detail level changes what safe detail is retained, displayed, compared and exported; it does not change which scanner families are planned. Everything is the deepest supported level, not exhaustive or unlimited.

Evidence boundary

Missing evidence is not a clean result

When an evidence source cannot be assessed, Security Observatory preserves that limitation explicitly. It does not convert unavailable evidence into a zero count, clean state or unsupported assurance claim.

Evidence terminology

One canonical rendered vocabulary

The rendered state is selected according to the limiting evidence dimension. Partial is a completeness qualifier, not a separate state. Coverage remains a separate axis for SBS and other supporting-framework review. A retained state is the point-in-time rendered state stored with a scan.

Current Evidence Terminology Contract · Canonical v1.1 · Frozen v1.0

Severity

  • Critical
  • High
  • Moderate

Rendered evidence state

  • Contextual metric or finding label
  • None found
  • Unavailable
  • Not assessed
  • Not retained at this evidence level
  • Not captured
  • Not applicable

Coverage

  • Automated
  • Partial Evidence
  • Manual Required
  • Not Covered
  • Extended Check
  • Partial Evidence is Coverage, never an outcome or evidence state

Completeness boundaries

  • Partial: shown beside a usable count
  • Not assessed: work not performed or outside assessed scope
  • Unavailable: required attempted source inaccessible or unusable
  • Not retained at this evidence level: detail intentionally not retained
  • Not captured: product intentionally does not capture that detail
  • Incomplete: separate licence-assignment capture status for bounded or truncated capture

Licence-assignment capture status

  • Complete
  • Incomplete — the full population could not be retained safely
  • Unavailable
  • Captured may be lower than expected or zero when safe transaction/DML headroom is exhausted
  • Zero captured is not zero assignments and must not be shown as Complete
  • An unknown expected count remains unknown

Read this before drawing conclusions

What a result does not mean

  • Not a compliance decision. An SBS control reference is a version-bound pointer to retained evidence, not a statement that the control is satisfied, compliant or secure.
  • None found is not a pass. It means no matching records were observed in the successfully assessed scope — not that the control passed or that exposure cannot exist outside the assessed sources.
  • Unavailable is not zero. When a required source was attempted but could not be obtained or used, the result remains Unavailable and carries a bounded reason and safe next action where available.
  • History is not rewritten. A later terminology, SBS registry or mapping change does not silently reinterpret retained historical evidence.
  • Counts are family-scoped. A retained family finding count belongs to the mapped scanner family, not uniquely to one SBS control.

SBS overlay

An independent benchmark, used as an optional layer

Security Observatory can attach Security Benchmark for Salesforce (SBS) control references to a scan. SBS is maintained by its own editorial board and contributors and published under CC BY-SA 4.0. It is not owned, published, endorsed or supported by Salesforce, and it is not a Fortmilo benchmark.

Optional, not required

Security Observatory works without SBS enabled. Turning it on adds a reference layer on top of the same underlying evidence — it does not change what is scanned.

Version-stamped per scan

A scan run with SBS enabled is stamped with the SBS registry version and mapping revision used for that run, so a later catalogue change does not silently rewrite what an older scan meant.

Bounded, not reproduced

Fortmilo stores only bounded registry fields — control ID, short title, domain, category and local mapping fields — never SBS audit procedures, remediation guidance or control prose.

Not a compliance decision: Security Observatory does not determine whether an SBS control is satisfied.

Current v1 SBS mapping

45 controls, fixed to one registry version

The current catalogue is bound to SBS version v0.4.0, mapping set M1, revision 1. Mapping is recorded per control; domain and category labels support grouping and review rather than replacing that per-control record. These figures describe Fortmilo’s retained SBS mapping catalogue for mapping set M1 revision 1; they do not assert completeness or equivalence to the current upstream SBS release.

45SBS controls in the v1 registry
47Mapping metadata records across those controls
11Distinct SBS control-key families mapped

Partial Evidence

  • 10 of 45 controls
  • Carry a retained scanner-family mapping

Manual Required

  • 17 of 45 controls
  • No automated mapping recorded

Not Covered

  • 18 of 45 controls
  • Explicitly outside current automated coverage

There are no Automated or Extended Check dispositions in the current M1 catalogue. Two controls, SBS-ACS-004 and SBS-ACS-008, each map to two scanner families, producing 12 ordered scanner-family associations across the 10 mapped controls. The catalogue spans 11 control-key families: ACS, AUTH, CODE, CPORTAL, DATA, DEP, FILE, FDNS, INT, OAUTH and SECCONF.

Retained traceability

Control, family, state and outcome stay linked — not merged

SBS traceability is not a control ID stamped directly onto every finding. It is a retained, version-bound relationship: retained SBS control → retained mapped scanner family or families → retained scanner-family evidence state → retained control outcome → related retained family findings. Every part of that chain comes from the selected scan's own retained records, so a later catalogue change cannot reinterpret an older scan.

What a traceability row shows

  • Control key, title, domain and category
  • Coverage type and mapped scanner families
  • Retained family evidence state
  • Retained control outcome
  • Retained family finding count
  • Degraded-source flag and sanitised cause
  • Limitation and safe next action

What mapping metadata is not

  • No numeric or High/Medium/Low confidence score
  • Coverage type describes how far available evidence can assess the mapped question. It is neither confidence nor the selected evidence detail level.
  • Coverage and Top Issues/Balanced/Everything are different axes
  • Family rank orders multiple families — it is not a score

Mapping metadata fields

  • Coverage type
  • Scanner-family key
  • Family rank (ordering)
  • Optional limitation code
  • Mapping revision

Findings are joined, not stamped: a finding carries its own identity and scanner-family relationship; the SBS control separately retains the mapped scanner-family reference. Traceability joins the two through the shared scanner-family key. It is accurate to say retained findings can be related to retained SBS controls through that relationship — not that every finding carries an SBS control ID.

Coverage and outcome tiles

Coverage tiles, including Partial Evidence, remain separate from control-outcome tiles and filters.

Control table

Control, coverage, outcome, mapped scanner families and retained family finding count in one row per control.

Family-state detail

Degraded cause, limitation and safe next action for each mapped scanner family.

Family-scoped, not control-exclusive: the retained family finding count is a family-scoped evidence count, not a claim that every finding is uniquely attributable to one SBS control.

Export controls

Each surface exports a fixed allow-list

There is no single universal retained-evidence CSV schema — each evidence surface exports the fixed allow-list of display fields appropriate to it. The current SBS control-traceability export uses its own fixed set of columns, and the shared JavaScript export helper applies the current sanitisation controls to the enumerated helper-backed export paths.

SBS control traceability export — 14 columns

Scan Number, Retained Registry Key, Retained SBS Version, Retained Mapping Revision, Control Key, Control Title, Domain, Coverage Type, Mapped Scanner Families, Retained Family Evidence State, Retained Control Outcome, Retained family findings, Degraded Source, Sanitised Cause.

Spreadsheet formula-trigger mitigation

In the reviewed JavaScript export helper, every header and cell is quoted, embedded quotes are doubled, and tabs, carriage returns and line feeds are flattened to spaces. If a cell begins with =, +, - or @, an apostrophe is prefixed. This applies to the reviewed export helper paths and the enumerated trigger characters. It is not a claim that every spreadsheet-injection technique or every export path is covered.

Sensitive-value boundary

The reviewed helper-backed export paths exclude raw Salesforce IDs, raw queries, stack traces, tokens, session identifiers, credentials and certificate bodies. Raw IP values are omitted or redacted; exact retained-data shape is validated per environment.

Comparison boundary

Current claims stop at supported source evidence

Completed scans from the same source Salesforce organisation can support comparison. Runtime terminology-version stamping and terminology-version comparison gating are future behaviour unless and until implementation evidence proves them; this site does not claim they are current.

Evidence detail level

Top Issues, Balanced and Everything describe retained detail. Any like-for-like comparison claim must be validated against the release build.

History is not rewritten

A terminology contract change does not by itself rewrite retained historical evidence or prove runtime version metadata.

Future comparison gate

Terminology-version comparison gating remains future until source evidence establishes scan stamping and enforced compatibility behaviour.