# Redaction rates of v1's engine (Redact with the baseline rules at run
# time, column v1) and of the current rules alone (column current), over the
# labelled corpus in corpus/ and the seeded generators of corpus_test.go.
# TestRates fails on any difference from what it measures. After a change to
# the rules, the corpus or the generators, rewrite this file with
#   go test ./internal/redact -run '^TestRates$' -args -rewrite-rates
# and commit it with the change.
#
# recall.<class>       the share of a class's secrets removed. A secret is
#                      kept when the output holds any 6 consecutive
#                      characters of it. Classes gen_* are generated.
# changed.<class>      the share of a class of negative cases changed at all.
# spans_replaced       the ranges replaced in every case, of the corpus files
#                      and generated.
# spans_on_a_secret    those of them that hold part of a labelled or
#                      generated secret.
# precision            spans_on_a_secret over spans_replaced.
# *.corpus             the same three, of the corpus files' cases only, whose
#                      negatives and code the generated cases outnumber.
# fact_loss_rate       the share of facts whose value the output no longer
#                      holds whole.
# hex32_kept_rate      the share of the 32-character hex keys after a trigger
#                      phrase (class gen_hex32) kept.
# oracle_violations    secrets, labelled or generated, that the baseline rules
#                      remove and the engine keeps (TestOracle).
# oracle_gap_violations
#                      those of them in the gap classes (*oracle_gap_*), the
#                      shapes recorded in corpus/oracle_gaps.txt, where the
#                      current rules alone keep on purpose code that the
#                      baseline rules remove, and so secrets in its shape.
#                      TestOracle fails on any other.
#
figure                                  n      v1      current
recall.audit_kept                       26     0.0769  0.0769
recall.auth_header                      6      1.0000  1.0000
recall.command_password                 8      0.8750  0.8750
recall.connection_string                4      1.0000  1.0000
recall.gen_hex32                        3000   0.6210  0.6210
recall.gen_oracle_gap_name_index        1000   1.0000  0.0000
recall.gen_oracle_gap_regex_value       1000   1.0000  0.6260
recall.gen_oracle_gap_type_annotation   1000   1.0000  0.1160
recall.gen_oracle_gap_yaml_alias        1000   1.0000  0.0000
recall.gen_oracle_near_name_index       21000  1.0000  1.0000
recall.gen_oracle_near_regex_value      4000   1.0000  1.0000
recall.gen_oracle_near_type_annotation  12000  1.0000  1.0000
recall.gen_oracle_near_yaml_alias       4500   1.0000  1.0000
recall.gen_password                     6000   0.9888  0.9888
recall.gen_provider                     5000   1.0000  1.0000
recall.gen_random_base32                1000   0.6690  0.6690
recall.gen_random_base62                1000   0.6570  0.6570
recall.gen_random_base64                1000   0.6820  0.6820
recall.gen_random_digits                1000   0.1110  0.1110
recall.gen_random_hex                   1000   0.4970  0.4970
recall.gen_random_urlsafe               1000   0.6900  0.6900
recall.jwt                              3      1.0000  1.0000
recall.key_value_json                   3      1.0000  1.0000
recall.key_value_query                  3      1.0000  1.0000
recall.key_value_shell                  4      1.0000  1.0000
recall.key_value_toml                   3      1.0000  1.0000
recall.key_value_yaml                   3      1.0000  1.0000
recall.oracle_gap_name_index            5      1.0000  0.0000
recall.oracle_gap_regex_value           5      1.0000  0.0000
recall.oracle_gap_type_annotation       5      1.0000  0.0000
recall.oracle_gap_yaml_alias            5      1.0000  0.0000
recall.oracle_near_name_index           96     1.0000  1.0000
recall.oracle_near_regex_value          12     1.0000  1.0000
recall.oracle_near_type_annotation      32     1.0000  1.0000
recall.oracle_near_yaml_alias           22     1.0000  1.0000
recall.private_key                      5      1.0000  1.0000
recall.prose_password                   3      0.3333  0.3333
recall.provider                         39     1.0000  1.0000
recall.random_near_trigger              6      0.6667  0.6667
recall.url_password                     6      1.0000  1.0000
changed.chat                            6      0.0000  0.0000
changed.code                            18     0.3889  0.1667
changed.docs                            4      0.0000  0.0000
changed.prose                           5      0.0000  0.0000
changed.reference                       5      0.0000  0.0000
changed.sha                             3      0.0000  0.0000
changed.uuid                            2      0.0000  0.0000
spans_replaced                          -      62290   59008
spans_on_a_secret                       -      61890   58612
precision                               -      0.9936  0.9933
spans_replaced.corpus                   -      299     275
spans_on_a_secret.corpus                -      276     256
precision.corpus                        -      0.9231  0.9309
fact_loss_rate                          18     0.8889  0.8889
hex32_kept_rate                         3000   0.3790  0.3790
oracle_violations                       65804  0       3278
oracle_gap_violations                   4020   0       3278
