0.1.0 shipped a LoadGuard that reported a block on a payload it had already run.
The fix is 4166e604; this publishes it.
While bumping: the publish workflow ran five of the seven suites, and
load_guard_test.rb was not one of them. The suite holding the regression test
for the bug being released was invisible to the pipeline releasing it. Added it
and psych/inspector_test.rb, so CI now runs all seven.
Local gate on the bumped version: 79 PASS, 0 FAIL.
Five documents per repository convention.
01-CONCEPTS.md opens with the Equifax debunk, because leading with a correction
is the strongest available demonstration that this repo checks its claims.
CVE-2017-5638 is OGNL expression injection, NVD CWE-755, CISA KEV CWE-20. Four
primary sources opened and text-searched (GAO-18-559, House Oversight, Equifax's
own release, the DOJ indictment) and "deserialization" appears in none of them.
The ASF published the correction 2017-09-14 and it lost to a three-day timing
coincidence with CVE-2017-9805. The twelve-item wrong-claims table closes the
chapter.
Honest impact numbers only: 69 of 1,653 CISA KEV entries are CWE-502, seventh
most common, 34.8% with known ransomware use against a 20.1% baseline. No
aggregate dollar figure, because none is credible.
Every transcript in the track was produced by execution, not recall. That
included discovering that ERB#def_method does not prepend its wrapper, it
inserts before the first non-comment line, so the payload's leading "#" is
forging the magic encoding comment a real compiled template carries. The first
draft described the wrapping wrong and the container corrected it.
README: the scanner figures were pre-taxonomy-fix and stale. Re-measured on
ruby:4.0-slim today: 124 ungated to 28 reachable, 142 unanalysable, not
119/29/135. Test and gate counts moved to 268 and 79, both measured rather
than carried forward.
LoadGuard#owner_name resolved the receiver with `receiver.is_a?(Module) ?
receiver : receiver.class`. The receiver is the gadget. A method-erased proxy
answers .class and .is_a? through method_missing, and method_missing is exactly
what the shipped erb-def-module chain enters through, so identifying the object
fired the chain it was about to veto.
TracePoint does not trace a handler's own nested calls, so that detonation was
invisible to the guard as well as unguarded by it. The observable result was a
guard reporting a block on a payload that had already written its canary:
strict guard: blocked -> deserialization hook (class with no name)#method_missing
canary created? true
"(class with no name)" was the tell: receiver.class had been answered by
method_missing, which returned the anonymous Module that ERB#def_module builds.
Resolve identity through Object#class, Object#is_a? and Module#name unbound and
bind_call'd onto the receiver, so nothing dispatches to it. Both hook sets now
veto with the canary absent and the real owner named.
Side effect, and it is the stronger behaviour: Module#name read unbound means a
class that overrides .name to raise is now named truthfully instead of reported
anonymous. test_an_owner_that_refuses_to_name_itself_fails_closed asserted the
old outcome and is rewritten to assert the new invariant.
test_the_guard_never_dispatches_a_method_on_the_receiver_it_inspects pins it,
and it isolates: instrumenting a wiped proxy shows [] against the fix and
[:is_a?, :class] against the revert.
Two independent audits of the same tree, one executed and one static. Both were
worth running: the static pass found seven real defects the executed pass missed,
including the worst one here, and the executed pass found three the static pass
could not see because seeing them required running Ruby.
The release workflow could publish from any branch. The publish job carried no ref
condition and its tag check read `[ tag != expected ] && [ event = push ]`, so on a
workflow_dispatch the second clause was false, the && never fired, and control fell
straight through to rubygems/release-gem. Anyone with the Actions tab could ship a
mutable branch checkout to rubygems.org. The job is now gated on a pushed
refs/tags/marshalsea-v* ref, the version check is unconditional, and the manifest is
audited before the push step with a negative control proving a drifted lib file
turns it red.
Three payloads ran attacker code while the detector reported proceed. A
String-subclass hash key reaching a user #eql?, measured {eql: 1}; a gadget nested
in a bare Array key, measured {hash: 1}, 22 bytes hand-built; and a Range whose
endpoints dispatch #<=>, measured {cmp: 1}. All three were accepted under
deny_sinks_only and under strict with the class allowlisted, which is the documented
normal usage. They are blocked under every policy now.
The oracle that missed them dumped `{ key => nil }`. One key means no bucket
collision, so #eql? could never fire in the probe no matter how many key shapes were
added. Blind by construction, the exact defect class this project already had a rule
about. The corpus went further and asserted the String-subclass case was a precision
control, a positive claim that rejecting it would be a false positive. It is a
reject now, and the key rules are re-derived from research 02 section 4.2 rather than
grown case by case.
Range endpoints marshal as bare `begin`/`end`, not `@begin`/`@end`, because Range
uses a marshal compat dumper. The first constant was wrong and the test caught it.
The scanner's reachability filter contradicted its own thesis. Requiring zero arity
for ungated entry points excluded eql?, ==, <=>, []=, method_missing and
respond_to_missing? entirely: 85 candidates across those rows, 0 reachable. Only
hash and to_s survived, and research 02 section 4.2 verified to_s is never an entry
point, so 11 of 18 reachable results were a method Marshal.load does not invoke.
Entry points are now a table carrying gate, format and the arity the deserializer
supplies; links are a third gate value and are reported separately instead of
scored as entry points. Gated hooks are arity-checked too, so an arity-0
marshal_load that would raise ArgumentError is no longer called reachable.
Marshal.load reaches a private self._load through rb_funcallv, which ignores
visibility, while singleton_methods(false) does not report it. Adding
singleton_class.private_instance_methods immediately found Time._load on a stock
image, a real stdlib sink the scanner had never seen. Prism is error tolerant and
parse_definitions consumed .value without checking failure?, so a tree recovered
from four syntax errors produced a confident touches_state verdict; it is a
suppression now.
The parser accepted ivar-name and struct-member-name slots holding a fixnum, a
string or an array, the detector said proceed, and CRuby then raised
ArgumentError, so the defended route answered 500. The parser stays forensic on
purpose, because a sink hidden where a symbol belongs must stay visible, so the
anomaly is labelled on Result and the detector rejects on it. The target also
rescues the loader rather than trusting inspection.
Three things the contract promised and did not have.
The headline payload was not a chain. It built an ERB object past the @_init guard
and then both demonstrations called def_method themselves, so the canary was not a
consequence of Marshal.load. Research 04 line 292 and 05 line 982 already said the
real chain reaches def_module through
ActiveSupport::Deprecation::DeprecatedInstanceVariableProxy, and 05 line 1143 listed
reproducing it as open. erb-def-module does that: the proxy sits in hash-key
position, ungated #hash dispatch lands in method_missing, target calls
@instance.__send__(:def_module), and ERB compiles the payload inside Marshal.load
with no application call. The old builder stays as erb-def-method and is labelled a
primitive. Three things only execution showed: the proxy undefines
instance_variable_set so setup has to go through a bound Object method; a real
deprecator holds a Proc and cannot be dumped, so the chain hand-builds one with
@silenced true and warn short-circuits before touching @behavior; and building
{proxy => 1} fires the payload in the builder's own process, so serialize splices a
key-position stream from a standalone dump and refuses any graph carrying an object
link, whose index would shift behind the hash node.
LoadGuard is the M6 runtime guard. A TracePoint on :call fires before a method body
runs, which is the veto a Marshal.load proc cannot give you. It watches the gated
hooks plus method_missing and respond_to_missing?, because a hook list without those
two is evaded by a respond_to_missing? proxy. hash and eql? are opt-in behind
strict:, since they are among the hottest methods in Ruby and BoundaryDetector
already catches key-position dispatch before any bytes load. It raises a
StandardError, never a SecurityError that would skip every rescue in the stack.
Its cost is not 1.4x. That figure is a property of the payload that was measured,
not of the guard. Enabling a TracePoint costs a near-constant ~46 microseconds per
load, so the ratio is decided by how much work the load does: 185.9x on 3 bytes,
40.4x on a 45-byte session cookie, 1.1x on 46 KB, 1.0x on 488 KB. The lab's own use
case is a session cookie, which is the worst case. A dated correction is written
back into research 03.
The Psych half exists now. Psych::Inspector reads a document through parse_stream,
revives nothing, and reports every !ruby/* tag with the method it would dispatch,
bounded on bytes, depth, nodes, aliases and documents. psych-init-with is the
matching chain. The target grows /yaml/unsafe and /yaml/safe so the spine of this
project is executable over HTTP: the same ERB object reaches code execution through
YAML.unsafe_load, and YAML.safe_load refuses it by checking the tag before revival.
The gate proves both layers independently, including a document the inspector
approves that Psych still refuses, so neither can alibi the other.
The target ran attacker Ruby on Docker's default bridge with outbound access and
installed sinatra, rackup and webrick unversioned. It now runs on an internal
network with cap-drop ALL, no-new-privileges, pid and memory ceilings and pinned
versions, with a control proving it cannot reach off the host. Creating that network
also proved --internal blocks the published port, so the gate drives the target from
a second container on the same network instead. The HTTP gate asserted body prefixes
and never captured status; it asserts exact status and body per endpoint now, which
immediately caught a bug in this very change where a nil sentinel conflated "the
loader refused" with the legitimate value nil.
Smaller: Chains.all filtered out a Base that was never registered, so the filter was
inert and its test vacuous; chains are discovered by directory glob now, per the
design's no-registry-to-rot contract. AFFECTED was shallow frozen, and mutating
metadata[:affected][2] flipped affects?("5.0.0") from true to false. Limits.permissive
keeps a depth cap on purpose and now says so, because lifting it trades a rescuable
DepthLimitError for an uncatchable SystemStackError. The README claimed a fixnum
width rejection its own test proves is unreachable. Regexp options were discarded
while the node still reported fully_decoded?.
The README is rebuilt to the repository's shape, and Deserialization Gadget Lab
takes project 41 in the root table, replacing Ghost on the Wire. CHANGELOG.md is
dropped from the gem manifest, the metadata and the packaging gate.
Full gate: 78 PASS, 0 FAIL across six stages, up from 58. 267 tests across seven
suites, from 194. Lint 0 across 37 files. Every rule added here ships with the
mutant that kills it.
rube has been on rubygems.org since 2009-08-05: Richard LeBer, 12,305 downloads,
and it is an ERB front-end, which is funny given the flagship CVE here is an ERB
gadget. The name was never publishable, so publishing required a rename first.
marshalsea. The Marshalsea was a London debtors' prison, 1373 to 1842, and the
name is the job description: hold untrusted objects at the gate and decide what
gets through before Marshal.load turns bytes into behaviour. It also carries
"Marshal", so the gem reads as on-topic without a subtitle.
module Rube is module Marshalsea, lib/rube/ is lib/marshalsea/, require "rube" is
require "marshalsea", RUBE_TARGET_PORT is MARSHALSEA_TARGET_PORT, and the canary
moved to /tmp/marshalsea-canary. 31 files, roughly 163 occurrences, every one a
hand edit. The single deliberate survivor is the README's "a Rube Goldberg
machine", which describes the gadget chain and not the gem.
Publishing is trusted publishing over OIDC, so no long-lived API key exists in
this repository to leak. A marshalsea-v* tag runs the five suites and the
standalone controls on Ruby 3.4 and 4.0, refuses to continue if the tag disagrees
with Marshalsea::VERSION or if the gemspec floor stops matching the tested
matrix, and then publishes with a Sigstore attestation. The attestation is
recorded as an auditable record and explicitly NOT as an install-time protection,
because neither gem install nor bundle install verifies one today.
Two things the primary source settled that the docs did not. rubygems/release-gem
does accept working-directory, which neither its README nor the RubyGems guide
mentions, so a monorepo subdirectory works. And it runs bundle exec rake release,
which this Rakefile had no task for at all.
Adding bundler/gem_tasks exposed a monorepo trap: Bundler::GemHelper tags a bare
v0.1.0, which says nothing about which of sixty projects it belongs to. Fixed
with tag_prefix. The catch is that rake -T still PRINTS "Create tag v0.1.0",
because that description is built when gem_tasks is required and the prefix is
assigned after. The tag actually created is marshalsea-v0.1.0. The label is
wrong and the behaviour is right, so the gate asserts the runtime value and
carries a control proving a Rakefile without the prefix line really does produce
the bare tag.
Full gate: 58 PASS, 0 FAIL across six stages, package now 25 of 25. 194 tests.
Lint 0 across 30 files.
required_ruby_version claimed ">= 3.3" while every gate stage ran on Ruby 4.0
images only. The claim was false. Marshal.load did not validate the bignum sign
byte until 3.4, so on 3.3 real Ruby accepts "!", "\x00", "\xFF" and "0" in the
sign position and reads them all as positive, where 3.4 and 4.0 raise
ArgumentError. The parser accepts "+" and "-" only, so it models 3.4+, and on
3.3 parser_test.rb goes red at its own liveness guard: the differential oracle
finds nothing rejected and says so instead of passing vacuously.
3.4.10 runs all five suites green at the same counts as 4.0 and prints ALL
CONTROLS PASSED. That makes 3.4 the oldest release actually proven, so the floor
is ">= 3.4". TargetRubyVersion moves with it, since those two must stay equal.
Teaching the parser two Marshal models to keep 3.3 was rejected. It buys a branch
in security maintenance only, and it pays with a second sign-validation path in
the one component whose whole job is modelling Marshal.load correctly.
The untracked rube-0.1.0.gem sitting in the repo root turned out to be built from
pre-B17 source: 12 lib files instead of 13, no float_body.rb, read_float still
using Float() with a bare rescue, no frozen_string_literal lines, declaring
">= 3.3". It installed and required without error, so nothing caught it. Two
artifacts with the same name and version and no way to tell them apart. Deleted.
package-gate.sh therefore asserts every shipped lib file is byte-identical to
the worktree rather than merely present, builds from the declared manifest alone
so an omitted file cannot produce a gem that builds anyway, installs the
artifact on the floor and current images and exercises it from the installed
copy, and re-proves the floor in both directions each run. Three negative
controls: a gem shipping the vulnerable target must be rejected, a gem with a
drifted lib file must be rejected, and RubyGems must refuse to install below the
declared floor. Aimed at the stale artifact it fails 6 of 23; on a fresh build it
passes 23 of 23. Both executed.
just build now writes to tmp/build as the invoking user instead of leaving a
root-owned gem in the tree, and just package audits an artifact you already have.
Full gate: 56 PASS, 0 FAIL across six stages. 194 tests. Lint 0 across 30 files.
B17 through B30, fourteen items. Every one reproduced before it was touched
and mutation-proven after. 194 tests from 119, rubocop 903 offenses to 0,
all six gate stages green.
One rule runs through all of it: nothing may present a guess, a default, or
a swallowed error as a verdict.
Decision states (B19). Under POLICY_OBSERVE_AND_LOG a payload carrying a live
Gem::Requirement#marshal_load snapshot reported accepted? true AND rejected?
false, so both obvious caller shapes loaded it. The predicate pair could not
express the third outcome, so there was no safe branch to pick - the prior
note claiming target/app.rb sidestepped this by branching on rejected? was
wrong, and both forms were byte-for-byte equivalent in outcome. accepted?,
rejected? and would_reject? are removed rather than redefined, so copying
`if d.accepted?` now raises NoMethodError instead of silently changing
meaning. One state validated in the constructor, three exclusive predicates,
and proceed? is the only one that may gate a Marshal.load. Observe-and-log
stays non-blocking and still hands back its snapshot; the monitoring caller
writes `proceed? || observed?` and names the state out loud.
Scanner error accounting (B18). Five rescues returned nil, [] or false and
told nobody. They now record site, subject and error class, and Report
exposes suppressed_count, suppressions_by_site, complete? and
candidates_lost?. Wiring the counter immediately surfaced 3 suppressions on
a stock image that had always been invisible: <internal:symbol>,
<internal:pathname_builtin> and <internal:ractor> all fail Prism.parse_file
with ENOENT because Ruby hands out those paths but they are not files.
The fourth state (B29). Those 3 suppressions were also 7 wrong answers -
candidates scored "does not touch state", indistinguishable from analysed
and inert, and 4 zero-arity ungated ones silently dropped from reachable.
touches_state is now four-state. A source that was given and could not be
parsed fails OPEN and stays reachable, because a scanner that discards what
it failed to analyse is the exact failure mode B18 names. A C-defined method
with no Ruby source at all is reported as unanalysable instead: 132 of 173
candidates, and failing open there would take reachable from 25 to 74 of 165
ungated and stop the filter filtering. Report#unanalysable and
#fully_analysed? state the real coverage - 33 of 173 - rather than implying
the filter saw everything. The 5 recovered candidates were verified by
executing them, not by reading source we could not read; one of them,
ERB::Compiler::PercentLine#to_s, is an alias of an attr_reader, which is a
second and distinct analysis gap.
Reason escaping (B20). Reject reasons interpolated raw attacker bytes into a
caller-supplied reporter. A class name carrying CR, LF, ESC and NUL turned
one reporter call into three log lines, the middle one forging a successful
authentication. All three interpolation sites now truncate at 96 bytes and
inspect the binary form, so no byte below 0x20 survives and the value is
quote-delimited.
Target hardening (B21, B22). The defended endpoint returned HTTP 500 with a
source line for three roots the detector had just accepted, leaking paths the
same way the B3 mutant did. show_exceptions is off, the shape is checked, and
the gate now greps every error body for source paths. ALLOWED_CLASSES could
never match anything, measured: a benign session cookie carries zero class
names. It is PERMITTED_CLASS_NAMES = [] now, which is what the app actually
requires and is strictly tighter - the old list admitted a C-wrapped String,
and under a mutant restoring it only the new shape check stopped that payload.
Gated agreement (B23). GATED_SINK_TAGS omitted TAG_DATA while the scanner
listed _load_data. The scanner was right, and this is now execution evidence
rather than a reading of marshal.c: a hand-built d stream naming Thread::Mutex,
a real C-level T_DATA, raises TypeError naming the missing _load_data, while
the same stream naming String dies earlier at "dump format error" - which is
why the previous attempt could not see it. A test compares both definitions
directly so they cannot drift again.
Float fidelity (B17). read_float returned nil for seven body forms
Marshal.load accepts, two more than the finding listed. Ruby uses its own
ruby_strtod, so "INF" is 0.0 while "inf" is Infinity, and String#to_f turns
out to be that same function. Ruby's legacy binary mantissa is NOT decoded:
a model fitted to four oracle points passed a 25-case table and then failed
930 of 5000 randomised cases, and marshal.c is not available in these images.
Since 0 of 209 Marshal.dump outputs contain a NUL, no living Ruby emits that
form, so the parser records the strtod prefix and flags Node#undecoded_tail
instead. A plausible wrong number is worse than a labelled non-answer.
Final differential: 2919 agreed exactly, 2081 flagged, 0 claimed-and-wrong.
Hygiene (B24, B25, B27, B28). PRIMITIVE_CLASS_NAMES and NAMESPACE_SEPARATOR
had one reference each, their own definition. BIGNUM_SIGN_POSITIVE is live
and stays. The width > FIXNUM_MAX_WIDTH guard is unreachable for all 256
possible marker bytes, checked exhaustively, and raised the wrong error class;
it is replaced by a test that derives widths from real Marshal.dump output.
The symlink and object-link bounds checks no longer lean on negative-index
wraparound. The parse graph is sealed before it is returned - every node, its
collections and its scalars frozen - and the whole suite stayed green first
try, which proves nothing downstream was mutating it. exploit-gate.sh pins
both sides now, 4.0.2-slim erb 6.0.1 FIRED against 4.0.6-slim erb 6.0.1.1
BLOCKED; the finding's claim about detector-gate.sh was wrong, it never had
a patched side.
Lint (B30). just lint used a --network none runner, so gem install could
never reach RubyGems, the && short-circuited, and the recipe exited 2 while
printing absolutely nothing. That is the fourth instance of a dropped return
value hiding a failure in this project. It is loud now. The config had never
been validated against a real run: 903 offenses, dominated by a quote style
the codebase does not use. frozen_string_literal was verified safe by running
the whole suite under RUBYOPT=--enable-frozen-string-literal BEFORE the change,
so ~247 offenses were retired by fixing code rather than silencing a cop.
Every remaining disabled cop carries a reason.
README's Status section listed the scanner, matrix, payload builder, target
and detector as "planned". All five have shipped. It also taught
Parser.new(blob) as the API, which was the unbounded form until this branch
made bounded limits the default.
Status now names all six components. Usage teaches the bounded default,
points at Limits.permissive for forensic parsing of a stream you already
trust, shows the detector alongside the parser, and points readers at
LIMITATION_NOTICE before they rely on an accept. Development lists all
eleven just recipes instead of five.
Every code example in the README was executed verbatim before this commit
and produces exactly the output it claims.
CHANGELOG gains Changed and Fixed sections covering the default-limits
change and the six defects closed on this branch.
Every item contracted to clear before M7 is closed. 151 tests from 119,
58 corpus cases from 48, all six gates green.
Depth accounting (B3, B4). TAG_IVAR charged no depth at all, so an I-chain
of any length parsed under any ceiling. Proven end to end against a rebuilt
target image: a 12,936-byte cookie returned HTTP 500 with a SystemStackError
that no rescue StreamError can catch, and a 724,287-byte response body
leaking absolute container paths for every file in lib/. Fixed, the same
cookie returns 400 DepthLimitError, and so does a 53,340-byte one.
read_userdef also hard-coded a depth of 1 for its class-name slot.
Budget axes (B12, B13, B14). Bignum magnitude bypassed the scalar budget
entirely and the sign byte accepted anything as positive where Marshal.load
raises ArgumentError. Added max_symbol_references, max_symbol_name_bytes,
max_class_name_bytes, max_instance_variables and max_struct_members.
Parser.new now enforces Limits.new instead of resolving to an unbounded
config; Limits.permissive became a class method.
Hash-key dispatch (B11). Nothing rejected an allowlisted class used as a
hash KEY, where #hash and #eql? run during load before any allowlist can
act. Measured against real Marshal.load: a key dispatches iff it carries a
class name and its underlying value is not a T_STRING. So TAG_REGEXP is not
a key-position risk and TAG_USERCLASS only conditionally - rejecting either
outright would have been a false positive. No opt-out allowlist was added,
because the dispatch happens before any check could run.
Fidelity (B15, B16, B9). The parser already matched Marshal.load on header
versions, so the 4.8 contradiction was resolved by giving the detector the
policy check and leaving the forensic parser permissive. Class-name slots
now accept only a symbol, an ivar-wrapped symbol, or a symlink. Wrapper
tags C and e no longer take an object-table slot, which Ruby does not give
them - link index 3 resolved to "bbb" for us and "ccc" for Ruby.
Gate soundness (B6, B7). Three discarded check() return values now register
as failures; section 6 no longer reports a vacuous 0/0; section 7 requires
reachable > 0, and prism absence is a named failure rather than a silent
zero. version-matrix.sh exits non-zero when any image produces no probe
result. control_check.rb no longer pulls in minitest, which was printing
"0 runs, 0 assertions" directly under ALL CONTROLS PASSED. Rewrote the
vacuous tests: the regexp options byte had zero minitest coverage and its
mutant survived the whole suite, and read_count's negative guard was
alibied by take's own guard.
The target app (B5) lost its hand-rolled copy of the sink-plus-allowlist
policy and now runs one BoundaryDetector with real limits, branching on
rejected? rather than accepted?.
Everything here is mutation-proven. Notable misses that mutation caught:
B14 had no test at all until reverting it stayed green, and a struct-member
test was vacuous on the first attempt because struct member names are always
symbols.
lib/rube/marshal/parser.rb carries eight backlog items at once and cannot be
split without interactive hunk staging, so this is one commit rather than
eight.
Duplicate ivar names deleted subtrees and five of six readers threw away the
class-name node, so a gadget in either position was invisible to the detector
while Marshal.load still fired it. Both had working proofs; the suite was green
the whole time.
The parse graph now has exactly one traversal owner. read_class_name and
read_instance_variables push every class-name node, ivar name and ivar value
into auxiliary, and Node#each no longer walks instance_variables_map. The map
stays as a lookup convenience with last-write-wins semantics, it just is not
load-bearing for security any more. Walking both would have double-counted
every ivar value.
Corpus entries take an optional allowlist. Without one every case ran through
an empty strict allowlist where any class name rejects, which is why the corpus
could not express B2 at all. The 40 existing cases default to [] and are
unchanged.
At the tag level there are seven class-name slots, not six: o S u U d C e, and
only o retained its node. u, U and d are themselves sink tags so a corpus case
there can never fail; those three are asserted at the parser level instead and
a test pins the exclusion as deliberate rather than an oversight.
Verified by mutation, since green means nothing on this project. Dropping the
class-name push, dropping the ivar value push, and restoring the map walk each
now fail 2, 4 and 3 tests. The first two previously survived the entire suite.
119 tests from 110, 48 corpus cases from 40. test, control, exploit, detector,
target and matrix all pass.
Forty explicit Marshal streams built from tag bytes rather than Marshal.dump,
so the corpus stays valid when Ruby changes what it emits. Thirteen must be
accepted, twenty-seven must be rejected across eight distinct error types.
The accept half is what gives the reject half meaning. A detector that
rejected everything would fail thirteen cases here, including legitimate
cycles, shared object references, symlink reuse, and fixnums at every encoding
width.
Provenance note, stated because it weakens the evidence: Codex was asked to
build this corpus and the request was blocked by its provider's content filter,
which reads constructing hostile streams as offensive tooling. The request was
not reworded to get around that. The corpus was therefore written by the same
model that wrote the detector, which is exactly the arrangement the split
exists to avoid, and it should be treated as weaker than the negative-count
findings Codex produced independently.
The boundary of what Codex can contribute here is now mapped. Defensive
research, packaging research, defensive design and acceptance criteria, and
probing the parser during a design task all succeeded. Researching gadget
chains and building a hostile corpus were both refused.
Also fixes a bug in the corpus fixnum encoder, which computed the width marker
as width plus 256 and raised RangeError for any positive multibyte value.
110 tests, 244 assertions across five suites.
Implements the detector Codex specified while it had no implementation to look
at, so neither model defined and graded the same thing.
Parser-level budgets rather than post-parse checks. Codex's architectural point
was that inspecting after parsing is too late because the allocation already
happened, so Limits and Budget enforce byte size, depth, node count, registered
objects, symbol definitions, collection entries, scalar bytes and object links
DURING recursive descent. The size ceiling is checked before the parser is
constructed and non-String input is rejected without ever calling to_s.
Three policies, STRICT_ALLOWLIST as the default, on the reasoning that people
keep defaults far longer than they intend. No enforcing mode accepts an
allowed_sinks option, because permitting a class-and-sink pair still authorizes
a callback during load. OBSERVE_AND_LOG refuses to construct without a
reporter. Allowlisting a class does NOT exempt its sinks, and that is a test.
No method is named safe?, trusted?, sanitized? or safe_load, and a test asserts
their absence. Those names claim a guarantee this cannot make.
The gate demonstrates the documented bypass rather than asserting it. Under
DENY_SINKS_ONLY the detector ACCEPTS the CVE-2026-41316 payload, and the gate
then loads that accepted snapshot on vulnerable erb and confirms the canary
fires. Our own detector, in a shipped mode, admits a payload that achieves code
execution. That is the limitation notice being true rather than decorative, and
if it ever stops being demonstrable the gate fails.
The notice ships verbatim and names the bypass concretely: a payload carrying
no sink tag can still reach dangerous code, the published chain produces zero
sink tags because ERB defines no marshal_load, and an application that
allowlists ERB will accept it.
106 tests, 235 assertions across four suites. Five gates: check, matrix,
exploit, detector, target.
Codex, working the defensive half without having written the parser, probed M1
with adversarial input and found defects my own tests missed. Verified
independently before fixing, and two more were found while confirming:
negative_array_count ACCEPTED as an empty array
negative_hash_count ACCEPTED as an empty hash
negative_bignum_words NoMethodError leaked outside StreamError
negative_ivar_count ACCEPTED
negative_string_len cursor moved BACKWARD, wrong error raised
The last is the worst. take(-5) does not trip the count > remaining guard, so
byteslice returns nil and @position decreases. A parser whose cursor can rewind
on attacker input is a loop primitive, not merely a wrong error.
The M1 gate claimed bignum length confusion was covered. It was not. Oversized
widths were tested and negative counts never were, because the same author
chose both the implementation and the cases it would face. That is the
negative-control failure one level up, and it is exactly what an author cannot
catch alone.
Fixes: every count and length now flows through read_count with a role label
and a nonnegative check, take rejects negative byte counts outright, and
MalformedCountError joins the StreamError hierarchy so nothing leaks a raw
NoMethodError. Negative link indices were already guarded; regression tests now
pin that.
Also closes a detection blind spot Codex identified. read_ivar and read_object
discarded the parsed name nodes after taking their values, so a sink tag placed
in an instance-variable-name position vanished from Result#sinks. Node now
carries an auxiliary collection that Node#each traverses, and name and class
nodes are retained. Proven: a stream with a userdef tag in the name position
now reports Evil#_load where it previously reported nothing.
46 parser tests, 80 assertions. All controls pass, exploit gate still passes.
Sinatra on webrick in a container, storing session state as a base64 Marshal
blob in a cookie. Three endpoints: /render deserializes and compiles the
session template, /render/safe inspects the stream first, /canary reports
execution. Runs read-only, unprivileged, with a 1MB noexec tmpfs, on a high
configurable host port.
Gate proves three things and the third is what stops the defense being a brick:
PASS HTTP request achieved code execution through Marshal.load
PASS defended endpoint rejected the identical payload
PASS defended endpoint still serves a legitimate session
Two findings that change the defensive design.
Sink tags do not catch this chain. The working payload produces ZERO sink-tag
hits. ERB defines no marshal_load, so it serializes as a plain object with
instance variables and carries no u, U or d tag. The defended endpoint rejected
it on the class allowlist, and an application that allowlisted ERB as a
legitimate template class would have passed it through untouched. A gadget does
not need a marshal_load hook, it needs an object whose ivars the application
later feeds to a dangerous method. The dangerous call site lives in the
application, not in the serialized class. Any policy treating absence of sink
tags as safe is defeated by this exact public payload.
A legitimately initialized ERB cannot be serialized at all. @_init holds
self.class.singleton_class and Marshal raises TypeError: singleton class can't
be dumped. So the guard is not a flag an attacker might satisfy, it is anchored
to a value the serializer physically cannot reproduce. Any ERB an attacker can
serialize necessarily lacks a valid @_init. The generalized pattern for learn/:
do not validate the untrusted object, anchor trust to something unreachable
through the channel.
Also fixes a gate that skipped a control silently. The benign-session check
produced no output because POST with no body returns WEBrick LengthRequired,
and the script treated an empty result as nothing to test rather than as a
failure. It now fails loudly.
70 tests, 151 assertions, 0 failures. Target app excluded from the gem
manifest, verified at 0 files.
test_generate_returns_an_object_not_bytes errored on roughly 1 run in 12
under Minitest's random ordering.
assert_kind_of ERB, chain.generate evaluates the ERB constant before calling
generate, and generate was performing a lazy require of erb inside itself. If
that test ran before anything else had loaded erb, the constant did not exist.
The lazy require was the defect rather than the test. A require buried in a
method makes constant availability depend on call order for every caller, not
just this one. Moved to the top of the file.
Verified across 40 randomized runs, 0 unstable. The previous commit shipped
this suite showing 1 error and should not have.
Chain registry modelled on PHPGGC: the class is the chain identity, metadata
carries the CVE and its affected version ranges, generate returns an object
rather than bytes, and serialization is a separate step.
Ships the ErbDefMethod chain for CVE-2026-41316. Ruby 2.7.0 added an @_init
guard to stop Marshal.load code execution on ERB objects, and def_method never
checked it. def_module and def_class delegate to def_method, so the single
missing check exposed all three entry points for six years.
The payload is an ERB built by allocate with @src, @filename and @lineno set
and @_init deliberately absent. @src opens with a comment line and a bare end
so that the def wrapper def_method injects is closed before the payload runs,
which puts execution at eval time rather than at call time.
Gate proves both halves and neither alone is sufficient:
4.0.2-slim erb=6.0.1 outcome=FIRED predicted=FIRED
4.0-slim erb=6.0.1.1 outcome=BLOCKED predicted=BLOCKED
The prediction column is the load-bearing one. affects? evaluates the CVE
ranges encoded in the chain metadata against the erb version present in the
image, before the payload runs. Observed behaviour matched on both, so the
registry is making falsifiable claims rather than carrying documentation.
Exploit containers run with no network, a read-only root filesystem, a 1MB
noexec tmpfs and an unprivileged user. The parser also inspects the payload
and reports ERB without deserializing it, so the offensive and defensive
halves meet on the same artifact.
70 tests, 151 assertions across four suites.
Explanatory prose in puts statements is commentary living in code. Removed
from control_check.rb and render_matrix.rb, leaving facts and verdicts.
Controls now report as a uniform PASS/FAIL table with a single exit status,
which also makes them usable as a gate rather than something a human reads.
Records the resolution on the two third-party findings: both dropped, no
disclosure, no further investigation. Neither was independently verified, no
obligation attaches to unpublished observations, and the lab has a stronger
flagship in CVE-2026-41316. Kept as unverified leads in gitignored docs.
Walks the live class graph and classifies auto-invoked methods along the
gated/ungated dispatch axis. Marshal calls respond_to? before invoking
marshal_load and _load, while hash, eql?, <=>, []= and to_s are dispatched
blind, so a class can be dead as a Marshal entry point and live as a #hash
entry point.
Rediscovers Gem::Requirement#marshal_load, Gem::Version#marshal_load and
Gem::Specification._load with source locations, from reflection alone, with no
class names hardcoded anywhere in the scanner.
A new control asserts the M1 parser and the M3 scanner agree. The parser reads
bytes off a payload, the scanner walks the class graph, neither consults the
other, and every sink the parser finds in a real Gem::Requirement payload is
one the scanner independently located. Two routes to the same fact.
Raw output was unusable at 163 ungated candidates dominated by to_s and hash,
which nearly every class defines. The reachability predicate from the research
narrows that to methods that are zero-arity AND touch instance state, cutting
163 to 12 and keeping 7.4 percent. Gated sinks are always reachable.
Two corrections found while building the predicate:
RubyVM::AbstractSyntaxTree.of does not work on Ruby 4.0. It raises
'cannot get AST for ISEQ compiled by prism' because prism is now the default
parser. Any tool reaching for that API on modern Ruby is broken. Replaced with
Prism, which ships as a default gem.
The literal reads-its-own-ivars predicate is too narrow. Gem::Requirement#hash
calls the requirements attr_reader rather than @requirements, so an instance
variable check reports false on a method that plainly operates on instance
state. Widened to instance variable reads OR implicit-self calls.
55 tests, 112 assertions, 0 failures across both suites.
Probes six pinned Ruby images with no network and renders a compatibility
matrix for the deserialization gadget surface. Reproducible with just matrix.
The matrix carries three controls, because a table that reports one value
everywhere cannot be distinguished from a probe that always returns the same
answer. Two axes must show more than one state, and the ERB guard column is
cross-checked against the published CVE-2026-41316 affected ranges. That third
control is the load-bearing one: the probe reads source and knows nothing about
NVD, and it agrees with the advisory on 6 of 6 images.
Findings recorded in the research docs:
The gadget surface moved rather than shrank. Net::WriteAdapter is reachable at
baseline on Ruby 3.1 and 3.2 and gone from 3.3 onward, which is why vakzz-era
chains needed no preloaded net/http on old Ruby. Gem::URI appears in the same
release that took it away, and Gem::URI reached through an autoloaded
Gem::SpecFetcher is exactly the bootstrap the 2024 chain relies on. One door
closed and another opened in the same version, so a defense reasoning about
the known gadget classes is reasoning about a moving target.
The ERB @_init guard sits only on def_method. def_module and def_class delegate
to it in both vulnerable and patched releases, so one check covers all three.
The first probe measured all three independently and reported the delegates as
unguarded even on patched erb 6.0.1.1. That was the probe being wrong, not the
patch being incomplete, and it is corrected here.
Marshal stream format is 4.8 on every image, so the M1 parser applies across
the whole range unchanged. make and git are absent from every slim image, so
rake is the only exec binary present on that family.
Scaffolds the Ruby deserialization security lab and lands its defensive core
first: a parser that extracts structure, referenced class names, and gadget
sinks from a Marshal stream without ever calling Marshal.load.
Sinks are classified along the gated/ungated dispatch axis. Marshal checks
respond_to? before invoking marshal_load and _load, while hash, eql?, <=> and
[]= are dispatched blind, so the same class can be dead as a Marshal entry
point and live as a #hash entry point.
Object links are ZERO-indexed. Ruby's Marshal format documentation says
one-indexed and is wrong: a self-referential array dumps as 04 08 5b 06 40 00
with the trailing 00 linking to the outermost object. Written against observed
bytes rather than the docs.
Validation rejects truncated streams, unsupported version bytes, unknown type
tags, out-of-bounds object links and symlinks, oversized fixnum widths,
trailing bytes, and nesting past a configurable depth limit.
A negative-control script accompanies the suite and caught a test that was
passing vacuously: the TracePoint oracle watched :c_call, but Marshal.load is
a Ruby-level method in Ruby 4.0 (<internal:marshal>:33) and fires :call, so
the test could never have failed. The suite now asserts the oracle observes a
real Marshal.load before the negative assertion is allowed to mean anything.
Gem manifest is an explicit allowlist rather than git ls-files, so the
deliberately vulnerable target cannot be swept into a published gem later.
34 tests, 62 assertions, 0 failures. 52/52 corpus round-trip. gem build
--strict clean. All execution in ruby:4.0-slim with --network none.
Vertically wrap the long table-driven case literals and one Fatalf call to
satisfy the project's golines linter (max-len 80, reformat-tags), which the CI
Go lint job flagged. Formatting only; no test behavior change.
The lint job installs ruff unpinned (pip install ruff), which reached 0.15.22
and, under preview = true, promoted the rule-codes-in-selectors diagnostic that
rejects rule codes like E501/S101 in lint.ignore. This project was the only one
with preview enabled. Dropping preview keeps the code-based selectors valid on
every ruff version in play (0.15.1 pre-commit pin, 0.15.7, 0.15.22 CI) and
aligns it with the other Python projects. No rule coverage change.
Add the five-part learn/ folder (overview, concepts, architecture,
implementation, challenges) grounded in docs/research, with the
QR-from-ISO/IEC-18004 Reed-Solomon injection as the showpiece and crypha's
real capacities cited from the binary. Update the root README row from a
Python synopsis to the built Go project with Source Code and Docs links,
matching the nadezhda row.
RealIP only reads XFF when the immediate peer sits in TRUSTED_PROXY_CIDRS,
so an untrusted client can no longer spoof its source IP while a real client
behind a known reverse proxy still resolves correctly. Parses the
comma-separated list via a knadh/koanf ProviderWithValue callback (blank or
whitespace-only input leaves the built-in default intact), wires the var
through both compose files and .env.example, and adds MYSQL_FAKE_* and
TURNSTILE_SECRET env aliases. Covered by config_test.go.
Guided hide/reveal/capacity wizard in internal/tui as a pure view over internal/engine (no carrier logic): HCL gradient engine, live capacity meter, embed animation, and a secure-options form. Bare crypha on a TTY launches it; piped or --help prints help. Adds engine.Overhead and engine.EnvelopeSize for exact payload-fit preflight (flate pass only, no KDF).
Reimplements the QR internals needed for the covert channel from
ISO/IEC 18004: the function-module map, format-info parse, data mask,
zigzag placement, and error-correction block de-interleave, plus a
from-scratch Reed-Solomon decoder over GF(2^8) (syndromes, Berlekamp-
Massey, Chien search, and a Vandermonde magnitude solve). skip2 generates
the clean symbol; crypha introspects it and reuses none of its internals.
The payload is hidden as up to floor(t/2) correctable codeword errors per
block in the data region, leaving the error-correction codewords intact,
so any scanner's Reed-Solomon decoder self-heals to the cover and never
sees it. Reveal reads the module grid, RS-decodes each block itself, and
diffs the corrected data against the stego to recover the payload. EC
level is fixed at H; versions 1-10 auto-select by cover and payload size.
Capacity is tens of bytes, so an encrypted envelope (which exceeds it) is
rejected cleanly.
Differentially tested: the extracted codewords match skip2 as valid RS
codewords across all supported versions, and every stego still scans back
to the cover via gozxing at the full injection budget. skip2 is a runtime
dependency (the generator); gozxing is test-only. No toolchain bump; the
go directive stays 1.25.0.
M4 audio: 16-bit PCM WAV LSB carrier. Cover input is WAV or FLAC
(FLAC decoded via the mewkiz decoder); output is always 16-bit PCM
WAV. Native FLAC output is deferred. uint32 length-prefix framing
with overflow-safe bounds; in-memory WriteSeeker so the WAV encoder
can seek back and patch chunk sizes through the io.Writer interface.
M5 pdf: three techniques behind one carrier. Attachment (default,
pdfcpu embedded-file, lossless), metadata (base64url payload chunked
across custom Info-dict keys), and append-after-EOF (raw trailing
bytes, O(1) end-seek). Reveal auto-tries all three; technique
selection is exposed via New(Technique). The package disables the
pdfcpu config directory for hermeticity.
Both carriers self-register and are blank-imported in carrier/all.
No toolchain bump (go directive stays 1.25.0).
M2 image carrier: LSB embedding in PNG and 24-bit BMP covers via the
mandatory NRGBA conversion (avoids the premultiply LSB-corruption trap on
both encode paths), RGB-only with alpha untouched, uint32 length prefix,
paletted/16-bit/JPEG rejection, Capacity and Sniff, self-registering.
M3 text carrier: encrypted payload hidden as zero-width Unicode
(U+200B/U+2060) appended after cover text, magic + length framing that
survives incidental zero-width in the cover and NFC/NFD/NFKC/NFKD
normalization. Frame extraction consumes exactly to the end of the
carrier-bit stream so nested stego reveals the last-hidden layer.
New internal/carrier/all sentinel blank-imports carriers into the
registry. Adds golang.org/x/image/bmp and (test-only) golang.org/x/text.
CarterPerez-dev
dependabot[bot]