From 3aa2065d084d6a29492aaa15e822b5d17c3a4266 Mon Sep 17 00:00:00 2001 From: Dotta <34892728+cryppadotta@users.noreply.github.com> Date: Mon, 31 Aug 2026 02:19:31 -0500 Subject: [PATCH] feat(runner): validate structured question responses (#12420) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The Rust runner now owns a bounded ACPX session and a fail-closed turn lifecycle > - Provider questions pause a turn and must return structured answers to the same persisted question set > - JSON Schema validates the wire shape, but it cannot validate identifiers and constraints across two documents > - Unknown questions, invalid choices, and malformed custom answers must fail before any provider receives them > - This pull request adds only the package-local response validator and tests > - The benefit is a small trust boundary that later request-resolution code can use without changing production selection ## Linked Issues or Issue Description Refs #12419 ## What Changed - Validate `paperclip.question_response.v1` against its versioned JSON Schema. - Bound serialized responses to 768 KiB before validation. - Require answer identifiers to match the exact persisted question set. - Require answers for required questions and reject unknown question identifiers. - Enforce text, single-select, and multi-select answer modes. - Match the existing TypeScript numeric syntax, including decimal, exponent, hexadecimal, octal, and binary input. - Match ECMAScript trimming exactly, including BOM whitespace while rejecting Unicode NEL rather than inheriting Rust-specific whitespace behavior. - Enforce known options, custom-answer policy, text length, pattern, and numeric constraints. - Validate duplicate option IDs, inverted bounds, and dynamic patterns before answer lookup so malformed optional questions fail closed even when unanswered. - Match JavaScript UTF-16 code-unit length semantics for text constraints and the 100,000-unit response-field bound. - Preserve the public optional `recommended` question-option field in the versioned schema, generated schema bundle, and Rust validation path. - Return typed validation errors for malformed inputs without panics. - Export the validator from the Rust runner core. - Add table-driven tests for valid, mismatched, malformed, oversized, and numeric-boundary responses. - Document the package-local structured-response boundary. - Add `num-bigint` 0.4 and `num-traits` 0.2 as direct runner-core dependencies for exact arbitrary-length radix parsing and one-step JavaScript Number rounding; update only the package-local runner Cargo lockfile. - Do not change the repository PNPM lockfile, workflows, runnerd selection, server behavior, UI, or migrations. ## Verification - Replay base: `9a9fdf06ee4142f77427db30efccc4c43056f64b` (`master` after #12419 merged). - Exact replay head: `fad92b3fb348b66ddb10dde44b7b060e55c4fe96`. - Stable patch ID: `4d6ffbd519dd081f7ea530977cd965bd4569fc75`; this is the prepared two-commit delta plus the focused cross-language parity fix found during replay review. - The exact delta is 10 files, 712 additions, and 2 deletions, all in `packages/paperclip-runner`. - The package-local `packages/paperclip-runner/runner/Cargo.lock` records the two direct runner-core dependencies; their already-resolved versions and checksums are unchanged. - The question-set schema source, generated TypeScript schema bundle, and protocol manifest hash are updated together; the schema SHA-256 is `42b5441a3d388851dacb6e4500dfd4a17d878eded2e724228078b647e7440d3f`. - GitHub Actions run `33366812025`, attempt 2: **PASSED** on the exact replay head (23/23 jobs passed; a failed-job-only retry cleared one unrelated ACPX runtime-host timeout). - Greptile: **5/5** on the exact replay head with zero unresolved review threads; Superagent, Socket, Snyk, and contributor-trust checks also passed. - No local test result is claimed. GitHub Actions is the authoritative verification environment for this replayed revision. ## Risks - The validator compiles the embedded response schema for each submission. Responses are user-paced and bounded, so this keeps the slice simple without affecting a hot event path. - The persisted question set is the source of truth for identifiers and constraints. A malformed persisted set fails closed. - Numeric input follows the existing structured-question contract, including JavaScript-prefixed syntax. Optional whitespace-only answers are rejected instead of being treated as an omitted value. - Error messages identify the invalid field but do not include answer text. - No production path invokes this validator in this pull request. Request resolution remains the next slice. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex with GPT-5.6, agentic reasoning, tool use, and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --- packages/paperclip-runner/README.md | 4 + .../paperclip-runner/protocol/manifest.json | 2 +- .../protocol/schemas/question-set.schema.json | 3 +- packages/paperclip-runner/runner/Cargo.lock | 2 + packages/paperclip-runner/runner/Cargo.toml | 2 + .../runner/crates/runner-core/Cargo.toml | 2 + .../runner/crates/runner-core/src/lib.rs | 1 + .../runner-core/src/question_response.rs | 375 ++++++++++++++++++ .../runner-core/tests/question_response.rs | 320 +++++++++++++++ .../src/protocol/generated/schema-bundle.ts | 3 + 10 files changed, 712 insertions(+), 2 deletions(-) create mode 100644 packages/paperclip-runner/runner/crates/runner-core/src/question_response.rs create mode 100644 packages/paperclip-runner/runner/crates/runner-core/tests/question_response.rs diff --git a/packages/paperclip-runner/README.md b/packages/paperclip-runner/README.md index a4b60c0e32..7c91f9a034 100644 --- a/packages/paperclip-runner/README.md +++ b/packages/paperclip-runner/README.md @@ -103,6 +103,10 @@ then resumes the same verified persistent identity in a fresh generation. This prevents a late session-lifetime MCP callback from inheriting the next turn's event authority. +The Rust question-response validator checks the versioned response envelope +against the exact persisted question IDs, answer modes, options, required +answers, custom-answer policy, and text constraints before provider delivery. + Run the complete contract gate with: ```sh diff --git a/packages/paperclip-runner/protocol/manifest.json b/packages/paperclip-runner/protocol/manifest.json index c7a5b5f0c3..64caa6a3e1 100644 --- a/packages/paperclip-runner/protocol/manifest.json +++ b/packages/paperclip-runner/protocol/manifest.json @@ -65,7 +65,7 @@ { "path": "schemas/question-set.schema.json", "id": "https://paperclip.dev/schemas/prp/v1/question-set.schema.json", - "sha256": "3bda40311ae3153bbf9d4a8c59f716680e17d711a4db3570193963918359751c" + "sha256": "42b5441a3d388851dacb6e4500dfd4a17d878eded2e724228078b647e7440d3f" }, { "path": "schemas/request.schema.json", diff --git a/packages/paperclip-runner/protocol/schemas/question-set.schema.json b/packages/paperclip-runner/protocol/schemas/question-set.schema.json index 995bfa8f72..a1bec0e67f 100644 --- a/packages/paperclip-runner/protocol/schemas/question-set.schema.json +++ b/packages/paperclip-runner/protocol/schemas/question-set.schema.json @@ -23,7 +23,8 @@ "properties": { "id": { "type": "string", "minLength": 1, "maxLength": 160 }, "label": { "type": "string", "minLength": 1, "maxLength": 1000 }, - "description": { "type": "string", "maxLength": 4000 } + "description": { "type": "string", "maxLength": 4000 }, + "recommended": { "type": "boolean" } }, "additionalProperties": false }, diff --git a/packages/paperclip-runner/runner/Cargo.lock b/packages/paperclip-runner/runner/Cargo.lock index d5a2e4c120..c429d5a3c2 100644 --- a/packages/paperclip-runner/runner/Cargo.lock +++ b/packages/paperclip-runner/runner/Cargo.lock @@ -663,6 +663,8 @@ dependencies = [ "getrandom 0.3.4", "hmac", "jsonschema", + "num-bigint", + "num-traits", "serde", "serde_json", "sha2", diff --git a/packages/paperclip-runner/runner/Cargo.toml b/packages/paperclip-runner/runner/Cargo.toml index ef4df0e2f3..d2d049bda4 100644 --- a/packages/paperclip-runner/runner/Cargo.toml +++ b/packages/paperclip-runner/runner/Cargo.toml @@ -13,6 +13,8 @@ aes-gcm = "0.10" getrandom = "0.3" hmac = "0.12" jsonschema = { version = "0.50", default-features = false } +num-bigint = "0.4" +num-traits = "0.2" serde = { version = "1.0", features = ["derive"] } serde_json = "1.0" sha2 = "0.10" diff --git a/packages/paperclip-runner/runner/crates/runner-core/Cargo.toml b/packages/paperclip-runner/runner/crates/runner-core/Cargo.toml index df141ae78e..d34f35ef48 100644 --- a/packages/paperclip-runner/runner/crates/runner-core/Cargo.toml +++ b/packages/paperclip-runner/runner/crates/runner-core/Cargo.toml @@ -10,6 +10,8 @@ aes-gcm.workspace = true getrandom.workspace = true hmac.workspace = true jsonschema.workspace = true +num-bigint.workspace = true +num-traits.workspace = true serde.workspace = true serde_json.workspace = true sha2.workspace = true diff --git a/packages/paperclip-runner/runner/crates/runner-core/src/lib.rs b/packages/paperclip-runner/runner/crates/runner-core/src/lib.rs index 48dbd09ad8..05b3ea4045 100644 --- a/packages/paperclip-runner/runner/crates/runner-core/src/lib.rs +++ b/packages/paperclip-runner/runner/crates/runner-core/src/lib.rs @@ -14,6 +14,7 @@ pub mod process_supervisor; pub mod provider_backend; pub mod provider_bridge; pub mod provider_events; +pub mod question_response; pub mod replay; use std::error::Error; diff --git a/packages/paperclip-runner/runner/crates/runner-core/src/question_response.rs b/packages/paperclip-runner/runner/crates/runner-core/src/question_response.rs new file mode 100644 index 0000000000..0987d0dd7c --- /dev/null +++ b/packages/paperclip-runner/runner/crates/runner-core/src/question_response.rs @@ -0,0 +1,375 @@ +use std::collections::BTreeSet; + +use num_bigint::BigUint; +use num_traits::ToPrimitive; +use serde_json::{json, Value}; + +use crate::local_runner::LocalRunnerError; + +const MAX_QUESTION_RESPONSE_BYTES: usize = 768 * 1024; +const MAX_QUESTION_ANSWER_CODE_UNITS: usize = 100_000; + +/// Validates a provider-neutral response against the exact persisted question +/// set that produced it. The JSON Schema owns the wire shape; this function +/// adds cross-document identifiers, modes, required answers, and constraints. +pub fn validate_question_response( + question_set: &Value, + response: &Value, +) -> Result<(), LocalRunnerError> { + let bytes = serde_json::to_vec(response).map_err(|error| { + LocalRunnerError::invalid(format!("question response is not serializable: {error}")) + })?; + if bytes.len() > MAX_QUESTION_RESPONSE_BYTES { + return Err(LocalRunnerError::invalid( + "question response exceeds its bounded transport contract", + )); + } + let question_set_schema: Value = serde_json::from_str(include_str!( + "../../../../protocol/schemas/question-set.schema.json" + )) + .map_err(|_| LocalRunnerError::invalid("embedded question-set schema is invalid"))?; + let question_set_validator = jsonschema::validator_for(&question_set_schema) + .map_err(|_| LocalRunnerError::invalid("embedded question-set schema cannot compile"))?; + if !question_set_validator.is_valid(question_set) { + return Err(LocalRunnerError::invalid( + "persisted question set failed the Paperclip question-set schema", + )); + } + let schema: Value = serde_json::from_str(include_str!( + "../../../../protocol/schemas/question-response.schema.json" + )) + .map_err(|_| LocalRunnerError::invalid("embedded question-response schema is invalid"))?; + let validator = jsonschema::validator_for(&schema).map_err(|_| { + LocalRunnerError::invalid("embedded question-response schema cannot compile") + })?; + if !validator.is_valid(response) { + return Err(LocalRunnerError::invalid( + "response failed the Paperclip question-response schema", + )); + } + + let questions = question_set + .get("questions") + .and_then(Value::as_array) + .ok_or_else(|| LocalRunnerError::invalid("persisted question set is malformed"))?; + let answers = response + .get("answers") + .and_then(Value::as_object) + .ok_or_else(|| LocalRunnerError::invalid("question response answers are malformed"))?; + let question_ids = questions + .iter() + .filter_map(|question| question.get("id").and_then(Value::as_str)) + .collect::>(); + if question_ids.len() != questions.len() { + return Err(LocalRunnerError::invalid( + "persisted question set has invalid or duplicate ids", + )); + } + for question in questions { + validate_persisted_question(question)?; + } + if answers.keys().any(|id| !question_ids.contains(id.as_str())) { + return Err(LocalRunnerError::invalid( + "question response contains an unknown question id", + )); + } + + for question in questions { + let question_id = question + .get("id") + .and_then(Value::as_str) + .ok_or_else(|| LocalRunnerError::invalid("persisted question id is malformed"))?; + validate_answer(question, answers.get(question_id))?; + } + Ok(()) +} + +fn validate_answer(question: &Value, answer: Option<&Value>) -> Result<(), LocalRunnerError> { + let question_id = question + .get("id") + .and_then(Value::as_str) + .ok_or_else(|| LocalRunnerError::invalid("persisted question id is malformed"))?; + let required = question + .get("required") + .and_then(Value::as_bool) + .ok_or_else(|| LocalRunnerError::invalid("persisted question requirement is malformed"))?; + let Some(answer) = answer else { + return if required { + Err(LocalRunnerError::invalid(format!( + "question response is missing required answer {question_id}" + ))) + } else { + Ok(()) + }; + }; + let answer = answer + .as_object() + .ok_or_else(|| LocalRunnerError::invalid("question response answer is malformed"))?; + let selected = answer + .get("selectedOptionIds") + .and_then(Value::as_array) + .map(|values| { + values + .iter() + .map(|value| { + value.as_str().ok_or_else(|| { + LocalRunnerError::invalid("question response option id is malformed") + }) + }) + .collect::, _>>() + }) + .transpose()?; + let text = answer.get("text").and_then(Value::as_str); + let custom = answer.get("customText").and_then(Value::as_str); + if [text, custom] + .into_iter() + .flatten() + .any(|value| javascript_string_length(value) > MAX_QUESTION_ANSWER_CODE_UNITS) + { + return Err(LocalRunnerError::invalid(format!( + "question response answer {question_id} exceeds its text bound" + ))); + } + let has_value = selected.as_ref().is_some_and(|values| !values.is_empty()) + || text.is_some_and(|value| !value.trim_matches(is_ecmascript_whitespace).is_empty()) + || custom.is_some_and(|value| !value.trim_matches(is_ecmascript_whitespace).is_empty()); + if !has_value { + return Err(LocalRunnerError::invalid(format!( + "question response answer {question_id} is {}", + if required { "required" } else { "empty" } + ))); + } + + match question.get("answerMode").and_then(Value::as_str) { + Some("text") => { + if selected.as_ref().is_some_and(|values| !values.is_empty()) || custom.is_some() { + return Err(LocalRunnerError::invalid(format!( + "text answer {question_id} cannot contain selection fields" + ))); + } + if let Some(text) = text { + validate_text_constraints(question_id, question, text)?; + } + } + Some("single_select" | "multi_select") => { + if text.is_some() { + return Err(LocalRunnerError::invalid(format!( + "select answer {question_id} cannot contain text" + ))); + } + let allowed = question + .get("options") + .and_then(Value::as_array) + .into_iter() + .flatten() + .filter_map(|option| option.get("id").and_then(Value::as_str)) + .collect::>(); + if selected + .iter() + .flatten() + .any(|option_id| !allowed.contains(option_id)) + { + return Err(LocalRunnerError::invalid(format!( + "select answer {question_id} contains an unknown option" + ))); + } + if question.get("answerMode").and_then(Value::as_str) == Some("single_select") + && selected.as_ref().is_some_and(|values| values.len() > 1) + { + return Err(LocalRunnerError::invalid(format!( + "single-select answer {question_id} chose multiple options" + ))); + } + if custom.is_some() + && question + .pointer("/customAnswer/enabled") + .and_then(Value::as_bool) + != Some(true) + { + return Err(LocalRunnerError::invalid(format!( + "select answer {question_id} does not allow custom text" + ))); + } + if question.get("answerMode").and_then(Value::as_str) == Some("single_select") + && selected.as_ref().is_some_and(|values| !values.is_empty()) + && custom + .is_some_and(|value| !value.trim_matches(is_ecmascript_whitespace).is_empty()) + { + return Err(LocalRunnerError::invalid(format!( + "single-select answer {question_id} cannot combine an option and custom text" + ))); + } + if let Some(custom) = custom { + validate_text_constraints(question_id, question, custom)?; + } + } + _ => { + return Err(LocalRunnerError::invalid( + "persisted question answer mode is malformed", + )) + } + } + Ok(()) +} + +fn validate_text_constraints( + question_id: &str, + question: &Value, + text: &str, +) -> Result<(), LocalRunnerError> { + let Some(validation) = question.get("textValidation") else { + return Ok(()); + }; + let length = javascript_string_length(text) as u64; + if validation + .get("minLength") + .and_then(Value::as_u64) + .is_some_and(|minimum| length < minimum) + || validation + .get("maxLength") + .and_then(Value::as_u64) + .is_some_and(|maximum| length > maximum) + { + return Err(LocalRunnerError::invalid(format!( + "answer {question_id} violates its text length constraint" + ))); + } + if let Some(pattern) = validation.get("pattern").and_then(Value::as_str) { + let pattern_schema = json!({"type":"string","pattern":pattern}); + let validator = jsonschema::validator_for(&pattern_schema) + .map_err(|_| LocalRunnerError::invalid("persisted question pattern cannot compile"))?; + if !validator.is_valid(&Value::String(text.to_owned())) { + return Err(LocalRunnerError::invalid(format!( + "answer {question_id} does not match its required pattern" + ))); + } + } + if matches!( + validation.get("inputType").and_then(Value::as_str), + Some("number" | "integer") + ) { + let number = parse_javascript_number(text).ok_or_else(|| { + LocalRunnerError::invalid(format!("answer {question_id} must be numeric")) + })?; + if !number.is_finite() + || (validation.get("inputType").and_then(Value::as_str) == Some("integer") + && number.fract() != 0.0) + || validation + .get("minimum") + .and_then(Value::as_f64) + .is_some_and(|minimum| number < minimum) + || validation + .get("maximum") + .and_then(Value::as_f64) + .is_some_and(|maximum| number > maximum) + { + return Err(LocalRunnerError::invalid(format!( + "answer {question_id} violates its numeric constraint" + ))); + } + } + Ok(()) +} + +fn validate_persisted_question(question: &Value) -> Result<(), LocalRunnerError> { + let question_id = question + .get("id") + .and_then(Value::as_str) + .ok_or_else(|| LocalRunnerError::invalid("persisted question id is malformed"))?; + if let Some(options) = question.get("options").and_then(Value::as_array) { + let option_ids = options + .iter() + .filter_map(|option| option.get("id").and_then(Value::as_str)) + .collect::>(); + if option_ids.len() != options.len() { + return Err(LocalRunnerError::invalid(format!( + "persisted question {question_id} has invalid or duplicate option ids" + ))); + } + } + let Some(validation) = question.get("textValidation") else { + return Ok(()); + }; + if validation + .get("minLength") + .and_then(Value::as_u64) + .zip(validation.get("maxLength").and_then(Value::as_u64)) + .is_some_and(|(minimum, maximum)| minimum > maximum) + { + return Err(LocalRunnerError::invalid(format!( + "persisted question {question_id} has inverted text length bounds" + ))); + } + if validation + .get("minimum") + .and_then(Value::as_f64) + .zip(validation.get("maximum").and_then(Value::as_f64)) + .is_some_and(|(minimum, maximum)| minimum > maximum) + { + return Err(LocalRunnerError::invalid(format!( + "persisted question {question_id} has inverted numeric bounds" + ))); + } + if let Some(pattern) = validation.get("pattern").and_then(Value::as_str) { + let pattern_schema = json!({"type":"string","pattern":pattern}); + jsonschema::validator_for(&pattern_schema).map_err(|_| { + LocalRunnerError::invalid(format!( + "persisted question {question_id} has an invalid text pattern" + )) + })?; + } + Ok(()) +} + +fn javascript_string_length(value: &str) -> usize { + value.encode_utf16().count() +} + +fn parse_javascript_number(value: &str) -> Option { + let value = value.trim_matches(is_ecmascript_whitespace); + if value.is_empty() { + return Some(0.0); + } + for (prefixes, radix) in [(["0x", "0X"], 16), (["0o", "0O"], 8), (["0b", "0B"], 2)] { + if let Some(digits) = prefixes + .iter() + .find_map(|prefix| value.strip_prefix(prefix)) + { + if digits.is_empty() { + return None; + } + let digits = digits + .chars() + .map(|character| character.to_digit(radix).map(|digit| digit as u8)) + .collect::>>()?; + // JavaScript parses the entire prefixed integer exactly and rounds + // once when converting it to Number. BigUint's f64 conversion uses + // round-to-odd before the final nearest-ties-to-even conversion, + // preserving that behavior without intermediate digit rounding. + return BigUint::from_radix_be(&digits, radix).and_then(|number| number.to_f64()); + } + } + value.parse::().ok() +} + +fn is_ecmascript_whitespace(character: char) -> bool { + matches!( + character, + '\u{0009}' + | '\u{000a}' + | '\u{000b}' + | '\u{000c}' + | '\u{000d}' + | '\u{0020}' + | '\u{00a0}' + | '\u{1680}' + | '\u{2000}' + ..='\u{200a}' + | '\u{2028}' + | '\u{2029}' + | '\u{202f}' + | '\u{205f}' + | '\u{3000}' + | '\u{feff}' + ) +} diff --git a/packages/paperclip-runner/runner/crates/runner-core/tests/question_response.rs b/packages/paperclip-runner/runner/crates/runner-core/tests/question_response.rs new file mode 100644 index 0000000000..acbf3cfa3c --- /dev/null +++ b/packages/paperclip-runner/runner/crates/runner-core/tests/question_response.rs @@ -0,0 +1,320 @@ +use num_bigint::BigUint; +use paperclip_runner_core::question_response::validate_question_response; +use serde_json::{json, Value}; + +fn question_set() -> Value { + json!({ + "schema":"paperclip.question_set.v1", + "questions":[ + { + "id":"target", + "prompt":"Which target?", + "required":true, + "answerMode":"single_select", + "options":[{"id":"first","label":"First","recommended":true},{"id":"second","label":"Second"}], + "customAnswer":{"enabled":true} + }, + { + "id":"regions", + "prompt":"Which regions?", + "required":false, + "answerMode":"multi_select", + "options":[{"id":"east","label":"East"},{"id":"west","label":"West"}] + }, + { + "id":"notes", + "prompt":"Add notes", + "required":true, + "answerMode":"text", + "textValidation":{"minLength":2,"maxLength":5,"pattern":"^[A-Z]+$"} + }, + { + "id":"count", + "prompt":"How many?", + "required":false, + "answerMode":"text", + "textValidation":{"inputType":"integer","minimum":1,"maximum":3} + } + ] + }) +} + +fn valid_response() -> Value { + json!({ + "schema":"paperclip.question_response.v1", + "answers":{ + "target":{"selectedOptionIds":["first"]}, + "regions":{"selectedOptionIds":["east","west"]}, + "notes":{"text":"YES"}, + "count":{"text":"2"} + } + }) +} + +#[test] +fn accepts_answers_that_match_the_exact_question_set() { + validate_question_response(&question_set(), &valid_response()).unwrap(); + + let mut custom = valid_response(); + custom["answers"]["target"] = json!({"customText":"another"}); + validate_question_response(&question_set(), &custom).unwrap(); + + let mut javascript_numeric_syntax = valid_response(); + javascript_numeric_syntax["answers"]["count"] = json!({"text":"\u{feff}0x2\u{feff}"}); + validate_question_response(&question_set(), &javascript_numeric_syntax).unwrap(); + + let mut empty_custom_with_selection = valid_response(); + empty_custom_with_selection["answers"]["target"] = + json!({"selectedOptionIds":["first"],"customText":"\u{feff}"}); + validate_question_response(&question_set(), &empty_custom_with_selection).unwrap(); +} + +#[test] +fn rounds_large_prefixed_integers_like_javascript_number() { + let mut bounded_set = question_set(); + bounded_set["questions"][3]["textValidation"]["minimum"] = json!(1_152_921_504_606_847_200_u64); + bounded_set["questions"][3]["textValidation"]["maximum"] = json!(1_152_921_504_606_847_200_u64); + + for value in [ + "0x1000000000000081", + "0o100000000000000000201", + "0b1000000000000000000000000000000000000000000000000000010000001", + ] { + let mut response = valid_response(); + response["answers"]["count"] = json!({"text":value}); + validate_question_response(&bounded_set, &response) + .unwrap_or_else(|error| panic!("{value} should match JavaScript Number: {error}")); + } +} + +#[test] +fn matches_javascript_radix_overflow_midpoint() { + let mut unbounded_set = question_set(); + let validation = unbounded_set["questions"][3]["textValidation"] + .as_object_mut() + .unwrap(); + validation.remove("minimum"); + validation.remove("maximum"); + + let overflow = BigUint::from(1_u8) << 1024_usize; + // Number.MAX_VALUE is 2^1024 - 2^971. The midpoint to the + // non-representable 2^1024 sentinel is 2^1024 - 2^970. At the midpoint, + // nearest-ties-to-even selects the sentinel, which JavaScript exposes as + // Infinity; the immediately preceding integer still rounds to MAX_VALUE. + let infinite_midpoint = &overflow - (BigUint::from(1_u8) << 970_usize); + let largest_finite = &infinite_midpoint - BigUint::from(1_u8); + let below_overflow_but_infinite = &overflow - BigUint::from(1_u8); + + for (prefix, radix) in [("0x", 16), ("0o", 8), ("0b", 2)] { + let mut response = valid_response(); + response["answers"]["count"] = + json!({"text":format!("{prefix}{}", largest_finite.to_str_radix(radix))}); + validate_question_response(&unbounded_set, &response).unwrap_or_else(|error| { + panic!("the largest finite-rounding base-{radix} integer was rejected: {error}") + }); + + for value in [&infinite_midpoint, &below_overflow_but_infinite, &overflow] { + response["answers"]["count"] = + json!({"text":format!("{prefix}{}", value.to_str_radix(radix))}); + assert!( + validate_question_response(&unbounded_set, &response).is_err(), + "base-{radix} value that rounds to Infinity was accepted" + ); + } + } +} + +#[test] +fn treats_ecmascript_bom_whitespace_as_an_empty_required_answer() { + let mut unconstrained_set = question_set(); + unconstrained_set["questions"][2] + .as_object_mut() + .unwrap() + .remove("textValidation"); + + let mut bom_text = valid_response(); + bom_text["answers"]["notes"] = json!({"text":"\u{feff}"}); + assert!(validate_question_response(&unconstrained_set, &bom_text).is_err()); + + let mut bom_custom = valid_response(); + bom_custom["answers"]["target"] = json!({"customText":"\u{feff}"}); + assert!(validate_question_response(&question_set(), &bom_custom).is_err()); +} + +#[test] +fn rejects_present_empty_optional_answers() { + let mut optional_text_set = question_set(); + optional_text_set["questions"][2]["required"] = json!(false); + optional_text_set["questions"][2] + .as_object_mut() + .unwrap() + .remove("textValidation"); + let mut empty_text = valid_response(); + empty_text["answers"]["notes"] = json!({"text":"\u{feff}\u{2009}"}); + assert!(validate_question_response(&optional_text_set, &empty_text).is_err()); + + let mut optional_custom_set = question_set(); + optional_custom_set["questions"][0]["required"] = json!(false); + let mut empty_custom = valid_response(); + empty_custom["answers"]["target"] = json!({"customText":"\u{feff}\u{2009}"}); + assert!(validate_question_response(&optional_custom_set, &empty_custom).is_err()); +} + +#[test] +fn rejects_malformed_persisted_text_constraints() { + let malformed_constraints = [ + json!("not-an-object"), + json!({"minLength":"2"}), + json!({"maxLength":2.5}), + json!({"pattern":false}), + json!({"inputType":false}), + json!({"minimum":"1"}), + json!({"maximum":{}}), + ]; + for constraint in malformed_constraints { + let mut malformed_set = question_set(); + malformed_set["questions"][2]["textValidation"] = constraint.clone(); + assert!( + validate_question_response(&malformed_set, &valid_response()).is_err(), + "malformed text constraint unexpectedly passed: {constraint}" + ); + } +} + +#[test] +fn rejects_semantically_malformed_unanswered_questions() { + let cases = [ + ("duplicate option ids", { + let mut value = question_set(); + value["questions"][0]["options"][1]["id"] = json!("first"); + value + }), + ("inverted text length bounds", { + let mut value = question_set(); + value["questions"][2]["textValidation"]["minLength"] = json!(6); + value + }), + ("inverted numeric bounds", { + let mut value = question_set(); + value["questions"][3]["textValidation"]["minimum"] = json!(4); + value + }), + ("invalid text pattern", { + let mut value = question_set(); + value["questions"][2]["textValidation"]["pattern"] = json!("["); + value + }), + ]; + let response = json!({"schema":"paperclip.question_response.v1","answers":{}}); + for (label, mut set) in cases { + for question in set["questions"].as_array_mut().unwrap() { + question["required"] = json!(false); + } + assert!( + validate_question_response(&set, &response).is_err(), + "{label} unexpectedly passed without an answer" + ); + } +} + +#[test] +fn counts_text_lengths_like_javascript_utf16() { + let mut bounded_set = question_set(); + bounded_set["questions"][2]["textValidation"] = json!({"minLength":2,"maxLength":2}); + let mut response = valid_response(); + response["answers"]["notes"] = json!({"text":"😀"}); + validate_question_response(&bounded_set, &response).unwrap(); + + bounded_set["questions"][2]["textValidation"] = json!({"maxLength":1}); + assert!(validate_question_response(&bounded_set, &response).is_err()); +} + +#[test] +fn rejects_cross_document_and_answer_mode_mismatches() { + let cases = [ + ("missing required", { + let mut value = valid_response(); + value["answers"].as_object_mut().unwrap().remove("target"); + value + }), + ("unknown question", { + let mut value = valid_response(); + value["answers"]["other"] = json!({"text":"x"}); + value + }), + ("unknown option", { + let mut value = valid_response(); + value["answers"]["target"] = json!({"selectedOptionIds":["other"]}); + value + }), + ("multiple single selections", { + let mut value = valid_response(); + value["answers"]["target"] = json!({"selectedOptionIds":["first","second"]}); + value + }), + ("combined single selection", { + let mut value = valid_response(); + value["answers"]["target"] = + json!({"selectedOptionIds":["first"],"customText":"other"}); + value + }), + ("selection on text", { + let mut value = valid_response(); + value["answers"]["notes"] = json!({"selectedOptionIds":["first"]}); + value + }), + ("pattern mismatch", { + let mut value = valid_response(); + value["answers"]["notes"] = json!({"text":"no"}); + value + }), + ("numeric mismatch", { + let mut value = valid_response(); + value["answers"]["count"] = json!({"text":"4"}); + value + }), + ("invalid numeric syntax", { + let mut value = valid_response(); + value["answers"]["count"] = json!({"text":"0xGG"}); + value + }), + ("non-ECMAScript numeric whitespace", { + let mut value = valid_response(); + value["answers"]["count"] = json!({"text":"\u{0085}2\u{0085}"}); + value + }), + ]; + for (label, response) in cases { + assert!( + validate_question_response(&question_set(), &response).is_err(), + "{label} unexpectedly passed" + ); + } +} + +#[test] +fn rejects_malformed_or_oversized_response_envelopes() { + assert!(validate_question_response( + &question_set(), + &json!({"schema":"paperclip.question_response.v2","answers":{}}) + ) + .is_err()); + assert!(validate_question_response( + &question_set(), + &json!({ + "schema":"paperclip.question_response.v1", + "answers":{"notes":{"text":"x".repeat(800_000)}} + }) + ) + .is_err()); + let mut unconstrained_set = question_set(); + unconstrained_set["questions"][2] + .as_object_mut() + .unwrap() + .remove("textValidation"); + let mut code_unit_bounded = valid_response(); + code_unit_bounded["answers"]["notes"] = json!({"text":"😀".repeat(50_000)}); + validate_question_response(&unconstrained_set, &code_unit_bounded).unwrap(); + code_unit_bounded["answers"]["notes"] = json!({"text":"😀".repeat(50_001)}); + assert!(validate_question_response(&unconstrained_set, &code_unit_bounded).is_err()); +} diff --git a/packages/paperclip-runner/src/protocol/generated/schema-bundle.ts b/packages/paperclip-runner/src/protocol/generated/schema-bundle.ts index 62948faf9e..fcfabda1b3 100644 --- a/packages/paperclip-runner/src/protocol/generated/schema-bundle.ts +++ b/packages/paperclip-runner/src/protocol/generated/schema-bundle.ts @@ -2439,6 +2439,9 @@ export const questionSetSchema = { "description": { "type": "string", "maxLength": 4000 + }, + "recommended": { + "type": "boolean" } }, "additionalProperties": false