feat(channels): add experimental iMessage Photon (#13299)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Channels connect external conversations to company tasks and agent execution. > - Slack, Discord, and AgentMail already provide durable delivery and access controls. > - People also need to reach an agent from Apple Messages and send photos. > - Photon provides shared Pro DMs, dedicated numbers, and authenticated event recovery. > - This pull request connects Photon to the existing channel services. > - People can message an agent while Paperclip retains task ownership and approval authority. ## Linked Issues or Issue Description **Subsystem affected** Cross-cutting: channel services, shared contracts, database constraints, Apps, and agent Channels UI. **Problem or motivation** Paperclip has no iMessage channel. A person cannot use Apple Messages to start a task, send a photo, or answer an agent's pending question. **Proposed solution** Add experimental **iMessage Photon** with Pro-compatible shared DMs or a dedicated Photon Cloud number per agent channel. Reuse channel admission, identity links, task generations, publication, and interaction continuation. Keep groups disabled for shared allocation. Dedicated lines support groups that an operator explicitly enables. Require a fresh linked message and a published agent response before setup completes. **Alternatives considered** Shared allocation has no owned phone number, so it reserves one project and allows DMs only. Dedicated allocation reserves one stable number. Local Mac access needs a separate deployment model. The upstream Photon Chat SDK adapter does not persist the poll mappings and send receipts required here. This change uses the lower-level SDK without adding another agent runtime. **Roadmap alignment** This extends Connected Apps and agent communication through the existing channel subsystem. It does not add a parallel tool connection or agent loop. GitHub searches for Photon and iMessage found no matching provider implementation. **Additional context** This ships behind the existing experimental channel gate. Dedicated-line release qualification remains incomplete. Real Photon Pro DMs passed task/reply, native poll, text answers, confirmation rejection, media, restart, pause, reconnect, revocation, and removal tests. An operator-supplied iPhone camera HEIC also passed the full round trip. Dedicated groups remain unqualified. See [the verification record](doc/connections/IMESSAGE-PHOTON-VERIFICATION.md) and [the implementation plan](doc/plans/2026-09-11-imessage-photon.md). ## What Changed - Add the provider catalog entry, shared setup contracts, and a forward migration. A global partial index reserves the dedicated number or shared project until its endpoint is archived. - Add Cloud project inspection, vaulted project credentials, selected-line token renewal, and a leased receiver. Persist checkpoint updates under the receiver lease. Shared project replay accepts sparse increasing sequences only after a complete recovery barrier. - Connect DMs and enabled groups to existing task generations, sender authorization, ordered delivery, and publication services. Keep each iMessage conversation on its task after completion; only explicit `/new` or `/close` releases the binding. Publish committed inbound comments live and label their human bubbles “Sent from iMessage” in both task-chat renderers. - Persist immutable text/file send identities, upload receipts, poll IDs, option IDs, per-person drafts, and canonical interaction continuation proofs. - Add source-bound file recovery, bounded HEIC/HEIF conversion, JPEG previews, and related Live Photo companion video retention. - Add the three-step setup flow and channel management surfaces with official branding. Preserve the experimental gate and existing pause/disconnect behavior. - Add interactive production-component Storybooks for setup, access, recovery, and ongoing conversations. Add provider, integration, catalog, and browser regression coverage. Document setup, recovery, supported boundaries, and qualification gaps. ## Verification - Live Photon Pro, SDK 2.1.0: linked iPhone messages create a task and receive native Codex replies in Apple Messages. Unlinked senders cannot start work. - Three real follow-ups each reopened the same completed task. Incoming bubbles appeared on its open page without reload and showed “Sent from iMessage.” The third follow-up ran after restarting the server on `4d7222110`; the agent correctly repeated its previous reply from before the restart. - Native polls after restart, sequential text drafts, required-field correction, explicit submission, approval rejection with a required reason, and native continuation passed against Photon. - PNG, text documents, synthetic HEIC, and a real iPhone camera HEIC passed in both directions. The camera photo produced a 3024×4032 JPEG preview. The native agent described it and returned the received HEIC byte-for-byte. - Pause/resume, reconnect, identity revocation, removal, `/status`, `/new`, `/close`, and stale answers after close passed live. Messages suppressed by pause did not become work on resume. Removal stopped intake and removed credential bindings. - All 304 focused tests passed on `4d7222110`. These cover Photon unit/integration behavior, both task-chat renderers, live comment hydration, completed-task continuity after restart, enabled groups, duplicate delivery, and explicit reset/close. The selected Teams completion-boundary regression also passed. Full workspace typecheck/build and token gates passed for the conversation fix; the final UI changes passed their affected typecheck/build and tests. - All 26 new Photon Storybook Playwright cases passed in light and dark themes, including the complete shared-DM setup journey and 390px mobile follow-ups. UI typecheck and the Storybook build passed. These stories use simulated Photon responses and do not replace the live evidence above. - The full chat-adapters browser suite previously passed all 39 cases. Migration checks passed, and migration 0275 applied to the isolated live instance with the earlier Photon migration already applied. - The local full Vitest run was previously interrupted by the host's embedded-Postgres shared-memory limit; it is not a full-suite pass. All 30 applicable CI checks passed on preceding head `7a5419cac`, with two skipped checks and Greptile 5/5. Head `24f8e1aae` adds an explicit required-story discovery guard to the 26 passing Storybook cases. Greptile rates this final head 5/5 with no unresolved review threads. All 30 applicable CI checks passed, with two optional checks skipped. - A repeated live send key suppressed the duplicate but returned gRPC 6 / SDK `internalError` without an original receipt. Paperclip keeps unknown delivery unresolved. This provider behavior is covered by a regression test. - See [the verification record](doc/connections/IMESSAGE-PHOTON-VERIFICATION.md) for package versions, redacted live evidence, deterministic coverage, and remaining qualification gaps. ## Risks - Dedicated group qualification remains unrun; groups are disabled for the approved Pro scope. Real iPhone camera HEIC passed transport, preview generation, agent inspection, and return. Keep the channel experimental; the dedicated-line release matrix remains incomplete. - Shared recovery and attachment aliases were verified against the live gateway. Duplicate writes currently return an error without the original receipt; unresolved sends require operator resolution. The implementation fails visibly on invalid replay ordering, a reset cursor, or changed identity. - The HEIF converter passed on macOS arm64 and in Linux CI. Windows HEIF binaries have not been executed in this work. Linux musl has no packaged converter. Unsupported conversion retains the original and reports the missing preview. - The migration adds a global reservation across companies for Photon numbers and shared projects. Paused and revoked endpoints keep that reservation until removal. - Integration touches shared channel services. Existing provider browser coverage passes; broad repository verification is recorded above. - `pnpm-lock.yaml` is intentionally excluded under repository policy. The repository bot owns lockfile updates. The additional Superagent supply-chain scan is neutral/inconclusive because these new dependencies are not yet in the committed lockfile. Its security scan passed; all required CI checks pass. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository inspection, code execution, browser testing, and tool use. The exact served model identifier and context-window size are not exposed in this session. No sub-agents were used. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Dotta
GitHub
parent
44f6312cd8
commit
4d317274ce
|
|
@ -161,6 +161,24 @@ Paperclip’s core identity is a **control plane for autonomous AI companies**,
|
|||
9. **Thin core, rich edges**
|
||||
Put optional chat, knowledge, and special surfaces into plugins/extensions rather than bloating the control plane.
|
||||
|
||||
### Experimental iMessage Photon channel
|
||||
|
||||
A Photon Cloud project can represent one agent through the existing
|
||||
experimental channel subsystem. DMs and explicitly enabled groups create or
|
||||
continue task-bound conversations. Linked sender identity is the default;
|
||||
telephone numbers, email addresses, names, and group membership do not grant
|
||||
Paperclip authority. Photos/files and ordinary questions/confirmations use the
|
||||
existing attachment, interaction, continuation, and publication contracts.
|
||||
Pause and Disconnect govern runtime behavior independently of the UI gate.
|
||||
Local Mac access, unsolicited conversations, and SMS/RCS
|
||||
fallback are excluded. Live qualification is required before release readiness.
|
||||
Pro shared allocation supports DMs only, with sender enrollment in Photon and
|
||||
separate identity linking in Paperclip. Shared channels reserve one project, not
|
||||
a pool phone number; group admission and publication are disabled. Dedicated
|
||||
allocation retains one selected number and individually enabled groups.
|
||||
|
||||
See [iMessage Photon](connections/IMESSAGE-PHOTON.md) for the implementation
|
||||
contract, setup, recovery, boundaries, and qualification status.
|
||||
### Experimental persistent agent conversations
|
||||
|
||||
Agent Chat is an opt-in core task presentation (`enableAgentChat`, off by default). Each person has one persistent task-backed conversation per agent and company, with ordinary company task visibility. The shared task composer, transcript, tools, files, and document panel remain the interaction surface. Agents clarify goals and hand substantial execution to linked, assigned tasks; a reply ends a turn without completing the conversation. `/new` starts fresh provider context in the same conversation while preserving visible history and artifacts. Healthy idle conversations wait for a message and do not count as unfinished execution work. See `doc/plans/2026-09-10-agent-chat.md` for the implementation contract.
|
||||
|
|
|
|||
|
|
@ -1617,6 +1617,30 @@ outcomes without a separate email composer. See
|
|||
[AgentMail connections](connections/AGENTMAIL.md) for setup, transports, recovery,
|
||||
authorization, and the API/CLI contract.
|
||||
|
||||
### Experimental iMessage Photon channel
|
||||
|
||||
A Photon Cloud project can represent one agent through the existing
|
||||
experimental channel subsystem. DMs and explicitly enabled groups create or
|
||||
continue task-bound conversations. Linked sender identity is the default;
|
||||
telephone numbers, email addresses, names, and group membership do not grant
|
||||
Paperclip authority. Photos/files and ordinary questions/confirmations use the
|
||||
existing attachment, interaction, continuation, and publication contracts.
|
||||
Pause and Disconnect govern runtime behavior independently of the UI gate.
|
||||
Local Mac access, unsolicited conversations, and SMS/RCS
|
||||
fallback are excluded. Live qualification is required before release readiness.
|
||||
Pro shared allocation supports DMs only, with sender enrollment in Photon and
|
||||
separate identity linking in Paperclip. Shared channels reserve one project, not
|
||||
a pool phone number; group admission and publication are disabled. Dedicated
|
||||
allocation retains one selected number and individually enabled groups.
|
||||
|
||||
iMessage task completion ends a turn, not its conversation. Subsequent messages
|
||||
reopen the same task, including after restart; only explicit `/new` or `/close`
|
||||
allows the next message to start another task. The open task receives committed
|
||||
inbound comments live, with “Sent from iMessage” attribution on user bubbles.
|
||||
|
||||
See [iMessage Photon](connections/IMESSAGE-PHOTON.md) for the implementation
|
||||
contract, setup, recovery, boundaries, and qualification status.
|
||||
|
||||
### Native task completion
|
||||
|
||||
For ordinary low-risk tasks, accept the current agent's structured `done` claim
|
||||
|
|
|
|||
19
doc/SPEC.md
19
doc/SPEC.md
|
|
@ -567,3 +567,22 @@ A paused task takes over the composer with an amber notice and a Resume action.
|
|||
Operators must release the effective task or ancestor pause before sending a new
|
||||
message. The draft stays intact. This applies to both task interfaces and to
|
||||
board comment API requests; an agent may still report interrupted work.
|
||||
|
||||
### Experimental iMessage Photon channel
|
||||
|
||||
A Photon Cloud project can represent one agent through the existing
|
||||
experimental channel subsystem. DMs and explicitly enabled groups create or
|
||||
continue task-bound conversations. Linked sender identity is the default;
|
||||
telephone numbers, email addresses, names, and group membership do not grant
|
||||
Paperclip authority. Photos/files and ordinary questions/confirmations use the
|
||||
existing attachment, interaction, continuation, and publication contracts.
|
||||
Pause and Disconnect govern runtime behavior independently of the UI gate.
|
||||
Local Mac access, unsolicited conversations, and SMS/RCS
|
||||
fallback are excluded. Live qualification is required before release readiness.
|
||||
Pro shared allocation supports DMs only, with sender enrollment in Photon and
|
||||
separate identity linking in Paperclip. Shared channels reserve one project, not
|
||||
a pool phone number; group admission and publication are disabled. Dedicated
|
||||
allocation retains one selected number and individually enabled groups.
|
||||
|
||||
See [iMessage Photon](connections/IMESSAGE-PHOTON.md) for the implementation
|
||||
contract, setup, recovery, boundaries, and qualification status.
|
||||
|
|
|
|||
|
|
@ -0,0 +1,286 @@
|
|||
# iMessage Photon verification
|
||||
|
||||
Date: 2026-09-11. Branch: `codex/imessage-photon`.
|
||||
Base inspected: `1c4bcff2b`; updated through master `ab15aff39`.
|
||||
Initial implementation checked: `7ada38eb7ef5dff5441f23c02131798b11d57712`.
|
||||
**Status: experimental; Pro shared-DM live journeys verified below. Dedicated groups and the remaining release matrix are not yet qualified.**
|
||||
|
||||
[PR #13299](https://github.com/paperclipai/paperclip/pull/13299) carries the current
|
||||
CI and review results. The Photon migration is `0275_easy_dragon_man.sql`, regenerated after master added its own 0274 agent-chat migration. Greptile reviewed the implementation commit at 5/5 with no
|
||||
actionable comments. This record distinguishes local evidence from live proof.
|
||||
|
||||
## Environment and versions
|
||||
|
||||
- Fresh worktree: `imessage-photon`; separate worktree configuration, instance,
|
||||
database/storage home, and application port 3109.
|
||||
- Browser tests use a disposable local-trusted instance on port 3319 with a new
|
||||
database and storage home. They mock the provider/control-plane responses.
|
||||
- Database integration tests use disposable embedded PostgreSQL with real channel,
|
||||
identity, task, attachment, publication, and interaction services.
|
||||
- Advanced SDK 2.1.0; grpc-js 1.14.4; nice-grpc 2.1.17; nice-grpc-common 2.0.4;
|
||||
heif2jpeg 0.1.6. Local converter execution: macOS arm64.
|
||||
- The synthetic HEIC fixture is generated from a solid-color 16×16 image. It has
|
||||
no personal photo content and does not qualify real iPhone HEIC/Live Photos.
|
||||
- Production credentials, line tokens, phone numbers, and participant identifiers
|
||||
are absent from this record. Test numbers/IDs in fixtures are synthetic.
|
||||
|
||||
The primary-instance seed attempt encountered existing source schema drift
|
||||
(`tool_connections_transport_check` missing), so the isolated worktree uses a clean
|
||||
instance. The primary database was not modified. Several test starts also reached
|
||||
macOS's 32-segment System V shared-memory limit. Only unattached IPC from this task's
|
||||
exited browser-test databases was eligible for cleanup; running instances were not
|
||||
stopped or altered.
|
||||
|
||||
## Deterministic acceptance evidence
|
||||
|
||||
`server/src/__tests__/photon/photon.test.ts` exercises Basic Cloud authentication,
|
||||
token redaction, dedicated/shared/missing allocation, immutable line identity,
|
||||
Unicode multipart publication, receipt recovery, unknown sends and explicit retry,
|
||||
upload receipt reuse, quota classification, per-part authorization, contiguous
|
||||
checkpoint recovery, ignored event frames, cutoff history, lease loss, real local
|
||||
gRPC framing/authentication, scoped state, duplicate-title poll IDs, poll creation
|
||||
before a local crash, answer parsing, source ownership, image bounds, and actual
|
||||
synthetic HEIC conversion.
|
||||
|
||||
`server/src/__tests__/photon/channel.integration.test.ts` composes the real channel
|
||||
service with the Photon adapter and synthetic provider responses. It proves the
|
||||
fresh linked-message/task/agent-publication setup requirement, restored DM reply,
|
||||
echo filtering, identity reservation, explicit group enablement, authorized poll
|
||||
resolution, per-person answer drafts, rejection reasons, exactly one canonical
|
||||
continuation record, delayed HEIC retry after restart, attachment provenance,
|
||||
quoted context, task generations, stale controls, retained pending input through
|
||||
pause, group removal, and a native continuation proof for a second group person.
|
||||
The checkpoint takeover test verifies the database lease and checkpoint update
|
||||
share one transaction.
|
||||
|
||||
The native continuation test caught a JSON key-order mismatch after JSONB storage.
|
||||
Both the recorded answer digest and reconstructed proof now use the existing
|
||||
canonical hash. This is a native authorization composition test, not evidence of
|
||||
a live model turn through Photon.
|
||||
|
||||
The Photon/OpenAPI follow-up also verifies safe setup credential, quota, network,
|
||||
and invalid-response errors, the complete board-only inspection contract, group
|
||||
participant response fields, and the unchanged credential binding after rejected
|
||||
replacement. All 39 Photon/OpenAPI tests and the server build passed after the
|
||||
review fix separating provider outages from invalid setup input.
|
||||
|
||||
The Photon browser cases in `tests/e2e/chat-adapters-ui.spec.ts` cover catalog
|
||||
discovery, multiple-line selection, password input, keyboard selection, vaulted
|
||||
credential payload shape, setup completion, group enablement, light/dark themes,
|
||||
mobile navigation/layout, and pause/resume. The surrounding suite covers existing
|
||||
Slack, Discord, GitHub, Teams, and Telegram surfaces.
|
||||
|
||||
| Check | Result |
|
||||
| --- | --- |
|
||||
| Photon targeted tests | 31 passed, including checkpoint takeover, Live Photo companion retention, and native continuation authorization. |
|
||||
| Token gates | Passed. All four gates clean. |
|
||||
| Workspace typecheck | Full `pnpm -r typecheck` passed before and after rebase. |
|
||||
| Full chat-adapters browser suite | 38 passed, including Photon light/dark/mobile coverage and existing providers. |
|
||||
| OpenAPI contract | 8 passed, including mounted-route completeness, board-only inspection, and token-free response schemas. |
|
||||
| Post-rebase channel/native checks | 96 passed across Photon, OpenAPI, explicit native continuation, and chat-control admission retry. |
|
||||
| Native session resume | 37 passed after building the required local fake-provider binary. |
|
||||
| UI Vitest project | 6,008 passed across 582 files after rebase. |
|
||||
| Shared catalog project | 727 passed, including exact catalog and branding coverage. |
|
||||
| Repository Vitest suite | The initial `pnpm test:run` overlapped edits/rebase and was stopped; it is not a final-commit pass. Fresh targeted and CI checks supersede it. The serialized route run found the missing Photon OpenAPI contract, which is fixed and passes its 8-case suite. Full gate status is recorded in the linked PR. |
|
||||
| Build | Full `pnpm build` passed before and after rebase. |
|
||||
| Generated forward migration | Generated through `pnpm db:generate`; `@paperclipai/db check:migrations` passed. Disposable database migrations exercised by integration tests. |
|
||||
| Native HEIF platform packages | macOS arm64 executed; other published platforms not executed. |
|
||||
|
||||
### Local test prerequisites
|
||||
|
||||
The standard `pnpm test:run` launcher isolates `PAPERCLIP_CONFIG`, `PAPERCLIP_HOME`,
|
||||
and temporary files. Direct heartbeat/continuation tests must use equivalent
|
||||
isolation; otherwise the worktree preview configuration suppresses execution.
|
||||
The actual runner-driver fixture also requires:
|
||||
|
||||
```sh
|
||||
cargo build --manifest-path packages/paperclip-runner/runner/Cargo.toml --bin fake-codex-app-server
|
||||
```
|
||||
|
||||
A run without that binary failed at provider startup; the complete 37-case native
|
||||
session-resume suite passed after building it. Catalog assertions were updated
|
||||
for the 42nd visible app, and the focused catalog/Browse/board-gallery tests pass.
|
||||
Some broad package runs encountered host embedded-Postgres startup limits during
|
||||
concurrent local development. These startup failures are not provider proof;
|
||||
inspect the linked PR for the current complete gate results.
|
||||
|
||||
## Pro shared-DM live qualification (2026-09-12)
|
||||
|
||||
The operator approved Pro-compatible shared DMs with groups disabled. The live
|
||||
test uses the isolated instance on port 3109, a Photon Pro project, its enrolled
|
||||
test participant, and the participant's actual iPhone. The test source was fully
|
||||
seeded through the worktree CLI; the primary instance remains untouched.
|
||||
|
||||
Observed with SDK 2.1.0 on implementation base `e556f7dbefd3ee738bde7830d69d5e30c4e96872`
|
||||
plus the shared-DM changes in this PR:
|
||||
|
||||
- Project inspection and vaulted setup succeeded against Photon Cloud's actual
|
||||
shared allocation. Shared credentials select the fixed shared gateway and a
|
||||
project-scoped identity, without inventing an owned phone number.
|
||||
- At 13:16 UTC, the participant sent a fresh iMessage from their iPhone. Photon
|
||||
delivered it through authenticated recovery. The project-filtered event feed
|
||||
jumped from an empty cursor to a non-adjacent sequence; the dedicated-only
|
||||
adjacency check initially stopped in Attention.
|
||||
- After the shared recovery fix and an isolated server restart, reconnect replayed
|
||||
the original message at 13:26 UTC. Paperclip discovered the exact sender but
|
||||
created no conversation/task while the identity was unlinked. The normal private
|
||||
confirmation flow then linked that identity to the isolated Board account.
|
||||
- At 13:27–13:28 UTC, the fresh linked request created a task, ran the native
|
||||
Codex runner, and delivered the requested response back to Apple Messages.
|
||||
- A native poll created at 13:29 UTC survived restart. Setup initially rejected
|
||||
interaction answers until the endpoint was active, deadlocking a clarifying
|
||||
question before final-reply qualification. Photon now permits those responses
|
||||
during its verified test step with the same identity, generation, and permission
|
||||
checks. After restart, a fresh vote at 13:33 UTC produced exactly one canonical
|
||||
answer and one native continuation. Its final reply arrived in Messages. A late
|
||||
unvote did not undo the answer. Setup then completed normally.
|
||||
- At 13:35–13:37 UTC, two free-text answers were collected sequentially. Early
|
||||
submission stayed pending; explicit submission of both drafts resumed the
|
||||
native agent with both exact values. The test also corrected the missing-answer
|
||||
hint to identify the unanswered question rather than always question 1.
|
||||
- At 13:38 UTC, the initial PNG/document import failed visibly because the shared
|
||||
gateway returns project attachment aliases in metadata and native UUIDs in
|
||||
stream headers. Shared downloads now retain authenticated source-message/chat
|
||||
checks and the exact alias-addressed RPC, validate the header metadata, and
|
||||
retain project aliases for provenance and restart. At 13:43–13:44 UTC, a fresh
|
||||
two-file message sent during the outage was recovered, imported, inspected by
|
||||
the agent, and returned as actual PNG and text-file attachments in Messages.
|
||||
The agent correctly identified the image and the document's verification word.
|
||||
- At 13:46–13:49 UTC, a canonical `request_confirmation` rejected a bare Reject
|
||||
reply with an actionable reason request. A correlated rejection with a reason
|
||||
resolved the canonical interaction and resumed the native agent, which returned
|
||||
the exact reason and confirmed that no further action ran.
|
||||
- At 13:49–13:50 UTC, a synthetic HEIC was sent through Apple Messages. Paperclip
|
||||
retained the 676-byte original and created a 633-byte JPEG derivative. The agent
|
||||
correctly described the solid blue 16×16 image and returned the original HEIC
|
||||
through Photon; the file appeared in Messages. This tests the real transport and
|
||||
converter together, but does not substitute for an actual iPhone camera photo.
|
||||
- At 13:51–13:53 UTC, Pause suppressed a delivered test message without creating
|
||||
a task. Resume did not replay it as work; a fresh request created the next task
|
||||
and received a reply. Reconnect reused the vaulted credentials and preserved
|
||||
project/allocation identity, then completed its fresh-message/reply test.
|
||||
- At 13:53–13:54 UTC, revoking the linked identity caused the next live message to
|
||||
be filtered with no task or agent run. The normal private confirmation flow
|
||||
restored the link. Completed tasks remained idle between fresh requests, and
|
||||
`/status` correctly reported no active task. `/new` requested a fresh message,
|
||||
and `/close` closed the next active conversation. Its late correlated answer
|
||||
left the old interaction unresolved and did not start another task.
|
||||
- At 13:56 UTC, Remove connection archived the test endpoint and its connection,
|
||||
cleared saved secret bindings, and stopped intake. A message sent while removed
|
||||
created no task. The same Photon project remained eligible in new setup.
|
||||
A replacement endpoint was linked normally and completed a fresh native
|
||||
task/reply test at 13:58 UTC. The test channel was left active.
|
||||
- At 18:13–18:14 UTC, an operator-supplied iPhone camera HEIC passed the same
|
||||
authorized Apple Messages conversation on code commit `fc4e4f0a3` (documentation
|
||||
head `a2a9319f3`). Messages transformed the 1,432,391-byte source into a
|
||||
1,132,602-byte HEIC before ingestion. Paperclip retained those received bytes
|
||||
and generated a 783,443-byte, 3024×4032 JPEG preview. The native agent correctly
|
||||
described the photo, then staged the HEIC with the same SHA-256 as the received
|
||||
original. Text and file publications each succeeded on their first attempt with
|
||||
provider receipts, and the returned photo appeared in Apple Messages. The native
|
||||
run succeeded and the task completed. The personal photo is not included in the
|
||||
repository or this report. This closes the real camera HEIC round-trip gap;
|
||||
Live Photo reassembly remains outside scope.
|
||||
- An identical published test send was repeated with its original key, exact
|
||||
payload digest, and reply target. Photon suppressed it but returned gRPC 6 with
|
||||
SDK `internalError` and an empty context, saying the operation was already
|
||||
processed. No new bubble appeared. Contrary to the documented original-result
|
||||
behavior, the shared gateway supplied no receipt. A regression test preserves
|
||||
delivery-unknown state in this case; no text matching or new key is used.
|
||||
- The shared receiver now commits only after the complete ordered replay barrier.
|
||||
Regression cases cover sparse events, interrupted/out-of-order replay, and cursor
|
||||
resets without advancing the saved checkpoint. Dedicated recovery remains strict.
|
||||
- All 39 chat-adapters browser tests passed, including shared setup after reload
|
||||
and existing provider coverage. The 20 Photon unit cases passed. An integration
|
||||
rerun initially hit the host's embedded-Postgres startup limit; this is a test
|
||||
environment failure, not a provider result.
|
||||
|
||||
The expanded unit suite has 22 passing cases, including shared attachment alias
|
||||
ownership, header validation, and missing duplicate receipts. After merging master
|
||||
and regenerating migration 0275, all 16 integration cases passed at code commit
|
||||
`fc4e4f0a32d35e41e56f6698404fca64cee3f32b`. Full workspace typecheck, build, token
|
||||
gates, and migration checks passed on that commit. The isolated instance then
|
||||
restarted successfully, reported startup ready on that commit, and retained the
|
||||
active shared-DM endpoint and linked identity. A broad `pnpm test:run` was started and stopped
|
||||
when the host's shared-memory limit prevented the live isolated PostgreSQL from
|
||||
restarting. Only this task's exited test database resources were removed. This
|
||||
interrupted run is not a full-suite pass; current CI must qualify the final commit.
|
||||
|
||||
All 30 applicable CI checks passed on `a2a9319f3`, with two skipped checks. One
|
||||
unchanged Cursor sandbox command-selection case initially exceeded its 10-second
|
||||
timeout. The exact case passed locally in 735 ms, and the failed CI server shard
|
||||
passed on its single rerun. Greptile rated that head 5/5 with no unresolved review
|
||||
threads. The 22 Photon unit cases also passed in Linux CI, including native HEIC
|
||||
conversion. Subsequent changes to this record add qualification evidence only;
|
||||
the linked PR shows their current check status.
|
||||
|
||||
Photon's CLI manages projects and users; its terminal provider simulates chat UI.
|
||||
Neither substitutes for actual Cloud iMessage delivery. The local Mac initially
|
||||
classified the assigned number as RCS, while the participant's iPhone sent the
|
||||
observed iMessage. No RCS/SMS fallback was enabled.
|
||||
|
||||
## Live qualification still required
|
||||
|
||||
Dedicated-line credentials were unavailable during the initial implementation.
|
||||
The Pro shared-DM journeys above passed; the remaining matrix must be completed
|
||||
before release readiness. Live inbound receipt alone is not full qualification.
|
||||
Record the tested commit, package versions, redacted project/line/chat IDs,
|
||||
participants, timestamps, and observable results when running it.
|
||||
|
||||
| Live case | Status |
|
||||
| --- | --- |
|
||||
| Linked DM creates task and receives actual agent response | Passed with Pro shared DMs and the native Codex runner. |
|
||||
| Enabled group with two linked people preserves attribution | Disabled for the approved Pro scope; dedicated-line live qualification remains unrun. |
|
||||
| Unlinked sender cannot start work | Passed for the live shared-DM probe; sender discovered, zero conversations/tasks created. |
|
||||
| Inbound/outbound photos and real iPhone HEIC | Passed for PNG, text file, synthetic HEIC, and an operator-supplied iPhone camera HEIC. The real photo produced a full-resolution JPEG preview and a byte-identical return of the received HEIC. |
|
||||
| Native poll and text answer resume correct interaction | Passed, including sequential drafts, incomplete submission, explicit submission, and one poll continuation. |
|
||||
| Approval rejection reason reaches canonical interaction | Passed, including missing-reason correction and native continuation. |
|
||||
| Restart preserves DM/group replies and pending questions | Shared DM recovery and pending native poll passed; dedicated groups remain unrun. |
|
||||
| Pause/resume/reconnect/removal enforce authority | Passed for Pro DMs. Removal archived the endpoint and connection, cleared secret bindings, and stopped intake. |
|
||||
| Completed turn stays idle until fresh input | Passed. September 12 correction: two successive real follow-ups reopened PHOTON-17, with no new task. |
|
||||
| Provider ambiguous-send/idempotency behavior | Real repeated key suppressed duplicates but returned no original receipt. Unknown-send recovery remains an operator action; no induced network-timeout test. |
|
||||
| HEIF conversion on Linux glibc/Windows and deployment packaging | macOS arm64 and Linux CI conversion passed. Windows execution remains unrun. Linux musl has no packaged converter. |
|
||||
|
||||
Keep this channel behind the existing experimental gate. Mocked tests, synthetic
|
||||
gRPC, and a visible catalog card do not establish these live results.
|
||||
|
||||
### September 12: persistent conversation and live task bubbles
|
||||
|
||||
The operator reported three messages creating PHOTON-15, PHOTON-16, and
|
||||
PHOTON-17. Task completion had incorrectly been treated as the end of an
|
||||
iMessage conversation, and channel admission did not emit the comment event
|
||||
used by an open task page. The fix preserves the latest task until an explicit
|
||||
`/new` or `/close`, publishes comment activity after its transaction commits,
|
||||
and labels inbound human bubbles in both task-chat renderers.
|
||||
|
||||
Tested the fix in the isolated `codex/imessage-photon` worktree on September 12,
|
||||
2026 at 13:56–13:57 America/Chicago, against the operator's existing Pro DM
|
||||
endpoint (`99bebf95…3884`) and PHOTON-17 (`bd6d8379…ba15`). Left the task page
|
||||
open and sent two authorized messages through Apple Messages to the same Photon
|
||||
conversation, waiting for completion between sends. Both appeared without a
|
||||
page reload and both reopened PHOTON-17. Its bubbles showed “Sent from
|
||||
iMessage”; the agent returned “PHOTON-17 live follow-up received” and
|
||||
“PHOTON-17 still one conversation” through Photon. The earlier task records
|
||||
were preserved as history. No live `/new` was sent to replace the operator's
|
||||
current conversation; explicit reset, close, stale controls, duplicate delivery,
|
||||
restart, and dedicated-group continuity are covered by integration fixtures.
|
||||
|
||||
The live server was then restarted on `4d7222110`. A third message asked the
|
||||
agent to repeat its previous reply. It appeared live on PHOTON-17 with its
|
||||
iMessage label, and the agent returned the exact previous reply through Photon.
|
||||
All 304 focused tests passed on that commit, and the existing Teams completion
|
||||
boundary passed its separate regression test.
|
||||
|
||||
Interactive Storybook coverage lives under **Connections / iMessage Photon**.
|
||||
It uses the production catalog card, three-step channel wizard, access and
|
||||
management pages, and task message bubbles with explicitly simulated provider
|
||||
actions. Thirteen stories cover catalog discovery, agent selection, credentials,
|
||||
shared setup, multiple dedicated lines, missing allocation, loading, connecting,
|
||||
outage recovery, reconnect, identity access, and persistent task follow-ups.
|
||||
All 26 light/dark Playwright cases passed, including the 390px mobile layout.
|
||||
The credential and mobile screenshots were inspected. Run with:
|
||||
|
||||
```sh
|
||||
pnpm build-storybook
|
||||
pnpm exec playwright test --config tests/storybook-visual/imessage-photon.config.ts
|
||||
```
|
||||
|
|
@ -0,0 +1,233 @@
|
|||
# iMessage Photon
|
||||
|
||||
**Status: experimental. Live-provider qualification is pending.**
|
||||
|
||||
This channel connects one agent to Photon Cloud. Pro shared allocation supports
|
||||
DMs; dedicated lines also support explicitly enabled groups. A linked
|
||||
Paperclip person can send a DM, exchange files, answer questions, and respond to
|
||||
ordinary confirmations. Each conversation remains attached to a Paperclip task.
|
||||
The connection is a channel with `chat_sdk` transport, not an MCP tool connection.
|
||||
|
||||
## Prerequisites and setup
|
||||
|
||||
1. Enable the existing experimental chat-connectors setting. Open **Apps →
|
||||
iMessage Photon**, or the agent's **Channels** panel.
|
||||
2. In the [Photon dashboard](https://app.photon.codes/), obtain a project ID
|
||||
and project secret. Paperclip checks the project's actual allocation. Pro
|
||||
shared allocation is eligible for DMs only. Enroll each sender in the Photon
|
||||
project's **Users** page and find their assigned number in **Get started**.
|
||||
This enrollment does not authorize them in Paperclip. See Photon's
|
||||
[line model](https://photon.codes/docs/spectrum-ts/providers/imessage/connection-and-routing).
|
||||
3. Choose one invokable agent. Enter the project ID and secret, inspect the
|
||||
allocation. Connect shared DMs, or select a dedicated line. A single eligible dedicated line is selected
|
||||
automatically. A number already reserved by any non-archived endpoint in
|
||||
this instance cannot be selected, including a paused or revoked endpoint.
|
||||
Shared projects have the same exclusive reservation by project ID. Their
|
||||
assigned numbers may differ by sender and are not represented as owned numbers.
|
||||
4. Send a fresh message to the displayed dedicated number, or to the sender's
|
||||
assigned number from Photon for shared DMs. Link the discovered Messages
|
||||
identity through Paperclip's identity confirmation flow. Send another fresh
|
||||
message from that linked person. Setup completes only after a task is created
|
||||
and an actual agent response is published successfully.
|
||||
5. With a dedicated line, to use a group, add the number in Apple Messages and send a message to discover
|
||||
it. Enable the group in Paperclip's Settings page, then send a fresh request.
|
||||
Discovery does not enable a group or replay the discovery message as work.
|
||||
|
||||
The server needs outbound HTTPS to `spectrum.photon.codes` and TLS gRPC to the
|
||||
selected `<line-id>.imsg.photon.codes:443` endpoint, or
|
||||
`imessage.spectrum.photon.codes:443` for a shared project. No public webhook, Mac Messages
|
||||
permissions, Spectrum application runtime, or additional agent loop is needed.
|
||||
|
||||
Project secrets are write-only and vaulted. Inspection is restricted to connection
|
||||
managers and returns project identity, line IDs, phone numbers, and eligibility;
|
||||
it does not return credentials or line tokens. Agents never receive the project
|
||||
secret. The server holds short-lived line tokens in memory, renews before expiry,
|
||||
and checks that the project, line, and number have not changed. Every operation
|
||||
uses that selected line. Replacing credentials must preserve the same identity;
|
||||
connect a different identity with a new endpoint.
|
||||
|
||||
Setup and inspection distinguish credential/allocation errors (HTTP 422), quota
|
||||
limits (429), temporary provider outages (503), and invalid upstream responses
|
||||
(502). An outage does not mean valid credentials need replacement. A failed
|
||||
reconnect leaves the existing credential binding intact.
|
||||
|
||||
## Conversation and access rules
|
||||
|
||||
DMs are enabled by default. Dedicated groups start disabled; shared channels
|
||||
reject groups at admission, publication, and settings changes. Unlinked people cannot
|
||||
start work unless an operator explicitly enables that setting. Identity links
|
||||
use the provider-authenticated sender address and service. A phone number and an
|
||||
Apple-account email are separate identities; names and group membership do not
|
||||
grant Paperclip authority. Revoked links and inactive/viewer memberships cannot
|
||||
answer interactions. Guest work retains the shared channel restrictions.
|
||||
|
||||
Enabling a group makes the agent's responses visible to everyone in that group.
|
||||
It does not authorize every participant to start work. Every authorized message
|
||||
in an enabled group can start or continue work without a mention. Group names
|
||||
and participants are displayed in Settings. If the agent's number leaves the
|
||||
group, that destination becomes unavailable and publication is blocked.
|
||||
|
||||
DMs and groups are linear conversations. An authorized request starts a task;
|
||||
follow-ups append to the current generation through the ordered delivery queue.
|
||||
Completing a task ends the current turn. The next message reopens that same task,
|
||||
including after a server restart. Incoming messages appear live on the open task
|
||||
as user bubbles labeled “Sent from iMessage.” `/status` shows the current task,
|
||||
`/close` closes the conversation,
|
||||
and `/new` closes the current generation so the next request starts a new task.
|
||||
Quoted message GUIDs and multipart references are retained as task context.
|
||||
Quotes do not create separate tasks. A quoted control from an older generation
|
||||
cannot close a newer task. Outgoing echoes, reactions, read receipts, typing,
|
||||
and nonhuman system messages do not start agent work.
|
||||
|
||||
Messages, tasks, assets, publications, identities, and state remain company-scoped.
|
||||
Number and shared-project reservations are deliberately instance-wide. Task assignment, budget
|
||||
limits, pauses, approvals, and native/legacy execution continue through the
|
||||
existing Paperclip services.
|
||||
|
||||
## Questions and confirmations
|
||||
|
||||
Ordinary `ask_user_questions` uses native polls for closed single-choice questions
|
||||
with 2–10 options. Prompts include a text alternative. Correlation uses the returned
|
||||
poll message GUID and option IDs; duplicate titles and option labels are not lookup
|
||||
keys. Responses from other devices, added options, missing actors, expired prompts,
|
||||
and later vote changes cannot undo a completed decision.
|
||||
|
||||
Reply to the exact prompt, or use `/answer <reference>[.<question>] <value>`.
|
||||
Numbered choices, comma-separated multiple choices, custom text, and optional
|
||||
`skip` answers are supported. Questions appear sequentially. Multiple-question
|
||||
sets save a separate draft for each person and require `/submit <reference>`.
|
||||
Paperclip's canonical validators check required answers and selection/numerical
|
||||
rules before resolution. Different people cannot contribute to the same draft.
|
||||
|
||||
Ordinary `request_confirmation` offers explicit Accept/Reject. A required rejection
|
||||
reason is collected through a correlated text response. Target revision, audience,
|
||||
current identity, task generation, endpoint status, and permissions are rechecked
|
||||
at submission. Responses resolve through the canonical interaction service and
|
||||
its durable continuation delivery. A terminal acknowledgement is published once.
|
||||
Arbitrary “yes” messages and tapbacks never constitute approval.
|
||||
|
||||
Credential proposals, connection authorization, governed tool actions, and review
|
||||
kinds that need the full review surface remain in Paperclip. The channel supplies
|
||||
a task link and instructions. No individual-iMessage web permalinks are fabricated.
|
||||
|
||||
## Photos and files
|
||||
|
||||
Text, JPEG/PNG/WebP/GIF, allowed documents, audio, and video use Paperclip's existing
|
||||
attachment policy and byte limits. Provider upload allowances do not raise those
|
||||
limits. Attachments are source-bound to the selected line, chat, message, and
|
||||
attachment GUID before downloading. The server verifies that ownership again on
|
||||
recovery, bounds metadata, streamed bytes, time, and decoded image dimensions,
|
||||
and reports rejected/unavailable files in the task. A not-yet-ready attachment
|
||||
retries before waking the agent, without creating another comment.
|
||||
|
||||
HEIC/HEIF are included in the default attachment policy; operator overrides still
|
||||
win. The original remains downloadable and a JPEG derivative supplies browser
|
||||
preview and image input to the agent. The derivative records its source attachment
|
||||
and hashes. Conversion runs in a separate process with input/output/pixel limits
|
||||
and a deadline. `heif2jpeg@0.1.6` publishes macOS, Windows, and Linux glibc packages
|
||||
for x64/arm64; it does not publish Linux musl binaries. A missing or failed converter
|
||||
retains the original and reports preview unavailability. Only macOS arm64 has been
|
||||
executed locally for this change; other platform binaries still require qualification.
|
||||
|
||||
Live Photo stills and policy-allowed companion videos are retained as attachments
|
||||
on the same message. Native Live Photo reconstruction is not implemented. Outbound
|
||||
files require the existing task/company/agent/originating-run authorization. The
|
||||
server uploads actual bytes; it never sends private storage URLs to Photon.
|
||||
|
||||
## Publication and recovery
|
||||
|
||||
Only output classified for external publication is sent. Internal commentary,
|
||||
reasoning, raw tool output, and credentials stay internal. Final responses use
|
||||
normal bubbles and the channel refreshes typing while work runs. Text is split at
|
||||
paragraph boundaries with a 4,000-Unicode-code-point target and preserved order.
|
||||
Source-message reply references are used when the originating run identifies one.
|
||||
Every text part, attachment message, poll, and explicitly staged correction has a
|
||||
stable `clientMessageId` and immutable payload. Upload completion is recorded before
|
||||
the attachment message is sent. Native edits have a bounded window; ordinary final
|
||||
responses and acknowledgements are separate messages, never token-by-token edits.
|
||||
|
||||
A timeout after transmission is **delivery unknown**. Inspect the activity record
|
||||
and known Photon receipts, then use Paperclip's operator resolution/retry controls.
|
||||
Do not retry by creating another publication or changing its key. Explicit retries
|
||||
reuse the original key and payload. Similar text is not evidence of delivery. An
|
||||
ambiguous upload without a recorded receipt also needs operator review.
|
||||
|
||||
One elected receiver holds the endpoint lease. Live streams notify a serial
|
||||
catch-up reader. The reader advances its checkpoint only after preceding events
|
||||
are durably admitted or classified, including irrelevant events. It deduplicates
|
||||
provider sequence and message identity independently and reconstructs chats,
|
||||
attachments, and poll mappings from persisted state after restart.
|
||||
|
||||
Dedicated recovery requires adjacent sequence numbers. The shared gateway's
|
||||
project-filtered feed has increasing, non-adjacent sequences. Shared recovery
|
||||
commits its checkpoint only after the complete replay barrier and every preceding
|
||||
admission succeed. Interrupted or out-of-order replay retains the previous cursor.
|
||||
Shared channels do not subscribe to the unsupported group stream.
|
||||
|
||||
The pinned SDK's public catch-up iterator discards sequence-only/unknown-variant
|
||||
frames. Paperclip's small authenticated gRPC recovery transport retains their
|
||||
sequence while delegating known event decoding to the SDK. This prevents false
|
||||
history gaps without silently skipping a frame. A missing/reset cursor or an
|
||||
actual history gap stops in Attention. Initial historical messages establish a
|
||||
checkpoint but do not create old tasks automatically.
|
||||
|
||||
Pause stops execution and external publication while retaining already accepted
|
||||
pending work. Resume establishes a new intake cutoff, so messages deliberately
|
||||
suppressed during pause do not become work. Outage recovery catches up eligible
|
||||
missed messages. Disconnect archives the endpoint, stops streams, invalidates
|
||||
interaction authority, and removes owned secret bindings. It does not delete the
|
||||
Photon project, number, subscription, or Messages history. Hiding experimental
|
||||
UI alone does not disconnect existing channels.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| State or symptom | Action |
|
||||
| --- | --- |
|
||||
| Invalid project credentials | Replace the vaulted secret for the same project/number and reconnect. |
|
||||
| Shared allocation | Connect shared DMs, enroll the sender in Photon, and use their assigned number. Groups require a dedicated line. |
|
||||
| No eligible dedicated lines | Review the project's line allocation in Photon, then inspect again. |
|
||||
| Number already owned | Use its existing endpoint or remove that endpoint before reconnecting the number. Pause retains the reservation. |
|
||||
| Number changes/disappears | Review the Photon allocation. Restore the original identity or create a new endpoint. |
|
||||
| No task from a group message | Groups are disabled for shared channels. For a dedicated channel, enable the discovered group, link the sender, and send a fresh request. |
|
||||
| Setup remains Verifying | Complete the linked fresh-message → task → actual agent reply loop; a credential check is insufficient. |
|
||||
| Quota/network interruption | Review Activity. Transient errors retry with bounded backoff; quotas are distinct from authentication failures. |
|
||||
| Attachment preparing | Let the durable delivery retry; do not resend the message to force another task. |
|
||||
| Preview unavailable | Download the original and verify converter support/policy on this deployment platform. |
|
||||
| Delivery unknown | Reconcile the exact provider receipt or explicitly retry the same immutable publication. |
|
||||
| Missing/reset cursor or history gap | Review the affected period before operator recovery. The service does not silently skip it. |
|
||||
| Old poll no longer works | Open the task's current interaction. Completed/expired polls cannot reverse a decision. |
|
||||
|
||||
Diagnostics use existing local activity and run records. This change adds no
|
||||
first-party Telemetry events. Persisted receipts/checkpoints are required for
|
||||
recovery; do not manually delete provider state to resolve an outage.
|
||||
|
||||
## Qualification and source versions
|
||||
|
||||
See [implementation and acceptance plan](../plans/2026-09-11-imessage-photon.md)
|
||||
and [verification record](IMESSAGE-PHOTON-VERIFICATION.md). Deterministic fixtures
|
||||
and synthetic gRPC are not live-provider proof. A dedicated test line, known
|
||||
participants, real iPhone HEIC, and native polls are required before claiming the
|
||||
full live acceptance loop.
|
||||
|
||||
Pinned dependencies: `@photon-ai/advanced-imessage@2.1.0`, `@grpc/grpc-js@1.14.4`,
|
||||
`nice-grpc@2.1.17`, `nice-grpc-common@2.0.4`, `heif2jpeg@0.1.6`.
|
||||
|
||||
First-party references inspected on 2026-09-11:
|
||||
[Cloud authentication](https://github.com/photon-hq/spectrum-ts/blob/main/packages/core/src/utils/cloud.ts),
|
||||
[SDK](https://github.com/photon-hq/advanced-imessage-ts),
|
||||
[events](https://photon.codes/docs/advanced-kits/imessage/events),
|
||||
[polls](https://photon.codes/docs/advanced-kits/imessage/polls),
|
||||
[attachments](https://photon.codes/docs/advanced-kits/imessage/attachments),
|
||||
[idempotency](https://photon.codes/docs/advanced-kits/imessage/error-handling), and
|
||||
[HEIF converter](https://photon.codes/docs/utilities/heif2jpeg).
|
||||
|
||||
### Shared-gateway duplicate receipts
|
||||
|
||||
The Pro shared gateway has been observed returning gRPC `ALREADY_EXISTS` as SDK
|
||||
`internalError`, without a receipt, when an identical `clientMessageId` is repeated.
|
||||
Paperclip retains delivery-unknown state if no stored receipt exists. Inspect the
|
||||
original conversation and use the existing operator resolution action. Do not
|
||||
create another idempotency key or infer delivery from matching text. Photon’s
|
||||
[documented idempotency behavior](https://photon.codes/docs/advanced-kits/imessage/error-handling)
|
||||
says repeated writes return the original result; the live shared-gateway result
|
||||
is recorded separately in the verification report.
|
||||
|
|
@ -9,7 +9,8 @@ manifest generation, branding, secrets, deterministic tests, real-account
|
|||
proof, and PR submission.
|
||||
|
||||
Provider notes: [Google Workspace](./GOOGLE-WORKSPACE.md),
|
||||
[Gmail](./GMAIL.md), [PostHog](./POSTHOG.md). Optional credential custody:
|
||||
[Gmail](./GMAIL.md), [PostHog](./POSTHOG.md),
|
||||
[AgentMail](./AGENTMAIL.md), and [iMessage Photon](./IMESSAGE-PHOTON.md). Optional credential custody:
|
||||
[Vercel Connect](./VERCEL-CONNECT.md).
|
||||
|
||||
Post-read action: classify a new integration request, pick the right Paperclip
|
||||
|
|
|
|||
|
|
@ -0,0 +1,239 @@
|
|||
# iMessage Photon channel
|
||||
|
||||
Date: 2026-09-11. Status: approved for implementation; qualification tracked below.
|
||||
Base: origin/master, 1c4bcff2b. Branch: codex/imessage-photon.
|
||||
|
||||
## Approved scope update — 2026-09-12
|
||||
|
||||
The operator approved Pro-compatible shared DMs with groups disabled. This
|
||||
supersedes the dedicated-only exclusions below for DMs. Shared setup uses the
|
||||
project token and fixed `imessage.spectrum.photon.codes:443` gateway, reserves the
|
||||
project across non-archived endpoints, and derives a project-scoped conversation
|
||||
and checkpoint namespace. It never claims ownership of a pool phone number.
|
||||
Sender enrollment in Photon and identity linking in Paperclip are separate gates.
|
||||
Dedicated lines retain the original behavior. Allocation changes require a new
|
||||
channel. Native groups remain unavailable on shared channels at every boundary.
|
||||
|
||||
The real qualification uses an isolated clean database, a test-only agent, the
|
||||
operator's enrolled Messages identity, and the existing Apps wizard. Photon offers
|
||||
a terminal development provider and control-plane CLI; neither substitutes for
|
||||
an iMessage Cloud round trip. Record actual live results separately from fixtures.
|
||||
|
||||
## Outcome and defaults
|
||||
|
||||
Add **iMessage Photon** (`imessage-photon`) behind the existing experimental
|
||||
chat-connectors UI gate, in Apps and each agent's Channels panel. Use Photon
|
||||
Cloud and one dedicated number per channel/agent. People initiate DMs and
|
||||
explicitly enabled groups; every authorized group message can start or continue
|
||||
work without a mention. Require linked Paperclip people by default. Unlinked
|
||||
senders require an explicit operator opt-in. Never infer authority from a phone
|
||||
number, email address, display name, or group membership, or merge those identities.
|
||||
|
||||
Messages remain bound to tasks through the existing chat subsystem: company
|
||||
scope, vaulted credentials, durable admission, endpoint/conversation leases,
|
||||
ordered wakeups, external principals, task generations, publication outbox,
|
||||
attachment provenance, budgets, pauses, native and legacy execution. Keep
|
||||
`connectionPurpose: channel` and `chat_sdk` transport. Chat-approved external
|
||||
responses publish automatically; AgentMail's explicit email-send policy stays
|
||||
specific to email. Do not add a second agent loop, Spectrum application runtime,
|
||||
generic Photon MCP connection, or arbitrary send API.
|
||||
|
||||
Excluded initially: local Mac access, shared-pool numbers, SMS/RCS, unsolicited
|
||||
new conversations, agent-created groups, calls, location, stickers, backgrounds,
|
||||
custom iMessage apps, and native Live Photo reassembly. Hiding experimental UI
|
||||
must not stop existing channels; Pause/Disconnect control runtime behavior.
|
||||
|
||||
## Provider and credentials
|
||||
|
||||
Implement an in-repo Chat SDK adapter using `@photon-ai/advanced-imessage@2.1.0`
|
||||
with explicit gRPC dependencies. Reference upstream Photon adapter behavior but
|
||||
persist correlation by IDs, not in-memory poll titles. Separate Cloud client,
|
||||
receiver/recovery, adapter, attachment, and interaction helpers.
|
||||
|
||||
Accept project ID and write-only project secret. Inspect Photon using Basic
|
||||
project authentication and its project/token endpoints. Return only project
|
||||
identity, allocation eligibility, line IDs, and numbers to managers through
|
||||
`POST /api/chat-endpoints/:endpointId/photon/inspect`; never return minted tokens.
|
||||
Verify dedicated allocation from actual token data. Auto-select a sole eligible
|
||||
number; require explicit selection for multiple numbers. Reject shared allocation.
|
||||
Vault the secret; store validated project/selected-line config separately. Mint
|
||||
line tokens in memory, renew before expiry, bind every RPC and stream to the
|
||||
selected instance, and recheck number/ownership on renewal. Missing, changed,
|
||||
or ineligible line enters Attention. Rotation preserves project and number;
|
||||
a new identity needs a new endpoint. Retire old ownership before replacement.
|
||||
|
||||
Extend provider contracts across db/shared/server/UI and generate a forward
|
||||
migration. Reserve the Photon number across companies for every non-archived
|
||||
endpoint, including paused/revoked endpoints. Persist checkpoints, typed poll
|
||||
bindings, per-person drafts, immutable send identities, upload receipts, and
|
||||
side effects in company/endpoint-scoped state/action stores.
|
||||
|
||||
## Setup and management
|
||||
|
||||
Three steps: choose an invokable agent; connect and inspect Photon credentials
|
||||
and select a dedicated line; test from Apple Messages. Link Photon dashboard and
|
||||
dedicated-line documentation. Defaults: DMs on, unlinked people off, individual
|
||||
groups disabled. Show actionable missing/shared/duplicate/invalid credential
|
||||
errors. Show the copyable number and discovered sender, support the existing
|
||||
identity-link confirmation. Only a linked sender's fresh message that creates a
|
||||
task and receives a successful publication completes setup. Credential verification
|
||||
alone does not. Optional group test: add number in Messages, discover group,
|
||||
enable it in Paperclip, send a fresh authorized request.
|
||||
|
||||
Management shows agent/project/number, health, receive/send timestamps, discovered
|
||||
groups and participants/availability/enablement, linked people/revocation,
|
||||
delivery retry/unknown outcomes, pause/resume/reconnect/disconnect. Explain that
|
||||
group replies are visible to all members while sender authorization is separate.
|
||||
Use existing design tokens/components, official branding with provenance, themes,
|
||||
keyboard/loading/error/narrow-screen states, task links and copyable number;
|
||||
never invent individual iMessage web permalinks.
|
||||
|
||||
## Conversations and authorization
|
||||
|
||||
DMs/groups are linear: one active task generation per endpoint/chat. Fresh
|
||||
authorized input creates work when none is active; follow-ups append and use
|
||||
existing ordered queue/coalescing. Per the September 12 product correction,
|
||||
terminal tasks reopen on the next message in the same conversation. Only an
|
||||
explicit `/new` or `/close` followed by a fresh message creates a new generation.
|
||||
Inbound comments appear live with “Sent from iMessage” attribution. Support `/status`.
|
||||
Preserve native reply GUID/part as context; it does not create a separate task.
|
||||
Chronology guards protect later generations from stale controls. Rename/avatar
|
||||
changes affect presentation only. Bot removal marks a group unavailable and
|
||||
blocks sends. Ignore outgoing echoes and system/read/typing/reaction/metadata
|
||||
events as work triggers. Linked identity/current membership and resource/endpoint
|
||||
policy are rechecked at admission and interaction resolution.
|
||||
|
||||
## Questions and approvals
|
||||
|
||||
Support ordinary ask_user_questions and eligible request_confirmation. Single
|
||||
select 2–10 canonical options uses native polls with persistent returned poll
|
||||
GUID/option-ID bindings, plus text fallback. Never match by title/label. Questions
|
||||
appear sequentially with a short reference. Accept exact native prompt replies
|
||||
or `/answer <reference> <value>`; support free text, custom/numbered multi-select,
|
||||
optional skipping, and canonical validation including numerical constraints.
|
||||
Per-person drafts cannot mix; multiple-question sets require `/submit <reference>`.
|
||||
A single question resolves on first valid authorized submission.
|
||||
|
||||
Confirmations show the approved external summary and Accept/Reject, collecting a
|
||||
required rejection reason via correlated text. Revalidate target revision,
|
||||
audience, linked user permission, generation and endpoint before using canonical
|
||||
interaction/continuation services. Arbitrary yes/tapbacks are never approvals.
|
||||
Credential disclosure, connection authorization, governed tool execution, and
|
||||
review kinds requiring the full Board surface get an explanation/task link.
|
||||
|
||||
Handle recognized responses before normal comment ingestion. Duplicate votes,
|
||||
missing actors, stale/expired links, changed membership, late poll changes and
|
||||
participant-added options cannot resolve. Unvotes only clear unsubmitted drafts.
|
||||
Exactly one canonical resolution and continuation; terminal acknowledgement;
|
||||
old polls inert. Test provider poll creation before local binding crash and votes
|
||||
before binding finalization without title matching or duplicate resolution.
|
||||
|
||||
## Photos, attachments, and publication
|
||||
|
||||
Support text and policy-allowed images/documents/audio/video. Persist a closed
|
||||
line/chat/message/attachment/optional-part locator before fetching. Authenticate
|
||||
over selected line and verify attachment ownership via message/chat. Bound
|
||||
metadata, bytes, time and decoded dimensions. Keep Paperclip's configured limits,
|
||||
not Photon's larger allowance. Retry attachmentNotReady before agent wake,
|
||||
without duplicate comments. Preserve attachment-only input, captions, multiple
|
||||
images and multipart order; surface unavailable/rejected files in the task.
|
||||
|
||||
Add HEIC/HEIF to default policy while honoring overrides. Preserve originals and
|
||||
produce a labeled JPEG derivative for preview/agent image input through bounded
|
||||
`heif2jpeg@0.1.6`; verify packaged platform binaries. Preserve provenance. Keep
|
||||
Live Photo still/allowed companion video as related files.
|
||||
|
||||
Outbound files need existing company/task/agent/originating-run authorization.
|
||||
Send bytes, never private storage URLs. Immutable outbox carries stable
|
||||
clientMessageId for each text part/file/poll/correction; retry same key/payload.
|
||||
Persist upload receipt before send. Split plain readable text at paragraph
|
||||
boundaries near 4,000 Unicode-safe characters, preserving order/URLs/code.
|
||||
Final messages plus typing, not token bubbles. Use native reply references.
|
||||
Edits within provider limits; expired edits fail visibly or require staged correction.
|
||||
Internal reasoning/commentary/tools/credentials remain internal.
|
||||
|
||||
Transmission timeout is delivery_unknown. Reconcile exact receipts, never
|
||||
similar text; operator resolution handles unresolved results, never silently
|
||||
mint a new idempotency key. Do not assume undocumented provider key retention.
|
||||
|
||||
## Receiving and recovery
|
||||
|
||||
Elect one receiver per endpoint with lease renewal/generation fencing. Subscribe
|
||||
to live message/chat/group/poll events concurrently with catch-up. Dedupe by line
|
||||
instance/event sequence and message GUID. Advance checkpoint only after all
|
||||
preceding events are durably admitted/classified, including irrelevant events.
|
||||
Bound intake; execution follows durable admission. Rebuild chats, attachments,
|
||||
and poll mappings after restart without SDK caches.
|
||||
|
||||
Initial activation cutoff suppresses historical work while allowing metadata and
|
||||
checkpoint establishment. Pause stops execution/publication and retains accepted
|
||||
work; record pause boundaries to suppress intentional pause interval input.
|
||||
Outages catch up eligible missed events. Missing/reset/gapped history enters
|
||||
Attention, never silently skips. Distinguish auth/line/quota/network/preparation/
|
||||
ambiguous send failures, bound retries, expose recovery actions. Disconnect stops
|
||||
streams, invalidates interaction authority, archives endpoint/removes owned secret
|
||||
bindings; never deletes Photon project/number/subscription/history. No webhook.
|
||||
Diagnostics stay local; any Telemetry change needs separate strict review.
|
||||
|
||||
## Delivery and verification
|
||||
|
||||
1. Worktree/isolation, source versions, shared contracts/state, generated migration.
|
||||
2. Cloud inspection, line auth/renewal, adapter lifecycle, synthetic gRPC fixtures.
|
||||
3. Durable ingress, authorization/linking/resources/generations/queue/publication.
|
||||
4. Attachments/recovery/HEIC, polls/text drafts/confirmations/continuation.
|
||||
5. Catalog/setup/management, browser checks, docs, live qualification, PR preparation.
|
||||
|
||||
Keep logical commits; existing Slack/Discord/AgentMail and native/legacy paths
|
||||
remain functional. Acceptance covers invalid/missing/shared/multiple/duplicate
|
||||
lines; interrupted setup; unlinked/revoked/viewer/wrong-company/wrong-line inputs;
|
||||
disabled/removed groups; concurrent/ordered bursts and generation commands;
|
||||
restart/duplicate/out-of-order/takeover/renewal/catch-up/pause; stable multipart
|
||||
keys, Unicode, rate limits, successful-send crash/unknown/partial uploads/edit
|
||||
expiry; image-only/multiple/HEIC/corrupt/oversize/delayed files; exact poll IDs,
|
||||
free/custom/multiselect/partial/submitted/invalid drafts; accept/reject/reason,
|
||||
stale target, Board race and one continuation; accessible responsive UI.
|
||||
|
||||
Run targeted suites, migration/package checks, affected chat-adapters browser
|
||||
suite, then `pnpm check:token-gates`, `pnpm -r typecheck`, `pnpm test:run`,
|
||||
`pnpm build`. Leave lockfile to the repository bot. Update channel setup,
|
||||
troubleshooting/feature boundaries and relevant spec addendum; regenerate catalog.
|
||||
Prepare every PR-template section and explicitly report unavailable credentials,
|
||||
failed checks and unqualified behavior.
|
||||
|
||||
Live proof requires a dedicated test line and known participants: linked DM and
|
||||
agent reply; enabled group with two linked participants/attribution; unlinked
|
||||
denial; both-direction photos including real iPhone HEIC; poll/text continuation;
|
||||
rejection reason; restart retaining DMs/groups/questions; pause/resume/reconnect/
|
||||
remove; terminal conversations idle until fresh input. Record tested commit,
|
||||
versions, redacted IDs, observable outcomes and limitations. Mocks are not live
|
||||
qualification. Full DM/group/media/interaction/restart/auth loop is required.
|
||||
|
||||
## Primary sources (verified during planning 2026-09-11)
|
||||
|
||||
- https://photon.codes/docs/spectrum-ts/providers/imessage/connection-and-routing
|
||||
- https://github.com/photon-hq/advanced-imessage-ts
|
||||
- https://github.com/photon-hq/spectrum-ts/blob/main/packages/core/src/utils/cloud.ts
|
||||
- https://photon.codes/docs/advanced-kits/imessage/polls
|
||||
- https://photon.codes/docs/advanced-kits/imessage/attachments
|
||||
- https://photon.codes/docs/advanced-kits/imessage/events
|
||||
- https://photon.codes/docs/advanced-kits/imessage/error-handling
|
||||
- https://photon.codes/docs/utilities/heif2jpeg
|
||||
|
||||
Inspected versions: advanced-imessage 2.1.0, Photon chat adapter 3.2.0,
|
||||
Spectrum core/iMessage 12.8.0, heif2jpeg 0.1.6. Cloud Basic auth project endpoint:
|
||||
`https://spectrum.photon.codes/projects/{projectId}/`; token exchange:
|
||||
`POST .../imessage/tokens`; dedicated response has `auth` and `numbers` maps keyed
|
||||
by instance ID and `expiresIn`. Line gRPC host: `{instanceId}.imsg.photon.codes:443`.
|
||||
|
||||
## Implementation evidence
|
||||
|
||||
Implemented in the fresh `codex/imessage-photon` worktree. Provider contracts,
|
||||
forward migration, Cloud inspection, leased recovery, source-bound media,
|
||||
immutable publication, native interaction continuation, and the setup/management
|
||||
UI are present. The channel remains experimental.
|
||||
|
||||
The [channel runbook](../connections/IMESSAGE-PHOTON.md) documents operation and
|
||||
supported boundaries. The [verification record](../connections/IMESSAGE-PHOTON-VERIFICATION.md)
|
||||
records automated results and the still-unrun live qualification matrix. No live
|
||||
Photon credentials or approved test participants were available. Implementation
|
||||
must not be represented as live-provider qualification.
|
||||
|
|
@ -1,6 +1,6 @@
|
|||
export interface PaperclipChatFilePreparationDelivery {
|
||||
readonly provider:
|
||||
"slack" | "github" | "discord" | "microsoft-teams" | "telegram" | null;
|
||||
"slack" | "github" | "discord" | "microsoft-teams" | "telegram" | "imessage-photon" | null;
|
||||
readonly mode: "provider_attachment" | "paperclip_task_only" | "unknown";
|
||||
readonly preparationState: "prepared";
|
||||
readonly providerDeliveryConfirmed: false;
|
||||
|
|
@ -35,7 +35,8 @@ export function paperclipChatFilePreparationDelivery(
|
|||
if (
|
||||
authenticatedProvider === "slack" ||
|
||||
authenticatedProvider === "discord" ||
|
||||
authenticatedProvider === "telegram"
|
||||
authenticatedProvider === "telegram" ||
|
||||
authenticatedProvider === "imessage-photon"
|
||||
) {
|
||||
return {
|
||||
...common,
|
||||
|
|
|
|||
|
|
@ -797,7 +797,7 @@ type PaperclipWakeRecovery = {
|
|||
};
|
||||
|
||||
export type PaperclipExternalChatProvider =
|
||||
"slack" | "github" | "discord" | "microsoft-teams" | "telegram";
|
||||
"slack" | "github" | "discord" | "microsoft-teams" | "telegram" | "imessage-photon";
|
||||
|
||||
type PaperclipWakePayload = {
|
||||
executionContinuation: ExecutionContinuationEnvelope | null;
|
||||
|
|
@ -1665,6 +1665,7 @@ const PAPERCLIP_EXTERNAL_CHAT_PROVIDERS =
|
|||
"discord",
|
||||
"microsoft-teams",
|
||||
"telegram",
|
||||
"imessage-photon",
|
||||
]);
|
||||
|
||||
function normalizePaperclipExternalChatProvider(
|
||||
|
|
|
|||
|
|
@ -0,0 +1,5 @@
|
|||
ALTER TABLE "chat_endpoints" DROP CONSTRAINT IF EXISTS "chat_endpoints_provider_check";--> statement-breakpoint
|
||||
ALTER TABLE "chat_external_principals" DROP CONSTRAINT IF EXISTS "chat_external_principals_provider_check";--> statement-breakpoint
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS "chat_endpoints_photon_number_uq" ON "chat_endpoints" USING btree ("bot_external_id") WHERE "chat_endpoints"."provider" = 'imessage-photon' and "chat_endpoints"."status" <> 'archived' and "chat_endpoints"."bot_external_id" is not null;--> statement-breakpoint
|
||||
ALTER TABLE "chat_endpoints" ADD CONSTRAINT "chat_endpoints_provider_check" CHECK ("chat_endpoints"."provider" in ('slack', 'github', 'discord', 'microsoft-teams', 'telegram', 'agentmail', 'imessage-photon'));--> statement-breakpoint
|
||||
ALTER TABLE "chat_external_principals" ADD CONSTRAINT "chat_external_principals_provider_check" CHECK ("chat_external_principals"."provider" in ('slack', 'github', 'discord', 'microsoft-teams', 'telegram', 'agentmail', 'imessage-photon'));
|
||||
File diff suppressed because it is too large
Load Diff
|
|
@ -1912,6 +1912,13 @@
|
|||
"when": 1789219070888,
|
||||
"tag": "0274_agent_chat",
|
||||
"breakpoints": true
|
||||
},
|
||||
{
|
||||
"idx": 275,
|
||||
"version": "7",
|
||||
"when": 1789221632037,
|
||||
"tag": "0275_easy_dragon_man",
|
||||
"breakpoints": true
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
|
|
@ -116,7 +116,7 @@ export const chatEndpoints = pgTable(
|
|||
check("chat_endpoints_email_policy_check", sql`${table.provider} <> 'agentmail' or (${table.publicationMode} = 'explicit' and ${table.externalExecutionPolicy} = 'agent')`),
|
||||
check(
|
||||
"chat_endpoints_provider_check",
|
||||
sql`${table.provider} in ('slack', 'github', 'discord', 'microsoft-teams', 'telegram', 'agentmail')`,
|
||||
sql`${table.provider} in ('slack', 'github', 'discord', 'microsoft-teams', 'telegram', 'agentmail', 'imessage-photon')`,
|
||||
),
|
||||
check(
|
||||
"chat_endpoints_status_check",
|
||||
|
|
@ -155,6 +155,9 @@ export const chatEndpoints = pgTable(
|
|||
// one native bot identity. Excluding providerAccountId closes the race
|
||||
// where concurrent setup in two guilds could otherwise claim that bot for
|
||||
// two Paperclip agents after both application-level prechecks passed.
|
||||
uniqueIndex("chat_endpoints_photon_number_uq")
|
||||
.on(table.botExternalId)
|
||||
.where(sql`${table.provider} = 'imessage-photon' and ${table.status} <> 'archived' and ${table.botExternalId} is not null`),
|
||||
uniqueIndex("chat_endpoints_live_discord_bot_external_uq")
|
||||
.on(table.provider, table.botExternalId)
|
||||
.where(
|
||||
|
|
@ -278,7 +281,7 @@ export const chatExternalPrincipals = pgTable(
|
|||
(table) => [
|
||||
check(
|
||||
"chat_external_principals_provider_check",
|
||||
sql`${table.provider} in ('slack', 'github', 'discord', 'microsoft-teams', 'telegram', 'agentmail')`,
|
||||
sql`${table.provider} in ('slack', 'github', 'discord', 'microsoft-teams', 'telegram', 'agentmail', 'imessage-photon')`,
|
||||
),
|
||||
check(
|
||||
"chat_external_principals_kind_check",
|
||||
|
|
|
|||
|
|
@ -3,66 +3,67 @@ import a1 from "./app-definitions/zapier.json" with { type: "json" };
|
|||
import a2 from "./app-definitions/github.json" with { type: "json" };
|
||||
import a3 from "./app-definitions/slack.json" with { type: "json" };
|
||||
import a4 from "./app-definitions/microsoft-teams.json" with { type: "json" };
|
||||
import a5 from "./app-definitions/telegram.json" with { type: "json" };
|
||||
import a6 from "./app-definitions/discord.json" with { type: "json" };
|
||||
import a7 from "./app-definitions/notion.json" with { type: "json" };
|
||||
import a8 from "./app-definitions/posthog.json" with { type: "json" };
|
||||
import a9 from "./app-definitions/linear.json" with { type: "json" };
|
||||
import a10 from "./app-definitions/context7.json" with { type: "json" };
|
||||
import a11 from "./app-definitions/shopify.json" with { type: "json" };
|
||||
import a12 from "./app-definitions/composio.json" with { type: "json" };
|
||||
import a13 from "./app-definitions/oauth-generic.json" with { type: "json" };
|
||||
import a14 from "./app-definitions/api-key-generic.json" with { type: "json" };
|
||||
import a15 from "./app-definitions/sentry.json" with { type: "json" };
|
||||
import a16 from "./app-definitions/vercel.json" with { type: "json" };
|
||||
import a17 from "./app-definitions/anthropic.json" with { type: "json" };
|
||||
import a18 from "./app-definitions/jira.json" with { type: "json" };
|
||||
import a19 from "./app-definitions/airtable.json" with { type: "json" };
|
||||
import a20 from "./app-definitions/beehiiv.json" with { type: "json" };
|
||||
import a21 from "./app-definitions/bitly.json" with { type: "json" };
|
||||
import a22 from "./app-definitions/candid.json" with { type: "json" };
|
||||
import a23 from "./app-definitions/cloudflare.json" with { type: "json" };
|
||||
import a24 from "./app-definitions/cloudinary.json" with { type: "json" };
|
||||
import a25 from "./app-definitions/coda.json" with { type: "json" };
|
||||
import a26 from "./app-definitions/hugging-face.json" with { type: "json" };
|
||||
import a27 from "./app-definitions/kernel.json" with { type: "json" };
|
||||
import a28 from "./app-definitions/local-falcon.json" with { type: "json" };
|
||||
import a29 from "./app-definitions/make.json" with { type: "json" };
|
||||
import a30 from "./app-definitions/manufact.json" with { type: "json" };
|
||||
import a31 from "./app-definitions/miro.json" with { type: "json" };
|
||||
import a32 from "./app-definitions/netlify.json" with { type: "json" };
|
||||
import a33 from "./app-definitions/oreilly.json" with { type: "json" };
|
||||
import a34 from "./app-definitions/planetscale.json" with { type: "json" };
|
||||
import a35 from "./app-definitions/resend.json" with { type: "json" };
|
||||
import a36 from "./app-definitions/ticktick.json" with { type: "json" };
|
||||
import a37 from "./app-definitions/todoist.json" with { type: "json" };
|
||||
import a38 from "./app-definitions/webflow.json" with { type: "json" };
|
||||
import a39 from "./app-definitions/wix.json" with { type: "json" };
|
||||
import a40 from "./app-definitions/brex.json" with { type: "json" };
|
||||
import a41 from "./app-definitions/clickhouse.json" with { type: "json" };
|
||||
import a42 from "./app-definitions/egnyte.json" with { type: "json" };
|
||||
import a43 from "./app-definitions/embat.json" with { type: "json" };
|
||||
import a44 from "./app-definitions/mixpanel.json" with { type: "json" };
|
||||
import a45 from "./app-definitions/postman.json" with { type: "json" };
|
||||
import a46 from "./app-definitions/razorpay.json" with { type: "json" };
|
||||
import a47 from "./app-definitions/sanity.json" with { type: "json" };
|
||||
import a48 from "./app-definitions/stripe.json" with { type: "json" };
|
||||
import a49 from "./app-definitions/supabase.json" with { type: "json" };
|
||||
import a50 from "./app-definitions/ticket-tailor.json" with { type: "json" };
|
||||
import a51 from "./app-definitions/asana.json" with { type: "json" };
|
||||
import a52 from "./app-definitions/box.json" with { type: "json" };
|
||||
import a53 from "./app-definitions/mem0.json" with { type: "json" };
|
||||
import a54 from "./app-definitions/pagerduty.json" with { type: "json" };
|
||||
import a55 from "./app-definitions/similarweb.json" with { type: "json" };
|
||||
import a56 from "./app-definitions/xero.json" with { type: "json" };
|
||||
import a57 from "./app-definitions/gmail.json" with { type: "json" };
|
||||
import a58 from "./app-definitions/google-drive.json" with { type: "json" };
|
||||
import a59 from "./app-definitions/google-docs.json" with { type: "json" };
|
||||
import a60 from "./app-definitions/google-sheets.json" with { type: "json" };
|
||||
import a61 from "./app-definitions/google-slides.json" with { type: "json" };
|
||||
import a62 from "./app-definitions/google-calendar.json" with { type: "json" };
|
||||
import a63 from "./app-definitions/google-chat.json" with { type: "json" };
|
||||
import a64 from "./app-definitions/google-people.json" with { type: "json" };
|
||||
import a65 from "./app-definitions/google-workspace-search.json" with { type: "json" };
|
||||
import a5 from "./app-definitions/imessage-photon.json" with { type: "json" };
|
||||
import a6 from "./app-definitions/telegram.json" with { type: "json" };
|
||||
import a7 from "./app-definitions/discord.json" with { type: "json" };
|
||||
import a8 from "./app-definitions/notion.json" with { type: "json" };
|
||||
import a9 from "./app-definitions/posthog.json" with { type: "json" };
|
||||
import a10 from "./app-definitions/linear.json" with { type: "json" };
|
||||
import a11 from "./app-definitions/context7.json" with { type: "json" };
|
||||
import a12 from "./app-definitions/shopify.json" with { type: "json" };
|
||||
import a13 from "./app-definitions/composio.json" with { type: "json" };
|
||||
import a14 from "./app-definitions/oauth-generic.json" with { type: "json" };
|
||||
import a15 from "./app-definitions/api-key-generic.json" with { type: "json" };
|
||||
import a16 from "./app-definitions/sentry.json" with { type: "json" };
|
||||
import a17 from "./app-definitions/vercel.json" with { type: "json" };
|
||||
import a18 from "./app-definitions/anthropic.json" with { type: "json" };
|
||||
import a19 from "./app-definitions/jira.json" with { type: "json" };
|
||||
import a20 from "./app-definitions/airtable.json" with { type: "json" };
|
||||
import a21 from "./app-definitions/beehiiv.json" with { type: "json" };
|
||||
import a22 from "./app-definitions/bitly.json" with { type: "json" };
|
||||
import a23 from "./app-definitions/candid.json" with { type: "json" };
|
||||
import a24 from "./app-definitions/cloudflare.json" with { type: "json" };
|
||||
import a25 from "./app-definitions/cloudinary.json" with { type: "json" };
|
||||
import a26 from "./app-definitions/coda.json" with { type: "json" };
|
||||
import a27 from "./app-definitions/hugging-face.json" with { type: "json" };
|
||||
import a28 from "./app-definitions/kernel.json" with { type: "json" };
|
||||
import a29 from "./app-definitions/local-falcon.json" with { type: "json" };
|
||||
import a30 from "./app-definitions/make.json" with { type: "json" };
|
||||
import a31 from "./app-definitions/manufact.json" with { type: "json" };
|
||||
import a32 from "./app-definitions/miro.json" with { type: "json" };
|
||||
import a33 from "./app-definitions/netlify.json" with { type: "json" };
|
||||
import a34 from "./app-definitions/oreilly.json" with { type: "json" };
|
||||
import a35 from "./app-definitions/planetscale.json" with { type: "json" };
|
||||
import a36 from "./app-definitions/resend.json" with { type: "json" };
|
||||
import a37 from "./app-definitions/ticktick.json" with { type: "json" };
|
||||
import a38 from "./app-definitions/todoist.json" with { type: "json" };
|
||||
import a39 from "./app-definitions/webflow.json" with { type: "json" };
|
||||
import a40 from "./app-definitions/wix.json" with { type: "json" };
|
||||
import a41 from "./app-definitions/brex.json" with { type: "json" };
|
||||
import a42 from "./app-definitions/clickhouse.json" with { type: "json" };
|
||||
import a43 from "./app-definitions/egnyte.json" with { type: "json" };
|
||||
import a44 from "./app-definitions/embat.json" with { type: "json" };
|
||||
import a45 from "./app-definitions/mixpanel.json" with { type: "json" };
|
||||
import a46 from "./app-definitions/postman.json" with { type: "json" };
|
||||
import a47 from "./app-definitions/razorpay.json" with { type: "json" };
|
||||
import a48 from "./app-definitions/sanity.json" with { type: "json" };
|
||||
import a49 from "./app-definitions/stripe.json" with { type: "json" };
|
||||
import a50 from "./app-definitions/supabase.json" with { type: "json" };
|
||||
import a51 from "./app-definitions/ticket-tailor.json" with { type: "json" };
|
||||
import a52 from "./app-definitions/asana.json" with { type: "json" };
|
||||
import a53 from "./app-definitions/box.json" with { type: "json" };
|
||||
import a54 from "./app-definitions/mem0.json" with { type: "json" };
|
||||
import a55 from "./app-definitions/pagerduty.json" with { type: "json" };
|
||||
import a56 from "./app-definitions/similarweb.json" with { type: "json" };
|
||||
import a57 from "./app-definitions/xero.json" with { type: "json" };
|
||||
import a58 from "./app-definitions/gmail.json" with { type: "json" };
|
||||
import a59 from "./app-definitions/google-drive.json" with { type: "json" };
|
||||
import a60 from "./app-definitions/google-docs.json" with { type: "json" };
|
||||
import a61 from "./app-definitions/google-sheets.json" with { type: "json" };
|
||||
import a62 from "./app-definitions/google-slides.json" with { type: "json" };
|
||||
import a63 from "./app-definitions/google-calendar.json" with { type: "json" };
|
||||
import a64 from "./app-definitions/google-chat.json" with { type: "json" };
|
||||
import a65 from "./app-definitions/google-people.json" with { type: "json" };
|
||||
import a66 from "./app-definitions/google-workspace-search.json" with { type: "json" };
|
||||
import type { AppDefinition } from "./types/app-definition.js";
|
||||
export const APP_DEFINITIONS=[a0,a1,a2,a3,a4,a5,a6,a7,a8,a9,a10,a11,a12,a13,a14,a15,a16,a17,a18,a19,a20,a21,a22,a23,a24,a25,a26,a27,a28,a29,a30,a31,a32,a33,a34,a35,a36,a37,a38,a39,a40,a41,a42,a43,a44,a45,a46,a47,a48,a49,a50,a51,a52,a53,a54,a55,a56,a57,a58,a59,a60,a61,a62,a63,a64,a65] as AppDefinition[];
|
||||
export const APP_DEFINITIONS=[a0,a1,a2,a3,a4,a5,a6,a7,a8,a9,a10,a11,a12,a13,a14,a15,a16,a17,a18,a19,a20,a21,a22,a23,a24,a25,a26,a27,a28,a29,a30,a31,a32,a33,a34,a35,a36,a37,a38,a39,a40,a41,a42,a43,a44,a45,a46,a47,a48,a49,a50,a51,a52,a53,a54,a55,a56,a57,a58,a59,a60,a61,a62,a63,a64,a65,a66] as AppDefinition[];
|
||||
|
|
|
|||
|
|
@ -679,7 +679,7 @@ describe("AppDefinition catalog", () => {
|
|||
"ticktick",
|
||||
"xero",
|
||||
]);
|
||||
expect(APP_STORE_DEFINITIONS).toHaveLength(41);
|
||||
expect(APP_STORE_DEFINITIONS).toHaveLength(42);
|
||||
const connectableSlugs = new Set(
|
||||
CONNECTABLE_APP_DEFINITIONS.map((entry) => entry.slug),
|
||||
);
|
||||
|
|
@ -691,7 +691,7 @@ describe("AppDefinition catalog", () => {
|
|||
expect(storeSlugs.has(slug), slug).toBe(false);
|
||||
}
|
||||
});
|
||||
it("ships complete local branding provenance for all 41 store-visible providers", () => {
|
||||
it("ships complete local branding provenance for all 42 store-visible providers", () => {
|
||||
const uiPublic = path.resolve(
|
||||
path.dirname(fileURLToPath(import.meta.url)),
|
||||
"../../../ui/public",
|
||||
|
|
@ -711,14 +711,14 @@ describe("AppDefinition catalog", () => {
|
|||
}>;
|
||||
};
|
||||
const visible = manifest.providers.filter((entry) => entry.catalogVisible);
|
||||
expect(visible).toHaveLength(41);
|
||||
expect(visible).toHaveLength(42);
|
||||
expect(new Set(visible.map((entry) => entry.slug))).toHaveProperty(
|
||||
"size",
|
||||
41,
|
||||
42,
|
||||
);
|
||||
expect(new Set(visible.map((entry) => entry.localAsset))).toHaveProperty(
|
||||
"size",
|
||||
41,
|
||||
42,
|
||||
);
|
||||
expect(new Set(APP_STORE_DEFINITIONS.map((entry) => entry.slug))).toEqual(
|
||||
new Set(visible.map((entry) => entry.slug)),
|
||||
|
|
|
|||
|
|
@ -27,6 +27,7 @@ export const CONNECTABLE_APP_SLUGS = new Set([
|
|||
"discord",
|
||||
"microsoft-teams",
|
||||
"telegram",
|
||||
"imessage-photon",
|
||||
]);
|
||||
|
||||
export const CONNECTABLE_APP_DEFINITIONS = APP_DEFINITIONS.filter((app) =>
|
||||
|
|
|
|||
|
|
@ -0,0 +1,50 @@
|
|||
{
|
||||
"schemaVersion": 1,
|
||||
"slug": "imessage-photon",
|
||||
"name": "iMessage Photon",
|
||||
"description": "Message a Paperclip agent from Apple Messages using Photon Cloud. Pro supports DMs; dedicated lines also support groups.",
|
||||
"categories": [
|
||||
"communication"
|
||||
],
|
||||
"featured": false,
|
||||
"branding": {
|
||||
"logoUrl": "/brands/apps/imessage-photon.png"
|
||||
},
|
||||
"urlPatterns": [
|
||||
"https://photon.codes/*"
|
||||
],
|
||||
"methods": [
|
||||
{
|
||||
"key": "chat-agent",
|
||||
"label": "Chat with an agent",
|
||||
"purpose": "channel",
|
||||
"provider": "imessage-photon",
|
||||
"transport": "chat_sdk",
|
||||
"auth": "api_key",
|
||||
"ownershipModes": [
|
||||
"customer"
|
||||
],
|
||||
"whenToUse": "Let people in iMessage Photon start and continue work with one Paperclip agent.",
|
||||
"credentialFields": [
|
||||
{
|
||||
"key": "projectSecret",
|
||||
"label": "Project secret",
|
||||
"type": "password",
|
||||
"required": true,
|
||||
"placeholder": "Photon project secret",
|
||||
"secret": true
|
||||
}
|
||||
],
|
||||
"guidanceMd": "Connect a Photon Cloud project. Pro shared lines support DMs after sender enrollment in Photon and identity linking in Paperclip. Dedicated lines also support individually enabled groups.",
|
||||
"consoleLinks": {
|
||||
"register": "https://photon.codes/",
|
||||
"docs": "https://photon.codes/docs/spectrum-ts/providers/imessage/connection-and-routing"
|
||||
},
|
||||
"riskTier": "S3",
|
||||
"requiredResourceFilters": [
|
||||
"direct_message",
|
||||
"group_chat"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
@ -2,7 +2,7 @@ import type { ConnectionGrantKind, ToolConnectionOwnership, ToolConnectionPurpos
|
|||
export type AppCategory = "ai"|"analytics"|"commerce"|"communication"|"content"|"data"|"developer"|"productivity"|"other";
|
||||
export type OAuthRedirectConstraints = "https-or-loopback-http";
|
||||
export interface FieldDef { key:string; label:string; type:"text"|"password"|"textarea"|"datetime"|"select"|"checkbox"; required?:boolean; advanced?:boolean; hidden?:boolean; placeholder?:string; helperMd?:string; secret?:boolean; prefix?:string; defaultValue?:string|boolean; validation?:{pattern?:string;maxLength?:number}; options?:Array<{value:string;label:string}>; transport?:{location:"query"|"header";name:string;format?:"string"|"csv"|"boolean";omitFalse?:boolean} }
|
||||
export interface ConnectionMethodDef { key:string; label?:string; purpose?:ToolConnectionPurpose; provider?:"slack"|"github"|"discord"|"microsoft-teams"|"telegram" | "agentmail"; transport:ToolConnectionTransport; auth:"oauth"|"api_key"|"none"; oauthStrategy?:"paperclip_cloud_connector"|"paperclip_id_connector"; connectorProfile?:string; capabilityProfile?:{key:string;label:string;description?:string}; grantKinds?:ConnectionGrantKind[]; ownershipModes:ToolConnectionOwnership[]; whenToUse:string; defaults?:{serverUrl?:string;serverUrlTemplate?:string;discoveryUrl?:string|null;serviceHost?:string;templateKey?:string;authorizationEndpoint?:string;tokenEndpoint?:string;metadataUrl?:string;scopesHint?:string[];oauthAuthorizationParams?:{access_type?:"offline";prompt?:"consent"};toolArgumentDefaults?:Record<string,unknown>}; tenantFields?:FieldDef[]; extensionFields?:FieldDef[]; configRequirements?:{atLeastOneOf?:string[]}; credentialFields?:FieldDef[]; keyPlacement?:{location:"header"|"query"|"body_json"|"env";name:string;prefix?:string|null}; credentialSources?:{vercelConnect?:{services:string[];principalModes:VercelConnectPrincipalMode[];scopes:string[];header:{name:string;prefix?:string|null}}}; guidanceMd:string; consoleLinks?:{register?:string;keys?:string;settings?:string;docs?:string}; warnings?:string[]; variants?:Array<{key:string;label:string;whenToUse:string;tenantFields?:FieldDef[]}>; riskTier:"S1"|"S2"|"S3"|"S4"; requiredResourceFilters?:string[] }
|
||||
export interface ConnectionMethodDef { key:string; label?:string; purpose?:ToolConnectionPurpose; provider?:"slack"|"github"|"discord"|"microsoft-teams"|"telegram" | "agentmail" | "imessage-photon"; transport:ToolConnectionTransport; auth:"oauth"|"api_key"|"none"; oauthStrategy?:"paperclip_cloud_connector"|"paperclip_id_connector"; connectorProfile?:string; capabilityProfile?:{key:string;label:string;description?:string}; grantKinds?:ConnectionGrantKind[]; ownershipModes:ToolConnectionOwnership[]; whenToUse:string; defaults?:{serverUrl?:string;serverUrlTemplate?:string;discoveryUrl?:string|null;serviceHost?:string;templateKey?:string;authorizationEndpoint?:string;tokenEndpoint?:string;metadataUrl?:string;scopesHint?:string[];oauthAuthorizationParams?:{access_type?:"offline";prompt?:"consent"};toolArgumentDefaults?:Record<string,unknown>}; tenantFields?:FieldDef[]; extensionFields?:FieldDef[]; configRequirements?:{atLeastOneOf?:string[]}; credentialFields?:FieldDef[]; keyPlacement?:{location:"header"|"query"|"body_json"|"env";name:string;prefix?:string|null}; credentialSources?:{vercelConnect?:{services:string[];principalModes:VercelConnectPrincipalMode[];scopes:string[];header:{name:string;prefix?:string|null}}}; guidanceMd:string; consoleLinks?:{register?:string;keys?:string;settings?:string;docs?:string}; warnings?:string[]; variants?:Array<{key:string;label:string;whenToUse:string;tenantFields?:FieldDef[]}>; riskTier:"S1"|"S2"|"S3"|"S4"; requiredResourceFilters?:string[] }
|
||||
export interface AppDefinition { schemaVersion:1; slug:string; name:string; description:string; categories:AppCategory[]; featured?:boolean; branding:{logoUrl:string;darkLogoUrl?:string;backgroundColor?:string;accentColor?:string}; urlPatterns:string[]; docsUrl?:string; setupPrerequisite?:{title:string;description:string;steps?:string[];actionLabel:string;actionUrl:string}; redirectConstraints?:OAuthRedirectConstraints; methods:ConnectionMethodDef[]; suggestable?:boolean; availability?:{available:boolean;reason?:string;robotEmail?:string}; ownershipAvailability?:Partial<Record<ToolConnectionOwnership,boolean>> }
|
||||
|
||||
export type SelfServeMcpAuthMode =
|
||||
|
|
|
|||
|
|
@ -6,6 +6,7 @@ export const CHAT_PROVIDERS = [
|
|||
"microsoft-teams",
|
||||
"telegram",
|
||||
"agentmail",
|
||||
"imessage-photon",
|
||||
] as const;
|
||||
export type ChatProvider = (typeof CHAT_PROVIDERS)[number];
|
||||
|
||||
|
|
@ -226,6 +227,7 @@ export interface ChatEndpoint {
|
|||
botUsername?: string | null;
|
||||
botLabel?: string | null;
|
||||
botAvatarUrl?: string | null;
|
||||
photonAllocation?: "dedicated" | "shared";
|
||||
allowDirectMessages: boolean;
|
||||
allowGroupChats: boolean;
|
||||
allowUnlinkedPeople: boolean;
|
||||
|
|
@ -255,6 +257,7 @@ export interface ChatEndpointResource {
|
|||
enabled: boolean;
|
||||
createdAt: string;
|
||||
updatedAt: string;
|
||||
participants?: string[];
|
||||
}
|
||||
|
||||
export interface ChatExternalPrincipal {
|
||||
|
|
@ -467,6 +470,7 @@ export interface UpdateChatEndpointInput {
|
|||
export interface ConfigureChatEndpointInput {
|
||||
action: "configure" | "verify" | "pause" | "resume" | "reconnect" | "remove";
|
||||
credentials?: Record<string, string>;
|
||||
photon?: PhotonChannelConfiguration;
|
||||
}
|
||||
|
||||
export interface NormalizedChatEvent {
|
||||
|
|
@ -503,3 +507,15 @@ export interface NormalizedChatEvent {
|
|||
};
|
||||
raw: Record<string, unknown>;
|
||||
}
|
||||
|
||||
/** Safe Photon project inspection; credentials and line tokens are never serialized. */
|
||||
export interface PhotonProjectInspection {
|
||||
projectId: string;
|
||||
projectName: string;
|
||||
allocation: "dedicated" | "shared";
|
||||
eligible: boolean;
|
||||
lines: Array<{ lineId: string; phoneNumber: string; eligible: boolean; unavailableReason?: string }>;
|
||||
}
|
||||
export type PhotonChannelConfiguration =
|
||||
| { allocation?: "dedicated"; projectId: string; lineId: string }
|
||||
| { allocation: "shared"; projectId: string };
|
||||
|
|
|
|||
|
|
@ -1057,6 +1057,8 @@ export interface IssueCommentMetadataSection {
|
|||
|
||||
export interface IssueCommentMetadata {
|
||||
version: 1;
|
||||
/** Inbound channel attribution; never an authorization input. */
|
||||
sourceChannel?: "imessage-photon";
|
||||
sourceRunId?: string | null;
|
||||
sourceIdentityContextId?: string | null;
|
||||
authorizationReason?: string | null;
|
||||
|
|
|
|||
|
|
@ -5,6 +5,6 @@ const appBrandAssetUrlSchema=z.string().refine((value)=>{
|
|||
try{return new URL(value).protocol==="https:";}catch{return false;}
|
||||
},{message:"Brand assets must be HTTPS URLs or local /brands/apps SVG/PNG paths"});
|
||||
const field=z.object({key:z.string().min(1),label:z.string().min(1),type:z.enum(["text","password","textarea","datetime","select","checkbox"]),required:z.boolean().optional(),advanced:z.boolean().optional(),hidden:z.boolean().optional(),placeholder:z.string().optional(),helperMd:z.string().optional(),secret:z.boolean().optional(),prefix:z.string().optional(),defaultValue:z.union([z.string(),z.boolean()]).optional(),validation:z.object({pattern:z.string().optional(),maxLength:z.number().int().positive().optional()}).optional(),options:z.array(z.object({value:z.string(),label:z.string()})).optional(),transport:z.object({location:z.enum(["query","header"]),name:z.string().min(1),format:z.enum(["string","csv","boolean"]).optional(),omitFalse:z.boolean().optional()}).optional()}).superRefine((v,c)=>{if(v.required&&v.type!=="checkbox"&&!v.placeholder)c.addIssue({code:"custom",message:"Required fields need placeholders",path:["placeholder"]});if(v.type==="select"&&(!v.options||v.options.length===0))c.addIssue({code:"custom",message:"Select fields need options",path:["options"]});if(v.hidden&&v.defaultValue===undefined)c.addIssue({code:"custom",message:"Hidden fields need defaults",path:["defaultValue"]})});
|
||||
export const connectionMethodDefSchema=z.object({key:z.string().min(1),label:z.string().min(1).optional(),purpose:toolConnectionPurposeSchema.optional(),provider:z.enum(["slack","github","discord","microsoft-teams","telegram","agentmail"]).optional(),transport:toolConnectionTransportSchema,auth:z.enum(["oauth","api_key","none"]),oauthStrategy:z.enum(["paperclip_cloud_connector","paperclip_id_connector"]).optional(),connectorProfile:z.string().regex(/^[a-z0-9]+(?:[.-][a-z0-9]+)*$/).optional(),capabilityProfile:z.object({key:z.string().min(1),label:z.string().min(1),description:z.string().min(1).optional()}).optional(),grantKinds:z.array(connectionGrantKindSchema).min(1).optional(),ownershipModes:z.array(toolConnectionOwnershipSchema).min(1),whenToUse:z.string().min(1),defaults:z.object({serverUrl:z.string().url().optional(),serverUrlTemplate:z.string().regex(/^https:\/\//).optional(),discoveryUrl:z.string().url().nullable().optional(),serviceHost:z.string().optional(),templateKey:z.string().optional(),authorizationEndpoint:z.string().url().optional(),tokenEndpoint:z.string().url().optional(),metadataUrl:z.string().url().optional(),scopesHint:z.array(z.string()).optional(),oauthAuthorizationParams:z.object({access_type:z.literal("offline").optional(),prompt:z.literal("consent").optional()}).optional(),toolArgumentDefaults:z.record(z.string(),z.unknown()).optional()}).optional(),tenantFields:z.array(field).optional(),extensionFields:z.array(field).optional(),configRequirements:z.object({atLeastOneOf:z.array(z.string().min(1)).min(1).optional()}).optional(),credentialFields:z.array(field).optional(),keyPlacement:z.object({location:z.enum(["header","query","body_json","env"]),name:z.string().min(1),prefix:z.string().nullable().optional()}).optional(),credentialSources:z.object({vercelConnect:z.object({services:z.array(z.string().min(1)).min(1),principalModes:z.array(z.enum(["app","user"])).min(1),scopes:z.array(z.string().min(1)).min(1),header:z.object({name:z.string().min(1),prefix:z.string().nullable().optional()})}).optional()}).optional(),guidanceMd:z.string().min(1),consoleLinks:z.object({register:z.string().url().optional(),keys:z.string().url().optional(),settings:z.string().url().optional(),docs:z.string().url().optional()}).optional(),warnings:z.array(z.string()).optional(),variants:z.array(z.object({key:z.string(),label:z.string(),whenToUse:z.string(),tenantFields:z.array(field).optional()})).optional(),riskTier:z.enum(["S1","S2","S3","S4"]),requiredResourceFilters:z.array(z.string()).optional()}).superRefine((v,c)=>{const purpose=v.purpose??"tool";if(v.transport==="chat_sdk"&&purpose!=="channel")c.addIssue({code:"custom",message:"Chat SDK methods must be channel connections",path:["purpose"]});if(purpose==="channel"&&v.transport!=="chat_sdk"&&!(v.provider==="agentmail"&&v.transport==="rest_api"))c.addIssue({code:"custom",message:"Channel connections must use the Chat SDK transport",path:["transport"]});if(purpose==="channel"&&!v.provider)c.addIssue({code:"custom",message:"Channel connections require a chat provider",path:["provider"]});if(v.auth==="api_key"&&!v.keyPlacement&&purpose!=="channel")c.addIssue({code:"custom",message:"API-key tool methods require keyPlacement",path:["keyPlacement"]});if(v.oauthStrategy&&v.auth!=="oauth")c.addIssue({code:"custom",message:"OAuth strategies require OAuth auth",path:["oauthStrategy"]});if(v.oauthStrategy&&!v.connectorProfile)c.addIssue({code:"custom",message:"Paperclip Cloud connector methods require connectorProfile",path:["connectorProfile"]});if(v.connectorProfile&&!v.oauthStrategy)c.addIssue({code:"custom",message:"connectorProfile requires a Paperclip Cloud OAuth strategy",path:["connectorProfile"]});if(v.credentialSources?.vercelConnect&&(v.transport!=="mcp_remote"||v.auth==="none"))c.addIssue({code:"custom",message:"Vercel Connect requires an authenticated remote MCP method",path:["credentialSources","vercelConnect"]});const keys=new Set([...(v.tenantFields??[]),...(v.extensionFields??[])].map((entry)=>entry.key));for(const key of v.configRequirements?.atLeastOneOf??[])if(!keys.has(key))c.addIssue({code:"custom",message:"Config requirement references an unknown field",path:["configRequirements","atLeastOneOf"]});if(v.defaults?.serverUrl&&v.defaults.serverUrlTemplate)c.addIssue({code:"custom",message:"Use either serverUrl or serverUrlTemplate",path:["defaults"]});for(const placeholder of v.defaults?.serverUrlTemplate?.matchAll(/\{([a-zA-Z0-9_-]+)\}/g)??[])if(!keys.has(placeholder[1]))c.addIssue({code:"custom",message:"Server URL template references an unknown field",path:["defaults","serverUrlTemplate"]})});
|
||||
export const connectionMethodDefSchema=z.object({key:z.string().min(1),label:z.string().min(1).optional(),purpose:toolConnectionPurposeSchema.optional(),provider:z.enum(["slack","github","discord","microsoft-teams","telegram","agentmail","imessage-photon"]).optional(),transport:toolConnectionTransportSchema,auth:z.enum(["oauth","api_key","none"]),oauthStrategy:z.enum(["paperclip_cloud_connector","paperclip_id_connector"]).optional(),connectorProfile:z.string().regex(/^[a-z0-9]+(?:[.-][a-z0-9]+)*$/).optional(),capabilityProfile:z.object({key:z.string().min(1),label:z.string().min(1),description:z.string().min(1).optional()}).optional(),grantKinds:z.array(connectionGrantKindSchema).min(1).optional(),ownershipModes:z.array(toolConnectionOwnershipSchema).min(1),whenToUse:z.string().min(1),defaults:z.object({serverUrl:z.string().url().optional(),serverUrlTemplate:z.string().regex(/^https:\/\//).optional(),discoveryUrl:z.string().url().nullable().optional(),serviceHost:z.string().optional(),templateKey:z.string().optional(),authorizationEndpoint:z.string().url().optional(),tokenEndpoint:z.string().url().optional(),metadataUrl:z.string().url().optional(),scopesHint:z.array(z.string()).optional(),oauthAuthorizationParams:z.object({access_type:z.literal("offline").optional(),prompt:z.literal("consent").optional()}).optional(),toolArgumentDefaults:z.record(z.string(),z.unknown()).optional()}).optional(),tenantFields:z.array(field).optional(),extensionFields:z.array(field).optional(),configRequirements:z.object({atLeastOneOf:z.array(z.string().min(1)).min(1).optional()}).optional(),credentialFields:z.array(field).optional(),keyPlacement:z.object({location:z.enum(["header","query","body_json","env"]),name:z.string().min(1),prefix:z.string().nullable().optional()}).optional(),credentialSources:z.object({vercelConnect:z.object({services:z.array(z.string().min(1)).min(1),principalModes:z.array(z.enum(["app","user"])).min(1),scopes:z.array(z.string().min(1)).min(1),header:z.object({name:z.string().min(1),prefix:z.string().nullable().optional()})}).optional()}).optional(),guidanceMd:z.string().min(1),consoleLinks:z.object({register:z.string().url().optional(),keys:z.string().url().optional(),settings:z.string().url().optional(),docs:z.string().url().optional()}).optional(),warnings:z.array(z.string()).optional(),variants:z.array(z.object({key:z.string(),label:z.string(),whenToUse:z.string(),tenantFields:z.array(field).optional()})).optional(),riskTier:z.enum(["S1","S2","S3","S4"]),requiredResourceFilters:z.array(z.string()).optional()}).superRefine((v,c)=>{const purpose=v.purpose??"tool";if(v.transport==="chat_sdk"&&purpose!=="channel")c.addIssue({code:"custom",message:"Chat SDK methods must be channel connections",path:["purpose"]});if(purpose==="channel"&&v.transport!=="chat_sdk"&&!(v.provider==="agentmail"&&v.transport==="rest_api"))c.addIssue({code:"custom",message:"Channel connections must use the Chat SDK transport",path:["transport"]});if(purpose==="channel"&&!v.provider)c.addIssue({code:"custom",message:"Channel connections require a chat provider",path:["provider"]});if(v.auth==="api_key"&&!v.keyPlacement&&purpose!=="channel")c.addIssue({code:"custom",message:"API-key tool methods require keyPlacement",path:["keyPlacement"]});if(v.oauthStrategy&&v.auth!=="oauth")c.addIssue({code:"custom",message:"OAuth strategies require OAuth auth",path:["oauthStrategy"]});if(v.oauthStrategy&&!v.connectorProfile)c.addIssue({code:"custom",message:"Paperclip Cloud connector methods require connectorProfile",path:["connectorProfile"]});if(v.connectorProfile&&!v.oauthStrategy)c.addIssue({code:"custom",message:"connectorProfile requires a Paperclip Cloud OAuth strategy",path:["connectorProfile"]});if(v.credentialSources?.vercelConnect&&(v.transport!=="mcp_remote"||v.auth==="none"))c.addIssue({code:"custom",message:"Vercel Connect requires an authenticated remote MCP method",path:["credentialSources","vercelConnect"]});const keys=new Set([...(v.tenantFields??[]),...(v.extensionFields??[])].map((entry)=>entry.key));for(const key of v.configRequirements?.atLeastOneOf??[])if(!keys.has(key))c.addIssue({code:"custom",message:"Config requirement references an unknown field",path:["configRequirements","atLeastOneOf"]});if(v.defaults?.serverUrl&&v.defaults.serverUrlTemplate)c.addIssue({code:"custom",message:"Use either serverUrl or serverUrlTemplate",path:["defaults"]});for(const placeholder of v.defaults?.serverUrlTemplate?.matchAll(/\{([a-zA-Z0-9_-]+)\}/g)??[])if(!keys.has(placeholder[1]))c.addIssue({code:"custom",message:"Server URL template references an unknown field",path:["defaults","serverUrlTemplate"]})});
|
||||
export const appDefinitionSchema=z.object({schemaVersion:z.literal(1),slug:z.string().regex(/^[a-z0-9]+(?:-[a-z0-9]+)*$/),name:z.string().min(1),description:z.string().min(1),categories:z.array(z.enum(["ai","analytics","commerce","communication","content","data","developer","productivity","other"])).min(1),featured:z.boolean().optional(),branding:z.object({logoUrl:appBrandAssetUrlSchema,darkLogoUrl:appBrandAssetUrlSchema.optional(),backgroundColor:z.string().optional(),accentColor:z.string().optional()}),urlPatterns:z.array(z.string()),docsUrl:z.string().url().optional(),setupPrerequisite:z.object({title:z.string().min(1),description:z.string().min(1),steps:z.array(z.string().min(1)).min(1).optional(),actionLabel:z.string().min(1),actionUrl:z.string().url()} ).optional(),redirectConstraints:z.enum(["https-or-loopback-http"]).optional(),methods:z.array(connectionMethodDefSchema).min(1),suggestable:z.boolean().optional(),availability:z.object({available:z.boolean(),reason:z.string().optional(),robotEmail:z.string().optional()}).optional(),ownershipAvailability:z.object({platform_shared:z.boolean().optional(),platform_provisioned:z.boolean().optional(),customer:z.boolean().optional(),dcr:z.boolean().optional()}).optional()});
|
||||
export const appDefinitionsSchema=z.array(appDefinitionSchema).superRefine((v,c)=>{const s=new Set<string>();v.forEach((a,i)=>{if(s.has(a.slug))c.addIssue({code:"custom",message:"Duplicate slug",path:[i,"slug"]});s.add(a.slug)})});
|
||||
|
|
|
|||
|
|
@ -87,6 +87,17 @@ export const updateChatEndpointSchema = z
|
|||
message: "At least one chat endpoint field is required",
|
||||
});
|
||||
|
||||
export const photonProjectIdSchema = z.string().trim().min(1).max(128).regex(/^[a-zA-Z0-9_-]+$/);
|
||||
export const photonLineIdSchema = z.string().trim().min(1).max(63).regex(/^[a-zA-Z0-9](?:[a-zA-Z0-9-]*[a-zA-Z0-9])?$/);
|
||||
export const photonChannelConfigurationSchema = z.union([
|
||||
z.object({ allocation: z.literal("dedicated").default("dedicated"), projectId: photonProjectIdSchema, lineId: photonLineIdSchema }).strict(),
|
||||
z.object({ allocation: z.literal("shared"), projectId: photonProjectIdSchema }).strict(),
|
||||
]);
|
||||
export const inspectPhotonProjectSchema = z.object({
|
||||
projectId: photonProjectIdSchema,
|
||||
projectSecret: z.string().min(1).max(4096),
|
||||
}).strict();
|
||||
|
||||
export const configureChatEndpointSchema = z
|
||||
.object({
|
||||
action: z.enum([
|
||||
|
|
@ -98,11 +109,12 @@ export const configureChatEndpointSchema = z
|
|||
"remove",
|
||||
]),
|
||||
credentials: chatEndpointCredentialsSchema.optional(),
|
||||
photon: photonChannelConfigurationSchema.optional(),
|
||||
})
|
||||
.strict()
|
||||
.superRefine((value, ctx) => {
|
||||
if (
|
||||
value.credentials &&
|
||||
(value.credentials || value.photon) &&
|
||||
value.action !== "configure" &&
|
||||
value.action !== "reconnect"
|
||||
) {
|
||||
|
|
|
|||
|
|
@ -1000,6 +1000,7 @@ export const issueCommentMetadataSectionSchema = z
|
|||
export const issueCommentMetadataSchema = z
|
||||
.object({
|
||||
version: z.literal(1),
|
||||
sourceChannel: z.literal("imessage-photon").optional(),
|
||||
sourceRunId: z.string().guid().nullable().optional(),
|
||||
authorizationReason: z
|
||||
.string()
|
||||
|
|
|
|||
|
|
@ -65,6 +65,7 @@ const chatProviderName = (provider) =>
|
|||
"microsoft-teams": "Microsoft Teams",
|
||||
slack: "Slack",
|
||||
telegram: "Telegram",
|
||||
"imessage-photon": "iMessage Photon",
|
||||
})[provider];
|
||||
const channelMethod = (
|
||||
provider,
|
||||
|
|
@ -364,6 +365,14 @@ const apps = [
|
|||
},
|
||||
),
|
||||
],
|
||||
[
|
||||
"imessage-photon", "iMessage Photon",
|
||||
"Message a Paperclip agent from Apple Messages using Photon Cloud. Pro supports DMs; dedicated lines also support groups.",
|
||||
"communication", "photon.codes", ["https://photon.codes/*"],
|
||||
channelMethod("imessage-photon", [field("projectSecret", "Project secret", "Photon project secret")], ["direct_message", "group_chat"],
|
||||
"Connect a Photon Cloud project. Pro shared lines support DMs after sender enrollment in Photon and identity linking in Paperclip. Dedicated lines also support individually enabled groups.",
|
||||
{ register: "https://photon.codes/", docs: "https://photon.codes/docs/spectrum-ts/providers/imessage/connection-and-routing" }),
|
||||
],
|
||||
[
|
||||
"telegram",
|
||||
"Telegram",
|
||||
|
|
|
|||
|
|
@ -51,6 +51,7 @@
|
|||
"@chat-adapter/teams": "4.39.0",
|
||||
"@chat-adapter/telegram": "4.39.0",
|
||||
"@discordjs/ws": "1.2.3",
|
||||
"@grpc/grpc-js": "1.14.4",
|
||||
"@opentelemetry/api": "^1.9.0",
|
||||
"@paperclipai/adapter-claude-local": "workspace:*",
|
||||
"@paperclipai/adapter-codex-local": "workspace:*",
|
||||
|
|
@ -68,6 +69,7 @@
|
|||
"@paperclipai/plugin-sdk": "workspace:*",
|
||||
"@paperclipai/shared": "workspace:*",
|
||||
"@paperclipai/skills-catalog": "workspace:*",
|
||||
"@photon-ai/advanced-imessage": "2.1.0",
|
||||
"@vercel/connect": "0.6.1",
|
||||
"acpx": "0.13.1",
|
||||
"ajv": "^8.20.0",
|
||||
|
|
@ -81,8 +83,11 @@
|
|||
"drizzle-orm": "^0.45.2",
|
||||
"embedded-postgres": "^18.1.0-beta.16",
|
||||
"express": "^5.1.0",
|
||||
"heif2jpeg": "0.1.6",
|
||||
"jsdom": "^30.0.1",
|
||||
"multer": "^2.2.0",
|
||||
"nice-grpc": "2.1.17",
|
||||
"nice-grpc-common": "2.0.4",
|
||||
"open": "^11.0.1",
|
||||
"pino": "^10.0.0",
|
||||
"pino-http": "^11.0.0",
|
||||
|
|
|
|||
|
|
@ -385,6 +385,7 @@ describe("openapi routes", () => {
|
|||
["post", "/api/chat-endpoints/{endpointId}/setup"],
|
||||
["post", "/api/chat-endpoints/{endpointId}/setup-secret"],
|
||||
["post", "/api/chat-endpoints/{endpointId}/test"],
|
||||
["post", "/api/chat-endpoints/{endpointId}/photon/inspect"],
|
||||
["get", "/api/chat-endpoints/{endpointId}/resources"],
|
||||
["put", "/api/chat-endpoints/{endpointId}/resources"],
|
||||
["get", "/api/chat-endpoints/{endpointId}/principals"],
|
||||
|
|
@ -449,7 +450,7 @@ describe("openapi routes", () => {
|
|||
properties: {
|
||||
provider: {
|
||||
type: "string",
|
||||
enum: ["slack", "github", "discord", "microsoft-teams", "telegram"],
|
||||
enum: ["slack", "github", "discord", "microsoft-teams", "telegram", "imessage-photon"],
|
||||
},
|
||||
assignedAgentId: { type: "string", format: "uuid" },
|
||||
},
|
||||
|
|
@ -509,6 +510,21 @@ describe("openapi routes", () => {
|
|||
);
|
||||
expect(setup.responses["409"]).toBeDefined();
|
||||
expect(setup.responses["422"]).toBeDefined();
|
||||
expect(setup.responses["502"]).toBeDefined();
|
||||
expect(setup.responses["503"]).toBeDefined();
|
||||
|
||||
const photon = spec.paths["/api/chat-endpoints/{endpointId}/photon/inspect"].post;
|
||||
expect(photon.requestBody.content["application/json"].schema.required).toEqual([
|
||||
"projectId", "projectSecret",
|
||||
]);
|
||||
const photonResponse = photon.responses["200"].content["application/json"].schema;
|
||||
expect(photonResponse.properties.allocation.enum).toEqual(["dedicated", "shared"]);
|
||||
expect(photonResponse.properties.lines.items.additionalProperties).toBe(false);
|
||||
expect(JSON.stringify(photonResponse)).not.toMatch(/projectSecret|token/);
|
||||
expect(photon.responses["422"]).toBeDefined();
|
||||
expect(photon.responses["429"]).toBeDefined();
|
||||
expect(photon.responses["502"]).toBeDefined();
|
||||
expect(photon.responses["503"]).toBeDefined();
|
||||
|
||||
const setupSecret =
|
||||
spec.paths["/api/chat-endpoints/{endpointId}/setup-secret"].post;
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load Diff
|
|
@ -0,0 +1,222 @@
|
|||
import { vi } from "vitest";
|
||||
import {
|
||||
TypedEventStream,
|
||||
type GrpcAdvancedIMessage,
|
||||
type Chat,
|
||||
type Message,
|
||||
type LiveEvent,
|
||||
type CatchUpEvent,
|
||||
} from "@photon-ai/advanced-imessage";
|
||||
import type {
|
||||
ChatSdkStatePersistence,
|
||||
ChatSdkStateRecord,
|
||||
} from "../../services/chat-sdk-state.js";
|
||||
import { PhotonState } from "../../services/photon/state.js";
|
||||
import {
|
||||
PhotonCloudClient,
|
||||
PhotonLineAuthentication,
|
||||
} from "../../services/photon/cloud.js";
|
||||
import {
|
||||
PhotonChatAdapter,
|
||||
photonThreadId,
|
||||
} from "../../services/photon/adapter.js";
|
||||
export function memoryPersistence(): ChatSdkStatePersistence {
|
||||
const rows = new Map<string, ChatSdkStateRecord>();
|
||||
const key = (scope: { companyId: string; endpointId: string }, key: string) =>
|
||||
`${scope.companyId}:${scope.endpointId}:${key}`;
|
||||
return {
|
||||
async read(scope, id) {
|
||||
return structuredClone(rows.get(key(scope, id)) ?? null);
|
||||
},
|
||||
async compareAndSet(input) {
|
||||
const id = key(input, input.key),
|
||||
current = rows.get(id);
|
||||
if ((current?.version ?? null) !== input.expectedVersion) return false;
|
||||
rows.set(id, {
|
||||
value: structuredClone(input.value),
|
||||
version: (current?.version ?? 0) + 1,
|
||||
expiresAt: input.expiresAt,
|
||||
});
|
||||
return true;
|
||||
},
|
||||
async deleteIfVersion(input) {
|
||||
const id = key(input, input.key);
|
||||
if (rows.get(id)?.version !== input.expectedVersion) return false;
|
||||
return rows.delete(id);
|
||||
},
|
||||
};
|
||||
}
|
||||
export function stream<T>(values: T[]): TypedEventStream<T> {
|
||||
return new TypedEventStream(
|
||||
(async function* () {
|
||||
for (const value of values) yield value;
|
||||
})(),
|
||||
async () => {},
|
||||
);
|
||||
}
|
||||
export function photonChat(
|
||||
guid = "iMessage;-;+15555550101",
|
||||
isGroup = false,
|
||||
): Chat {
|
||||
return {
|
||||
guid,
|
||||
service: "iMessage",
|
||||
isGroup,
|
||||
isArchived: false,
|
||||
displayName: isGroup ? "Test group" : "",
|
||||
participants: [{ address: "+15555550101", service: "iMessage" }],
|
||||
properties: {},
|
||||
} as unknown as Chat;
|
||||
}
|
||||
export function photonMessage(
|
||||
guid: string,
|
||||
chatGuid: string,
|
||||
text = "Hello",
|
||||
fromMe = false,
|
||||
): Message {
|
||||
return {
|
||||
guid,
|
||||
chatGuids: [chatGuid],
|
||||
content: { text, attachments: [], parts: [], mentions: [] },
|
||||
sender: { address: "+15555550101", service: "iMessage" },
|
||||
dateCreated: new Date(),
|
||||
isFromMe: fromMe,
|
||||
isSystemMessage: false,
|
||||
isServiceMessage: false,
|
||||
} as unknown as Message;
|
||||
}
|
||||
export function photonEvent(
|
||||
sequence: number,
|
||||
chat = photonChat(),
|
||||
text = "Hello",
|
||||
): LiveEvent {
|
||||
return {
|
||||
type: "message.received",
|
||||
chatGuid: chat.guid,
|
||||
message: photonMessage(`message-${sequence}`, chat.guid, text),
|
||||
sequence,
|
||||
occurredAt: new Date(),
|
||||
isFromMe: false,
|
||||
} as LiveEvent;
|
||||
}
|
||||
export function photonFixture() {
|
||||
const persistence = memoryPersistence();
|
||||
const state = new PhotonState(
|
||||
{ companyId: "company", endpointId: "endpoint" },
|
||||
persistence,
|
||||
);
|
||||
const cloud = new PhotonCloudClient();
|
||||
const allocation = vi.spyOn(cloud, "allocation").mockResolvedValue({
|
||||
inspection: {
|
||||
projectId: "project",
|
||||
projectName: "Tests",
|
||||
allocation: "dedicated",
|
||||
eligible: true,
|
||||
lines: [{ lineId: "line", phoneNumber: "+15555550100", eligible: true }],
|
||||
},
|
||||
tokens: new Map([["line", "private-line-token"]]),
|
||||
expiresIn: 300,
|
||||
});
|
||||
const authentication = new PhotonLineAuthentication(
|
||||
{ projectId: "project", lineId: "line", phoneNumber: "+15555550100" },
|
||||
"private-project-secret",
|
||||
cloud,
|
||||
);
|
||||
const chat = photonChat();
|
||||
const receipts = new Map<string, Message>();
|
||||
const polls = new Map<
|
||||
string,
|
||||
{
|
||||
pollMessageGuid: string;
|
||||
options: { optionIdentifier: string; text: string }[];
|
||||
}
|
||||
>();
|
||||
const events: CatchUpEvent[] = [];
|
||||
const sendText = vi.fn(
|
||||
async (
|
||||
chatGuid: string,
|
||||
text: string,
|
||||
opts: { clientMessageId: string },
|
||||
) => {
|
||||
if (!receipts.has(opts.clientMessageId))
|
||||
receipts.set(
|
||||
opts.clientMessageId,
|
||||
photonMessage(`sent-${receipts.size}`, chatGuid, text, true),
|
||||
);
|
||||
return receipts.get(opts.clientMessageId)!;
|
||||
},
|
||||
);
|
||||
const client = {
|
||||
close: vi.fn(async () => {}),
|
||||
chats: {
|
||||
get: vi.fn(async () => chat),
|
||||
setTyping: vi.fn(async () => {}),
|
||||
subscribeEvents: vi.fn(() => stream([])),
|
||||
},
|
||||
messages: {
|
||||
sendText,
|
||||
sendAttachment: vi.fn(
|
||||
async (
|
||||
chatGuid: string,
|
||||
attachment: string,
|
||||
opts: { clientMessageId: string },
|
||||
) => sendText(chatGuid, attachment, opts),
|
||||
),
|
||||
edit: vi.fn(),
|
||||
get: vi.fn(async (id) => photonMessage(id, chat.guid)),
|
||||
subscribeEvents: vi.fn(() => stream([])),
|
||||
},
|
||||
attachments: {
|
||||
upload: vi.fn(async () => ({ attachment: { guid: "uploaded-file" } })),
|
||||
downloadStream: vi.fn(),
|
||||
},
|
||||
groups: { subscribeEvents: vi.fn(() => stream([])) },
|
||||
polls: {
|
||||
create: vi.fn(
|
||||
async (
|
||||
_chat: string,
|
||||
_title: string,
|
||||
labels: string[],
|
||||
opts: { clientMessageId: string },
|
||||
) => {
|
||||
if (!polls.has(opts.clientMessageId))
|
||||
polls.set(opts.clientMessageId, {
|
||||
pollMessageGuid: `poll-${polls.size}`,
|
||||
options: labels.map((text, index) => ({
|
||||
optionIdentifier: `${polls.size}-option-${index}`,
|
||||
text,
|
||||
})),
|
||||
});
|
||||
return polls.get(opts.clientMessageId)!;
|
||||
},
|
||||
),
|
||||
subscribeEvents: vi.fn(() => stream([])),
|
||||
},
|
||||
events: { catchUp: vi.fn(() => stream(events)) },
|
||||
};
|
||||
const adapter = new PhotonChatAdapter(
|
||||
"Agent",
|
||||
authentication,
|
||||
state,
|
||||
client as unknown as GrpcAdvancedIMessage,
|
||||
);
|
||||
const threadId = photonThreadId({
|
||||
lineId: "line",
|
||||
chatGuid: chat.guid,
|
||||
isGroup: false,
|
||||
});
|
||||
return {
|
||||
state,
|
||||
persistence,
|
||||
allocation,
|
||||
cloud,
|
||||
authentication,
|
||||
chat,
|
||||
client,
|
||||
adapter,
|
||||
threadId,
|
||||
receipts,
|
||||
events,
|
||||
polls,
|
||||
};
|
||||
}
|
||||
|
|
@ -0,0 +1 @@
|
|||
Synthetic HEIC fixture generated on macOS from a 16 × 16 solid RGB (40, 120, 180) PNG with `sips -s format heic`. Contains no personal photo or metadata. The test exercises the installed heif2jpeg native binary. This does not replace qualification with an actual iPhone photo.
|
||||
Binary file not shown.
|
|
@ -0,0 +1,712 @@
|
|||
import { describe, it, expect, vi } from "vitest";
|
||||
import { Client, Server, ServerCredentials, credentials } from "@grpc/grpc-js";
|
||||
import { IMessageError } from "@photon-ai/advanced-imessage";
|
||||
import type { AskUserQuestionsInteraction } from "@paperclipai/shared";
|
||||
import {
|
||||
PhotonCloudClient,
|
||||
PhotonLineAuthentication, photonSharedIdentity, photonSharedScope,
|
||||
PhotonError,
|
||||
photonFailure,
|
||||
} from "../../services/photon/cloud.js";
|
||||
import { PhotonReceiver } from "../../services/photon/receiver.js";
|
||||
import {
|
||||
PhotonChatAdapter,
|
||||
splitPhotonText,
|
||||
} from "../../services/photon/adapter.js";
|
||||
import { PhotonState } from "../../services/photon/state.js";
|
||||
import {
|
||||
photonCatchUpRequest,
|
||||
photonEnvelopeSequence,
|
||||
decodePhotonRecoveryFrame,
|
||||
PhotonRecoveryTransport,
|
||||
PHOTON_CATCHUP_PATH,
|
||||
} from "../../services/photon/recovery-transport.js";
|
||||
import {
|
||||
publishPhotonPrompt,
|
||||
parsePhotonQuestionAnswer,
|
||||
photonResponseCommand,
|
||||
} from "../../services/photon/interactions.js";
|
||||
import { photonFixture, photonEvent, stream } from "./fixture.js";
|
||||
import {
|
||||
downloadPhotonAttachment,
|
||||
takePhotonCompanion,
|
||||
} from "../../services/photon/attachments.js";
|
||||
import {
|
||||
photonHeifPreview,
|
||||
validateHeifDimensions,
|
||||
validatePhotonImage,
|
||||
} from "../../services/photon/media.js";
|
||||
import sharp from "sharp";
|
||||
import { readFile } from "node:fs/promises";
|
||||
|
||||
const question = (id = "interaction"): AskUserQuestionsInteraction =>
|
||||
({
|
||||
id,
|
||||
kind: "ask_user_questions",
|
||||
payload: {
|
||||
questions: [
|
||||
{
|
||||
id: "q1",
|
||||
prompt: "Same title",
|
||||
options: [
|
||||
{ id: "a", label: "Same" },
|
||||
{ id: "b", label: "Same" },
|
||||
],
|
||||
selectionMode: "single",
|
||||
allowOther: false,
|
||||
required: true,
|
||||
},
|
||||
],
|
||||
},
|
||||
}) as AskUserQuestionsInteraction;
|
||||
const guard = async () => {};
|
||||
|
||||
describe("Photon Cloud and fixed line identity", () => {
|
||||
it("uses Basic project auth, verifies dedicated allocation, and never exposes tokens", async () => {
|
||||
const fetcher = vi.fn(
|
||||
async (url: string | URL | Request, init?: RequestInit) => {
|
||||
expect(String(url)).toMatch(
|
||||
/^https:\/\/spectrum.photon.codes\/projects\/p\//,
|
||||
);
|
||||
expect(new Headers(init?.headers).get("authorization")).toBe(
|
||||
`Basic ${Buffer.from("p:secret").toString("base64")}`,
|
||||
);
|
||||
return Response.json({
|
||||
succeed: true,
|
||||
data: String(url).endsWith("tokens")
|
||||
? {
|
||||
type: "dedicated",
|
||||
auth: { one: "TOKEN", two: "TOKEN2" },
|
||||
numbers: { one: "+15555550100", two: "+15555550102" },
|
||||
expiresIn: 300,
|
||||
}
|
||||
: { id: "p", name: "Project" },
|
||||
});
|
||||
},
|
||||
);
|
||||
const result = await new PhotonCloudClient(fetcher).inspect("p", "secret");
|
||||
expect(result.lines).toHaveLength(2);
|
||||
expect(JSON.stringify(result)).not.toMatch(/TOKEN|secret/);
|
||||
});
|
||||
it("rejects credentials and missing dedicated lines while inspecting shared DMs without exposing tokens", async () => {
|
||||
const client = new PhotonCloudClient(
|
||||
async () => new Response("secret=BAD", { status: 401 }),
|
||||
);
|
||||
await expect(client.inspect("p", "secret")).rejects.toMatchObject({
|
||||
code: "credentials",
|
||||
});
|
||||
const shared = new PhotonCloudClient(async () =>
|
||||
Response.json({
|
||||
succeed: true,
|
||||
data: { type: "shared", token: "PRIVATE", expiresIn: 300 },
|
||||
}),
|
||||
);
|
||||
expect(await shared.inspect("p", "secret")).toMatchObject({
|
||||
eligible: true,
|
||||
allocation: "shared",
|
||||
lines: [],
|
||||
});
|
||||
const missing = new PhotonCloudClient(async () =>
|
||||
Response.json({
|
||||
succeed: true,
|
||||
data: { type: "dedicated", auth: {}, numbers: {}, expiresIn: 300 },
|
||||
}),
|
||||
);
|
||||
expect(await missing.inspect("p", "secret")).toMatchObject({
|
||||
eligible: false,
|
||||
lines: [],
|
||||
});
|
||||
});
|
||||
it("binds shared gateway tokens to one project and fences renewal and group access", async () => {
|
||||
let now = 0;
|
||||
const cloud = new PhotonCloudClient();
|
||||
const allocation = vi.spyOn(cloud, "allocation").mockResolvedValue({
|
||||
inspection: {projectId: "p", projectName: "Shared", allocation: "shared", eligible: true, lines: []},
|
||||
tokens: new Map(), sharedToken: "first", expiresIn: 60,
|
||||
});
|
||||
const identity = {allocation: "shared" as const, projectId: "p", lineId: photonSharedScope("p"), phoneNumber: photonSharedIdentity("p")};
|
||||
const auth = new PhotonLineAuthentication(identity, "secret", cloud, () => now);
|
||||
expect(auth.address).toBe("imessage.spectrum.photon.codes:443");
|
||||
await expect(auth.token()).resolves.toBe("first");
|
||||
expect(photonSharedScope("other")).not.toBe(identity.lineId);
|
||||
expect(() => new PhotonLineAuthentication({...identity, projectId: "other"}, "secret", cloud)).toThrow(/match its project/);
|
||||
const f = photonFixture();
|
||||
const adapter = new PhotonChatAdapter("Shared", auth, f.state, f.adapter.client);
|
||||
expect(() => adapter.encodeThreadId({lineId: identity.lineId, chatGuid: "group", isGroup: true})).toThrow(/direct messages only/);
|
||||
expect(() => adapter.decodeThreadId(`imessage-photon:${identity.lineId}:g:Z3JvdXA`)).toThrow(/direct messages only/);
|
||||
now = 60_000;
|
||||
allocation.mockResolvedValueOnce({inspection: {projectId:"p", projectName:"Moved", allocation:"dedicated", eligible:true, lines:[]}, tokens:new Map(), expiresIn:60});
|
||||
await expect(auth.token()).rejects.toMatchObject({code:"line_unavailable"});
|
||||
});
|
||||
it("renews only the selected line and refuses replacement identity or retired ownership", async () => {
|
||||
const f = photonFixture();
|
||||
await expect(f.authentication.token()).resolves.toBe("private-line-token");
|
||||
f.authentication.retire();
|
||||
await expect(f.authentication.token()).rejects.toMatchObject({
|
||||
code: "credentials",
|
||||
});
|
||||
const other = photonFixture();
|
||||
other.allocation.mockResolvedValueOnce({
|
||||
inspection: {
|
||||
projectId: "project",
|
||||
projectName: "Test",
|
||||
allocation: "dedicated",
|
||||
eligible: true,
|
||||
lines: [
|
||||
{ lineId: "line", phoneNumber: "+15555550999", eligible: true },
|
||||
],
|
||||
},
|
||||
tokens: new Map([["line", "TOKEN"]]),
|
||||
expiresIn: 60,
|
||||
});
|
||||
await expect(other.authentication.token()).rejects.toMatchObject({
|
||||
code: "line_unavailable",
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("Photon publication receipts", () => {
|
||||
it("keeps Unicode and paragraph order and reuses immutable receipts across restart", async () => {
|
||||
const f = photonFixture();
|
||||
const text = "😀".repeat(3999) + "\n\n" + "tail";
|
||||
expect(splitPhotonText(text).join("")).toBe(text);
|
||||
expect(
|
||||
splitPhotonText(text).every((part) => Array.from(part).length <= 4000),
|
||||
).toBe(true);
|
||||
const first = await f.adapter.publish(
|
||||
f.threadId,
|
||||
"pub",
|
||||
{ markdown: text },
|
||||
{ assertCurrent: guard },
|
||||
);
|
||||
const restarted = new PhotonChatAdapter(
|
||||
"Agent",
|
||||
f.authentication,
|
||||
new PhotonState(f.state.scope, f.persistence),
|
||||
f.adapter.client,
|
||||
);
|
||||
expect(
|
||||
await restarted.publish(
|
||||
f.threadId,
|
||||
"pub",
|
||||
{ markdown: text },
|
||||
{ assertCurrent: guard },
|
||||
),
|
||||
).toEqual(first);
|
||||
expect(f.client.messages.sendText).toHaveBeenCalledTimes(2);
|
||||
await expect(
|
||||
restarted.publish(
|
||||
f.threadId,
|
||||
"pub",
|
||||
{ markdown: "changed" },
|
||||
{ assertCurrent: guard },
|
||||
),
|
||||
).rejects.toThrow("payload changed");
|
||||
});
|
||||
it("holds an unknown send until explicit retry and reuses the same key", async () => {
|
||||
const f = photonFixture();
|
||||
const original = f.client.messages.sendText.getMockImplementation()!;
|
||||
f.client.messages.sendText.mockImplementationOnce(async (...args) => {
|
||||
await original(...args);
|
||||
throw new Error("socket closed after send");
|
||||
});
|
||||
await expect(
|
||||
f.adapter.publish(f.threadId, "pub", "hello", { assertCurrent: guard }),
|
||||
).rejects.toMatchObject({ code: "delivery_unknown" });
|
||||
await expect(
|
||||
f.adapter.publish(f.threadId, "pub", "hello", { assertCurrent: guard }),
|
||||
).rejects.toMatchObject({ code: "delivery_unknown" });
|
||||
expect(f.client.messages.sendText).toHaveBeenCalledTimes(1);
|
||||
await f.adapter.publish(f.threadId, "pub", "hello", {
|
||||
assertCurrent: guard,
|
||||
retryUnknown: true,
|
||||
});
|
||||
expect(f.receipts.size).toBe(1);
|
||||
expect(f.client.messages.sendText.mock.calls[0][2]).toEqual(
|
||||
f.client.messages.sendText.mock.calls[1][2],
|
||||
);
|
||||
});
|
||||
it("stores an upload before a failed send and never uploads it twice", async () => {
|
||||
const f = photonFixture();
|
||||
f.client.messages.sendAttachment.mockRejectedValueOnce(
|
||||
new Error("lost receipt"),
|
||||
);
|
||||
const message = {
|
||||
markdown: "",
|
||||
files: [{ filename: "photo.png", data: Buffer.from("data") }],
|
||||
};
|
||||
await expect(
|
||||
f.adapter.publish(f.threadId, "filepub", message, {
|
||||
assertCurrent: guard,
|
||||
}),
|
||||
).rejects.toMatchObject({ code: "delivery_unknown" });
|
||||
await f.adapter.publish(f.threadId, "filepub", message, {
|
||||
assertCurrent: guard,
|
||||
retryUnknown: true,
|
||||
});
|
||||
expect(f.client.attachments.upload).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
it("retains unknown delivery when the shared gateway rejects a duplicate without a receipt", async () => {
|
||||
const f = photonFixture();
|
||||
f.client.messages.sendText.mockRejectedValueOnce(new Error("lost receipt"));
|
||||
await expect(f.adapter.publish(f.threadId, "duplicate", "hello", {
|
||||
assertCurrent: guard,
|
||||
})).rejects.toMatchObject({ code: "delivery_unknown" });
|
||||
// Observed on Photon Pro: ALREADY_EXISTS arrives as internalError, without
|
||||
// a message GUID. Neither the wording nor duplicate status proves a receipt.
|
||||
f.client.messages.sendText.mockRejectedValueOnce(new IMessageError(
|
||||
"[upstream] Operation already processed with this client message ID",
|
||||
{ code: "internalError", grpcCode: 6, retryable: false },
|
||||
));
|
||||
await expect(f.adapter.publish(f.threadId, "duplicate", "hello", {
|
||||
assertCurrent: guard, retryUnknown: true,
|
||||
})).rejects.toMatchObject({ code: "delivery_unknown" });
|
||||
await expect(f.adapter.publish(f.threadId, "duplicate", "hello", {
|
||||
assertCurrent: guard,
|
||||
})).rejects.toMatchObject({ code: "delivery_unknown" });
|
||||
expect(f.client.messages.sendText).toHaveBeenCalledTimes(2);
|
||||
expect(f.client.messages.sendText.mock.calls[0][2]).toEqual(
|
||||
f.client.messages.sendText.mock.calls[1][2],
|
||||
);
|
||||
});
|
||||
it("distinguishes explicit quota errors and rechecks authorization between parts", async () => {
|
||||
const f = photonFixture();
|
||||
f.client.messages.sendText.mockRejectedValueOnce(
|
||||
new IMessageError("secret upstream text", {
|
||||
code: "dailyLimitExceeded",
|
||||
grpcCode: 8,
|
||||
retryable: true,
|
||||
retryAfter: 12345,
|
||||
}),
|
||||
);
|
||||
await expect(
|
||||
f.adapter.publish(f.threadId, "quota", "hello", { assertCurrent: guard }),
|
||||
).rejects.toMatchObject({ code: "quota", retryAfterMs: 12345 });
|
||||
await f.adapter.publish(f.threadId, "quota", "hello", {
|
||||
assertCurrent: guard,
|
||||
});
|
||||
let calls = 0;
|
||||
await expect(
|
||||
f.adapter.publish(f.threadId, "parts", "x".repeat(9000), {
|
||||
assertCurrent: async () => {
|
||||
if (++calls === 2) throw new Error("revoked");
|
||||
},
|
||||
}),
|
||||
).rejects.toThrow("revoked");
|
||||
expect(f.client.messages.sendText).toHaveBeenCalledTimes(3);
|
||||
});
|
||||
});
|
||||
|
||||
describe("Photon durable event recovery", () => {
|
||||
it("records preceding events, deduplicates, and does not checkpoint an unadmitted event", async () => {
|
||||
const f = photonFixture();
|
||||
const events = [photonEvent(1), photonEvent(2)];
|
||||
let crash = true;
|
||||
const admitted = vi.fn(async (event) => {
|
||||
if (event.sequence === 2 && crash) throw new Error("crash");
|
||||
});
|
||||
const receiver = new PhotonReceiver({
|
||||
client: f.adapter.client,
|
||||
state: f.state,
|
||||
lineId: "line",
|
||||
intakeAfter: 0,
|
||||
assertOwned: guard,
|
||||
admit: admitted,
|
||||
failure: guard,
|
||||
catchUp: () =>
|
||||
stream([...events, { type: "catchup.complete", headSequence: 2 }]),
|
||||
});
|
||||
await expect(receiver.catchUp()).rejects.toThrow("crash");
|
||||
expect(await f.state.read("checkpoint")).toMatchObject({ sequence: 1 });
|
||||
crash = false;
|
||||
await receiver.catchUp();
|
||||
expect(await f.state.read("checkpoint")).toMatchObject({ sequence: 2 });
|
||||
expect(admitted.mock.calls.map(([event]) => event.sequence)).toEqual([
|
||||
1, 2, 2,
|
||||
]);
|
||||
});
|
||||
it("advances irrelevant frames and cutoff history, and stops at genuine gaps or lost leases", async () => {
|
||||
const f = photonFixture();
|
||||
const admit = vi.fn();
|
||||
const receiver = new PhotonReceiver({
|
||||
client: f.adapter.client,
|
||||
state: f.state,
|
||||
lineId: "line",
|
||||
intakeAfter: Date.now() + 1000,
|
||||
assertOwned: guard,
|
||||
admit,
|
||||
failure: guard,
|
||||
catchUp: () =>
|
||||
stream([
|
||||
photonEvent(10),
|
||||
{ type: "photon.ignored", sequence: 11 },
|
||||
{ type: "catchup.complete", headSequence: 11 },
|
||||
]),
|
||||
});
|
||||
await receiver.catchUp();
|
||||
expect(admit).not.toHaveBeenCalled();
|
||||
expect(await f.state.read("checkpoint")).toMatchObject({ sequence: 11 });
|
||||
const gap = new PhotonReceiver({
|
||||
client: f.adapter.client,
|
||||
state: f.state,
|
||||
lineId: "line",
|
||||
intakeAfter: 0,
|
||||
assertOwned: guard,
|
||||
admit,
|
||||
failure: guard,
|
||||
catchUp: () =>
|
||||
stream([
|
||||
photonEvent(13),
|
||||
{ type: "catchup.complete", headSequence: 13 },
|
||||
]),
|
||||
});
|
||||
await expect(gap.catchUp()).rejects.toMatchObject({ code: "history_gap" });
|
||||
const lost = new PhotonReceiver({
|
||||
client: f.adapter.client,
|
||||
state: f.state,
|
||||
lineId: "line",
|
||||
intakeAfter: 0,
|
||||
assertOwned: async () => {
|
||||
throw new Error("lease lost");
|
||||
},
|
||||
admit,
|
||||
failure: guard,
|
||||
});
|
||||
await expect(lost.catchUp()).rejects.toThrow("lease lost");
|
||||
expect(admit).not.toHaveBeenCalled();
|
||||
});
|
||||
it("recovers sparse shared-project sequences and commits only a complete ordered replay", async () => {
|
||||
const f = photonFixture();
|
||||
const admit = vi.fn(guard);
|
||||
await f.state.update("checkpoint", () => ({schema:1, lineId:"line", sequence:0}));
|
||||
let events: any[] = [photonEvent(1_008_648_034), {type:"catchup.complete",headSequence:1_008_648_034}];
|
||||
const receiver = new PhotonReceiver({client:f.adapter.client,state:f.state,lineId:"line",allocation:"shared",intakeAfter:0,assertOwned:guard,admit,failure:guard,catchUp:()=>stream(events)});
|
||||
await receiver.catchUp();
|
||||
expect(admit).toHaveBeenCalledTimes(1);
|
||||
expect(await f.state.read("checkpoint")).toMatchObject({sequence:1_008_648_034});
|
||||
events=[photonEvent(1_008_648_090),photonEvent(1_008_648_080),{type:"catchup.complete",headSequence:1_008_648_090}];
|
||||
await expect(receiver.catchUp()).rejects.toThrow(/out of order/);
|
||||
expect(await f.state.read("checkpoint")).toMatchObject({sequence:1_008_648_034});
|
||||
events=[photonEvent(1_008_648_080)];
|
||||
await expect(receiver.catchUp()).rejects.toMatchObject({code:"network"});
|
||||
expect(await f.state.read("checkpoint")).toMatchObject({sequence:1_008_648_034});
|
||||
events=[{type:"catchup.complete",headSequence:0}];
|
||||
await expect(receiver.catchUp()).rejects.toMatchObject({code:"history_gap"});
|
||||
});
|
||||
it("reads sequence-only and complete frames over authenticated synthetic gRPC", async () => {
|
||||
const f = photonFixture();
|
||||
const server = new Server();
|
||||
server.addService(
|
||||
{
|
||||
catchUp: {
|
||||
path: PHOTON_CATCHUP_PATH,
|
||||
requestStream: false,
|
||||
responseStream: true,
|
||||
requestSerialize: (b: Buffer) => b,
|
||||
requestDeserialize: (b: Buffer) => b,
|
||||
responseSerialize: (b: Buffer) => b,
|
||||
responseDeserialize: (b: Buffer) => b,
|
||||
},
|
||||
},
|
||||
{
|
||||
catchUp: (call: any) => {
|
||||
expect(call.metadata.get("authorization")).toEqual([
|
||||
"Bearer private-line-token",
|
||||
]);
|
||||
expect(call.request).toEqual(photonCatchUpRequest(2));
|
||||
call.write(Buffer.from([8, 3]));
|
||||
call.write(Buffer.from([162, 1, 2, 8, 3]));
|
||||
call.end();
|
||||
},
|
||||
},
|
||||
);
|
||||
const port = await new Promise<number>((resolve, reject) =>
|
||||
server.bindAsync(
|
||||
"127.0.0.1:0",
|
||||
ServerCredentials.createInsecure(),
|
||||
(error, port) => (error ? reject(error) : resolve(port)),
|
||||
),
|
||||
);
|
||||
const transport = new PhotonRecoveryTransport(
|
||||
f.authentication,
|
||||
new Client(`127.0.0.1:${port}`, credentials.createInsecure()),
|
||||
);
|
||||
try {
|
||||
const result = [];
|
||||
for await (const event of transport.catchUp(2)) result.push(event);
|
||||
expect(result).toEqual([
|
||||
{ type: "photon.ignored", sequence: 3 },
|
||||
{ type: "catchup.complete", headSequence: 3 },
|
||||
]);
|
||||
} finally {
|
||||
transport.close();
|
||||
server.forceShutdown();
|
||||
}
|
||||
});
|
||||
it("keeps state company scoped and rejects corrupt cursor envelopes", async () => {
|
||||
const f = photonFixture();
|
||||
await f.state.update("checkpoint", () => ({ sequence: 1 }));
|
||||
expect(
|
||||
await new PhotonState(
|
||||
{ ...f.state.scope, companyId: "other" },
|
||||
f.persistence,
|
||||
).read("checkpoint"),
|
||||
).toBeNull();
|
||||
expect(() => photonEnvelopeSequence(Buffer.from([8, 128]))).toThrow();
|
||||
expect(() =>
|
||||
decodePhotonRecoveryFrame(Buffer.from([8, 1, 8, 2])),
|
||||
).toThrow();
|
||||
});
|
||||
});
|
||||
|
||||
describe("Photon native prompts and media", () => {
|
||||
it("binds duplicate titles and option labels only by returned IDs across restart", async () => {
|
||||
const f = photonFixture();
|
||||
const binding = {
|
||||
version: 1 as const,
|
||||
reference: "referenceA",
|
||||
interactionId: "one",
|
||||
publicationId: "publication1",
|
||||
sessionGeneration: 1,
|
||||
expiresAt: new Date(Date.now() + 60000).toISOString(),
|
||||
};
|
||||
const first = await publishPhotonPrompt({
|
||||
adapter: f.adapter,
|
||||
threadId: f.threadId,
|
||||
binding,
|
||||
interaction: question("one"),
|
||||
questionIndex: 0,
|
||||
assertCurrent: guard,
|
||||
});
|
||||
const second = await publishPhotonPrompt({
|
||||
adapter: f.adapter,
|
||||
threadId: f.threadId,
|
||||
binding: {
|
||||
...binding,
|
||||
reference: "referenceB",
|
||||
interactionId: "two",
|
||||
publicationId: "publication2",
|
||||
},
|
||||
interaction: question("two"),
|
||||
questionIndex: 0,
|
||||
assertCurrent: guard,
|
||||
});
|
||||
expect(first.pollMessageGuid).not.toBe(second.pollMessageGuid);
|
||||
expect(Object.values(first.options)).toEqual(["a", "b"]);
|
||||
expect(
|
||||
await f.state.read(`poll-message:${first.pollMessageGuid}`),
|
||||
).toMatchObject({ reference: "referenceA" });
|
||||
await publishPhotonPrompt({
|
||||
adapter: f.adapter,
|
||||
threadId: f.threadId,
|
||||
binding,
|
||||
interaction: question("one"),
|
||||
questionIndex: 0,
|
||||
assertCurrent: guard,
|
||||
});
|
||||
expect(f.client.polls.create).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
it("reuses poll clientMessageId after an accepted create loses its local receipt", async () => {
|
||||
const f = photonFixture();
|
||||
const create = f.client.polls.create.getMockImplementation()!;
|
||||
f.client.polls.create.mockImplementationOnce(async (...args) => {
|
||||
await create(...args);
|
||||
throw new Error("crash");
|
||||
});
|
||||
const input = {
|
||||
adapter: f.adapter,
|
||||
threadId: f.threadId,
|
||||
binding: {
|
||||
version: 1 as const,
|
||||
reference: "referenceA",
|
||||
interactionId: "one",
|
||||
publicationId: "p",
|
||||
sessionGeneration: 1,
|
||||
expiresAt: new Date().toISOString(),
|
||||
},
|
||||
interaction: question(),
|
||||
questionIndex: 0,
|
||||
assertCurrent: guard,
|
||||
};
|
||||
await expect(publishPhotonPrompt(input)).rejects.toMatchObject({
|
||||
code: "delivery_unknown",
|
||||
});
|
||||
await expect(publishPhotonPrompt(input)).rejects.toMatchObject({
|
||||
code: "delivery_unknown",
|
||||
});
|
||||
await publishPhotonPrompt({ ...input, retryUnknown: true });
|
||||
expect(f.polls.size).toBe(1);
|
||||
});
|
||||
it("validates numbered selection, custom answers, optional skips and explicit references", () => {
|
||||
expect(parsePhotonQuestionAnswer(question(), 0, "2")).toEqual({
|
||||
questionId: "q1",
|
||||
optionIds: ["b"],
|
||||
});
|
||||
expect(() => parsePhotonQuestionAnswer(question(), 0, "yes")).toThrow();
|
||||
expect(() => parsePhotonQuestionAnswer(question(), 0, "1,2")).toThrow();
|
||||
expect(photonResponseCommand("yes")).toBeNull();
|
||||
expect(
|
||||
photonResponseCommand("/answer referenceA.2 custom answer"),
|
||||
).toMatchObject({
|
||||
reference: "referenceA",
|
||||
questionIndex: 1,
|
||||
value: "custom answer",
|
||||
});
|
||||
const multi = question();
|
||||
multi.payload.questions[0].selectionMode = "multi";
|
||||
expect(parsePhotonQuestionAnswer(multi, 0, "1,2").optionIds).toEqual([
|
||||
"a",
|
||||
"b",
|
||||
]);
|
||||
multi.payload.questions[0].required = false;
|
||||
expect(parsePhotonQuestionAnswer(multi, 0, "skip").optionIds).toEqual([]);
|
||||
});
|
||||
it("rejects foreign attachment provenance before downloading and bounds decoded images", async () => {
|
||||
const f = photonFixture();
|
||||
await expect(
|
||||
downloadPhotonAttachment(f.adapter.client, "line", {
|
||||
kind: "photon_attachment",
|
||||
lineId: "other",
|
||||
chatGuid: f.chat.guid,
|
||||
messageGuid: "m",
|
||||
attachmentGuid: "a",
|
||||
}),
|
||||
).rejects.toThrow("another line");
|
||||
await expect(
|
||||
downloadPhotonAttachment(f.adapter.client, "line", {
|
||||
kind: "photon_attachment",
|
||||
lineId: "line",
|
||||
chatGuid: "other",
|
||||
messageGuid: "m",
|
||||
attachmentGuid: "a",
|
||||
}),
|
||||
).rejects.toThrow("does not belong");
|
||||
const wrongMessage = await f.client.messages.get("different-message");
|
||||
wrongMessage.content.attachments = [{ guid: "a" }] as typeof wrongMessage.content.attachments;
|
||||
f.client.messages.get.mockResolvedValueOnce(wrongMessage);
|
||||
await expect(downloadPhotonAttachment(f.adapter.client, "line", {
|
||||
kind: "photon_attachment",
|
||||
lineId: "line",
|
||||
chatGuid: f.chat.guid,
|
||||
messageGuid: "m",
|
||||
attachmentGuid: "a",
|
||||
})).rejects.toThrow("does not belong");
|
||||
expect(f.client.attachments.downloadStream).not.toHaveBeenCalled();
|
||||
const png = await sharp({
|
||||
create: { width: 2, height: 2, channels: 3, background: "white" },
|
||||
})
|
||||
.png()
|
||||
.toBuffer();
|
||||
await validatePhotonImage(png, "image/png");
|
||||
await expect(validatePhotonImage(png, "image/jpeg")).rejects.toThrow(
|
||||
"declared",
|
||||
);
|
||||
expect(() => validateHeifDimensions(png)).toThrow();
|
||||
});
|
||||
it("preserves numerical free-text answers for canonical numeric validation", () => {
|
||||
const interaction = question();
|
||||
interaction.payload.questions[0].allowOther = true;
|
||||
interaction.payload.questions[0].options = [
|
||||
{ id: "number", label: "Enter a number", freeText: true },
|
||||
];
|
||||
expect(parsePhotonQuestionAnswer(interaction, 0, "42")).toEqual({
|
||||
questionId: "q1",
|
||||
optionIds: ["number"],
|
||||
otherText: "42",
|
||||
});
|
||||
});
|
||||
it("downloads a shared project alias with a native header UUID only after source ownership is verified", async () => {
|
||||
const f = photonFixture();
|
||||
const body = Buffer.from("synthetic shared attachment");
|
||||
const alias = "spc-att-11111111-1111-4111-8111-111111111111";
|
||||
const message = photonEvent(1).message;
|
||||
const info = {guid: alias, totalBytes: body.length, fileName: "test.txt", mimeType: "text/plain", isHidden: false, isSticker: false};
|
||||
(message.content.attachments as any[]).push(info);
|
||||
f.client.messages.get.mockResolvedValue(message);
|
||||
const native = {...info, guid: "22222222-2222-4222-8222-222222222222"};
|
||||
f.client.attachments.downloadStream.mockImplementation(() => stream([
|
||||
{type: "header", info: native}, {type: "primaryChunk", data: body},
|
||||
]));
|
||||
const locator = {kind: "photon_attachment" as const, lineId: "line", chatGuid: f.chat.guid, messageGuid: message.guid, attachmentGuid: alias};
|
||||
await expect(downloadPhotonAttachment(f.adapter.client, "line", locator)).rejects.toThrow("metadata changed");
|
||||
await expect(downloadPhotonAttachment(f.adapter.client, "line", locator, "shared")).resolves.toEqual(body);
|
||||
expect(f.client.attachments.downloadStream).toHaveBeenLastCalledWith(alias);
|
||||
native.mimeType = "application/octet-stream";
|
||||
await expect(downloadPhotonAttachment(f.adapter.client, "line", locator, "shared")).rejects.toThrow("metadata changed");
|
||||
f.client.attachments.downloadStream.mockClear();
|
||||
await expect(downloadPhotonAttachment(f.adapter.client, "line", {...locator, chatGuid: "wrong-chat"}, "shared")).rejects.toThrow("does not belong");
|
||||
expect(f.client.attachments.downloadStream).not.toHaveBeenCalled();
|
||||
});
|
||||
it("retains source-bound Live Photo companion bytes and waits for incomplete companions", async () => {
|
||||
const f = photonFixture();
|
||||
const photo = Buffer.from("primary"),
|
||||
video = Buffer.from("companion");
|
||||
const message = photonEvent(1).message;
|
||||
(message.content.attachments as any[]).push({
|
||||
guid: "live-photo",
|
||||
totalBytes: photo.length,
|
||||
isHidden: false,
|
||||
isSticker: false,
|
||||
});
|
||||
f.client.messages.get.mockResolvedValue(message);
|
||||
const locator = {
|
||||
kind: "photon_attachment" as const,
|
||||
lineId: "line",
|
||||
chatGuid: f.chat.guid,
|
||||
messageGuid: message.guid,
|
||||
attachmentGuid: "live-photo",
|
||||
};
|
||||
const header = {
|
||||
type: "header",
|
||||
info: { guid: "live-photo", totalBytes: photo.length },
|
||||
companionInfo: {
|
||||
kind: "live-photo-video",
|
||||
mimeType: "video/quicktime",
|
||||
fileName: "photo.mov",
|
||||
totalBytes: video.length,
|
||||
},
|
||||
};
|
||||
f.client.attachments.downloadStream.mockImplementationOnce(() =>
|
||||
stream([header, { type: "primaryChunk", data: photo }]),
|
||||
);
|
||||
await expect(
|
||||
downloadPhotonAttachment(f.adapter.client, "line", locator),
|
||||
).rejects.toMatchObject({ code: "attachment_not_ready" });
|
||||
f.client.attachments.downloadStream.mockImplementationOnce(() =>
|
||||
stream([
|
||||
header,
|
||||
{ type: "primaryChunk", data: photo },
|
||||
{ type: "companionChunk", data: video },
|
||||
]),
|
||||
);
|
||||
const result = await downloadPhotonAttachment(
|
||||
f.adapter.client,
|
||||
"line",
|
||||
locator,
|
||||
);
|
||||
expect(result).toEqual(photo);
|
||||
expect(takePhotonCompanion(result)).toEqual({
|
||||
unavailable: false,
|
||||
fileName: "photo.mov",
|
||||
mimeType: "video/quicktime",
|
||||
data: video,
|
||||
});
|
||||
expect(takePhotonCompanion(result)).toBeUndefined();
|
||||
});
|
||||
it("converts a real synthetic HEIC with the installed native binary and bounds its dimensions", async () => {
|
||||
const heic = await readFile(
|
||||
new URL("./fixtures/synthetic.heic", import.meta.url),
|
||||
);
|
||||
const jpeg = await photonHeifPreview(heic);
|
||||
expect(await sharp(jpeg).metadata()).toMatchObject({
|
||||
format: "jpeg",
|
||||
width: 16,
|
||||
height: 16,
|
||||
});
|
||||
const oversized = Buffer.from(heic);
|
||||
const at = oversized.indexOf("ispe");
|
||||
expect(at).toBeGreaterThan(0);
|
||||
oversized.writeUInt32BE(100_000, at + 8);
|
||||
expect(() => validateHeifDimensions(oversized)).toThrow("pixel limit");
|
||||
});
|
||||
});
|
||||
|
|
@ -4974,6 +4974,7 @@ describeEmbeddedPostgres("tool access service", () => {
|
|||
expect(res.body.apps.map((app: { slug: string }) => app.slug)).toEqual(
|
||||
expect.arrayContaining([
|
||||
"agentmail",
|
||||
"imessage-photon",
|
||||
"jira",
|
||||
"airtable",
|
||||
"asana",
|
||||
|
|
@ -4994,7 +4995,7 @@ describeEmbeddedPostgres("tool access service", () => {
|
|||
"github",
|
||||
]),
|
||||
);
|
||||
expect(res.body.apps).toHaveLength(41);
|
||||
expect(res.body.apps).toHaveLength(42);
|
||||
expect(
|
||||
res.body.apps.find((app: { slug: string }) => app.slug === "gmail")
|
||||
.ownershipAvailability,
|
||||
|
|
|
|||
|
|
@ -20,6 +20,10 @@ export const DEFAULT_ALLOWED_TYPES: readonly string[] = [
|
|||
"image/jpg",
|
||||
"image/webp",
|
||||
"image/gif",
|
||||
"image/heic",
|
||||
"image/heif",
|
||||
"image/heic-sequence",
|
||||
"image/heif-sequence",
|
||||
"audio/mpeg",
|
||||
"audio/mp4",
|
||||
"audio/ogg",
|
||||
|
|
|
|||
|
|
@ -7,6 +7,7 @@ import type { Db } from "@paperclipai/db";
|
|||
import {
|
||||
CHAT_PROVIDERS,
|
||||
configureChatEndpointSchema,
|
||||
inspectPhotonProjectSchema,
|
||||
confirmChatIdentityLinkSchema,
|
||||
createChatEndpointSchema,
|
||||
createChatIdentityLinkIntentSchema,
|
||||
|
|
@ -153,6 +154,12 @@ export function chatChannelRoutes(db: Db, options: ChatChannelRouteOptions) {
|
|||
},
|
||||
);
|
||||
|
||||
router.post("/chat-endpoints/:endpointId/photon/inspect", validate(inspectPhotonProjectSchema), async (req, res) => {
|
||||
if (!(await assertEndpointManagementAccess(req, res))) return;
|
||||
res.set("Cache-Control", "no-store");
|
||||
res.json(await service.inspectPhoton(endpointId(req), req.body));
|
||||
});
|
||||
|
||||
router.post(
|
||||
"/chat-endpoints/:endpointId/setup",
|
||||
validate(configureChatEndpointSchema),
|
||||
|
|
|
|||
|
|
@ -262,6 +262,9 @@ import {
|
|||
confirmChatIdentityLinkSchema,
|
||||
createChatEndpointSchema,
|
||||
createChatIdentityLinkIntentSchema,
|
||||
inspectPhotonProjectSchema,
|
||||
photonProjectIdSchema,
|
||||
photonLineIdSchema,
|
||||
publishChatPublicationSchema,
|
||||
resolveChatActionSchema,
|
||||
resolveChatPublicationSchema,
|
||||
|
|
@ -787,6 +790,7 @@ const chatEndpointResponseSchema = z
|
|||
providerAccountId: z.string().nullable(),
|
||||
providerAccountLabel: z.string().nullable(),
|
||||
botExternalId: z.string().nullable(),
|
||||
photonAllocation: z.enum(["dedicated", "shared"]).optional(),
|
||||
botUsername: z.string().nullable(),
|
||||
botLabel: z.string().nullable(),
|
||||
botAvatarUrl: z.string().nullable(),
|
||||
|
|
@ -820,6 +824,7 @@ const chatEndpointResourceResponseSchema = z
|
|||
availability: chatResourceAvailabilitySchema,
|
||||
enabled: z.boolean(),
|
||||
metadata: z.record(z.string(), z.unknown()),
|
||||
participants: z.array(z.string()).optional(),
|
||||
createdAt: z.string().datetime(),
|
||||
updatedAt: z.string().datetime(),
|
||||
})
|
||||
|
|
@ -1448,6 +1453,7 @@ const BOARD_ONLY_OPERATIONS = new Set([
|
|||
"POST /api/chat-endpoints/{endpointId}/setup",
|
||||
"POST /api/chat-endpoints/{endpointId}/setup-secret",
|
||||
"POST /api/chat-endpoints/{endpointId}/test",
|
||||
"POST /api/chat-endpoints/{endpointId}/photon/inspect",
|
||||
"GET /api/chat-endpoints/{endpointId}/resources",
|
||||
"PUT /api/chat-endpoints/{endpointId}/resources",
|
||||
"GET /api/chat-endpoints/{endpointId}/principals",
|
||||
|
|
@ -2124,7 +2130,7 @@ registry.registerPath({
|
|||
tags: ["chat-channels"],
|
||||
summary: "Configure or change chat endpoint lifecycle state",
|
||||
description:
|
||||
"Runs a setup or lifecycle action. `configure` and `reconnect` accept provider credentials (Slack: `botToken`, `signingSecret`; GitHub: `appId`, `privateKey` after Paperclip generates the webhook secret; Discord: `applicationId`, `guildId`, `botToken`; Microsoft Teams: `clientId`, `tenantId`, `clientSecret`; Telegram: `botToken`). Credentials are stored as Paperclip secret references and are never returned. Other actions do not require credentials.",
|
||||
"Runs a setup or lifecycle action. `configure` and `reconnect` accept provider credentials (Slack: `botToken`, `signingSecret`; GitHub: `appId`, `privateKey` after Paperclip generates the webhook secret; Discord: `applicationId`, `guildId`, `botToken`; Microsoft Teams: `clientId`, `tenantId`, `clientSecret`; Telegram: `botToken`; iMessage Photon: `projectSecret`, with nonsecret `photon.projectId` and `photon.lineId` configuration). Credentials are stored as Paperclip secret references and are never returned. Other actions do not require credentials.",
|
||||
request: {
|
||||
params: z.object({ endpointId: z.string().uuid() }),
|
||||
body: jsonBody(configureChatEndpointSchema),
|
||||
|
|
@ -2137,6 +2143,9 @@ registry.registerPath({
|
|||
404: r.notFound,
|
||||
409: r.conflict,
|
||||
422: r.unprocessable,
|
||||
429: { description: "Provider request limit reached; retry later" },
|
||||
502: { description: "Provider returned an invalid response; inspect provider health" },
|
||||
503: { description: "Provider temporarily unavailable; retry later" },
|
||||
},
|
||||
});
|
||||
|
||||
|
|
@ -2158,13 +2167,48 @@ registry.registerPath({
|
|||
},
|
||||
});
|
||||
|
||||
registry.registerPath({
|
||||
method: "post",
|
||||
path: "/api/chat-endpoints/{endpointId}/photon/inspect",
|
||||
tags: ["chat-channels"],
|
||||
summary: "Inspect Photon shared project or dedicated numbers for channel setup",
|
||||
description:
|
||||
"Requires a board user with connection-management access. The project secret is write-only input. Returns the project's actual allocation and eligibility for shared DMs or dedicated lines, never project secrets or minted line tokens. Responses are not cached. Inspection alone does not activate the channel.",
|
||||
request: {
|
||||
params: z.object({ endpointId: z.string().uuid() }),
|
||||
body: jsonBody(inspectPhotonProjectSchema),
|
||||
},
|
||||
responses: {
|
||||
200: r.ok(z.object({
|
||||
projectId: photonProjectIdSchema,
|
||||
projectName: z.string(),
|
||||
allocation: z.enum(["dedicated", "shared"]),
|
||||
eligible: z.boolean(),
|
||||
lines: z.array(z.object({
|
||||
lineId: photonLineIdSchema,
|
||||
phoneNumber: z.string(),
|
||||
eligible: z.boolean(),
|
||||
unavailableReason: z.string().optional(),
|
||||
}).strict()),
|
||||
}).strict()),
|
||||
400: r.badRequest,
|
||||
401: r.unauthorized,
|
||||
403: r.forbidden,
|
||||
404: r.notFound,
|
||||
422: r.unprocessable,
|
||||
429: { description: "Photon request limit reached; retry later" },
|
||||
502: { description: "Photon returned an invalid response; inspect provider health" },
|
||||
503: { description: "Photon temporarily unavailable; retry later" },
|
||||
},
|
||||
});
|
||||
|
||||
registry.registerPath({
|
||||
method: "post",
|
||||
path: "/api/chat-endpoints/{endpointId}/test",
|
||||
tags: ["chat-channels"],
|
||||
summary: "Complete a chat endpoint setup test",
|
||||
description:
|
||||
"Activates a verifying endpoint only after Paperclip has received a real provider event since the server-issued setup test boundary.",
|
||||
"Activates a verifying endpoint only after Paperclip has received a real provider event since the server-issued setup test boundary. iMessage Photon additionally requires a fresh linked sender's task and a successful outbound agent publication.",
|
||||
request: { params: z.object({ endpointId: z.string().uuid() }) },
|
||||
responses: {
|
||||
200: r.ok(chatEndpointResponseSchema),
|
||||
|
|
@ -2181,7 +2225,7 @@ registry.registerPath({
|
|||
tags: ["chat-channels"],
|
||||
summary: "List destinations discovered for a chat endpoint",
|
||||
description:
|
||||
"Lists provider destinations such as Slack and Discord channels, Teams channels, GitHub repositories, and Telegram chats. Direct-message resources are intentionally omitted.",
|
||||
"Lists provider destinations such as Slack and Discord channels, Teams channels, GitHub repositories, Telegram chats, and iMessage Photon groups. Direct-message resources are intentionally omitted.",
|
||||
request: { params: z.object({ endpointId: z.string().uuid() }) },
|
||||
responses: {
|
||||
200: r.ok(z.array(chatEndpointResourceResponseSchema)),
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load Diff
|
|
@ -1,3 +1,4 @@
|
|||
import { nativePhotonInteraction } from "./photon/interactions.js";
|
||||
import { randomBytes } from "node:crypto";
|
||||
import { and, eq, inArray, sql } from "drizzle-orm";
|
||||
import type { Db } from "@paperclipai/db";
|
||||
|
|
@ -313,6 +314,7 @@ export async function enqueueIssueInteractionChatPublications(
|
|||
)
|
||||
: null;
|
||||
const supportsCallbacks =
|
||||
endpoint.provider !== "imessage-photon" &&
|
||||
formDraft === null &&
|
||||
question !== null &&
|
||||
endpoint.capabilities.actions === true;
|
||||
|
|
@ -430,7 +432,13 @@ export async function enqueueIssueInteractionChatPublications(
|
|||
.onConflictDoNothing()
|
||||
.returning();
|
||||
const publication = rows[0];
|
||||
if (publication && formDraft) {
|
||||
if (publication && endpoint.provider === "imessage-photon" && nativePhotonInteraction(interaction)) {
|
||||
const reference = randomBytes(9).toString("base64url");
|
||||
await db.insert(chatActions).values({ companyId: interaction.companyId, endpointId: endpoint.id, conversationId: conversation.id,
|
||||
kind: "photon_interaction", providerActionId: `photon:${reference}`,
|
||||
payload: { version: 1, reference, interactionId: interaction.id, publicationId: publication.id, sessionGeneration: conversation.sessionGeneration,
|
||||
expiresAt: new Date(publication.createdAt.getTime() + CHAT_QUESTION_ACTION_TOKEN_TTL_MS).toISOString() }, status: "issued" });
|
||||
} else if (publication && formDraft) {
|
||||
await db.insert(chatActions).values(
|
||||
chatQuestionFormActionRecords(formDraft, {
|
||||
companyId: interaction.companyId,
|
||||
|
|
@ -540,10 +548,12 @@ export async function enqueueTerminalIssueInteractionChatPublications(
|
|||
and(
|
||||
eq(chatActions.companyId, interaction.companyId),
|
||||
inArray(chatActions.kind, [
|
||||
"photon_interaction",
|
||||
"question_answer",
|
||||
"question_form_open",
|
||||
"question_form_submit",
|
||||
"confirmation_response",
|
||||
"photon_interaction",
|
||||
]),
|
||||
eq(chatActions.status, "issued"),
|
||||
eq(
|
||||
|
|
@ -763,6 +773,7 @@ export async function cancelPendingIssueInteractionChatPublications(
|
|||
"question_form_open",
|
||||
"question_form_submit",
|
||||
"confirmation_response",
|
||||
"photon_interaction",
|
||||
]),
|
||||
eq(chatActions.status, "issued"),
|
||||
inArray(sql<string>`${chatActions.payload}->>'interactionId'`, [
|
||||
|
|
|
|||
|
|
@ -493,6 +493,7 @@ export function parseChatProviderLifecycle(
|
|||
input: ParseChatProviderLifecycleInput,
|
||||
): ChatProviderLifecycleEffect[] {
|
||||
switch (input.provider) {
|
||||
case "imessage-photon": return []; // Authenticated gRPC events own lifecycle.
|
||||
case "agentmail": return [];
|
||||
case "slack":
|
||||
return parseSlackLifecycle(input);
|
||||
|
|
|
|||
|
|
@ -192,6 +192,23 @@ export function classifyChatPublicationError(
|
|||
.filter((candidate): candidate is string => typeof candidate === "string")
|
||||
.map((candidate) => candidate.toLowerCase());
|
||||
const reason = text(error);
|
||||
if (names.includes("PhotonError")) {
|
||||
if (codes.includes("delivery_unknown")) return { kind: "delivery_unknown", reason };
|
||||
if (codes.includes("credentials") || codes.includes("line_unavailable") || codes.includes("history_gap")) {
|
||||
return { kind: "endpoint_attention", reason };
|
||||
}
|
||||
if (codes.includes("quota") || codes.includes("network") || codes.includes("attachment_not_ready")) {
|
||||
return {
|
||||
kind: "retry",
|
||||
retryAfterMs: values
|
||||
.map((value) => finitePositive(value.retryAfterMs))
|
||||
.find((value) => value !== null) ?? Math.min(60_000, 1000 * 2 ** Math.min(attempt, 6)),
|
||||
providerRateLimit: codes.includes("quota"),
|
||||
reason,
|
||||
};
|
||||
}
|
||||
return { kind: "failed", reason };
|
||||
}
|
||||
const responseHeaders = values
|
||||
.map((value) => value.response?.headers)
|
||||
.find((headers) => headers !== undefined);
|
||||
|
|
|
|||
|
|
@ -1,3 +1,9 @@
|
|||
import { PhotonChatAdapter, parsePhotonThreadId } from "./photon/adapter.js";
|
||||
import { PhotonLineAuthentication } from "./photon/cloud.js";
|
||||
import { PhotonState } from "./photon/state.js";
|
||||
import { PhotonReceiver } from "./photon/receiver.js";
|
||||
import { photonAttachmentLocator, photonAttachmentLocatorSchema, downloadPhotonAttachment, type PhotonAttachmentLocator } from "./photon/attachments.js";
|
||||
import type { LiveEvent as PhotonEvent } from "@photon-ai/advanced-imessage";
|
||||
import {
|
||||
createGitHubAdapter,
|
||||
type GitHubAdapter,
|
||||
|
|
@ -130,8 +136,8 @@ const DISCORD_GATEWAY_HEALTHY_SESSION_MS = 60_000;
|
|||
|
||||
/** Public Paperclip provider ids. The Teams SDK name remains an internal detail. */
|
||||
export type ChatSdkProvider =
|
||||
"slack" | "github" | "discord" | "microsoft-teams" | "telegram";
|
||||
type ChatSdkAdapterKey = "slack" | "github" | "discord" | "teams" | "telegram";
|
||||
"slack" | "github" | "discord" | "microsoft-teams" | "telegram" | "imessage-photon";
|
||||
type ChatSdkAdapterKey = "slack" | "github" | "discord" | "teams" | "telegram" | "imessage-photon";
|
||||
|
||||
interface ProviderConfigBase {
|
||||
/** Agent-derived native bot display/mention name. */
|
||||
|
|
@ -200,7 +206,13 @@ export interface ResolvedTelegramChatConfig extends ProviderConfigBase {
|
|||
};
|
||||
}
|
||||
|
||||
export interface ResolvedPhotonChatConfig extends ProviderConfigBase {
|
||||
provider: "imessage-photon";
|
||||
intakeAfter: number;
|
||||
credentials: { allocation?: "dedicated" | "shared"; projectId: string; projectSecret: string; lineId: string; phoneNumber: string };
|
||||
}
|
||||
export type ResolvedChatSdkProviderConfig =
|
||||
| ResolvedPhotonChatConfig
|
||||
| ResolvedSlackChatConfig
|
||||
| ResolvedGitHubChatConfig
|
||||
| ResolvedDiscordChatConfig
|
||||
|
|
@ -222,6 +234,7 @@ interface DurableAttachmentMetadata {
|
|||
}
|
||||
|
||||
type ChatSdkAttachmentLocator =
|
||||
| PhotonAttachmentLocator
|
||||
| GitHubPublicAttachmentLocator
|
||||
| TeamsInlineImageLocator
|
||||
| TelegramMediaLocator
|
||||
|
|
@ -453,6 +466,10 @@ export interface DiscordGatewayCallbackEvent extends ChatSdkCallbackEvent<Discor
|
|||
* Chat SDK event escape hatches; callers must never publish them directly.
|
||||
*/
|
||||
export interface ChatSdkRuntimeCallbacks {
|
||||
onPhotonEvent?(event: PhotonEvent): Promise<void>;
|
||||
onPhotonAssertOwned?(activeThreadId?: string): Promise<void>;
|
||||
onPhotonCheckpoint?(sequence: number): Promise<void>;
|
||||
onPhotonFailure?(error: unknown): Promise<void>;
|
||||
onMessage(event: ChatSdkMessageCallbackEvent): Promise<void> | void;
|
||||
onTelegramGenerationStopped?(
|
||||
event: ChatSdkCallbackEvent<TelegramGenerationStoppedProof>,
|
||||
|
|
@ -1339,6 +1356,7 @@ function createProviderAdapter(
|
|||
): Adapter {
|
||||
const resolvedLogger = adapterLogger(logger);
|
||||
switch (config.provider) {
|
||||
case "imessage-photon": throw new Error("Photon adapter requires scoped persistence");
|
||||
case "slack": {
|
||||
const adapterConfig: SlackAdapterConfig = {
|
||||
...config.credentials,
|
||||
|
|
@ -2023,10 +2041,13 @@ export class ChatSdkEndpointRuntime {
|
|||
private discordGatewayFatal = false;
|
||||
private initialization: Promise<void> | null = null;
|
||||
private retired = false;
|
||||
private photonReceiver?: PhotonReceiver;
|
||||
private readonly runtimeOptions: CreateChatSdkEndpointRuntimeOptions;
|
||||
private shutdownTask: Promise<void> | null = null;
|
||||
private shutdownCompleted = false;
|
||||
|
||||
constructor(options: CreateChatSdkEndpointRuntimeOptions) {
|
||||
this.runtimeOptions = options;
|
||||
this.companyId = options.companyId;
|
||||
this.endpointId = options.endpointId;
|
||||
this.provider = options.providerConfig.provider;
|
||||
|
|
@ -2076,7 +2097,11 @@ export class ChatSdkEndpointRuntime {
|
|||
1,
|
||||
Math.min(options.webhookIngressTimeoutMs ?? 2_500, 10_000),
|
||||
);
|
||||
this.adapter = createProviderAdapter(
|
||||
this.adapter = options.providerConfig.provider === "imessage-photon"
|
||||
? new PhotonChatAdapter(options.providerConfig.userName,
|
||||
new PhotonLineAuthentication(options.providerConfig.credentials, options.providerConfig.credentials.projectSecret),
|
||||
new PhotonState({ companyId: options.companyId, endpointId: options.endpointId }, options.persistence))
|
||||
: createProviderAdapter(
|
||||
options.providerConfig,
|
||||
options.logger,
|
||||
options.callbacks,
|
||||
|
|
@ -2256,6 +2281,18 @@ export class ChatSdkEndpointRuntime {
|
|||
async initialize(): Promise<void> {
|
||||
await this.initializeChat();
|
||||
this.assertNotRetired();
|
||||
if (this.adapter instanceof PhotonChatAdapter && this.discordGatewayEnabled && !this.photonReceiver) {
|
||||
const options = this.runtimeOptions;
|
||||
const config = options.providerConfig as ResolvedPhotonChatConfig;
|
||||
if (!options.callbacks.onPhotonCheckpoint || !options.callbacks.onPhotonAssertOwned || !options.callbacks.onPhotonEvent || !options.callbacks.onPhotonFailure) throw new Error("Photon receiver requires durable admission callbacks");
|
||||
this.adapter.typingGuard = async (activeThreadId) => { this.assertNotRetired(); await options.callbacks.onPhotonAssertOwned!(activeThreadId); };
|
||||
this.photonReceiver = new PhotonReceiver({ client: this.adapter.client, state: this.adapter.state,
|
||||
lineId: config.credentials.lineId, intakeAfter: config.intakeAfter, allocation: config.credentials.allocation,
|
||||
catchUp: (sequence) => (this.adapter as PhotonChatAdapter).recoveryStream(sequence),
|
||||
assertOwned: async () => { this.assertNotRetired(); await (this.adapter as PhotonChatAdapter).authentication.token(); await options.callbacks.onPhotonAssertOwned!(); },
|
||||
commitCheckpoint: options.callbacks.onPhotonCheckpoint, admit: options.callbacks.onPhotonEvent, failure: options.callbacks.onPhotonFailure });
|
||||
this.photonReceiver.start();
|
||||
}
|
||||
if (this.provider === "discord" && this.discordGatewayEnabled) {
|
||||
this.startDiscordGateway();
|
||||
}
|
||||
|
|
@ -2706,6 +2743,12 @@ export class ChatSdkEndpointRuntime {
|
|||
const metadata = durableAttachmentMetadata(attachment);
|
||||
return locator && metadata ? { version: 1, provider: "telegram", attachment: metadata, locator } : null;
|
||||
}
|
||||
if (this.adapter instanceof PhotonChatAdapter && source?.message) {
|
||||
const thread = this.adapter.decodeThreadId(source.threadId);
|
||||
const locator = photonAttachmentLocator(attachment, thread.lineId, thread.chatGuid, source.message);
|
||||
const metadata = durableAttachmentMetadata(attachment);
|
||||
return locator && metadata ? { version: 1, provider: "imessage-photon", attachment: metadata, locator } : null;
|
||||
}
|
||||
const retained = this.teamsInlineImageDescriptors.get(attachment);
|
||||
if (retained) {
|
||||
if (!source) return retained;
|
||||
|
|
@ -2875,6 +2918,14 @@ export class ChatSdkEndpointRuntime {
|
|||
descriptor: unknown,
|
||||
source?: ChatSdkAttachmentSource,
|
||||
): Attachment | null {
|
||||
if (this.adapter instanceof PhotonChatAdapter && isRecord(descriptor) && descriptor.version === 1 && descriptor.provider === this.provider && source) {
|
||||
const parsed = photonAttachmentLocatorSchema.safeParse(descriptor.locator);
|
||||
const thread = this.adapter.decodeThreadId(source.threadId);
|
||||
const metadata = isRecord(descriptor.attachment) ? durableAttachmentMetadata(descriptor.attachment as unknown as Attachment) : null;
|
||||
if (!parsed.success || !metadata || parsed.data.lineId !== thread.lineId || parsed.data.chatGuid !== thread.chatGuid || parsed.data.messageGuid !== source.messageId) return null;
|
||||
const adapter = this.adapter;
|
||||
return { ...metadata, fetchData: () => downloadPhotonAttachment(adapter.client, thread.lineId, parsed.data, adapter.authentication.identity.allocation) };
|
||||
}
|
||||
if (
|
||||
this.provider === "microsoft-teams" &&
|
||||
isRecord(descriptor) &&
|
||||
|
|
@ -3048,6 +3099,7 @@ export class ChatSdkEndpointRuntime {
|
|||
connectorOrigin: validated.locator.connectorOrigin,
|
||||
};
|
||||
break;
|
||||
case "photon_attachment":
|
||||
case "teams_inline_image":
|
||||
return null; // Only the exact source-bound branch above may authorize it.
|
||||
case "telegram_media":
|
||||
|
|
@ -3086,6 +3138,7 @@ export class ChatSdkEndpointRuntime {
|
|||
await this.initialization?.catch(() => undefined);
|
||||
await this.discordGatewayTask?.catch(() => undefined);
|
||||
this.discordGatewayAbort = null;
|
||||
await this.photonReceiver?.close();
|
||||
await this.chat.shutdown();
|
||||
this.shutdownCompleted = true;
|
||||
})();
|
||||
|
|
|
|||
|
|
@ -422,6 +422,7 @@ export async function authorizeChatConversationForBoundRun(
|
|||
"discord",
|
||||
"microsoft-teams",
|
||||
"telegram",
|
||||
"imessage-photon",
|
||||
].find(
|
||||
(candidate) =>
|
||||
source === `chat:${candidate}` || source === `chat:${candidate}:recovery`,
|
||||
|
|
@ -552,6 +553,7 @@ function externalChatWaitCandidate(
|
|||
"discord",
|
||||
"microsoft-teams",
|
||||
"telegram",
|
||||
"imessage-photon",
|
||||
].find(
|
||||
(candidate) =>
|
||||
source === `chat:${candidate}` || source === `chat:${candidate}:recovery`,
|
||||
|
|
|
|||
|
|
@ -32,6 +32,7 @@ const EXTERNAL_CHAT_PROVIDERS = new Set([
|
|||
"discord",
|
||||
"microsoft-teams",
|
||||
"telegram",
|
||||
"imessage-photon",
|
||||
]);
|
||||
const ATTACHMENT_OMISSION_REASONS = new Set([
|
||||
"attachment_limit",
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
import { and, eq, inArray, notExists, sql } from "drizzle-orm";
|
||||
import { photonAnswersMatch } from "../photon/interactions.js";
|
||||
import { and, or, eq, inArray, notExists, sql } from "drizzle-orm";
|
||||
import type { Db } from "@paperclipai/db";
|
||||
import {
|
||||
agentWakeupRequests,
|
||||
|
|
@ -338,7 +339,7 @@ async function resolveQuestionResponseChain(
|
|||
const provider =
|
||||
parent?.provider ??
|
||||
(
|
||||
["slack", "github", "discord", "microsoft-teams", "telegram"] as const
|
||||
["slack", "github", "discord", "microsoft-teams", "telegram", "imessage-photon"] as const
|
||||
).find(
|
||||
(candidate) =>
|
||||
sourceContext.source === `chat:${candidate}` ||
|
||||
|
|
@ -448,7 +449,7 @@ async function resolveQuestionResponseChain(
|
|||
eq(chatDeliveries.companyId, chatActions.companyId),
|
||||
eq(chatDeliveries.endpointId, chatActions.endpointId),
|
||||
eq(chatDeliveries.conversationId, chatActions.conversationId),
|
||||
eq(chatDeliveries.principalId, chatActions.principalId),
|
||||
or(eq(chatActions.kind, "photon_interaction"), eq(chatDeliveries.principalId, chatActions.principalId)),
|
||||
),
|
||||
)
|
||||
.innerJoin(
|
||||
|
|
@ -485,7 +486,7 @@ async function resolveQuestionResponseChain(
|
|||
.where(
|
||||
and(
|
||||
eq(chatActions.companyId, binding.companyId),
|
||||
inArray(chatActions.kind, ["question_answer", "question_form_submit"]),
|
||||
inArray(chatActions.kind, ["question_answer", "question_form_submit", "photon_interaction"]),
|
||||
eq(chatActions.status, "processed"),
|
||||
sql`${chatActions.payload}->>'interactionId' = ${interaction.id}`,
|
||||
sql`${chatActions.result}->>'interactionId' = ${interaction.id}`,
|
||||
|
|
@ -686,6 +687,8 @@ async function resolveQuestionResponseChain(
|
|||
)
|
||||
)
|
||||
return null;
|
||||
} else if (action.kind === "photon_interaction") {
|
||||
if (provider !== "imessage-photon" || !photonAnswersMatch(interaction as unknown as AskUserQuestionsInteraction, action.result) || action.payload.version !== 1 || action.payload.sessionGeneration !== conversation.sessionGeneration || typeof action.payload.expiresAt !== "string" || Date.parse(action.payload.expiresAt) <= interaction.resolvedAt.getTime() || !Number.isFinite(Date.parse(action.payload.expiresAt))) return null;
|
||||
} else if (
|
||||
!completedQuestionFormMatchesInteraction(
|
||||
interaction as unknown as AskUserQuestionsInteraction,
|
||||
|
|
@ -697,12 +700,13 @@ async function resolveQuestionResponseChain(
|
|||
if (
|
||||
inbound.state !== "processed" ||
|
||||
comment.deletedAt !== null ||
|
||||
comment.authorUserId !== interaction.resolvedByUserId ||
|
||||
(action?.kind !== "photon_interaction" && comment.authorUserId !== interaction.resolvedByUserId) ||
|
||||
conversation.issueId !== binding.issueId ||
|
||||
!["active", "waiting"].includes(conversation.state) ||
|
||||
endpoint.provider !== provider ||
|
||||
endpoint.assignedAgentId !== binding.agentId ||
|
||||
endpoint.status !== "active" ||
|
||||
(endpoint.status !== "active" &&
|
||||
!(provider === "imessage-photon" && endpoint.status === "verifying" && record(endpoint.setup).step === "test")) ||
|
||||
publication.state !== "published" ||
|
||||
!publication.providerMessageId ||
|
||||
publication.issueId !== binding.issueId ||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,576 @@
|
|||
import { PhotonRecoveryTransport } from "./recovery-transport.js";
|
||||
import { createHash } from "node:crypto";
|
||||
import {
|
||||
createGrpcClient,
|
||||
type GrpcAdvancedIMessage,
|
||||
type Message as PhotonMessage,
|
||||
type Chat as PhotonChat,
|
||||
} from "@photon-ai/advanced-imessage";
|
||||
import {
|
||||
Message,
|
||||
parseMarkdown,
|
||||
stringifyMarkdown,
|
||||
type Adapter,
|
||||
type AdapterPostableMessage,
|
||||
type ChatInstance,
|
||||
type FileUpload,
|
||||
type FormattedContent,
|
||||
type FetchOptions,
|
||||
type ThreadInfo,
|
||||
type Attachment,
|
||||
} from "chat";
|
||||
import {
|
||||
PhotonLineAuthentication,
|
||||
PhotonError,
|
||||
photonFailure,
|
||||
} from "./cloud.js";
|
||||
import { PhotonState } from "./state.js";
|
||||
import {
|
||||
downloadPhotonAttachment,
|
||||
type PhotonAttachmentLocator,
|
||||
} from "./attachments.js";
|
||||
import { MAX_ATTACHMENT_BYTES } from "../../attachment-types.js";
|
||||
|
||||
export interface PhotonThread {
|
||||
lineId: string;
|
||||
chatGuid: string;
|
||||
isGroup: boolean;
|
||||
}
|
||||
interface SendRecord {
|
||||
schema: 1;
|
||||
digest: string;
|
||||
phase: "prepared" | "uploading" | "uploaded" | "sending" | "sent";
|
||||
attachmentGuid?: string;
|
||||
messageGuid?: string;
|
||||
}
|
||||
export function photonThreadId(value: PhotonThread): string {
|
||||
return `imessage-photon:${value.lineId}:${value.isGroup ? "g" : "d"}:${Buffer.from(value.chatGuid).toString("base64url")}`;
|
||||
}
|
||||
export function parsePhotonThreadId(value: string): PhotonThread {
|
||||
const match =
|
||||
/^imessage-photon:([a-zA-Z0-9-]{1,63}):(d|g):([a-zA-Z0-9_-]{1,1024})$/.exec(
|
||||
value,
|
||||
);
|
||||
if (!match) throw new Error("Invalid Photon conversation identity");
|
||||
const chatGuid = Buffer.from(match[3], "base64url").toString("utf8");
|
||||
if (!chatGuid || Buffer.from(chatGuid).toString("base64url") !== match[3])
|
||||
throw new Error("Invalid Photon chat identity");
|
||||
return { lineId: match[1], chatGuid, isGroup: match[2] === "g" };
|
||||
}
|
||||
/** Closed quote context survives durable admission without retaining SDK objects. */
|
||||
export function photonReplyReference(
|
||||
raw: unknown,
|
||||
): { guid: string; part?: string } | null {
|
||||
if (!raw || typeof raw !== "object") return null;
|
||||
const value = raw as Partial<PhotonMessage>;
|
||||
const guid = value.replyTargetGuid ?? value.threadOriginatorGuid;
|
||||
if (typeof guid !== "string" || !guid || guid.length > 512) return null;
|
||||
return {
|
||||
guid,
|
||||
...(typeof value.threadOriginatorPart === "string" &&
|
||||
value.threadOriginatorPart.length <= 64
|
||||
? { part: value.threadOriginatorPart }
|
||||
: {}),
|
||||
};
|
||||
}
|
||||
export function splitPhotonText(text: string, maximum = 4000): string[] {
|
||||
const points = Array.from(text);
|
||||
const result: string[] = [];
|
||||
while (points.length > maximum) {
|
||||
const candidate = points.slice(0, maximum).join("");
|
||||
const paragraph = candidate.lastIndexOf("\n\n");
|
||||
const count =
|
||||
paragraph >= maximum / 2
|
||||
? Array.from(candidate.slice(0, paragraph + 2)).length
|
||||
: maximum;
|
||||
result.push(points.splice(0, count).join(""));
|
||||
}
|
||||
if (points.length) result.push(points.join(""));
|
||||
return result;
|
||||
}
|
||||
function textOf(message: AdapterPostableMessage): string {
|
||||
if (typeof message === "string") return message;
|
||||
if ("markdown" in message) return message.markdown;
|
||||
if ("raw" in message) return message.raw;
|
||||
if ("ast" in message) return stringifyMarkdown(message.ast);
|
||||
if ("fallbackText" in message) return message.fallbackText ?? "";
|
||||
throw new Error("Photon requires a reviewed text or poll publication");
|
||||
}
|
||||
|
||||
export class PhotonChatAdapter implements Adapter<PhotonThread, PhotonMessage> {
|
||||
readonly name = "imessage-photon";
|
||||
readonly lockScope = "channel" as const;
|
||||
readonly botUserId: string;
|
||||
readonly client: GrpcAdvancedIMessage;
|
||||
private closed = false;
|
||||
private recovery?: PhotonRecoveryTransport;
|
||||
private readonly typing = new Map<string, ReturnType<typeof setTimeout>>();
|
||||
typingGuard?: (activeThreadId?: string) => Promise<void>;
|
||||
recoveryStream(sequence?: number) {
|
||||
this.recovery ??= new PhotonRecoveryTransport(this.authentication);
|
||||
return this.recovery.catchUp(sequence);
|
||||
}
|
||||
constructor(
|
||||
readonly userName: string,
|
||||
readonly authentication: PhotonLineAuthentication,
|
||||
readonly state: PhotonState,
|
||||
client?: GrpcAdvancedIMessage,
|
||||
) {
|
||||
this.botUserId = authentication.identity.phoneNumber;
|
||||
this.client =
|
||||
client ??
|
||||
createGrpcClient({
|
||||
address: authentication.address,
|
||||
token: () => authentication.token(),
|
||||
tls: true,
|
||||
retry: false,
|
||||
autoIdempotency: false,
|
||||
timeout: 25_000,
|
||||
channelOptions: {
|
||||
"grpc.max_receive_message_length": MAX_ATTACHMENT_BYTES + 1024 * 1024,
|
||||
},
|
||||
});
|
||||
}
|
||||
async initialize(_chat: ChatInstance): Promise<void> {
|
||||
await this.authentication.token();
|
||||
}
|
||||
async disconnect(): Promise<void> {
|
||||
this.closed = true;
|
||||
for (const timer of this.typing.values()) clearTimeout(timer);
|
||||
this.typing.clear();
|
||||
this.authentication.retire();
|
||||
this.recovery?.close();
|
||||
await this.client.close();
|
||||
}
|
||||
encodeThreadId(value: PhotonThread): string {
|
||||
if (value.isGroup && this.authentication.identity.allocation === "shared")
|
||||
throw new PhotonError("rejected", "Photon shared channels support direct messages only");
|
||||
if (value.lineId !== this.authentication.identity.lineId)
|
||||
throw new Error("Wrong Photon line");
|
||||
return photonThreadId(value);
|
||||
}
|
||||
decodeThreadId(id: string): PhotonThread {
|
||||
const value = parsePhotonThreadId(id);
|
||||
if (value.isGroup && this.authentication.identity.allocation === "shared")
|
||||
throw new PhotonError("rejected", "Photon shared channels support direct messages only");
|
||||
if (value.lineId !== this.authentication.identity.lineId)
|
||||
throw new Error("Wrong Photon line");
|
||||
return value;
|
||||
}
|
||||
channelIdFromThreadId(id: string): string {
|
||||
this.decodeThreadId(id);
|
||||
return id;
|
||||
}
|
||||
isDM(id: string): boolean {
|
||||
return !this.decodeThreadId(id).isGroup;
|
||||
}
|
||||
async chatInfo(id: string): Promise<PhotonChat> {
|
||||
if (this.closed) throw new Error("Photon runtime retired");
|
||||
const identity = this.decodeThreadId(id);
|
||||
const chat = await this.client.chats
|
||||
.get(identity.chatGuid)
|
||||
.catch((error) => {
|
||||
throw photonFailure(error);
|
||||
});
|
||||
if (
|
||||
chat.guid !== identity.chatGuid ||
|
||||
chat.isGroup !== identity.isGroup ||
|
||||
chat.service !== "iMessage"
|
||||
)
|
||||
throw new Error("Photon conversation identity changed");
|
||||
return chat;
|
||||
}
|
||||
async fetchThread(id: string): Promise<ThreadInfo> {
|
||||
const chat = await this.chatInfo(id);
|
||||
return {
|
||||
id,
|
||||
channelId: id,
|
||||
channelName:
|
||||
chat.displayName || chat.participants.map((p) => p.address).join(", "),
|
||||
isDM: !chat.isGroup,
|
||||
metadata: {
|
||||
participants: chat.participants.map((p) => ({
|
||||
address: p.address,
|
||||
service: p.service,
|
||||
})),
|
||||
isArchived: chat.isArchived,
|
||||
},
|
||||
};
|
||||
}
|
||||
async fetchChannelInfo(id: string) {
|
||||
const info = await this.fetchThread(id);
|
||||
return {
|
||||
id,
|
||||
name: info.channelName,
|
||||
isDM: info.isDM,
|
||||
metadata: info.metadata,
|
||||
};
|
||||
}
|
||||
parseMessage(raw: PhotonMessage): Message<PhotonMessage> {
|
||||
if (raw.chatGuids.length !== 1)
|
||||
throw new Error(
|
||||
"Photon message must have one authenticated conversation",
|
||||
);
|
||||
// The receiver supplies the authoritative chat shape; fetch paths set it too.
|
||||
const isGroup = (raw as PhotonMessage & { paperclipIsGroup?: boolean })
|
||||
.paperclipIsGroup;
|
||||
if (typeof isGroup !== "boolean")
|
||||
throw new Error("Photon message is missing its authenticated chat shape");
|
||||
const chatGuid = raw.chatGuids[0];
|
||||
const threadId = this.encodeThreadId({
|
||||
lineId: this.authentication.identity.lineId,
|
||||
chatGuid,
|
||||
isGroup,
|
||||
});
|
||||
const address = raw.sender?.address ?? (raw.isFromMe ? this.botUserId : "");
|
||||
const service = raw.sender?.service;
|
||||
const authorId = raw.isFromMe ? this.botUserId : `${service}:${address}`;
|
||||
const attachments: Attachment[] = raw.content.attachments
|
||||
.filter((a) => !a.isHidden && !a.isSticker)
|
||||
.map((a) => {
|
||||
const locator: PhotonAttachmentLocator = {
|
||||
kind: "photon_attachment",
|
||||
lineId: this.authentication.identity.lineId,
|
||||
chatGuid,
|
||||
messageGuid: raw.guid,
|
||||
attachmentGuid: a.guid,
|
||||
};
|
||||
return {
|
||||
type: a.mimeType.startsWith("image/")
|
||||
? "image"
|
||||
: a.mimeType.startsWith("audio/")
|
||||
? "audio"
|
||||
: a.mimeType.startsWith("video/")
|
||||
? "video"
|
||||
: "file",
|
||||
name: a.fileName,
|
||||
mimeType: a.mimeType,
|
||||
size: a.totalBytes,
|
||||
fetchMetadata: {
|
||||
kind: locator.kind,
|
||||
lineId: locator.lineId,
|
||||
chatGuid: locator.chatGuid,
|
||||
messageGuid: locator.messageGuid,
|
||||
attachmentGuid: locator.attachmentGuid,
|
||||
},
|
||||
fetchData: () =>
|
||||
downloadPhotonAttachment(
|
||||
this.client,
|
||||
this.authentication.identity.lineId,
|
||||
locator,
|
||||
this.authentication.identity.allocation,
|
||||
),
|
||||
};
|
||||
});
|
||||
const text = raw.content.text ?? "";
|
||||
return new Message({
|
||||
id: raw.guid,
|
||||
threadId,
|
||||
text,
|
||||
formatted: parseMarkdown(text),
|
||||
raw,
|
||||
attachments,
|
||||
author: {
|
||||
userId: authorId,
|
||||
userName: address,
|
||||
fullName: address,
|
||||
isBot: false,
|
||||
isMe: raw.isFromMe,
|
||||
isSystem:
|
||||
raw.isSystemMessage ||
|
||||
raw.isServiceMessage ||
|
||||
!address ||
|
||||
(service !== "iMessage" && !raw.isFromMe),
|
||||
},
|
||||
metadata: {
|
||||
dateSent: new Date(raw.dateCreated),
|
||||
edited: !!raw.dateEdited,
|
||||
editedAt: raw.dateEdited ? new Date(raw.dateEdited) : undefined,
|
||||
},
|
||||
});
|
||||
}
|
||||
normalize(raw: PhotonMessage, chat: PhotonChat): Message<PhotonMessage> {
|
||||
if (!raw.chatGuids.includes(chat.guid))
|
||||
throw new Error("Photon message belongs to another chat");
|
||||
return this.parseMessage({
|
||||
...raw,
|
||||
chatGuids: [chat.guid],
|
||||
paperclipIsGroup: chat.isGroup,
|
||||
} as PhotonMessage);
|
||||
}
|
||||
async fetchMessage(id: string, messageId: string) {
|
||||
const chat = await this.chatInfo(id);
|
||||
return this.normalize(
|
||||
await this.client.messages.get(messageId).catch((error) => {
|
||||
throw photonFailure(error);
|
||||
}),
|
||||
chat,
|
||||
);
|
||||
}
|
||||
async fetchMessages(id: string, options?: FetchOptions) {
|
||||
const chat = await this.chatInfo(id);
|
||||
const page = await this.client.messages
|
||||
.listInChat(chat.guid, {
|
||||
pageSize: Math.min(options?.limit ?? 50, 100),
|
||||
pageToken: options?.cursor,
|
||||
})
|
||||
.catch((error) => {
|
||||
throw photonFailure(error);
|
||||
});
|
||||
return {
|
||||
messages: page.messages
|
||||
.map((message) => this.normalize(message, chat))
|
||||
.sort(
|
||||
(a, b) =>
|
||||
a.metadata.dateSent.getTime() - b.metadata.dateSent.getTime(),
|
||||
),
|
||||
nextCursor: page.nextPageToken,
|
||||
};
|
||||
}
|
||||
async getUser(userId: string) {
|
||||
return {
|
||||
userId,
|
||||
fullName: userId.replace(/^iMessage:/, ""),
|
||||
userName: userId.replace(/^iMessage:/, ""),
|
||||
isBot: false,
|
||||
};
|
||||
}
|
||||
async handleWebhook(): Promise<Response> {
|
||||
return new Response("Photon uses authenticated streams", { status: 405 });
|
||||
}
|
||||
renderFormatted(content: FormattedContent): string {
|
||||
return stringifyMarkdown(content);
|
||||
}
|
||||
async startTyping(id: string): Promise<void> {
|
||||
await this.refreshTyping(id, false);
|
||||
}
|
||||
private async refreshTyping(id: string, refresh: boolean): Promise<void> {
|
||||
if (this.closed) return;
|
||||
if (this.typing.has(id)) clearTimeout(this.typing.get(id));
|
||||
await this.typingGuard?.(refresh ? id : undefined);
|
||||
await this.client.chats
|
||||
.setTyping(this.decodeThreadId(id).chatGuid, true)
|
||||
.catch((error) => {
|
||||
throw photonFailure(error);
|
||||
});
|
||||
if (this.closed) return;
|
||||
// Refresh while work is running; final/prompt publication or runtime
|
||||
// retirement clears this timer. Every refresh checks endpoint ownership.
|
||||
const timer = setTimeout(() => {
|
||||
this.typing.delete(id);
|
||||
void this.refreshTyping(id, true).catch(() => {});
|
||||
}, 8_000);
|
||||
timer.unref();
|
||||
this.typing.set(id, timer);
|
||||
}
|
||||
async endTyping(id: string): Promise<void> {
|
||||
if (this.typing.has(id)) clearTimeout(this.typing.get(id));
|
||||
this.typing.delete(id);
|
||||
if (!this.closed)
|
||||
await this.client.chats
|
||||
.setTyping(this.decodeThreadId(id).chatGuid, false)
|
||||
.catch(() => {});
|
||||
}
|
||||
async postMessage(
|
||||
_id: string,
|
||||
_message: AdapterPostableMessage,
|
||||
): Promise<never> {
|
||||
throw new Error(
|
||||
"Photon sends require an immutable Paperclip publication identity",
|
||||
);
|
||||
}
|
||||
async editMessage(
|
||||
_id: string,
|
||||
_messageId: string,
|
||||
_message: AdapterPostableMessage,
|
||||
): Promise<never> {
|
||||
throw new Error(
|
||||
"Photon edits require an immutable Paperclip publication identity",
|
||||
);
|
||||
}
|
||||
async deleteMessage(): Promise<never> {
|
||||
throw new Error("Photon message deletion is not supported");
|
||||
}
|
||||
async addReaction(): Promise<void> {
|
||||
/* Receipt reactions are intentionally not published. */
|
||||
}
|
||||
async removeReaction(): Promise<void> {
|
||||
/* Reactions never represent approvals. */
|
||||
}
|
||||
|
||||
async publish(
|
||||
id: string,
|
||||
publicationId: string,
|
||||
message: AdapterPostableMessage,
|
||||
options: {
|
||||
replyTo?: string;
|
||||
replaceMessageId?: string;
|
||||
retryUnknown?: boolean;
|
||||
assertCurrent(): Promise<void>;
|
||||
},
|
||||
): Promise<{ id: string; messageIds: string[] }> {
|
||||
const chat = await this.chatInfo(id);
|
||||
if (chat.isArchived) throw new Error("Photon conversation is unavailable");
|
||||
const text = textOf(message);
|
||||
const files =
|
||||
typeof message !== "string" && "files" in message
|
||||
? (message.files ?? [])
|
||||
: [];
|
||||
const parts = splitPhotonText(text);
|
||||
if (options.replaceMessageId && (parts.length !== 1 || files.length))
|
||||
throw new Error("Photon cannot edit a multipart publication");
|
||||
let lastId: string | undefined;
|
||||
const messageIds: string[] = [];
|
||||
for (let index = 0; index < parts.length; index++) {
|
||||
lastId = await this.sendPart(
|
||||
publicationId,
|
||||
`text-${index}`,
|
||||
{
|
||||
chatGuid: chat.guid,
|
||||
text: parts[index],
|
||||
replyTo: options.replyTo,
|
||||
replaceMessageId: options.replaceMessageId,
|
||||
},
|
||||
undefined,
|
||||
options,
|
||||
);
|
||||
messageIds.push(lastId);
|
||||
}
|
||||
for (let index = 0; index < files.length; index++) {
|
||||
const file = files[index];
|
||||
const bytes =
|
||||
file.data instanceof Blob
|
||||
? Buffer.from(await file.data.arrayBuffer())
|
||||
: Buffer.from(file.data as ArrayBuffer);
|
||||
if (bytes.length > MAX_ATTACHMENT_BYTES)
|
||||
throw new Error("Attachment exceeds the configured size limit");
|
||||
lastId = await this.sendPart(
|
||||
publicationId,
|
||||
`file-${index}`,
|
||||
{
|
||||
chatGuid: chat.guid,
|
||||
filename: file.filename,
|
||||
sha256: createHash("sha256").update(bytes).digest("hex"),
|
||||
replyTo: options.replyTo,
|
||||
},
|
||||
{ ...file, data: bytes },
|
||||
options,
|
||||
);
|
||||
messageIds.push(lastId);
|
||||
}
|
||||
if (!lastId) throw new Error("Photon publication is empty");
|
||||
await this.endTyping(id);
|
||||
return { id: lastId, messageIds };
|
||||
}
|
||||
private async sendPart(
|
||||
publicationId: string,
|
||||
part: string,
|
||||
payload: {
|
||||
chatGuid: string;
|
||||
text?: string;
|
||||
filename?: string;
|
||||
sha256?: string;
|
||||
replyTo?: string;
|
||||
replaceMessageId?: string;
|
||||
},
|
||||
file: FileUpload | undefined,
|
||||
options: { retryUnknown?: boolean; assertCurrent(): Promise<void> },
|
||||
): Promise<string> {
|
||||
const key = `send:${publicationId}:${part}`;
|
||||
const digest = createHash("sha256")
|
||||
.update(JSON.stringify(payload))
|
||||
.digest("hex");
|
||||
let record = await this.state.update<SendRecord>(key, (current) => {
|
||||
if (current && current.digest !== digest)
|
||||
throw new Error("Photon publication payload changed after preparation");
|
||||
return current ?? { schema: 1, digest, phase: "prepared" };
|
||||
});
|
||||
if (record.phase === "sent" && record.messageGuid)
|
||||
return record.messageGuid;
|
||||
if (
|
||||
(record.phase === "sending" || record.phase === "uploading") &&
|
||||
!options.retryUnknown
|
||||
)
|
||||
throw new PhotonError(
|
||||
"delivery_unknown",
|
||||
"Photon delivery is unknown; resolve this publication before retrying",
|
||||
);
|
||||
const save = async (patch: Partial<SendRecord>) => {
|
||||
record = await this.state.update<SendRecord>(key, (current) => {
|
||||
if (!current || current.digest !== digest)
|
||||
throw new Error("Photon send identity changed");
|
||||
return { ...current, ...patch };
|
||||
});
|
||||
};
|
||||
if (file && !record.attachmentGuid) {
|
||||
await options.assertCurrent();
|
||||
await save({ phase: "uploading" });
|
||||
try {
|
||||
const uploaded = await this.client.attachments.upload({
|
||||
fileName: file.filename,
|
||||
data: file.data as Buffer,
|
||||
});
|
||||
if (!uploaded.attachment.guid)
|
||||
throw new PhotonError(
|
||||
"delivery_unknown",
|
||||
"Photon upload receipt is missing",
|
||||
);
|
||||
await save({
|
||||
phase: "uploaded",
|
||||
attachmentGuid: uploaded.attachment.guid,
|
||||
});
|
||||
} catch (error) {
|
||||
const failure = photonFailure(error, true);
|
||||
if (failure.code !== "delivery_unknown")
|
||||
await save({ phase: "prepared" });
|
||||
throw failure;
|
||||
}
|
||||
}
|
||||
await options.assertCurrent();
|
||||
if (payload.replaceMessageId) {
|
||||
const original = await this.client.messages.get(payload.replaceMessageId);
|
||||
if (
|
||||
!original.isFromMe ||
|
||||
!original.chatGuids.includes(payload.chatGuid) ||
|
||||
Date.now() - new Date(original.dateCreated).getTime() >= 15 * 60_000
|
||||
)
|
||||
throw new PhotonError(
|
||||
"rejected",
|
||||
"The iMessage edit window expired; stage a correction as a new publication",
|
||||
);
|
||||
}
|
||||
await save({ phase: "sending" });
|
||||
const clientMessageId = createHash("sha256")
|
||||
.update(`${this.state.scope.endpointId}:${publicationId}:${part}`)
|
||||
.digest("hex");
|
||||
try {
|
||||
const result = record.attachmentGuid
|
||||
? await this.client.messages.sendAttachment(
|
||||
payload.chatGuid,
|
||||
record.attachmentGuid,
|
||||
{ clientMessageId, replyTo: payload.replyTo },
|
||||
)
|
||||
: payload.replaceMessageId
|
||||
? await this.client.messages.edit(
|
||||
payload.chatGuid,
|
||||
payload.replaceMessageId,
|
||||
payload.text!,
|
||||
{ clientMessageId },
|
||||
)
|
||||
: await this.client.messages.sendText(
|
||||
payload.chatGuid,
|
||||
payload.text!,
|
||||
{ clientMessageId, replyTo: payload.replyTo },
|
||||
);
|
||||
if (!result.guid || !result.chatGuids.includes(payload.chatGuid))
|
||||
throw new Error("Photon returned an invalid send receipt");
|
||||
await save({ phase: "sent", messageGuid: result.guid });
|
||||
return result.guid;
|
||||
} catch (error) {
|
||||
const failure = photonFailure(error, true);
|
||||
if (failure.code !== "delivery_unknown")
|
||||
await save({ phase: record.attachmentGuid ? "uploaded" : "prepared" });
|
||||
throw failure;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,208 @@
|
|||
import type { Attachment, Message as ChatMessage } from "chat";
|
||||
import type {
|
||||
GrpcAdvancedIMessage,
|
||||
CompanionInfo,
|
||||
Message as PhotonMessage,
|
||||
} from "@photon-ai/advanced-imessage";
|
||||
import { z } from "zod";
|
||||
import { MAX_ATTACHMENT_BYTES } from "../../attachment-types.js";
|
||||
import { PhotonError, photonFailure } from "./cloud.js";
|
||||
const guid = z.string().min(1).max(512);
|
||||
export const photonAttachmentLocatorSchema = z
|
||||
.object({
|
||||
kind: z.literal("photon_attachment"),
|
||||
lineId: guid,
|
||||
chatGuid: guid,
|
||||
messageGuid: guid,
|
||||
attachmentGuid: guid,
|
||||
partIndex: z.number().int().nonnegative().max(100).optional(),
|
||||
})
|
||||
.strict();
|
||||
export type PhotonAttachmentLocator = z.infer<
|
||||
typeof photonAttachmentLocatorSchema
|
||||
>;
|
||||
|
||||
type PhotonCompanion =
|
||||
| { unavailable: true }
|
||||
| { unavailable: false; data: Buffer; fileName: string; mimeType: string };
|
||||
const companions = new WeakMap<Buffer, PhotonCompanion>();
|
||||
/** Bytes are ephemeral. Recovery downloads the source-bound primary and companion again. */
|
||||
export function takePhotonCompanion(body: Buffer): PhotonCompanion | undefined {
|
||||
const companion = companions.get(body);
|
||||
companions.delete(body);
|
||||
return companion;
|
||||
}
|
||||
|
||||
export function photonAttachmentLocator(
|
||||
attachment: Attachment,
|
||||
lineId: string,
|
||||
chatGuid: string,
|
||||
message: ChatMessage,
|
||||
): PhotonAttachmentLocator | null {
|
||||
const parsed = photonAttachmentLocatorSchema.safeParse(
|
||||
attachment.fetchMetadata
|
||||
? {
|
||||
kind: attachment.fetchMetadata.kind,
|
||||
lineId: attachment.fetchMetadata.lineId,
|
||||
chatGuid: attachment.fetchMetadata.chatGuid,
|
||||
messageGuid: attachment.fetchMetadata.messageGuid,
|
||||
attachmentGuid: attachment.fetchMetadata.attachmentGuid,
|
||||
}
|
||||
: null,
|
||||
);
|
||||
if (
|
||||
!parsed.success ||
|
||||
parsed.data.lineId !== lineId ||
|
||||
parsed.data.chatGuid !== chatGuid ||
|
||||
parsed.data.messageGuid !== message.id
|
||||
)
|
||||
return null;
|
||||
const raw = message.raw as PhotonMessage;
|
||||
if (
|
||||
!raw.chatGuids?.includes(chatGuid) ||
|
||||
!raw.content?.attachments.some(
|
||||
(candidate) => candidate.guid === parsed.data.attachmentGuid,
|
||||
)
|
||||
)
|
||||
return null;
|
||||
return parsed.data;
|
||||
}
|
||||
export async function downloadPhotonAttachment(
|
||||
client: GrpcAdvancedIMessage,
|
||||
lineId: string,
|
||||
locator: PhotonAttachmentLocator,
|
||||
allocation: "dedicated" | "shared" = "dedicated",
|
||||
): Promise<Buffer> {
|
||||
photonAttachmentLocatorSchema.parse(locator);
|
||||
if (locator.lineId !== lineId)
|
||||
throw new Error("Photon attachment belongs to another line");
|
||||
const source = await client.messages
|
||||
.get(locator.messageGuid)
|
||||
.catch((error) => {
|
||||
throw photonFailure(error);
|
||||
});
|
||||
const attachment = source.content.attachments.find(
|
||||
(candidate) => candidate.guid === locator.attachmentGuid,
|
||||
);
|
||||
if (
|
||||
source.guid !== locator.messageGuid ||
|
||||
!source.chatGuids.includes(locator.chatGuid) ||
|
||||
!attachment ||
|
||||
attachment.isSticker ||
|
||||
attachment.isHidden
|
||||
)
|
||||
throw new Error(
|
||||
"Photon attachment does not belong to this message and chat",
|
||||
);
|
||||
if (
|
||||
!Number.isSafeInteger(attachment.totalBytes) ||
|
||||
attachment.totalBytes < 0 ||
|
||||
attachment.totalBytes > MAX_ATTACHMENT_BYTES
|
||||
)
|
||||
throw new Error("Attachment exceeds the configured size limit");
|
||||
const stream = client.attachments.downloadStream(locator.attachmentGuid);
|
||||
let timedOut = false;
|
||||
const timer = setTimeout(() => {
|
||||
timedOut = true;
|
||||
void stream.close();
|
||||
}, 30_000);
|
||||
timer.unref();
|
||||
const chunks: Uint8Array[] = [];
|
||||
let length = 0;
|
||||
let header = false;
|
||||
let companionInfo: CompanionInfo | undefined;
|
||||
let companionUnavailable = false;
|
||||
let companionStarted = false;
|
||||
let companionLength = 0;
|
||||
const companionChunks: Uint8Array[] = [];
|
||||
try {
|
||||
for await (const part of stream) {
|
||||
if (part.type === "header") {
|
||||
// The shared gateway rewrites message/metadata attachment IDs to opaque
|
||||
// project aliases, but streams the native UUID in download headers.
|
||||
// Ownership comes from the authenticated source-message lookup above
|
||||
// and this exact alias-addressed RPC, never from matching filenames.
|
||||
const sharedAlias = allocation === "shared" &&
|
||||
/^spc-att-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i.test(locator.attachmentGuid);
|
||||
const matchingSharedHeader = sharedAlias &&
|
||||
/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i.test(part.info.guid) &&
|
||||
part.info.totalBytes === attachment.totalBytes &&
|
||||
part.info.mimeType === attachment.mimeType &&
|
||||
part.info.fileName === attachment.fileName;
|
||||
if (
|
||||
header ||
|
||||
(part.info.guid !== locator.attachmentGuid && !matchingSharedHeader) ||
|
||||
part.info.isHidden || part.info.isSticker ||
|
||||
!Number.isSafeInteger(part.info.totalBytes) || part.info.totalBytes < 0 ||
|
||||
part.info.totalBytes > MAX_ATTACHMENT_BYTES
|
||||
)
|
||||
throw new Error("Photon attachment metadata changed");
|
||||
header = true;
|
||||
companionInfo = part.companionInfo;
|
||||
companionUnavailable = Boolean(
|
||||
companionInfo &&
|
||||
(companionInfo.kind !== "live-photo-video" ||
|
||||
!["video/quicktime", "video/mp4"].includes(
|
||||
companionInfo.mimeType,
|
||||
) ||
|
||||
!Number.isSafeInteger(companionInfo.totalBytes) ||
|
||||
companionInfo.totalBytes <= 0 ||
|
||||
companionInfo.totalBytes > MAX_ATTACHMENT_BYTES),
|
||||
);
|
||||
} else if (part.type === "primaryChunk") {
|
||||
if (companionStarted)
|
||||
throw new Error("Photon attachment chunks arrived out of order");
|
||||
if (!header) throw new Error("Photon attachment header is missing");
|
||||
length += part.data.length;
|
||||
if (length > MAX_ATTACHMENT_BYTES)
|
||||
throw new Error("Attachment exceeds the configured size limit");
|
||||
chunks.push(part.data);
|
||||
} else if (part.type === "companionChunk") {
|
||||
if (!header || !companionInfo)
|
||||
throw new Error("Photon companion metadata is missing");
|
||||
companionStarted = true;
|
||||
companionLength += part.data.length;
|
||||
if (companionUnavailable || companionLength > MAX_ATTACHMENT_BYTES) {
|
||||
companionUnavailable = true;
|
||||
break;
|
||||
}
|
||||
companionChunks.push(part.data);
|
||||
}
|
||||
}
|
||||
if (timedOut || !header || !length)
|
||||
throw new PhotonError(
|
||||
"attachment_not_ready",
|
||||
"Photon attachment is still being prepared; retry download",
|
||||
);
|
||||
if (attachment.totalBytes > 0 && length !== attachment.totalBytes)
|
||||
throw new PhotonError(
|
||||
"attachment_not_ready",
|
||||
"Photon attachment transfer is incomplete",
|
||||
);
|
||||
if (
|
||||
companionInfo &&
|
||||
!companionUnavailable &&
|
||||
companionLength !== companionInfo.totalBytes
|
||||
)
|
||||
throw new PhotonError(
|
||||
"attachment_not_ready",
|
||||
"Photon Live Photo companion is still being prepared",
|
||||
);
|
||||
const body = Buffer.concat(chunks);
|
||||
if (companionUnavailable) companions.set(body, { unavailable: true });
|
||||
else if (companionInfo)
|
||||
companions.set(body, {
|
||||
unavailable: false,
|
||||
data: Buffer.concat(companionChunks),
|
||||
fileName: companionInfo.fileName,
|
||||
mimeType: companionInfo.mimeType,
|
||||
});
|
||||
return body;
|
||||
} catch (error) {
|
||||
if (error instanceof Error && !("code" in error)) throw error;
|
||||
throw photonFailure(error);
|
||||
} finally {
|
||||
clearTimeout(timer);
|
||||
await stream.close();
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,350 @@
|
|||
import { createHash } from "node:crypto";
|
||||
import {
|
||||
photonLineIdSchema,
|
||||
photonProjectIdSchema,
|
||||
type PhotonProjectInspection,
|
||||
} from "@paperclipai/shared";
|
||||
import { IMessageError } from "@photon-ai/advanced-imessage";
|
||||
|
||||
const CLOUD_ORIGIN = "https://spectrum.photon.codes";
|
||||
const MAX_RESPONSE_BYTES = 256 * 1024;
|
||||
export class PhotonError extends Error {
|
||||
constructor(
|
||||
readonly code:
|
||||
| "credentials"
|
||||
| "line_unavailable"
|
||||
| "quota"
|
||||
| "network"
|
||||
| "history_gap"
|
||||
| "delivery_unknown"
|
||||
| "attachment_not_ready"
|
||||
| "invalid_response"
|
||||
| "rejected",
|
||||
message: string,
|
||||
readonly retryAfterMs?: number,
|
||||
) {
|
||||
super(message);
|
||||
this.name = "PhotonError";
|
||||
}
|
||||
}
|
||||
|
||||
/** Only documented semantic rejections prove a write did not happen. */
|
||||
export function photonFailure(error: unknown, writing = false): PhotonError {
|
||||
if (error instanceof PhotonError) return error;
|
||||
if (error instanceof IMessageError) {
|
||||
if (
|
||||
[
|
||||
"unauthenticated",
|
||||
"tokenExpired",
|
||||
"tokenBlocked",
|
||||
"unauthorized",
|
||||
].includes(error.code)
|
||||
)
|
||||
return new PhotonError(
|
||||
"credentials",
|
||||
"Photon rejected the selected line credentials; reconnect the channel",
|
||||
);
|
||||
if (
|
||||
[
|
||||
"dailyLimitExceeded",
|
||||
"recipientLimitExceeded",
|
||||
"uploadRateExceeded",
|
||||
"recipientCoolingDown",
|
||||
"recipientLocked",
|
||||
"sendReceiveRatioExceeded",
|
||||
"contentDuplicateExceeded",
|
||||
].includes(error.code)
|
||||
)
|
||||
return new PhotonError(
|
||||
"quota",
|
||||
"Photon has temporarily limited this line",
|
||||
error.retryAfter,
|
||||
);
|
||||
if (error.code === "attachmentNotReady")
|
||||
return new PhotonError(
|
||||
"attachment_not_ready",
|
||||
"Photon attachment is still being prepared",
|
||||
);
|
||||
if (
|
||||
[
|
||||
"chatNotFound",
|
||||
"messageNotFound",
|
||||
"attachmentNotFound",
|
||||
"invalidArgument",
|
||||
"preconditionFailed",
|
||||
"operationNotSupported",
|
||||
].includes(error.code)
|
||||
)
|
||||
return new PhotonError(
|
||||
"rejected",
|
||||
`Photon rejected the operation (${error.code})`,
|
||||
);
|
||||
}
|
||||
return writing
|
||||
? new PhotonError(
|
||||
"delivery_unknown",
|
||||
"Photon delivery is unknown; reconcile its receipt before retrying",
|
||||
)
|
||||
: new PhotonError("network", "Photon connection interrupted; retrying");
|
||||
}
|
||||
/** Shared credentials own one project, not any number in the provider pool. */
|
||||
export function photonSharedIdentity(projectId: string): string {
|
||||
photonProjectIdSchema.parse(projectId);
|
||||
return `photon-project:${projectId}`;
|
||||
}
|
||||
export function photonSharedScope(projectId: string): string {
|
||||
photonProjectIdSchema.parse(projectId);
|
||||
return `shared-${createHash("sha256").update(projectId).digest("hex").slice(0, 48)}`;
|
||||
}
|
||||
interface CloudAllocation {
|
||||
sharedToken?: string;
|
||||
inspection: PhotonProjectInspection;
|
||||
tokens: ReadonlyMap<string, string>;
|
||||
expiresIn: number;
|
||||
}
|
||||
function record(value: unknown): value is Record<string, unknown> {
|
||||
return !!value && typeof value === "object" && !Array.isArray(value);
|
||||
}
|
||||
|
||||
/** Only this module ever sees a Cloud project secret or minted line tokens. */
|
||||
export class PhotonCloudClient {
|
||||
constructor(private readonly fetchImpl: typeof fetch = fetch) {}
|
||||
private async request(
|
||||
projectId: string,
|
||||
projectSecret: string,
|
||||
suffix: string,
|
||||
method: string,
|
||||
): Promise<unknown> {
|
||||
photonProjectIdSchema.parse(projectId);
|
||||
if (!projectSecret || projectSecret.length > 4096)
|
||||
throw new PhotonError("credentials", "Enter a Photon project secret");
|
||||
let response: Response;
|
||||
try {
|
||||
response = await this.fetchImpl(
|
||||
`${CLOUD_ORIGIN}/projects/${encodeURIComponent(projectId)}/${suffix}`,
|
||||
{
|
||||
method,
|
||||
redirect: "error",
|
||||
signal: AbortSignal.timeout(15_000),
|
||||
headers: {
|
||||
authorization: `Basic ${Buffer.from(`${projectId}:${projectSecret}`).toString("base64")}`,
|
||||
accept: "application/json",
|
||||
},
|
||||
},
|
||||
);
|
||||
} catch {
|
||||
throw new PhotonError(
|
||||
"network",
|
||||
"Photon Cloud could not be reached; retry the connection",
|
||||
);
|
||||
}
|
||||
if (response.status === 401 || response.status === 403)
|
||||
throw new PhotonError(
|
||||
"credentials",
|
||||
"Photon rejected this project ID or secret",
|
||||
);
|
||||
if (response.status === 429)
|
||||
throw new PhotonError(
|
||||
"quota",
|
||||
"Photon Cloud request limit reached; retry later",
|
||||
);
|
||||
if (!response.ok)
|
||||
throw new PhotonError(
|
||||
"network",
|
||||
`Photon Cloud returned HTTP ${response.status}`,
|
||||
);
|
||||
const reader = response.body?.getReader();
|
||||
if (!reader)
|
||||
throw new PhotonError(
|
||||
"invalid_response",
|
||||
"Photon returned an empty response",
|
||||
);
|
||||
const chunks: Uint8Array[] = [];
|
||||
let length = 0;
|
||||
try {
|
||||
for (;;) {
|
||||
const next = await reader.read();
|
||||
if (next.done) break;
|
||||
length += next.value.length;
|
||||
if (length > MAX_RESPONSE_BYTES)
|
||||
throw new PhotonError(
|
||||
"invalid_response",
|
||||
"Photon project response exceeds the supported size",
|
||||
);
|
||||
chunks.push(next.value);
|
||||
}
|
||||
const body: unknown = JSON.parse(Buffer.concat(chunks).toString("utf8"));
|
||||
if (!record(body) || body.succeed !== true || !record(body.data))
|
||||
throw new PhotonError(
|
||||
"invalid_response",
|
||||
"Photon did not return a valid project response",
|
||||
);
|
||||
return body.data;
|
||||
} catch (error) {
|
||||
if (error instanceof PhotonError) throw error;
|
||||
throw new PhotonError(
|
||||
"invalid_response",
|
||||
"Photon returned an invalid project response",
|
||||
);
|
||||
} finally {
|
||||
await reader.cancel().catch(() => {});
|
||||
}
|
||||
}
|
||||
async allocation(
|
||||
projectId: string,
|
||||
projectSecret: string,
|
||||
): Promise<CloudAllocation> {
|
||||
const project = await this.request(projectId, projectSecret, "", "GET");
|
||||
if (
|
||||
record(project) &&
|
||||
typeof project.id === "string" &&
|
||||
project.id !== projectId
|
||||
)
|
||||
throw new PhotonError(
|
||||
"credentials",
|
||||
"Photon returned a different project identity",
|
||||
);
|
||||
const data = await this.request(
|
||||
projectId,
|
||||
projectSecret,
|
||||
"imessage/tokens",
|
||||
"POST",
|
||||
);
|
||||
if (!record(data))
|
||||
throw new PhotonError("invalid_response", "Photon allocation is missing");
|
||||
const projectName =
|
||||
record(project) && typeof project.name === "string"
|
||||
? project.name.slice(0, 160)
|
||||
: projectId;
|
||||
const inspection: PhotonProjectInspection = {
|
||||
projectId,
|
||||
projectName,
|
||||
allocation: data.type === "dedicated" ? "dedicated" : "shared",
|
||||
eligible: false,
|
||||
lines: [],
|
||||
};
|
||||
if (data.type === "shared") {
|
||||
if (typeof data.token !== "string" || !data.token || data.token.length > 16_384 ||
|
||||
typeof data.expiresIn !== "number" || !Number.isFinite(data.expiresIn) || data.expiresIn < 30)
|
||||
throw new PhotonError("invalid_response", "Photon returned invalid shared project credentials");
|
||||
inspection.eligible = true;
|
||||
return { inspection, sharedToken: data.token, tokens: new Map(), expiresIn: Math.min(data.expiresIn, 86_400) };
|
||||
}
|
||||
if (
|
||||
data.type !== "dedicated" ||
|
||||
!record(data.auth) ||
|
||||
!record(data.numbers) ||
|
||||
typeof data.expiresIn !== "number" ||
|
||||
!Number.isFinite(data.expiresIn) ||
|
||||
data.expiresIn < 30
|
||||
) {
|
||||
throw new PhotonError(
|
||||
"line_unavailable",
|
||||
"Photon did not supply a dedicated line with a stable phone number",
|
||||
);
|
||||
}
|
||||
const tokens = new Map<string, string>();
|
||||
for (const [lineId, token] of Object.entries(data.auth)) {
|
||||
if (
|
||||
!photonLineIdSchema.safeParse(lineId).success ||
|
||||
typeof token !== "string" ||
|
||||
!token ||
|
||||
token.length > 16_384
|
||||
)
|
||||
continue;
|
||||
const phoneNumber = data.numbers[lineId];
|
||||
if (
|
||||
typeof phoneNumber !== "string" ||
|
||||
!/^\+[1-9]\d{6,14}$/.test(phoneNumber)
|
||||
)
|
||||
continue;
|
||||
tokens.set(lineId, token);
|
||||
inspection.lines.push({ lineId, phoneNumber, eligible: true });
|
||||
}
|
||||
if (inspection.lines.length > 256)
|
||||
throw new PhotonError(
|
||||
"invalid_response",
|
||||
"Photon returned too many lines",
|
||||
);
|
||||
inspection.eligible = inspection.lines.length > 0;
|
||||
return { inspection, tokens, expiresIn: Math.min(data.expiresIn, 86_400) };
|
||||
}
|
||||
async inspect(
|
||||
projectId: string,
|
||||
projectSecret: string,
|
||||
): Promise<PhotonProjectInspection> {
|
||||
return (await this.allocation(projectId, projectSecret)).inspection;
|
||||
}
|
||||
}
|
||||
|
||||
/** A fixed identity; renewal can replace its token, never its project/line/number. */
|
||||
export class PhotonLineAuthentication {
|
||||
private current: { token: string; renewAt: number } | undefined;
|
||||
private renewing: Promise<string> | undefined;
|
||||
private retired = false;
|
||||
constructor(
|
||||
readonly identity: {
|
||||
allocation?: "shared" | "dedicated";
|
||||
projectId: string;
|
||||
lineId: string;
|
||||
phoneNumber: string;
|
||||
},
|
||||
private readonly secret: string,
|
||||
private readonly cloud = new PhotonCloudClient(),
|
||||
private readonly now = Date.now,
|
||||
) {
|
||||
photonProjectIdSchema.parse(identity.projectId);
|
||||
photonLineIdSchema.parse(identity.lineId);
|
||||
if (identity.allocation === "shared" && (identity.lineId !== photonSharedScope(identity.projectId) || identity.phoneNumber !== photonSharedIdentity(identity.projectId)))
|
||||
throw new PhotonError("credentials", "Photon shared identity must match its project");
|
||||
}
|
||||
get address(): string {
|
||||
return this.identity.allocation === "shared" ? "imessage.spectrum.photon.codes:443" : `${this.identity.lineId}.imsg.photon.codes:443`;
|
||||
}
|
||||
retire(): void {
|
||||
this.retired = true;
|
||||
this.current = undefined;
|
||||
}
|
||||
async token(): Promise<string> {
|
||||
if (this.retired)
|
||||
throw new PhotonError("credentials", "Photon runtime has been retired");
|
||||
if (this.current && this.current.renewAt > this.now())
|
||||
return this.current.token;
|
||||
this.renewing ??= this.renew().finally(() => {
|
||||
this.renewing = undefined;
|
||||
});
|
||||
return this.renewing;
|
||||
}
|
||||
private async renew(): Promise<string> {
|
||||
this.current = undefined;
|
||||
const allocation = await this.cloud.allocation(
|
||||
this.identity.projectId,
|
||||
this.secret,
|
||||
);
|
||||
if (allocation.inspection.allocation !== (this.identity.allocation ?? "dedicated"))
|
||||
throw new PhotonError("line_unavailable", "Photon project allocation changed; create a new channel for the new identity");
|
||||
if (this.identity.allocation === "shared") {
|
||||
if (!allocation.sharedToken || this.retired)
|
||||
throw new PhotonError("credentials", "Photon shared credentials are unavailable");
|
||||
this.current = { token: allocation.sharedToken, renewAt: this.now() + allocation.expiresIn * 800 };
|
||||
return allocation.sharedToken;
|
||||
}
|
||||
const line = allocation.inspection.lines.find(
|
||||
(candidate) => candidate.lineId === this.identity.lineId,
|
||||
);
|
||||
const token = allocation.tokens.get(this.identity.lineId);
|
||||
if (
|
||||
!line?.eligible ||
|
||||
line.phoneNumber !== this.identity.phoneNumber ||
|
||||
!token
|
||||
)
|
||||
throw new PhotonError(
|
||||
"line_unavailable",
|
||||
"The selected Photon number is no longer available; reconnect the same line or create a new channel",
|
||||
);
|
||||
if (this.retired)
|
||||
throw new PhotonError("credentials", "Photon runtime has been retired");
|
||||
this.current = { token, renewAt: this.now() + allocation.expiresIn * 800 };
|
||||
return token;
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,365 @@
|
|||
import { nativeSha256 } from "../native-runtime/canonical.js";
|
||||
import { createHash } from "node:crypto";
|
||||
import type {
|
||||
AskUserQuestionsInteraction,
|
||||
AskUserQuestionsAnswer,
|
||||
IssueThreadInteraction,
|
||||
RequestConfirmationInteraction,
|
||||
} from "@paperclipai/shared";
|
||||
import { PhotonChatAdapter } from "./adapter.js";
|
||||
import { PhotonError, photonFailure } from "./cloud.js";
|
||||
import { projectSafeChatPublicationText } from "../chat-publication-projection.js";
|
||||
|
||||
export class PhotonAnswerValidationError extends Error {}
|
||||
|
||||
export interface PhotonInteractionBinding {
|
||||
version: 1;
|
||||
reference: string;
|
||||
interactionId: string;
|
||||
publicationId: string;
|
||||
sessionGeneration: number;
|
||||
expiresAt: string;
|
||||
}
|
||||
export interface PhotonPromptReceipt {
|
||||
schema: 1;
|
||||
reference: string;
|
||||
questionIndex: number;
|
||||
publicationId: string;
|
||||
promptMessageGuid: string;
|
||||
promptMessageGuids?: string[];
|
||||
pollMessageGuid?: string;
|
||||
options: Record<string, string>;
|
||||
}
|
||||
export interface PhotonDraft {
|
||||
schema: 1;
|
||||
interactionId: string;
|
||||
userId: string;
|
||||
principalId: string;
|
||||
answers: AskUserQuestionsAnswer[];
|
||||
decision?: "accept" | "reject";
|
||||
reason?: string;
|
||||
lastSequence: number;
|
||||
}
|
||||
export function nativePhotonInteraction(
|
||||
interaction: IssueThreadInteraction,
|
||||
): interaction is AskUserQuestionsInteraction | RequestConfirmationInteraction {
|
||||
if (interaction.kind === "ask_user_questions")
|
||||
return (
|
||||
interaction.payload.questions.length > 0 &&
|
||||
interaction.payload.questions.length <= 64
|
||||
);
|
||||
return (
|
||||
interaction.kind === "request_confirmation" &&
|
||||
!interaction.payload.toolAction &&
|
||||
!interaction.payload.secretProposal &&
|
||||
!interaction.payload.connectionAuthorization
|
||||
);
|
||||
}
|
||||
export function photonResponseCommand(text: string): {
|
||||
command: "answer" | "submit";
|
||||
reference: string;
|
||||
questionIndex: number;
|
||||
value: string;
|
||||
} | null {
|
||||
const match =
|
||||
/^\/(answer|submit)\s+([a-zA-Z0-9_-]{8,24})(?:\.(\d{1,2}))?(?:\s+([\s\S]*))?$/i.exec(
|
||||
text.trim(),
|
||||
);
|
||||
if (!match) return null;
|
||||
return {
|
||||
command: match[1].toLowerCase() as "answer" | "submit",
|
||||
reference: match[2],
|
||||
questionIndex: Number(match[3] ?? 1) - 1,
|
||||
value: match[4]?.trim() ?? "",
|
||||
};
|
||||
}
|
||||
export function parsePhotonQuestionAnswer(
|
||||
interaction: AskUserQuestionsInteraction,
|
||||
questionIndex: number,
|
||||
value: string,
|
||||
): AskUserQuestionsAnswer {
|
||||
const question = interaction.payload.questions[questionIndex];
|
||||
if (!question)
|
||||
throw new PhotonAnswerValidationError("That question does not exist");
|
||||
if (value.toLowerCase() === "skip") {
|
||||
if (question.required !== false)
|
||||
throw new PhotonAnswerValidationError("This question requires an answer");
|
||||
return { questionId: question.id, optionIds: [] };
|
||||
}
|
||||
if (!value || value.length > 100_000)
|
||||
throw new PhotonAnswerValidationError(
|
||||
"Enter a nonempty answer within the task’s length limit",
|
||||
);
|
||||
const numbers =
|
||||
/^\d+(?:[ ,]+\d+)*$/.test(value) &&
|
||||
question.options.some((option) => !option.freeText)
|
||||
? value.split(/[ ,]+/).map(Number)
|
||||
: null;
|
||||
if (numbers) {
|
||||
if (
|
||||
new Set(numbers).size !== numbers.length ||
|
||||
numbers.some((index) => index < 1 || index > question.options.length)
|
||||
)
|
||||
throw new PhotonAnswerValidationError(
|
||||
"Choose valid option numbers without duplicates",
|
||||
);
|
||||
if (question.selectionMode === "single" && numbers.length !== 1)
|
||||
throw new PhotonAnswerValidationError("Choose one option");
|
||||
if (numbers.some((index) => question.options[index - 1].freeText))
|
||||
throw new PhotonAnswerValidationError("Write your custom answer as text");
|
||||
return {
|
||||
questionId: question.id,
|
||||
optionIds: numbers.map((index) => question.options[index - 1].id),
|
||||
};
|
||||
}
|
||||
const freeText = question.options.find((option) => option.freeText);
|
||||
if (question.options.length && question.allowOther === false && !freeText)
|
||||
throw new PhotonAnswerValidationError(
|
||||
"Answer with the option number(s) shown in the prompt",
|
||||
);
|
||||
return {
|
||||
questionId: question.id,
|
||||
optionIds: freeText ? [freeText.id] : [],
|
||||
otherText: value,
|
||||
};
|
||||
}
|
||||
export function photonAnswersMatch(
|
||||
interaction: AskUserQuestionsInteraction,
|
||||
result: unknown,
|
||||
): boolean {
|
||||
if (!result || typeof result !== "object") return false;
|
||||
const record = result as {
|
||||
code?: unknown;
|
||||
interactionId?: unknown;
|
||||
answersSha256?: unknown;
|
||||
};
|
||||
return (
|
||||
record.code === "photon_question_answered" &&
|
||||
record.interactionId === interaction.id &&
|
||||
record.answersSha256 === nativeSha256(interaction.result?.answers)
|
||||
);
|
||||
}
|
||||
|
||||
/** Prompt and poll receipts persist independently, before the outbox is settled.
|
||||
* A vote arriving before this binding is finalized must wait for this record. */
|
||||
export async function publishPhotonPrompt(input: {
|
||||
adapter: PhotonChatAdapter;
|
||||
threadId: string;
|
||||
binding: PhotonInteractionBinding;
|
||||
interaction: AskUserQuestionsInteraction | RequestConfirmationInteraction;
|
||||
questionIndex: number;
|
||||
taskUrl?: string | null;
|
||||
retryUnknown?: boolean;
|
||||
assertCurrent(): Promise<void>;
|
||||
}): Promise<PhotonPromptReceipt> {
|
||||
const { adapter, binding, interaction, questionIndex, assertCurrent } = input;
|
||||
const key = `prompt:${binding.reference}:${questionIndex}`;
|
||||
const digest = nativeSha256({
|
||||
binding,
|
||||
threadId: input.threadId,
|
||||
kind: interaction.kind,
|
||||
payload: interaction.payload,
|
||||
questionIndex,
|
||||
taskUrl: input.taskUrl ?? null,
|
||||
});
|
||||
await adapter.state.update<{ digest: string }>(
|
||||
`${key}:identity`,
|
||||
(current) => {
|
||||
if (current && current.digest !== digest)
|
||||
throw new PhotonError(
|
||||
"rejected",
|
||||
"Photon prompt changed after preparation",
|
||||
);
|
||||
return current ?? { digest };
|
||||
},
|
||||
);
|
||||
const existing = await adapter.state.read<PhotonPromptReceipt>(key);
|
||||
if (existing) {
|
||||
for (const guid of existing.promptMessageGuids ?? [
|
||||
existing.promptMessageGuid,
|
||||
])
|
||||
await adapter.state.update<PhotonPromptReceipt>(
|
||||
`prompt-message:${guid}`,
|
||||
(current) => current ?? existing,
|
||||
);
|
||||
if (existing.pollMessageGuid)
|
||||
await adapter.state.update<PhotonPromptReceipt>(
|
||||
`poll-message:${existing.pollMessageGuid}`,
|
||||
(current) => current ?? existing,
|
||||
);
|
||||
return existing;
|
||||
}
|
||||
const question =
|
||||
interaction.kind === "ask_user_questions"
|
||||
? interaction.payload.questions[questionIndex]
|
||||
: null;
|
||||
if (interaction.kind === "ask_user_questions" && !question)
|
||||
throw new PhotonAnswerValidationError("Photon question index is invalid");
|
||||
const reference = `${binding.reference}.${questionIndex + 1}`;
|
||||
const title =
|
||||
projectSafeChatPublicationText(
|
||||
question?.prompt ??
|
||||
(interaction as RequestConfirmationInteraction).payload.prompt,
|
||||
).trim() || "Input needed";
|
||||
const choices = question
|
||||
? question.options.map((option) => ({
|
||||
id: option.id,
|
||||
label:
|
||||
projectSafeChatPublicationText(option.label).trim() ||
|
||||
`Choice ${question.options.indexOf(option) + 1}`,
|
||||
}))
|
||||
: [
|
||||
{
|
||||
id: "accept",
|
||||
label: projectSafeChatPublicationText(
|
||||
(interaction as RequestConfirmationInteraction).payload
|
||||
.acceptLabel ?? "Accept",
|
||||
),
|
||||
},
|
||||
{
|
||||
id: "reject",
|
||||
label: projectSafeChatPublicationText(
|
||||
(interaction as RequestConfirmationInteraction).payload
|
||||
.rejectLabel ?? "Reject",
|
||||
),
|
||||
},
|
||||
];
|
||||
const nativePoll =
|
||||
(!question ||
|
||||
(question.selectionMode === "single" &&
|
||||
!question.allowOther &&
|
||||
!question.options.some((o) => o.freeText))) &&
|
||||
choices.length >= 2 &&
|
||||
choices.length <= 10;
|
||||
const text = [
|
||||
title,
|
||||
interaction.kind === "request_confirmation" &&
|
||||
interaction.payload.detailsMarkdown
|
||||
? projectSafeChatPublicationText(interaction.payload.detailsMarkdown)
|
||||
: "",
|
||||
question?.helpText ? projectSafeChatPublicationText(question.helpText) : "",
|
||||
...choices.map((choice, index) => `${index + 1}. ${choice.label}`),
|
||||
`Reply to this message, or send /answer ${reference} ${question?.selectionMode === "multi" ? "<numbers separated by commas>" : "<answer>"}.`,
|
||||
question?.required === false ? `Optional: /answer ${reference} skip` : "",
|
||||
interaction.kind === "request_confirmation" &&
|
||||
interaction.payload.rejectRequiresReason
|
||||
? "To reject, send Reject followed by your reason."
|
||||
: "",
|
||||
interaction.kind === "ask_user_questions" &&
|
||||
interaction.payload.questions.length > 1
|
||||
? `Answers are saved for you. Finish with /submit ${binding.reference}.`
|
||||
: "",
|
||||
input.taskUrl ? `Open this Paperclip task: ${input.taskUrl}` : "",
|
||||
]
|
||||
.filter(Boolean)
|
||||
.join("\n\n");
|
||||
const prompt = await adapter.publish(
|
||||
input.threadId,
|
||||
`${binding.publicationId}:question:${questionIndex}`,
|
||||
{ markdown: text },
|
||||
{ assertCurrent, retryUnknown: input.retryUnknown },
|
||||
);
|
||||
const receipt: PhotonPromptReceipt = {
|
||||
schema: 1,
|
||||
reference: binding.reference,
|
||||
publicationId: binding.publicationId,
|
||||
questionIndex,
|
||||
promptMessageGuid: prompt.id,
|
||||
promptMessageGuids: prompt.messageIds,
|
||||
options: {},
|
||||
};
|
||||
if (nativePoll) {
|
||||
const stateKey = `poll:${binding.publicationId}:${questionIndex}`;
|
||||
type PollSend = {
|
||||
schema: 1;
|
||||
phase: "prepared" | "creating" | "created";
|
||||
receipt?: {
|
||||
pollMessageGuid: string;
|
||||
options: Array<{ optionIdentifier: string; text: string }>;
|
||||
};
|
||||
};
|
||||
let saved = await adapter.state.read<PollSend>(stateKey);
|
||||
if (saved?.phase === "creating" && !input.retryUnknown)
|
||||
throw new PhotonError(
|
||||
"delivery_unknown",
|
||||
"Photon poll creation is unknown; reconcile this publication before retrying",
|
||||
);
|
||||
if (!saved || saved.phase !== "created") {
|
||||
await assertCurrent();
|
||||
await adapter.state.update<PollSend>(stateKey, (current) => {
|
||||
if (current?.phase === "creating" && !input.retryUnknown)
|
||||
throw new PhotonAnswerValidationError("Photon poll already claimed");
|
||||
return { schema: 1, phase: "creating" };
|
||||
});
|
||||
const poll = await adapter.client.polls
|
||||
.create(
|
||||
adapter.decodeThreadId(input.threadId).chatGuid,
|
||||
title,
|
||||
choices.map((choice) => choice.label),
|
||||
{
|
||||
clientMessageId: createHash("sha256")
|
||||
.update(`${adapter.state.scope.endpointId}:${stateKey}`)
|
||||
.digest("hex"),
|
||||
},
|
||||
)
|
||||
.catch(async (error) => {
|
||||
const failure = photonFailure(error, true);
|
||||
if (failure.code !== "delivery_unknown")
|
||||
await adapter.state.update<PollSend>(stateKey, () => ({
|
||||
schema: 1,
|
||||
phase: "prepared",
|
||||
}));
|
||||
throw failure;
|
||||
});
|
||||
if (
|
||||
!poll.pollMessageGuid ||
|
||||
poll.options.length !== choices.length ||
|
||||
new Set(poll.options.map((option) => option.optionIdentifier)).size !==
|
||||
choices.length ||
|
||||
poll.options.some(
|
||||
(option, index) =>
|
||||
!option.optionIdentifier ||
|
||||
option.text.trim() !== choices[index].label.trim(),
|
||||
)
|
||||
)
|
||||
throw new PhotonError(
|
||||
"delivery_unknown",
|
||||
"Photon returned an incomplete poll receipt",
|
||||
);
|
||||
saved = await adapter.state.update<PollSend>(stateKey, () => ({
|
||||
schema: 1,
|
||||
phase: "created",
|
||||
receipt: {
|
||||
pollMessageGuid: poll.pollMessageGuid,
|
||||
options: poll.options.map((option) => ({
|
||||
optionIdentifier: option.optionIdentifier,
|
||||
text: option.text,
|
||||
})),
|
||||
},
|
||||
}));
|
||||
}
|
||||
receipt.pollMessageGuid = saved.receipt!.pollMessageGuid;
|
||||
// Creation response preserves requested option order; labels/titles are never a lookup key.
|
||||
receipt.options = Object.fromEntries(
|
||||
saved.receipt!.options.map((option, index) => [
|
||||
option.optionIdentifier,
|
||||
choices[index].id,
|
||||
]),
|
||||
);
|
||||
}
|
||||
await adapter.state.update<PhotonPromptReceipt>(
|
||||
key,
|
||||
(current) => current ?? receipt,
|
||||
);
|
||||
for (const guid of prompt.messageIds)
|
||||
await adapter.state.update<PhotonPromptReceipt>(
|
||||
`prompt-message:${guid}`,
|
||||
(current) => current ?? receipt,
|
||||
);
|
||||
if (receipt.pollMessageGuid)
|
||||
await adapter.state.update<PhotonPromptReceipt>(
|
||||
`poll-message:${receipt.pollMessageGuid}`,
|
||||
(current) => current ?? receipt,
|
||||
);
|
||||
return receipt;
|
||||
}
|
||||
|
|
@ -0,0 +1,140 @@
|
|||
import { spawn } from "node:child_process";
|
||||
import { createRequire } from "node:module";
|
||||
import sharp from "sharp";
|
||||
import { MAX_ATTACHMENT_BYTES } from "../../attachment-types.js";
|
||||
|
||||
const require = createRequire(import.meta.url);
|
||||
const MAX_PIXELS = 50_000_000;
|
||||
export const HEIF_CONTENT_TYPES = new Set([
|
||||
"image/heic",
|
||||
"image/heif",
|
||||
"image/heic-sequence",
|
||||
"image/heif-sequence",
|
||||
]);
|
||||
|
||||
/** Validate bounded ISO-BMFF structure before invoking any native decoder. */
|
||||
export function validateHeifDimensions(body: Buffer): void {
|
||||
let boxes = 0;
|
||||
let dimensions = 0;
|
||||
let totalPixels = 0;
|
||||
let branded = false;
|
||||
const visit = (start: number, end: number, depth: number) => {
|
||||
if (depth > 8) throw new Error("HEIF metadata nesting is too deep");
|
||||
for (let at = start; at < end; ) {
|
||||
if (++boxes > 4096 || end - at < 8)
|
||||
throw new Error("Invalid HEIF box structure");
|
||||
let size = body.readUInt32BE(at);
|
||||
const type = body.toString("ascii", at + 4, at + 8);
|
||||
let header = 8;
|
||||
if (size === 1) {
|
||||
if (end - at < 16) throw new Error("Invalid HEIF box length");
|
||||
const extended = body.readBigUInt64BE(at + 8);
|
||||
if (extended > BigInt(body.length))
|
||||
throw new Error("HEIF box exceeds file bounds");
|
||||
size = Number(extended);
|
||||
header = 16;
|
||||
} else if (size === 0) size = end - at;
|
||||
if (size < header || at + size > end)
|
||||
throw new Error("HEIF box exceeds file bounds");
|
||||
const content = at + header;
|
||||
if (type === "ftyp") {
|
||||
if (size < header + 8) throw new Error("HEIF file type is missing");
|
||||
const brands = body.toString("ascii", content, at + size);
|
||||
branded = /heic|heix|hevc|hevx|mif1|msf1/.test(brands);
|
||||
} else if (type === "ispe") {
|
||||
if (size !== header + 12)
|
||||
throw new Error("Invalid HEIF image dimensions");
|
||||
const width = body.readUInt32BE(content + 4);
|
||||
const height = body.readUInt32BE(content + 8);
|
||||
if (
|
||||
!width ||
|
||||
!height ||
|
||||
width > 16_384 ||
|
||||
height > 16_384 ||
|
||||
width * height > MAX_PIXELS
|
||||
)
|
||||
throw new Error("HEIF decoded image exceeds the pixel limit");
|
||||
totalPixels += width * height;
|
||||
if (totalPixels > MAX_PIXELS * 3 || ++dimensions > 512)
|
||||
throw new Error("HEIF image collection exceeds the pixel limit");
|
||||
} else if (["meta", "iprp", "ipco"].includes(type)) {
|
||||
visit(content + (type === "meta" ? 4 : 0), at + size, depth + 1);
|
||||
}
|
||||
at += size;
|
||||
}
|
||||
};
|
||||
if (!body.length || body.length > MAX_ATTACHMENT_BYTES)
|
||||
throw new Error("HEIF exceeds the attachment byte limit");
|
||||
visit(0, body.length, 0);
|
||||
if (!branded || !dimensions)
|
||||
throw new Error("HEIF dimensions could not be verified");
|
||||
}
|
||||
|
||||
export async function validatePhotonImage(
|
||||
body: Buffer,
|
||||
contentType: string,
|
||||
): Promise<void> {
|
||||
if (!contentType.startsWith("image/")) return;
|
||||
if (HEIF_CONTENT_TYPES.has(contentType)) return validateHeifDimensions(body);
|
||||
const metadata = await sharp(body, {
|
||||
limitInputPixels: MAX_PIXELS,
|
||||
failOn: "error",
|
||||
}).metadata();
|
||||
if (
|
||||
!metadata.width ||
|
||||
!metadata.height ||
|
||||
metadata.width * metadata.height * (metadata.pages ?? 1) > MAX_PIXELS
|
||||
)
|
||||
throw new Error("Decoded image exceeds the pixel limit");
|
||||
const formats: Record<string, string[]> = {
|
||||
"image/jpeg": ["jpeg"],
|
||||
"image/jpg": ["jpeg"],
|
||||
"image/png": ["png"],
|
||||
"image/webp": ["webp"],
|
||||
"image/gif": ["gif"],
|
||||
};
|
||||
if (!formats[contentType]?.includes(metadata.format ?? ""))
|
||||
throw new Error("Image bytes do not match the declared content type");
|
||||
}
|
||||
|
||||
/** Isolate the native converter with a deadline and bounded input/output.
|
||||
* Native packages exist for macOS, Windows and Linux glibc x64/arm64.
|
||||
* Unsupported hosts retain the original and report an unavailable preview. */
|
||||
export async function photonHeifPreview(body: Buffer): Promise<Buffer> {
|
||||
validateHeifDimensions(body);
|
||||
const modulePath = require.resolve("heif2jpeg");
|
||||
const script = `const {heifToJpeg}=require(process.argv[1]);const chunks=[];process.stdin.on('data',c=>chunks.push(c));process.stdin.on('end',async()=>{try{const jpeg=await heifToJpeg(Buffer.concat(chunks),{quality:80});process.stdout.end(jpeg);}catch{process.exitCode=1;}});`;
|
||||
const jpeg = await new Promise<Buffer>((resolve, reject) => {
|
||||
const child = spawn(
|
||||
process.execPath,
|
||||
["--max-old-space-size=256", "--eval", script, modulePath],
|
||||
{ stdio: ["pipe", "pipe", "ignore"], windowsHide: true },
|
||||
);
|
||||
let length = 0;
|
||||
const chunks: Buffer[] = [];
|
||||
const timer = setTimeout(() => {
|
||||
child.kill("SIGKILL");
|
||||
reject(new Error("HEIF preview conversion timed out"));
|
||||
}, 10_000);
|
||||
child.stdout.on("data", (chunk: Buffer) => {
|
||||
length += chunk.length;
|
||||
if (length > MAX_ATTACHMENT_BYTES) {
|
||||
child.kill("SIGKILL");
|
||||
reject(new Error("HEIF preview exceeds the attachment byte limit"));
|
||||
} else chunks.push(chunk);
|
||||
});
|
||||
child.on("error", () => {
|
||||
clearTimeout(timer);
|
||||
reject(new Error("HEIF preview converter is unavailable"));
|
||||
});
|
||||
child.on("close", (code) => {
|
||||
clearTimeout(timer);
|
||||
if (code === 0 && length > 0) resolve(Buffer.concat(chunks));
|
||||
else reject(new Error("HEIF preview conversion failed on this host"));
|
||||
});
|
||||
child.stdin.on("error", () => {});
|
||||
child.stdin.end(body);
|
||||
});
|
||||
await validatePhotonImage(jpeg, "image/jpeg");
|
||||
return jpeg;
|
||||
}
|
||||
|
|
@ -0,0 +1,217 @@
|
|||
import type { PhotonRecoveryEvent } from "./recovery-transport.js";
|
||||
import type {
|
||||
GrpcAdvancedIMessage,
|
||||
LiveEvent,
|
||||
TypedEventStream,
|
||||
} from "@photon-ai/advanced-imessage";
|
||||
import { PhotonError } from "./cloud.js";
|
||||
import { PhotonState } from "./state.js";
|
||||
|
||||
interface Checkpoint {
|
||||
schema: 1;
|
||||
lineId: string;
|
||||
sequence: number;
|
||||
}
|
||||
export interface PhotonReceiverOptions {
|
||||
client: GrpcAdvancedIMessage;
|
||||
state: PhotonState;
|
||||
lineId: string;
|
||||
intakeAfter: number;
|
||||
allocation?: "dedicated" | "shared";
|
||||
catchUp?(sequence?: number): TypedEventStream<PhotonRecoveryEvent>;
|
||||
/** Renewal verifies both the selected identity and the current endpoint lease. */
|
||||
assertOwned(): Promise<void>;
|
||||
admit(event: LiveEvent): Promise<void>;
|
||||
failure(error: unknown): Promise<void>;
|
||||
/** Production persists the cursor under the same transaction as its lease fence. */
|
||||
commitCheckpoint?(sequence: number): Promise<void>;
|
||||
}
|
||||
|
||||
/** Live streams wake a serialized catch-up reader. Only that complete event log
|
||||
* advances the checkpoint, so cross-stream ordering can never skip an event. */
|
||||
export class PhotonReceiver {
|
||||
private stopped = false;
|
||||
private streams: Array<TypedEventStream<LiveEvent>> = [];
|
||||
private running?: Promise<void>;
|
||||
private requested = false;
|
||||
private catchUpStream?: TypedEventStream<PhotonRecoveryEvent>;
|
||||
private timer?: ReturnType<typeof setInterval>;
|
||||
constructor(private readonly options: PhotonReceiverOptions) {}
|
||||
start(): void {
|
||||
if (this.stopped || this.timer) return;
|
||||
const { client } = this.options;
|
||||
this.streams = [
|
||||
client.messages.subscribeEvents(),
|
||||
client.chats.subscribeEvents(),
|
||||
...(this.options.allocation === "shared" ? [] : [client.groups.subscribeEvents()]),
|
||||
client.polls.subscribeEvents(),
|
||||
];
|
||||
for (const stream of this.streams) {
|
||||
void (async () => {
|
||||
try {
|
||||
for await (const _event of stream) {
|
||||
if (this.stopped) break;
|
||||
this.request();
|
||||
}
|
||||
if (!this.stopped)
|
||||
await this.options.failure(
|
||||
new PhotonError("network", "Photon event stream disconnected"),
|
||||
);
|
||||
} catch (error) {
|
||||
if (!this.stopped) await this.options.failure(error);
|
||||
}
|
||||
})().catch(() => {});
|
||||
}
|
||||
this.timer = setInterval(() => this.request(), 15_000);
|
||||
this.timer.unref();
|
||||
this.request();
|
||||
}
|
||||
private request(): void {
|
||||
if (this.stopped) return;
|
||||
this.requested = true;
|
||||
if (this.running) return;
|
||||
this.running = (async () => {
|
||||
while (this.requested && !this.stopped) {
|
||||
this.requested = false;
|
||||
await this.catchUp();
|
||||
}
|
||||
})()
|
||||
.catch(async (error) => {
|
||||
if (!this.stopped) await this.options.failure(error);
|
||||
})
|
||||
.finally(() => {
|
||||
this.running = undefined;
|
||||
});
|
||||
}
|
||||
/** Exposed for deterministic recovery tests, never an external route. */
|
||||
async catchUp(): Promise<void> {
|
||||
const { state, lineId, client, assertOwned, admit, intakeAfter } =
|
||||
this.options;
|
||||
await assertOwned();
|
||||
const original = await state.read<Checkpoint>("checkpoint");
|
||||
if (!original && (await state.read("receiver-initialized")))
|
||||
throw new PhotonError(
|
||||
"history_gap",
|
||||
"Photon receiver checkpoint is missing; operator recovery is required",
|
||||
);
|
||||
if (
|
||||
original &&
|
||||
(original.schema !== 1 ||
|
||||
original.lineId !== lineId ||
|
||||
!Number.isSafeInteger(original.sequence) ||
|
||||
original.sequence < 0)
|
||||
)
|
||||
throw new PhotonError(
|
||||
"history_gap",
|
||||
"Photon checkpoint is invalid; operator recovery is required",
|
||||
);
|
||||
const shared = this.options.allocation === "shared";
|
||||
let sequence = original?.sequence;
|
||||
let batchEvents = 0;
|
||||
const stream =
|
||||
this.options.catchUp?.(sequence) ?? client.events.catchUp(sequence);
|
||||
this.catchUpStream = stream;
|
||||
let completed = false;
|
||||
try {
|
||||
for await (const event of stream) {
|
||||
if (this.stopped) return;
|
||||
await assertOwned();
|
||||
if (event.type === "catchup.complete") {
|
||||
if (
|
||||
!Number.isSafeInteger(event.headSequence) ||
|
||||
event.headSequence < 0 ||
|
||||
(sequence !== undefined && (shared ? event.headSequence < sequence : event.headSequence !== sequence))
|
||||
)
|
||||
throw new PhotonError(
|
||||
"history_gap",
|
||||
"Photon history has a gap or reset; operator recovery is required",
|
||||
);
|
||||
// Shared gateway replay is project-filtered: sequence numbers are
|
||||
// increasing but not adjacent (the first live project event may be
|
||||
// > 1 billion). A complete replay barrier covers the filtered tail.
|
||||
// Commit shared batches only here, after every admission succeeds;
|
||||
// malformed ordering or interrupted replay keeps the previous cursor.
|
||||
sequence = shared ? event.headSequence : sequence ?? event.headSequence;
|
||||
await this.checkpoint(sequence);
|
||||
completed = true;
|
||||
break;
|
||||
}
|
||||
if (!Number.isSafeInteger(event.sequence) || event.sequence < 1)
|
||||
throw new PhotonError(
|
||||
"history_gap",
|
||||
"Photon returned an invalid event sequence",
|
||||
);
|
||||
if (++batchEvents > 100_000)
|
||||
throw new PhotonError("history_gap", "Photon replay exceeds the supported recovery window");
|
||||
if (shared && sequence !== undefined && event.sequence < sequence && event.sequence > (original?.sequence ?? -1))
|
||||
throw new PhotonError("history_gap", "Photon replay arrived out of order; the saved cursor was retained");
|
||||
if (sequence !== undefined && event.sequence <= sequence) continue;
|
||||
if (!shared && sequence !== undefined && event.sequence !== sequence + 1)
|
||||
throw new PhotonError(
|
||||
"history_gap",
|
||||
"Photon event history is incomplete; operator recovery is required",
|
||||
);
|
||||
// On the first connection, the server may retain only a tail of history.
|
||||
// Establish that boundary explicitly; it is never allowed after a cursor.
|
||||
sequence ??= event.sequence - 1;
|
||||
if (event.type !== "photon.ignored") {
|
||||
const occurredAt = new Date(event.occurredAt).getTime();
|
||||
if (!Number.isFinite(occurredAt))
|
||||
throw new PhotonError(
|
||||
"invalid_response",
|
||||
"Photon event timestamp is invalid",
|
||||
);
|
||||
if (occurredAt >= intakeAfter) await admit(event);
|
||||
}
|
||||
// admission must durably store or classify even irrelevant events.
|
||||
if (!shared) await this.checkpoint(event.sequence);
|
||||
sequence = event.sequence;
|
||||
}
|
||||
if (!completed && !this.stopped)
|
||||
throw new PhotonError(
|
||||
"network",
|
||||
"Photon catch-up ended before its checkpoint barrier",
|
||||
);
|
||||
} finally {
|
||||
await stream.close();
|
||||
if (this.catchUpStream === stream) this.catchUpStream = undefined;
|
||||
}
|
||||
}
|
||||
private async checkpoint(sequence: number): Promise<void> {
|
||||
await this.options.assertOwned();
|
||||
if (this.options.commitCheckpoint)
|
||||
return this.options.commitCheckpoint(sequence);
|
||||
await writePhotonCheckpoint(
|
||||
this.options.state,
|
||||
this.options.lineId,
|
||||
sequence,
|
||||
);
|
||||
}
|
||||
|
||||
async close(): Promise<void> {
|
||||
this.stopped = true;
|
||||
if (this.timer) clearInterval(this.timer);
|
||||
await this.catchUpStream?.close();
|
||||
await Promise.allSettled(this.streams.map((stream) => stream.close()));
|
||||
// client.close interrupts a blocked catch-up RPC during runtime shutdown.
|
||||
}
|
||||
}
|
||||
|
||||
export async function writePhotonCheckpoint(
|
||||
state: PhotonState,
|
||||
lineId: string,
|
||||
sequence: number,
|
||||
): Promise<void> {
|
||||
await state.update<Checkpoint>("checkpoint", (current) => {
|
||||
if (current && (current.lineId !== lineId || current.sequence > sequence))
|
||||
throw new PhotonError(
|
||||
"history_gap",
|
||||
"Photon checkpoint ownership changed",
|
||||
);
|
||||
return { schema: 1, lineId, sequence };
|
||||
});
|
||||
await state.update(
|
||||
"receiver-initialized",
|
||||
(current) => current ?? { schema: 1, lineId },
|
||||
);
|
||||
}
|
||||
|
|
@ -0,0 +1,178 @@
|
|||
import { Client, credentials, Metadata, status } from "@grpc/grpc-js";
|
||||
import {
|
||||
decodeCatchUpEvent,
|
||||
TypedEventStream,
|
||||
type CatchUpEvent,
|
||||
} from "@photon-ai/advanced-imessage";
|
||||
import {
|
||||
PhotonError,
|
||||
photonFailure,
|
||||
type PhotonLineAuthentication,
|
||||
} from "./cloud.js";
|
||||
import { MAX_ATTACHMENT_BYTES } from "../../attachment-types.js";
|
||||
|
||||
export type PhotonRecoveryEvent =
|
||||
| CatchUpEvent
|
||||
| { type: "photon.ignored"; sequence: number };
|
||||
export const PHOTON_CATCHUP_PATH =
|
||||
"/photon.imessage.v1.EventService/CatchUpEvents";
|
||||
|
||||
/** Pinned v2.1.0 protobuf envelope. The SDK decoder owns the event graph; this
|
||||
* reader retains sequence-only/new-variant frames that its public API drops. */
|
||||
export function photonEnvelopeSequence(bytes: Uint8Array): number | undefined {
|
||||
let offset = 0;
|
||||
const integer = () => {
|
||||
let value = 0n;
|
||||
for (let shift = 0n; shift < 70n; shift += 7n) {
|
||||
if (offset >= bytes.length)
|
||||
throw new PhotonError(
|
||||
"invalid_response",
|
||||
"Truncated Photon recovery frame",
|
||||
);
|
||||
const byte = bytes[offset++];
|
||||
value |= BigInt(byte & 127) << shift;
|
||||
if (!(byte & 128)) {
|
||||
if (value > BigInt(Number.MAX_SAFE_INTEGER))
|
||||
throw new PhotonError(
|
||||
"history_gap",
|
||||
"Photon recovery sequence exceeds the supported range",
|
||||
);
|
||||
return Number(value);
|
||||
}
|
||||
}
|
||||
throw new PhotonError(
|
||||
"invalid_response",
|
||||
"Invalid Photon recovery integer",
|
||||
);
|
||||
};
|
||||
let sequence: number | undefined;
|
||||
while (offset < bytes.length) {
|
||||
const tag = integer();
|
||||
if (tag === 8) {
|
||||
if (sequence !== undefined)
|
||||
throw new PhotonError(
|
||||
"invalid_response",
|
||||
"Duplicate Photon sequence field",
|
||||
);
|
||||
sequence = integer();
|
||||
continue;
|
||||
}
|
||||
const wire = tag & 7;
|
||||
if (wire === 0) integer();
|
||||
else if (wire === 1) offset += 8;
|
||||
else if (wire === 2) {
|
||||
const length = integer();
|
||||
offset += length;
|
||||
} else if (wire === 5) offset += 4;
|
||||
else
|
||||
throw new PhotonError(
|
||||
"invalid_response",
|
||||
"Unsupported Photon recovery frame",
|
||||
);
|
||||
if (offset > bytes.length)
|
||||
throw new PhotonError(
|
||||
"invalid_response",
|
||||
"Photon recovery field exceeds frame bounds",
|
||||
);
|
||||
}
|
||||
return sequence;
|
||||
}
|
||||
export function photonCatchUpRequest(sequence?: number): Buffer {
|
||||
if (sequence === undefined) return Buffer.alloc(0);
|
||||
if (!Number.isSafeInteger(sequence) || sequence < 0)
|
||||
throw new PhotonError("history_gap", "Invalid Photon recovery cursor");
|
||||
let remaining = BigInt(sequence);
|
||||
const bytes = [8];
|
||||
do {
|
||||
const byte = Number(remaining & 127n);
|
||||
remaining >>= 7n;
|
||||
bytes.push(byte | (remaining ? 128 : 0));
|
||||
} while (remaining);
|
||||
return Buffer.from(bytes);
|
||||
}
|
||||
export function decodePhotonRecoveryFrame(
|
||||
bytes: Buffer,
|
||||
): PhotonRecoveryEvent | null {
|
||||
const sequence = photonEnvelopeSequence(bytes);
|
||||
let event: CatchUpEvent | undefined;
|
||||
try {
|
||||
event = decodeCatchUpEvent(bytes);
|
||||
} catch {
|
||||
throw new PhotonError(
|
||||
"invalid_response",
|
||||
"Photon recovery event could not be decoded",
|
||||
);
|
||||
}
|
||||
return (
|
||||
event ??
|
||||
(sequence !== undefined ? { type: "photon.ignored", sequence } : null)
|
||||
);
|
||||
}
|
||||
|
||||
export class PhotonRecoveryTransport {
|
||||
private readonly client: Client;
|
||||
constructor(
|
||||
private readonly authentication: PhotonLineAuthentication,
|
||||
client?: Client,
|
||||
) {
|
||||
this.client =
|
||||
client ??
|
||||
new Client(authentication.address, credentials.createSsl(), {
|
||||
"grpc.max_receive_message_length": MAX_ATTACHMENT_BYTES + 1024 * 1024,
|
||||
});
|
||||
}
|
||||
catchUp(sequence?: number): TypedEventStream<PhotonRecoveryEvent> {
|
||||
const controller = new AbortController();
|
||||
const { client, authentication } = this;
|
||||
async function* receive() {
|
||||
const metadata = new Metadata();
|
||||
metadata.set("authorization", `Bearer ${await authentication.token()}`);
|
||||
if (controller.signal.aborted) return;
|
||||
const call = client.makeServerStreamRequest(
|
||||
PHOTON_CATCHUP_PATH,
|
||||
photonCatchUpRequest,
|
||||
(bytes: Buffer) => bytes,
|
||||
sequence,
|
||||
metadata,
|
||||
{ deadline: Date.now() + 60_000 },
|
||||
);
|
||||
const abort = () => call.cancel();
|
||||
controller.signal.addEventListener("abort", abort, { once: true });
|
||||
try {
|
||||
for await (const bytes of call) {
|
||||
const frame = decodePhotonRecoveryFrame(bytes as Buffer);
|
||||
if (frame) yield frame;
|
||||
}
|
||||
} catch (error) {
|
||||
if (controller.signal.aborted) return;
|
||||
const code = (error as { code?: number }).code;
|
||||
if (code === status.OUT_OF_RANGE || code === status.FAILED_PRECONDITION)
|
||||
throw new PhotonError(
|
||||
"history_gap",
|
||||
"Photon cannot recover the saved cursor; reconnect after reviewing the history gap",
|
||||
);
|
||||
if (
|
||||
code === status.UNAUTHENTICATED ||
|
||||
code === status.PERMISSION_DENIED
|
||||
)
|
||||
throw new PhotonError(
|
||||
"credentials",
|
||||
"Photon rejected the selected line credentials; reconnect the channel",
|
||||
);
|
||||
if (code === status.RESOURCE_EXHAUSTED)
|
||||
throw new PhotonError(
|
||||
"quota",
|
||||
"Photon recovery is temporarily rate limited",
|
||||
);
|
||||
throw photonFailure(error);
|
||||
} finally {
|
||||
controller.signal.removeEventListener("abort", abort);
|
||||
call.cancel();
|
||||
}
|
||||
}
|
||||
return new TypedEventStream(receive(), async () => controller.abort());
|
||||
}
|
||||
close(): void {
|
||||
this.client.close();
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,39 @@
|
|||
import { createHash } from "node:crypto";
|
||||
import type {
|
||||
ChatSdkStatePersistence,
|
||||
ChatSdkStateScope,
|
||||
} from "../chat-sdk-state.js";
|
||||
|
||||
/** Typed provider records share the existing company/endpoint scoped CAS store. */
|
||||
export class PhotonState {
|
||||
constructor(
|
||||
readonly scope: ChatSdkStateScope,
|
||||
private readonly persistence: ChatSdkStatePersistence,
|
||||
) {}
|
||||
private key(key: string): string {
|
||||
return `photon:${createHash("sha256").update(key).digest("hex")}`;
|
||||
}
|
||||
async read<T>(key: string): Promise<T | null> {
|
||||
const row = await this.persistence.read(this.scope, this.key(key));
|
||||
return row ? (row.value as T) : null;
|
||||
}
|
||||
async update<T>(key: string, update: (current: T | null) => T): Promise<T> {
|
||||
for (let attempt = 0; attempt < 32; attempt++) {
|
||||
const row = await this.persistence.read(this.scope, this.key(key));
|
||||
const value = update(row ? (row.value as T) : null);
|
||||
if (Buffer.byteLength(JSON.stringify(value)) > 512 * 1024)
|
||||
throw new Error("Photon state record is too large");
|
||||
if (
|
||||
await this.persistence.compareAndSet({
|
||||
...this.scope,
|
||||
key: this.key(key),
|
||||
expectedVersion: row?.version ?? null,
|
||||
expiresAt: null,
|
||||
value,
|
||||
})
|
||||
)
|
||||
return value;
|
||||
}
|
||||
throw new Error("Photon state changed concurrently; retry");
|
||||
}
|
||||
}
|
||||
|
|
@ -17,7 +17,7 @@ import {
|
|||
* because those checks require real accounts and publicly reachable ingress.
|
||||
*/
|
||||
|
||||
type Provider = "slack" | "github" | "discord" | "microsoft-teams" | "telegram";
|
||||
type Provider = "slack" | "github" | "discord" | "microsoft-teams" | "telegram" | "imessage-photon";
|
||||
|
||||
const GITHUB_PRIVATE_KEY_FIXTURE =
|
||||
"-----BEGIN PRIVATE KEY-----\nlocal-e2e-key\n-----END PRIVATE KEY-----\n";
|
||||
|
|
@ -125,6 +125,7 @@ const PROVIDER_LIFECYCLE_COPY: Record<
|
|||
Provider,
|
||||
{ reconnect: string; remove: string }
|
||||
> = {
|
||||
"imessage-photon": { reconnect: "Reconnect the same dedicated Photon number", remove: "does not delete the Photon project" },
|
||||
slack: {
|
||||
reconnect:
|
||||
"Reconnect verifies or replaces credentials for this same Slack app. It does not reinstall the app or change its workspace or channel membership.",
|
||||
|
|
@ -335,7 +336,8 @@ async function installChatControlPlaneMock(
|
|||
{
|
||||
enableChatConnectors,
|
||||
resourceCount = 2,
|
||||
}: { enableChatConnectors: boolean; resourceCount?: number },
|
||||
photonShared = false,
|
||||
}: { enableChatConnectors: boolean; resourceCount?: number; photonShared?: boolean },
|
||||
): Promise<ChatMock> {
|
||||
const endpoint = endpointFixture(provider, seed);
|
||||
const state: ChatMock & {
|
||||
|
|
@ -543,12 +545,16 @@ async function installChatControlPlaneMock(
|
|||
} else {
|
||||
expect(provider.provider).toBe("slack");
|
||||
}
|
||||
if (provider.provider === "imessage-photon") {
|
||||
expect(body.photon).toEqual(photonShared ? {allocation:"shared",projectId:"project-e2e"} : {allocation:"dedicated",projectId:"project-e2e",lineId:"line-one"});
|
||||
}
|
||||
Object.assign(endpoint, {
|
||||
...(provider.provider === "imessage-photon" ? {photonAllocation: photonShared ? "shared" : "dedicated", allowGroupChats: !photonShared} : {}),
|
||||
status: "verifying",
|
||||
providerAccountId: `account-${provider.provider}`,
|
||||
providerAccountLabel: provider.accountLabel,
|
||||
botExternalId: `bot-${provider.provider}`,
|
||||
botUsername: provider.botUsername,
|
||||
botExternalId: provider.provider === "imessage-photon" ? photonShared ? "photon-project:project-e2e" : "+15555550100" : `bot-${provider.provider}`,
|
||||
botUsername: photonShared ? null : provider.botUsername,
|
||||
botLabel: provider.botLabel,
|
||||
setup: {
|
||||
...endpoint.setup,
|
||||
|
|
@ -562,6 +568,10 @@ async function installChatControlPlaneMock(
|
|||
return;
|
||||
}
|
||||
|
||||
if (pathname === `/api/chat-endpoints/${endpoint.id}/photon/inspect` && method === "POST") {
|
||||
expect(bodyOf(route)).toEqual({projectId:"project-e2e",projectSecret:"photon-test-secret"});
|
||||
await fulfill(route,{projectId:"project-e2e",projectName:"Photon Test",allocation:photonShared ? "shared" : "dedicated",eligible:true,lines:photonShared ? [] : [{lineId:"line-one",phoneNumber:"+15555550100",eligible:true},{lineId:"line-two",phoneNumber:"+15555550102",eligible:true}]}); return;
|
||||
}
|
||||
if (
|
||||
pathname === `/api/chat-endpoints/${endpoint.id}/test` &&
|
||||
method === "POST"
|
||||
|
|
@ -681,7 +691,7 @@ async function installChatControlPlaneMock(
|
|||
externalConversationId: `external-conversation-${provider.provider}`,
|
||||
externalThreadId: `external-thread-${provider.provider}`,
|
||||
externalLabel: provider.resourceLabel,
|
||||
externalUrl: provider.externalUrl,
|
||||
externalUrl: provider.provider === "imessage-photon" ? null : provider.externalUrl,
|
||||
isDirectMessage: provider.provider === "telegram",
|
||||
state: state.conversationState,
|
||||
lastPublicationStatus: "published",
|
||||
|
|
@ -1531,7 +1541,7 @@ test.describe.serial("native chat adapter UI", () => {
|
|||
).toBeVisible();
|
||||
await expectSetupRail(page);
|
||||
await selectMaya(page);
|
||||
expect(mock.createdWithAgentId).toBe(seed.agentId);
|
||||
await expect.poll(() => mock.createdWithAgentId).toBe(seed.agentId);
|
||||
expect(mock.createdWithAgentId).not.toBe(seed.otherAgentId);
|
||||
await expect(
|
||||
page.getByRole("button", { name: "Choose an active agent" }),
|
||||
|
|
@ -3340,3 +3350,136 @@ test.describe("Exact failed chat run retry", () => {
|
|||
}
|
||||
}
|
||||
});
|
||||
|
||||
|
||||
test.describe("iMessage Photon setup and management", () => {
|
||||
let seed: Seed;
|
||||
const photon: ProviderCase = {
|
||||
provider: "imessage-photon",
|
||||
slug: "imessage-photon",
|
||||
name: "iMessage Photon",
|
||||
accountLabel: "Photon Test",
|
||||
botLabel: "Maya",
|
||||
botUsername: "+15555550100",
|
||||
resourceLabel: "Family project",
|
||||
secondaryResourceLabel: "Second group",
|
||||
resourceType: "group_chat",
|
||||
externalUrl: "https://app.photon.codes/",
|
||||
setupHeading: /Connect iMessage Photon/,
|
||||
setupButton: "Connect selected number",
|
||||
chatAndTool: false,
|
||||
};
|
||||
test.beforeAll(async ({ request }) => {
|
||||
seed = await seedCompanyAndAgent(request);
|
||||
});
|
||||
test("connects Pro shared DMs without presenting a fake owned number or enabling groups", async ({page}) => {
|
||||
await installChatControlPlaneMock(page, photon, seed, { enableChatConnectors: true, photonShared: true });
|
||||
await page.goto(`/${seed.prefix}/apps`);
|
||||
await page.getByRole("button", {name:"Connect iMessage Photon",exact:true}).click();
|
||||
await selectMaya(page);
|
||||
await page.getByLabel("Project ID").fill("project-e2e");
|
||||
await page.getByLabel("Project secret").fill("photon-test-secret");
|
||||
await page.getByRole("button", {name:"Inspect Photon project"}).click();
|
||||
await expect(page.getByText(/Groups cannot be enabled on this channel/)).toBeVisible();
|
||||
await page.getByRole("button", {name:"Connect shared DMs"}).click();
|
||||
await expect(page.getByRole("heading", {name:"Try Maya in iMessage Photon"})).toBeVisible();
|
||||
await page.reload();
|
||||
await expect(page.getByText(/enroll your sender in Users/)).toBeVisible();
|
||||
await expect(page.getByRole("button", {name:/Copy \+1555/})).toHaveCount(0);
|
||||
await page.getByRole("button", {name:"I've sent the test message"}).click();
|
||||
await page.getByRole("tab", {name:"Settings"}).click();
|
||||
await expect(page.getByText(/Shared Photon project · direct messages only/)).toBeVisible();
|
||||
await expect(page.getByRole("switch", {name:"Enable Family project"})).toBeDisabled();
|
||||
await expect(page.getByRole("button", {name:"Copy dedicated number"})).toHaveCount(0);
|
||||
});
|
||||
for (const theme of ["light", "dark"] as const) {
|
||||
test(`discovers dedicated lines and completes the channel wizard (${theme})`, async ({
|
||||
page,
|
||||
}, testInfo) => {
|
||||
const mock = await installChatControlPlaneMock(page, photon, seed, {
|
||||
enableChatConnectors: true,
|
||||
});
|
||||
await page.addInitScript(
|
||||
(value) => localStorage.setItem("paperclip.theme", value),
|
||||
theme,
|
||||
);
|
||||
await page.goto(`/${seed.prefix}/apps`);
|
||||
const card = page.locator(
|
||||
'[role="listitem"][data-app-slug="imessage-photon"]',
|
||||
);
|
||||
await expect(card).toBeVisible();
|
||||
await card
|
||||
.getByRole("button", { name: "Connect iMessage Photon" })
|
||||
.click();
|
||||
await selectMaya(page);
|
||||
await expectSetupRail(page);
|
||||
await expect(
|
||||
page.getByRole("heading", { name: "Connect iMessage Photon" }),
|
||||
).toBeVisible();
|
||||
await page.getByLabel("Project ID").fill("project-e2e");
|
||||
await page.getByLabel("Project secret").fill("photon-test-secret");
|
||||
await expect(page.getByLabel("Project secret")).toHaveAttribute(
|
||||
"type",
|
||||
"password",
|
||||
);
|
||||
await page
|
||||
.getByRole("button", { name: "Inspect Photon project" })
|
||||
.click();
|
||||
await expect(
|
||||
page.getByRole("button", { name: "Connect selected number" }),
|
||||
).toBeDisabled();
|
||||
await page
|
||||
.getByRole("radio", { name: "+15555550100", exact: true })
|
||||
.focus();
|
||||
await page.keyboard.press("Space");
|
||||
await page
|
||||
.getByRole("button", { name: "Connect selected number" })
|
||||
.click();
|
||||
expect(mock.configuredCredentialKeys).toEqual(["projectSecret"]);
|
||||
await expect(
|
||||
page.getByRole("heading", { name: "Try Maya in iMessage Photon" }),
|
||||
).toBeVisible();
|
||||
await expect(
|
||||
page.getByRole("button", { name: "Copy +15555550100" }),
|
||||
).toBeVisible();
|
||||
await expect(
|
||||
page.getByText("Link your Messages identity", { exact: true }),
|
||||
).toBeVisible();
|
||||
await page
|
||||
.getByRole("button", { name: "I've sent the test message" })
|
||||
.click();
|
||||
await page.getByRole("tab", { name: "Settings" }).click();
|
||||
await expect(
|
||||
page.getByText(/replies are visible to everyone in that group/),
|
||||
).toBeVisible();
|
||||
const group = page.getByRole("switch", { name: "Enable Family project" });
|
||||
await expect(group).not.toBeChecked();
|
||||
await group.click();
|
||||
await expect(group).toBeChecked();
|
||||
await page.setViewportSize({ width: 390, height: 844 });
|
||||
await page
|
||||
.getByRole("combobox", { name: "Page section" })
|
||||
.selectOption("activity");
|
||||
await expect(
|
||||
page.getByRole("button", { name: "Pause", exact: true }),
|
||||
).toBeVisible();
|
||||
await page.screenshot({
|
||||
path: testInfo.outputPath(`photon-${theme}-mobile.png`),
|
||||
fullPage: true,
|
||||
});
|
||||
expect(
|
||||
await page.evaluate(
|
||||
() => document.documentElement.scrollWidth <= window.innerWidth,
|
||||
),
|
||||
).toBe(true);
|
||||
await page.getByRole("button", { name: "Pause", exact: true }).click();
|
||||
await expect(
|
||||
page.getByRole("button", { name: "Resume", exact: true }),
|
||||
).toBeVisible();
|
||||
await page.getByRole("button", { name: "Resume", exact: true }).click();
|
||||
await expect(
|
||||
page.getByRole("button", { name: "Pause", exact: true }),
|
||||
).toBeVisible();
|
||||
});
|
||||
}
|
||||
});
|
||||
|
|
|
|||
|
|
@ -0,0 +1,12 @@
|
|||
import { defineConfig } from "@playwright/test";
|
||||
|
||||
export default defineConfig({
|
||||
testDir: ".", testMatch: "imessage-photon.spec.ts", workers: 2,
|
||||
timeout: 30_000, retries: 0, outputDir: "./test-results/imessage-photon",
|
||||
reporter: [["list"]],
|
||||
use: { browserName: "chromium", baseURL: "http://127.0.0.1:6128", reducedMotion: "reduce" },
|
||||
webServer: {
|
||||
command: "node ../../scripts/serve-storybook-static.mjs --port 6128",
|
||||
url: "http://127.0.0.1:6128/index.json", reuseExistingServer: false,
|
||||
},
|
||||
});
|
||||
|
|
@ -0,0 +1,39 @@
|
|||
import { readFileSync } from "node:fs";
|
||||
import { resolve } from "node:path";
|
||||
import { test, expect } from "@playwright/test";
|
||||
|
||||
const index = JSON.parse(readFileSync(resolve(import.meta.dirname, "../../ui/storybook-static/index.json"), "utf8"));
|
||||
type StoryEntry = { id: string; name: string; type: string };
|
||||
const stories = (Object.values(index.entries) as StoryEntry[]).filter((entry) => entry.type === "story" && entry.id.startsWith("connections-imessage-photon--"));
|
||||
const requiredStories = [
|
||||
"catalog", "choose-agent", "credentials", "inspecting", "connecting",
|
||||
"multiple-dedicated-numbers", "no-eligible-line", "provider-outage-recovery",
|
||||
"reconnect", "access", "incoming-follow-ups", "shared-dm-walkthrough", "narrow-mobile",
|
||||
];
|
||||
const discovered = new Set(stories.map((story) => story.id));
|
||||
const missing = requiredStories.filter((name) => !discovered.has(`connections-imessage-photon--${name}`));
|
||||
if (missing.length) throw new Error(`Missing required iMessage Photon stories: ${missing.join(", ")}. Rebuild Storybook and check story discovery.`);
|
||||
for (const story of stories) for (const theme of ["light", "dark"]) {
|
||||
test(`${story.name} / ${theme}`, async ({ page }, info) => {
|
||||
const errors: string[] = [];
|
||||
page.on("pageerror", (error) => errors.push(error.message));
|
||||
await page.setViewportSize({ width: /Narrow/.test(story.name) ? 390 : 1200, height: 844 });
|
||||
await page.goto(`/iframe.html?id=${story.id}&viewMode=story&globals=theme:${theme}`);
|
||||
await page.waitForFunction((id) => document.body.dataset.photonStoryReady === id || document.body.dataset.photonStoryError, story.id);
|
||||
expect(await page.locator("body").getAttribute("data-photon-story-error")).toBeNull();
|
||||
await expect(page.locator(".sb-errordisplay")).not.toBeVisible();
|
||||
await expect(page.getByText("Simulated Photon demo.", { exact: true })).toBeVisible();
|
||||
if (/incoming-follow-ups|narrow-mobile/.test(story.id)) {
|
||||
await expect(page.getByText("Sent from iMessage", { exact: false })).toHaveCount(2);
|
||||
}
|
||||
if (story.id.endsWith("shared-dm-walkthrough")) {
|
||||
await expect(page.getByRole("heading", { name: "DEMO-1 · Launch plan" })).toBeVisible();
|
||||
await expect(page.getByText("Sent from iMessage", { exact: false })).toBeVisible();
|
||||
}
|
||||
if (story.id.endsWith("multiple-dedicated-numbers")) {
|
||||
await expect(page.getByRole("radio", { name: "+15555550112" })).toBeChecked();
|
||||
}
|
||||
expect(errors).toEqual([]);
|
||||
await page.screenshot({ path: info.outputPath(`${story.id}-${theme}.png`), fullPage: true, animations: "disabled" });
|
||||
});
|
||||
}
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 32 KiB |
|
|
@ -634,6 +634,16 @@
|
|||
"upstreamAssetUrl": "https://github.com/simple-icons/simple-icons/blob/16.28.0/icons/zapier.svg",
|
||||
"assetType": "svg",
|
||||
"darkVariantRequired": false
|
||||
},
|
||||
{
|
||||
"slug": "imessage-photon",
|
||||
"provider": "iMessage Photon",
|
||||
"catalogVisible": true,
|
||||
"localAsset": "/brands/apps/imessage-photon.png",
|
||||
"officialSourceUrl": "https://photon.codes/",
|
||||
"upstreamAssetUrl": "https://framerusercontent.com/images/XbFmp7b9ze39qL1GdqVbmJXbiS8.png?height=512&width=512",
|
||||
"assetType": "png",
|
||||
"darkVariantRequired": false
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,5 +1,7 @@
|
|||
import { api } from "./client";
|
||||
import type {
|
||||
PhotonProjectInspection,
|
||||
PhotonChannelConfiguration,
|
||||
ChatPublicationBatchStatus,
|
||||
ChatPublicationState,
|
||||
ChatPublicationSummary,
|
||||
|
|
@ -12,7 +14,7 @@ export type {
|
|||
} from "@paperclipai/shared";
|
||||
|
||||
export type ChatProvider =
|
||||
"slack" | "github" | "discord" | "microsoft-teams" | "telegram" | "agentmail";
|
||||
"slack" | "github" | "discord" | "microsoft-teams" | "telegram" | "agentmail" | "imessage-photon";
|
||||
export type ChatEndpointStatus =
|
||||
| "draft"
|
||||
| "verifying"
|
||||
|
|
@ -33,6 +35,7 @@ export interface ChatEndpointResource {
|
|||
availability: "available" | "unavailable" | "removed";
|
||||
enabled: boolean;
|
||||
detail?: string | null;
|
||||
participants?: string[];
|
||||
}
|
||||
|
||||
export interface ChatIdentityLink {
|
||||
|
|
@ -97,6 +100,7 @@ export interface ChatEndpoint {
|
|||
botLabel?: string | null;
|
||||
botUsername?: string | null;
|
||||
botExternalId?: string | null;
|
||||
photonAllocation?: "dedicated" | "shared";
|
||||
allowDirectMessages?: boolean;
|
||||
allowGroupChats?: boolean;
|
||||
allowUnlinkedPeople: boolean;
|
||||
|
|
@ -188,8 +192,11 @@ export const chatEndpointsApi = {
|
|||
input: {
|
||||
action: ChatEndpointSetupAction;
|
||||
credentials?: Record<string, string>;
|
||||
photon?: PhotonChannelConfiguration;
|
||||
},
|
||||
) => api.post<ChatEndpoint>(`/chat-endpoints/${endpointId}/setup`, input),
|
||||
inspectPhoton: (endpointId: string, input: { projectId: string; projectSecret: string }) =>
|
||||
api.post<PhotonProjectInspection>(`/chat-endpoints/${endpointId}/photon/inspect`, input),
|
||||
generateSetupSecret: (endpointId: string) =>
|
||||
api.post<ChatEndpointSetupSecret>(
|
||||
`/chat-endpoints/${endpointId}/setup-secret`,
|
||||
|
|
|
|||
|
|
@ -426,6 +426,45 @@ describe("IssueChatThread", () => {
|
|||
});
|
||||
});
|
||||
|
||||
it("labels incoming iMessage bubbles without labeling board replies", () => {
|
||||
const root = createRoot(container);
|
||||
act(() => {
|
||||
root.render(
|
||||
<MemoryRouter>
|
||||
<IssueChatThread
|
||||
comments={["imessage", "board"].map((source) => ({
|
||||
id: `comment-${source}`,
|
||||
companyId: "company-1",
|
||||
issueId: "issue-1",
|
||||
authorAgentId: null,
|
||||
authorUserId: "user-board",
|
||||
authorType: "user" as const,
|
||||
body: `Reply from ${source}`,
|
||||
presentation: null,
|
||||
metadata: source === "imessage" ? {
|
||||
version: 1 as const,
|
||||
sourceChannel: "imessage-photon" as const,
|
||||
sections: [{ title: "iMessage Photon sender", rows: [{ type: "text" as const, text: "Linked person" }] }],
|
||||
} : null,
|
||||
createdAt: new Date("2026-09-12T12:00:00Z"),
|
||||
updatedAt: new Date("2026-09-12T12:00:00Z"),
|
||||
}))}
|
||||
linkedRuns={[]}
|
||||
timelineEvents={[]}
|
||||
liveRuns={[]}
|
||||
currentUserId="user-board"
|
||||
onAdd={async () => {}}
|
||||
showComposer={false}
|
||||
enableLiveTranscriptPolling={false}
|
||||
/>
|
||||
</MemoryRouter>,
|
||||
);
|
||||
});
|
||||
expect(container.querySelector("#comment-comment-imessage")?.textContent).toContain("Sent from iMessage");
|
||||
expect(container.querySelector("#comment-comment-board")?.textContent).not.toContain("Sent from iMessage");
|
||||
act(() => root.unmount());
|
||||
});
|
||||
|
||||
it("uses accent-safe markdown color in the current user's blue message bubble", () => {
|
||||
const root = createRoot(container);
|
||||
|
||||
|
|
|
|||
|
|
@ -2019,6 +2019,8 @@ function IssueChatUserMessage({
|
|||
? custom.sourceTrust
|
||||
: null;
|
||||
const followUpRequested = custom.followUpRequested === true;
|
||||
const sentFromIMessage = isIssueCommentMetadata(custom.commentMetadata) &&
|
||||
custom.commentMetadata.sourceChannel === "imessage-photon";
|
||||
const queueReason =
|
||||
typeof custom.queueReason === "string" ? custom.queueReason : null;
|
||||
const queueBadgeLabel =
|
||||
|
|
@ -2151,6 +2153,11 @@ function IssueChatUserMessage({
|
|||
)}
|
||||
</div>
|
||||
|
||||
{sentFromIMessage && !deleted ? (
|
||||
<div className="mt-1 px-1 text-xs text-muted-foreground">
|
||||
Sent from iMessage
|
||||
</div>
|
||||
) : null}
|
||||
{pending ? (
|
||||
<div
|
||||
className={cn(
|
||||
|
|
|
|||
|
|
@ -13,6 +13,7 @@ const providerNames: Record<ChatProvider, string> = {
|
|||
discord: "Discord",
|
||||
"microsoft-teams": "Microsoft Teams",
|
||||
telegram: "Telegram",
|
||||
"imessage-photon": "iMessage Photon",
|
||||
agentmail: "AgentMail",
|
||||
};
|
||||
|
||||
|
|
|
|||
|
|
@ -38,6 +38,7 @@ const providerNames: Record<ChatProvider, string> = {
|
|||
"microsoft-teams": "Microsoft Teams",
|
||||
telegram: "Telegram",
|
||||
agentmail: "AgentMail",
|
||||
"imessage-photon": "iMessage Photon",
|
||||
};
|
||||
|
||||
type PublicationFeedback = {
|
||||
|
|
|
|||
|
|
@ -41,6 +41,19 @@ describe("TaskChatBubble attachment chips", () => {
|
|||
);
|
||||
}
|
||||
|
||||
it("shows persistent iMessage attribution only on inbound human bubbles", () => {
|
||||
for (const author of ["human", "agent"] as const) {
|
||||
flushSync(() => root!.render(
|
||||
<ThemeProvider>
|
||||
<TaskChatBubble item={{ id: "photon", kind: "message", author, text: "A reply", timestamp: "1:56 PM", sourceChannel: "imessage-photon" }} />
|
||||
</ThemeProvider>,
|
||||
));
|
||||
expect(container.textContent?.includes("Sent from iMessage")).toBe(author === "human");
|
||||
}
|
||||
renderMessage("Board reply");
|
||||
expect(container.textContent).not.toContain("Sent from iMessage");
|
||||
});
|
||||
|
||||
it("opens attachment images in the shared task gallery", () => {
|
||||
const openGallery = vi.fn(() => true);
|
||||
const contentPath = "/api/attachments/shared-image/content";
|
||||
|
|
|
|||
|
|
@ -186,6 +186,7 @@ function TaskChatBubbleContent({
|
|||
}
|
||||
|
||||
const isHuman = item.author === "human";
|
||||
const sentFromIMessage = isHuman && item.sourceChannel === "imessage-photon";
|
||||
// Non-image file references ("[name](/api/attachments/…/content)") render as
|
||||
// attachment chips under the bubble; link-only lines leave the body text.
|
||||
const { refs: linkedRefs, text: bodyWithoutAttachmentLinks } =
|
||||
|
|
@ -418,9 +419,11 @@ function TaskChatBubbleContent({
|
|||
) : null}
|
||||
</div>
|
||||
)
|
||||
) : item.timestamp ? (
|
||||
) : item.timestamp || sentFromIMessage ? (
|
||||
// Timestamps are always visible (round 9) — no longer hover-revealed.
|
||||
<span className="px-1 text-(length:--text-micro) text-muted-foreground">
|
||||
{sentFromIMessage ? "Sent from iMessage" : null}
|
||||
{sentFromIMessage && item.timestamp ? " · " : null}
|
||||
{item.timestamp}
|
||||
</span>
|
||||
) : null}
|
||||
|
|
|
|||
|
|
@ -6,6 +6,18 @@ import {
|
|||
} from "./task-chat-adapter";
|
||||
|
||||
describe("commentsToTaskChatItems", () => {
|
||||
it("carries inbound channel attribution into the human bubble", () => {
|
||||
expect(commentsToTaskChatItems([{
|
||||
id: "photon-comment",
|
||||
body: "From my phone",
|
||||
authorType: "user",
|
||||
authorUserId: "local-board",
|
||||
metadata: { version: 1, sourceChannel: "imessage-photon", sections: [] },
|
||||
createdAt: "2026-09-12T12:00:00Z",
|
||||
} as unknown as IssueChatComment])).toMatchObject([{
|
||||
author: "human", sourceChannel: "imessage-photon", text: "From my phone",
|
||||
}]);
|
||||
});
|
||||
it("classifies a recovered local-board comment as an agent bubble", () => {
|
||||
const items = commentsToTaskChatItems([{
|
||||
id: "c-recovered",
|
||||
|
|
|
|||
|
|
@ -121,6 +121,7 @@ export function commentsToTaskChatItems(
|
|||
author: kind,
|
||||
authorName,
|
||||
text: comment.body,
|
||||
sourceChannel: kind === "human" ? comment.metadata?.sourceChannel : undefined,
|
||||
timestamp: formatTaskChatCommentTimestamp(comment, kind),
|
||||
optimistic,
|
||||
queueTargetRunId: queued ? comment.queueTargetRunId ?? null : null,
|
||||
|
|
|
|||
|
|
@ -109,6 +109,8 @@ export interface TaskChatMessageItem {
|
|||
text: string;
|
||||
/** Runner-authored output channel. Legacy adapters leave this unset. */
|
||||
channel?: "progress" | "final" | "unknown";
|
||||
/** Transport attribution for an inbound human comment. */
|
||||
sourceChannel?: IssueCommentMetadata["sourceChannel"];
|
||||
timestamp?: string;
|
||||
/** Show a streaming cursor and suppress collapse while true. */
|
||||
streaming?: boolean;
|
||||
|
|
|
|||
|
|
@ -25,7 +25,8 @@ export function attachmentDownloadPath(attachment: AttachmentPathLike) {
|
|||
}
|
||||
|
||||
export function isImageAttachment(attachment: Pick<IssueAttachment, "contentType">) {
|
||||
return normalizedContentType(attachment).startsWith("image/");
|
||||
const type = normalizedContentType(attachment);
|
||||
return type.startsWith("image/") && !/^image\/hei[cf](?:-sequence)?$/.test(type);
|
||||
}
|
||||
|
||||
export function isVideoAttachment(
|
||||
|
|
|
|||
|
|
@ -257,6 +257,7 @@ describe("Connectors landing page", () => {
|
|||
"discord",
|
||||
"github",
|
||||
"gmail",
|
||||
"imessage-photon",
|
||||
"jira",
|
||||
"microsoft-teams",
|
||||
"notion",
|
||||
|
|
|
|||
|
|
@ -412,6 +412,7 @@ export function Browse() {
|
|||
});
|
||||
}
|
||||
const nativeChatProviders = [
|
||||
{ provider: "imessage-photon", name: "iMessage Photon", description: "Message agents and share photos from Apple Messages with a dedicated Photon number." },
|
||||
{
|
||||
provider: "slack",
|
||||
name: "Slack",
|
||||
|
|
@ -534,6 +535,7 @@ export function Browse() {
|
|||
discord: "Discord",
|
||||
"microsoft-teams": "Microsoft Teams",
|
||||
telegram: "Telegram",
|
||||
"imessage-photon": "iMessage Photon",
|
||||
agentmail: "AgentMail",
|
||||
} as const;
|
||||
target = {
|
||||
|
|
|
|||
|
|
@ -54,6 +54,7 @@ const providerNames: Record<ChatProvider, string> = {
|
|||
discord: "Discord",
|
||||
"microsoft-teams": "Microsoft Teams",
|
||||
telegram: "Telegram",
|
||||
"imessage-photon": "iMessage Photon",
|
||||
};
|
||||
|
||||
const providerLifecycleGuidance: Record<
|
||||
|
|
@ -85,6 +86,10 @@ const providerLifecycleGuidance: Record<
|
|||
remove:
|
||||
"Paperclip archives the endpoint, stops new ingress, and retires its saved client secret. It does not uninstall the Teams app: the Entra app registration, Azure Bot, custom Teams app, and Teams installations remain until you remove them in Microsoft.",
|
||||
},
|
||||
"imessage-photon": {
|
||||
reconnect: "Reconnect verifies the same Photon project and line allocation, then recovers eligible missed messages.",
|
||||
remove: "Disconnect archives this channel and removes its saved secret. Your Photon project, number, subscription, and Messages history remain in Photon.",
|
||||
},
|
||||
telegram: {
|
||||
reconnect:
|
||||
"Reconnect verifies this same BotFather bot and automatically refreshes its Paperclip webhook and command menu.",
|
||||
|
|
@ -223,6 +228,7 @@ export function ChatEndpointDetail() {
|
|||
: false,
|
||||
});
|
||||
const endpoint = endpointQuery.data;
|
||||
const [copyStatus, setCopyStatus] = useState<string | null>(null);
|
||||
|
||||
useEffect(() => {
|
||||
if (!endpoint || !activeTab) return;
|
||||
|
|
@ -276,6 +282,16 @@ export function ChatEndpointDetail() {
|
|||
<p className="mt-1 text-sm text-muted-foreground">
|
||||
{endpoint.providerAccountLabel ?? "Chat connection"}
|
||||
</p>
|
||||
{endpoint.provider === "imessage-photon" && endpoint.botExternalId && endpoint.photonAllocation !== "shared" && (
|
||||
<div className="mt-2 flex flex-wrap items-center gap-2 text-sm">
|
||||
<span>{endpoint.botExternalId}</span>
|
||||
<Button variant="ghost" size="sm" aria-label="Copy dedicated number" onClick={async () => {
|
||||
try { await copyTextToClipboard(endpoint.botExternalId!); setCopyStatus("Number copied"); }
|
||||
catch { setCopyStatus("Could not copy the number. Select and copy it manually."); }
|
||||
}}><Copy className="size-4" />Copy number</Button>
|
||||
<span role="status" className="text-muted-foreground">{copyStatus}</span>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
<div className="flex items-center gap-2">
|
||||
{setupIncomplete ? (
|
||||
|
|
@ -376,6 +392,7 @@ function Settings({
|
|||
saveResources.mutate({ id: resource.id, enabled });
|
||||
return (
|
||||
<section className="max-w-3xl space-y-7">
|
||||
{endpoint.provider === "imessage-photon" && <p className="text-sm text-muted-foreground">{endpoint.photonAllocation === "shared" ? "Shared Photon project · direct messages only. Enroll senders in Photon and link their Messages identities in Access. Groups cannot be enabled." : "Enable each group individually. Agent replies are visible to everyone in that group; only authorized senders can start work."}</p>}
|
||||
{endpoint.provider === "slack" && endpoint.setup?.command && (
|
||||
<div className="space-y-2">
|
||||
<h2 className="text-lg font-semibold">Slack command</h2>
|
||||
|
|
@ -437,11 +454,13 @@ function Settings({
|
|||
? (resource.detail ?? resource.type)
|
||||
: "Unavailable at the provider"}
|
||||
</p>
|
||||
{resource.participants?.length ? <p className="mt-1 break-words text-xs text-muted-foreground">Participants: {resource.participants.join(", ")}</p> : null}
|
||||
</div>
|
||||
<ToggleSwitch
|
||||
aria-label={`Enable ${resource.label}`}
|
||||
checked={resource.enabled}
|
||||
disabled={
|
||||
endpoint.photonAllocation === "shared" ||
|
||||
resource.availability !== "available" ||
|
||||
saveResources.isPending
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,3 +1,4 @@
|
|||
import { PhotonConnectStep } from "./PhotonConnectStep";
|
||||
import { EmailEndpointSetup } from "./EmailEndpointSetup";
|
||||
import {
|
||||
useEffect,
|
||||
|
|
@ -42,6 +43,7 @@ const providerNames: Record<ChatProvider, string> = {
|
|||
discord: "Discord",
|
||||
"microsoft-teams": "Microsoft Teams",
|
||||
telegram: "Telegram",
|
||||
"imessage-photon": "iMessage Photon",
|
||||
};
|
||||
|
||||
const knownProviders = new Set(Object.keys(providerNames));
|
||||
|
|
@ -128,7 +130,7 @@ export function ChatEndpointSetup() {
|
|||
return params.get("provider") === "agentmail" ? <EmailEndpointSetup /> : <ChatSdkEndpointSetup />;
|
||||
}
|
||||
function ChatSdkEndpointSetup() {
|
||||
const [params] = useSearchParams();
|
||||
const [params, setParams] = useSearchParams();
|
||||
const navigate = useNavigate();
|
||||
const queryClient = useQueryClient();
|
||||
const { selectedCompanyId } = useCompany();
|
||||
|
|
@ -233,7 +235,14 @@ function ChatSdkEndpointSetup() {
|
|||
provider: provider!,
|
||||
assignedAgentId: agentId,
|
||||
}),
|
||||
onSuccess: syncEndpointSnapshot,
|
||||
onSuccess: (next) => {
|
||||
syncEndpointSnapshot(next);
|
||||
if (next.provider === "imessage-photon") {
|
||||
const resumed = new URLSearchParams(params);
|
||||
resumed.set("resume", next.id);
|
||||
setParams(resumed, { replace: true });
|
||||
}
|
||||
},
|
||||
onError: (error) =>
|
||||
pushToast({
|
||||
title: "Couldn't start setup",
|
||||
|
|
@ -248,11 +257,16 @@ function ChatSdkEndpointSetup() {
|
|||
}: {
|
||||
action: ChatEndpointSetupAction;
|
||||
values?: Record<string, string>;
|
||||
}) => chatEndpointsApi.setup(endpoint!.id, { action, credentials: values }),
|
||||
}) => chatEndpointsApi.setup(endpoint!.id, provider === "imessage-photon" ? {
|
||||
action,
|
||||
...(values?.projectSecret ? { credentials: { projectSecret: values.projectSecret } } : {}),
|
||||
...(values?.projectId && values.allocation === "shared" ? { photon: { allocation: "shared" as const, projectId: values.projectId } } : values?.projectId && values?.lineId ? { photon: { allocation: "dedicated" as const, projectId: values.projectId, lineId: values.lineId } } : {}),
|
||||
} : { action, credentials: values }),
|
||||
onMutate: () => setSetupError(null),
|
||||
onSuccess: (next) => {
|
||||
setSetupError(null);
|
||||
syncEndpointSnapshot(next);
|
||||
setCredentials({});
|
||||
},
|
||||
onError: (error, variables) =>
|
||||
setSetupError(sanitizedSetupErrorMessage(error, variables.values)),
|
||||
|
|
@ -473,6 +487,7 @@ function ChatSdkEndpointSetup() {
|
|||
agentName={selectedAgent?.name ?? endpoint.assignedAgentName}
|
||||
botLabel={endpoint.botLabel}
|
||||
botUsername={endpoint.botUsername}
|
||||
photonAllocation={endpoint.photonAllocation}
|
||||
providerUrl={endpoint.setup?.providerUrl}
|
||||
guestIsolationState={
|
||||
experimentalSettingsQuery.isPending
|
||||
|
|
@ -740,6 +755,7 @@ settings:
|
|||
null,
|
||||
2,
|
||||
);
|
||||
if (provider === "imessage-photon") return <PhotonConnectStep endpoint={endpoint} agentName={agentName} repairing={repairing} pending={pending} onAction={onAction} />;
|
||||
if (provider === "discord") {
|
||||
const applicationId = credentials.applicationId?.trim() ?? "";
|
||||
const guildId = credentials.guildId?.trim() ?? "";
|
||||
|
|
@ -756,7 +772,7 @@ settings:
|
|||
: "Create one dedicated Discord application and bot for this Paperclip agent."}
|
||||
</p>
|
||||
</div>
|
||||
<ol className="list-decimal space-y-2 pl-5 text-sm">
|
||||
<ol className="list-decimal space-y-2 pl-5 text-sm">
|
||||
<li>
|
||||
In Discord Developer Portal, create an application. Copy its
|
||||
Application ID from General Information.
|
||||
|
|
@ -1468,6 +1484,7 @@ function TryStep({
|
|||
agentName,
|
||||
botLabel,
|
||||
botUsername,
|
||||
photonAllocation,
|
||||
providerUrl,
|
||||
guestIsolationState,
|
||||
pending,
|
||||
|
|
@ -1479,6 +1496,7 @@ function TryStep({
|
|||
agentName: string;
|
||||
botLabel?: string | null;
|
||||
botUsername?: string | null;
|
||||
photonAllocation?: "dedicated" | "shared";
|
||||
providerUrl?: string | null;
|
||||
guestIsolationState: "loading" | "enabled" | "disabled" | "unknown";
|
||||
pending: boolean;
|
||||
|
|
@ -1490,19 +1508,23 @@ function TryStep({
|
|||
queryFn: () => chatEndpointsApi.listPrincipals(endpointId),
|
||||
refetchInterval: 1_500,
|
||||
});
|
||||
const [numberCopied, setNumberCopied] = useState(false);
|
||||
const [copyError, setCopyError] = useState<string | null>(null);
|
||||
const identities = principalsQuery.data ?? [];
|
||||
const unlinkedIdentities = identities.filter(
|
||||
(identity) => identity.status !== "linked",
|
||||
);
|
||||
const freshConversationInstruction =
|
||||
provider === "telegram"
|
||||
provider === "imessage-photon" ? "send a fresh message to your Photon number" : provider === "telegram"
|
||||
? "start a fresh conversation with /new and send the test message again"
|
||||
: provider === "github"
|
||||
? "start a new issue or pull request conversation and mention the agent again"
|
||||
: provider === "microsoft-teams"
|
||||
? "start a new channel post and mention the agent again"
|
||||
: "send a new root mention to the agent";
|
||||
const identityGuidance = principalsQuery.isError
|
||||
const identityGuidance = provider === "imessage-photon" && principalsQuery.isSuccess && (identities.length === 0 || unlinkedIdentities.length > 0)
|
||||
? { tone: "info" as const, title: "Link your Messages identity", body: "Send one message to discover your phone number or Apple account address, then link that exact identity in Access. Send a fresh request after linking; earlier messages do not start work." }
|
||||
: principalsQuery.isError
|
||||
? {
|
||||
tone: "warning" as const,
|
||||
title: "Identity readiness could not be checked",
|
||||
|
|
@ -1550,7 +1572,12 @@ function TryStep({
|
|||
? `@${normalizedBotUsername}`
|
||||
: (botLabel ?? agentName);
|
||||
const instructions =
|
||||
provider === "discord"
|
||||
provider === "imessage-photon" ? [
|
||||
photonAllocation === "shared" ? "In your Photon project, enroll your sender in Users and find its assigned number in Get started. Send a fresh message to that number from Apple Messages." : `Open Apple Messages and send a fresh message to ${botUsername ?? botLabel ?? "the dedicated number"}.`,
|
||||
"Link the discovered sender to a Paperclip person in Access, then send a fresh request.",
|
||||
"Wait for the agent’s actual reply. Setup completes after that reply is delivered.",
|
||||
...(photonAllocation === "shared" ? ["This Pro-compatible channel supports DMs only. Group messages cannot start work."] : ["For a group: add the number in Messages, send a message, enable the discovered group in Settings, then send a fresh request."]),
|
||||
] : provider === "discord"
|
||||
? [
|
||||
"Open a text channel where the bot is installed.",
|
||||
`Mention ${botMention} in a new root message.`,
|
||||
|
|
@ -1616,6 +1643,7 @@ function TryStep({
|
|||
</Button>
|
||||
</div>
|
||||
) : null}
|
||||
{provider === "imessage-photon" && botUsername && <div className="space-y-2"><Button variant="outline" onClick={() => { void copyTextToClipboard(botUsername).then(() => { setNumberCopied(true); setCopyError(null); }, () => setCopyError("Could not copy the number. Select it in the instructions below.")); }}>{numberCopied ? "Number copied" : `Copy ${botUsername}`}</Button>{copyError && <p role="alert" className="text-sm text-destructive">{copyError}</p>}</div>}
|
||||
<ol className="list-decimal space-y-2 pl-5 text-sm">
|
||||
{instructions.map((item) => (
|
||||
<li key={item}>{item}</li>
|
||||
|
|
|
|||
|
|
@ -14,6 +14,7 @@ const providerNames: Record<ChatProvider, string> = {
|
|||
"microsoft-teams": "Microsoft Teams",
|
||||
telegram: "Telegram",
|
||||
agentmail: "AgentMail",
|
||||
"imessage-photon": "iMessage Photon",
|
||||
};
|
||||
|
||||
export function ChatIdentityConfirm() {
|
||||
|
|
|
|||
|
|
@ -0,0 +1,192 @@
|
|||
import { useState } from "react";
|
||||
import { useMutation } from "@tanstack/react-query";
|
||||
import { Button } from "@/components/ui/button";
|
||||
import { Input } from "@/components/ui/input";
|
||||
import {
|
||||
chatEndpointsApi,
|
||||
type ChatEndpoint,
|
||||
type ChatEndpointSetupAction,
|
||||
} from "@/api/chatEndpoints";
|
||||
import { sanitizedSetupErrorMessage } from "./chat-setup-error";
|
||||
|
||||
export function PhotonConnectStep({
|
||||
endpoint,
|
||||
agentName,
|
||||
repairing,
|
||||
pending,
|
||||
onAction,
|
||||
}: {
|
||||
endpoint: ChatEndpoint;
|
||||
agentName: string;
|
||||
repairing: boolean;
|
||||
pending: boolean;
|
||||
onAction(
|
||||
action: ChatEndpointSetupAction,
|
||||
values?: Record<string, string>,
|
||||
): void;
|
||||
}) {
|
||||
const [projectId, setProjectId] = useState(endpoint.providerAccountId ?? "");
|
||||
const [projectSecret, setProjectSecret] = useState("");
|
||||
const [lineId, setLineId] = useState("");
|
||||
const inspection = useMutation({
|
||||
mutationFn: () =>
|
||||
chatEndpointsApi.inspectPhoton(endpoint.id, {
|
||||
projectId: projectId.trim(),
|
||||
projectSecret,
|
||||
}),
|
||||
onSuccess: (result) => {
|
||||
const eligible = result.lines.filter((line) => line.eligible);
|
||||
setLineId(eligible.length === 1 ? eligible[0].lineId : "");
|
||||
},
|
||||
});
|
||||
const resetInspection = () => {
|
||||
inspection.reset();
|
||||
setLineId("");
|
||||
};
|
||||
return (
|
||||
<div className="space-y-5">
|
||||
<div className="space-y-2">
|
||||
<h1 className="text-xl font-bold">Connect iMessage Photon</h1>
|
||||
<p className="text-sm text-muted-foreground">
|
||||
Connect {agentName} to Photon Cloud. Pro supports direct messages through
|
||||
a shared line. Dedicated numbers also support individually enabled groups.
|
||||
</p>
|
||||
<p className="text-sm">
|
||||
<a
|
||||
className="underline"
|
||||
href="https://app.photon.codes/"
|
||||
target="_blank"
|
||||
rel="noreferrer"
|
||||
>
|
||||
Photon dashboard
|
||||
</a>
|
||||
{" · "}
|
||||
<a
|
||||
className="underline"
|
||||
href="https://photon.codes/docs/spectrum-ts/providers/imessage/connection-and-routing"
|
||||
target="_blank"
|
||||
rel="noreferrer"
|
||||
>
|
||||
Photon line setup
|
||||
</a>
|
||||
</p>
|
||||
</div>
|
||||
{repairing && (
|
||||
<p className="text-sm text-muted-foreground">
|
||||
Reconnect keeps this project and{" "}
|
||||
{endpoint.photonAllocation === "shared" ? "shared DM allocation" : endpoint.botExternalId ?? "dedicated number"}. Leave the secret blank
|
||||
to reuse the saved connection.
|
||||
</p>
|
||||
)}
|
||||
<label className="grid gap-2 text-sm font-medium">
|
||||
Project ID
|
||||
<Input
|
||||
value={projectId}
|
||||
autoComplete="off"
|
||||
disabled={pending || inspection.isPending || !!endpoint.botExternalId}
|
||||
onChange={(event) => {
|
||||
setProjectId(event.target.value);
|
||||
resetInspection();
|
||||
}}
|
||||
/>
|
||||
</label>
|
||||
<label className="grid gap-2 text-sm font-medium">
|
||||
Project secret
|
||||
<Input
|
||||
type="password"
|
||||
value={projectSecret}
|
||||
autoComplete="new-password"
|
||||
disabled={pending || inspection.isPending}
|
||||
onChange={(event) => {
|
||||
setProjectSecret(event.target.value);
|
||||
resetInspection();
|
||||
}}
|
||||
/>
|
||||
</label>
|
||||
<Button
|
||||
variant="outline"
|
||||
disabled={
|
||||
pending || inspection.isPending || !projectId.trim() || !projectSecret
|
||||
}
|
||||
onClick={() => inspection.mutate()}
|
||||
>
|
||||
{inspection.isPending ? "Inspecting Photon…" : "Inspect Photon project"}
|
||||
</Button>
|
||||
{inspection.isError && (
|
||||
<p role="alert" className="text-sm text-destructive">
|
||||
{sanitizedSetupErrorMessage(inspection.error, { projectSecret })}
|
||||
</p>
|
||||
)}
|
||||
{inspection.data && (
|
||||
<fieldset className="space-y-3">
|
||||
<legend className="text-sm font-medium">
|
||||
{inspection.data.allocation === "shared" ? "Shared DMs" : "Dedicated numbers"} in {inspection.data.projectName}
|
||||
</legend>
|
||||
{!inspection.data.eligible && (
|
||||
<p role="alert" className="text-sm text-destructive">
|
||||
{inspection.data.allocation === "shared"
|
||||
? "This shared project already belongs to another channel. Use a separate Photon project for each agent."
|
||||
: "No eligible dedicated number is available. Check the line allocation in Photon and existing Paperclip channels."}
|
||||
</p>
|
||||
)}
|
||||
{inspection.data.allocation === "shared" && inspection.data.eligible && (
|
||||
<p className="text-sm text-muted-foreground">
|
||||
Direct messages only. Enroll each test sender in your Photon project's Users page,
|
||||
then use the number Photon assigns to that sender. Paperclip identity linking is
|
||||
still required. Groups cannot be enabled on this channel.
|
||||
</p>
|
||||
)}
|
||||
{inspection.data.lines.map((line) => (
|
||||
<label
|
||||
key={line.lineId}
|
||||
className="flex items-center gap-2 text-sm"
|
||||
>
|
||||
<input
|
||||
type="radio"
|
||||
name="photon-line"
|
||||
value={line.lineId}
|
||||
checked={lineId === line.lineId}
|
||||
disabled={
|
||||
!line.eligible ||
|
||||
pending ||
|
||||
(!!endpoint.botExternalId &&
|
||||
endpoint.botExternalId !== line.phoneNumber)
|
||||
}
|
||||
onChange={() => setLineId(line.lineId)}
|
||||
/>
|
||||
<span>
|
||||
{line.phoneNumber}
|
||||
{line.unavailableReason ? ` — ${line.unavailableReason}` : ""}
|
||||
</span>
|
||||
</label>
|
||||
))}
|
||||
</fieldset>
|
||||
)}
|
||||
<div>
|
||||
<Button
|
||||
disabled={
|
||||
pending ||
|
||||
inspection.isPending ||
|
||||
(!(inspection.data?.eligible && (inspection.data.allocation === "shared" || lineId)) && !(repairing && !projectSecret))
|
||||
}
|
||||
onClick={() =>
|
||||
onAction(
|
||||
repairing ? "reconnect" : "configure",
|
||||
inspection.data?.eligible && inspection.data.allocation === "shared"
|
||||
? { projectId: projectId.trim(), projectSecret, allocation: "shared" }
|
||||
: lineId
|
||||
? { projectId: projectId.trim(), projectSecret, lineId, allocation: "dedicated" }
|
||||
: undefined,
|
||||
)
|
||||
}
|
||||
>
|
||||
{pending
|
||||
? "Connecting…"
|
||||
: repairing
|
||||
? "Reconnect Photon"
|
||||
: inspection.data?.allocation === "shared" ? "Connect shared DMs" : "Connect selected number"}
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
|
@ -0,0 +1,206 @@
|
|||
import { useEffect, useState } from "react";
|
||||
import type { Meta, StoryObj } from "@storybook/react-vite";
|
||||
import { addons } from "storybook/preview-api";
|
||||
import { expect, userEvent, within, waitFor } from "storybook/test";
|
||||
import { QueryClient, QueryClientProvider, useQueryClient } from "@tanstack/react-query";
|
||||
import { Routes, Route, useNavigate } from "@/lib/router";
|
||||
import { ChatEndpointSetup } from "@/pages/apps/chat/ChatEndpointSetup";
|
||||
import { ChatEndpointDetail } from "@/pages/apps/chat/ChatEndpointDetail";
|
||||
import { ConnectorCard } from "@/pages/apps/Browse";
|
||||
import { TaskChatBubble } from "@/components/task-chat/TaskChatBubble";
|
||||
import { Button } from "@/components/ui/button";
|
||||
import type { ChatEndpoint, ChatIdentityLink } from "@/api/chatEndpoints";
|
||||
import type { TaskChatMessageItem } from "@/components/task-chat/task-chat-model";
|
||||
import { storybookAgents } from "../fixtures/paperclipData";
|
||||
|
||||
type Screen = "catalog" | "agent" | "credentials" | "access" | "task" | "reconnect";
|
||||
type Scenario = { screen?: Screen; dedicated?: boolean; outage?: boolean; noLine?: boolean; loading?: boolean; connecting?: boolean };
|
||||
const endpointId = "photon-story-endpoint";
|
||||
const agent = storybookAgents[0];
|
||||
const setupUrl = `/apps/chat/new?provider=imessage-photon&purpose=chat&agentId=${agent.id}`;
|
||||
const sender: ChatIdentityLink = { id: "demo-link", principalId: "demo-person", externalLabel: "+15555550101", status: "pending" };
|
||||
let endpoint: ChatEndpoint;
|
||||
let principal: ChatIdentityLink | null;
|
||||
let received = false;
|
||||
let messages: TaskChatMessageItem[] = [];
|
||||
|
||||
function resetFixture(scenario: Scenario) {
|
||||
const established = ["access", "task", "reconnect"].includes(scenario.screen ?? "catalog");
|
||||
endpoint = {
|
||||
id: endpointId, companyId: agent.companyId, provider: "imessage-photon",
|
||||
status: established ? "verifying" : "draft", assignedAgentId: agent.id,
|
||||
assignedAgentName: agent.name, allowDirectMessages: true, allowUnlinkedPeople: false,
|
||||
allowGroupChats: false, photonAllocation: scenario.dedicated ? "dedicated" : "shared",
|
||||
setup: { step: established ? "test" : "provider_setup" },
|
||||
...(established ? { providerAccountId: "demo-project", providerAccountLabel: "Demo project", botExternalId: "photon-project:demo-project" } : {}),
|
||||
};
|
||||
principal = established ? { ...sender } : null;
|
||||
received = false;
|
||||
messages = [];
|
||||
if (scenario.screen === "task") {
|
||||
principal = { ...sender, status: "linked", paperclipUserLabel: "Alex" };
|
||||
simulateIncoming();
|
||||
}
|
||||
if (scenario.screen === "reconnect") endpoint.status = "attention";
|
||||
}
|
||||
|
||||
function simulateIncoming() {
|
||||
principal ??= { ...sender };
|
||||
if (principal.status !== "linked") return;
|
||||
received = true;
|
||||
const followUp = messages.length > 0;
|
||||
messages.push(
|
||||
{ id: `input-${messages.length}`, kind: "message", author: "human", text: followUp ? "And what about the next step?" : "Help me plan the launch.", sourceChannel: "imessage-photon", timestamp: "2:00 PM" },
|
||||
{ id: `reply-${messages.length}`, kind: "message", author: "agent", authorName: agent.name, text: followUp ? "Continuing our plan in DEMO-1. Next, review the launch checklist." : "Let’s start with the launch checklist. This conversation is DEMO-1.", timestamp: "2:00 PM" },
|
||||
);
|
||||
}
|
||||
|
||||
function Journey({ screen = "catalog" }: { screen?: Screen }) {
|
||||
const navigate = useNavigate();
|
||||
const queryClient = useQueryClient();
|
||||
const [, rerender] = useState(0);
|
||||
useEffect(() => {
|
||||
navigate(screen === "catalog" ? "/apps" : screen === "task" ? "/issues/DEMO-1" : screen === "access" ? `/apps/chat/${endpointId}/access` : `${setupUrl}${screen === "agent" ? "" : `&resume=${endpointId}`}${screen === "reconnect" ? "&reconnect=1" : ""}`, { replace: true });
|
||||
}, [screen]);
|
||||
const refresh = () => { void queryClient.invalidateQueries(); rerender((n) => n + 1); };
|
||||
return <div className="mx-auto max-w-4xl space-y-6 p-4">
|
||||
<aside className="space-y-2 rounded-lg border border-border bg-muted p-3 text-sm">
|
||||
<p><strong>Simulated Photon demo.</strong> Production catalog, setup, access, management, and message components use local fixtures. No credentials are saved and no real messages are sent.</p>
|
||||
<div className="flex flex-wrap gap-2">
|
||||
<Button size="sm" variant="outline" onClick={() => { simulateIncoming(); refresh(); }}>Simulate incoming iMessage</Button>
|
||||
<Button size="sm" variant="outline" onClick={() => { principal = { ...sender, status: "linked", paperclipUserLabel: "Alex" }; refresh(); }}>Simulate identity confirmation</Button>
|
||||
<Button size="sm" variant="outline" onClick={() => navigate(`${setupUrl}&resume=${endpointId}`)}>Return to setup</Button>
|
||||
<Button size="sm" variant="outline" onClick={() => navigate("/issues/DEMO-1")}>View simulated task</Button>
|
||||
</div>
|
||||
</aside>
|
||||
<Routes>
|
||||
<Route path="/:companyPrefix/apps" element={<div role="list" aria-label="Connectors"><ConnectorCard
|
||||
row={{ key: "imessage-photon", slug: "imessage-photon", name: "iMessage Photon", description: "Message an agent from Apple Messages. Send photos and keep one task conversation.", brandKey: "imessage-photon", entry: null, applications: [], connections: [], chatEndpoints: [] }}
|
||||
allConnections={[]} userProfileById={new Map()} chatConnectorsEnabled
|
||||
onNavigate={() => navigate(setupUrl)} onRequestRemove={() => {}}
|
||||
/></div>} />
|
||||
<Route path="/:companyPrefix/apps/chat/new" element={<ChatEndpointSetup />} />
|
||||
<Route path="/:companyPrefix/apps/chat/:endpointId/:tab" element={<ChatEndpointDetail />} />
|
||||
<Route path="/:companyPrefix/issues/DEMO-1" element={<section className="space-y-4" aria-label="Simulated task conversation">
|
||||
<h1 className="text-xl font-bold">DEMO-1 · Launch plan</h1>
|
||||
<p className="text-sm text-muted-foreground">Each simulated reply completes a turn. Follow-ups stay on DEMO-1. Use /new or /close in Messages when you want a new task.</p>
|
||||
{messages.length ? messages.map((message) => <TaskChatBubble key={message.id} item={message} />) : <p>Link the sender, then simulate a fresh message to start work.</p>}
|
||||
</section>} />
|
||||
</Routes>
|
||||
</div>;
|
||||
}
|
||||
|
||||
function Host(props: { screen?: Screen }) {
|
||||
const [client] = useState(() => new QueryClient({ defaultOptions: { queries: { retry: false }, mutations: { retry: false } } }));
|
||||
return <QueryClientProvider client={client}><Journey {...props} /></QueryClientProvider>;
|
||||
}
|
||||
|
||||
const meta = {
|
||||
title: "Connections/iMessage Photon", component: Host,
|
||||
parameters: { layout: "fullscreen" },
|
||||
beforeEach: ({ parameters }) => {
|
||||
const scenario = (parameters.photonScenario ?? {}) as Scenario;
|
||||
resetFixture(scenario);
|
||||
delete document.body.dataset.photonStoryReady;
|
||||
delete document.body.dataset.photonStoryError;
|
||||
const channel = addons.getChannel();
|
||||
const report = (error: unknown) => { document.body.dataset.photonStoryError = JSON.stringify(error); };
|
||||
channel.on("playFunctionThrewException", report);
|
||||
channel.on("unhandledErrorsWhilePlaying", report);
|
||||
const original = window.fetch;
|
||||
let inspections = 0;
|
||||
window.fetch = async (input, init) => {
|
||||
const url = new URL(typeof input === "string" ? input : input instanceof URL ? input.href : input.url, window.location.origin);
|
||||
if (url.pathname.endsWith("/agents")) return Response.json([agent]);
|
||||
if (url.pathname === "/api/instance/settings/experimental") return Response.json({ enableChatConnectors: true, enableIsolatedWorkspaces: false });
|
||||
if (url.pathname.includes("/chat-endpoints")) {
|
||||
const body = init?.body ? JSON.parse(String(init.body)) : {};
|
||||
if (url.pathname.endsWith("/photon/inspect")) {
|
||||
if (scenario.loading) return new Promise<Response>(() => {});
|
||||
if (scenario.outage && inspections++ === 0) return Response.json({ error: "Photon is temporarily unavailable. Try again.", details: { code: "photon_network" } }, { status: 503 });
|
||||
return Response.json({ projectId: "demo-project", projectName: "Demo project", allocation: scenario.dedicated ? "dedicated" : "shared", eligible: !scenario.noLine, lines: scenario.dedicated && !scenario.noLine ? [
|
||||
{ lineId: "line-a", phoneNumber: "+15555550111", eligible: true },
|
||||
{ lineId: "line-b", phoneNumber: "+15555550112", eligible: true },
|
||||
] : [] });
|
||||
}
|
||||
if (url.pathname.endsWith("/setup")) {
|
||||
if (scenario.connecting) return new Promise<Response>(() => {});
|
||||
const number = body.photon?.lineId === "line-b" ? "+15555550112" : "+15555550111";
|
||||
endpoint = { ...endpoint, status: "verifying", setup: { step: "test" }, providerAccountId: "demo-project", providerAccountLabel: "Demo project", botExternalId: scenario.dedicated ? number : "photon-project:demo-project", botUsername: scenario.dedicated ? number : null };
|
||||
}
|
||||
if (url.pathname.endsWith("/test")) {
|
||||
if (!received) return Response.json({ error: "A linked sender must send a fresh test message and receive an agent reply." }, { status: 422 });
|
||||
endpoint = { ...endpoint, status: "active", setup: { step: "complete" } };
|
||||
}
|
||||
if (url.pathname.endsWith("/principals")) return Response.json(principal ? [principal] : []);
|
||||
if (url.pathname.endsWith("/resources") || url.pathname.endsWith("/activity")) return Response.json([]);
|
||||
if (url.pathname.endsWith("/conversations")) return Response.json(received ? [{ id: "demo-conversation", externalLabel: sender.externalLabel, issueId: "DEMO-1", issueIdentifier: "DEMO-1", issueTitle: "Launch plan", state: "active" }] : []);
|
||||
if (url.pathname.endsWith("/link-intent")) return Response.json({ confirmationUrl: `${window.location.origin}/simulated-photon-confirmation` });
|
||||
if (init?.method === "PATCH") endpoint = { ...endpoint, ...body };
|
||||
return Response.json(endpoint);
|
||||
}
|
||||
return original(input, init);
|
||||
};
|
||||
return () => { window.fetch = original; channel.off("playFunctionThrewException", report); channel.off("unhandledErrorsWhilePlaying", report); };
|
||||
},
|
||||
afterEach: ({ id }) => { document.body.dataset.photonStoryReady = id; },
|
||||
} satisfies Meta<typeof Host>;
|
||||
export default meta;
|
||||
type Story = StoryObj<typeof meta>;
|
||||
const step = (screen: Screen, scenario: Scenario = {}): Story => ({ args: { screen }, parameters: { photonScenario: { ...scenario, screen } } });
|
||||
const inspect: Story["play"] = async ({ canvasElement }) => {
|
||||
const canvas = within(canvasElement);
|
||||
await userEvent.type(await canvas.findByLabelText("Project ID"), "demo-project");
|
||||
await userEvent.type(canvas.getByLabelText("Project secret"), "simulated-secret");
|
||||
await userEvent.click(canvas.getByRole("button", { name: "Inspect Photon project" }));
|
||||
};
|
||||
export const Catalog: Story = step("catalog");
|
||||
export const ChooseAgent: Story = step("agent");
|
||||
export const Credentials: Story = step("credentials");
|
||||
export const Inspecting: Story = { ...step("credentials", { loading: true }), play: inspect };
|
||||
export const Connecting: Story = { ...step("credentials", { connecting: true }), play: async (context) => {
|
||||
await inspect!(context); await userEvent.click(await within(context.canvasElement).findByRole("button", { name: "Connect shared DMs" }));
|
||||
} };
|
||||
export const MultipleDedicatedNumbers: Story = { ...step("credentials", { dedicated: true }), play: async (context) => {
|
||||
await inspect!(context); const canvas = within(context.canvasElement);
|
||||
await expect(await canvas.findByRole("button", { name: "Connect selected number" })).toBeDisabled();
|
||||
await userEvent.click(await canvas.findByRole("radio", { name: "+15555550112" }));
|
||||
await expect(canvas.getByRole("button", { name: "Connect selected number" })).toBeEnabled();
|
||||
} };
|
||||
export const NoEligibleLine: Story = { ...step("credentials", { dedicated: true, noLine: true }), play: async (context) => {
|
||||
await inspect!(context); await expect(await within(context.canvasElement).findByRole("alert")).toHaveTextContent("No eligible dedicated number");
|
||||
} };
|
||||
export const ProviderOutageRecovery: Story = { ...step("credentials", { outage: true }), play: async (context) => {
|
||||
await inspect!(context); const canvas = within(context.canvasElement);
|
||||
await expect(await canvas.findByRole("alert")).toHaveTextContent("temporarily unavailable");
|
||||
await userEvent.click(canvas.getByRole("button", { name: "Inspect Photon project" }));
|
||||
await expect(await canvas.findByRole("button", { name: "Connect shared DMs" })).toBeEnabled();
|
||||
} };
|
||||
export const Reconnect: Story = step("reconnect");
|
||||
export const Access: Story = step("access");
|
||||
export const IncomingFollowUps: Story = { ...step("task"), play: async ({ canvasElement }) => {
|
||||
const canvas = within(canvasElement);
|
||||
await waitFor(() => expect(canvas.getByText("Sent from iMessage", { exact: false })).toBeVisible());
|
||||
await userEvent.click(canvas.getByRole("button", { name: "Simulate incoming iMessage" }));
|
||||
await expect(canvas.getByRole("heading", { name: "DEMO-1 · Launch plan" })).toBeVisible();
|
||||
await expect(canvas.getAllByText("Sent from iMessage", { exact: false })).toHaveLength(2);
|
||||
} };
|
||||
export const SharedDmWalkthrough: Story = { ...step("catalog"), play: async (context) => {
|
||||
const canvas = within(context.canvasElement);
|
||||
await userEvent.click(await canvas.findByRole("button", { name: "Connect iMessage Photon" }));
|
||||
await userEvent.click(await canvas.findByRole("button", { name: "Continue" }));
|
||||
await inspect!(context);
|
||||
await userEvent.click(await canvas.findByRole("button", { name: "Connect shared DMs" }));
|
||||
await expect(await canvas.findByRole("heading", { name: /Try .* in iMessage Photon/ })).toBeVisible();
|
||||
await userEvent.click(canvas.getByRole("button", { name: "Simulate incoming iMessage" }));
|
||||
await userEvent.click(await canvas.findByRole("button", { name: "Review identity access" }));
|
||||
await expect(await canvas.findByRole("heading", { name: "External identity access" })).toBeVisible();
|
||||
await userEvent.click(canvas.getByRole("button", { name: "Simulate identity confirmation" }));
|
||||
await userEvent.click(canvas.getByRole("button", { name: "Return to setup" }));
|
||||
await userEvent.click(canvas.getByRole("button", { name: "Simulate incoming iMessage" }));
|
||||
await userEvent.click(await canvas.findByRole("button", { name: "I've sent the test message" }));
|
||||
await expect(await canvas.findByText(/Shared Photon project · direct messages only/)).toBeVisible();
|
||||
await userEvent.click(canvas.getByRole("button", { name: "View simulated task" }));
|
||||
await waitFor(() => expect(canvas.getByText("Sent from iMessage", { exact: false })).toBeVisible());
|
||||
} };
|
||||
export const NarrowMobile: Story = { ...IncomingFollowUps, globals: { viewport: { value: "mobile1", isRotated: false } } };
|
||||
Loading…
Reference in New Issue