feat(stats): post the Discord stats reports in Spanish, opened by an AI summary (#752)
## What
The admin Discord channel is Spanish, but the four scheduled `stats:*`
reports posted English. They now post Spanish, and the three
cohort/experiment reports open with a short AI-written summary so a
reader understands the situation without decoding the table.
### 1. Spanish reports
`stats:daily-report`, `stats:subscription-funnel`,
`stats:experiment-funnel` and `stats:ai-cohort-report` now post Spanish
embed titles, field names, ASCII table headers, legends and disclaimers,
with Spanish dates (`sáb., 13 jun. 2026`). Table headers and cells stay
ASCII (`Semana`, `Variante`, `UMad`, `pdte`, …) so `sprintf`'s byte
padding keeps the columns aligned inside the code block.
Hardcoded, not `__()`: this is an internal channel, not user-facing UI,
so it never needs a second language and doesn't belong in
`lang/es.json`. Everything else — code, comments, PHPDoc, command
descriptions, `$this->info()` — stays English.
### 2. AI summary (best-effort)
New `ReportSummarizer` + `ReportSummaryAgent` (laravel/ai, same
Gemini-Flash pattern and `AI_PROVIDER` switch as the other AI features,
config in `config/ai_reports.php`). It prepends a few sentences to the
embed description that compare the current period against the previous
one (week over week for the two weekly reports, month over month for the
monthly cohort one), say what got better or worse, and call out
explicitly when a figure isn't conclusive — small sample, immature
cohort, signup surge week, or no previous period.
- **Data**: only what each collector already computes. The two cohort
reports pass their weekly series with the per-metric maturity flags; the
experiment report passes the per-variant figures plus the
already-rendered significance verdict (one source of truth), with money
figures nulled exactly where the table renders `—`, so the summary can't
report a zero where the reader sees no data.
- **Previous period**: the experiment report has no time series, so each
run caches its figures (with a capture timestamp) as the next run's
baseline. A same-day manual re-run doesn't overwrite it, and the model
is told to flag a gap that isn't roughly one period.
- **Degrades safely**: no API key, a provider outage, a bad provider
name or a slow response (30s timeout) → the report is posted unchanged,
without the summary. Transient provider errors are logged; anything else
is reported, matching `CategorizeTransactions` /
`LaravelAiRuleSuggestionGenerator`.
### 3. Fix found while reviewing: Discord's embed limits
The translated "Cómo leerlo" field came out at 1152 characters, past
Discord's 1024-character cap on a field value — Discord rejects the
**whole** payload, so the experiment funnel would have silently stopped
appearing in the channel every Monday (`DiscordWebhook` only logs the
400). Both long fields are now tighter (745 and 898 chars),
`DiscordWebhook` trims anything still over the limit instead of losing
the report, and a test measures every scheduled report's embed so
growing copy fails in CI rather than in production.
## Testing
- `tests/Feature/Ai/ReportSummarizerTest.php`: baseline in/out, same-day
re-run, dry run, backtick stripping, truncation, empty answer,
transient-vs-reported failures.
- `tests/Feature/DiscordReportEmbedLimitsTest.php`: all four embeds
inside Discord's limits, plus the trimming fallback.
- The four command tests: Spanish assertions, summary is the first thing
in the description, and the report is still posted when the AI throws.
- 64 tests / 338 assertions pass locally, plus PHPStan and Pint.
## QA
Ran all four commands against the local DB with `--no-discord` (and
dumped the real webhook payloads with the Discord call faked). Real
Gemini output, e.g. the experiment funnel on the second run:
> No se ha producido ningún cambio en las métricas del experimento
respecto a la ejecución previa del 10 de agosto de 2026. La tasa de
conversión sobre usuarios maduros se mantiene en el 4,7 % para la
variante reducida, el 4,0 % para pay_now y el 3,7 % para el control. Las
cifras no son conclusivas ya que las diferencias entre variantes no son
estadísticamente significativas, con un p-valor de 0,593 frente al
umbral α de 0,017.
Also verified with an unreachable provider: the report renders in full,
without the summary.
## Not included
- `stats:stuck-cohort-report` is still English. It posts to the same
webhook but isn't scheduled in `routes/console.php`, so it was out of
the four active reports; worth its own decision.
- The `—` and `⚡` cells are still a couple of bytes off inside the code
block (multibyte in `sprintf`). Pre-existing on `main`, unchanged here.
This commit is contained in:
Víctor Falcón
GitHub
parent
da9032a76e
commit
fe747c4472
|
|
@ -151,6 +151,7 @@ DEMO_ENCRYPTION_KEY=demo
|
|||
# AI_PROVIDER=gemini
|
||||
# AI_SUGGESTIONS_PROVIDER=gemini
|
||||
# AI_CATEGORIZATION_PROVIDER=gemini
|
||||
# AI_REPORTS_PROVIDER=gemini
|
||||
|
||||
# --- Gemini (default provider) ---
|
||||
GEMINI_API_KEY=
|
||||
|
|
@ -185,6 +186,12 @@ AI_SUGGESTIONS_REPORT_WEEKS=16
|
|||
# account that plants the release-anchor consent on deploy).
|
||||
AI_SUGGESTIONS_REPORT_EXCLUDED_EMAILS=
|
||||
|
||||
# AI summary opening the scheduled stats reports (subscription funnel,
|
||||
# experiment funnel and AI cohort). Best-effort: without a key, or if the
|
||||
# provider fails or is slow, the report is still posted without the summary.
|
||||
# AI_REPORTS_MODEL=gemini-flash-latest
|
||||
# AI_REPORTS_TIMEOUT=30
|
||||
|
||||
# Discord webhooks
|
||||
DISCORD_WEBHOOK_URL=
|
||||
# Optional dedicated channel for the AI cohort report; falls back to DISCORD_WEBHOOK_URL.
|
||||
|
|
|
|||
|
|
@ -190,8 +190,11 @@ unknown or non-text provider fails fast when the AI feature runs.
|
|||
| `AI_PROVIDER` | `gemini` | Provider for all AI features. Set once to switch everything. |
|
||||
| `AI_SUGGESTIONS_PROVIDER` | `AI_PROVIDER` | Override the provider for rule suggestions only. |
|
||||
| `AI_CATEGORIZATION_PROVIDER` | `AI_PROVIDER` | Override the provider for transaction categorization only. |
|
||||
| `AI_REPORTS_PROVIDER` | `AI_PROVIDER` | Override the provider for the stats-report summaries only. |
|
||||
| `AI_SUGGESTIONS_MODEL` | `gemini-flash-latest`| Model used for rule suggestions. |
|
||||
| `AI_CATEGORIZATION_MODEL` | `gemini-flash-latest`| Model used for transaction categorization. |
|
||||
| `AI_REPORTS_MODEL` | `gemini-flash-latest`| Model used for the stats-report summaries. |
|
||||
| `AI_REPORTS_TIMEOUT` | `30` | Seconds before a report is posted without its AI summary. |
|
||||
| `GEMINI_API_KEY` | - | Required when the provider is `gemini`. |
|
||||
| `OLLAMA_URL` | `http://localhost:11434` | Ollama server URL (used when the provider is `ollama`). |
|
||||
| `OLLAMA_API_KEY` | - | Optional; only needed behind an authenticating proxy. |
|
||||
|
|
|
|||
|
|
@ -0,0 +1,51 @@
|
|||
<?php
|
||||
|
||||
namespace App\Ai\Agents;
|
||||
|
||||
use Laravel\Ai\Contracts\Agent;
|
||||
use Laravel\Ai\Promptable;
|
||||
use Stringable;
|
||||
|
||||
/**
|
||||
* Writes the short summary that opens a scheduled stats report on Discord. The
|
||||
* answer is Spanish because the admin channel is Spanish; the instructions stay
|
||||
* English like the rest of the codebase. Callers pass the report-specific
|
||||
* context: what the report measures and which period it is compared against.
|
||||
*/
|
||||
class ReportSummaryAgent implements Agent
|
||||
{
|
||||
use Promptable;
|
||||
|
||||
public function __construct(private string $context) {}
|
||||
|
||||
public function instructions(): Stringable|string
|
||||
{
|
||||
return <<<PROMPT
|
||||
You are the data analyst of a personal-finance app. You write the opening
|
||||
summary of an internal report posted to an admin channel, so that a reader
|
||||
understands the situation without having to interpret the table below it.
|
||||
|
||||
{$this->context}
|
||||
|
||||
You are given a JSON object with "current" (the figures of the report about
|
||||
to be posted), "previous" (the same report measured on an earlier run, null
|
||||
when there is none yet) and "previous_captured_at" (when that earlier run
|
||||
happened). Rows are keyed by period and freeze once they are scored, so a
|
||||
row that went from null to a value in "previous" has simply matured — that
|
||||
is not an improvement. Compare like-for-like periods only, and if
|
||||
"previous_captured_at" is not roughly one period back, say so.
|
||||
|
||||
Rules:
|
||||
- Answer in Spanish (Spain), as plain text. No markdown, no bullet lists,
|
||||
no emoji, no greeting, no sign-off, and never restate the whole table.
|
||||
- At most 4 short sentences, and every sentence must carry information. No
|
||||
filler, no flourish, no advice nobody asked for.
|
||||
- Say what changed against the previous period, whether it is getting
|
||||
better or worse, and the most likely reason.
|
||||
- Say explicitly when a figure is not conclusive: small sample, immature
|
||||
cohort, signup surge week, or no previous period to compare against.
|
||||
- Use only the figures in the payload. Never invent a metric or a number;
|
||||
when something cannot be derived from the payload, say so instead.
|
||||
PROMPT;
|
||||
}
|
||||
}
|
||||
|
|
@ -4,6 +4,7 @@ namespace App\Console\Commands;
|
|||
|
||||
use App\Console\Commands\Concerns\RendersReportToConsole;
|
||||
use App\Services\Ai\AiCohortReportCollector;
|
||||
use App\Services\Ai\ReportSummarizer;
|
||||
use App\Services\Discord\DiscordWebhook;
|
||||
use Carbon\CarbonImmutable;
|
||||
use Illuminate\Console\Command;
|
||||
|
|
@ -16,8 +17,24 @@ class SendAiCohortReportCommand extends Command
|
|||
|
||||
protected $description = 'Post the weekly AI-suggestions cohort retention/conversion report to Discord';
|
||||
|
||||
public function __construct(private AiCohortReportCollector $collector)
|
||||
{
|
||||
/**
|
||||
* What the AI summary is looking at, and which periods it must compare.
|
||||
*/
|
||||
private const SUMMARY_CONTEXT = <<<'CONTEXT'
|
||||
The report tracks the weekly cohorts of users eligible for the AI suggestions
|
||||
feature (retention, trial, paid and AI-consent rates): one row per signup
|
||||
week, every metric measured at the same cohort age, with "phase" telling
|
||||
whether the cohort signed up before or after the feature was released. Each
|
||||
rate is null until its own horizon has elapsed: "retention_mature" governs
|
||||
retained_rate and trial_rate, "paid_mature" governs paid_rate. The report is
|
||||
posted on the first day of each month, so compare the last four weeks against
|
||||
the four before them — month against previous month, not week against week.
|
||||
CONTEXT;
|
||||
|
||||
public function __construct(
|
||||
private AiCohortReportCollector $collector,
|
||||
private ReportSummarizer $summarizer,
|
||||
) {
|
||||
parent::__construct();
|
||||
}
|
||||
|
||||
|
|
@ -32,7 +49,15 @@ class SendAiCohortReportCommand extends Command
|
|||
$weeks = $this->option('weeks') !== null ? (int) $this->option('weeks') : null;
|
||||
|
||||
$report = $this->collector->collect($weeks);
|
||||
$embed = $this->buildEmbed($report);
|
||||
|
||||
$summary = $this->summarizer->summarize(
|
||||
'ai-cohort-report',
|
||||
self::SUMMARY_CONTEXT,
|
||||
$this->summaryPayload($report),
|
||||
remember: ! $this->option('no-discord'),
|
||||
);
|
||||
|
||||
$embed = $this->buildEmbed($report, $summary);
|
||||
|
||||
if ($this->option('no-discord')) {
|
||||
$this->printEmbeds([$embed]);
|
||||
|
|
@ -52,12 +77,37 @@ class SendAiCohortReportCommand extends Command
|
|||
}
|
||||
|
||||
/**
|
||||
* The figures the AI summary may talk about — the same ones the table shows.
|
||||
*
|
||||
* @param array{releaseAt: ?CarbonImmutable, releaseWeek: ?string, weeks: list<array<string, mixed>>} $report
|
||||
* @return array<string, mixed>
|
||||
*/
|
||||
private function buildEmbed(array $report): array
|
||||
private function summaryPayload(array $report): array
|
||||
{
|
||||
$lines = [sprintf('%-9s %5s %6s %6s %6s %5s', 'Week', 'Elig', 'Ret', 'Trial', 'Paid', 'AI')];
|
||||
return [
|
||||
'release_week' => $report['releaseWeek'],
|
||||
'weeks' => array_map(fn (array $row): array => [
|
||||
'week' => $row['week'],
|
||||
'eligible' => $row['eligible'],
|
||||
'retained_rate' => $row['retainedRate'],
|
||||
'trial_rate' => $row['trialRate'],
|
||||
'paid_rate' => $row['paidRate'],
|
||||
'ai_accepted_rate' => $row['aiAcceptedRate'],
|
||||
'phase' => $row['phase'],
|
||||
'retention_mature' => $row['retentionMature'],
|
||||
'paid_mature' => $row['paidMature'],
|
||||
'signup_surge' => $row['surge'],
|
||||
], $report['weeks']),
|
||||
];
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array{releaseAt: ?CarbonImmutable, releaseWeek: ?string, weeks: list<array<string, mixed>>} $report
|
||||
* @return array<string, mixed>
|
||||
*/
|
||||
private function buildEmbed(array $report, ?string $summary): array
|
||||
{
|
||||
$lines = [sprintf('%-9s %5s %6s %6s %6s %5s', 'Semana', 'Eleg', 'Ret', 'Prueba', 'Pago', 'IA')];
|
||||
|
||||
foreach ($report['weeks'] as $row) {
|
||||
$flags = '';
|
||||
|
|
@ -83,27 +133,29 @@ class SendAiCohortReportCommand extends Command
|
|||
}
|
||||
|
||||
$release = $report['releaseAt'] !== null
|
||||
? 'First AI consent (release anchor): '.$report['releaseAt']->format('D, d M Y').' · week '.$report['releaseWeek'].' 🚀'
|
||||
: 'No AI consent recorded yet — feature not live in production.';
|
||||
? 'Primer consentimiento de IA (ancla de lanzamiento): '.$report['releaseAt']->copy()->locale('es')->translatedFormat('D, d M Y').' · semana '.$report['releaseWeek'].' 🚀'
|
||||
: 'Todavía no hay ningún consentimiento de IA — la función no está activa en producción.';
|
||||
|
||||
$table = "```\n".implode("\n", $lines)."\n```";
|
||||
|
||||
return [
|
||||
'title' => '🤖 AI Suggestions — Weekly Cohort Report',
|
||||
'description' => "```\n".implode("\n", $lines)."\n```",
|
||||
'title' => '🤖 Sugerencias con IA — informe de cohortes semanales',
|
||||
'description' => $summary !== null ? $summary."\n\n".$table : $table,
|
||||
'color' => 0x5865F2,
|
||||
'fields' => [
|
||||
[
|
||||
'name' => 'Release anchor',
|
||||
'name' => 'Ancla de lanzamiento',
|
||||
'value' => $release,
|
||||
'inline' => false,
|
||||
],
|
||||
[
|
||||
'name' => 'Legend',
|
||||
'value' => 'Elig = users with ≥50 transactions in their first 7 days · Ret = active ≥14d after signup · Trial = subscribed ≤14d · Paid = active subscription ≤30d · AI = accepted AI consent · `pend` = cohort too young to score · ⚡ = signup surge',
|
||||
'name' => 'Leyenda',
|
||||
'value' => 'Eleg = usuarios con ≥50 transacciones en sus primeros 7 días · Ret = activos ≥14d después de registrarse · Prueba = se suscribieron ≤14d · Pago = suscripción activa ≤30d · IA = aceptaron el consentimiento de IA · `pdte` = cohorte demasiado joven para puntuarla · ⚡ = pico de registros',
|
||||
'inline' => false,
|
||||
],
|
||||
[
|
||||
'name' => '⚠️ Directional only',
|
||||
'value' => 'Pre/post comparison, not a randomised test. Cohorts are compared at equal age. Surge weeks (⚡, e.g. launch/YouTube) differ in acquisition channel and are not controlled — compare organic weeks like-for-like. Confidence builds over a quarter, not a single month.',
|
||||
'name' => '⚠️ Solo orientativo',
|
||||
'value' => 'Es una comparación antes/después, no un test aleatorizado. Las cohortes se comparan a la misma edad. Las semanas con pico (⚡, p. ej. lanzamiento o YouTube) llegan por otro canal de adquisición y no están controladas — compara semanas orgánicas entre sí. La confianza se construye a lo largo de un trimestre, no en un solo mes.',
|
||||
'inline' => false,
|
||||
],
|
||||
],
|
||||
|
|
@ -117,7 +169,7 @@ class SendAiCohortReportCommand extends Command
|
|||
}
|
||||
|
||||
if (! $mature || $rate === null) {
|
||||
return 'pend';
|
||||
return 'pdte';
|
||||
}
|
||||
|
||||
return ((int) round($rate * 100)).'%';
|
||||
|
|
|
|||
|
|
@ -79,8 +79,8 @@ class SendDailyStatsReportCommand extends Command
|
|||
{
|
||||
$fields = [
|
||||
[
|
||||
'name' => '👥 Users',
|
||||
'value' => "New yesterday: **{$newUsers}**\nTotal: **{$totalUsers}**",
|
||||
'name' => '👥 Usuarios',
|
||||
'value' => "Nuevos ayer: **{$newUsers}**\nTotal: **{$totalUsers}**",
|
||||
'inline' => false,
|
||||
],
|
||||
];
|
||||
|
|
@ -98,10 +98,10 @@ class SendDailyStatsReportCommand extends Command
|
|||
$fields[] = [
|
||||
'name' => '💳 '.strtoupper($currency),
|
||||
'value' => implode("\n", [
|
||||
"Active: **{$active['count']}** ({$this->money($currentMrr, $currency)} MRR)",
|
||||
"Trialing: **{$trialing['count']}** ({$this->money($trialing['mrr'], $currency)} MRR)",
|
||||
"Current MRR/ARR: **{$this->money($currentMrr, $currency)}** / **{$this->money($currentMrr * 12, $currency)}**",
|
||||
"Projected MRR/ARR: **{$this->money($projectedMrr, $currency)}** / **{$this->money($projectedMrr * 12, $currency)}**",
|
||||
"Activas: **{$active['count']}** ({$this->money($currentMrr, $currency)} de MRR)",
|
||||
"En prueba: **{$trialing['count']}** ({$this->money($trialing['mrr'], $currency)} de MRR)",
|
||||
"MRR/ARR actual: **{$this->money($currentMrr, $currency)}** / **{$this->money($currentMrr * 12, $currency)}**",
|
||||
"MRR/ARR previsto: **{$this->money($projectedMrr, $currency)}** / **{$this->money($projectedMrr * 12, $currency)}**",
|
||||
]),
|
||||
'inline' => false,
|
||||
];
|
||||
|
|
@ -109,14 +109,14 @@ class SendDailyStatsReportCommand extends Command
|
|||
|
||||
if ($currencies === []) {
|
||||
$fields[] = [
|
||||
'name' => '💳 Subscriptions',
|
||||
'value' => 'No active or trialing subscriptions.',
|
||||
'name' => '💳 Suscripciones',
|
||||
'value' => 'No hay suscripciones activas ni en prueba.',
|
||||
'inline' => false,
|
||||
];
|
||||
}
|
||||
|
||||
return [
|
||||
'title' => '📊 Daily Stats — '.$day->format('D, d M Y'),
|
||||
'title' => '📊 Estadísticas diarias — '.$day->copy()->locale('es')->translatedFormat('D, d M Y'),
|
||||
'color' => 0x5865F2,
|
||||
'fields' => $fields,
|
||||
];
|
||||
|
|
|
|||
|
|
@ -3,6 +3,7 @@
|
|||
namespace App\Console\Commands;
|
||||
|
||||
use App\Features\SubscriptionExperiment;
|
||||
use App\Services\Ai\ReportSummarizer;
|
||||
use App\Services\Discord\DiscordWebhook;
|
||||
use App\Services\Stats\BinomialProportion;
|
||||
use App\Services\Stats\ExperimentFunnelCollector;
|
||||
|
|
@ -15,7 +16,7 @@ class SendExperimentFunnelReportCommand extends Command
|
|||
{
|
||||
protected $signature = 'stats:experiment-funnel
|
||||
{--no-discord : Print the report to the console only, without posting to Discord}
|
||||
{--cost-per-connection=0.4 : Estimated cost (in the Cashier currency) per bank connection, used for the Cost/Burn/CM columns}';
|
||||
{--cost-per-connection=0.4 : Estimated cost (in the Cashier currency) per bank connection, used for the cost, burn and contribution-margin columns}';
|
||||
|
||||
protected $description = 'Post the trial/pricing experiment funnel (per variant) to Discord';
|
||||
|
||||
|
|
@ -25,9 +26,25 @@ class SendExperimentFunnelReportCommand extends Command
|
|||
SubscriptionExperiment::PAY_NOW => 'pay_now',
|
||||
];
|
||||
|
||||
/**
|
||||
* What the AI summary is looking at, and which periods it must compare.
|
||||
*/
|
||||
private const SUMMARY_CONTEXT = <<<'CONTEXT'
|
||||
The report is the A/B/C trial-and-pricing experiment: one row per variant
|
||||
(control, reduced, pay_now), cumulative since the experiment started. Conv%
|
||||
(conversions over matured users) and ARPU are the comparable metrics; the
|
||||
absolute MRR, cost and margin totals scale with how many users have matured,
|
||||
which differs per variant by design. Monetary amounts are in cents of the
|
||||
report currency, and null means there is no data yet rather than zero. The
|
||||
report is posted every Monday: compare against the previous run to say what
|
||||
moved this week, and never call a winner the significance block does not
|
||||
support.
|
||||
CONTEXT;
|
||||
|
||||
public function __construct(
|
||||
private ExperimentFunnelCollector $collector,
|
||||
private ProportionSignificance $significance,
|
||||
private ReportSummarizer $summarizer,
|
||||
) {
|
||||
parent::__construct();
|
||||
}
|
||||
|
|
@ -49,6 +66,18 @@ class SendExperimentFunnelReportCommand extends Command
|
|||
return self::SUCCESS;
|
||||
}
|
||||
|
||||
$summary = $this->summarizer->summarize(
|
||||
'experiment-funnel',
|
||||
self::SUMMARY_CONTEXT,
|
||||
$this->summaryPayload($report),
|
||||
remember: ! $this->option('no-discord'),
|
||||
);
|
||||
|
||||
if ($summary !== null) {
|
||||
$this->line($summary);
|
||||
$this->newLine();
|
||||
}
|
||||
|
||||
foreach ($this->tableLines($report) as $line) {
|
||||
$this->line($line);
|
||||
}
|
||||
|
|
@ -66,7 +95,7 @@ class SendExperimentFunnelReportCommand extends Command
|
|||
$webhookUrl = config('services.discord.ai_cohort_webhook_url')
|
||||
?: config('services.discord.webhook_url');
|
||||
|
||||
(new DiscordWebhook($webhookUrl))->send('', [$this->buildEmbed($report)]);
|
||||
(new DiscordWebhook($webhookUrl))->send('', [$this->buildEmbed($report, $summary)]);
|
||||
|
||||
$this->info('Experiment funnel report sent to Discord.');
|
||||
|
||||
|
|
@ -83,7 +112,7 @@ class SendExperimentFunnelReportCommand extends Command
|
|||
$currency = $report['currency'];
|
||||
$lines = [sprintf(
|
||||
'%-8s %5s %5s %5s %5s %5s %6s %7s %7s %7s %7s %7s',
|
||||
'Variant', 'Assg', 'Actd', 'Card', 'MatU', 'Conv', 'Conv%', 'ARPU', 'MRR', 'Cost', 'Burn', 'CM',
|
||||
'Variante', 'Asig', 'Actv', 'Tarj', 'UMad', 'Conv', 'Conv%', 'ARPU', 'MRR', 'Coste', 'Quema', 'MC',
|
||||
)];
|
||||
|
||||
foreach (self::LABELS as $key => $label) {
|
||||
|
|
@ -99,7 +128,7 @@ class SendExperimentFunnelReportCommand extends Command
|
|||
$row['subscribed'],
|
||||
$row['assignedMature'],
|
||||
$row['convertedMature'],
|
||||
$mature ? ((int) round($row['conversionRate'] * 100)).'%' : 'pend',
|
||||
$mature ? ((int) round($row['conversionRate'] * 100)).'%' : 'pdte',
|
||||
$showMoney && $row['arpuCents'] !== null ? Money::format($row['arpuCents'], $currency) : '—',
|
||||
$showMoney ? Money::format($row['mrrCents'], $currency) : '—',
|
||||
$mature ? Money::format($row['costCents'], $currency) : '—',
|
||||
|
|
@ -122,7 +151,7 @@ class SendExperimentFunnelReportCommand extends Command
|
|||
*/
|
||||
private function significanceLines(array $report): array
|
||||
{
|
||||
$lines = ['', 'Significance (95% Wilson CI on Conv%, n = MatU):'];
|
||||
$lines = ['', 'Significancia (IC de Wilson al 95% sobre Conv%, n = UMad):'];
|
||||
$arms = [];
|
||||
|
||||
foreach (self::LABELS as $key => $label) {
|
||||
|
|
@ -131,7 +160,7 @@ class SendExperimentFunnelReportCommand extends Command
|
|||
$k = (int) $row['convertedMature'];
|
||||
|
||||
if ($n <= 0) {
|
||||
$lines[] = sprintf(' %-8s pend (n=0)', $label);
|
||||
$lines[] = sprintf(' %-8s pdte (n=0)', $label);
|
||||
|
||||
continue;
|
||||
}
|
||||
|
|
@ -142,7 +171,7 @@ class SendExperimentFunnelReportCommand extends Command
|
|||
}
|
||||
|
||||
if (count($arms) < 2) {
|
||||
$lines[] = 'Not enough matured variants to compare yet.';
|
||||
$lines[] = 'Aún no hay suficientes variantes maduras para comparar.';
|
||||
|
||||
return $lines;
|
||||
}
|
||||
|
|
@ -152,20 +181,20 @@ class SendExperimentFunnelReportCommand extends Command
|
|||
$result = $this->significance->compare($leader, $runnerUp);
|
||||
|
||||
$lines[] = sprintf(
|
||||
'Leader %s vs %s: Δ %+.1f pts (95%% CI %+.1f … %+.1f pts, Newcombe).',
|
||||
'Líder %s vs %s: Δ %+.1f pts (IC 95%% %+.1f … %+.1f pts, Newcombe).',
|
||||
$leader->label, $runnerUp->label,
|
||||
($leader->rate() - $runnerUp->rate()) * 100, $result['diffLow'] * 100, $result['diffHigh'] * 100,
|
||||
);
|
||||
$lines[] = sprintf(
|
||||
'Fisher exact p=%.3f %s α=%.3f (Bonferroni×3) -> %s.%s',
|
||||
'Test exacto de Fisher p=%.3f %s α=%.3f (Bonferroni×3) -> %s.%s',
|
||||
$result['fisherP'], $result['significant'] ? '<' : '≥', $result['alpha'],
|
||||
$result['significant'] ? 'significant' : 'not significant',
|
||||
$result['significant'] ? '' : ' Keep running.',
|
||||
$result['significant'] ? 'significativo' : 'no significativo',
|
||||
$result['significant'] ? '' : ' Hay que seguir midiendo.',
|
||||
);
|
||||
|
||||
if ($result['minExpectedCount'] < 5.0) {
|
||||
$lines[] = sprintf(
|
||||
'(Small sample: min expected conversions %.1f < 5, so the normal-approx z=%.2f overstates — exact test used.)',
|
||||
'(Muestra pequeña: el mínimo de conversiones esperadas es %.1f < 5, así que la aproximación normal z=%.2f sobreestima — se usa el test exacto.)',
|
||||
$result['minExpectedCount'], $result['z'],
|
||||
);
|
||||
}
|
||||
|
|
@ -179,37 +208,84 @@ class SendExperimentFunnelReportCommand extends Command
|
|||
}
|
||||
|
||||
/**
|
||||
* The figures the AI summary may talk about — the same ones the table shows,
|
||||
* nulled under exactly the conditions that render them as "—", so the summary
|
||||
* can't claim a zero where the reader sees no data. The significance verdict
|
||||
* is handed over as the rendered lines, keeping one source of truth for it.
|
||||
*
|
||||
* @param array{startedAt: ?CarbonImmutable, currency: string, revenueAvailable: bool, costPerConnectionCents: int, variants: array<string, array<string, mixed>>} $report
|
||||
* @return array<string, mixed>
|
||||
*/
|
||||
private function buildEmbed(array $report): array
|
||||
private function summaryPayload(array $report): array
|
||||
{
|
||||
$variants = [];
|
||||
|
||||
foreach (self::LABELS as $key => $label) {
|
||||
$row = $report['variants'][$key];
|
||||
$mature = $row['assignedMature'] > 0;
|
||||
$showMoney = $report['revenueAvailable'] && $mature;
|
||||
|
||||
$variants[$label] = [
|
||||
'assigned' => $row['assigned'],
|
||||
'activated' => $row['activated'],
|
||||
'carded' => $row['subscribed'],
|
||||
'matured_users' => $row['assignedMature'],
|
||||
'converted_mature' => $row['convertedMature'],
|
||||
'conversion_rate' => $row['conversionRate'],
|
||||
'arpu_cents' => $showMoney ? $row['arpuCents'] : null,
|
||||
'mrr_cents' => $showMoney ? $row['mrrCents'] : null,
|
||||
'cost_cents' => $mature ? $row['costCents'] : null,
|
||||
'wasted_cost_cents' => $mature ? $row['wastedCostCents'] : null,
|
||||
'contribution_margin_cents' => $showMoney ? $row['contributionMarginCents'] : null,
|
||||
];
|
||||
}
|
||||
|
||||
return [
|
||||
'title' => '🧪 Trial/Pricing Experiment — Funnel by Variant',
|
||||
'description' => "```\n".implode("\n", $this->tableLines($report))."\n```",
|
||||
'currency' => $report['currency'],
|
||||
'revenue_available' => $report['revenueAvailable'],
|
||||
'started_at' => $report['startedAt']?->toDateString(),
|
||||
'variants' => $variants,
|
||||
'significance' => array_values(array_filter($this->significanceLines($report))),
|
||||
];
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array{startedAt: ?CarbonImmutable, currency: string, revenueAvailable: bool, costPerConnectionCents: int, variants: array<string, array<string, mixed>>} $report
|
||||
* @return array<string, mixed>
|
||||
*/
|
||||
private function buildEmbed(array $report, ?string $summary): array
|
||||
{
|
||||
$table = "```\n".implode("\n", $this->tableLines($report))."\n```";
|
||||
|
||||
return [
|
||||
'title' => '🧪 Experimento de prueba/precio — embudo por variante',
|
||||
'description' => $summary !== null ? $summary."\n\n".$table : $table,
|
||||
'color' => 0xFEE75C,
|
||||
'fields' => [
|
||||
[
|
||||
'name' => 'Started',
|
||||
'value' => $report['startedAt']->format('D, d M Y').' · new signups split evenly into the three variants.',
|
||||
'name' => 'Inicio',
|
||||
'value' => $report['startedAt']->copy()->locale('es')->translatedFormat('D, d M Y').' · los nuevos registros se reparten a partes iguales entre las tres variantes.',
|
||||
'inline' => false,
|
||||
],
|
||||
[
|
||||
'name' => '📊 Significance',
|
||||
'name' => '📊 Significancia',
|
||||
'value' => "```\n".implode("\n", $this->significanceLines($report))."\n```",
|
||||
'inline' => false,
|
||||
],
|
||||
[
|
||||
'name' => 'Legend',
|
||||
// Keep this and "Cómo leerlo" tight: they are the only fields
|
||||
// anywhere near Discord's 1024-character limit per field,
|
||||
// beyond which DiscordWebhook has to trim them.
|
||||
'name' => 'Leyenda',
|
||||
'value' => sprintf(
|
||||
'Assg = signups · Actd = activated (connected a bank or enabled AI = cost triggered) · Card = completed checkout (card on file) · MatU = matured assigned (cohort old enough to score for this variant) · Conv = matured users who ever converted (were charged, net of refund) — time-invariant, so it does not shrink as an older cohort has longer to churn · Conv%% = Conv ÷ MatU (always ≤100%%, comparable across variants) · ARPU = MRR ÷ MatU (revenue per matured user) · MRR = monthly run-rate of *currently* paying subs (yearly ÷ 12); Conv above MRR is churn · Cost = est. connection cost of MatU (%s/connection) · Burn = connection cost of matured users who never earned net revenue (connected a bank but never paid, or paid then refunded) · CM = MRR − Cost · `pend`/`—` = no matured data yet.',
|
||||
'Asig = registros · Actv = activados (conectaron un banco o activaron la IA = coste disparado) · Tarj = checkout completado (tarjeta guardada) · UMad = asignados maduros (cohorte con edad para puntuarla en esta variante) · Conv = maduros que llegaron a convertir (se les cobró, menos devoluciones); no depende del momento, así que no baja porque una cohorte antigua haya tenido más tiempo para cancelar · Conv%% = Conv ÷ UMad · ARPU = MRR ÷ UMad · MRR = ritmo mensual de quienes pagan *ahora* (anuales ÷ 12); si Conv va por encima del MRR, es churn · Coste = coste estimado de conexiones de UMad (%s por conexión) · Quema = coste de conexión de maduros que nunca dejaron ingreso neto · MC = MRR − Coste · `pdte`/`—` = todavía sin datos maduros.',
|
||||
Money::format($report['costPerConnectionCents'], $report['currency']),
|
||||
),
|
||||
'inline' => false,
|
||||
],
|
||||
[
|
||||
'name' => '⚠️ How to read it',
|
||||
'value' => 'Each variant matures on its own decision window (control 15d, reduced 7d, pay_now 3d, +3d settle), so at any moment MatU differs a lot between variants (pay_now matures first). **Compare variants on Conv% and ARPU — normalized per matured user — not on the absolute MRR/Cost/Burn/CM totals, which scale with MatU and so mechanically favour whichever variant has matured more.** Assg/Actd/Card are lifetime counts; everything from MatU rightward covers the matured cohort only, so the raw Actd→Card→Conv funnel mixes cohorts (immature carded users can\'t have matured yet) — read it for volume. Conv counts anyone ever charged (net of refund), so it is not depressed for older cohorts the way a live-active snapshot would be. Per-user CM is sub-cent at current volume, so treat CM as directional context, not the decision. Check significance (sample size = MatU) before calling a winner. Cost is a flat per-connection estimate across all providers, not per-provider billing.',
|
||||
'name' => '⚠️ Cómo leerlo',
|
||||
'value' => 'Cada variante madura con su propia ventana de decisión (control 15d, reduced 7d, pay_now 3d, +3d de liquidación), así que UMad difiere mucho entre variantes (pay_now madura antes). **Compara por Conv% y ARPU — normalizados por usuario maduro — y no por los totales de MRR/Coste/Quema/MC, que escalan con UMad y favorecen a la variante que más ha madurado.** Asig/Actv/Tarj son recuentos de toda la vida; de UMad hacia la derecha solo cuenta la cohorte madura, así que el embudo Actv→Tarj→Conv mezcla cohortes: léelo como volumen. Conv cuenta a quien se le cobró alguna vez, así que no se hunde en las cohortes antiguas como haría una foto de activos de hoy. El MC por usuario está por debajo del céntimo con este volumen: es contexto, no la decisión. Comprueba la significancia (n = UMad) antes de dar un ganador. El coste es una estimación plana por conexión, no la factura real de cada proveedor.',
|
||||
'inline' => false,
|
||||
],
|
||||
],
|
||||
|
|
|
|||
|
|
@ -2,6 +2,7 @@
|
|||
|
||||
namespace App\Console\Commands;
|
||||
|
||||
use App\Services\Ai\ReportSummarizer;
|
||||
use App\Services\Discord\DiscordWebhook;
|
||||
use App\Services\Stats\SubscriptionFunnelCollector;
|
||||
use Illuminate\Console\Command;
|
||||
|
|
@ -12,8 +13,23 @@ class SendSubscriptionFunnelReportCommand extends Command
|
|||
|
||||
protected $description = 'Post the weekly registration -> subscription -> paid funnel to Discord';
|
||||
|
||||
public function __construct(private SubscriptionFunnelCollector $collector)
|
||||
{
|
||||
/**
|
||||
* What the AI summary is looking at, and which periods it must compare.
|
||||
*/
|
||||
private const SUMMARY_CONTEXT = <<<'CONTEXT'
|
||||
The report is a weekly signup -> subscription -> paid funnel: one row per
|
||||
signup-week cohort, every stage measured at the same cohort age. Each rate is
|
||||
null until its own horizon has elapsed: "subscribed_mature" governs
|
||||
subscribed_rate, "paid_mature" governs paid_rate and trial_to_paid_rate. The
|
||||
report is posted every Monday, so compare the most recent week that is mature
|
||||
for a given rate against the one before it, and put it in the context of the
|
||||
trend across the mature weeks.
|
||||
CONTEXT;
|
||||
|
||||
public function __construct(
|
||||
private SubscriptionFunnelCollector $collector,
|
||||
private ReportSummarizer $summarizer,
|
||||
) {
|
||||
parent::__construct();
|
||||
}
|
||||
|
||||
|
|
@ -29,6 +45,18 @@ class SendSubscriptionFunnelReportCommand extends Command
|
|||
|
||||
$report = $this->collector->collect($weeks);
|
||||
|
||||
$summary = $this->summarizer->summarize(
|
||||
'subscription-funnel',
|
||||
self::SUMMARY_CONTEXT,
|
||||
$this->summaryPayload($report),
|
||||
remember: ! $this->option('no-discord'),
|
||||
);
|
||||
|
||||
if ($summary !== null) {
|
||||
$this->line($summary);
|
||||
$this->newLine();
|
||||
}
|
||||
|
||||
foreach ($this->tableLines($report) as $line) {
|
||||
$this->line($line);
|
||||
}
|
||||
|
|
@ -42,7 +70,7 @@ class SendSubscriptionFunnelReportCommand extends Command
|
|||
$webhookUrl = config('services.discord.ai_cohort_webhook_url')
|
||||
?: config('services.discord.webhook_url');
|
||||
|
||||
(new DiscordWebhook($webhookUrl))->send('', [$this->buildEmbed($report)]);
|
||||
(new DiscordWebhook($webhookUrl))->send('', [$this->buildEmbed($report, $summary)]);
|
||||
|
||||
$this->info('Subscription funnel report sent to Discord.');
|
||||
|
||||
|
|
@ -55,7 +83,7 @@ class SendSubscriptionFunnelReportCommand extends Command
|
|||
*/
|
||||
private function tableLines(array $report): array
|
||||
{
|
||||
$lines = [sprintf('%-9s %5s %5s %5s %5s %5s %5s', 'Week', 'Reg', 'Sub', 'Sub%', 'Paid', 'Pd%', 'T2P')];
|
||||
$lines = [sprintf('%-9s %5s %5s %5s %5s %5s %5s', 'Semana', 'Reg', 'Sub', 'Sub%', 'Pago', 'Pag%', 'P/S')];
|
||||
|
||||
foreach ($report['weeks'] as $row) {
|
||||
$lines[] = sprintf(
|
||||
|
|
@ -75,10 +103,35 @@ class SendSubscriptionFunnelReportCommand extends Command
|
|||
}
|
||||
|
||||
/**
|
||||
* The figures the AI summary may talk about — the same ones the table shows.
|
||||
*
|
||||
* @param array{trialDays: int, weeks: list<array<string, mixed>>} $report
|
||||
* @return array<string, mixed>
|
||||
*/
|
||||
private function buildEmbed(array $report): array
|
||||
private function summaryPayload(array $report): array
|
||||
{
|
||||
return [
|
||||
'trial_days' => $report['trialDays'],
|
||||
'weeks' => array_map(fn (array $row): array => [
|
||||
'week' => $row['week'],
|
||||
'registered' => $row['registered'],
|
||||
'subscribed' => $row['subscribed'],
|
||||
'subscribed_rate' => $row['subscribedRate'],
|
||||
'paid' => $row['paid'],
|
||||
'paid_rate' => $row['paidRate'],
|
||||
'trial_to_paid_rate' => $row['trialToPaidRate'],
|
||||
'subscribed_mature' => $row['subscribedMature'],
|
||||
'paid_mature' => $row['paidMature'],
|
||||
'signup_surge' => $row['surge'],
|
||||
], $report['weeks']),
|
||||
];
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array{trialDays: int, weeks: list<array<string, mixed>>} $report
|
||||
* @return array<string, mixed>
|
||||
*/
|
||||
private function buildEmbed(array $report, ?string $summary): array
|
||||
{
|
||||
$mature = array_values(array_filter($report['weeks'], fn (array $row): bool => $row['paidMature']));
|
||||
|
||||
|
|
@ -88,7 +141,7 @@ class SendSubscriptionFunnelReportCommand extends Command
|
|||
|
||||
$totals = $registered > 0
|
||||
? sprintf(
|
||||
"Registered %d\nSubscribed %d (%s%%)\nPaid %d (%s%% of reg · %s%% of subs)",
|
||||
"Registrados %d\nSuscritos %d (%s%%)\nPagando %d (%s%% de registros · %s%% de suscritos)",
|
||||
$registered,
|
||||
$subscribed,
|
||||
$this->pct($subscribed / $registered),
|
||||
|
|
@ -96,26 +149,28 @@ class SendSubscriptionFunnelReportCommand extends Command
|
|||
$this->pct($paid / $registered),
|
||||
$subscribed > 0 ? $this->pct($paid / $subscribed) : '—',
|
||||
)
|
||||
: 'No mature cohorts yet.';
|
||||
: 'Aún no hay cohortes maduras.';
|
||||
|
||||
$table = "```\n".implode("\n", $this->tableLines($report))."\n```";
|
||||
|
||||
return [
|
||||
'title' => '💸 Subscription Funnel — Weekly Cohorts',
|
||||
'description' => "```\n".implode("\n", $this->tableLines($report))."\n```",
|
||||
'title' => '💸 Embudo de suscripción — cohortes semanales',
|
||||
'description' => $summary !== null ? $summary."\n\n".$table : $table,
|
||||
'color' => 0x57F287,
|
||||
'fields' => [
|
||||
[
|
||||
'name' => 'Mature cohorts (baseline)',
|
||||
'name' => 'Cohortes maduras (referencia)',
|
||||
'value' => "```\n".$totals."\n```",
|
||||
'inline' => false,
|
||||
],
|
||||
[
|
||||
'name' => 'Legend',
|
||||
'value' => 'Reg = signups · Sub = started a plan ≤30d after signup · Paid = that plan billed past the '.$report['trialDays'].'d trial (active, or canceled only after billing) · Sub%/Pd% of signups · T2P = paid ÷ subscribed · `pend` = cohort too young to score · ⚡ = signup surge',
|
||||
'name' => 'Leyenda',
|
||||
'value' => 'Reg = registros · Sub = empezó un plan ≤30d después de registrarse · Pago = ese plan se cobró al pasar la prueba de '.$report['trialDays'].'d (activo, o cancelado solo después de cobrarse) · Sub%/Pag% sobre registros · P/S = pago ÷ suscritos · `pdte` = cohorte demasiado joven para puntuarla · ⚡ = pico de registros',
|
||||
'inline' => false,
|
||||
],
|
||||
[
|
||||
'name' => '⚠️ Directional only',
|
||||
'value' => 'Cohorts compared at equal age. Surge weeks (⚡, e.g. launch/marketing) differ in acquisition channel and are not controlled — compare organic weeks like-for-like. This is the pre-A/B baseline, not a randomised test.',
|
||||
'name' => '⚠️ Solo orientativo',
|
||||
'value' => 'Las cohortes se comparan a la misma edad. Las semanas con pico (⚡, p. ej. lanzamiento o marketing) llegan por otro canal de adquisición y no están controladas — compara semanas orgánicas entre sí. Esta es la referencia previa al A/B, no un test aleatorizado.',
|
||||
'inline' => false,
|
||||
],
|
||||
],
|
||||
|
|
@ -129,7 +184,7 @@ class SendSubscriptionFunnelReportCommand extends Command
|
|||
}
|
||||
|
||||
if (! $mature || $rate === null) {
|
||||
return 'pend';
|
||||
return 'pdte';
|
||||
}
|
||||
|
||||
return $this->pct($rate).'%';
|
||||
|
|
|
|||
|
|
@ -0,0 +1,109 @@
|
|||
<?php
|
||||
|
||||
namespace App\Services\Ai;
|
||||
|
||||
use App\Ai\Agents\ReportSummaryAgent;
|
||||
use Illuminate\Support\Facades\Cache;
|
||||
use Illuminate\Support\Facades\Log;
|
||||
use Illuminate\Support\Str;
|
||||
use Laravel\Ai\Enums\Lab;
|
||||
use Laravel\Ai\Exceptions\FailoverableException;
|
||||
use Throwable;
|
||||
|
||||
/**
|
||||
* Best-effort AI summary opening a scheduled stats report.
|
||||
*
|
||||
* The summary is a nice-to-have: a missing API key, a provider outage or a slow
|
||||
* response must never keep the report itself from being posted, so every failure
|
||||
* is swallowed and the caller simply gets null.
|
||||
*/
|
||||
class ReportSummarizer
|
||||
{
|
||||
/**
|
||||
* How long the previous run's figures are kept so the next run has a period
|
||||
* to compare against — comfortably longer than the monthly report's cadence.
|
||||
*
|
||||
* ponytail: the cache is enough for a directional summary; if a lost baseline
|
||||
* ever matters, persist the snapshots in a table instead.
|
||||
*/
|
||||
private const BASELINE_DAYS = 70;
|
||||
|
||||
/**
|
||||
* Hard bound (ellipsis included) on the summary length: a Discord embed
|
||||
* description is capped and the report table takes most of it, so a runaway
|
||||
* answer is trimmed rather than eating the table's room.
|
||||
*/
|
||||
private const MAX_SUMMARY_LENGTH = 900;
|
||||
|
||||
/**
|
||||
* @param string $reportKey identifies the report, so each keeps its own baseline
|
||||
* @param string $context what the report measures and which period to compare
|
||||
* @param array<string, mixed> $payload the figures the summary may talk about
|
||||
* @param bool $remember keep this payload as the baseline for the next run
|
||||
*/
|
||||
public function summarize(string $reportKey, string $context, array $payload, bool $remember = true): ?string
|
||||
{
|
||||
try {
|
||||
$previous = $this->baseline($reportKey, $payload, $remember);
|
||||
|
||||
$response = (new ReportSummaryAgent($context))->prompt(
|
||||
(string) json_encode([
|
||||
'current' => $payload,
|
||||
'previous' => $previous['payload'] ?? null,
|
||||
'previous_captured_at' => $previous['captured_at'] ?? null,
|
||||
], JSON_UNESCAPED_UNICODE | JSON_UNESCAPED_SLASHES),
|
||||
provider: Lab::from((string) config('ai_reports.provider')),
|
||||
model: (string) config('ai_reports.model'),
|
||||
timeout: (int) config('ai_reports.timeout'),
|
||||
);
|
||||
} catch (FailoverableException $exception) {
|
||||
// An overloaded or rate-limited provider is an expected transient
|
||||
// condition, not a bug, so it stays out of the error reports.
|
||||
Log::warning('Report AI summary skipped: provider transient failure.', [
|
||||
'report' => $reportKey,
|
||||
'exception' => $exception->getMessage(),
|
||||
]);
|
||||
|
||||
return null;
|
||||
} catch (Throwable $exception) {
|
||||
// Anything else (a misconfigured provider, an SDK change, a cache
|
||||
// outage) is a real bug: report it, but still let the report post.
|
||||
report($exception);
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
// The summary is prepended to a fenced table, so a stray backtick from
|
||||
// the model would swallow the table into its own code block.
|
||||
$summary = trim(str_replace('`', '', $response->text));
|
||||
|
||||
return $summary === '' ? null : Str::limit($summary, self::MAX_SUMMARY_LENGTH - 1, '…');
|
||||
}
|
||||
|
||||
/**
|
||||
* Read the previous run's figures and, unless this is a dry run, leave the
|
||||
* current ones behind as the next run's baseline. A same-day re-run keeps
|
||||
* the stored baseline, so re-running a report by hand can't overwrite the
|
||||
* period the next scheduled run needs to compare against.
|
||||
*
|
||||
* @param array<string, mixed> $payload
|
||||
* @return array{captured_at: string, payload: array<string, mixed>}|null
|
||||
*/
|
||||
private function baseline(string $reportKey, array $payload, bool $remember): ?array
|
||||
{
|
||||
$key = "report_summary_baseline:{$reportKey}";
|
||||
$previous = Cache::get($key);
|
||||
|
||||
$capturedToday = isset($previous['captured_at'])
|
||||
&& Str::startsWith($previous['captured_at'], now()->toDateString());
|
||||
|
||||
if ($remember && ! $capturedToday) {
|
||||
Cache::put($key, [
|
||||
'captured_at' => now()->toIso8601String(),
|
||||
'payload' => $payload,
|
||||
], now()->addDays(self::BASELINE_DAYS));
|
||||
}
|
||||
|
||||
return is_array($previous) ? $previous : null;
|
||||
}
|
||||
}
|
||||
|
|
@ -7,6 +7,15 @@ use Illuminate\Support\Facades\Log;
|
|||
|
||||
class DiscordWebhook
|
||||
{
|
||||
/**
|
||||
* Discord's own caps on embed text. Exceeding either one makes Discord
|
||||
* reject the whole request, so an over-long report would vanish from the
|
||||
* channel with nothing but a logged 400 behind it.
|
||||
*/
|
||||
private const DESCRIPTION_LIMIT = 4096;
|
||||
|
||||
private const FIELD_VALUE_LIMIT = 1024;
|
||||
|
||||
public function __construct(private ?string $webhookUrl) {}
|
||||
|
||||
/**
|
||||
|
|
@ -22,7 +31,7 @@ class DiscordWebhook
|
|||
|
||||
$payload = array_filter([
|
||||
'content' => $content !== '' ? $content : null,
|
||||
'embeds' => $embeds !== [] ? $embeds : null,
|
||||
'embeds' => $embeds !== [] ? $this->withinLimits($embeds) : null,
|
||||
]);
|
||||
|
||||
$response = Http::asJson()->post($this->webhookUrl, $payload);
|
||||
|
|
@ -34,4 +43,42 @@ class DiscordWebhook
|
|||
]);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Trim embed text that would break Discord's limits. Losing the tail of a
|
||||
* legend is a much better outcome than losing the entire report, and the
|
||||
* warning says which text needs shortening.
|
||||
*
|
||||
* @param array<int, array<string, mixed>> $embeds
|
||||
* @return array<int, array<string, mixed>>
|
||||
*/
|
||||
private function withinLimits(array $embeds): array
|
||||
{
|
||||
foreach ($embeds as $index => $embed) {
|
||||
if (isset($embed['description'])) {
|
||||
$embeds[$index]['description'] = $this->trim($embed['description'], self::DESCRIPTION_LIMIT, 'description');
|
||||
}
|
||||
|
||||
foreach ($embed['fields'] ?? [] as $field => $data) {
|
||||
$embeds[$index]['fields'][$field]['value'] = $this->trim($data['value'], self::FIELD_VALUE_LIMIT, $data['name']);
|
||||
}
|
||||
}
|
||||
|
||||
return $embeds;
|
||||
}
|
||||
|
||||
private function trim(string $text, int $limit, string $label): string
|
||||
{
|
||||
if (mb_strlen($text) <= $limit) {
|
||||
return $text;
|
||||
}
|
||||
|
||||
Log::warning('Discord embed text trimmed to fit the limit.', [
|
||||
'label' => $label,
|
||||
'length' => mb_strlen($text),
|
||||
'limit' => $limit,
|
||||
]);
|
||||
|
||||
return mb_substr($text, 0, $limit - 1).'…';
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,36 @@
|
|||
<?php
|
||||
|
||||
return [
|
||||
|
||||
/*
|
||||
|--------------------------------------------------------------------------
|
||||
| Provider & model
|
||||
|--------------------------------------------------------------------------
|
||||
|
|
||||
| Provider and model for the AI summary that opens the scheduled stats
|
||||
| reports (`stats:subscription-funnel`, `stats:experiment-funnel`,
|
||||
| `stats:ai-cohort-report`). The provider defaults to Gemini but accepts any
|
||||
| laravel/ai provider (a valid Laravel\Ai\Enums\Lab case); a Flash-tier
|
||||
| model is plenty for summarising a handful of pre-computed figures. See the
|
||||
| README "AI Provider" section for the shared AI_PROVIDER switch.
|
||||
|
|
||||
*/
|
||||
|
||||
'provider' => env('AI_REPORTS_PROVIDER', env('AI_PROVIDER', 'gemini')),
|
||||
|
||||
'model' => env('AI_REPORTS_MODEL', 'gemini-flash-latest'),
|
||||
|
||||
/*
|
||||
|--------------------------------------------------------------------------
|
||||
| Timeout
|
||||
|--------------------------------------------------------------------------
|
||||
|
|
||||
| Seconds to wait for the summary before giving up and posting the report
|
||||
| without it. The summary is a nice-to-have, so a slow provider must never
|
||||
| hold up the report itself.
|
||||
|
|
||||
*/
|
||||
|
||||
'timeout' => (int) env('AI_REPORTS_TIMEOUT', 30),
|
||||
|
||||
];
|
||||
|
|
@ -0,0 +1,105 @@
|
|||
<?php
|
||||
|
||||
use App\Ai\Agents\ReportSummaryAgent;
|
||||
use App\Services\Ai\ReportSummarizer;
|
||||
use Illuminate\Support\Facades\Cache;
|
||||
use Illuminate\Support\Facades\Exceptions;
|
||||
use Illuminate\Support\Facades\Log;
|
||||
use Laravel\Ai\Exceptions\ProviderOverloadedException;
|
||||
|
||||
function summarize(bool $remember = true): ?string
|
||||
{
|
||||
return app(ReportSummarizer::class)->summarize('test-report', 'Test report.', ['weeks' => [['week' => '2026-W26']]], $remember);
|
||||
}
|
||||
|
||||
it('feeds the model the previous run figures and when they were captured', function () {
|
||||
ReportSummaryAgent::fake(['Los registros bajan.']);
|
||||
|
||||
Cache::put('report_summary_baseline:test-report', [
|
||||
'captured_at' => now()->subWeek()->toIso8601String(),
|
||||
'payload' => ['weeks' => [['week' => '2026-W25']]],
|
||||
], now()->addDay());
|
||||
|
||||
expect(summarize())->toBe('Los registros bajan.');
|
||||
|
||||
ReportSummaryAgent::assertPrompted(function ($prompt): bool {
|
||||
$payload = json_decode($prompt->prompt, true);
|
||||
|
||||
return $payload['previous']['weeks'][0]['week'] === '2026-W25'
|
||||
&& $payload['current']['weeks'][0]['week'] === '2026-W26'
|
||||
&& str_starts_with($payload['previous_captured_at'], now()->subWeek()->toDateString());
|
||||
});
|
||||
});
|
||||
|
||||
it('leaves the current figures as the baseline for the next run', function () {
|
||||
ReportSummaryAgent::fake(['Resumen.']);
|
||||
|
||||
summarize();
|
||||
|
||||
expect(Cache::get('report_summary_baseline:test-report'))
|
||||
->toMatchArray(['payload' => ['weeks' => [['week' => '2026-W26']]]]);
|
||||
});
|
||||
|
||||
it('keeps the stored baseline when the report is re-run the same day', function () {
|
||||
ReportSummaryAgent::fake(['Resumen.', 'Resumen.']);
|
||||
|
||||
Cache::put('report_summary_baseline:test-report', [
|
||||
'captured_at' => now()->toIso8601String(),
|
||||
'payload' => ['weeks' => [['week' => 'last-week']]],
|
||||
], now()->addDay());
|
||||
|
||||
summarize();
|
||||
|
||||
// A manual re-run must not overwrite the period the next scheduled run
|
||||
// compares against, otherwise the summary reports "no change".
|
||||
expect(Cache::get('report_summary_baseline:test-report')['payload'])
|
||||
->toBe(['weeks' => [['week' => 'last-week']]]);
|
||||
});
|
||||
|
||||
it('does not touch the baseline on a dry run', function () {
|
||||
ReportSummaryAgent::fake(['Resumen.']);
|
||||
|
||||
summarize(remember: false);
|
||||
|
||||
expect(Cache::get('report_summary_baseline:test-report'))->toBeNull();
|
||||
});
|
||||
|
||||
it('strips backticks so the summary cannot swallow the table below it', function () {
|
||||
ReportSummaryAgent::fake(['Los pagos suben ```mucho```.']);
|
||||
|
||||
expect(summarize())->toBe('Los pagos suben mucho.');
|
||||
});
|
||||
|
||||
it('trims a runaway answer', function () {
|
||||
ReportSummaryAgent::fake([str_repeat('a', 2000)]);
|
||||
|
||||
expect(mb_strlen((string) summarize()))->toBeLessThanOrEqual(900);
|
||||
});
|
||||
|
||||
it('returns null for an empty answer', function () {
|
||||
ReportSummaryAgent::fake([' ']);
|
||||
|
||||
expect(summarize())->toBeNull();
|
||||
});
|
||||
|
||||
it('returns null and reports the failure when the provider fails', function () {
|
||||
Exceptions::fake();
|
||||
ReportSummaryAgent::fake(fn () => throw new RuntimeException('provider down'));
|
||||
|
||||
expect(summarize())->toBeNull();
|
||||
|
||||
Exceptions::assertReported(fn (RuntimeException $exception): bool => $exception->getMessage() === 'provider down');
|
||||
});
|
||||
|
||||
it('logs a transient provider failure without reporting it', function () {
|
||||
Exceptions::fake();
|
||||
Log::spy();
|
||||
ReportSummaryAgent::fake(fn () => throw ProviderOverloadedException::forProvider('gemini'));
|
||||
|
||||
expect(summarize())->toBeNull();
|
||||
|
||||
Exceptions::assertNothingReported();
|
||||
Log::shouldHaveReceived('warning')
|
||||
->withArgs(fn (string $message): bool => str_contains($message, 'provider transient failure'))
|
||||
->once();
|
||||
});
|
||||
|
|
@ -0,0 +1,76 @@
|
|||
<?php
|
||||
|
||||
use App\Ai\Agents\ReportSummaryAgent;
|
||||
use App\Services\Discord\DiscordWebhook;
|
||||
use Illuminate\Support\Facades\Http;
|
||||
|
||||
use function Pest\Laravel\artisan;
|
||||
|
||||
/**
|
||||
* Discord rejects an entire webhook payload when an embed field value is longer
|
||||
* than 1024 characters or a description longer than 4096 — the report would just
|
||||
* stop appearing in the channel, with only a logged 400 behind it. The legend and
|
||||
* "how to read it" fields sit closest to that cap, so every scheduled report is
|
||||
* measured here, and the trimming that saves an over-long one is covered too.
|
||||
*/
|
||||
beforeEach(function () {
|
||||
config([
|
||||
'subscriptions.enabled' => true,
|
||||
'subscriptions.experiment.started_at' => '2026-06-01',
|
||||
'services.discord.webhook_url' => 'https://discord.test/hook',
|
||||
'services.discord.ai_cohort_webhook_url' => 'https://discord.test/hook',
|
||||
]);
|
||||
|
||||
Http::fake(['discord.test/*' => Http::response('', 204)]);
|
||||
ReportSummaryAgent::fake(['Sin cambios relevantes frente al periodo anterior.']);
|
||||
});
|
||||
|
||||
it('keeps every scheduled report inside Discord\'s embed limits', function (string $command) {
|
||||
if ($command === 'stats:daily-report') {
|
||||
bindMockStripeClientForStats(['active' => [], 'trialing' => []]);
|
||||
}
|
||||
|
||||
artisan($command)->assertSuccessful();
|
||||
|
||||
$embed = null;
|
||||
|
||||
Http::assertSent(function ($request) use (&$embed) {
|
||||
$embed = $request['embeds'][0];
|
||||
|
||||
return true;
|
||||
});
|
||||
|
||||
expect(mb_strlen($embed['description'] ?? ''))->toBeLessThanOrEqual(4096);
|
||||
|
||||
foreach ($embed['fields'] ?? [] as $field) {
|
||||
expect(mb_strlen($field['value']))->toBeLessThanOrEqual(1024)
|
||||
// The webhook trims over-long text and marks it with an ellipsis, so
|
||||
// a trimmed field means the copy itself has to be shortened.
|
||||
->and($field['value'])->not->toEndWith('…');
|
||||
}
|
||||
})->with([
|
||||
'stats:daily-report',
|
||||
'stats:subscription-funnel',
|
||||
'stats:experiment-funnel',
|
||||
'stats:ai-cohort-report',
|
||||
]);
|
||||
|
||||
it('trims over-long embed text instead of letting Discord reject the report', function () {
|
||||
(new DiscordWebhook('https://discord.test/hook'))->send('', [[
|
||||
'title' => 'Informe',
|
||||
'description' => str_repeat('a', 4106),
|
||||
'fields' => [
|
||||
['name' => 'Leyenda', 'value' => str_repeat('b', 1034), 'inline' => false],
|
||||
['name' => 'Corto', 'value' => 'cabe de sobra', 'inline' => false],
|
||||
],
|
||||
]]);
|
||||
|
||||
Http::assertSent(function ($request) {
|
||||
$embed = $request['embeds'][0];
|
||||
|
||||
return mb_strlen($embed['description']) === 4096
|
||||
&& mb_strlen($embed['fields'][0]['value']) === 1024
|
||||
&& str_ends_with($embed['fields'][0]['value'], '…')
|
||||
&& $embed['fields'][1]['value'] === 'cabe de sobra';
|
||||
});
|
||||
});
|
||||
|
|
@ -1,5 +1,6 @@
|
|||
<?php
|
||||
|
||||
use App\Ai\Agents\ReportSummaryAgent;
|
||||
use App\Models\Account;
|
||||
use App\Models\AiConsent;
|
||||
use App\Models\Category;
|
||||
|
|
@ -79,6 +80,7 @@ beforeEach(function () {
|
|||
config(['subscriptions.enabled' => true]);
|
||||
config(['ai_suggestions.eligibility_min_transactions' => 3]);
|
||||
config(['ai_suggestions.report.excluded_emails' => []]);
|
||||
ReportSummaryAgent::fake(['La retención se mantiene plana frente al mes anterior.']);
|
||||
});
|
||||
|
||||
it('skips the report and hits no external service when subscriptions are disabled', function () {
|
||||
|
|
@ -218,19 +220,54 @@ it('posts the cohort report embed to the configured discord webhook', function (
|
|||
artisan('stats:ai-cohort-report')->assertSuccessful();
|
||||
|
||||
Http::assertSent(function ($request) {
|
||||
$embed = $request['embeds'][0];
|
||||
|
||||
return $request->url() === 'https://discord.test/hook'
|
||||
&& isset($request['embeds'][0]['title'])
|
||||
&& str_contains($request['embeds'][0]['title'], 'AI Suggestions');
|
||||
&& str_contains($embed['title'], 'Sugerencias con IA')
|
||||
// Spanish table header and legend, no leftover English.
|
||||
&& str_contains($embed['description'], 'Semana')
|
||||
&& str_contains($embed['description'], 'Prueba')
|
||||
&& collect($embed['fields'])->contains(fn ($field) => $field['name'] === 'Leyenda');
|
||||
});
|
||||
});
|
||||
|
||||
it('opens the embed with the AI summary, above the table', function () {
|
||||
config(['services.discord.ai_cohort_webhook_url' => 'https://discord.test/hook']);
|
||||
Http::fake(['discord.test/*' => Http::response('', 204)]);
|
||||
ReportSummaryAgent::fake(['La cohorte de este mes convierte peor que la anterior.']);
|
||||
|
||||
cohortUser(referenceNow()->subWeeks(6), ['transactions' => 3, 'lastActiveAt' => referenceNow()->subWeeks(3)]);
|
||||
|
||||
artisan('stats:ai-cohort-report')->assertSuccessful();
|
||||
|
||||
Http::assertSent(fn ($request) => str_starts_with(
|
||||
$request['embeds'][0]['description'],
|
||||
"La cohorte de este mes convierte peor que la anterior.\n\n```",
|
||||
));
|
||||
});
|
||||
|
||||
it('still posts the report when the AI summary fails', function () {
|
||||
config(['services.discord.ai_cohort_webhook_url' => 'https://discord.test/hook']);
|
||||
Http::fake(['discord.test/*' => Http::response('', 204)]);
|
||||
ReportSummaryAgent::fake(fn () => throw new RuntimeException('provider down'));
|
||||
|
||||
cohortUser(referenceNow()->subWeeks(6), ['transactions' => 3, 'lastActiveAt' => referenceNow()->subWeeks(3)]);
|
||||
|
||||
artisan('stats:ai-cohort-report')->assertSuccessful();
|
||||
|
||||
Http::assertSent(fn ($request) => str_starts_with($request['embeds'][0]['description'], '```')
|
||||
&& str_contains($request['embeds'][0]['description'], 'Semana'));
|
||||
});
|
||||
|
||||
it('prints to the console without posting when --no-discord is set', function () {
|
||||
config(['services.discord.ai_cohort_webhook_url' => 'https://discord.test/hook']);
|
||||
Http::fake(['discord.test/*' => Http::response('', 204)]);
|
||||
|
||||
cohortUser(referenceNow()->subWeeks(6), ['transactions' => 3, 'lastActiveAt' => referenceNow()->subWeeks(3)]);
|
||||
|
||||
artisan('stats:ai-cohort-report', ['--no-discord' => true])->assertSuccessful();
|
||||
artisan('stats:ai-cohort-report', ['--no-discord' => true])
|
||||
->expectsOutputToContain('La retención se mantiene plana frente al mes anterior.')
|
||||
->assertSuccessful();
|
||||
|
||||
Http::assertNothingSent();
|
||||
});
|
||||
|
|
|
|||
|
|
@ -36,10 +36,10 @@ test('posts yesterday user counts and stripe stats to discord', function () {
|
|||
|
||||
Http::assertSent(function ($request) {
|
||||
$embed = $request['embeds'][0];
|
||||
$users = collect($embed['fields'])->firstWhere('name', '👥 Users');
|
||||
$users = collect($embed['fields'])->firstWhere('name', '👥 Usuarios');
|
||||
|
||||
return $request->url() === 'https://discord.test/webhook'
|
||||
&& str_contains($users['value'], 'New yesterday: **1**')
|
||||
&& str_contains($users['value'], 'Nuevos ayer: **1**')
|
||||
&& str_contains($users['value'], 'Total: **2**')
|
||||
&& collect($embed['fields'])->contains(fn ($f) => str_contains($f['value'], '€10.00'));
|
||||
});
|
||||
|
|
@ -71,9 +71,9 @@ test('total reflects users at end of yesterday and excludes users created today'
|
|||
$this->artisan('stats:daily-report')->assertSuccessful();
|
||||
|
||||
Http::assertSent(function ($request) {
|
||||
$users = collect($request['embeds'][0]['fields'])->firstWhere('name', '👥 Users');
|
||||
$users = collect($request['embeds'][0]['fields'])->firstWhere('name', '👥 Usuarios');
|
||||
|
||||
return str_contains($users['value'], 'New yesterday: **1**')
|
||||
return str_contains($users['value'], 'Nuevos ayer: **1**')
|
||||
&& str_contains($users['value'], 'Total: **2**');
|
||||
});
|
||||
});
|
||||
|
|
@ -87,9 +87,9 @@ test('reports zero new users when none were created yesterday', function () {
|
|||
$this->artisan('stats:daily-report')->assertSuccessful();
|
||||
|
||||
Http::assertSent(function ($request) {
|
||||
$users = collect($request['embeds'][0]['fields'])->firstWhere('name', '👥 Users');
|
||||
$users = collect($request['embeds'][0]['fields'])->firstWhere('name', '👥 Usuarios');
|
||||
|
||||
return str_contains($users['value'], 'New yesterday: **0**')
|
||||
return str_contains($users['value'], 'Nuevos ayer: **0**')
|
||||
&& str_contains($users['value'], 'Total: **1**');
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -1,5 +1,6 @@
|
|||
<?php
|
||||
|
||||
use App\Ai\Agents\ReportSummaryAgent;
|
||||
use App\Features\SubscriptionExperiment;
|
||||
use App\Models\AiConsent;
|
||||
use App\Models\BankingConnection;
|
||||
|
|
@ -29,6 +30,8 @@ beforeEach(function () {
|
|||
|
||||
// Seed the price→monthly-equivalent map so revenue is computed without Stripe.
|
||||
Cache::put('experiment_funnel_monthly_equiv', ['price_test' => 399], now()->addHour());
|
||||
|
||||
ReportSummaryAgent::fake(['pay_now sigue por delante, pero la muestra es demasiado pequeña.']);
|
||||
});
|
||||
|
||||
it('skips the report and hits no external service when subscriptions are disabled', function () {
|
||||
|
|
@ -258,7 +261,7 @@ it('reports a matured-cohort conversion capped at 100% and prints the matured de
|
|||
Artisan::call('stats:experiment-funnel', ['--no-discord' => true]);
|
||||
$output = Artisan::output();
|
||||
|
||||
expect($output)->toContain('MatU') // matured denominator column is printed
|
||||
expect($output)->toContain('UMad') // matured denominator column is printed
|
||||
->toContain('Conv%')
|
||||
->toContain('100%') // capped, not the old 200% A2P%
|
||||
->not->toContain('200%');
|
||||
|
|
@ -275,11 +278,11 @@ it('reports a Wilson confidence interval and defers the verdict while samples ar
|
|||
Artisan::call('stats:experiment-funnel', ['--no-discord' => true]);
|
||||
$output = Artisan::output();
|
||||
|
||||
expect($output)->toContain('Significance')
|
||||
expect($output)->toContain('Significancia')
|
||||
->toContain('Wilson')
|
||||
->toContain('Fisher exact') // verdict uses the exact test, not the z-approx
|
||||
->toContain('not significant') // equal 50/50 rates, n=2 per arm → nowhere near
|
||||
->toContain('Small sample'); // min expected conversions < 5
|
||||
->toContain('exacto de Fisher') // verdict uses the exact test, not the z-approx
|
||||
->toContain('no significativo') // equal 50/50 rates, n=2 per arm → nowhere near
|
||||
->toContain('Muestra pequeña'); // min expected conversions < 5
|
||||
});
|
||||
|
||||
it('declares significance via the exact test when the separation is real', function () {
|
||||
|
|
@ -295,8 +298,8 @@ it('declares significance via the exact test when the separation is real', funct
|
|||
Artisan::call('stats:experiment-funnel', ['--no-discord' => true]);
|
||||
$output = Artisan::output();
|
||||
|
||||
expect($output)->toContain('Fisher exact')
|
||||
->not->toContain('not significant'); // the exact test clears the corrected bar
|
||||
expect($output)->toContain('exacto de Fisher')
|
||||
->not->toContain('no significativo'); // the exact test clears the corrected bar
|
||||
});
|
||||
|
||||
it('measures conversion as ever-charged, not active-now, so churn does not bias it', function () {
|
||||
|
|
@ -395,8 +398,68 @@ it('posts the experiment funnel embed to discord', function () {
|
|||
|
||||
artisan('stats:experiment-funnel')->assertSuccessful();
|
||||
|
||||
Http::assertSent(fn ($request) => $request->url() === 'https://discord.test/hook'
|
||||
&& str_contains($request['embeds'][0]['title'], 'Experiment'));
|
||||
Http::assertSent(function ($request) {
|
||||
$embed = $request['embeds'][0];
|
||||
|
||||
return $request->url() === 'https://discord.test/hook'
|
||||
&& str_contains($embed['title'], 'Experimento')
|
||||
// Spanish table header and legend, no leftover English.
|
||||
&& str_contains($embed['description'], 'Variante')
|
||||
&& collect($embed['fields'])->contains(fn ($field) => $field['name'] === 'Leyenda');
|
||||
});
|
||||
});
|
||||
|
||||
it('opens the embed with the AI summary, above the table', function () {
|
||||
config(['services.discord.ai_cohort_webhook_url' => 'https://discord.test/hook']);
|
||||
Http::fake(['discord.test/*' => Http::response('', 204)]);
|
||||
ReportSummaryAgent::fake(['control convierte mejor, pero la diferencia no es significativa.']);
|
||||
|
||||
experimentUser(SubscriptionExperiment::CONTROL, CarbonImmutable::parse('2026-06-05'), [
|
||||
'status' => 'active',
|
||||
'at' => CarbonImmutable::parse('2026-06-05'),
|
||||
]);
|
||||
|
||||
artisan('stats:experiment-funnel')->assertSuccessful();
|
||||
|
||||
Http::assertSent(fn ($request) => str_starts_with(
|
||||
$request['embeds'][0]['description'],
|
||||
"control convierte mejor, pero la diferencia no es significativa.\n\n```",
|
||||
));
|
||||
});
|
||||
|
||||
it('feeds the summary the variant figures and the significance verdict', function () {
|
||||
Http::fake();
|
||||
|
||||
experimentUser(SubscriptionExperiment::CONTROL, CarbonImmutable::parse('2026-06-05'), [
|
||||
'status' => 'active',
|
||||
'at' => CarbonImmutable::parse('2026-06-05'),
|
||||
]);
|
||||
|
||||
artisan('stats:experiment-funnel', ['--no-discord' => true])->assertSuccessful();
|
||||
|
||||
ReportSummaryAgent::assertPrompted(function ($prompt): bool {
|
||||
$payload = json_decode($prompt->prompt, true)['current'];
|
||||
|
||||
return $payload['variants']['control']['converted_mature'] === 1
|
||||
&& $payload['currency'] === 'EUR'
|
||||
&& collect($payload['significance'])->contains(fn (string $line) => str_contains($line, 'Wilson'));
|
||||
});
|
||||
});
|
||||
|
||||
it('still posts the report when the AI summary fails', function () {
|
||||
config(['services.discord.ai_cohort_webhook_url' => 'https://discord.test/hook']);
|
||||
Http::fake(['discord.test/*' => Http::response('', 204)]);
|
||||
ReportSummaryAgent::fake(fn () => throw new RuntimeException('provider down'));
|
||||
|
||||
experimentUser(SubscriptionExperiment::CONTROL, CarbonImmutable::parse('2026-06-05'), [
|
||||
'status' => 'active',
|
||||
'at' => CarbonImmutable::parse('2026-06-05'),
|
||||
]);
|
||||
|
||||
artisan('stats:experiment-funnel')->assertSuccessful();
|
||||
|
||||
Http::assertSent(fn ($request) => str_starts_with($request['embeds'][0]['description'], '```')
|
||||
&& str_contains($request['embeds'][0]['description'], 'Variante'));
|
||||
});
|
||||
|
||||
it('does not post when the experiment has not started', function () {
|
||||
|
|
@ -405,6 +468,7 @@ it('does not post when the experiment has not started', function () {
|
|||
|
||||
artisan('stats:experiment-funnel')->assertSuccessful();
|
||||
|
||||
ReportSummaryAgent::assertNeverPrompted();
|
||||
Http::assertNothingSent();
|
||||
});
|
||||
|
||||
|
|
|
|||
|
|
@ -1,9 +1,11 @@
|
|||
<?php
|
||||
|
||||
use App\Ai\Agents\ReportSummaryAgent;
|
||||
use App\Models\User;
|
||||
use App\Services\Stats\SubscriptionFunnelCollector;
|
||||
use Carbon\CarbonImmutable;
|
||||
use Illuminate\Support\Carbon;
|
||||
use Illuminate\Support\Facades\Cache;
|
||||
use Illuminate\Support\Facades\Http;
|
||||
use Illuminate\Support\Str;
|
||||
|
||||
|
|
@ -61,6 +63,7 @@ beforeEach(function () {
|
|||
Carbon::setTestNow(funnelNow());
|
||||
config(['subscriptions.enabled' => true]);
|
||||
config(['ai_suggestions.report.excluded_emails' => []]);
|
||||
ReportSummaryAgent::fake(['Los registros bajan respecto a la semana anterior.']);
|
||||
});
|
||||
|
||||
it('skips the report and hits no external service when subscriptions are disabled', function () {
|
||||
|
|
@ -150,19 +153,66 @@ it('posts the funnel embed to the configured discord webhook', function () {
|
|||
artisan('stats:subscription-funnel')->assertSuccessful();
|
||||
|
||||
Http::assertSent(function ($request) {
|
||||
$embed = $request['embeds'][0];
|
||||
|
||||
return $request->url() === 'https://discord.test/hook'
|
||||
&& isset($request['embeds'][0]['title'])
|
||||
&& str_contains($request['embeds'][0]['title'], 'Subscription Funnel');
|
||||
&& str_contains($embed['title'], 'Embudo de suscripción')
|
||||
// Spanish table header and legend, no leftover English.
|
||||
&& str_contains($embed['description'], 'Semana')
|
||||
&& collect($embed['fields'])->contains(fn ($field) => $field['name'] === 'Leyenda');
|
||||
});
|
||||
});
|
||||
|
||||
it('opens the embed with the AI summary, above the table', function () {
|
||||
config(['services.discord.ai_cohort_webhook_url' => 'https://discord.test/hook']);
|
||||
Http::fake(['discord.test/*' => Http::response('', 204)]);
|
||||
ReportSummaryAgent::fake(['Los pagos suben respecto a la semana anterior.']);
|
||||
|
||||
funnelUser(funnelNow()->subWeeks(10), ['status' => 'active', 'at' => funnelNow()->subWeeks(10)->addDays(2)]);
|
||||
|
||||
artisan('stats:subscription-funnel')->assertSuccessful();
|
||||
|
||||
Http::assertSent(fn ($request) => str_starts_with(
|
||||
$request['embeds'][0]['description'],
|
||||
"Los pagos suben respecto a la semana anterior.\n\n```",
|
||||
));
|
||||
});
|
||||
|
||||
it('still posts the report when the AI summary fails', function () {
|
||||
config(['services.discord.ai_cohort_webhook_url' => 'https://discord.test/hook']);
|
||||
Http::fake(['discord.test/*' => Http::response('', 204)]);
|
||||
ReportSummaryAgent::fake(fn () => throw new RuntimeException('provider down'));
|
||||
|
||||
funnelUser(funnelNow()->subWeeks(10), ['status' => 'active', 'at' => funnelNow()->subWeeks(10)->addDays(2)]);
|
||||
|
||||
artisan('stats:subscription-funnel')->assertSuccessful();
|
||||
|
||||
Http::assertSent(fn ($request) => str_starts_with($request['embeds'][0]['description'], '```')
|
||||
&& str_contains($request['embeds'][0]['description'], 'Semana'));
|
||||
});
|
||||
|
||||
it('prints to the console without posting when --no-discord is set', function () {
|
||||
config(['services.discord.ai_cohort_webhook_url' => 'https://discord.test/hook']);
|
||||
Http::fake(['discord.test/*' => Http::response('', 204)]);
|
||||
|
||||
funnelUser(funnelNow()->subWeeks(10), ['status' => 'active', 'at' => funnelNow()->subWeeks(10)->addDays(2)]);
|
||||
|
||||
artisan('stats:subscription-funnel', ['--no-discord' => true])->assertSuccessful();
|
||||
artisan('stats:subscription-funnel', ['--no-discord' => true])
|
||||
->expectsOutputToContain('Los registros bajan respecto a la semana anterior.')
|
||||
->assertSuccessful();
|
||||
|
||||
Http::assertNothingSent();
|
||||
});
|
||||
|
||||
it('does not overwrite the comparison baseline on a --no-discord run', function () {
|
||||
Http::fake();
|
||||
Cache::put('report_summary_baseline:subscription-funnel', ['weeks' => ['sentinel']], now()->addDay());
|
||||
|
||||
artisan('stats:subscription-funnel', ['--no-discord' => true])->assertSuccessful();
|
||||
|
||||
expect(Cache::get('report_summary_baseline:subscription-funnel'))->toBe(['weeks' => ['sentinel']]);
|
||||
|
||||
artisan('stats:subscription-funnel')->assertSuccessful();
|
||||
|
||||
expect(Cache::get('report_summary_baseline:subscription-funnel'))->not->toBe(['weeks' => ['sentinel']]);
|
||||
});
|
||||
|
|
|
|||
Loading…
Reference in New Issue