RomanNum3ral
/
whisper-money
mirror of https://github.com/whisper-money/whisper-money.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
whisper-money/database/migrations
History
Víctor Falcón 4c83cb8b33
feat(import): persist per-account import configuration on the backend (#698) 
## What & why

Import configuration (the CSV/Excel column mapping and date format a
user
sets up per account) was stored in the browser's `localStorage`, so it
never
followed the user to another device. Importing the same account's
statement on
mobile — or on a new machine — meant reconfiguring the mapping from
scratch.

This moves that configuration to the backend, keyed per account and per
import
type, so it's preconfigured and loaded automatically wherever the user
imports.

## Demo

Cross-device QA: import transactions on "device 1" with a custom column
mapping, then clear **all** cookies + local/session storage (simulating
a
fresh device), log back in, and start a new import — the mapping
(Description → *Movement*, Amount → *How Much*, date format
*DD-MM-YYYY*)
is auto-loaded from the backend with no manual setup.


https://github.com/user-attachments/assets/e449dfe9-3970-48d4-97ca-9e415c73835b



## How

- New `account_import_configs` table: one row per `(account_id, type)`
where
`type` is `transaction` or `balance`. The mapping + date format are
stored as
  an opaque `config` JSON blob (exactly what the client sends).
- `GET/PUT /api/accounts/{account}/import-config`
(`AccountImportConfigController`),
authorized via the existing `AccountPolicy` (`view`/`update`) — mirrors
  `AccountBalanceController`. `PUT` upserts on `(account_id, type)`.
- Frontend: the two import-config storage helpers now read/write the
endpoint
via `axios` instead of `localStorage` (merged into a single module — the
  transaction and balance variants shared the same logic).
- The saved config is fetched **off the file-parse critical path**: the
parsed
file shows immediately with auto-detected columns, and the saved mapping
is
applied when it arrives (guarded so a slow load can't clobber a file
picked
afterwards). A slow or hanging request never blocks the preview or Next
  button. When no config exists or the request fails, it falls back to
  auto-detection exactly as before.

## Notes / decisions

- **No localStorage migration.** Existing per-device configs aren't
migrated;
on the next import the mapping is auto-detected and re-saved to the
backend,
so it self-heals after one import. The old `import_config_account_*`
keys are
  simply no longer read.
- The mapping-validity check against the actual file headers stays
client-side
  (it depends on the just-parsed file).

## Testing

- `tests/Feature/AccountImportConfigTest.php` (9 tests): auth required,
  cross-user 403 on read and write, save→persist→load round-trip, upsert
  de-duplication, transaction/balance independence, and validation
  (unknown type, missing column mapping).
- Sibling suites green (`AccountBalanceControllerTest`,
`SavedFilterTest`) —
no regression from the new `Account::importConfigs()` relation or
routes.
 		2026-07-18 16:10:40 +02:00
..
0001_01_01_000000_create_users_table.php Set up a fresh Laravel app 2025-11-07 12:01:36 +00:00
0001_01_01_000001_create_cache_table.php Set up a fresh Laravel app 2025-11-07 12:01:36 +00:00
0001_01_01_000002_create_jobs_table.php Set up a fresh Laravel app 2025-11-07 12:01:36 +00:00
2025_08_26_100418_add_two_factor_columns_to_users_table.php Set up a fresh Laravel app 2025-11-07 12:01:36 +00:00
2025_11_07_135255_add_encryption_salt_to_users_table.php E2E Encryption 2025-11-07 14:21:25 +00:00
2025_11_07_135256_create_encrypted_messages_table.php E2E Encryption 2025-11-07 14:21:25 +00:00
2025_11_07_150038_create_banks_table.php feat: Add financial models and seeders 2025-11-07 15:45:28 +00:00
2025_11_07_150122_create_accounts_table.php feat: Add financial models and seeders 2025-11-07 15:45:28 +00:00
2025_11_07_150613_create_categories_table.php feat: Add financial models and seeders 2025-11-07 15:45:28 +00:00
2025_11_07_150659_create_transactions_table.php feat: Add financial models and seeders 2025-11-07 15:45:28 +00:00
2025_11_07_184056_modify_banks_table_add_user_id_and_simplify_name.php Banks accounts 2025-11-07 19:19:29 +00:00
2025_11_07_185018_change_banks_logo_to_text.php Banks accounts 2025-11-07 19:19:29 +00:00
2025_11_08_140230_change_transactions_id_to_uuid.php Update transaction amount to bigint 2025-11-15 20:53:47 +01:00
2025_11_08_141344_make_category_id_nullable_in_transactions_table.php Sync transactions 2025-11-08 14:17:16 +00:00
2025_11_08_144530_add_soft_deletes_to_transactions_table.php Transactions page 2025-11-08 23:38:35 +00:00
2025_11_10_110100_create_automation_rules_table.php Automation rules 2025-11-10 12:08:58 +00:00
2025_11_15_172640_create_account_balances_table.php Add account balances 2025-11-15 20:27:18 +01:00
2025_11_15_175636_change_transactions_amount_to_bigint.php Update transaction amount to bigint 2025-11-15 20:53:47 +01:00
2025_11_15_195739_convert_all_ids_to_uuid.php fix: migration history 2025-12-30 07:22:19 +01:00
2025_11_18_184409_create_user_leads_table.php Merge pull request #1 from whisper-money/landing-page 2025-11-24 12:11:35 +01:00
2025_11_28_104227_add_source_to_transactions_table.php Save transaction source (manually_created, or imported) 2025-11-28 12:36:48 +01:00
2025_11_29_170353_add_type_to_categories_table.php Add category type field support (#2) 2025-12-01 20:19:47 +01:00
2025_11_29_170955_fix_account_balances_unique_constraint.php fix: migration history 2025-12-30 07:22:19 +01:00
2025_12_01_101443_add_unique_constraint_to_categories_table.php Add unique constraint to categories table and update factory data 2025-12-01 11:19:16 +01:00
2025_12_06_112515_create_customer_columns.php Subscriptions (#15) 2025-12-06 19:09:56 +01:00
2025_12_06_112516_create_subscriptions_table.php Subscriptions (#15) 2025-12-06 19:09:56 +01:00
2025_12_06_112517_create_subscription_items_table.php Subscriptions (#15) 2025-12-06 19:09:56 +01:00
2025_12_06_112518_add_meter_id_to_subscription_items_table.php Subscriptions (#15) 2025-12-06 19:09:56 +01:00
2025_12_06_112519_add_meter_event_name_to_subscription_items_table.php Subscriptions (#15) 2025-12-06 19:09:56 +01:00
2025_12_10_142006_add_onboarded_at_to_users_table.php User Onboarding Flow (#23) 2025-12-12 13:06:08 +01:00
2025_12_12_092647_create_labels_table.php feat: add transaction labels feature (#24) 2025-12-13 13:02:19 +01:00
2025_12_12_092650_create_label_transaction_table.php feat: add transaction labels feature (#24) 2025-12-13 13:02:19 +01:00
2025_12_12_092651_create_automation_rule_labels_table.php feat: add transaction labels feature (#24) 2025-12-13 13:02:19 +01:00
2025_12_12_141955_update_labels_unique_constraint_include_deleted_at.php feat: add transaction labels feature (#24) 2025-12-13 13:02:19 +01:00
2025_12_16_151952_create_user_mail_logs_table.php feat: Implement drip email campaign system (#35) 2025-12-30 07:22:18 +01:00
2025_12_19_092437_create_budgets_table.php Add Budgeting Feature to Track and Manage Spending (#36) 2026-01-21 15:25:50 +01:00
2025_12_19_092443_create_budget_periods_table.php Add Budgeting Feature to Track and Manage Spending (#36) 2026-01-21 15:25:50 +01:00
2025_12_19_092446_create_budget_transactions_table.php Add Budgeting Feature to Track and Manage Spending (#36) 2026-01-21 15:25:50 +01:00
2025_12_20_154221_create_features_table.php Add Cashflow Analytics Feature (#49) 2026-01-05 13:06:50 +01:00
2025_12_29_063338_add_currency_code_to_users_table.php Add Budgeting Feature to Track and Manage Spending (#36) 2026-01-21 15:25:50 +01:00
2026_01_08_192757_add_email_identifier_to_user_mail_logs.php feat: Send custom emails to users (#52) 2026-01-09 09:33:19 +01:00
2026_01_08_204223_update_user_mail_logs_unique_constraint.php feat: Send custom emails to users (#52) 2026-01-09 09:33:19 +01:00
2026_01_21_151353_add_indexes_for_budget_transaction_assignment.php feat: Load transactions history on budget created (#72) 2026-01-22 11:10:15 +01:00
2026_01_22_071325_add_processing_historical_to_budget_periods.php feat: Load transactions history on budget created (#72) 2026-01-22 11:10:15 +01:00
2026_01_22_143232_add_locale_to_users_table.php feat: Spanish localization (#74) 2026-02-08 11:58:08 +01:00
2026_02_08_103953_create_user_settings_table.php Add chart color scheme setting (#101) 2026-02-28 12:58:21 +01:00
2026_02_09_110624_add_encrypted_to_accounts_table.php Remove encryption from bank account names (#104) 2026-02-09 14:15:26 +01:00
2026_02_09_131403_make_description_iv_nullable_on_transactions.php feat: Plaintext transactions behind feature flag (#105) 2026-02-10 14:31:24 +01:00
2026_02_09_144915_create_banking_connections_table.php feat: Integrate EnableBanking as open banking provider (#106) 2026-02-12 09:09:28 +01:00
2026_02_09_144959_add_banking_connection_fields_to_accounts_table.php feat: Integrate EnableBanking as open banking provider (#106) 2026-02-12 09:09:28 +01:00
2026_02_09_144959_add_external_transaction_id_to_transactions_table.php feat: Integrate EnableBanking as open banking provider (#106) 2026-02-12 09:09:28 +01:00
2026_02_10_085759_add_aspsp_logo_to_banking_connections_table.php feat: Integrate EnableBanking as open banking provider (#106) 2026-02-12 09:09:28 +01:00
2026_02_10_095639_add_raw_data_to_transactions_table.php feat: Integrate EnableBanking as open banking provider (#106) 2026-02-12 09:09:28 +01:00
2026_02_11_113115_add_pending_accounts_data_to_banking_connections_table.php feat: Integrate EnableBanking as open banking provider (#106) 2026-02-12 09:09:28 +01:00
2026_02_11_113119_add_linked_at_to_accounts_table.php feat: Integrate EnableBanking as open banking provider (#106) 2026-02-12 09:09:28 +01:00
2026_02_13_083716_add_original_description_to_transactions_table.php feat: Add per-bank description formatter for bank-synced transactions (#120) 2026-02-13 09:54:58 +01:00
2026_02_18_084241_add_api_token_to_banking_connections_table.php feat: Add Indexa Capital integration (#130) 2026-02-18 10:42:13 +01:00
2026_02_18_110556_add_api_secret_to_banking_connections_table.php feat: Add Binance integration (#131) 2026-02-18 15:23:46 +01:00
2026_02_20_074846_create_exchange_rates_table.php feat: Add multi-currency conversion for net worth charts (#138) 2026-02-20 10:45:23 +01:00
2026_02_22_180317_add_invested_amount_to_account_balances_table.php feat: investment benefits — show gains/losses on investment accounts (#140) 2026-02-23 13:59:10 +01:00
2026_02_24_193117_fix_budget_transaction_refund_amounts.php fix(budgets): handle refunds correctly in budget spending calculations (#152) 2026-02-24 21:12:36 +01:00
2026_03_03_114620_add_paywall_seen_at_to_users_table.php feat(subscription): allow free plan for open banking users without connected banks (#188) 2026-03-03 22:28:50 +00:00
2026_03_04_085843_add_waitlist_columns_to_user_leads_table.php fix(migration): make add_waitlist_columns migration idempotent (#200) 2026-03-04 12:06:38 +00:00
2026_03_04_094926_add_locale_to_user_leads_table.php feat(waitlist): waiting list with referral system (#199) 2026-03-04 12:36:47 +01:00
2026_03_12_095411_add_iban_to_accounts_table.php fix(banking): update external_account_id on reconnect and store IBAN (#220) 2026-03-12 10:28:23 +00:00
2026_03_18_112307_add_cashflow_direction_to_categories_table.php feat(cashflow): track transfer categories in trends (#236) 2026-03-18 14:02:47 +00:00
2026_03_19_151039_add_include_loans_in_net_worth_chart_to_user_settings_table.php fix(dashboard): treat loans as debt in net worth (#238) 2026-03-20 09:55:53 +00:00
2026_03_20_110659_create_real_estate_details_table.php feat(accounts): add real estate asset tracking (#241) 2026-03-24 10:21:32 +00:00
2026_03_20_132058_make_bank_id_nullable_on_accounts_table.php feat(accounts): add real estate asset tracking (#241) 2026-03-24 10:21:32 +00:00
2026_03_20_154755_add_include_real_estate_in_net_worth_chart_to_user_settings_table.php feat(accounts): add real estate asset tracking (#241) 2026-03-24 10:21:32 +00:00
2026_03_25_125717_add_revaluation_percentage_to_real_estate_details_table.php feat(accounts): add market value and annual revaluation to real estate accounts (#245) 2026-03-26 11:02:20 +01:00
2026_03_25_132244_create_loan_details_table.php feat(accounts): add loan amortization projections for loan accounts (#246) 2026-03-26 15:06:09 +01:00
2026_03_30_123011_add_consecutive_sync_failures_to_banking_connections_table.php fix(banking): retry failed sync connections and log every sync attempt (#251) 2026-03-31 11:34:35 +01:00
2026_03_30_123011_create_banking_sync_logs_table.php fix(banking): retry failed sync connections and log every sync attempt (#251) 2026-03-31 11:34:35 +01:00
2026_04_14_120000_add_email_verification_to_user_leads_table.php feat: verify waitlist leads (#285) 2026-04-14 11:26:01 +01:00
2026_04_16_063515_add_bank_transactions_email_cutoff_at_to_banking_connections_table.php fix(open-banking): skip silent sync emails (#295) 2026-04-16 08:28:44 +01:00
2026_04_16_092644_add_timezone_to_users_table.php fix(user): persist detected timezones (#296) 2026-04-16 11:36:57 +01:00
2026_04_22_084113_add_deleted_at_to_users_table.php Support soft-deleted users with reusable emails (#316) 2026-04-22 11:41:41 +01:00
2026_04_25_092824_add_launch_columns_to_user_leads_table.php feat(leads): cohort-based launch invitations with per-user Stripe coupons (#333) 2026-04-30 15:10:28 +01:00
2026_05_05_070442_add_rate_limited_until_to_banking_connections_table.php feat(banking): back off scheduler when EnableBanking returns 429 (#352) 2026-05-05 09:39:32 +02:00
2026_05_05_132023_convert_custom_budgets_to_monthly.php fix(budgets): remove Custom period type to fix duplicate-key crash (#355) 2026-05-05 16:07:16 +02:00
2026_05_13_085027_add_dedup_fingerprint_to_transactions_table.php fix(banking): dedup EnableBanking transactions by deterministic fingerprint (#390) 2026-05-13 11:30:11 +01:00
2026_05_25_115100_update_default_saving_and_investment_category_types.php feat(cashflow): add savings and period views (#424) 2026-05-25 16:41:00 +02:00
2026_05_26_061232_add_reinvitation_columns_to_user_leads_table.php feat(leads): add user lead re-invite campaign (#432) 2026-05-26 08:35:31 +02:00
2026_05_27_061513_add_counterparty_names_to_transactions_table.php feat(transactions): add counterparty fields (#440) 2026-05-27 16:20:55 +02:00
2026_05_28_073203_update_categories_unique_constraint_for_soft_deletes.php fix(categories): allow recreate after delete (#444) 2026-05-28 09:46:02 +02:00
2026_05_29_085835_update_saving_and_investment_category_cashflow_direction.php fix(categories): expose cashflow setting on create (#448) 2026-05-29 11:05:04 +02:00
2026_06_01_085554_create_budget_category_table.php feat(budgets): track multiple categories and labels per budget (#466) 2026-06-01 12:32:23 +02:00
2026_06_01_085554_create_budget_label_table.php feat(budgets): track multiple categories and labels per budget (#466) 2026-06-01 12:32:23 +02:00
2026_06_01_085554_migrate_budget_category_label_to_pivots.php feat(budgets): track multiple categories and labels per budget (#466) 2026-06-01 12:32:23 +02:00
2026_06_02_090000_add_parent_id_to_categories_table.php feat: parent/child category tree (#474) 2026-06-03 19:30:12 +02:00
2026_06_02_094823_add_notify_on_bank_transactions_synced_to_user_settings_table.php feat(settings): let users disable bank transactions email (#472) 2026-06-02 12:24:39 +02:00
2026_06_05_120000_create_saved_filters_table.php feat(transactions): save and reuse transaction filters (#496) 2026-06-05 18:00:14 +02:00
2026_06_05_185212_add_state_token_to_banking_connections_table.php feat(open-banking): finalize bank connection without a session via state token (#498) 2026-06-06 12:12:07 +02:00
2026_06_08_114044_add_analysis_days_to_saved_filters_table.php feat(transactions): filtered analysis dashboard (#507) 2026-06-08 14:04:00 +02:00
2026_06_09_124010_add_analysis_mode_to_saved_filters_table.php feat(analysis): project-aware transaction analysis (#513) 2026-06-09 15:32:07 +02:00
2026_06_10_083644_add_last_logged_in_at_to_users_table.php feat(users): track last login and last active timestamps (#516) 2026-06-10 11:01:30 +02:00
2026_06_10_084055_add_last_active_at_to_users_table.php feat(users): track last login and last active timestamps (#516) 2026-06-10 11:01:30 +02:00
2026_06_12_134225_create_ai_consents_table.php feat(ai): suggest automation rules during onboarding (#523) 2026-06-13 22:51:15 +02:00
2026_06_12_134523_create_suggestion_runs_table.php feat(ai): suggest automation rules during onboarding (#523) 2026-06-13 22:51:15 +02:00
2026_06_12_134524_create_rule_suggestions_table.php feat(ai): suggest automation rules during onboarding (#523) 2026-06-13 22:51:15 +02:00
2026_06_13_000000_add_is_catch_all_to_budgets_table.php feat: add catch-all budgets (#527) 2026-06-15 16:07:19 +00:00
2026_06_15_120911_add_ai_categorization_fields_to_transactions_table.php feat(ai): auto-categorize transactions with AI (behind flag) (#535) 2026-06-15 16:35:20 +02:00
2026_06_15_120911_add_origin_to_automation_rules_table.php feat(ai): auto-categorize transactions with AI (behind flag) (#535) 2026-06-15 16:35:20 +02:00
2026_06_15_120911_create_category_corrections_table.php feat(ai): auto-categorize transactions with AI (behind flag) (#535) 2026-06-15 16:35:20 +02:00
2026_06_17_081319_add_ai_suggested_category_to_transactions_table.php feat(ai): persist AI categorization suggestions below the label bar (#547) 2026-06-17 08:24:30 +00:00
2026_06_17_114601_create_integration_requests_table.php feat(integration-requests): community board to request & vote bank integrations (#550) 2026-06-17 12:50:51 +00:00
2026_06_17_114602_create_integration_request_votes_table.php feat(integration-requests): community board to request & vote bank integrations (#550) 2026-06-17 12:50:51 +00:00
2026_06_17_120850_add_status_to_integration_requests_table.php feat(integration-requests): community board to request & vote bank integrations (#550) 2026-06-17 12:50:51 +00:00
2026_06_17_122001_seed_initial_integration_requests.php feat(integration-requests): community board to request & vote bank integrations (#550) 2026-06-17 12:50:51 +00:00
2026_06_17_141827_add_comment_to_integration_requests_table.php feat(integration-requests): add not-doable status with a public comment (#552) 2026-06-17 16:36:32 +02:00
2026_06_18_092830_drop_unique_user_vote_from_integration_request_votes_table.php feat(integration-requests): multi-vote, per-plan quota and undo (#554) 2026-06-18 10:06:08 +00:00
2026_06_19_134854_create_stuck_cohort_snapshots_table.php feat(stats): weekly paywall stuck-cohort report to Discord (#563) 2026-06-19 14:12:51 +00:00
2026_06_20_105609_align_accounts_encrypted_flag_with_plaintext_names.php refactor(encryption): strip client-side transaction encryption (#514) 2026-06-20 16:13:26 +00:00
2026_06_20_165235_add_position_to_accounts_table.php feat(accounts): reorder accounts with drag-and-drop (#575) 2026-06-21 11:17:45 +02:00
2026_06_26_124609_add_ai_model_to_transactions_table.php feat(ai): record the model behind each AI categorization (#594) 2026-06-26 12:53:55 +00:00
2026_06_27_000000_backfill_xxx_account_currencies.php fix(open-banking): stop storing the XXX no-currency placeholder on accounts (#602) 2026-06-27 16:01:21 +00:00
2026_06_27_114559_add_refunded_at_to_subscriptions_table.php feat(subscriptions): trial/pricing A/B/C experiment (#600) 2026-06-27 18:00:15 +02:00
2026_06_27_154041_add_hidden_on_dashboard_to_accounts_table.php feat(dashboard): add accounts manager dialog with visibility toggle and reorder (#604) 2026-06-27 16:11:25 +00:00
2026_06_29_161916_add_transactions_last_visited_at_to_users_table.php feat(transactions): make new-transaction marker cross-device (#611) 2026-06-29 19:11:37 +02:00
2026_07_01_064324_add_ai_consent_prompt_dismissed_at_to_users_table.php feat(ai): dismissable AI consent banner that stops after the first decision (#617) 2026-07-01 07:26:36 +00:00
2026_07_02_133321_add_user_source_created_at_index_to_transactions_table.php perf(db): index transactions for the daily synced-email slow query (PHP-LARAVEL-3X) (#622) 2026-07-02 15:51:15 +02:00
2026_07_06_120000_create_spaces_table.php feat(spaces): phase 0 — multi-tenant Space foundation (no behaviour change) (#650) 2026-07-09 14:26:07 +02:00
2026_07_06_120001_create_space_user_table.php feat(spaces): phase 0 — multi-tenant Space foundation (no behaviour change) (#650) 2026-07-09 14:26:07 +02:00
2026_07_06_120002_create_space_invitations_table.php feat(spaces): phase 0 — multi-tenant Space foundation (no behaviour change) (#650) 2026-07-09 14:26:07 +02:00
2026_07_06_120003_add_current_space_id_to_users_table.php feat(spaces): phase 0 — multi-tenant Space foundation (no behaviour change) (#650) 2026-07-09 14:26:07 +02:00
2026_07_06_120004_add_space_id_to_owned_tables.php feat(spaces): phase 0 — multi-tenant Space foundation (no behaviour change) (#650) 2026-07-09 14:26:07 +02:00
2026_07_06_120005_backfill_spaces.php feat(spaces): phase 0 — multi-tenant Space foundation (no behaviour change) (#650) 2026-07-09 14:26:07 +02:00
2026_07_17_125642_create_personal_access_tokens_table.php feat(mcp): read-only MCP server for Pro accounts (#689) 2026-07-17 16:54:15 +02:00
2026_07_17_154027_create_oauth_auth_codes_table.php feat(mcp): add OAuth 2.1 for Claude Desktop & ChatGPT connectors (Phase 3) (#691) 2026-07-17 19:10:48 +02:00
2026_07_17_154028_create_oauth_access_tokens_table.php feat(mcp): add OAuth 2.1 for Claude Desktop & ChatGPT connectors (Phase 3) (#691) 2026-07-17 19:10:48 +02:00
2026_07_17_154029_create_oauth_refresh_tokens_table.php feat(mcp): add OAuth 2.1 for Claude Desktop & ChatGPT connectors (Phase 3) (#691) 2026-07-17 19:10:48 +02:00
2026_07_17_154030_create_oauth_clients_table.php feat(mcp): add OAuth 2.1 for Claude Desktop & ChatGPT connectors (Phase 3) (#691) 2026-07-17 19:10:48 +02:00
2026_07_17_154031_create_oauth_device_codes_table.php feat(mcp): add OAuth 2.1 for Claude Desktop & ChatGPT connectors (Phase 3) (#691) 2026-07-17 19:10:48 +02:00
2026_07_18_104433_create_account_import_configs_table.php feat(import): persist per-account import configuration on the backend (#698) 2026-07-18 16:10:40 +02:00
2026_07_18_111320_add_upsell_source_to_subscriptions_table.php feat: reuse the upgrade modal at more upsell points and attribute revenue (#699) 2026-07-18 12:53:20 +00:00