RomanNum3ral
/
whisper-money
mirror of https://github.com/whisper-money/whisper-money.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

feat(import): persist per-account import configuration on the backend (#698) 

## What & why

Import configuration (the CSV/Excel column mapping and date format a
user
sets up per account) was stored in the browser's `localStorage`, so it
never
followed the user to another device. Importing the same account's
statement on
mobile — or on a new machine — meant reconfiguring the mapping from
scratch.

This moves that configuration to the backend, keyed per account and per
import
type, so it's preconfigured and loaded automatically wherever the user
imports.

## Demo

Cross-device QA: import transactions on "device 1" with a custom column
mapping, then clear **all** cookies + local/session storage (simulating
a
fresh device), log back in, and start a new import — the mapping
(Description → *Movement*, Amount → *How Much*, date format
*DD-MM-YYYY*)
is auto-loaded from the backend with no manual setup.


https://github.com/user-attachments/assets/e449dfe9-3970-48d4-97ca-9e415c73835b



## How

- New `account_import_configs` table: one row per `(account_id, type)`
where
`type` is `transaction` or `balance`. The mapping + date format are
stored as
  an opaque `config` JSON blob (exactly what the client sends).
- `GET/PUT /api/accounts/{account}/import-config`
(`AccountImportConfigController`),
authorized via the existing `AccountPolicy` (`view`/`update`) — mirrors
  `AccountBalanceController`. `PUT` upserts on `(account_id, type)`.
- Frontend: the two import-config storage helpers now read/write the
endpoint
via `axios` instead of `localStorage` (merged into a single module — the
  transaction and balance variants shared the same logic).
- The saved config is fetched **off the file-parse critical path**: the
parsed
file shows immediately with auto-detected columns, and the saved mapping
is
applied when it arrives (guarded so a slow load can't clobber a file
picked
afterwards). A slow or hanging request never blocks the preview or Next
  button. When no config exists or the request fails, it falls back to
  auto-detection exactly as before.

## Notes / decisions

- **No localStorage migration.** Existing per-device configs aren't
migrated;
on the next import the mapping is auto-detected and re-saved to the
backend,
so it self-heals after one import. The old `import_config_account_*`
keys are
  simply no longer read.
- The mapping-validity check against the actual file headers stays
client-side
  (it depends on the just-parsed file).

## Testing

- `tests/Feature/AccountImportConfigTest.php` (9 tests): auth required,
  cross-user 403 on read and write, save→persist→load round-trip, upsert
  de-duplication, transaction/balance independence, and validation
  (unknown type, missing column mapping).
- Sibling suites green (`AccountBalanceControllerTest`,
`SavedFilterTest`) —
no regression from the new `Account::importConfigs()` relation or
routes.
This commit is contained in:
Víctor Falcón 2026-07-18 16:10:40 +02:00 committed by GitHub
parent 2b2e4c4a87
commit 4c83cb8b33
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
13 changed files with 433 additions and 136 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
app/Enums/ImportConfigType.php Normal file
View File

@ -0,0 +1,9 @@
<?php
namespace App\Enums;
enum ImportConfigType: string
{
case Transaction = 'transaction';
case Balance = 'balance';
}

44
app/Http/Controllers/Api/AccountImportConfigController.php Normal file
View File

@ -0,0 +1,44 @@
<?php
namespace App\Http\Controllers\Api;
use App\Enums\ImportConfigType;
use App\Http\Controllers\Controller;
use App\Http\Requests\Api\UpdateAccountImportConfigRequest;
use App\Models\Account;
use Illuminate\Foundation\Auth\Access\AuthorizesRequests;
use Illuminate\Http\JsonResponse;
use Illuminate\Http\Request;
use Illuminate\Validation\Rule;
class AccountImportConfigController extends Controller
{
use AuthorizesRequests;
public function show(Request $request, Account $account): JsonResponse
{
$this->authorize('view', $account);
$validated = $request->validate([
'type' => ['required', Rule::enum(ImportConfigType::class)],
]);
$config = $account->importConfigs()
->where('type', $validated['type'])
->first();
return response()->json(['data' => $config?->config]);
}
public function update(UpdateAccountImportConfigRequest $request, Account $account): JsonResponse
{
$this->authorize('update', $account);
$config = $account->importConfigs()->updateOrCreate(
['type' => $request->validated('type')],
['config' => $request->validated('config')],
);
return response()->json(['data' => $config->config]);
}
}

32
app/Http/Requests/Api/UpdateAccountImportConfigRequest.php Normal file
View File

@ -0,0 +1,32 @@
<?php
namespace App\Http\Requests\Api;
use App\Enums\ImportConfigType;
use Illuminate\Contracts\Validation\ValidationRule;
use Illuminate\Foundation\Http\FormRequest;
use Illuminate\Validation\Rule;
class UpdateAccountImportConfigRequest extends FormRequest
{
/**
* Authorization is handled by the controller via the AccountPolicy.
*/
public function authorize(): bool
{
return true;
}
/**
* @return array<string, ValidationRule|array<mixed>|string>
*/
public function rules(): array
{
return [
'type' => ['required', Rule::enum(ImportConfigType::class)],
'config' => ['required', 'array'],
'config.columnMapping' => ['required', 'array'],
'config.dateFormat' => ['required', 'string', 'max:20'],
];
}
}

6
app/Models/Account.php
View File

@ -104,6 +104,12 @@ class Account extends Model
return $this->hasMany(AccountBalance::class);
}
/** @return HasMany<AccountImportConfig, $this> */
public function importConfigs(): HasMany
{
return $this->hasMany(AccountImportConfig::class);
}
/** @return BelongsTo<BankingConnection, $this> */
public function bankingConnection(): BelongsTo
{

36
app/Models/AccountImportConfig.php Normal file
View File

@ -0,0 +1,36 @@
<?php
namespace App\Models;
use App\Enums\ImportConfigType;
use Database\Factories\AccountImportConfigFactory;
use Illuminate\Database\Eloquent\Concerns\HasUuids;
use Illuminate\Database\Eloquent\Factories\HasFactory;
use Illuminate\Database\Eloquent\Model;
use Illuminate\Database\Eloquent\Relations\BelongsTo;
class AccountImportConfig extends Model
{
/** @use HasFactory<AccountImportConfigFactory> */
use HasFactory, HasUuids;
protected $fillable = [
'account_id',
'type',
'config',
];
protected function casts(): array
{
return [
'type' => ImportConfigType::class,
'config' => 'array',
];
}
/** @return BelongsTo<Account, $this> */
public function account(): BelongsTo
{
return $this->belongsTo(Account::class);
}
}

36
database/factories/AccountImportConfigFactory.php Normal file
View File

@ -0,0 +1,36 @@
<?php
namespace Database\Factories;
use App\Enums\ImportConfigType;
use App\Models\Account;
use App\Models\AccountImportConfig;
use Illuminate\Database\Eloquent\Factories\Factory;
/**
* @extends Factory<AccountImportConfig>
*/
class AccountImportConfigFactory extends Factory
{
/**
* @return array<string, mixed>
*/
public function definition(): array
{
return [
'account_id' => Account::factory(),
'type' => ImportConfigType::Transaction,
'config' => [
'columnMapping' => [
'transaction_date' => 'Date',
'description' => 'Description',
'amount' => 'Amount',
'balance' => null,
'creditor_name' => null,
'debtor_name' => null,
],
'dateFormat' => 'YYYY-MM-DD',
],
];
}
}

32
database/migrations/2026_07_18_104433_create_account_import_configs_table.php Normal file
View File

@ -0,0 +1,32 @@
<?php
use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;
return new class extends Migration
{
/**
* Run the migrations.
*/
public function up(): void
{
Schema::create('account_import_configs', function (Blueprint $table) {
$table->uuid('id')->primary();
$table->foreignUuid('account_id')->constrained()->cascadeOnDelete();
$table->string('type');
$table->json('config');
$table->timestamps();
$table->unique(['account_id', 'type']);
});
}
/**
* Reverse the migrations.
*/
public function down(): void
{
Schema::dropIfExists('account_import_configs');
}
};

66
resources/js/components/accounts/import-balances-drawer.tsx
View File

@ -8,10 +8,6 @@ import {
DrawerTitle,
} from '@/components/ui/drawer';
import { Progress } from '@/components/ui/progress';
import {
loadBalanceImportConfig,
saveBalanceImportConfig,
} from '@/lib/balance-import-config-storage';
import { getCsrfToken } from '@/lib/csrf';
import {
detectDateFormat,
@ -19,6 +15,10 @@ import {
parseDate,
parseFile,
} from '@/lib/file-parser';
import {
loadBalanceImportConfig,
saveBalanceImportConfig,
} from '@/lib/import-config-storage';
import { type SharedData } from '@/types';
import { supportsInvestedAmount, type Account } from '@/types/account';
import {
@ -279,13 +279,23 @@ export function ImportBalancesDrawer({
}
}
let finalMapping = autoMapping;
let finalDateFormat = detectedFormat;
// Show the parsed file immediately with auto-detected columns; the
// saved per-account config is fetched off the critical path so a
// slow or hanging network never blocks the preview or Next button.
setState((prev) => ({
...prev,
file,
parsedData: data,
columnHeaders: headers,
columnOptions,
columnMapping: autoMapping,
dateFormat: detectedFormat,
dateFormatDetected: formatDetected,
}));
if (state.selectedAccountId) {
const savedConfig = loadBalanceImportConfig(
state.selectedAccountId,
);
const accountId = state.selectedAccountId;
if (accountId) {
const savedConfig = await loadBalanceImportConfig(accountId);
if (savedConfig) {
const isValidMapping = (
@ -300,26 +310,24 @@ export function ImportBalancesDrawer({
};
if (isValidMapping(savedConfig.columnMapping)) {
finalMapping = savedConfig.columnMapping;
finalDateFormat = savedConfig.dateFormat;
// Keep the saved format as the default, but still show
// the selector when the dates are ambiguous so a
// Only apply if this file is still the selected one, so a
// slow load can't clobber a file picked afterwards. Keep
// the saved format as the default, but still show the
// selector when the dates are ambiguous so a
// previously-saved wrong format can be corrected.
formatDetected = !formatAmbiguous;
}
}
}
setState((prev) => ({
setState((prev) =>
prev.file === file
? {
...prev,
file,
parsedData: data,
columnHeaders: headers,
columnOptions,
columnMapping: finalMapping,
dateFormat: finalDateFormat,
dateFormatDetected: formatDetected,
}));
columnMapping: savedConfig.columnMapping,
dateFormat: savedConfig.dateFormat,
dateFormatDetected: !formatAmbiguous,
}
: prev,
);
}
}
}
} catch (err) {
setError(
err instanceof Error ? err.message : 'Failed to parse file',
@ -390,7 +398,7 @@ export function ImportBalancesDrawer({
}
if (state.selectedAccountId) {
saveBalanceImportConfig(state.selectedAccountId, {
void saveBalanceImportConfig(state.selectedAccountId, {
columnMapping: state.columnMapping,
dateFormat: state.dateFormat,
});

56
resources/js/components/transactions/import-transactions-drawer.tsx
View File

@ -223,11 +223,23 @@ export function ImportTransactionsDrawer({
}
}
let finalMapping = autoMapping;
let finalDateFormat = detectedFormat;
// Show the parsed file immediately with auto-detected columns; the
// saved per-account config is fetched off the critical path so a
// slow or hanging network never blocks the preview or Next button.
setState((prev) => ({
...prev,
file,
parsedData: data,
columnHeaders: headers,
columnOptions,
columnMapping: autoMapping,
dateFormat: detectedFormat,
dateFormatDetected: formatDetected,
}));
if (state.selectedAccountId) {
const savedConfig = loadImportConfig(state.selectedAccountId);
const accountId = state.selectedAccountId;
if (accountId) {
const savedConfig = await loadImportConfig(accountId);
if (savedConfig) {
const isValidMapping = (
@ -247,26 +259,24 @@ export function ImportTransactionsDrawer({
};
if (isValidMapping(savedConfig.columnMapping)) {
finalMapping = savedConfig.columnMapping;
finalDateFormat = savedConfig.dateFormat;
// Keep the saved format as the default, but still show
// the selector when the dates are ambiguous so a
// Only apply if this file is still the selected one, so a
// slow load can't clobber a file picked afterwards. Keep
// the saved format as the default, but still show the
// selector when the dates are ambiguous so a
// previously-saved wrong format can be corrected.
formatDetected = !formatAmbiguous;
}
}
}
setState((prev) => ({
setState((prev) =>
prev.file === file
? {
...prev,
file,
parsedData: data,
columnHeaders: headers,
columnOptions,
columnMapping: finalMapping,
dateFormat: finalDateFormat,
dateFormatDetected: formatDetected,
}));
columnMapping: savedConfig.columnMapping,
dateFormat: savedConfig.dateFormat,
dateFormatDetected: !formatAmbiguous,
}
: prev,
);
}
}
}
} catch (err) {
setError(
err instanceof Error ? err.message : 'Failed to parse file',
@ -458,7 +468,7 @@ export function ImportTransactionsDrawer({
}
if (state.selectedAccountId) {
saveImportConfig(state.selectedAccountId, {
void saveImportConfig(state.selectedAccountId, {
columnMapping: state.columnMapping,
dateFormat: state.dateFormat,
});

61
resources/js/lib/balance-import-config-storage.ts
View File

@ -1,61 +0,0 @@
import type { BalanceColumnMapping } from '@/types/balance-import';
import { DateFormat } from '@/types/import';
import type { UUID } from '@/types/uuid';
interface BalanceImportConfig {
columnMapping: BalanceColumnMapping;
dateFormat: DateFormat;
}
const STORAGE_KEY_PREFIX = 'balance_import_config_account_';
export function saveBalanceImportConfig(
accountId: UUID,
config: BalanceImportConfig,
): void {
if (typeof window === 'undefined') return;
try {
const key = `${STORAGE_KEY_PREFIX}${accountId}`;
localStorage.setItem(key, JSON.stringify(config));
} catch (error) {
console.error('Failed to save balance import configuration:', error);
}
}
export function loadBalanceImportConfig(
accountId: UUID,
): BalanceImportConfig | null {
if (typeof window === 'undefined') return null;
try {
const key = `${STORAGE_KEY_PREFIX}${accountId}`;
const stored = localStorage.getItem(key);
if (!stored) {
return null;
}
const config = JSON.parse(stored) as BalanceImportConfig;
if (!config.columnMapping || !config.dateFormat) {
return null;
}
return config;
} catch (error) {
console.error('Failed to load balance import configuration:', error);
return null;
}
}
export function clearBalanceImportConfig(accountId: UUID): void {
if (typeof window === 'undefined') return;
try {
const key = `${STORAGE_KEY_PREFIX}${accountId}`;
localStorage.removeItem(key);
} catch (error) {
console.error('Failed to clear balance import configuration:', error);
}
}

76
resources/js/lib/import-config-storage.ts
View File

@ -1,55 +1,81 @@
import type { BalanceColumnMapping } from '@/types/balance-import';
import { type ColumnMapping, DateFormat } from '@/types/import';
import { type UUID } from '@/types/uuid';
import axios from 'axios';
interface ImportConfig {
columnMapping: ColumnMapping;
dateFormat: DateFormat;
}
const STORAGE_KEY_PREFIX = 'import_config_account_';
interface BalanceImportConfig {
columnMapping: BalanceColumnMapping;
dateFormat: DateFormat;
}
export function saveImportConfig(accountId: UUID, config: ImportConfig): void {
if (typeof window === 'undefined') return;
type ImportConfigType = 'transaction' | 'balance';
function configUrl(accountId: UUID): string {
return `/api/accounts/${accountId}/import-config`;
}
async function saveConfig(
accountId: UUID,
type: ImportConfigType,
config: ImportConfig | BalanceImportConfig,
): Promise<void> {
try {
const key = `${STORAGE_KEY_PREFIX}${accountId}`;
localStorage.setItem(key, JSON.stringify(config));
await axios.put(configUrl(accountId), { type, config });
} catch (error) {
console.error('Failed to save import configuration:', error);
console.error(`Failed to save ${type} import configuration:`, error);
}
}
export function loadImportConfig(accountId: UUID): ImportConfig | null {
if (typeof window === 'undefined') return null;
async function loadConfig<T extends ImportConfig | BalanceImportConfig>(
accountId: UUID,
type: ImportConfigType,
): Promise<T | null> {
try {
const key = `${STORAGE_KEY_PREFIX}${accountId}`;
const stored = localStorage.getItem(key);
const { data } = await axios.get<{ data: T | null }>(
configUrl(accountId),
{ params: { type } },
);
if (!stored) {
return null;
}
const config = data.data;
const config = JSON.parse(stored) as ImportConfig;
if (!config.columnMapping || !config.dateFormat) {
if (!config || !config.columnMapping || !config.dateFormat) {
return null;
}
return config;
} catch (error) {
console.error('Failed to load import configuration:', error);
console.error(`Failed to load ${type} import configuration:`, error);
return null;
}
}
export function clearImportConfig(accountId: UUID): void {
if (typeof window === 'undefined') return;
export function saveImportConfig(
accountId: UUID,
config: ImportConfig,
): Promise<void> {
return saveConfig(accountId, 'transaction', config);
}
try {
const key = `${STORAGE_KEY_PREFIX}${accountId}`;
localStorage.removeItem(key);
} catch (error) {
console.error('Failed to clear import configuration:', error);
export function loadImportConfig(
accountId: UUID,
): Promise<ImportConfig | null> {
return loadConfig<ImportConfig>(accountId, 'transaction');
}
export function saveBalanceImportConfig(
accountId: UUID,
config: BalanceImportConfig,
): Promise<void> {
return saveConfig(accountId, 'balance', config);
}
export function loadBalanceImportConfig(
accountId: UUID,
): Promise<BalanceImportConfig | null> {
return loadConfig<BalanceImportConfig>(accountId, 'balance');
}

5
routes/api.php
View File

@ -2,6 +2,7 @@
use App\Http\Controllers\AccountBalanceController;
use App\Http\Controllers\Api\AccountController;
use App\Http\Controllers\Api\AccountImportConfigController;
use App\Http\Controllers\Api\CashflowAnalyticsController;
use App\Http\Controllers\Api\CategoryMonthlyBreakdownController;
use App\Http\Controllers\Api\DashboardAnalyticsController;
@ -38,6 +39,10 @@ Route::middleware(['web', 'auth', 'throttle:300,1'])->group(function () {
Route::get('accounts', [AccountController::class, 'index'])->name('api.accounts.index');
Route::put('accounts/{account}', [AccountController::class, 'update'])->name('api.accounts.update');
// Account import configuration (per-account column mapping, synced across devices)
Route::get('accounts/{account}/import-config', [AccountImportConfigController::class, 'show'])->name('api.accounts.import-config.show');
Route::put('accounts/{account}/import-config', [AccountImportConfigController::class, 'update'])->name('api.accounts.import-config.update');
// Account Balances
Route::put('accounts/{account}/balance/current', [AccountBalanceController::class, 'updateCurrent'])->name('api.accounts.balance.update-current');
Route::get('accounts/{account}/balances', [AccountBalanceController::class, 'index'])->name('api.accounts.balances.index');

114
tests/Feature/AccountImportConfigTest.php Normal file
View File

@ -0,0 +1,114 @@
<?php
use App\Models\Account;
use App\Models\AccountImportConfig;
use App\Models\User;
beforeEach(function () {
$this->user = User::factory()->create();
$this->account = Account::factory()->create(['user_id' => $this->user->id]);
$this->actingAs($this->user);
});
$transactionConfig = [
'columnMapping' => [
'transaction_date' => 'Date',
'description' => 'Concept',
'amount' => 'Amount',
'balance' => null,
'creditor_name' => null,
'debtor_name' => null,
],
'dateFormat' => 'DD-MM-YYYY',
];
test('import config endpoints require authentication', function () {
auth()->logout();
$this->getJson("/api/accounts/{$this->account->id}/import-config?type=transaction")
->assertUnauthorized();
});
test('returns null when no config is saved yet', function () {
$this->getJson("/api/accounts/{$this->account->id}/import-config?type=transaction")
->assertOk()
->assertJsonPath('data', null);
});
test('saves and returns an import config for an account', function () use ($transactionConfig) {
$this->putJson("/api/accounts/{$this->account->id}/import-config", [
'type' => 'transaction',
'config' => $transactionConfig,
])
->assertOk()
->assertJsonPath('data.dateFormat', 'DD-MM-YYYY')
->assertJsonPath('data.columnMapping.description', 'Concept');
$this->assertDatabaseHas('account_import_configs', [
'account_id' => $this->account->id,
'type' => 'transaction',
]);
$this->getJson("/api/accounts/{$this->account->id}/import-config?type=transaction")
->assertOk()
->assertJsonPath('data.dateFormat', 'DD-MM-YYYY');
});
test('upserts the config instead of creating duplicates', function () use ($transactionConfig) {
$url = "/api/accounts/{$this->account->id}/import-config";
$this->putJson($url, ['type' => 'transaction', 'config' => $transactionConfig])->assertOk();
$updated = $transactionConfig;
$updated['dateFormat'] = 'YYYY-MM-DD';
$this->putJson($url, ['type' => 'transaction', 'config' => $updated])->assertOk();
expect(AccountImportConfig::where('account_id', $this->account->id)->count())->toBe(1);
$this->getJson("{$url}?type=transaction")
->assertJsonPath('data.dateFormat', 'YYYY-MM-DD');
});
test('transaction and balance configs are stored independently', function () use ($transactionConfig) {
$url = "/api/accounts/{$this->account->id}/import-config";
$balanceConfig = [
'columnMapping' => ['balance_date' => 'Date', 'balance' => 'Saldo', 'invested_amount' => null],
'dateFormat' => 'YYYY-MM-DD',
];
$this->putJson($url, ['type' => 'transaction', 'config' => $transactionConfig])->assertOk();
$this->putJson($url, ['type' => 'balance', 'config' => $balanceConfig])->assertOk();
expect(AccountImportConfig::where('account_id', $this->account->id)->count())->toBe(2);
$this->getJson("{$url}?type=balance")
->assertJsonPath('data.columnMapping.balance', 'Saldo');
});
test('cannot read the import config of another user account', function () {
$otherAccount = Account::factory()->create();
$this->getJson("/api/accounts/{$otherAccount->id}/import-config?type=transaction")
->assertForbidden();
});
test('cannot write the import config of another user account', function () use ($transactionConfig) {
$otherAccount = Account::factory()->create();
$this->putJson("/api/accounts/{$otherAccount->id}/import-config", [
'type' => 'transaction',
'config' => $transactionConfig,
])->assertForbidden();
});
test('rejects an unknown config type', function () use ($transactionConfig) {
$this->putJson("/api/accounts/{$this->account->id}/import-config", [
'type' => 'nonsense',
'config' => $transactionConfig,
])->assertJsonValidationErrors('type');
});
test('rejects a config without a column mapping', function () {
$this->putJson("/api/accounts/{$this->account->id}/import-config", [
'type' => 'transaction',
'config' => ['dateFormat' => 'YYYY-MM-DD'],
])->assertJsonValidationErrors('config.columnMapping');
});