Compare commits

...

31 Commits
v4.7.0 ... main

Author SHA1 Message Date
Martin Hauser 2f79d987a8 fix(virtualization): Restore Cluster Group in VM placement panel
Restore the Cluster Group and its link alongside the Cluster name,
matching the display before the declarative UI conversion.

Fixes #23181
2026-09-16 09:06:29 -04:00
github-actions 17a15204de Update source translation strings 2026-09-16 05:02:53 +00:00
Jeremy Stretch abccf4e036
Release v4.7.1 (#23179) 2026-09-15 14:35:13 -04:00
Jeremy Stretch a4ade2e7ae
Fixes #23112: Initiate SSO logins via a script-driven navigation (#23177)
Rendering the SSO buttons as POST forms (#23042) made every SSO login a form
submission which NetBox answers with a redirect to the identity provider.
Chromium-based browsers evaluate the CSP form-action directive against every hop
in a form submission's redirect chain, so a deployment which serves NetBox with
`form-action 'self'` blocks that redirect and the button silently does nothing.

Add SocialAuthBeginView, which wraps python-social-auth's begin view and returns
the identity provider's URL as JSON to clients which request it. The login page
now submits the form via fetch() and assigns window.location, which form-action
does not govern. The upstream view is reused as-is, so CSRF protection, the
callback URL, and the session state recorded for the identity provider are
unchanged; clients which do not request JSON (a browser without JavaScript, or a
backend which renders an HTML form rather than redirecting) receive the
unmodified response as before.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-15 13:00:11 -04:00
Jason Novinger e15b3d9080
Fixes #23166: Apply zero-valued numeric bounds to profile attribute form fields 2026-09-15 11:08:58 -04:00
Jeremy Stretch 74fbc90c69 Closes #23110: Upgrade to redis 8.1 2026-09-15 11:01:39 -04:00
bctiemann 1793874dd9
Merge pull request #23174 from netbox-community/23167-module-type-profile-desc-not-sanitized
Fixes #23167: Sanitize JSON schema property descriptions used as form help text
2026-09-15 10:30:35 -04:00
bctiemann 321a2fbf26
Fixes #23154: Correct required=False mismatch on L2VPN.type and RackType.form_factor (#23175) 2026-09-15 09:01:46 -05:00
github-actions 9fcd744c90 Update source translation strings 2026-09-15 05:02:23 +00:00
Jason Novinger f96e6f86b9 Assert the full help text rather than a fragment of it
The sanitization tests used `assertInHTML`, which checks containment. Each
needle happened to span the whole output, so they passed, but a payload
surviving outside the `rendered-markdown` div would not have failed them.
`assertHTMLEqual` is what the docstring already claimed these tests did.
2026-09-14 12:24:02 -05:00
Jason Novinger 8e68d91124 Fixes #23167: Sanitize JSON schema property descriptions used as form help text
`JSONSchemaProperty.to_form_field()` assigned a schema property's description
directly to the form field's `help_text`, which `form_helpers/render_field.html`
renders through the safe filter. Any element a module type profile schema
author put in a property description reached the DOM intact, including ones
outside `HTML_ALLOWED_TAGS`.

Pass the description through `render_markdown()`, which applies the allowlist
via `clean_html()` before `mark_safe()`. This matches how custom field
descriptions are handled in `CustomField.to_form_field()`, so both kinds of
user-defined attribute render their help text the same way.

Descriptions stored since v4.3.0 are now interpreted as Markdown, so one
beginning with "#" renders as a heading and one beginning with "1." renders
as a list item. Custom field descriptions took the same change in #12685.

Sanitizing inside `to_form_field()` rather than at the call site in
`dcim/forms/model_forms.py` means plugins calling this utility are covered too.
2026-09-14 11:52:53 -05:00
Jason Novinger fc5172f170
Closes #22999: Add a default module profile for transceivers (#23165) 2026-09-14 09:00:37 -07:00
Jason Novinger 9d96894f4e
Fixes #23130: Ensure ltree cascade triggers can be restored from a pg_dump (#23137) 2026-09-14 10:35:05 -05:00
Jason Novinger 64ce9e2db4
Closes #23041: Add InfiniBand 2X interface types (#23163)
Add a 2X (two-lane) InfiniBand group for HDR and later generations, covering
the HDR100, NDR200, and XDR400 breakout links formed by splitting a four-lane
port into two independent two-lane links.

SDR through EDR are omitted deliberately: two-lane breakout only became a
shipping configuration once per-lane rates reached 50 Gbps, so there are no
corresponding products at those generations.
2026-09-14 08:20:07 -07:00
bctiemann 07975fda34
Merge pull request #23164 from netbox-community/23012-eszett-in-search
Fixes #23012: Respect column collation when filtering case-insensitively
2026-09-11 05:26:23 -04:00
Jason Novinger 9782be4cd5 Assert the collation reaches the query, and clarify the documented behaviour
The existing tests assert on filter results, which stay correct for ASCII values
even when the collation is never applied. Add a test which asserts on the
lookup's own compiled output, so that the mechanism failing open is caught
rather than passing silently.

Explain why the placeholder is compared literally: a field declaring its own
get_placeholder() compiles to something other than '%s', and splicing a COLLATE
clause into that is not safe, so any other right-hand side is left alone.

The documentation note described the folding as specific to the German
eszett. It is the common example rather than the rule: the collation treats a
character as equivalent to the sequence it expands to in upper case, which also
covers ligatures. Note too that a case-insensitive exact match on a collated
field may now return more than one object.
2026-09-11 03:50:47 -05:00
Jason Novinger 2769e3d9d9 Fixes #23012: Respect column collation when filtering case-insensitively
Django's PostgreSQL backend compiles icontains, iexact, istartswith and
iendswith as UPPER(col::text) LIKE UPPER(%s). UPPER() folds according to the
collation of its argument, and the two sides do not share one: the column folds
under its own collation while the parameter folds under the database default.

For a column using natural_sort, UPPER('ß') is 'SS' on the left and 'ß' on the
right, so searching for 'ß' matched nothing. This affects the name field of most
models and all four case-insensitive lookups, which are also exposed through the
REST API as __ic, __ie, __isw and __iew.

Apply the column's collation to the parameter as well, inside the UPPER() call,
so that both sides fold the same way. Matching on those fields becomes
bidirectional, so 'Strasse' finds 'Straße' and vice versa. Fields without the
collation are unchanged.

The lookups only collate a bare column compared against a simple value. An
expression which already carries an explicit collation, such as Collate() or
CollateAsChar(), would otherwise raise a collation mismatch error.
2026-09-11 03:18:31 -05:00
github-actions 6385c09837 Update source translation strings 2026-09-10 05:02:27 +00:00
bctiemann 5de246563b
Merge pull request #23144 from netbox-community/23096-partial-cable-length-saves-leave-the-normalized-length-out
Fixes #23096: Keep normalized cable length in sync during partial saves
2026-09-09 14:03:29 -04:00
github-actions d2191e0fb3 Update source translation strings 2026-09-09 05:02:18 +00:00
Martin Hauser dfb99e1f69
Fixes #23125: Add missing standard REST API fields for VLAN Translation Policies and Rules (#23127) 2026-09-08 13:40:29 -05:00
Martin Hauser 90675dbbab
fix(dcim): Persist normalized cable length on partial saves
Cable.save() recomputed _abs_length in memory but never added it to
update_fields, so a save naming length or length_unit left the stored
normalized value stale. Derive it from the values the row will hold
after the save and persist it alongside its source fields.

Fixes #23096
2026-09-08 18:53:51 +02:00
Martin Hauser 7b56158d47
Closes #23145: Prevent advisory lock cleanup races in Custom Field tests (#23146) 2026-09-08 11:43:20 -05:00
Martin Hauser 7ae8e4461f
fix(filters): Preserve contains lookup for negated multiselect filters (#23128)
Add FILTER_ARRAY_BASED_LOOKUP_MAP to maintain 'contains' lookup under
negation for MultiValueArrayFilter, preventing fallback to exact match.
Negation now correctly excludes objects whose array contains the value
rather than matching it exactly.

Fixes #23117
2026-09-08 08:43:08 -07:00
Martin Hauser 5685c5218e
Revert "Fixes #23097: Prevent duplicate Cable Paths when Cable Terminations …" (#23149)
This reverts commit 1745a7d9aa.
2026-09-08 17:25:35 +02:00
Martin Hauser 6895fb76c0
Fixes #23120: Fix REST API serialization and assignment of Data Source tags (#23126) 2026-09-08 09:44:47 -05:00
github-actions 46b6a17ae0 Update source translation strings 2026-09-08 05:02:43 +00:00
Arthur Hanson c9a62254d7
Fixes #22750: Validate Custom Script input and resolve object IDs in the REST API (#23119)
Validate REST script input before enqueueing jobs. Resolve ObjectVar
IDs to model instances and MultiObjectVar IDs to querysets, returning
HTTP 400 with errors nested under data when validation fails.

Share form preparation between the API and UI, including multi-value
defaults, while keeping validation out of the job runner to preserve
other execution paths. Exclude only known execution fields from script
data and prevent _notifications from leaking into CLI script input.

Document the REST compatibility changes, including required-field
validation and discarded undeclared keys. Add regression coverage for
object resolution, defaults, validation errors, and execution options.

Co-authored-by: Martin Burggraf <martin.burggraf@netclusive.com>
2026-09-07 13:11:14 +02:00
github-actions eaf30a6fb0 Update source translation strings 2026-09-05 05:02:38 +00:00
Martin Hauser 1745a7d9aa
Fixes #23097: Prevent duplicate Cable Paths when Cable Terminations are unchanged (#23100)
* fix(dcim): Prevent path rebuild when Cable Terminations unchanged

Compare Cable Terminations against stored values instead of the empty
cache when checking for modifications, so a freshly loaded Cable that is
resaved with the same terminations no longer rebuilds its paths. Raise
the flag whenever update_terminations() force-recreates an end, since
the edit form warms the cache that gated it and a profile change then
tore every path down without rebuilding it. Add regression tests for
both.

Fixes #23097

* fix(dcim): Preserve cable end order when terminations unchanged

Compare cable terminations against stored values instead of potentially
stale prefetched relations when checking for modifications. Skip setting
terminations in the form's clean() when a saved cable's members are
unchanged, preserving the connector order assigned by the profile.
2026-09-04 14:02:59 -04:00
Jason Novinger 2d519ece58 Fixes #22569: Ensures that Script Run OpenAPI operation is present
Does two things:
1. Adds a regression test to ensure that the `extras_scripts_run`
   operation is always present in contrib/openapi.json. This has
   regressed at least once since original implementation, so I wanted to
   make sure we catch it quickly in the future.
2. Overrides the Django `CACHES` setting for the OpenAPISchemaTestCase,
   which contains the new test, so that caching of the schema is
   disabled. This caused problems by masking whether or not the
   regression test (and other existing tests) were failing/succeeding in
   response to changes or not.
2026-09-03 08:55:12 -04:00
102 changed files with 8484 additions and 5050 deletions

View File

@ -18,10 +18,6 @@ django-debug-toolbar
# https://github.com/carltongibson/django-filter/blob/main/CHANGES.rst
django-filter
# Django Debug Toolbar extension for GraphiQL
# https://github.com/flavors/django-graphiql-debug-toolbar/blob/main/CHANGES.rst
django-graphiql-debug-toolbar
# HTMX utilities for Django
# https://django-htmx.readthedocs.io/en/latest/changelog.html
django-htmx
@ -143,8 +139,7 @@ PyYAML
# redis-py
# https://github.com/redis/redis-py
# Default protocol changes to RESP3 in v8.0; see #22388
redis<8.0
redis
# Requests
# https://github.com/psf/requests/blob/main/HISTORY.md

View File

@ -512,6 +512,9 @@
"infiniband-hdr",
"infiniband-ndr",
"infiniband-xdr",
"infiniband-hdr-2x",
"infiniband-ndr-2x",
"infiniband-xdr-2x",
"infiniband-sdr-4x",
"infiniband-ddr-4x",
"infiniband-qdr-4x",

File diff suppressed because one or more lines are too long

View File

@ -41,6 +41,12 @@ NetBox supports single sign-on authentication via the [python-social-auth](https
Most remote authentication backends require some additional configuration through settings prefixed with `SOCIAL_AUTH_`. These will be automatically imported from NetBox's `configuration.py` file. Additionally, the [authentication pipeline](https://python-social-auth.readthedocs.io/en/latest/pipeline.html) can be customized via the `SOCIAL_AUTH_PIPELINE` parameter. (NetBox's default pipeline is defined in `netbox/settings.py` for your reference.)
!!! note "Content Security Policy"
Beginning an SSO login requires the browser to make a request back to NetBox before it is sent
on to the identity provider. If you serve NetBox with a Content Security Policy which does not
permit same-origin connections, SSO logins will fail: add `connect-src 'self'` (or a
`default-src` which covers it) to your policy.
#### Configuring the SSO module's appearance
The way a remote authentication backend is displayed to the user on the login

View File

@ -44,6 +44,60 @@ By default, only those objects whose cache is empty are rendered, so the command
python3 netbox/manage.py rebuild_config_context_cache [--force]
```
## rebuild_ltree_paths
Recompute the `path` and `sort_path` columns of the hierarchical models (regions, site groups, locations, device roles, platforms, tenant groups, contact groups, wireless LAN groups, module bays, inventory items, and inventory item templates) from their parent relationships. These columns are maintained by PostgreSQL triggers, so this is needed only where a write bypassed them: a bulk `COPY`, a direct `UPDATE`, or a database restored from a NetBox v4.7.0 dump (see [#23130](https://github.com/netbox-community/netbox/issues/23130)).
The command has two modes. Both operate on every hierarchical model by default, or on those named as `app_label.ModelName`.
### Reporting
`--check` compares each object's stored `path` and `sort_path` against its parent's and reports which models disagree. It modifies nothing and takes no locks, so it can be run on a live system or against a replica.
```
python3 netbox/manage.py rebuild_ltree_paths --check
```
```no-highlight
dcim.location: 5 path, 5 sort_path row(s) out of date
dcim.region: 2 sort_path row(s) out of date
...
Needs rebuilding: dcim.location dcim.region
```
The counts answer whether a model needs rebuilding, not how many of its objects are wrong. Where an object has moved, the objects beneath it still agree with their own parent and are not counted, though they are equally stale. Rebuild the whole model rather than acting on the number.
A model can also be damaged in a way `--check` does not report: an object which no root reaches by following `parent_id` is compared against a parent that is itself unreachable, so it may agree and be counted clean. The rebuild detects that case and refuses (see below).
### Rebuilding
With no `--check`, each named model is rebuilt: every row's `path` and `sort_path` are recomputed from the hierarchy.
```
python3 netbox/manage.py rebuild_ltree_paths [app_label.ModelName ...]
```
```no-highlight
dcim.region: rebuilding... done
Finished.
```
A rebuild derives each object's path by walking down from the roots, so it can only repair an object which some root reaches. Where a model contains an object no root reaches — one in a cycle, one parented to itself, or one whose parent no longer exists — the command reports how many and stops without modifying that model, because a rebuild would silently skip exactly those objects:
```no-highlight
CommandError: dcim.region: 5 row(s) cannot be reached from a root by following
parent_id, so a rebuild would skip them: 1, 2, 3, 4, 5. Correct the parent
relationships, then re-run.
```
One of the listed objects is in a cycle, parented to itself, or pointing at an object which no longer exists; the rest are descended from it and are otherwise intact. Correcting the relationship is left to the operator, as only they can say what the hierarchy was meant to be. Each model is checked and rebuilt in its own transaction, so a refusal leaves that model untouched, and models already rebuilt stay rebuilt.
!!! warning
A rebuild rewrites every row of each named model in a single statement, locking those rows until it commits. On a large table this blocks concurrent writes for minutes, so run it during a maintenance window. Use `--check` first to limit the rebuild to the models which need it.
A rebuild also assumes nothing else is changing the hierarchy while it runs. An object reparented after the command has checked the model, but before it rewrites it, is not accounted for, and the check which refuses unreachable objects cannot see it either. This is another reason to run the command with writes paused rather than against a live system.
## rebuild_prefixes
Rebuild the IPAM prefix hierarchy, recalculating the depth and child counts for all prefixes.

View File

@ -0,0 +1,74 @@
# Repairing Hierarchical Paths
NetBox stores each hierarchical object's position in its tree in a PostgreSQL [`ltree`](https://www.postgresql.org/docs/current/ltree.html) column named `path`, and most such models additionally maintain a `sort_path` used to order children by name. Both columns are maintained by database triggers which cascade a change to an object's name or parent down to its descendants.
This page covers detecting and repairing stale values in those columns. It applies to the nested group models (region, site group, location, device role, platform, tenant group, contact group, wireless LAN group) as well as module bays, inventory items, and inventory item templates.
## Databases Restored From a v4.7.0 Dump
In NetBox v4.7.0, the cascade triggers could not be recreated when restoring a `pg_dump` of the database, because `pg_dump` resets the `search_path` and the triggers' `WHEN` clause depended on it. As `psql` does not stop on error by default, such a restore reported success while leaving the database without those triggers. Renaming or moving an affected object therefore did not update its descendants, and the stored paths drifted out of sync with the actual hierarchy. This was corrected in NetBox v4.7.1 ([#23130](https://github.com/netbox-community/netbox/issues/23130)).
Upgrading to v4.7.1 or later reinstalls the triggers, so all subsequent changes are cascaded correctly. It does **not** repair values which have already gone stale — use the checks below to determine whether a repair is needed.
!!! tip
To avoid this class of failure in general, always restore a dump with `psql -v ON_ERROR_STOP=1` (or `pg_restore --exit-on-error`), as described under [Replicating NetBox](./replicating-netbox.md#load-an-exported-database).
## Checking for Stale Paths
### After Upgrading
The [`rebuild_ltree_paths`](./management-commands.md#rebuild_ltree_paths) management command reports which models are affected without modifying anything or taking any locks:
```no-highlight
python netbox/manage.py rebuild_ltree_paths --check
```
### Before Upgrading
The same test can be run as SQL against a deployment which has not yet been upgraded. Substitute each hierarchical table in turn: `dcim_region`, `dcim_sitegroup`, `dcim_location`, `dcim_devicerole`, `dcim_platform`, `dcim_modulebay`, `dcim_inventoryitem`, `dcim_inventoryitemtemplate`, `tenancy_tenantgroup`, `tenancy_contactgroup`, and `wireless_wirelesslangroup`.
```no-highlight
SELECT count(*) FROM (
SELECT id FROM dcim_region WHERE parent_id IS NULL
AND path <> lpad(id::text, 19, '0')::ltree
UNION ALL
SELECT c.id FROM dcim_region c JOIN dcim_region p ON c.parent_id = p.id
WHERE c.path <> p.path || lpad(c.id::text, 19, '0')::ltree
) x;
```
Treat any non-zero result as "this table needs rebuilding" rather than as a count of the damage: an object whose ancestor moved is reported, but its own descendants are consistent with it and so are not, even though they are equally stale.
### Checking `sort_path`
The nine tables which order their children by name additionally maintain a `sort_path`, which can go stale on a rename even when `path` is correct. Every table in the list above except `dcim_inventoryitem` and `dcim_inventoryitemtemplate` carries one, and is checked with:
```no-highlight
SELECT count(*) FROM (
SELECT id FROM dcim_region WHERE parent_id IS NULL AND sort_path <> name
UNION ALL
SELECT c.id FROM dcim_region c JOIN dcim_region p ON c.parent_id = p.id
WHERE c.sort_path <> p.sort_path || chr(9) || c.name
) x;
```
Stale `sort_path` values affect only the order in which objects are listed. A stale `path`, by contrast, misplaces an object within the hierarchy, so it can be omitted from its ancestor's list of descendants.
## Repairing
Repair an affected table with the [`rebuild_ltree_paths`](./management-commands.md#rebuild_ltree_paths) management command, naming the models the queries above flagged:
```no-highlight
python netbox/manage.py rebuild_ltree_paths dcim.region
```
!!! warning
A rebuild rewrites every row of the named tables, locking those rows until it commits, so run it during a maintenance window.
Should the command report that a table contains rows unreachable from any root, the parent relationships themselves need correcting first: a rebuild walks down from the roots and would skip those rows.
## Plugins
Plugins which maintain their own `ltree` models via the `InstallLtreeTriggers` migration operation are affected in the same way, and their tables are not touched by NetBox's own corrective migrations. Where such a database was restored from a dump, the plugin's cascade triggers are missing entirely; where it was upgraded in place, they carry the old definition and will be lost by its next dump.
Either way, a new plugin migration applying `ReinstallLtreeTriggers` (passing the same `name_column` as the original) installs the corrected definitions. Use that operation rather than `InstallLtreeTriggers`: both drop each trigger before recreating it, so either works going forwards, but reversing the corrective migration should not undo the original installation. `InstallLtreeTriggers` reverses by dropping both triggers and their functions, which would leave the table with no path maintenance while the migration that first installed them remains applied. `ReinstallLtreeTriggers` reverses to a no-op instead.

View File

@ -34,9 +34,16 @@ When restoring a database from a file, it's recommended to delete any existing d
```no-highlight
psql -c 'drop database netbox'
psql -c 'create database netbox'
psql netbox < netbox.sql
psql -v ON_ERROR_STOP=1 netbox < netbox.sql
```
!!! warning "Always restore with ON_ERROR_STOP"
By default, `psql` continues after an error and still exits with status 0. A restore which failed partway through, leaving out an index, a function, or a trigger, therefore reports success and yields a database which looks healthy but is incomplete. Passing `-v ON_ERROR_STOP=1` makes `psql` abort on the first error and exit non-zero, so check the exit status before putting the restored database into service.
This changes the behavior of the restore: a dump which previously appeared to restore successfully will now abort on its first error, including errors unrelated to NetBox's own schema (a role which already exists, an extension owned by another user, and so on). That is the intended outcome, but expect a restore which used to "succeed" to start reporting failures which were there all along.
For a dump in one of `pg_dump`'s non-plain formats, restore it with `pg_restore --exit-on-error` instead.
Keep in mind that PostgreSQL user accounts and permissions are not included with the dump: You will need to create those manually if you want to fully replicate the original database (see the [installation docs](../installation/1-postgresql.md)). When setting up a development instance of NetBox, it's strongly recommended to use different credentials anyway.
### Export the Database Schema

View File

@ -21,14 +21,14 @@ flowchart BT
modulebay1 & modulebay2 & modulebay3 --> device[Device]
```
### 1. Create an SFP Module Type Profile
### 1. Select an SFP Module Type Profile
If one has not already been defined, create a [module type profile](../models/dcim/moduletypeprofile.md) for SFPs. This profile will be assigned for all module types which represent a pluggable transceiver. Typically, you will need only one profile for all pluggable transceivers.
New NetBox installations include a "Transceiver" [module type profile](../models/dcim/moduletypeprofile.md), which you can select for all module types which represent a pluggable transceiver. Typically, you will need only one profile for all pluggable transceivers. If this profile is not present, or if you prefer a different set of attributes, create your own profile for SFPs instead.
You might opt to define custom attributes for the profile by defining a custom [JSON schema](https://json-schema.org/). Profile attributes might be used to define characteristics unique to transceivers, such as optical wavelength and power ranges. Adding profile attributes is optional, and can be done at a later point.
The default profile defines attributes for form factor, media, PHY, data rate, reach, and connector type. You might opt to add or replace these by editing the profile's [JSON schema](https://json-schema.org/). Profile attributes might be used to define characteristics unique to transceivers, such as optical wavelength and power ranges. Adding profile attributes is optional, and can be done at a later point.
!!! note
Creating a module type profile is optional, but recommended as it allows for defining custom module attributes.
Assigning a module type profile is optional, but recommended as it allows for defining custom module attributes.
### 2. Create a Module Type for Each SFP Model in Inventory

View File

@ -301,6 +301,9 @@ All custom script variables support the following default options:
* `required` - Indicates whether the field is mandatory (all fields are required by default)
* `widget` - The class of form widget to use (see the [Django documentation](https://docs.djangoproject.com/en/stable/ref/forms/widgets/))
!!! warning "Reserved variable names"
The names `_commit`, `_schedule_at`, `_interval`, and `_notifications` are reserved for the execution parameters which NetBox renders alongside a script's own fields. A variable declared with one of these names shadows its execution parameter, and its value is not passed to `run()`. Choose a different name.
### StringVar
Stores a string of characters (i.e. text). Options include:
@ -546,6 +549,9 @@ http://netbox/api/extras/scripts/example.MyReport/ \
Optionally `schedule_at` can be passed in the form data with a datetime string to schedule a script at the specified date and time.
!!! note
Script input submitted through the REST API is validated against the variables declared by the script. Missing required variables or invalid values result in an HTTP 400 response, and undeclared keys are discarded rather than passed to `run()`. Existing API clients that relied on the previous pass-through behavior may need to update their requests. Scripts declaring a `FileVar` must be run via a `multipart/form-data` request, passing `data` as a JSON string alongside the uploaded file.
### Via the CLI
Scripts can be run on the CLI by invoking the management command:

View File

@ -101,6 +101,16 @@ Here is an example of a lookup expression on a string field that will return all
GET /api/dcim/devices/?name__ic=switch
```
!!! note "Case-insensitive matching depends on the field's collation"
Most `name` fields use a database collation which sorts them in natural order, so that
`device-2` precedes `device-10`. Case-insensitive matching on those fields follows the
same collation, which treats a character as equivalent to the sequence it expands to in
upper case. The German `ß` is the common example: a search for `Strasse` matches a
device named `Straße`, and vice versa. Ligatures such as `fi` behave the same way. One
consequence is that a case-insensitive exact match on such a field may return more than
one object. Fields which do not use this collation, such as `serial` and `description`,
match these characters literally.
### Foreign Keys & Other Fields
Certain other fields, namely foreign key relationships support just the negation

View File

@ -1,5 +1,31 @@
# NetBox v4.7
## v4.7.1 (2026-09-15)
!!! warning "Databases Restored From a v4.7.0 Dump"
The triggers which cascade a hierarchical object's path to its descendants could not be recreated when restoring a `pg_dump` of a v4.7.0 database, so such a restore reported success while leaving the database without those triggers. Renaming or moving a region, site group, location, device role, platform, tenant group, contact group, wireless LAN group, module bay, inventory item, or inventory item template then did not update its descendants. Upgrading reinstalls the triggers so that all subsequent changes cascade correctly, but does **not** repair values which have already gone stale. See [Repairing Hierarchical Paths](../administration/repairing-hierarchical-paths.md) for how to detect and correct them, and for the steps plugins maintaining their own `ltree` models must take.
### Enhancements
* [#22999](https://github.com/netbox-community/netbox/issues/22999) - Add a default module type profile for transceivers
* [#23041](https://github.com/netbox-community/netbox/issues/23041) - Add InfiniBand 2X interface types for HDR and later generations (HDR100, NDR200, and XDR400)
### Bug Fixes
* [#22750](https://github.com/netbox-community/netbox/issues/22750) - Resolve object IDs to model instances and validate the submitted data when executing a custom script via the REST API
* [#23012](https://github.com/netbox-community/netbox/issues/23012) - Apply a field's collation to both sides of a case-insensitive comparison, so that names containing characters such as `ß` can be matched
* [#23096](https://github.com/netbox-community/netbox/issues/23096) - Persist a cable's normalized length when saving only its `length` or `length_unit` field
* [#23112](https://github.com/netbox-community/netbox/issues/23112) - Initiate SSO logins via script-driven navigation, so that they are not blocked by a restrictive `form-action` content security policy
* [#23117](https://github.com/netbox-community/netbox/issues/23117) - Fix the negation (`__n`) filter lookup for multiple selection custom fields
* [#23120](https://github.com/netbox-community/netbox/issues/23120) - Include tags in the REST API representation of a data source, and honor them on write
* [#23125](https://github.com/netbox-community/netbox/issues/23125) - Add the missing standard fields to the VLAN translation policy & rule REST API serializers
* [#23130](https://github.com/netbox-community/netbox/issues/23130) - Ensure that the cascade triggers for hierarchical models can be restored from a `pg_dump` (see the warning above)
* [#23154](https://github.com/netbox-community/netbox/issues/23154) - Correct the optional/required mismatch on the L2VPN `type` and rack type `form_factor` fields
* [#23166](https://github.com/netbox-community/netbox/issues/23166) - Apply a zero-valued minimum or maximum bound from a module type profile attribute to its form field
* [#23167](https://github.com/netbox-community/netbox/issues/23167) - Sanitize the JSON schema property descriptions used as form help text for module type profile attributes
---
## v4.7.0 (2026-09-02)
!!! warning "PostgreSQL 15 or Later Required"

View File

@ -176,6 +176,7 @@ nav:
- Error Reporting: 'administration/error-reporting.md'
- Management Commands: 'administration/management-commands.md'
- Replicating NetBox: 'administration/replicating-netbox.md'
- Repairing Hierarchical Paths: 'administration/repairing-hierarchical-paths.md'
- NetBox Shell: 'administration/netbox-shell.md'
- Data Model:
- Circuits:

View File

@ -9,14 +9,16 @@ from django.contrib.auth.forms import AuthenticationForm, PasswordChangeForm
from django.contrib.auth.mixins import LoginRequiredMixin
from django.contrib.auth.models import update_last_login
from django.contrib.auth.signals import user_logged_in
from django.http import HttpResponseRedirect
from django.http import HttpResponseRedirect, JsonResponse
from django.shortcuts import get_object_or_404, redirect, render, resolve_url
from django.urls import reverse, reverse_lazy
from django.utils.decorators import method_decorator
from django.utils.translation import gettext_lazy as _
from django.views.decorators.cache import never_cache
from django.views.decorators.debug import sensitive_post_parameters
from django.views.generic import View
from social_core.backends.utils import load_backends
from social_django.views import auth as social_auth_begin
from account.models import UserToken
from core.models import ObjectChange
@ -78,7 +80,7 @@ class LoginView(View):
request_data = request.POST if request.method == 'POST' else request.GET
for name in load_backends(settings.AUTHENTICATION_BACKENDS).keys():
url = reverse('social:begin', args=[name])
url = reverse('social_auth_begin', args=[name])
params = {}
if next := request_data.get('next'):
params['next'] = next
@ -188,6 +190,50 @@ class LogoutView(View):
return response
class SocialAuthBeginView(View):
"""
Initiate authentication against a social auth (SSO) backend.
This wraps python-social-auth's "begin" view, which responds with an HTTP redirect to the
identity provider. Chromium-based browsers evaluate the CSP `form-action` directive against
every hop in a form submission's redirect chain, so a deployment which serves NetBox with
`form-action 'self'` (a common reverse proxy default) blocks that redirect and the SSO button
appears to do nothing. A client which asks for JSON is given the identity provider's URL in the
response body instead, and navigates to it itself: `form-action` does not govern a navigation
initiated by a script. Any other client (e.g. a browser with JavaScript disabled) receives the
unmodified response from python-social-auth.
A backend which does not redirect (`uses_redirect()` is False, as for OpenID 2.0) renders its
own HTML instead, which is returned in the response body for the client to render in place so
that it need not repeat the request. This is not a way around `form-action`: that document
carries a form which submits itself to the identity provider, and such a submission is governed
by the policy wherever the document is rendered. Deployments using one of these backends still
require a `form-action` which admits the identity provider.
The underlying view is reused as-is so that CSRF protection, the callback URL, and the session
state recorded for the identity provider all remain identical to a direct form submission.
"""
@method_decorator(never_cache)
def dispatch(self, *args, **kwargs):
return super().dispatch(*args, **kwargs)
def post(self, request, backend):
response = social_auth_begin(request, backend)
if 'application/json' in request.headers.get('Accept', ''):
if url := response.headers.get('Location'):
return JsonResponse({'url': url})
if response.status_code == 200:
# Some backends render an HTML form (which submits itself to the identity provider)
# rather than redirecting. Hand that document to the client to render, so that it
# need not repeat the request and initiate the login a second time.
return JsonResponse({'html': response.content.decode(response.charset)})
# Anything else (including the response to a client which has not asked for JSON) is passed
# through unchanged.
return response
#
# User profiles
#

View File

@ -26,7 +26,7 @@ class DataSourceSerializer(PrimaryModelSerializer):
model = DataSource
fields = [
'id', 'url', 'display_url', 'display', 'name', 'type', 'source_url', 'enabled', 'status', 'description',
'sync_interval', 'parameters', 'ignore_rules', 'owner', 'comments', 'custom_fields', 'created',
'sync_interval', 'parameters', 'ignore_rules', 'owner', 'comments', 'tags', 'custom_fields', 'created',
'last_updated', 'last_synced', 'file_count',
]
brief_fields = ('id', 'url', 'display', 'name', 'description')

View File

@ -1,7 +1,7 @@
{
"datafile:api_list_objects": 10,
"datafile:list_objects_with_permission": 17,
"datasource:api_list_objects": 11,
"datasource:api_list_objects": 12,
"datasource:list_objects_with_permission": 17,
"job:api_list_objects": 12,
"job:list_objects_with_permission": 19

View File

@ -12,7 +12,7 @@ from rq.registry import FailedJobRegistry, StartedJobRegistry
from users.constants import TOKEN_PREFIX
from users.models import Token
from utilities.testing import APITestCase, APIViewTestCases, GraphQLQueryTest, TestCase
from utilities.testing import APITestCase, APIViewTestCases, GraphQLQueryTest, TestCase, create_tags
from utilities.testing.mixins import RQQueueTestMixin
from utilities.testing.utils import disable_logging
@ -100,6 +100,57 @@ class DataSourceTestCase(APIViewTestCases.APIViewTestCase):
},
]
def test_tags_in_representation(self):
"""Assigned tags are rendered in the detail representation."""
data_source = DataSource.objects.first()
data_source.tags.set(create_tags('Alpha'))
self.add_permissions('core.view_datasource')
response = self.client.get(self._get_detail_url(data_source), **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
self.assertIn('tags', response.data)
self.assertEqual([tag['slug'] for tag in response.data['tags']], ['alpha'])
def test_create_with_tags(self):
"""Tags supplied on creation are assigned to the new data source."""
create_tags('Alpha')
self.add_permissions('core.add_datasource', 'extras.view_tag')
data = {
'name': 'Data Source 7',
'type': 'git',
'source_url': 'https://example.com/git/source7',
'tags': [{'slug': 'alpha'}],
}
response = self.client.post(self._get_list_url(), data, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_201_CREATED)
data_source = DataSource.objects.get(pk=response.data['id'])
self.assertEqual(list(data_source.tags.values_list('slug', flat=True)), ['alpha'])
def test_update_tags(self):
"""Tags supplied on update replace the existing assignment."""
data_source = DataSource.objects.first()
tags = create_tags('Alpha', 'Bravo')
data_source.tags.set([tags[0]])
self.add_permissions('core.change_datasource', 'extras.view_tag')
data = {'tags': [{'slug': 'bravo'}]}
response = self.client.patch(self._get_detail_url(data_source), data, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
self.assertEqual(list(data_source.tags.values_list('slug', flat=True)), ['bravo'])
def test_clear_tags(self):
"""An empty tag list clears the existing assignment."""
data_source = DataSource.objects.first()
data_source.tags.set(create_tags('Alpha'))
self.add_permissions('core.change_datasource')
data = {'tags': []}
response = self.client.patch(self._get_detail_url(data_source), data, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
self.assertEqual(list(data_source.tags.values_list('slug', flat=True)), [])
def assert_only_source_1(self, data):
"""The JSON lookup returns exactly the source carrying the matching value."""
ids = sorted(result['id'] for result in data['data_source_list'])

View File

@ -5,7 +5,7 @@ Refs: #20638
"""
import json
from django.test import SimpleTestCase, TestCase
from django.test import SimpleTestCase, TestCase, override_settings
from core.api.schema import FixSerializedPKRelatedField, NetBoxAutoSchema
from dcim.api.serializers import SiteSerializer
@ -15,6 +15,11 @@ from netbox.api.fields import SerializedPKRelatedField
from netbox.api.serializers import BulkOperationErrorSerializer
@override_settings(CACHES={
'default': {
'BACKEND': 'django.core.cache.backends.dummy.DummyCache'
}
})
class OpenAPISchemaTestCase(TestCase):
"""Tests for OpenAPI schema generation."""
@ -331,6 +336,22 @@ class OpenAPISchemaTestCase(TestCase):
with self.subTest(component=component, field=field):
self.assertEqual(components[component]['properties'][field]['items']['type'], 'integer')
def test_script_run_operation_exists(self):
"""
Encodes presence of extras_scripts_run operation in schema as expected.
Refs: #22569
"""
paths = self.schema['paths']
resource_path = paths['/api/extras/scripts/{id}/']
self.assertIn('post', resource_path)
run_operation = resource_path['post']
self.assertEqual(run_operation['operationId'], 'extras_scripts_run')
self.assertEqual(len(run_operation['responses']), 1)
self.assertIn('200', run_operation['responses'])
class WritableFieldRebuildTestCase(TestCase):
"""

View File

@ -86,6 +86,9 @@ class RackBaseSerializer(PrimaryModelSerializer):
class RackTypeSerializer(RackBaseSerializer):
# Unlike Rack.form_factor (optional & nullable), RackType.form_factor is required
# (blank=False, no default), so override RackBaseSerializer's optional declaration.
form_factor = ChoiceField(choices=RackFormFactorChoices, required=True)
manufacturer = ManufacturerSerializer(nested=True)
rack_count = serializers.IntegerField(read_only=True)

View File

@ -1137,6 +1137,11 @@ class InterfaceTypeChoices(ChoiceSet):
TYPE_INFINIBAND_NDR = 'infiniband-ndr'
TYPE_INFINIBAND_XDR = 'infiniband-xdr'
# InfiniBand 2X
TYPE_INFINIBAND_HDR_2X = 'infiniband-hdr-2x'
TYPE_INFINIBAND_NDR_2X = 'infiniband-ndr-2x'
TYPE_INFINIBAND_XDR_2X = 'infiniband-xdr-2x'
# InfiniBand 4X
TYPE_INFINIBAND_SDR_4X = 'infiniband-sdr-4x'
TYPE_INFINIBAND_DDR_4X = 'infiniband-ddr-4x'
@ -1480,6 +1485,14 @@ class InterfaceTypeChoices(ChoiceSet):
Choice(TYPE_INFINIBAND_XDR, 'XDR (200 Gbps)'),
)
),
(
'InfiniBand 2X',
(
Choice(TYPE_INFINIBAND_HDR_2X, 'HDR 2X (100 Gbps)'),
Choice(TYPE_INFINIBAND_NDR_2X, 'NDR 2X (200 Gbps)'),
Choice(TYPE_INFINIBAND_XDR_2X, 'XDR 2X (400 Gbps)'),
)
),
(
'InfiniBand 4X',
(

View File

@ -225,7 +225,6 @@ class RackTypeImportForm(PrimaryModelImportForm):
form_factor = CSVChoiceField(
label=_('Type'),
choices=RackFormFactorChoices,
required=False,
help_text=_('Form factor')
)
starting_unit = forms.IntegerField(

View File

@ -20,7 +20,8 @@ def load_initial_data(apps, schema_editor):
'hard_disk',
'memory',
'power_supply',
'expansion_card'
'expansion_card',
'transceiver'
)
profile_objects = []

View File

@ -0,0 +1,53 @@
"""Reinstall the ltree cascade triggers with a restore-safe WHEN clause.
The cascade triggers installed by 0242_ltree_paths compared two ltree values with
`IS DISTINCT FROM`, which resolves the `ltree = ltree` operator through search_path at
CREATE TRIGGER time. pg_dump emits `set_config('search_path', '', false)`, so restoring a
v4.7.0 dump could not create these triggers and because psql does not stop on error by
default, the restore reported success with the triggers silently missing. See #23130.
Reinstalling covers both affected databases: one restored from such a dump (the triggers
are absent) and one upgraded in place (they exist with the old definition, which would
fail its own next restore). InstallLtreeTriggers drops before creating, so this applies
cleanly in either state.
This reinstalls triggers only, so it takes ACCESS EXCLUSIVE on each table for the DDL
itself and performs no table scan. Note that this is a stronger lock than the ROW
EXCLUSIVE held by 0242's backfill, and it blocks readers as well as writers: it is brief,
but on a busy table it queues behind any long-running query and holds everything behind
it for that query's duration.
It does not repair path/sort_path values which went stale while the triggers were
missing; see the v4.7.1 release notes for detection and repair.
Reversing this migration is a no-op. Reversing 0242_ltree_paths in turn drops these
triggers rather than recreating them, which is that migration's business; what matters
here is that undoing a corrective reinstall has no target state of its own, since the
definition it replaced is the broken one.
"""
from django.db import migrations
from utilities.ltree import ReinstallLtreeTriggers
# The tables carrying a sort_path column, maintained from `name`.
SORT_TABLES = (
'dcim_region',
'dcim_sitegroup',
'dcim_location',
'dcim_devicerole',
'dcim_platform',
'dcim_modulebay',
)
class Migration(migrations.Migration):
dependencies = [
('dcim', '0250_cooling_infrastructure'),
]
operations = [
*[ReinstallLtreeTriggers(t, name_column='name') for t in SORT_TABLES],
ReinstallLtreeTriggers('dcim_inventoryitem'),
ReinstallLtreeTriggers('dcim_inventoryitemtemplate'),
]

View File

@ -0,0 +1,37 @@
{
"name": "Transceiver",
"schema": {
"properties": {
"form_factor": {
"type": "string",
"title": "Form factor",
"description": "Physical form factor e.g. SFP+, QSFP28, OSFP"
},
"media": {
"type": "string",
"title": "Media",
"description": "Physical media e.g. single-mode fiber, multimode fiber, DAC"
},
"phy": {
"type": "string",
"title": "PHY",
"description": "Physical layer standard e.g. 10GBASE-LR, 400ZR"
},
"data_rate": {
"type": "string",
"title": "Data rate",
"description": "Nominal line rate e.g. 10G, 400G"
},
"reach": {
"type": "number",
"title": "Maximum reach (m)",
"description": "Maximum supported distance in meters, or fixed assembly length for DAC and AOC cables"
},
"connector_type": {
"type": "string",
"title": "Connector type",
"description": "Connector type e.g. LC/UPC, MPO-12, 8P8C"
}
}
}
}

View File

@ -9,7 +9,7 @@ from django.contrib.postgres.fields import ArrayField
from django.contrib.postgres.indexes import GinIndex
from django.core.exceptions import ValidationError
from django.core.validators import MaxValueValidator, MinValueValidator
from django.db import models
from django.db import models, router
from django.dispatch import Signal
from django.urls import reverse
from django.utils.translation import gettext_lazy as _
@ -328,15 +328,32 @@ class Cable(PrimaryModel):
}
update_fields = normalize_update_fields(save_kwargs)
# Store the given length (if any) in meters for use in database ordering
if self.length is not None and self.length_unit:
self._abs_length = to_meters(self.length, self.length_unit)
else:
self._abs_length = None
length_written = update_fields is None or 'length' in update_fields
length_unit_written = update_fields is None or 'length_unit' in update_fields
# Clear length_unit if no length is defined
if self.length is None:
self.length_unit = None
if length_written or length_unit_written:
if length_written and length_unit_written:
stored = {}
else:
# Read from the database this save will write, so a router cannot split the two
db = using or router.db_for_write(Cable, instance=self)
stored = Cable.objects.using(db).filter(pk=self.pk).values('length', 'length_unit').first() or {}
length = self.length if length_written else stored.get('length')
length_unit = self.length_unit if length_unit_written else stored.get('length_unit')
# Clear length_unit if no length is defined
if length is None and length_unit_written:
self.length_unit = None
# Store the given length (if any) in meters for use in database ordering
if length is not None and length_unit:
self._abs_length = to_meters(length, length_unit)
else:
self._abs_length = None
# _abs_length is a denormalized cache of length and length_unit, so persist them together
if update_fields is not None:
save_kwargs['update_fields'] = update_fields | {'_abs_length'}
# A field counts as changed only when this save actually writes it
status_written = update_fields is None or 'status' in update_fields

View File

@ -1206,6 +1206,28 @@ class RackTypeTestCase(APIViewTestCases.APIViewTestCase):
},
]
def test_form_factor_required(self):
"""
form_factor must be reported as required by OPTIONS, and a POST omitting it
must be rejected with a normal "required" validation error rather than a
model-level "cannot be blank" error.
"""
self.add_permissions('dcim.add_racktype')
response = self.client.options(self._get_list_url(), **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
self.assertTrue(response.data['actions']['POST']['form_factor']['required'])
manufacturer = Manufacturer.objects.first()
data = {
'manufacturer': manufacturer.pk,
'model': 'Rack Type Missing Form Factor',
'slug': 'rack-type-missing-form-factor',
}
response = self.client.post(self._get_list_url(), data, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_400_BAD_REQUEST)
self.assertEqual(response.data['form_factor'][0].code, 'required')
class RackTestCase(APIViewTestCases.APIViewTestCase):
model = Rack

View File

@ -2,6 +2,7 @@ from decimal import Decimal
from django.conf import settings
from django.contrib.contenttypes.models import ContentType
from django.db import DEFAULT_DB_ALIAS, connection
from django.test import TestCase
from circuits.models import Circuit, CircuitTermination, CircuitType, Provider
@ -18,6 +19,7 @@ from netbox.choices import (
)
from tenancy.models import Tenant, TenantGroup
from users.models import User
from utilities.query_functions import CollateAsChar
from utilities.testing import ChangeLoggedFilterSetTestMixin, create_test_device, create_test_virtualmachine
from virtualization.models import Cluster, ClusterGroup, ClusterType, VirtualMachine, VMInterface
from wireless.choices import WirelessChannelChoices, WirelessRoleChoices
@ -3375,6 +3377,117 @@ class DeviceTestCase(TestCase, ChangeLoggedFilterSetTestMixin):
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)
class DeviceCollatedFilterTestCase(TestCase):
"""
Case-insensitive filtering against a column which carries the natural_sort collation.
UPPER() folds according to the collation of its argument, so a collated column and an
uncollated parameter disagree: UPPER('ß') is 'SS' under natural_sort but 'ß' under the
database default. Searching for 'ß' therefore matched nothing at all (#23012).
"""
queryset = Device.objects.all()
filterset = DeviceFilterSet
@classmethod
def setUpTestData(cls):
manufacturer = Manufacturer.objects.create(name='Manufacturer 1', slug='manufacturer-1')
device_type = DeviceType.objects.create(manufacturer=manufacturer, model='Model 1', slug='model-1')
role = DeviceRole.objects.create(name='Device Role 1', slug='device-role-1')
site = Site.objects.create(name='Site 1', slug='site-1')
Device.objects.bulk_create((
# The reported case: an eszett within the name.
Device(name='Straße Switch 1', device_type=device_type, role=role, site=site),
# The same word spelled 'ss', which must match the name above and vice versa.
Device(name='Strasse Switch 2', device_type=device_type, role=role, site=site),
# An eszett alongside an umlaut, to pin that umlauts are not folded.
Device(name='Grüße Router 3', device_type=device_type, role=role, site=site),
# Plain ASCII control.
Device(name='Device 4', device_type=device_type, role=role, site=site),
))
def assertFilterReturns(self, params, expected_names):
names = self.filterset(params, self.queryset).qs.values_list('name', flat=True)
self.assertEqual(sorted(names), sorted(expected_names))
def test_icontains_eszett(self):
self.assertFilterReturns(
{'name__ic': ['straße']}, ['Straße Switch 1', 'Strasse Switch 2']
)
def test_icontains_ss_matches_eszett(self):
self.assertFilterReturns(
{'name__ic': ['strasse']}, ['Straße Switch 1', 'Strasse Switch 2']
)
def test_icontains_bare_eszett(self):
self.assertFilterReturns(
{'name__ic': ['ß']}, ['Straße Switch 1', 'Strasse Switch 2', 'Grüße Router 3']
)
def test_iexact_eszett(self):
self.assertFilterReturns({'name__ie': ['strasse switch 1']}, ['Straße Switch 1'])
def test_istartswith_eszett(self):
self.assertFilterReturns(
{'name__isw': ['Strasse']}, ['Straße Switch 1', 'Strasse Switch 2']
)
def test_iendswith_eszett(self):
self.assertFilterReturns({'name__iew': ['ße Router 3']}, ['Grüße Router 3'])
def test_ascii_matching_is_unchanged(self):
self.assertFilterReturns({'name__ic': ['device']}, ['Device 4'])
self.assertFilterReturns({'name__ic': ['SWITCH']}, ['Straße Switch 1', 'Strasse Switch 2'])
def test_umlauts_are_not_folded(self):
# Only the eszett is folded; 'ü' must not match 'u'. Otherwise this would be
# blanket accent stripping, which is a much broader change than intended.
self.assertFilterReturns({'name__ic': ['grusse']}, [])
def test_q_search_finds_eszett(self):
# The surface reported in #23012: the object list's quick search, which is also
# what the REST API uses.
self.assertFilterReturns(
{'q': 'straße'}, ['Straße Switch 1', 'Strasse Switch 2']
)
def test_uncollated_field_is_unaffected(self):
# serial carries no collation, so it keeps plain case-insensitive matching.
Device.objects.filter(name='Device 4').update(serial='Straße')
self.assertFilterReturns({'serial__ic': ['straße']}, ['Device 4'])
self.assertFilterReturns({'serial__ic': ['strasse']}, [])
def test_explicit_lhs_collation_does_not_error(self):
# Applying an explicit collation to the left-hand side must not collide with the
# collation applied to the parameter: PostgreSQL rejects two explicit collations
# in one comparison, which would turn a working query into a 500.
qs = Device.objects.annotate(collated=CollateAsChar('name')).filter(collated__icontains='switch')
self.assertEqual(qs.count(), 2)
def test_collation_is_applied_to_parameter(self):
# The tests above assert on results, which stay correct for ASCII values even if
# the collation is never applied. This asserts on the lookup's own output instead,
# so that the mechanism failing open is caught rather than passing silently.
for lookup in ('icontains', 'iexact', 'istartswith', 'iendswith'):
with self.subTest(lookup=lookup):
self.assertEqual(
self._compiled_rhs(Device, 'name', lookup),
'%s COLLATE "natural_sort"'
)
self.assertEqual(self._compiled_rhs(Device, 'serial', lookup), '%s')
@staticmethod
def _compiled_rhs(model, field_name, lookup):
"""
Compile a single filter's right-hand side and return its SQL.
"""
query = model.objects.filter(**{f'{field_name}__{lookup}': 'x'}).query
compiler = query.get_compiler(using=DEFAULT_DB_ALIAS)
rhs, _ = query.where.children[0].process_rhs(compiler, connection)
return rhs
class ModuleTestCase(TestCase, ChangeLoggedFilterSetTestMixin):
queryset = Module.objects.all()
filterset = ModuleFilterSet

View File

@ -1,6 +1,7 @@
from unittest.mock import patch
from django import forms
from django.template.loader import render_to_string
from django.test import TestCase
from dcim.choices import (
@ -228,6 +229,73 @@ class ModuleTypeFormTestCase(TestCase):
module_type = form.save()
self.assertEqual(module_type.attribute_data, {'media': ['copper', 'qsfp28']})
def test_zero_bound_attribute_is_enforced_by_the_form(self):
profile = ModuleTypeProfile.objects.create(
name='Module Type Profile 2',
schema={
'properties': {
'offset': {
'title': 'Offset',
'type': 'number',
'minimum': 0,
'maximum': 0,
},
},
},
)
form = ModuleTypeForm(data={
'manufacturer': self.manufacturer.pk,
'model': 'Module Type 2',
'profile': profile.pk,
'attr_offset': -5,
})
self.assertEqual(form.fields['attr_offset'].min_value, 0)
self.assertEqual(form.fields['attr_offset'].max_value, 0)
with patch('utilities.forms.fields.dynamic.get_action_url', return_value='/'):
self.assertFalse(form.is_valid())
self.assertIn('attr_offset', form.errors)
class ModuleTypeProfileDescriptionRenderingTestCase(TestCase):
"""
A profile schema property's description is rendered as the attribute field's help text via the
`safe` filter, so markup outside HTML_ALLOWED_TAGS must not reach the DOM as a live element.
Verified end to end because the sanitization and the `safe` filter that makes it necessary sit
in different layers.
"""
@classmethod
def setUpTestData(cls):
cls.manufacturer = Manufacturer.objects.create(name='Manufacturer 1', slug='manufacturer-1')
cls.profile = ModuleTypeProfile.objects.create(
name='Disk',
schema={
'properties': {
'capacity': {
'type': 'integer',
'title': 'Capacity (GB)',
'description': 'Gross disk size <iframe src="https://example.com"></iframe>',
},
},
},
)
def test_help_text_is_rendered_without_disallowed_markup(self):
form = ModuleTypeForm(data={
'manufacturer': self.manufacturer.pk,
'model': 'Module Type 1',
'profile': self.profile.pk,
'attr_capacity': 500,
})
rendered = render_to_string('form_helpers/render_field.html', {'field': form['attr_capacity']})
self.assertInHTML(
'<span class="form-text" id="id_attr_capacity_helptext">'
'<div class="rendered-markdown"><p>Gross disk size</p></div></span>',
rendered,
)
class ModuleBayTemplateImportFormTestCase(TestCase):

View File

@ -1,9 +1,11 @@
from decimal import Decimal
from django.core.exceptions import ValidationError
from django.db import connection
from django.db.models import ProtectedError
from django.db.models.signals import post_save
from django.test import TestCase, tag
from django.test.utils import CaptureQueriesContext
from circuits.models import *
from core.models import ObjectType
@ -2700,6 +2702,213 @@ class CableTestCase(TestCase):
self.assertEqual(cable._abs_length, Decimal('1609343983.9066'))
def test_partial_save_persists_normalized_length(self):
"""
A save naming only length must persist the normalized length alongside it.
"""
cable = Cable.objects.first()
cable.length = Decimal('1')
cable.length_unit = CableLengthUnitChoices.UNIT_METER
cable.save()
cable.length = Decimal('2')
cable.save(update_fields=['length'])
cable.refresh_from_db()
self.assertEqual(cable.length, Decimal('2.00'))
self.assertEqual(cable.length_unit, CableLengthUnitChoices.UNIT_METER)
self.assertEqual(cable._abs_length, Decimal('2.0000'))
def test_partial_save_persists_normalized_length_for_a_unit_change(self):
"""
A save naming only length_unit must renormalize against the stored length.
"""
cable = Cable.objects.first()
cable.length = Decimal('10')
cable.length_unit = CableLengthUnitChoices.UNIT_METER
cable.save()
cable.length_unit = CableLengthUnitChoices.UNIT_FOOT
cable.save(update_fields=['length_unit'])
cable.refresh_from_db()
self.assertEqual(cable.length, Decimal('10.00'))
self.assertEqual(cable.length_unit, CableLengthUnitChoices.UNIT_FOOT)
self.assertEqual(cable._abs_length, Decimal('3.0480'))
def test_partial_save_persists_normalized_length_for_both_source_fields(self):
"""
A save naming both source fields must normalize from the values being written.
"""
cable = Cable.objects.first()
cable.length = Decimal('2')
cable.length_unit = CableLengthUnitChoices.UNIT_KILOMETER
cable.save(update_fields=['length', 'length_unit'])
cable.refresh_from_db()
self.assertEqual(cable._abs_length, Decimal('2000.0000'))
def test_partial_save_normalizes_against_an_unwritten_length_unit(self):
"""
A save naming only length must normalize against the stored unit, not an unwritten one.
"""
cable = Cable.objects.first()
cable.length = Decimal('1')
cable.length_unit = CableLengthUnitChoices.UNIT_METER
cable.save()
cable.length = Decimal('2')
cable.length_unit = CableLengthUnitChoices.UNIT_KILOMETER
cable.save(update_fields=['length'])
cable.refresh_from_db()
self.assertEqual(cable.length_unit, CableLengthUnitChoices.UNIT_METER)
self.assertEqual(cable._abs_length, Decimal('2.0000'))
def test_partial_save_normalizes_against_an_unwritten_length(self):
"""
A save naming only length_unit must normalize against the stored length, not an unwritten one.
"""
cable = Cable.objects.first()
cable.length = Decimal('1')
cable.length_unit = CableLengthUnitChoices.UNIT_METER
cable.save()
cable.length = Decimal('2')
cable.length_unit = CableLengthUnitChoices.UNIT_CENTIMETER
cable.save(update_fields=['length_unit'])
cable.refresh_from_db()
self.assertEqual(cable.length, Decimal('1.00'))
self.assertEqual(cable.length_unit, CableLengthUnitChoices.UNIT_CENTIMETER)
self.assertEqual(cable._abs_length, Decimal('0.0100'))
def test_partial_save_normalizes_against_an_unwritten_cleared_length(self):
"""
A save naming only length_unit must keep the unit when the excluded length is cleared in memory.
"""
cable = Cable.objects.first()
cable.length = Decimal('1')
cable.length_unit = CableLengthUnitChoices.UNIT_METER
cable.save()
cable.length = None
cable.length_unit = CableLengthUnitChoices.UNIT_CENTIMETER
cable.save(update_fields=['length_unit'])
cable.refresh_from_db()
self.assertEqual(cable.length, Decimal('1.00'))
self.assertEqual(cable.length_unit, CableLengthUnitChoices.UNIT_CENTIMETER)
self.assertEqual(cable._abs_length, Decimal('0.0100'))
def test_partial_save_clearing_length_keeps_the_stored_unit(self):
"""
A save naming only length must clear the normalized length without writing length_unit.
"""
cable = Cable.objects.first()
cable.length = Decimal('1')
cable.length_unit = CableLengthUnitChoices.UNIT_METER
cable.save()
cable.length = None
cable.save(update_fields=['length'])
# The unit was not written, so the instance must still agree with the row
self.assertEqual(cable.length_unit, CableLengthUnitChoices.UNIT_METER)
cable.refresh_from_db()
self.assertIsNone(cable.length)
self.assertEqual(cable.length_unit, CableLengthUnitChoices.UNIT_METER)
self.assertIsNone(cable._abs_length)
def test_partial_save_leaves_an_unwritten_length_alone(self):
"""
A save naming an unrelated field must not persist an in-memory length change.
"""
cable = Cable.objects.first()
cable.length = Decimal('1')
cable.length_unit = CableLengthUnitChoices.UNIT_METER
cable.save()
cable.length = Decimal('99')
cable.label = 'Renamed'
cable.save(update_fields=['label'])
cable.refresh_from_db()
self.assertEqual(cable.label, 'Renamed')
self.assertEqual(cable.length, Decimal('1.00'))
self.assertEqual(cable._abs_length, Decimal('1.0000'))
def test_partial_save_normalizes_against_an_out_of_band_length(self):
"""
A save naming only length_unit must read the stored length, not one cached on the instance.
"""
cable = Cable.objects.first()
cable.length = Decimal('1')
cable.length_unit = CableLengthUnitChoices.UNIT_METER
cable.save()
Cable.objects.filter(pk=cable.pk).update(length=Decimal('7'))
cable.length_unit = CableLengthUnitChoices.UNIT_FOOT
cable.save(update_fields=['length_unit'])
cable.refresh_from_db()
self.assertEqual(cable.length, Decimal('7.00'))
self.assertEqual(cable._abs_length, Decimal('2.1336'))
def test_partial_save_clears_a_unit_written_without_a_stored_length(self):
"""
A save naming only length_unit must drop the unit when the row holds no length.
"""
cable = Cable.objects.first()
cable.length_unit = CableLengthUnitChoices.UNIT_METER
cable.save(update_fields=['length_unit'])
cable.refresh_from_db()
self.assertIsNone(cable.length)
self.assertIsNone(cable.length_unit)
self.assertIsNone(cable._abs_length)
def test_full_save_clears_the_unit_when_the_length_is_removed(self):
"""
A full save with no length must clear the stored unit.
"""
cable = Cable.objects.first()
cable.length = Decimal('1')
cable.length_unit = CableLengthUnitChoices.UNIT_METER
cable.save()
cable.length = None
cable.save()
cable.refresh_from_db()
self.assertIsNone(cable.length)
self.assertIsNone(cable.length_unit)
self.assertIsNone(cable._abs_length)
def test_partial_save_reads_the_stored_pair_only_for_an_excluded_field(self):
"""
A save writing both source fields must normalize without reading the row back.
"""
cable = Cable.objects.first()
cable.length = Decimal('1')
cable.length_unit = CableLengthUnitChoices.UNIT_METER
cable.save()
with CaptureQueriesContext(connection) as both_written:
cable.save(update_fields=['length', 'length_unit'])
with CaptureQueriesContext(connection) as one_written:
cable.save(update_fields=['length'])
def cable_reads(queries):
return [q for q in queries if q['sql'].startswith('SELECT') and 'FROM "dcim_cable"' in q['sql']]
self.assertEqual(cable_reads(both_written), [])
self.assertEqual(len(cable_reads(one_written)), 1)
class CableTerminationTestCase(TestCase):

View File

@ -503,10 +503,10 @@ class RackTypeTestCase(ViewTestCases.PrimaryObjectViewTestCase):
}
cls.csv_data = (
"manufacturer,model,slug,width,u_height,weight,max_weight,weight_unit",
"Manufacturer 1,RackType 4,rack-type-4,19,42,100,2000,kg",
"Manufacturer 1,RackType 5,rack-type-5,19,42,100,2000,kg",
"Manufacturer 1,RackType 6,rack-type-6,19,42,100,2000,kg",
"manufacturer,model,slug,form_factor,width,u_height,weight,max_weight,weight_unit",
f"Manufacturer 1,RackType 4,rack-type-4,{RackFormFactorChoices.TYPE_CABINET},19,42,100,2000,kg",
f"Manufacturer 1,RackType 5,rack-type-5,{RackFormFactorChoices.TYPE_CABINET},19,42,100,2000,kg",
f"Manufacturer 1,RackType 6,rack-type-6,{RackFormFactorChoices.TYPE_CABINET},19,42,100,2000,kg",
)
cls.csv_update_data = (
@ -531,6 +531,30 @@ class RackTypeTestCase(ViewTestCases.PrimaryObjectViewTestCase):
'comments': 'New comments',
}
def test_bulk_import_objects_without_form_factor(self):
"""
A CSV import row omitting form_factor must be rejected, not silently saved
with form_factor=''.
"""
obj_perm = ObjectPermission(name='Test permission', actions=['add'])
obj_perm.save()
obj_perm.users.add(self.user)
obj_perm.object_types.add(ObjectType.objects.get_for_model(self.model))
initial_count = self._get_queryset().count()
csv_data = (
"manufacturer,model,slug,width,u_height,weight,max_weight,weight_unit",
"Manufacturer 1,RackType Missing Form Factor,rack-type-missing-form-factor,19,42,100,2000,kg",
)
data = {
'data': '\n'.join(csv_data),
'format': ImportFormatChoices.CSV,
'csv_delimiter': CSVDelimiterChoices.AUTO,
}
response = self.client.post(self._get_url('bulk_import'), data)
self.assertHttpStatus(response, 200)
self.assertEqual(self._get_queryset().count(), initial_count)
class RackTestCase(ViewTestCases.PrimaryObjectViewTestCase):
model = Rack

View File

@ -188,6 +188,16 @@ class ScriptInputSerializer(serializers.Serializer):
if script and script.python_class:
self.fields['notifications'].default = script.python_class.notifications_default
def validate_data(self, value):
"""
Validates that the script input is an object mapping variable names to values.
"""
if not isinstance(value, dict):
raise serializers.ValidationError(
_('Invalid data payload; expected an object mapping variable names to values.')
)
return value
def validate_schedule_at(self, value):
"""
Validates the specified schedule time for a script execution.

View File

@ -1,9 +1,9 @@
from django.core.exceptions import NON_FIELD_ERRORS
from django.core.exceptions import ValidationError as DjangoValidationError
from django.http import Http404
from django.shortcuts import get_object_or_404
from django.utils.translation import gettext_lazy as _
from drf_spectacular.utils import OpenApiResponse, OpenApiTypes, extend_schema
from rest_framework import status
from rest_framework.decorators import action
from rest_framework.exceptions import PermissionDenied, ValidationError
from rest_framework.generics import RetrieveUpdateDestroyAPIView
@ -16,6 +16,7 @@ from core.choices import ManagedFileRootPathChoices
from extras import filtersets
from extras.jobs import ScriptJob
from extras.models import *
from extras.scripts import EXEC_PARAM_FIELDS, prepare_script_form
from netbox.api.authentication import IsAuthenticatedOrLoginNotRequired, TokenWritePermission
from netbox.api.features import SyncedDataMixin
from netbox.api.metadata import ContentTypeMetadata
@ -404,29 +405,56 @@ class ScriptViewSet(ListModelMixin, RetrieveModelMixin, BaseViewSet):
if not any_workers_for_queue('default'):
raise RQWorkerNotRunningException()
if input_serializer.is_valid():
try:
ScriptJob.enqueue(
instance=script,
user=request.user,
data=input_serializer.data['data'],
request=copy_safe_request(request),
commit=input_serializer.data['commit'],
job_timeout=script.python_class.job_timeout,
schedule_at=input_serializer.validated_data.get('schedule_at'),
interval=input_serializer.validated_data.get('interval'),
notifications=input_serializer.validated_data.get('notifications'),
)
except DjangoValidationError as e:
# The script's execution configuration is invalid (see #22872). Surface it as a 400 rather than
# allowing the exception to bubble up as an HTTP 500. These are script-level config errors, not
# request-field errors, so report them under the non-field "detail" key.
raise ValidationError({'detail': e.messages}) from e
serializer = serializers.ScriptDetailSerializer(script, context={'request': request})
input_serializer.is_valid(raise_exception=True)
return Response(serializer.data)
validated = input_serializer.validated_data
return Response(input_serializer.errors, status=status.HTTP_400_BAD_REQUEST)
payload = validated['data']
# Guaranteed non-None by the is_executable check above
script_class = script.python_class
script_instance = script_class()
form = prepare_script_form(script_instance, payload, files=request.FILES)
if not form.is_valid():
# Exec params are validated separately via ScriptInputSerializer. Excluded by name
# rather than by '_' prefix, which would also strip Django's NON_FIELD_ERRORS
# key ('__all__').
errors = {k: v for k, v in form.errors.items() if k not in EXEC_PARAM_FIELDS}
if not errors:
# Every error was on an exec-param field, which a client can bind by naming one
# in 'data' (e.g. {"_interval": "abc"}). NON_FIELD_ERRORS is never among them --
# the filter above retains '__all__' -- so there is nothing to re-surface here;
# report a generic message rather than an empty body.
errors = {NON_FIELD_ERRORS: [_('Invalid script input.')]}
# Nest under 'data' so script-variable errors can't collide with the
# serializer's own top-level fields (commit, schedule_at, interval, ...).
raise ValidationError({'data': errors})
data = form.cleaned_data.copy()
for k in EXEC_PARAM_FIELDS:
data.pop(k, None)
try:
ScriptJob.enqueue(
instance=script,
user=request.user,
data=data,
request=copy_safe_request(request),
commit=validated.get('commit'),
job_timeout=script_class.job_timeout,
schedule_at=validated.get('schedule_at'),
interval=validated.get('interval'),
notifications=validated.get('notifications'),
)
except DjangoValidationError as e:
# The script's execution configuration is invalid (see #22872). Surface it as a 400 rather than
# allowing the exception to bubble up as an HTTP 500. These are script-level config errors, not
# request-field errors, so report them under the non-field "detail" key.
raise ValidationError({'detail': e.messages}) from e
serializer = serializers.ScriptDetailSerializer(script, context={'request': request})
return Response(serializer.data)
#

View File

@ -1,12 +1,18 @@
from django.contrib.postgres.fields import ArrayField
from django.contrib.postgres.fields.ranges import RangeField
from django.db.models import CharField, JSONField, Lookup
from django.db.models.expressions import Col
from django.db.models.fields.json import KeyTextTransform
from django.db.models.lookups import IContains, IEndsWith, IExact, IStartsWith
from .fields import CachedValueField, ChoiceSetField
__all__ = (
'ChoiceValueLookup',
'CollatedIContains',
'CollatedIEndsWith',
'CollatedIExact',
'CollatedIStartsWith',
'Empty',
'JSONEmpty',
'NetContainsOrEquals',
@ -14,6 +20,10 @@ __all__ = (
'RangeContains',
)
# The ICU collation created by dcim.migrations.0197_natural_sort_collation and applied to
# the name field of most models.
NATURAL_SORT_COLLATION = 'natural_sort'
class RangeContains(Lookup):
"""
@ -123,9 +133,70 @@ class NetContainsOrEquals(Lookup):
return f'CAST({lhs} AS INET) >>= {rhs}', params
class CollatedCaseInsensitiveMixin:
"""
Apply the column's collation to the right-hand side of a case-insensitive comparison.
UPPER() folds according to the collation of its argument. Django uppercases the column
under the column's own collation but the parameter under the database default, so for a
column using natural_sort the two sides disagree: UPPER('ß') is 'SS' on the left and
'ß' on the right, and the comparison silently matches nothing (#23012).
The COLLATE clause must sit inside UPPER(), not after the comparison, or it applies to
the comparison's result rather than to its operand and has no effect.
Tested in dcim.tests.test_filtersets.DeviceCollatedFilterTestCase, which is where the
collated fields these lookups act upon are defined.
"""
def process_rhs(self, compiler, connection):
rhs, params = super().process_rhs(compiler, connection)
collation = getattr(self.lhs.output_field, 'db_collation', None)
# Restricted to a bare column compared against a single placeholder. An expression
# wrapping the column (Collate() and CollateAsChar() in particular) may already
# carry an explicit collation, and PostgreSQL rejects two explicit collations in
# one comparison. Requiring a Col also avoids reading a collation from an
# annotation's output_field which the annotation itself does not carry, as Concat()
# and Coalesce() both do.
#
# The placeholder is compared literally rather than inspected structurally: a field
# declaring its own get_placeholder() compiles to something other than '%s', and
# splicing a COLLATE clause into that is not safe. Any other rhs is a deliberate
# opt-out which leaves the lookup at its previous behaviour.
if collation == NATURAL_SORT_COLLATION and rhs == '%s' and isinstance(self.lhs, Col):
# The collation name cannot be passed as a query parameter, but it originates
# from the field definition rather than from user input.
rhs = f'%s COLLATE "{collation}"'
return rhs, params
class CollatedIContains(CollatedCaseInsensitiveMixin, IContains):
pass
class CollatedIExact(CollatedCaseInsensitiveMixin, IExact):
pass
class CollatedIStartsWith(CollatedCaseInsensitiveMixin, IStartsWith):
pass
class CollatedIEndsWith(CollatedCaseInsensitiveMixin, IEndsWith):
pass
ArrayField.register_lookup(RangeContains)
ChoiceSetField.register_lookup(ChoiceValueLookup)
CharField.register_lookup(Empty)
JSONField.register_lookup(JSONEmpty)
CachedValueField.register_lookup(NetHost)
CachedValueField.register_lookup(NetContainsOrEquals)
# Override the built-in case-insensitive lookups so that they respect the collation of the
# column being searched.
CharField.register_lookup(CollatedIContains)
CharField.register_lookup(CollatedIExact)
CharField.register_lookup(CollatedIStartsWith)
CharField.register_lookup(CollatedIEndsWith)

View File

@ -7,7 +7,7 @@ from django.core.exceptions import ValidationError
from django.core.management.base import BaseCommand, CommandError
from extras.jobs import ScriptJob
from extras.scripts import get_module_and_script
from extras.scripts import EXEC_PARAM_FIELDS, get_module_and_script
from users.models import User
from utilities.request import NetBoxFakeRequest
@ -82,11 +82,11 @@ class Command(BaseCommand):
logger.error(f'\t{field}: {error.get("message")}')
raise CommandError()
# Remove extra fields from ScriptForm before passing data to script
form.cleaned_data.pop('_schedule_at')
form.cleaned_data.pop('_interval')
form.cleaned_data.pop('_commit')
notifications = form.cleaned_data.pop('_notifications')
# Remove exec-parameter fields from ScriptForm before passing data to the script
cleaned_data = form.cleaned_data.copy()
notifications = cleaned_data.pop('_notifications')
for key in EXEC_PARAM_FIELDS:
cleaned_data.pop(key, None)
# Execute the script.
try:
@ -94,7 +94,7 @@ class Command(BaseCommand):
instance=script_obj,
user=user,
immediate=True,
data=form.cleaned_data,
data=cleaned_data,
notifications=notifications,
request=NetBoxFakeRequest({
'META': {},

View File

@ -1091,6 +1091,7 @@ class CustomField(CloningMixin, ExportTemplatesMixin, OwnerMixin, ChangeLoggedMo
# Multiselect
elif self.type == CustomFieldTypeChoices.TYPE_MULTISELECT:
# Do not pin lookup_expr: FILTER_ARRAY_BASED_LOOKUP_MAP preserves the class default under negation
filter_class = filters.MultiValueArrayFilter
# Object

View File

@ -46,6 +46,11 @@ __all__ = (
'get_module_and_script',
)
# Internal ScriptForm fields used to carry execution parameters (see ScriptForm in
# extras/forms/scripts.py). These are validated/sourced separately from the script's own
# declared variables and must never be treated as script data or surfaced as script errors.
EXEC_PARAM_FIELDS = ('_commit', '_schedule_at', '_interval', '_notifications')
# Sentinel distinguishing "argument not supplied" from an explicit None in validate_meta().
_UNSET = object()
@ -706,3 +711,27 @@ def get_module_and_script(module_name, script_name):
module = ScriptModule.objects.get(file_path=f'{module_name}.py')
script = module.scripts.get(name=script_name)
return module, script
def prepare_script_form(script_instance, data, files=None):
"""
Return a bound ScriptForm for the given Script instance, back-filling the declared
`default` of any variable omitted from `data`.
`data` is copied rather than coerced to a plain dict, so a QueryDict retains the
multi-value semantics a MultiObjectVar's multi-select field depends on.
"""
data = data.copy() if data is not None else {}
for name, var in script_instance._get_vars().items():
if name in data:
continue
if (initial := var.field_attrs.get('initial')) is None:
continue
if isinstance(initial, (list, tuple)) and hasattr(data, 'setlist'):
# Assigning a list to a QueryDict stores it as a single nested value, which a
# multi-select widget reads back as one bogus choice. Set the values individually
# so a MultiChoiceVar/MultiObjectVar default binds as it does for a plain dict.
data.setlist(name, list(initial))
else:
data[name] = initial
return script_instance.as_form(data=data, files=files)

View File

@ -13,14 +13,14 @@ from django.urls import reverse
from django.utils.timezone import make_aware, now
from rest_framework import status
from core.choices import ManagedFileRootPathChoices
from core.choices import JobNotificationChoices, ManagedFileRootPathChoices
from core.events import *
from core.models import DataFile, DataSource, Job, ObjectType
from dcim.models import Device, DeviceRole, DeviceType, Location, Manufacturer, Rack, RackRole, Site
from extras.api.serializers import EventRuleSerializer
from extras.choices import *
from extras.models import *
from extras.scripts import BooleanVar, IntegerVar, StringVar
from extras.scripts import BooleanVar, IntegerVar, MultiObjectVar, ObjectVar, StringVar
from extras.scripts import Script as PythonClass
from netbox.event_rules import EventRuleAction, register_event_rule_action
from netbox.registry import registry
@ -1890,6 +1890,208 @@ class ScriptTestCase(APITestCase):
self.assertHttpStatus(response, status.HTTP_404_NOT_FOUND)
class ScriptRunExecutionTestCase(APITestCase):
"""
Exercises ScriptViewSet.run() end-to-end: request -> serializer -> form -> enqueue.
"""
class TestScriptClass(PythonClass):
class Meta:
name = 'Test run script'
site = ObjectVar(model=Site)
sites = MultiObjectVar(model=Site, required=False)
label = StringVar(default='hello')
flag = BooleanVar(default=True)
def run(self, data, commit=True):
return 'ok'
@classmethod
def setUpTestData(cls):
cls.sites = [
Site.objects.create(name=f'Test Site {i}', slug=f'test-site-{i}') for i in range(1, 3)
]
with patch.object(ScriptModule, 'sync_classes'):
module = ScriptModule.objects.create(
file_root=ManagedFileRootPathChoices.SCRIPTS,
file_path='run_script.py',
)
script = Script.objects.create(
module=module,
name='Test run script',
is_executable=True,
)
cls.url = reverse('extras-api:script-detail', kwargs={'pk': script.pk})
def setUp(self):
super().setUp()
self.add_permissions('extras.run_script')
# Monkey-patch the Script model to return our TestScriptClass above, restoring
# the real property afterwards so later tests aren't left with our stub.
python_class_patch = patch.object(Script, 'python_class', new=self.TestScriptClass)
python_class_patch.start()
self.addCleanup(python_class_patch.stop)
# The script-run endpoint gates on a live RQ worker. Tests run without one, so
# bypass the check to exercise validation and the enqueue path.
worker_patch = patch('extras.api.views.any_workers_for_queue', return_value=True)
worker_patch.start()
self.addCleanup(worker_patch.stop)
@patch('extras.jobs.ScriptJob.enqueue')
def test_run_forwards_commit_value(self, mock_enqueue):
for commit_value in (True, False):
with self.subTest(commit=commit_value):
mock_enqueue.reset_mock()
payload = {'data': {'site': self.sites[0].pk}, 'commit': commit_value}
response = self.client.post(self.url, payload, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
mock_enqueue.assert_called_once()
self.assertIs(mock_enqueue.call_args.kwargs['commit'], commit_value)
@patch('extras.jobs.ScriptJob.enqueue')
def test_run_forwards_notifications_value(self, mock_enqueue):
# Regression: ScriptForm.clean() overwrites an empty '_notifications' with the
# field's own initial, so a client-supplied value never reached ScriptJob.enqueue.
payload = {
'data': {'site': self.sites[0].pk},
'commit': True,
'notifications': JobNotificationChoices.NOTIFICATION_NEVER,
}
response = self.client.post(self.url, payload, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
self.assertEqual(
mock_enqueue.call_args.kwargs['notifications'],
JobNotificationChoices.NOTIFICATION_NEVER,
)
@patch('extras.jobs.ScriptJob.enqueue')
def test_run_forwards_schedule_at_and_interval(self, mock_enqueue):
# Regression: schedule_at/interval were likewise read from the form (always
# absent there) instead of the validated request
schedule_at = now() + datetime.timedelta(hours=1)
payload = {
'data': {'site': self.sites[0].pk},
'commit': True,
'schedule_at': schedule_at,
'interval': 60,
}
response = self.client.post(self.url, payload, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
kwargs = mock_enqueue.call_args.kwargs
self.assertEqual(kwargs['schedule_at'], schedule_at)
self.assertEqual(kwargs['interval'], 60)
@patch('extras.jobs.ScriptJob.enqueue')
def test_run_converts_objectvar_and_multiobjectvar_ids(self, mock_enqueue):
payload = {
'data': {
'site': self.sites[0].pk,
'sites': [site.pk for site in self.sites],
},
'commit': True,
}
response = self.client.post(self.url, payload, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
data = mock_enqueue.call_args.kwargs['data']
self.assertEqual(data['site'], self.sites[0])
self.assertEqual(
set(data['sites'].values_list('pk', flat=True)),
{site.pk for site in self.sites},
)
@patch('extras.jobs.ScriptJob.enqueue')
def test_run_backfills_default_for_omitted_required_var(self, mock_enqueue):
# Regression: required vars declaring `default=` were not back-filled before
# binding the form on the API path (unlike the UI path), so they 400'd even
# though the client legitimately omitted them.
payload = {'data': {'site': self.sites[0].pk}, 'commit': True} # 'label' omitted
response = self.client.post(self.url, payload, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
self.assertEqual(mock_enqueue.call_args.kwargs['data']['label'], 'hello')
def test_run_rejects_non_dict_payload(self):
payload = {'data': 'not-a-dict', 'commit': True}
response = self.client.post(self.url, payload, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_400_BAD_REQUEST)
@patch('extras.jobs.ScriptJob.enqueue')
def test_run_rejects_nonexistent_object_id(self, mock_enqueue):
# This is the primary new failure mode introduced by converting raw IDs to model
# instances: a PK that doesn't resolve must 400 cleanly, not enqueue a broken job
# or raise an unhandled DoesNotExist.
nonexistent_pk = Site.objects.order_by('-pk').first().pk + 1000
payload = {'data': {'site': nonexistent_pk}, 'commit': True}
response = self.client.post(self.url, payload, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_400_BAD_REQUEST)
mock_enqueue.assert_not_called()
@patch('extras.jobs.ScriptJob.enqueue')
def test_run_ignores_undeclared_keys(self, mock_enqueue):
# Binding 'data' to the script's form means keys which don't correspond to a declared
# variable are dropped rather than forwarded to run(). This is the contract documented
# under "Running Custom Scripts > Via the API"; pin it so it can't regress silently.
payload = {
'data': {'site': self.sites[0].pk, 'bogus': 'ignored', 'id': 99},
'commit': True,
}
response = self.client.post(self.url, payload, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
data = mock_enqueue.call_args.kwargs['data']
self.assertEqual(data['site'], self.sites[0])
self.assertNotIn('bogus', data)
self.assertNotIn('id', data)
@patch('extras.jobs.ScriptJob.enqueue')
def test_run_rejects_omitted_required_var(self, mock_enqueue):
# 'site' is required and declares no default, so unlike 'label' it cannot be
# back-filled: omitting it must 400 rather than enqueue a job that fails at runtime.
payload = {'data': {'label': 'hi'}, 'commit': True}
response = self.client.post(self.url, payload, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_400_BAD_REQUEST)
self.assertIn('site', response.data['data'])
mock_enqueue.assert_not_called()
@patch('extras.jobs.ScriptJob.enqueue')
def test_run_resolves_booleanvar_default_and_explicit_values(self, mock_enqueue):
# BooleanVar renders as a checkbox, and CheckboxInput reads a missing key as False.
# An omitted BooleanVar must therefore pick up its declared default, while an
# explicitly supplied False must not be overwritten by that default.
for case, supplied, expected in (
('omitted', {}, True),
('explicit False', {'flag': False}, False),
('explicit True', {'flag': True}, True),
):
with self.subTest(case=case):
mock_enqueue.reset_mock()
payload = {'data': {'site': self.sites[0].pk, **supplied}, 'commit': True}
response = self.client.post(self.url, payload, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
self.assertIs(mock_enqueue.call_args.kwargs['data']['flag'], expected)
class CreatedUpdatedFilterTestCase(APITestCase):
@classmethod

View File

@ -2752,6 +2752,7 @@ class CustomFieldModelFilterTestCase(TestCase):
'cf4': None,
'cf6': None,
'cf7': None,
'cf10': None,
})
for filter_name, value in (
@ -2766,6 +2767,7 @@ class CustomFieldModelFilterTestCase(TestCase):
('cf_cf7__nic', 'a'),
('cf_cf7__nisw', 'http://'),
('cf_cf7__niew', '.com'),
('cf_cf10__n', 'A'),
):
with self.subTest(filter_name):
pks = set(
@ -2836,6 +2838,15 @@ class CustomFieldModelFilterTestCase(TestCase):
def test_filter_multiselect(self):
self.assertEqual(self.filterset({'cf_cf10': ['A']}, self.queryset).qs.count(), 1)
self.assertEqual(self.filterset({'cf_cf10': ['A', 'C']}, self.queryset).qs.count(), 2)
# Negation excludes the objects whose array holds the value, not those whose array equals it
self.assertEqual(
set(self.filterset({'cf_cf10__n': ['A']}, self.queryset).qs.values_list('slug', flat=True)),
{'site-2', 'site-3', 'site-4'}
)
self.assertEqual(
set(self.filterset({'cf_cf10__n': ['A', 'C']}, self.queryset).qs.values_list('slug', flat=True)),
{'site-3', 'site-4'}
)
# Matches both the object holding a literal null and the one carrying no key, as `empty` does
self.assertEqual(self.filterset({'cf_cf10': ['null']}, self.queryset).qs.count(), 2)
self.assertEqual(self.filterset({'cf_cf10__empty': True}, self.queryset).qs.count(), 2)
@ -2876,9 +2887,18 @@ def hold_data_lock(custom_field):
cursor.execute('SELECT pg_try_advisory_lock(%s, %s)', lock_key)
if not cursor.fetchone()[0]:
raise RuntimeError(f"Failed to acquire the data lock for {custom_field}")
yield
released = False
try:
yield
finally:
# Closing the connection releases the lock asynchronously, so the next deletion can race it
with connection.cursor() as cursor:
cursor.execute('SELECT pg_advisory_unlock(%s, %s)', lock_key)
released = cursor.fetchone()[0]
# Outside the finally, so a failing body is reported as itself
if not released:
raise RuntimeError(f"Failed to release the data lock for {custom_field}")
finally:
# Closing the session releases any advisory lock held on it
connection.close()
@ -3739,7 +3759,7 @@ class DeferredCustomFieldDataTestCase(TestCase):
# delete() has returned and its own atomic block has exited, but the enclosing transaction
# has yet to commit, so the lock must still be held
with self.assertRaises(RuntimeError):
with self.assertRaisesMessage(RuntimeError, "Failed to acquire the data lock"):
with hold_data_lock(cf):
pass

View File

@ -14,7 +14,7 @@ from core.models import Job, ObjectType
from dcim.models import DeviceType, Manufacturer, Site
from extras.choices import *
from extras.models import *
from extras.scripts import BooleanVar, IntegerVar
from extras.scripts import BooleanVar, IntegerVar, MultiChoiceVar, StringVar
from extras.scripts import Script as PythonClass
from users.models import Group, ObjectPermission, User
from utilities.testing import TestCase, ViewTestCases
@ -1279,6 +1279,62 @@ class ScriptModuleCreateViewTestCase(TestCase):
self.assertEqual(response.context['return_url'], reverse('extras:script_list'))
class ScriptDefaultBackfillTestCase(TestCase):
"""
The UI and the REST API now share prepare_script_form(), so the UI's back-filling of
declared defaults (previously inline in ScriptView.post()) must keep working after
that logic moved into the helper.
"""
user_permissions = ['extras.view_script', 'extras.run_script']
class TestScriptClass(PythonClass):
class Meta:
name = 'Backfill test'
commit_default = False
label = StringVar(default='hello')
flag = BooleanVar(default=True)
picks = MultiChoiceVar(choices=(('a', 'A'), ('b', 'B'), ('c', 'C')), default=['a', 'b'])
def run(self, data, commit):
return 'Complete'
@classmethod
def setUpTestData(cls):
with patch.object(ScriptModule, 'sync_classes'):
module = ScriptModule.objects.create(
file_root=ManagedFileRootPathChoices.SCRIPTS,
file_path='backfill_script.py',
)
cls.script = Script.objects.create(module=module, name='Backfill test', is_executable=True)
def setUp(self):
super().setUp()
python_class_patch = patch.object(Script, 'python_class', new=self.TestScriptClass)
python_class_patch.start()
self.addCleanup(python_class_patch.stop)
@tag('regression')
def test_ui_backfills_declared_defaults(self):
url = reverse('extras:script', kwargs={'pk': self.script.pk})
with (
patch('extras.views.any_workers_for_queue', return_value=True),
patch('extras.jobs.ScriptJob.enqueue') as mock_enqueue,
):
mock_enqueue.return_value.pk = 1
response = self.client.post(url, {'_commit': 'true'})
self.assertEqual(response.status_code, 302)
data = mock_enqueue.call_args.kwargs['data']
self.assertEqual(data['label'], 'hello')
self.assertIs(data['flag'], True)
# A multi-value default must be set on the QueryDict with setlist(): a plain
# assignment stores the list as one nested value, which the multi-select widget
# then rejects as a single invalid choice.
self.assertEqual(data['picks'], ['a', 'b'])
class ScriptValidationErrorTestCase(TestCase):
user_permissions = ['extras.view_script', 'extras.run_script']

View File

@ -24,6 +24,7 @@ from dcim.models import Device, DeviceRole, Platform
from extras.choices import LogLevelChoices
from extras.dashboard.forms import DashboardWidgetAddForm, DashboardWidgetForm
from extras.dashboard.utils import get_widget_class
from extras.scripts import prepare_script_form
from extras.utils import SharedObjectViewMixin
from netbox.object_actions import *
from netbox.ui import layout
@ -1739,13 +1740,7 @@ class ScriptView(BaseScriptView):
'script': script,
})
# Populate missing variables with their default values, if defined
post_data = request.POST.copy()
for name, var in script_class._get_vars().items():
if name not in post_data and (initial := var.field_attrs.get('initial')) is not None:
post_data[name] = initial
form = script_class.as_form(post_data, request.FILES)
form = prepare_script_form(script_class, request.POST, request.FILES)
# Allow execution only if RQ worker process is running
if not any_workers_for_queue('default'):

View File

@ -117,13 +117,20 @@ class VLANTranslationRuleSerializer(NetBoxModelSerializer):
class Meta:
model = VLANTranslationRule
fields = ['id', 'url', 'display', 'policy', 'local_vid', 'remote_vid', 'description']
fields = [
'id', 'url', 'display_url', 'display', 'policy', 'local_vid', 'remote_vid', 'description', 'tags',
'custom_fields', 'created', 'last_updated',
]
brief_fields = ('id', 'url', 'display', 'policy', 'local_vid', 'remote_vid', 'description')
class VLANTranslationPolicySerializer(PrimaryModelSerializer):
rules = VLANTranslationRuleSerializer(many=True, read_only=True)
rules = VLANTranslationRuleSerializer(nested=True, many=True, read_only=True)
class Meta:
model = VLANTranslationPolicy
fields = ['id', 'url', 'display', 'name', 'description', 'display', 'rules', 'owner', 'comments']
fields = [
'id', 'url', 'display_url', 'display', 'name', 'description', 'rules', 'owner', 'comments', 'tags',
'custom_fields', 'created', 'last_updated',
]
brief_fields = ('id', 'url', 'display', 'name', 'description')

View File

@ -28,9 +28,9 @@
"vlan:list_objects_with_permission": 21,
"vlangroup:api_list_objects": 12,
"vlangroup:list_objects_with_permission": 22,
"vlantranslationpolicy:api_list_objects": 12,
"vlantranslationpolicy:api_list_objects": 13,
"vlantranslationpolicy:list_objects_with_permission": 17,
"vlantranslationrule:api_list_objects": 12,
"vlantranslationrule:api_list_objects": 13,
"vlantranslationrule:list_objects_with_permission": 18,
"vrf:api_list_objects": 20,
"vrf:list_objects_with_permission": 17

View File

@ -11,7 +11,7 @@ from ipam.choices import *
from ipam.models import *
from tenancy.models import Tenant
from utilities.data import string_to_ranges
from utilities.testing import APITestCase, APIViewTestCases, create_test_device, disable_logging
from utilities.testing import APITestCase, APIViewTestCases, create_tags, create_test_device, disable_logging
class AppTestCase(APITestCase):
@ -1422,6 +1422,39 @@ class VLANTranslationPolicyTestCase(APIViewTestCases.APIViewTestCase):
},
]
def test_standard_fields_in_representation(self):
"""The standard URL, tag, custom-field and change-tracking names appear in the representation."""
policy = VLANTranslationPolicy.objects.first()
self.add_permissions('ipam.view_vlantranslationpolicy')
response = self.client.get(self._get_detail_url(policy), **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
expected = {'display_url', 'tags', 'custom_fields', 'created', 'last_updated'}
self.assertEqual(expected - set(response.data), set())
def test_nested_rules_are_brief(self):
"""Rules nested in a policy representation carry only the rule serializer's brief fields."""
policy = VLANTranslationPolicy.objects.first()
VLANTranslationRule.objects.create(policy=policy, local_vid=100, remote_vid=200)
self.add_permissions('ipam.view_vlantranslationpolicy')
response = self.client.get(self._get_detail_url(policy), **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
self.assertEqual(sorted(response.data['rules'][0]), VLANTranslationRuleTestCase.brief_fields)
def test_update_tags(self):
"""Tags supplied on update are assigned and rendered in the response."""
policy = VLANTranslationPolicy.objects.first()
create_tags('Alpha')
self.add_permissions('ipam.change_vlantranslationpolicy', 'extras.view_tag')
data = {'tags': [{'slug': 'alpha'}]}
response = self.client.patch(self._get_detail_url(policy), data, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
self.assertIn('tags', response.data)
self.assertEqual([tag['slug'] for tag in response.data['tags']], ['alpha'])
self.assertEqual(list(policy.tags.values_list('slug', flat=True)), ['alpha'])
class VLANTranslationRuleTestCase(APIViewTestCases.APIViewTestCase):
model = VLANTranslationRule
@ -1491,6 +1524,29 @@ class VLANTranslationRuleTestCase(APIViewTestCases.APIViewTestCase):
'description': 'New description',
}
def test_standard_fields_in_representation(self):
"""The standard URL, tag, custom-field and change-tracking names appear in the representation."""
rule = VLANTranslationRule.objects.first()
self.add_permissions('ipam.view_vlantranslationrule')
response = self.client.get(self._get_detail_url(rule), **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
expected = {'display_url', 'tags', 'custom_fields', 'created', 'last_updated'}
self.assertEqual(expected - set(response.data), set())
def test_update_tags(self):
"""Tags supplied on update are assigned and rendered in the response."""
rule = VLANTranslationRule.objects.first()
create_tags('Alpha')
self.add_permissions('ipam.change_vlantranslationrule', 'extras.view_tag')
data = {'tags': [{'slug': 'alpha'}]}
response = self.client.patch(self._get_detail_url(rule), data, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
self.assertIn('tags', response.data)
self.assertEqual([tag['slug'] for tag in response.data['tags']], ['alpha'])
self.assertEqual(list(rule.tags.values_list('slug', flat=True)), ['alpha'])
class ServiceTemplateTestCase(APIViewTestCases.APIViewTestCase):
model = ServiceTemplate

View File

@ -18,6 +18,7 @@ from extras.models import CustomField, SavedFilter
from users.filterset_mixins import OwnerFilterMixin
from utilities import filters
from utilities.constants import (
FILTER_ARRAY_BASED_LOOKUP_MAP,
FILTER_CHAR_BASED_LOOKUP_MAP,
FILTER_NEGATION_LOOKUP_MAP,
FILTER_NUMERIC_BASED_LOOKUP_MAP,
@ -170,6 +171,12 @@ class BaseFilterSet(django_filters.FilterSet):
# These filter types support only negation
return FILTER_NEGATION_LOOKUP_MAP
if isinstance(existing_filter, (
filters.MultiValueArrayFilter,
)):
# Must precede the char-based branch below, which would otherwise shadow this subclass
return FILTER_ARRAY_BASED_LOOKUP_MAP
if isinstance(existing_filter, (
django_filters.filters.CharFilter,
django_filters.ChoiceFilter,

View File

@ -6,6 +6,7 @@ from unittest.mock import MagicMock, patch
from django.conf import settings
from django.contrib.messages.storage.fallback import FallbackStorage
from django.http import HttpResponse
from django.test import Client, RequestFactory, SimpleTestCase
from django.test import TestCase as DjangoTestCase
from django.test.utils import override_settings
@ -861,7 +862,7 @@ class SSOLoginButtonTestCase(DjangoTestCase):
Return the body of the rendered SSO form. The password login form renders its own hidden
`next` field, so assertions about the SSO parameters must be scoped to this form.
"""
begin_url = reverse('social:begin', args=['google-oauth2'])
begin_url = reverse('social_auth_begin', args=['google-oauth2'])
match = re.search(
rf'<form[^>]*action="{re.escape(begin_url)}"[^>]*>(.*?)</form>',
response.content.decode(),
@ -876,7 +877,7 @@ class SSOLoginButtonTestCase(DjangoTestCase):
"""
Each SSO button must be rendered as a POST form (including a CSRF token) rather than a link.
"""
begin_url = reverse('social:begin', args=['google-oauth2'])
begin_url = reverse('social_auth_begin', args=['google-oauth2'])
response = self.client.get(reverse('login'))
self.assertEqual(response.status_code, 200)
@ -951,6 +952,158 @@ class SSOLoginButtonTestCase(DjangoTestCase):
self.assertEqual(auth_backend['params'].get('next'), '/dcim/sites/')
class SocialAuthBeginViewTestCase(DjangoTestCase):
"""
Verify the view which initiates an SSO login. Chromium-based browsers evaluate the CSP
`form-action` directive against every hop in a form submission's redirect chain, so redirecting
the submission to the identity provider is blocked wherever `form-action 'self'` is enforced.
Clients which ask for JSON are handed the identity provider's URL to navigate to instead
(see #23112).
"""
SSO_BACKENDS = [
'social_core.backends.google.GoogleOAuth2',
'netbox.authentication.ObjectPermissionBackend',
]
AUTHORIZATION_URL = 'https://accounts.google.com/o/oauth2/auth'
# Stands in for the document rendered by a backend which does not redirect (see BaseAuth.start())
AUTH_HTML = '<html><body><form id="openid_message" action="https://idp.example.com/"></form></body></html>'
def setUp(self):
# load_backends() caches the discovered backends in a module-level dict, so isolate the
# backends overridden below from the remainder of the test suite.
cache_patcher = patch.dict('social_core.backends.utils.BACKENDSCACHE', {}, clear=True)
cache_patcher.start()
self.addCleanup(cache_patcher.stop)
self.url = reverse('social_auth_begin', args=['google-oauth2'])
@override_settings(AUTHENTICATION_BACKENDS=SSO_BACKENDS)
def test_json_request_returns_authorization_url(self):
"""
A client which requests JSON receives the identity provider's URL in the response body
rather than an HTTP redirect, so that it can navigate there itself.
"""
response = self.client.post(self.url, headers={'accept': 'application/json'})
self.assertEqual(response.status_code, 200)
self.assertEqual(response.headers['Content-Type'], 'application/json')
self.assertNotIn('Location', response.headers)
self.assertTrue(response.json()['url'].startswith(self.AUTHORIZATION_URL))
@override_settings(AUTHENTICATION_BACKENDS=SSO_BACKENDS)
def test_form_submission_returns_redirect(self):
"""
A client which has not asked for JSON (e.g. a browser with JavaScript disabled) receives the
unmodified redirect from python-social-auth.
"""
response = self.client.post(self.url, headers={'accept': 'text/html'})
self.assertEqual(response.status_code, 302)
self.assertTrue(response.headers['Location'].startswith(self.AUTHORIZATION_URL))
@override_settings(AUTHENTICATION_BACKENDS=SSO_BACKENDS)
def test_session_state_recorded(self):
"""
The anti-forgery state conveyed to the identity provider must be recorded in the session, as
the completion view compares the two. This is what makes the JSON response safe to follow:
the session established here is the one the callback is validated against.
"""
response = self.client.post(self.url, headers={'accept': 'application/json'})
state = self.client.session['google-oauth2_state']
self.assertIn(f'state={state}', response.json()['url'])
@override_settings(AUTHENTICATION_BACKENDS=SSO_BACKENDS)
def test_next_recorded_in_session(self):
"""
The post-login URL is read from the POST data by do_auth() and stashed in the session; the
wrapper must not interfere with the form fields rendered on the login page.
"""
self.client.post(self.url, {'next': '/dcim/sites/'}, headers={'accept': 'application/json'})
self.assertEqual(self.client.session['next'], '/dcim/sites/')
@override_settings(AUTHENTICATION_BACKENDS=SSO_BACKENDS)
def test_get_request_not_allowed(self):
"""
Authentication must be initiated by POST: a GET request is trivially forgeable, which is why
social-auth-app-django restricts its own begin view to POST.
"""
response = self.client.get(self.url)
self.assertEqual(response.status_code, 405)
@override_settings(AUTHENTICATION_BACKENDS=SSO_BACKENDS)
def test_csrf_token_required(self):
"""
CSRF protection must be retained, so that a third party cannot silently initiate an SSO
login on the user's behalf.
"""
client = Client(enforce_csrf_checks=True)
response = client.post(self.url, headers={'accept': 'application/json'})
self.assertEqual(response.status_code, 403)
@override_settings(AUTHENTICATION_BACKENDS=SSO_BACKENDS)
def test_response_is_not_cached(self):
"""
The authorization URL embeds a single-use state parameter and must never be cached.
"""
response = self.client.post(self.url, headers={'accept': 'application/json'})
self.assertIn('no-store', response.headers['Cache-Control'])
@override_settings(AUTHENTICATION_BACKENDS=SSO_BACKENDS)
def test_unknown_backend(self):
"""
An unconfigured backend yields an HTTP 404, as it does via python-social-auth directly.
"""
response = self.client.post(reverse('social_auth_begin', args=['nosuchbackend']))
self.assertEqual(response.status_code, 404)
@override_settings(AUTHENTICATION_BACKENDS=SSO_BACKENDS)
@patch('social_core.backends.google.GoogleOAuth2.uses_redirect', return_value=False)
@patch('social_core.backends.google.GoogleOAuth2.auth_html', return_value=AUTH_HTML)
def test_json_request_returns_html_for_non_redirecting_backend(self, _auth_html, _uses_redirect):
"""
A backend which renders its own HTML rather than redirecting has that document returned in
the response body. The client renders it in place: were it made to submit the form to fetch
the document again, the login would be initiated a second time.
"""
response = self.client.post(self.url, headers={'accept': 'application/json'})
self.assertEqual(response.status_code, 200)
self.assertEqual(response.headers['Content-Type'], 'application/json')
self.assertNotIn('Location', response.headers)
self.assertEqual(response.json()['html'], self.AUTH_HTML)
@override_settings(AUTHENTICATION_BACKENDS=SSO_BACKENDS)
@patch('social_core.backends.google.GoogleOAuth2.uses_redirect', return_value=False)
@patch('social_core.backends.google.GoogleOAuth2.auth_html', return_value=AUTH_HTML)
def test_html_passed_through_for_non_redirecting_backend(self, _auth_html, _uses_redirect):
"""
A client which has not asked for JSON receives that same document unmodified.
"""
response = self.client.post(self.url, headers={'accept': 'text/html'})
self.assertEqual(response.status_code, 200)
self.assertEqual(response.content.decode(), self.AUTH_HTML)
@override_settings(AUTHENTICATION_BACKENDS=SSO_BACKENDS)
@patch('account.views.social_auth_begin', side_effect=lambda *args, **kwargs: HttpResponse(status=502))
def test_json_request_passes_through_error_response(self, _begin):
"""
Only a redirect or a rendered document is translated to JSON. An unsuccessful response is
passed through as-is, so that the client reports the failure rather than mistaking the
response for a login it can act on.
"""
response = self.client.post(self.url, headers={'accept': 'application/json'})
self.assertEqual(response.status_code, 502)
self.assertNotEqual(response.headers.get('Content-Type'), 'application/json')
class SocialAuthExceptionMiddlewareTestCase(SimpleTestCase):
"""
Verify that SSO/SAML authentication failures are surfaced as a login-page message rather than

View File

@ -4,7 +4,7 @@ from django.urls import path
from django.views.decorators.cache import cache_page
from drf_spectacular.views import SpectacularAPIView, SpectacularRedocView, SpectacularSwaggerView
from account.views import LoginView, LogoutView
from account.views import LoginView, LogoutView, SocialAuthBeginView
from netbox.api.views import APIRootView, AuthenticationCheckView, StatusView
from netbox.graphql.schema import schema
from netbox.graphql.views import NetBoxGraphQLView
@ -20,6 +20,7 @@ _patterns = [
# Login/logout
path('login/', LoginView.as_view(), name='login'),
path('logout/', LogoutView.as_view(), name='logout'),
path('oauth/begin/<str:backend>/', SocialAuthBeginView.as_view(), name='social_auth_begin'),
path('oauth/', include('social_django.urls', namespace='social')),
# Apps

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@ -33,7 +33,7 @@
"markdown-it": "^15.0.1",
"picomatch": "4.0.7",
"query-string": "9.5.1",
"sass": "1.103.1",
"sass": "1.104.1",
"tom-select": "2.6.2",
"typeface-inter": "3.18.1",
"typeface-roboto-mono": "1.1.13"
@ -45,11 +45,11 @@
"@types/bootstrap": "5.2.11",
"@types/cookie": "^1.0.0",
"@types/node": "^24.10.1",
"@typescript-eslint/eslint-plugin": "^8.68.0",
"@typescript-eslint/parser": "^8.68.0",
"@typescript-eslint/eslint-plugin": "^8.70.0",
"@typescript-eslint/parser": "^8.70.0",
"esbuild": "^0.28.2",
"esbuild-sass-plugin": "^3.7.0",
"eslint": "^10.9.1",
"eslint": "^10.10.0",
"eslint-config-prettier": "^10.1.8",
"eslint-import-resolver-typescript": "^4.4.5",
"eslint-plugin-import": "^2.32.0",

View File

@ -14,6 +14,7 @@ import { initRackElevation } from './racks';
import { initHtmx } from './htmx';
import { initSavedFilterSelect } from './forms/savedFiltersSelect';
import { initHotkeys } from './hotkeys';
import { initSSOForms } from './sso';
function initDocument(): void {
for (const init of [
@ -33,6 +34,7 @@ function initDocument(): void {
initHtmx,
initSavedFilterSelect,
initHotkeys,
initSSOForms,
]) {
init();
}

View File

@ -0,0 +1,124 @@
import { getElement, getElements } from './util';
// Abandon a login which has gone unanswered for this many milliseconds, so that a request which
// hangs (a backend retrieving identity provider metadata of its own, for instance) surfaces an
// error rather than leaving the SSO buttons disabled indefinitely.
const REQUEST_TIMEOUT = 15000;
// Whether a login is already being initiated. Each login is issued a single-use state parameter,
// which the completion view compares against the value recorded in the session, so a second login
// would invalidate the state of the one being navigated to. This is scoped to the document rather
// than to the form: SAML renders one form per identity provider, and all of them write the session
// keys of the same backend.
let loginPending = false;
/**
* Enable or disable every SSO button on the page. All of them are disabled while a login is being
* initiated, as only one login can be in flight at a time.
*/
function setButtonsDisabled(disabled: boolean): void {
for (const form of getElements<HTMLFormElement>('form.sso-login-form')) {
for (const button of form.querySelectorAll<HTMLButtonElement>('button[type="submit"]')) {
button.disabled = disabled;
}
}
}
function setErrorVisible(visible: boolean): void {
const error = getElement('sso-error');
if (error === null) return;
error.classList.toggle('d-none', !visible);
if (visible) {
// Revealing an alert whose text has not itself changed is not reliably announced by a screen
// reader, so move focus to it (the login the user asked for did not begin, and the reason for
// that is the only thing worth their attention).
error.focus();
}
}
/**
* Begin an SSO login by navigating to the identity provider, rather than by submitting the form.
*
* The social auth "begin" endpoint responds with a redirect to the identity provider. Chromium-based
* browsers evaluate the CSP `form-action` directive against every hop in a form submission's
* redirect chain, so a deployment which serves NetBox with `form-action 'self'` blocks that redirect
* and the SSO button appears to do nothing. Requesting the identity provider's URL and navigating to
* it here sidesteps the directive, which does not govern a navigation initiated by a script.
*/
async function beginLogin(form: HTMLFormElement): Promise<void> {
const body = new URLSearchParams();
for (const [name, value] of new FormData(form).entries()) {
if (typeof value === 'string') {
body.append(name, value);
}
}
// Browsers predating AbortSignal.timeout() issue the request without a deadline, which is no
// worse than the behavior they had before it was imposed.
const signal =
typeof AbortSignal.timeout === 'function' ? AbortSignal.timeout(REQUEST_TIMEOUT) : null;
const res = await fetch(form.action, {
method: 'POST',
headers: { Accept: 'application/json' },
body,
credentials: 'same-origin',
signal,
});
if (!res.ok || !(res.headers.get('Content-Type') ?? '').includes('application/json')) {
throw new Error(`The login request returned an unexpected response (HTTP ${res.status})`);
}
const { url, html } = (await res.json()) as { url?: string; html?: string };
if (typeof url === 'string') {
window.location.assign(url);
} else if (typeof html === 'string') {
// The backend renders its own HTML (an auto-submitting form, for instance) instead of
// redirecting. That document has already been generated by the request above, so render it in
// place; submitting the form to fetch it again would initiate the login a second time. Note
// that this does not evade `form-action`: the document is written into NetBox's own, so the
// form it carries is submitted under NetBox's policy exactly as it would have been otherwise.
document.open();
document.write(html);
document.close();
} else {
throw new Error('The login response contained neither a URL nor a document');
}
}
export function initSSOForms(): void {
for (const form of getElements<HTMLFormElement>('form.sso-login-form')) {
form.addEventListener('submit', event => {
event.preventDefault();
if (loginPending) return;
loginPending = true;
setErrorVisible(false);
setButtonsDisabled(true);
beginLogin(form).catch(error => {
// Report the failure rather than falling back to submitting the form: a deployment which
// serves NetBox with `form-action 'self'` — the very condition this indirection exists to
// work around — blocks that submission silently, leaving a dead button and no explanation.
// The alert shown to the user cannot say why the login failed, so log the reason.
console.error(error);
loginPending = false;
setButtonsDisabled(false);
setErrorVisible(true);
});
});
}
// A browser which restores the login page from its back/forward cache (having navigated to the
// identity provider and returned) preserves both the DOM and the state above, which would
// otherwise leave every SSO button permanently disabled. Only a restore may clear the guard:
// pageshow also fires on an ordinary load, after `load` and so after a login begun in the
// meantime, where resetting it would readmit the very double-submission it exists to prevent.
window.addEventListener('pageshow', event => {
if (!event.persisted) return;
loginPending = false;
setButtonsDisabled(false);
});
}

View File

@ -2,6 +2,24 @@
# yarn lockfile v1
"@cacheable/memory@^2.2.0":
version "2.2.0"
resolved "https://registry.yarnpkg.com/@cacheable/memory/-/memory-2.2.0.tgz#72aeb8b051f6d597d10ac8595b94ad8302bd2239"
integrity sha512-CTLKqLItRCEixEAewD3/j9DB3/o96gpTPD4eJ1v+DGOlxZRZncRQkGYqqnAGCscYd6RNeXfGeiuCphsPtqyIfQ==
dependencies:
"@cacheable/utils" "^2.5.0"
"@keyv/bigmap" "^1.3.1"
hookified "^1.15.1"
keyv "^5.6.0"
"@cacheable/utils@^2.5.0":
version "2.5.0"
resolved "https://registry.yarnpkg.com/@cacheable/utils/-/utils-2.5.0.tgz#534c91113aa48fe43baedb169550b6ee070ef303"
integrity sha512-buipgOVDkkPXNR5+xBpDw7Zk2n1EvU7qBJCNUcL7rhQ//kfpOXPAvQ511Os0vpLYJ1pZnvudNytkQt2hst3wqA==
dependencies:
hashery "^1.5.1"
keyv "^5.6.0"
"@emnapi/core@^1.4.3":
version "1.7.1"
resolved "https://registry.yarnpkg.com/@emnapi/core/-/core-1.7.1.tgz#3a79a02dbc84f45884a1806ebb98e5746bdfaac4"
@ -174,9 +192,9 @@
integrity sha512-EriSTlt5OC9/7SXkRSCAhfSxxoSUgBm33OH+IkwbdpgoqsSsUg7y3uh+IICI/Qg4BBWr3U2i39RpmycbxMq4ew==
"@eslint/compat@^2.1.0":
version "2.1.0"
resolved "https://registry.yarnpkg.com/@eslint/compat/-/compat-2.1.0.tgz#8c66110f95cf0fdd864b76ae4d534042dea7bb7f"
integrity sha512-LgaSCymEpw7tF53xvDw9SNsraPb1IBHxpdABIOM0hW8UAlP8znrjYtuxfR58FSJ3L9BhwD+FaPRFQpZq84Nh6g==
version "2.1.1"
resolved "https://registry.yarnpkg.com/@eslint/compat/-/compat-2.1.1.tgz#0284d648b62c44f7653804d615d2b4b3b8eaeef1"
integrity sha512-rMcy8GSrwNzcISX/BlTDY/GLB4eCopEuy9woIls3To+15OLxykZrxxq+WUcylCPCQ6F4MujjBM1DX5V1aqI3Vw==
dependencies:
"@eslint/core" "^1.2.1"
@ -204,9 +222,9 @@
"@types/json-schema" "^7.0.15"
"@eslint/eslintrc@^3.3.6":
version "3.3.6"
resolved "https://registry.yarnpkg.com/@eslint/eslintrc/-/eslintrc-3.3.6.tgz#d22bfd6b3a7d8e1f2c0b2f2e6de111b53ec6e13e"
integrity sha512-l2Ul9PrHsPCKcEY/ac7VgFj9D80C7S68sOKc618SyHDPK36s1XcFebXY0iTzUVn4Yq+YbwvSnDmCz9yxjX+QrA==
version "3.3.7"
resolved "https://registry.yarnpkg.com/@eslint/eslintrc/-/eslintrc-3.3.7.tgz#76d3dedec4a30ea32df797bf8a0085c1311b7316"
integrity sha512-F42g89Qd5oAWtp0k0nnSrjziAKza7w8SVT4mStc18LZMaRb4J1HQAHLCalEtDCxrTuksx7NU9qsmeLwpOfPqWw==
dependencies:
ajv "^6.14.0"
debug "^4.3.2"
@ -214,7 +232,7 @@
globals "^14.0.0"
ignore "^5.2.0"
import-fresh "^3.2.1"
js-yaml "^4.3.0"
js-yaml "^4.3.2"
minimatch "^3.1.5"
strip-json-comments "^3.1.1"
@ -228,10 +246,10 @@
resolved "https://registry.yarnpkg.com/@eslint/object-schema/-/object-schema-3.0.5.tgz#88e9bf4d11d2b19c082e78ebe7ce88724a5eb091"
integrity sha512-vqTaUEgxzm+YDSdElad6PiRoX4t8VGDjCtt05zn4nU810UIx/uNEV7/lZJ6KwFThKZOzOxzXy48da+No7HZaMw==
"@eslint/plugin-kit@^0.7.2":
version "0.7.2"
resolved "https://registry.yarnpkg.com/@eslint/plugin-kit/-/plugin-kit-0.7.2.tgz#4b0962f3f2c7ce8bc98b3ecfe34525c09d2cb729"
integrity sha512-+CNAzxglkrpNf/kKywqQfk74QjtceuOE7Qm+AF8miRvPF/wmmK5+OJOgVh3AVTT3RP2mH3+FOaxlE5v72owk0A==
"@eslint/plugin-kit@^0.7.3":
version "0.7.3"
resolved "https://registry.yarnpkg.com/@eslint/plugin-kit/-/plugin-kit-0.7.3.tgz#cc7268cc36405b331ef92db1bc37971f21d66fe1"
integrity sha512-IkO+/KEUvwbVpiURZg+P7zF74z5Jxe0UgJxVni+RtoHQ6IZieXaO02kmadomap/q+l6bc/jdPGGqTjhuZnuz1Q==
dependencies:
"@eslint/core" "^1.2.1"
levn "^0.4.1"
@ -393,6 +411,19 @@
dependencies:
"@swc/helpers" "^0.5.0"
"@keyv/bigmap@^1.3.1":
version "1.3.1"
resolved "https://registry.yarnpkg.com/@keyv/bigmap/-/bigmap-1.3.1.tgz#fc82fa83947e7ff68c6798d08907db842771ef2c"
integrity sha512-WbzE9sdmQtKy8vrNPa9BRnwZh5UF4s1KTmSK0KUVLo3eff5BlQNNWDnFOouNpKfPKDnms9xynJjsMYjMaT/aFQ==
dependencies:
hashery "^1.4.0"
hookified "^1.15.0"
"@keyv/serialize@^1.1.1":
version "1.1.1"
resolved "https://registry.yarnpkg.com/@keyv/serialize/-/serialize-1.1.1.tgz#0c01dd3a3483882af7cf3878d4e71d505c81fc4a"
integrity sha512-dXn3FZhPv0US+7dtJsIi2R+c7qWYiReoEh5zUntWCf4oSpMNib8FDhSoed6m3QyZdx5hK7iLFkYk3rNxwt8vTA==
"@mdi/font@7.4.47":
version "7.4.47"
resolved "https://registry.npmjs.org/@mdi/font/-/font-7.4.47.tgz"
@ -924,100 +955,100 @@
dependencies:
"@types/estree" "*"
"@typescript-eslint/eslint-plugin@^8.68.0":
version "8.69.0"
resolved "https://registry.yarnpkg.com/@typescript-eslint/eslint-plugin/-/eslint-plugin-8.69.0.tgz#bf74cc392ebcaaf096bc8b4c4d7bbeb0677687b8"
integrity sha512-t5jQTKPIgVW1PE6dR6H6Qz5gm8zjMlX5/2gRaOGd9eO6V7J+tQc6iWKukEe7dY8u9HyYasQ0yfF0/FSSTEO2gA==
"@typescript-eslint/eslint-plugin@^8.70.0":
version "8.70.0"
resolved "https://registry.yarnpkg.com/@typescript-eslint/eslint-plugin/-/eslint-plugin-8.70.0.tgz#59f635c74dd1e2bffb6aecbda557b24dadffd3dc"
integrity sha512-/v8HZt6RlyIZxB3ntehELOcUcfxKPVGWXnQdJuHRmzrqgF8nQypcC/oxGW+Ot4VGKDq81XugPKxx0n5PBtf9PA==
dependencies:
"@eslint-community/regexpp" "^4.12.2"
"@typescript-eslint/scope-manager" "8.69.0"
"@typescript-eslint/type-utils" "8.69.0"
"@typescript-eslint/utils" "8.69.0"
"@typescript-eslint/visitor-keys" "8.69.0"
"@typescript-eslint/scope-manager" "8.70.0"
"@typescript-eslint/type-utils" "8.70.0"
"@typescript-eslint/utils" "8.70.0"
"@typescript-eslint/visitor-keys" "8.70.0"
ignore "^7.0.5"
natural-compare "^1.4.0"
ts-api-utils "^2.5.0"
"@typescript-eslint/parser@^8.68.0":
version "8.69.0"
resolved "https://registry.yarnpkg.com/@typescript-eslint/parser/-/parser-8.69.0.tgz#de3ead2b35e5c71580eda40820adb4fd14834ca1"
integrity sha512-l4b0DhWioGg6Gt2ebGlvfkFMOjRsauxtsnDRwUSRX1qHq3HdTfQHV8wW9zEXeciai6HfeaKOedQn2Zoofx3WBw==
"@typescript-eslint/parser@^8.70.0":
version "8.70.0"
resolved "https://registry.yarnpkg.com/@typescript-eslint/parser/-/parser-8.70.0.tgz#96ce2de96c06c8442fea1a8f7b07855a56b3c5e3"
integrity sha512-zYvrmj9Yxd63UGaXw+kdt6A0F0s0qveJyuatIM77bYC2DE4pgmg7a50u8LR7PRtXd0x+h+Tl3eXabGm06SWd3Q==
dependencies:
"@typescript-eslint/scope-manager" "8.69.0"
"@typescript-eslint/types" "8.69.0"
"@typescript-eslint/typescript-estree" "8.69.0"
"@typescript-eslint/visitor-keys" "8.69.0"
"@typescript-eslint/scope-manager" "8.70.0"
"@typescript-eslint/types" "8.70.0"
"@typescript-eslint/typescript-estree" "8.70.0"
"@typescript-eslint/visitor-keys" "8.70.0"
debug "^4.4.3"
"@typescript-eslint/project-service@8.69.0":
version "8.69.0"
resolved "https://registry.yarnpkg.com/@typescript-eslint/project-service/-/project-service-8.69.0.tgz#cf728554436a50e644a5214a89fe02cb1ffa9af8"
integrity sha512-yi4obFrHMmnsesWehHbkg9zMA7Jt8cXT+mKM08G999pH1yT6nqgsHx7MYm0uY1wAj8CqiBXYRJ7WAT0QdQHQXg==
"@typescript-eslint/project-service@8.70.0":
version "8.70.0"
resolved "https://registry.yarnpkg.com/@typescript-eslint/project-service/-/project-service-8.70.0.tgz#a62e837362f26c604ad15b20bacce1c3f4d6b552"
integrity sha512-hFHbTNqhU9G+2eKFXCBVb1tjFT/LceiJ4+HfLO4pTpDI0KHi6iajpcFFkaSQ9gXmCh7n82A0PthaayEdN6mspQ==
dependencies:
"@typescript-eslint/tsconfig-utils" "^8.69.0"
"@typescript-eslint/types" "^8.69.0"
"@typescript-eslint/tsconfig-utils" "^8.70.0"
"@typescript-eslint/types" "^8.70.0"
debug "^4.4.3"
"@typescript-eslint/scope-manager@8.69.0":
version "8.69.0"
resolved "https://registry.yarnpkg.com/@typescript-eslint/scope-manager/-/scope-manager-8.69.0.tgz#13f3d1e25108e95a9ceb5a198806d1fa558f8c7a"
integrity sha512-ewfspqWvSxKSOaplqAUNbaSFO0eB6w1EtQ+esfYFRm3614Ty4uNtExkcbgd6nWsXphbqKyf9ZYdbZdv2xEoWEQ==
"@typescript-eslint/scope-manager@8.70.0":
version "8.70.0"
resolved "https://registry.yarnpkg.com/@typescript-eslint/scope-manager/-/scope-manager-8.70.0.tgz#b77628b03c9c56ef21fb5a6bc2f4a4335163b999"
integrity sha512-8nP3Kwh5hlgZ4FicGvmznAmJe8UL4sdU8tLukrPaMuQmDuk4Y8xYfzu/aYZW4xT2JCgc7H/TpDI5cGlxcWJSqQ==
dependencies:
"@typescript-eslint/types" "8.69.0"
"@typescript-eslint/visitor-keys" "8.69.0"
"@typescript-eslint/types" "8.70.0"
"@typescript-eslint/visitor-keys" "8.70.0"
"@typescript-eslint/tsconfig-utils@8.69.0", "@typescript-eslint/tsconfig-utils@^8.69.0":
version "8.69.0"
resolved "https://registry.yarnpkg.com/@typescript-eslint/tsconfig-utils/-/tsconfig-utils-8.69.0.tgz#d3b0ccc781ab252a90a0b3989b9d1eb85ab59469"
integrity sha512-xNqK7YTDZsLniQMV/4rpFR8Z5JlqeRvVjuG1YgF/mdPVH84HSD19L8CczMA0qg2RfwEV231GHH3VnToJDo4MfQ==
"@typescript-eslint/tsconfig-utils@8.70.0", "@typescript-eslint/tsconfig-utils@^8.70.0":
version "8.70.0"
resolved "https://registry.yarnpkg.com/@typescript-eslint/tsconfig-utils/-/tsconfig-utils-8.70.0.tgz#f583ca72159c4fd8e775c153da3241de6b77974f"
integrity sha512-adnkeeNq9Sq1sUf4+FRVc0KdgYghzsgFpZSQVZVvY0LCuUuN0FnQgyGzCJeC4fW1cdXseBAjU2EOqUIjbNcZUw==
"@typescript-eslint/type-utils@8.69.0":
version "8.69.0"
resolved "https://registry.yarnpkg.com/@typescript-eslint/type-utils/-/type-utils-8.69.0.tgz#7ce68d2ebcbedd8421806c27a7f360755017159f"
integrity sha512-ZfoJAVg3JZndQEpEl9petVlxau3lRuElc4HRMuAlLCf8to04/iHz692RUSNmXKDjEuJmIL+KZ2/BsOcBc16dsA==
"@typescript-eslint/type-utils@8.70.0":
version "8.70.0"
resolved "https://registry.yarnpkg.com/@typescript-eslint/type-utils/-/type-utils-8.70.0.tgz#7f9c01c24e56bfad2a089167926c7cdded9de5f6"
integrity sha512-NUMKIhYVaVIVLnRL9CRt+VVcuLgSHUCpXn4/+K8wql+vdInUzvx8BjUO1oJ7cG9shjFJKtF8F8Hh2kCh3/KBVw==
dependencies:
"@typescript-eslint/types" "8.69.0"
"@typescript-eslint/typescript-estree" "8.69.0"
"@typescript-eslint/utils" "8.69.0"
"@typescript-eslint/types" "8.70.0"
"@typescript-eslint/typescript-estree" "8.70.0"
"@typescript-eslint/utils" "8.70.0"
debug "^4.4.3"
ts-api-utils "^2.5.0"
"@typescript-eslint/types@8.69.0", "@typescript-eslint/types@^8.69.0":
version "8.69.0"
resolved "https://registry.yarnpkg.com/@typescript-eslint/types/-/types-8.69.0.tgz#5d9ad3f707c2e4f70a2db540031104df3e63bcf5"
integrity sha512-K3VrubUPhlo9VDBS6QdI8YB5j7ClpqLRdefcz6PFrhnwicehBweqQ9Evhl4l+FYz0HdDmMqIiSX0aldGRYtDCA==
"@typescript-eslint/types@8.70.0", "@typescript-eslint/types@^8.70.0":
version "8.70.0"
resolved "https://registry.yarnpkg.com/@typescript-eslint/types/-/types-8.70.0.tgz#9ee52888cdeca604fe9436935219b967fa7f6053"
integrity sha512-asTOIYhDg4zdzOScCyaytrsV3cR6B4ecPQlXw/dJIm7J/MZTtCtfVII9JD8Geh4jTCrK/Xe6cg5UevoleMcoJQ==
"@typescript-eslint/typescript-estree@8.69.0":
version "8.69.0"
resolved "https://registry.yarnpkg.com/@typescript-eslint/typescript-estree/-/typescript-estree-8.69.0.tgz#efa915913ffe2049bbfd26092b95d1bc7c9c454f"
integrity sha512-AdFkgqck3Vudb/kWnxlyafU/4aBhHrbQ9locP2N4psXTy5mOBg0SHJumnLvx7r6g1gV4DKvUFwV2nJZBoqOD8w==
"@typescript-eslint/typescript-estree@8.70.0":
version "8.70.0"
resolved "https://registry.yarnpkg.com/@typescript-eslint/typescript-estree/-/typescript-estree-8.70.0.tgz#79cfcd9678ee28ea69cc13032c0f89bb1d298917"
integrity sha512-d9NmHMPEKQ7QCLLm1jI3zmoQBwT5KwFYjXBJ9ymZfKCUU+5rmTRykKAFvH5Qn/ZCds3CEAFS9OC9M/jkl0X2bA==
dependencies:
"@typescript-eslint/project-service" "8.69.0"
"@typescript-eslint/tsconfig-utils" "8.69.0"
"@typescript-eslint/types" "8.69.0"
"@typescript-eslint/visitor-keys" "8.69.0"
"@typescript-eslint/project-service" "8.70.0"
"@typescript-eslint/tsconfig-utils" "8.70.0"
"@typescript-eslint/types" "8.70.0"
"@typescript-eslint/visitor-keys" "8.70.0"
debug "^4.4.3"
minimatch "^10.2.2"
semver "^7.7.3"
tinyglobby "^0.2.15"
ts-api-utils "^2.5.0"
"@typescript-eslint/utils@8.69.0":
version "8.69.0"
resolved "https://registry.yarnpkg.com/@typescript-eslint/utils/-/utils-8.69.0.tgz#67ad9c00edf12fe2fbc0bf0a71b00822a8d02e97"
integrity sha512-tUbx60BBqQa31kXF5MCsOOLL5E/WzUuxIn7YpAvq+eaUlqvk8/NXnXMBNAdLCr0icjkzem7iUA5QqWHe/hJ1aw==
"@typescript-eslint/utils@8.70.0":
version "8.70.0"
resolved "https://registry.yarnpkg.com/@typescript-eslint/utils/-/utils-8.70.0.tgz#78a78b4c52dd8523e5321993cb46ebe6d7934510"
integrity sha512-oZmtKJz/4fufZ2p3+Cn3ijEojcdfR+1zYDH2xKYrEly0dR/Q/1xUPRCOlKGxod78nWlU2UnDe09GZ3TaknBFGA==
dependencies:
"@eslint-community/eslint-utils" "^4.9.1"
"@typescript-eslint/scope-manager" "8.69.0"
"@typescript-eslint/types" "8.69.0"
"@typescript-eslint/typescript-estree" "8.69.0"
"@typescript-eslint/scope-manager" "8.70.0"
"@typescript-eslint/types" "8.70.0"
"@typescript-eslint/typescript-estree" "8.70.0"
"@typescript-eslint/visitor-keys@8.69.0":
version "8.69.0"
resolved "https://registry.yarnpkg.com/@typescript-eslint/visitor-keys/-/visitor-keys-8.69.0.tgz#f659785dbb79733c40499f71a65439e2033966b5"
integrity sha512-+rmdgPA+EXkNgKYvHvFfhrs35utXbwaC5PGpDquSXcoXQDKUA5UjV0LmTucG/4JXkM31BTu4TilHtrN8IVBe8w==
"@typescript-eslint/visitor-keys@8.70.0":
version "8.70.0"
resolved "https://registry.yarnpkg.com/@typescript-eslint/visitor-keys/-/visitor-keys-8.70.0.tgz#b451c8aea76dc97fc768b8d9d7f61019bf789628"
integrity sha512-BoC8PiO4Hkdo0TVJh9Ntxr5MxPDI7/oFsrygN5ADelFSeXG/qgNuucIGA+L5Z6JpPTE/uRfcTWtscjbUaufepQ==
dependencies:
"@typescript-eslint/types" "8.69.0"
"@typescript-eslint/types" "8.70.0"
eslint-visitor-keys "^5.0.0"
"@unrs/resolver-binding-android-arm-eabi@1.11.1":
@ -1123,9 +1154,9 @@ acorn-jsx@^5.3.2:
integrity sha512-rq9s+JNhf0IChjtDXxllJ7g41oZk5SlXtp0LHwyA5cejwn7vKmKp4pPri6YEePv2PU65sAsegbXtIinmDFDXgQ==
acorn@^8.15.0:
version "8.15.0"
resolved "https://registry.npmjs.org/acorn/-/acorn-8.15.0.tgz"
integrity sha512-NZyJarBfL7nWwIq+FDL6Zp/yHEhePMNnnJ0y3qfieCrmNvYct8uvtiV41UvlSe6apAfk0fY1FbWx+NwfmpvtTg==
version "8.18.0"
resolved "https://registry.yarnpkg.com/acorn/-/acorn-8.18.0.tgz#4faf01b2d6d326bfeed97aea1f52220b5f4c1940"
integrity sha512-lGq+9yr1/GuAWaVYIHRjvvySG5/4VfKIvC8EWxStPdcDh/Ka7FG3twP6v4d5BkravUilhIAsG4Qj83t02LWUPQ==
acorn@^8.16.0:
version "8.16.0"
@ -1282,9 +1313,9 @@ bootstrap@5.3.8:
integrity sha512-HP1SZDqaLDPwsNiqRqi5NcP0SSXciX2s9E+RyqJIIqGo+vJeN5AJVM98CXmW/Wux0nQ5L7jeWUdplCEf0Ee+tg==
brace-expansion@^1.1.7:
version "1.1.14"
resolved "https://registry.yarnpkg.com/brace-expansion/-/brace-expansion-1.1.14.tgz#d9de602370d91347cd9ddad1224d4fd701eb348b"
integrity sha512-MWPGfDxnyzKU7rNOW9SP/c50vi3xrmrua/+6hfPbCS2ABNWfx24vPidzvC7krjU/RTo235sV776ymlsMtGKj8g==
version "1.1.21"
resolved "https://registry.yarnpkg.com/brace-expansion/-/brace-expansion-1.1.21.tgz#edf4fab5c64d051aea5a8def49aba1c7522279f3"
integrity sha512-9zeA+KLZNNzglF2TPKRQEDyx6Yby7daAkuy8MiPzpXPsYDWi/DRM8jmwUDxokQjYqBpv5DgPiwD4h4ZZSy1Ujw==
dependencies:
balanced-match "^1.0.0"
concat-map "0.0.1"
@ -1303,6 +1334,17 @@ braces@^3.0.3:
dependencies:
fill-range "^7.1.1"
cacheable@^2.5.0:
version "2.5.0"
resolved "https://registry.yarnpkg.com/cacheable/-/cacheable-2.5.0.tgz#d142d41043e5a865f6053cc70ef4e3ad068bd5ce"
integrity sha512-60cyAOytib/OzBw1JNSoSV/boK1AtHryDIjvVBk7XbN4ugfkM3+Sry7fEjNgPMGgOjuaZPAp8ruZ0Cxafwyq9g==
dependencies:
"@cacheable/memory" "^2.2.0"
"@cacheable/utils" "^2.5.0"
hookified "^1.15.0"
keyv "^5.6.0"
qified "^0.10.1"
call-bind-apply-helpers@^1.0.0, call-bind-apply-helpers@^1.0.1, call-bind-apply-helpers@^1.0.2:
version "1.0.2"
resolved "https://registry.npmjs.org/call-bind-apply-helpers/-/call-bind-apply-helpers-1.0.2.tgz"
@ -1342,7 +1384,7 @@ call-bound@^1.0.2, call-bound@^1.0.3, call-bound@^1.0.4:
callsites@^3.0.0:
version "3.1.0"
resolved "https://registry.npmjs.org/callsites/-/callsites-3.1.0.tgz"
resolved "https://registry.yarnpkg.com/callsites/-/callsites-3.1.0.tgz#b3630abd8943432f54b3f0519238e33cd7df2f73"
integrity sha512-P8BjAsXvZS+VIDUI11hHCQEv74YT67YUi5JJFNWIqL235sBmjX4+qx9Muvls5ivyNENctx46xQLQ3aTuE7ssaQ==
chokidar@^4.0.0:
@ -1890,7 +1932,7 @@ eslint-visitor-keys@^3.4.3:
eslint-visitor-keys@^4.2.1:
version "4.2.1"
resolved "https://registry.npmjs.org/eslint-visitor-keys/-/eslint-visitor-keys-4.2.1.tgz"
resolved "https://registry.yarnpkg.com/eslint-visitor-keys/-/eslint-visitor-keys-4.2.1.tgz#4cfea60fe7dd0ad8e816e1ed026c1d5251b512c1"
integrity sha512-Uhdk5sfqcee/9H/rCOJikYz67o0a2Tw2hGRPOG2Y1R2dg7brRe1uG0yaNQDHu+TO/uQPF/5eCapvYSmHUjt7JQ==
eslint-visitor-keys@^5.0.0, eslint-visitor-keys@^5.0.1:
@ -1898,17 +1940,17 @@ eslint-visitor-keys@^5.0.0, eslint-visitor-keys@^5.0.1:
resolved "https://registry.yarnpkg.com/eslint-visitor-keys/-/eslint-visitor-keys-5.0.1.tgz#9e3c9489697824d2d4ce3a8ad12628f91e9f59be"
integrity sha512-tD40eHxA35h0PEIZNeIjkHoDR4YjjJp34biM0mDvplBe//mB+IHCqHDGV7pxF+7MklTvighcCPPZC7ynWyjdTA==
eslint@^10.9.1:
version "10.9.1"
resolved "https://registry.yarnpkg.com/eslint/-/eslint-10.9.1.tgz#409da5c41a5536d5a849f8555a18ca7ef1eb963b"
integrity sha512-9VaAkDURekixUQJy0oJYl2DcN6oKMfxay7XzaGYAWQwsb6qfKf+x76R2k1L8kb1boc+FyCAaTA9GmiKaaiaF+A==
eslint@^10.10.0:
version "10.10.0"
resolved "https://registry.yarnpkg.com/eslint/-/eslint-10.10.0.tgz#14b1d1849eedb2ee6805f3759050479ce2b23d71"
integrity sha512-NPXn6r5zl4uET1DAVPaOwzX3rut4c0wcmw3dWJAfOsTM5+TogXo0DDjz8pwm/hL8cyVNpHqeK4JpN0NjnyFFNw==
dependencies:
"@eslint-community/eslint-utils" "^4.8.0"
"@eslint-community/regexpp" "^4.12.2"
"@eslint/config-array" "^0.23.5"
"@eslint/config-helpers" "^0.7.0"
"@eslint/core" "^1.2.1"
"@eslint/plugin-kit" "^0.7.2"
"@eslint/plugin-kit" "^0.7.3"
"@humanfs/node" "^0.16.6"
"@humanwhocodes/module-importer" "^1.0.1"
"@humanwhocodes/retry" "^0.4.2"
@ -1923,7 +1965,7 @@ eslint@^10.9.1:
esquery "^1.7.0"
esutils "^2.0.2"
fast-deep-equal "^3.1.3"
file-entry-cache "^8.0.0"
file-entry-cache "11.1.5 || >11.1.6 <12"
find-up "^5.0.0"
glob-parent "^6.0.2"
ignore "^5.2.0"
@ -1936,7 +1978,7 @@ eslint@^10.9.1:
espree@^10.0.1:
version "10.4.0"
resolved "https://registry.npmjs.org/espree/-/espree-10.4.0.tgz"
resolved "https://registry.yarnpkg.com/espree/-/espree-10.4.0.tgz#d54f4949d4629005a1fa168d937c3ff1f7e2a837"
integrity sha512-j6PAQ2uUr79PZhBjP5C5fhl8e39FmRnOjsD5lGnWrFU8i2G776tBK7+nP8KuQUTTyAZUwfQqXAgrVH5MbH9CYQ==
dependencies:
acorn "^8.15.0"
@ -2001,12 +2043,12 @@ fdir@^6.5.0:
resolved "https://registry.yarnpkg.com/fdir/-/fdir-6.5.0.tgz#ed2ab967a331ade62f18d077dae192684d50d350"
integrity sha512-tIbYtZbucOs0BRGqPJkshJUYdL+SDH7dVM8gjy+ERp3WAUjLEFJE+02kanyHtwjWOnwrKYBiwAmM0p4kLJAnXg==
file-entry-cache@^8.0.0:
version "8.0.0"
resolved "https://registry.yarnpkg.com/file-entry-cache/-/file-entry-cache-8.0.0.tgz#7787bddcf1131bffb92636c69457bbc0edd6d81f"
integrity sha512-XXTUwCvisa5oacNGRP9SfNtYBNAMi+RPwBFmblZEF7N7swHYQS6/Zfk7SRwx4D5j3CH211YNRco1DEMNVfZCnQ==
"file-entry-cache@11.1.5 || >11.1.6 <12":
version "11.1.5"
resolved "https://registry.yarnpkg.com/file-entry-cache/-/file-entry-cache-11.1.5.tgz#c8210eb055de63e68685ccfb6a017e386d4577d0"
integrity sha512-+PFTHITI08JIGhnNpGNI8T8inUpgZfk3GNEqfT9R2zZV2iFXg3CvqzSl/uEhs7TSGujYRELEANyDvS8Fj7+S7Q==
dependencies:
flat-cache "^4.0.0"
flat-cache "^6.1.23"
fill-range@^7.1.1:
version "7.1.1"
@ -2028,23 +2070,24 @@ find-up@^5.0.0:
locate-path "^6.0.0"
path-exists "^4.0.0"
flat-cache@^4.0.0:
version "4.0.1"
resolved "https://registry.yarnpkg.com/flat-cache/-/flat-cache-4.0.1.tgz#0ece39fcb14ee012f4b0410bd33dd9c1f011127c"
integrity sha512-f7ccFPK3SXFHpx15UIGyRJ/FJQctuKZ0zVuN3frBo4HnK3cay9VEW0R6yPYFHC0AgqhukPzKjq22t5DmAyqGyw==
flat-cache@^6.1.23:
version "6.1.23"
resolved "https://registry.yarnpkg.com/flat-cache/-/flat-cache-6.1.23.tgz#735dc888c271868d7301b3bbb8edba5028afb61d"
integrity sha512-f++BY9pTk+983xK1FLzlLpmM0i0z+jHmx3QESGkURMXujQZz1k5wzwX6hjnQ8goaD0B+sYnDK1yZ6MTyZfUaqA==
dependencies:
flatted "^3.2.9"
keyv "^4.5.4"
cacheable "^2.5.0"
flatted "^3.4.2"
hookified "^1.15.0"
flatpickr@4.6.13:
version "4.6.13"
resolved "https://registry.npmjs.org/flatpickr/-/flatpickr-4.6.13.tgz"
integrity sha512-97PMG/aywoYpB4IvbvUJi0RQi8vearvU0oov1WW3k0WZPBMrTQVqekSX5CjSG/M4Q3i6A/0FKXC7RyAoAUUSPw==
flatted@^3.2.9:
version "3.4.2"
resolved "https://registry.yarnpkg.com/flatted/-/flatted-3.4.2.tgz#f5c23c107f0f37de8dbdf24f13722b3b98d52726"
integrity sha512-PjDse7RzhcPkIJwy5t7KPWQSZ9cAbzQXcafsetQoD7sOJRQlGikNbx7yZp2OotDnJyrDcbyRq3Ttb18iYOqkxA==
flatted@^3.4.2:
version "3.4.4"
resolved "https://registry.yarnpkg.com/flatted/-/flatted-3.4.4.tgz#aeeca2a506303f0cee61c59e6c9f2a88d2f29fc6"
integrity sha512-5+ybhBZANEJxaH3X5evAFatUxLfEHSr7n6kYJ+1Qd0mUqr4eu9gIf6GDbWHf8RJijHrjjO8G+la14SlL2SeS1Q==
for-each@^0.3.3:
version "0.3.3"
@ -2182,7 +2225,7 @@ glob-parent@^6.0.2:
globals@^14.0.0:
version "14.0.0"
resolved "https://registry.npmjs.org/globals/-/globals-14.0.0.tgz"
resolved "https://registry.yarnpkg.com/globals/-/globals-14.0.0.tgz#898d7413c29babcf6bafe56fcadded858ada724e"
integrity sha512-oahGvuMGQlPw/ivIYBjVSrWAfWLBeku5tpPE2fOPLi+WHffIWbuh2tCjhyQhTBPMf5E9jDEH4FOmTYgYwbKwtQ==
globals@^17.11.0:
@ -2292,6 +2335,13 @@ has-tostringtag@^1.0.0, has-tostringtag@^1.0.2:
dependencies:
has-symbols "^1.0.3"
hashery@^1.4.0, hashery@^1.5.1:
version "1.5.1"
resolved "https://registry.yarnpkg.com/hashery/-/hashery-1.5.1.tgz#4ba82ad54911ac617467870845d57a9fe508a400"
integrity sha512-iZyKG96/JwPz1N55vj2Ie2vXbhu440zfUfJvSwEqEbeLluk7NnapfGqa7LH0mOsnDxTF85Mx8/dyR6HfqcbmbQ==
dependencies:
hookified "^1.15.0"
hasown@^2.0.0, hasown@^2.0.1, hasown@^2.0.2:
version "2.0.2"
resolved "https://registry.npmjs.org/hasown/-/hasown-2.0.2.tgz"
@ -2299,6 +2349,16 @@ hasown@^2.0.0, hasown@^2.0.1, hasown@^2.0.2:
dependencies:
function-bind "^1.1.2"
hookified@^1.15.0, hookified@^1.15.1:
version "1.15.1"
resolved "https://registry.yarnpkg.com/hookified/-/hookified-1.15.1.tgz#b1fafeaa5489cdc29cb85546a8f837ed4ffbbcb6"
integrity sha512-MvG/clsADq1GPM2KGo2nyfaWVyn9naPiXrqIe4jYjXNZQt238kWyOGrsyc/DmRAQ+Re6yeo6yX/yoNCG5KAEVg==
hookified@^2.1.1:
version "2.2.0"
resolved "https://registry.yarnpkg.com/hookified/-/hookified-2.2.0.tgz#1d024ac1668973dd5bcc4a96ab9ccdb7639ef8d4"
integrity sha512-p/LgFzRN5FeoD3DLS6bkUapeye6E4SI6yJs6KetENd18S+FBthqYq2amJUWpt5z0EQwwHemidjY5OqJGEKm5uA==
htmx.org@2.0.10:
version "2.0.10"
resolved "https://registry.yarnpkg.com/htmx.org/-/htmx.org-2.0.10.tgz#62442b0e2952a885ae2e50a7654b8b20d0981134"
@ -2321,7 +2381,7 @@ immutable@^5.1.5:
import-fresh@^3.2.1:
version "3.3.1"
resolved "https://registry.npmjs.org/import-fresh/-/import-fresh-3.3.1.tgz"
resolved "https://registry.yarnpkg.com/import-fresh/-/import-fresh-3.3.1.tgz#9cecb56503c0ada1f2741dbbd6546e4b13b57ccf"
integrity sha512-TR3KfrTZTYLPB6jUjfx6MF9WcWrHL9su5TObK4ZkYgBdWKPOFoSoQIdEuTuR82pmtxH2spWG9h6etwfr1pLBqQ==
dependencies:
parent-module "^1.0.0"
@ -2670,18 +2730,13 @@ js-cookie@3.0.8:
resolved "https://registry.yarnpkg.com/js-tokens/-/js-tokens-4.0.0.tgz#19203fb59991df98e3a287050d4647cdeaf32499"
integrity sha512-RdJUflcE3cUzKiMqQgsCu06FPu9UdIJO0beYbPhHN4k6apgJtifcoCtT9bcxOpYBtpD2kCM6Sbzg4CausW/PKQ==
js-yaml@^4.3.0:
version "4.3.0"
resolved "https://registry.yarnpkg.com/js-yaml/-/js-yaml-4.3.0.tgz#d1900572a7f7cf0b5f540c83673e60bad3436592"
integrity sha512-1td788aAnnZ5qs7V2QIRl1owjtYpbKt749Y3xauqQgwIIGF/xXWz1wMTEBx5O3LK3lXLVuqXPdPxj2BoFHaW9Q==
js-yaml@^4.3.2:
version "4.3.2"
resolved "https://registry.yarnpkg.com/js-yaml/-/js-yaml-4.3.2.tgz#8e44fb14a2643c59726bb15787b5f1512cb3d3fb"
integrity sha512-SFNOvSJ+Dgf/9An904Yx+CgSlIPCkIpao4qo51lpee25TIRejdH3rhR4EZMGoNx3/TP3O+wzWuiTFl4sqbltzA==
dependencies:
argparse "^2.0.1"
json-buffer@3.0.1:
version "3.0.1"
resolved "https://registry.yarnpkg.com/json-buffer/-/json-buffer-3.0.1.tgz#9338802a30d3b6605fbe0613e094008ca8c05a13"
integrity sha512-4bV5BfR2mqfQTJm+V5tPPdf+ZpuhiIvTuAB5g8kcrXOZpTT/QwwVRWBywX1ozr6lEuPdbHxwaJlm9G6mI2sfSQ==
json-schema-traverse@^0.4.1:
version "0.4.1"
resolved "https://registry.yarnpkg.com/json-schema-traverse/-/json-schema-traverse-0.4.1.tgz#69f6a87d9513ab8bb8fe63bdb0979c448e684660"
@ -2699,12 +2754,12 @@ json5@^1.0.2:
dependencies:
minimist "^1.2.0"
keyv@^4.5.4:
version "4.5.4"
resolved "https://registry.yarnpkg.com/keyv/-/keyv-4.5.4.tgz#a879a99e29452f942439f2a405e3af8b31d4de93"
integrity sha512-oxVHkHR/EJf2CNXnWxRLW6mg7JyCCUcG0DtEGmL2ctUo1PNTin1PUil+r/+4r5MpVgC/fn1kjsx7mjSujKqIpw==
keyv@^5.6.0:
version "5.6.0"
resolved "https://registry.yarnpkg.com/keyv/-/keyv-5.6.0.tgz#03044074c6b4d072d0a62c7b9fa649537baf0105"
integrity sha512-CYDD3SOtsHtyXeEORYRx2qBtpDJFjRTGXUtmNEMGyzYOKj1TE3tycdlho7kA1Ufx9OYWZzg52QFBGALTirzDSw==
dependencies:
json-buffer "3.0.1"
"@keyv/serialize" "^1.1.1"
levn@^0.4.1:
version "0.4.1"
@ -2742,7 +2797,7 @@ loose-envify@^1.1.0:
dependencies:
js-tokens "^3.0.0 || ^4.0.0"
markdown-it@^14.1.0, markdown-it@^14.2.0:
markdown-it@^14.1.0, markdown-it@^14.2.0, markdown-it@^15.0.1:
version "14.2.0"
resolved "https://registry.yarnpkg.com/markdown-it/-/markdown-it-14.2.0.tgz#06d48d9035e77d5b1c85adb315482fc8240289ef"
integrity sha512-1TGiQiJVRQ3NPmZH6sx5Cfnmg6GQm9jvC1ch4TK511NjSJvjzKLzn5pPfZRNZkRPZP0HqCioSndqH8v2nRaWVQ==
@ -2754,18 +2809,6 @@ markdown-it@^14.1.0, markdown-it@^14.2.0:
punycode.js "^2.3.1"
uc.micro "^2.1.0"
markdown-it@^15.0.1:
version "15.0.1"
resolved "https://registry.yarnpkg.com/markdown-it/-/markdown-it-15.0.1.tgz#33b87eafff0feb08cbef5edfd9bc20b3d920f5d8"
integrity sha512-9/7gE95FNPkfUWrjJIoHZza2iLmuJlPD0UNMxPi7bxUrbCR525YZY0r+zyfes0dZI5ZZ/uNIXUJca0pJvtw41g==
dependencies:
argparse "^3.0.0"
entities "^8.0.0"
linkify-it "^6.0.0"
mdurl "^2.1.0"
punycode.js "^2.3.1"
uc.micro "^3.0.0"
math-intrinsics@^1.1.0:
version "1.1.0"
resolved "https://registry.npmjs.org/math-intrinsics/-/math-intrinsics-1.1.0.tgz"
@ -2948,7 +2991,7 @@ p-locate@^5.0.0:
parent-module@^1.0.0:
version "1.0.1"
resolved "https://registry.npmjs.org/parent-module/-/parent-module-1.0.1.tgz"
resolved "https://registry.yarnpkg.com/parent-module/-/parent-module-1.0.1.tgz#691d2709e78c79fae3a156622452d00762caaaa2"
integrity sha512-GQ2EWRpQV8/o+Aw8YqtfZZPfNRWZYkbidE9k5rpl/hC3vtHHBfGm2Ifi6qWV+coDGkrUKZAxE3Lot5kcsRlh+g==
dependencies:
callsites "^3.0.0"
@ -3015,6 +3058,13 @@ punycode@^2.1.0:
resolved "https://registry.yarnpkg.com/punycode/-/punycode-2.3.1.tgz#027422e2faec0b25e1549c3e1bd8309b9133b6e5"
integrity sha512-vYt7UD1U9Wg6138shLtLOvdAu+8DsC/ilFtEVHcH+wydcSpNE20AfSOduf6MkRFahL5FY7X1oU7nKVZFtfq8Fg==
qified@^0.10.1:
version "0.10.1"
resolved "https://registry.yarnpkg.com/qified/-/qified-0.10.1.tgz#0640bf21bbe6ca540db290ad8f5fde4d870b8bda"
integrity sha512-+Owyggi9IxT1ePKGafcI87ubSmxol6smwJ+RAHDQlx9+9cPwFWDiKFFCPuWhr9ignlGpZ9vDQLw67N4dcTVFEA==
dependencies:
hookified "^2.1.1"
query-string@9.5.1:
version "9.5.1"
resolved "https://registry.yarnpkg.com/query-string/-/query-string-9.5.1.tgz#aecdc091d3dc7ce293eed83957e220d523a3d0f7"
@ -3146,7 +3196,7 @@ regexp.prototype.flags@^1.5.4:
resolve-from@^4.0.0:
version "4.0.0"
resolved "https://registry.npmjs.org/resolve-from/-/resolve-from-4.0.0.tgz"
resolved "https://registry.yarnpkg.com/resolve-from/-/resolve-from-4.0.0.tgz#4abcd852ad32dd7baabfe9b40e00a36db5f392e6"
integrity sha512-pb/MYmXstAkysRFx8piNI1tGFNQIFA3vkE3Gq4EuA1dF6gHp/+vgZqsCGJapvy8N3Q+4o7FwvquPJcnZ7RYy4g==
resolve-pkg-maps@^1.0.0:
@ -3219,10 +3269,10 @@ safe-regex-test@^1.1.0:
es-errors "^1.3.0"
is-regex "^1.2.1"
sass@1.103.1:
version "1.103.1"
resolved "https://registry.yarnpkg.com/sass/-/sass-1.103.1.tgz#13b70f5ff69288db956dc27d27c8fb79f3a27c61"
integrity sha512-9icZURbP51S6S0QGoyaeqk9uB06GNWxsFYWfH5RgpFgqK5FA8tJcM3AdVxrZEVJ7dz+L87nG95gBKf4VuaMHGw==
sass@1.104.1:
version "1.104.1"
resolved "https://registry.yarnpkg.com/sass/-/sass-1.104.1.tgz#29a4bdb33c48a8bb656e300a33afcafbfcbc8491"
integrity sha512-yDA+1aIG3EHgN4V/BvuhCvu61FF6hEd4e+9DxikUm9U0CAGuvdIZ/UYy7qbOxjhbbWQraoyLlMuzdRGOSV5Bmw==
dependencies:
chokidar "^5.0.0"
immutable "^5.1.5"
@ -3450,7 +3500,7 @@ strip-bom@^3.0.0:
strip-json-comments@^3.1.1:
version "3.1.1"
resolved "https://registry.npmjs.org/strip-json-comments/-/strip-json-comments-3.1.1.tgz"
resolved "https://registry.yarnpkg.com/strip-json-comments/-/strip-json-comments-3.1.1.tgz#31f1281b3832630434831c310c01cccda8cbe006"
integrity sha512-6fPc+R4ihwqP6N/aIv2f1gMH8lOVtWQHoqC4yK6oSDVVocumAsfCqjkXnqiYMhmMwS/mEHLp7Vehlt3ql6lEig==
supports-preserve-symlinks-flag@^1.0.0:

View File

@ -1,3 +1,3 @@
version: "4.7.0"
version: "4.7.1"
edition: "Community"
published: "2026-09-02"
published: "2026-09-15"

View File

@ -83,11 +83,15 @@
{% if login_form_hidden %}
<h2 class="text-center mb-4">{% trans "Log In" %}</h2>
{% endif %}
{# Revealed and focused by initSSOForms() when a login cannot be initiated #}
<div id="sso-error" class="alert alert-danger d-none" role="alert" tabindex="-1">
{% trans "Unable to begin single sign-on. Please try again." %}
</div>
<div class="row">
{% for backend in auth_backends %}
<div class="col">
{# The social auth begin view accepts only POST requests #}
<form action="{{ backend.url }}" method="post">
{# SSO logins are initiated by POST; see SocialAuthBeginView #}
<form action="{{ backend.url }}" method="post" class="sso-login-form">
{% csrf_token %}
{% for param, value in backend.params.items %}
<input type="hidden" name="{{ param }}" value="{{ value }}" />

View File

@ -0,0 +1,35 @@
"""Reinstall the ltree cascade triggers with a restore-safe WHEN clause.
The cascade triggers installed by 0025_ltree_paths compared two ltree values with
`IS DISTINCT FROM`, which resolves the `ltree = ltree` operator through search_path at
CREATE TRIGGER time. pg_dump emits `set_config('search_path', '', false)`, so restoring a
v4.7.0 dump could not create these triggers and because psql does not stop on error by
default, the restore reported success with the triggers silently missing. See #23130.
Reinstalling covers both affected databases: one restored from such a dump (the triggers
are absent) and one upgraded in place (they exist with the old definition, which would
fail its own next restore). InstallLtreeTriggers drops before creating, so this applies
cleanly in either state.
This does not repair path/sort_path values which went stale while the triggers were
missing; see the v4.7.1 release notes for detection and repair.
Reversing this migration is a no-op: the triggers it replaces belong to 0025_ltree_paths,
which recreates them (from the corrected template) when reversed in turn.
"""
from django.db import migrations
from utilities.ltree import ReinstallLtreeTriggers
TABLES = ('tenancy_tenantgroup', 'tenancy_contactgroup')
class Migration(migrations.Migration):
dependencies = [
('tenancy', '0026_consolidate_unique_constraints'),
]
operations = [
*[ReinstallLtreeTriggers(t, name_column='name') for t in TABLES],
]

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@ -17,6 +17,12 @@ FILTER_CHAR_BASED_LOOKUP_MAP = dict(
iregex='iregex',
)
# A member is a scalar inside a stored array, so negation cannot fall back to equality
FILTER_ARRAY_BASED_LOOKUP_MAP = {
**FILTER_CHAR_BASED_LOOKUP_MAP,
'n': 'contains',
}
FILTER_NUMERIC_BASED_LOOKUP_MAP = dict(
n='exact',
lte='lte',

View File

@ -11,6 +11,7 @@ from jsonschema.exceptions import SchemaError
from jsonschema.validators import validator_for
from utilities.string import title
from utilities.templatetags.builtins.filters import render_markdown
from utilities.validators import MultipleOfValidator
__all__ = (
@ -86,9 +87,10 @@ class JSONSchemaProperty:
"""
Instantiate and return a Django form field suitable for editing the property's value.
"""
field_class = self.field_class
field_kwargs = {
'label': self.title or title(name),
'help_text': self.description,
'help_text': render_markdown(self.description),
'required': required,
'initial': self.default,
}
@ -110,10 +112,14 @@ class JSONSchemaProperty:
# String validation
if self.type == PropertyTypeEnum.STRING.value:
if self.minLength is not None:
field_kwargs['min_length'] = self.minLength
if self.maxLength is not None:
field_kwargs['max_length'] = self.maxLength
# Checking against CharField is safe because the other CharField-derived fields are
# ruled out by the "is a string" check above. UUIDField is the exception: it cleans to
# a uuid.UUID, which the length validators can't call len() on.
if issubclass(field_class, forms.CharField) and not issubclass(field_class, forms.UUIDField):
if self.minLength is not None:
field_kwargs['min_length'] = self.minLength
if self.maxLength is not None:
field_kwargs['max_length'] = self.maxLength
if self.pattern is not None:
field_kwargs['validators'] = [
RegexValidator(regex=self.pattern)
@ -121,11 +127,12 @@ class JSONSchemaProperty:
# Integer/number validation
elif self.type in (PropertyTypeEnum.INTEGER.value, PropertyTypeEnum.NUMBER.value):
field_kwargs['widget'] = forms.NumberInput(attrs={'step': 'any'})
if self.minimum:
field_kwargs['min_value'] = self.minimum
if self.maximum:
field_kwargs['max_value'] = self.maximum
if issubclass(field_class, forms.IntegerField):
field_kwargs['widget'] = forms.NumberInput(attrs={'step': 'any'})
if self.minimum is not None:
field_kwargs['min_value'] = self.minimum
if self.maximum is not None:
field_kwargs['max_value'] = self.maximum
if self.multipleOf:
field_kwargs['validators'] = [
MultipleOfValidator(multiple=self.multipleOf)

View File

@ -9,11 +9,43 @@ not on the model definitions.
The paths maintained by these triggers are never computed or mutated from Python;
the model layer only reads `path`/`sort_path` back from the database.
Trigger DDL and search_path
---------------------------
Everything this module emits is replayed verbatim by `pg_restore`, which runs with
`search_path` set to the empty string and schema-qualifies every name it can (a
CVE-2018-1058 hardening). Unqualified names in the SQL below therefore have to
resolve without help from the path, and the two halves of a trigger differ in when
that resolution happens:
* A trigger's WHEN clause is resolved at CREATE TRIGGER time. `IS DISTINCT FROM`
(like `=`, `<`, ...) is grammar which expands to the operand type's operator, and
there is no syntax to schema-qualify it. An extension type whose operators live
outside `pg_catalog` `ltree` installs into `public` makes that CREATE TRIGGER
unrestorable, and because `psql` does not stop on error by default the restore
appears to succeed with the trigger silently missing (#23130).
* A trigger FUNCTION's body survives only because pg_dump emits
`SET check_function_bodies = false`, which suppresses the validation that would
otherwise reject the unqualified `ltree` declarations below at CREATE FUNCTION
time. Under the default `check_function_bodies = on` they fail with
`type "ltree" does not exist`. So the bodies are not inherently path-independent;
they are exempted by the restore's own configuration. Anything replaying this DDL
outside a pg_dump context must either put the extension's schema on the path or
set that GUC itself.
Rule: keep WHEN-clause operand types inside `pg_catalog`. Cast an
extension-typed column with `::text` (ltree's text I/O is byte-canonical, so
`::text` equality is exactly ltree equality).
`RestoreUnderRestrictedSearchPathTests` in `utilities/tests/test_ltree.py` enforces
this by creating the generated DDL with the extension's schema off the search_path.
"""
from django.db import migrations
__all__ = (
'InstallLtreeTriggers',
'ReinstallLtreeTriggers',
'ltree_trigger_sql',
)
@ -220,10 +252,35 @@ CREATE TRIGGER "{table}_ltree_compute_path"
# because that statement does not touch parent_id or {name_col}, the AFTER
# trigger does not re-fire on those descendant rows. This prevents the
# quadratic re-cascade that would otherwise occur for any deep subtree.
#
# `path` is compared as text rather than as ltree. `IS DISTINCT FROM` is SQL
# grammar, not an operator: it has no schema-qualification syntax, and it expands
# to the operand type's `=` operator, which is resolved from search_path at
# CREATE TRIGGER time. `ltree =` lives in whichever schema the extension was
# installed into (normally `public`), so a CREATE TRIGGER replayed by pg_restore
# — which runs with `search_path` set to the empty string and schema-qualifies
# every name it can — cannot resolve it and fails with "operator does not exist:
# public.ltree = public.ltree", silently dropping the cascade trigger from the
# restored database (#23130). `text =` is in `pg_catalog`, which is always on the
# effective path, so the cast makes the DDL search_path-independent.
#
# The comparison is equivalent: ltree's text I/O is byte-preserving (parse_ltree
# and deparse_ltree copy label bytes with memcpy, and ltree_eq is a memcmp over
# those same bytes), so two ltree values are equal iff their text renderings are
# — see contrib/ltree/ltree_io.c and ltree_op.c. PostgreSQL publishes no explicit
# guarantee of this; it is a property of the implementation, which cannot change
# without breaking ltree's on-disk format and every existing ltree index.
#
# `sort_path` needs no cast: it is already a text column.
#
# See also the module docstring ("Trigger DDL and search_path") and
# RestoreUnderRestrictedSearchPathTests in utilities/tests/test_ltree.py, which
# enforces this by creating the generated DDL with the extension's schema off the
# search_path.
_AFTER_TRIGGER_PATH_ONLY = '''
CREATE TRIGGER "{table}_ltree_cascade_path"
AFTER UPDATE OF parent_id ON "{table}"
FOR EACH ROW WHEN (OLD.path IS DISTINCT FROM NEW.path)
FOR EACH ROW WHEN (OLD.path::text IS DISTINCT FROM NEW.path::text)
EXECUTE FUNCTION "{table}_ltree_cascade_path_fn"();
'''
@ -231,13 +288,57 @@ _AFTER_TRIGGER_PATH_AND_SORT = '''
CREATE TRIGGER "{table}_ltree_cascade_path"
AFTER UPDATE OF parent_id, "{name_col}" ON "{table}"
FOR EACH ROW WHEN (
OLD.path IS DISTINCT FROM NEW.path
OLD.path::text IS DISTINCT FROM NEW.path::text
OR OLD.sort_path IS DISTINCT FROM NEW.sort_path
)
EXECUTE FUNCTION "{table}_ltree_cascade_path_fn"();
'''
def ltree_trigger_sql(table, name_column=None):
"""
Return the DDL statements which install ltree path-maintenance triggers on `table`.
Two functions and two triggers, in dependency order. If `name_column` is given, the
table is expected to carry a `sort_path` column and gets the variants which maintain
it alongside `path`.
The triggers are dropped before being created, so re-running this SQL converges
instead of failing: a plain `CREATE TRIGGER` raises 42710 when the trigger already
exists, which a re-run, a partially-applied migration, or a later migration
reinstalling a corrected definition (#23130) would all hit. The functions already use
CREATE OR REPLACE. This mirrors `utilities.migration.InstallDenormalizationTrigger`.
`InstallLtreeTriggers` executes exactly this SQL, so tests can assert against the
statements migrations really run rather than a copy which can drift.
"""
if name_column:
function_sql = (
_COMPUTE_PATH_AND_SORT_FN.format(table=table, name_col=name_column),
_CASCADE_PATH_AND_SORT_FN.format(table=table),
)
trigger_sql = (
_BEFORE_TRIGGER_PATH_AND_SORT.format(table=table, name_col=name_column),
_AFTER_TRIGGER_PATH_AND_SORT.format(table=table, name_col=name_column),
)
else:
function_sql = (
_COMPUTE_PATH_ONLY_FN.format(table=table),
_CASCADE_PATH_ONLY_FN.format(table=table),
)
trigger_sql = (
_BEFORE_TRIGGER_PATH_ONLY.format(table=table),
_AFTER_TRIGGER_PATH_ONLY.format(table=table),
)
return (
*function_sql,
f'DROP TRIGGER IF EXISTS "{table}_ltree_cascade_path" ON "{table}";',
f'DROP TRIGGER IF EXISTS "{table}_ltree_compute_path" ON "{table}";',
*trigger_sql,
)
class InstallLtreeTriggers(migrations.operations.base.Operation):
"""
Install per-table ltree path-maintenance triggers.
@ -252,6 +353,10 @@ class InstallLtreeTriggers(migrations.operations.base.Operation):
ancestor names. This implements MPTT's `order_insertion_by=(name,)`
semantics: insert, reparent, and rename all honor the current value of
`name_column`, with renames cascaded into descendants' sort_paths.
Applying this operation is idempotent (see `ltree_trigger_sql`), so it can be
re-run to reinstall a corrected trigger definition on a table which already has
one.
"""
reversible = True
@ -263,24 +368,8 @@ class InstallLtreeTriggers(migrations.operations.base.Operation):
pass
def database_forwards(self, app_label, schema_editor, from_state, to_state):
if self.name_column:
schema_editor.execute(_COMPUTE_PATH_AND_SORT_FN.format(
table=self.table_name, name_col=self.name_column,
))
schema_editor.execute(_CASCADE_PATH_AND_SORT_FN.format(
table=self.table_name,
))
schema_editor.execute(_BEFORE_TRIGGER_PATH_AND_SORT.format(
table=self.table_name, name_col=self.name_column,
))
schema_editor.execute(_AFTER_TRIGGER_PATH_AND_SORT.format(
table=self.table_name, name_col=self.name_column,
))
else:
schema_editor.execute(_COMPUTE_PATH_ONLY_FN.format(table=self.table_name))
schema_editor.execute(_CASCADE_PATH_ONLY_FN.format(table=self.table_name))
schema_editor.execute(_BEFORE_TRIGGER_PATH_ONLY.format(table=self.table_name))
schema_editor.execute(_AFTER_TRIGGER_PATH_ONLY.format(table=self.table_name))
for sql in ltree_trigger_sql(self.table_name, self.name_column):
schema_editor.execute(sql)
def database_backwards(self, app_label, schema_editor, from_state, to_state):
t = self.table_name
@ -291,3 +380,23 @@ class InstallLtreeTriggers(migrations.operations.base.Operation):
def describe(self):
return f"Install ltree path triggers on {self.table_name}"
class ReinstallLtreeTriggers(InstallLtreeTriggers):
"""
Reinstall a table's ltree path-maintenance triggers, replacing an earlier definition.
Identical to `InstallLtreeTriggers` going forwards, but a no-op in reverse. The
parent operation's reverse drops both triggers and both functions, which is right
when reversing the migration that first installed them and wrong when reversing one
that merely corrected them: it would leave the table with no path maintenance at all
a state no release ever shipped and every subsequent INSERT failing on `path`'s
NOT NULL constraint. The triggers this replaces are recreated by reversing back to
the migration which installed them, so there is nothing for this operation to undo.
"""
def database_backwards(self, app_label, schema_editor, from_state, to_state):
pass
def describe(self):
return f"Reinstall ltree path triggers on {self.table_name}"

View File

@ -0,0 +1,154 @@
from django.apps import apps
from django.core.management.base import BaseCommand, CommandError
from django.db import connection, transaction
from netbox.models.ltree import LtreeModel
from netbox.plugins import PluginConfig
from utilities.mptt_to_ltree import (
count_stale_rows_sql,
populate_paths_sql,
unreachable_rows_sql,
)
class Command(BaseCommand):
help = (
"Recompute the trigger-maintained path (and sort_path) columns of hierarchical models "
"from their parent relationships"
)
# How many offending ids a refusal names. Enough to start from, short enough to read.
REPORTED_IDS = 10
def add_arguments(self, parser):
parser.add_argument(
'model', nargs='*',
help="Limit the rebuild to these models, as app_label.ModelName (default: all)",
)
parser.add_argument(
'--check', action='store_true',
help="Report which models need rebuilding, without modifying anything",
)
def get_models(self, names):
"""
Return the concrete core hierarchical models to operate on: those named, in the
order given, or every one of them ordered by table name.
Plugin models are excluded, including when named explicitly: the SQL which rebuilds
`sort_path` reads the name column by name, while `InstallLtreeTriggers` lets a plugin
maintain it from any column, so rebuilding one is not something this command can do
correctly. A plugin in that position needs its own repair path.
"""
def concrete_subclasses(base):
for subclass in base.__subclasses__():
if subclass._meta.abstract:
yield from concrete_subclasses(subclass)
elif not isinstance(apps.get_app_config(subclass._meta.app_label), PluginConfig):
yield subclass
candidates = {
model._meta.label_lower: model for model in concrete_subclasses(LtreeModel)
}
if not names:
return sorted(candidates.values(), key=lambda model: model._meta.db_table)
models = []
for name in names:
model = candidates.get(name.lower())
if model is None:
raise CommandError(f"{name} is not a core hierarchical (ltree-backed) model")
models.append(model)
return models
def check_reachable(self, cursor, model):
"""
Raise unless every row is reachable from a root by following `parent_id`.
The rebuild walks down from `parent_id IS NULL`, so a row no root can reach is one
it silently leaves alone. Reporting success in that case would be the same failure
this command exists to repair: an operation which appears to have worked while the
data is still wrong. Refuse instead, and leave correcting the parent relationships
to the operator, since only they can say what the intended hierarchy was.
Takes the caller's cursor so a refusal rolls back with the transaction the rebuild
would have run in. That does not make the pair atomic with respect to other
writers: under READ COMMITTED every statement takes a fresh snapshot, so a
reparent committed between the check and the rebuild is still missed. Pause writes
for the duration, as the documentation says to.
"""
cursor.execute(unreachable_rows_sql(model._meta.db_table, self.REPORTED_IDS))
unreachable, ids = cursor.fetchone()
if unreachable:
listed = ', '.join(str(pk) for pk in ids)
if unreachable > len(ids):
listed += ', ...'
raise CommandError(
f'{model._meta.label_lower}: {unreachable} row(s) cannot be reached from a '
f'root by following parent_id, so a rebuild would skip them: {listed}. '
f'Correct the parent relationships, then re-run.'
)
def report_stale(self, model):
"""
Report whether a model's stored paths disagree with its parent relationships.
Read-only, and takes no locks, so it can be run outside a maintenance window or
against a replica. It answers which models need rebuilding, not how many rows are
damaged: see `count_stale_rows_sql()` for why the counts understate a deep tree.
"""
with connection.cursor() as cursor:
cursor.execute(
count_stale_rows_sql(model._meta.db_table, sort_path=model._has_sort_path())
)
stale_paths, stale_sort_paths = cursor.fetchone()
if not (stale_paths or stale_sort_paths):
self.stdout.write(f'{model._meta.label_lower}: OK')
return False
damage = []
if stale_paths:
damage.append(f'{stale_paths} path')
if stale_sort_paths:
damage.append(f'{stale_sort_paths} sort_path')
self.stdout.write(self.style.WARNING(
f"{model._meta.label_lower}: {', '.join(damage)} row(s) out of date"
))
return True
def handle(self, *args, **options):
models = self.get_models(options['model'])
if options['check']:
stale = [model for model in models if self.report_stale(model)]
if stale:
names = ' '.join(model._meta.label_lower for model in stale)
self.stdout.write(f'\nNeeds rebuilding: {names}')
else:
self.stdout.write(self.style.SUCCESS('Nothing to rebuild.'))
return
# Each table is checked and rebuilt in its own transaction. Tables already done
# stay done if a later one fails or is refused: rebuilding one table cannot leave
# another inconsistent, and holding every table's row locks until the last one
# finished would turn several short blocking windows into one long one.
for model in models:
with transaction.atomic(), connection.cursor() as cursor:
# Announce the rebuild only once the check has passed, so a refusal does
# not print "rebuilding..." for a table left untouched.
self.check_reachable(cursor, model)
self.stdout.write(f'{model._meta.label_lower}: rebuilding... ', ending='')
self.stdout.flush()
# populate_paths_sql() is the same SQL which backfilled these columns
# during the ltree migrations. It relies on SET LOCAL, so it must run
# inside a transaction, and the UPDATE it emits locks every row in the
# table until it commits.
cursor.execute(
populate_paths_sql(model._meta.db_table, sort_path=model._has_sort_path())
)
self.stdout.write(self.style.SUCCESS('done'))
self.stdout.write(self.style.SUCCESS('Finished.'))

View File

@ -67,6 +67,16 @@ class InstallDenormalizationTrigger(migrations.operations.base.Operation):
newly created source row has no dependents yet) and it does not recurse: the dependent tables carry no
triggers of their own.
!!! warning "Watched columns must be of a type whose `=` lives in `pg_catalog`"
The generated WHEN clause compares each watched column with `IS DISTINCT FROM`, which expands to
that column type's `=` operator, resolved from `search_path` at CREATE TRIGGER time and with no
syntax available to schema-qualify it. Every current caller watches integer FK columns, whose `=` is
a built-in in `pg_catalog` and therefore always resolvable. Do NOT pass a column of an extension
type (`ltree`, `hstore`, PostGIS `geometry`, ...): its operators live in the extension's schema, so
the resulting trigger would fail to restore from a `pg_dump`, which replays DDL with an empty
`search_path` and because `psql` does not stop on error by default, the restore would appear to
succeed with the trigger silently missing. See `utilities/ltree.py` and #23130.
Example: refresh a CircuitTermination's cached region/sitegroup when its Site's region or group changes::
InstallDenormalizationTrigger(

View File

@ -35,7 +35,9 @@ ancestor `name` values. Keep the two modules in sync if either changes.
__all__ = (
'assert_paths_populated_sql',
'count_stale_rows_sql',
'populate_paths_sql',
'unreachable_rows_sql',
)
# Width to which each PK is zero-padded when used as an ltree label. Must match
@ -117,6 +119,80 @@ UPDATE "{table}" SET path = t.path FROM t WHERE "{table}".id = t.id;
""" + _RESTORE_SEARCH_PATH
def count_stale_rows_sql(table, sort_path=False):
"""
Return SQL counting the rows in `table` whose `path` disagrees with the hierarchy, and
(when `sort_path` is set) the rows whose `sort_path` does.
A reparent leaves `path` wrong, a rename leaves `sort_path` wrong, and while the
cascade trigger is missing either can happen without the other, so both are counted
separately. Roots are checked against what `populate_paths_sql()` would give them (a
path of their own padded id, and a sort_path of their own name) and every other row
against its parent: a root has no parent to compare with, but it can still be wrong.
This answers "does this table need rebuilding", not "how many rows are damaged". Where
an object has moved, the objects below it agree with their own parent and are not
counted, though they are equally stale. Treat any non-zero result as the whole table
needing a rebuild, and do not use it to decide which rows to touch.
"""
root_path = (
f'SELECT id FROM "{table}"'
f" WHERE parent_id IS NULL"
f" AND path <> lpad(id::text, {_PATH_LABEL_WIDTH}, '0')::ltree"
)
child_path = (
f'SELECT c.id FROM "{table}" c JOIN "{table}" p ON c.parent_id = p.id'
f" WHERE c.path <> p.path || lpad(c.id::text, {_PATH_LABEL_WIDTH}, '0')::ltree"
)
if sort_path:
root_sort_path = (
f'SELECT id FROM "{table}" WHERE parent_id IS NULL AND sort_path <> name'
)
child_sort_path = (
f'SELECT c.id FROM "{table}" c JOIN "{table}" p ON c.parent_id = p.id'
f' WHERE c.sort_path <> p.sort_path || chr(9) || c.name'
)
stale_sort_path = f'SELECT count(*) FROM ({root_sort_path} UNION ALL {child_sort_path}) s'
else:
stale_sort_path = 'SELECT 0'
return f"""
SELECT
(SELECT count(*) FROM ({root_path} UNION ALL {child_path}) p) AS stale_paths,
({stale_sort_path}) AS stale_sort_paths;
"""
def unreachable_rows_sql(table, limit):
"""
Return SQL reporting the rows in `table` which no root can reach by following
`parent_id`: how many there are, and the first `limit` of their ids.
`populate_paths_sql()` seeds from `parent_id IS NULL` and walks downward, so it
rewrites only the rows reachable that way. Anything else it leaves untouched, which
makes an unreachable row an unrepaired one. Three shapes cause it: a cycle, a row
whose `parent_id` is its own id, and a `parent_id` referencing a row which does not
exist.
Callers which repair a populated table (rather than backfilling a fresh column, where
`assert_paths_populated_sql()` catches the same condition via the NULLs left behind)
should run this first and refuse if the count is non-zero: the parent relationships
have to be corrected before any path rebuild can produce a correct answer. The ids are
returned so that refusal can name rows to start from, rather than leaving the operator
to search the table for them.
"""
return f"""
WITH RECURSIVE reachable(id) AS (
SELECT id FROM "{table}" WHERE parent_id IS NULL
UNION ALL
SELECT c.id FROM "{table}" c JOIN reachable r ON c.parent_id = r.id
)
SELECT count(*), (array_agg(t.id ORDER BY t.id))[:{limit}]
FROM "{table}" t
WHERE NOT EXISTS (SELECT 1 FROM reachable r WHERE r.id = t.id);
"""
def assert_paths_populated_sql(table):
"""
Return SQL that raises if any row in `table` still has a NULL `path` after

View File

@ -28,6 +28,7 @@ from ipam.filtersets import ASNFilterSet
from ipam.models import ASN, RIR
from netbox.filtersets import BaseFilterSet
from utilities.filters import (
MultiValueArrayFilter,
MultiValueCharFilter,
MultiValueDateFilter,
MultiValueDateTimeFilter,
@ -209,6 +210,9 @@ class BaseFilterSetTestCase(TestCase):
multiplechoicefield = django_filters.MultipleChoiceFilter(
field_name='choicefield'
)
multivaluearrayfield = MultiValueArrayFilter(
field_name='charfield' # We're pretending this is an array field
)
multivaluecharfield = MultiValueCharFilter(
field_name='charfield'
)
@ -326,6 +330,13 @@ class BaseFilterSetTestCase(TestCase):
self.assertEqual(self.filters['modelmultiplechoicefield__n'].lookup_expr, 'exact')
self.assertEqual(self.filters['modelmultiplechoicefield__n'].exclude, True)
def test_multi_value_array_filter(self):
self.assertIsInstance(self.filters['multivaluearrayfield'], MultiValueArrayFilter)
self.assertEqual(self.filters['multivaluearrayfield'].lookup_expr, 'contains')
self.assertEqual(self.filters['multivaluearrayfield'].exclude, False)
self.assertEqual(self.filters['multivaluearrayfield__n'].lookup_expr, 'contains')
self.assertEqual(self.filters['multivaluearrayfield__n'].exclude, True)
def test_multi_value_char_filter(self):
self.assertIsInstance(self.filters['multivaluecharfield'], MultiValueCharFilter)
self.assertEqual(self.filters['multivaluecharfield'].lookup_expr, 'exact')

View File

@ -1,8 +1,13 @@
from uuid import UUID
from django import forms
from django.contrib.postgres.forms import SimpleArrayField
from django.core.exceptions import ValidationError
from django.core.validators import RegexValidator
from django.test import TestCase
from utilities.jsonschema import JSONSchemaProperty
from utilities.validators import MultipleOfValidator
class JSONSchemaPropertyTestCase(TestCase):
@ -44,3 +49,342 @@ class JSONSchemaPropertyTestCase(TestCase):
self.assertIsInstance(field, SimpleArrayField)
self.assertIsInstance(field.base_field, forms.CharField)
self.assertEqual(field.clean('ge-0/0/0,ge-0/0/1'), ['ge-0/0/0', 'ge-0/0/1'])
def test_zero_minimum_is_applied_to_form_field(self):
prop = JSONSchemaProperty(type='number', title='Offset', minimum=0)
field = prop.to_form_field('offset')
self.assertEqual(field.min_value, 0)
with self.assertRaises(ValidationError):
field.clean(-5)
self.assertEqual(field.clean(0), 0)
def test_zero_maximum_is_applied_to_form_field(self):
prop = JSONSchemaProperty(type='number', title='Offset', maximum=0)
field = prop.to_form_field('offset')
self.assertEqual(field.max_value, 0)
with self.assertRaises(ValidationError):
field.clean(5)
self.assertEqual(field.clean(0), 0)
def test_zero_bounds_are_applied_to_integer_form_field(self):
prop = JSONSchemaProperty(type='integer', title='Slots', minimum=0, maximum=0)
field = prop.to_form_field('slots')
self.assertEqual(field.min_value, 0)
self.assertEqual(field.max_value, 0)
with self.assertRaises(ValidationError):
field.clean(-1)
with self.assertRaises(ValidationError):
field.clean(1)
self.assertEqual(field.clean(0), 0)
def test_nonzero_bounds_are_applied_to_form_field(self):
prop = JSONSchemaProperty(type='number', title='Offset', minimum=1, maximum=10)
field = prop.to_form_field('offset')
self.assertEqual(field.min_value, 1)
self.assertEqual(field.max_value, 10)
with self.assertRaises(ValidationError):
field.clean(0)
with self.assertRaises(ValidationError):
field.clean(11)
def test_omitted_bounds_are_not_applied_to_form_field(self):
prop = JSONSchemaProperty(type='number', title='Offset')
field = prop.to_form_field('offset')
self.assertIsNone(field.min_value)
self.assertIsNone(field.max_value)
self.assertEqual(field.clean(-100), -100)
def test_numeric_enum_with_zero_bound_builds_choice_field(self):
"""A numeric property carrying both an enum and a zero bound resolves to a ChoiceField.
ChoiceField accepts neither min_value nor max_value, so the numeric bounds must not be
passed through when an enum is present.
"""
prop = JSONSchemaProperty(type='integer', title='Slots', enum=[0, 1, 2], minimum=0)
field = prop.to_form_field('slots')
self.assertIsInstance(field, forms.ChoiceField)
self.assertEqual(list(field.choices), [(None, ''), (0, 0), (1, 1), (2, 2)])
def test_numeric_enum_with_nonzero_bound_builds_choice_field(self):
prop = JSONSchemaProperty(type='integer', title='Slots', enum=[1, 2], minimum=1, maximum=2)
field = prop.to_form_field('slots')
self.assertIsInstance(field, forms.ChoiceField)
self.assertEqual(list(field.choices), [(None, ''), (1, 1), (2, 2)])
def test_numeric_enum_with_multiple_of_builds_choice_field(self):
"""An enum suppresses the numeric bounds but retains the multipleOf validator.
Field.__init__() accepts validators, so a MultipleOfValidator remains applicable to a
ChoiceField even though min_value and max_value are not.
"""
prop = JSONSchemaProperty(type='integer', title='Slots', enum=[2, 4], multipleOf=2)
field = prop.to_form_field('slots')
self.assertIsInstance(field, forms.ChoiceField)
self.assertEqual(list(field.choices), [(None, ''), (2, 2), (4, 4)])
self.assertEqual(len(field.validators), 1)
self.assertIsInstance(field.validators[0], MultipleOfValidator)
def test_string_enum_with_min_length_builds_choice_field(self):
"""A string property carrying both an enum and a length bound resolves to a ChoiceField.
ChoiceField accepts neither min_length nor max_length, so the length bounds must not be
passed through when an enum is present.
"""
prop = JSONSchemaProperty(type='string', title='Media', enum=['a', 'bb'], minLength=1)
field = prop.to_form_field('media')
self.assertIsInstance(field, forms.ChoiceField)
self.assertEqual(list(field.choices), [(None, ''), ('a', 'a'), ('bb', 'bb')])
def test_string_enum_with_max_length_builds_choice_field(self):
prop = JSONSchemaProperty(type='string', title='Media', enum=['a', 'bb'], maxLength=2)
field = prop.to_form_field('media')
self.assertIsInstance(field, forms.ChoiceField)
self.assertEqual(list(field.choices), [(None, ''), ('a', 'a'), ('bb', 'bb')])
def test_string_enum_retains_pattern_validator(self):
"""Dropping the length bounds for an enum must not also drop the pattern validator.
Field.__init__() accepts validators, so a RegexValidator remains applicable to a
ChoiceField even though min_length and max_length are not.
"""
prop = JSONSchemaProperty(
type='string', title='Media', enum=['a', 'bb'], minLength=1, pattern='^[ab]+$'
)
field = prop.to_form_field('media')
self.assertIsInstance(field, forms.ChoiceField)
self.assertEqual(len(field.validators), 1)
self.assertIsInstance(field.validators[0], RegexValidator)
self.assertEqual(field.validators[0].regex.pattern, '^[ab]+$')
def test_string_bounds_are_applied_without_an_enum(self):
prop = JSONSchemaProperty(type='string', title='Media', minLength=1, maxLength=4)
field = prop.to_form_field('media')
self.assertIsInstance(field, forms.CharField)
self.assertEqual(field.min_length, 1)
self.assertEqual(field.max_length, 4)
with self.assertRaises(ValidationError):
field.clean('toolong')
def test_string_format_with_length_bound_builds_format_field(self):
"""A string format resolves to a field class which accepts no length bounds.
DateField, TimeField and DateTimeField do not subclass CharField, so passing minLength
or maxLength to one raises TypeError.
"""
for string_format, expected_class in (
('date', forms.DateField),
('time', forms.TimeField),
('datetime', forms.DateTimeField),
):
with self.subTest(format=string_format):
prop = JSONSchemaProperty(
type='string', title='Timestamp', format=string_format, minLength=10, maxLength=30
)
field = prop.to_form_field('timestamp')
self.assertIsInstance(field, expected_class)
def test_string_format_retains_pattern_validator(self):
prop = JSONSchemaProperty(type='string', title='Timestamp', format='date', pattern='^x$')
field = prop.to_form_field('timestamp')
self.assertIsInstance(field, forms.DateField)
self.assertEqual(len(field.validators), 1)
self.assertIsInstance(field.validators[0], RegexValidator)
def test_charfield_derived_format_retains_length_bounds(self):
"""EmailField and URLField clean to a string, so the length bounds apply to them."""
for string_format, expected_class, value in (
('email', forms.EmailField, 'user@example.com'),
('uri', forms.URLField, 'https://example.com/x'),
):
with self.subTest(format=string_format):
prop = JSONSchemaProperty(
type='string', title='Contact', format=string_format, minLength=5, maxLength=40
)
field = prop.to_form_field('contact')
self.assertIsInstance(field, expected_class)
self.assertEqual(field.min_length, 5)
self.assertEqual(field.max_length, 40)
self.assertEqual(field.clean(value), value)
def test_uuid_format_omits_length_bounds(self):
"""UUIDField subclasses CharField but cleans to a uuid.UUID, which has no length.
CharField.__init__() installs a MinLengthValidator and MaxLengthValidator for the bounds,
and those call len() on the cleaned value, so a UUID raises TypeError at clean time.
"""
value = '12345678-1234-5678-1234-567812345678'
prop = JSONSchemaProperty(type='string', title='Serial', format='uuid', minLength=5, maxLength=40)
field = prop.to_form_field('serial')
self.assertIsInstance(field, forms.UUIDField)
self.assertIsNone(field.min_length)
self.assertIsNone(field.max_length)
self.assertEqual(field.clean(value), UUID(value))
class JSONSchemaPropertyDescriptionSanitizationTestCase(TestCase):
"""
A property's description becomes the form field's help_text, which is rendered through the
`safe` filter in form_helpers/render_field.html. It is passed through render_markdown(), which
applies the HTML_ALLOWED_TAGS allowlist, matching the custom field path in
extras.models.customfields.CustomField.to_form_field().
Each test compares the entire help text, so a payload surviving anywhere in it fails the
assertion. Asserting only on the absence of a substring would not, because stripping an
element leaves its text behind as character data.
"""
def test_disallowed_element_is_stripped(self):
prop = JSONSchemaProperty(
type='integer',
title='Capacity (GB)',
description='Gross disk size <iframe src="https://example.com"></iframe>',
)
field = prop.to_form_field('capacity')
self.assertHTMLEqual(
'<div class="rendered-markdown"><p>Gross disk size</p></div>',
field.help_text,
)
def test_script_element_is_stripped(self):
prop = JSONSchemaProperty(
type='string',
description='Vendor code <script>alert(1)</script>',
)
field = prop.to_form_field('vendor_code')
self.assertHTMLEqual(
'<div class="rendered-markdown"><p>Vendor code</p></div>',
field.help_text,
)
def test_event_handler_attribute_is_stripped(self):
"""An allowed tag carrying a disallowed attribute keeps the tag but loses the attribute."""
prop = JSONSchemaProperty(
type='string',
description='<b onmouseover="alert(1)">Vendor code</b>',
)
field = prop.to_form_field('vendor_code')
self.assertHTMLEqual(
'<div class="rendered-markdown"><p><b>Vendor code</b></p></div>',
field.help_text,
)
def test_javascript_uri_is_stripped(self):
prop = JSONSchemaProperty(
type='string',
description='<a href="javascript:alert(1)">Vendor code</a>',
)
field = prop.to_form_field('vendor_code')
self.assertHTMLEqual(
'<div class="rendered-markdown">'
'<p><a rel="noopener noreferrer">Vendor code</a></p></div>',
field.help_text,
)
def test_disallowed_element_is_stripped_from_mixed_markup(self):
"""A disallowed element is dropped while its allowed siblings are kept."""
prop = JSONSchemaProperty(
type='string',
description='<b>Vendor</b> code <iframe src="https://example.com"></iframe>',
)
field = prop.to_form_field('vendor_code')
self.assertHTMLEqual(
'<div class="rendered-markdown"><p><b>Vendor</b> code</p></div>',
field.help_text,
)
def test_allowed_markup_is_preserved(self):
"""
render_markdown() applies the HTML_ALLOWED_TAGS allowlist, so markup inside it survives.
This is the behavior that keeps schema descriptions consistent with custom field
descriptions.
"""
prop = JSONSchemaProperty(
type='integer',
description='Gross disk size in <code>GB</code>',
)
field = prop.to_form_field('capacity')
self.assertHTMLEqual(
'<div class="rendered-markdown"><p>Gross disk size in <code>GB</code></p></div>',
field.help_text,
)
def test_markdown_is_rendered(self):
"""Descriptions are interpreted as Markdown, matching the custom field path."""
prop = JSONSchemaProperty(
type='integer',
description='Gross disk size in **GB**',
)
field = prop.to_form_field('capacity')
self.assertHTMLEqual(
'<div class="rendered-markdown">'
'<p>Gross disk size in <strong>GB</strong></p></div>',
field.help_text,
)
def test_description_text_is_retained(self):
"""Sanitization must not discard the author's actual help text."""
prop = JSONSchemaProperty(
type='string',
description='Gross disk size in gigabytes',
)
field = prop.to_form_field('capacity')
self.assertHTMLEqual(
'<div class="rendered-markdown"><p>Gross disk size in gigabytes</p></div>',
field.help_text,
)
def test_absent_description_yields_no_help_text(self):
"""A property without a description must not gain help text from the sanitizer."""
prop = JSONSchemaProperty(type='string', title='Vendor Code')
field = prop.to_form_field('vendor_code')
self.assertFalse(field.help_text)

View File

@ -1,11 +1,15 @@
"""Tests for the ltree-based hierarchical model infrastructure."""
from django.apps import apps
from django.contrib.contenttypes.models import ContentType
from django.db import connection
from django.test import TestCase
from django.test import SimpleTestCase, TestCase, TransactionTestCase
from core.models import ObjectChange
from dcim.models import Region, Site
from netbox.models.ltree import LtreeModel
from netbox.plugins import PluginConfig
from tenancy.models import Contact, ContactGroup
from utilities.ltree import ReinstallLtreeTriggers, ltree_trigger_sql
from utilities.mptt_to_ltree import populate_paths_sql
@ -992,3 +996,240 @@ class RestrictedSearchPathBackfillTests(TestCase):
self.assertEqual(len(plain_rows), 2)
self.assertEqual([r[0] for r in plain_rows], [_path(1), _path(1, 2)])
class RestoreUnderRestrictedSearchPathTests(TestCase):
"""
The generated trigger DDL must create with the ltree extension's schema off the
search_path, because that is how pg_dump replays it: dumps begin with
`set_config('search_path', '', false)` and schema-qualify every name they can.
`IS DISTINCT FROM` cannot be schema-qualified it expands to the operand type's
`=` operator, resolved at CREATE TRIGGER time so comparing two ltree values in a
WHEN clause produced a cascade trigger which silently failed to restore, leaving
descendant paths to go stale on the next rename or reparent (#23130). Comparing
`path::text` resolves `pg_catalog.text =` instead, which is always available.
The trigger functions are created with the extension's schema on the path: they
legitimately declare `parent_path ltree`, and pg_dump schema-qualifies those
declarations, so only the CREATE TRIGGER statements are under test here.
"""
def _install_with_extension_off_path(self, schema, table, name_column):
with connection.cursor() as cursor:
cursor.execute(f'CREATE SCHEMA {schema}')
columns = 'id bigint PRIMARY KEY, parent_id bigint, path ltree, name text'
if name_column:
columns += ', sort_path text'
cursor.execute(f'SET LOCAL search_path = {schema}, public')
cursor.execute(f'CREATE TABLE {schema}.{table} ({columns})')
statements = ltree_trigger_sql(table, name_column)
functions = [s for s in statements if 'CREATE OR REPLACE FUNCTION' in s]
triggers = [s for s in statements if 'CREATE TRIGGER' in s]
self.assertEqual(len(functions), 2)
self.assertEqual(len(triggers), 2)
# Pass an empty parameter list, as schema_editor.execute() does during a
# migration: the function bodies double their literal percent signs for
# .format(), and psycopg only collapses `%%` to `%` when parameters are
# given. Executing them without it fails to compile the plpgsql.
for statement in functions:
cursor.execute(statement, ())
# Drop the extension's schema, as a pg_dump restore does, and create only
# the triggers.
cursor.execute(f'SET LOCAL search_path = {schema}')
for statement in triggers:
cursor.execute(statement, ())
cursor.execute(
'SELECT tgname FROM pg_trigger t '
'JOIN pg_class c ON t.tgrelid = c.oid '
'JOIN pg_namespace n ON c.relnamespace = n.oid '
'WHERE n.nspname = %s AND NOT t.tgisinternal ORDER BY tgname',
[schema],
)
return [row[0] for row in cursor.fetchall()]
def test_path_and_sort_triggers_create_with_extension_off_search_path(self):
installed = self._install_with_extension_off_path('sp_sorted', 'sorted', 'name')
self.assertEqual(installed, ['sorted_ltree_cascade_path', 'sorted_ltree_compute_path'])
def test_path_only_triggers_create_with_extension_off_search_path(self):
installed = self._install_with_extension_off_path('sp_plain', 'plain', None)
self.assertEqual(installed, ['plain_ltree_cascade_path', 'plain_ltree_compute_path'])
class CascadeTriggerDefinitionTests(TestCase):
"""
Every core LtreeModel's cascade trigger must compare `path` as text.
This covers the templates as they are installed, catching a new hierarchical model
which ships without triggers at all. It cannot tell whether the corrective migrations
reached a given table: a test database is built by migrating forward, so the original
migrations install the current, already-corrected definitions. See
`CorrectiveMigrationTests` for the seeded states which do exercise that.
The expected tables are derived from the model layer and plugin models are excluded,
so installing a plugin with its own ltree model cannot fail this.
"""
@staticmethod
def _core_ltree_tables():
def concrete_subclasses(base):
for subclass in base.__subclasses__():
if subclass._meta.abstract:
yield from concrete_subclasses(subclass)
elif not isinstance(apps.get_app_config(subclass._meta.app_label), PluginConfig):
yield subclass
return {model._meta.db_table for model in concrete_subclasses(LtreeModel)}
def test_core_cascade_triggers_compare_path_as_text(self):
expected = self._core_ltree_tables()
self.assertTrue(expected, 'no core LtreeModel subclasses found')
with connection.cursor() as cursor:
cursor.execute(
'SELECT c.relname, pg_get_triggerdef(t.oid) FROM pg_trigger t '
'JOIN pg_class c ON t.tgrelid = c.oid '
'WHERE NOT t.tgisinternal AND t.tgname = c.relname || %s',
['_ltree_cascade_path'],
)
definitions = dict(cursor.fetchall())
self.assertSetEqual(
expected - set(definitions), set(),
msg='these core ltree tables have no cascade trigger installed',
)
# Assert on the cast rather than on PostgreSQL's exact rendering of the clause:
# the parenthesization pg_get_triggerdef() emits is an implementation detail.
for table in sorted(expected):
definition = definitions[table]
self.assertIn(
'::text IS DISTINCT FROM', definition,
msg=f'{table}: the cascade trigger compares ltree values directly, so it '
f'will not survive a pg_dump restore (see #23130)',
)
self.assertNotRegex(
definition, r'old\.path\s+IS DISTINCT FROM\s+new\.path',
msg=f'{table}: the cascade trigger compares path without a cast to text',
)
class LtreeTriggerSqlTests(SimpleTestCase):
"""The generated cascade DDL must not compare ltree values directly (#23130)."""
def test_cascade_when_clause_casts_path_to_text(self):
for name_column in ('name', None):
with self.subTest(name_column=name_column):
sql = '\n'.join(ltree_trigger_sql('probe', name_column))
self.assertIn('OLD.path::text IS DISTINCT FROM NEW.path::text', sql)
self.assertNotIn('OLD.path IS DISTINCT FROM NEW.path', sql)
def test_triggers_are_dropped_before_creation(self):
sql = ltree_trigger_sql('probe', 'name')
for trigger in ('probe_ltree_cascade_path', 'probe_ltree_compute_path'):
drop = f'DROP TRIGGER IF EXISTS "{trigger}" ON "probe";'
self.assertIn(drop, sql)
create = next(s for s in sql if f'CREATE TRIGGER "{trigger}"' in s)
self.assertLess(sql.index(drop), sql.index(create))
class CorrectiveMigrationTests(TransactionTestCase):
"""
`ReinstallLtreeTriggers` must repair both states a v4.7.0 database can be in.
A test database is built by migrating forward, so `0242_ltree_paths` installs its
triggers from the current templates and every table already carries the corrected
definition before `0251_fix_ltree_cascade_triggers` runs. Nothing asserted about the
end state of that database says whether the corrective migration did anything. Seed
each state a real database can be in instead:
- the definition v4.7.0 shipped, which an upgraded-in-place database still carries
- no cascade trigger, which is what a database restored from a v4.7.0 dump has
then apply the operation those migrations are built from and assert the repair.
Scope: this covers the operation, not the migrations which call it. The tables each
corrective migration names are hand-maintained lists, and a table omitted from one
would not fail here. Catching that needs the pre-migration state a forward-migrated
test database does not have, i.e. replaying `0250 -> 0251` against a seeded fixture.
TransactionTestCase, because the seeded DDL has to be committed for the operation's
own transaction to see it.
"""
TABLE = 'dcim_region'
TRIGGER = 'dcim_region_ltree_cascade_path'
def tearDown(self):
# Leave the trigger as the migrations would have it, for whatever runs next.
self.apply_corrective_operation()
def cascade_triggerdef(self):
with connection.cursor() as cursor:
cursor.execute(
'SELECT pg_get_triggerdef(oid) FROM pg_trigger '
'WHERE tgname = %s AND NOT tgisinternal',
[self.TRIGGER],
)
row = cursor.fetchone()
return row[0] if row else None
def drop_cascade_trigger(self):
"""Leave the table as a database restored from a v4.7.0 dump: no cascade trigger."""
with connection.cursor() as cursor:
cursor.execute(f'DROP TRIGGER IF EXISTS "{self.TRIGGER}" ON "{self.TABLE}"')
def install_v470_cascade_trigger(self):
"""
Install the definition v4.7.0 shipped: bare ltree comparisons, which a dump cannot
restore because the `ltree` operator is unresolvable under an empty search_path.
"""
self.drop_cascade_trigger()
with connection.cursor() as cursor:
cursor.execute(
f'CREATE TRIGGER "{self.TRIGGER}" '
f'AFTER UPDATE OF parent_id, "name" ON "{self.TABLE}" '
f'FOR EACH ROW WHEN ('
f' OLD.path IS DISTINCT FROM NEW.path'
f' OR OLD.sort_path IS DISTINCT FROM NEW.sort_path'
f') EXECUTE FUNCTION "{self.TABLE}_ltree_cascade_path_fn"()'
)
def apply_corrective_operation(self):
with connection.schema_editor() as schema_editor:
ReinstallLtreeTriggers(self.TABLE, name_column='name').database_forwards(
'dcim', schema_editor, None, None,
)
def test_replaces_the_definition_shipped_in_v470(self):
self.install_v470_cascade_trigger()
self.assertNotIn('::text', self.cascade_triggerdef())
self.apply_corrective_operation()
self.assertIn('::text IS DISTINCT FROM', self.cascade_triggerdef())
def test_reinstalls_a_cascade_trigger_lost_in_a_restore(self):
self.drop_cascade_trigger()
self.assertIsNone(self.cascade_triggerdef())
self.apply_corrective_operation()
self.assertIn('::text IS DISTINCT FROM', self.cascade_triggerdef())
def test_the_repaired_trigger_cascades_a_rename(self):
"""The reinstalled trigger has to work, not merely exist."""
self.drop_cascade_trigger()
self.apply_corrective_operation()
parent = Region.objects.create(name='Before', slug='before-cm')
child = Region.objects.create(name='Child', slug='child-cm', parent=parent)
parent.name = 'After'
parent.save()
child.refresh_from_db()
self.assertEqual(child.sort_path, f'After{chr(9)}Child')

View File

@ -2,8 +2,12 @@ from io import StringIO
from unittest.mock import MagicMock, patch
from django.core.management import call_command
from django.core.management.base import CommandError
from django.db import connection
from django.test import TestCase
from dcim.models import Region
from tenancy.models import TenantGroup
from utilities.management.commands.calculate_cached_counts import Command
@ -49,3 +53,232 @@ class CalculateCachedCountsTestCase(TestCase):
ChildModel._meta.get_field.assert_called_once_with('parent')
fk_field.related_query_name.assert_called_once_with()
self.assertEqual(dict(models), {ParentModel: {'child_count': 'children'}})
class RebuildLtreePathsTestCase(TestCase):
"""
The command must repair path/sort_path values the triggers did not maintain.
Corruption is injected by writing the path columns directly: the triggers fire on
parent_id and the name column, so a raw UPDATE of path bypasses them, reproducing a
database whose cascade trigger went missing across a restore.
"""
@classmethod
def setUpTestData(cls):
cls.parent = Region.objects.create(name='Alpha', slug='alpha-rlp')
cls.child = Region.objects.create(name='Beta', slug='beta-rlp', parent=cls.parent)
@staticmethod
def _set_parent_bypassing_triggers(pk, parent_pk):
"""
Repoint a row's parent_id without firing the ltree triggers.
The BEFORE trigger recomputes `path` and rejects a move which its own cycle guard
can see, so the ORM cannot produce these states directly. Suppressing the triggers
for the statement reproduces what #23130 leaves behind: a database whose parent_id
graph has drifted from the paths stored alongside it.
`ALTER TABLE ... DISABLE TRIGGER` needs only ownership of the table, which the
role running the tests has, where `session_replication_role` needs SUPERUSER or an
explicit grant. It does refuse while the transaction holds pending trigger events,
which the rows created in setUpTestData leave behind, so flush those first: the
events are the deferred foreign key checks, and firing them early is harmless.
`netbox/tests/test_search.py` does the same to reach its own schema states.
"""
with connection.cursor() as cursor:
cursor.execute('SET CONSTRAINTS ALL IMMEDIATE')
cursor.execute('ALTER TABLE dcim_region DISABLE TRIGGER USER')
try:
cursor.execute(
'UPDATE dcim_region SET parent_id = %s WHERE id = %s', [parent_pk, pk]
)
finally:
cursor.execute('ALTER TABLE dcim_region ENABLE TRIGGER USER')
def test_rebuilds_stale_path_and_sort_path(self):
Region.objects.filter(pk=self.child.pk).update(
path='9999999999999999999', sort_path='stale',
)
call_command('rebuild_ltree_paths', 'dcim.region', stdout=StringIO())
self.child.refresh_from_db()
self.assertEqual(
self.child.path,
f'{str(self.parent.pk).zfill(19)}.{str(self.child.pk).zfill(19)}',
)
self.assertEqual(self.child.sort_path, f'Alpha{chr(9)}Beta')
def test_rebuilds_a_stale_sort_path_alone(self):
# What a rename leaves behind: the renamed row's own sort_path is rewritten by the
# BEFORE trigger, its descendants' are not, and no path changes.
Region.objects.filter(pk=self.child.pk).update(sort_path='stale')
call_command('rebuild_ltree_paths', 'dcim.region', stdout=StringIO())
self.child.refresh_from_db()
self.assertEqual(self.child.sort_path, f'Alpha{chr(9)}Beta')
def test_rebuilds_every_core_hierarchical_model_by_default(self):
out = StringIO()
call_command('rebuild_ltree_paths', stdout=out)
output = out.getvalue()
for label in ('dcim.region', 'dcim.inventoryitem', 'dcim.inventoryitemtemplate',
'tenancy.tenantgroup', 'wireless.wirelesslangroup'):
self.assertIn(label, output)
self.assertIn('Finished.', output)
def test_check_reports_a_model_needing_a_rebuild(self):
Region.objects.filter(pk=self.child.pk).update(sort_path='stale')
out = StringIO()
call_command('rebuild_ltree_paths', 'dcim.region', '--check', stdout=out)
output = out.getvalue()
self.assertIn('sort_path', output)
self.assertIn('Needs rebuilding: dcim.region', output)
def test_check_reports_a_healthy_model_as_ok(self):
out = StringIO()
call_command('rebuild_ltree_paths', 'dcim.region', '--check', stdout=out)
self.assertIn('dcim.region: OK', out.getvalue())
self.assertIn('Nothing to rebuild.', out.getvalue())
def test_check_modifies_nothing(self):
Region.objects.filter(pk=self.child.pk).update(sort_path='stale')
call_command('rebuild_ltree_paths', 'dcim.region', '--check', stdout=StringIO())
self.child.refresh_from_db()
self.assertEqual(self.child.sort_path, 'stale')
def test_check_reports_a_stale_path_where_sort_path_is_correct(self):
# A reparent leaves path wrong on its own, so the two counts are separate.
Region.objects.filter(pk=self.child.pk).update(path='9999999999999999999')
out = StringIO()
call_command('rebuild_ltree_paths', 'dcim.region', '--check', stdout=out)
self.assertIn('1 path', out.getvalue())
def test_check_reports_a_stale_root(self):
"""
A root has no parent to be compared against, so a check which only joins children
to parents never examines it and reports a corrupt root as clean.
"""
root = Region.objects.create(name='Solo', slug='solo-rlp')
Region.objects.filter(pk=root.pk).update(
path='9999999999999999999', sort_path='WRONG',
)
out = StringIO()
call_command('rebuild_ltree_paths', 'dcim.region', '--check', stdout=out)
output = out.getvalue()
self.assertIn('1 path, 1 sort_path', output)
self.assertNotIn('dcim.region: OK', output)
def test_rebuilds_a_stale_root(self):
root = Region.objects.create(name='Solo', slug='solo-rlp')
Region.objects.filter(pk=root.pk).update(
path='9999999999999999999', sort_path='WRONG',
)
call_command('rebuild_ltree_paths', 'dcim.region', stdout=StringIO())
root.refresh_from_db()
self.assertEqual(root.path, str(root.pk).zfill(19))
self.assertEqual(root.sort_path, 'Solo')
def test_check_reports_a_stale_root_whose_child_agrees_with_it(self):
"""
The child of a corrupt root can be consistent with that root, so a parent-only
comparison sees nothing wrong anywhere in the subtree.
"""
root = Region.objects.create(name='Solo', slug='solo-rlp')
child = Region.objects.create(name='Sub', slug='sub-rlp', parent=root)
Region.objects.filter(pk=root.pk).update(path='9999999999999999999')
Region.objects.filter(pk=child.pk).update(
path=f'9999999999999999999.{str(child.pk).zfill(19)}',
)
out = StringIO()
call_command('rebuild_ltree_paths', 'dcim.region', '--check', stdout=out)
self.assertIn('1 path', out.getvalue())
def test_rejects_a_model_which_is_not_hierarchical(self):
with self.assertRaises(CommandError):
call_command('rebuild_ltree_paths', 'dcim.site')
def test_refuses_a_table_containing_a_cycle(self):
"""
A rebuild walks down from the roots, so rows in a cycle are never reached and keep
whatever paths they have. Refuse rather than report success, and name the rows to
start from: "correct the parent relationships" is not actionable without them.
"""
self._set_parent_bypassing_triggers(self.parent.pk, self.child.pk)
with self.assertRaises(CommandError) as ctx:
call_command('rebuild_ltree_paths', 'dcim.region')
message = str(ctx.exception)
self.assertIn(str(self.parent.pk), message)
self.assertIn(str(self.child.pk), message)
def test_refuses_a_table_containing_a_self_parented_row(self):
self._set_parent_bypassing_triggers(self.child.pk, self.child.pk)
with self.assertRaises(CommandError):
call_command('rebuild_ltree_paths', 'dcim.region')
def test_refuses_a_table_whose_parent_id_references_a_missing_row(self):
"""
Not a cycle, but equally unreachable, so a cycle-specific check would miss it.
Disabling the triggers leaves the foreign key enforced, so drop it for this row as
well. Such a row does occur in practice: `pg_restore --disable-triggers` and
logical replication both load rows without enforcing it.
"""
with connection.cursor() as cursor:
cursor.execute('SET CONSTRAINTS ALL IMMEDIATE')
cursor.execute(
"SELECT conname FROM pg_constraint "
"WHERE conrelid = 'dcim_region'::regclass AND contype = 'f' "
"AND conkey = ARRAY[(SELECT attnum FROM pg_attribute "
"WHERE attrelid = 'dcim_region'::regclass AND attname = 'parent_id')]"
)
constraint = cursor.fetchone()[0]
cursor.execute(f'ALTER TABLE dcim_region DROP CONSTRAINT "{constraint}"')
self._set_parent_bypassing_triggers(self.child.pk, self.parent.pk + 10000)
with self.assertRaises(CommandError):
call_command('rebuild_ltree_paths', 'dcim.region')
def test_refusing_a_table_leaves_that_table_untouched(self):
"""
A refusal rolls back the transaction it was raised in, so the refused table keeps
the paths it had. Tables already rebuilt stay rebuilt: each is its own transaction,
which is what keeps one table's row locks from being held while the rest run.
"""
group = TenantGroup.objects.create(name='Unrelated', slug='unrelated-rlp')
TenantGroup.objects.filter(pk=group.pk).update(sort_path='stale')
Region.objects.filter(pk=self.child.pk).update(sort_path='also-stale')
# dcim.region is named second and is the table which fails the check.
self._set_parent_bypassing_triggers(self.parent.pk, self.child.pk)
with self.assertRaises(CommandError):
call_command('rebuild_ltree_paths', 'tenancy.tenantgroup', 'dcim.region')
# The refused table is untouched: no partial rebuild, nothing to undo by hand.
self.child.refresh_from_db()
self.assertEqual(self.child.sort_path, 'also-stale')
# The table which passed its own check was rebuilt and committed.
group.refresh_from_db()
self.assertEqual(group.sort_path, 'Unrelated')

View File

@ -66,7 +66,7 @@ class VirtualMachinePlacementPanel(panels.ObjectAttributesPanel):
title = _('Placement')
site = attrs.RelatedObjectAttr('site', linkify=True, grouped_by='group')
cluster = attrs.RelatedObjectAttr('cluster', linkify=True)
cluster = attrs.RelatedObjectAttr('cluster', linkify=True, grouped_by='group')
cluster_type = attrs.RelatedObjectAttr('cluster.type', linkify=True)
device = attrs.RelatedObjectAttr('device', linkify=True)

View File

@ -16,7 +16,7 @@ __all__ = (
class L2VPNSerializer(PrimaryModelSerializer):
type = ChoiceField(choices=L2VPNTypeChoices, required=False)
type = ChoiceField(choices=L2VPNTypeChoices, required=True)
import_targets = SerializedPKRelatedField(
queryset=RouteTarget.objects.all(),
serializer=RouteTargetSerializer,

View File

@ -601,6 +601,26 @@ class L2VPNTestCase(APIViewTestCases.APIViewTestCase):
self.assertHttpStatus(response, status.HTTP_200_OK)
self.assertEqual(response_data['count'], 1)
def test_type_required(self):
"""
type must be reported as required by OPTIONS, and a POST omitting it must
be rejected with a normal "required" validation error rather than a
model-level "cannot be blank" error.
"""
self.add_permissions('vpn.add_l2vpn')
response = self.client.options(self._get_list_url(), **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
self.assertTrue(response.data['actions']['POST']['type']['required'])
data = {
'name': 'L2VPN Missing Type',
'slug': 'l2vpn-missing-type',
}
response = self.client.post(self._get_list_url(), data, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_400_BAD_REQUEST)
self.assertEqual(response.data['type'][0].code, 'required')
class L2VPNTerminationTestCase(APIViewTestCases.APIViewTestCase):
model = L2VPNTermination

Some files were not shown because too many files have changed in this diff Show More