RomanNum3ral
/
netbox
mirror of https://github.com/netbox-community/netbox.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'main' into feature

This commit is contained in:
Jeremy Stretch 2026-06-16 14:09:27 -04:00
parent 288c47d445 0c26f973ff
commit bf954f08d6
172 changed files with 52394 additions and 12170 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.github/ISSUE_TEMPLATE/01-feature_request.yaml vendored
View File

@ -15,7 +15,7 @@ body:
attributes:
label: NetBox version
description: What version of NetBox are you currently running?
placeholder: v4.6.2
placeholder: v4.6.3
validations:
required: true
- type: dropdown

2
.github/ISSUE_TEMPLATE/02-bug_report.yaml vendored
View File

@ -27,7 +27,7 @@ body:
attributes:
label: NetBox Version
description: What version of NetBox are you currently running?
placeholder: v4.6.2
placeholder: v4.6.3
validations:
required: true
- type: dropdown

2
.github/ISSUE_TEMPLATE/03-performance.yaml vendored
View File

@ -8,7 +8,7 @@ body:
attributes:
label: NetBox Version
description: What version of NetBox are you currently running?
placeholder: v4.6.2
placeholder: v4.6.3
validations:
required: true
- type: dropdown

21
.github/workflows/no-blank-issue.yml vendored Normal file
View File

@ -0,0 +1,21 @@
name: Enforce issue templates
on:
issues:
types:
- opened
- reopened
permissions:
issues: write
jobs:
no-blank-issue:
name: No Blank Issue
runs-on: ubuntu-slim
steps:
- name: Close new issues without labels
uses: ldez/no-blank-issue@800e2d0c81c9e0ca7bdb58f3e7480a74602d91e0 # v1.2.0
with:
github-token: ${{ secrets.GITHUB_TOKEN }}

5
base_requirements.txt
View File

@ -141,6 +141,11 @@ psycopg[c,pool]
# https://github.com/yaml/pyyaml/blob/master/CHANGES
PyYAML
# redis-py
# https://github.com/redis/redis-py
# Default protocol changes to RESP3 in v8.0; see #22388
redis<8.0
# Requests
# https://github.com/psf/requests/blob/main/HISTORY.md
requests

8
contrib/generated_schema.json
View File

@ -604,6 +604,10 @@
"lc-pc",
"lc-upc",
"lc-apc",
"mu",
"mu-pc",
"mu-upc",
"mu-apc",
"lsh",
"lsh-pc",
"lsh-upc",
@ -672,6 +676,10 @@
"lc-pc",
"lc-upc",
"lc-apc",
"mu",
"mu-pc",
"mu-upc",
"mu-apc",
"lsh",
"lsh-pc",
"lsh-upc",

34467
contrib/openapi.json
View File

File diff suppressed because one or more lines are too long

5
docs/installation/2-redis.md
View File

@ -8,7 +8,10 @@
sudo apt install -y redis-server
```
Before continuing, verify that your installed version of Redis is at least v4.0:
Before continuing, verify that your installed version of Redis is at least v6.0:
!!! warning "Redis v5.x is deprecated"
Support for Redis versions older than 6.0 is deprecated and will be removed in NetBox v4.7.
```no-highlight
redis-server -v

8
docs/installation/upgrading.md
View File

@ -40,10 +40,10 @@ NetBox requires the following dependencies:
| NetBox Version | Python min | Python max | PostgreSQL min | Redis min | Documentation |
|:--------------:|:----------:|:----------:|:--------------:|:---------:|:-----------------------------------------------------------------------------------------:|
| 4.6 | 3.12 | 3.14 | 14 | 4.0 | [Link](https://github.com/netbox-community/netbox/blob/v4.6.0/docs/installation/index.md) |
| 4.5 | 3.12 | 3.14 | 14 | 4.0 | [Link](https://github.com/netbox-community/netbox/blob/v4.5.0/docs/installation/index.md) |
| 4.4 | 3.10 | 3.12 | 14 | 4.0 | [Link](https://github.com/netbox-community/netbox/blob/v4.4.0/docs/installation/index.md) |
| 4.3 | 3.10 | 3.12 | 14 | 4.0 | [Link](https://github.com/netbox-community/netbox/blob/v4.3.0/docs/installation/index.md) |
| 4.6 | 3.12 | 3.14 | 14 | 5.0 | [Link](https://github.com/netbox-community/netbox/blob/v4.6.0/docs/installation/index.md) |
| 4.5 | 3.12 | 3.14 | 14 | 5.0 | [Link](https://github.com/netbox-community/netbox/blob/v4.5.0/docs/installation/index.md) |
| 4.4 | 3.10 | 3.12 | 14 | 5.0 | [Link](https://github.com/netbox-community/netbox/blob/v4.4.0/docs/installation/index.md) |
| 4.3 | 3.10 | 3.12 | 14 | 5.0 | [Link](https://github.com/netbox-community/netbox/blob/v4.3.0/docs/installation/index.md) |
| 4.2 | 3.10 | 3.12 | 13 | 4.0 | [Link](https://github.com/netbox-community/netbox/blob/v4.2.0/docs/installation/index.md) |
| 4.1 | 3.10 | 3.12 | 12 | 4.0 | [Link](https://github.com/netbox-community/netbox/blob/v4.1.0/docs/installation/index.md) |
| 4.0 | 3.10 | 3.12 | 12 | 4.0 | [Link](https://github.com/netbox-community/netbox/blob/v4.0.0/docs/installation/index.md) |

6
docs/models/ipam/vlan.md
View File

@ -2,6 +2,12 @@
A Virtual LAN (VLAN) represents an isolated layer two domain, identified by a name and a numeric ID (1-4094) as defined in [IEEE 802.1Q](https://en.wikipedia.org/wiki/IEEE_802.1Q). VLANs are arranged into [VLAN groups](./vlangroup.md) to define scope and to enforce uniqueness.
## Bulk Creation
Multiple VLANs can be created at once by selecting the "Bulk Create" tab on the VLAN creation form. Enter the desired VLAN IDs and/or ID ranges as a comma-separated list (e.g. `100,200-210,4000-4010`). The string `{vid}` may be embedded in the name field as a placeholder for each VLAN's ID; for example, `VLAN-{vid}` yields `VLAN-100`, `VLAN-200`, and so on. All other attributes (status, role, tenant, etc.) are applied to every new VLAN.
The operation is atomic: if any VLAN fails validation (for example, a VLAN ID falling outside the assigned group's permitted ranges), no VLANs are created.
## Fields
### ID

39
docs/plugins/development/ui-components.md
View File

@ -135,29 +135,32 @@ panels.ObjectsTablePanel(
The following classes are available to represent object attributes within an ObjectAttributesPanel. Additionally, plugins can subclass `netbox.ui.attrs.ObjectAttribute` to create custom classes.
| Class | Description |
|------------------------------------------|--------------------------------------------------|
| `netbox.ui.attrs.AddressAttr` | A physical or mailing address. |
| `netbox.ui.attrs.BooleanAttr` | A boolean value |
| `netbox.ui.attrs.ChoiceAttr` | A selection from a set of choices |
| `netbox.ui.attrs.ColorAttr` | A color expressed in RGB |
| `netbox.ui.attrs.DateTimeAttr` | A date or datetime value |
| `netbox.ui.attrs.GenericForeignKeyAttr` | A related object via a generic foreign key |
| `netbox.ui.attrs.GPSCoordinatesAttr` | GPS coordinates (latitude and longitude) |
| `netbox.ui.attrs.ImageAttr` | An attached image (displays the image) |
| `netbox.ui.attrs.NestedObjectAttr` | A related nested object (includes ancestors) |
| `netbox.ui.attrs.NumericAttr` | An integer or float value |
| `netbox.ui.attrs.RelatedObjectAttr` | A related object |
| `netbox.ui.attrs.RelatedObjectListAttr` | A list of related objects |
| `netbox.ui.attrs.TemplatedAttr` | Renders an attribute using a custom template |
| `netbox.ui.attrs.TextAttr` | A string (text) value |
| `netbox.ui.attrs.TimezoneAttr` | A timezone with annotated offset |
| `netbox.ui.attrs.UtilizationAttr` | A numeric value expressed as a utilization graph |
| Class | Description |
|-----------------------------------------|-----------------------------------------------------|
| `netbox.ui.attrs.AddressAttr` | A physical or mailing address. |
| `netbox.ui.attrs.ArrayAttr` | An array of values, shown as a comma-separated list |
| `netbox.ui.attrs.BooleanAttr` | A boolean value |
| `netbox.ui.attrs.ChoiceAttr` | A selection from a set of choices |
| `netbox.ui.attrs.ColorAttr` | A color expressed in RGB |
| `netbox.ui.attrs.DateTimeAttr` | A date or datetime value |
| `netbox.ui.attrs.GenericForeignKeyAttr` | A related object via a generic foreign key |
| `netbox.ui.attrs.GPSCoordinatesAttr` | GPS coordinates (latitude and longitude) |
| `netbox.ui.attrs.ImageAttr` | An attached image (displays the image) |
| `netbox.ui.attrs.NestedObjectAttr` | A related nested object (includes ancestors) |
| `netbox.ui.attrs.NumericAttr` | An integer or float value |
| `netbox.ui.attrs.RelatedObjectAttr` | A related object |
| `netbox.ui.attrs.RelatedObjectListAttr` | A list of related objects |
| `netbox.ui.attrs.TemplatedAttr` | Renders an attribute using a custom template |
| `netbox.ui.attrs.TextAttr` | A string (text) value |
| `netbox.ui.attrs.TimezoneAttr` | A timezone with annotated offset |
| `netbox.ui.attrs.UtilizationAttr` | A numeric value expressed as a utilization graph |
::: netbox.ui.attrs.ObjectAttribute
::: netbox.ui.attrs.AddressAttr
::: netbox.ui.attrs.ArrayAttr
::: netbox.ui.attrs.BooleanAttr
::: netbox.ui.attrs.ChoiceAttr

46
docs/release-notes/version-4.6.md
View File

@ -1,5 +1,51 @@
# NetBox v4.6
## v4.6.3 (2026-06-16)
### Enhancements
* [#17598](https://github.com/netbox-community/netbox/issues/17598) - Add bulk creation support for VLANs
* [#21666](https://github.com/netbox-community/netbox/issues/21666) - Add MU connector type for fiber ports and cables
* [#22361](https://github.com/netbox-community/netbox/issues/22361) - Introduce an `ArrayAttr` UI panel attribute for rendering array field values
* [#22457](https://github.com/netbox-community/netbox/issues/22457) - Use `hmac.compare_digest()` for constant-time authentication of API tokens
### Performance Improvements
* [#21870](https://github.com/netbox-community/netbox/issues/21870) - Optimize prefix availability calculations
* [#22375](https://github.com/netbox-community/netbox/issues/22375) - Improve efficiency of filtering VLANs by interface
### Bug Fixes
* [#21338](https://github.com/netbox-community/netbox/issues/21338) - Include connected endpoint data in interface webhooks generated during cable creation
* [#21895](https://github.com/netbox-community/netbox/issues/21895) - Restore pagination controls for job log entries (previously limited to 50 rows)
* [#22210](https://github.com/netbox-community/netbox/issues/22210) - Respect saved filters when rendering IPAM child availability views in additional tabs
* [#22237](https://github.com/netbox-community/netbox/issues/22237) - Fix server error when opening the standalone "Add Table Configuration" page
* [#22245](https://github.com/netbox-community/netbox/issues/22245) - Include the `id` field in the OpenAPI request schemas for bulk PATCH/PUT endpoints
* [#22251](https://github.com/netbox-community/netbox/issues/22251) - Re-parent child module bays when a multi-bay module is moved to a new bay
* [#22273](https://github.com/netbox-community/netbox/issues/22273) - Fix migration failure when a service has several thousand ports defined
* [#22303](https://github.com/netbox-community/netbox/issues/22303) - Add the missing `fields` parameter to the OpenAPI schema
* [#22324](https://github.com/netbox-community/netbox/issues/22324) - Fix GraphQL filtering of custom field choice set extra choices
* [#22340](https://github.com/netbox-community/netbox/issues/22340) - Display a token's allowed IPs as comma-separated strings rather than `IPNetwork` objects
* [#22346](https://github.com/netbox-community/netbox/issues/22346) - Render SSO/SAML authentication failures as a login page message instead of an HTTP 500 error
* [#22357](https://github.com/netbox-community/netbox/issues/22357) - Remove the unused `local_context_data` field from `dcim.Module` (which no longer inherits from `ConfigContextModel`)
* [#22376](https://github.com/netbox-community/netbox/issues/22376) - Fix `AssertionError` in event rule script jobs when a device type has an image attached
* [#22388](https://github.com/netbox-community/netbox/issues/22388) - Pin redis-py to <8.0 to avoid a startup failure on older Redis releases
* [#22397](https://github.com/netbox-community/netbox/issues/22397) - Fix `AttributeError` exception when an unauthenticated user attempts to export devices
* [#22399](https://github.com/netbox-community/netbox/issues/22399) - Enforce object permissions on the related object when serving static media
* [#22427](https://github.com/netbox-community/netbox/issues/22427) - Validate `JSONFilter.path` to prevent ORM operator injection over JSONField contents in the GraphQL API
* [#22429](https://github.com/netbox-community/netbox/issues/22429) - Enforce `ObjectPermission` constraints on `grant_token` in the REST API
* [#22431](https://github.com/netbox-community/netbox/issues/22431) - Use a cryptographically secure random number generator when generating API tokens
* [#22444](https://github.com/netbox-community/netbox/issues/22444) - Fix `KeyError` exception on the power feed detail view when the locale is not English
* [#22448](https://github.com/netbox-community/netbox/issues/22448) - Ensure all object representations are escaped under `handle_protectederror()`
* [#22454](https://github.com/netbox-community/netbox/issues/22454) - Fix serialization of decimal custom field values to avoid spurious changelog entries
* [#22466](https://github.com/netbox-community/netbox/issues/22466) - Fix test failure against SSL-enabled PosgtreSQL
### Deprecations
* [#22392](https://github.com/netbox-community/netbox/issues/22392) - Deprecate support for Redis 5.x (to be removed in v4.7)
---
## v4.6.2 (2026-06-02)
### Enhancements

29
netbox/circuits/graphql/filters.py
View File

@ -1,4 +1,3 @@
from datetime import date
from typing import TYPE_CHECKING, Annotated
import strawberry
@ -62,9 +61,9 @@ class CircuitTerminationFilter(
upstream_speed: Annotated['IntegerLookup', strawberry.lazy('netbox.graphql.filter_lookups')] | None = (
strawberry_django.filter_field()
)
xconnect_id: StrFilterLookup[str] | None = strawberry_django.filter_field()
pp_info: StrFilterLookup[str] | None = strawberry_django.filter_field()
description: StrFilterLookup[str] | None = strawberry_django.filter_field()
xconnect_id: StrFilterLookup | None = strawberry_django.filter_field()
pp_info: StrFilterLookup | None = strawberry_django.filter_field()
description: StrFilterLookup | None = strawberry_django.filter_field()
# Cached relations
_provider_network: Annotated['ProviderNetworkFilter', strawberry.lazy('circuits.graphql.filters')] | None = (
@ -92,7 +91,7 @@ class CircuitFilter(
TenancyFilterMixin,
PrimaryModelFilter
):
cid: StrFilterLookup[str] | None = strawberry_django.filter_field()
cid: StrFilterLookup | None = strawberry_django.filter_field()
provider: Annotated['ProviderFilter', strawberry.lazy('circuits.graphql.filters')] | None = (
strawberry_django.filter_field()
)
@ -108,8 +107,8 @@ class CircuitFilter(
status: BaseFilterLookup[Annotated['CircuitStatusEnum', strawberry.lazy('circuits.graphql.enums')]] | None = (
strawberry_django.filter_field()
)
install_date: DateFilterLookup[date] | None = strawberry_django.filter_field()
termination_date: DateFilterLookup[date] | None = strawberry_django.filter_field()
install_date: DateFilterLookup | None = strawberry_django.filter_field()
termination_date: DateFilterLookup | None = strawberry_django.filter_field()
commit_rate: Annotated['IntegerLookup', strawberry.lazy('netbox.graphql.filter_lookups')] | None = (
strawberry_django.filter_field()
)
@ -145,8 +144,8 @@ class CircuitGroupAssignmentFilter(CustomFieldsFilterMixin, TagsFilterMixin, Cha
@strawberry_django.filter_type(models.Provider, lookups=True)
class ProviderFilter(ContactFilterMixin, PrimaryModelFilter):
name: StrFilterLookup[str] | None = strawberry_django.filter_field()
slug: StrFilterLookup[str] | None = strawberry_django.filter_field()
name: StrFilterLookup | None = strawberry_django.filter_field()
slug: StrFilterLookup | None = strawberry_django.filter_field()
asns: Annotated['ASNFilter', strawberry.lazy('ipam.graphql.filters')] | None = strawberry_django.filter_field()
circuits: Annotated['CircuitFilter', strawberry.lazy('circuits.graphql.filters')] | None = (
strawberry_django.filter_field()
@ -159,18 +158,18 @@ class ProviderAccountFilter(ContactFilterMixin, PrimaryModelFilter):
strawberry_django.filter_field()
)
provider_id: ID | None = strawberry_django.filter_field()
account: StrFilterLookup[str] | None = strawberry_django.filter_field()
name: StrFilterLookup[str] | None = strawberry_django.filter_field()
account: StrFilterLookup | None = strawberry_django.filter_field()
name: StrFilterLookup | None = strawberry_django.filter_field()
@strawberry_django.filter_type(models.ProviderNetwork, lookups=True)
class ProviderNetworkFilter(PrimaryModelFilter):
name: StrFilterLookup[str] | None = strawberry_django.filter_field()
name: StrFilterLookup | None = strawberry_django.filter_field()
provider: Annotated['ProviderFilter', strawberry.lazy('circuits.graphql.filters')] | None = (
strawberry_django.filter_field()
)
provider_id: ID | None = strawberry_django.filter_field()
service_id: StrFilterLookup[str] | None = strawberry_django.filter_field()
service_id: StrFilterLookup | None = strawberry_django.filter_field()
@strawberry_django.filter_type(models.VirtualCircuitType, lookups=True)
@ -180,7 +179,7 @@ class VirtualCircuitTypeFilter(CircuitTypeFilterMixin, OrganizationalModelFilter
@strawberry_django.filter_type(models.VirtualCircuit, lookups=True)
class VirtualCircuitFilter(TenancyFilterMixin, PrimaryModelFilter):
cid: StrFilterLookup[str] | None = strawberry_django.filter_field()
cid: StrFilterLookup | None = strawberry_django.filter_field()
provider_network: Annotated['ProviderNetworkFilter', strawberry.lazy('circuits.graphql.filters')] | None = (
strawberry_django.filter_field()
)
@ -218,4 +217,4 @@ class VirtualCircuitTerminationFilter(CustomFieldsFilterMixin, TagsFilterMixin,
strawberry_django.filter_field()
)
interface_id: ID | None = strawberry_django.filter_field()
description: StrFilterLookup[str] | None = strawberry_django.filter_field()
description: StrFilterLookup | None = strawberry_django.filter_field()

55
netbox/core/api/schema.py
View File

@ -14,7 +14,7 @@ from drf_spectacular.plumbing import (
get_doc,
)
from drf_spectacular.types import OpenApiTypes
from drf_spectacular.utils import Direction
from drf_spectacular.utils import Direction, OpenApiParameter
from netbox.api.fields import ChoiceField
from netbox.api.serializers import WritableNestedSerializer
@ -138,14 +138,30 @@ class NetBoxAutoSchema(AutoSchema):
return super().get_operation_id()
def get_request_serializer(self) -> typing.Any:
# bulk operations should specify a list
serializer = super().get_request_serializer()
# Bulk update/partial-update has a special request shape: a list of
# writable objects plus a required `id` field. The normal writable
# serializer omits `id` because it is read-only, so don't use the generic
# bulk handling for these actions.
action = getattr(self.view, 'action', None)
if action in ('bulk_update', 'bulk_partial_update'):
get_bulk_update_request_serializer = getattr(
self.view,
'get_bulk_update_request_serializer',
None,
)
if get_bulk_update_request_serializer is not None:
return get_bulk_update_request_serializer(
partial=(action == 'bulk_partial_update' or self.method == 'PATCH')
)
# Bulk creates/deletes should specify a list.
if self.is_bulk_action:
return type(serializer)(many=True)
# handle mapping for Writable serializers - adapted from dansheps original code
# for drf-yasg
# handle mapping for Writable serializers - adapted from dansheps original
# code for drf-yasg.
if serializer is not None and self.method in WRITABLE_ACTIONS:
writable_class = self.get_writable_class(serializer)
if writable_class is not None:
@ -258,6 +274,37 @@ class NetBoxAutoSchema(AutoSchema):
writable_class = self.writable_serializers[type(serializer)]
return writable_class
def get_override_parameters(self):
params = super().get_override_parameters()
# Expose the ?fields, ?omit, and ?brief query parameters supported by NetBoxModelViewSet
# for all non-bulk GET operations (both list and detail).
if not self.is_bulk_action and self.method == 'GET':
params = list(params) + [
OpenApiParameter(
name='fields',
location=OpenApiParameter.QUERY,
required=False,
type=OpenApiTypes.STR,
description='Comma-separated list of fields to include in the response. Example: `fields=id,name`.',
),
OpenApiParameter(
name='omit',
location=OpenApiParameter.QUERY,
required=False,
type=OpenApiTypes.STR,
description='Comma-separated list of fields to exclude from the response. '
'Example: `omit=description,tags`.',
),
OpenApiParameter(
name='brief',
location=OpenApiParameter.QUERY,
required=False,
type=OpenApiTypes.BOOL,
description='Return only brief fields for each object.',
),
]
return params
def get_filter_backends(self):
# bulk operations don't have filter params
if self.is_bulk_action:

2
netbox/core/apps.py
View File

@ -22,7 +22,7 @@ class CoreConfig(AppConfig):
def ready(self):
from core.api import schema # noqa: F401
from core.checks import check_duplicate_indexes, check_postgresql_version # noqa: F401
from core.checks import check_duplicate_indexes, check_postgresql_version, check_redis_version # noqa: F401
from netbox import context_managers # noqa: F401
from netbox.models.features import register_models

26
netbox/core/checks.py
View File

@ -1,4 +1,5 @@
from django.apps import apps
from django.core.cache import cache
from django.core.checks import Error, Tags, Warning, register
from django.db import connection
from django.db.models import Index, UniqueConstraint
@ -6,6 +7,7 @@ from django.db.models import Index, UniqueConstraint
__all__ = (
'check_duplicate_indexes',
'check_postgresql_version',
'check_redis_version',
)
@ -67,3 +69,27 @@ def check_postgresql_version(app_configs, **kwargs):
except Exception:
pass
return warnings
@register(Tags.caches)
def check_redis_version(app_configs, **kwargs):
"""
Warn if the Redis version is less than 6.0, as support for Redis older than 6.0
will be removed in NetBox v4.7.
"""
warnings = []
try:
client = cache.client.get_client()
redis_version = tuple(int(x) for x in client.info()['redis_version'].split('.'))
if redis_version < (6, 0):
warnings.append(
Warning(
f'Support for Redis {".".join(str(x) for x in redis_version)} is deprecated and will be '
f'removed in NetBox v4.7.',
hint='Please upgrade to Redis 6.0 or later.',
id='netbox.W002',
)
)
except Exception:
pass
return warnings

9
netbox/core/forms/model_forms.py
View File

@ -115,15 +115,6 @@ class ManagedFileForm(SyncedDataMixin, NetBoxModelForm):
return self.cleaned_data
def save(self, *args, **kwargs):
# If a file was uploaded, save it to disk
if self.cleaned_data['upload_file']:
self.instance.file_path = self.cleaned_data['upload_file'].name
with open(self.instance.full_path, 'wb+') as new_file:
new_file.write(self.cleaned_data['upload_file'].read())
return super().save(*args, **kwargs)
class ConfigFormMetaclass(forms.models.ModelFormMetaclass):

5
netbox/core/graphql/filter_mixins.py
View File

@ -1,5 +1,4 @@
from dataclasses import dataclass
from datetime import datetime
from typing import TYPE_CHECKING, Annotated
import strawberry
@ -20,5 +19,5 @@ class ChangeLoggingMixin:
changelog: Annotated['ObjectChangeFilter', strawberry.lazy('core.graphql.filters')] | None = (
strawberry_django.filter_field()
)
created: DatetimeFilterLookup[datetime] | None = strawberry_django.filter_field()
last_updated: DatetimeFilterLookup[datetime] | None = strawberry_django.filter_field()
created: DatetimeFilterLookup | None = strawberry_django.filter_field()
last_updated: DatetimeFilterLookup | None = strawberry_django.filter_field()

31
netbox/core/graphql/filters.py
View File

@ -1,4 +1,3 @@
from datetime import datetime
from typing import TYPE_CHECKING, Annotated
import strawberry
@ -26,33 +25,33 @@ __all__ = (
@strawberry_django.filter_type(models.DataFile, lookups=True)
class DataFileFilter(BaseModelFilter):
created: DatetimeFilterLookup[datetime] | None = strawberry_django.filter_field()
last_updated: DatetimeFilterLookup[datetime] | None = strawberry_django.filter_field()
created: DatetimeFilterLookup | None = strawberry_django.filter_field()
last_updated: DatetimeFilterLookup | None = strawberry_django.filter_field()
source: Annotated['DataSourceFilter', strawberry.lazy('core.graphql.filters')] | None = (
strawberry_django.filter_field()
)
source_id: ID | None = strawberry_django.filter_field()
path: StrFilterLookup[str] | None = strawberry_django.filter_field()
path: StrFilterLookup | None = strawberry_django.filter_field()
size: Annotated['IntegerLookup', strawberry.lazy('netbox.graphql.filter_lookups')] | None = (
strawberry_django.filter_field()
)
hash: StrFilterLookup[str] | None = strawberry_django.filter_field()
hash: StrFilterLookup | None = strawberry_django.filter_field()
@strawberry_django.filter_type(models.DataSource, lookups=True)
class DataSourceFilter(PrimaryModelFilter):
name: StrFilterLookup[str] | None = strawberry_django.filter_field()
type: StrFilterLookup[str] | None = strawberry_django.filter_field()
source_url: StrFilterLookup[str] | None = strawberry_django.filter_field()
name: StrFilterLookup | None = strawberry_django.filter_field()
type: StrFilterLookup | None = strawberry_django.filter_field()
source_url: StrFilterLookup | None = strawberry_django.filter_field()
status: (
BaseFilterLookup[Annotated['DataSourceStatusEnum', strawberry.lazy('core.graphql.enums')]] | None
) = strawberry_django.filter_field()
enabled: FilterLookup[bool] | None = strawberry_django.filter_field()
ignore_rules: StrFilterLookup[str] | None = strawberry_django.filter_field()
ignore_rules: StrFilterLookup | None = strawberry_django.filter_field()
parameters: Annotated['JSONFilter', strawberry.lazy('netbox.graphql.filter_lookups')] | None = (
strawberry_django.filter_field()
)
last_synced: DatetimeFilterLookup[datetime] | None = strawberry_django.filter_field()
last_synced: DatetimeFilterLookup | None = strawberry_django.filter_field()
datafiles: Annotated['DataFileFilter', strawberry.lazy('core.graphql.filters')] | None = (
strawberry_django.filter_field()
)
@ -60,10 +59,10 @@ class DataSourceFilter(PrimaryModelFilter):
@strawberry_django.filter_type(models.ObjectChange, lookups=True)
class ObjectChangeFilter(BaseModelFilter):
time: DatetimeFilterLookup[datetime] | None = strawberry_django.filter_field()
time: DatetimeFilterLookup | None = strawberry_django.filter_field()
user: Annotated['UserFilter', strawberry.lazy('users.graphql.filters')] | None = strawberry_django.filter_field()
user_name: StrFilterLookup[str] | None = strawberry_django.filter_field()
request_id: StrFilterLookup[str] | None = strawberry_django.filter_field()
user_name: StrFilterLookup | None = strawberry_django.filter_field()
request_id: StrFilterLookup | None = strawberry_django.filter_field()
action: (
BaseFilterLookup[Annotated['ObjectChangeActionEnum', strawberry.lazy('core.graphql.enums')]] | None
) = strawberry_django.filter_field()
@ -76,7 +75,7 @@ class ObjectChangeFilter(BaseModelFilter):
strawberry_django.filter_field()
)
related_object_id: ID | None = strawberry_django.filter_field()
object_repr: StrFilterLookup[str] | None = strawberry_django.filter_field()
object_repr: StrFilterLookup | None = strawberry_django.filter_field()
prechange_data: Annotated['JSONFilter', strawberry.lazy('netbox.graphql.filter_lookups')] | None = (
strawberry_django.filter_field()
)
@ -87,5 +86,5 @@ class ObjectChangeFilter(BaseModelFilter):
@strawberry_django.filter_type(DjangoContentType, lookups=True)
class ContentTypeFilter(BaseModelFilter):
app_label: StrFilterLookup[str] | None = strawberry_django.filter_field()
model: StrFilterLookup[str] | None = strawberry_django.filter_field()
app_label: StrFilterLookup | None = strawberry_django.filter_field()
model: StrFilterLookup | None = strawberry_django.filter_field()

49
netbox/core/tests/test_api.py
View File

@ -12,7 +12,7 @@ from rq.registry import FailedJobRegistry, StartedJobRegistry
from users.constants import TOKEN_PREFIX
from users.models import Token
from utilities.testing import APITestCase, APIViewTestCases, TestCase
from utilities.testing import APITestCase, APIViewTestCases, GraphQLQueryTest, TestCase
from utilities.testing.mixins import RQQueueTestMixin
from utilities.testing.utils import disable_logging
@ -39,12 +39,49 @@ class DataSourceTestCase(APIViewTestCases.APIViewTestCase):
@classmethod
def setUpTestData(cls):
data_sources = (
DataSource(name='Data Source 1', type='local', source_url='file:///var/tmp/source1/'),
DataSource(
name='Data Source 1', type='local', source_url='file:///var/tmp/source1/',
parameters={
'sync_date': '2024-01-01',
'sync_datetime': '2024-01-01T12:30:00+00:00',
'sync_time': '12:30:00',
},
),
DataSource(name='Data Source 2', type='local', source_url='file:///var/tmp/source2/'),
DataSource(name='Data Source 3', type='local', source_url='file:///var/tmp/source3/'),
)
DataSource.objects.bulk_create(data_sources)
cls.graphql_query_tests = (
GraphQLQueryTest(
name='parameters_json_date_lookup',
query=(
'{ data_source_list(filters: {parameters: '
'{path: "sync_date", lookup: {date_lookup: {exact: "2024-01-01"}}}}) '
'{ id } }'
),
assert_result=cls.assert_only_source_1,
),
GraphQLQueryTest(
name='parameters_json_datetime_lookup',
query=(
'{ data_source_list(filters: {parameters: '
'{path: "sync_datetime", lookup: {datetime_lookup: {exact: "2024-01-01T12:30:00+00:00"}}}}) '
'{ id } }'
),
assert_result=cls.assert_only_source_1,
),
GraphQLQueryTest(
name='parameters_json_time_lookup',
query=(
'{ data_source_list(filters: {parameters: '
'{path: "sync_time", lookup: {time_lookup: {exact: "12:30:00"}}}}) '
'{ id } }'
),
assert_result=cls.assert_only_source_1,
),
)
cls.create_data = [
{
'name': 'Data Source 4',
@ -63,6 +100,11 @@ class DataSourceTestCase(APIViewTestCases.APIViewTestCase):
},
]
def assert_only_source_1(self, data):
"""The JSON lookup returns exactly the source carrying the matching value."""
ids = sorted(result['id'] for result in data['data_source_list'])
self.assertEqual(ids, [str(DataSource.objects.get(name='Data Source 1').pk)])
class DataFileTestCase(
APIViewTestCases.GetObjectViewTestCase,
@ -279,9 +321,8 @@ class BackgroundTaskTestCase(RQQueueTestMixin, TestCase):
# Enqueue & run a job that will fail
queue = get_queue('default')
job = queue.enqueue(self.dummy_job_failing)
worker = get_worker('default')
with disable_logging():
worker.work(burst=True)
self.run_rq_jobs('default')
self.assertTrue(job.is_failed)
url = reverse('core-api:rqtask-requeue', args=[job.id])

76
netbox/core/tests/test_views.py
View File

@ -1,7 +1,7 @@
import json
import urllib.parse
import uuid
from datetime import datetime
from datetime import UTC, datetime
from django.contrib.contenttypes.models import ContentType
from django.urls import reverse
@ -152,6 +152,77 @@ class JobTestCase(
)
class JobLogViewTestCase(TestCase):
user_permissions = (
'core.view_job',
)
@classmethod
def setUpTestData(cls):
cls.job = Job.objects.create(
name='Test Job',
job_id=uuid.uuid4(),
)
cls.job.log_entries = [
{
'level': 'info',
'message': f'log line {i}',
'timestamp': datetime(2026, 1, 1, tzinfo=UTC),
}
for i in range(120)
]
cls.job.save()
def setUp(self):
super().setUp()
# UserConfig.set() mutates self.data in place, which can mutate DEFAULT_USER_PREFERENCES
# (the signal in users/signals.py initializes data with a shared reference). Assign a
# fresh literal instead. Pin per_page so page-boundary assertions don't depend on PAGINATE_COUNT.
self.user.config.data = {'pagination': {'per_page': 50}}
self.user.config.save()
def test_log_page_renders_table_inline(self):
"""The full page renders the first log page inside an HTMX container."""
url = reverse('core:job_log', kwargs={'pk': self.job.pk})
response = self.client.get(url)
self.assertHttpStatus(response, 200)
self.assertContains(response, 'htmx-container')
self.assertContains(response, 'log line 0')
self.assertContains(response, 'Showing 1-50 of 120')
def test_log_page_table_is_embedded(self):
"""The embedded table never pushes page/per_page into the browser URL."""
url = reverse('core:job_log', kwargs={'pk': self.job.pk})
response = self.client.get(url)
self.assertHttpStatus(response, 200)
self.assertNotContains(response, 'hx-push-url="true"')
def test_log_table_htmx_renders_partial(self):
"""An HTMX request returns the paginated table partial."""
url = reverse('core:job_log', kwargs={'pk': self.job.pk})
response = self.client.get(url, headers={'hx-request': 'true'})
self.assertHttpStatus(response, 200)
self.assertContains(response, 'log line 0')
self.assertContains(response, 'Showing 1-50 of 120')
self.assertContains(response, 'Per Page')
def test_log_table_htmx_page_navigation(self):
"""`?page=2` advances the embedded table to the second page."""
url = reverse('core:job_log', kwargs={'pk': self.job.pk})
response = self.client.get(f'{url}?page=2', headers={'hx-request': 'true'})
self.assertHttpStatus(response, 200)
self.assertContains(response, 'log line 50')
self.assertNotContains(response, 'log line 49')
def test_log_table_htmx_per_page(self):
"""`?per_page=100` widens the embedded table page size."""
url = reverse('core:job_log', kwargs={'pk': self.job.pk})
response = self.client.get(f'{url}?per_page=100', headers={'hx-request': 'true'})
self.assertHttpStatus(response, 200)
self.assertContains(response, 'log line 99')
self.assertNotContains(response, 'log line 100')
# TODO: Convert to StandardTestCases.Views
class ObjectChangeTestCase(TestCase):
user_permissions = (
@ -315,9 +386,8 @@ class BackgroundTaskTestCase(RQQueueTestMixin, TestCase):
# Enqueue & run a job that will fail
job = queue.enqueue(self.dummy_job_failing)
worker = get_worker('default')
with disable_logging():
worker.work(burst=True)
self.run_rq_jobs('default')
self.assertTrue(job.is_failed)
# Re-enqueue the failed job and check that its status has been reset

21
netbox/core/views.py
View File

@ -39,7 +39,6 @@ from netbox.plugins.utils import get_installed_plugins
from netbox.ui import layout
from netbox.ui.panels import (
CommentsPanel,
ContextTablePanel,
JSONPanel,
ObjectsTablePanel,
PluginContentPanel,
@ -269,7 +268,7 @@ class JobLogView(generic.ObjectView):
layout = layout.Layout(
layout.Row(
layout.Column(
ContextTablePanel('table', title=_('Log Entries')),
TemplatePanel('core/job/log_entries.html', title=_('Log Entries')),
PluginContentPanel('left_page'),
),
),
@ -280,13 +279,27 @@ class JobLogView(generic.ObjectView):
),
)
def get_extra_context(self, request, instance):
def get_table(self, request, instance):
table = JobLogEntryTable(instance.log_entries)
table.embedded = True
table.htmx_url = reverse('core:job_log', kwargs={'pk': instance.pk})
table.configure(request)
return table
def get_extra_context(self, request, instance):
return {
'table': table,
'table': self.get_table(request, instance),
}
def get(self, request, **kwargs):
if htmx_partial(request):
instance = self.get_object(**kwargs)
return render(request, 'htmx/table.html', {
'object': instance,
'table': self.get_table(request, instance),
})
return super().get(request, **kwargs)
@register_model_view(Job, 'delete')
class JobDeleteView(generic.ObjectDeleteView):

8
netbox/dcim/choices.py
View File

@ -1637,6 +1637,10 @@ class PortTypeChoices(ChoiceSet):
TYPE_LC_PC = 'lc-pc'
TYPE_LC_UPC = 'lc-upc'
TYPE_LC_APC = 'lc-apc'
TYPE_MU = 'mu'
TYPE_MU_PC = 'mu-pc'
TYPE_MU_UPC = 'mu-upc'
TYPE_MU_APC = 'mu-apc'
TYPE_MTRJ = 'mtrj'
TYPE_MPO = 'mpo'
TYPE_LSH = 'lsh'
@ -1700,6 +1704,10 @@ class PortTypeChoices(ChoiceSet):
(TYPE_LC_PC, 'LC/PC'),
(TYPE_LC_UPC, 'LC/UPC'),
(TYPE_LC_APC, 'LC/APC'),
(TYPE_MU, 'MU'),
(TYPE_MU_PC, 'MU/PC'),
(TYPE_MU_UPC, 'MU/UPC'),
(TYPE_MU_APC, 'MU/APC'),
(TYPE_LSH, 'LSH'),
(TYPE_LSH_PC, 'LSH/PC'),
(TYPE_LSH_UPC, 'LSH/UPC'),

3
netbox/dcim/forms/bulk_import.py
View File

@ -1583,7 +1583,8 @@ class CableImportForm(PrimaryModelImportForm):
:param side: 'a' or 'b'
"""
assert side in 'ab', f"Invalid side designation: {side}"
if side not in ('a', 'b'):
raise ValueError(_("Invalid side designation: {side}").format(side=side))
device = self.cleaned_data.get(f'side_{side}_device')
power_panel = self.cleaned_data.get(f'side_{side}_power_panel')

12
netbox/dcim/graphql/filter_mixins.py
View File

@ -66,9 +66,9 @@ class ComponentModelFilterMixin:
)
device: Annotated['DeviceFilter', strawberry.lazy('dcim.graphql.filters')] | None = strawberry_django.filter_field()
device_id: ID | None = strawberry_django.filter_field()
name: StrFilterLookup[str] | None = strawberry_django.filter_field()
label: StrFilterLookup[str] | None = strawberry_django.filter_field()
description: StrFilterLookup[str] | None = strawberry_django.filter_field()
name: StrFilterLookup | None = strawberry_django.filter_field()
label: StrFilterLookup | None = strawberry_django.filter_field()
description: StrFilterLookup | None = strawberry_django.filter_field()
@dataclass
@ -96,9 +96,9 @@ class ComponentTemplateFilterMixin:
strawberry_django.filter_field()
)
device_type_id: ID | None = strawberry_django.filter_field()
name: StrFilterLookup[str] | None = strawberry_django.filter_field()
label: StrFilterLookup[str] | None = strawberry_django.filter_field()
description: StrFilterLookup[str] | None = strawberry_django.filter_field()
name: StrFilterLookup | None = strawberry_django.filter_field()
label: StrFilterLookup | None = strawberry_django.filter_field()
description: StrFilterLookup | None = strawberry_django.filter_field()
@dataclass

82
netbox/dcim/graphql/filters.py
View File

@ -116,7 +116,7 @@ __all__ = (
@strawberry_django.filter_type(models.CableBundle, lookups=True)
class CableBundleFilter(PrimaryModelFilter):
name: StrFilterLookup[str] | None = strawberry_django.filter_field()
name: StrFilterLookup | None = strawberry_django.filter_field()
@strawberry_django.filter_type(models.Cable, lookups=True)
@ -127,7 +127,7 @@ class CableFilter(TenancyFilterMixin, PrimaryModelFilter):
status: BaseFilterLookup[Annotated['LinkStatusEnum', strawberry.lazy('dcim.graphql.enums')]] | None = (
strawberry_django.filter_field()
)
label: StrFilterLookup[str] | None = strawberry_django.filter_field()
label: StrFilterLookup | None = strawberry_django.filter_field()
color: BaseFilterLookup[Annotated['ColorEnum', strawberry.lazy('netbox.graphql.enums')]] | None = (
strawberry_django.filter_field()
)
@ -223,9 +223,9 @@ class DeviceFilter(
platform: Annotated['PlatformFilter', strawberry.lazy('dcim.graphql.filters')] | None = (
strawberry_django.filter_field()
)
name: StrFilterLookup[str] | None = strawberry_django.filter_field()
serial: StrFilterLookup[str] | None = strawberry_django.filter_field()
asset_tag: StrFilterLookup[str] | None = strawberry_django.filter_field()
name: StrFilterLookup | None = strawberry_django.filter_field()
serial: StrFilterLookup | None = strawberry_django.filter_field()
asset_tag: StrFilterLookup | None = strawberry_django.filter_field()
site: Annotated['SiteFilter', strawberry.lazy('dcim.graphql.filters')] | None = strawberry_django.filter_field()
site_id: ID | None = strawberry_django.filter_field()
location: Annotated['LocationFilter', strawberry.lazy('dcim.graphql.filters')] | None = (
@ -353,7 +353,7 @@ class InventoryItemTemplateFilter(ComponentTemplateFilterMixin, ChangeLoggedMode
strawberry_django.filter_field()
)
manufacturer_id: ID | None = strawberry_django.filter_field()
part_id: StrFilterLookup[str] | None = strawberry_django.filter_field()
part_id: StrFilterLookup | None = strawberry_django.filter_field()
@strawberry_django.filter_type(models.DeviceRole, lookups=True)
@ -370,13 +370,13 @@ class DeviceTypeFilter(ImageAttachmentFilterMixin, WeightFilterMixin, PrimaryMod
strawberry_django.filter_field()
)
manufacturer_id: ID | None = strawberry_django.filter_field()
model: StrFilterLookup[str] | None = strawberry_django.filter_field()
slug: StrFilterLookup[str] | None = strawberry_django.filter_field()
model: StrFilterLookup | None = strawberry_django.filter_field()
slug: StrFilterLookup | None = strawberry_django.filter_field()
default_platform: Annotated['PlatformFilter', strawberry.lazy('dcim.graphql.filters')] | None = (
strawberry_django.filter_field()
)
default_platform_id: ID | None = strawberry_django.filter_field()
part_number: StrFilterLookup[str] | None = strawberry_django.filter_field()
part_number: StrFilterLookup | None = strawberry_django.filter_field()
instances: Annotated['DeviceFilter', strawberry.lazy('dcim.graphql.filters')] | None = (
strawberry_django.filter_field()
)
@ -493,7 +493,7 @@ class PortTemplateMappingFilter(BaseModelFilter):
@strawberry_django.filter_type(models.MACAddress, lookups=True)
class MACAddressFilter(PrimaryModelFilter):
mac_address: StrFilterLookup[str] | None = strawberry_django.filter_field()
mac_address: StrFilterLookup | None = strawberry_django.filter_field()
assigned_object_type: Annotated['ContentTypeFilter', strawberry.lazy('core.graphql.filters')] | None = (
strawberry_django.filter_field()
)
@ -539,7 +539,7 @@ class InterfaceFilter(
duplex: BaseFilterLookup[Annotated['InterfaceDuplexEnum', strawberry.lazy('dcim.graphql.enums')]] | None = (
strawberry_django.filter_field()
)
wwn: StrFilterLookup[str] | None = strawberry_django.filter_field()
wwn: StrFilterLookup | None = strawberry_django.filter_field()
parent: Annotated['InterfaceFilter', strawberry.lazy('dcim.graphql.filters')] | None = (
strawberry_django.filter_field()
)
@ -659,9 +659,9 @@ class InventoryItemFilter(ComponentModelFilterMixin, NetBoxModelFilter):
strawberry_django.filter_field()
)
manufacturer_id: ID | None = strawberry_django.filter_field()
part_id: StrFilterLookup[str] | None = strawberry_django.filter_field()
serial: StrFilterLookup[str] | None = strawberry_django.filter_field()
asset_tag: StrFilterLookup[str] | None = strawberry_django.filter_field()
part_id: StrFilterLookup | None = strawberry_django.filter_field()
serial: StrFilterLookup | None = strawberry_django.filter_field()
asset_tag: StrFilterLookup | None = strawberry_django.filter_field()
discovered: FilterLookup[bool] | None = strawberry_django.filter_field()
@ -679,7 +679,7 @@ class LocationFilter(ContactFilterMixin, ImageAttachmentFilterMixin, TenancyFilt
status: BaseFilterLookup[Annotated['LocationStatusEnum', strawberry.lazy('dcim.graphql.enums')]] | None = (
strawberry_django.filter_field()
)
facility: StrFilterLookup[str] | None = strawberry_django.filter_field()
facility: StrFilterLookup | None = strawberry_django.filter_field()
prefixes: Annotated['PrefixFilter', strawberry.lazy('ipam.graphql.filters')] | None = (
strawberry_django.filter_field()
)
@ -708,8 +708,8 @@ class ModuleFilter(ConfigContextFilterMixin, PrimaryModelFilter):
status: BaseFilterLookup[Annotated['ModuleStatusEnum', strawberry.lazy('dcim.graphql.enums')]] | None = (
strawberry_django.filter_field()
)
serial: StrFilterLookup[str] | None = strawberry_django.filter_field()
asset_tag: StrFilterLookup[str] | None = strawberry_django.filter_field()
serial: StrFilterLookup | None = strawberry_django.filter_field()
asset_tag: StrFilterLookup | None = strawberry_django.filter_field()
consoleports: Annotated['ConsolePortFilter', strawberry.lazy('dcim.graphql.filters')] | None = (
strawberry_django.filter_field(name='console_ports')
)
@ -748,19 +748,19 @@ class ModuleBayFilter(ModularComponentFilterMixin, NetBoxModelFilter):
strawberry_django.filter_field()
)
parent_id: ID | None = strawberry_django.filter_field()
position: StrFilterLookup[str] | None = strawberry_django.filter_field()
position: StrFilterLookup | None = strawberry_django.filter_field()
enabled: FilterLookup[bool] | None = strawberry_django.filter_field()
@strawberry_django.filter_type(models.ModuleBayTemplate, lookups=True)
class ModuleBayTemplateFilter(ModularComponentTemplateFilterMixin, ChangeLoggedModelFilter):
position: StrFilterLookup[str] | None = strawberry_django.filter_field()
position: StrFilterLookup | None = strawberry_django.filter_field()
enabled: FilterLookup[bool] | None = strawberry_django.filter_field()
@strawberry_django.filter_type(models.ModuleTypeProfile, lookups=True)
class ModuleTypeProfileFilter(PrimaryModelFilter):
name: StrFilterLookup[str] | None = strawberry_django.filter_field()
name: StrFilterLookup | None = strawberry_django.filter_field()
@strawberry_django.filter_type(models.ModuleType, lookups=True)
@ -773,8 +773,8 @@ class ModuleTypeFilter(ImageAttachmentFilterMixin, WeightFilterMixin, PrimaryMod
strawberry_django.filter_field()
)
profile_id: ID | None = strawberry_django.filter_field()
model: StrFilterLookup[str] | None = strawberry_django.filter_field()
part_number: StrFilterLookup[str] | None = strawberry_django.filter_field()
model: StrFilterLookup | None = strawberry_django.filter_field()
part_number: StrFilterLookup | None = strawberry_django.filter_field()
instances: Annotated['ModuleFilter', strawberry.lazy('dcim.graphql.filters')] | None = (
strawberry_django.filter_field()
)
@ -831,7 +831,7 @@ class PowerFeedFilter(CabledObjectModelFilterMixin, TenancyFilterMixin, PrimaryM
power_panel_id: ID | None = strawberry_django.filter_field()
rack: Annotated['RackFilter', strawberry.lazy('dcim.graphql.filters')] | None = strawberry_django.filter_field()
rack_id: ID | None = strawberry_django.filter_field()
name: StrFilterLookup[str] | None = strawberry_django.filter_field()
name: StrFilterLookup | None = strawberry_django.filter_field()
status: BaseFilterLookup[Annotated['PowerFeedStatusEnum', strawberry.lazy('dcim.graphql.enums')]] | None = (
strawberry_django.filter_field()
)
@ -902,7 +902,7 @@ class PowerPanelFilter(ContactFilterMixin, ImageAttachmentFilterMixin, PrimaryMo
location_id: Annotated['TreeNodeFilter', strawberry.lazy('netbox.graphql.filter_lookups')] | None = (
strawberry_django.filter_field()
)
name: StrFilterLookup[str] | None = strawberry_django.filter_field()
name: StrFilterLookup | None = strawberry_django.filter_field()
@strawberry_django.filter_type(models.PowerPort, lookups=True)
@ -940,8 +940,8 @@ class RackTypeFilter(ImageAttachmentFilterMixin, RackFilterMixin, WeightFilterMi
strawberry_django.filter_field()
)
manufacturer_id: ID | None = strawberry_django.filter_field()
model: StrFilterLookup[str] | None = strawberry_django.filter_field()
slug: StrFilterLookup[str] | None = strawberry_django.filter_field()
model: StrFilterLookup | None = strawberry_django.filter_field()
slug: StrFilterLookup | None = strawberry_django.filter_field()
racks: Annotated['RackFilter', strawberry.lazy('dcim.graphql.filters')] | None = strawberry_django.filter_field()
rack_count: ComparisonFilterLookup[int] | None = strawberry_django.filter_field()
@ -962,8 +962,8 @@ class RackFilter(
strawberry_django.filter_field()
)
rack_type_id: ID | None = strawberry_django.filter_field()
name: StrFilterLookup[str] | None = strawberry_django.filter_field()
facility_id: StrFilterLookup[str] | None = strawberry_django.filter_field()
name: StrFilterLookup | None = strawberry_django.filter_field()
facility_id: StrFilterLookup | None = strawberry_django.filter_field()
site: Annotated['SiteFilter', strawberry.lazy('dcim.graphql.filters')] | None = strawberry_django.filter_field()
site_id: ID | None = strawberry_django.filter_field()
location: Annotated['LocationFilter', strawberry.lazy('dcim.graphql.filters')] | None = (
@ -981,8 +981,8 @@ class RackFilter(
)
role: Annotated['RackRoleFilter', strawberry.lazy('dcim.graphql.filters')] | None = strawberry_django.filter_field()
role_id: ID | None = strawberry_django.filter_field()
serial: StrFilterLookup[str] | None = strawberry_django.filter_field()
asset_tag: StrFilterLookup[str] | None = strawberry_django.filter_field()
serial: StrFilterLookup | None = strawberry_django.filter_field()
asset_tag: StrFilterLookup | None = strawberry_django.filter_field()
airflow: BaseFilterLookup[Annotated['RackAirflowEnum', strawberry.lazy('dcim.graphql.enums')]] | None = (
strawberry_django.filter_field()
)
@ -1006,7 +1006,7 @@ class RackReservationFilter(TenancyFilterMixin, PrimaryModelFilter):
unit_count: ComparisonFilterLookup[int] | None = strawberry_django.filter_field()
user: Annotated['UserFilter', strawberry.lazy('users.graphql.filters')] | None = strawberry_django.filter_field()
user_id: ID | None = strawberry_django.filter_field()
description: StrFilterLookup[str] | None = strawberry_django.filter_field()
description: StrFilterLookup | None = strawberry_django.filter_field()
status: BaseFilterLookup[Annotated['RackReservationStatusEnum', strawberry.lazy('dcim.graphql.enums')]] | None = (
strawberry_django.filter_field()
)
@ -1057,8 +1057,8 @@ class RegionFilter(ContactFilterMixin, NestedGroupModelFilter):
@strawberry_django.filter_type(models.Site, lookups=True)
class SiteFilter(ContactFilterMixin, ImageAttachmentFilterMixin, TenancyFilterMixin, PrimaryModelFilter):
name: StrFilterLookup[str] | None = strawberry_django.filter_field()
slug: StrFilterLookup[str] | None = strawberry_django.filter_field()
name: StrFilterLookup | None = strawberry_django.filter_field()
slug: StrFilterLookup | None = strawberry_django.filter_field()
status: BaseFilterLookup[Annotated['SiteStatusEnum', strawberry.lazy('dcim.graphql.enums')]] | None = (
strawberry_django.filter_field()
)
@ -1072,11 +1072,11 @@ class SiteFilter(ContactFilterMixin, ImageAttachmentFilterMixin, TenancyFilterMi
group_id: Annotated['TreeNodeFilter', strawberry.lazy('netbox.graphql.filter_lookups')] | None = (
strawberry_django.filter_field()
)
facility: StrFilterLookup[str] | None = strawberry_django.filter_field()
facility: StrFilterLookup | None = strawberry_django.filter_field()
asns: Annotated['ASNFilter', strawberry.lazy('ipam.graphql.filters')] | None = strawberry_django.filter_field()
time_zone: StrFilterLookup[str] | None = strawberry_django.filter_field()
physical_address: StrFilterLookup[str] | None = strawberry_django.filter_field()
shipping_address: StrFilterLookup[str] | None = strawberry_django.filter_field()
time_zone: StrFilterLookup | None = strawberry_django.filter_field()
physical_address: StrFilterLookup | None = strawberry_django.filter_field()
shipping_address: StrFilterLookup | None = strawberry_django.filter_field()
latitude: Annotated['FloatLookup', strawberry.lazy('netbox.graphql.filter_lookups')] | None = (
strawberry_django.filter_field()
)
@ -1105,8 +1105,8 @@ class SiteGroupFilter(ContactFilterMixin, NestedGroupModelFilter):
class VirtualChassisFilter(PrimaryModelFilter):
master: Annotated['DeviceFilter', strawberry.lazy('dcim.graphql.filters')] | None = strawberry_django.filter_field()
master_id: ID | None = strawberry_django.filter_field()
name: StrFilterLookup[str] | None = strawberry_django.filter_field()
domain: StrFilterLookup[str] | None = strawberry_django.filter_field()
name: StrFilterLookup | None = strawberry_django.filter_field()
domain: StrFilterLookup | None = strawberry_django.filter_field()
members: (
Annotated['DeviceFilter', strawberry.lazy('dcim.graphql.filters')] | None
) = strawberry_django.filter_field()
@ -1117,7 +1117,7 @@ class VirtualChassisFilter(PrimaryModelFilter):
class VirtualDeviceContextFilter(TenancyFilterMixin, PrimaryModelFilter):
device: Annotated['DeviceFilter', strawberry.lazy('dcim.graphql.filters')] | None = strawberry_django.filter_field()
device_id: ID | None = strawberry_django.filter_field()
name: StrFilterLookup[str] | None = strawberry_django.filter_field()
name: StrFilterLookup | None = strawberry_django.filter_field()
status: (
BaseFilterLookup[Annotated['VirtualDeviceContextStatusEnum', strawberry.lazy('dcim.graphql.enums')]] | None
) = (
@ -1134,7 +1134,7 @@ class VirtualDeviceContextFilter(TenancyFilterMixin, PrimaryModelFilter):
strawberry_django.filter_field()
)
primary_ip6_id: ID | None = strawberry_django.filter_field()
comments: StrFilterLookup[str] | None = strawberry_django.filter_field()
comments: StrFilterLookup | None = strawberry_django.filter_field()
interfaces: (
Annotated['InterfaceFilter', strawberry.lazy('dcim.graphql.filters')] | None
) = strawberry_django.filter_field()

2
netbox/dcim/migrations/0239_device__config_context_data.py → netbox/dcim/migrations/0240_device__config_context_data.py
View File

@ -3,7 +3,7 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('dcim', '0238_ltree_paths'),
('dcim', '0239_denormalization_triggers'),
]
operations = [

5
netbox/dcim/models/cables.py
View File

@ -675,7 +675,10 @@ class CableTermination(ChangeLoggedModel):
Cache objects related to the termination (e.g. device, rack, site) directly on the object to
enable efficient filtering.
"""
assert self.termination is not None
if self.termination is None:
raise ValueError(
_("Invalid cable termination: the assigned termination object does not exist.")
)
# Device components
if getattr(self.termination, 'device', None):

23
netbox/dcim/models/device_components.py
View File

@ -384,12 +384,27 @@ class PathEndpoint(models.Model):
a stale in-memory `_path` relation while the database already points to
a different CablePath (or to no path at all).
If the cached relation points to a CablePath that has just been
deleted, refresh only the `_path` field from the database and retry.
This keeps the fix cheap and narrowly scoped to the denormalized FK.
Two stale cases are repaired by refreshing only the `_path` field
from the database:
1. The endpoint is linked (by cable or wireless link) but `_path` is
unset, because the instance was loaded before its path was traced
(e.g. while queued for event serialization during link creation).
2. The cached relation points to a CablePath row that has just been
deleted.
Repairing case 1 costs one query per access for a linked endpoint
whose path is genuinely absent in the database. That state is
transient outside of tracing failures, so no result caching is
attempted here.
"""
if self._path_id is None:
return None
has_link = self.cable_id is not None or getattr(self, 'wireless_link_id', None) is not None
if self.pk and has_link:
self.refresh_from_db(fields=['_path'])
if self._path_id is None:
return None
try:
return self._path

11
netbox/dcim/models/modules.py
View File

@ -325,9 +325,20 @@ class Module(TrackingModelMixin, PrimaryModel):
def save(self, *args, **kwargs):
is_new = self.pk is None
old_module_bay_id = None
if not is_new:
old_module_bay_id = Module.objects.filter(pk=self.pk).values_list(
'module_bay_id', flat=True
).first()
super().save(*args, **kwargs)
if old_module_bay_id is not None and old_module_bay_id != self.module_bay_id:
for child_bay in self.modulebays.select_related('module__module_bay'):
child_bay.snapshot()
child_bay.save()
adopt_components = getattr(self, '_adopt_components', False)
disable_replication = getattr(self, '_disable_replication', False)

3
netbox/dcim/models/racks.py
View File

@ -638,9 +638,6 @@ class Rack(ContactsMixin, ImageAttachmentsMixin, TrackingModelMixin, RackBase):
return elevation.render(face)
def get_0u_devices(self):
return self.devices.filter(position=0)
def get_utilization(self):
"""
Determine the utilization rate of the rack and return it as a percentage. Occupied and reserved units both count

2
netbox/dcim/tests/query_counts.json
View File

@ -72,7 +72,7 @@
"rearporttemplate:api_list_objects": 12,
"region:api_list_objects": 13,
"region:list_objects_with_permission": 20,
"site:api_list_objects": 16,
"site:api_list_objects": 17,
"site:list_objects_with_permission": 22,
"sitegroup:api_list_objects": 13,
"sitegroup:list_objects_with_permission": 20,

73
netbox/dcim/tests/test_api.py
View File

@ -20,6 +20,8 @@ from users.models import ObjectPermission, Token, User
from utilities.testing import (
APITestCase,
APIViewTestCases,
GraphQLFilterTest,
GraphQLQueryTest,
create_test_device,
create_test_nat_ip_pair,
disable_logging,
@ -146,6 +148,19 @@ class SiteTestCase(APIViewTestCases.APIViewTestCase):
bulk_update_data = {
'status': 'planned',
}
graphql_filter_tests = (
GraphQLFilterTest(
name='tenant__name__exact',
filters='tenant: {name: {exact: "Tenant 1"}}',
expected=lambda qs: qs.filter(tenant__name='Tenant 1'),
permissions=('tenancy.view_tenant',),
),
)
def assert_nested_locations_active(self, data):
site_data = data.get('site') or {}
location_names = sorted(location['name'] for location in site_data.get('locations', []))
self.assertEqual(location_names, ['Site1 Active A', 'Site1 Active B'])
@classmethod
def setUpTestData(cls):
@ -160,15 +175,32 @@ class SiteTestCase(APIViewTestCases.APIViewTestCase):
SiteGroup.objects.create(name='Site Group 2', slug='site-group-2'),
)
tenant = Tenant.objects.create(name='Tenant 1', slug='tenant-1')
# Site 1's tenant activates the dynamic tenant prefetch (+1 in api_list_objects baseline).
sites = (
Site(region=regions[0], group=groups[0], name='Site 1', slug='site-1'),
Site(region=regions[0], group=groups[0], tenant=tenant, name='Site 1', slug='site-1'),
Site(region=regions[0], group=groups[0], name='Site 2', slug='site-2'),
Site(region=regions[0], group=groups[0], name='Site 3', slug='site-3'),
)
Site.objects.bulk_create(sites)
nested_site = Site.objects.get(slug='site-1')
cls.nested_site_pk = nested_site.pk
Location.objects.create(
site=nested_site, name='Site1 Active A', slug='site1-active-a',
status=LocationStatusChoices.STATUS_ACTIVE,
)
Location.objects.create(
site=nested_site, name='Site1 Active B', slug='site1-active-b',
status=LocationStatusChoices.STATUS_ACTIVE,
)
Location.objects.create(
site=nested_site, name='Site1 Planned', slug='site1-planned',
status=LocationStatusChoices.STATUS_PLANNED,
)
rir = RIR.objects.create(name='RFC 6996', is_private=True)
tenant = Tenant.objects.create(name='Tenant 1', slug='tenant-1')
asns = [
ASN(asn=65000 + i, rir=rir) for i in range(8)
@ -203,6 +235,19 @@ class SiteTestCase(APIViewTestCases.APIViewTestCase):
},
]
cls.graphql_query_tests = (
GraphQLQueryTest(
name='nested_locations_by_status',
query=(
'{ site(id: ' + str(cls.nested_site_pk) + ') { '
'locations(filters: {status: {exact: STATUS_ACTIVE}}) { name } '
'} }'
),
assert_result=cls.assert_nested_locations_active,
permissions=('dcim.view_location',),
),
)
def test_add_tags(self):
"""
Add tags to an existing object via the add_tags field.
@ -427,6 +472,16 @@ class LocationTestCase(APIViewTestCases.APIViewTestCase):
'description': 'New description',
}
user_permissions = ('dcim.view_site',)
graphql_filter_tests = (
GraphQLFilterTest(
name='status__in_list',
filters='status: {in_list: [STATUS_PLANNED, STATUS_STAGING]}',
expected=lambda qs: qs.filter(status__in=[
LocationStatusChoices.STATUS_PLANNED,
LocationStatusChoices.STATUS_STAGING,
]),
),
)
@classmethod
def setUpTestData(cls):
@ -476,6 +531,20 @@ class LocationTestCase(APIViewTestCases.APIViewTestCase):
parent=parent_locations[0],
status=LocationStatusChoices.STATUS_ACTIVE,
)
Location.objects.create(
site=sites[0],
name='GraphQL Planned Location',
slug='graphql-planned-location',
parent=parent_locations[0],
status=LocationStatusChoices.STATUS_PLANNED,
)
Location.objects.create(
site=sites[0],
name='GraphQL Staging Location',
slug='graphql-staging-location',
parent=parent_locations[0],
status=LocationStatusChoices.STATUS_STAGING,
)
cls.create_data = [
{

9
netbox/dcim/tests/test_forms.py
View File

@ -503,6 +503,15 @@ class InterfaceTestCase(TestCase):
self.assertNotIn('qinq_svlan', form.cleaned_data.keys())
class CableTestCase(TestCase):
def test_invalid_side_designation_raises_value_error(self):
"""_clean_side rejects a side other than 'a' or 'b' with ValueError."""
form = CableImportForm.__new__(CableImportForm)
with self.assertRaisesMessage(ValueError, "Invalid side designation: c"):
form._clean_side('c')
class SiteFormTestCase(TestCase):
"""
Tests for M2MAddRemoveFields using Site ASN assignments as the test case.

167
netbox/dcim/tests/test_models.py
View File

@ -1,4 +1,5 @@
from django.core.exceptions import ValidationError
from django.db.models.signals import post_save
from django.test import TestCase, tag
from circuits.models import *
@ -1187,6 +1188,105 @@ class ModuleBayTestCase(TestCase):
# tree: its path is now a strict descendant of host_bay's path.
self.assertTrue(str(movable_bay.path).startswith(f'{host_bay.path}.'))
@tag('regression') # #22251
def test_moving_module_reparents_child_module_bays(self):
"""
When a module is moved to a different module bay, each child ModuleBay
(a bay that belongs to the module) must have its parent updated to the
new host bay. Without the fix the children stay parented to the old bay
even though Module.module_bay_id has changed.
"""
device_type = DeviceType.objects.first()
device_role = DeviceRole.objects.first()
site = Site.objects.first()
device = Device.objects.create(
name='Move Module Device',
device_type=device_type,
role=device_role,
site=site,
)
bay_a = ModuleBay.objects.create(device=device, name='Bay A')
bay_b = ModuleBay.objects.create(device=device, name='Bay B')
manufacturer = Manufacturer.objects.first()
module_type = ModuleType.objects.create(
manufacturer=manufacturer, model='Move Module Type'
)
module = Module.objects.create(
device=device, module_bay=bay_a, module_type=module_type
)
child_1 = ModuleBay.objects.create(device=device, module=module, name='Child Bay 1')
child_2 = ModuleBay.objects.create(device=device, module=module, name='Child Bay 2')
self.assertEqual(child_1.parent_id, bay_a.pk)
self.assertEqual(child_2.parent_id, bay_a.pk)
# Move the module to bay_b.
module.module_bay = bay_b
module.save()
child_1.refresh_from_db()
child_2.refresh_from_db()
self.assertEqual(child_1.parent_id, bay_b.pk)
self.assertEqual(child_2.parent_id, bay_b.pk)
# Children must be re-rooted under bay_b in the ltree hierarchy.
bay_b.refresh_from_db()
self.assertTrue(str(child_1.path).startswith(f'{bay_b.path}.'))
self.assertTrue(str(child_2.path).startswith(f'{bay_b.path}.'))
@tag('regression') # #22251
def test_moving_module_reparents_grandchild_module_bays(self):
"""
When a module is moved, grandchild ModuleBays (bays inside a module
that is itself installed inside a child bay of the moved module) must
also land in the new ltree subtree. The trigger cascade moves subtrees
atomically, so calling save() only on direct children is sufficient —
this test documents and preserves that invariant for future tree-backend
changes.
"""
device_type = DeviceType.objects.first()
device_role = DeviceRole.objects.first()
site = Site.objects.first()
device = Device.objects.create(
name='Grandchild Move Device',
device_type=device_type,
role=device_role,
site=site,
)
bay_a = ModuleBay.objects.create(device=device, name='Bay A')
bay_b = ModuleBay.objects.create(device=device, name='Bay B')
manufacturer = Manufacturer.objects.first()
module_type = ModuleType.objects.create(
manufacturer=manufacturer, model='Grandchild Move Type'
)
# Depth-1: module installed in bay_a, with one child bay.
module_1 = Module.objects.create(device=device, module_bay=bay_a, module_type=module_type)
child_bay = ModuleBay.objects.create(device=device, module=module_1, name='Child Bay')
# Depth-2: module installed in child_bay, with one grandchild bay.
module_2 = Module.objects.create(device=device, module_bay=child_bay, module_type=module_type)
grandchild_bay = ModuleBay.objects.create(device=device, module=module_2, name='Grandchild Bay')
self.assertEqual(child_bay.parent_id, bay_a.pk)
self.assertEqual(grandchild_bay.parent_id, child_bay.pk)
bay_a.refresh_from_db()
self.assertTrue(str(grandchild_bay.path).startswith(f'{bay_a.path}.'))
# Move the top-level module to bay_b.
module_1.module_bay = bay_b
module_1.save()
child_bay.refresh_from_db()
grandchild_bay.refresh_from_db()
bay_b.refresh_from_db()
self.assertEqual(child_bay.parent_id, bay_b.pk)
self.assertTrue(str(child_bay.path).startswith(f'{bay_b.path}.'))
# Grandchild's direct parent (child_bay) is unchanged; only tree placement moves.
self.assertEqual(grandchild_bay.parent_id, child_bay.pk)
self.assertTrue(str(grandchild_bay.path).startswith(f'{bay_b.path}.'))
def test_single_module_token(self):
device_type = DeviceType.objects.first()
device_role = DeviceRole.objects.first()
@ -2066,6 +2166,73 @@ class CableTestCase(TestCase):
self.assertIsNone(data['connected_endpoints_type'])
self.assertFalse(data['connected_endpoints_reachable'])
@tag('regression') # #21338
def test_path_refreshes_unset_cablepath_reference(self):
"""
An endpoint instance saved during cable creation, before path tracing,
should resolve its path and connected endpoints.
The stale-instance preconditions rely on Cable.save() saving each
CableTermination (which re-saves the endpoint) before trace_paths
creates the CablePath records.
"""
device = Device.objects.get(name='TestDevice2')
interface_a = Interface.objects.create(device=device, name='eth2')
interface_b = Interface.objects.create(device=device, name='eth3')
# Capture the instances handed to the event machinery on save
saved_instances = []
def capture(sender, instance, **kwargs):
saved_instances.append(instance)
post_save.connect(capture, sender=Interface)
try:
Cable(a_terminations=[interface_a], b_terminations=[interface_b]).save()
finally:
post_save.disconnect(capture, sender=Interface)
self.assertEqual(len(saved_instances), 2)
captured_a = next(i for i in saved_instances if i.pk == interface_a.pk)
captured_b = next(i for i in saved_instances if i.pk == interface_b.pk)
# The captured instances predate path tracing: cabled, but no path yet
self.assertIsNotNone(captured_a.cable_id)
self.assertIsNone(captured_a._path_id)
self.assertIsNone(captured_b._path_id)
# The accessor must repair the unset denormalized reference
self.assertIsNotNone(captured_a.path)
self.assertEqual(captured_a.connected_endpoints, [interface_b])
# Serialization as performed by the event queue must see the peer
data = serialize_for_event(captured_b)
self.assertEqual([endpoint['id'] for endpoint in data['connected_endpoints']], [interface_a.pk])
self.assertEqual([peer['id'] for peer in data['link_peers']], [interface_a.pk])
self.assertTrue(data['connected_endpoints_reachable'])
def test_path_returns_none_for_unsaved_endpoint(self):
"""
An unsaved endpoint with a link assigned should report no path rather
than attempting a database refresh.
"""
device = Device.objects.get(name='TestDevice1')
cable = Cable.objects.first()
interface = Interface(device=device, name='tmp', cable=cable)
self.assertIsNone(interface.path)
class CableTerminationTestCase(TestCase):
def test_cache_related_objects_requires_resolvable_termination(self):
"""cache_related_objects raises ValueError when the termination cannot be resolved."""
cable_termination = CableTermination(
termination_type=ObjectType.objects.get_for_model(Interface),
termination_id=0,
)
with self.assertRaises(ValueError):
cable_termination.cache_related_objects()
class VirtualDeviceContextTestCase(TestCase):

6
netbox/dcim/ui/panels.py
View File

@ -32,7 +32,7 @@ class RackDimensionsPanel(panels.ObjectAttributesPanel):
outer_width = attrs.NumericAttr('outer_width', unit_accessor='get_outer_unit_display')
outer_height = attrs.NumericAttr('outer_height', unit_accessor='get_outer_unit_display')
outer_depth = attrs.NumericAttr('outer_depth', unit_accessor='get_outer_unit_display')
mounting_depth = attrs.TextAttr('mounting_depth', format_string=_('{} millimeters'))
mounting_depth = attrs.TextAttr('mounting_depth', format_string=_('{0} millimeters'))
class RackNumberingPanel(panels.ObjectAttributesPanel):
@ -355,8 +355,8 @@ class PowerFeedElectricalPanel(panels.ObjectAttributesPanel):
title = _('Electrical Characteristics')
supply = attrs.ChoiceAttr('supply')
voltage = attrs.TextAttr('voltage', format_string=_('{}V'))
amperage = attrs.TextAttr('amperage', format_string=_('{}A'))
voltage = attrs.TextAttr('voltage', format_string='{}V')
amperage = attrs.TextAttr('amperage', format_string='{}A')
phase = attrs.ChoiceAttr('phase')
max_utilization = attrs.TextAttr('max_utilization', format_string='{}%')

10
netbox/extras/events.py
View File

@ -125,7 +125,13 @@ def enqueue_event(queue, instance, request, event_type):
app_label = instance._meta.app_label
model_name = instance._meta.model_name
assert instance.pk is not None
if instance.pk is None:
raise ValueError(
_("Cannot enqueue an event for an unsaved {app_label}.{model} instance.").format(
app_label=app_label,
model=model_name,
)
)
key = f'{app_label}.{model_name}:{instance.pk}'
if key in queue:
@ -251,7 +257,7 @@ def process_event_rules(event_rules, object_type, event):
if 'snapshots' in event:
params['snapshots'] = event['snapshots']
if 'request' in event:
params['request'] = copy_safe_request(event['request'])
params['request'] = copy_safe_request(event['request'], include_files=False)
# Enqueue the job
ScriptJob.enqueue(**params)

23
netbox/extras/fields.py
View File

@ -1,4 +1,10 @@
from django.db.models import TextField
from django.contrib.postgres.fields import ArrayField
from django.db.models import CharField, TextField
__all__ = (
'CachedValueField',
'ChoiceSetField',
)
class CachedValueField(TextField):
@ -6,3 +12,18 @@ class CachedValueField(TextField):
Currently a dummy field to prevent custom lookups being applied globally to TextField.
"""
pass
class ChoiceSetField(ArrayField):
"""
An ArrayField of two-element [value, label] string pairs representing custom field choices.
"""
def __init__(self, **kwargs):
kwargs['base_field'] = ArrayField(base_field=CharField(max_length=100), size=2)
super().__init__(**kwargs)
def deconstruct(self):
name, path, args, kwargs = super().deconstruct()
# base_field is fixed by __init__ and omitted from migrations
del kwargs['base_field']
return name, path, args, kwargs

2
netbox/extras/forms/bulk_edit.py
View File

@ -212,7 +212,7 @@ class SavedFilterBulkEditForm(ChangelogMessageMixin, OwnerMixin, BulkEditForm):
nullable_fields = ('description',)
class TableConfigBulkEditForm(BulkEditForm):
class TableConfigBulkEditForm(ChangelogMessageMixin, BulkEditForm):
pk = forms.ModelMultipleChoiceField(
queryset=TableConfig.objects.all(),
widget=forms.MultipleHiddenInput

23
netbox/extras/forms/model_forms.py
View File

@ -403,7 +403,7 @@ class SavedFilterForm(ChangelogMessageMixin, OwnerMixin, forms.ModelForm):
super().__init__(*args, initial=initial, **kwargs)
class TableConfigForm(forms.ModelForm):
class TableConfigForm(ChangelogMessageMixin, forms.ModelForm):
object_type = ContentTypeChoiceField(
label=_('Object type'),
queryset=ObjectType.objects.all()
@ -439,10 +439,29 @@ class TableConfigForm(forms.ModelForm):
def __init__(self, data=None, *args, **kwargs):
super().__init__(data, *args, **kwargs)
object_type = ObjectType.objects.get(pk=get_field_value(self, 'object_type'))
self.fields['available_columns'].widget.choices = ()
self.fields['columns'].widget.choices = ()
# Table context may be absent e.g. when the add view is requested directly
object_type_pk = get_field_value(self, 'object_type')
object_type_pk = getattr(object_type_pk, 'pk', object_type_pk)
if not object_type_pk:
return
try:
object_type = ObjectType.objects.get(pk=object_type_pk)
except (ObjectType.DoesNotExist, TypeError, ValueError):
return
model = object_type.model_class()
if model is None:
return
table_name = get_field_value(self, 'table')
table_class = get_table_for_model(model, table_name)
if table_class is None:
return
table = table_class([])
if columns := self._get_columns():

3
netbox/extras/forms/scripts.py
View File

@ -112,5 +112,4 @@ class ScriptFileForm(ManagedFileForm):
data = self.cleaned_data['upload_file']
storage.save(filename, data)
# need to skip ManagedFileForm save method
return super(ManagedFileForm, self).save(*args, **kwargs)
return super().save(*args, **kwargs)

30
netbox/extras/graphql/filter_lookups.py Normal file
View File

@ -0,0 +1,30 @@
import strawberry
import strawberry_django
from django.db.models import Q, QuerySet
from strawberry.directive import DirectiveValue
from strawberry.types import Info
__all__ = (
'ExtraChoicesLookup',
)
@strawberry.input(
one_of=True,
description='Lookup for extra choices defined on a choice set. Only one of the lookup fields can be set.',
)
class ExtraChoicesLookup:
contains: str | None = strawberry.field(
default=strawberry.UNSET, description='Has an extra choice with this value'
)
length: int | None = strawberry.field(
default=strawberry.UNSET, description='Number of extra choices'
)
@strawberry_django.filter_field
def filter(self, info: Info, queryset: QuerySet, prefix: DirectiveValue[str] = '') -> tuple[QuerySet, Q]:
if self.contains is not strawberry.UNSET and self.contains is not None:
return queryset, Q(**{f'{prefix}choice_value': self.contains})
if self.length is not strawberry.UNSET and self.length is not None:
return queryset, Q(**{f'{prefix}len': self.length})
return queryset, Q()

116
netbox/extras/graphql/filters.py
View File

@ -1,4 +1,3 @@
from datetime import datetime
from typing import TYPE_CHECKING, Annotated
import strawberry
@ -23,6 +22,7 @@ if TYPE_CHECKING:
SiteFilter,
SiteGroupFilter,
)
from extras.graphql.filter_lookups import ExtraChoicesLookup
from netbox.graphql.enums import ColorEnum
from netbox.graphql.filter_lookups import FloatLookup, IntegerLookup, JSONFilter, StringArrayLookup, TreeNodeFilter
from tenancy.graphql.filters import TenantFilter, TenantGroupFilter
@ -54,11 +54,11 @@ __all__ = (
@strawberry_django.filter_type(models.ConfigContext, lookups=True)
class ConfigContextFilter(SyncedDataFilterMixin, ChangeLoggedModelFilter):
name: StrFilterLookup[str] | None = strawberry_django.filter_field()
name: StrFilterLookup | None = strawberry_django.filter_field()
weight: Annotated['IntegerLookup', strawberry.lazy('netbox.graphql.filter_lookups')] | None = (
strawberry_django.filter_field()
)
description: StrFilterLookup[str] | None = strawberry_django.filter_field()
description: StrFilterLookup | None = strawberry_django.filter_field()
is_active: FilterLookup[bool] | None = strawberry_django.filter_field()
regions: Annotated['RegionFilter', strawberry.lazy('dcim.graphql.filters')] | None = (
strawberry_django.filter_field()
@ -111,22 +111,22 @@ class ConfigContextFilter(SyncedDataFilterMixin, ChangeLoggedModelFilter):
@strawberry_django.filter_type(models.ConfigContextProfile, lookups=True)
class ConfigContextProfileFilter(SyncedDataFilterMixin, PrimaryModelFilter):
name: StrFilterLookup[str] | None = strawberry_django.filter_field()
description: StrFilterLookup[str] | None = strawberry_django.filter_field()
name: StrFilterLookup | None = strawberry_django.filter_field()
description: StrFilterLookup | None = strawberry_django.filter_field()
tags: Annotated['TagFilter', strawberry.lazy('extras.graphql.filters')] | None = strawberry_django.filter_field()
@strawberry_django.filter_type(models.ConfigTemplate, lookups=True)
class ConfigTemplateFilter(SyncedDataFilterMixin, ChangeLoggedModelFilter):
name: StrFilterLookup[str] | None = strawberry_django.filter_field()
description: StrFilterLookup[str] | None = strawberry_django.filter_field()
template_code: StrFilterLookup[str] | None = strawberry_django.filter_field()
name: StrFilterLookup | None = strawberry_django.filter_field()
description: StrFilterLookup | None = strawberry_django.filter_field()
template_code: StrFilterLookup | None = strawberry_django.filter_field()
environment_params: Annotated['JSONFilter', strawberry.lazy('netbox.graphql.filter_lookups')] | None = (
strawberry_django.filter_field()
)
mime_type: StrFilterLookup[str] | None = strawberry_django.filter_field()
file_name: StrFilterLookup[str] | None = strawberry_django.filter_field()
file_extension: StrFilterLookup[str] | None = strawberry_django.filter_field()
mime_type: StrFilterLookup | None = strawberry_django.filter_field()
file_name: StrFilterLookup | None = strawberry_django.filter_field()
file_extension: StrFilterLookup | None = strawberry_django.filter_field()
as_attachment: FilterLookup[bool] | None = strawberry_django.filter_field()
@ -141,10 +141,10 @@ class CustomFieldFilter(ChangeLoggedModelFilter):
related_object_type: Annotated['ContentTypeFilter', strawberry.lazy('core.graphql.filters')] | None = (
strawberry_django.filter_field()
)
name: StrFilterLookup[str] | None = strawberry_django.filter_field()
label: StrFilterLookup[str] | None = strawberry_django.filter_field()
group_name: StrFilterLookup[str] | None = strawberry_django.filter_field()
description: StrFilterLookup[str] | None = strawberry_django.filter_field()
name: StrFilterLookup | None = strawberry_django.filter_field()
label: StrFilterLookup | None = strawberry_django.filter_field()
group_name: StrFilterLookup | None = strawberry_django.filter_field()
description: StrFilterLookup | None = strawberry_django.filter_field()
required: FilterLookup[bool] | None = strawberry_django.filter_field()
unique: FilterLookup[bool] | None = strawberry_django.filter_field()
search_weight: Annotated['IntegerLookup', strawberry.lazy('netbox.graphql.filter_lookups')] | None = (
@ -170,7 +170,7 @@ class CustomFieldFilter(ChangeLoggedModelFilter):
validation_maximum: Annotated['FloatLookup', strawberry.lazy('netbox.graphql.filter_lookups')] | None = (
strawberry_django.filter_field()
)
validation_regex: StrFilterLookup[str] | None = strawberry_django.filter_field()
validation_regex: StrFilterLookup | None = strawberry_django.filter_field()
choice_set: Annotated['CustomFieldChoiceSetFilter', strawberry.lazy('extras.graphql.filters')] | None = (
strawberry_django.filter_field()
)
@ -186,19 +186,19 @@ class CustomFieldFilter(ChangeLoggedModelFilter):
strawberry_django.filter_field()
)
is_cloneable: FilterLookup[bool] | None = strawberry_django.filter_field()
comments: StrFilterLookup[str] | None = strawberry_django.filter_field()
comments: StrFilterLookup | None = strawberry_django.filter_field()
@strawberry_django.filter_type(models.CustomFieldChoiceSet, lookups=True)
class CustomFieldChoiceSetFilter(ChangeLoggedModelFilter):
name: StrFilterLookup[str] | None = strawberry_django.filter_field()
description: StrFilterLookup[str] | None = strawberry_django.filter_field()
name: StrFilterLookup | None = strawberry_django.filter_field()
description: StrFilterLookup | None = strawberry_django.filter_field()
base_choices: (
BaseFilterLookup[Annotated['CustomFieldChoiceSetBaseEnum', strawberry.lazy('extras.graphql.enums')]] | None
) = (
strawberry_django.filter_field()
)
extra_choices: Annotated['StringArrayLookup', strawberry.lazy('netbox.graphql.filter_lookups')] | None = (
extra_choices: Annotated['ExtraChoicesLookup', strawberry.lazy('extras.graphql.filter_lookups')] | None = (
strawberry_django.filter_field()
)
order_alphabetically: FilterLookup[bool] | None = strawberry_django.filter_field()
@ -233,14 +233,14 @@ class CustomFieldChoiceSetFilter(ChangeLoggedModelFilter):
@strawberry_django.filter_type(models.CustomLink, lookups=True)
class CustomLinkFilter(ChangeLoggedModelFilter):
name: StrFilterLookup[str] | None = strawberry_django.filter_field()
name: StrFilterLookup | None = strawberry_django.filter_field()
enabled: FilterLookup[bool] | None = strawberry_django.filter_field()
link_text: StrFilterLookup[str] | None = strawberry_django.filter_field()
link_url: StrFilterLookup[str] | None = strawberry_django.filter_field()
link_text: StrFilterLookup | None = strawberry_django.filter_field()
link_url: StrFilterLookup | None = strawberry_django.filter_field()
weight: Annotated['IntegerLookup', strawberry.lazy('netbox.graphql.filter_lookups')] | None = (
strawberry_django.filter_field()
)
group_name: StrFilterLookup[str] | None = strawberry_django.filter_field()
group_name: StrFilterLookup | None = strawberry_django.filter_field()
button_class: (
BaseFilterLookup[Annotated['CustomLinkButtonClassEnum', strawberry.lazy('extras.graphql.enums')]] | None
) = (
@ -251,15 +251,15 @@ class CustomLinkFilter(ChangeLoggedModelFilter):
@strawberry_django.filter_type(models.ExportTemplate, lookups=True)
class ExportTemplateFilter(SyncedDataFilterMixin, ChangeLoggedModelFilter):
name: StrFilterLookup[str] | None = strawberry_django.filter_field()
description: StrFilterLookup[str] | None = strawberry_django.filter_field()
template_code: StrFilterLookup[str] | None = strawberry_django.filter_field()
name: StrFilterLookup | None = strawberry_django.filter_field()
description: StrFilterLookup | None = strawberry_django.filter_field()
template_code: StrFilterLookup | None = strawberry_django.filter_field()
environment_params: Annotated['JSONFilter', strawberry.lazy('netbox.graphql.filter_lookups')] | None = (
strawberry_django.filter_field()
)
mime_type: StrFilterLookup[str] | None = strawberry_django.filter_field()
file_name: StrFilterLookup[str] | None = strawberry_django.filter_field()
file_extension: StrFilterLookup[str] | None = strawberry_django.filter_field()
mime_type: StrFilterLookup | None = strawberry_django.filter_field()
file_name: StrFilterLookup | None = strawberry_django.filter_field()
file_extension: StrFilterLookup | None = strawberry_django.filter_field()
as_attachment: FilterLookup[bool] | None = strawberry_django.filter_field()
@ -275,7 +275,7 @@ class ImageAttachmentFilter(ChangeLoggedModelFilter):
image_width: Annotated['IntegerLookup', strawberry.lazy('netbox.graphql.filter_lookups')] | None = (
strawberry_django.filter_field()
)
name: StrFilterLookup[str] | None = strawberry_django.filter_field()
name: StrFilterLookup | None = strawberry_django.filter_field()
@strawberry_django.filter_type(models.JournalEntry, lookups=True)
@ -291,13 +291,13 @@ class JournalEntryFilter(CustomFieldsFilterMixin, TagsFilterMixin, ChangeLoggedM
kind: BaseFilterLookup[Annotated['JournalEntryKindEnum', strawberry.lazy('extras.graphql.enums')]] | None = (
strawberry_django.filter_field()
)
comments: StrFilterLookup[str] | None = strawberry_django.filter_field()
comments: StrFilterLookup | None = strawberry_django.filter_field()
@strawberry_django.filter_type(models.Notification, lookups=True)
class NotificationFilter(BaseModelFilter):
created: DatetimeFilterLookup[datetime] | None = strawberry_django.filter_field()
read: DatetimeFilterLookup[datetime] | None = strawberry_django.filter_field()
created: DatetimeFilterLookup | None = strawberry_django.filter_field()
read: DatetimeFilterLookup | None = strawberry_django.filter_field()
user: Annotated['UserFilter', strawberry.lazy('users.graphql.filters')] | None = strawberry_django.filter_field()
user_id: ID | None = strawberry_django.filter_field()
object_type: Annotated['ContentTypeFilter', strawberry.lazy('core.graphql.filters')] | None = (
@ -305,23 +305,23 @@ class NotificationFilter(BaseModelFilter):
)
object_type_id: ID | None = strawberry_django.filter_field()
object_id: ID | None = strawberry_django.filter_field()
object_repr: StrFilterLookup[str] | None = strawberry_django.filter_field()
event_type: StrFilterLookup[str] | None = strawberry_django.filter_field()
object_repr: StrFilterLookup | None = strawberry_django.filter_field()
event_type: StrFilterLookup | None = strawberry_django.filter_field()
@strawberry_django.filter_type(models.NotificationGroup, lookups=True)
class NotificationGroupFilter(ChangeLoggedModelFilter):
name: StrFilterLookup[str] | None = strawberry_django.filter_field()
description: StrFilterLookup[str] | None = strawberry_django.filter_field()
name: StrFilterLookup | None = strawberry_django.filter_field()
description: StrFilterLookup | None = strawberry_django.filter_field()
groups: Annotated['GroupFilter', strawberry.lazy('users.graphql.filters')] | None = strawberry_django.filter_field()
users: Annotated['UserFilter', strawberry.lazy('users.graphql.filters')] | None = strawberry_django.filter_field()
@strawberry_django.filter_type(models.SavedFilter, lookups=True)
class SavedFilterFilter(ChangeLoggedModelFilter):
name: StrFilterLookup[str] | None = strawberry_django.filter_field()
slug: StrFilterLookup[str] | None = strawberry_django.filter_field()
description: StrFilterLookup[str] | None = strawberry_django.filter_field()
name: StrFilterLookup | None = strawberry_django.filter_field()
slug: StrFilterLookup | None = strawberry_django.filter_field()
description: StrFilterLookup | None = strawberry_django.filter_field()
user: Annotated['UserFilter', strawberry.lazy('users.graphql.filters')] | None = strawberry_django.filter_field()
user_id: ID | None = strawberry_django.filter_field()
weight: Annotated['IntegerLookup', strawberry.lazy('netbox.graphql.filter_lookups')] | None = (
@ -336,7 +336,7 @@ class SavedFilterFilter(ChangeLoggedModelFilter):
@strawberry_django.filter_type(models.Subscription, lookups=True)
class SubscriptionFilter(BaseModelFilter):
created: DatetimeFilterLookup[datetime] | None = strawberry_django.filter_field()
created: DatetimeFilterLookup | None = strawberry_django.filter_field()
user: Annotated['UserFilter', strawberry.lazy('users.graphql.filters')] | None = strawberry_django.filter_field()
user_id: ID | None = strawberry_django.filter_field()
object_type: Annotated['ContentTypeFilter', strawberry.lazy('core.graphql.filters')] | None = (
@ -348,8 +348,8 @@ class SubscriptionFilter(BaseModelFilter):
@strawberry_django.filter_type(models.TableConfig, lookups=True)
class TableConfigFilter(ChangeLoggedModelFilter):
name: StrFilterLookup[str] | None = strawberry_django.filter_field()
description: StrFilterLookup[str] | None = strawberry_django.filter_field()
name: StrFilterLookup | None = strawberry_django.filter_field()
description: StrFilterLookup | None = strawberry_django.filter_field()
user: Annotated['UserFilter', strawberry.lazy('users.graphql.filters')] | None = strawberry_django.filter_field()
user_id: ID | None = strawberry_django.filter_field()
weight: Annotated['IntegerLookup', strawberry.lazy('netbox.graphql.filter_lookups')] | None = (
@ -361,30 +361,30 @@ class TableConfigFilter(ChangeLoggedModelFilter):
@strawberry_django.filter_type(models.Tag, lookups=True)
class TagFilter(ChangeLoggedModelFilter):
name: StrFilterLookup[str] | None = strawberry_django.filter_field()
slug: StrFilterLookup[str] | None = strawberry_django.filter_field()
name: StrFilterLookup | None = strawberry_django.filter_field()
slug: StrFilterLookup | None = strawberry_django.filter_field()
color: BaseFilterLookup[Annotated['ColorEnum', strawberry.lazy('netbox.graphql.enums')]] | None = (
strawberry_django.filter_field()
)
description: StrFilterLookup[str] | None = strawberry_django.filter_field()
description: StrFilterLookup | None = strawberry_django.filter_field()
@strawberry_django.filter_type(models.Webhook, lookups=True)
class WebhookFilter(CustomFieldsFilterMixin, TagsFilterMixin, ChangeLoggedModelFilter):
name: StrFilterLookup[str] | None = strawberry_django.filter_field()
description: StrFilterLookup[str] | None = strawberry_django.filter_field()
payload_url: StrFilterLookup[str] | None = strawberry_django.filter_field()
name: StrFilterLookup | None = strawberry_django.filter_field()
description: StrFilterLookup | None = strawberry_django.filter_field()
payload_url: StrFilterLookup | None = strawberry_django.filter_field()
http_method: (
BaseFilterLookup[Annotated['WebhookHttpMethodEnum', strawberry.lazy('extras.graphql.enums')]] | None
) = (
strawberry_django.filter_field()
)
http_content_type: StrFilterLookup[str] | None = strawberry_django.filter_field()
additional_headers: StrFilterLookup[str] | None = strawberry_django.filter_field()
body_template: StrFilterLookup[str] | None = strawberry_django.filter_field()
secret: StrFilterLookup[str] | None = strawberry_django.filter_field()
http_content_type: StrFilterLookup | None = strawberry_django.filter_field()
additional_headers: StrFilterLookup | None = strawberry_django.filter_field()
body_template: StrFilterLookup | None = strawberry_django.filter_field()
secret: StrFilterLookup | None = strawberry_django.filter_field()
ssl_verification: FilterLookup[bool] | None = strawberry_django.filter_field()
ca_file_path: StrFilterLookup[str] | None = strawberry_django.filter_field()
ca_file_path: StrFilterLookup | None = strawberry_django.filter_field()
events: Annotated['EventRuleFilter', strawberry.lazy('extras.graphql.filters')] | None = (
strawberry_django.filter_field()
)
@ -392,8 +392,8 @@ class WebhookFilter(CustomFieldsFilterMixin, TagsFilterMixin, ChangeLoggedModelF
@strawberry_django.filter_type(models.EventRule, lookups=True)
class EventRuleFilter(CustomFieldsFilterMixin, TagsFilterMixin, ChangeLoggedModelFilter):
name: StrFilterLookup[str] | None = strawberry_django.filter_field()
description: StrFilterLookup[str] | None = strawberry_django.filter_field()
name: StrFilterLookup | None = strawberry_django.filter_field()
description: StrFilterLookup | None = strawberry_django.filter_field()
event_types: Annotated['StringArrayLookup', strawberry.lazy('netbox.graphql.filter_lookups')] | None = (
strawberry_django.filter_field()
)
@ -412,4 +412,4 @@ class EventRuleFilter(CustomFieldsFilterMixin, TagsFilterMixin, ChangeLoggedMode
action_data: Annotated['JSONFilter', strawberry.lazy('netbox.graphql.filter_lookups')] | None = (
strawberry_django.filter_field()
)
comments: StrFilterLookup[str] | None = strawberry_django.filter_field()
comments: StrFilterLookup | None = strawberry_django.filter_field()

28
netbox/extras/lookups.py
View File

@ -3,7 +3,16 @@ from django.contrib.postgres.fields.ranges import RangeField
from django.db.models import CharField, JSONField, Lookup
from django.db.models.fields.json import KeyTextTransform
from .fields import CachedValueField
from .fields import CachedValueField, ChoiceSetField
__all__ = (
'ChoiceValueLookup',
'Empty',
'JSONEmpty',
'NetContainsOrEquals',
'NetHost',
'RangeContains',
)
class RangeContains(Lookup):
@ -34,6 +43,22 @@ class RangeContains(Lookup):
return sql, params
class ChoiceValueLookup(Lookup):
"""
Match rows where any [value, label] pair in a ChoiceSetField has the given value.
Compares the RHS against the first element (the value) of each pair.
"""
lookup_name = 'choice_value'
prepare_rhs = False
def as_sql(self, compiler, connection):
lhs, lhs_params = self.process_lhs(compiler, connection)
rhs, rhs_params = self.process_rhs(compiler, connection)
# Slice the value column of the two-dimensional array and match any element
return f'{rhs} = ANY({lhs}[:][1:1])', [*rhs_params, *lhs_params]
class Empty(Lookup):
"""
Filter on whether a string is empty.
@ -99,6 +124,7 @@ class NetContainsOrEquals(Lookup):
ArrayField.register_lookup(RangeContains)
ChoiceSetField.register_lookup(ChoiceValueLookup)
CharField.register_lookup(Empty)
JSONField.register_lookup(JSONEmpty)
CachedValueField.register_lookup(NetHost)

18
netbox/extras/migrations/0139_alter_customfieldchoiceset_extra_choices.py Normal file
View File

@ -0,0 +1,18 @@
from django.db import migrations
import extras.fields
class Migration(migrations.Migration):
dependencies = [
('extras', '0138_customfieldchoiceset_choice_colors'),
]
operations = [
migrations.AlterField(
model_name='customfieldchoiceset',
name='extra_choices',
field=extras.fields.ChoiceSetField(blank=True, null=True),
),
]

10
netbox/extras/models/customfields.py
View File

@ -7,7 +7,6 @@ import django_filters
import jsonschema
from django import forms
from django.conf import settings
from django.contrib.postgres.fields import ArrayField
from django.core.validators import RegexValidator, ValidationError
from django.db import models
from django.db.models import F, Func, Value
@ -21,6 +20,7 @@ from jsonschema.exceptions import ValidationError as JSONValidationError
from core.models import ObjectType
from extras.choices import *
from extras.data import CHOICE_SETS
from extras.fields import ChoiceSetField
from netbox.context import query_cache
from netbox.models import ChangeLoggedModel
from netbox.models.features import CloningMixin, ExportTemplatesMixin
@ -461,6 +461,8 @@ class CustomField(CloningMixin, ExportTemplatesMixin, OwnerMixin, ChangeLoggedMo
"""
if value is None:
return value
if self.type == CustomFieldTypeChoices.TYPE_DECIMAL:
return float(value)
if self.type == CustomFieldTypeChoices.TYPE_DATE and type(value) is date:
return value.isoformat()
if self.type == CustomFieldTypeChoices.TYPE_DATETIME and type(value) is datetime:
@ -877,11 +879,7 @@ class CustomFieldChoiceSet(CloningMixin, ExportTemplatesMixin, OwnerMixin, Chang
null=True,
help_text=_('Base set of predefined choices (optional)')
)
extra_choices = ArrayField(
ArrayField(
base_field=models.CharField(max_length=100),
size=2
),
extra_choices = ChoiceSetField(
blank=True,
null=True
)

6
netbox/extras/models/models.py
View File

@ -649,6 +649,10 @@ class TableConfig(CloningMixin, ChangeLoggedModel):
def clean(self):
super().clean()
# Skip table validation until the object type and table have been set
if not self.object_type_id or not self.table:
return
# Validate table
if self.table_class is None:
raise ValidationError({
@ -667,7 +671,7 @@ class TableConfig(CloningMixin, ChangeLoggedModel):
})
# Validate selected columns
for name in self.columns:
for name in self.columns or []:
if name not in table.columns:
raise ValidationError({
'columns': _('Unknown column: {name}').format(name=name)

58
netbox/extras/tests/test_api.py
View File

@ -325,6 +325,64 @@ class CustomFieldChoiceSetTestCase(APIViewTestCases.APIViewTestCase):
response = self.client.post(self._get_list_url(), data, format='json', **self.header)
self.assertEqual(response.status_code, 400)
def test_graphql_filter_extra_choices(self):
"""Filter choice sets by choice value and by number of choices."""
self.add_permissions('extras.view_customfieldchoiceset')
# '1A' appears here only as a label, so it must not match contains
CustomFieldChoiceSet.objects.create(
name='Choice Set Labels',
extra_choices=[['sel1', 'Selection 1'], ['other', '1A']],
)
def run(lookup):
query = '{ custom_field_choice_set_list(filters: {extra_choices: ' + lookup + '}) { name } }'
response = self.client.post(reverse('graphql'), data={'query': query}, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
data = response.json()
self.assertNotIn('errors', data)
return sorted(row['name'] for row in data['data']['custom_field_choice_set_list'])
# contains matches choice values only, never labels
self.assertEqual(run('{contains: "1A"}'), ['Choice Set 1'])
self.assertEqual(run('{contains: "sel1"}'), ['Choice Set Labels'])
self.assertEqual(run('{contains: "Selection 1"}'), [])
# length is the number of [value, label] pairs
self.assertEqual(run('{length: 2}'), ['Choice Set Labels'])
self.assertEqual(run('{length: 1}'), [])
def test_graphql_filter_extra_choices_rejects_array_operands(self):
"""The legacy flat and nested array operand shapes fail schema validation."""
self.add_permissions('extras.view_customfieldchoiceset')
def run_invalid(lookup):
query = '{ custom_field_choice_set_list(filters: {extra_choices: ' + lookup + '}) { name } }'
response = self.client.post(reverse('graphql'), data={'query': query}, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
self.assertIn('errors', response.json())
# shapes advertised or attempted before #22324
run_invalid('{contains: ["1A"]}')
run_invalid('{contains: [["1A", "Choice 1A"]]}')
def test_graphql_filter_extra_choices_via_relation(self):
"""The extra_choices lookup composes through the choice_set relation prefix."""
self.add_permissions('extras.view_customfield')
for choice_set in CustomFieldChoiceSet.objects.filter(name__in=['Choice Set 1', 'Choice Set 2']):
CustomField.objects.create(
name=f'cf_{choice_set.name[-1]}',
type=CustomFieldTypeChoices.TYPE_SELECT,
choice_set=choice_set,
)
query = '{ custom_field_list(filters: {choice_set: {extra_choices: {contains: "1A"}}}) { name } }'
response = self.client.post(reverse('graphql'), data={'query': query}, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
data = response.json()
self.assertNotIn('errors', data)
self.assertEqual([row['name'] for row in data['data']['custom_field_list']], ['cf_1'])
class CustomLinkTestCase(APIViewTestCases.APIViewTestCase):
model = CustomLink

212
netbox/extras/tests/test_event_rules.py
View File

@ -1,30 +1,36 @@
import json
import logging
import uuid
from io import BytesIO
from unittest import skipIf
from unittest.mock import Mock, patch
import django_rq
from django.conf import settings
from django.http import HttpResponse
from django.test import RequestFactory
from django.test import RequestFactory, tag
from django.urls import reverse
from PIL import Image
from requests import Session
from rest_framework import status
from core.choices import ManagedFileRootPathChoices
from core.events import *
from core.models import ObjectType
from core.models import Job, ObjectType
from dcim.choices import SiteStatusChoices
from dcim.models import Site
from dcim.models import DeviceType, Interface, Manufacturer, Site
from extras.choices import EventRuleActionChoices
from extras.events import enqueue_event, flush_events, serialize_for_event
from extras.models import EventRule, Script, Tag, Webhook
from extras.models import EventRule, Script, ScriptModule, Tag, Webhook
from extras.scripts import Script as ScriptBase
from extras.signals import process_job_end_event_rules
from extras.webhooks import generate_signature, send_webhook
from netbox.context_managers import event_tracking
from utilities.testing import APITestCase
from utilities.testing import APITestCase, create_test_device
from utilities.testing.mixins import RQQueueTestMixin
class EventRuleTestCase(APITestCase):
class EventRuleTestCase(RQQueueTestMixin, APITestCase):
def setUp(self):
super().setUp()
@ -39,6 +45,16 @@ class EventRuleTestCase(APITestCase):
# Clear the queue so leftover jobs do not leak to the next test suite
self.queue.empty()
def test_enqueue_event_requires_saved_instance(self):
"""enqueue_event raises ValueError for an unsaved instance."""
request = RequestFactory().get('/')
request.id = uuid.uuid4()
request.user = self.user
site = Site(name='Site 1', slug='site-1')
with patch('extras.events.has_feature', return_value=True):
with self.assertRaises(ValueError):
enqueue_event({}, site, request, OBJECT_CREATED)
@classmethod
def setUpTestData(cls):
@ -531,6 +547,48 @@ class EventRuleTestCase(APITestCase):
self.assertEqual(event['data']['name'], 'Site 1')
self.assertIsNone(event['snapshots']['postchange'])
@tag('regression') # #21338
def test_cable_creation_event_payload_includes_connected_endpoints(self):
"""
Interface update events queued during cable creation must include the
peer interface in connected_endpoints and link_peers.
"""
webhook = Webhook.objects.get(name='Webhook 1')
event_rule = EventRule.objects.create(
name='Interface Update Rule',
event_types=[OBJECT_UPDATED],
action_type=EventRuleActionChoices.WEBHOOK,
action_object_type=ObjectType.objects.get_for_model(Webhook),
action_object_id=webhook.id,
)
event_rule.object_types.set([ObjectType.objects.get_for_model(Interface)])
device = create_test_device('Device 1')
interface_a = Interface.objects.create(device=device, name='eth0')
interface_b = Interface.objects.create(device=device, name='eth1')
# Create a cable between the two interfaces via the REST API
data = {
'a_terminations': [{'object_type': 'dcim.interface', 'object_id': interface_a.pk}],
'b_terminations': [{'object_type': 'dcim.interface', 'object_id': interface_b.pk}],
}
url = reverse('dcim-api:cable-list')
self.add_permissions('dcim.add_cable')
response = self.client.post(url, data, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_201_CREATED)
# One update event was queued for each interface
self.assertEqual(self.queue.count, 2)
payloads = {job.kwargs['data']['id']: job.kwargs['data'] for job in self.queue.jobs}
peers = {interface_a.pk: interface_b.pk, interface_b.pk: interface_a.pk}
self.assertEqual(set(payloads), set(peers))
for interface_id, payload in payloads.items():
peer_id = peers[interface_id]
self.assertIsNotNone(payload['connected_endpoints'])
self.assertEqual([endpoint['id'] for endpoint in payload['connected_endpoints']], [peer_id])
self.assertEqual([peer['id'] for peer in payload['link_peers']], [peer_id])
self.assertTrue(payload['connected_endpoints_reachable'])
def test_duplicate_triggers(self):
"""
Test for erroneous duplicate event triggers resulting from saving an object multiple times
@ -601,3 +659,145 @@ class EventRuleTestCase(APITestCase):
self.add_permissions('dcim.add_site')
response = self.client.post(url, {'name': 'Site X', 'slug': 'site-x'}, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_201_CREATED)
@tag('regression')
def test_eventrule_script_action_with_object_image_files(self):
"""
Verify that a Script event-rule action can be enqueued and executed cleanly when the
triggering object carries uploaded files (e.g. DeviceType images).
This is a regression test for issue #22376.
"""
# Create a dummy script class and an instance of it
class DummyScript(ScriptBase):
class Meta:
name = "Dummy Script"
def run(self, data, commit=True):
return "finished successfully"
dummy_script = DummyScript()
# Create ScriptModule and Script
with patch.object(ScriptModule, 'sync_classes'):
module = ScriptModule.objects.create(
file_root=ManagedFileRootPathChoices.SCRIPTS,
file_path='dummy_script.py',
)
script = Script.objects.create(
module=module,
name='Dummy Script',
is_executable=True,
)
script_type = ObjectType.objects.get_for_model(Script)
# Create an event rule that triggers on DeviceType update with Script action
devicetype_type = ObjectType.objects.get_for_model(DeviceType)
event_rule = EventRule.objects.create(
name='Test Script Event Rule with Files',
event_types=[OBJECT_UPDATED],
action_type=EventRuleActionChoices.SCRIPT,
action_object_type=script_type,
action_object_id=script.pk,
)
event_rule.object_types.set([devicetype_type])
# Create a manufacturer and DeviceType
manufacturer = Manufacturer.objects.create(
name='Test Manufacturer',
slug='test-manufacturer',
)
devicetype = DeviceType.objects.create(
model='Test DeviceType',
slug="test-devicetype",
manufacturer=manufacturer,
)
# Create an image file
image = BytesIO()
Image.new('RGB', (1, 1)).save(image, format='PNG')
image.name = 'test_image.png'
image.seek(0)
# PATCH the DeviceType via REST API to add the image
data = {
'front_image': image,
}
url = reverse('dcim-api:devicetype-detail', kwargs={'pk': devicetype.pk})
self.add_permissions('dcim.change_devicetype')
# Mock the script's python_class to prevent the test from trying to load from disk
with patch.object(Script, 'python_class') as mock:
mock.return_value = dummy_script
# Since in core/models/jobs.py Jobs are enqueued with a transaction.on_commit-handler
# we simulate commit by using captureOnCommitCallbacks context manager
with self.captureOnCommitCallbacks(execute=True):
response = self.client.patch(url, data, format='multipart', **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
# Assert that the script job was enqueued cleanly and is waiting for execution
self.assertEqual(self.queue.count, 1)
script_job = Job.objects.filter(name=dummy_script.name).last()
self.assertEqual(script_job.status, "pending")
# silence rqworker (cleaner output) and trigger job execution
logging.getLogger('rq.worker').setLevel(logging.ERROR)
self.run_rq_jobs('default')
# Assert that our script was executed without any errors
script_job.refresh_from_db()
self.assertEqual(script_job.status, "completed")
self.assertEqual(script_job.data.get('output', ''), "finished successfully")
@tag('regression')
def test_eventrule_webhook_action_with_object_image_files(self):
"""
Verify that a Webhook event-rule action can be enqueued and executed cleanly when
the triggering object carries uploaded files (e.g. DeviceType images).
This is a regression test for issue #20873.
"""
# Create an event rule that triggers on DeviceType update with Script action
webhook = Webhook.objects.get(name='Webhook 1')
webhook_type = ObjectType.objects.get_for_model(Webhook)
devicetype_type = ObjectType.objects.get_for_model(DeviceType)
event_rule = EventRule.objects.create(
name='Test Webhook Event Rule with Files',
event_types=[OBJECT_UPDATED],
action_type=EventRuleActionChoices.WEBHOOK,
action_object_type=webhook_type,
action_object_id=webhook.pk,
)
event_rule.object_types.set([devicetype_type])
# Create a manufacturer and DeviceType
manufacturer = Manufacturer.objects.create(
name='Test Manufacturer',
slug='test-manufacturer',
)
devicetype = DeviceType.objects.create(
model='Test DeviceType',
slug="test-devicetype",
manufacturer=manufacturer,
)
# Create an image file
image = BytesIO()
Image.new('RGB', (1, 1)).save(image, format='PNG')
image.name = 'test_image.png'
image.seek(0)
# PATCH the DeviceType via REST API to add the image
data = {
'front_image': image,
}
url = reverse('dcim-api:devicetype-detail', kwargs={'pk': devicetype.pk})
self.add_permissions('dcim.change_devicetype')
response = self.client.patch(url, data, format='multipart', **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
# Assert that the webhook job was enqueued cleanly
self.assertEqual(self.queue.count, 1)
job = self.queue.jobs[0]
self.assertEqual(job.kwargs['event_rule'], event_rule)
self.assertEqual(job.kwargs['event_type'], OBJECT_UPDATED)

51
netbox/extras/tests/test_forms.py
View File

@ -10,7 +10,7 @@ from core.models import DataSource, ObjectType
from dcim.forms import SiteForm
from dcim.models import Site
from extras.choices import CustomFieldTypeChoices
from extras.forms import SavedFilterForm
from extras.forms import SavedFilterForm, TableConfigBulkEditForm, TableConfigForm
from extras.forms.model_forms import CustomFieldChoiceSetForm
from extras.forms.scripts import ScriptFileForm
from extras.models import CustomField, CustomFieldChoiceSet, ScriptModule
@ -288,3 +288,52 @@ class ScriptFileFormTestCase(TestCase):
form = ScriptFileForm(files={'upload_file': upload_file}, instance=self._new_module())
self.assertTrue(form.is_valid())
class TableConfigFormTestCase(TestCase):
def test_form_without_table_context(self):
"""The form must be constructible without an object type."""
form = TableConfigForm()
self.assertEqual(list(form.fields['available_columns'].widget.choices), [])
self.assertEqual(list(form.fields['columns'].widget.choices), [])
def test_form_with_invalid_object_type(self):
"""An unknown object type must yield empty column choices."""
last_pk = ObjectType.objects.order_by('pk').last().pk
form = TableConfigForm(initial={'object_type': last_pk + 1})
self.assertEqual(list(form.fields['available_columns'].widget.choices), [])
self.assertEqual(list(form.fields['columns'].widget.choices), [])
def test_form_with_unknown_table(self):
"""An unresolvable table name must yield empty column choices."""
object_type = ObjectType.objects.get_for_model(Site)
form = TableConfigForm(initial={'object_type': object_type.pk, 'table': 'NoSuchTable'})
self.assertEqual(list(form.fields['columns'].widget.choices), [])
def test_form_with_table_context(self):
"""Column choices must be populated from the resolved table."""
object_type = ObjectType.objects.get_for_model(Site)
form = TableConfigForm(initial={
'object_type': object_type.pk,
'table': 'SiteTable',
'columns': ['name', 'status'],
})
self.assertEqual(
[name for name, _ in form.fields['columns'].widget.choices],
['name', 'status']
)
self.assertIn('region', dict(form.fields['available_columns'].widget.choices))
def test_form_includes_changelog_message(self):
"""The model form must expose the changelog_message meta field."""
object_type = ObjectType.objects.get_for_model(Site)
form = TableConfigForm(initial={'object_type': object_type.pk, 'table': 'SiteTable'})
self.assertIn('changelog_message', form.fields)
self.assertIn('changelog_message', form.meta_fields)
def test_bulk_edit_form_includes_changelog_message(self):
"""The bulk edit form must expose the changelog_message meta field."""
form = TableConfigBulkEditForm()
self.assertIn('changelog_message', form.fields)
self.assertIn('changelog_message', form.meta_fields)

31
netbox/extras/tests/test_lookups.py Normal file
View File

@ -0,0 +1,31 @@
from django.core.exceptions import FieldError
from django.test import TestCase
from extras.choices import CustomFieldChoiceSetBaseChoices
from extras.models import CustomFieldChoiceSet, EventRule
class ChoiceValueLookupTestCase(TestCase):
def test_choice_value_matches_values_only(self):
"""choice_value matches the value element of a pair, never the label."""
CustomFieldChoiceSet.objects.create(
name='Choice Set 1',
extra_choices=[['sel1', 'Selection 1'], ['other', 'sel2']],
)
self.assertEqual(CustomFieldChoiceSet.objects.filter(extra_choices__choice_value='sel1').count(), 1)
self.assertEqual(CustomFieldChoiceSet.objects.filter(extra_choices__choice_value='sel2').count(), 0)
def test_choice_value_excludes_null_extra_choices(self):
"""Choice sets without extra choices are excluded without raising."""
CustomFieldChoiceSet.objects.create(
name='Base Only',
base_choices=CustomFieldChoiceSetBaseChoices.IATA,
)
self.assertEqual(CustomFieldChoiceSet.objects.filter(extra_choices__choice_value='sel1').count(), 0)
self.assertEqual(CustomFieldChoiceSet.objects.filter(extra_choices__len=2).count(), 0)
def test_choice_value_not_registered_on_plain_array_fields(self):
"""choice_value is scoped to ChoiceSetField and unavailable on other ArrayFields."""
with self.assertRaises(FieldError):
EventRule.objects.filter(event_types__choice_value='x').exists()

20
netbox/extras/tests/test_models.py
View File

@ -210,6 +210,26 @@ class TableConfigTestCase(TestCase):
# Must not raise TypeError: 'NoneType' object is not iterable
tc.full_clean()
def test_clean_without_object_type(self):
"""full_clean() on an instance missing its object type must raise ValidationError."""
tc = TableConfig(
table=self.table_name,
name='No object type',
columns=['name'],
)
with self.assertRaises(ValidationError):
tc.full_clean()
def test_clean_accepts_columns_none(self):
"""full_clean() must report missing columns rather than raise TypeError."""
tc = TableConfig(
object_type=self.site_ct,
table=self.table_name,
name='No columns',
)
with self.assertRaises(ValidationError):
tc.full_clean()
class TagTestCase(TestCase):

58
netbox/extras/tests/test_views.py
View File

@ -2,6 +2,7 @@ import uuid
from unittest.mock import PropertyMock, patch
from django.contrib.contenttypes.models import ContentType
from django.contrib.messages import get_messages
from django.test import tag
from django.urls import reverse
@ -277,13 +278,16 @@ class SavedFilterTestCase(ViewTestCases.PrimaryObjectViewTestCase):
class TableConfigTestCase(
ViewTestCases.GetObjectViewTestCase,
ViewTestCases.GetObjectChangelogViewTestCase,
ViewTestCases.ListObjectsViewTestCase,
ViewTestCases.CreateObjectViewTestCase,
ViewTestCases.EditObjectViewTestCase,
ViewTestCases.DeleteObjectViewTestCase,
ViewTestCases.ListObjectsViewTestCase,
ViewTestCases.BulkEditObjectsViewTestCase,
ViewTestCases.BulkDeleteObjectsViewTestCase,
):
# Add/Edit/BulkEdit views require an object_type pre-context from the source
# table view, so they are not exercised here.
model = TableConfig
# Selected columns are POSTed as a list but compared as a CSV string
validation_excluded_fields = ('columns',)
@classmethod
def setUpTestData(cls):
@ -320,6 +324,45 @@ class TableConfigTestCase(
)
TableConfig.objects.bulk_create(table_configs)
cls.form_data = {
'name': 'Table Config X',
'object_type': site_type.pk,
'table': 'SiteTable',
'description': 'A table config',
'weight': 100,
'enabled': True,
'shared': True,
'columns': ['name', 'status'],
'ordering': 'name',
}
cls.bulk_edit_data = {
'weight': 999,
}
def _get_url(self, action, instance=None):
url = super()._get_url(action, instance)
# The add view requires the table context from the source table view
if action == 'add':
site_type = ObjectType.objects.get_for_model(Site)
url = f'{url}?object_type={site_type.pk}&table=SiteTable'
return url
def test_add_view_without_table_context(self):
"""A GET without the table context params must redirect to the home page."""
self.add_permissions('extras.add_tableconfig')
response = self.client.get(reverse('extras:tableconfig_add'))
self.assertRedirects(response, reverse('home'))
messages_list = list(get_messages(response.wsgi_request))
self.assertEqual(len(messages_list), 1)
self.assertEqual(str(messages_list[0]), 'Table configurations must be created from an object list view.')
def test_add_view_post_without_table_context(self):
"""A POST without the table context must return form errors rather than a server error."""
self.add_permissions('extras.add_tableconfig')
response = self.client.post(reverse('extras:tableconfig_add'), data={})
self.assertHttpStatus(response, 200)
class BookmarkTestCase(
ViewTestCases.DeleteObjectViewTestCase,
@ -365,6 +408,9 @@ class BookmarkTestCase(
def test_list_objects_anonymous(self):
return
def test_export_objects_anonymous(self):
return
def test_list_objects_with_constrained_permission(self):
return
@ -919,6 +965,9 @@ class SubscriptionTestCase(
login_url = reverse('login')
self.assertRedirects(self.client.get(url), f'{login_url}?next={url}')
def test_export_objects_anonymous(self):
return
def test_list_objects_with_permission(self):
return
@ -1027,6 +1076,9 @@ class NotificationTestCase(
login_url = reverse('login')
self.assertRedirects(self.client.get(url), f'{login_url}?next={url}')
def test_export_objects_anonymous(self):
return
def test_list_objects_with_permission(self):
return

8
netbox/extras/views.py
View File

@ -481,6 +481,14 @@ class TableConfigEditView(SharedObjectViewMixin, generic.ObjectEditView):
form = forms.TableConfigForm
template_name = 'extras/tableconfig_edit.html'
def get(self, request, *args, **kwargs):
# The add view requires the object_type & table parameters from the source table view
if not kwargs and not (request.GET.get('object_type') and request.GET.get('table')):
messages.warning(request, _('Table configurations must be created from an object list view.'))
return redirect('home')
return super().get(request, *args, **kwargs)
def alter_object(self, obj, request, url_args, url_kwargs):
if not obj.pk:
obj.user = request.user

2
netbox/ipam/api/views.py
View File

@ -407,7 +407,7 @@ class AvailableIPAddressesView(AvailableObjectsView):
def get_available_objects(self, parent, limit=None):
# Calculate available IPs within the parent
ip_list = []
for index, ip in enumerate(parent.get_available_ips(), start=1):
for index, ip in enumerate(parent.iter_available_ips(), start=1):
ip_list.append(ip)
if index == limit:
break

6
netbox/ipam/fields.py
View File

@ -42,7 +42,10 @@ class BaseIPField(models.Field):
raise ValidationError(e)
def get_prep_value(self, value):
if not value:
# Membership check; `not value` incorrectly treats the valid zero addresses
# 0.0.0.0 and :: as empty. netaddr objects compare unequal to all three
# sentinels; raw int 0 stays "empty" for backward compatibility.
if value in (None, '', 0):
return None
if isinstance(value, list):
return [str(self.to_python(v)) for v in value]
@ -107,6 +110,7 @@ IPAddressField.register_lookup(lookups.NetContainsOrEquals)
IPAddressField.register_lookup(lookups.NetHost)
IPAddressField.register_lookup(lookups.NetIn)
IPAddressField.register_lookup(lookups.NetHostContained)
IPAddressField.register_lookup(lookups.NetHostBetween)
IPAddressField.register_lookup(lookups.NetFamily)
IPAddressField.register_lookup(lookups.NetMaskLength)
IPAddressField.register_lookup(lookups.Host)

14
netbox/ipam/filtersets.py
View File

@ -1142,19 +1142,17 @@ class VLANFilterSet(PrimaryModelFilterSet, TenancyFilterSet):
def filter_interface_id(self, queryset, name, value):
if value is None:
return queryset.none()
return queryset.filter(
Q(interfaces_as_tagged=value) |
Q(interfaces_as_untagged=value)
).distinct()
tagged = queryset.filter(interfaces_as_tagged=value)
untagged = queryset.filter(interfaces_as_untagged=value)
return queryset.filter(pk__in=tagged.union(untagged).values('pk'))
@extend_schema_field(OpenApiTypes.INT)
def filter_vminterface_id(self, queryset, name, value):
if value is None:
return queryset.none()
return queryset.filter(
Q(vminterfaces_as_tagged=value) |
Q(vminterfaces_as_untagged=value)
).distinct()
tagged = queryset.filter(vminterfaces_as_tagged=value)
untagged = queryset.filter(vminterfaces_as_untagged=value)
return queryset.filter(pk__in=tagged.union(untagged).values('pk'))
@register_filterset

18
netbox/ipam/forms/bulk_create.py
View File

@ -1,10 +1,12 @@
from django import forms
from django.utils.translation import gettext_lazy as _
from utilities.forms.fields import ExpandableIPNetworkField
from ipam.constants import VLAN_VID_MAX, VLAN_VID_MIN
from utilities.forms.fields import ExpandableIPNetworkField, NumericArrayField
__all__ = (
'IPNetworkBulkCreateForm',
'VLANIDBulkCreateForm',
)
@ -15,3 +17,17 @@ class IPNetworkBulkCreateForm(forms.Form):
pattern = ExpandableIPNetworkField(
label=_('Pattern')
)
class VLANIDBulkCreateForm(forms.Form):
pattern = NumericArrayField(
base_field=forms.IntegerField(
min_value=VLAN_VID_MIN,
max_value=VLAN_VID_MAX
),
label=_('VLAN IDs'),
help_text=_(
'Enter VLAN IDs and ranges separated by commas. '
'Example: 100,200-210,3100-3299'
)
)

21
netbox/ipam/forms/model_forms.py
View File

@ -45,6 +45,7 @@ __all__ = (
'ServiceCreateForm',
'ServiceForm',
'ServiceTemplateForm',
'VLANBulkAddForm',
'VLANForm',
'VLANGroupForm',
'VLANTranslationPolicyForm',
@ -727,6 +728,26 @@ class VLANForm(TenancyForm, PrimaryModelForm):
]
class VLANBulkAddForm(VLANForm):
"""
Subclass of VLANForm for bulk creation.
The VID field is inherited but excluded from the visible fieldsets, as it is
populated programmatically by BulkCreateView from the expanded pattern.
"""
fieldsets = (
FieldSet('group', 'site', 'name', 'status', 'role', 'description', 'tags', name=_('VLAN')),
FieldSet('qinq_role', 'qinq_svlan', name=_('Q-in-Q/802.1ad')),
FieldSet('tenant_group', 'tenant', name=_('Tenancy')),
)
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.fields['name'].help_text = _(
'Use {vid} as a placeholder for the VLAN ID. Example: VLAN-{vid}.'
)
class VLANTranslationPolicyForm(PrimaryModelForm):
fieldsets = (

41
netbox/ipam/graphql/filters.py
View File

@ -1,4 +1,3 @@
from datetime import date
from typing import TYPE_CHECKING, Annotated
import netaddr
@ -72,8 +71,8 @@ class ASNFilter(TenancyFilterMixin, PrimaryModelFilter):
@strawberry_django.filter_type(models.ASNRange, lookups=True)
class ASNRangeFilter(TenancyFilterMixin, OrganizationalModelFilter):
name: StrFilterLookup[str] | None = strawberry_django.filter_field()
slug: StrFilterLookup[str] | None = strawberry_django.filter_field()
name: StrFilterLookup | None = strawberry_django.filter_field()
slug: StrFilterLookup | None = strawberry_django.filter_field()
rir: Annotated['RIRFilter', strawberry.lazy('ipam.graphql.filters')] | None = strawberry_django.filter_field()
rir_id: ID | None = strawberry_django.filter_field()
start: Annotated['BigIntegerLookup', strawberry.lazy('netbox.graphql.filter_lookups')] | None = (
@ -86,10 +85,10 @@ class ASNRangeFilter(TenancyFilterMixin, OrganizationalModelFilter):
@strawberry_django.filter_type(models.Aggregate, lookups=True)
class AggregateFilter(ContactFilterMixin, TenancyFilterMixin, PrimaryModelFilter):
prefix: StrFilterLookup[str] | None = strawberry_django.filter_field()
prefix: StrFilterLookup | None = strawberry_django.filter_field()
rir: Annotated['RIRFilter', strawberry.lazy('ipam.graphql.filters')] | None = strawberry_django.filter_field()
rir_id: ID | None = strawberry_django.filter_field()
date_added: DateFilterLookup[date] | None = strawberry_django.filter_field()
date_added: DateFilterLookup | None = strawberry_django.filter_field()
@strawberry_django.filter_field()
def contains(self, value: list[str], prefix) -> Q:
@ -122,14 +121,14 @@ class FHRPGroupFilter(PrimaryModelFilter):
group_id: Annotated['IntegerLookup', strawberry.lazy('netbox.graphql.filter_lookups')] | None = (
strawberry_django.filter_field()
)
name: StrFilterLookup[str] | None = strawberry_django.filter_field()
name: StrFilterLookup | None = strawberry_django.filter_field()
protocol: BaseFilterLookup[Annotated['FHRPGroupProtocolEnum', strawberry.lazy('ipam.graphql.enums')]] | None = (
strawberry_django.filter_field()
)
auth_type: BaseFilterLookup[Annotated['FHRPGroupAuthTypeEnum', strawberry.lazy('ipam.graphql.enums')]] | None = (
strawberry_django.filter_field()
)
auth_key: StrFilterLookup[str] | None = strawberry_django.filter_field()
auth_key: StrFilterLookup | None = strawberry_django.filter_field()
ip_addresses: Annotated['IPAddressFilter', strawberry.lazy('ipam.graphql.filters')] | None = (
strawberry_django.filter_field()
)
@ -140,7 +139,7 @@ class FHRPGroupAssignmentFilter(ChangeLoggedModelFilter):
interface_type: Annotated['ContentTypeFilter', strawberry.lazy('core.graphql.filters')] | None = (
strawberry_django.filter_field()
)
interface_id: StrFilterLookup[str] | None = strawberry_django.filter_field()
interface_id: StrFilterLookup | None = strawberry_django.filter_field()
group: Annotated['FHRPGroupFilter', strawberry.lazy('ipam.graphql.filters')] | None = (
strawberry_django.filter_field()
)
@ -176,7 +175,7 @@ class FHRPGroupAssignmentFilter(ChangeLoggedModelFilter):
@strawberry_django.filter_type(models.IPAddress, lookups=True)
class IPAddressFilter(ContactFilterMixin, TenancyFilterMixin, PrimaryModelFilter):
address: StrFilterLookup[str] | None = strawberry_django.filter_field()
address: StrFilterLookup | None = strawberry_django.filter_field()
vrf: Annotated['VRFFilter', strawberry.lazy('ipam.graphql.filters')] | None = strawberry_django.filter_field()
vrf_id: ID | None = strawberry_django.filter_field()
status: BaseFilterLookup[Annotated['IPAddressStatusEnum', strawberry.lazy('ipam.graphql.enums')]] | None = (
@ -197,7 +196,7 @@ class IPAddressFilter(ContactFilterMixin, TenancyFilterMixin, PrimaryModelFilter
strawberry_django.filter_field()
)
nat_outside_id: ID | None = strawberry_django.filter_field()
dns_name: StrFilterLookup[str] | None = strawberry_django.filter_field()
dns_name: StrFilterLookup | None = strawberry_django.filter_field()
@strawberry_django.filter_field()
def assigned(self, value: bool, prefix) -> Q:
@ -227,8 +226,8 @@ class IPAddressFilter(ContactFilterMixin, TenancyFilterMixin, PrimaryModelFilter
@strawberry_django.filter_type(models.IPRange, lookups=True)
class IPRangeFilter(ContactFilterMixin, TenancyFilterMixin, PrimaryModelFilter):
start_address: StrFilterLookup[str] | None = strawberry_django.filter_field()
end_address: StrFilterLookup[str] | None = strawberry_django.filter_field()
start_address: StrFilterLookup | None = strawberry_django.filter_field()
end_address: StrFilterLookup | None = strawberry_django.filter_field()
size: Annotated['IntegerLookup', strawberry.lazy('netbox.graphql.filter_lookups')] | None = (
strawberry_django.filter_field()
)
@ -281,7 +280,7 @@ class IPRangeFilter(ContactFilterMixin, TenancyFilterMixin, PrimaryModelFilter):
@strawberry_django.filter_type(models.Prefix, lookups=True)
class PrefixFilter(ContactFilterMixin, ScopedFilterMixin, TenancyFilterMixin, PrimaryModelFilter):
prefix: StrFilterLookup[str] | None = strawberry_django.filter_field()
prefix: StrFilterLookup | None = strawberry_django.filter_field()
vrf: Annotated['VRFFilter', strawberry.lazy('ipam.graphql.filters')] | None = strawberry_django.filter_field()
vrf_id: ID | None = strawberry_django.filter_field()
vlan: Annotated['VLANFilter', strawberry.lazy('ipam.graphql.filters')] | None = strawberry_django.filter_field()
@ -330,7 +329,7 @@ class RoleFilter(OrganizationalModelFilter):
@strawberry_django.filter_type(models.RouteTarget, lookups=True)
class RouteTargetFilter(TenancyFilterMixin, PrimaryModelFilter):
name: StrFilterLookup[str] | None = strawberry_django.filter_field()
name: StrFilterLookup | None = strawberry_django.filter_field()
importing_vrfs: Annotated['VRFFilter', strawberry.lazy('ipam.graphql.filters')] | None = (
strawberry_django.filter_field()
)
@ -347,7 +346,7 @@ class RouteTargetFilter(TenancyFilterMixin, PrimaryModelFilter):
@strawberry_django.filter_type(models.Service, lookups=True)
class ServiceFilter(ContactFilterMixin, ServiceFilterMixin, PrimaryModelFilter):
name: StrFilterLookup[str] | None = strawberry_django.filter_field()
name: StrFilterLookup | None = strawberry_django.filter_field()
ip_addresses: Annotated['IPAddressFilter', strawberry.lazy('ipam.graphql.filters')] | None = (
strawberry_django.filter_field()
)
@ -359,7 +358,7 @@ class ServiceFilter(ContactFilterMixin, ServiceFilterMixin, PrimaryModelFilter):
@strawberry_django.filter_type(models.ServiceTemplate, lookups=True)
class ServiceTemplateFilter(ServiceFilterMixin, PrimaryModelFilter):
name: StrFilterLookup[str] | None = strawberry_django.filter_field()
name: StrFilterLookup | None = strawberry_django.filter_field()
@strawberry_django.filter_type(models.VLAN, lookups=True)
@ -373,7 +372,7 @@ class VLANFilter(TenancyFilterMixin, PrimaryModelFilter):
vid: Annotated['IntegerLookup', strawberry.lazy('netbox.graphql.filter_lookups')] | None = (
strawberry_django.filter_field()
)
name: StrFilterLookup[str] | None = strawberry_django.filter_field()
name: StrFilterLookup | None = strawberry_django.filter_field()
status: BaseFilterLookup[Annotated['VLANStatusEnum', strawberry.lazy('ipam.graphql.enums')]] | None = (
strawberry_django.filter_field()
)
@ -404,7 +403,7 @@ class VLANGroupFilter(ScopedFilterMixin, OrganizationalModelFilter):
@strawberry_django.filter_type(models.VLANTranslationPolicy, lookups=True)
class VLANTranslationPolicyFilter(PrimaryModelFilter):
name: StrFilterLookup[str] | None = strawberry_django.filter_field()
name: StrFilterLookup | None = strawberry_django.filter_field()
@strawberry_django.filter_type(models.VLANTranslationRule, lookups=True)
@ -413,7 +412,7 @@ class VLANTranslationRuleFilter(NetBoxModelFilter):
strawberry_django.filter_field()
)
policy_id: ID | None = strawberry_django.filter_field()
description: StrFilterLookup[str] | None = strawberry_django.filter_field()
description: StrFilterLookup | None = strawberry_django.filter_field()
local_vid: Annotated['IntegerLookup', strawberry.lazy('netbox.graphql.filter_lookups')] | None = (
strawberry_django.filter_field()
)
@ -424,8 +423,8 @@ class VLANTranslationRuleFilter(NetBoxModelFilter):
@strawberry_django.filter_type(models.VRF, lookups=True)
class VRFFilter(TenancyFilterMixin, PrimaryModelFilter):
name: StrFilterLookup[str] | None = strawberry_django.filter_field()
rd: StrFilterLookup[str] | None = strawberry_django.filter_field()
name: StrFilterLookup | None = strawberry_django.filter_field()
rd: StrFilterLookup | None = strawberry_django.filter_field()
enforce_unique: FilterLookup[bool] | None = strawberry_django.filter_field()
import_targets: Annotated['RouteTargetFilter', strawberry.lazy('ipam.graphql.filters')] | None = (
strawberry_django.filter_field()

4
netbox/ipam/graphql/types.py
View File

@ -244,7 +244,7 @@ class RouteTargetType(PrimaryObjectType):
@strawberry_django.type(
models.Service,
exclude=('parent_object_type', 'parent_object_id'),
exclude=('_ports_lowest', 'parent_object_type', 'parent_object_id'),
filters=ServiceFilter,
pagination=True
)
@ -264,7 +264,7 @@ class ServiceType(ContactsMixin, PrimaryObjectType):
@strawberry_django.type(
models.ServiceTemplate,
fields='__all__',
exclude=('_ports_lowest',),
filters=ServiceTemplateFilter,
pagination=True
)

35
netbox/ipam/lookups.py
View File

@ -1,3 +1,4 @@
import netaddr
from django.db.models import IntegerField, Lookup, Transform, lookups
@ -99,7 +100,8 @@ class NetHost(Lookup):
if rhs_params:
rhs_params[0] = rhs_params[0].split('/')[0]
params = list(lhs_params) + rhs_params
return f'HOST({lhs}) = {rhs}', params
# Cast to INET so the predicate matches the inet ipam_ipaddress_host index.
return f'CAST(HOST({lhs}) AS INET) = {rhs}', params
class NetIn(Lookup):
@ -120,7 +122,8 @@ class NetIn(Lookup):
without_mask.append(address)
address_in_clause = self.create_in_clause('{} IN ('.format(lhs), len(with_mask))
host_in_clause = self.create_in_clause('HOST({}) IN ('.format(lhs), len(without_mask))
# Cast to INET so the predicate matches the inet ipam_ipaddress_host index.
host_in_clause = self.create_in_clause('CAST(HOST({}) AS INET) IN ('.format(lhs), len(without_mask))
if with_mask and not without_mask:
return address_in_clause, with_mask
@ -156,6 +159,34 @@ class NetHostContained(Lookup):
return f'CAST(HOST({lhs}) AS INET) <<= {rhs}', params
class NetHostBetween(Lookup):
"""
Match host addresses (mask ignored) falling inclusively between two bounds. The left-hand
side is kept as an inet-typed host expression so PostgreSQL can use the host expression
indexes on the IPAM address and range tables; the CAST(HOST(...) AS INET) spelling matches
NetHost/NetIn for consistency (PostgreSQL canonicalizes the INET(HOST(...)) function form
to the same expression).
"""
lookup_name = 'host_between'
def get_prep_lookup(self):
if not isinstance(self.rhs, (list, tuple)) or len(self.rhs) != 2:
raise ValueError('The host_between lookup requires a (lower, upper) pair of bounds')
try:
# Normalize to bare hosts; reject malformed values before they reach SQL.
lower, upper = (netaddr.IPNetwork(str(bound)).ip for bound in self.rhs)
except (netaddr.AddrFormatError, ValueError) as e:
raise ValueError(f'Invalid host_between bound: {e}') from e
if lower.version != upper.version:
raise ValueError('host_between bounds must not mix address families')
return lower, upper
def as_sql(self, qn, connection):
lhs, lhs_params = self.process_lhs(qn, connection)
params = list(lhs_params) + [str(bound) for bound in self.rhs]
return f'CAST(HOST({lhs}) AS INET) BETWEEN %s AND %s', params
class NetFamily(Transform):
lookup_name = 'family'
function = 'FAMILY'

4
netbox/ipam/managers.py
View File

@ -1,10 +1,10 @@
from django.db.models import Manager
from ipam.lookups import Host, Inet
from utilities.querysets import RestrictedQuerySet
from ipam.querysets import IPAddressQuerySet
class IPAddressManager(Manager.from_queryset(RestrictedQuerySet)):
class IPAddressManager(Manager.from_queryset(IPAddressQuerySet)):
def get_queryset(self):
"""

5
netbox/ipam/migrations/0089_default_ordering_indexes.py
View File

@ -32,9 +32,12 @@ class Migration(migrations.Migration):
model_name='role',
index=models.Index(fields=['weight', 'name'], name='ipam_role_weight_01396b_idx'),
),
# Adding a dummy index, to allow a safe migration in case updating users already have services
# with a large number of ports configured (see issue #22273)
# Will get removed in 0091_alter_service_index_and_ordering
migrations.AddIndex(
model_name='service',
index=models.Index(fields=['protocol', 'ports', 'id'], name='ipam_servic_protoco_687d13_idx'),
index=models.Index(fields=['id'], name='ipam_servic_protoco_687d13_idx'),
),
migrations.AddIndex(
model_name='vlangroup',

58
netbox/ipam/migrations/0091_alter_service_index_and_ordering.py Normal file
View File

@ -0,0 +1,58 @@
from django.db import migrations, models
def populate__ports_lowest(apps, schema_editor):
Service = apps.get_model('ipam', 'Service')
ServiceTemplate = apps.get_model('ipam', 'ServiceTemplate')
CHUNK_SIZE = 500
for model in (Service, ServiceTemplate):
chunk = []
qs = model.objects.filter(_ports_lowest__isnull=True).only('id', 'ports', '_ports_lowest')
for obj in qs.iterator(chunk_size=CHUNK_SIZE):
if obj.ports:
obj._ports_lowest = min(obj.ports)
chunk.append(obj)
if len(chunk) >= CHUNK_SIZE:
model.objects.bulk_update(chunk, ['_ports_lowest'])
chunk = []
if chunk:
model.objects.bulk_update(chunk, ['_ports_lowest'])
class Migration(migrations.Migration):
dependencies = [
('ipam', '0090_vlangroup_recompute_total_vlan_ids'),
]
operations = [
migrations.RemoveIndex(
model_name='service',
name='ipam_servic_protoco_687d13_idx',
),
migrations.AddField(
model_name='service',
name='_ports_lowest',
field=models.PositiveIntegerField(blank=True, null=True),
),
migrations.AddField(
model_name='servicetemplate',
name='_ports_lowest',
field=models.PositiveIntegerField(blank=True, null=True),
),
migrations.RunPython(populate__ports_lowest, migrations.RunPython.noop),
migrations.AddIndex(
model_name='service',
index=models.Index(
fields=['protocol', '_ports_lowest', 'id'],
name='ipam_servic_protoco_e2901d_idx'
),
),
migrations.AlterModelOptions(
name='service',
options={
'ordering': ('protocol', '_ports_lowest', 'id')
},
),
]

34
netbox/ipam/migrations/0092_iprange_host_indexes.py Normal file
View File

@ -0,0 +1,34 @@
import django.db.models.functions.comparison
from django.db import migrations, models
import ipam.fields
import ipam.lookups
class Migration(migrations.Migration):
dependencies = [
('ipam', '0091_alter_service_index_and_ordering'),
]
operations = [
migrations.AddIndex(
model_name='iprange',
index=models.Index(
django.db.models.functions.comparison.Cast(
ipam.lookups.Host('start_address'),
output_field=ipam.fields.IPAddressField(),
),
name='ipam_iprange_start_host',
),
),
migrations.AddIndex(
model_name='iprange',
index=models.Index(
django.db.models.functions.comparison.Cast(
ipam.lookups.Host('end_address'),
output_field=ipam.fields.IPAddressField(),
),
name='ipam_iprange_end_host',
),
),
]

2
netbox/ipam/migrations/0091_denormalization_triggers.py → netbox/ipam/migrations/0093_denormalization_triggers.py
View File

@ -10,7 +10,7 @@ from utilities.migration import cached_scope_triggers
class Migration(migrations.Migration):
dependencies = [
('ipam', '0090_vlangroup_recompute_total_vlan_ids'),
('ipam', '0092_iprange_host_indexes'),
# Source tables (dcim_site, dcim_location) must already exist.
('dcim', '0238_ltree_paths'),
]

339
netbox/ipam/models/ip.py
View File

@ -15,7 +15,7 @@ from ipam.constants import *
from ipam.fields import IPAddressField, IPNetworkField
from ipam.lookups import Host
from ipam.managers import IPAddressManager
from ipam.querysets import PrefixQuerySet
from ipam.querysets import IPRangeQuerySet, PrefixQuerySet
from ipam.validators import DNSValidator
from netbox.config import get_config
from netbox.models import OrganizationalModel, PrimaryModel
@ -425,14 +425,63 @@ class Prefix(ContactsMixin, GetAvailablePrefixesMixin, CachedScopeMixin, Primary
return Prefix.objects.filter(prefix__net_contained=str(self.prefix))
return Prefix.objects.filter(prefix__net_contained=str(self.prefix), vrf=self.vrf)
@property
def usable_ip_bounds(self):
"""
Return the first and last IPs considered usable for available-IP calculations.
Pools and IPv4 /31-/32 / IPv6 /127-/128 are fully usable; otherwise IPv4 excludes
network and broadcast, IPv6 excludes the subnet-router anycast address.
"""
network = netaddr.IPNetwork(self.prefix)
family = network.version
first = network.first
last = network.last
mask_length = network.prefixlen
if (
self.is_pool
or (family == 4 and mask_length >= 31)
or (family == 6 and mask_length >= 127)
):
return (
netaddr.IPAddress(first, version=family),
netaddr.IPAddress(last, version=family),
)
if family == 4:
return (
netaddr.IPAddress(first + 1, version=family),
netaddr.IPAddress(last - 1, version=family),
)
return (
netaddr.IPAddress(first + 1, version=family),
netaddr.IPAddress(last, version=family),
)
@property
def usable_size(self):
"""
The number of usable host addresses within the prefix (excludes reserved addresses).
"""
first_ip, last_ip = self.usable_ip_bounds
return int(last_ip) - int(first_ip) + 1
def get_child_ranges(self, **kwargs):
"""
Return all IPRanges within this Prefix and VRF.
"""
# A host BETWEEN over the prefix span uses the ipam_iprange_*_host btree indexes.
prefix = netaddr.IPNetwork(self.prefix)
bounds = (
netaddr.IPAddress(prefix.first, version=prefix.version),
netaddr.IPAddress(prefix.last, version=prefix.version),
)
return IPRange.objects.filter(
vrf=self.vrf,
start_address__net_host_contained=str(self.prefix),
end_address__net_host_contained=str(self.prefix),
start_address__host_between=bounds,
end_address__host_between=bounds,
**kwargs
)
@ -441,52 +490,106 @@ class Prefix(ContactsMixin, GetAvailablePrefixesMixin, CachedScopeMixin, Primary
Return all IPAddresses within this Prefix and VRF. If this Prefix is a container in the global table, return
child IPAddresses belonging to any VRF.
"""
# A host BETWEEN over the prefix span is index-sargable without the <<= containment recheck.
prefix = netaddr.IPNetwork(self.prefix)
bounds = (
netaddr.IPAddress(prefix.first, version=prefix.version),
netaddr.IPAddress(prefix.last, version=prefix.version),
)
if self.vrf is None and self.status == PrefixStatusChoices.STATUS_CONTAINER:
return IPAddress.objects.filter(address__net_host_contained=str(self.prefix))
return IPAddress.objects.filter(address__net_host_contained=str(self.prefix), vrf=self.vrf)
return IPAddress.objects.filter(address__host_between=bounds)
return IPAddress.objects.filter(address__host_between=bounds, vrf=self.vrf)
def get_available_ips(self):
"""
Return all available IPs within this prefix as an IPSet.
"""
prefix = netaddr.IPSet(self.prefix)
child_ips = netaddr.IPSet([
ip.address.ip for ip in self.get_child_ips()
])
child_ranges = netaddr.IPSet([
iprange.range for iprange in self.get_child_ranges().filter(mark_populated=True)
])
available_ips = prefix - child_ips - child_ranges
return netaddr.IPSet(
cidr
for start, end in self._available_intervals()
for cidr in netaddr.iprange_to_cidrs(start, end)
)
# Pool, IPv4 /31-/32 or IPv6 /127-/128 sets are fully usable
if (
self.is_pool
or (self.family == 4 and self.prefix.prefixlen >= 31)
or (self.family == 6 and self.prefix.prefixlen >= 127)
):
return available_ips
def iter_available_ips(self):
"""
Yield the available IPs within this prefix as netaddr.IPAddress objects, in
ascending order. Unlike get_available_ips(), consumption is lazy: stopping
early stops reading from the database.
"""
for start, end in self._available_intervals():
yield from netaddr.iter_iprange(start, end)
if self.family == 4:
# For "normal" IPv4 prefixes, omit first and last addresses
available_ips -= netaddr.IPSet([
netaddr.IPAddress(self.prefix.first),
netaddr.IPAddress(self.prefix.last),
])
else:
# For IPv6 prefixes, omit the Subnet-Router anycast address
# per RFC 4291
available_ips -= netaddr.IPSet([netaddr.IPAddress(self.prefix.first)])
def get_available_ip_count(self):
"""
Return the number of available IPs within the prefix.
"""
first_ip, last_ip = self.usable_ip_bounds
usable_size = int(last_ip) - int(first_ip) + 1
return available_ips
populated_intervals = self.get_child_ranges(mark_populated=True).get_intervals(first_ip, last_ip)
populated_count = sum(int(end) - int(start) + 1 for start, end in populated_intervals)
# Populated ranges already cover the usable span; skip the child-IP count entirely.
if populated_count >= usable_size:
return 0
child_ip_count = (
self.get_child_ips()
.filter(address__host_between=(first_ip, last_ip))
.count_distinct_hosts(exclude_intervals=populated_intervals)
)
return max(usable_size - populated_count - child_ip_count, 0)
def get_ip_usage_summary(self):
"""
Return the available IP count and utilization together as a dict, sharing a
single distinct-host scan. Intended for detail views rendering both values;
list views should call get_utilization() alone, which is cheaper per row.
"""
# Marked-utilized and container utilization need no host scan; delegate.
if self.mark_utilized or self.status == PrefixStatusChoices.STATUS_CONTAINER:
return {
'available_ip_count': self.get_available_ip_count(),
'utilization': self.get_utilization(),
}
first_ip, last_ip = self.usable_ip_bounds
usable_size = int(last_ip) - int(first_ip) + 1
populated_intervals = self.get_child_ranges(mark_populated=True).get_intervals(first_ip, last_ip)
utilized_intervals = self.get_child_ranges(mark_utilized=True).get_intervals()
counts = self.get_child_ips().count_distinct_hosts_pair(
bounds=(first_ip, last_ip),
bounded_exclude=populated_intervals,
total_exclude=utilized_intervals,
)
populated_count = sum(int(end) - int(start) + 1 for start, end in populated_intervals)
utilized_range_count = sum(int(end) - int(start) + 1 for start, end in utilized_intervals)
prefix_size = self._get_utilization_denominator()
return {
'available_ip_count': max(usable_size - populated_count - counts['bounded'], 0),
'utilization': min(float(utilized_range_count + counts['total']) / prefix_size * 100, 100),
}
def get_first_available_ip(self):
"""
Return the first available IP within the prefix (or None).
"""
available_ips = self.get_available_ips()
if not available_ips:
first_ip, last_ip = self.usable_ip_bounds
populated_intervals = self.get_child_ranges(mark_populated=True).get_intervals(first_ip, last_ip)
first_available_ip = self.get_child_ips().first_available_host(
first_ip, last_ip, exclude_intervals=populated_intervals,
)
if first_available_ip is None:
return None
return '{}/{}'.format(next(available_ips.__iter__()), self.prefix.prefixlen)
return f'{first_available_ip}/{self.prefix.prefixlen}'
def get_utilization(self):
"""
@ -504,20 +607,43 @@ class Prefix(ContactsMixin, GetAvailablePrefixesMixin, CachedScopeMixin, Primary
child_prefixes = netaddr.IPSet([p.prefix for p in queryset])
utilization = float(child_prefixes.size) / self.prefix.size * 100
else:
# Compile an IPSet to avoid counting duplicate IPs
child_ips = netaddr.IPSet()
for iprange in self.get_child_ranges().filter(mark_utilized=True):
child_ips.add(iprange.range)
for ip in self.get_child_ips():
child_ips.add(ip.address.ip)
prefix_size = self._get_utilization_denominator()
utilized_intervals = self.get_child_ranges(mark_utilized=True).get_intervals()
utilized_range_count = sum(int(end) - int(start) + 1 for start, end in utilized_intervals)
prefix_size = self.prefix.size
if self.prefix.version == 4 and self.prefix.prefixlen < 31 and not self.is_pool:
prefix_size -= 2
utilization = float(child_ips.size) / prefix_size * 100
# Utilized ranges already saturate the prefix; skip the child-IP count.
if utilized_range_count >= prefix_size:
return 100
child_ip_count = self.get_child_ips().count_distinct_hosts(
exclude_intervals=utilized_intervals,
)
utilization = float(utilized_range_count + child_ip_count) / prefix_size * 100
return min(utilization, 100)
def _available_intervals(self):
"""
Yield the available (start, end) host intervals within the prefix.
"""
first_ip, last_ip = self.usable_ip_bounds
populated_intervals = self.get_child_ranges(mark_populated=True).get_intervals(first_ip, last_ip)
return self.get_child_ips().available_intervals(
first_ip, last_ip, exclude_intervals=populated_intervals,
)
def _get_utilization_denominator(self):
"""
The address count utilization is measured against (IPv4 non-pool prefixes
exclude the network and broadcast addresses; IPv6 uses the full prefix size).
"""
prefix_size = self.prefix.size
if self.prefix.version == 4 and self.prefix.prefixlen < 31 and not self.is_pool:
return prefix_size - 2
return prefix_size
class IPRange(ContactsMixin, PrimaryModel):
"""
@ -576,12 +702,24 @@ class IPRange(ContactsMixin, PrimaryModel):
help_text=_("Report space as fully utilized")
)
objects = IPRangeQuerySet.as_manager()
clone_fields = (
'vrf', 'tenant', 'status', 'role', 'description', 'mark_populated', 'mark_utilized',
)
class Meta:
ordering = (F('vrf').asc(nulls_first=True), 'start_address', 'pk') # (vrf, start_address) may be non-unique
indexes = (
models.Index(
Cast(Host('start_address'), output_field=IPAddressField()),
name='ipam_iprange_start_host',
),
models.Index(
Cast(Host('end_address'), output_field=IPAddressField()),
name='ipam_iprange_end_host',
),
)
verbose_name = _('IP range')
verbose_name_plural = _('IP ranges')
@ -709,40 +847,14 @@ class IPRange(ContactsMixin, PrimaryModel):
def get_status_color(self):
return IPRangeStatusChoices.colors.get(self.status)
def get_child_ips(self):
"""
Return all IPAddresses within this IPRange and VRF.
"""
return IPAddress.objects.filter(
address__gte=self.start_address,
address__lte=self.end_address,
vrf=self.vrf
)
def get_available_ips(self):
"""
Return all available IPs within this range as an IPSet.
"""
if self.mark_populated:
return netaddr.IPSet()
range = netaddr.IPRange(self.start_address.ip, self.end_address.ip)
child_ips = netaddr.IPSet([ip.address.ip for ip in self.get_child_ips()])
return netaddr.IPSet(range) - child_ips
@cached_property
def first_available_ip(self):
"""
Return the first available IP within the range (or None).
"""
available_ips = self.get_available_ips()
if not available_ips:
return None
return self.get_first_available_ip()
return '{}/{}'.format(next(available_ips.__iter__()), self.start_address.prefixlen)
@cached_property
@property
def utilization(self):
"""
Determine the utilization of the range and return it as a percentage.
@ -750,12 +862,79 @@ class IPRange(ContactsMixin, PrimaryModel):
if self.mark_utilized:
return 100
# Compile an IPSet to avoid counting duplicate IPs
child_count = netaddr.IPSet([
ip.address.ip for ip in self.get_child_ips()
]).size
return min(float(self._occupied_host_count) / self.size * 100, 100)
return min(float(child_count) / self.size * 100, 100)
def get_child_ips(self):
"""
Return all IPAddresses within this IPRange and VRF.
"""
return IPAddress.objects.filter(
vrf=self.vrf,
address__host_between=(self.start_address.ip, self.end_address.ip),
)
def get_available_ips(self):
"""
Return all available IPs within this range as an IPSet.
"""
return netaddr.IPSet(
cidr
for start, end in self._available_intervals()
for cidr in netaddr.iprange_to_cidrs(start, end)
)
def iter_available_ips(self):
"""
Yield the available IPs within this range as netaddr.IPAddress objects, in
ascending order. Unlike get_available_ips(), consumption is lazy: stopping
early stops reading from the database.
"""
for start, end in self._available_intervals():
yield from netaddr.iter_iprange(start, end)
def get_available_ip_count(self):
"""
Return the number of available IPs within the range.
"""
if self.mark_populated:
return 0
return max(self.size - self._occupied_host_count, 0)
def get_first_available_ip(self):
"""
Return the first available IP within the range (or None).
"""
if self.mark_populated:
return None
first_available_ip = self.get_child_ips().first_available_host(
self.start_address.ip, self.end_address.ip,
)
if first_available_ip is None:
return None
return f'{first_available_ip}/{self.start_address.prefixlen}'
def _available_intervals(self):
"""
Yield the available (start, end) host intervals within the range.
"""
if self.mark_populated:
return iter(())
return self.get_child_ips().available_intervals(
self.start_address.ip, self.end_address.ip,
)
@cached_property
def _occupied_host_count(self):
"""
The number of distinct occupied hosts within the range, cached for the
lifetime of the instance.
"""
return self.get_child_ips().count_distinct_hosts()
class IPAddress(ContactsMixin, PrimaryModel):
@ -948,10 +1127,10 @@ class IPAddress(ContactsMixin, PrimaryModel):
# Disallow the creation of IPAddresses within an IPRange with mark_populated=True
parent_range_qs = IPRange.objects.filter(
start_address__lte=self.address,
end_address__gte=self.address,
start_address__host__inet__lte=self.address.ip,
end_address__host__inet__gte=self.address.ip,
vrf=self.vrf,
mark_populated=True
mark_populated=True,
)
if not self.pk and (parent_range := parent_range_qs.first()):
raise ValidationError({

17
netbox/ipam/models/services.py
View File

@ -31,10 +31,22 @@ class ServiceBase(models.Model):
),
verbose_name=_('port numbers')
)
_ports_lowest = models.PositiveIntegerField(
null=True,
blank=True,
)
class Meta:
abstract = True
def save(self, *args, **kwargs):
# On saving find the smallest port and save for default ordering
self._ports_lowest = min(self.ports) if self.ports else None
update_fields = kwargs.get('update_fields')
if update_fields is not None and '_ports_lowest' not in update_fields:
kwargs['update_fields'] = list(update_fields) + ['_ports_lowest']
super().save(*args, **kwargs)
def __str__(self):
return f'{self.name} ({self.get_protocol_display()}/{self.port_list})'
@ -74,7 +86,6 @@ class Service(ContactsMixin, ServiceBase, PrimaryModel):
ct_field='parent_object_type',
fk_field='parent_object_id'
)
name = models.CharField(
max_length=100,
verbose_name=_('name')
@ -93,9 +104,9 @@ class Service(ContactsMixin, ServiceBase, PrimaryModel):
class Meta:
indexes = (
models.Index(fields=('protocol', 'ports', 'id')), # Default ordering
models.Index(fields=('protocol', '_ports_lowest', 'id')), # Default ordering
models.Index(fields=('parent_object_type', 'parent_object_id')),
)
ordering = ('protocol', 'ports', 'pk') # (protocol, port) may be non-unique
ordering = ('protocol', '_ports_lowest', 'id')
verbose_name = _('application service')
verbose_name_plural = _('application services')

2
netbox/ipam/models/vlans.py
View File

@ -330,7 +330,7 @@ class VLAN(PrimaryModel):
)
# Check that the VLAN ID is permitted in the assigned group (if any)
if self.group:
if self.group and self.vid is not None:
if not any([self.vid in r for r in self.group.vid_ranges]):
raise ValidationError({
'vid': _(

191
netbox/ipam/querysets.py
View File

@ -1,18 +1,51 @@
import heapq
import netaddr
from django.contrib.contenttypes.models import ContentType
from django.db.models import Count, F, OuterRef, Q, Subquery, Value
from django.db.models.expressions import RawSQL
from django.db.models.functions import NullIf, Round
from django.db.models.functions import Cast, NullIf, Round
from utilities.query import count_related
from utilities.querysets import RestrictedQuerySet
from .fields import IPAddressField
from .lookups import Host
__all__ = (
'ASNRangeQuerySet',
'IPAddressQuerySet',
'IPRangeQuerySet',
'PrefixQuerySet',
'VLANGroupQuerySet',
'VLANQuerySet',
)
# The host portion of an IP address (mask ignored), in the same form as the
# ipam_ipaddress_host expression index.
HOST_ADDRESS = Cast(Host('address'), output_field=IPAddressField())
def _merge_intervals(intervals):
"""
Return the union of (start, end) netaddr.IPAddress intervals, merged and sorted.
"""
if not intervals:
return []
intervals = sorted(intervals)
merged = [intervals[0]]
for start, end in intervals[1:]:
current_start, current_end = merged[-1]
# Adjacency math in int space; netaddr raises at the address-space maximum.
if start.version == current_end.version and int(start) <= int(current_end) + 1:
merged[-1] = (current_start, max(current_end, end))
else:
merged.append((start, end))
return merged
class ASNRangeQuerySet(RestrictedQuerySet):
@ -32,6 +65,162 @@ class ASNRangeQuerySet(RestrictedQuerySet):
return self.annotate(asn_count=Subquery(asns))
class IPAddressQuerySet(RestrictedQuerySet):
def count_distinct_hosts(self, exclude_intervals=()):
"""
Count distinct host addresses, optionally excluding (start, end) netaddr.IPAddress intervals.
"""
queryset = self
for start, end in exclude_intervals:
queryset = queryset.exclude(address__host_between=(start, end))
return queryset.aggregate(count=Count(HOST_ADDRESS, distinct=True))['count']
def count_distinct_hosts_pair(self, bounds, bounded_exclude=(), total_exclude=()):
"""
Return two distinct host counts computed in a single scan, as a dict:
'bounded' counts hosts within the (first_ip, last_ip) bounds excluding the
bounded_exclude intervals; 'total' counts all hosts excluding the
total_exclude intervals. Interval arguments match the output of
IPRangeQuerySet.get_intervals(). Avoids a second scan of the host expression
index when both counts are needed. Use only when both counts are needed (e.g.
Prefix.get_ip_usage_summary()); single-purpose callers should prefer
count_distinct_hosts().
"""
# The deduplicated column is already a bare host; plain comparisons beat
# the host_between lookup here, which would re-wrap it in HOST()::inet.
bounded_q = Q(host_address__range=(str(bounds[0]), str(bounds[1])))
for start, end in bounded_exclude:
bounded_q &= ~Q(host_address__range=(str(start), str(end)))
total_q = Q()
for start, end in total_exclude:
total_q &= ~Q(host_address__range=(str(start), str(end)))
hosts = self.order_by().annotate(host_address=HOST_ADDRESS).values('host_address').distinct()
return hosts.aggregate(
bounded=Count('host_address', filter=bounded_q),
# An empty Q is falsy; fall back to a plain count of all hosts.
total=Count('host_address', filter=total_q or None),
)
def _iter_distinct_hosts(self, first_ip, last_ip, batch_size):
"""
Yield the distinct occupied hosts in [first_ip, last_ip] in ascending order,
fetched in LIMIT batches that resume just past the last seen host. (A
server-side cursor is unsuitable here: on autocommit connections Django
declares it WITH HOLD, which materializes the full result at DECLARE.)
"""
resume = first_ip
while True:
# order_by() first clears the default ordering, which would otherwise
# leak into SELECT and break distinct().
hosts = list(
self.filter(address__host_between=(resume, last_ip))
.order_by()
.annotate(host_address=HOST_ADDRESS)
.values_list('host_address', flat=True)
.distinct()
.order_by('host_address')[:batch_size]
)
for host in hosts:
yield host.ip
if len(hosts) < batch_size:
return
last_host = hosts[-1].ip
if int(last_host) >= int(last_ip):
return
resume = netaddr.IPAddress(int(last_host) + 1, version=last_host.version)
def available_intervals(self, first_ip, last_ip, exclude_intervals=(), batch_size=5000):
"""
Yield the unoccupied (start, end) netaddr.IPAddress intervals (inclusive)
within [first_ip, last_ip], in ascending order. exclude_intervals are
(start, end) netaddr.IPAddress pairs; they are merged and sorted internally,
intervals of a foreign address family are ignored, and addresses they cover
count as occupied. Consumption is lazy: a caller that stops early stops
fetching host batches.
"""
if batch_size < 1:
raise ValueError('batch_size must be greater than zero')
first_int, last_int = int(first_ip), int(last_ip)
version = first_ip.version
if first_int > last_int:
return
# Normalize: the sweep below requires sorted, non-overlapping, same-family intervals.
exclude_intervals = _merge_intervals([
(start, end)
for start, end in exclude_intervals
if start.version == end.version == version
])
intervals = [(int(start), int(end)) for start, end in exclude_intervals]
# Fast path: one merged excluded interval covers the entire span.
if intervals and intervals[0][0] <= first_int and intervals[0][1] >= last_int:
return
hosts = (
(int(host), int(host))
for host in self._iter_distinct_hosts(first_ip, last_ip, batch_size)
)
candidate = first_int
# Ties on `start` are harmless; the sweep handles overlapping intervals.
for start, end in heapq.merge(intervals, hosts):
if end < candidate:
continue
if start > candidate:
yield (
netaddr.IPAddress(candidate, version=version),
netaddr.IPAddress(min(start - 1, last_int), version=version),
)
candidate = max(candidate, end + 1)
if candidate > last_int:
return
if candidate <= last_int:
yield (
netaddr.IPAddress(candidate, version=version),
netaddr.IPAddress(last_int, version=version),
)
def first_available_host(self, first_ip, last_ip, exclude_intervals=()):
"""
Return the first host in [first_ip, last_ip] neither present nor in an excluded interval (or None).
"""
interval = next(self.available_intervals(first_ip, last_ip, exclude_intervals), None)
return interval[0] if interval else None
class IPRangeQuerySet(RestrictedQuerySet):
def get_intervals(self, first_ip=None, last_ip=None):
"""
Return ranges as merged (start, end) netaddr.IPAddress intervals, optionally clipped to the bounds.
"""
intervals = []
# order_by() clears the default ordering; _merge_intervals() sorts anyway.
for start_address, end_address in self.order_by().values_list('start_address', 'end_address'):
start, end = start_address.ip, end_address.ip
if first_ip is not None:
if end < first_ip:
continue
start = max(start, first_ip)
if last_ip is not None:
if start > last_ip:
continue
end = min(end, last_ip)
intervals.append((start, end))
return _merge_intervals(intervals)
class PrefixQuerySet(RestrictedQuerySet):
def annotate_hierarchy(self):

29
netbox/ipam/tests/test_fields.py Normal file
View File

@ -0,0 +1,29 @@
from django.test import TestCase
from netaddr import IPAddress
from ipam.fields import IPAddressField, IPNetworkField
class BaseIPFieldTestCase(TestCase):
"""
Regression coverage for BaseIPField.get_prep_value() — zero addresses such as
0.0.0.0 and :: are valid hosts and must not be treated as empty values.
"""
def test_get_prep_value_accepts_ipv4_zero_address(self):
# Regression: 0.0.0.0 is a valid host, not an empty value.
self.assertEqual(IPAddressField().get_prep_value(IPAddress('0.0.0.0')), '0.0.0.0')
def test_get_prep_value_accepts_ipv6_zero_address(self):
# Regression: :: is a valid host, not an empty value.
self.assertEqual(IPAddressField().get_prep_value(IPAddress('::')), '::')
def test_get_prep_value_passes_through_empty(self):
self.assertIsNone(IPNetworkField().get_prep_value(None))
self.assertIsNone(IPAddressField().get_prep_value(''))
def test_get_prep_value_preserves_raw_zero_as_empty(self):
# Raw int 0 is preserved as the legacy "empty" sentinel; Django's ORM never
# passes it directly, but the previous `not value` check returned None for it.
self.assertIsNone(IPAddressField().get_prep_value(0))
self.assertIsNone(IPNetworkField().get_prep_value(0))

41
netbox/ipam/tests/test_filtersets.py
View File

@ -4,7 +4,7 @@ from django.test import TestCase
from netaddr import IPNetwork
from circuits.models import Provider
from dcim.choices import InterfaceTypeChoices
from dcim.choices import InterfaceModeChoices, InterfaceTypeChoices
from dcim.models import Device, DeviceRole, DeviceType, Interface, Location, Manufacturer, Rack, Region, Site, SiteGroup
from ipam.choices import *
from ipam.filtersets import *
@ -2206,11 +2206,50 @@ class VLANTestCase(TestCase, ChangeLoggedFilterSetTests):
params = {'interface_id': interface_id}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 1)
# An interface untagged on one VLAN and tagged on a different VLAN should return both (UNION across paths)
vlans = self.queryset.all()[:2]
interface = Interface.objects.create(
device=Device.objects.first(),
name='Interface X',
type=InterfaceTypeChoices.TYPE_1GE_FIXED,
mode=InterfaceModeChoices.MODE_TAGGED,
untagged_vlan=vlans[0],
)
interface.tagged_vlans.add(vlans[1])
params = {'interface_id': interface.pk}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)
# A VLAN that is both untagged and tagged on the same interface should be returned only once (deduplication)
interface.tagged_vlans.add(vlans[0])
params = {'interface_id': interface.pk}
qs = self.filterset(params, self.queryset).qs
self.assertEqual(qs.count(), 2)
self.assertEqual(len(qs), len(set(qs.values_list('pk', flat=True))))
def test_vminterface(self):
vminterface_id = VMInterface.objects.first().pk
params = {'vminterface_id': vminterface_id}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 1)
# A VM interface untagged on one VLAN and tagged on a different VLAN should return both (UNION across paths)
vlans = self.queryset.all()[:2]
vminterface = VMInterface.objects.create(
virtual_machine=VirtualMachine.objects.first(),
name='VM Interface X',
mode=InterfaceModeChoices.MODE_TAGGED,
untagged_vlan=vlans[0],
)
vminterface.tagged_vlans.add(vlans[1])
params = {'vminterface_id': vminterface.pk}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)
# A VLAN that is both untagged and tagged on the same interface should be returned only once (deduplication)
vminterface.tagged_vlans.add(vlans[0])
params = {'vminterface_id': vminterface.pk}
qs = self.filterset(params, self.queryset).qs
self.assertEqual(qs.count(), 2)
self.assertEqual(len(qs), len(set(qs.values_list('pk', flat=True))))
def test_qinq_role(self):
params = {'qinq_role': [VLANQinQRoleChoices.ROLE_SERVICE, VLANQinQRoleChoices.ROLE_CUSTOMER]}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 6)

30
netbox/ipam/tests/test_forms.py
View File

@ -3,7 +3,7 @@ from django.test import TestCase
from dcim.constants import InterfaceTypeChoices
from dcim.models import Device, DeviceRole, DeviceType, Interface, Location, Manufacturer, Region, Site, SiteGroup
from ipam.forms import PrefixForm
from ipam.forms import PrefixForm, VLANIDBulkCreateForm
from ipam.forms.bulk_import import IPAddressImportForm
@ -96,3 +96,31 @@ class IPAddressImportFormTestCase(TestCase):
self.device.refresh_from_db()
self.assertEqual(self.device.oob_ip, ip1, "OOB IP was incorrectly cleared by a row with is_oob=False")
class VLANFormTestCase(TestCase):
def test_bulk_create_valid_patterns(self):
"""Single values, ranges, and combinations expand to sorted, deduplicated VLAN IDs."""
cases = (
('100', [100]),
('5,10,20', [5, 10, 20]),
('10-20', list(range(10, 21))),
('1,10-20,300-305', [1, *range(10, 21), *range(300, 306)]),
(' 5 , 7 - 9 ', [5, 7, 8, 9]),
('5,5,4-6', [4, 5, 6]),
)
for pattern, expected in cases:
with self.subTest(pattern=pattern):
form = VLANIDBulkCreateForm({'pattern': pattern})
self.assertTrue(form.is_valid(), form.errors)
self.assertEqual(form.cleaned_data['pattern'], expected)
def test_bulk_create_invalid_patterns(self):
"""Malformed, descending, or out-of-range patterns are rejected with an error on the pattern field."""
cases = ('', 'abc', '10,abc', '20-10', '10-', '5,', '-5', '0', '4095')
for pattern in cases:
with self.subTest(pattern=pattern):
form = VLANIDBulkCreateForm({'pattern': pattern})
self.assertFalse(form.is_valid())
self.assertIn('pattern', form.errors)

135
netbox/ipam/tests/test_lookups.py
View File

@ -1,7 +1,9 @@
import netaddr
from django.db.backends.postgresql.psycopg_any import NumericRange
from django.test import TestCase
from netaddr import IPNetwork
from ipam.models import VLANGroup
from ipam.models import IPAddress, VLANGroup
class VLANGroupRangeContainsLookupTestCase(TestCase):
@ -65,3 +67,134 @@ class VLANGroupRangeContainsLookupTestCase(TestCase):
specific condition.
"""
self.assertFalse(VLANGroup.objects.filter(pk=self.g_empty.pk, vid_ranges__range_contains=1).exists())
class IPAddressHostBetweenLookupTestCase(TestCase):
@classmethod
def setUpTestData(cls):
IPAddress.objects.bulk_create((
IPAddress(address=IPNetwork('192.0.2.0/24')),
IPAddress(address=IPNetwork('192.0.2.1/24')),
IPAddress(address=IPNetwork('192.0.2.5/32')),
IPAddress(address=IPNetwork('192.0.2.10/25')),
IPAddress(address=IPNetwork('192.0.2.11/24')),
IPAddress(address=IPNetwork('2001:db8::1/64')),
IPAddress(address=IPNetwork('2001:db8::5/128')),
IPAddress(address=IPNetwork('2001:db8::10/64')),
))
def test_ipv4_boundaries_inclusive(self):
"""
Tests that both bounds are included and hosts outside the window are excluded.
"""
queryset = IPAddress.objects.filter(
address__host_between=(netaddr.IPAddress('192.0.2.1'), netaddr.IPAddress('192.0.2.10'))
)
self.assertEqual(
sorted(str(ip.address) for ip in queryset),
['192.0.2.1/24', '192.0.2.10/25', '192.0.2.5/32'],
)
def test_mask_insensitive(self):
"""
Tests that hosts match regardless of their mask length.
"""
queryset = IPAddress.objects.filter(
address__host_between=(netaddr.IPAddress('192.0.2.5'), netaddr.IPAddress('192.0.2.5'))
)
self.assertEqual(queryset.count(), 1)
def test_ipv6(self):
"""
Tests that IPv6 hosts filter by host portion.
"""
queryset = IPAddress.objects.filter(
address__host_between=(netaddr.IPAddress('2001:db8::1'), netaddr.IPAddress('2001:db8::5'))
)
self.assertEqual(queryset.count(), 2)
def test_bounds_mask_stripped(self):
"""
Tests that bounds supplied with a mask compare by host portion only.
"""
queryset = IPAddress.objects.filter(
address__host_between=(IPNetwork('192.0.2.1/24'), IPNetwork('192.0.2.10/24'))
)
self.assertEqual(queryset.count(), 3)
def test_invalid_bounds_raise(self):
"""
Tests that a bounds value which is not a two-item pair raises ValueError.
"""
with self.assertRaises(ValueError):
IPAddress.objects.filter(address__host_between=(netaddr.IPAddress('192.0.2.1'),))
def test_invalid_bound_value_raises(self):
"""
Tests that a bound which is not a valid IP address raises ValueError.
"""
with self.assertRaises(ValueError):
IPAddress.objects.filter(address__host_between=('invalid', netaddr.IPAddress('192.0.2.10')))
def test_mixed_family_bounds_raise(self):
"""
Tests that bounds from different address families raise ValueError.
"""
with self.assertRaises(ValueError):
IPAddress.objects.filter(
address__host_between=(netaddr.IPAddress('192.0.2.1'), netaddr.IPAddress('2001:db8::1'))
)
def test_sql_uses_cast_host_expression(self):
"""
Tests that the compiled SQL matches the ipam_ipaddress_host index expression.
"""
queryset = IPAddress.objects.filter(
address__host_between=(netaddr.IPAddress('192.0.2.1'), netaddr.IPAddress('192.0.2.10'))
)
self.assertIn('CAST(HOST(', str(queryset.query))
class IPAddressNetLookupsTestCase(TestCase):
@classmethod
def setUpTestData(cls):
IPAddress.objects.bulk_create((
IPAddress(address='10.0.0.1/24'),
IPAddress(address='10.0.0.2/24'),
IPAddress(address='10.0.0.1/25'), # Same host as the first, different mask
IPAddress(address='2001:db8::1/64'),
))
def test_net_host_matches_host_ignoring_mask(self):
"""net_host matches every address whose host portion equals the value."""
qs = IPAddress.objects.filter(address__net_host='10.0.0.1')
self.assertEqual(qs.count(), 2)
def test_net_host_predicate_is_inet_typed(self):
"""net_host casts the host expression to inet so the inet host index applies."""
sql = str(IPAddress.objects.filter(address__net_host='10.0.0.1').query)
self.assertIn('CAST(HOST(', sql)
self.assertIn('AS INET) =', sql)
def test_net_in_without_mask(self):
"""net_in matches host values supplied without a mask."""
qs = IPAddress.objects.filter(address__net_in=['10.0.0.1', '10.0.0.2'])
self.assertEqual(qs.count(), 3)
def test_net_in_with_mask(self):
"""net_in matches an exact address/mask value."""
qs = IPAddress.objects.filter(address__net_in=['10.0.0.1/25'])
self.assertEqual(qs.count(), 1)
def test_net_in_normalizes_ipv6(self):
"""net_in matches an expanded IPv6 form against the canonical host value."""
qs = IPAddress.objects.filter(
address__net_in=['2001:0db8:0000:0000:0000:0000:0000:0001']
)
self.assertEqual(qs.count(), 1)
def test_net_in_predicate_is_inet_typed(self):
"""net_in casts the host expression to inet so the inet host index applies."""
sql = str(IPAddress.objects.filter(address__net_in=['10.0.0.1']).query)
self.assertIn('CAST(HOST(', sql)
self.assertIn('AS INET) IN', sql)

1003
netbox/ipam/tests/test_models.py
View File

File diff suppressed because it is too large Load Diff

344
netbox/ipam/tests/test_querysets.py Normal file
View File

@ -0,0 +1,344 @@
import netaddr
from django.test import TestCase
from netaddr import IPNetwork
from ipam.models import IPAddress, IPRange
class IPAddressQuerySetTestCase(TestCase):
@classmethod
def setUpTestData(cls):
IPAddress.objects.bulk_create((
IPAddress(address=IPNetwork('192.0.2.1/24')),
IPAddress(address=IPNetwork('192.0.2.1/32')),
IPAddress(address=IPNetwork('192.0.2.2/24')),
))
def test_count_distinct_hosts(self):
"""
Tests that duplicate hosts with different masks are counted once.
"""
self.assertEqual(IPAddress.objects.count_distinct_hosts(), 2)
def test_count_distinct_hosts_empty(self):
"""
Tests that an empty queryset counts zero hosts.
"""
self.assertEqual(IPAddress.objects.none().count_distinct_hosts(), 0)
def test_count_distinct_hosts_exclude_intervals(self):
"""
Tests that hosts covered by an excluded interval are not counted.
"""
interval = (netaddr.IPAddress('192.0.2.1'), netaddr.IPAddress('192.0.2.1'))
self.assertEqual(IPAddress.objects.count_distinct_hosts(exclude_intervals=[interval]), 1)
def test_count_distinct_hosts_pair(self):
"""
Tests that the bounded and total distinct host counts are computed correctly.
"""
counts = IPAddress.objects.count_distinct_hosts_pair(
bounds=(netaddr.IPAddress('192.0.2.2'), netaddr.IPAddress('192.0.2.10')),
bounded_exclude=[(netaddr.IPAddress('192.0.2.2'), netaddr.IPAddress('192.0.2.2'))],
total_exclude=[(netaddr.IPAddress('192.0.2.1'), netaddr.IPAddress('192.0.2.1'))],
)
self.assertEqual(counts, {'bounded': 0, 'total': 1})
def test_count_distinct_hosts_pair_no_excludes(self):
"""
Tests that both counts dedupe hosts and respect the bounds without excludes.
"""
counts = IPAddress.objects.count_distinct_hosts_pair(
bounds=(netaddr.IPAddress('192.0.2.2'), netaddr.IPAddress('192.0.2.10')),
)
self.assertEqual(counts, {'bounded': 1, 'total': 2})
def test_first_available_host(self):
"""
Tests that occupied hosts and excluded intervals are skipped, including hosts behind the sweep.
"""
interval = (netaddr.IPAddress('192.0.2.1'), netaddr.IPAddress('192.0.2.5'))
self.assertEqual(
IPAddress.objects.first_available_host(
netaddr.IPAddress('192.0.2.1'), netaddr.IPAddress('192.0.2.10'), exclude_intervals=[interval]
),
netaddr.IPAddress('192.0.2.6'),
)
def test_first_available_host_inverted_bounds(self):
"""
Tests that an inverted bounds pair yields None.
"""
self.assertIsNone(
IPAddress.objects.first_available_host(netaddr.IPAddress('192.0.2.10'), netaddr.IPAddress('192.0.2.5'))
)
def test_available_intervals(self):
"""
Tests that gaps around occupied hosts and excluded intervals are yielded in order.
"""
interval = (netaddr.IPAddress('192.0.2.5'), netaddr.IPAddress('192.0.2.6'))
self.assertEqual(
list(IPAddress.objects.available_intervals(
netaddr.IPAddress('192.0.2.1'), netaddr.IPAddress('192.0.2.10'), exclude_intervals=[interval]
)),
[
(netaddr.IPAddress('192.0.2.3'), netaddr.IPAddress('192.0.2.4')),
(netaddr.IPAddress('192.0.2.7'), netaddr.IPAddress('192.0.2.10')),
],
)
def test_available_intervals_leading_gap(self):
"""
Tests that the gap before the first occupied host is yielded.
"""
self.assertEqual(
list(IPAddress.objects.available_intervals(
netaddr.IPAddress('192.0.2.0'), netaddr.IPAddress('192.0.2.2')
)),
[(netaddr.IPAddress('192.0.2.0'), netaddr.IPAddress('192.0.2.0'))],
)
def test_available_intervals_empty_queryset(self):
"""
Tests that an empty queryset yields the full span.
"""
self.assertEqual(
list(IPAddress.objects.none().available_intervals(
netaddr.IPAddress('192.0.2.1'), netaddr.IPAddress('192.0.2.3')
)),
[(netaddr.IPAddress('192.0.2.1'), netaddr.IPAddress('192.0.2.3'))],
)
def test_available_intervals_inverted_bounds(self):
"""
Tests that an inverted bounds pair yields nothing.
"""
self.assertEqual(
list(IPAddress.objects.available_intervals(
netaddr.IPAddress('192.0.2.10'), netaddr.IPAddress('192.0.2.5')
)),
[],
)
def test_available_intervals_fully_excluded(self):
"""
Tests that a span covered by an excluded interval yields nothing.
"""
interval = (netaddr.IPAddress('192.0.2.0'), netaddr.IPAddress('192.0.2.20'))
self.assertEqual(
list(IPAddress.objects.available_intervals(
netaddr.IPAddress('192.0.2.1'), netaddr.IPAddress('192.0.2.10'), exclude_intervals=[interval]
)),
[],
)
def test_available_intervals_mixed_family_exclude(self):
"""
Tests that an exclude interval spanning address families is ignored.
"""
interval = (netaddr.IPAddress('192.0.2.5'), netaddr.IPAddress('2001:db8::5'))
self.assertEqual(
list(IPAddress.objects.available_intervals(
netaddr.IPAddress('192.0.2.1'), netaddr.IPAddress('192.0.2.10'), exclude_intervals=[interval]
)),
[(netaddr.IPAddress('192.0.2.3'), netaddr.IPAddress('192.0.2.10'))],
)
def test_available_intervals_invalid_batch_size(self):
"""
Tests that a non-positive batch size raises ValueError.
"""
intervals = IPAddress.objects.available_intervals(
netaddr.IPAddress('192.0.2.1'), netaddr.IPAddress('192.0.2.10'), batch_size=0
)
with self.assertRaises(ValueError):
next(intervals)
def test_available_intervals_first_interval_single_query(self):
"""
Tests that consuming only the first interval issues a single batch query.
"""
IPAddress.objects.bulk_create((
IPAddress(address=IPNetwork('192.0.2.12/24')),
IPAddress(address=IPNetwork('192.0.2.14/24')),
IPAddress(address=IPNetwork('192.0.2.16/24')),
))
intervals = IPAddress.objects.available_intervals(
netaddr.IPAddress('192.0.2.10'), netaddr.IPAddress('192.0.2.20'), batch_size=1
)
with self.assertNumQueries(1):
self.assertEqual(
next(intervals),
(netaddr.IPAddress('192.0.2.10'), netaddr.IPAddress('192.0.2.11')),
)
def test_available_intervals_unsorted_exclude_intervals(self):
"""
Tests that unsorted, overlapping exclude intervals are normalized internally.
"""
intervals = list(IPAddress.objects.none().available_intervals(
netaddr.IPAddress('192.0.2.1'),
netaddr.IPAddress('192.0.2.40'),
exclude_intervals=[
(netaddr.IPAddress('192.0.2.20'), netaddr.IPAddress('192.0.2.30')),
(netaddr.IPAddress('192.0.2.1'), netaddr.IPAddress('192.0.2.10')),
(netaddr.IPAddress('192.0.2.25'), netaddr.IPAddress('192.0.2.30')),
],
))
self.assertEqual(intervals, [
(netaddr.IPAddress('192.0.2.11'), netaddr.IPAddress('192.0.2.19')),
(netaddr.IPAddress('192.0.2.31'), netaddr.IPAddress('192.0.2.40')),
])
def test_available_intervals_batching(self):
"""
Tests that gaps spanning multiple fetch batches are yielded completely and in order.
"""
IPAddress.objects.bulk_create(
IPAddress(address=IPNetwork(f'192.0.3.{i}/24')) for i in range(2, 82, 2)
)
expected = [
(netaddr.IPAddress(f'192.0.3.{i}'), netaddr.IPAddress(f'192.0.3.{i}'))
for i in range(1, 83, 2)
]
self.assertEqual(
list(IPAddress.objects.available_intervals(
netaddr.IPAddress('192.0.3.1'), netaddr.IPAddress('192.0.3.81'), batch_size=8
)),
expected,
)
def test_iter_distinct_hosts_stops_at_upper_bound(self):
"""
Tests that batch resumption stops once the last fetched host reaches the upper bound.
"""
IPAddress.objects.bulk_create(
IPAddress(address=IPNetwork(f'192.0.4.{i}/24')) for i in (2, 4)
)
self.assertEqual(
list(IPAddress.objects.all()._iter_distinct_hosts(
netaddr.IPAddress('192.0.4.2'), netaddr.IPAddress('192.0.4.4'), batch_size=1
)),
[netaddr.IPAddress('192.0.4.2'), netaddr.IPAddress('192.0.4.4')],
)
def test_available_intervals_batch_size_one(self):
"""
Tests that fetching one host per batch still terminates and yields every gap.
"""
IPAddress.objects.bulk_create(
IPAddress(address=IPNetwork(f'192.0.3.{i}/24')) for i in (2, 3, 5)
)
self.assertEqual(
list(IPAddress.objects.available_intervals(
netaddr.IPAddress('192.0.3.1'), netaddr.IPAddress('192.0.3.6'), batch_size=1
)),
[
(netaddr.IPAddress('192.0.3.1'), netaddr.IPAddress('192.0.3.1')),
(netaddr.IPAddress('192.0.3.4'), netaddr.IPAddress('192.0.3.4')),
(netaddr.IPAddress('192.0.3.6'), netaddr.IPAddress('192.0.3.6')),
],
)
class IPRangeQuerySetTestCase(TestCase):
@classmethod
def setUpTestData(cls):
IPRange.objects.bulk_create((
IPRange(start_address=IPNetwork('192.0.2.10/24'), end_address=IPNetwork('192.0.2.19/24'), size=10),
IPRange(start_address=IPNetwork('192.0.2.15/24'), end_address=IPNetwork('192.0.2.24/24'), size=10),
IPRange(start_address=IPNetwork('192.0.2.40/24'), end_address=IPNetwork('192.0.2.49/24'), size=10),
))
def test_get_intervals_merges_overlaps(self):
"""
Tests that overlapping ranges merge and disjoint ranges stay separate.
"""
self.assertEqual(
IPRange.objects.get_intervals(),
[
(netaddr.IPAddress('192.0.2.10'), netaddr.IPAddress('192.0.2.24')),
(netaddr.IPAddress('192.0.2.40'), netaddr.IPAddress('192.0.2.49')),
],
)
def test_get_intervals_clips_to_bounds(self):
"""
Tests that ranges are clipped to the bounds and out-of-bounds ranges are dropped.
"""
self.assertEqual(
IPRange.objects.get_intervals(netaddr.IPAddress('192.0.2.20'), netaddr.IPAddress('192.0.2.30')),
[(netaddr.IPAddress('192.0.2.20'), netaddr.IPAddress('192.0.2.24'))],
)
def test_get_intervals_drops_ranges_below_bounds(self):
"""
Tests that ranges entirely below the lower bound are dropped.
"""
self.assertEqual(
IPRange.objects.get_intervals(netaddr.IPAddress('192.0.2.30'), netaddr.IPAddress('192.0.2.60')),
[(netaddr.IPAddress('192.0.2.40'), netaddr.IPAddress('192.0.2.49'))],
)
def test_get_intervals_drops_ranges_above_bounds(self):
"""
Tests that ranges entirely above the upper bound are dropped.
"""
self.assertEqual(
IPRange.objects.get_intervals(netaddr.IPAddress('192.0.2.0'), netaddr.IPAddress('192.0.2.30')),
[(netaddr.IPAddress('192.0.2.10'), netaddr.IPAddress('192.0.2.24'))],
)
def test_get_intervals_clips_to_upper_bound(self):
"""
Tests that a range straddling the upper bound is clipped to it.
"""
self.assertEqual(
IPRange.objects.get_intervals(netaddr.IPAddress('192.0.2.0'), netaddr.IPAddress('192.0.2.15')),
[(netaddr.IPAddress('192.0.2.10'), netaddr.IPAddress('192.0.2.15'))],
)
def test_get_intervals_mixed_families(self):
"""
Tests that int-adjacent intervals of different address families are not merged.
"""
IPRange.objects.bulk_create((
IPRange(
start_address=IPNetwork('255.255.255.254/32'),
end_address=IPNetwork('255.255.255.255/32'),
size=2,
),
IPRange(start_address=IPNetwork('::1/128'), end_address=IPNetwork('::2/128'), size=2),
))
self.assertEqual(
IPRange.objects.get_intervals(),
[
(netaddr.IPAddress('192.0.2.10'), netaddr.IPAddress('192.0.2.24')),
(netaddr.IPAddress('192.0.2.40'), netaddr.IPAddress('192.0.2.49')),
(netaddr.IPAddress('255.255.255.254'), netaddr.IPAddress('255.255.255.255')),
(netaddr.IPAddress('::1'), netaddr.IPAddress('::2')),
],
)
def test_get_intervals_ipv6(self):
"""
Tests that IPv6 ranges merge and clip by host address.
"""
IPRange.objects.create(
start_address=IPNetwork('2001:db8::10/64'),
end_address=IPNetwork('2001:db8::1f/64'),
)
IPRange.objects.create(
start_address=IPNetwork('2001:db8::18/64'),
end_address=IPNetwork('2001:db8::2f/64'),
)
self.assertEqual(
IPRange.objects.get_intervals(netaddr.IPAddress('2001:db8::'), netaddr.IPAddress('2001:db8::ffff')),
[(netaddr.IPAddress('2001:db8::10'), netaddr.IPAddress('2001:db8::2f'))],
)

475
netbox/ipam/tests/test_views.py
View File

@ -1,6 +1,8 @@
import datetime
from django.contrib.contenttypes.models import ContentType
from django.db.backends.postgresql.psycopg_any import NumericRange
from django.test import RequestFactory
from django.urls import reverse
from netaddr import IPNetwork
@ -8,8 +10,10 @@ from core.choices import ObjectChangeActionChoices
from core.models import ObjectChange, ObjectType
from dcim.constants import InterfaceTypeChoices
from dcim.models import Device, DeviceRole, DeviceType, Interface, Manufacturer, Site
from extras.models import SavedFilter
from ipam.choices import *
from ipam.models import *
from ipam.views import AggregatePrefixesView
from netbox.choices import CSVDelimiterChoices, ImportFormatChoices
from tenancy.models import Tenant
from users.models import ObjectPermission
@ -353,6 +357,101 @@ class AggregateTestCase(ViewTestCases.PrimaryObjectViewTestCase):
url = reverse('ipam:aggregate_prefixes', kwargs={'pk': aggregate.pk})
self.assertHttpStatus(self.client.get(url), 200)
def test_aggregate_prefixes_filter_suppresses_available_prefixes(self):
self.add_permissions('ipam.view_aggregate', 'ipam.view_prefix')
tenants = (
Tenant(name='Aggregate Tenant 1', slug='aggregate-tenant-1'),
Tenant(name='Aggregate Tenant 2', slug='aggregate-tenant-2'),
)
Tenant.objects.bulk_create(tenants)
aggregate = Aggregate.objects.create(
prefix=IPNetwork('203.0.113.0/24'),
rir=RIR.objects.first()
)
prefixes = (
Prefix(prefix=IPNetwork('203.0.113.0/26'), tenant=tenants[0]),
Prefix(prefix=IPNetwork('203.0.113.64/26'), tenant=tenants[1]),
)
Prefix.objects.bulk_create(prefixes)
url = reverse('ipam:aggregate_prefixes', kwargs={'pk': aggregate.pk})
response = self.client.get(url, {'tenant_id': tenants[0].pk})
self.assertHttpStatus(response, 200)
self.assertEqual(len(response.context['table'].data), 1)
self.assertContains(response, '203.0.113.0/26')
self.assertNotContains(response, '203.0.113.64/26')
def test_aggregate_prefixes_saved_filter(self):
self.add_permissions('ipam.view_aggregate', 'ipam.view_prefix')
tenants = (
Tenant(name='Aggregate Saved Tenant 1', slug='aggregate-saved-tenant-1'),
Tenant(name='Aggregate Saved Tenant 2', slug='aggregate-saved-tenant-2'),
)
Tenant.objects.bulk_create(tenants)
aggregate = Aggregate.objects.create(
prefix=IPNetwork('203.0.114.0/24'),
rir=RIR.objects.first()
)
prefixes = (
Prefix(prefix=IPNetwork('203.0.114.0/26'), tenant=tenants[0]),
Prefix(prefix=IPNetwork('203.0.114.64/26'), tenant=tenants[1]),
)
Prefix.objects.bulk_create(prefixes)
saved_filter = SavedFilter.objects.create(
name='Aggregate Tenant 1 prefixes',
slug='aggregate-tenant-1-prefixes',
parameters={
'tenant_id': [str(tenants[0].pk)],
},
)
saved_filter.object_types.add(ObjectType.objects.get_for_model(Prefix))
url = reverse('ipam:aggregate_prefixes', kwargs={'pk': aggregate.pk})
response = self.client.get(url, {'filter_id': saved_filter.pk})
self.assertHttpStatus(response, 200)
self.assertEqual(len(response.context['table'].data), 1)
self.assertContains(response, '203.0.114.0/26')
self.assertNotContains(response, '203.0.114.64/26')
def test_children_are_filtered_fallback(self):
"""_children_are_filtered() rebuilds the queryset when prep_table_data() has not cached a result."""
self.add_permissions('ipam.view_aggregate', 'ipam.view_prefix')
aggregate = Aggregate.objects.create(
prefix=IPNetwork('203.0.115.0/24'),
rir=RIR.objects.first()
)
tenant = Tenant.objects.create(name='Aggregate Fallback Tenant', slug='aggregate-fallback-tenant')
Prefix.objects.create(prefix=IPNetwork('203.0.115.0/26'), tenant=tenant)
Prefix.objects.create(prefix=IPNetwork('203.0.115.64/26'))
# No cached value: the fallback path rebuilds the filtered queryset and detects the filter.
view = AggregatePrefixesView()
request = RequestFactory().get('/', {'tenant_id': tenant.pk})
request.user = self.user
self.assertFalse(hasattr(view, '_child_queryset_is_filtered'))
self.assertTrue(view._children_are_filtered(request, aggregate))
# No cached value and no filter: the fallback path reports no filtering.
view = AggregatePrefixesView()
request = RequestFactory().get('/')
request.user = self.user
self.assertFalse(view._children_are_filtered(request, aggregate))
# A cached value takes precedence over the actual request state.
view = AggregatePrefixesView()
view._set_children_filtered(False)
request = RequestFactory().get('/', {'tenant_id': tenant.pk})
request.user = self.user
self.assertFalse(view._children_are_filtered(request, aggregate))
class RoleTestCase(ViewTestCases.OrganizationalObjectViewTestCase):
model = Role
@ -588,6 +687,63 @@ class PrefixTestCase(ViewTestCases.PrimaryObjectViewTestCase):
url = reverse('ipam:prefix_prefixes', kwargs={'pk': prefixes[0].pk})
self.assertHttpStatus(self.client.get(url), 200)
def test_prefix_prefixes_filter_suppresses_available_prefixes(self):
self.add_permissions('ipam.view_prefix')
tenants = (
Tenant(name='Prefix Tenant 1', slug='prefix-tenant-1'),
Tenant(name='Prefix Tenant 2', slug='prefix-tenant-2'),
)
Tenant.objects.bulk_create(tenants)
parent = Prefix.objects.create(prefix=IPNetwork('198.51.100.0/24'))
prefixes = (
Prefix(prefix=IPNetwork('198.51.100.0/26'), tenant=tenants[0]),
Prefix(prefix=IPNetwork('198.51.100.64/26'), tenant=tenants[1]),
)
Prefix.objects.bulk_create(prefixes)
url = reverse('ipam:prefix_prefixes', kwargs={'pk': parent.pk})
response = self.client.get(url, {'tenant_id': tenants[0].pk})
self.assertHttpStatus(response, 200)
self.assertEqual(len(response.context['table'].data), 1)
self.assertContains(response, '198.51.100.0/26')
self.assertNotContains(response, '198.51.100.64/26')
def test_prefix_prefixes_saved_filter_suppresses_available_prefixes(self):
self.add_permissions('ipam.view_prefix')
tenants = (
Tenant(name='Prefix Saved Tenant 1', slug='prefix-saved-tenant-1'),
Tenant(name='Prefix Saved Tenant 2', slug='prefix-saved-tenant-2'),
)
Tenant.objects.bulk_create(tenants)
parent = Prefix.objects.create(prefix=IPNetwork('198.51.101.0/24'))
prefixes = (
Prefix(prefix=IPNetwork('198.51.101.0/26'), tenant=tenants[0]),
Prefix(prefix=IPNetwork('198.51.101.64/26'), tenant=tenants[1]),
)
Prefix.objects.bulk_create(prefixes)
saved_filter = SavedFilter.objects.create(
name='Prefix Tenant 1 prefixes',
slug='prefix-tenant-1-prefixes',
parameters={
'tenant_id': [str(tenants[0].pk)],
},
)
saved_filter.object_types.add(ObjectType.objects.get_for_model(Prefix))
url = reverse('ipam:prefix_prefixes', kwargs={'pk': parent.pk})
response = self.client.get(url, {'filter_id': saved_filter.pk})
self.assertHttpStatus(response, 200)
self.assertEqual(len(response.context['table'].data), 1)
self.assertContains(response, '198.51.101.0/26')
self.assertNotContains(response, '198.51.101.64/26')
def test_prefix_ipranges(self):
self.add_permissions('ipam.view_prefix', 'ipam.view_iprange')
prefix = Prefix.objects.create(prefix=IPNetwork('192.168.0.0/16'))
@ -616,6 +772,89 @@ class PrefixTestCase(ViewTestCases.PrimaryObjectViewTestCase):
url = reverse('ipam:prefix_ipaddresses', kwargs={'pk': prefix.pk})
self.assertHttpStatus(self.client.get(url), 200)
def test_prefix_ipaddresses_filter(self):
self.add_permissions('ipam.view_prefix', 'ipam.view_ipaddress', 'ipam.view_iprange')
tenants = (
Tenant(name='IP Address Tenant 1', slug='ip-address-tenant-1'),
Tenant(name='IP Address Tenant 2', slug='ip-address-tenant-2'),
)
Tenant.objects.bulk_create(tenants)
prefix = Prefix.objects.create(prefix=IPNetwork('192.0.2.0/24'))
ip_addresses = (
IPAddress(address=IPNetwork('192.0.2.1/24'), tenant=tenants[0]),
IPAddress(address=IPNetwork('192.0.2.2/24'), tenant=tenants[1]),
)
IPAddress.objects.bulk_create(ip_addresses)
url = reverse('ipam:prefix_ipaddresses', kwargs={'pk': prefix.pk})
response = self.client.get(url, {'tenant_id': tenants[0].pk})
self.assertHttpStatus(response, 200)
self.assertEqual(len(response.context['table'].data), 1)
self.assertContains(response, '192.0.2.1/24')
self.assertNotContains(response, '192.0.2.2/24')
def test_prefix_ipaddresses_saved_filter(self):
self.add_permissions('ipam.view_prefix', 'ipam.view_ipaddress', 'ipam.view_iprange')
tenants = (
Tenant(name='Saved Filter Tenant 1', slug='saved-filter-tenant-1'),
Tenant(name='Saved Filter Tenant 2', slug='saved-filter-tenant-2'),
)
Tenant.objects.bulk_create(tenants)
prefix = Prefix.objects.create(prefix=IPNetwork('192.0.2.0/24'))
ip_addresses = (
IPAddress(address=IPNetwork('192.0.2.1/24'), tenant=tenants[0]),
IPAddress(address=IPNetwork('192.0.2.2/24'), tenant=tenants[1]),
)
IPAddress.objects.bulk_create(ip_addresses)
saved_filter = SavedFilter.objects.create(
name='Tenant 1 IP addresses',
slug='tenant-1-ip-addresses',
parameters={
'tenant_id': [str(tenants[0].pk)],
},
)
saved_filter.object_types.add(ObjectType.objects.get_for_model(IPAddress))
url = reverse('ipam:prefix_ipaddresses', kwargs={'pk': prefix.pk})
response = self.client.get(url, {'filter_id': saved_filter.pk})
self.assertHttpStatus(response, 200)
self.assertEqual(len(response.context['table'].data), 1)
self.assertContains(response, '192.0.2.1/24')
self.assertNotContains(response, '192.0.2.2/24')
def test_prefix_ipaddresses_unfiltered_shows_available_space(self):
"""An unfiltered IP Addresses tab injects synthetic available-space rows."""
self.add_permissions('ipam.view_prefix', 'ipam.view_ipaddress', 'ipam.view_iprange')
prefix = Prefix.objects.create(prefix=IPNetwork('192.0.2.0/29'))
IPAddress.objects.create(address=IPNetwork('192.0.2.1/29'))
url = reverse('ipam:prefix_ipaddresses', kwargs={'pk': prefix.pk})
response = self.client.get(url)
self.assertHttpStatus(response, 200)
self.assertGreater(len(response.context['table'].data), 1)
def test_prefix_prefixes_unfiltered_shows_available_prefixes(self):
"""An unfiltered Child Prefixes tab injects synthetic available-prefix rows."""
self.add_permissions('ipam.view_prefix')
parent = Prefix.objects.create(prefix=IPNetwork('198.51.102.0/24'))
Prefix.objects.create(prefix=IPNetwork('198.51.102.0/26'))
url = reverse('ipam:prefix_prefixes', kwargs={'pk': parent.pk})
response = self.client.get(url)
self.assertHttpStatus(response, 200)
self.assertGreater(len(response.context['table'].data), 1)
def test_prefix_ipaddresses_with_single_address_range(self):
self.add_permissions('ipam.view_prefix', 'ipam.view_ipaddress', 'ipam.view_iprange')
# The IP Addresses tab annotates child IP addresses alongside any
@ -1130,6 +1369,70 @@ class VLANGroupTestCase(ViewTestCases.OrganizationalObjectViewTestCase):
'description': 'New description',
}
def test_vlans_filter_suppresses_available_vlans(self):
self.add_permissions('ipam.view_vlangroup', 'ipam.view_vlan')
group = VLANGroup.objects.create(
name='Filtered VLAN Group',
slug='filtered-vlan-group'
)
vlans = (
VLAN(group=group, vid=100, name='VLAN100'),
VLAN(group=group, vid=200, name='VLAN200'),
)
VLAN.objects.bulk_create(vlans)
url = reverse('ipam:vlangroup_vlans', kwargs={'pk': group.pk})
response = self.client.get(url, {'vid': 100})
self.assertHttpStatus(response, 200)
self.assertEqual(len(response.context['table'].data), 1)
self.assertContains(response, 'VLAN100')
self.assertNotContains(response, 'VLAN200')
def test_vlans_saved_filter_suppresses_available_vlans(self):
self.add_permissions('ipam.view_vlangroup', 'ipam.view_vlan')
group = VLANGroup.objects.create(
name='Saved Filter VLAN Group',
slug='saved-filter-vlan-group'
)
vlans = (
VLAN(group=group, vid=100, name='VLAN100'),
VLAN(group=group, vid=200, name='VLAN200'),
)
VLAN.objects.bulk_create(vlans)
saved_filter = SavedFilter.objects.create(
name='VLAN 100',
slug='vlan-100',
parameters={
'vid': ['100'],
},
)
saved_filter.object_types.add(ObjectType.objects.get_for_model(VLAN))
url = reverse('ipam:vlangroup_vlans', kwargs={'pk': group.pk})
response = self.client.get(url, {'filter_id': saved_filter.pk})
self.assertHttpStatus(response, 200)
self.assertEqual(len(response.context['table'].data), 1)
self.assertContains(response, 'VLAN100')
self.assertNotContains(response, 'VLAN200')
def test_vlans_unfiltered_shows_available_vlans(self):
"""An unfiltered VLANs tab injects synthetic available-VLAN rows."""
self.add_permissions('ipam.view_vlangroup', 'ipam.view_vlan')
group = VLANGroup.objects.create(name='Unfiltered VLAN Group', slug='unfiltered-vlan-group')
VLAN.objects.create(group=group, vid=1, name='VLAN0001')
url = reverse('ipam:vlangroup_vlans', kwargs={'pk': group.pk})
response = self.client.get(url)
self.assertHttpStatus(response, 200)
self.assertGreater(len(response.context['table'].data), 1)
class VLANTestCase(ViewTestCases.PrimaryObjectViewTestCase):
model = VLAN
@ -1199,6 +1502,178 @@ class VLANTestCase(ViewTestCases.PrimaryObjectViewTestCase):
'description': 'New description',
}
def test_bulk_add_vlans(self):
self.add_permissions('ipam.add_vlan')
group = VLANGroup.objects.get(name='VLAN Group 1')
initial_count = VLAN.objects.count()
expected_vids = (110, 120, 121, 122)
form_data = {
'pattern': '110,120-122',
'group': group.pk,
'name': 'Pool-{vid}',
'status': VLANStatusChoices.STATUS_RESERVED,
}
response = self.client.post(reverse('ipam:vlan_bulk_add'), form_data)
self.assertHttpStatus(response, 302)
self.assertEqual(VLAN.objects.count(), initial_count + len(expected_vids))
for vid in expected_vids:
self.assertTrue(
VLAN.objects.filter(
group=group,
vid=vid,
name=f'Pool-{vid}'
).exists()
)
def test_bulk_add_vlans_rolls_back_on_duplicate_name(self):
self.add_permissions('ipam.add_vlan')
group = VLANGroup.objects.get(name='VLAN Group 1')
initial_count = VLAN.objects.count()
form_data = {
'pattern': '110-112',
'group': group.pk,
'name': 'Duplicate name',
'status': VLANStatusChoices.STATUS_RESERVED,
}
response = self.client.post(reverse('ipam:vlan_bulk_add'), form_data)
self.assertHttpStatus(response, 200)
self.assertEqual(VLAN.objects.count(), initial_count)
self.assertFalse(VLAN.objects.filter(group=group, vid=110).exists())
def test_bulk_add_vlans_rolls_back_when_any_id_outside_group_range(self):
self.add_permissions('ipam.add_vlan')
group = VLANGroup.objects.create(
name='Restricted VLAN Group',
slug='restricted-vlan-group',
vid_ranges=[NumericRange(200, 204)] # Valid VIDs: 200-203
)
initial_count = VLAN.objects.count()
form_data = {
'pattern': '200-203,500',
'group': group.pk,
'name': 'Restricted-{vid}',
'status': VLANStatusChoices.STATUS_RESERVED,
}
response = self.client.post(reverse('ipam:vlan_bulk_add'), form_data)
self.assertHttpStatus(response, 200)
self.assertEqual(VLAN.objects.count(), initial_count)
self.assertFalse(VLAN.objects.filter(group=group, vid=200).exists())
self.assertFalse(VLAN.objects.filter(group=group, vid=203).exists())
self.assertFalse(VLAN.objects.filter(group=group, vid=500).exists())
def test_bulk_add_vlans_pattern_shapes(self):
"""Single values, multiple values, ranges, and combinations create the expected VLANs."""
self.add_permissions('ipam.add_vlan')
# The combination runs against a second group: subTests share one transaction, and VIDs
# 10 & 20 would otherwise collide with the multiple-values case via the (group, vid) constraint.
cases = (
('500', (500,), 'VLAN Group 1'),
('5,10,20', (5, 10, 20), 'VLAN Group 1'),
('600-605', tuple(range(600, 606)), 'VLAN Group 1'),
('1,10-20,300-305', (1, *range(10, 21), *range(300, 306)), 'VLAN Group 2'),
)
for pattern, expected_vids, group_name in cases:
with self.subTest(pattern=pattern):
group = VLANGroup.objects.get(name=group_name)
initial_count = VLAN.objects.count()
form_data = {
'pattern': pattern,
'group': group.pk,
'name': 'Pool-{vid}',
'status': VLANStatusChoices.STATUS_ACTIVE,
}
response = self.client.post(reverse('ipam:vlan_bulk_add'), form_data)
self.assertHttpStatus(response, 302)
self.assertEqual(VLAN.objects.count(), initial_count + len(expected_vids))
for vid in expected_vids:
self.assertTrue(VLAN.objects.filter(group=group, vid=vid, name=f'Pool-{vid}').exists())
def test_bulk_add_vlans_invalid_pattern(self):
"""An invalid pattern re-renders the form with a pattern error and creates nothing."""
self.add_permissions('ipam.add_vlan')
initial_count = VLAN.objects.count()
for pattern in ('abc', '20-10', '0', '4095', '10-'):
with self.subTest(pattern=pattern):
form_data = {
'pattern': pattern,
'name': 'Pool-{vid}',
'status': VLANStatusChoices.STATUS_ACTIVE,
}
response = self.client.post(reverse('ipam:vlan_bulk_add'), form_data)
self.assertHttpStatus(response, 200)
self.assertIn('pattern', response.context['form'].errors)
self.assertEqual(VLAN.objects.count(), initial_count)
def test_bulk_add_vlans_static_name_without_group(self):
"""A static name (no {vid} placeholder) is permitted across VLANs not assigned to a group."""
self.add_permissions('ipam.add_vlan')
initial_count = VLAN.objects.count()
form_data = {
'pattern': '710-712',
'name': 'Same name',
'status': VLANStatusChoices.STATUS_ACTIVE,
}
response = self.client.post(reverse('ipam:vlan_bulk_add'), form_data)
self.assertHttpStatus(response, 302)
self.assertEqual(VLAN.objects.count(), initial_count + 3)
self.assertEqual(VLAN.objects.filter(name='Same name').count(), 3)
def test_bulk_add_vlans_rolls_back_on_constrained_permission(self):
"""Bulk creation rolls back when a generated VLAN falls outside the user's add constraints."""
obj_perm = ObjectPermission(
name='Test permission',
actions=['add'],
constraints={'vid__lt': 120}
)
obj_perm.save()
obj_perm.users.add(self.user)
obj_perm.object_types.add(ObjectType.objects.get_for_model(VLAN))
initial_count = VLAN.objects.count()
form_data = {
'pattern': '110,120-122',
'name': 'Pool-{vid}',
'status': VLANStatusChoices.STATUS_ACTIVE,
}
response = self.client.post(reverse('ipam:vlan_bulk_add'), form_data)
self.assertHttpStatus(response, 200)
self.assertEqual(VLAN.objects.count(), initial_count)
self.assertTrue(response.context['form'].non_field_errors())
def test_bulk_add_vlans_propagates_field_errors(self):
"""A per-object validation error on a non-pattern field is reported on the bulk-create form."""
self.add_permissions('ipam.add_vlan')
initial_count = VLAN.objects.count()
form_data = {
'pattern': '800',
'name': 'Pool-{vid}',
'status': VLANStatusChoices.STATUS_ACTIVE,
'qinq_role': VLANQinQRoleChoices.ROLE_CUSTOMER, # Requires an SVLAN
}
response = self.client.post(reverse('ipam:vlan_bulk_add'), form_data)
self.assertHttpStatus(response, 200)
self.assertEqual(VLAN.objects.count(), initial_count)
self.assertTrue(response.context['form'].non_field_errors())
class VLANTranslationPolicyTestCase(ViewTestCases.PrimaryObjectViewTestCase):
model = VLANTranslationPolicy

33
netbox/ipam/utils.py
View File

@ -1,10 +1,10 @@
from dataclasses import dataclass
import netaddr
from django.apps import apps
from django.utils.translation import gettext_lazy as _
from .constants import *
from .models import VLAN, Prefix
__all__ = (
'AvailableIPSpace',
@ -39,7 +39,7 @@ def add_requested_prefixes(parent, prefix_list, show_available=True, show_assign
requested, create fake Prefix objects for all unallocated space within a prefix.
:param parent: Parent Prefix instance
:param prefix_list: Child prefixes list
:param prefix_list: Child prefixes list (or queryset)
:param show_available: Include available prefixes.
:param show_assigned: Show assigned prefixes.
"""
@ -47,6 +47,7 @@ def add_requested_prefixes(parent, prefix_list, show_available=True, show_assign
# Add available prefixes to the table if requested
if prefix_list and show_available:
Prefix = apps.get_model('ipam', 'Prefix')
# Find all unallocated space, add fake Prefix objects to child_prefixes.
# IMPORTANT: These are unsaved Prefix instances (pk=None). If this is ever changed to use
@ -78,22 +79,7 @@ def annotate_ip_space(prefix):
records = sorted(records, key=lambda x: x[0])
# Determine the first & last valid IP addresses in the prefix
if (
prefix.is_pool
or (prefix.family == 4 and prefix.mask_length >= 31)
or (prefix.family == 6 and prefix.mask_length >= 127)
):
# Pool, IPv4 /31-/32 or IPv6 /127-/128 sets are fully usable
first_ip_in_prefix = netaddr.IPAddress(prefix.prefix.first)
last_ip_in_prefix = netaddr.IPAddress(prefix.prefix.last)
elif prefix.family == 4:
# Ignore the network and broadcast addresses for non-pool IPv4 prefixes larger than /31
first_ip_in_prefix = netaddr.IPAddress(prefix.prefix.first + 1)
last_ip_in_prefix = netaddr.IPAddress(prefix.prefix.last - 1)
else:
# For IPv6 prefixes, omit the Subnet-Router anycast address (RFC 4291)
first_ip_in_prefix = netaddr.IPAddress(prefix.prefix.first + 1)
last_ip_in_prefix = netaddr.IPAddress(prefix.prefix.last)
first_ip_in_prefix, last_ip_in_prefix = prefix.usable_ip_bounds
if not records:
return [
@ -195,7 +181,7 @@ def add_available_vlans(vlans, vlan_group):
new_vlans.extend(available_vlans_from_range(vlans, vlan_group, vid_range))
vlans = list(vlans) + new_vlans
vlans.sort(key=lambda v: v.vid if type(v) is VLAN else v['vid'])
vlans.sort(key=lambda v: v['vid'] if isinstance(v, dict) else v.vid)
return vlans
@ -204,6 +190,9 @@ def rebuild_prefixes(vrf):
"""
Rebuild the prefix hierarchy for all prefixes in the specified VRF (or global table).
"""
Prefix = apps.get_model('ipam', 'Prefix')
prefix_queryset = Prefix.objects.filter(vrf=vrf)
def contains(parent, child):
return child in parent and child != parent
@ -219,10 +208,10 @@ def rebuild_prefixes(vrf):
stack = []
update_queue = []
prefixes = Prefix.objects.filter(vrf=vrf).values('pk', 'prefix')
prefixes = prefix_queryset.order_by('prefix', 'pk').values('pk', 'prefix')
# Iterate through all Prefixes in the VRF, growing and shrinking the stack as we go
for i, p in enumerate(prefixes):
# Iterate through all Prefixes in the table, growing and shrinking the stack as we go
for p in prefixes:
# Grow the stack if this is a child of the most recent prefix
if not stack or contains(stack[-1]['prefix'], p['prefix']):

105
netbox/ipam/views.py
View File

@ -547,8 +547,64 @@ class AggregateView(generic.ObjectView):
)
class ChildAvailabilityMixin:
"""
Mixin for ObjectChildrenView subclasses that render synthetic "available" rows
(available IP space, prefixes, or VLANs) and must suppress them when the child
queryset has been narrowed by a direct or saved filter.
"""
@staticmethod
def _where_signature(queryset):
# query.where is Django-internal, but it is the closest signal for "narrowed by a filter".
return str(queryset.query.where)
def _set_children_filtered(self, is_filtered):
self._child_queryset_is_filtered = is_filtered
return is_filtered
def _queryset_is_filtered(self, request, queryset, parent):
"""
Return True if the filtered child queryset differs from the unfiltered one.
Compares WHERE clauses rather than testing queryset.query.where for truthiness,
because child querysets are already scoped to their parent object and carry WHERE
clauses before any user filter is applied. The result is cached on the view instance
so get_extra_context() can reuse it without rebuilding the queryset.
"""
if self.filterset is None:
return self._set_children_filtered(False)
unfiltered = self.get_children(request, parent)
return self._set_children_filtered(
self._where_signature(queryset) != self._where_signature(unfiltered)
)
def _children_are_filtered(self, request, parent):
"""
Return whether child objects are filtered.
In the normal ObjectChildrenView flow prep_table_data() runs first and caches the
result, so this returns the cached value. Fall back to rebuilding the queryset for
direct calls where prep_table_data() has not run.
"""
if hasattr(self, '_child_queryset_is_filtered'):
return self._child_queryset_is_filtered
if self.filterset is None:
return self._set_children_filtered(False)
unfiltered = self.get_children(request, parent)
filtered = self.filterset(request.GET, unfiltered, request=request).qs
return self._set_children_filtered(
self._where_signature(filtered) != self._where_signature(unfiltered)
)
@register_model_view(Aggregate, 'prefixes')
class AggregatePrefixesView(generic.ObjectChildrenView):
class AggregatePrefixesView(ChildAvailabilityMixin, generic.ObjectChildrenView):
queryset = Aggregate.objects.all()
child_model = Prefix
table = tables.PrefixTable
@ -572,13 +628,21 @@ class AggregatePrefixesView(generic.ObjectChildrenView):
show_available = bool(request.GET.get('show_available', 'true') == 'true')
show_assigned = bool(request.GET.get('show_assigned', 'true') == 'true')
if self._queryset_is_filtered(request, queryset, parent):
show_available = False
return add_requested_prefixes(parent.prefix, queryset, show_available, show_assigned)
def get_extra_context(self, request, instance):
show_available = (
bool(request.GET.get('show_available', 'true') == 'true') and
not self._children_are_filtered(request, instance)
)
return {
'bulk_querystring': f'within={instance.prefix}',
'first_available_prefix': instance.get_first_available_prefix(),
'show_available': bool(request.GET.get('show_available', 'true') == 'true'),
'show_available': show_available,
'show_assigned': bool(request.GET.get('show_assigned', 'true') == 'true'),
}
@ -770,7 +834,7 @@ class PrefixView(generic.ObjectView):
@register_model_view(Prefix, 'prefixes')
class PrefixPrefixesView(generic.ObjectChildrenView):
class PrefixPrefixesView(ChildAvailabilityMixin, generic.ObjectChildrenView):
queryset = Prefix.objects.all()
child_model = Prefix
table = tables.PrefixTable
@ -794,13 +858,21 @@ class PrefixPrefixesView(generic.ObjectChildrenView):
show_available = bool(request.GET.get('show_available', 'true') == 'true')
show_assigned = bool(request.GET.get('show_assigned', 'true') == 'true')
if self._queryset_is_filtered(request, queryset, parent):
show_available = False
return add_requested_prefixes(parent.prefix, queryset, show_available, show_assigned)
def get_extra_context(self, request, instance):
show_available = (
bool(request.GET.get('show_available', 'true') == 'true') and
not self._children_are_filtered(request, instance)
)
return {
'bulk_querystring': f"vrf_id={instance.vrf.pk if instance.vrf else '0'}&within={instance.prefix}",
'first_available_prefix': instance.get_first_available_prefix(),
'show_available': bool(request.GET.get('show_available', 'true') == 'true'),
'show_available': show_available,
'show_assigned': bool(request.GET.get('show_assigned', 'true') == 'true'),
}
@ -833,7 +905,7 @@ class PrefixIPRangesView(generic.ObjectChildrenView):
@register_model_view(Prefix, 'ipaddresses', path='ip-addresses')
class PrefixIPAddressesView(generic.ObjectChildrenView):
class PrefixIPAddressesView(ChildAvailabilityMixin, generic.ObjectChildrenView):
queryset = Prefix.objects.all()
child_model = IPAddress
table = tables.AnnotatedIPAddressTable
@ -851,9 +923,10 @@ class PrefixIPAddressesView(generic.ObjectChildrenView):
return parent.get_child_ips().restrict(request.user, 'view').prefetch_related('vrf', 'tenant', 'tenant__group')
def prep_table_data(self, request, queryset, parent):
if not request.GET.get('q') and not get_table_ordering(request, self.table):
if not self._queryset_is_filtered(request, queryset, parent) and not get_table_ordering(request, self.table):
return annotate_ip_space(parent)
return queryset
return super().prep_table_data(request, queryset, parent)
def get_extra_context(self, request, instance):
return {
@ -1292,7 +1365,7 @@ class VLANGroupBulkDeleteView(generic.BulkDeleteView):
@register_model_view(VLANGroup, 'vlans')
class VLANGroupVLANsView(generic.ObjectChildrenView):
class VLANGroupVLANsView(ChildAvailabilityMixin, generic.ObjectChildrenView):
queryset = VLANGroup.objects.all()
child_model = VLAN
table = tables.VLANTable
@ -1312,9 +1385,11 @@ class VLANGroupVLANsView(generic.ObjectChildrenView):
)
def prep_table_data(self, request, queryset, parent):
if not get_table_ordering(request, self.table):
# Skip synthetic available rows under active filters: filtered-out VLANs would otherwise look available.
if not self._queryset_is_filtered(request, queryset, parent) and not get_table_ordering(request, self.table):
return add_available_vlans(queryset, parent)
return queryset
return super().prep_table_data(request, queryset, parent)
#
@ -1718,6 +1793,16 @@ class VLANDeleteView(generic.ObjectDeleteView):
queryset = VLAN.objects.all()
@register_model_view(VLAN, 'bulk_add', path='bulk-add', detail=False)
class VLANBulkCreateView(generic.BulkCreateView):
queryset = VLAN.objects.all()
form = forms.VLANIDBulkCreateForm
model_form = forms.VLANBulkAddForm
pattern_target = 'vid'
pattern_template_fields = ('name',)
template_name = 'ipam/vlan_bulk_add.html'
@register_model_view(VLAN, 'bulk_import', path='import', detail=False)
class VLANBulkImportView(generic.BulkImportView):
queryset = VLAN.objects.all()

68
netbox/netbox/api/serializers/bulk.py
View File

@ -1,11 +1,79 @@
import copy
import functools
from rest_framework import serializers
from .features import ChangeLogMessageSerializer
__all__ = (
'BulkOperationSerializer',
'BulkPartialUpdateSchemaMixin',
'BulkUpdateSchemaMixin',
'get_bulk_update_serializer_class'
)
class BulkOperationSerializer(ChangeLogMessageSerializer):
id = serializers.IntegerField()
class BulkUpdateSchemaMixin:
def get_fields(self):
fields = super().get_fields()
# Reuse the runtime bulk-operation ID field so the schema stays in sync
# with the validator that consumes `id` before model serialization.
_id = copy.deepcopy(BulkOperationSerializer().fields['id'])
_id.required = True
fields['id'] = _id
return fields
class BulkPartialUpdateSchemaMixin(BulkUpdateSchemaMixin):
def get_fields(self):
fields = super().get_fields()
for name, field in fields.items():
if name != 'id':
field.required = False
return fields
@functools.cache
def get_bulk_update_serializer_class(serializer_class, *, partial=False):
"""
Return a schema-only serializer for bulk PUT/PATCH requests.
Bulk update requests to a list endpoint require each object to include
the target object's numeric ID, even though `id` is read-only on the
normal model serializer. The runtime code consumes `id` before invoking
the model serializer for each object.
"""
meta = getattr(serializer_class, 'Meta')
if meta.fields == '__all__':
fields = '__all__'
else:
fields = ('id', *[f for f in meta.fields if f != 'id'])
class Meta(meta):
pass
# intentional; this is different than setting fields = fields within class Meta above
Meta.fields = fields
bases = (
(BulkPartialUpdateSchemaMixin, serializer_class)
if partial
else (BulkUpdateSchemaMixin, serializer_class)
)
attrs = {
'Meta': Meta,
'__module__': serializer_class.__module__,
}
prefix = 'PatchedBulk' if partial else 'Bulk'
return type(f'{prefix}{serializer_class.__name__}', bases, attrs)

15
netbox/netbox/api/viewsets/mixins.py
View File

@ -10,6 +10,7 @@ from rest_framework.reverse import reverse
from core.models import ObjectType
from extras.models import ExportTemplate
from netbox.api.serializers import BulkOperationSerializer
from netbox.api.serializers.bulk import get_bulk_update_serializer_class
from netbox.jobs import AsyncAPIJob
from utilities.exceptions import RQWorkerNotRunningException
from utilities.rqworker import any_workers_for_queue
@ -240,6 +241,20 @@ class BulkUpdateModelMixin:
return updated_pks
def get_bulk_update_serializer_class(self, *, partial=False):
return get_bulk_update_serializer_class(
self.get_serializer_class(),
partial=partial,
)
def get_bulk_update_request_serializer(self, *, partial=False):
serializer_class = self.get_bulk_update_serializer_class(partial=partial)
# Important: do NOT pass partial=True here. The partial schema class already
# makes non-id fields optional, and passing partial=True would also make id
# appear optional in OpenAPI.
return serializer_class(many=True)
def bulk_partial_update(self, request, *args, **kwargs):
kwargs['partial'] = True
return self.bulk_update(request, *args, **kwargs)

123
netbox/netbox/graphql/filter_lookups.py
View File

@ -1,3 +1,4 @@
import re
from enum import Enum
from typing import Generic, TypeVar
@ -11,17 +12,48 @@ from strawberry.directive import DirectiveValue
from strawberry.types import Info
from strawberry_django import (
ComparisonFilterLookup,
DateFilterLookup,
DatetimeFilterLookup,
FilterLookup,
RangeLookup,
StrFilterLookup,
TimeFilterLookup,
process_filters,
)
from netbox.graphql.scalars import BigInt
# ------------------------------------------------------------------
# JSON path validation (VM-323)
# ------------------------------------------------------------------
# Each segment of a JSON path may only contain alphanumerics, underscores, and
# hyphens. Hyphens are included because JSON keys commonly use them; leading
# underscores are permitted (e.g. _foo is a valid key name).
_JSON_PATH_SEGMENT_RE = re.compile(r'^[A-Za-z0-9_][A-Za-z0-9_-]*$')
def _validate_json_path(path: str) -> str:
"""Validate a JSON traversal path for use in ORM lookups.
Each ``__``-separated segment must match ``[A-Za-z0-9_][A-Za-z0-9_-]*``.
Raises ``ValueError`` on an empty path, empty segment, or segment with
disallowed characters.
ORM operator names (``date``, ``regex``, etc.) are intentionally *not*
blocked here: ``JSONFilter.filter()`` always appends ``__`` to the path
before handing it to ``process_filters``, so a segment named ``regex``
becomes another level of JSON key traversal (``data__key__regex__exact``),
not the ORM regex transform (``data__key__regex=…``).
"""
if not path:
raise ValueError("JSON path cannot be empty")
for segment in path.split('__'):
if not segment:
raise ValueError("JSON path contains consecutive or trailing '__'")
if not _JSON_PATH_SEGMENT_RE.match(segment):
raise ValueError(f"Invalid JSON path segment: {segment!r}")
return path
__all__ = (
'ArrayLookup',
'BigIntegerLookup',
@ -31,6 +63,8 @@ __all__ = (
'IntegerLookup',
'IntegerRangeArrayLookup',
'JSONFilter',
'JSONLookup',
'JSONStringLookup',
'StringArrayLookup',
'TreeNodeFilter',
)
@ -39,16 +73,82 @@ T = TypeVar('T')
SKIP_MSG = 'Filter will be skipped on `null` value'
# These JSON lookup types intentionally mirror the legacy DateFilterLookup[str],
# TimeFilterLookup[str], and DatetimeFilterLookup[str] schema. JSON values are
# string-backed, so the concrete strawberry-django date/time lookup classes
# (which now ignore type parameters and warn) are deliberately not used here.
@strawberry.input(name='StrDateFilterLookup')
class JSONDateFilterLookup(ComparisonFilterLookup[str]):
year: ComparisonFilterLookup[int] | None = strawberry.UNSET
month: ComparisonFilterLookup[int] | None = strawberry.UNSET
day: ComparisonFilterLookup[int] | None = strawberry.UNSET
week_day: ComparisonFilterLookup[int] | None = strawberry.UNSET
iso_week_day: ComparisonFilterLookup[int] | None = strawberry.UNSET
week: ComparisonFilterLookup[int] | None = strawberry.UNSET
iso_year: ComparisonFilterLookup[int] | None = strawberry.UNSET
quarter: ComparisonFilterLookup[int] | None = strawberry.UNSET
@strawberry.input(name='StrTimeFilterLookup')
class JSONTimeFilterLookup(ComparisonFilterLookup[str]):
hour: ComparisonFilterLookup[int] | None = strawberry.UNSET
minute: ComparisonFilterLookup[int] | None = strawberry.UNSET
second: ComparisonFilterLookup[int] | None = strawberry.UNSET
date: ComparisonFilterLookup[int] | None = strawberry.UNSET
time: ComparisonFilterLookup[int] | None = strawberry.UNSET
@strawberry.input(name='StrDatetimeFilterLookup')
class JSONDatetimeFilterLookup(ComparisonFilterLookup[str]):
year: ComparisonFilterLookup[int] | None = strawberry.UNSET
month: ComparisonFilterLookup[int] | None = strawberry.UNSET
day: ComparisonFilterLookup[int] | None = strawberry.UNSET
week_day: ComparisonFilterLookup[int] | None = strawberry.UNSET
iso_week_day: ComparisonFilterLookup[int] | None = strawberry.UNSET
week: ComparisonFilterLookup[int] | None = strawberry.UNSET
iso_year: ComparisonFilterLookup[int] | None = strawberry.UNSET
quarter: ComparisonFilterLookup[int] | None = strawberry.UNSET
hour: ComparisonFilterLookup[int] | None = strawberry.UNSET
minute: ComparisonFilterLookup[int] | None = strawberry.UNSET
second: ComparisonFilterLookup[int] | None = strawberry.UNSET
date: ComparisonFilterLookup[int] | None = strawberry.UNSET
time: ComparisonFilterLookup[int] | None = strawberry.UNSET
@strawberry.input(description='String lookups for JSON field values.')
class JSONStringLookup:
"""
String-filter type for use inside JSONLookup.
Equivalent to ``StrFilterLookup`` but defined explicitly so that the type
name remains stable and any future per-field restrictions are easy to add.
``regex`` / ``i_regex`` are included: they provide no additional oracle
power beyond ``starts_with``, which is also present.
"""
exact: str | None = strawberry_django.filter_field()
i_exact: str | None = strawberry_django.filter_field()
contains: str | None = strawberry_django.filter_field()
i_contains: str | None = strawberry_django.filter_field()
starts_with: str | None = strawberry_django.filter_field()
i_starts_with: str | None = strawberry_django.filter_field()
ends_with: str | None = strawberry_django.filter_field()
i_ends_with: str | None = strawberry_django.filter_field()
in_: list[str] | None = strawberry_django.filter_field()
isnull: bool | None = strawberry_django.filter_field()
regex: str | None = strawberry_django.filter_field()
i_regex: str | None = strawberry_django.filter_field()
@strawberry.input(one_of=True, description='Lookup for JSON field. Only one of the lookup fields can be set.')
class JSONLookup:
string_lookup: StrFilterLookup[str] | None = strawberry_django.filter_field()
string_lookup: JSONStringLookup | None = strawberry_django.filter_field()
int_range_lookup: RangeLookup[int] | None = strawberry_django.filter_field()
int_comparison_lookup: ComparisonFilterLookup[int] | None = strawberry_django.filter_field()
float_range_lookup: RangeLookup[float] | None = strawberry_django.filter_field()
float_comparison_lookup: ComparisonFilterLookup[float] | None = strawberry_django.filter_field()
date_lookup: DateFilterLookup[str] | None = strawberry_django.filter_field()
datetime_lookup: DatetimeFilterLookup[str] | None = strawberry_django.filter_field()
time_lookup: TimeFilterLookup[str] | None = strawberry_django.filter_field()
date_lookup: JSONDateFilterLookup | None = strawberry_django.filter_field()
datetime_lookup: JSONDatetimeFilterLookup | None = strawberry_django.filter_field()
time_lookup: JSONTimeFilterLookup | None = strawberry_django.filter_field()
boolean_lookup: FilterLookup[bool] | None = strawberry_django.filter_field()
def get_filter(self):
@ -119,7 +219,12 @@ class JSONFilter:
if not filters:
return queryset, Q()
json_path = f'{prefix}{self.path}__'
try:
safe_path = _validate_json_path(self.path)
except ValueError:
return queryset, Q()
json_path = f'{prefix}{safe_path}__'
return process_filters(filters=filters, queryset=queryset, info=info, prefix=json_path)

5
netbox/netbox/graphql/filter_mixins.py
View File

@ -1,5 +1,4 @@
from dataclasses import dataclass
from datetime import datetime
from typing import TYPE_CHECKING, Annotated, TypeVar
import strawberry
@ -48,9 +47,9 @@ class SyncedDataFilterMixin:
strawberry_django.filter_field()
)
data_file_id: FilterLookup[int] | None = strawberry_django.filter_field()
data_path: StrFilterLookup[str] | None = strawberry_django.filter_field()
data_path: StrFilterLookup | None = strawberry_django.filter_field()
auto_sync_enabled: FilterLookup[bool] | None = strawberry_django.filter_field()
data_synced: DatetimeFilterLookup[datetime] | None = strawberry_django.filter_field()
data_synced: DatetimeFilterLookup | None = strawberry_django.filter_field()
@dataclass

18
netbox/netbox/graphql/filters.py
View File

@ -42,21 +42,21 @@ class NetBoxModelFilter(
@dataclass
class NestedGroupModelFilter(NetBoxModelFilter):
name: StrFilterLookup[str] | None = strawberry_django.filter_field()
slug: StrFilterLookup[str] | None = strawberry_django.filter_field()
description: StrFilterLookup[str] | None = strawberry_django.filter_field()
name: StrFilterLookup | None = strawberry_django.filter_field()
slug: StrFilterLookup | None = strawberry_django.filter_field()
description: StrFilterLookup | None = strawberry_django.filter_field()
parent_id: ID | None = strawberry_django.filter_field()
@dataclass
class OrganizationalModelFilter(NetBoxModelFilter):
name: StrFilterLookup[str] | None = strawberry_django.filter_field()
slug: StrFilterLookup[str] | None = strawberry_django.filter_field()
description: StrFilterLookup[str] | None = strawberry_django.filter_field()
comments: StrFilterLookup[str] | None = strawberry_django.filter_field()
name: StrFilterLookup | None = strawberry_django.filter_field()
slug: StrFilterLookup | None = strawberry_django.filter_field()
description: StrFilterLookup | None = strawberry_django.filter_field()
comments: StrFilterLookup | None = strawberry_django.filter_field()
@dataclass
class PrimaryModelFilter(NetBoxModelFilter):
description: StrFilterLookup[str] | None = strawberry_django.filter_field()
comments: StrFilterLookup[str] | None = strawberry_django.filter_field()
description: StrFilterLookup | None = strawberry_django.filter_field()
comments: StrFilterLookup | None = strawberry_django.filter_field()

18
netbox/netbox/graphql/schema.py
View File

@ -1,3 +1,5 @@
from collections.abc import Callable
import strawberry
from django.conf import settings
from strawberry.extensions import MaxAliasesLimiter, QueryDepthLimiter, SchemaExtension
@ -18,6 +20,8 @@ from wireless.graphql.schema import WirelessQuery
from .scalars import BigInt, BigIntScalar
SchemaExtensionFactory = type[SchemaExtension] | Callable[[], SchemaExtension]
@strawberry.type
class Query(
@ -36,14 +40,16 @@ class Query(
pass
def get_schema_extensions() -> list[SchemaExtension]:
extensions: list[SchemaExtension] = [
DjangoOptimizerExtension(prefetch_custom_queryset=True),
MaxAliasesLimiter(max_alias_count=settings.GRAPHQL_MAX_ALIASES),
]
def get_schema_extensions() -> list[SchemaExtensionFactory]:
max_aliases = settings.GRAPHQL_MAX_ALIASES
max_depth = settings.GRAPHQL_MAX_QUERY_DEPTH
extensions: list[SchemaExtensionFactory] = [
lambda: DjangoOptimizerExtension(prefetch_custom_queryset=True),
lambda: MaxAliasesLimiter(max_alias_count=max_aliases),
]
if max_depth and max_depth > 0:
extensions.append(QueryDepthLimiter(max_depth=max_depth))
extensions.append(lambda: QueryDepthLimiter(max_depth=max_depth))
return extensions

13
netbox/netbox/middleware.py
View File

@ -10,7 +10,9 @@ from django.db import ProgrammingError, connection
from django.db.utils import InternalError
from django.http import Http404, HttpResponseRedirect
from django.middleware.common import CommonMiddleware as DjangoCommonMiddleware
from django.utils.translation import gettext_lazy as _
from django_prometheus import middleware
from social_django.middleware import SocialAuthExceptionMiddleware as SocialAuthExceptionMiddleware_
from netbox.config import clear_config, get_config
from netbox.metrics import Metrics
@ -26,6 +28,7 @@ __all__ = (
'PrometheusAfterMiddleware',
'PrometheusBeforeMiddleware',
'RemoteUserMiddleware',
'SocialAuthExceptionMiddleware',
)
@ -286,3 +289,13 @@ class MaintenanceModeMiddleware:
messages.error(request, error_message)
return HttpResponseRedirect(request.path_info)
return None
class SocialAuthExceptionMiddleware(SocialAuthExceptionMiddleware_):
"""
Subclass of python-social-auth's exception middleware which surfaces a generic, user-friendly
message rather than exposing the raw social_core exception text to (typically unauthenticated)
users when an SSO/SAML login fails.
"""
def get_message(self, request, exception):
return _("Single sign-on failed. Please try again or contact your administrator.")

14
netbox/netbox/models/deletion.py
View File

@ -3,6 +3,7 @@ import logging
from django.contrib.contenttypes.fields import GenericRelation
from django.db import router
from django.db.models.deletion import CASCADE, Collector
from django.utils.translation import gettext as _
logger = logging.getLogger("netbox.models.deletion")
@ -45,7 +46,7 @@ class CustomCollector(Collector):
# Add GenericRelations to the dependency graph
processed_relations = set()
for _, instances in list(self.data.items()):
for _model, instances in list(self.data.items()):
for instance in instances:
# Get all GenericRelations for this model
for field in instance._meta.private_fields:
@ -70,10 +71,13 @@ class DeleteMixin:
Override delete to use our custom collector.
"""
using = using or router.db_for_write(self.__class__, instance=self)
assert self._get_pk_val() is not None, (
f"{self._meta.object_name} object can't be deleted because its "
f"{self._meta.pk.attname} attribute is set to None."
)
if self._get_pk_val() is None:
raise ValueError(
_("{object_name} object can't be deleted because its {pk_attname} attribute is set to None.").format(
object_name=self._meta.object_name,
pk_attname=self._meta.pk.attname,
)
)
collector = CustomCollector(using=using)
collector.collect([self], keep_parents=keep_parents)

8
netbox/netbox/settings.py
View File

@ -514,6 +514,7 @@ MIDDLEWARE = [
'netbox.middleware.RemoteUserMiddleware',
'netbox.middleware.CoreMiddleware',
'netbox.middleware.MaintenanceModeMiddleware',
'netbox.middleware.SocialAuthExceptionMiddleware',
]
if DEBUG:
@ -713,6 +714,13 @@ SOCIAL_AUTH_PIPELINE = (
'social_core.pipeline.user.user_details',
)
# Redirect users back to the login page (surfacing the error via the messages framework) when an
# SSO/SAML authentication failure occurs, rather than raising an HTTP 500. Full exceptions are still
# raised when DEBUG is enabled. LOGIN_URL is an absolute path which respects BASE_PATH; the social
# auth middleware passes this value directly to an HttpResponseRedirect without reversing it.
SOCIAL_AUTH_LOGIN_ERROR_URL = LOGIN_URL
SOCIAL_AUTH_RAISE_EXCEPTIONS = DEBUG
# Load all SOCIAL_AUTH_* settings from the user configuration
for param in dir(configuration):
if param.startswith('SOCIAL_AUTH_'):

58
netbox/netbox/tests/test_authentication.py
View File

@ -1,13 +1,16 @@
import datetime
from django.conf import settings
from django.test import Client
from django.contrib.messages.storage.fallback import FallbackStorage
from django.test import Client, RequestFactory, SimpleTestCase
from django.test.utils import override_settings
from django.urls import reverse
from rest_framework.test import APIClient
from social_core.exceptions import AuthFailed
from core.models import ObjectType
from dcim.models import Rack, Site
from netbox.middleware import SocialAuthExceptionMiddleware
from users.constants import TOKEN_PREFIX
from users.models import Group, ObjectPermission, Token, User
from utilities.testing import TestCase
@ -697,3 +700,56 @@ class ObjectPermissionAPIViewTestCase(TestCase):
url = reverse('dcim-api:rack-detail', kwargs={'pk': self.racks[0].pk})
response = self.client.delete(url, format='json', **self.header)
self.assertEqual(response.status_code, 204)
class SocialAuthExceptionMiddlewareTestCase(SimpleTestCase):
"""
Verify that SSO/SAML authentication failures are surfaced as a login-page message rather than
bubbling up as an HTTP 500 (see #22346).
"""
GENERIC_MESSAGE = "Single sign-on failed. Please try again or contact your administrator."
class FakeStrategy:
# Mirror social_core's DjangoStrategy.setting(), which reads SOCIAL_AUTH_<NAME> from Django
# settings. This ensures the test exercises the real configured values (e.g.
# SOCIAL_AUTH_LOGIN_ERROR_URL) rather than hardcoded stand-ins.
def setting(self, name, default=None, backend=None):
return getattr(settings, f'SOCIAL_AUTH_{name}', default)
class FakeBackend:
name = 'saml'
def setUp(self):
self.factory = RequestFactory()
self.middleware = SocialAuthExceptionMiddleware(lambda request: None)
def _make_request(self):
request = self.factory.get('/')
request.social_strategy = self.FakeStrategy()
request.backend = self.FakeBackend()
# Attach message storage (normally provided by MessageMiddleware)
setattr(request, 'session', {})
request._messages = FallbackStorage(request)
return request
def test_generic_message(self):
"""
The raw exception text should never be surfaced to the user.
"""
request = self._make_request()
exception = AuthFailed(self.FakeBackend(), 'raw internal SAML detail')
self.assertEqual(self.middleware.get_message(request, exception), self.GENERIC_MESSAGE)
def test_redirect_on_failure(self):
"""
A SocialAuthBaseException should redirect to the login page with the generic message set.
"""
request = self._make_request()
exception = AuthFailed(self.FakeBackend(), 'raw internal SAML detail')
response = self.middleware.process_exception(request, exception)
self.assertEqual(response.status_code, 302)
self.assertEqual(response.url, settings.SOCIAL_AUTH_LOGIN_ERROR_URL)
self.assertEqual(response.url, settings.LOGIN_URL)
messages = [str(m) for m in request._messages]
self.assertEqual(messages, [self.GENERIC_MESSAGE])

202
netbox/netbox/tests/test_graphql.py
View File

@ -1,4 +1,5 @@
import json
import re
import strawberry
from django.contrib.contenttypes.models import ContentType
@ -10,15 +11,18 @@ from strawberry.schema.config import StrawberryConfig
from dcim.choices import LocationStatusChoices
from dcim.models import Device, DeviceRole, DeviceType, Location, Manufacturer, Site, VirtualChassis
from extras.models import TableConfig
from extras.models import TableConfig, Tag
from netbox.graphql.scalars import BigInt, BigIntScalar
from netbox.graphql.schema import Query, get_schema_extensions
from netbox.graphql.schema import Query, get_schema_extensions, schema
from utilities.tables import get_table_for_model
from utilities.testing import APITestCase, TestCase, disable_warnings
class GraphQLTestCase(TestCase):
def _schema_extension_instances(self):
return [factory() for factory in get_schema_extensions()]
@override_settings(GRAPHQL_ENABLED=False)
def test_graphql_enabled(self):
"""
@ -32,21 +36,21 @@ class GraphQLTestCase(TestCase):
"""
QueryDepthLimiter should not be installed when GRAPHQL_MAX_QUERY_DEPTH is unset.
"""
self.assertFalse(any(isinstance(ext, QueryDepthLimiter) for ext in get_schema_extensions()))
self.assertFalse(any(isinstance(ext, QueryDepthLimiter) for ext in self._schema_extension_instances()))
@override_settings(GRAPHQL_MAX_QUERY_DEPTH=0)
def test_graphql_max_query_depth_disabled_when_zero(self):
"""
QueryDepthLimiter should not be installed when GRAPHQL_MAX_QUERY_DEPTH is zero.
"""
self.assertFalse(any(isinstance(ext, QueryDepthLimiter) for ext in get_schema_extensions()))
self.assertFalse(any(isinstance(ext, QueryDepthLimiter) for ext in self._schema_extension_instances()))
@override_settings(GRAPHQL_MAX_QUERY_DEPTH=-1)
def test_graphql_max_query_depth_disabled_when_negative(self):
"""
QueryDepthLimiter should not be installed when GRAPHQL_MAX_QUERY_DEPTH is negative.
"""
self.assertFalse(any(isinstance(ext, QueryDepthLimiter) for ext in get_schema_extensions()))
self.assertFalse(any(isinstance(ext, QueryDepthLimiter) for ext in self._schema_extension_instances()))
@override_settings(GRAPHQL_MAX_QUERY_DEPTH=3)
def test_graphql_max_query_depth_enforced(self):
@ -54,9 +58,9 @@ class GraphQLTestCase(TestCase):
Queries exceeding GRAPHQL_MAX_QUERY_DEPTH should be rejected.
"""
extensions = get_schema_extensions()
self.assertTrue(any(isinstance(ext, QueryDepthLimiter) for ext in extensions))
self.assertTrue(any(isinstance(ext, QueryDepthLimiter) for ext in self._schema_extension_instances()))
# Build a temporary schema with the configured extensions and execute a deep query
# Build a temporary schema with the configured extension factories and execute a deep query
test_schema = strawberry.Schema(
query=Query,
config=StrawberryConfig(auto_camel_case=False, scalar_map={BigInt: BigIntScalar}),
@ -87,6 +91,30 @@ class GraphQLTestCase(TestCase):
with disable_warnings('django.request'):
self.assertHttpStatus(response, 302) # Redirect to login page
def test_json_lookup_schema_is_string_backed(self):
"""JSONLookup date/time lookups keep the legacy string-backed input types and fields."""
sdl = schema.as_str()
def input_block(name):
match = re.search(rf'^input {re.escape(name)}\b.*?^\}}', sdl, re.DOTALL | re.MULTILINE)
self.assertIsNotNone(match, f'{name} not found in schema')
return match.group(0)
# JSONLookup points at the legacy string-backed lookup type names
json_lookup = input_block('JSONLookup')
self.assertIn('date_lookup: StrDateFilterLookup', json_lookup)
self.assertIn('datetime_lookup: StrDatetimeFilterLookup', json_lookup)
self.assertIn('time_lookup: StrTimeFilterLookup', json_lookup)
# Value fields are string-backed, not Date/DateTime/Time scalars
self.assertIn('exact: String', input_block('StrDateFilterLookup'))
# Legacy date/time sub-lookups remain integer comparison lookups
for name in ('StrTimeFilterLookup', 'StrDatetimeFilterLookup'):
block = input_block(name)
self.assertIn('date: IntComparisonFilterLookup', block)
self.assertIn('time: IntComparisonFilterLookup', block)
class GraphQLAPITestCase(APITestCase):
@ -185,6 +213,72 @@ class GraphQLAPITestCase(APITestCase):
self.assertNotIn('errors', data)
self.assertEqual(len(data['data']['site']['locations']), 0)
@override_settings(LOGIN_REQUIRED=True)
def test_graphql_nested_filter_objects(self):
"""
Test filtering of nested GraphQL object lists.
"""
self.add_permissions('dcim.view_site', 'dcim.view_location', 'extras.view_tag')
site = Site.objects.create(
name='Nested Filter Site',
slug='nested-filter-site'
)
# Location is MPTT-managed; bulk_create skips tree-init hooks. Use per-instance create.
Location.objects.create(
site=site,
name='Nested Active 1',
slug='nested-active-1',
status=LocationStatusChoices.STATUS_ACTIVE,
)
Location.objects.create(
site=site,
name='Nested Active 2',
slug='nested-active-2',
status=LocationStatusChoices.STATUS_ACTIVE,
)
Location.objects.create(
site=site,
name='Nested Planned',
slug='nested-planned',
status=LocationStatusChoices.STATUS_PLANNED,
)
planned = Tag.objects.create(name='Planned', slug='planned')
production = Tag.objects.create(name='Production', slug='production')
staging = Tag.objects.create(name='Staging', slug='staging')
site.tags.add(planned, production, staging)
url = reverse('graphql')
query = f"""
{{
site(id: {site.pk}) {{
locations(filters: {{status: {{exact: STATUS_ACTIVE}}}}) {{
name
}}
tags(filters: {{name: {{i_starts_with: "P"}}}}) {{
name
}}
}}
}}
"""
response = self.client.post(url, data={'query': query}, format="json", **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
data = json.loads(response.content)
self.assertNotIn('errors', data)
self.assertEqual(
{location['name'] for location in data['data']['site']['locations']},
{'Nested Active 1', 'Nested Active 2'}
)
self.assertEqual(
{tag['name'] for tag in data['data']['site']['tags']},
{'Planned', 'Production'}
)
def test_graphql_integer_range_lookup(self):
"""
Test that range_lookup works for integer fields (e.g. vc_position). Regression test for #20468.
@ -411,3 +505,97 @@ class GraphQLAPITestCase(APITestCase):
data = json.loads(response.content)
self.assertIn('errors', data)
self.assertEqual(data['errors'][0]['message'], 'Cannot specify both `start` and `offset` in pagination.')
class JSONPathValidationTestCase(TestCase):
"""Unit tests for _validate_json_path (VM-323 security fix)."""
def setUp(self):
from netbox.graphql.filter_lookups import _validate_json_path
self.validate = _validate_json_path
# --- Valid paths ---
def test_single_key(self):
self.assertEqual(self.validate('key'), 'key')
def test_nested_key(self):
self.assertEqual(self.validate('parent__child'), 'parent__child')
def test_deeply_nested(self):
self.assertEqual(self.validate('a__b__c'), 'a__b__c')
def test_key_with_underscores(self):
self.assertEqual(self.validate('my_key'), 'my_key')
def test_key_with_hyphens(self):
self.assertEqual(self.validate('my-key'), 'my-key')
def test_numeric_array_index(self):
self.assertEqual(self.validate('items__0'), 'items__0')
def test_alphanumeric_segment(self):
self.assertEqual(self.validate('key123'), 'key123')
def test_key_with_leading_underscore(self):
# JSON keys may start with underscore (e.g. _foo)
self.assertEqual(self.validate('_key'), '_key')
def test_orm_operator_name_as_key(self):
# 'date', 'regex' etc. are valid JSON key names; the path validator
# must not block them. The ORM injection risk is neutralised by the
# trailing __ that JSONFilter always appends before process_filters.
self.assertEqual(self.validate('date'), 'date')
self.assertEqual(self.validate('key__regex'), 'key__regex')
self.assertEqual(self.validate('key__exact'), 'key__exact')
# --- Invalid paths ---
def test_rejects_empty_string(self):
with self.assertRaises(ValueError):
self.validate('')
def test_rejects_all_underscores(self):
# '___' splits into segments ['', '', ''] via '__' — empty segments rejected
with self.assertRaises(ValueError):
self.validate('___')
def test_accepts_trailing_single_underscore(self):
# A single trailing underscore is a valid JSON key character
self.assertEqual(self.validate('key_'), 'key_')
def test_rejects_trailing_double_underscore(self):
with self.assertRaises(ValueError):
self.validate('key__')
def test_rejects_leading_double_underscore(self):
with self.assertRaises(ValueError):
self.validate('__key')
def test_rejects_consecutive_double_underscores(self):
with self.assertRaises(ValueError):
self.validate('key1____key2')
def test_rejects_segment_starting_with_special_char(self):
with self.assertRaises(ValueError):
self.validate('$secret')
def test_rejects_path_with_spaces(self):
with self.assertRaises(ValueError):
self.validate('key one')
def test_rejects_path_with_dot(self):
with self.assertRaises(ValueError):
self.validate('key.subkey')
class JSONStringLookupTestCase(TestCase):
"""Verify JSONStringLookup exposes the expected set of string operators."""
def test_string_operators_present(self):
from netbox.graphql.filter_lookups import JSONStringLookup
field_names = {f.name for f in JSONStringLookup.__strawberry_definition__.fields}
for expected in ('exact', 'i_exact', 'contains', 'i_contains',
'starts_with', 'i_starts_with', 'ends_with', 'i_ends_with',
'in_', 'isnull', 'regex', 'i_regex'):
self.assertIn(expected, field_names, f"{expected!r} must be present on JSONStringLookup")

10
netbox/netbox/tests/test_models.py
View File

@ -4,6 +4,7 @@ from django.conf import settings
from django.test import TestCase
from core.models import ObjectChange
from dcim.models import Site
from netbox.tests.dummy_plugin.models import DummyNetBoxModel
@ -21,3 +22,12 @@ class ModelTestCase(TestCase):
m.pk = 123
self.assertEqual(m.get_absolute_url(), f'/plugins/dummy-plugin/netboxmodel/{m.pk}/')
class DeleteMixinTestCase(TestCase):
def test_delete_unsaved_instance_raises_value_error(self):
"""Deleting an instance with no primary key raises ValueError."""
site = Site(name='Site 1', slug='site-1')
with self.assertRaises(ValueError):
site.delete()

24
netbox/netbox/tests/test_ui.py
View File

@ -3,6 +3,7 @@ from types import SimpleNamespace
from django.template import Context, Template
from django.test import RequestFactory, SimpleTestCase, TestCase
from netaddr import IPNetwork
from circuits.choices import CircuitStatusChoices, VirtualCircuitTerminationRoleChoices
from circuits.models import (
@ -250,6 +251,29 @@ class TextAttrTestCase(TestCase):
self.assertTrue(context['copy_button'])
class ArrayAttrTestCase(TestCase):
def test_get_value(self):
attr = attrs.ArrayAttr('allowed_ips')
obj = SimpleNamespace(allowed_ips=[IPNetwork('192.168.1.1/32'), IPNetwork('2001:db8::/64')])
self.assertEqual(attr.get_value(obj), '192.168.1.1/32, 2001:db8::/64')
def test_get_value_empty(self):
attr = attrs.ArrayAttr('allowed_ips')
obj = SimpleNamespace(allowed_ips=[])
self.assertIsNone(attr.get_value(obj))
def test_get_value_none(self):
attr = attrs.ArrayAttr('allowed_ips')
obj = SimpleNamespace(allowed_ips=None)
self.assertIsNone(attr.get_value(obj))
def test_get_value_with_format_string(self):
attr = attrs.ArrayAttr('ports', format_string='{}/tcp')
obj = SimpleNamespace(ports=[80, 443])
self.assertEqual(attr.get_value(obj), '80/tcp, 443/tcp')
class NumericAttrTestCase(TestCase):
def test_get_context_with_unit_accessor(self):

69
netbox/netbox/tests/test_views.py
View File

@ -1,9 +1,13 @@
import urllib.parse
from unittest.mock import patch
from django.contrib.contenttypes.models import ContentType
from django.http import HttpResponse
from django.test import Client, override_settings
from django.urls import reverse
from dcim.models import Site
from dcim.models import DeviceType, Manufacturer, Site
from extras.models import ImageAttachment
from netbox.constants import EMPTY_TABLE_TEXT
from netbox.search.backends import search_backend
from utilities.testing import TestCase
@ -78,6 +82,27 @@ class SearchViewTestCase(TestCase):
class MediaViewTestCase(TestCase):
@classmethod
def setUpTestData(cls):
site = Site.objects.create(name='Site 1', slug='site-1')
ct = ContentType.objects.get_for_model(Site)
cls.image_attachment = ImageAttachment.objects.create(
object_type=ct,
object_id=site.pk,
name='Test Image',
image='image-attachments/site_1_test.jpg',
image_height=100,
image_width=100,
)
manufacturer = Manufacturer.objects.create(name='Manufacturer 1', slug='manufacturer-1')
cls.device_type = DeviceType.objects.create(
model='Device Type 1',
slug='device-type-1',
manufacturer=manufacturer,
front_image='devicetype-images/front.jpg',
)
def test_media_login_required(self):
url = reverse('media', kwargs={'path': 'foo.txt'})
response = Client().get(url)
@ -92,3 +117,45 @@ class MediaViewTestCase(TestCase):
# Unauthenticated request should return a 404 (not found)
self.assertHttpStatus(response, 404)
def test_image_attachment_with_permission(self):
self.add_permissions('extras.view_imageattachment')
url = reverse('media', kwargs={'path': self.image_attachment.image.name})
with patch('netbox.views.misc.serve', return_value=HttpResponse(status=200)):
response = self.client.get(url)
self.assertHttpStatus(response, 200)
self.assertEqual(response['Content-Disposition'], 'attachment')
self.assertEqual(response['X-Content-Type-Options'], 'nosniff')
def test_image_attachment_without_permission(self):
url = reverse('media', kwargs={'path': self.image_attachment.image.name})
response = self.client.get(url)
self.assertHttpStatus(response, 404)
def test_image_attachment_traversal_without_permission(self):
# A traversal path that normalizes to a protected directory must still be denied.
traversal_path = 'foo/../' + self.image_attachment.image.name
url = reverse('media', kwargs={'path': traversal_path})
response = self.client.get(url)
self.assertHttpStatus(response, 404)
def test_device_type_with_permission(self):
self.add_permissions('dcim.view_devicetype')
url = reverse('media', kwargs={'path': self.device_type.front_image.name})
with patch('netbox.views.misc.serve', return_value=HttpResponse(status=200)):
response = self.client.get(url)
self.assertHttpStatus(response, 200)
self.assertEqual(response['Content-Disposition'], 'attachment')
self.assertEqual(response['X-Content-Type-Options'], 'nosniff')
def test_device_type_without_permission(self):
url = reverse('media', kwargs={'path': self.device_type.front_image.name})
response = self.client.get(url)
self.assertHttpStatus(response, 404)
def test_device_type_traversal_without_permission(self):
# A traversal path that normalizes to a protected directory must still be denied.
traversal_path = 'foo/../' + self.device_type.front_image.name
url = reverse('media', kwargs={'path': traversal_path})
response = self.client.get(url)
self.assertHttpStatus(response, 404)

17
netbox/netbox/ui/attrs.py
View File

@ -8,6 +8,7 @@ from utilities.data import resolve_attr_path
__all__ = (
'AddressAttr',
'ArrayAttr',
'BooleanAttr',
'ChoiceAttr',
'ColorAttr',
@ -141,6 +142,22 @@ class TextAttr(ObjectAttribute):
}
class ArrayAttr(TextAttr):
"""
An attribute comprising an array of values, rendered as a comma-separated list. If specified, `format_string`
is applied to each item individually. Null and empty arrays are treated as equivalent: both render as the
placeholder.
"""
def get_value(self, obj):
value = resolve_attr_path(obj, self.accessor)
if not value:
return None
if self.format_string:
return ', '.join(self.format_string.format(v) for v in value)
return ', '.join(str(v) for v in value)
class NumericAttr(ObjectAttribute):
"""
An integer or float attribute.

113
netbox/netbox/views/generic/bulk_views.py
View File

@ -8,7 +8,7 @@ from types import SimpleNamespace
from django.conf import settings
from django.contrib import messages
from django.contrib.contenttypes.fields import GenericForeignKey, GenericRel
from django.core.exceptions import FieldDoesNotExist, ObjectDoesNotExist, ValidationError
from django.core.exceptions import FieldDoesNotExist, ImproperlyConfigured, ObjectDoesNotExist, ValidationError
from django.db import IntegrityError, router, transaction
from django.db.models import ManyToManyField, ProtectedError, RestrictedError
from django.db.models.fields.reverse_related import ManyToManyRel
@ -27,7 +27,7 @@ from netbox.forms.bulk_rename import NetBoxModelBulkRenameForm
from netbox.models.features import ChangeLoggingMixin
from netbox.object_actions import AddObject, BulkDelete, BulkEdit, BulkExport, BulkImport, BulkRename
from utilities.error_handlers import handle_protectederror
from utilities.exceptions import AbortRequest, PermissionsViolation
from utilities.exceptions import AbortRequest, AbortTransaction, PermissionsViolation
from utilities.export import TableExport, stream_table_csv_response
from utilities.forms import BulkDeleteForm, BulkRenameForm, restrict_form_fields
from utilities.forms.bulk_import import BulkImportForm
@ -210,7 +210,7 @@ class ObjectListView(BaseMultiObjectView, ActionsMixin, TableMixin):
if request.GET['export'] == 'table':
table = self.get_table(self.queryset, request, has_table_actions)
columns = [name for name, _ in table.selected_columns]
delimiter = request.user.config.get('csv_delimiter')
delimiter = request.user.config.get('csv_delimiter') if request.user.is_authenticated else None
return self.export_table(table, columns, delimiter=delimiter)
# Render an ExportTemplate
@ -231,7 +231,7 @@ class ObjectListView(BaseMultiObjectView, ActionsMixin, TableMixin):
# Fall back to default table/YAML export
table = self.get_table(self.queryset, request, has_table_actions)
delimiter = request.user.config.get('csv_delimiter')
delimiter = request.user.config.get('csv_delimiter') if request.user.is_authenticated else None
return self.export_table(table, delimiter=delimiter)
# Render the objects table
@ -274,11 +274,96 @@ class BulkCreateView(GetReturnURLMixin, BaseMultiObjectView):
form = None
model_form = None
pattern_target = ''
pattern_template_fields = ()
htmx_template_name = 'htmx/bulk_add_form.html'
def get_required_permission(self):
return get_permission_for_model(self.queryset.model, 'add')
def get_pattern_context(self, value):
"""
Return a context mapping for substituting the generated pattern value into
model form fields.
By default, the field named by ``pattern_target`` is supported as a
placeholder, e.g. ``{vid}``.
"""
if not self.pattern_target:
return {}
return {
self.pattern_target: str(value),
}
def render_pattern_template(self, template, value):
"""
Replace pattern placeholders in a single form field value.
"""
rendered = str(template)
for key, replacement in self.get_pattern_context(value).items():
rendered = rendered.replace(f'{{{key}}}', replacement)
return rendered
def apply_pattern_template_fields(self, data, value):
"""
Apply the generated pattern value to any configured template fields.
"""
for field_name in self.pattern_template_fields:
if field_name not in data:
continue
# QueryDict values may be multi-valued; preserve that behavior.
if hasattr(data, 'getlist') and hasattr(data, 'setlist'):
data.setlist(field_name, [
self.render_pattern_template(field_value, value)
for field_value in data.getlist(field_name)
])
else:
data[field_name] = self.render_pattern_template(data[field_name], value)
return data
def get_model_form_data(self, form, request, value):
"""
Return the submitted data to use when instantiating the model form for a
single generated pattern value.
"""
data = request.POST.copy()
data[self.pattern_target] = value
return self.apply_pattern_template_fields(data, value)
def add_model_form_errors(self, form, model_form, value):
"""
Copy validation errors from the generated object's model form back onto
the pattern form for display.
"""
errors = model_form.errors.as_data()
if errors.get(self.pattern_target):
form.add_error('pattern', errors.pop(self.pattern_target))
for field_name, field_errors in errors.items():
if field_name == '__all__':
field_label = _('General')
elif field_name in model_form.fields:
field_label = model_form.fields[field_name].label
else:
field_label = field_name
for error in field_errors:
for message in error.messages:
form.add_error(
None,
_('{value}: {field}: {error}').format(
value=value,
field=field_label,
error=message,
)
)
def _create_objects(self, form, request):
new_objects = []
@ -287,8 +372,7 @@ class BulkCreateView(GetReturnURLMixin, BaseMultiObjectView):
# Reinstantiate the model form each time to avoid overwriting the same instance. Use a mutable
# copy of the POST QueryDict so that we can update the target field value.
model_form = self.model_form(request.POST.copy())
model_form.data[self.pattern_target] = value
model_form = self.model_form(self.get_model_form_data(form, request, value))
# Validate each new object independently.
if model_form.is_valid():
@ -296,12 +380,10 @@ class BulkCreateView(GetReturnURLMixin, BaseMultiObjectView):
obj = model_form.save()
new_objects.append(obj)
else:
# Copy any errors on the pattern target field to the pattern form.
errors = model_form.errors.as_data()
if errors.get(self.pattern_target):
form.add_error('pattern', errors[self.pattern_target])
# Raise an IntegrityError to break the for loop and abort the transaction.
raise IntegrityError()
self.add_model_form_errors(form, model_form, value)
# Abort the transaction and break out of the loop.
raise AbortTransaction()
return new_objects
@ -372,7 +454,7 @@ class BulkCreateView(GetReturnURLMixin, BaseMultiObjectView):
return redirect(request.path)
return redirect(self.get_return_url(request))
except IntegrityError:
except (AbortTransaction, IntegrityError):
pass
except (AbortRequest, PermissionsViolation) as e:
@ -758,7 +840,10 @@ class BulkEditView(GetReturnURLMixin, BaseMultiObjectView):
# Update custom fields
for name, customfield in custom_fields.items():
assert name.startswith('cf_')
if not name.startswith('cf_'):
raise ImproperlyConfigured(
_("Custom field form field name must begin with 'cf_': {name}").format(name=name)
)
cf_name = name[3:] # Strip cf_ prefix
if name in form.nullable_fields and name in nullified_fields:
obj.custom_field_data[cf_name] = None

31
netbox/netbox/views/misc.py
View File

@ -1,4 +1,5 @@
import logging
import posixpath
import re
from collections import namedtuple
@ -6,6 +7,8 @@ from django.conf import settings
from django.contrib import messages
from django.contrib.contenttypes.models import ContentType
from django.core.cache import cache
from django.db.models import Q
from django.http import Http404
from django.shortcuts import redirect, render
from django.utils.translation import gettext_lazy as _
from django.views.generic import View
@ -13,8 +16,10 @@ from django.views.static import serve
from django_tables2 import RequestConfig
from packaging import version
from dcim.models import DeviceType
from extras.constants import DEFAULT_DASHBOARD
from extras.dashboard.utils import get_dashboard, get_default_dashboard
from extras.models import ImageAttachment
from netbox.forms import SearchForm
from netbox.search import LookupTypes
from netbox.search.backends import search_backend
@ -131,7 +136,29 @@ class SearchView(ConditionalLoginRequiredMixin, View):
class MediaView(TokenConditionalLoginRequiredMixin, View):
"""
Wrap Django's serve() view to enforce LOGIN_REQUIRED for static media.
Serve uploaded media files, enforcing authentication and view permission on the associated object.
"""
def get(self, request, path):
return serve(request, path, document_root=settings.MEDIA_ROOT)
# Normalize the path to prevent traversal sequences (e.g. "foo/../image-attachments/...")
# from bypassing the directory checks below.
path = posixpath.normpath(path).lstrip('/')
# For known upload directories, resolve the path to an owning record and
# enforce object-level view permission. restrict() returns .none() when the
# user lacks permission, so a denial and a missing file are both 404s.
# Paths outside these directories (e.g. plugin uploads) fall through
# to the original behaviour.
if path.startswith('image-attachments/'):
if not ImageAttachment.objects.restrict(request.user, 'view').filter(image=path).exists():
raise Http404
elif path.startswith('devicetype-images/'):
if not DeviceType.objects.restrict(request.user, 'view').filter(
Q(front_image=path) | Q(rear_image=path)
).exists():
raise Http404
response = serve(request, path, document_root=settings.MEDIA_ROOT)
response['Content-Disposition'] = 'attachment'
response['X-Content-Type-Options'] = 'nosniff'
return response

2
netbox/project-static/dist/netbox.js vendored
View File

File diff suppressed because one or more lines are too long

2
netbox/project-static/dist/netbox.js.map vendored
View File

File diff suppressed because one or more lines are too long

12
netbox/project-static/package.json
View File

@ -31,7 +31,7 @@
"gridstack": "12.6.0",
"htmx.org": "2.0.10",
"query-string": "9.4.0",
"sass": "1.100.0",
"sass": "1.101.0",
"tom-select": "2.6.1",
"typeface-inter": "3.18.1",
"typeface-roboto-mono": "1.1.13"
@ -43,17 +43,17 @@
"@types/bootstrap": "5.2.11",
"@types/cookie": "^1.0.0",
"@types/node": "^24.10.1",
"@typescript-eslint/eslint-plugin": "^8.60.1",
"@typescript-eslint/parser": "^8.60.1",
"esbuild": "^0.28.0",
"@typescript-eslint/eslint-plugin": "^8.61.1",
"@typescript-eslint/parser": "^8.61.1",
"esbuild": "^0.28.1",
"esbuild-sass-plugin": "^3.7.0",
"eslint": "^10.4.1",
"eslint": "^10.5.0",
"eslint-config-prettier": "^10.1.8",
"eslint-import-resolver-typescript": "^4.4.5",
"eslint-plugin-import": "^2.32.0",
"eslint-plugin-prettier": "^5.5.6",
"globals": "^17.5.0",
"prettier": "^3.8.3",
"prettier": "^3.8.4",
"typescript": "^5.9.3"
},
"resolutions": {

Some files were not shown because too many files have changed in this diff Show More