#20285 - Support multiple protocols per application service via port mappings (#22692)

This commit is contained in:
Arthur Hanson 2026-08-03 07:58:01 -07:00 committed by GitHub
parent d2024a1edc
commit 0270cf1495
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
37 changed files with 2957 additions and 392 deletions

View File

@ -23,14 +23,62 @@ The parent object to which the application service is assigned. This must be one
A service or protocol name.
### Protocol
### Port Mappings
The wire protocol on which the service runs. Choices include UDP, TCP, and SCTP.
The protocols and ports on which the service runs. A service may expose the same port on multiple protocols — for example, DNS listening on both `tcp/53` and `udp/53`. In the UI, ports for a given protocol may be entered together using commas and/or hyphens (e.g. `80,8001-8003`).
### Ports
In the REST and GraphQL APIs, port mappings are represented as a flat list of `protocol/port` strings — matching how they are stored:
One or more numeric ports to which the service is bound. Multiple ports can be expressed using commas and/or hyphens. For example, `80,8001-8003` specifies ports 80, 8001, 8002, and 8003.
```json
[
"tcp/80",
"tcp/443",
"udp/53"
]
```
!!! note "Changed in NetBox v4.7"
The single-protocol `protocol` and `ports` fields have been replaced by the unified `port_mappings` field, which supports multiple protocols per service. For backward compatibility, the REST and GraphQL APIs still expose the legacy `protocol` and `ports` fields, and the REST API still accepts them on write as an alternative to `port_mappings`. They are populated for single-protocol services; a service with multiple protocols cannot be represented in the legacy format and returns `null` for both, while a service with no mappings returns `protocol: null` and `ports: []`. In other words, `ports: null` specifically signals "multiple protocols — read `port_mappings` instead." **These legacy fields are deprecated and will be removed in NetBox v5.0; use `port_mappings` instead.**
On write, `port_mappings` and the legacy `protocol`/`ports` fields may be submitted together only when they agree — as in a full-object round-trip that echoes back a read. A request whose legacy fields contradict `port_mappings` (for example, an edited `port_mappings` sent alongside the stale `protocol`/`ports` from the original read) is rejected as ambiguous; send `port_mappings` alone, or keep the legacy fields consistent with it.
At the ORM level (custom scripts and plugins), `protocol` and `ports` are now **read-only** properties derived from `port_mappings`. Assign `port_mappings` directly — e.g. `Service(parent=device, name='http', port_mappings=['tcp/80'])` — since passing `protocol=`/`ports=` to the model raises `TypeError` and setting `service.ports = [...]` raises `AttributeError`.
### Filtering by Port Mapping, Protocol, and Port
`port_mappings`, `protocol`, and `port` are all filtered against the `port_mappings` array. Each accepts multiple values (matching any of them), and `port` supports the usual numeric lookups:
| Parameter | Matches services having a mapping… |
|---|---|
| `?port_mappings=tcp/80` | that is exactly `tcp/80` |
| `?port_mappings__n=tcp/80` | *(negated)* that is exactly `tcp/80` |
| `?protocol=tcp` | whose protocol is TCP |
| `?protocol__n=tcp` | *(negated)* whose protocol is TCP |
| `?port=80` | whose port is 80 |
| `?port__n=80` | *(negated)* whose port is 80 |
| `?port__gt=` / `?port__gte=` / `?port__lt=` / `?port__lte=` | whose port is above/below the given value |
`port_mappings` is the most direct way to ask "which services expose this exact protocol and port?" — `?port_mappings=tcp/80` will not match a service that exposes only `udp/80`. Protocols may be given in any case, and leading zeros are ignored, so `?port_mappings=TCP/080` finds `tcp/80`. A value naming an unknown protocol or a malformed pair simply matches nothing rather than returning an error.
When `protocol` and one or more `port` lookups are combined, they must all be satisfied by a **single** mapping. So `?protocol=tcp&port__gt=1000` does not match a service whose only TCP mapping is `tcp/80` (even if it also exposes `udp/9999`), and `?port__gte=1000&port__lte=2000` does not match a service exposing only ports 500 and 5000. Each `port_mappings` value already names one complete pair, so it needs no such correlation and is simply combined with the other parameters.
All of these parameters are available as GraphQL filters too, under the same names — `port_mappings`, `protocol`, `port`, `port__gt`, `port__gte`, `port__lt`, `port__lte` — each accepting a list of values. For example, `filters: {port_mappings: ["tcp/80"]}` or `filters: {protocol: [TCP], port__gt: [1000]}`. The single-mapping correlation rule described above applies identically.
!!! warning "GraphQL filter change in NetBox v4.7"
The GraphQL filters for `Service` and `ServiceTemplate` have changed shape. The former `protocol` lookup and `ports` integer lookup (which nested their comparisons, e.g. `ports: {gt: 1000}`) are replaced by the flat `protocol`, `port`, `port__gt`, `port__gte`, `port__lt`, `port__lte`, and `port_mappings` parameters, each accepting a list of values and spelled the same way as the corresponding REST query parameter. Rewrite `ports: {gt: 1000}` as `port__gt: [1000]`, and `ports: {exact: 80}` as `port: [80]`. The `range` and `i_exact` lookups previously offered by the integer lookup have no direct equivalent; express a range as `port__gte`/`port__lte`, which — unlike the old lookup — requires a single mapping to satisfy both bounds.
The members of the `ServiceProtocolEnum` used by the `protocol` filter have also been renamed to drop a spurious `ROLE_` prefix: `ROLE_TCP`, `ROLE_UDP`, and `ROLE_SCTP` are now `TCP`, `UDP`, and `SCTP`.
!!! warning "REST filter change in NetBox v4.7"
Because `protocol` is now filtered against the `port_mappings` array rather than a dedicated model field, the character-based lookup variants previously auto-generated for it — `protocol__ic`, `protocol__nic`, `protocol__isw`, `protocol__empty`, etc. — are no longer available; `protocol` and `protocol__n` remain. The `port__empty` lookup is likewise gone, as a service always has at least one port mapping. As with any unrecognized query parameter, the REST API silently ignores a removed lookup rather than raising an error, so update any saved filters or scripts that relied on them.
### IP Addresses
The [IP address(es)](./ipaddress.md) to which this service is bound. If no IP addresses are bound, the service is assumed to be reachable via any assigned IP address.
## Bulk Import (CSV)
When importing application services or [application service templates](./servicetemplate.md) via CSV, all port mappings for a row are given in a single `port_mappings` column as a comma-separated list of `protocol/port` pairs enclosed in double quotes. For example, `"tcp/80,tcp/443,udp/53"`. Protocols may be specified in any case.

View File

@ -12,10 +12,10 @@ Application service templates can be used to instantiate [application services](
A service or protocol name.
### Protocol
### Port Mappings
The wire protocol on which the service runs. Choices include UDP, TCP, and SCTP.
The protocols and ports on which the service runs. See [Port Mappings](./service.md#port-mappings) on the application service model for details.
### Ports
## Bulk Import (CSV)
One or more numeric ports to which the service is bound. Multiple ports can be expressed using commas and/or hyphens. For example, `80,8001-8003` specifies ports 80, 8001, 8002, and 8003.
Application service templates are imported via CSV using the same `port_mappings` column format as application services. See [Bulk Import (CSV)](./service.md#bulk-import-csv) on the application service model for details.

View File

@ -1,8 +1,13 @@
from django.contrib.contenttypes.models import ContentType
from django.core.exceptions import ValidationError as DjangoValidationError
from django.utils.translation import gettext as _
from rest_framework import serializers
from ipam.choices import *
from ipam.constants import SERVICE_ASSIGNMENT_MODELS
from ipam.constants import SERVICE_ASSIGNMENT_MODELS, SERVICE_PORT_MAX, SERVICE_PORT_MIN
from ipam.models import IPAddress, Service, ServiceTemplate
from ipam.utils import legacy_protocol_and_ports
from ipam.validators import validate_port_mappings
from netbox.api.fields import ChoiceField, ContentTypeField, SerializedPKRelatedField
from netbox.api.gfk_fields import GFKSerializerField
from netbox.api.serializers import PrimaryModelSerializer
@ -15,20 +20,126 @@ __all__ = (
)
class ServiceTemplateSerializer(PrimaryModelSerializer):
protocol = ChoiceField(choices=ServiceProtocolChoices, required=False)
class PortMappingsField(serializers.ListField):
"""
A service's port mappings as a flat list of ``protocol/port`` strings (e.g. ``["tcp/80", "udp/53"]``),
matching how they are stored. Each entry is validated (and normalized) on write.
"""
child = serializers.CharField()
def to_internal_value(self, data):
mappings = super().to_internal_value(data)
try:
return validate_port_mappings(mappings)
except DjangoValidationError as exc:
raise serializers.ValidationError(exc.messages)
class PortMappingsSerializerMixin(serializers.Serializer):
"""
Shared port-mapping handling for the Service and ServiceTemplate serializers, including backward
compatibility for the legacy single-protocol ``protocol``/``ports`` representation.
Read: alongside the ``port_mappings`` list, a service that uses a single protocol also reports the
legacy ``protocol`` and ``ports`` fields; a multi-protocol service reports ``null`` for both (it
cannot be expressed in the old single-protocol format).
Write: either format is accepted. When the legacy ``protocol``/``ports`` pair is supplied (and
``port_mappings`` is not), it is translated into ``port_mappings``. Supplying both together is
accepted only when the legacy fields agree with what ``port_mappings`` implies (e.g. a full-object
round-trip that echoes back the read representation); a genuine conflict is rejected as ambiguous.
Subclassing ``serializers.Serializer`` (rather than a plain mixin) lets DRF's metaclass collect the
fields declared here into the inheriting serializers.
"""
port_mappings = PortMappingsField(required=False)
# Legacy single-protocol fields, retained for backward compatibility. They are read straight off the
# model's protocol/ports properties (which share these field names), so DRF sources them directly —
# matching the {"value", "label"} shape every other choice field uses. default=None applies only on
# write, where validate() consumes them.
# TODO: Remove protocol/ports in v5.0 along with the legacy handling in validate().
protocol = ChoiceField(
choices=ServiceProtocolChoices,
required=False,
allow_null=True,
default=None,
help_text=_("Deprecated; use port_mappings. Reported only for single-protocol services."),
)
ports = serializers.ListField(
child=serializers.IntegerField(min_value=SERVICE_PORT_MIN, max_value=SERVICE_PORT_MAX),
required=False,
allow_null=True,
default=None,
help_text=_("Deprecated; use port_mappings. Reported only for single-protocol services."),
)
def validate(self, data):
# Consume the legacy fields and translate them into port_mappings *before* calling super(),
# which instantiates the model (via full_clean()) and would choke on these now-nonexistent kwargs.
legacy_protocol = data.pop('protocol', None)
legacy_ports = data.pop('ports', None)
# protocol/ports carry default=None, so an omitted field arrives as None; an explicitly-supplied
# value (including a falsy ports=[]) is a legacy write and must be handled — checking `is not None`
# rather than truthiness so an intentional empty list isn't silently dropped.
if legacy_protocol is not None or legacy_ports is not None:
# `port_mappings` and `protocol`/`ports` are mutually exclusive as *representations*, but a
# full-object round-trip (GET then PUT/PATCH) legitimately resubmits port_mappings alongside
# the legacy protocol/ports the read emitted. Only reject a genuine *conflict*: when the legacy
# fields agree with what port_mappings already implies they're merely redundant, so accept the
# request and let port_mappings win.
if 'port_mappings' in data:
expected_protocol, expected_ports = legacy_protocol_and_ports(data['port_mappings'])
protocol_agrees = legacy_protocol is None or legacy_protocol == expected_protocol
ports_agree = legacy_ports is None or sorted(legacy_ports) == (expected_ports or [])
if not (protocol_agrees and ports_agree):
raise serializers.ValidationError(_(
"Specify either 'port_mappings' or the deprecated 'protocol'/'ports' fields, not both."
))
return super().validate(data)
# The old API accepted an empty ports list (the ArrayField had no minimum length); the new
# model requires at least one mapping. Report that directly — both fields may have been
# supplied, so the "both are required" message below would be misleading.
if legacy_ports == []:
raise serializers.ValidationError(
{'ports': _("At least one port mapping is required.")}
)
# The legacy API let either field be updated on its own (e.g. a PATCH that adjusts only the
# port list). Preserve that by backfilling the omitted field from the instance's current
# single-protocol representation.
if not (legacy_protocol and legacy_ports):
legacy_protocol = legacy_protocol or (self.instance.protocol if self.instance else None)
if legacy_ports is None:
legacy_ports = self.instance.ports if self.instance else None
# If the pair still can't be resolved — a create, or an existing multi-protocol service that
# has no single-protocol form — the request can't be expressed in the legacy format.
if not (legacy_protocol and legacy_ports):
raise serializers.ValidationError(_(
"Both 'protocol' and 'ports' are required when writing via the deprecated legacy "
"format; use port_mappings instead."
))
try:
data['port_mappings'] = validate_port_mappings(
[f'{legacy_protocol}/{port}' for port in legacy_ports]
)
except DjangoValidationError as exc:
raise serializers.ValidationError({'ports': exc.messages})
return super().validate(data)
class ServiceTemplateSerializer(PortMappingsSerializerMixin, PrimaryModelSerializer):
class Meta:
model = ServiceTemplate
fields = [
'id', 'url', 'display_url', 'display', 'name', 'protocol', 'ports', 'description', 'owner', 'comments',
'tags', 'custom_fields', 'created', 'last_updated',
'id', 'url', 'display_url', 'display', 'name', 'port_mappings', 'protocol', 'ports', 'description',
'owner', 'comments', 'tags', 'custom_fields', 'created', 'last_updated',
]
brief_fields = ('id', 'url', 'display', 'name', 'protocol', 'ports', 'description')
brief_fields = ('id', 'url', 'display', 'name', 'port_mappings', 'description')
class ServiceSerializer(PrimaryModelSerializer):
protocol = ChoiceField(choices=ServiceProtocolChoices, required=False)
class ServiceSerializer(PortMappingsSerializerMixin, PrimaryModelSerializer):
ipaddresses = SerializedPKRelatedField(
queryset=IPAddress.objects.all(),
serializer=IPAddressSerializer,
@ -45,7 +156,7 @@ class ServiceSerializer(PrimaryModelSerializer):
model = Service
fields = [
'id', 'url', 'display_url', 'display', 'parent_object_type', 'parent_object_id', 'parent', 'name',
'protocol', 'ports', 'ipaddresses', 'description', 'owner', 'comments', 'tags', 'custom_fields',
'created', 'last_updated',
'port_mappings', 'protocol', 'ports', 'ipaddresses', 'description', 'owner', 'comments', 'tags',
'custom_fields', 'created', 'last_updated',
]
brief_fields = ('id', 'url', 'display', 'name', 'protocol', 'ports', 'description')
brief_fields = ('id', 'url', 'display', 'name', 'port_mappings', 'description')

View File

@ -22,7 +22,6 @@ from utilities.filters import (
MultiValueCharFilter,
MultiValueContentTypeFilter,
MultiValueNumberFilter,
NumericArrayFilter,
TreeNodeMultipleChoiceFilter,
)
from utilities.filtersets import register_filterset
@ -31,6 +30,7 @@ from vpn.models import L2VPN
from .choices import *
from .models import *
from .utils import normalize_port_mapping, port_mapping_q
__all__ = (
'ASNFilterSet',
@ -1214,16 +1214,139 @@ class VLANTranslationRuleFilterSet(NetBoxModelFilterSet):
return queryset.filter(qs_filter)
@register_filterset
class ServiceTemplateFilterSet(PrimaryModelFilterSet):
port = NumericArrayFilter(
field_name='ports',
lookup_expr='contains'
# Service/ServiceTemplate port filter name -> the port lookup it applies, in the order the conditions
# are built. See ServicePortMappingFilterMixin and ipam.utils.PORT_MAPPING_LOOKUPS.
SERVICE_PORT_FILTERS = {
'port': 'exact',
'port__gt': 'gt',
'port__gte': 'gte',
'port__lt': 'lt',
'port__lte': 'lte',
}
class ServicePortMappingFilterMixin(django_filters.FilterSet):
"""
Shared ``port_mappings``, ``protocol`` and ``port`` filtering for Service and ServiceTemplate, all
operating on the ``port_mappings`` array. ``protocol`` and every active ``port`` lookup are
correlated: they must all be satisfied by one single mapping, so ``?protocol=tcp&port__gt=1000`` does
not match a service whose only tcp mapping is tcp/80, and ``?port__gte=1000&port__lte=2000`` does not
match a service exposing only ports 500 and 5000. See ``ipam.utils.port_mapping_q``.
"""
# Whole-mapping lookup, e.g. ?port_mappings=tcp/80. Each value already names one complete
# protocol/port pair, so this needs none of the protocol/port correlation machinery below and is
# simply ANDed with the other filters.
port_mappings = MultiValueCharFilter(
method='filter_port_mappings',
label=_('Port mapping (protocol/port)'),
)
port_mappings__n = MultiValueCharFilter(
method='filter_port_mappings_negated',
label=_('Port mapping (protocol/port)'),
)
protocol = django_filters.MultipleChoiceFilter(
choices=ServiceProtocolChoices,
method='filter_noop',
)
# Negation lookup retained from when `protocol` was a model field: method-based filters don't get
# the char-based lookups (protocol__n, __ic, ...) auto-generated, and silently dropping protocol__n
# would widen existing saved filters/scripts rather than error. The __ic/__nic/__empty variants were
# never meaningful on a small fixed choice set and are intentionally left gone.
protocol__n = django_filters.MultipleChoiceFilter(
choices=ServiceProtocolChoices,
method='filter_protocol_negated',
)
# `port` and its range lookups. These are declared explicitly because a method-based filter gets no
# auto-generated lookups (BaseFilterSet.get_additional_lookups() skips filters with a method), and
# they must be correlated with `protocol` rather than applied independently. `port__empty` is
# intentionally absent: port_mappings is never empty on a validated object, so it was never
# meaningful. See ipam.utils.PORT_MAPPING_LOOKUPS for the lookup -> SQL operator mapping.
#
# `protocol` above and every `port*` lookup below (except the negations, which stand alone) are
# deliberately no-ops: because they must be correlated with one another they cannot be applied as each
# filter runs. filter_queryset() applies them together, once, after super() has applied the rest.
port = MultiValueNumberFilter(
method='filter_noop',
)
port__n = MultiValueNumberFilter(
method='filter_port_negated',
)
port__gt = MultiValueNumberFilter(
method='filter_noop',
)
port__gte = MultiValueNumberFilter(
method='filter_noop',
)
port__lt = MultiValueNumberFilter(
method='filter_noop',
)
port__lte = MultiValueNumberFilter(
method='filter_noop',
)
def filter_queryset(self, queryset):
"""
Apply `protocol` and every active `port*` lookup as a single correlated predicate.
These can't be applied per-filter the way django-filter normally works: they must all be satisfied
by one single mapping, and a query combining N of them would otherwise emit N independent (and
redundant) copies of the same scan. So the individual filters are no-ops and the combined
predicate is built here, from the cleaned data, exactly once per call.
"""
queryset = super().filter_queryset(queryset)
cleaned_data = self.form.cleaned_data
protocols = cleaned_data.get('protocol') or []
port_tests = [
(lookup, values)
for lookup, values in (
(lookup, cleaned_data.get(name) or [])
for name, lookup in SERVICE_PORT_FILTERS.items()
)
if values
]
if not protocols and not port_tests:
return queryset
return queryset.filter(port_mapping_q(protocols, port_tests))
def filter_noop(self, queryset, name, value):
# See filter_queryset(), which applies `protocol` and the port lookups as one correlated predicate.
return queryset
def filter_port_mappings(self, queryset, name, value: list[str]):
# Array overlap (&&) is served by the GIN index on port_mappings and gives the multi-value OR
# semantics used throughout NetBox: ?port_mappings=tcp/80&port_mappings=udp/53 matches either.
if not value:
return queryset
return queryset.filter(port_mappings__overlap=[normalize_port_mapping(v) for v in value])
def filter_port_mappings_negated(self, queryset, name, value: list[str]):
if not value:
return queryset
return queryset.exclude(port_mappings__overlap=[normalize_port_mapping(v) for v in value])
def filter_protocol_negated(self, queryset, name, value: list[str]):
# Exclude services exposing any of the given protocols (negation of the protocol-only lookup).
if not value:
return queryset
return queryset.exclude(port_mapping_q(value))
def filter_port_negated(self, queryset, name, value: list[int]):
# Exclude services exposing any of the given ports. Correlated with `protocol` when supplied, so
# ?protocol=tcp&port__n=80 excludes only services exposing tcp/80 (not those exposing udp/80).
if not value:
return queryset
protocols = self.form.cleaned_data.get('protocol') or []
return queryset.exclude(port_mapping_q(protocols, [('exact', value)]))
@register_filterset
class ServiceTemplateFilterSet(ServicePortMappingFilterMixin, PrimaryModelFilterSet):
class Meta:
model = ServiceTemplate
fields = ('id', 'name', 'protocol', 'description')
fields = ('id', 'name', 'description')
def search(self, queryset, name, value):
if not value.strip():
@ -1236,7 +1359,7 @@ class ServiceTemplateFilterSet(PrimaryModelFilterSet):
@register_filterset
class ServiceFilterSet(ContactModelFilterSet, PrimaryModelFilterSet):
class ServiceFilterSet(ServicePortMappingFilterMixin, ContactModelFilterSet, PrimaryModelFilterSet):
parent_object_type = MultiValueContentTypeFilter()
device = MultiValueCharFilter(
method='filter_device',
@ -1279,14 +1402,10 @@ class ServiceFilterSet(ContactModelFilterSet, PrimaryModelFilterSet):
to_field_name='address',
label=_('IP address'),
)
port = NumericArrayFilter(
field_name='ports',
lookup_expr='contains'
)
class Meta:
model = Service
fields = ('id', 'name', 'protocol', 'description', 'parent_object_type', 'parent_object_id')
fields = ('id', 'name', 'description', 'parent_object_type', 'parent_object_id')
def search(self, queryset, name, value):
if not value.strip():

View File

@ -1,5 +1,7 @@
from .bulk_create import *
from .bulk_edit import *
from .bulk_import import *
from .fields import *
from .filtersets import *
from .model_forms import *
from .widgets import *

View File

@ -6,6 +6,7 @@ from dcim.forms.mixins import ScopedBulkEditForm
from dcim.models import Region, Site, SiteGroup
from ipam.choices import *
from ipam.constants import *
from ipam.forms.fields import PortMappingField
from ipam.models import *
from ipam.models import ASN
from netbox.forms import NetBoxModelBulkEditForm, OrganizationalModelBulkEditForm, PrimaryModelBulkEditForm
@ -16,7 +17,6 @@ from utilities.forms.fields import (
DynamicModelChoiceField,
DynamicModelMultipleChoiceField,
GenericObjectChoiceField,
NumericArrayField,
NumericRangeArrayField,
)
from utilities.forms.rendering import FieldSet
@ -476,23 +476,20 @@ class VLANTranslationRuleBulkEditForm(NetBoxModelBulkEditForm):
class ServiceTemplateBulkEditForm(PrimaryModelBulkEditForm):
protocol = ChoiceField(
label=_('Protocol'),
choices=add_blank_choice(ServiceProtocolChoices),
required=False
add_port_mappings = PortMappingField(
label=_('Add port mappings'),
required=False,
help_text=_("Port mappings to add to each selected object"),
)
ports = NumericArrayField(
label=_('Ports'),
base_field=forms.IntegerField(
min_value=SERVICE_PORT_MIN,
max_value=SERVICE_PORT_MAX
),
required=False
remove_port_mappings = PortMappingField(
label=_('Remove port mappings'),
required=False,
help_text=_("Port mappings to remove from each selected object (if present)"),
)
model = ServiceTemplate
fieldsets = (
FieldSet('protocol', 'ports', 'description'),
FieldSet('add_port_mappings', 'remove_port_mappings', 'description'),
)
nullable_fields = ('description', 'comments')

View File

@ -1,5 +1,7 @@
from django import forms
from django.contrib.contenttypes.models import ContentType
from django.contrib.postgres.forms import SimpleArrayField
from django.core.exceptions import ValidationError as DjangoValidationError
from django.utils.translation import gettext_lazy as _
from dcim.forms.mixins import ScopedImportForm
@ -7,6 +9,7 @@ from dcim.models import Device, Interface, Site
from ipam.choices import *
from ipam.constants import *
from ipam.models import *
from ipam.validators import validate_port_mappings
from netbox.forms import NetBoxModelImportForm, OrganizationalModelImportForm, PrimaryModelImportForm
from tenancy.models import Tenant
from utilities.forms.fields import (
@ -586,19 +589,41 @@ class VLANTranslationRuleImportForm(NetBoxModelImportForm):
fields = ('policy', 'local_vid', 'remote_vid')
class ServiceTemplateImportForm(PrimaryModelImportForm):
protocol = CSVChoiceField(
label=_('Protocol'),
choices=ServiceProtocolChoices,
help_text=_('IP protocol')
class ServicePortMappingsImportMixin(forms.Form):
"""
Adds a ``port_mappings`` CSV column parsed from a comma-separated list of ``protocol/port`` pairs
(e.g. "tcp/80,udp/53") into the model's flat ``['tcp/80', 'udp/53']`` list.
"""
port_mappings = SimpleArrayField(
base_field=forms.CharField(),
label=_('Port mappings'),
required=True,
help_text=_('Comma-separated list of protocol/port pairs in double quotes (e.g. "tcp/80,udp/53").')
)
def clean_port_mappings(self):
mappings = self.cleaned_data.get('port_mappings')
if not mappings:
return []
# Strip surrounding whitespace from each CSV token; validate_port_mappings matches the protocol
# case-insensitively and returns the normalized (canonical) list, so protocols may be given in
# any case (e.g. "TCP/80") without folding here.
mappings = [mapping.strip() for mapping in mappings]
try:
mappings = validate_port_mappings(mappings)
except DjangoValidationError as exc:
raise forms.ValidationError(exc.messages)
return mappings
class ServiceTemplateImportForm(ServicePortMappingsImportMixin, PrimaryModelImportForm):
class Meta:
model = ServiceTemplate
fields = ('name', 'protocol', 'ports', 'description', 'owner', 'comments', 'tags')
fields = ('name', 'port_mappings', 'description', 'owner', 'comments', 'tags')
class ServiceImportForm(PrimaryModelImportForm):
class ServiceImportForm(ServicePortMappingsImportMixin, PrimaryModelImportForm):
parent_object_type = CSVContentTypeField(
queryset=ContentType.objects.filter(SERVICE_ASSIGNMENT_MODELS),
required=True,
@ -615,11 +640,6 @@ class ServiceImportForm(PrimaryModelImportForm):
required=False,
help_text=_('Parent object ID'),
)
protocol = CSVChoiceField(
label=_('Protocol'),
choices=ServiceProtocolChoices,
help_text=_('IP protocol')
)
ipaddresses = CSVModelMultipleChoiceField(
queryset=IPAddress.objects.all(),
required=False,
@ -630,7 +650,7 @@ class ServiceImportForm(PrimaryModelImportForm):
class Meta:
model = Service
fields = (
'ipaddresses', 'name', 'protocol', 'ports', 'description', 'owner', 'comments', 'tags',
'ipaddresses', 'name', 'port_mappings', 'description', 'owner', 'comments', 'tags',
)
def __init__(self, data=None, *args, **kwargs):

103
netbox/ipam/forms/fields.py Normal file
View File

@ -0,0 +1,103 @@
import json
from django import forms
from django.core.exceptions import ValidationError
from django.utils.translation import gettext_lazy as _
from ipam.forms.widgets import PortMappingWidget
from ipam.utils import expand_port_mapping, group_port_mapping_rows
from ipam.validators import validate_port_mappings
__all__ = (
'PortMappingField',
)
class PortMappingField(forms.Field):
"""
A form field for editing a service's port mappings. Presents one row per protocol (each with a
comma/range list of ports) but cleans to the model's flat list of ``protocol/port`` strings, e.g.
``['tcp/80', 'tcp/443', 'udp/53']``.
"""
widget = PortMappingWidget
def prepare_value(self, value):
# Group the flat ['tcp/80', 'tcp/443', 'udp/53'] list back into per-protocol rows for the widget.
if value in (None, ''):
return '[]'
if isinstance(value, str):
# An already-grouped JSON string (e.g. re-rendering a bound form) is passed through. A bare
# 'protocol/port' string arrives when cloning a single-mapping object: the querystring
# single-value collapse (normalize_querydict) yields a str rather than a list, so group it
# like the list case instead of handing the widget unparseable JSON (which blanks the row).
try:
json.loads(value)
except (TypeError, ValueError):
return json.dumps(group_port_mapping_rows([value]))
return value
return json.dumps(group_port_mapping_rows(value))
def to_python(self, value):
if value in (None, ''):
return []
# A list is assumed to already be the flat ['tcp/80', ...] form (e.g. set programmatically)
if isinstance(value, list):
mappings = value
else:
try:
rows = json.loads(value)
except (TypeError, ValueError):
raise ValidationError(_("Invalid port mapping data."))
if not isinstance(rows, list):
raise ValidationError(_("Invalid port mapping data."))
mappings = []
for position, row in enumerate(rows, start=1):
# The widget's JS always submits a list of {protocol, ports} objects, but the hidden
# input is just POST data: a hand-crafted payload can put anything here, so validate the
# shape rather than letting a non-dict row raise AttributeError (a 500) on .get() below.
if not isinstance(row, dict):
raise ValidationError(_("Invalid port mapping data."))
protocol = row.get('protocol')
raw_ports = row.get('ports')
# Likewise `protocol` is only ever a string, and `ports` either a string (the widget's
# comma/range format) or a list of ports (set programmatically); anything else would reach
# expand_port_mapping() and fail there on .strip().
if (
(protocol is not None and not isinstance(protocol, str))
or (raw_ports is not None and not isinstance(raw_ports, (str, list)))
):
raise ValidationError(_("Invalid port mapping data."))
if isinstance(raw_ports, str):
raw_ports = raw_ports.strip()
# Ignore entirely-empty rows (e.g. the default blank row on an untouched form)
if not protocol and not raw_ports:
continue
# Expand via the shared helper, which accepts either the widget's comma/range string or an
# already-expanded list, rejects a blank protocol, and preserves a protocol-without-ports
# row as a bare 'protocol/' token. Errors are re-raised with the row's position (among the
# submitted rows — the widget omits entirely-blank ones), since it renders one row per
# protocol and an unqualified "Select a protocol" gives no clue which row to fix. Errors
# from validate_port_mappings() below are deliberately left unqualified: each quotes the
# offending mapping already, and a duplicate spans two rows.
try:
mappings.extend(expand_port_mapping(protocol, raw_ports))
except ValidationError as e:
raise ValidationError([
_("Row {position}: {error}").format(position=position, error=message)
for message in e.messages
])
# Shared validation returns the canonical (normalized) list of protocol/port strings
return validate_port_mappings(mappings)
def has_changed(self, initial, data):
# Compare the parsed mappings rather than raw strings, so cosmetic differences (row/port
# ordering, whitespace) don't register as a change.
def normalize(value):
try:
return sorted(self.to_python(value))
except ValidationError:
return None
return normalize(self.prepare_value(initial)) != normalize(data)

View File

@ -640,12 +640,24 @@ class ServiceTemplateFilterForm(PrimaryModelFilterSetForm):
model = ServiceTemplate
fieldsets = (
FieldSet('q', 'filter_id', 'tag'),
FieldSet('protocol', 'port', name=_('Attributes')),
FieldSet('port_mappings', 'protocol', 'port', name=_('Attributes')),
FieldSet('owner_group_id', 'owner_id', name=_('Ownership')),
)
protocol = forms.ChoiceField(
# A complete protocol/port pair, matched as a whole. Unlike `protocol` and `port` (which are
# correlated but independently specified), this is a single free-text value: an unknown protocol or
# malformed pair simply matches nothing, so no client-side validation is needed here.
port_mappings = forms.CharField(
label=_('Port mapping'),
required=False,
widget=forms.TextInput(
attrs={
'placeholder': 'e.g. tcp/80',
}
)
)
protocol = forms.MultipleChoiceField(
label=_('Protocol'),
choices=add_blank_choice(ServiceProtocolChoices),
choices=ServiceProtocolChoices,
required=False
)
port = forms.IntegerField(
@ -659,7 +671,7 @@ class ServiceFilterForm(ContactModelFilterForm, ServiceTemplateFilterForm):
model = Service
fieldsets = (
FieldSet('q', 'filter_id', 'tag'),
FieldSet('protocol', 'port', name=_('Attributes')),
FieldSet('port_mappings', 'protocol', 'port', name=_('Attributes')),
FieldSet('device_id', 'virtual_machine_id', 'fhrpgroup_id', name=_('Assignment')),
FieldSet('owner_group_id', 'owner_id', name=_('Ownership')),
FieldSet('contact', 'contact_role', 'contact_group', name=_('Contacts')),

View File

@ -9,6 +9,7 @@ from dcim.models import Device, Interface, Site, SiteGroup
from ipam.choices import *
from ipam.constants import *
from ipam.formfields import IPNetworkFormField
from ipam.forms.fields import PortMappingField
from ipam.models import *
from netbox.forms import NetBoxModelForm, OrganizationalModelForm, PrimaryModelForm
from tenancy.forms import TenancyForm
@ -19,11 +20,10 @@ from utilities.forms.fields import (
DynamicModelChoiceField,
DynamicModelMultipleChoiceField,
GenericObjectChoiceField,
NumericArrayField,
NumericRangeArrayField,
TypedChoiceField,
)
from utilities.forms.rendering import FieldSet, InlineFields, ObjectAttribute, TabbedGroups
from utilities.forms.rendering import FieldSet, ObjectAttribute, TabbedGroups
from utilities.forms.widgets import DatePicker
from virtualization.models import VirtualMachine, VMInterface
@ -808,34 +808,31 @@ class VLANTranslationRuleForm(NetBoxModelForm):
]
class ServiceTemplateForm(PrimaryModelForm):
protocol = ChoiceField(
label=_('Protocol'),
choices=ServiceProtocolChoices,
)
ports = NumericArrayField(
label=_('Ports'),
base_field=forms.IntegerField(
min_value=SERVICE_PORT_MIN,
max_value=SERVICE_PORT_MAX
class ServicePortMappingsMixin(forms.Form):
"""
Adds a ``port_mappings`` field (protocol + ports rows) to a Service/ServiceTemplate form. The field
maps directly to the model's ``port_mappings`` ArrayField, so no custom save handling is required.
"""
port_mappings = PortMappingField(
label=_('Port Mappings'),
help_text=_(
"One protocol per row, each with one or more port numbers. A range may be specified using a "
"hyphen (e.g. 80,443,8000-8010)."
),
help_text=_("Comma-separated list of one or more port numbers. A range may be specified using a hyphen.")
)
class ServiceTemplateForm(ServicePortMappingsMixin, PrimaryModelForm):
fieldsets = (
FieldSet('name', 'protocol', 'ports', 'description', 'tags', name=_('Application Service Template')),
FieldSet('name', 'port_mappings', 'description', 'tags', name=_('Application Service Template')),
)
class Meta:
model = ServiceTemplate
fields = ('name', 'protocol', 'ports', 'description', 'owner', 'comments', 'tags')
fields = ('name', 'port_mappings', 'description', 'owner', 'comments', 'tags')
class ServiceForm(GenericObjectFormMixin, PrimaryModelForm):
protocol = ChoiceField(
label=_('Protocol'),
choices=ServiceProtocolChoices,
)
class ServiceForm(ServicePortMappingsMixin, GenericObjectFormMixin, PrimaryModelForm):
parent = GenericObjectChoiceField(
label=_('Parent'),
content_type_queryset=ContentType.objects.filter(SERVICE_ASSIGNMENT_MODELS),
@ -843,14 +840,6 @@ class ServiceForm(GenericObjectFormMixin, PrimaryModelForm):
selector=True,
hx_target_id='service',
)
ports = NumericArrayField(
label=_('Ports'),
base_field=forms.IntegerField(
min_value=SERVICE_PORT_MIN,
max_value=SERVICE_PORT_MAX
),
help_text=_("Comma-separated list of one or more port numbers. A range may be specified using a hyphen.")
)
ipaddresses = DynamicModelMultipleChoiceField(
queryset=IPAddress.objects.all(),
required=False,
@ -859,8 +848,7 @@ class ServiceForm(GenericObjectFormMixin, PrimaryModelForm):
fieldsets = (
FieldSet(
'parent', 'name',
InlineFields('protocol', 'ports', label=_('Port(s)')),
'parent', 'name', 'port_mappings',
'ipaddresses', 'description', 'tags', name=_('Application Service'),
html_id='service',
),
@ -869,7 +857,7 @@ class ServiceForm(GenericObjectFormMixin, PrimaryModelForm):
class Meta:
model = Service
fields = [
'name', 'protocol', 'ports', 'ipaddresses', 'description', 'owner', 'comments', 'tags',
'name', 'port_mappings', 'ipaddresses', 'description', 'owner', 'comments', 'tags',
]
def __init__(self, *args, **kwargs):
@ -898,7 +886,7 @@ class ServiceCreateForm(ServiceForm):
'parent',
TabbedGroups(
FieldSet('service_template', name=_('From Template')),
FieldSet('name', 'protocol', 'ports', name=_('Custom')),
FieldSet('name', 'port_mappings', name=_('Custom')),
),
'ipaddresses', 'description', 'tags', name=_('Application Service'),
html_id='service',
@ -907,7 +895,7 @@ class ServiceCreateForm(ServiceForm):
class Meta(ServiceForm.Meta):
fields = [
'service_template', 'name', 'protocol', 'ports', 'ipaddresses', 'description',
'service_template', 'name', 'port_mappings', 'ipaddresses', 'description',
'comments', 'tags',
]
@ -915,7 +903,7 @@ class ServiceCreateForm(ServiceForm):
super().__init__(*args, **kwargs)
# Fields which may be populated from a ServiceTemplate are not required
for field in ('name', 'protocol', 'ports'):
for field in ('name', 'port_mappings'):
self.fields[field].required = False
self.fields[field].widget.is_required = False
@ -925,11 +913,10 @@ class ServiceCreateForm(ServiceForm):
# Create a new Service from the specified template
service_template = self.cleaned_data['service_template']
self.cleaned_data['name'] = service_template.name
self.cleaned_data['protocol'] = service_template.protocol
self.cleaned_data['ports'] = service_template.ports
self.cleaned_data['port_mappings'] = list(service_template.port_mappings)
if not self.cleaned_data['description']:
self.cleaned_data['description'] = service_template.description
elif not all(self.cleaned_data[f] for f in ('name', 'protocol', 'ports')):
elif not self.cleaned_data.get('name') or not self.cleaned_data.get('port_mappings'):
raise forms.ValidationError(
_("Must specify name, protocol, and port(s) if not using an application service template.")
_("Must specify name and port mapping(s) if not using an application service template.")
)

View File

@ -0,0 +1,63 @@
import json
from django import forms
from django.forms.utils import flatatt
from ipam.choices import ServiceProtocolChoices
__all__ = (
'PortMappingWidget',
)
class PortMappingWidget(forms.Widget):
"""
Renders a dynamic set of (protocol, ports) rows. The rows are serialized to a JSON string held in a
single hidden input (client-side JS keeps the hidden input in sync as rows are added/removed). Each
row's ``ports`` value is a raw comma/range string (e.g. "80,443,8000-8010"); the server expands it.
"""
template_name = 'ipam/widgets/port_mappings.html'
# aria-* attributes which render_field_with_aria() sets per-field, and which must be copied onto the
# row controls: the wrapping <div> isn't a form control, so assistive technology ignores them there.
CONTROL_ATTRS = ('aria-describedby', 'aria-invalid')
def id_for_label(self, id_):
# The field's <label for="..."> must point at a real form control, not the wrapping <div> which
# carries the widget's id. Target the first row's protocol <select>; the client-side widget keeps
# this id on whichever row is first as rows are added and removed.
return f'{id_}_protocol_0' if id_ else ''
def get_context(self, name, value, attrs):
attrs = attrs or {}
rows = []
if value:
try:
rows = json.loads(value)
except (TypeError, ValueError):
rows = []
# Re-rendering an invalid bound form hands us back the raw POST value, which need not be the list
# of {protocol, ports} objects the JS produces: a crafted payload of e.g. `5` or `{"a": 1}` parses
# as valid JSON but would break the template's row loop. Discard anything of the wrong shape and
# fall through to the blank row below.
if not isinstance(rows, list) or not all(isinstance(row, dict) for row in rows):
rows = []
# Always render at least one (blank) row so the entry fields are visible on an empty form
if not rows:
rows = [{'protocol': '', 'ports': ''}]
return {
'widget': {
'name': name,
'value': value or '[]',
'rows': rows,
'attrs': attrs,
'label_id': self.id_for_label(attrs.get('id')),
'control_attrs': flatatt({
key: value for key, value in attrs.items() if key in self.CONTROL_ATTRS
}),
},
'protocol_choices': list(ServiceProtocolChoices),
}
def value_from_datadict(self, data, files, name):
return data.get(name)

View File

@ -22,6 +22,6 @@ IPAddressRoleEnum = strawberry.enum(IPAddressRoleChoices.as_enum(prefix='role'))
IPAddressStatusEnum = strawberry.enum(IPAddressStatusChoices.as_enum(prefix='status'))
IPRangeStatusEnum = strawberry.enum(IPRangeStatusChoices.as_enum(prefix='status'))
PrefixStatusEnum = strawberry.enum(PrefixStatusChoices.as_enum(prefix='status'))
ServiceProtocolEnum = strawberry.enum(ServiceProtocolChoices.as_enum(prefix='role'))
ServiceProtocolEnum = strawberry.enum(ServiceProtocolChoices.as_enum())
VLANStatusEnum = strawberry.enum(VLANStatusChoices.as_enum(prefix='status'))
VLANQinQRoleEnum = strawberry.enum(VLANQinQRoleChoices.as_enum(prefix='role'))

View File

@ -1,25 +0,0 @@
from dataclasses import dataclass
from typing import TYPE_CHECKING, Annotated
import strawberry
import strawberry_django
from strawberry_django import BaseFilterLookup
if TYPE_CHECKING:
from netbox.graphql.filter_lookups import IntegerLookup
from .enums import *
__all__ = (
'ServiceFilterMixin',
)
@dataclass
class ServiceFilterMixin:
protocol: BaseFilterLookup[Annotated['ServiceProtocolEnum', strawberry.lazy('ipam.graphql.enums')]] | None = (
strawberry_django.filter_field()
)
ports: Annotated['IntegerLookup', strawberry.lazy('netbox.graphql.filter_lookups')] | None = (
strawberry_django.filter_field()
)

View File

@ -11,7 +11,7 @@ from strawberry_django import BaseFilterLookup, ComparisonFilterLookup, DateFilt
from dcim.graphql.filter_mixins import ScopedFilterMixin
from dcim.models import Device
from ipam import models
from ipam.graphql.filter_mixins import ServiceFilterMixin
from ipam.utils import normalize_port_mapping, port_mapping_q
from netbox.graphql.filters import (
ChangeLoggedModelFilter,
NetBoxModelFilter,
@ -345,8 +345,136 @@ class RouteTargetFilter(TenancyFilterMixin, PrimaryModelFilter):
)
# Custom (method-based) GraphQL filters can't be inherited from a mixin — strawberry_django only picks
# up filter_field methods declared on the filter_type class itself — so the filters below keep thin
# wrappers here. Each method reads *all* of its siblings' values off ``self`` so a combined
# protocol/port query matches a single mapping rather than each condition independently.
# GraphQL port filter name -> the port lookup it applies, in the order the conditions are built. The
# names deliberately carry the same `port__gt` double-underscore form as their REST counterparts in
# ipam.filtersets.SERVICE_PORT_FILTERS, so both APIs offer an identically-spelled set of lookups.
# (GraphQL reserves only *leading* double underscores, for introspection.) The schema is built with
# auto_camel_case=False, so these names reach the schema verbatim.
# See ipam.utils.PORT_MAPPING_LOOKUPS for the lookup -> SQL operator mapping.
GRAPHQL_PORT_FILTERS = {
'port': 'exact',
'port__gt': 'gt',
'port__gte': 'gte',
'port__lt': 'lt',
'port__lte': 'lte',
}
# `protocol` plus every port lookup, in the order their conditions are built. All of these must be
# satisfied by one single mapping, so they can't be applied as each resolver runs; the first one
# actually supplied owns the combined predicate (see _owns_predicate).
CORRELATED_PORT_FILTERS = ('protocol', *GRAPHQL_PORT_FILTERS)
def _supplied(filters, name):
"""
Return the list of values supplied for a sibling filter field, or None if it was not supplied. Only
a list is a real value: an omitted field is None/UNSET, and an unset method-based filter can resolve
to the bound method itself, so anything non-list is reported as absent rather than surfacing as a
TypeError downstream.
"""
value = getattr(filters, name, None)
return list(value) if isinstance(value, (list, tuple)) else None
def _port_mapping_args(filters):
"""
Collect the correlated protocol/port arguments from every sibling field on the filter instance, in
the ``(protocols, port_tests)`` shapes ``port_mapping_q()`` expects.
"""
protocols = [v.value for v in _supplied(filters, 'protocol') or ()]
port_tests = [
(lookup, values)
for name, lookup in GRAPHQL_PORT_FILTERS.items()
if (values := _supplied(filters, name))
]
return protocols, port_tests
def _owns_predicate(filters, name):
"""
True for exactly one of the correlated filter fields: the first one supplied, in
``CORRELATED_PORT_FILTERS`` order. The others contribute nothing, so a query combining N of them
builds the (deliberately sequential) scan once rather than ANDing N identical copies of it.
"""
for candidate in CORRELATED_PORT_FILTERS:
if _supplied(filters, candidate) is not None:
return candidate == name
return False
def _port_mapping_prefix_q(model, protocols, port_tests, prefix):
qs_filter = port_mapping_q(protocols, port_tests)
if prefix:
# Nested relation (e.g. prefix='services__'): the incoming queryset is a *different* model, so
# resolve the matching PKs on the target model and match them through the prefix.
return Q(**{f'{prefix}pk__in': model.objects.filter(qs_filter).values('pk')})
# Root query: the incoming queryset already targets this model, so return the lookup directly rather
# than wrapping it in an extra pk__in self-subquery.
return qs_filter
def _make_port_mapping_filters(model):
# strawberry_django only collects filter_field methods declared on the filter_type class itself (not
# from a mixin), so the Service/ServiceTemplate filters are produced by this factory and assigned
# into each class body. This keeps the protocol/port correlation logic in a single place.
def correlated(filters, name, prefix):
# Deliberately ignores the resolver's own `value` in favour of reading every sibling off
# `filters`: only the owning field applies the predicate, and it needs them all.
if not _owns_predicate(filters, name):
return Q()
protocols, port_tests = _port_mapping_args(filters)
return _port_mapping_prefix_q(model, protocols, port_tests, prefix)
@strawberry_django.filter_field
def protocol(
self,
queryset,
value: list[Annotated['ServiceProtocolEnum', strawberry.lazy('ipam.graphql.enums')]],
prefix,
):
return correlated(self, 'protocol', prefix)
# `port` and its range lookups. Values within one lookup are OR'd (as ?port=80&port=443 is on the
# REST API); the lookups themselves are AND'd, and so must hold for one single mapping.
@strawberry_django.filter_field
def port(self, queryset, value: list[int], prefix):
return correlated(self, 'port', prefix)
@strawberry_django.filter_field
def port__gt(self, queryset, value: list[int], prefix):
return correlated(self, 'port__gt', prefix)
@strawberry_django.filter_field
def port__gte(self, queryset, value: list[int], prefix):
return correlated(self, 'port__gte', prefix)
@strawberry_django.filter_field
def port__lt(self, queryset, value: list[int], prefix):
return correlated(self, 'port__lt', prefix)
@strawberry_django.filter_field
def port__lte(self, queryset, value: list[int], prefix):
return correlated(self, 'port__lte', prefix)
@strawberry_django.filter_field
def port_mappings(self, queryset, value: list[str], prefix):
# Whole-mapping lookup (e.g. ["tcp/80", "udp/53"], matching any). Each value names one complete
# protocol/port pair, so unlike protocol/port this needs no correlation and reduces to a
# GIN-indexable array overlap. Values are normalized so 'TCP/080' finds the stored 'tcp/80'.
mappings = [normalize_port_mapping(mapping) for mapping in value]
return Q(**{f'{prefix}port_mappings__overlap': mappings})
return protocol, port, port__gt, port__gte, port__lt, port__lte, port_mappings
@register_filter(models.Service, lookups=True)
class ServiceFilter(ContactFilterMixin, ServiceFilterMixin, PrimaryModelFilter):
class ServiceFilter(ContactFilterMixin, PrimaryModelFilter):
name: StrFilterLookup | None = strawberry_django.filter_field()
ip_addresses: Annotated['IPAddressFilter', strawberry.lazy('ipam.graphql.filters')] | None = (
strawberry_django.filter_field()
@ -355,11 +483,17 @@ class ServiceFilter(ContactFilterMixin, ServiceFilterMixin, PrimaryModelFilter):
strawberry_django.filter_field()
)
parent_object_id: ID | None = strawberry_django.filter_field()
protocol, port, port__gt, port__gte, port__lt, port__lte, port_mappings = (
_make_port_mapping_filters(models.Service)
)
@register_filter(models.ServiceTemplate, lookups=True)
class ServiceTemplateFilter(ServiceFilterMixin, PrimaryModelFilter):
class ServiceTemplateFilter(PrimaryModelFilter):
name: StrFilterLookup | None = strawberry_django.filter_field()
protocol, port, port__gt, port__gte, port__lt, port__lte, port_mappings = (
_make_port_mapping_filters(models.ServiceTemplate)
)
@register_filter(models.VLAN, lookups=True)

View File

@ -248,16 +248,31 @@ class RouteTargetType(PrimaryObjectType):
exporting_vrfs: list[Annotated["VRFType", strawberry.lazy('ipam.graphql.types')]]
# Shared deprecation reason for the legacy port-mapping GraphQL fields. The fields themselves are
# declared on each type (rather than via a mixin) so they reliably override the auto-generated model
# field of the same name; each delegates to the model's protocol/ports properties (the single source of
# truth for the legacy view, derived from port_mappings on each access).
_LEGACY_DEPRECATION = "Deprecated; use port_mappings. Populated only for single-protocol services."
@register_type(
models.Service,
exclude=('_ports_lowest', 'parent_object_type', 'parent_object_id'),
exclude=['parent_object_type', 'parent_object_id'],
filters=ServiceFilter,
pagination=True
)
class ServiceType(ContactsMixin, PrimaryObjectType):
ports: list[int]
port_mappings: list[str]
ipaddresses: list[Annotated['IPAddressType', strawberry.lazy('ipam.graphql.types')]]
@strawberry_django.field(deprecation_reason=_LEGACY_DEPRECATION)
def protocol(self) -> str | None:
return self.protocol
@strawberry_django.field(deprecation_reason=_LEGACY_DEPRECATION)
def ports(self) -> list[int] | None:
return self.ports
@strawberry_django.field(prefetch_related='parent')
def parent(self) -> Annotated[
Annotated['DeviceType', strawberry.lazy('dcim.graphql.types')]
@ -270,12 +285,20 @@ class ServiceType(ContactsMixin, PrimaryObjectType):
@register_type(
models.ServiceTemplate,
exclude=('_ports_lowest',),
fields='__all__',
filters=ServiceTemplateFilter,
pagination=True
)
class ServiceTemplateType(PrimaryObjectType):
ports: list[int]
port_mappings: list[str]
@strawberry_django.field(deprecation_reason=_LEGACY_DEPRECATION)
def protocol(self) -> str | None:
return self.protocol
@strawberry_django.field(deprecation_reason=_LEGACY_DEPRECATION)
def ports(self) -> list[int] | None:
return self.ports
@register_type(

View File

@ -0,0 +1,126 @@
import django.contrib.postgres.fields
import django.contrib.postgres.indexes
from django.db import migrations, models
POPULATE_PORT_MAPPINGS_SQL = """
UPDATE {table} SET port_mappings = ARRAY(
SELECT protocol || '/' || port
FROM (
-- Dedupe while preserving first-seen order: the legacy ports array wasn't guaranteed unique
-- (the REST API accepted any list), and a duplicated port would otherwise produce a duplicate
-- mapping that fails validation on the object's next save. NULL elements — only reachable via
-- raw DB writes are dropped rather than concatenated into a 'tcp/' mapping.
SELECT port, MIN(ordinality) AS ordinality
FROM unnest(ports) WITH ORDINALITY AS unnested(port, ordinality)
WHERE port IS NOT NULL
GROUP BY port
) AS deduped
ORDER BY deduped.ordinality
)
WHERE cardinality(ports) > 0 AND protocol <> ''
"""
def populate_port_mappings(apps, schema_editor):
"""
Build the new ``port_mappings`` array (e.g. ['tcp/80', 'tcp/443']) from the legacy protocol/ports
fields on each Service/ServiceTemplate. Done as a single set-based UPDATE per table rather than a
row-by-row rewrite, so the maintenance window stays bounded on large installs (this runs over every
service and service template in the database).
The ports column is an integer array, so every mapping this produces is already in the canonical form
validate_port_mappings() enforces no leading zeros to strip, and the protocol was constrained to
ServiceProtocolChoices.
Rows which cannot be converted an empty ``ports`` array, or ports that are all NULL, both
technically invalid under the old schema but possible via direct DB writes are left with
``port_mappings=[]``, which the new model rejects on the next save. Nothing is discarded that the old
schema considered valid. Operators can find any such records post-migration with, e.g.:
SELECT id, name FROM ipam_service WHERE port_mappings = '{}';
SELECT id, name FROM ipam_servicetemplate WHERE port_mappings = '{}';
"""
for model_name in ('Service', 'ServiceTemplate'):
# Table names come from the historical model state, not from user input
table = apps.get_model('ipam', model_name)._meta.db_table
with schema_editor.connection.cursor() as cursor:
cursor.execute(POPULATE_PORT_MAPPINGS_SQL.format(table=table))
class Migration(migrations.Migration):
dependencies = [
("ipam", "0094_denormalization_triggers"),
]
operations = [
# Add the new field to both models first
migrations.AddField(
model_name="service",
name="port_mappings",
field=django.contrib.postgres.fields.ArrayField(
base_field=models.CharField(max_length=63), blank=True, default=list
),
),
migrations.AddField(
model_name="servicetemplate",
name="port_mappings",
field=django.contrib.postgres.fields.ArrayField(
base_field=models.CharField(max_length=63), blank=True, default=list
),
),
# Migrate existing protocol/ports data into port_mappings before dropping the old fields.
migrations.RunPython(populate_port_mappings),
migrations.AlterModelOptions(
name="service",
options={"ordering": ("name", "id")},
),
migrations.RemoveIndex(
model_name="service",
name="ipam_servic_protoco_e2901d_idx",
),
migrations.AddIndex(
model_name="service",
index=models.Index(
fields=["name", "id"], name="ipam_servic_name_b3260b_idx"
),
),
migrations.RemoveField(
model_name="servicetemplate",
name="_ports_lowest",
),
migrations.RemoveField(
model_name="servicetemplate",
name="ports",
),
migrations.RemoveField(
model_name="servicetemplate",
name="protocol",
),
migrations.RemoveField(
model_name="service",
name="_ports_lowest",
),
migrations.RemoveField(
model_name="service",
name="ports",
),
migrations.RemoveField(
model_name="service",
name="protocol",
),
# GIN indexes supporting exact protocol/port lookups (port_mappings && ['tcp/80']).
# Protocol-only and range lookups are served by a correlated scan (GIN array_ops supports
# only =, &&, @> and <@, so no array index can answer them) — see ipam.utils.PortMappingMatch.
migrations.AddIndex(
model_name="service",
index=django.contrib.postgres.indexes.GinIndex(
fields=["port_mappings"], name="ipam_servic_port_ma_a3d51d_gin"
),
),
migrations.AddIndex(
model_name="servicetemplate",
index=django.contrib.postgres.indexes.GinIndex(
fields=["port_mappings"], name="ipam_servic_port_ma_39e070_gin"
),
),
]

View File

@ -1,58 +1,125 @@
from django.contrib.contenttypes.fields import GenericForeignKey
from django.contrib.postgres.fields import ArrayField
from django.core.validators import MaxValueValidator, MinValueValidator
from django.contrib.postgres.indexes import GinIndex
from django.core.exceptions import ValidationError
from django.db import models
from django.utils.translation import gettext_lazy as _
from ipam.choices import *
from ipam.constants import *
from ipam.utils import legacy_protocol_and_ports, split_port_mapping
from ipam.validators import validate_port_mappings
from netbox.models import PrimaryModel
from netbox.models.features import ContactsMixin
from utilities.data import array_to_string
__all__ = (
'Service',
'ServiceTemplate',
)
# Fixed protocol value -> label map, built once (the choice set is static per process) rather than
# rebuilt on every port_mappings_list render.
SERVICE_PROTOCOL_LABELS = dict(ServiceProtocolChoices)
class ServiceBase(models.Model):
protocol = models.CharField(
verbose_name=_('protocol'),
max_length=50,
choices=ServiceProtocolChoices
)
ports = ArrayField(
base_field=models.PositiveIntegerField(
validators=[
MinValueValidator(SERVICE_PORT_MIN),
MaxValueValidator(SERVICE_PORT_MAX)
]
),
verbose_name=_('port numbers')
)
_ports_lowest = models.PositiveIntegerField(
null=True,
"""
Shared behavior for Service and ServiceTemplate. Protocol/port data is stored as a single array of
``protocol/port`` strings (e.g. ``['tcp/80', 'tcp/443', 'udp/53']``), allowing a service to expose
the same port on multiple protocols.
"""
port_mappings = ArrayField(
base_field=models.CharField(max_length=63),
verbose_name=_('port mappings'),
help_text=_("Protocol/port pairs, e.g. tcp/80"),
blank=True,
default=list,
)
class Meta:
abstract = True
def save(self, *args, **kwargs):
# On saving find the smallest port and save for default ordering
self._ports_lowest = min(self.ports) if self.ports else None
update_fields = kwargs.get('update_fields')
if update_fields is not None and '_ports_lowest' not in update_fields:
kwargs['update_fields'] = list(update_fields) + ['_ports_lowest']
super().save(*args, **kwargs)
def __str__(self):
return f'{self.name} ({self.get_protocol_display()}/{self.port_list})'
return self.name
def clean(self):
super().clean()
# validate_port_mappings returns the canonical form (integer ports), so storing its result
# normalizes any entry that bypassed the form field (e.g. a raw REST payload of 'tcp/080'). Key
# its errors to the field — it raises unkeyed, which full_clean() would otherwise file as a
# non-field (__all__) error rather than against port_mappings.
try:
self.port_mappings = validate_port_mappings(self.port_mappings)
except ValidationError as e:
raise ValidationError({'port_mappings': e.messages})
if not self.port_mappings:
raise ValidationError({'port_mappings': _("At least one port mapping is required.")})
@staticmethod
def _normalize_mapping(mapping):
# Normalize a stored/incoming mapping's port to an integer so a non-canonical value (e.g. a
# raw-DB 'tcp/080') compares equal to its canonical form ('tcp/80').
protocol, port = split_port_mapping(mapping)
return f'{protocol}/{int(port)}' if port.isdigit() else mapping
def _add_port_mappings(self, mappings):
"""
Add the given canonical ``protocol/port`` strings to ``port_mappings``, skipping any already
present (matched by normalized form). The merged list is left for ``clean()`` to validate.
Internal helper called from the Service/ServiceTemplate bulk-edit view's pre_save_operations()
hook, so the merge is part of the single bulk-edit save (one change-log entry) and the model
stays unaware of the bulk-edit form. Underscore-prefixed to keep it out of the way of the
identically-named ``add_port_mappings`` bulk-edit form field (which the generic bulk-edit view
assigns onto the object).
"""
existing = {self._normalize_mapping(mapping) for mapping in self.port_mappings}
self.port_mappings = list(self.port_mappings) + [
mapping for mapping in mappings if self._normalize_mapping(mapping) not in existing
]
def _remove_port_mappings(self, mappings):
"""
Remove the given canonical ``protocol/port`` strings from ``port_mappings`` (matched by
normalized form). The result is left for ``clean()`` to validate (range, duplicates, and the
at-least-one rule). Internal helper called from the bulk-edit view's pre_save_operations() hook
(see ``_add_port_mappings``).
"""
remove = {self._normalize_mapping(mapping) for mapping in mappings}
self.port_mappings = [
mapping for mapping in self.port_mappings if self._normalize_mapping(mapping) not in remove
]
@property
def port_list(self):
return array_to_string(self.ports)
def port_mappings_list(self):
"""
Return a user-friendly list of port mappings, e.g. "TCP/80, TCP/443, UDP/53".
"""
return ', '.join(
f'{SERVICE_PROTOCOL_LABELS.get(protocol, protocol)}/{port}'
for protocol, port in (split_port_mapping(mapping) for mapping in self.port_mappings)
)
# Read-only legacy accessors mirroring the deprecated REST/GraphQL protocol/ports fields, retained
# for backward compatibility with code that read the old single-protocol fields. A multi-protocol
# service has no single-protocol form, so both return None (ports=[] when there are no mappings).
# TODO: Remove these in v5.0 once backward compatibility is dropped.
@property
def _legacy_protocol_ports(self):
# Recomputed on access (grouping a handful of strings is cheap) rather than cached, so a mutation
# of port_mappings — e.g. via _add_port_mappings()/_remove_port_mappings() — is always reflected
# by the protocol/ports accessors, with no cache to invalidate.
return legacy_protocol_and_ports(self.port_mappings)
# Return types are annotated so drf-spectacular can resolve these properties when it builds the
# write-side serializer schema (without them it warns and falls back to string).
@property
def protocol(self) -> str | None:
return self._legacy_protocol_ports[0]
@property
def ports(self) -> list[int] | None:
return self._legacy_protocol_ports[1]
class ServiceTemplate(ServiceBase, PrimaryModel):
@ -65,7 +132,12 @@ class ServiceTemplate(ServiceBase, PrimaryModel):
unique=True
)
clone_fields = ('port_mappings', 'description')
class Meta:
indexes = (
GinIndex(fields=('port_mappings',)),
)
ordering = ('name',)
verbose_name = _('application service template')
verbose_name_plural = _('application service templates')
@ -99,14 +171,15 @@ class Service(ContactsMixin, ServiceBase, PrimaryModel):
)
clone_fields = (
'protocol', 'ports', 'description', 'parent', 'ipaddresses',
'port_mappings', 'description', 'parent', 'ipaddresses',
)
class Meta:
indexes = (
models.Index(fields=('protocol', '_ports_lowest', 'id')), # Default ordering
models.Index(fields=('name', 'id')), # Default ordering
models.Index(fields=('parent_object_type', 'parent_object_id')),
GinIndex(fields=('port_mappings',)),
)
ordering = ('protocol', '_ports_lowest', 'id')
ordering = ('name', 'id')
verbose_name = _('application service')
verbose_name_plural = _('application services')

View File

@ -16,10 +16,13 @@ class ServiceTemplateTable(PrimaryModelTable):
verbose_name=_('Name'),
linkify=True
)
# Column key kept as 'ports' (not renamed to 'port_mappings') so existing saved table configs keep
# working; the removed single 'protocol' column, however, will silently drop out of any saved config
# or export template that referenced it.
ports = tables.Column(
verbose_name=_('Ports'),
accessor=tables.A('port_list'),
order_by=tables.A('ports'),
verbose_name=_('Port Mappings'),
accessor=tables.A('port_mappings_list'),
orderable=False,
)
tags = columns.TagColumn(
url_name='ipam:servicetemplate_list'
@ -28,9 +31,9 @@ class ServiceTemplateTable(PrimaryModelTable):
class Meta(PrimaryModelTable.Meta):
model = ServiceTemplate
fields = (
'pk', 'id', 'name', 'protocol', 'ports', 'description', 'comments', 'tags', 'created', 'last_updated',
'pk', 'id', 'name', 'ports', 'description', 'comments', 'tags', 'created', 'last_updated',
)
default_columns = ('pk', 'name', 'protocol', 'ports', 'description')
default_columns = ('pk', 'name', 'ports', 'description')
class ServiceTable(ContactsColumnMixin, PrimaryModelTable):
@ -43,10 +46,13 @@ class ServiceTable(ContactsColumnMixin, PrimaryModelTable):
linkify=True,
order_by=('device', 'virtual_machine')
)
# Column key kept as 'ports' (not renamed to 'port_mappings') so existing saved table configs keep
# working; the removed single 'protocol' column, however, will silently drop out of any saved config
# or export template that referenced it.
ports = tables.Column(
verbose_name=_('Ports'),
accessor=tables.A('port_list'),
order_by=tables.A('ports'),
verbose_name=_('Port Mappings'),
accessor=tables.A('port_mappings_list'),
orderable=False,
)
tags = columns.TagColumn(
url_name='ipam:service_list'
@ -55,7 +61,7 @@ class ServiceTable(ContactsColumnMixin, PrimaryModelTable):
class Meta(PrimaryModelTable.Meta):
model = Service
fields = (
'pk', 'id', 'name', 'parent', 'protocol', 'ports', 'ipaddresses', 'description', 'contacts', 'comments',
'pk', 'id', 'name', 'parent', 'ports', 'ipaddresses', 'description', 'contacts', 'comments',
'tags', 'created', 'last_updated',
)
default_columns = ('pk', 'name', 'parent', 'protocol', 'ports', 'description')
default_columns = ('pk', 'name', 'parent', 'ports', 'description')

View File

@ -0,0 +1,66 @@
{% load i18n %}
<div class="port-mapping-widget" id="{{ widget.attrs.id }}" data-name="{{ widget.name }}"
role="group"{% if widget.attrs.required %} aria-required="true"{% endif %}>
<input type="hidden" name="{{ widget.name }}" value="{{ widget.value }}">
<table class="table table-sm mb-1">
<thead>
<tr>
<th scope="col" style="width: 40%;">{% trans "Protocol" %}</th>
<th scope="col">{% trans "Ports" %}</th>
<th scope="col" style="width: 1%;"><span class="visually-hidden">{% trans "Actions" %}</span></th>
</tr>
</thead>
<tbody data-port-mapping-rows>
{% for row in widget.rows %}
<tr data-port-mapping-row>
<td>
{# Only the first row's select carries the field's label target; the JS moves it as rows change. #}
<select class="form-select form-select-sm port-mapping-protocol"
{% if forloop.first %}id="{{ widget.label_id }}" {% endif %}
aria-label="{% trans "Protocol" %}"{{ widget.control_attrs }}>
<option value=""{% if not row.protocol %} selected{% endif %}>---------</option>
{% for value, label in protocol_choices %}
<option value="{{ value }}"{% if value == row.protocol %} selected{% endif %}>{{ label }}</option>
{% endfor %}
</select>
</td>
<td>
<input type="text" class="form-control form-control-sm port-mapping-ports" value="{{ row.ports }}"
aria-label="{% trans "Ports" %}"{{ widget.control_attrs }}
placeholder="{% trans 'e.g. 80,443,8000-8010' %}">
</td>
<td>
<button type="button" class="btn btn-sm btn-outline-danger" data-port-mapping-remove
aria-label="{% trans 'Remove mapping' %}"><i class="mdi mdi-close"></i></button>
</td>
</tr>
{% endfor %}
</tbody>
</table>
<button type="button" class="btn btn-sm btn-outline-primary" data-port-mapping-add>
<i class="mdi mdi-plus-thick"></i> {% trans "Add mapping" %}
</button>
<template data-port-mapping-template>
<tr data-port-mapping-row>
<td>
<select class="form-select form-select-sm port-mapping-protocol"
aria-label="{% trans "Protocol" %}"{{ widget.control_attrs }}>
<option value="">---------</option>
{% for value, label in protocol_choices %}
<option value="{{ value }}">{{ label }}</option>
{% endfor %}
</select>
</td>
<td>
<input type="text" class="form-control form-control-sm port-mapping-ports"
aria-label="{% trans "Ports" %}"{{ widget.control_attrs }}
placeholder="{% trans 'e.g. 80,443,8000-8010' %}">
</td>
<td>
<button type="button" class="btn btn-sm btn-outline-danger" data-port-mapping-remove
aria-label="{% trans 'Remove mapping' %}"><i class="mdi mdi-close"></i></button>
</td>
</tr>
</template>
</div>

View File

@ -1442,7 +1442,7 @@ class VLANTranslationRuleTestCase(APIViewTestCases.APIViewTestCase):
class ServiceTemplateTestCase(APIViewTestCases.APIViewTestCase):
model = ServiceTemplate
brief_fields = ['description', 'display', 'id', 'name', 'ports', 'protocol', 'url']
brief_fields = ['description', 'display', 'id', 'name', 'port_mappings', 'url']
bulk_update_data = {
'description': 'New description',
}
@ -1450,35 +1450,283 @@ class ServiceTemplateTestCase(APIViewTestCases.APIViewTestCase):
@classmethod
def setUpTestData(cls):
service_templates = (
ServiceTemplate(name='Service Template 1', protocol=ServiceProtocolChoices.PROTOCOL_TCP, ports=[1, 2]),
ServiceTemplate(name='Service Template 2', protocol=ServiceProtocolChoices.PROTOCOL_TCP, ports=[3, 4]),
ServiceTemplate(name='Service Template 3', protocol=ServiceProtocolChoices.PROTOCOL_TCP, ports=[5, 6]),
)
ServiceTemplate.objects.bulk_create(service_templates)
ServiceTemplate.objects.bulk_create([
ServiceTemplate(name='Service Template 1', port_mappings=['tcp/1', 'tcp/2']),
ServiceTemplate(name='Service Template 2', port_mappings=['tcp/3', 'tcp/4']),
ServiceTemplate(name='Service Template 3', port_mappings=['tcp/5', 'tcp/6']),
])
cls.create_data = [
{
'name': 'Service Template 4',
'protocol': ServiceProtocolChoices.PROTOCOL_TCP,
'ports': [7, 8],
'port_mappings': ['tcp/7', 'tcp/8'],
},
{
'name': 'Service Template 5',
'protocol': ServiceProtocolChoices.PROTOCOL_TCP,
'ports': [9, 10],
'port_mappings': ['tcp/53', 'udp/53'],
},
{
'name': 'Service Template 6',
'protocol': ServiceProtocolChoices.PROTOCOL_TCP,
'ports': [11, 12],
'port_mappings': ['tcp/11', 'tcp/12'],
},
]
def test_graphql_port_mappings(self):
"""port_mappings is exposed over GraphQL as a flat list of protocol/port strings."""
self.add_permissions('ipam.view_servicetemplate')
template = ServiceTemplate.objects.create(name='GQL Mappings', port_mappings=['tcp/80', 'udp/53'])
url = reverse('graphql')
query = f'{{ service_template(id: {template.pk}) {{ port_mappings }} }}'
response = self.client.post(url, data={'query': query}, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
data = json.loads(response.content)
self.assertNotIn('errors', data)
self.assertEqual(data['data']['service_template']['port_mappings'], ['tcp/80', 'udp/53'])
def test_graphql_protocol_and_port_filter(self):
"""Combined protocol+port filtering works for ServiceTemplate over GraphQL."""
self.add_permissions('ipam.view_servicetemplate')
url = reverse('graphql')
query = '{ service_template_list(filters: {protocol: [TCP], port: [1]}) { name } }'
response = self.client.post(url, data={'query': query}, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
data = json.loads(response.content)
self.assertNotIn('errors', data)
# Only Service Template 1 exposes tcp/1.
self.assertEqual([t['name'] for t in data['data']['service_template_list']], ['Service Template 1'])
def test_graphql_port_only_filter(self):
"""A port-only GraphQL filter (no protocol) works for ServiceTemplate."""
self.add_permissions('ipam.view_servicetemplate')
url = reverse('graphql')
query = '{ service_template_list(filters: {port: [3]}) { name } }'
response = self.client.post(url, data={'query': query}, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
data = json.loads(response.content)
self.assertNotIn('errors', data)
# Only Service Template 2 exposes port 3 (tcp/3).
self.assertEqual([t['name'] for t in data['data']['service_template_list']], ['Service Template 2'])
def test_graphql_port_mappings_filter(self):
"""The whole-mapping GraphQL filter matches an exact protocol/port pair for ServiceTemplate."""
self.add_permissions('ipam.view_servicetemplate')
url = reverse('graphql')
query = '{ service_template_list(filters: {port_mappings: ["tcp/3"]}) { name } }'
response = self.client.post(url, data={'query': query}, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
data = json.loads(response.content)
self.assertNotIn('errors', data)
self.assertEqual([t['name'] for t in data['data']['service_template_list']], ['Service Template 2'])
# udp/3 does not exist, though tcp/3 does
query = '{ service_template_list(filters: {port_mappings: ["udp/3"]}) { name } }'
response = self.client.post(url, data={'query': query}, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
data = json.loads(response.content)
self.assertNotIn('errors', data)
self.assertEqual(data['data']['service_template_list'], [])
def test_graphql_port_range_lookups(self):
"""The port range lookups are available on ServiceTemplate too, and stay correlated."""
self.add_permissions('ipam.view_servicetemplate')
url = reverse('graphql')
# Templates 1-3 expose tcp/1-2, tcp/3-4 and tcp/5-6 respectively
query = '{ service_template_list(filters: {port__gte: [3], port__lte: [4]}) { name } }'
response = self.client.post(url, data={'query': query}, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
data = json.loads(response.content)
self.assertNotIn('errors', data)
self.assertEqual([t['name'] for t in data['data']['service_template_list']], ['Service Template 2'])
# A protocol which no template exposes narrows the same range to nothing
query = '{ service_template_list(filters: {protocol: [UDP], port__gte: [3], port__lte: [4]}) { name } }'
response = self.client.post(url, data={'query': query}, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
data = json.loads(response.content)
self.assertNotIn('errors', data)
self.assertEqual(data['data']['service_template_list'], [])
def test_create_duplicate_mapping_rejected(self):
"""A duplicate protocol/port entry is rejected with a clean 400 (not a 500)."""
self.add_permissions('ipam.add_servicetemplate')
data = {'name': 'Duplicate', 'port_mappings': ['tcp/80', 'tcp/80']}
response = self.client.post(self._get_list_url(), data, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_400_BAD_REQUEST)
def test_create_port_out_of_range_rejected(self):
"""Ports outside SERVICE_PORT_MIN..SERVICE_PORT_MAX are rejected with a 400."""
self.add_permissions('ipam.add_servicetemplate')
data = {'name': 'OutOfRange', 'port_mappings': ['tcp/70000']}
response = self.client.post(self._get_list_url(), data, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_400_BAD_REQUEST)
def test_create_without_port_mappings_rejected(self):
"""A service (template) must define at least one port mapping (400, not a portless object)."""
self.add_permissions('ipam.add_servicetemplate')
data = {'name': 'Portless', 'port_mappings': []}
response = self.client.post(self._get_list_url(), data, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_400_BAD_REQUEST)
def test_create_normalizes_port_mappings(self):
"""Input is normalized (e.g. leading zeros stripped) into the model's canonical form."""
self.add_permissions('ipam.add_servicetemplate')
data = {'name': 'Normalized', 'port_mappings': ['tcp/443', 'tcp/080']}
response = self.client.post(self._get_list_url(), data, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_201_CREATED)
template = ServiceTemplate.objects.get(name='Normalized')
self.assertEqual(template.port_mappings, ['tcp/443', 'tcp/80'])
def test_port_mappings_read(self):
"""port_mappings reads back as the stored flat list of protocol/port strings."""
self.add_permissions('ipam.view_servicetemplate')
template = ServiceTemplate.objects.create(name='Mappings', port_mappings=['tcp/443', 'tcp/80', 'udp/53'])
response = self.client.get(self._get_detail_url(template), **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
self.assertEqual(response.data['port_mappings'], ['tcp/443', 'tcp/80', 'udp/53'])
def test_legacy_read_single_protocol(self):
"""A single-protocol service reports the deprecated protocol/ports fields for compatibility."""
self.add_permissions('ipam.view_servicetemplate')
template = ServiceTemplate.objects.create(name='Legacy Single', port_mappings=['tcp/80', 'tcp/443'])
response = self.client.get(self._get_detail_url(template), **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
# The legacy protocol field keeps the standard choice-field {value, label} read shape.
self.assertEqual(response.data['protocol'], {'value': 'tcp', 'label': 'TCP'})
self.assertEqual(response.data['ports'], [80, 443])
self.assertEqual(response.data['port_mappings'], ['tcp/80', 'tcp/443'])
def test_legacy_read_multiple_protocols_null(self):
"""A multi-protocol service can't be expressed in the old format, so protocol/ports are null."""
self.add_permissions('ipam.view_servicetemplate')
template = ServiceTemplate.objects.create(name='Legacy Multi', port_mappings=['tcp/53', 'udp/53'])
response = self.client.get(self._get_detail_url(template), **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
self.assertIsNone(response.data['protocol'])
self.assertIsNone(response.data['ports'])
self.assertEqual(response.data['port_mappings'], ['tcp/53', 'udp/53'])
def test_legacy_read_empty_distinct_from_multiple(self):
"""An empty service is distinguishable from a multi-protocol one: ports=[] vs ports=null."""
self.add_permissions('ipam.view_servicetemplate')
# A mapping-less template is normally prevented by validation, but can exist via migrated data;
# objects.create() bypasses full_clean() so we can exercise the read path here.
template = ServiceTemplate.objects.create(name='Legacy Empty', port_mappings=[])
response = self.client.get(self._get_detail_url(template), **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
self.assertIsNone(response.data['protocol'])
self.assertEqual(response.data['ports'], [])
self.assertEqual(response.data['port_mappings'], [])
def test_create_via_legacy_format(self):
"""The deprecated protocol/ports format is accepted on write and translated to port_mappings."""
self.add_permissions('ipam.add_servicetemplate')
data = {'name': 'Legacy Create', 'protocol': 'tcp', 'ports': [80, 443]}
response = self.client.post(self._get_list_url(), data, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_201_CREATED)
template = ServiceTemplate.objects.get(name='Legacy Create')
self.assertEqual(template.port_mappings, ['tcp/80', 'tcp/443'])
def test_legacy_empty_ports_reports_at_least_one(self):
"""
A legacy write with an explicitly-empty ports list (allowed by the old API) is rejected with the
at-least-one-mapping message keyed to ports, not the misleading "both are required" error.
"""
self.add_permissions('ipam.add_servicetemplate')
data = {'name': 'Legacy Empty', 'protocol': 'tcp', 'ports': []}
response = self.client.post(self._get_list_url(), data, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_400_BAD_REQUEST)
self.assertIn('ports', response.data)
def test_create_port_mappings_case_insensitive(self):
"""port_mappings accepts protocols in any case (e.g. 'TCP/80') and stores the canonical value."""
self.add_permissions('ipam.add_servicetemplate')
data = {'name': 'Case Insensitive', 'port_mappings': ['TCP/80', 'UDP/53']}
response = self.client.post(self._get_list_url(), data, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_201_CREATED)
template = ServiceTemplate.objects.get(name='Case Insensitive')
self.assertEqual(template.port_mappings, ['tcp/80', 'udp/53'])
def test_both_formats_rejected(self):
"""Supplying both port_mappings and the legacy protocol/ports is ambiguous and must 400."""
self.add_permissions('ipam.add_servicetemplate')
# port_mappings is a well-formed flat list so it passes field-level parsing and actually reaches
# the validate() mutual-exclusion guard (rather than 400ing on a malformed value first).
data = {
'name': 'Both Formats',
'port_mappings': ['udp/53'],
'protocol': 'tcp',
'ports': [80],
}
response = self.client.post(self._get_list_url(), data, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_400_BAD_REQUEST)
self.assertFalse(ServiceTemplate.objects.filter(name='Both Formats').exists())
def test_create_legacy_port_out_of_range_rejected(self):
"""A legacy ports value outside the permitted range is rejected with a 400 (not a 500)."""
self.add_permissions('ipam.add_servicetemplate')
data = {'name': 'Legacy OOR', 'protocol': 'tcp', 'ports': [70000]}
response = self.client.post(self._get_list_url(), data, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_400_BAD_REQUEST)
def test_create_legacy_protocol_without_ports_rejected(self):
"""One half of the legacy pair is ambiguous and must 400, not silently drop the input."""
self.add_permissions('ipam.add_servicetemplate')
data = {'name': 'Legacy Half', 'protocol': 'tcp'}
response = self.client.post(self._get_list_url(), data, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_400_BAD_REQUEST)
def test_update_legacy_ports_only(self):
"""A partial update supplying only legacy 'ports' keeps the existing single protocol."""
self.add_permissions('ipam.change_servicetemplate')
template = ServiceTemplate.objects.create(name='Legacy Patch', port_mappings=['tcp/80'])
response = self.client.patch(
self._get_detail_url(template), {'ports': [8080]}, format='json', **self.header
)
self.assertHttpStatus(response, status.HTTP_200_OK)
template.refresh_from_db()
self.assertEqual(template.port_mappings, ['tcp/8080'])
def test_update_legacy_protocol_only(self):
"""A partial update supplying only legacy 'protocol' keeps the existing ports."""
self.add_permissions('ipam.change_servicetemplate')
template = ServiceTemplate.objects.create(name='Legacy Patch', port_mappings=['tcp/80', 'tcp/443'])
response = self.client.patch(
self._get_detail_url(template), {'protocol': 'udp'}, format='json', **self.header
)
self.assertHttpStatus(response, status.HTTP_200_OK)
template.refresh_from_db()
self.assertEqual(template.port_mappings, ['udp/80', 'udp/443'])
def test_update_legacy_single_field_multiprotocol_rejected(self):
"""A single legacy field can't patch a multi-protocol service (no single-protocol form)."""
self.add_permissions('ipam.change_servicetemplate')
template = ServiceTemplate.objects.create(name='Legacy Patch', port_mappings=['tcp/80', 'udp/53'])
response = self.client.patch(
self._get_detail_url(template), {'ports': [8080]}, format='json', **self.header
)
self.assertHttpStatus(response, status.HTTP_400_BAD_REQUEST)
template.refresh_from_db()
self.assertEqual(template.port_mappings, ['tcp/80', 'udp/53'])
def test_read_malformed_port_mapping_degrades(self):
"""A malformed stored mapping (validation bypassed) must degrade on API read, not raise a 500."""
self.add_permissions('ipam.view_servicetemplate')
# objects.create bypasses full_clean, simulating a raw-DB/plugin write of a non-numeric port
template = ServiceTemplate.objects.create(name='Malformed', port_mappings=['tcp/80', 'tcp/abc'])
response = self.client.get(self._get_detail_url(template), **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
# port_mappings echoes the stored values verbatim (no reformatting). The legacy view can't
# faithfully represent a mapping that fails integer coercion, so rather than silently returning
# a subset it reports ports=null — the same "not representable" signal used for multi-protocol.
self.assertEqual(response.data['port_mappings'], ['tcp/80', 'tcp/abc'])
self.assertIsNone(response.data['ports'])
self.assertIsNone(response.data['protocol'])
class ServiceTestCase(APIViewTestCases.APIViewTestCase):
model = Service
brief_fields = ['description', 'display', 'id', 'name', 'ports', 'protocol', 'url']
brief_fields = ['description', 'display', 'id', 'name', 'port_mappings', 'url']
bulk_update_data = {
'description': 'New description',
}
@ -1497,33 +1745,254 @@ class ServiceTestCase(APIViewTestCases.APIViewTestCase):
)
Device.objects.bulk_create(devices)
services = (
Service(parent=devices[0], name='Service 1', protocol=ServiceProtocolChoices.PROTOCOL_TCP, ports=[1]),
Service(parent=devices[0], name='Service 2', protocol=ServiceProtocolChoices.PROTOCOL_TCP, ports=[2]),
Service(parent=devices[0], name='Service 3', protocol=ServiceProtocolChoices.PROTOCOL_TCP, ports=[3]),
)
Service.objects.bulk_create(services)
Service.objects.bulk_create([
Service(parent=devices[0], name='Service 1', port_mappings=['tcp/1']),
Service(parent=devices[0], name='Service 2', port_mappings=['tcp/2']),
Service(parent=devices[0], name='Service 3', port_mappings=['tcp/3']),
])
cls.create_data = [
{
'parent_object_id': devices[1].pk,
'parent_object_type': 'dcim.device',
'name': 'Service 4',
'protocol': ServiceProtocolChoices.PROTOCOL_TCP,
'ports': [4],
'port_mappings': ['tcp/4'],
},
{
'parent_object_id': devices[1].pk,
'parent_object_type': 'dcim.device',
'name': 'Service 5',
'protocol': ServiceProtocolChoices.PROTOCOL_TCP,
'ports': [5],
'name': 'dns',
'port_mappings': ['tcp/53', 'udp/53'],
},
{
'parent_object_id': devices[1].pk,
'parent_object_type': 'dcim.device',
'name': 'Service 6',
'protocol': ServiceProtocolChoices.PROTOCOL_TCP,
'ports': [6],
'port_mappings': ['tcp/6'],
},
]
def test_graphql_protocol_and_port_filter(self):
"""Combined protocol + port filtering works over GraphQL (port mappings live in an array)."""
self.add_permissions('ipam.view_service')
url = reverse('graphql')
query = '{ service_list(filters: {protocol: [TCP], port: [1]}) { id name } }'
response = self.client.post(url, data={'query': query}, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
data = json.loads(response.content)
self.assertNotIn('errors', data)
self.assertEqual(len(data['data']['service_list']), 1)
self.assertEqual(data['data']['service_list'][0]['name'], 'Service 1')
def test_graphql_protocol_and_port_filter_multiprotocol(self):
"""
A combined protocol+port filter must match a single mapping, not protocol and port matched
independently across different mappings on the same object (GraphQL parity with the FilterSet).
"""
self.add_permissions('ipam.view_service')
device = Device.objects.first()
Service.objects.create(parent=device, name='dns-multi', port_mappings=['tcp/8080', 'udp/53'])
url = reverse('graphql')
# tcp/8080 exists on the service -> matches
query = '{ service_list(filters: {protocol: [TCP], port: [8080]}) { name } }'
response = self.client.post(url, data={'query': query}, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
data = json.loads(response.content)
self.assertNotIn('errors', data)
self.assertEqual([s['name'] for s in data['data']['service_list']], ['dns-multi'])
# udp/8080 does not exist, even though the service has udp (on 53) and 8080 (on tcp)
query = '{ service_list(filters: {protocol: [UDP], port: [8080]}) { name } }'
response = self.client.post(url, data={'query': query}, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
data = json.loads(response.content)
self.assertNotIn('errors', data)
self.assertEqual(data['data']['service_list'], [])
def test_graphql_port_mappings(self):
"""port_mappings is exposed over GraphQL as a flat list of protocol/port strings."""
self.add_permissions('ipam.view_service')
device = Device.objects.first()
service = Service.objects.create(parent=device, name='GQL Mappings', port_mappings=['tcp/80', 'udp/53'])
url = reverse('graphql')
query = f'{{ service(id: {service.pk}) {{ port_mappings }} }}'
response = self.client.post(url, data={'query': query}, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
data = json.loads(response.content)
self.assertNotIn('errors', data)
self.assertEqual(data['data']['service']['port_mappings'], ['tcp/80', 'udp/53'])
def test_graphql_port_only_filter(self):
"""A port-only GraphQL filter (no protocol) matches the port across any protocol."""
self.add_permissions('ipam.view_service')
device = Device.objects.first()
Service.objects.create(parent=device, name='udp-on-1', port_mappings=['udp/1'])
url = reverse('graphql')
query = '{ service_list(filters: {port: [1]}) { name } }'
response = self.client.post(url, data={'query': query}, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
data = json.loads(response.content)
self.assertNotIn('errors', data)
# Service 1 (tcp/1) and the new udp-on-1 both expose port 1, on different protocols.
self.assertEqual({s['name'] for s in data['data']['service_list']}, {'Service 1', 'udp-on-1'})
def test_graphql_port_mappings_filter(self):
"""The whole-mapping GraphQL filter matches an exact protocol/port pair, OR'd across values."""
self.add_permissions('ipam.view_service')
device = Device.objects.first()
Service.objects.create(parent=device, name='udp-on-1', port_mappings=['udp/1'])
url = reverse('graphql')
# tcp/1 must not match the udp-only service, even though both expose port 1
query = '{ service_list(filters: {port_mappings: ["tcp/1"]}) { name } }'
response = self.client.post(url, data={'query': query}, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
data = json.loads(response.content)
self.assertNotIn('errors', data)
self.assertEqual([s['name'] for s in data['data']['service_list']], ['Service 1'])
# Multiple values are OR'd, and input is normalized ('UDP/001' -> 'udp/1')
query = '{ service_list(filters: {port_mappings: ["tcp/1", "UDP/001"]}) { name } }'
response = self.client.post(url, data={'query': query}, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
data = json.loads(response.content)
self.assertNotIn('errors', data)
self.assertEqual({s['name'] for s in data['data']['service_list']}, {'Service 1', 'udp-on-1'})
def test_graphql_protocol_only_filter(self):
"""A protocol-only GraphQL filter matches services exposing that protocol on any port."""
self.add_permissions('ipam.view_service')
device = Device.objects.first()
Service.objects.create(parent=device, name='udp-svc', port_mappings=['udp/9'])
url = reverse('graphql')
query = '{ service_list(filters: {protocol: [UDP]}) { name } }'
response = self.client.post(url, data={'query': query}, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
data = json.loads(response.content)
self.assertNotIn('errors', data)
# Only the udp service matches; the seeded Service 1-3 are all tcp.
self.assertEqual([s['name'] for s in data['data']['service_list']], ['udp-svc'])
def test_graphql_port_range_lookups(self):
"""The port__gt/gte/lt/lte GraphQL lookups mirror their identically-named REST counterparts."""
self.add_permissions('ipam.view_service')
url = reverse('graphql')
# Seeded services expose tcp/1, tcp/2 and tcp/3 respectively
for filters, expected in (
('{port__gt: [2]}', {'Service 3'}),
('{port__gte: [2]}', {'Service 2', 'Service 3'}),
('{port__lt: [2]}', {'Service 1'}),
('{port__lte: [2]}', {'Service 1', 'Service 2'}),
):
query = f'{{ service_list(filters: {filters}) {{ name }} }}'
response = self.client.post(url, data={'query': query}, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
data = json.loads(response.content)
self.assertNotIn('errors', data)
self.assertEqual({s['name'] for s in data['data']['service_list']}, expected, msg=filters)
def test_graphql_port_range_bounds_correlated(self):
"""
Combined range bounds must be satisfied by a *single* mapping, so a service straddling the range
without any port inside it does not match (GraphQL parity with the FilterSet).
"""
self.add_permissions('ipam.view_service')
device = Device.objects.first()
Service.objects.create(parent=device, name='straddles', port_mappings=['tcp/500', 'tcp/5000'])
Service.objects.create(parent=device, name='inside', port_mappings=['tcp/1500'])
url = reverse('graphql')
query = '{ service_list(filters: {port__gte: [1000], port__lte: [2000]}) { name } }'
response = self.client.post(url, data={'query': query}, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
data = json.loads(response.content)
self.assertNotIn('errors', data)
self.assertEqual([s['name'] for s in data['data']['service_list']], ['inside'])
def test_graphql_protocol_and_port_range_correlated(self):
"""A protocol combined with a range lookup must also be satisfied by a single mapping."""
self.add_permissions('ipam.view_service')
device = Device.objects.first()
Service.objects.create(parent=device, name='mixed', port_mappings=['tcp/80', 'udp/9999'])
Service.objects.create(parent=device, name='tcp-high', port_mappings=['tcp/9999'])
url = reverse('graphql')
# 'mixed' has a tcp mapping and a mapping above 1000, but no tcp mapping above 1000
query = '{ service_list(filters: {protocol: [TCP], port__gt: [1000]}) { name } }'
response = self.client.post(url, data={'query': query}, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
data = json.loads(response.content)
self.assertNotIn('errors', data)
self.assertEqual([s['name'] for s in data['data']['service_list']], ['tcp-high'])
def test_port_mapping_prefix_branch(self):
"""
The nested-relation (prefix) branch of the shared port filter resolves matches through a
relation. No GraphQL type currently exposes a nested Service filter, so exercise the helper
directly via the IPAddress -> services reverse relation.
"""
from ipam.graphql.filters import _port_mapping_prefix_q
device = Device.objects.first()
service = Service.objects.create(parent=device, name='svc-with-ip', port_mappings=['tcp/1'])
ip = IPAddress.objects.create(address='192.0.2.1/32')
service.ipaddresses.add(ip)
match = _port_mapping_prefix_q(Service, ['tcp'], [('exact', [1])], 'services__')
self.assertIn(ip, IPAddress.objects.filter(match))
miss = _port_mapping_prefix_q(Service, ['tcp'], [('exact', [999])], 'services__')
self.assertNotIn(ip, IPAddress.objects.filter(miss))
def test_update_full_body_roundtrip(self):
"""
A full-object round-trip (GET then PUT of the same body, including the legacy protocol/ports the
read emitted alongside port_mappings) succeeds; only a genuine conflict is rejected.
"""
self.add_permissions('ipam.view_service', 'ipam.change_service')
service = Service.objects.get(name='Service 1') # tcp/1
read = self.client.get(self._get_detail_url(service), **self.header).data
put_data = {
'parent_object_type': 'dcim.device',
'parent_object_id': service.parent_object_id,
'name': service.name,
'port_mappings': read['port_mappings'],
# The legacy protocol field reads as {value, label}; on write NetBox choice fields take the
# raw value, so a well-behaved round-trip resubmits read['protocol']['value'].
'protocol': read['protocol']['value'],
'ports': read['ports'],
}
response = self.client.put(self._get_detail_url(service), put_data, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
# A legacy field that disagrees with port_mappings is still rejected as a conflict.
put_data['protocol'] = 'udp'
response = self.client.put(self._get_detail_url(service), put_data, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_400_BAD_REQUEST)
def test_legacy_read_single_protocol(self):
"""A single-protocol service reports the deprecated protocol/ports fields for compatibility."""
self.add_permissions('ipam.view_service')
service = Service.objects.get(name='Service 1') # port_mappings=['tcp/1']
response = self.client.get(self._get_detail_url(service), **self.header)
self.assertHttpStatus(response, status.HTTP_200_OK)
# The legacy protocol field keeps the standard choice-field {value, label} read shape.
self.assertEqual(response.data['protocol'], {'value': 'tcp', 'label': 'TCP'})
self.assertEqual(response.data['ports'], [1])
def test_create_via_legacy_format(self):
"""The deprecated protocol/ports format is accepted on write and translated to port_mappings."""
self.add_permissions('ipam.add_service')
device = Device.objects.first()
data = {
'parent_object_type': 'dcim.device',
'parent_object_id': device.pk,
'name': 'Legacy Service',
'protocol': 'udp',
'ports': [53, 67],
}
response = self.client.post(self._get_list_url(), data, format='json', **self.header)
self.assertHttpStatus(response, status.HTTP_201_CREATED)
service = Service.objects.get(name='Legacy Service')
self.assertEqual(service.port_mappings, ['udp/53', 'udp/67'])

View File

@ -1312,24 +1312,9 @@ class IPAddressTestCase(TestCase, ChangeLoggedFilterSetTests):
)
services = (
Service(
parent=devices[0],
name='Service 1',
protocol=ServiceProtocolChoices.PROTOCOL_TCP,
ports=[1],
),
Service(
parent=devices[1],
name='Service 2',
protocol=ServiceProtocolChoices.PROTOCOL_TCP,
ports=[1],
),
Service(
parent=devices[2],
name='Service 3',
protocol=ServiceProtocolChoices.PROTOCOL_TCP,
ports=[1],
),
Service(parent=devices[0], name='Service 1', port_mappings=['tcp/80']),
Service(parent=devices[1], name='Service 2', port_mappings=['tcp/80']),
Service(parent=devices[2], name='Service 3', port_mappings=['tcp/80']),
)
Service.objects.bulk_create(services)
services[0].ipaddresses.add(ipaddresses[0])
@ -2383,46 +2368,17 @@ class VLANTranslationRuleTestCase(TestCase, ChangeLoggedFilterSetTests):
class ServiceTemplateTestCase(TestCase, ChangeLoggedFilterSetTests):
queryset = ServiceTemplate.objects.all()
filterset = ServiceTemplateFilterSet
ignore_fields = ('ports',)
@classmethod
def setUpTestData(cls):
service_templates = (
ServiceTemplate(
name='Service Template 1',
protocol=ServiceProtocolChoices.PROTOCOL_TCP,
ports=[1001],
description='foobar1'
),
ServiceTemplate(
name='Service Template 2',
protocol=ServiceProtocolChoices.PROTOCOL_TCP,
ports=[1002],
description='foobar2'
),
ServiceTemplate(
name='Service Template 3',
protocol=ServiceProtocolChoices.PROTOCOL_UDP,
ports=[1003],
description='foobar3'
),
ServiceTemplate(
name='Service Template 4',
protocol=ServiceProtocolChoices.PROTOCOL_TCP,
ports=[2001]
),
ServiceTemplate(
name='Service Template 5',
protocol=ServiceProtocolChoices.PROTOCOL_TCP,
ports=[2002]
),
ServiceTemplate(
name='Service Template 6',
protocol=ServiceProtocolChoices.PROTOCOL_UDP,
ports=[2003]
),
)
ServiceTemplate.objects.bulk_create(service_templates)
ServiceTemplate.objects.bulk_create((
ServiceTemplate(name='Service Template 1', description='foobar1', port_mappings=['tcp/1001']),
ServiceTemplate(name='Service Template 2', description='foobar2', port_mappings=['tcp/1002']),
ServiceTemplate(name='Service Template 3', description='foobar3', port_mappings=['udp/1003']),
ServiceTemplate(name='Service Template 4', port_mappings=['tcp/2001']),
ServiceTemplate(name='Service Template 5', port_mappings=['tcp/2002']),
ServiceTemplate(name='Service Template 6', port_mappings=['udp/2003']),
))
def test_q(self):
params = {'q': 'foobar1'}
@ -2432,14 +2388,121 @@ class ServiceTemplateTestCase(TestCase, ChangeLoggedFilterSetTests):
params = {'name': ['Service Template 1', 'Service Template 2']}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)
def test_port_mappings(self):
# A whole-mapping lookup matches only that exact protocol/port pair.
params = {'port_mappings': ['tcp/1001']}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 1)
# udp/1001 does not exist, though tcp/1001 does
params = {'port_mappings': ['udp/1001']}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 0)
# Multiple values are OR'd
params = {'port_mappings': ['tcp/1001', 'udp/1003']}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)
def test_port_mappings_normalized(self):
# Input is canonicalized for lookup, so case and leading zeros still match stored values.
ServiceTemplate.objects.create(name='Padded', port_mappings=['tcp/80'])
for value in ('TCP/80', 'tcp/080', 'TCP/080'):
params = {'port_mappings': [value]}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 1, msg=value)
def test_port_mappings_unmatchable(self):
# An unknown protocol or malformed pair yields no results rather than a validation error.
for value in ('bogus/80', 'tcp/notaport', 'tcp'):
params = {'port_mappings': [value]}
filterset = self.filterset(params, self.queryset)
self.assertTrue(filterset.is_valid(), msg=value)
self.assertEqual(filterset.qs.count(), 0, msg=value)
# An empty value is a no-op, as it is for every other filter
filterset = self.filterset({'port_mappings': ['']}, self.queryset)
self.assertTrue(filterset.is_valid())
self.assertEqual(filterset.qs.count(), self.queryset.count())
def test_port_mappings_negated(self):
# port_mappings__n excludes objects exposing the given mapping (1 of 6 templates has tcp/1001).
params = {'port_mappings__n': ['tcp/1001']}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 5)
def test_port_mappings_multiprotocol(self):
# A mapping lookup is satisfied by any one of an object's mappings.
ServiceTemplate.objects.create(name='DNS', port_mappings=['tcp/53', 'udp/53'])
params = {'port_mappings': ['udp/53']}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 1)
# Combined with the correlated protocol/port filters, both must hold (independently)
params = {'port_mappings': ['udp/53'], 'protocol': [ServiceProtocolChoices.PROTOCOL_TCP]}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 1)
params = {'port_mappings': ['udp/53'], 'port': [1001]}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 0)
def test_protocol(self):
params = {'protocol': ServiceProtocolChoices.PROTOCOL_TCP}
params = {'protocol': [ServiceProtocolChoices.PROTOCOL_TCP]}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 4)
def test_protocol_negated(self):
# protocol__n excludes objects exposing the given protocol (2 of 6 templates are udp-only).
params = {'protocol__n': [ServiceProtocolChoices.PROTOCOL_TCP]}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)
def test_port(self):
params = {'port': '1001'}
params = {'port': [1001]}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 1)
def test_protocol_and_port(self):
# A combined protocol+port filter must match a single mapping, not protocol and port matched
# independently across different mappings on the same object.
ServiceTemplate.objects.create(name='DNS', port_mappings=['tcp/8080', 'udp/53'])
params = {'protocol': [ServiceProtocolChoices.PROTOCOL_TCP], 'port': [8080]}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 1)
# udp/8080 does not exist, even though this template has udp (on 53) and 8080 (on tcp)
params = {'protocol': [ServiceProtocolChoices.PROTOCOL_UDP], 'port': [8080]}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 0)
# Single-mapping composition still works
params = {'protocol': [ServiceProtocolChoices.PROTOCOL_TCP], 'port': [1001]}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 1)
def test_port_negated(self):
# port__n excludes objects exposing the given port (1 of 6 templates uses 1001).
params = {'port__n': [1001]}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 5)
def test_port_range_lookups(self):
# Ports in play: tcp/1001, tcp/1002, udp/1003, tcp/2001, tcp/2002, udp/2003
params = {'port__gt': [2000]}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 3)
params = {'port__gte': [2001]}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 3)
params = {'port__lt': [1003]}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)
params = {'port__lte': [1003]}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 3)
def test_port_range_bounds_combined(self):
# gte + lte describe one range, so both bounds must hold for the same mapping.
ServiceTemplate.objects.create(name='Straddling', port_mappings=['tcp/500', 'tcp/5000'])
params = {'port__gte': [1000], 'port__lte': [1003]}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 3)
def test_protocol_and_port_range(self):
# A range lookup is correlated with protocol, so this template's udp mapping (53) must not be
# matched by way of its tcp mapping (8080).
ServiceTemplate.objects.create(name='DNS', port_mappings=['tcp/8080', 'udp/53'])
params = {'protocol': [ServiceProtocolChoices.PROTOCOL_TCP], 'port__gt': [2000]}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 3)
params = {'protocol': [ServiceProtocolChoices.PROTOCOL_UDP], 'port__gt': [1000]}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)
def test_port_mappings_filter_is_idempotent(self):
# The correlated protocol/port predicate is applied by filter_queryset() rather than by the
# individual filters. Filtering twice with the same FilterSet instance must yield the same result,
# i.e. that must stay free of per-instance state which would drop or double the predicate.
filterset = self.filterset({'protocol': [ServiceProtocolChoices.PROTOCOL_UDP]}, self.queryset)
self.assertEqual(filterset.qs.count(), 2)
self.assertEqual(filterset.filter_queryset(self.queryset).count(), 2)
def test_description(self):
params = {'description': ['foobar1', 'foobar2']}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)
@ -2448,7 +2511,6 @@ class ServiceTemplateTestCase(TestCase, ChangeLoggedFilterSetTests):
class ServiceTestCase(TestCase, ChangeLoggedFilterSetTests):
queryset = Service.objects.all()
filterset = ServiceFilterSet
ignore_fields = ('ports',)
@classmethod
def setUpTestData(cls):
@ -2494,50 +2556,13 @@ class ServiceTestCase(TestCase, ChangeLoggedFilterSetTests):
)
services = (
Service(
parent=devices[0],
name='Service 1',
protocol=ServiceProtocolChoices.PROTOCOL_TCP,
ports=[1001],
description='foobar1',
),
Service(
parent=devices[1],
name='Service 2',
protocol=ServiceProtocolChoices.PROTOCOL_TCP,
ports=[1002],
description='foobar2',
),
Service(
parent=devices[2],
name='Service 3',
protocol=ServiceProtocolChoices.PROTOCOL_UDP,
ports=[1003]
),
Service(
parent=virtual_machines[0],
name='Service 4',
protocol=ServiceProtocolChoices.PROTOCOL_TCP,
ports=[2001],
),
Service(
parent=virtual_machines[1],
name='Service 5',
protocol=ServiceProtocolChoices.PROTOCOL_TCP,
ports=[2002],
),
Service(
parent=virtual_machines[2],
name='Service 6',
protocol=ServiceProtocolChoices.PROTOCOL_UDP,
ports=[2003],
),
Service(
parent=fhrp_group,
name='Service 7',
protocol=ServiceProtocolChoices.PROTOCOL_UDP,
ports=[2004],
),
Service(parent=devices[0], name='Service 1', description='foobar1', port_mappings=['tcp/1001']),
Service(parent=devices[1], name='Service 2', description='foobar2', port_mappings=['tcp/1002']),
Service(parent=devices[2], name='Service 3', port_mappings=['udp/1003']),
Service(parent=virtual_machines[0], name='Service 4', port_mappings=['tcp/2001']),
Service(parent=virtual_machines[1], name='Service 5', port_mappings=['tcp/2002']),
Service(parent=virtual_machines[2], name='Service 6', port_mappings=['udp/2003']),
Service(parent=fhrp_group, name='Service 7', port_mappings=['udp/2004']),
)
Service.objects.bulk_create(services)
services[0].ipaddresses.add(ip_addresses[0])
@ -2552,18 +2577,120 @@ class ServiceTestCase(TestCase, ChangeLoggedFilterSetTests):
params = {'name': ['Service 1', 'Service 2']}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)
def test_port_mappings(self):
# A whole-mapping lookup matches only that exact protocol/port pair.
params = {'port_mappings': ['tcp/1001']}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 1)
# udp/1001 does not exist, though tcp/1001 does
params = {'port_mappings': ['udp/1001']}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 0)
# Multiple values are OR'd
params = {'port_mappings': ['tcp/1001', 'udp/1003']}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)
def test_port_mappings_normalized(self):
# Input is canonicalized for lookup, so case and leading zeros still match stored values.
Service.objects.create(parent=Device.objects.first(), name='Padded', port_mappings=['tcp/80'])
for value in ('TCP/80', 'tcp/080', 'TCP/080'):
params = {'port_mappings': [value]}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 1, msg=value)
def test_port_mappings_unmatchable(self):
# An unknown protocol or malformed pair yields no results rather than a validation error.
for value in ('bogus/80', 'tcp/notaport', 'tcp'):
params = {'port_mappings': [value]}
filterset = self.filterset(params, self.queryset)
self.assertTrue(filterset.is_valid(), msg=value)
self.assertEqual(filterset.qs.count(), 0, msg=value)
# An empty value is a no-op, as it is for every other filter
filterset = self.filterset({'port_mappings': ['']}, self.queryset)
self.assertTrue(filterset.is_valid())
self.assertEqual(filterset.qs.count(), self.queryset.count())
def test_port_mappings_negated(self):
# port_mappings__n excludes objects exposing the given mapping (1 of 7 services has tcp/1001).
params = {'port_mappings__n': ['tcp/1001']}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 6)
def test_port_mappings_multiprotocol(self):
# A mapping lookup is satisfied by any one of an object's mappings.
Service.objects.create(parent=Device.objects.first(), name='DNS', port_mappings=['tcp/53', 'udp/53'])
params = {'port_mappings': ['udp/53']}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 1)
# Combined with the correlated protocol/port filters, both must hold (independently)
params = {'port_mappings': ['udp/53'], 'protocol': [ServiceProtocolChoices.PROTOCOL_TCP]}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 1)
params = {'port_mappings': ['udp/53'], 'port': [1001]}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 0)
def test_protocol(self):
params = {'protocol': ServiceProtocolChoices.PROTOCOL_TCP}
params = {'protocol': [ServiceProtocolChoices.PROTOCOL_TCP]}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 4)
def test_protocol_negated(self):
# protocol__n excludes objects exposing the given protocol (3 of 7 services are udp-only).
params = {'protocol__n': [ServiceProtocolChoices.PROTOCOL_TCP]}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 3)
def test_description(self):
params = {'description': ['foobar1', 'foobar2']}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)
def test_port(self):
params = {'port': '1001'}
params = {'port': [1001]}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 1)
def test_protocol_and_port(self):
# A combined protocol+port filter must match a single mapping, not protocol and port matched
# independently across different mappings on the same object.
device = Device.objects.first()
Service.objects.create(parent=device, name='DNS', port_mappings=['tcp/8080', 'udp/53'])
params = {'protocol': [ServiceProtocolChoices.PROTOCOL_TCP], 'port': [8080]}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 1)
# udp/8080 does not exist, even though this service has udp (on 53) and 8080 (on tcp)
params = {'protocol': [ServiceProtocolChoices.PROTOCOL_UDP], 'port': [8080]}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 0)
# Single-mapping composition still works
params = {'protocol': [ServiceProtocolChoices.PROTOCOL_TCP], 'port': [1001]}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 1)
def test_port_negated(self):
# port__n excludes objects exposing the given port (1 of 7 services uses 1001).
params = {'port__n': [1001]}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 6)
def test_port_range_lookups(self):
# Ports in play: tcp/1001, tcp/1002, udp/1003, tcp/2001, tcp/2002, udp/2003, udp/2004
params = {'port__gt': [2000]}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 4)
params = {'port__gte': [2001]}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 4)
params = {'port__lt': [1003]}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)
params = {'port__lte': [1003]}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 3)
def test_port_range_bounds_combined(self):
# gte + lte describe one range, so both bounds must hold for the same mapping.
device = Device.objects.first()
Service.objects.create(parent=device, name='Straddling', port_mappings=['tcp/500', 'tcp/5000'])
params = {'port__gte': [1000], 'port__lte': [1003]}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 3)
def test_protocol_and_port_range(self):
# A range lookup is correlated with protocol, so this service's udp mapping (53) must not be
# matched by way of its tcp mapping (8080).
device = Device.objects.first()
Service.objects.create(parent=device, name='DNS', port_mappings=['tcp/8080', 'udp/53'])
params = {'protocol': [ServiceProtocolChoices.PROTOCOL_TCP], 'port__gt': [2000]}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 3)
params = {'protocol': [ServiceProtocolChoices.PROTOCOL_UDP], 'port__gt': [1000]}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 3)
def test_device(self):
devices = Device.objects.all()[:2]
params = {'device_id': [devices[0].pk, devices[1].pk]}

View File

@ -1,10 +1,17 @@
from django.contrib.contenttypes.models import ContentType
from django.core.exceptions import ValidationError
from django.template import Context, Template
from django.test import TestCase
from dcim.constants import InterfaceTypeChoices
from dcim.models import Device, DeviceRole, DeviceType, Interface, Location, Manufacturer, Region, Site, SiteGroup
from ipam.constants import SERVICE_PORT_MAX
from ipam.filtersets import ServiceFilterSet, ServiceTemplateFilterSet
from ipam.forms import PrefixForm, VLANIDBulkCreateForm
from ipam.forms.bulk_import import IPAddressImportForm
from ipam.forms.bulk_import import IPAddressImportForm, ServiceTemplateImportForm
from ipam.forms.fields import PortMappingField
from ipam.forms.filtersets import ServiceFilterForm, ServiceTemplateFilterForm
from ipam.forms.widgets import PortMappingWidget
class PrefixFormTestCase(TestCase):
@ -204,3 +211,226 @@ class VLANFormTestCase(TestCase):
form = VLANIDBulkCreateForm({'pattern': pattern})
self.assertFalse(form.is_valid())
self.assertIn('pattern', form.errors)
class PortMappingFieldTestCase(TestCase):
def test_ports_and_ranges_expand(self):
"""A protocol row's comma/range port string expands into individual protocol/port mappings."""
field = PortMappingField()
value = field.clean('[{"protocol": "tcp", "ports": "80,443,8000-8002"}]')
self.assertEqual(value, ['tcp/80', 'tcp/443', 'tcp/8000', 'tcp/8001', 'tcp/8002'])
def test_out_of_range_rejected_without_expanding(self):
"""
An out-of-bounds range is rejected before it is expanded, so a pathological range cannot
exhaust memory (regression guard for the unbounded parse_numeric_range expansion).
"""
field = PortMappingField()
with self.assertRaises(ValidationError):
field.clean('[{"protocol": "tcp", "ports": "1-9999999999"}]')
with self.assertRaises(ValidationError):
field.clean(f'[{{"protocol": "tcp", "ports": "1-{SERVICE_PORT_MAX + 1}"}}]')
def test_malformed_payload_rejected(self):
"""
The hidden input is ordinary POST data, so a hand-crafted payload need not be the list of
{protocol, ports} objects the widget's JS produces. Anything else must raise a ValidationError
(a 400) rather than an unhandled AttributeError/TypeError (a 500).
"""
field = PortMappingField()
for value in (
'5', # a JSON scalar
'"tcp/80"', # a JSON string
'{"protocol": "tcp", "ports": "80"}', # an object rather than a list of them
'[5]', # a list of non-objects
'[[1, 2]]', # a list of lists
'[null]', # a null row
'[{"protocol": "tcp", "ports": {"a": 1}}]', # ports of the wrong type
'[{"protocol": ["tcp"], "ports": "80"}]', # protocol of the wrong type
):
with self.subTest(value=value), self.assertRaises(ValidationError):
field.clean(value)
def test_widget_tolerates_malformed_value(self):
"""
Re-rendering an invalid bound form hands the widget back the raw POST value, which may be valid
JSON of the wrong shape. It must fall back to a blank row rather than raise while rendering.
"""
widget = PortMappingWidget()
for value in ('5', '"tcp/80"', '{"a": 1}', '[5]', '[[1, 2]]', 'not json at all'):
with self.subTest(value=value):
context = widget.get_context('port_mappings', value, {})
self.assertEqual(context['widget']['rows'], [{'protocol': '', 'ports': ''}])
def test_ports_as_list_requires_protocol(self):
"""
A programmatically-set list of ports still gets the blank-protocol check, rather than emitting a
'/80' token that surfaces as a blank "Invalid protocol:" message.
"""
field = PortMappingField()
self.assertEqual(field.clean('[{"protocol": "tcp", "ports": [80, 443]}]'), ['tcp/80', 'tcp/443'])
with self.assertRaises(ValidationError) as ctx:
field.clean('[{"protocol": "", "ports": [80]}]')
self.assertTrue(any('protocol' in msg.lower() for msg in ctx.exception.messages))
self.assertFalse(any(msg.strip().endswith('Invalid protocol:') for msg in ctx.exception.messages))
def test_protocol_without_ports_reports_clear_error(self):
"""A protocol chosen with no ports reports the 'protocol/port' error, not 'Range \"\" is invalid'."""
field = PortMappingField()
with self.assertRaises(ValidationError) as ctx:
field.clean('[{"protocol": "tcp", "ports": ""}]')
self.assertTrue(any('tcp/' in msg for msg in ctx.exception.messages))
def test_ports_without_protocol_reports_clear_error(self):
"""Ports entered with no protocol (e.g. the blank initial row) report a clear protocol error."""
field = PortMappingField()
with self.assertRaises(ValidationError) as ctx:
field.clean('[{"protocol": "", "ports": "80"}]')
self.assertTrue(any('protocol' in msg.lower() for msg in ctx.exception.messages))
# Specifically not the confusing blank "Invalid protocol:" message.
self.assertFalse(any(msg.strip().endswith('Invalid protocol:') for msg in ctx.exception.messages))
def test_row_errors_identify_the_row(self):
"""
A per-row error names the offending row, since the widget renders one row per protocol and an
unqualified message gives no clue which of several rows to fix.
"""
field = PortMappingField()
# A row with ports but no protocol
rows = '[{"protocol": "tcp", "ports": "80"}, {"protocol": "", "ports": "53"}]'
with self.assertRaises(ValidationError) as ctx:
field.clean(rows)
self.assertTrue(
any(msg.startswith('Row 2:') for msg in ctx.exception.messages), ctx.exception.messages
)
# A row whose port range is invalid
rows = '[{"protocol": "tcp", "ports": "80"}, {"protocol": "udp", "ports": "9000-53"}]'
with self.assertRaises(ValidationError) as ctx:
field.clean(rows)
self.assertTrue(
any(msg.startswith('Row 2:') for msg in ctx.exception.messages), ctx.exception.messages
)
def test_whole_field_errors_are_not_row_attributed(self):
"""
Errors raised by validate_port_mappings() are left unqualified: each already quotes the offending
mapping, and a duplicate spans two rows so attributing it to one would be misleading.
"""
field = PortMappingField()
with self.assertRaises(ValidationError) as ctx:
field.clean('[{"protocol": "tcp", "ports": "80"}, {"protocol": "udp", "ports": ""}]')
self.assertFalse(any(msg.startswith('Row ') for msg in ctx.exception.messages))
self.assertTrue(any('udp/' in msg for msg in ctx.exception.messages), ctx.exception.messages)
def test_reversed_range_rejected(self):
"""A reversed range must raise rather than silently expanding to an empty (dropped) list."""
field = PortMappingField()
with self.assertRaises(ValidationError):
field.clean('[{"protocol": "tcp", "ports": "9000-53"}]')
def test_invalid_subrange_alongside_valid_rejected(self):
"""
An invalid range combined with a valid one must raise rather than silently dropping the
invalid sub-range (the valid range would otherwise mask the empty expansion).
"""
field = PortMappingField()
with self.assertRaises(ValidationError):
field.clean('[{"protocol": "tcp", "ports": "80,9000-53"}]')
with self.assertRaises(ValidationError):
field.clean('[{"protocol": "tcp", "ports": "80,70000-80"}]')
def test_normalizes_leading_zero_ports(self):
"""Leading-zero ports are normalized so they remain matchable by the port filter."""
field = PortMappingField()
self.assertEqual(field.clean('[{"protocol": "tcp", "ports": "080"}]'), ['tcp/80'])
def test_prepare_value_grouped_json_passthrough(self):
"""An already-grouped JSON string (bound-form re-render) is passed to the widget unchanged."""
field = PortMappingField()
self.assertEqual(
field.prepare_value('[{"protocol": "tcp", "ports": "80"}]'),
'[{"protocol": "tcp", "ports": "80"}]',
)
def test_prepare_value_flat_list_grouped(self):
"""A flat protocol/port list (e.g. a multi-mapping clone) is grouped into widget rows."""
field = PortMappingField()
self.assertEqual(
field.prepare_value(['tcp/80', 'tcp/443']),
'[{"protocol": "tcp", "ports": "80,443"}]',
)
def test_prepare_value_bare_string_grouped(self):
"""
Cloning a single-mapping object collapses port_mappings to a bare 'protocol/port' string
(normalize_querydict single-value collapse); it must group into a row, not blank the widget.
Regression guard for the single-protocol clone losing its port mapping.
"""
field = PortMappingField()
self.assertEqual(
field.prepare_value('tcp/80'),
'[{"protocol": "tcp", "ports": "80"}]',
)
class ServiceTemplateImportFormTestCase(TestCase):
def test_valid_port_mappings_parsed_and_normalized(self):
form = ServiceTemplateImportForm(data={'name': 'X', 'port_mappings': 'tcp/080,tcp/443,udp/53'})
self.assertTrue(form.is_valid(), form.errors)
self.assertEqual(form.cleaned_data['port_mappings'], ['tcp/80', 'tcp/443', 'udp/53'])
def test_protocol_lowercased(self):
"""Protocols may be given in any case; the input is lowercased before validation."""
form = ServiceTemplateImportForm(data={'name': 'X', 'port_mappings': 'TCP/80,UDP/53'})
self.assertTrue(form.is_valid(), form.errors)
self.assertEqual(form.cleaned_data['port_mappings'], ['tcp/80', 'udp/53'])
def test_invalid_protocol_rejected(self):
form = ServiceTemplateImportForm(data={'name': 'X', 'port_mappings': 'tcp/80,bogus/53'})
self.assertFalse(form.is_valid())
self.assertIn('port_mappings', form.errors)
def test_duplicate_mapping_rejected(self):
form = ServiceTemplateImportForm(data={'name': 'X', 'port_mappings': 'tcp/80,tcp/080'})
self.assertFalse(form.is_valid())
self.assertIn('port_mappings', form.errors)
def test_empty_port_rejected(self):
form = ServiceTemplateImportForm(data={'name': 'X', 'port_mappings': 'tcp/'})
self.assertFalse(form.is_valid())
self.assertIn('port_mappings', form.errors)
class ServiceFilterFormTestCase(TestCase):
"""
`port_mappings` matches a complete protocol/port pair, which the correlated `protocol`/`port` pair
cannot express on its own, so it must be reachable from the UI and not only from the API.
"""
forms_and_filtersets = (
(ServiceTemplateFilterForm, ServiceTemplateFilterSet),
(ServiceFilterForm, ServiceFilterSet),
)
def test_port_mappings_field_present(self):
# ServiceFilterForm inherits the field from ServiceTemplateFilterForm but redeclares fieldsets,
# so both must be checked.
for form_class, filterset_class in self.forms_and_filtersets:
with self.subTest(form=form_class.__name__):
fieldset_items = [item for fieldset in form_class.fieldsets for item in fieldset.items]
self.assertIn('port_mappings', fieldset_items)
self.assertIn('port_mappings', form_class().fields)
# The form field's name must match the filter's, or the rendered query does nothing
self.assertIn('port_mappings', filterset_class.get_filters())
# Render the form to confirm the fieldset entry resolves to a real field
template = Template('{% load form_helpers %}{% render_form form %}')
html = template.render(Context({'form': form_class()}))
self.assertIn('id_port_mappings', html)
def test_port_mappings_value_cleans(self):
for form_class, _ in self.forms_and_filtersets:
with self.subTest(form=form_class.__name__):
form = form_class(data={'port_mappings': 'tcp/80'})
self.assertTrue(form.is_valid(), form.errors)
self.assertEqual(form.cleaned_data['port_mappings'], 'tcp/80')

View File

@ -9,7 +9,7 @@ from dcim.models import Location, Region, Site, SiteGroup
from ipam.choices import *
from ipam.constants import SERVICE_PORT_MAX, SERVICE_PORT_MIN
from ipam.models import *
from ipam.utils import rebuild_prefixes
from ipam.utils import port_mapping_q, rebuild_prefixes
from utilities.data import string_to_ranges
from virtualization.models import VirtualMachine
@ -1931,42 +1931,92 @@ class PrefixGetChildIPsTestCase(TestCase):
class ServiceTemplateTestCase(TestCase):
def test_servicetemplate_lowest_port(self):
def test_multiple_protocols_same_port(self):
"""
Test lowest port setting for servicetemplate
A template may expose the same port on multiple protocols (e.g. DNS on tcp/53 and udp/53).
"""
template = ServiceTemplate(
name='Template 1',
protocol=ServiceProtocolChoices.PROTOCOL_TCP,
ports=[80, 443, 22, 8080], # small test list
)
template = ServiceTemplate(name='DNS', port_mappings=['tcp/53', 'udp/53'])
template.full_clean()
template.save()
self.assertEqual(template._ports_lowest, 22)
self.assertEqual(template.port_mappings, ['tcp/53', 'udp/53'])
def test_servicetemplate_single_port(self):
"""
Test with a single port
"""
template = ServiceTemplate(
name='Template 2',
protocol=ServiceProtocolChoices.PROTOCOL_UDP,
ports=[53],
)
template.full_clean()
template.save()
self.assertEqual(template._ports_lowest, 53)
def test_duplicate_mapping_not_allowed(self):
template = ServiceTemplate(name='Duplicate', port_mappings=['tcp/80', 'tcp/80'])
with self.assertRaises(ValidationError):
template.full_clean()
def test_servicetemplate_empty_ports(self):
"""
Test with empty ports list
"""
template = ServiceTemplate(
name='Template 3',
protocol=ServiceProtocolChoices.PROTOCOL_TCP,
ports=[],
def test_invalid_protocol(self):
template = ServiceTemplate(name='Bad Protocol', port_mappings=['bogus/80'])
with self.assertRaises(ValidationError):
template.full_clean()
def test_port_out_of_range(self):
template = ServiceTemplate(name='Out Of Range', port_mappings=[f'tcp/{SERVICE_PORT_MAX + 1}'])
with self.assertRaises(ValidationError):
template.full_clean()
def test_empty_port_mappings(self):
# A service (template) must define at least one port mapping
template = ServiceTemplate(name='Empty', port_mappings=[])
with self.assertRaises(ValidationError):
template.full_clean()
def test_duplicate_normalized_port(self):
# tcp/80 and tcp/080 are the same mapping and must be rejected as a duplicate
template = ServiceTemplate(name='DupNorm', port_mappings=['tcp/80', 'tcp/080'])
with self.assertRaises(ValidationError):
template.full_clean()
def _matching_names(self, protocols=(), port_tests=()):
return set(
ServiceTemplate.objects.filter(port_mapping_q(protocols, port_tests))
.values_list('name', flat=True)
)
self.assertRaises(ValidationError, template.full_clean)
def test_protocol_query_reads_port_mappings_directly(self):
# Protocol filtering is derived from port_mappings on every query rather than from a
# denormalized column, so it is correct for any write path -- including ones that bypass
# Model.save() such as bulk_create() and queryset.update().
ServiceTemplate.objects.create(name='Saved', port_mappings=['tcp/80', 'tcp/443'])
ServiceTemplate.objects.bulk_create([
ServiceTemplate(name='Bulk', port_mappings=['udp/53', 'tcp/53']),
])
updated = ServiceTemplate.objects.create(name='Updated', port_mappings=['tcp/53', 'udp/53'])
ServiceTemplate.objects.filter(pk=updated.pk).update(port_mappings=['udp/53'])
self.assertEqual(self._matching_names(protocols=['tcp']), {'Saved', 'Bulk'})
self.assertEqual(self._matching_names(protocols=['udp']), {'Bulk', 'Updated'})
def test_port_range_query_is_correlated_with_protocol(self):
# A range lookup must be satisfied by the same mapping as the protocol, so a template whose only
# tcp mapping is tcp/80 must not match protocol=tcp with port > 1000.
ServiceTemplate.objects.create(name='Low TCP', port_mappings=['tcp/80', 'udp/9999'])
ServiceTemplate.objects.create(name='High TCP', port_mappings=['tcp/8080'])
self.assertEqual(self._matching_names(port_tests=[('gt', [1000])]), {'Low TCP', 'High TCP'})
self.assertEqual(
self._matching_names(protocols=['tcp'], port_tests=[('gt', [1000])]), {'High TCP'}
)
def test_port_range_bounds_must_hold_for_one_mapping(self):
# gte + lte together describe a single port range, so a template exposing only ports outside it
# must not match by satisfying each bound with a different mapping.
ServiceTemplate.objects.create(name='Straddling', port_mappings=['tcp/500', 'tcp/5000'])
ServiceTemplate.objects.create(name='Inside', port_mappings=['tcp/1500'])
self.assertEqual(
self._matching_names(port_tests=[('gte', [1000]), ('lte', [2000])]), {'Inside'}
)
def test_port_query_tolerates_malformed_mapping(self):
# A mapping written outside the ORM (raw SQL, a plugin) whose port isn't numeric must not abort
# the query; it simply never matches a port comparison.
template = ServiceTemplate.objects.create(name='Malformed', port_mappings=['tcp/80'])
ServiceTemplate.objects.filter(pk=template.pk).update(port_mappings=['tcp/nope', 'udp/53'])
self.assertEqual(self._matching_names(port_tests=[('gt', [1])]), {'Malformed'})
self.assertEqual(self._matching_names(protocols=['tcp'], port_tests=[('gt', [1])]), set())
self.assertEqual(self._matching_names(protocols=['tcp']), {'Malformed'})
class ServiceTestCase(TestCase):
@ -1984,16 +2034,45 @@ class ServiceTestCase(TestCase):
def test_large_service(self):
"""
Test creation of service with large number of ports.
Test creation of a service with a large number of port mappings.
Related to issue #22273
"""
service = Service(
name='Service 1',
protocol=ServiceProtocolChoices.PROTOCOL_TCP,
ports=list(range(SERVICE_PORT_MIN, SERVICE_PORT_MAX)),
parent=VirtualMachine.objects.first(),
port_mappings=[f'tcp/{port}' for port in range(SERVICE_PORT_MIN, SERVICE_PORT_MAX)],
)
service.full_clean()
# Testing .save() is the important part, to check for database problems
service.save()
self.assertEqual(service._ports_lowest, SERVICE_PORT_MIN)
self.assertEqual(len(service.port_mappings), SERVICE_PORT_MAX - SERVICE_PORT_MIN)
def test_port_mappings_list_summary(self):
"""
The port_mappings_list property renders each protocol/port pair individually for display.
"""
service = Service.objects.create(
name='dns',
parent=VirtualMachine.objects.first(),
port_mappings=['tcp/53', 'udp/53'],
)
self.assertEqual(service.port_mappings_list, 'TCP/53, UDP/53')
def test_legacy_protocol_ports_properties(self):
"""The read-only protocol/ports properties expose the deprecated single-protocol representation."""
vm = VirtualMachine.objects.first()
# Single protocol: reported as (protocol, sorted ports)
single = Service.objects.create(name='http', parent=vm, port_mappings=['tcp/443', 'tcp/80'])
self.assertEqual(single.protocol, 'tcp')
self.assertEqual(single.ports, [80, 443])
# Multiple protocols: not representable in the legacy format, so both are None
multi = Service.objects.create(name='dns', parent=vm, port_mappings=['tcp/53', 'udp/53'])
self.assertIsNone(multi.protocol)
self.assertIsNone(multi.ports)
# No mappings: protocol is None but ports is an empty list (as the old API always reported)
empty = Service.objects.create(name='empty', parent=vm, port_mappings=[])
self.assertIsNone(empty.protocol)
self.assertEqual(empty.ports, [])

View File

@ -2,7 +2,7 @@ import datetime
from django.contrib.contenttypes.models import ContentType
from django.db.backends.postgresql.psycopg_any import NumericRange
from django.test import RequestFactory
from django.test import RequestFactory, override_settings
from django.urls import reverse
from netaddr import IPNetwork
@ -17,7 +17,7 @@ from ipam.views import AggregatePrefixesView
from netbox.choices import CSVDelimiterChoices, ImportFormatChoices
from tenancy.models import Tenant
from users.models import ObjectPermission
from utilities.testing import ViewTestCases, create_tags
from utilities.testing import ViewTestCases, create_tags, post_data
class ASNRangeTestCase(ViewTestCases.PrimaryObjectViewTestCase):
@ -1845,13 +1845,16 @@ class VLANTranslationRuleTestCase(ViewTestCases.PrimaryObjectViewTestCase):
class ServiceTemplateTestCase(ViewTestCases.PrimaryObjectViewTestCase):
model = ServiceTemplate
# port_mappings is an ArrayField, but the form submits it as a JSON string rather than a list, so
# the value isn't directly comparable to the stored field during the view test's edit assertions
validation_excluded_fields = ('port_mappings',)
@classmethod
def setUpTestData(cls):
service_templates = (
ServiceTemplate(name='Service Template 1', protocol=ServiceProtocolChoices.PROTOCOL_TCP, ports=[101]),
ServiceTemplate(name='Service Template 2', protocol=ServiceProtocolChoices.PROTOCOL_TCP, ports=[102]),
ServiceTemplate(name='Service Template 3', protocol=ServiceProtocolChoices.PROTOCOL_TCP, ports=[103]),
ServiceTemplate(name='Service Template 1', port_mappings=['tcp/101']),
ServiceTemplate(name='Service Template 2', port_mappings=['tcp/102']),
ServiceTemplate(name='Service Template 3', port_mappings=['tcp/103']),
)
ServiceTemplate.objects.bulk_create(service_templates)
@ -1859,17 +1862,16 @@ class ServiceTemplateTestCase(ViewTestCases.PrimaryObjectViewTestCase):
cls.form_data = {
'name': 'Service Template X',
'protocol': ServiceProtocolChoices.PROTOCOL_UDP,
'ports': '104,105',
'port_mappings': '[{"protocol": "udp", "ports": "104,105"}]',
'description': 'A new service template',
'tags': [t.pk for t in tags],
}
cls.csv_data = (
"name,protocol,ports,description",
"Service Template 4,tcp,1,First service template",
"Service Template 5,tcp,2,Second service template",
"Service Template 6,tcp,3,Third service template",
"name,port_mappings,description",
"Service Template 4,tcp/1,First service template",
"Service Template 5,tcp/2,Second service template",
'Service Template 6,"udp/3,tcp/4",Third service template',
)
cls.csv_update_data = (
@ -1880,16 +1882,75 @@ class ServiceTemplateTestCase(ViewTestCases.PrimaryObjectViewTestCase):
)
cls.bulk_edit_data = {
'protocol': ServiceProtocolChoices.PROTOCOL_UDP,
'ports': '106,107',
'description': 'New description',
}
def test_port_mappings_stored_from_form(self):
# port_mappings is in validation_excluded_fields (the form submits JSON rows, not a list), so the
# standard create/edit view tests can't compare it. Assert the form round-trip explicitly.
self.add_permissions('ipam.add_servicetemplate', 'ipam.change_servicetemplate')
# Create: a row's comma-separated ports expand into individual mappings
data = {
'name': 'Mappings Via Form',
'port_mappings': '[{"protocol": "udp", "ports": "104,105"}]',
}
self.assertHttpStatus(self.client.post(self._get_url('add'), data=post_data(data)), 302)
template = ServiceTemplate.objects.get(name='Mappings Via Form')
self.assertEqual(template.port_mappings, ['udp/104', 'udp/105'])
# Edit: multiple rows, a range, and a non-canonical port are expanded and normalized
data['port_mappings'] = '[{"protocol": "tcp", "ports": "080,8000-8002"}, {"protocol": "udp", "ports": "53"}]'
self.assertHttpStatus(
self.client.post(self._get_url('edit', template), data=post_data(data)), 302
)
template.refresh_from_db()
self.assertEqual(template.port_mappings, ['tcp/80', 'tcp/8000', 'tcp/8001', 'tcp/8002', 'udp/53'])
def test_bulk_edit_port_mappings(self):
# Bulk add/remove port mappings across selected templates (tags-style). ServiceTemplate is where
# the add/remove fields are declared (ServiceBulkEditForm inherits them), so cover it directly.
self.add_permissions('ipam.view_servicetemplate', 'ipam.change_servicetemplate')
templates = list(
ServiceTemplate.objects.filter(name__in=['Service Template 1', 'Service Template 2']).order_by('name')
)
data = {
'pk': [t.pk for t in templates],
'add_port_mappings': '[{"protocol": "udp", "ports": "53"}]',
'remove_port_mappings': '[{"protocol": "tcp", "ports": "101"}]',
'_apply': '',
}
response = self.client.post(self._get_url('bulk_edit'), data)
self.assertHttpStatus(response, 302)
# Service Template 1 (was tcp/101): tcp/101 removed, udp/53 added
self.assertEqual(ServiceTemplate.objects.get(pk=templates[0].pk).port_mappings, ['udp/53'])
# Service Template 2 (was tcp/102): remove is a no-op, udp/53 added
self.assertEqual(
ServiceTemplate.objects.get(pk=templates[1].pk).port_mappings, ['tcp/102', 'udp/53']
)
def test_bulk_edit_removing_all_mappings_is_rejected(self):
# Emptying an object's mappings must fail validation rather than persist an invalid object.
self.add_permissions('ipam.view_servicetemplate', 'ipam.change_servicetemplate')
template = ServiceTemplate.objects.get(name='Service Template 1')
data = {
'pk': [template.pk],
'remove_port_mappings': '[{"protocol": "tcp", "ports": "101"}]',
'_apply': '',
}
response = self.client.post(self._get_url('bulk_edit'), data)
self.assertHttpStatus(response, 200) # Re-rendered with the error, not a redirect
template.refresh_from_db()
self.assertEqual(template.port_mappings, ['tcp/101'])
class ServiceTestCase(ViewTestCases.PrimaryObjectViewTestCase):
model = Service
# TODO, related to #9816, cannot validate GFK
validation_excluded_fields = ('device',)
# port_mappings is an ArrayField, but the form submits it as a JSON string rather than a list, so
# the value isn't directly comparable to the stored field during the view test's edit assertions
validation_excluded_fields = ('device', 'port_mappings')
@classmethod
def setUpTestData(cls):
@ -1905,9 +1966,9 @@ class ServiceTestCase(ViewTestCases.PrimaryObjectViewTestCase):
)
services = (
Service(parent=device, name='Service 1', protocol=ServiceProtocolChoices.PROTOCOL_TCP, ports=[101]),
Service(parent=device, name='Service 2', protocol=ServiceProtocolChoices.PROTOCOL_TCP, ports=[102]),
Service(parent=device, name='Service 3', protocol=ServiceProtocolChoices.PROTOCOL_TCP, ports=[103]),
Service(parent=device, name='Service 1', port_mappings=['tcp/101']),
Service(parent=device, name='Service 2', port_mappings=['tcp/102']),
Service(parent=device, name='Service 3', port_mappings=['tcp/103']),
)
Service.objects.bulk_create(services)
@ -1924,19 +1985,18 @@ class ServiceTestCase(ViewTestCases.PrimaryObjectViewTestCase):
'parent_content_type': ContentType.objects.get_for_model(Device).pk,
'parent_object_id': device.pk,
'name': 'Service X',
'protocol': ServiceProtocolChoices.PROTOCOL_TCP,
'ports': '104,105',
'port_mappings': '[{"protocol": "tcp", "ports": "104,105"}, {"protocol": "udp", "ports": "104"}]',
'ipaddresses': [],
'description': 'A new service',
'tags': [t.pk for t in tags],
}
cls.csv_data = (
"parent_object_type,parent,name,protocol,ports,ipaddresses,description",
"dcim.device,Device 1,Service 1,tcp,1,192.0.2.1/24,First service",
"dcim.device,Device 1,Service 2,tcp,2,192.0.2.2/24,Second service",
"dcim.device,Device 1,Service 3,udp,3,,Third service",
"ipam.fhrpgroup,Group 1,Service 4,udp,4,192.0.2.3/24,Fourth service",
"parent_object_type,parent,name,port_mappings,ipaddresses,description",
"dcim.device,Device 1,Service 1,tcp/1,192.0.2.1/24,First service",
"dcim.device,Device 1,Service 2,tcp/2,192.0.2.2/24,Second service",
"dcim.device,Device 1,Service 3,udp/3,,Third service",
'ipam.fhrpgroup,Group 1,Service 4,"tcp/4,udp/4",192.0.2.3/24,Fourth service',
)
cls.csv_update_data = (
@ -1947,8 +2007,6 @@ class ServiceTestCase(ViewTestCases.PrimaryObjectViewTestCase):
)
cls.bulk_edit_data = {
'protocol': ServiceProtocolChoices.PROTOCOL_UDP,
'ports': '106,107',
'description': 'New description',
}
@ -1957,8 +2015,8 @@ class ServiceTestCase(ViewTestCases.PrimaryObjectViewTestCase):
device = Device.objects.first()
addr = IPAddress.objects.create(address='192.0.2.4/24')
csv_data = (
"parent_object_type,parent_object_id,name,protocol,ports,ipaddresses,description",
f"dcim.device,{device.pk},Service 11,tcp,10,{addr.address},Eleventh service",
"parent_object_type,parent_object_id,name,port_mappings,ipaddresses,description",
f"dcim.device,{device.pk},Service 11,tcp/10,{addr.address},Eleventh service",
)
initial_count = self._get_queryset().count()
@ -1988,8 +2046,8 @@ class ServiceTestCase(ViewTestCases.PrimaryObjectViewTestCase):
interface = device.interfaces.first()
addr = IPAddress.objects.create(assigned_object=interface, address='192.0.2.3/24')
csv_data = (
"parent_object_type,parent_object_id,name,protocol,ports,ipaddresses,description",
f"dcim.device,{device.pk},Service 11,tcp,10,{addr.address},Eleventh service",
"parent_object_type,parent_object_id,name,port_mappings,ipaddresses,description",
f"dcim.device,{device.pk},Service 11,tcp/10,{addr.address},Eleventh service",
)
initial_count = self._get_queryset().count()
@ -2023,8 +2081,7 @@ class ServiceTestCase(ViewTestCases.PrimaryObjectViewTestCase):
device = Device.objects.first()
service_template = ServiceTemplate.objects.create(
name='HTTP',
protocol=ServiceProtocolChoices.PROTOCOL_TCP,
ports=[80],
port_mappings=['tcp/80'],
description='Hypertext transfer protocol'
)
@ -2041,6 +2098,48 @@ class ServiceTestCase(ViewTestCases.PrimaryObjectViewTestCase):
instance = self._get_queryset().order_by('pk').last()
self.assertEqual(instance.parent, device)
self.assertEqual(instance.name, service_template.name)
self.assertEqual(instance.protocol, service_template.protocol)
self.assertEqual(instance.ports, service_template.ports)
self.assertEqual(instance.description, service_template.description)
# Port mappings should be copied from the template
self.assertEqual(instance.port_mappings, ['tcp/80'])
# EXEMPT_VIEW_PERMISSIONS matches the standard create/edit view tests: without it the form's related
# object fields (parent, tags) reject choices the test user has no view permission for.
@override_settings(EXEMPT_VIEW_PERMISSIONS=['*'], EXEMPT_EXCLUDE_MODELS=[])
def test_port_mappings_stored_from_form(self):
# port_mappings is in validation_excluded_fields (the form submits JSON rows, not a list), so the
# standard create/edit view tests can't compare it. Assert the form round-trip explicitly, using a
# multi-protocol payload — the whole point of the field — including the same port on two protocols.
self.add_permissions('ipam.add_service', 'ipam.change_service')
data = {**self.form_data, 'name': 'Mappings Via Form'}
self.assertHttpStatus(self.client.post(self._get_url('add'), data=post_data(data)), 302)
service = Service.objects.get(name='Mappings Via Form')
self.assertEqual(service.port_mappings, ['tcp/104', 'tcp/105', 'udp/104'])
# Edit: a range and a non-canonical port are expanded and normalized
data['port_mappings'] = '[{"protocol": "tcp", "ports": "080,8000-8002"}]'
self.assertHttpStatus(
self.client.post(self._get_url('edit', service), data=post_data(data)), 302
)
service.refresh_from_db()
self.assertEqual(service.port_mappings, ['tcp/80', 'tcp/8000', 'tcp/8001', 'tcp/8002'])
def test_bulk_edit_port_mappings(self):
# Bulk add/remove port mappings across selected services (tags-style)
self.add_permissions('ipam.view_service', 'ipam.change_service')
services = list(Service.objects.filter(name__in=['Service 1', 'Service 2']).order_by('name'))
data = {
'pk': [s.pk for s in services],
'add_port_mappings': '[{"protocol": "udp", "ports": "53"}]',
'remove_port_mappings': '[{"protocol": "tcp", "ports": "101"}]',
'_apply': '',
}
response = self.client.post(self._get_url('bulk_edit'), data)
self.assertHttpStatus(response, 302)
# Service 1 (was tcp/101): tcp/101 removed, udp/53 added
service1 = Service.objects.get(pk=services[0].pk)
self.assertEqual(service1.port_mappings, ['udp/53'])
# Service 2 (was tcp/102): remove is a no-op, udp/53 added
service2 = Service.objects.get(pk=services[1].pk)
self.assertEqual(service2.port_mappings, ['tcp/102', 'udp/53'])

View File

@ -236,16 +236,14 @@ class VLANCustomerVLANsPanel(panels.ObjectsTablePanel):
class ServiceTemplatePanel(panels.ObjectAttributesPanel):
name = attrs.TextAttr('name')
protocol = attrs.ChoiceAttr('protocol')
ports = attrs.TextAttr('port_list', label=_('Ports'))
port_mappings = attrs.TextAttr('port_mappings_list', label=_('Port Mappings'))
description = attrs.TextAttr('description')
class ServicePanel(panels.ObjectAttributesPanel):
name = attrs.TextAttr('name')
parent = attrs.RelatedObjectAttr('parent', linkify=True)
protocol = attrs.ChoiceAttr('protocol')
ports = attrs.TextAttr('port_list', label=_('Ports'))
port_mappings = attrs.TextAttr('port_mappings_list', label=_('Port Mappings'))
ip_addresses = attrs.TemplatedAttr(
'ipaddresses',
template_name='ipam/service/attrs/ip_addresses.html',

View File

@ -2,17 +2,29 @@ from dataclasses import dataclass
import netaddr
from django.apps import apps
from django.core.exceptions import ValidationError
from django.db.models import BooleanField, F, Func, Q
from django.utils.translation import gettext_lazy as _
from .constants import *
__all__ = (
'PORT_MAPPING_LOOKUPS',
'AvailableIPSpace',
'PortMappingMatch',
'add_available_vlans',
'add_requested_prefixes',
'annotate_ip_space',
'expand_port_mapping',
'get_next_available_prefix',
'group_port_mapping_rows',
'group_port_mappings',
'legacy_protocol_and_ports',
'normalize_port_mapping',
'port_mapping_q',
'rebuild_prefixes',
'sorted_int_ports',
'split_port_mapping',
)
@ -259,3 +271,265 @@ def get_next_available_prefix(ipset, prefix_size):
ipset.remove(allocated_prefix)
return allocated_prefix
return None
#
# Service port mappings
#
def split_port_mapping(mapping):
"""
Split a ``protocol/port`` string (e.g. ``'tcp/80'``) into its ``(protocol, port)`` parts. A missing
separator or port yields an empty string for that part, leaving validation to report the problem.
"""
protocol, _sep, port = mapping.partition('/')
return protocol, port
def normalize_port_mapping(mapping):
"""
Canonicalize a single ``protocol/port`` string as far as possible *without raising*: the protocol is
lowercased and a numeric port loses any leading zeros, so ``'TCP/080'`` becomes ``'tcp/80'``. Anything
unrecognized is returned unchanged, in which case it simply won't match a stored (always-canonical)
mapping.
This is the lookup-side counterpart to ``validate_port_mappings()``, which enforces the same
canonical form on write but rejects bad input. Filtering must not 400 on an unknown protocol or a
malformed pair an empty result set is the right answer there hence the separate, lenient variant.
"""
# Imported lazily to avoid a circular import during settings load (ipam.choices reads
# settings.FIELD_CHOICES), matching validate_port_mappings().
from ipam.choices import ServiceProtocolChoices
protocol, port = split_port_mapping(mapping)
if not port or not port.isdigit():
return mapping
protocol = protocol.lower()
if protocol not in ServiceProtocolChoices.values():
return mapping
return f'{protocol}/{int(port)}'
def group_port_mappings(mappings):
"""
Group a flat ``['tcp/80', 'tcp/443', 'udp/53']`` list into an ordered ``{protocol: [ports]}`` dict,
preserving first-seen protocol order. Shared by the display property and the form widget so the
``protocol/port`` string is parsed in exactly one place.
"""
grouped = {}
for mapping in mappings:
protocol, port = split_port_mapping(mapping)
grouped.setdefault(protocol, []).append(port)
return grouped
def group_port_mapping_rows(mappings):
"""
Group a flat ``['tcp/80', 'tcp/443', 'udp/53']`` list into per-protocol rows
``[{'protocol': 'tcp', 'ports': '80,443'}, {'protocol': 'udp', 'ports': '53'}]`` the shape the
port-mapping form widget renders, one row per protocol.
"""
return [
{'protocol': protocol, 'ports': ','.join(ports)}
for protocol, ports in group_port_mappings(mappings).items()
]
def sorted_int_ports(ports):
"""
Sort a protocol's port strings numerically and return them as integers. Any entry that bypassed
validation (a raw SQL write, a plugin, or an unmigrated row) and isn't a plain integer is skipped
rather than raising, so a single malformed mapping degrades gracefully on API reads instead of
raising a 500 mirroring the tolerance of ``ServiceBase.port_mappings_list``.
"""
return sorted(int(port) for port in ports if str(port).isdigit())
def legacy_protocol_and_ports(mappings):
"""
Collapse port mappings into the deprecated single-protocol ``(protocol, ports)`` representation.
Single source of truth for the backward-compatibility contract shared by the REST serializers and
the GraphQL types:
* single protocol -> ``(protocol, [sorted int ports])``
* no mappings -> ``(None, [])`` (representable as an empty legacy ports list)
* multiple protocols -> ``(None, None)`` (not representable; ``ports=None`` signals "read
port_mappings instead")
* single protocol, but a port fails integer coercion (malformed raw/plugin data) -> ``(None, None)``
(a subset would be plausible-but-wrong, so signal "not representable" rather than silently
dropping the bad mapping)
"""
grouped = group_port_mappings(mappings)
if len(grouped) == 1:
protocol, ports = next(iter(grouped.items()))
int_ports = sorted_int_ports(ports)
# If any port was dropped by coercion, the legacy single-protocol view can't faithfully
# represent this service; signal "not representable" instead of returning a partial list.
if len(int_ports) != len(ports):
return None, None
return protocol, int_ports
return (None, []) if not grouped else (None, None)
# Whitelisted SQL comparison operators for the port half of a mapping, keyed by the django-filter
# lookup name. Only these five names are ever interpolated into SQL by PortMappingMatch, so the
# operator can never originate from user input.
PORT_MAPPING_LOOKUPS = {
'exact': '=',
'gt': '>',
'gte': '>=',
'lt': '<',
'lte': '<=',
}
# The port half of an unnested mapping, as an integer. Guarded by a numeric test so a malformed mapping
# written outside the ORM (raw SQL, a plugin) evaluates to NULL — which no comparison matches — instead
# of aborting the whole query with an invalid-input-syntax error. Mirrors the tolerance that
# sorted_int_ports() and ServiceBase.port_mappings_list already apply on reads.
_PORT_MAPPING_PORT_SQL = (
"CASE WHEN split_part(port_mapping, '/', 2) ~ '^[0-9]+$' "
"THEN split_part(port_mapping, '/', 2)::integer END"
)
class PortMappingMatch(Func):
"""
A boolean expression which is true for services having at least one port mapping that satisfies the
given protocol and port tests:
EXISTS (
SELECT 1 FROM unnest(port_mappings) AS port_mapping
WHERE split_part(port_mapping, '/', 1) = ANY(<protocols>)
AND <port> >= <value> AND <port> <= <value> ...
)
Testing every condition against the *same* unnested mapping is what keeps protocol and port
correlated: a service exposing tcp/80 and udp/9999 must not match ``protocol=tcp&port__gt=1000``,
and one exposing tcp/500 and tcp/5000 must not match ``port__gte=1000&port__lte=2000``.
This is deliberately a sequential scan. GIN's ``array_ops`` opclass supports only ``=``, ``&&``,
``@>`` and ``<@``, so no array index can serve a range comparison, and the alternatives (a
trigger-maintained denormalized column, or a related table) either cannot express the correlation or
cost far more than the scan measured at ~200 ms over 400k services and ~1 s over 2M.
``port_mapping_q()`` therefore reserves this for the cases an array overlap cannot express and uses
the GIN-indexable overlap for exact protocol+port lookups.
"""
output_field = BooleanField()
def __init__(self, protocols=(), port_tests=()):
"""
Args:
protocols: protocol values to match, OR'd together.
port_tests: ``(lookup, values)`` pairs, where ``lookup`` is a key of
``PORT_MAPPING_LOOKUPS``. Pairs are AND'd (and so must hold for one single mapping);
the values within a pair are OR'd, matching how django-filter's multi-value filters
combine ``?port=80&port=443``.
"""
self.protocols = list(protocols or ())
self.port_tests = [
(lookup, list(values)) for lookup, values in (port_tests or ()) if values
]
for lookup, _values in self.port_tests:
if lookup not in PORT_MAPPING_LOOKUPS:
raise ValueError(f"Unsupported port mapping lookup: {lookup}")
super().__init__(F('port_mappings'))
def as_sql(self, compiler, connection, **extra_context):
mappings_sql, mappings_params = compiler.compile(self.source_expressions[0])
conditions = []
params = list(mappings_params)
if self.protocols:
conditions.append("split_part(port_mapping, '/', 1) = ANY(%s)")
params.append(self.protocols)
for lookup, values in self.port_tests:
operator = PORT_MAPPING_LOOKUPS[lookup]
conditions.append('({})'.format(
' OR '.join(f'{_PORT_MAPPING_PORT_SQL} {operator} %s' for _value in values)
))
params.extend(values)
if not conditions:
# port_mapping_q() never builds an unconstrained match, but be explicit rather than emit an
# EXISTS with an empty WHERE clause.
return 'TRUE', []
sql = (
f"EXISTS (SELECT 1 FROM unnest({mappings_sql}) AS port_mapping "
f"WHERE {' AND '.join(conditions)})"
)
return sql, params
def port_mapping_q(protocols=(), port_tests=()):
"""
Build a ``Q`` filtering services by protocol and/or port, correlated so that a combined query must
be satisfied by a *single* mapping. See ``PortMappingMatch`` for the argument shapes.
A lone exact port test reduces to a GIN-indexable array overlap on ``port_mappings``
(``port_mappings && ['tcp/80', ...]`` each element is one whole mapping, so an overlap means
"shares any mapping"); for a port-only query each port is paired with every valid protocol to keep
it a single overlap. Everything else a protocol-only query, whose ports are unbounded and cannot
be enumerated, and any range lookup, which no array index can serve falls back to
``PortMappingMatch``. Shared by the FilterSet and the GraphQL filters.
"""
# Imported lazily to avoid a circular import during settings load (ipam.choices reads
# settings.FIELD_CHOICES), matching ipam.validators.
from ipam.choices import ServiceProtocolChoices
protocols = list(protocols or ())
port_tests = [(lookup, list(values)) for lookup, values in (port_tests or ()) if values]
if not protocols and not port_tests:
return Q()
if len(port_tests) == 1 and port_tests[0][0] == 'exact':
# Every stored mapping's protocol is validated against ServiceProtocolChoices, so enumerating
# the (small, fixed) protocol set covers all valid data for a port-only query.
ports = port_tests[0][1]
mapping_protocols = protocols or ServiceProtocolChoices.values()
combos = [f'{protocol}/{port}' for protocol in mapping_protocols for port in ports]
return Q(port_mappings__overlap=combos)
return Q(PortMappingMatch(protocols=protocols, port_tests=port_tests))
def expand_port_mapping(protocol, ports):
"""
Expand a single protocol plus its ports into the model's flat ``['tcp/80', 'tcp/443', ...]`` tokens.
``ports`` may be a comma/range string (the form widget's format, e.g. ``('tcp', '80,443,8000-8010')``)
or an already-expanded list of ports (e.g. set programmatically).
An empty ``ports`` yields a single bare ``'protocol/'`` token so ``validate_port_mappings`` reports a
clear "expected protocol/port" error (rather than ``parse_numeric_range`` raising a confusing
'Range "" is invalid'). An empty ``protocol`` raises a clear error rather than producing a ``'/80'``
token that surfaces as "Invalid protocol:" with a blank value. Shared by the model form field so the
protocol/port pairing is built in one place, and so every entry path gets the blank-protocol check.
"""
# Imported lazily to avoid pulling the forms layer in at module load.
from utilities.forms.utils import parse_numeric_range
# No case-folding here: validate_port_mappings (which every token below flows through) matches the
# protocol case-insensitively and stores the canonical value.
protocol = (protocol or '').strip()
if not protocol:
# A row with ports but no protocol (e.g. the initial blank row where the user typed a port but
# never picked a protocol) would otherwise expand to '/80' and surface as a confusing
# "Invalid protocol:" with a blank value. Report the real problem instead.
raise ValidationError(_("Select a protocol for each port mapping."))
if isinstance(ports, (list, tuple)):
# Already-expanded ports are paired as-is; validate_port_mappings() checks each value's range.
if not ports:
return [f'{protocol}/']
return [f'{protocol}/{port}' for port in ports]
ports_str = (ports or '').strip()
if not ports_str:
return [f'{protocol}/']
# parse_numeric_range validates each range against the port bounds (rejecting reversed and
# out-of-range values before expansion), so a non-empty string always yields >=1 port.
return [
f'{protocol}/{port}'
for port in parse_numeric_range(ports_str, min_value=SERVICE_PORT_MIN, max_value=SERVICE_PORT_MAX)
]

View File

@ -3,6 +3,62 @@ from django.core.validators import BaseValidator, RegexValidator
from django.utils.translation import gettext_lazy as _
def validate_port_mappings(mappings):
"""
Validate a list of service port mappings, i.e. ``protocol/port`` strings such as ``'tcp/80'``.
Ensures each entry is well-formed, uses a known protocol, falls within the permitted port range,
and is not duplicated. Raises a ``ValidationError`` describing the first problem found.
Returns the list in a canonical, normalized form (integer ports, so ``'tcp/080'`` becomes
``'tcp/80'``, and the protocol lowercased, so ``'TCP/80'`` becomes ``'tcp/80'``); callers should
persist the returned value so every entry path stores identical strings and remains matchable by the
port filters. Protocol matching is case-insensitive, so all paths (REST, CSV import, model form)
accept any case without each having to fold it first. Shared by the model (``ServiceBase.clean()``),
the model form field (``PortMappingField``), the CSV import form, and the REST API serializers so all
paths enforce identical rules.
"""
# Imported lazily to avoid a circular import during settings load (this module is imported by
# ipam.models, and ipam.constants pulls in ipam.choices, which reads settings.FIELD_CHOICES).
from ipam.choices import ServiceProtocolChoices
from ipam.constants import SERVICE_PORT_MAX, SERVICE_PORT_MIN
from ipam.utils import split_port_mapping
# A set, since this is consulted once per mapping and a service may define thousands
valid_protocols = set(ServiceProtocolChoices.values())
seen = set()
normalized_mappings = []
for mapping in mappings:
protocol, port = split_port_mapping(mapping)
if not port:
raise ValidationError(
_("Invalid port mapping '{mapping}'. Expected format protocol/port (e.g. tcp/80).").format(
mapping=mapping
)
)
# The error reports the protocol as supplied rather than the folded form.
if (canonical_protocol := protocol.lower()) not in valid_protocols:
raise ValidationError(_("Invalid protocol: {protocol}").format(protocol=protocol))
try:
port_number = int(port)
except ValueError:
raise ValidationError(_("Invalid port number: {port}").format(port=port))
if not SERVICE_PORT_MIN <= port_number <= SERVICE_PORT_MAX:
raise ValidationError(
_("Port {port} is not within the permitted range ({min}-{max}).").format(
port=port_number, min=SERVICE_PORT_MIN, max=SERVICE_PORT_MAX
)
)
# Normalize the port to an integer so e.g. tcp/80 and tcp/080 count as duplicates and are
# stored identically (leaving the raw string would make tcp/080 invisible to the port filter).
normalized = f'{canonical_protocol}/{port_number}'
if normalized in seen:
raise ValidationError(_("Duplicate port mapping: {mapping}").format(mapping=mapping))
seen.add(normalized)
normalized_mappings.append(normalized)
return normalized_mappings
def prefix_validator(prefix):
if prefix.ip != prefix.cidr.ip:
raise ValidationError(

View File

@ -1914,8 +1914,23 @@ class ServiceTemplateBulkImportView(generic.BulkImportView):
model_form = forms.ServiceTemplateImportForm
class ServicePortMappingsBulkEditMixin:
"""
Fold the ``add_port_mappings`` / ``remove_port_mappings`` bulk-edit deltas into each object before it
is validated and saved, keeping the model unaware of the bulk-edit form's fields.
"""
def pre_save_operations(self, form, obj):
super().pre_save_operations(form, obj)
add = form.cleaned_data.get('add_port_mappings')
remove = form.cleaned_data.get('remove_port_mappings')
if add:
obj._add_port_mappings(add)
if remove:
obj._remove_port_mappings(remove)
@register_model_view(ServiceTemplate, 'bulk_edit', path='edit', detail=False)
class ServiceTemplateBulkEditView(generic.BulkEditView):
class ServiceTemplateBulkEditView(ServicePortMappingsBulkEditMixin, generic.BulkEditView):
queryset = ServiceTemplate.objects.all()
filterset = filtersets.ServiceTemplateFilterSet
table = tables.ServiceTemplateTable
@ -1998,7 +2013,7 @@ class ServiceBulkImportView(generic.BulkImportView):
@register_model_view(Service, 'bulk_edit', path='edit', detail=False)
class ServiceBulkEditView(generic.BulkEditView):
class ServiceBulkEditView(ServicePortMappingsBulkEditMixin, generic.BulkEditView):
queryset = Service.objects.prefetch_related('parent')
filterset = filtersets.ServiceFilterSet
table = tables.ServiceTable

View File

@ -807,6 +807,15 @@ class BulkEditView(GetReturnURLMixin, BaseMultiObjectView):
def get_required_permission(self):
return get_permission_for_model(self.queryset.model, 'change')
def pre_save_operations(self, form, obj):
"""
This method is called for each object in _update_objects immediately before full_clean() and
save(). Override to modify the object from form fields that don't map directly to a model field
(e.g. add/remove-style deltas), so the change is validated and persisted within the single
bulk-edit save. No-op by default.
"""
pass
def post_save_operations(self, form, obj):
"""
This method is called for each object in _update_objects. Override to perform additional object-level
@ -884,6 +893,10 @@ class BulkEditView(GetReturnURLMixin, BaseMultiObjectView):
elif field.name in nullified_fields:
obj._m2m_values[field.name] = []
# Apply any form-driven modifications that don't map directly to a model field (e.g.
# add/remove deltas) before validation, so they're part of this single save.
self.pre_save_operations(form, obj)
obj.full_clean()
obj.save()
updated_objects.append(obj)

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@ -1,10 +1,17 @@
import { initClearField } from './clearField';
import { initFormElements } from './elements';
import { initFilterModifiers } from './filterModifiers';
import { initPortMappings } from './portMappings';
import { initSpeedSelector } from './speedSelector';
export function initForms(): void {
for (const func of [initFormElements, initSpeedSelector, initFilterModifiers, initClearField]) {
for (const func of [
initFormElements,
initSpeedSelector,
initFilterModifiers,
initClearField,
initPortMappings,
]) {
func();
}
}

View File

@ -0,0 +1,232 @@
import type TomSelect from 'tom-select';
import { NetBoxTomSelect } from '../select/classes/netboxTomSelect';
import { getPlugins } from '../select/config';
import { getElements } from '../util';
/**
* Return the TomSelect instance attached to a protocol <select>, if it has been initialized.
*/
function getProtocolSelect(select: HTMLSelectElement): TomSelect | undefined {
return (select as HTMLSelectElement & { tomselect?: TomSelect }).tomselect;
}
/**
* Apply NetBox's standard TomSelect styling to a protocol <select>. The widget manages these instances
* itself (rather than leaving them to the global static-select initializer) so it can keep TomSelect in
* sync as rows are added, removed, and their available protocols change.
*/
function initProtocolSelect(select: HTMLSelectElement, widget: HTMLElement): void {
if (getProtocolSelect(select)) return;
new NetBoxTomSelect(select, {
...getPlugins(select),
maxOptions: undefined,
// TomSelect emits its own (non-DOM) change event rather than a bubbling native one, so the widget's
// delegated 'change' listener never sees protocol selections. Refresh/serialize from here instead.
onChange: () => {
refreshProtocolOptions(widget);
serialize(widget);
},
});
}
/**
* Serialize the visible protocol/port rows of a widget into its hidden input as a JSON array of
* `{ protocol, ports }` objects. `ports` is kept as the raw comma/range string; the server expands it.
*/
function serialize(widget: HTMLElement): void {
const hidden = widget.querySelector<HTMLInputElement>('input[type="hidden"]');
if (hidden === null) return;
const rows: Array<{ protocol: string; ports: string }> = [];
for (const row of widget.querySelectorAll<HTMLElement>('[data-port-mapping-row]')) {
const protocol =
row.querySelector<HTMLSelectElement>('select.port-mapping-protocol')?.value ?? '';
const ports = row.querySelector<HTMLInputElement>('.port-mapping-ports')?.value.trim() ?? '';
if (protocol === '' && ports === '') continue;
rows.push({ protocol, ports });
}
hidden.value = JSON.stringify(rows);
}
/**
* Keep the field's `<label for="...">` target on whichever protocol `<select>` is currently first, so
* removing the first row doesn't leave the label pointing at an element that no longer exists. The id is
* derived the same way as the server-side widget's `id_for_label()`.
*/
function syncLabelTarget(widget: HTMLElement): void {
if (!widget.id) return;
const targetId = `${widget.id}_protocol_0`;
const selects = Array.from(
widget.querySelectorAll<HTMLSelectElement>('select.port-mapping-protocol'),
);
for (const select of selects) {
// Clear the id everywhere first so it is never present on two rows at once.
if (select.id === targetId) select.removeAttribute('id');
}
if (selects.length > 0) selects[0].id = targetId;
}
/**
* The set of protocol values currently selected across the widget's rows.
*/
function usedProtocols(widget: HTMLElement): Set<string> {
const selects = widget.querySelectorAll<HTMLSelectElement>('select.port-mapping-protocol');
return new Set(
Array.from(selects)
.map(select => select.value)
.filter(value => value !== ''),
);
}
/**
* The full list of protocol (value, label) choices, read from the widget's pristine `<template>` so it
* remains available even after options have been removed from the live selects.
*/
function protocolChoices(widget: HTMLElement): Array<{ value: string; text: string }> {
const template = widget.querySelector<HTMLTemplateElement>(
'template[data-port-mapping-template]',
);
const source = template?.content.querySelector<HTMLSelectElement>('select.port-mapping-protocol');
return Array.from(source?.options ?? [])
.filter(option => option.value !== '')
.map(option => ({ value: option.value, text: option.textContent?.trim() ?? option.value }));
}
/**
* Ensure each row offers only protocols not already chosen in another row (so each protocol can be
* selected at most once), and disable the "Add mapping" button when every protocol is in use.
*/
function refreshProtocolOptions(widget: HTMLElement): void {
const choices = protocolChoices(widget);
const selects = Array.from(
widget.querySelectorAll<HTMLSelectElement>('select.port-mapping-protocol'),
);
const used = new Set(selects.map(select => select.value).filter(value => value !== ''));
for (const select of selects) {
const current = select.value;
const ts = getProtocolSelect(select);
if (ts) {
// TomSelect renders its dropdown from its own option map, ignoring later changes to the native
// <option> disabled attribute — so add/remove options in that map to control what's offered.
// A protocol is offered only if it's free or already chosen in this row (never remove the row's
// own selection).
choices.forEach(({ value, text }, index) => {
const allowed = value === current || !used.has(value);
const exists = Object.prototype.hasOwnProperty.call(ts.options, value);
if (allowed && !exists) {
// Preserve the original ordering by mirroring the choice's index as TomSelect's $order.
ts.addOption({ value, text, $order: index + 1 });
} else if (!allowed && exists) {
ts.removeOption(value, true);
}
});
ts.refreshOptions(false);
} else {
// Fallback for a not-yet-enhanced select: toggle the native disabled attribute.
for (const option of Array.from(select.options)) {
if (option.value === '') continue;
option.disabled = used.has(option.value) && option.value !== current;
}
}
}
const addButton = widget.querySelector<HTMLButtonElement>('[data-port-mapping-add]');
if (addButton !== null) {
addButton.disabled = choices.length > 0 && used.size >= choices.length;
}
}
/**
* Add a new empty row by cloning the widget's `<template>`, defaulting it to the first protocol that
* is not already in use.
*/
function addRow(widget: HTMLElement): void {
const template = widget.querySelector<HTMLTemplateElement>(
'template[data-port-mapping-template]',
);
const body = widget.querySelector<HTMLElement>('[data-port-mapping-rows]');
if (template === null || body === null) return;
const used = usedProtocols(widget);
const fragment = template.content.cloneNode(true) as DocumentFragment;
body.appendChild(fragment);
// Default the new row to the first protocol not already selected elsewhere
const rows = body.querySelectorAll<HTMLElement>('[data-port-mapping-row]');
const newSelect = rows[rows.length - 1]?.querySelector<HTMLSelectElement>(
'select.port-mapping-protocol',
);
if (newSelect) {
// Cloned rows come from an inert <template>, so their select is a plain element that hasn't been
// enhanced yet; style it before setting a value so the change is reflected in the TomSelect control.
initProtocolSelect(newSelect, widget);
const available = protocolChoices(widget).find(choice => !used.has(choice.value));
if (available) {
const ts = getProtocolSelect(newSelect);
if (ts) {
ts.setValue(available.value, true);
} else {
newSelect.value = available.value;
}
}
}
syncLabelTarget(widget);
refreshProtocolOptions(widget);
serialize(widget);
}
/**
* Wire up a single port-mapping widget: add/remove row controls plus re-serialization on any change
* and on form submission.
*/
function initWidget(widget: HTMLElement): void {
// Guard against re-initialization: initForms() runs on every document-wide htmx:afterSettle, so an
// unrelated htmx swap on this (htmx-heavy) form would otherwise stack duplicate listeners on the
// persistent widget — making "Add mapping" insert several rows per click and re-serializing N times.
if (widget.dataset.portMappingInitialized === 'true') return;
widget.dataset.portMappingInitialized = 'true';
// Style the server-rendered protocol selects. TomSelect emits change events through its own callback
// (wired in initProtocolSelect), so the widget-level 'change' listener below only handles the ports
// inputs.
for (const select of widget.querySelectorAll<HTMLSelectElement>('select.port-mapping-protocol')) {
initProtocolSelect(select, widget);
}
const addButton = widget.querySelector<HTMLButtonElement>('[data-port-mapping-add]');
addButton?.addEventListener('click', () => addRow(widget));
// Remove-row buttons (event delegation, since rows are added dynamically)
widget.addEventListener('click', event => {
const target = event.target as HTMLElement;
const removeButton = target.closest('[data-port-mapping-remove]');
if (removeButton === null) return;
removeButton.closest('[data-port-mapping-row]')?.remove();
syncLabelTarget(widget);
refreshProtocolOptions(widget);
serialize(widget);
});
// Keep the hidden input in sync as the user edits the ports fields
widget.addEventListener('input', () => serialize(widget));
widget.addEventListener('change', () => {
refreshProtocolOptions(widget);
serialize(widget);
});
// Ensure the hidden input is current at submit time
widget.closest('form')?.addEventListener('submit', () => serialize(widget));
// Initialize option state and the hidden input from whatever rows are present on load
syncLabelTarget(widget);
refreshProtocolOptions(widget);
serialize(widget);
}
export function initPortMappings(): void {
for (const widget of getElements<HTMLElement>('.port-mapping-widget')) {
initWidget(widget);
}
}

View File

@ -180,3 +180,23 @@ html[data-bs-theme='dark'] {
width: fit-content;
}
// /Sticky action bars
// Port mapping widget
.port-mapping-widget {
// TomSelect keeps its single control's full-size vertical padding even when it inherits the
// `.form-select-sm` sizing, leaving the protocol dropdown taller than the sibling ports input.
// Trim it to the small padding so the two fields align.
.ts-wrapper.single .ts-control {
padding-top: $input-padding-y-sm;
padding-bottom: $input-padding-y-sm;
}
// Only divide multiple mappings; a lone row shouldn't have a line drawn beneath it.
table > tbody > tr > td {
border-bottom: 0;
}
table > tbody > tr + tr > td {
border-top: var(--#{$prefix}border-width) solid var(--#{$prefix}border-color);
}
}
// /Port mapping widget

View File

@ -25,13 +25,24 @@ __all__ = (
)
def parse_numeric_range(string, base=10):
def parse_numeric_range(string, base=10, min_value=None, max_value=None):
"""
Expand a numeric range (continuous or not) into a decimal or
hexadecimal list, as specified by the base parameter
'0-3,5' => [0, 1, 2, 3, 5]
'2,8-b,d,f' => [2, 8, 9, a, b, d, f]
Pass BOTH ``min_value`` and ``max_value`` to validate each range against those bounds *before* it is
expanded: a reversed or out-of-bounds range then raises rather than materializing a huge list or
silently expanding to nothing (which would be swallowed when combined with valid ranges, e.g.
"80,9000-53"). Bounds are all-or-nothing supplying only one raises ``ValueError`` so a caller
can't opt into a lower bound while leaving the expansion size uncapped. With no bounds (e.g.
IP/pattern expansion) a reversed range yields an empty list, as before.
"""
bounded = min_value is not None or max_value is not None
if bounded and (min_value is None or max_value is None):
raise ValueError("parse_numeric_range() requires both min_value and max_value, or neither.")
values = list()
for dash_range in string.split(','):
try:
@ -42,6 +53,16 @@ def parse_numeric_range(string, base=10):
begin, end = int(begin.strip(), base=base), int(end.strip(), base=base) + 1
except ValueError:
raise forms.ValidationError(_('Range "{value}" is invalid.').format(value=dash_range))
if bounded:
# Reject reversed ranges and endpoints outside the permitted range before expanding.
if begin > end - 1:
raise forms.ValidationError(_('Range "{value}" is invalid.').format(value=dash_range))
if begin < min_value or end - 1 > max_value:
raise forms.ValidationError(
_('Range "{value}" is not within the permitted range ({min}-{max}).').format(
value=dash_range, min=min_value, max=max_value
)
)
values.extend(range(begin, end))
return sorted(set(values))

View File

@ -22,6 +22,7 @@ from utilities.forms.utils import (
expand_ipnetwork_pattern,
get_capacity_unit_label,
get_field_value,
parse_numeric_range,
)
from utilities.forms.widgets.select import AvailableOptions, HTMXSelect, Select, SelectedOptions
@ -315,6 +316,65 @@ class ExpandAlphanumericTestCase(TestCase):
sorted(expand_alphanumeric_pattern('r[a,,b]a'))
class ParseNumericRangeTestCase(TestCase):
"""
Validate the operation of parse_numeric_range(), in particular its optional min_value/max_value
bounds checking.
"""
def test_unbounded(self):
self.assertEqual(parse_numeric_range('0-3,5'), [0, 1, 2, 3, 5])
self.assertEqual(parse_numeric_range('2,8-b,d,f', base=16), [2, 8, 9, 10, 11, 13, 15])
def test_unbounded_reversed_range_yields_nothing(self):
# Without bounds a reversed range expands to an empty list, as it always has. Callers which can't
# tolerate that (e.g. port mappings) pass bounds to get an error instead.
self.assertEqual(parse_numeric_range('9-5'), [])
self.assertEqual(parse_numeric_range('1,9-5'), [1])
def test_bounded_within_range(self):
self.assertEqual(parse_numeric_range('80,443,8000-8002', min_value=1, max_value=65535),
[80, 443, 8000, 8001, 8002])
# The bounds are inclusive at both ends
self.assertEqual(parse_numeric_range('1,65535', min_value=1, max_value=65535), [1, 65535])
self.assertEqual(parse_numeric_range('1-3', min_value=1, max_value=3), [1, 2, 3])
def test_bounded_below_min(self):
with self.assertRaises(forms.ValidationError):
parse_numeric_range('0', min_value=1, max_value=65535)
with self.assertRaises(forms.ValidationError):
parse_numeric_range('0-80', min_value=1, max_value=65535)
def test_bounded_above_max(self):
with self.assertRaises(forms.ValidationError):
parse_numeric_range('70000', min_value=1, max_value=65535)
with self.assertRaises(forms.ValidationError):
parse_numeric_range('80-70000', min_value=1, max_value=65535)
def test_bounded_rejects_before_expanding(self):
# A pathological range must be rejected on its endpoints rather than materialized first
with self.assertRaises(forms.ValidationError):
parse_numeric_range('1-9999999999', min_value=1, max_value=65535)
def test_bounded_reversed_range_raises(self):
# With bounds, a reversed range is an error rather than a silent empty expansion — otherwise it
# would be swallowed when combined with a valid range (e.g. '80,9000-53')
with self.assertRaises(forms.ValidationError):
parse_numeric_range('9000-53', min_value=1, max_value=65535)
with self.assertRaises(forms.ValidationError):
parse_numeric_range('80,9000-53', min_value=1, max_value=65535)
def test_bounded_invalid_value(self):
with self.assertRaises(forms.ValidationError):
parse_numeric_range('80,abc', min_value=1, max_value=65535)
def test_bounds_are_all_or_nothing(self):
# Supplying only one bound would opt into a lower bound while leaving the expansion size uncapped
with self.assertRaises(ValueError):
parse_numeric_range('80', min_value=1)
with self.assertRaises(ValueError):
parse_numeric_range('80', max_value=65535)
class ImportFormTestCase(TestCase):
def test_format_detection(self):