* Introduce reusable BackgroundJob framework
A new abstract class can be used to implement job function classes. It
handles the necessary logic for starting and stopping jobs, including
exception handling and rescheduling of recurring jobs.
This commit also includes the migration of data source jobs to the new
framework.
* Restore using import_string for jobs
Using the 'import_string()' utility from Django allows the job script
class to be simplified, as module imports no longer need to avoid loops.
This should make it easier to queue and maintain jobs.
* Use SyncDataSourceJob for management command
Instead of maintaining two separate job execution logics, the same job
is now used for both background and interactive execution.
* Implement BackgroundJob for running scripts
The independent implementations of interactive and background script
execution have been merged into a single BackgroundJob implementation.
* Fix documentation of model features
* Ensure consitent code style
* Introduce reusable ScheduledJob
A new abstract class can be used to implement job function classes that
specialize in scheduling. These use the same logic as regular
BackgroundJobs, but ensure that they are only scheduled once at any given
time.
* Introduce reusable SystemJob
A new abstract class can be used to implement job function classes that
specialize in system background tasks (e.g. synchronization or
housekeeping). In addition to the features of the BackgroundJob and
ScheduledJob classes, these implement additional logic to not need to be
bound to an existing NetBox object and to setup job schedules on plugin
load instead of an interactive request.
* Add documentation for jobs framework
* Revert "Use SyncDataSourceJob for management"
This partially reverts commit db591d4. The 'run_now' parameter of
'enqueue()' remains, as its being used by following commits.
* Merge enqueued status into JobStatusChoices
* Fix logger for ScriptJob
* Remove job name for scripts
Because scripts are already linked through the Job Instance field, the
name is displayed twice. Removing this reduces redundancy and opens up
the possibility of simplifying the BackgroundJob framework in future
commits.
* Merge ScheduledJob into BackgroundJob
Instead of using separate classes, the logic of ScheduledJob is now
merged into the generic BackgroundJob class. This allows reusing the
same logic, but dynamically deciding whether to enqueue the same job
once or multiple times.
* Add name attribute for BackgroundJob
Instead of defining individual names on enqueue, BackgroundJob classes
can now set a job name in their meta class. This is equivalent to other
Django classes and NetBox scripts.
* Drop enqueue_sync_job() method from DataSource
* Import ScriptJob directly
* Relax requirement for Jobs to reference a specific object
* Rename 'run_now' arg on Job.enqueue() to 'immediate'
* Fix queue lookup in Job enqueue
* Collapse SystemJob into BackgroundJob
* Remove legacy JobResultStatusChoices
ChoiceSet was moved to core in 40572b5.
* Use queue 'low' for system jobs by default
System jobs usually perform low-priority background tasks and therefore
can use a different queue than 'default', which is used for regular jobs
related to specific objects.
* Add test cases for BackgroundJob handling
* Fix enqueue interval jobs
As the job's name is set by enqueue(), it must not be passed in handle()
to avoid duplicate kwargs with the same name.
* Honor schedule_at for job's enqueue_once
Not only can a job's interval change, but so can the time at which it is
scheduled to run. If a specific scheduled time is set, it will also be
checked against the current job schedule. If there are any changes, the
job is rescheduled with the new time.
* Switch BackgroundJob to regular methods
Instead of using a class method for run(), a regular method is used for
this purpose. This gives the possibility to add more convenience methods
in the future, e.g. for interacting with the job object or for logging,
as implemented for scripts.
* Fix background tasks documentation
* Test enqueue in combination with enqueue_once
* Rename background jobs to tasks (to differentiate from RQ)
* Touch up docs
* Revert "Use queue 'low' for system jobs by default"
This reverts commit b17b2050df.
* Remove system background job
This commit reverts commits 4880d81 and 0b15ecf. Using the database
'connection_created' signal for job registration feels a little wrong at
this point, as it would trigger registration very often. However, the
background job framework is prepared for this use case and can be used
by plugins once the auto-registration of jobs is solved.
* Fix runscript management command
Defining names for background jobs was disabled with fb75389. The
preceeding changes in 257976d did forget the management command.
* Use regular imports for ScriptJob
* Rename BackgroundJob to JobRunner
---------
Co-authored-by: Jeremy Stretch <jstretch@netboxlabs.com>
* Initial work on #15621
* Signal receiver should ignore models which don't support notifications
* Flesh out NotificationGroup functionality
* Add NotificationGroup filters for users & groups
* Separate read & dimiss actions
* Enable one-click dismissals from notifications list
* Include total notification count in dropdown
* Drop 'kind' field from Notification model
* Register event types in the registry; add colors & icons
* Enable event rules to target notification groups
* Define dynamic choices for Notification.event_name
* Move event registration to core
* Add more job events
* Misc cleanup
* Misc cleanup
* Correct absolute URLs for notifications & subscriptions
* Optimize subscriber notifications
* Use core event types when queuing events
* Standardize queued event attribute to event_type; change content_type to object_type
* Rename Notification.event_name to event_type
* Restore NotificationGroupBulkEditView
* Add API tests
* Add view & filterset tests
* Add model documentation
* Fix tests
* Update notification bell when notifications have been cleared
* Ensure subscribe button appears only on relevant models
* Notifications/subscriptions cannot be ordered by object
* Misc cleanup
* Add event icon & type to notifications table
* Adjust icon sizing
* Mute color of read notifications
* Misc cleanup
* 16145 script api use module.script name instead of pk
* 16145 fix test
* 16145 allow both pk and script name
* 16145 update doc string
* Simplify retrieval logic
---------
Co-authored-by: Jeremy Stretch <jstretch@netboxlabs.com>
- Introduces the Script model to represent individual Python classes within a ScriptModule file
- Automatically migrates jobs & event rules
---------
Co-authored-by: Jeremy Stretch <jstretch@netboxlabs.com>
* Introduce RelatedObjectCountField
* Introduce get_annotations_for_serializer() and enable dynamic annotations
* Add RelatedObjectCountFields to serializers; remove static annotations from querysets
* Remove annotations cleanup logic from BriefModeMixin
* Annotate type for RelatedObjectCountField
* Remove redundant field on TagSerializer
* Add missing reverse relationship for power feeds to rack
* Refactor RelatedObjectCountField to take a single relationship name
* Enable dynamic field inclusion for REST API serializers
* Recurse through nested serializer when resolving prefetches
* Remove obsolete calls to prefetch_related() for API views
* Remove support for brief_prefetch_fields viewset attribute
* Rename query parameter
* Fixes#15133: Fix FHRP group representation on assignments endpoint under brief mode (#15134)
* Fixes#15133: Fix FHRP group representation on assignments endpoint under brief mode
* Update API test
* Restore get_queryset() on BriefModeMixin, minus prefetch logic
* get_prefetches_for_serializer() should reference serializer field source if set
In PR #13958 (commit 8224644) _get_report was modified to do the call on the variable without changing the call later on.
This commit fixes that and removes the call on the variable.
Signed-off-by: Josef Johansson <josef@oderland.se>
* adds standardized list API for scripts and reports #13037
* adds standardized list API for scripts and reports #13037
* adds standardized list API for scripts and reports #13037
* adds module name to the display #13037
* Initial work on custom field choice sets
* Rename choices to extra_choices (prep for #12194)
* Remove CustomField.choices
* Add & update tests
* Clean up table columns
* Add order_alphanetically boolean for choice sets
* Introduce ArrayColumn for choice lists
* Show dependent custom fields on choice set view
* Update custom fields documentation
* Introduce ArrayWidget for more convenient editing of choices
* Incorporate PR feedback
* Misc cleanup
* Initial work on predefined choices for custom fields
* Misc cleanup
* Add IATA airport codes
* #13241: Add support for custom field choice labels
* Restore ArrayColumn
* Misc cleanup
* Change extra_choices back to a nested ArrayField to preserve choice ordering
* Hack to bypass GraphQL API test utility absent support for nested ArrayFields
* Fixes#11335: Default manager for ObjectChange should filter by installed apps
* Employ canonical model discovery mechanism
* Move filtering logic to valid_models() queryset method
* fixed import to avoid content type does not exist
* Cleanup
---------
Co-authored-by: Abhimanyu Saharan <desk.abhimanyu@gmail.com>
* Reference database object by GFK when running scripts & reports via UI
* Reference database object by GFK when running scripts & reports via API
* Remove old enqueue_job() method
* Enable filtering jobs by object
* Introduce ObjectJobsView
* Add tabbed views for report & script jobs
* Add object_id to JobSerializer
* Move generic relation to JobsMixin
* Clean up old naming
* Replace masonry with gridstack
* Initial work on dashboard widgets
* Implement function to save dashboard layout
* Define a default dashboard
* Clean up widgets
* Implement widget configuration views & forms
* Permit merging dict value with existing dict in user config
* Add widget deletion view
* Enable HTMX for widget configuration
* Implement view to add dashboard widgets
* ObjectCountsWidget: Identify models by app_label & name
* Add color customization to dashboard widgets
* Introduce Dashboard model to store user dashboard layout & config
* Clean up utility functions
* Remove hard-coded API URL
* Use fixed grid cell height
* Add modal close button
* Clean up dashboard views
* Rebuild JS
* WIP
* Add config_template field to Device
* Pre-fetch referenced templates
* Correct up_to_date callable
* Add config_template FK to Device
* Update & merge migrations
* Add config_template FK to Platform
* Add tagging support for ConfigTemplate
* Catch exceptions when rendering device templates in UI
* Refactor ConfigTemplate.render()
* Add support for returning plain text content
* Add ConfigTemplate model documentation
* Add feature documentation for config rendering
* Add interval to JobResult
* Accept a recurrence interval when executing scripts & reports
* Cleaned up jobs list display
* Schedule next job only if a reference start time can be determined
* Improve validation for scheduled jobs
Arthur Hanson
Alexander Haase
Jeremy Stretch
Josef Johansson
Abhimanyu Saharan
Fabian Geisberger
kkthxbye-code
Jeremy Stretch
kkthxbye