Database is not reindexed on upgrade from 4.0.3 to 4.0.5

[pv2b]

Current Behavior

There was a defect in Netbox 4.0.3 where circuit provider accounts were not indexed (netbox-community/netbox#16286) which was resolved in Netbox 4.0.5.

So, as usual, I edit my Dockerfile-Plugins to bump the version number, do a docker-compose build and a docker-compose up -d to go up to the newest version.

However, the bug was still there, in that searching for a provider account does nothing. It turns out that a required step when performing a Netbox upgrade is to run upgrade.sh which among other things re-build the search cache:

https://github.com/netbox-community/netbox/blob/56b6b1b9d864a12d05b8cd6d9f4f1a476fa00e85/upgrade.sh#L110-L113

However, netbox-docker never calls this script, instead it has its own update logic:

netbox-docker/docker/docker-entrypoint.sh

Lines 39 to 51 in 0c99ff8

	# Check if update is needed
	if ! ./manage.py migrate --check >/dev/null 2>&1; then
	echo "⚙️ Applying database migrations"
	./manage.py migrate --no-input
	echo "⚙️ Running trace_paths"
	./manage.py trace_paths --no-input
	echo "⚙️ Removing stale content types"
	./manage.py remove_stale_contenttypes --no-input
	echo "⚙️ Removing expired user sessions"
	./manage.py clearsessions
	echo "⚙️ Building search index (lazy)"
	./manage.py reindex --lazy
	fi

This will re-index the search cache, but only if there were any changes made in the database models (there were none between 4.0.3 and 4.0.5) and thus the re-index never happened.

Manually running the rebuild using the command docker exec netbox-docker_netbox_1 ./manage.py reindex --lazy seems to have fixed the issue.

Expected Behavior

The upgrade logic (or, ideally, running upgrade.sh) should run on all version updates, and checking for database migrations is not a sufficient heuristic. For example, a persistent file could be created inside the container recording the version on startup, and if that version changes, the upgrade script is ran.

Docker Compose Version

docker-compose up -d

Docker Version

Client: Docker Engine - Community
 Version:           23.0.5
 API version:       1.42
 Go version:        go1.19.8
 Git commit:        bc4487a
 Built:             Wed Apr 26 16:21:07 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          23.0.5
  API version:      1.42 (minimum version 1.12)
  Go version:       go1.19.8
  Git commit:       94d3ad6
  Built:            Wed Apr 26 16:21:07 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.21
  GitCommit:        3dce8eb055cbb6872793272b4f20ed16117344f8
 runc:
  Version:          1.1.7
  GitCommit:        v1.1.7-0-g860f061
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

The git Revision

8692d54b838522ef947246ec5d62c2308efff7aa

The git Status

driftadm@dsdocker02:~/netbox-docker$ git status
On branch plugins
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   docker-compose.yml

Untracked files:
  (use "git add <file>..." to include in what will be committed)
        Dockerfile-Plugins
        a.log
        build.log
        configuration/plugins.py
        netbox-plugin-mclag/
        plugin_requirements.txt

no changes added to commit (use "git add" and/or "git commit -a")
driftadm@dsdocker02:~/netbox-docker$

Startup Command

docker-compose up -d

NetBox Logs

driftadm@dsdocker02:~/netbox-docker$ docker-compose logs netbox
Attaching to netbox-docker_netbox_1
netbox_1               | ↩️ Skip creating the superuser
netbox_1               | 🧬 loaded config '/etc/netbox/config/configuration.py'
netbox_1               | 🧬 loaded config '/etc/netbox/config/extra.py'
netbox_1               | 🧬 loaded config '/etc/netbox/config/logging.py'
netbox_1               | 🧬 loaded config '/etc/netbox/config/plugins.py'
netbox_1               | 🧬 loaded config '/etc/netbox/config/social-auth.py'
netbox_1               | ✅ Initialisation is done.
netbox_1               | ⏳ Waiting for control socket to be created... (1/10)
netbox_1               | 2024/06/10 08:47:25 [warn] 6#6 Unit is running unprivileged, then it cannot use arbitrary user and group.
netbox_1               | 2024/06/10 08:47:25 [info] 6#6 unit 1.32.0 started
netbox_1               | 2024/06/10 08:47:25 [info] 84#84 discovery started
netbox_1               | 2024/06/10 08:47:25 [notice] 84#84 module: python 3.11.6 "/usr/lib/unit/modules/python3.11.unit.so"
netbox_1               | 2024/06/10 08:47:25 [info] 6#6 controller started
netbox_1               | 2024/06/10 08:47:25 [notice] 6#6 process 84 exited with code 0
netbox_1               | 2024/06/10 08:47:25 [info] 86#86 router started
netbox_1               | 2024/06/10 08:47:25 [info] 86#86 OpenSSL 3.0.10 1 Aug 2023, 300000a0
netbox_1               | ⚙️ Applying configuration from /etc/unit/nginx-unit.json
netbox_1               | 2024/06/10 08:47:26 [info] 89#89 "netbox" prototype started
netbox_1               | 2024/06/10 08:47:26 [info] 90#90 "netbox" application started
netbox_1               | ✅ Unit configuration loaded successfully
netbox_1               | 2024/06/10 08:47:28 [notice] 6#6 process 82 exited with code 0
netbox_1               | 2024/06/10 08:47:33 [info] 122#122 "netbox" application started

Content of docker-compose.override.yml

driftadm@dsdocker02:~/netbox-docker$ cat docker-compose.override.yml
version: '3.4'
services:
  netbox:
    image: netbox:latest-plugins
    ports:
    - 8080:8080
    environment:
      DEBUG: "false"
    build:
      context: .
      dockerfile: Dockerfile-Plugins
  netbox-worker:
    image: netbox:latest-plugins
    build:
      context: .
      dockerfile: Dockerfile-Plugins
  netbox-housekeeping:
    image: netbox:latest-plugins
    build:
      context: .
      dockerfile: Dockerfile-Plugins

networks:
  default:
    name: dsnet
    external: true