Commit Graph

124 Commits

Author SHA1 Message Date
Jens L a987846c76
root: celery refactor (#6095)
* root: celery refactor

cleanup deprecation messages by configuring celery with a single object

run celery as django management command

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* improve debug experience

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* fix lint

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* add debugpy to dev dependencies

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* fix task_always_eager

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

---------

Signed-off-by: Jens Langhammer <jens@goauthentik.io>
2023-06-28 16:44:50 +02:00
Jens L 5b0cc3672b
root: add method to get install_id without django being loaded (#5755)
Signed-off-by: Jens Langhammer <jens@goauthentik.io>
2023-05-25 18:52:21 +02:00
Jens L 0d0bb1a559
root: add install ID (#5717)
* root: add install ID

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* fix tests

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* add fallback when no migrations table exists

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* fix lint

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

---------

Signed-off-by: Jens Langhammer <jens@goauthentik.io>
2023-05-22 17:24:12 +02:00
Jens L 54d508ae8c
ci: fix pyright errors (#5392)
* ci: fix pyright errors

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* fix error in oauth 1 source

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* remove redundant blueprint fixtures

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

---------

Signed-off-by: Jens Langhammer <jens@goauthentik.io>
2023-04-27 17:33:47 +03:00
Jens L 94ae490284
lifecycle: migrate internal healthcheck to use go (#5322)
Signed-off-by: Jens Langhammer <jens@goauthentik.io>
2023-04-20 19:46:49 +03:00
Jens L dfa80543b5
root: add ruff linter (#5240)
* root: add ruff linter

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* actually add ruff

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* fix lint

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

---------

Signed-off-by: Jens Langhammer <jens@goauthentik.io>
2023-04-18 13:28:19 +02:00
Jens L 1a4c640835
lifecycle: fix worker healthcheck (#5259)
closes #5258

Signed-off-by: Jens Langhammer <jens@goauthentik.io>
2023-04-14 14:39:31 +02:00
Jens L 38bf0ee740
lifecycle: re-add exec to ak wrapper (#5253)
Signed-off-by: Jens Langhammer <jens@goauthentik.io>
2023-04-14 13:52:11 +02:00
Jens L f84a10b59b
core: revert django update (#5236)
* Revert "core: bump django from 4.1.7 to 4.2 (#5151)"

This reverts commit 18a4eac527.

* run unittests with postgres 11 and 12

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

---------

Signed-off-by: Jens Langhammer <jens@goauthentik.io>
2023-04-13 14:10:12 +02:00
dependabot[bot] 18a4eac527
core: bump django from 4.1.7 to 4.2 (#5151)
* core: bump django from 4.1.7 to 4.2

Bumps [django](https://github.com/django/django) from 4.1.7 to 4.2.
- [Release notes](https://github.com/django/django/releases)
- [Commits](https://github.com/django/django/compare/4.1.7...4.2)

---
updated-dependencies:
- dependency-name: django
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* upgrade to psycopg3, use custom engine for prometheus metrics

See https://github.com/korfuri/django-prometheus/issues/350

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* make scripts use pscopg3

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Jens Langhammer <jens@goauthentik.io>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jens Langhammer <jens@goauthentik.io>
2023-04-11 15:00:27 +02:00
Jens L 02f75a92ce
lifecycle: don't use celery ping for worker healthcheck (#5153)
Signed-off-by: Jens Langhammer <jens@goauthentik.io>
2023-04-03 18:15:31 +02:00
Ongy adcd11b1f8
core: extend postgres configuration (#5138)
Add postgres configuration options to control
TLS verification and client certificates.
2023-04-02 17:39:36 +02:00
Jens L 9666d407b4
lifecycle: also migrate before starting worker, trap exit to cleanup mode (#5123)
Signed-off-by: Jens Langhammer <jens@goauthentik.io>
2023-03-30 14:04:38 +02:00
dependabot[bot] 18cfe67719
core: bump black from 22.12.0 to 23.1.0 (#4584)
* core: bump black from 22.12.0 to 23.1.0

Bumps [black](https://github.com/psf/black) from 22.12.0 to 23.1.0.
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](https://github.com/psf/black/compare/22.12.0...23.1.0)

---
updated-dependencies:
- dependency-name: black
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* re-format

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* fix

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Jens Langhammer <jens@goauthentik.io>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jens Langhammer <jens@goauthentik.io>
2023-02-01 11:31:32 +01:00
Jens L 9568f4dbd6
root: improve code style (#4436)
* cleanup pylint comments

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* remove more

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* fix url name

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* *: use ExtractHour instead of ExtractDay

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

Signed-off-by: Jens Langhammer <jens@goauthentik.io>
2023-01-15 17:02:31 +01:00
Jens L e6b5810e03
polices/hibp: remove deprecated (#4363)
* remove hibp

Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>

* don't save event matcher apps in migrations

Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>

* cleanup migrations

Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>

* update docs, update some phrasing

Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>

Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>
2023-01-05 13:19:26 +01:00
Jens Langhammer 6ca1654129 lifecycle: don't set user/group in gunicorn
closes #4098 closes #3236

the user and group are inherited from the parent process so this isnt required

Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>
2022-12-02 12:42:55 +02:00
Jens L 55aa1897af
root: use single redis db (#4009)
* use single redis db

Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>

* cleanup prefixes

Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>

* ensure __str__ always returns string

Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>

* fix remaining old prefixes

Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>

* add release notes

Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>

Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>
2022-11-15 14:31:29 +01:00
Jens Langhammer ee6dc45a30 lifecycle: fix incorrect messages looped
Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>
2022-11-12 16:50:03 +01:00
Jens Langhammer 6882445937 *: handle PermissionError when saving files, ensure permission bits are set correctly
closes #3817

Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>
2022-10-19 20:24:28 +02:00
Jens L b16a3d5697
internal: use config system for workers/threads, document the settings (#3626)
use config system for workers/threads, document the settings

Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>

Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>
2022-09-21 09:59:03 +02:00
Jens Langhammer 28835fbca7 root: re-use custom log helper from config and cleanup duplicate functions
Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>
2022-09-10 13:24:31 +02:00
Jens L 62f93c83d4
ci: update pyright (#3546) 2022-09-07 00:23:25 +02:00
Jens Langhammer d8a68407f9 lifecycle: add worker-status command to debug worker cpu usage issues
Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>

#3125
2022-08-18 20:13:30 +02:00
Jens L ec42d378ab
blueprints/cleanup (#3369) 2022-08-05 08:39:00 +02:00
Jens L db1dd196e0
lifecycle: optimise container lifecycle and process signals (#3332)
* add dumb-init, use exec in wrapper

Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>

* fix exec?

Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>
2022-07-30 22:41:29 +02:00
Jens Langhammer 658dc63c4c lifecycle: revert waiting for lock, launch managed reconcile on app import
Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>
2022-07-19 12:06:57 +02:00
Jens Langhammer 4edec5f666 lifecycle: connect to database first
Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>
2022-07-19 10:54:56 +02:00
Jens L e9d9d658c4
lifecycle: make worker wait for migrations to be done (#3254)
* lifecycle: make worker wait for migrations to be done

* retry managed reconcile task
2022-07-15 19:44:45 +02:00
Jens Langhammer d56ddb16b1 lifecycle: fix confusing success messages in startup healthiness check
Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>
2022-06-20 21:26:21 +02:00
Jens Langhammer 7beebe030d lifecycle: run bootstrap tasks inline when using automated install
Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>
2022-06-09 20:09:55 +02:00
Jens Langhammer 6739ded5a9 root: remove flower dependency
Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>
2022-06-04 14:22:25 +02:00
Jens L 3eb466ff4b
lifecycle: cleanup prometheus (#2972)
* remove high cardinality labels

Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>

* retry worker number for prometheus multiprocess id

Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>

* revert to pid, use subdirectories

Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>

* cleanup more

Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>

* use worker id based off of https://github.com/benoitc/gunicorn/issues/1352

Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>

* fix missing app label

Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>

* tests/e2e: remove static names

Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>

* fix

Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>
2022-05-29 21:45:25 +02:00
Jens L a7a7b5aacb
root: export poetry deps to requirements.txt so we don't need poetry … (#2823)
* root: export poetry deps to requirements.txt so we don't need poetry installed, removed packages we don't need anymore

Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>

* update docs

Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>
2022-05-07 21:22:33 +02:00
Jens Langhammer 15d8988569 Revert "lifecycle: use worker nr instead of process id to keep number of prometheus database files low"
This reverts commit 254249e38b.

Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>
2022-04-29 18:14:04 +02:00
Jens Langhammer 254249e38b lifecycle: use worker nr instead of process id to keep number of prometheus database files low
Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>

#2784
2022-04-28 23:09:39 +02:00
Jens Langhammer 4cf00ed5cf lifecycle: fix password and hostname not properly quoted
#2623
2022-03-31 16:37:54 +00:00
Jens Langhammer d9675695fe root: remove backup remainders
Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>
2022-03-09 21:56:18 +01:00
Jens L d5e04a2301
*: remove deprecated backup (#2129)
* *: remove backup

Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>

* fix lint

Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>

* website/docs: add docs

Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>

* *: final cleanup

Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>

* ci: use correct pyproject when migrating from stable

Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>

* website/docs: fix broken docs

Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>
2022-02-05 18:54:15 +01:00
Jens Langhammer 0232c4e162 lifecycle: send analytics in gunicorn config to decrease outgoing requests when workers get restarted
Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>
2022-02-01 15:01:43 +01:00
Jens Langhammer f679250edd lifecycle: remove gunicorn reload option
should help with #2159
2022-01-31 12:06:08 +00:00
Jens Langhammer c2586557d8 root: fix redis passwords not being encoded correctly
closes #2130

Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>
2022-01-26 20:45:45 +01:00
Jens Langhammer 0d47654651 root: add max-requests for gunicorn and max tasks for celery
Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>
2022-01-26 10:04:58 +01:00
Jens Langhammer fe39e39dcd lifecycle: make secret_key warning more prominent
Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>

#2131
2022-01-24 21:52:16 +01:00
Jens Langhammer 883f213b03 lifecycle: wait for db in worker
Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>
2022-01-24 21:51:57 +01:00
Jens Langhammer 33567b56d7 lifecycle: replace lowercase, deprecated prometheus_multiproc_dir
Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>
2022-01-23 21:21:06 +01:00
Jens Langhammer 41e7b9b73f outposts/proxyv2: fix before-redirect url not being saved in proxy mode
closes #2109

Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>
2022-01-19 19:16:30 +01:00
Jens Langhammer 89abd44b76 lifecycle: add early check for missing/invalid secret key
Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>
2022-01-19 09:53:53 +01:00
Jens Langhammer 111fbf119b *: refactor prometheus gauges to directly updating metrics view
Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>
2022-01-16 13:57:07 +01:00
Jens Langhammer b6f518ffe6 lifecycle: fix tests in container not working
Signed-off-by: Jens Langhammer <jens.langhammer@beryju.org>
2022-01-06 21:29:08 +01:00