You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Listener to services should be on a separate thread and resilient by default #682
Generated applications for django have a default behaviour to sync users which requires kafka and accounts to be fully operative at startup. If they're not fully operative, pods go on a crashloop.
This is not ideal because the user facing part of the application does not necessarily requires user login functionality.
Ideally we should set this kind of listeners on a separate pod/container.
A quicker proposal is the following:
if os.environ.get('KUBERNETES_SERVICE_HOST', None):
# init the auth service when running in/for k8s
import threading
from cloudharness_django.services import get_auth_service, init_services
from cloudharness import log
import time
def start_auth_service():
try:
init_services()
except:
log.exception("Error initializing services. Retrying in 5 seconds...")
time.sleep(5)
start_auth_service()
threading.Thread(target=start_auth_service).start()
# start the kafka event listener when running in/for k8s
def start_event_listener():
try:
import cloudharness_django.services.events
log.info("User sync events listener started")
except:
log.exception("Error initializing event queue. Retrying in 5 seconds...")
time.sleep(5)
start_event_listener()
threading.Thread(target=start_event_listener).start()
Yes the consume is on a different thread, but the initialization of the client isn't, so making the main application thread file failing by default is kafka is not there.
That's not necessarily wrong shouldn't be the default behaviour in my opinion as kafka events are usually related to specific features and are preventing/delaying the application startup this way.
Activity
zsinnema commentedon Mar 29, 2023
@filippomc the listener already is on a separate thread (using async_consume), see
cloud-harness/infrastructure/common-images/cloudharness-django/libraries/cloudharness-django/cloudharness_django/services/events.py
Line 61 in 5e5052f
filippomc commentedon Mar 29, 2023
Yes the consume is on a different thread, but the initialization of the client isn't, so making the main application thread file failing by default is kafka is not there.
That's not necessarily wrong shouldn't be the default behaviour in my opinion as kafka events are usually related to specific features and are preventing/delaying the application startup this way.