Description
After we upgraded to the version 6 we are facing several container crashes (in total 4) when working with the platform. We cannot reproduce it with a special action, it seems to be happening over time.
We havent had these issues before, and after downgrading to version 5, the issue dissapeared again. So the code has not changed.
In the Docker logs i see an "Out of memory" exception, in the log of the Container itself i do not see any anomalies.
I know that these kind of issues are very tricky to find - so let me know if i can somehow provide more infos which help you analyse the issue
Docker logs
root@docker-ce-ubuntu-4gb-nbg1-1:~# dmesg --ctime | grep -i -B 2 -A 2 'killed process'
[Mon May 12 16:16:15 2025] [1557847] 0 1557847 3860 685 77824 0 0 sshd
[Mon May 12 16:16:15 2025] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=ssh.service,mems_allowed=0,global_oom,task_memcg=/system.slice/docker-3d642c321f90d377970f9f3689d5ff549844ec87faa2000735e57932a53ceaaf.scope,task=java,pid=1531921,uid=1001
[Mon May 12 16:16:15 2025] Out of memory: Killed process 1531921 (java) total-vm:6357576kB, anon-rss:1873576kB, file-rss:0kB, shmem-rss:0kB, UID:1001 pgtables:4080kB oom_score_adj:0
[Mon May 12 16:17:20 2025] br-e8fcbc5a5776: port 1(veth14aeaea) entered disabled state
[Mon May 12 16:17:20 2025] veth958a19f: renamed from eth0
[Thu May 15 16:21:33 2025] [1644542] 999 1644542 54027 1428 176128 0 0 postgres
[Thu May 15 16:21:33 2025] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=qemu-guest-agent.service,mems_allowed=0,global_oom,task_memcg=/system.slice/docker-3d642c321f90d377970f9f3689d5ff549844ec87faa2000735e57932a53ceaaf.scope,task=java,pid=1558374,uid=1001
[Thu May 15 16:21:33 2025] Out of memory: Killed process 1558374 (java) total-vm:6362860kB, anon-rss:1999820kB, file-rss:0kB, shmem-rss:0kB, UID:1001 pgtables:4384kB oom_score_adj:0
[Thu May 15 18:37:49 2025] br-e8fcbc5a5776: port 1(vethf9fbc2a) entered disabled state
[Thu May 15 18:37:49 2025] veth6716826: renamed from eth0
[Sat May 17 06:23:43 2025] [1697979] 0 1697979 3657 276 57344 0 0 sshd
[Sat May 17 06:23:43 2025] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=init.scope,mems_allowed=0,global_oom,task_memcg=/system.slice/docker-3d642c321f90d377970f9f3689d5ff549844ec87faa2000735e57932a53ceaaf.scope,task=java,pid=1647159,uid=1001
[Sat May 17 06:23:43 2025] Out of memory: Killed process 1647159 (java) total-vm:6357728kB, anon-rss:1979492kB, file-rss:0kB, shmem-rss:0kB, UID:1001 pgtables:4348kB oom_score_adj:0
[Sat May 17 06:52:28 2025] br-e8fcbc5a5776: port 1(veth5dc9911) entered disabled state
[Sat May 17 06:52:28 2025] veth55c6d12: renamed from eth0
[Sat May 17 09:47:13 2025] [1712246] 0 1712246 3657 312 61440 0 0 sshd
[Sat May 17 09:47:13 2025] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=docker.service,mems_allowed=0,global_oom,task_memcg=/system.slice/docker-3ac9845643f11175442a7bbfa8d59d3c3275e23ec997de182084821b9d463d79.scope,task=java,pid=1705704,uid=1001
[Sat May 17 09:47:13 2025] Out of memory: Killed process 1705704 (java) total-vm:6358472kB, anon-rss:1883164kB, file-rss:0kB, shmem-rss:0kB, UID:1001 pgtables:4120kB oom_score_adj:0
[Sat May 17 10:15:28 2025] br-e8fcbc5a5776: port 1(veth22653a8) entered disabled state
[Sat May 17 10:15:28 2025] veth4c4c654: renamed from eth0
Container logs
___/ __ \____ ______/ /_ (_)___ (_)___ _/ / / / __/ / __ \ / / __ / / __
/ // / // (_ ) / / / / / // / / / / /
/_____/_,/// /// /_/// //
/___/
Powered by Quarkus 3.15.2
2025-05-17 06:53:10,046 INFO [org.das.uti.Home] (main) DASHJOIN_HOME = wooltest_dev_1
2025-05-17 06:53:10,046 INFO [org.das.uti.Home] (main) git clone ...
2025-05-17 06:53:21,472 INFO [com.das.lau.LocalAuthManager] (main) Loading /deployments/wooltest_dev_1/djusers.properties
2025-05-17 06:53:21,472 INFO [com.das.lau.LocalAuthManager] (main) Loading /deployments/wooltest_dev_1/djroles.properties
2025-05-17 06:53:21,472 INFO [com.das.lau.LocalAuthManager] (main) System already set up, will not provision initial user
2025-05-17 06:53:22,341 INFO [com.das.ser.goo.AppEngineService] (main) Warming up instance version=6.0.4-20b4b86-28b8b3b
2025-05-17 06:53:22,342 INFO [org.das.ser.Services] (main) Starting metadata collection for tenant=dj
2025-05-17 06:53:24,775 INFO [org.das.ser.Services] (main) Done metadata collection for tenant=dj
2025-05-17 06:53:25,173 INFO [com.das.ser.goo.AppEngineService] (main) Warming up instance done.
2025-05-17 06:53:25,315 INFO [io.quarkus] (main) djplatform 6.0.4-20b4b86-28b8b3b on JVM (powered by Quarkus 3.15.2) started in 17.262s. Listening on: http://0.0.0.0:8080
2025-05-17 06:53:25,316 INFO [io.quarkus] (main) Profile prod activated.
2025-05-17 06:53:25,316 INFO [io.quarkus] (main) Installed features: [cdi, google-cloud-firebase-admin, google-cloud-firestore, google-cloud-secret-manager, jdbc-db2, jdbc-h2, jdbc-mssql, jdbc-postgresql, jgit, jsch, reactive-routes, resteasy, resteasy-client, resteasy-client-jackson, resteasy-jackson, resteasy-multipart, security, security-properties-file, smallrye-context-propagation, smallrye-jwt, smallrye-openapi, swagger-ui, vertx]
2025-05-17 06:54:24,205 INFO [com.das.ser.sta.StaticContentHandler] (vert.x-eventloop-thread-2) DASHJOIN_DEVMODE active. Serving webroot without cache: /deployments/wooltest_dev_1/assets/public_html
2025-05-17 06:54:57,076 INFO [com.das.ser.ten.TenantService] (executor-thread-2) Login profile uid=admin email=admin@localhost roles=[admin] username=admin
2025-05-17 08:37:40,908 INFO [org.das.ser.Services] (ForkJoinPool.commonPool-worker-3) evicting pojo db - closing connection pools for [dj/wooltest, dj/config]
2025-05-17 08:37:40,908 INFO [org.das.ser.Services] (executor-thread-1) Starting metadata collection for tenant=dj
2025-05-17 08:37:42,124 INFO [org.das.ser.Services] (executor-thread-1) Done metadata collection for tenant=dj
2025-05-17 08:37:42,342 INFO [com.das.ser.ten.TenantService] (executor-thread-1) Login profile uid=cecile email=cecile@localhost roles=[authenticated, master] username=cecile
2025-05-17 09:47:02,380 INFO [com.das.ser.ten.TenantService] (executor-thread-74) Login profile uid=cecile email=cecile@localhost roles=[authenticated, master] username=cecile
`