New user's registration have been closed due to high spamming and low trafic on this forum. Please contact forum admins directly if you need an account. Thanks !
No, I wasn't using the mail system. I had webmin installed and using virtualmin. The latter used a cronjob to send messages which ended up in the maildrop. Since I'm not using webmin anymore, I completely uninstalled everything related to that and removed the cronjob.
So now everything is running smoothely again!