• 0 Posts
  • 9 Comments
Joined 18 days ago
cake
Cake day: September 14th, 2025

help-circle
  • tal@olio.cafetoSelfhosted@lemmy.worldhow do I find process that leads to oom?
    link
    fedilink
    English
    arrow-up
    13
    arrow-down
    1
    ·
    edit-2
    5 hours ago

    OOMs happen because your system is out of memory.

    You asked how to know which process is responsible. There is no correct answer to which process is “wrong” in using more memory — all one can say is that processes are in aggregate asking for too much memory. The kernel tries to “blame” a process and will kill it, as you’ve seen, to let your system continue to function, but ultimately, you may know better than it which is acting in a way you don’t want.

    It should log something to the kernel log when it OOM kills something.

    It may be that you simply don’t have enough memory to do what you want to do. You could take a glance in top (sort by memory usage with shift-M). You might be able to get by by adding more paging (swap) space. You can do this with a paging file if it’s problematic to create a paging partition.

    EDIT: I don’t know if there’s a way to get a dump of processes that are using memory at exactly the instant of the OOM, but if you want to get an idea of what memory usage looks at at that time, you can certainly do something like leave a top -o %MEM -b >log.txt process running to get a snapshot every two seconds of process memory use. top will print a timestamp at the top of each entry, and between the timestamped OOM entry in the kernel log and the timestamped dump, you should be able to look at what’s using memory.

    There are also various other packages for logging resource usage that provide less information, but also don’t use so much space, if you want to view historical resource usage. sysstat is what I usually use, with the sar command to view logged data, though that’s very elderly. Things like that won’t dump a list of all processes, but they will let you know if, over a given period of time, a server is running low on available memory.



  • Is your concern compromise of your data or loss of the server?

    My guess is that most burglaries don’t wind up with people trying to make use of the data on computers.

    As to loss, I mean, do an off-site backup of stuff that you can’t handle losing and in the unlikely case that it gets stolen, be prepared to replace hardware.

    If you just want to keep the hardware out of sight and create a minimal barrier, you can get locking, ventillated racks. I don’t know how cost-effective that is; I’d think that that might cost more than the expected value of the loss from theft. If a computer costs $1000 and you have a 1% chance of it being stolen, you should not spend more than $10 on prevention in terms of reducing cost of hardware loss, even if that method is 100% effective.

    EDIT: My guess is also that





  • I’d also bet against the CMOS battery, if the pre-reboot logs were off by 10 days.

    The CMOS battery is used to maintain the clock when the PC is powered off. But he has a discrepancy between current time and pre-reboot logs. He shouldn’t see that if the clock only got messed up during the power loss.

    I’d think that the time was off by 10 days prior to power loss.

    I don’t know why it’d be off by 10 days. I don’t know uptime of the system, but that seems like an implausible amount of drift for a PC RTC, from what I see online as lilely RTC drift.

    It might be that somehow, the system was set up to use some other time source, and that was off.

    It looks like chrony is using the Debian NTP pool at boot, though, and I donpt know why it’d change.

    Can DHCP serve an NTP server, maybe?

    kagis

    This says that it can, and at least when the comment was written, 12 years ago, Linux used it.

    https://superuser.com/questions/656695/which-clients-accept-dhcp-option-42-to-configure-their-ntp-servers

    The ISC DHCP client (which is used in almost any Linux distribution) and its variants accept the NTP field. There isn’t another well known/universal client that accepts this value.

    If I have to guess about why OSX nor Windows supports this option, I would say is due the various flaws that the base DHCP protocol has, like no Authentification Method, since mal intentioned DHCP servers could change your systems clocks, etc. Also, there aren’t lots of DHCP clients out there (I only know Windows and ISC-based clients), so that leave little (or no) options where to pick.

    Maybe OS X allows you to install another DHCP client, Windows isn’t so easy, but you could be sure that Linux does.

    My Debian trixie system has the ISC DHCP client installed in 2025, so might still be a factor. Maybe a consumer broadband router on your network was configured to tell the Proxmox box to use it as a NTP server or something? I mean, bit of a long shot, but nothing else that would change the NTP time source immediately comes to mind, unless you changed NTP config and didn’t restart chrony, and the power loss did it.



  • I have not done so in the traditional sense in quite some years. My experience was that it was an increasing headache due to crashing into a wide variety of anti-spam efforts. Get email past one and crash into another.

    Depending upon your use case – using the “forward to a smarthost” feature in some mail server packages to forward to a mailserver run by a SMTP service provider with whom you have an account might work for you. Then it still looks to local software like you have a local mailserver.

    If I were going to do a conventional, no-smarthost mailserver today, I think that I would probably start out by setting up a bunch of spam-filtering stuff — SpamAssassin, I dunno what-all gets used these days on a “regular” account — and then emailing stuff from my server and seeing what throws up red flags. That’d let me actually see the scoring and stuff that’s killing email. Once I had it as clean as I could get it, I’d get a variety of people I know on different mail servers and ask them to respond back to a test email, and see what made it out.