Starting in v3, more info here: https://github.com/immich-app/immich/discussions/29439

Love immich, but recently have been cleaning up my server and watching for stray processor and io use. This was one thing that jumped out. Personally, I don't want my full image library checked each night and fixed it to something that made sense for me.

Because my last post about just having some beers and discussing homelabbing with dutch people from this community got more attention than expected, we have actually set up an event!

First edition of homelabbing + beers NL will be held in Utrecht on the 21st of November.
Price will be 19 euros and drinks are included. This is to prevent that i have to pay a massive bill by the end lol. More info on exact location and time can be found on discord.

If you intend to show up, please set yourself as interested for the event in our newly created discord server: https://discord.gg/qbCrzfpMU
Looking forward to seeing you guys at the meetup

My wife and I have, for many years, used and relied on Google family location sharing in Google maps, to see each other's location. it worked well for many years, while we all had Android phones.

Recently, my wife has moved to an iPhone, and it no longer works. iOS is just a **** when it comes to arbitrarily putting apps to sleep. Despite giving Google maps every single permission and set it on the background always on for location etc, it still doesn't work properly.

Sometimes, after a few days of not using, I'll try to check if she made it safely to work, and it will say last reported location like 8 hours ago.

We then tried using Home assistant for this. again, same thing happens. after a few days, location just stops being reported. With Google maps it was instant.

Many people have recommended Life360 app, but I thought I'd check for a good self-hosted one before trying yet another proprietary third party app.

I'm building a video server and keep hitting a hard freeze I can't pin down. I'd appreciate ideas from anyone who knows Arc (DG2) on Linux, VT-d, or older Xeon platforms.

Hardware:

ASUS Z10PE-D8 WS, BIOS 4301 (latest)

2× Xeon E5-2696 v4 (Broadwell-EP, 22 cores each), microcode 0xb000040

2× Sparkle Arc A310 4 GB (4×HDMI "OMNI View", 8086:56a6 rev 05) - both on CPU 2's slots, PCIe Gen3 x8 and each card has its own PCIe switch in front of the GPU (8086:4fa1), No Resizable BAR on this board (256 MiB BAR)

6 × Kingston valueram 16GB ddr4 2Rx4 PC4-2133P-RA0-11 ECC RDIMM; memtest clean, EDAC 0

Other PCIE: Intel E810 NIC and a AMD FPGA card

Cooler master 1000w Elite Gold (BRAND NEW)

Software

Ubuntu 24.04, HWE kernel 7.0 (7.0.0-34 and 7.0.0-42 both tested)

i915 driver (xe is loaded but not bound)

Firmware: GuC 70.53.0, HuC 7.10.16, DMC 2.08, GSC DG02_2.2357

intel-media-driver (iHD) 24.1.0, later 25.2.4; FFmpeg with VA-API

VT-d on (translated default domain, lazy flush-queue invalidation), interrupt remapping on (x2APIC)

Workload/operation

6 live 1080p50 encodes split across both cards: H.264, HEVC 8/10-bit, AV1 and VP9. Also tested with 8 (4 per card). Each card draws only 14–18 W, at 56–75 °C.

Issue:

The whole server basically stops at once; every network interface goes silent within 0.2 s.

No panic and no reboot, even with kernel.panic=30 and hardlockup_panic=1. My NMI status never reports.

netconsole receives nothing.

After a reset there are no MCE, BERT, EDAC or AER records, with WHEA on or off.

Its not like it turns off, the server still stays powered, fans running; the Q-Code display shows the same AA POST code so nothings different from normal operation.

The board's CATT (CATERR) LED is on and ERR_CPU is off in every freeze where we looked (10 of them). I'm thinking its something to do with the CPU IERR, which explain why the OS logs nothing? I'm very confused.

Tests:

22 freezes in about total of 113 hours of GPU-encode load. The mean is about 5 h; the range is 24 seconds to 14 hours.

I've done the same load with the encodes moved to the CPU and both GPUs idle: ive done a run for 8 hours and one 24 h run and no freeze issues with them...

The GPU encodes alone (no capture cards, no DPDK) still freeze, so im pretty sure its the A310 encodes

Tried, and it still froze:

Kernel 7.0.0-34 → 7.0.0-42, iHD 24 → 25, newer HuC

i915.enable_dc=0

Every OS power saving off: ASPM, runtime PM, D3cold, C-states deeper than C1, SATA LPM, EEE, EPB

Core C6 off, uncore clock capped, turbo off.

Assumptions went to temperature, cooled my room as much as I could the GPU was at 56 °C and still froze.

A310 links forced to Gen2: 8 h clean, then froze in the repeat

intremap=off (interrupt remapping only)

WHEA off

Encode threads and buffers moved to the GPUs' NUMA node

Could test, but haven't (I've done so much testing so I've just tried to post here if anyone has had similar issues and what was the fix):

iommu=pt (GPUs untranslated), or iommu.strict=1 (no lazy flush queue)

The xe driver instead of i915

GSC firmware 2359 (how do you update it on Linux?)

A power limit on the cards through i915's hwmon

A different PSU, or one GPU per CPU socket

My guess:

Both GPUs sit behind the same VT-d unit: the one for CPU 2's root ports, which also covers the E810. My top theory is that two GPUs' constant map/unmap traffic floods that unit with page lookups and invalidations and wedges CPU 2's I/O hub. That would give an IERR with no PCIe error logged.

Or maybe:

cross-socket posted-write credit loss (Broadwell-EP erratum BDF83);

GPU power-state transitions on a lightly loaded card while DMA is in flight.

I've had endless hours and considering just returning the csrds, not sure if return is possible with Overclockers UK ... Alternatively I'll have to have a chat with the manufacturers to get some type of support/repair

Just wondering if anyone else had these issues?

CATERR/IERR hard locks with Arc (DG2) on Broadwell-EP or other older Xeons?

Are there known issues with DG2 under VT-d translation, or with two DG2 cards behind one IOMMU unit?

Is xe on DG2 mature enough for VA-API encodes (HuC/BRC) to be worth testing?

No BMC is fitted (currently shipping from china, but not exactly sure it will be able to identify my exact issue).

Is there any way to capture CPU state at an IERR, such as reading the MCA banks after a warm reset before the BIOS clears them?

What would you try first?

I can post lspci -vvv, dmesg or the kernel config if that helps.

Tldr: I'm stuck, just like my server after any GPU encodes.

I replaced almost all services with selfhosted solutions, but I really struggle to find gmail replacements.

Recently I saw some interesting projects like plmail or mailflow. But I'm kind of reluctant due to the vibe coded nature of such projects.

Some features I'd like to have: webmail and Android app (pwa or native), usable search, snooze. Easy in and export... I'm not switching to proton to be locked in the next golden cage.

Any opinions on mail?

edit: great confusion. I'm not looking for a Mail server, just a client