So we have a problem with $ sudo shutdown now
hanging on the line systemd-shutdown[1]: Detaching DM devices
every time we try to turn off or reboot the device. This is the output we get over the UART debug at shutdown:
[15:00:34:097] logout␍␊
[15:00:34:293] [<0x1b>[0;32m OK <0x1b>[0m] Stopped <0x1b>[0;1;39mUser Manager for UID 0<0x1b>[0m.␍␍␊
[15:00:34:293] [<0x1b>[0;32m OK <0x1b>[0m] Stopped <0x1b>[0;1;39mSession 4 of user guard<0x1b>[0m.␍␍␊
[15:00:34:293] Stopping <0x1b>[0;1;39mLogin Service<0x1b>[0m...␍␍␊
[15:00:34:293] Stopping <0x1b>[0;1;39mUser Manager for UID 1000<0x1b>[0m...␍␍␊
[15:00:34:293] Stopping <0x1b>[0;1;39mUser Runtime Directory /run/user/0<0x1b>[0m...␍␍␊
[15:00:34:293] Stopping <0x1b>[0;1;39mLSB: set CPUFreq kernel parameters<0x1b>[0m...␍␍␊
[15:00:34:293] [<0x1b>[0;32m OK <0x1b>[0m] Stopped target <0x1b>[0;1;39mSystem Time Synchronized<0x1b>[0m.␍␍␊
[15:00:34:293] [<0x1b>[0;32m OK <0x1b>[0m] Stopped target <0x1b>[0;1;39mNetwork is Online<0x1b>[0m.␍␍␊
[15:00:34:336] [<0x1b>[0;32m OK <0x1b>[0m] Unmounted <0x1b>[0;1;39m/run/user/0<0x1b>[0m.␍␍␊
[15:00:34:336] [<0x1b>[0;32m OK <0x1b>[0m] Started <0x1b>[0;1;39mShow Plymouth Power Off Screen<0x1b>[0m.␍␍␊
[15:00:34:336] [<0x1b>[0;32m OK <0x1b>[0m] Stopped <0x1b>[0;1;39mUser Runtime Directory /run/user/0<0x1b>[0m.␍␍␊
[15:00:34:336] [<0x1b>[0;32m OK <0x1b>[0m] Stopped <0x1b>[0;1;39mUser Manager for UID 1000<0x1b>[0m.␍␍␊
[15:00:34:336] Stopping <0x1b>[0;1;39mUser Runtime Directory /run/user/1000<0x1b>[0m...␍␍␊
[15:00:34:336] [<0x1b>[0;32m OK <0x1b>[0m] Removed slice <0x1b>[0;1;39mUser Slice of UID 0<0x1b>[0m.␍␍␊
[15:00:34:336] [<0x1b>[0;32m OK <0x1b>[0m] Stopped <0x1b>[0;1;39mLSB: set CPUFreq kernel parameters<0x1b>[0m.␍␍␊
[15:00:34:379] [<0x1b>[0;32m OK <0x1b>[0m] Unmounted <0x1b>[0;1;39m/run/user/1000<0x1b>[0m.␍␍␊
[15:00:34:379] [<0x1b>[0;32m OK <0x1b>[0m] Stopped <0x1b>[0;1;39mLogin Service<0x1b>[0m.␍␍␊
[15:00:34:379] [<0x1b>[0;32m OK <0x1b>[0m] Reached target <0x1b>[0;1;39mUnmount All Filesystems<0x1b>[0m.␍␍␊
[15:00:34:379] Stopping <0x1b>[0;1;39mLSB: Load kernel <0xe2><0x80><0xa6>d to enable cpufreq scaling<0x1b>[0m...␍␍␊
[15:00:34:379] [<0x1b>[0;32m OK <0x1b>[0m] Stopped <0x1b>[0;1;39mUser Runtime Directory /run/user/1000<0x1b>[0m.␍␍␊
[15:00:34:379] [<0x1b>[0;32m OK <0x1b>[0m] Removed slice <0x1b>[0;1;39mUser Slice of UID 1000<0x1b>[0m.␍␍␊
[15:00:34:379] Stopping <0x1b>[0;1;39mPermit User Sessions<0x1b>[0m...␍␍␊
[15:00:34:442] [<0x1b>[0;32m OK <0x1b>[0m] Stopped <0x1b>[0;1;39mPermit User Sessions<0x1b>[0m.␍␍␊
[15:00:34:442] [<0x1b>[0;32m OK <0x1b>[0m] Stopped <0x1b>[0;1;39mLSB: Load kernel m<0xe2><0x80><0xa6>ded to enable cpufreq scaling<0x1b>[0m.␍␍␊
[15:00:34:442] [<0x1b>[0;32m OK <0x1b>[0m] Stopped target <0x1b>[0;1;39mRemote File Systems<0x1b>[0m.␍␍␊
[15:00:34:442] [<0x1b>[0;32m OK <0x1b>[0m] Stopped target <0x1b>[0;1;39mNetwork<0x1b>[0m.␍␍␊
[15:00:34:442] Stopping <0x1b>[0;1;39mRaise network interfaces<0x1b>[0m...␍␍␊
[15:00:34:442] Stopping <0x1b>[0;1;39mifup for wlp1s0<0x1b>[0m...␍␍␊
[15:00:34:442] Stopping <0x1b>[0;1;39mNetwork Manager<0x1b>[0m...␍␍␊
[15:00:34:442] Stopping <0x1b>[0;1;39mWPA supplicant<0x1b>[0m...␍␍␊
[15:00:34:463] [<0x1b>[0;32m OK <0x1b>[0m] Stopped <0x1b>[0;1;39mWPA supplicant<0x1b>[0m.␍␍␊
[15:00:34:542] [<0x1b>[0;32m OK <0x1b>[0m] Stopped <0x1b>[0;1;39mRaise network interfaces<0x1b>[0m.␍␍␊
[15:00:34:542] [<0x1b>[0;32m OK <0x1b>[0m] Stopped <0x1b>[0;1;39mNetwork Manager<0x1b>[0m.␍␍␊
[15:00:34:542] Stopping <0x1b>[0;1;39mD-Bus System Message Bus<0x1b>[0m...␍␍␊
[15:00:34:542] [<0x1b>[0;32m OK <0x1b>[0m] Stopped <0x1b>[0;1;39mD-Bus System Message Bus<0x1b>[0m.␍␍␊
[15:00:34:542] [<0x1b>[0;32m OK <0x1b>[0m] Stopped target <0x1b>[0;1;39mBasic System<0x1b>[0m.␍␍␊
[15:00:34:542] [<0x1b>[0;32m OK <0x1b>[0m] Stopped <0x1b>[0;1;39mForward Password R<0xe2><0x80><0xa6>s to Plymouth Directory Watch<0x1b>[0m.␍␍␊
[15:00:34:542] [<0x1b>[0;32m OK <0x1b>[0m] Stopped target <0x1b>[0;1;39mSockets<0x1b>[0m.␍␍␊
[15:00:34:542] [<0x1b>[0;32m OK <0x1b>[0m] Closed <0x1b>[0;1;39mSyslog Socket<0x1b>[0m.␍␍␊
[15:00:34:585] [<0x1b>[0;32m OK <0x1b>[0m] Closed <0x1b>[0;1;39mGPS (Global Positioning System) Daemon Sockets<0x1b>[0m.␍␍␊
[15:00:34:585] [<0x1b>[0;32m OK <0x1b>[0m] Stopped target <0x1b>[0;1;39mSlices<0x1b>[0m.␍␍␊
[15:00:34:585] [<0x1b>[0;32m OK <0x1b>[0m] Removed slice <0x1b>[0;1;39mUser and Session Slice<0x1b>[0m.␍␍␊
[15:00:34:585] [<0x1b>[0;32m OK <0x1b>[0m] Stopped target <0x1b>[0;1;39mPaths<0x1b>[0m.␍␍␊
[15:00:34:585] [<0x1b>[0;32m OK <0x1b>[0m] Closed <0x1b>[0;1;39mD-Bus System Message Bus Socket<0x1b>[0m.␍␍␊
[15:00:34:585] [<0x1b>[0;32[ 593.265576] wlp1s0: deauthenticating from cc:2d:e0:e7:42:4b by local choice (Reason: 3=DEAUTH_LEAVING)␍␊
[15:00:34:628] m OK <0x1b>[0m] Stopped target <0x1b>[0;1;39mSystem Initialization<0x1b>[0m.␍␍␊
[15:00:34:628] Stopping <0x1b>[0;1;39mNetwork Time Synchronization<0x1b>[0m...␍␍␊
[15:00:34:628] [<0x1b>[0;32m OK <0x1b>[0m] Stopped target <0x1b>[0;1;39mLocal Encrypted Volumes<0x1b>[0m.␍␍␊
[15:00:34:628] [<0x1b>[0;32m OK <0x1b>[0m] Stopped <0x1b>[0;1;39mForward Password R<0xe2><0x80><0xa6>uests to Wall Directory Watch<0x1b>[0m.␍␍␊
[15:00:34:628] [<0x1b>[0;32m OK <0x1b>[0m] Stopped target <0x1b>[0;1;39mSwap<0x1b>[0m.␍␍␊
[15:00:34:628] Stopping <0x1b>[0;1;39mUpdate UTMP about System Boot/Shutdown<0x1b>[0m...␍␍␊
[15:00:34:628] Stopping <0x1b>[0;1;39mLoad/Save Random Seed<0x1b>[0m...␍␍␊
[15:00:34:665] [<0x1b>[0;32m OK <0x1b>[0m] Stopped <0x1b>[0;1;39mNetwork Time Synchronization<0x1b>[0m.␍␍␊
[15:00:34:665] [<0x1b>[0;32m OK <0x1b>[0m] Stopped <0x1b>[0;1;39mLoad/Save Random Seed<0x1b>[0m.␍␍␊
[15:00:34:665] [<0x1b>[0;32m OK <0x1b>[0m] Stopped <0x1b>[0;1;39mUpdate UTMP about System Boot/Shutdown<0x1b>[0m.␍␍␊
[15:00:34:665] [<0x1b>[0;32m OK <0x1b>[0m] Stopped <0x1b>[0;1;39mCreate Volatile Files and Directories<0x1b>[0m.␍␍␊
[15:00:34:825] [<0x1b>[0;32m OK <0x1b>[0m] Stopped <0x1b>[0;1;39mifup for wlp1s0<0x1b>[0m.␍␍␊
[15:00:34:868] [<0x1b>[0;32m OK <0x1b>[0m] Stopped target <0x1b>[0;1;39mLocal File Systems<0x1b>[0m.␍␍␊
[15:00:34:868] [<0x1b>[0;32m OK <0x1b>[0m] Stopped target <0x1b>[0;1;39mLocal File Systems (Pre)<0x1b>[0m.␍␍␊
[15:00:34:868] [<0x1b>[0;32m OK <0x1b>[0m] Stopped <0x1b>[0;1;39mCreate Static Device Nodes in /dev<0x1b>[0m.␍␍␊
[15:00:34:868] [<0x1b>[0;32m OK <0x1b>[0m] Stopped <0x1b>[0;1;39mCreate System Users<0x1b>[0m.␍␍␊
[15:00:34:868] [<0x1b>[0;32m OK <0x1b>[0m] Stopped <0x1b>[0;1;39mRemount Root and Kernel File Systems<0x1b>[0m.␍␍␊
[15:00:34:868] [<0x1b>[0;32m OK <0x1b>[0m] Stopped <0x1b>[0;1;39mApply Kernel Variables<0x1b>[0m.␍␍␊
[15:00:34:868] [<0x1b>[0;32m OK <0x1b>[0m] Stopped <0x1b>[0;1;39mLoad Kernel Modules<0x1b>[0m.␍␍␊
[15:00:34:905] [<0x1b>[0;32m OK <0x1b>[0m] Reached target <0x1b>[0;1;39mShutdown<0x1b>[0m.␍␍␊
[15:00:34:905] [<0x1b>[0;32m OK <0x1b>[0m] Reached target <0x1b>[0;1;39mFinal Step<0x1b>[0m.␍␍␊
[15:00:34:905] [<0x1b>[0;32m OK <0x1b>[0m] Started <0x1b>[0;1;39mPower-Off<0x1b>[0m.␍␍␊
[15:00:34:905] [<0x1b>[0;32m OK <0x1b>[0m] Reached target <0x1b>[0;1;39mPower-Off<0x1b>[0m.␍␍␊
[15:00:34:965] [ 593.639395] systemd-shutdow: 30 output lines suppressed due to ratelimiting␍␊
[15:00:35:006] [ 593.680082] systemd-shutdown[1]: Syncing filesystems and block devices.␍␊
[15:00:35:034] [ 593.703054] systemd-shutdown[1]: Sending SIGTERM to remaining processes...␍␊
[15:00:35:034] [ 593.708312] systemd-journald[1941]: Received SIGTERM from PID 1 (systemd-shutdow).␍␊
[15:00:35:087] [ 593.748891] systemd-shutdown[1]: Sending SIGKILL to remaining processes...␍␊
[15:00:35:087] [ 593.757580] systemd-shutdown[1]: Unmounting file systems.␍␊
[15:00:35:087] [ 593.763034] [2731]: Remounting '/' read-only in with options 'data=ordered'.␍␊
[15:00:35:127] [ 593.800996] EXT4-fs (sda10): re-mounted. Opts: data=ordered␍␊
[15:00:35:127] [ 593.820361] systemd-shutdown[1]: All filesystems unmounted.␍␊
[15:00:35:172] [ 593.820454] systemd-shutdown[1]: Deactivating swaps.␍␊
[15:00:35:172] [ 593.825390] systemd-shutdown[1]: All swaps deactivated.␍␊
[15:00:35:172] [ 593.830081] systemd-shutdown[1]: Detaching loop devices.␍␊
[15:00:35:172] [ 593.844133] systemd-shutdown[1]: All loop devices detached.␍␊
[15:00:35:172] [ 593.844174] systemd-shutdown[1]: Detaching DM devices.␍␊
This might have to do something with the fact that we created a custom rootfs image from the generated one with chroot following this guide. This doesn’t happen on the original image.
$ sudo poweroff -f
also hangs even if we wait a few minutes.
Might not be related, but it also fails to start the Qualcomm remotefs service at boot:
● rmtfs.service - Qualcomm remotefs service
Loaded: loaded (/lib/systemd/system/rmtfs.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Tue 2021-05-25 15:38:08 CEST; 5min ago
Process: 2597 ExecStart=/usr/bin/rmtfs -r -P -s (code=exited, status=1/FAILURE)
Main PID: 2597 (code=exited, status=1/FAILURE)
May 25 15:38:08 user systemd[1]: rmtfs.service: Service RestartSec=1s expired, scheduling restart.
May 25 15:38:08 user systemd[1]: rmtfs.service: Scheduled restart job, restart counter is at 5.
May 25 15:38:08 user systemd[1]: Stopped Qualcomm remotefs service.
May 25 15:38:08 user systemd[1]: rmtfs.service: Start request repeated too quickly.
May 25 15:38:08 user systemd[1]: rmtfs.service: Failed with result 'exit-code'.
May 25 15:38:08 user systemd[1]: Failed to start Qualcomm remotefs service.
Any suggestions?