Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

systemd is destructive

Here you can discuss every aspect of Debian. Note: not for support requests!
Locked
Message
Author
M51
Posts: 397
Joined: 2013-05-13 01:38

systemd is destructive

#1 Post by M51 »

I've been running linux generally, and Debian specifically for years and I have never seen a bigger cluster f&(% than systemd. I'd gotten lazy with one of my most recent setups of Debian Jessie and allowed the cancerous systemd to be installed...and regretted it.

What follows is my setup and what happened:

I have a machine set up to store recorded hdtv shows on btrfs. It runs a minimal XFCE desktop as I can't stand Gnome. An hour show is about 5GB and I have about 100 shows at any given time. The directory where the shows are stored is bind mounted to where my nfs exports are and it is exported as read only for a KODI client to view. I never had any trouble with this setup under Wheezy, but under Jessie with systemd I ran into an issue when clearing out old recordings. First, deleting them through Thunar took forever, and when deleting multiple files at the same time the weird crap started. A few of the files would get deleted, but at some point I'd get an error that the file couldn't be deleted because it was read-only. Skipping it would give the error on every subsequent file. Opening a terminal and running mount revealed that the entire root filesystem had been remounted read-only! Attempts to remount it read-write failed. Nothing relevant shows in the logs. Rebooting would begin normally but at some point (usually the point nfs exports are started) I'd see a btrfs error flash past and the system would stop booting, or it might make it all the way to a login prompt (but no X11). Logging in would reveal the entire root filesystem is irrevocably mounted read-only. Booting from a rescue disk and checking the btrfs filesystem reveals no problems. Rebooting the system repeatedly eventually will randomly allow it to make it to X11 normally like nothing ever happened! The next time I would clean multiple files off, the same thing would happen. The only recovery is a random number of reboots. One time it took nearly 20!

After that I was pissed off enough to completely remove systemd and replace it with sysvinit. Guess what? No problems ever since. I can delete any number of files without any slowdowns or bizarre remounting of my entire system as read-only without so much as a log message. My btrfs checks always come back clean.

F systemd !

User avatar
Danielsan
Posts: 659
Joined: 2010-10-10 22:36
Has thanked: 5 times

Re: systemd is destructive

#2 Post by Danielsan »

Currently systemd is f* my hp printer and I am having issue with cups because of it, I hope to resolve it soon. Just for the sake of the information with a laptop with Wheezy the printer is super fine... :roll:

deborah-and-ian
Posts: 182
Joined: 2016-07-13 08:40

Re: systemd is destructive

#3 Post by deborah-and-ian »

Is there a bug report about this?
Debian GNU/Linux 9 Stretch w/Openbox

Acer Aspire E5-521G
AMD A8-6410 APU
4 GB RAM
integrated AMD Mullins
dedicated AMD Hainan Radeon R5 M240 2 GB
240 GB Toshiba Q300 SSD
Realtek RTL8111/8168/8411 ethernet
Qualcomm Atheros QCA9565 wireless

M51
Posts: 397
Joined: 2013-05-13 01:38

Re: systemd is destructive

#4 Post by M51 »

I honestly don't care if there is a bug report. Such a non-deterministic, over-complex, 'big ball of mud' non-design isn't worth troubleshooting or fixing.

I will never make the mistake of allowing it on my machines again. Even a suggested dependency is enough to make me look for alternative software.

For anyone developing software:

Using systemd's public API is poison.

User avatar
Ardouos
Posts: 1075
Joined: 2013-11-03 00:30
Location: Elicoor II
Has thanked: 1 time
Been thanked: 4 times

Re: systemd is destructive

#5 Post by Ardouos »

M51 wrote:I honestly don't care if there is a bug report. Such a non-deterministic, over-complex, 'big ball of mud' non-design isn't worth troubleshooting or fixing.

I will never make the mistake of allowing it on my machines again. Even a suggested dependency is enough to make me look for alternative software.

For anyone developing software:

Using systemd's public API is poison.
I guess you will be keen on the release of Devuan.
There is only one Debian | Do not break Debian | Stability and Debian | Backports

⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Debian - The universal operating system
⢿⡄⠘⠷⠚⠋⠀ https://www.debian.org
⠈⠳⣄⠀

M51
Posts: 397
Joined: 2013-05-13 01:38

Re: systemd is destructive

#6 Post by M51 »

I might check out Devuan for curiosity's sake, but these days I am running more and more on my own personal distro built out of LFS (no systemd). I still use Debian for some things I haven't yet finished, but that will change soon enough.

I was lazy in allowing systemd on the machine. Since all it really had to do was host some files I figured "How could it screw that up?" Apparently the answer is: "Completely".

Bulkley
Posts: 6382
Joined: 2006-02-11 18:35
Has thanked: 2 times
Been thanked: 39 times

Re: systemd is destructive

#7 Post by Bulkley »

Systemd is not destructive as such but it does seem to be that one has to opt all in or all out.

M51
Posts: 397
Joined: 2013-05-13 01:38

Re: systemd is destructive

#8 Post by M51 »

Bulkley wrote:Systemd is not destructive as such but it does seem to be that one has to opt all in or all out.
It certainly seems destructive when it yanks your system out from under you for no good reason.

User avatar
phenest
Posts: 1702
Joined: 2010-03-09 09:38
Location: The Matrix

Re: systemd is destructive

#9 Post by phenest »

Danielsan wrote:Currently systemd is f* my hp printer and I am having issue with cups because of it, I hope to resolve it soon. Just for the sake of the information with a laptop with Wheezy the printer is super fine... :roll:
I use an HP printer via blue-tooth and it works fine.
M51 wrote:
Bulkley wrote:Systemd is not destructive as such but it does seem to be that one has to opt all in or all out.
It certainly seems destructive when it yanks your system out from under you for no good reason.
And you're certain that systemd is at fault? Despite what you say in your first post about replacing it with sysvinit (thus curing the problem) I've not seen any of the problems that anyone has ever mentioned using systemd.
ASRock H77 Pro4-M i7 3770K - 32GB RAM - Pioneer BDR-209D

M51
Posts: 397
Joined: 2013-05-13 01:38

Re: systemd is destructive

#10 Post by M51 »

phenest wrote: I use an HP printer via blue-tooth and it works fine.
Congratulations, but anecdotal "It works for me with my specific setup." doesn't help anyone and certainly doesn't preclude problems from existing. Funny, but that seems to be the general attitude of the systemd developers: "Tough luck. Works for me!"
phenest wrote: And you're certain that systemd is at fault? Despite what you say in your first post about replacing it with sysvinit (thus curing the problem) I've not seen any of the problems that anyone has ever mentioned using systemd.
No doubt it is systemd. I recreated it multiple times. Bringing back systemd brings back the problem. It could easily be that no one else has reported such a situation because no one else has my particular setup: UEFI, GPT partition, LVM2 volumes, Luks encryption, BTRFS, etc. The truth is systemd depends on f&(%ing everything, so who the hell knows what can trigger a bug in its bloated codebase. I have seen people with systemd problems where the system comes up stuck with a read-only root, but that is only part of the problem in this case.

For some technical examples of why systemd sucks:

http://ewontfix.com/14/

http://suckless.org/sucks/systemd

User avatar
dasein
Posts: 7680
Joined: 2011-03-04 01:06
Location: Terra Incantationum

Re: systemd is destructive

#11 Post by dasein »

M51 wrote:Congratulations, but anecdotal
So long as your N=1, so's yours.

(Just sayin')

M51
Posts: 397
Joined: 2013-05-13 01:38

Re: systemd is destructive

#12 Post by M51 »

dasein wrote:
M51 wrote:Congratulations, but anecdotal
So long as your N=1, so's yours.

(Just sayin')
Sure. :)

Though if you read the rest of the sentence I was saying that an anecdote doesn't preclude the possibility of an actual problem. Also, claiming "works for me" doesn't help the guy with the printer problem, seeing as phenest doesn't even know if it is the same model, interface, etc.

Could it be that my anecdotal evidence doesn't really indicate a problem with systemd? Sure, I'd grant that (remote) possibility. Plenty of people have f''d up systems. However I've tested it (on this one system, of course) pretty thoroughly and I am certain it is a problem with systemd. I also know of no other component that can remount root out from under you automatically. The fact that it does so under unknowable (without reading the source) conditions reveals one of systemd's biggest failings: It implements system policy in opaque code rather than leaving it to the system owner. That is simply 100% shitty-ass design. No reasonable developer can debate that and systemd is chock full of examples of exactly that kind of behavior. Reading the systemd changelog is some scary crap.

Using Debian with sysvinit will hold me over nicely until I can get everything moved over to my private distro.

User avatar
phenest
Posts: 1702
Joined: 2010-03-09 09:38
Location: The Matrix

Re: systemd is destructive

#13 Post by phenest »

M51 wrote:Also, claiming "works for me" doesn't help the guy with the printer problem, seeing as phenest doesn't even know if it is the same model, interface, etc.
What printer problem? That "guy" didn't specify. That "guy" just blamed systemd. How can someone give help when they don't specify what the problem is.
M51 wrote:That is simply 100% shitty-ass design. No reasonable developer can debate that and systemd is chock full of examples of exactly that kind of behavior. Reading the systemd changelog is some scary crap.
If systemd is that bad "100% shitty-ass design", should everyone be affected? If some are affected and some are not, then perhaps it's the setup that's at fault. After all, if you not satisfied with keeping Debian as the Debian devs intended, then you're asking for trouble.
ASRock H77 Pro4-M i7 3770K - 32GB RAM - Pioneer BDR-209D

User avatar
golinux
Posts: 1579
Joined: 2010-12-09 00:56
Location: not a 'buntard!
Been thanked: 1 time

Re: systemd is destructive

#14 Post by golinux »

phenest wrote:After all, if you not satisfied with keeping Debian as the Debian devs intended, then you're asking for trouble.
And here I thought Linux was about modularity and choice and doing things YOUR way. When did that change into 'do it our way if you want it to work'? Monolithic and zero choice comes to mind . . .
May the FORK be with you!

M51
Posts: 397
Joined: 2013-05-13 01:38

Re: systemd is destructive

#15 Post by M51 »

phenest wrote:
M51 wrote:Also, claiming "works for me" doesn't help the guy with the printer problem, seeing as phenest doesn't even know if it is the same model, interface, etc.
What printer problem? That "guy" didn't specify. That "guy" just blamed systemd. How can someone give help when they don't specify what the problem is.
He wasn't really asking for help. Assuming everyone who has trouble with systemd must be wrong is stupid.
phenest wrote:
M51 wrote:That is simply 100% shitty-ass design. No reasonable developer can debate that and systemd is chock full of examples of exactly that kind of behavior. Reading the systemd changelog is some scary crap.
If systemd is that bad "100% shitty-ass design", should everyone be affected?
If you really think that, then you probably ought not to comment on technical issues. Systems are complex, and systemd depends on everything including the kitchen sink when it comes to system state so it is largely non-deterministic, like Windows.
phenest wrote: If some are affected and some are not, then perhaps it's the setup that's at fault.
The systemd developers are often guilty of thinking that there is a "right way" and a "wrong way" for a system to be set up. They are typically wrong.
phenest wrote: After all, if you not satisfied with keeping Debian as the Debian devs intended, then you're asking for trouble.
LOL, that's the stupidest and funniest thing I've heard all day. Thanks! :lol:

User avatar
phenest
Posts: 1702
Joined: 2010-03-09 09:38
Location: The Matrix

Re: systemd is destructive

#16 Post by phenest »

golinux wrote:
phenest wrote:After all, if you not satisfied with keeping Debian as the Debian devs intended, then you're asking for trouble.
And here I thought Linux was about modularity and choice and doing things YOUR way. When did that change into 'do it our way if you want it to work'? Monolithic and zero choice comes to mind . . .
I'm not saying you don't have choice. But if you do change things, things could break.
M51 wrote:
phenest wrote:
M51 wrote:Also, claiming "works for me" doesn't help the guy with the printer problem, seeing as phenest doesn't even know if it is the same model, interface, etc.
What printer problem? That "guy" didn't specify. That "guy" just blamed systemd. How can someone give help when they don't specify what the problem is.
He wasn't really asking for help. Assuming everyone who has trouble with systemd must be wrong is stupid.
If he wasn't asking for help, then why did you accuse me of not being helpful?
M51 wrote:
phenest wrote:
M51 wrote:That is simply 100% shitty-ass design. No reasonable developer can debate that and systemd is chock full of examples of exactly that kind of behavior. Reading the systemd changelog is some scary crap.
If systemd is that bad "100% shitty-ass design", should everyone be affected?
If you really think that, then you probably ought not to comment on technical issues.
But calling it a "100% shitty-ass design" is technical, is it?
M51 wrote:
phenest wrote: After all, if you not satisfied with keeping Debian as the Debian devs intended, then you're asking for trouble.
LOL, that's the stupidest and funniest thing I've heard all day. Thanks! :lol:
Is that the best comment you can make? I haven't seen any evidence of you putting up a technical argument. Perhaps you could give me an example of how systemd can break, along with methodology to achieve it. If I don't see it for myself, then I can't sympathise.
I don't have a problem with systemd, because I haven't done anything to break it. That doesn't mean I haven't changed anything about my setup, it's just that so far, nothing has broken due to what I've done. The default Debian installation works for me too.
Again, help me out. What do I do to make systemd "destructive"?
ASRock H77 Pro4-M i7 3770K - 32GB RAM - Pioneer BDR-209D

robert-e
Posts: 44
Joined: 2011-12-09 21:29

Re: systemd is destructive

#17 Post by robert-e »

@golinux: Forget phenest; if he were on another forum, he would be known as a "systemd fanboy", with all that entails.
@phenest: Keep on using systemd; you deserve it.

Regards,
Bob

M51
Posts: 397
Joined: 2013-05-13 01:38

Re: systemd is destructive

#18 Post by M51 »

phenest wrote:
M51 wrote:
phenest wrote: If systemd is that bad "100% shitty-ass design", should everyone be affected?
If you really think that, then you probably ought not to comment on technical issues.
But calling it a "100% shitty-ass design" is technical, is it?
Yes...yes it is. It also seems to be beyond you.
Denying the possibility of bugs because they don't affect everyone belies an ignorance too deep to comment on.
phenest wrote:
M51 wrote:
phenest wrote: After all, if you not satisfied with keeping Debian as the Debian devs intended, then you're asking for trouble.
LOL, that's the stupidest and funniest thing I've heard all day. Thanks! :lol:
Is that the best comment you can make? I haven't seen any evidence of you putting up a technical argument. Perhaps you could give me an example of how systemd can break, along with methodology to achieve it. If I don't see it for myself, then I can't sympathise.
No, I can do much more, but you certainly aren't worth it, you tool. That you think there's a single right 'path' to follow and straying invites disaster is comical in the extreme. How do you get the courage to press keys on your keyboard?

No technical argument? Ha, I've already given you the basics...why don't you run off and do some testing? I don't care if you never see it and I certainly don't need your sympathy. But you do have mine. It must be difficult living so far below the intellectual capacity of others.
phenest wrote: I don't have a problem with systemd, because I haven't done anything to break it.
Ah yes, the famed systemd supporter rhetoric "It can't be systemd, it must be you." Sure, systemd is bug free and is made of ground unicorn infused with rainbows.
phenest wrote: Again, help me out. What do I do to make systemd "destructive"?
Just keep using it.
In fact, I insist.

pendrachken
Posts: 1394
Joined: 2007-03-04 21:10
Location: U.S.A. - WI.

Re: systemd is destructive

#19 Post by pendrachken »

M51 wrote:
dasein wrote:
I also know of no other component that can remount root out from under you automatically.

It's called the kernel. Something is going wrong with I/O and the kernel remounts the FS as RO and blocks RW remounts.

There are a few choices here:

1: kernel regression. maybe you pulled in the latest security updated kernel and it has a regression when you were lazy and let SystemD in.
2: crappy HDD controller. Either was crappy and was recoverable prior to now, or it just gave out recently.
3: dying HDD. I would take a damn close look at the S.M.A.R.T. stats on that drive / RAID array. Run both short and long tests, any failures means backup what data you want and toss the drive before it completely falls on its face.
4: combo of the above.
1 billionth place: SystemD is actually at fault. Not very likely since it doesn't have its own mount tool, just a tool for calling the system mount application.


Not that that means SystemD is great, I personally don't care one way or the other for or against it. I don't LIKE the bloat and throw everything, including the kitchen sink, into one big lump that the devs have with it; but that doesn't mean I actively hate it either.

The one thing I consider extremely stupid and short sighted is the damn binary logging, but at least I can dual log to the standard syslogs too.
fortune -o
Your love life will be... interesting.
:twisted: How did it know?

The U.S. uses the metric system too, we have tenths, hundredths and thousandths of inches :-P

M51
Posts: 397
Joined: 2013-05-13 01:38

Re: systemd is destructive

#20 Post by M51 »

pendrachken wrote:
M51 wrote:
dasein wrote:
I also know of no other component that can remount root out from under you automatically.

It's called the kernel. Something is going wrong with I/O and the kernel remounts the FS as RO and blocks RW remounts.

There are a few choices here:

1: kernel regression. maybe you pulled in the latest security updated kernel and it has a regression when you were lazy and let SystemD in.
2: crappy HDD controller. Either was crappy and was recoverable prior to now, or it just gave out recently.
3: dying HDD. I would take a damn close look at the S.M.A.R.T. stats on that drive / RAID array. Run both short and long tests, any failures means backup what data you want and toss the drive before it completely falls on its face.
4: combo of the above.
1 billionth place: SystemD is actually at fault. Not very likely since it doesn't have its own mount tool, just a tool for calling the system mount application.


Not that that means SystemD is great, I personally don't care one way or the other for or against it. I don't LIKE the bloat and throw everything, including the kitchen sink, into one big lump that the devs have with it; but that doesn't mean I actively hate it either.

The one thing I consider extremely stupid and short sighted is the damn binary logging, but at least I can dual log to the standard syslogs too.

1) I think you misunderstood, I was lazy and allowed systemd to be init when the machine was originally installed, not pulled in during an update. If it's a kernel regression it's a very interesting one because it only affects the system if systemd is acting as init.

2) Again, a hdd controller that only fails when systemd is acting as init?

3) SMART stats are fine. BTRFS checks are clean...again the problem only surfaces when systemd is acting as init.

4) See all of the above.

1 billionth place: But systemd can and will call tools to alter system state as it sees fit based on opaque policies baked into itself.

I can recreate or remove this problem repeatedly at will simply by changing from sysvinit to systemd.

Locked