Any Elasticsearch gurus here? I have a box with too many young gen GCs (one per 2 or 3 seconds), and irregular, very long old gen GCs (One per several hours, taking around a minute and freeing about 2/3's of the old gen space) -- I was thinking changing the new gen ratio from 2/3 to something like 3/4 or 4/5.

However, after reading an elastic article about settings to never touch... I'm no longer so sure...

Only other option I was considering is going from CMS to G1GC to cut back on the old gen GC time... A minute long downtime for Elastic is rather problematic.

Any thoughts? The box is rather old - running Elastic 5.6 with 20 GBs of heap, 207 shards and 306k docs.

There are 2 kinds of websites:
1 - The bad kind where not accepting their cookies boots you off the site (And so are in breach of GDPR... IIRC)
2 - Sites that continue working, albeit in a degraded / suboptimal state, even when you refuse their cookies.

I wish more sites were of the second variety. I'm even the only person among my friends who actually bothers going through the consent forms and disallowing everything marketing-related.

OneTrust is good. It at least remembers my preferences.

Lemme just say... Wow. Wireguard... It's so incredibly simple and elegant. I cannot believe how easy and how little reading it needed to set it up.

And unlike OpenVPN, the Android client is even able to override the system's DNS servers, meaning I can finally start blocking nosy apps from contacting their big brothers in the cloud via DNS blackholing!

Wow. Wireguard... 10/10. Simple, fast to set up, elegant.

So, despite being pretty experienced with Linux server management, today, I failed, even after hours spent tinkering, to get Bumblebee working on an older laptop of mine (Intel i3 + Geforce 960m).

What's funnier is that before I wiped that laptop with a clean install, it was working, albeit it on an out of date kernel / driver combo.

Though curiously, despite using the newest release of Xubuntu, the Bumblebee PPA repo wasn't signed (Missing InRelease file), and further lacked one of the Package index files (For i386 i believe)

I'm about to sell the laptop tomorrow. Anyone has any hints or things I could have missed? I still have a day to work on it, and if I don't manage, I'll just put on a clean win install...

After reading through an article by EFF about Google's new FLoC technology, I finally woke up and realized that Google isn't any different from the other huge organizations, and that it isn't here to fight for greater user privacy and rights... As such, I made a switch, going from Google to DuckDuckGo.

Sure, I still use YT, but other than that, I don't have a gmail address that'd be of importance, don't use their search engine, and now...

Got any tips on how to minimize tracking potential of an android-based rooted smartphone? (Stock rom, Xiaomi Mi 10T Pro). I know using a custom rom is an option, however, Lineage isn't officially supported for this model of phone, and I still value functionality over anything else, so... Unofficial daily drive OS is sub-optimal.

Alternative hardware solutions like PinePhone, although nice in theory, are way too behind the commercial brands in terms of technology and features, so that's a no go.

Okay, yes, modsecurity WAF is amazing and all, but... When one tries to implement its rules atop an existing app that wasn't developed in accordance to the rules... That hurts.

How tf am I supposed to parse and present a 6.5GB / 22M line audit log to the client?! Just parsing that monstrosity once takes *minutes*, let alone doing any sort of sorting / analysis!

I feel sick. This is exactly why I am a sysadmin and not a programmer, I don't like writing analysis stuff, or programs more complex than a few hundred lines of bash... :|

It makes me want to cry in frustration that I... actually love SystemD, as an *init* system. But with all the crap it brings along with that core part, it just makes it so much harder for me to really enjoy! Why can't it be modular? Why can't it be broken down into independently-installable packages, with the init system as a core? Is there some sort of internal API issue? Or does mister Pottering just does not want that to happen? The Linux world has always stayed by the idea of "1 package = 1 task", and it made the system management so much easier!

But now... When I switch to SystemD from SysVInit, I get... What SysVInit did + so much more I didn't ask for... I just... Don't understand it.

Not fair! How come Apache gets its modsecurity module pre-compiled and available in public repositories, but Nginx hides it behind a paywall of their "Plus" package/version?

Ugh... At least they provide the module's source, so one can compile and deploy it by himself...

I think I finally reached a point where I Have to completely reinstall my RPi.

Running Raspbian, I was under the impression their kernel releases worked the same way a pure Debian release worked - That the kernel was somewhere in the system repository.

Turns out it was, but in a different pool. And also turns out the new kernel and initramfs won't fit into my /boot as, for some reason, it is under 50 MBs in size. I dunno why, but I don't have any unallocated space left to grow the partition...

I have no idea why the boot is so small (Probably because, when I was setting the system up, I wasn't really that good with Linux yet, and just went with defaults).

What do you guys think - Is it better to run the native Raspbian system (Formerly RaspberryPi OS), or go with a pureblood Debian for Arm? (Yes, I already checked, my HW revision [3B+] is already compatible])

>Client complains about a 30 minute downtime around midnight
>Client also pays only for a single VM on a HV that they don't even own themselves

>Replies with an offer of how to make the setup more resilient, going from 1 VM to 2 LBs/FE loadbalanced through BGP, and distributing traffic through HaProxy onto 2 BE machines that in turn talk to a Postgres Cluster with RepMgr for dynamic failover.

>No reply so far

Hmmm :^)

So, none other than the father of our beloved Linux kernel - Linus Torvalds, just totally put an antivax guy down in the public kernel mailing list.

I think I love Linus even more now. He may not be a people person, but he sure does know how to totally rip people into shreds lol.

https://lore.kernel.org/ksummit/...

Okay, so, I have a functional snort agent instance, and it's spewing out alerts in it's "brilliant" unified2 log format.

I'm able to dump the log contents using the "u2spewfoo" utility (wtf even is that name lol... Unified2... something foo) but... It gives me... data. With no actual hint as to *what* rule made it log this. What is it that it found?

All I see are IDs and numbers and timings and stuff... How do I get this

(Event)
sensor id: 0 event id: 5540 event second: 1621329398 event microsecond: 388969
sig id: 366 gen id: 1 revision: 7 classification: 29
priority: 3 ip source: *src-ip* ip destination: *my-ip*
src port: 8 dest port: 0 protocol: 1 impact_flag: 0 blocked: 0
mpls label: 0 vland id: 0 policy id: 0

into information like "SYN flood from src-ip to destination-ip"

So... I got a simple task of choosing the best fitting NIDS/MIDS, as well as deploying it, configuring to fit a specific use case and monitor its outputs for one client at work today...

I'm a little... Anxious. At a first glance, setting up like... Snort... Doesn't seem all that difficult, but I have no idea where this takes me and if what I come up with will ultimately be useful or not... Until now I did simple service configuration changes like apache, nginx, php... And a bit of database management with things like mariadb, mysql, postgresql, mongo or elastic... I feel so... Out of my usual waters.

Do you guys thing a person without a title in network security (or... Any title for that matter) can even manage this?...

I seriously love rsync. Whoever made that utility is my hero. Not only that its CLI client is amazing and full of features, but rsync in daemon mode makes secure file synchronization a breeze! <3

Can someone, anyone, explain to me, how can Microsoft get away with *charging extra* for additional concurrent RDP sessions on a self-hosted instance of Windows Server?

And not only that, but apparently also charges extra once the box gets over a certain amount of system users, too.

As a Linux admin that's used to working in teams over SSH, it just completely baffles me.

It would be terrible if such a practice was in free software... But a system, that one already *pays* for to run?

Or did I understand something wrong from a colleague that claims that this is the reason why I can't get an account on one of our Windows Servers?

It is currently 5:30 AM, I've been trying to upgrade a server by one single major OS release for the last 3.5 hours. All major apps were no issue, except one.

Redmine.

That thing runs on Ruby.

Ruby, as it turns out, doesn't really like the way Debian handles package management.

And now, I have Redmine that, even if I completely uninstall / reinstall, won't start

I went down several rabbit holes, trying to operatively find what the issue is. But I never got to the key issue.

Fuck ruby. Fuck Redmine. Fuck nothing-saying error messages. Fuck bundle. Fuck gem. Fuck it all.

I'm redirecting the Redmine domain onto the server backup I've made. Upgrading that thing is a nightmare.

Maybe now I can finally go to bed...

You know your software's documentation sucks when it forces its users go snoop through the source code to find out how to set it up...

After two years of being in (metaphorical) jail, I once again was given the a privilege of unlocking and rooting my phone. Damn. Frick Huawei, never coming back to that experience.

I gotta say, rooting... Feels a tad less accessible nowadays than when I last practiced it. All this boot image backup, patch, copy, reflash is crying to be automised, only reason I can think of why that changed and magisk can no longer patch itself into the phone's initrd is that it's somehow locked? Was it a security concern? Or can sideloaded twrp no longer do that?

Oh, and the war... The war never changes, only exploits do - fruck safety net... Good for Google that they now have an *almost* unfoolable solution (almost). The new hardware-based check is annoying af, but luckily, can still be forced to downgrade back to the old basic check that can be fooled... Still, am I the only one who feels Google is kinda weird? On one hand, they support unlocking of their own brand of phones, but then they continuously try to come up with frameworks to make life with a rooted or unlocked phone more annoying...

On the other hand, I do like having my data encrypted in a way that even sideloading twrp doesn't give full access to all my stuff, including password manager cache...

Any recommendations what to install? I do love the basic tools like adaway (rip ads), greenify (yay battery life!), viper4android (More music out of my music!) and quite honestly even lucky patcher for apps where the dev studio practices disgust me and don't make me want to support them...

>Finds an URL that causes some sort of internal bug in a client's webapp
>Subsequent requests fill up the server's PHP-FPM slots, waiting for a session exclusive lock that never comes
>Effectively DoS's the server
>Sends it to a colleague to discuss the possible causes
>Uses slack
>Forgets Slack happily indexes any link it's given
>Slack almost DoS the service

FUN

That awkward feeling when you try to make an easy to pick up and use UI and fail so horrendously, that even a person otherwise skilled in computer management fails to grasp it...

I'm looking at you Synology and your fancy DSM bullshit that I just spent 2 hours trying to make available on WAN.

I almost gave up... Then realised I can log onto there through SSH, sudo su onto root and check out the webserver configuration (nginx) manually to make heads and tails of how to use it!

God... Its just tuesday, and I already feel like I need a shot of something strong...

Anytime I operate with hardware RAIDs on prod servers, I still sweat from the nerves of sending a wrong command that would wipe the RAID metadata clean and make all the data disappear.

Doesn't help that the CLI tools (MegaCli / StorCli) are both kinda terrible. The prior has a terrible documentation / switch design and the latter cannot do everything the prior can...

In last episode of "How SystemD screwed me over", we talked about Systemd's PrivateTMP and how it stopped me from generating SSL certificates.

In today's episode - SystemD vs CGroups!

Mister Pottering and his team apparently felt that CGroups are underused (As they can be quite difficult to set up), and so decided to integrate them into SystemD by default. As well as to provide a friendlier interface to control their values.

One can read about these interactions in the manual page "systemd.resource-control"

All is cool so far. So what happened to me today?

Imagine you did a major system release upgrade of a production server, previously tested on a standalone server. This upgrade doesn't only upgrade the distribution however, it also includes the switch from SysVInit to SystemD. Still, everything went smooth before, nothing to worry now then, right? Wrong.

The test server was never properly stress-tested. This would prove to be an issue.

When the upgrade finishes, it is 4 AM. I am happy to go to bed at last. At 6 AM, however, I am woken up again as the server's webservices are unavailable, and the machine is under 100% CPU load. Weird, I check htop and see that Apache now eats up all 32 virtual cores. So I restart it, casting it off to some weird bug or something as the load returns to normal.

2 hours later, however, the same situation occurs. This time, I scour all the logs I can, and find something weird - Many mentions that Apache couldn't create a worker thread? That's weird.

Several hours of research and tinkering later, I found out the following:
1 - By default, all processes of a system that runs SystemD are part of several CGroups. One of these CGroups is the PID CGroup, meant to stop a runaway process from exhausting all PIDs/TIDs of a system.

This limit is, by default, set to a certain amount of the total available PIDs. If a process exhausts this limit, it can no longer perform operations like fork().

So now, I know the how and why, but how should I solve this? The sanest option would be to get a rough estimate of just how many threads the Apache webserver might need. This option, though, is harder, than apparent. I cannot just take the MaxRequestsWorkers number... The instance has roughly double the amount of threads already. The cause being, as I found out, the HTTP/2 module, which spawns additional threads that do not count towards this limit. So I have no idea what limit to set.

Or I could... Disable the limit for just the webserver via the TasksAccounting switch. I thought this would work. And it did seem to... Until I ran out of TIDs again - Although systemctl status apache2.service no longer reported the number of tasks or a task limit of the process, the PID CGroup stayed set to the previous limit. Later I found out that I can only really disable the Task Accounting for all the units of a given slice and its parents.

This, though, systemctl somewhat didn't make apparent (And I skimmed the manual, that part was my fault)

So... The only remaining option I had was to... Just set the limit to infinite. And that worked, at last.

It took me several hours to debug this issue. And I once again feel like uninstalling systemd again, in favor of sysvinit.

What did I learn? RTFM, carefully, everything is important, it is not enough to read *half* the paragraph of a given configuration option...

Oh, and apache + http/2 = huge TID sink.

Dear MyISAM,

please die a painful and horrible death already, you living corpse of the times long gone. You're taking way too long.

Thank you,
signed, myself.

((Seriously, MyISAM is so bad, yet so many people still use it because they don't know better))

It's not everyday that I give Microsoft praise, but damn, the new Windows Terminal is... Surprisingly decent.

Together with WSL2, it allowed me to switch from working in a VM to working fully from Windows.

And with little tweaking of the settings file, it acts exactly the way I like.

Good job creating something modern, almost universal and usable Microsoft!

mod-php is weird and should never have existed.

I hate having to deal with it, even if it's only still in use in years old legacy systems. FPM is so much nicer.

You know a server is having a jolly'ol time when, while logging through the serial console, it lags... Then, a few seconds later, you get a message

[time.seconds] Out of memory: Kill process PID (login) score 0 or sacrifice child
[time.seconds] Killed process PID (login) total-vm:65400kB, anon-rss:488kB, file-rss:0kB

10/10, only way to bring the server back to life was by a hard-reset :|

WHAT. THE. FUCK.

Fucking UCEPROTECT blacklist, who the hell blacklists a whole fucking ASN when they detect even a large amount of spam coming from it? For all they know, it could be just a couple of IPs. But nooooo, instead of blacklisting IPs, they blacklist the whole ASN, so now, even some of our machines are on the list, without us ever doing anything. Just because the IP is from the DigitalOcean prefix. UGH.

Am I the only one who hates when I enter a simple question like "PHP memory limit" and the first link *isn't* the official PHP documentation? Who gives a flip about some fancy third-party webpage where they write a whole flippin' article about a simple directive?

Ugh... The priority Google...

You know your cmdline utility sucks when you have to publish a cheat sheet yourself, too, along the manual.

I'm looking at you, Broadcom, and that horrible MegaCLI raid management utility. Storcli is superior.

https://broadcom.com/support/...

A client asked us today to disable TLS 1.0 and 1.1 across their servers.

Its not often that I say this. But this makes me proud. It's a good client. Going with the changin times. I wish all clients were like this one.

RIP TLS 1.0/1.1, took you long enough.