Articles in Linux Journal cover the range from desktop how-tos to kernel hacking, always balanced to give both newcomers and long-term Linux users maximum enjoyment. See our Web site for an author's guide and list of upcoming topics and deadlines: http://linuxjournal.com/author/index.
It's always better to learn from someone else's mistakes than from your own. This weekend we feature Kyle Rankin and Bill Childers as they tell stories from their years as systems administrators. It's a win-win: you get to learn from their experiences, and they get to make snide comments to each other.
We also want to hear your scary server room stories. E-mail us, firstname.lastname@example.org, with yours (just a few sentences or even a few paragraphs is fine), and we'll publish every one we receive on October 31...spooky.
by Kyle Rankin and Bill Childers
Sometimes events and equipment conspire against you and your team to cause a problem. Occasionally, however, it's lack of understanding or foresight that can turn around and bite you. Unfortunately, this is a tale of where we failed to spot all the possible things that might go wrong.
by Kyle Rankin and Bill Childers
I was now at the next phase of troubleshooting: prayer. Somewhere around this time, I had my big breakthrough...
by Kyle Rankin and Bill Childers
I was suffering, badly. We had just finished an all-night switch migration on our production Storage Area Network while I was hacking up a lung fighting walking pneumonia. Even though I did my part of the all-nighter from home, I was exhausted. So when my pager went off at 9am that morning, allowing me a mere four hours of sleep, I was treading dangerously close to zombie territory...
by Kyle Rankin and Bill Childers
As much as I love working with Linux and configuring software, one major part of being a sysadmin that always has appealed to me is working with actual hardware. There's something about working with tangible, physical servers that gives my job an extra dimension and grounds it from what might otherwise be a completely abstract job even further disconnected from reality. On top of all that, when you get a large shipment of servers, and you view the servers at your company as your servers, there is a similar anticipation and excitement when you open a server box as when you open Christmas presents at home. This story so happens to start during the Christmas season...
News briefs September 21, 2018.
Purism yesterday launched Librem Key, the "first and only OpenPGP smart card providing a Heads-firmware-integrated tamper-evident boot process". The Librem key is the size of an average thumb drive, allows you to keep your secret encryption keys in your pocket, and it alerts you if anyone tampers with your kernel or BIOS while you're away from your laptop. The key works with all laptops but has extended features with Purism's Librem laptop line. You can order one from here for $59. See also Kyle Rankin's post for more details on the Librem key.
The Mir team announces the milestone release of the Mir 1.0 display server today. This release is "targeted at IoT device makers and enthusiasts looking to build thenext-generation of graphical solutions". Mir's goal is to "unify the graphical environment across all devices, including desktop, TV, and mobile devices and continues to be developed with new features and modern standards". See the Mir website for more information.
Solus 3 ISO Refresh was released yesterday. This refresh of the operating system designed for home computing "enables support for a variety of new hardware released since Solus 3, introduces an updated set of default applications and theming, as well as enables users to immediately take advantage of new Solus infrastructure". You can download Solus Budgie, Solus GNOME or Solus MATE from here.
A new botnet in the "Malware as a Service" arena has been discovered that touts "Android-based payloads to potential cybercriminals". The botnet was developed by a Russian-speaking group called "The Lucy Game", which already has provided demos for potential subscribers. See ZDNet for more details.
New install ISO images of Sparky 5.5 "Nibiru", which is based on Debian testing "Buster", are now available for download. Changes include Linux kernel 4.18.6, Calamares installer updated to v. 3.2.1, GCC 8 is now the default and much more. You can download new ISO images from here.
Nitrux is a Linux distribution with a focus on portable, application formats like AppImages. Nitrux uses KDE Plasma 5 and KDE Applications, and it also uses our in-house software suite Nomad Desktop.
Well, just about anything! You can surf the internet, word-process, send email, create spreadsheets, listen to music, watch movies, chat, play games, code, do photo editing, create content—whatever you want!
Nitrux's main feature is the Nomad Desktop, which aims to extend Plasma to suit new users without compromising its power and flexibility for experts. Nomad's features:
Nitrux is a complete operating system that ships the essential apps and services for daily use: office applications, PDF reader, image editor, music and video players and so on. We also include non-KDE or Qt applications like Chromium and LibreOffice that together create a friendly user experience.
Nitrux includes a selection of applications carefully chosen to perform the best when using your computer:
The NX Software Center is a free application that provides Linux users with a modern and easy way to manage the software installed on their open-source operating systems. Its features allow you to search, install and manage AppImages. AppImages are faster to install, easier to create and safer to run. AppImages aim to work on any distribution or device, from IoT devices to servers, desktops and mobile devices.
Figure 1. The Nomad Software Center
News briefs for September 20, 2018.
Canonical yesterday announced the Extended Security Maintenance for Ubuntu 14.04 LTS "Trusty Tahr", which means critical and important security patches will be available beyond the Ubuntu 14.04 end-of-life date (April 2019).
Mozilla to hold a high-level panel discussion on "the future of advertising in an open and sustainable internet ecosystem" at the 40th annual International Conference of Data Protection and Privacy Conference in Brussels, Belgium October 22–26, 2018. The discussion is titled "Online advertising is broken: Can ethics fix it?", and it's scheduled for October 23, 2018.
Attackers stole credit-card information from Newegg by injecting 15 lines of skimming code on the online payments page, which remained undetected from August 14th to September 18, 2018, TechCrunch reports. Yonathan Klijnsma, threat researcher at RiskIQ, told TechCrunch that "These attacks are not confined to certain geolocations or specific industries—any organization that processes payments online is a target." If you entered your credit-card data during that period, contact your bank immediately.
MetaCase this morning announced the launch of MetaEdit+ 5.5 for Linux, which brings collaborated models to Git and other version control systems. It's "aimed at expert developers looking to gain productivity and quality by generating tight code directly from domain-specific models". You can download a free trial from here.
MariaDB has acquired Clustrix, the "pioneer in distributed database technology". According to the press release, this acquisition gives "MariaDB's open source database the scalability and high-availability that rivals or exceeds Oracle and Amazon while foregoing the need for expensive computing platforms or high licensing fees."
Recently while refreshing my memory on the use of Bash's coproc feature, I came across a reference to a pitfall that described what I thought was some quite unexpected behavior. This post describes my quick investigation of the pitfall and suggests a workaround (although I don't really recommend using it).
News briefs for September 19, 2018.
Ampere, in partnership with Lenovo, announced availability of the Ampere eMAG for hyperscale cloud computing. The first-generation Armv8-A 64-bit processors provide "high-performance compute, high memory capacity, and rich I/O to address cloud workloads including big data, web tier and in-memory databases". Pricing is 32 cores at up to 3.3GHz Turbo for $850 or 16 cores at up to 3.3GHz Turbo for $550.
LLVM 7.0.0 is out. This release is the result of six months of work by the community and includes "function multiversioning in Clang with the 'target' attribute for ELF-based x86/x86_64 targets, improved PCH support in clang-cl, preliminary DWARF v5 support, basic support for OpenMP 4.5 offloading to NVPTX, OpenCL C++ support, MSan, X-Ray and libFuzzer support for FreeBSD, early UBSan, X-Ray and libFuzzer support for OpenBSD, UBSan checks for implicit conversions, many long-tail compatibility issues fixed in lld which is now production ready for ELF, COFF and MinGW, new tools llvm-exegesis, llvm-mca and diagtool." See the release notes for details, and go here to download.
Alibaba Cloud and MariaDB announce AsparaDB RDS for MariaDB TX, which is "the first public cloud to incorporate the enterprise version of MariaDB and provide customer support directly from the two companies. ApsaraDB RDS for MariaDB TX provides Alibaba Cloud customers the latest database innovations and most secure enterprise solution for mission-critical transactional workloads." See the press release for more information.
Unit 42 researchers have discovered a new malware family called Xbash, which they have connected to the Iron Group, that targets Linux and Microsoft Windows severs. Besides ransomware and coin-mining capabilities, "Xbash also has self-propagating capabilities (meaning it has worm-like characteristics similar to WannaCry or Petya/NotPetya). It also has capabilities not currently implemented that, when implemented, could enable it to spread very quickly within an organizations' network (again, much like WannaCry or Petya/NotPetya)." See the Palo Alto Networks post for more details on the attack and how to protect your servers.
Kong Inc. yesterday announced the launch of Kong 1.0, the "only open-source API purpose built for microservices, cloud native and server less architectures". According to the press release, Kong 1.0 is feature-complete: "it combines sub-millisecond low latency, linear scalability and unparalleled flexibility with a robust feature set, support for service mesh patterns, Kubernetes Ingress controller and backward compatibility between versions." See also the Kong GitHub page.
The Linux kernel config system, Kconfig, uses a macro language very similar to the make build tool's macro language. There are a few differences, however. And of course, make is designed as a general-purpose build tool while Kconfig is Linux-kernel-specific. But, why would the kernel developers create a whole new macro language so closely resembling that of an existing general-purpose tool?
One reason became clear recently when Linus Torvalds asked developers to add an entirely new system of dependency checks to the Kconfig language, specifically testing the capabilities of the GCC compiler.
It's actually an important issue. The Linux kernel wants to support as many versions of GCC as possible—so long as doing so would not require too much insanity in the kernel code itself—but different versions of GCC support different features. The GCC developers always are tweaking and adjusting, and GCC releases also sometimes have bugs that need to be worked around. Some Linux kernel features can only be built using one version of the compiler or another. And, some features build better or faster if they can take advantage of various GCC features that exist only in certain versions.
Up until this year, the kernel build system has had to check all those compiler features by hand, using many hacky methods. The art of probing a tool to find out if it supports a given feature dates back decades and is filled with insanity. Imagine giving a command that you know will fail, but giving it anyway because the specific manner of failure will tell you what you need to know for a future command to work. Now imagine hundreds of hacks like that in the Linux kernel build system.
Part of the problem with having those hacky checks in the build system is that you find out about them only during the build—not during configuration. But since some kernel features require certain GCC versions, the proper place to learn about the GCC version is at config time. If the user's compiler doesn't support a given feature, there's no reason to show that feature in the config system. It should just silently not exist.
Linus requested that developers migrate those checks into the Kconfig system and regularize them into the macro language itself. This way, kernel features with particular GCC dependencies could identify those dependencies and then show up or not show up at config time, according to whether those dependencies had been met.
That's the reason simply using
make wouldn't work. The config language had
to represent the results of all those ugly hacks in a friendly way that
developers could make use of.
News briefs for September 18, 2018.
Following Linus Torvalds' apology for his behavior, the Linux Community has announced it will adopt a "Code of Conduct", which pledges to make "participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, religion, or sexual identity and orientation."
Mozilla announced this morning that its new Firefox Reality browser, "designed from the ground up to work on stand-alone virtual and augmented reality (or mixed reality) headsets", is now available in the Viveport, Oculus and Daydream app stores. See the Mozilla blog for more information, how to participate and download links.
The new game Lamplight City, "a steampunk-ish detective adventure" was released recently for Linux, Windows and macOS. See the Steam store for more info and to purchase.
It's now much easier to run Ubuntu VMs on Windows 10 via Hyper-V Quick Create. According to ZDNet, Canonical and Microsoft partnered to release "an optimized Ubuntu Desktop image that's available through Microsoft's Hyper-V Gallery".
In any programming language, idioms may be used that may not seem obvious from reading the manual. Often these usages of the language represent ways to make your code more compact (as in requiring fewer lines of code). Of course, some will eschew these idioms believing they represent bad style. Style, of course, is in the eyes of beholder, and this article is not intended as an exercise in defining good or bad style. So for those who may be tempted to comment on the grounds of style I would (re)direct your attention to /dev/null.
Grepping is awesome, as long as you don't glob it up! This article covers some grep and regex basics.
There are generally two types of coffee drinkers. The first type buys a can of pre-ground beans and uses the included scoop to make their automatic drip coffee in the morning. The second type picks single-origin beans from various parts of the world, accepts only beans that have been roasted within the past week and grinds those beans with a conical burr grinder moments before brewing in any number of complicated methods. Text searching is a bit like that.
For most things on the command line, people think of *.* or *.txt and are happy to use file globbing to select the files they want. When it comes to grepping a log file, however, you need to get a little fancier. The confusing part is when the syntax of globbing and regex overlap. Thankfully, it's not hard to figure out when to use which construct.
The command shell uses globbing for filename completion. If you type
ls *.txt, you'll get a list of all the files that end in
.txt in the current directory. If you do
ls R*.txt, you'll get all the
files that start with capital R and have the .txt extension. The asterisk
is a wild card that lets you quickly filter which files you mean.
You also can use a question mark in globbing if you want to specify a
single character. So, typing
ls read??.txt will list readme.txt, but not
read.txt. That's different from
ls read*.txt, which will match both
readme.txt and read.txt, because the asterisk means "zero or more
characters" in the file glob.
Here's the easy way to remember if you're using globbing (which is very simple) vs. regular expressions: globbing is done to filenames by the shell, and regex is used for searching text. The only frustrating exception to this is that sometimes the shell is too smart and conveniently does globbing when you don't want it to—for example:
grep file* README.TXT
In most cases, this will search the file README.TXT looking for the regular
file*, which is what you normally want. But if there happens to
be a file in the current folder that matches the
file* glob (let's say
filename.txt), the shell will assume you meant to pass that to
grep, and so
grep actually will see:
grep filename.txt README.TXT
Gee, thank you so much Mr. Shell, but that's not what I wanted to do. For
that reason, I recommend always using quotation marks when using
of the time you won't get an accidental glob match, but that 1% can be
infuriating. So when using
grep, this is much safer:
grep "file*" README.TXT
Because even if there is a filename.txt, the shell won't substitute it automatically.