Articles in Linux Journal cover the range from desktop how-tos to kernel hacking, always balanced to give both newcomers and long-term Linux users maximum enjoyment. See our Web site for an author's guide and list of upcoming topics and deadlines: http://linuxjournal.com/author/index.
News briefs for November 13, 2018.
Red Hat this morning released Red Hat OpenStack Platform 14, delivering "enhanced Kubernetes integration, bare metal management and additional automation". According to the press release, it will be available in the coming weeks via the Red Hat Customer Portal and as a component of both Red Hat Cloud Infrastructure and Red Hat Cloud Suite.
Red Hat also announced a new virtual office solution today. This solution "provides a blueprint for modernizing telecommunications operations at the network edge via an open, software-defined infrastructure platform". Learn more about it here.
ownCloud yesterday announced SUSE Enterprise Storage Ceph/S3 API as a certified storage backend for ownCloud Enterprise Edition. The press release notes that the "SUSE Ceph/S3 Storage integration reduces dependency on proprietary hardware by replacing an organization's storage infrastructure with an open, unified and smarter software-defined storage solution". For more information on ownCloud, visit here.
There's a new project called iSH that lets you run a Linux shell on an iOS device. Bleeping Computer reports that the project is available as a TestFlight beta for iOS devices, and it is based on Alpine Linux. It allows you to "transfer files, write shell scripts, or simply to use Vi to develop code or edit files". You first need to install the TestFlight app, and then you can start testing the app by visiting this page: https://testflight.apple.com/join/97i7KM8O.
The Firefox Test Pilot Team announces two new features: Price Wise and Email Tabs. Price Wise lets you add products to your Price Watcher list, and you'll receive desktop notifications whenever the price drops. With Email Tabs, you can "select and send links to one or many open tabs all within Firefox in a few short steps, making it easier than ever to share your holiday gift list, Thanksgiving recipes or just about anything else". See the Mozilla Blog for details.
Using Python's os.walk function to walk through a tree of files and directories.
I'm a web guy; I put together my first site in early 1993. And so, when I started to do Python training, I assumed that most of my students also were going to be web developers or aspiring web developers. Nothing could be further from the truth. Although some of my students certainly are interested in web applications, the majority of them are software engineers, testers, data scientists and system administrators.
This last group, the system administrators, usually comes into my course with the same story. The company they work for has been writing Bash scripts for several years, but they want to move to a higher-level language with greater expressiveness and a large number of third-party add-ons. (No offense to Bash users is intended; you can do amazing things with Bash, but I hope you'll agree that the scripts can become unwieldy and hard to maintain.)
It turns out that with a few simple tools and ideas, these system administrators can use Python to do more with less code, as well as create reports and maintain servers. So in this article, I describe one particularly useful tool that's often overlooked: os.walk, a function that lets you walk through a tree of files and directories.
Linux users are used to the
ls command to get a list of files in a
directory. Python comes with two different functions that can return
the list of files. One is
os.listdir, which means the "listdir"
function in the "os" package. If you want, you can pass the name of a
os.listdir. If you don't do that, you'll get the names
of files in the current directory. So, you can say:
In : import os
When I do that on my computer, in the current directory, I get the following:
In : os.listdir('.') Out: ['.git', '.gitignore', '.ipynb_checkpoints', '.mypy_cache', 'Archive', 'Files']
As you can see,
os.listdir returns a list of strings, with each
string being a filename. Of course, in UNIX-type systems, directories
are files too—so along with files, you'll also see subdirectories
without any obvious indication of which is which.
I gave up on
os.listdir long ago, in favor of
glob.glob, which means
the "glob" function in the "glob" module. Command-line users are used
to using "globbing", although they often don't know its name. Globbing
means using the * and ? characters, among others, for more flexible
matching of filenames. Although
os.listdir can return the list of
files in a directory, it cannot filter them. You can though with
News briefs for November 12, 2018.
The Fedora team announces that the latest version of the Nest simulator is now available in Fedora as part of the NeuroFedora initiative. Nest allows computational neuroscientists to "make large scale computer models of the brain that are needed to investigate among other things, how the brain processes information". Nest provides an easy to use Python interface and it can be run on both laptops and supercomputing clusters.
Cloudflare's 18.104.22.168 DNS service comes to Android and iOS. According to The Verge, "The mobile app uses features like VPN support to push your mobile traffic towards the 22.214.171.124 DNS servers and improve speeds. It will also prevent your carrier from tracking your browsing history and potentially selling it. Cloudflare is promising not to track 126.96.36.199 users or sell ads, and the company has retained KPMG to perform an annual audit and publish a public report." You can download it for Android here.
The Ceph storage project receives a dedicated open-source foundation, hosted by The Linux Foundation. TechCrunch quotes Sage Weil, Ceph's co-creator, project leader, and chief architect at Red Hat for Ceph: "Today's launch of the Ceph Foundation is a testament to the strength of a diverse open source community coming together to address the explosive growth in data storage and services."
Valve appears to be making its own VR headset. GamingOnLinux reports that a leaked imgur album shows several photos of the new hardware with a Valve logo. Valve also is apparently working on new Half-Life title for VR.
Sparky Linux 4.9 has been released, which celebrates 100 years of Poland's independence. Sparky 4.9 offers the LXDE desktop environment and minimal images of MinimalGUI (Openbox) and MinimalCLI (text mode), so you can "install the base system with a desktop of your choice with a minimal set of applications, via the Sparky Advanced Installer". In addition to added packages and updates, this new version has the code name "100", commemorating the 100 anniversary of Poland's independence, and it provides information about Polish history and also includes new Poland nature wallpapers.
Python is easy to use, powerful, versatile and a Linux Journal reader favorite. We've round up some of the most popular recent Python-related articles for your weekend reading.
Introducing PyInstaller by Reuven M. Lerner: Want to distribute Python programs to your Python-less clients? PyInstaller is the answer.
Bytes, Characters and Python 2 by Reuven M. Lerner: Moving from Python 2 to 3? Here's what you need to know about strings and their role in in your upgrade.
Introducing Python 3.7's Dataclasses by Reuven M. Lerner: Python 3.7's dataclasses reduce repetition in your class definitions.
Examining Data Using Pandas by Reuven M. Lerner: You don't need to be a data scientist to use Pandas for some basic analysis.
Multiprocessing in Python by Reuven M. Lerner: Python's "multiprocessing" module feels like threads, but actually launches processes.
Launching External Processes in Python by Reuven M. Lerner: Think it's complex to connect your Python program to the UNIX shell? Think again!
Thinking Concurrently: How Modern Network Applications Handle Multiple Connections by Reuven M. Lerner: exploring different types of multiprocessing and looks at the advantages and disadvantages of each.
Threading in Python by Reuven M. Lerner: threads can provide concurrency, even if they're not truly parallel.
Using Python for Science by Joey Bernard: introducing Anaconda, a Python distribution for scientific research.
Visualizing Molecules with Python by Joey Bernard: introducing PyMOL, a Python package for studying chemical structures.
Novelty and Outlier Detection by Reuven M. Lerner: we look at a number of ways you can try to identify outliers using the tools and libraries that Python provides for working with data: NumPy, Pandas and scikit-learn.
News briefs for November 9, 2018.
System76 announces it will donate a portion of its profits from laptop sales to open-source projects until January 3, 2019. Projects include KiCad, Electronic Frontier Foundation (EFF), Free Software Foundation (FSF) and the Open Source Hardware Association (OSHWA). In addition, System76 is holding a laptop sale—you can save $30–$100 on a laptop or $160–$370 with upgraded components.
Red Hat announces that 16 additional companies have joined GPL Commitment. The new companies are Adobe, Alibaba, Amadeus, Ant Financial, Atlassian, Atos, AT&T, Bandwidth, Etsy, GitHub, Hitachi, NVIDIA, Oath, Renesas, Tencent and Twitter. Red Hat writes that these companies "are strengthening long-standing community norms of fairness, pragmatism, and predictability in open source license compliance".
NVIDIA 415.3 yesterday released its first beta release for Linux in the 415 release stream. According to Phoronix, highlights include NVIDIA installer improvments, NVIDIA settings improvements, as well as various Vulkan, OpenGL and X.Org server bug fixes. Download the NVIDIA 415.13 Linux driver from here.
Samsung announces Linux on DeX with Ubuntu: "Linux on DeX empowers developers to build apps within a Linux development environment by connecting their Galaxy device to a larger screen for a PC-like experience." The Linux on DeX app is available as a private beta; interested developers can sign up here.
Kdenlive 18.08.3 is out. This release mainly updates the build script and has some compilation fixes, but has more major breakthroughs on the Windows side. A bug-squash day is being organized for early December. You can download it from here.
The goal here is to remove duplicate entries from the PATH variable. But before I begin, let's be clear: there's no compelling reason to to do this. The shell will, in essence, ignore duplicates PATH entries; only the first occurrence of any one path is important. Two motivations drive this exercise. The first is to look at an awk one-liner that initially doesn't really appear to do much at all. The second is to feed the needs of those who are annoyed by such things as having duplicate PATH entries.
News briefs for November 8, 2018.
PostgreSQL 11.1 was released today. In addition, updates are available for all supported versions, including 10.6, 9.6.11, 9.5.15, 9.4.20 and 9.3.25. The updates address a security issue as well as several bugfixes, so update as soon as possible.
openSUSE announces a new legal review system called Cavil, "designed to help Linux/GNU distributions with the legal review process of licenses". The new project is "collectively beneficial not only for the openSUSE Project, but distributions and projects that want to use it".
Gumstix has added a free new service called Board Builder to its Geppetto Design-to-Order (D2O) custom-board design service. According to LinuxGizmos, Board Builder makes "the drag-and-drop Geppetto interface even easier to use, enabling customization of ports, layout and other features. With Board Builder, you can select items from a checklist, including computer-on-modules, memory, network, sensors, audio, USB, and other features. You can then select a custom size, and you're presented with 2D and 3D views of board diagrams that you can further manipulate." Visit the Board Builder page for more information.
Creative Commons urges the EU Parliament and Council to delete Article 11 that would "require news aggregators that wish to index or incorporate links and snippets of journalistic content to first get a license or pay a fee to the publisher for their use online". According to Creative Commons, the link tax "would undermine the intention of authors who wish to share without additional strings attached, such as creators who want to share works under open licenses. This could be especially harmful to Creative Commons licensors if it means that remuneration must be granted notwithstanding the terms of the CC license."
Unreal Engine 4.21 released yesterday. Unreal Engine's goal is the "relentless pursuit of greater efficiency, performance, and stability for every project on any platform". New features include the "Niagara effects toolset is now even more powerful and easier to use"; "you can build multiplayer experiences on a scale not previously possible using now production-ready Replication Graph functionality"; an up to "60% speed increase when cooking content"; "usability improvements to the Animation system, Blueprint Visual Scripting, Sequencer, and more."
How music listeners can fill the industry's "value gap".
From the 1940s to the 1960s, countless millions of people would put a dime in a jukebox to have a single piece of music played for them, one time. If they wanted to hear it again, or to play another song, they'd put in another dime.
In today's music business, companies such as Spotify, Apple and Pandora pay fractions of a penny to stream songs to listeners. While this is a big business that continues to become bigger, it fails to cover what the music industry calls a "value gap".
They have an idea for filling that gap. So do I. The difference is that mine can make them more money, with a strong hint from the old jukebox business.
For background, let's start with this graph from the IFPI's Global Music Report 2018:
Figure 1. Global Music Report 2018
You can see why IFPI no longer gives its full name: International Federation of the Phonographic Industry. That phonographic stuff is what they now call "physical". And you see where that's going (or mostly gone). You also can see that what once threatened the industry—"digital"—now accounts for most of its rebound (Figure 2).
Figure 2. Global Recorded Music Revenues by Segment (2016)
The graphic shown in Figure 2 is also a call-out from the first. Beside it is this text: "Before seeing a return to growth in 2015, the global recording industry lost nearly 40% in revenues from 1999 to 2014."
Later, the report says:
However, significant challenges need to be overcome if the industry is going to move to sustainable growth. The whole music sector has united in its effort to fix the fundamental flaw in today's music market, known as the "value gap", where fair revenues are not being returned to those who are creating and investing in music.
They want to solve this by lobbying: "The value gap is now the industry's single highest legislative priority as it seeks to create a level playing field for the digital market and secure the future of the industry." This has worked before. Revenues from streaming and performance rights owe a lot to royalty and copyright rates and regulations guided by the industry. (In the early 2000s, I covered this like a rug in Linux Journal. See here.)
Katherine Druckman talks to Doc Searls about Freenode Live, conferences, and the Linux community.