Friday, May 22, 2020

Thread-local stack on Linux

Threads on POSIX systems like Linux share an address space, but have their own registers and execution state like their own stacks. We also know that the stack is mapped in the same virtual memory space.

The illusion of thread-local stacks is achieved by having the stacks of the different threads start at different locations in virtual memory.

threadStorage.c prints the location of the stack.

Here are numbers from a few architectures:
x86-64: (kernel v4.15)
$ ./threadStorage
one: Address of first variable 0x7fc7bbc90edc
two: Address of first variable 0x7fc7bb48fedc
main: Address of first variable 0x7ffe79c59e1c
three: Address of first variable 0x7fc7bac8eedc

The difference between one and two, and two and three is 0x8392704

The stack of the main thread starts at 0x7ffe79c59e1c, which is much higher than the stack of the first thread. Since most processes are single-threaded, they get much larger stack frames.

Numbers from other architectures I had lying around:

arm32: (kernel v4.19.66)
$ ./threadStorage
one: Address of first variable 0x76e3ee68
two: Address of first variable 0x7663de68
main: Address of first variable 0x7e9c6c30
three: Address of first variable 0x75e3ce68

arm64: (kernel v4.15)
$ ./threadStorage 
one: Address of first variable 0xffffab10a9dc
two: Address of first variable 0xffffaa9099dc
main: Address of first variable 0xffffe26742ec
three: Address of first variable 0xffffaa1089dc

MIPS: (kernel v3.18.140)
$ ./threadStorage
main: Address of first variable 0x7fe384a4
three: Address of first variable 0x7603ef3c
two: Address of first variable 0x7683ff3c
one: Address of first variable 0x77040f3c


In all these systems, the addresses are randomized, so successive runs of the same program produce slightly different results.  The offsets between the locations in successive runs is constant.

The repository is here: https://github.com/youngelf/linux-innards

Tuesday, March 03, 2020

Steam on Windows 10

tl:dr; The Steam client on Windows 10 does not work for me.

This computer has great network connectivity, I can login to Steam on the browser on that machine, but the Steam client doesn't think I'm authenticated. All downloads are stuck, perhaps waiting for authentication.

The forums don't mention anything, and my Steam client on Linux on a separate computer is just fine. I suspect their login servers are alright, this is a combination of their client, or Windows 10.

Steam support is non-existent. I have tried re-installing Steam, and while that was able to authenticate and download games, it got itself wedged after the next reboot. Steam's network troubleshooting steps are totally generic. In this case, they are completely unhelpful.



It is frustrating to 'purchase' content from a store, and struggle to use the content. By comparison, the  CDs of games that I purchased many years ago continue to work reliably.

This is a machine that is only used for gaming. This is past the point where it is productive.


A lot has been written about the benefits of online stores, but a failure like this makes me anxious. Needless to say, I doubt I will purchase games from Steam. The Windows-only games I have purchased till now are a sunk cost, partly because of Steam and partly because of Windows itself. The cost of keeping a Windows machine around, updated and in good order is not worth the games themselves.

Thursday, December 19, 2019

Home audio recording studio (Digital Audio Workstation) for electric guitar

Old computers can be turned into an audio recording studio with relatively little effort and little cost.

If you play a musical instrument, sing, and want to practice or record yourself, you want a Digital Audio Workstation (DAW). This is computer software that allows recording, monitoring (listening), and editing of music. Some DAWs support plugins to modify the sound like guitar effects pedals or amplifiers, or reverb to make your singing sound more deep and 'warm'. Depending on your needs, DAWs can work on relatively modest computers, and an old computer might suffice.

This is my setup, at a glance:
2009 Thinkpad (Core 2 Duo, 8G RAM)
Solid State Disk
Behringer USB2 Audio Interface
Behringer microphone (Ultravoice Xm8500)
Windows 10
Tracktion 7


Most of this utilized spare components. It saves an old laptop from the landfill, and this setup provides decent audio with no dropouts.

Hardware: Set up the computer


Set up Windows from a fresh disk. Tracktion 7 will work on Windows 7 onwards, and either Windows 7 or a recent Windows 10 are stable enough. It is important to have the minimum possible software on the computer, because every additional service or 'utility' can interrupt your computer during the recording session, causing annoying clicks (called audio drop-outs). Preinstalled Windows software that comes with computers is notorious for using system resources, causing these drop-outs.

Tracktion runs on Macintosh and Linux as well, so if you are using a different operating system, consider cleaning out unused software. If your computer is relatively modern with a fast processor, it might manage with other software running on it.

If using an aging laptop, buy a decent USB keyboard or mouse. Tracktion has great keyboard support, and you will be using both keyboard and mouse while you have your guitar slung around you. USB keyboards and mice don't 'interfere' with USB audio input.

Update the computer, install Microsoft Security Essentials, or other virus protection software. Run the initial scan.

To reduce Audio drop-outs (clicks or other aberrations during recording), you need to streamline the computer as far as possible:

Turn off power throttling on a laptop, so the CPU is running at max performance all through. Turn off the screensaver, and power management. You'll suspend it manually.

Turn off wifi. Network drivers can be poorly written, and can interrupt audio processing. You don't want that. If possible, use a wired connection (ethernet). If you don't need the network, disable network entirely. Your computer is going to be used to record audio primarily, and turning off the network reduces the potential for other harm. If you need Ethernet to copy music from this machine, you can always use a USB stick, or connect it to the network just when copying music out.

Turn off bluetooth. If you aren't using any bluetooth peripherals, the same caveats apply. Bluetooth drivers can sometimes keep Windows from processing audio in time. If you only have a bluetooth keyboard/mouse, you'll want USB keyboards/mice instead.

If you have disabled the network: turn off the anti-virus, turn off the firewall, and pause System Updates. In order to keep the machine in working order, you will have to routinely get the machine online (wifi or ethernet), run updates and anti-virus scans manually. If the system started out updated and with no malware, it won't get any if it stays disconnected.

Your goal is to set up the computer as an appliance. The more isolated it is, the less you will have to maintain it. Copy files out of it using a USB stick and be careful when copying files in to this system.

Hardware: Set up the USB interface

Plug your guitar/mic/piano into the interface. Install ASIO drivers (for Windows), as they provide lower latency than conventional Windows drivers. 
Update the device drivers, if your operating system recommends it.

DAW: Tracktion 7

Download Tracktion 7. This is available for free, but still requires creating an account and registering. Once you do that, you get a license key for free. You don't need a credit card. Once you create an account, you enter that account in Tracktion 7, and that allows the full software to be used with no payment. Tracktion 7 runs on Windows, Mac and Linux.

Pedals, Amplifiers: Amplitube 4 Custom Shop

If you are a guitarist, you probably need additional amplifier simulators, pedals, etc. You can look for VST and VST3 plugins online as Tracktion 7 supports these. A decent starting point is Amplitube 4 Custom Shop which is also free. To download it, create an account with IK Multimedia, which gives you the option to get software, and also license keys for the free software. You will need to enter the license keys into Amplitube when you start out. Amplitube 4 is relatively light on CPU consumption, and the included cabinet and classic amplifier is sufficient for a beginner. Of course it isn't a perfect copy of a Marshall or a Fender Blues Junior, but works well enough till you know what you want and are ready to have a dedicated music setup.

Amplitube 4 has a basic tuner built into the amplifier, as the first effect, and you can use this to tune a guitar.

Metronome/Effects: MFreeFXBundle from Melda Production

If you need a Metronome and a combination of core plugins, download MFreeFXBundle from Melda Production. It includes a compressor, a metronome, an auto-pan, auto-pitch, flanger, a cross-fader, a recorder, an equalizer, noise generator, frequency-shifter, loudness meter, reverb, and much more. Amplitube and Tracktion already have a reasonably good reverb, and you can record using Tracktion. But the free bundle provides all these in a single location, you can add plugins one after the other, and these are a starting-point for future experimentation. These plugins are relatively light on CPU usage.

Drums: MT Power Drum kit

If you need a drum kit, a relatively basic but powerful drum kit is available at MT Power Drum Kit. This is provided as a VST plugin. You can either create your own drum loops as MIDI segments that play through the drum kit, or you can use its own samples as MIDI drum segments. You can also schedule different samples so you have a combination of intros, verse, chorus, bridge, and outro. If you read through the Tracktion 7 documentation, you can also learn how to loop various segments, or modify the MIDI segments, or how to add plugins (effects) to the drums that you get from MT Power Drum kit. Again, it is available for free.

Learn how to use a DAW

DAWs are formidable software, combining mixing, effects/plugins, editing, exporting, monitoring. Since a DAW combines the features of a mixing board, amplifiers, pedal boards, keyboards/synthesizers, equalizers, you have a lot of terminology to catch up on. Tracktion has quite some documentation that can be read. Much of this documentation will help you learn the concepts, and these concepts transfer to other DAWs or audio systems.

A comprehensive set of video tutorials covering the basics.
Tracktion 7 quick start guide.
Bill Edstrom's Guide to Tracktion 7, a full book about the functionality. This is an exhaustive book on Tracktion7, with lots of screenshots of the UI and how to use it. The key-bindings for the shortcuts were incorrect for me, so do check if they are correct for you.

When I wrote this, all the above guides and tutorials were free.

Tracktion 7 is a full-featured DAW: supporting MIDI surfaces, automation, folder tracks. It also has a full programming language that allows you to write custom macros, and then assign them to a single keystroke. Some of these features are beyond what you can find in many more expensive DAWs. As you progress in your musical journey, you can purchase higher-quality plugins, and graduate to a more complicated setup. A good understanding of the basics and a knowledge of Tracktion's capabilities can be a valuable skill in the long run.

Scratchy audio!!

If your sound quality isn't clean, here are ways to isolate the issue.
  1. Is the microphone/guitar itself producing scratchy audio? Connect it to a known good amplifier/PA system and check if the problem persists.
  2. Is the cable connecting the guitar/mic frayed? Does that give clean audio connected to a different system?
  3. Is the speaker-out or headphone-out jack bad? When you play pre-recorded music from an album, do you still hear scratchy sound?
  4. Is the system under contention? You can run Task Manager (under Windows, or its equivalent tool on other systems) and see if the CPU usage spikes during audio recording. If something other than Tracktion or plugins is taking significant CPU, find out what the process is, and try to eliminate it. 
  5. Is there something keeping the CPU from processing audio with low latency. Install LatencyMon.  Close all programs, run LatencyMon and see what it says.
  6. Are your plugins too CPU intensive? Turn off plugins, or use Tracktion's system usage meter to find which plugins are taking the most CPU and see if removing those plugins reduces drop-outs. You can also reduce over-sampling in plugins, as a first step.
  7. Windows 10 has a lot of superfluous services that aren't required on an audio-only system. You can evaluate the system services and turn them off. Be careful when doing this unless you know what the service does, as you can end up with an unusable system.
  8. Ensure you are using ASIO drivers in Tracktion (Windows).
  9. Ensure you have a reasonable audio buffer size. 256 samples should give you a latency of 5.8ms with 44.1khz sampling, and that should be enough to handle the occasional Windows service running in the background.
  10. If the system is RAM constrained, the Task Monitor will show a high RAM utilization. This might lead to the system swapping memory to disk. If that is the case, buying some additional RAM might help.
  11. One approach that works for me is to log in, run updates virus checks etc, and confirm that the Task Manager shows no extra tasks running. After that, I start up Tracktion, Amplitube and the MT Drums kit. At this point if the Task Manager shows no large background activity, I start recording audio. This enables Windows' background services to settle down and the remaining audio session to proceed without interruptions.
  12. Tracktion allows the audio system to be complete unused when playback and recording is stopped. This setting is available in the Options menu, and as a keyboard shortcut. Options -> "Run audio engine when stopped" when checked means that the audio system is fully running even when the playback is stopped. In the keyboard shortcuts screen this is called: Turn playback-in-stop on/off, and is mapped by default to Shift-E. When playback-in-stop is turned off (Press Shift-E once), the audio system and all plugins are unused when the audio is stopped. So Tracktion will be running, the user interface will be functional, yet the system usage by Tracktion should reduce to near-zero. At this point, you can investigate if anything else in Tracktion or elsewhere is consuming system resources.

Monday, July 15, 2019

Check if a screen is turned on in Windows 10

To check that a monitor is powered on in Windows 10, here is the command-line:


C:\Users\Vikram>wmic path win32_desktopmonitor GET Availability,Caption
Availability  Caption
8             LCD 1280x800
3             Default Monitor


It returns a line with column headers and a single row for each monitor you have. The column 'Availability' tells you whether the monitor is turned on or not. 8 means turned off, 3 means ON.

In the above output, I have an actual display LCD 128x800, which is off, and a virtual display called 'Default Monitor' (from Microsoft Remote Desktop) that is turned on.

You can get the full object as well, showing useful info like DPI, monitor size, etc:


C:\Users\Vikram>wmic path win32_desktopmonitor


Availability  Bandwidth  Caption          ConfigManagerErrorCode  ConfigManagerUserConfig  CreationClassName     Description      DeviceID         DisplayType  ErrorCleared  ErrorDescription  InstallDate  IsLocked  LastErrorCode  MonitorManufacturer  MonitorType      Name             PixelsPerXLogicalInch  PixelsPerYLogicalInch  PNPDeviceID                               PowerManagementCapabilities  PowerManagementSupported  ScreenHeight  ScreenWidth  Status  StatusInfo  SystemCreationClassName  SystemName
8                        LCD 1280x800     0                       FALSE                    Win32_DesktopMonitor  LCD 1280x800     DesktopMonitor1                                                                                     Lenovo               LCD 1280x800     LCD 1280x800     96                     96                     DISPLAY\LEN4010\4&3A89619C&0&UID67568640                                                                                    OK                  Win32_ComputerSystem     DESKTOP-DNTB7R7
3                        Default Monitor                                                   Win32_DesktopMonitor  Default Monitor  DesktopMonitor2                                                                                                          Default Monitor  Default Monitor  96                     96                                                                                                                      800           1280         OK                  Win32_ComputerSystem     DESKTOP-DNTB7R7

Friday, March 29, 2019

Untrusted: A superb programming game

I recently came across a wonderful programming game: Untrusted.

It is not a game you install, you just play it on your web browser. You don't need to buy it, you don't need any in-app currency. It requires some knowledge of programming.

It's a weird game. You edit code to finish a level, and very few instructions are provided.  The games require some critical thinking, and some brute-force. Some of the levels are cunning.



Do try it! It is one of the most innovative and brilliant games I have played recently.



Screenshot of the ending. It's the journey, not the destination.


Sunday, March 03, 2019

Unix from 1972... on a Raspberry Pi 3

Want to run Unix on a PDP 11, and all you have is a Raspberry Pi 2/3 handy?


Here's a Docker image I created with the first version of Unix, and SIMH (a CPU level simulator) that simulates a real PDP-11. Once you have Docker, getting it to work on Raspberry Pi is trivial:

docker run  -it youngelf/unix-72-raspi3:1.0

That should download the image, and run simh. The user is 'root', no password. Yes, this is very secure.

It is a fun system to play with. No vi (Bill Joy hadn't written it yet), but there is 'ed'. There is a C compiler which is 4k in size. The original Unix paper talks about the system.

The system is ancient: no backspace support, no command history, no control characters (Ctrl-p, Ctrl-n, Ctrl-a). Pipes on the shell have not been invented yet, though the system supports them. Redirection with > works, but the name of the file has to come immediately after the '>' character. Spaces aren't allowed between them. You have to be sure of yourself.


It is a full CPU emulator, so you can write PDP-11 assembly on this computer and assemble it.


Take a moment to think of the magic that brings you this: 
  1. The Raspberry Pi: A $35 computer.
  2. SIMH software emulating a 1970s PDP-11.
  3. Old source code for Unix, which was made available.
  4. Docker, a system to allow pre-packaged software to run with minimal effort.
  5. The Internet, that moves all these bits around.
  6. Linux, that runs on the Raspberry Pi and is available free of charge.

Many of these systems are built, powered, or influenced by Unix.


This builds on top of bahamat's git repository, which had to be recompiled for arm32.