private software development topic home

Here's a list of tools, side projects, applications and major investigations that I've done and implemented as a hobby, for my personal benefit, or to broaden my knowledge. Most of these have not been published or made available to the general public (in contrast to my open source contributions).
The list goes back to my first programming experiments on the PC, and lists the project name, its main programming language, and a short summary:

2022

Mar-2022 general.css
Every couple of years I extend my Cascading Style Sheet that powers my website. Now two decades old and in its 124th revision, custom classes for highlighting, dropcaps next to images, indication of links that open in a new window or parent frame have been added, driven by needs in my restarted blog and news items. I've got companion HTML pages that showcase and test-drive all features, and build automation in the form of a Makefile that minifies the CSS and copies it to my document root. That (and the fact that after the initial turmoil of the browser wars, CSS has stabilized) has helped a lot with the long-term maintenance.
16-Mar-2022 Vim plugins
While converting all of my 120 released Vim plugins to Git in January, 53 unreleased ones were also converted, but a huge number was still clumped together; mostly as a triple of plugin/NAME.vim, autoload/NAME.vim, and doc/NAME.txt files. With some additional streamlining of my half-automated conversion process, I finally extracted (thanks to Double Commander for the efficient file handling!) converted the whole remainder in two days; now there are a total of 300 separate projects (so 180 potential new plugins to polish and release). The only scripts left in the legacy directories are private customizations.
Mar-2020 private notebook setup
I now have two notebooks running Ubuntu 20.04 with completely equivalent software setups (automated through my software definitions); one is a Pinebook Pro with an ARM 64-bit processor, the other an el cheapo Teclast F6 notebook (with US-English keyboard layout; these unfortunately are still hard to find in German online shops). I was impressed by the independent and open design of the former, but initially struggled with its sketchy touchpad (I managed to tweak it a bit) and found out that some crucial applications were not available on the non-Intel architecture. I then considered buying a Microsoft Surface Pro 7, but even there it was difficult to obtain the separate keyboard with a US layout. Then suddenly the Teclast was on sale — I was impressed with a similar LincPlus P1 low-cost notebook I had bought for my daughter. Unfortunately, the Teclast has a different keyboard annoyance with the function / Fn-keys (cannot be toggled in the BIOS), and the touchpad is poor as well (but in a different way). So, my immediate needs are met, but I'm still on the lookout for the perfect machine™. I still have an ACEPC AK1 small form-factor PC lying around that I could set up in the same way (meant as a minimal home server). Maybe I should have bought just one powerful machine instead of three cheap ones?!
After the installation of the operating system, I install and configure via my software definitions, sync my Unixhome (configuration, Vim plugins, and my personal toolset) from my cloud server (with my NAS functioning as a local fallback), and obtain the more stable public share that has some common downloaded tools (mostly for Windows) and sounds from my NAS. My personal data is stored in a TrueCrypt container on a mini USB thumb drive; I can easily carry that with me and plug into any system I'm working on if required.
I'm pretty happy with this setup, especially since it got started decades ago and has mostly been built with a corporate work setup in mind, and is now serving my own distributed personal environment.
Mar-2022 dynamicsource
I needed a mechanism to augment the PATH of existing shells with a directory, for when I insert the USB thumb drive that holds my personal data — it has a bin/ subdir with commands like mount-data and backup-data that usually get triggered automatically (there's an autostarted watcher running in the background), but sometimes need to be executed manually. I ended up with a generic mechanism that hooks into Bash's PROMPT_COMMAND to run commands on the next shell prompt, and wrote an interface consisting of dynamic-{command,environment,message,path} — the latter adding the bin/ dir. I soon found another use case for it: dynamicenvironment to add the subsidiary locate database to LOCATE_PATH, so that the locate command will also find files on that data drive when it's available. So the added genericity payed off very soon.

2020

Nov..Dec-2020 software definitions for installation and configuration
A lot of Linux system setups were needed: I finally got a replacement for my 8-year-old notebook at work, my dev workstation was waiting for an upgrade from Ubuntu 16.04, a personal notebook (as I wanted to have more separation between work and private stuff), my daughter got a small notebook for home schooling during the pandemic that I would have to administer, and I purchased a small standalone PC to serve as a mini-server at home to get rid of Dropbox. As I went through the installation steps I had previously recorded manually in a log file, I translated those into a DSL and generic toolkit that would automate the installations and configurations as much as possible — with so many installations, I would greatly benefit from the savings in manual steps and the resulting consistency. Starting small, this gradually became very powerful and flexible — so much that I eventually split the engine from the definitions and published it. (And then found more uses, replacing a set of scripts for setting up cloud systems at work.)

2019

Unixhome restructuring and Git conversion
About 500 shell scripts had accumulated in my ~/bin/ folder, to the point it became unwieldy. As there was no end in sight to my prolific customization and extension, some restructuring was necessary. Roughly based on Vim's transition from a single plugin folder to per-project configuration, I've extended my ~/.bashrc to consider subdirectories, and then gradually extracted sets of coherent commands into separate projects. Like my Vim plugins, the writebackup-based version control data got automatically migrated into Git repositories. Some very generic ones that have no or few dependencies have even been published as open source; dependencies are still the main impetus here — the biggest downside of writing shell scripts as extremely modular, object-oriented programs.

2018

Mar,Dec-2018 website utilities ported to Linux
Building a house kept me busy, so I didn't update my homepage for a long time. In the meantime, I've completely switched all of my systems from Microsoft Windows to (Ubuntu) Linux, so the tools I had developed to edit and process my static web sites (in 2006) had to be ported to Linux. Fortunately, the Linux tooling is very amenable to this, and I had already acquired a lot of experience in this.
Oct-2018 inbox utilities
Since I do a lot of work now on Linux test systems, I often need to collect log files or other data. For that, my dev system exports a writable Samba share which my configuration on other systems mount. I've written an inbox command that copies any passed data (or standard input) to that share; it also adds a timestamp and hostname information to the filename. If the share isn't available, it also can automatically fall back to SSH, or use a local cache until later. For systems that don't run my configuration, I've implemented an analog scp-inbox command that fetches the remote data from the local dev system instead.

2017

23-Mar-2017 tmux customization
Installed some plugins (through Tmux Plugin Manager) and implemented Vim-style named registers. Added short SSH wrappers tssh and ssht that use a local / remote tmux session.

2016

25-Feb-2016 notebook on Linux
After successfully switching my workstation to Linux, I wanted to do the same with my notebook. A challenge here was that it's used for a larger variety of tasks (versus just development work), still needed to run Windows as well for company applications, and even though it already was 3.5 years old (and not powerful from the start, as I chose the light form factor over bulky performance), no replacement was in sight.
I settled for another VirtualBox VM, hosted by my workstation for performance, then moved to a USB memory stick because the notebook's SSD could only hold the Linux system and my personal data, but not another operating system installation. Externalizing the VM was both great in terms of flexibility and ease of doing backups, but also caused a lot of headaches with poor disk IO. I later had to migrate to an SSD in an external USB3 drive enclosure, and finally settled on putting it back on the internal SSD, and externalizing the private data to a tiny USB3 thumb drive instead.

2015

03-Nov-2015 development workstation on Linux
It was a difficult decision to make for someone who grew up with MS-DOS 6 and Windows 3.1 on a 386 PC, then used Windows NT 4.0 and later Windows 2000 Professional at work (and became one of the Windows gurus). But I had also been working on HP-UX workstations, a bit of Solaris and AIX, and always dabbled in Linux on the side. I think it was the attraction of Vim and its inviting eco system on the one hand, and many (bad) decisions made under Steve Ballmer's reign of Microsoft (Windows Vista, Silverlight, elimination of the Start Menu) that made me switch. From the UI consistency, Windows had peaked for me with Windows 2000, and from a stability perspective with Windows 7. All those proposed changes towards "Windows apps" and the unification with mobile touch devices didn't appeal to me at all, but rather forced me to change without perceived benefits. I had already successfully established Linux at home (on my wife's notebooks, after horrific experiences with the Symantec anti-virus products, and on the NAS and OLPC laptops).
When I got my brand-new development workstation at work, I set aside the factory Windows 7 installation and installed Ubuntu 14.04 LTS instead, under the hostname nashi, which means "without" (Windows, in this case) in Japanese. I had already found adequate replacements for most of my trusted Windows tools:

2013

mvn-log.cmd
Because the default output of Maven builds is overly verbose and hard to follow when running interactive builds, I've written a wrapper script that filters out unimportant stuff, and transforms long (multi-line) output into a shorter summary (using sed).
05-Apr-2013 commenter.wsf
This filter reacts to passed patterns in the pipe and then notifies the user about the occurrence, either via a simple beep, playing a custom sound file, or speaking parts of the pattern. I've added this to the mvn-log.cmd to be informed about successful builds (and failures, with a different sound); runtime exceptions of the web app are spoken.

2012

03-Feb-2012 git-writebackup-import
To facilitate the migration of more and more of my home-grown tools and side projects to published, collaborative open source projects (prompted both by individual requests and the conveniences of modern platforms like GitHub), I've written a Bash script that automatically converts my writebackup archives into a Git repository with complete revision history.

2011

19-Oct-2011 Quake-Terminal
Took a simple AutoHotkey script found on the Internet that implemented a pop-up console window (similar to the Tilda tool for Linux), and adapted and polished it to my advanced needs. After making it much more robust, I added the capability to toggle the console from another executable (assigned to a custom keyboard key) and devised a mechanism to allow both local and Remote Desktop Connection triggering (by offering two variants of the script that bound to either both or just the left CTRL key). The instant, everywhere access to a plain text console is crucial for a keyboard-focused, multitasking, remote system-juggling hacker like me.
May..Jun-2011 Subversion and Git command-line aliases addonly alias diff dp help lg ll log lsprivate prune rmprivate sh show statistics unprune
It started with various Git aliases that I had picked up from various blog postings. After these considerably enhanced my Git workflow and expanded my Git capabilities, I desired to have something similar for Subversion (and eventually Mercurial etc.), to have a more uniform revision-control use, more customizable and powerful tools than the ubiquitous Tortoise GUI, and to paper over deficiencies in the rather limited svn command-line tool.
This continues the tradition I started with my ClearCase tools in 2005. I complement these tools with various plugins for version control-handling in Vim.
07-Feb-2011 Imake archiverules, fprules, retrievalrules
Most of my colleagues are either too young or too afraid of the Imake-based build process, even though it's actually quite powerful. I reduced the sprawling growth of custom build scripts by extracting several general-purpose Imake rules for retrieving packages from two internal FTP locations and our private Maven repository, handling file archives, and finally generating file placement information automatically from these packages. This made the complete product build a one-step process triggered by a single make again, and avoided the error-prone choreography of various script invocations that had to be executed in the right order and in the right places.

2010

17-Jun-2010 AutoHotkey
Published a set of scripts that ease task switching inside Remote Desktop Connections, allow quick minimizing of windows via mouse and keyboard, and allow entering of special characters like umlauts via digraphs.
12-May-2010 application activation.vbs
Re-activated an old utility script from 2006, refactored and enhanced for use with my new dev PC. Assigned to the multi-purpose keys of my Microsoft Natural Keyboard, this allows to launch or activate several of my favorite applications with a single keystroke. Now also works with minimized windows and minimizes the window if applied to an active one.
12-Apr-2010 ssh-copy-id.cmd
Enhanced the waitup suite of Windows shell scripts to a complete suite that allows me to manage and launch any remote connections via a central control shell, including a Windows port of ssh-copy-id to install my public key to any new test systems, and helper scripts such as 2ssh to make two separate shell connections (my typical setup on test systems) at once.

2009

14-Dec-2009 syncUnixhomeProxy
Refactored my synchronization script to also support access through an SSH proxy system, since I didn't get any socks software to function on Windows Vista / 7.
11-Nov-2009 opr-checkout.sh
Bash shell script to checkout relevant parts of the 6+ Subversion repositories at work. Many of those contain staged binaries and the tool chain for various platforms, so a full checkout would be prohibitive. I use this to keep my three dev systems in sync and to be able to automatically set up a new dev system.
20-Jul-2009 syncUnixhomeExternal
Exposed my main file server at home to the Internet (only SSH access) to be able to access and synchronize files while at work. Wrappers around the syncUnixhome scripts with reconfigure the Unison tool to use the externally visible hostname. The wrappers themselves can be socksified to allow Internet access from a restricted company Intranet.
15-Mar-2009 photoshow
JavaScript function that adds controls to sequentially view photos to album web pages. Implemented with the JQuery library.
12-Jan-2009 runVimTests
A testing framework for the Vim editor, consisting of a Windows shell / Unix Bash shell driver, a Vim evaluation script and an optional Vim testing library. Also integrates with the TAP test protocol provided by another author. I use this to automate the regression testing for my Vim scripts.

2008

18-Dec-2008 MAF patch
JavaScript patch to the Mozilla Archive Format addon to make it compatible with the File Title addon (and massage the suggested file name into something that is close to my personal naming preferences.
15-Oct-2008 OLPC XO customizations
Customizations and enhancements to the user interface and window manager of the 100 Dollar laptop, which is written in Python.
17-Sep-2008 syncUnixhome
Set of Windows and Bash shell scripts to synchronize common parts of my home directory with my main file server (using the Unison tool for the sync). Based on unison-unixhome from 2006. Together with the writebackup tool, this implements a primitive Distributed Revision Control System.
24-Jun-2008 NSLU2 customization
Installed Unslung 6.10 Linux distribution on a small NAS device from Linksys and configured Samba, web server, TWiki and various tools.
09-Apr-2008 waitup
Windows shell script which allows to monitor a host as it goes down and up again during a reboot. An external command (like VNC or Remote Desktop Connection) can be launched once the host (or a specified remote-enabled service) is up again.
01-Jan-2008 sitemenu
JavaScript function that fetches a sitemenu HTML page via AJAX and embeds it into the page. Allows custom styling via CSS and fallback menu (without duplication!) if JavaScript is disabled. Implemented with the Prototype library.

2007

12-Jun-2007 toolbox
Windows shell scripts that package everything that is contained in my tool share (sysinternals.com, GNU Unix tools for Win32, Java web server, PuTTY, directory statistics, Wink screen capture, Total Commander, IrfanView, and Vim editor (of course!)) and put it in a TrueCrypt container. This container is deployed to (test-)systems and kept up-to-date via my automatic updater. With this, all tools start faster than through the network share and do not hang when the VM their run in is suspended.
05-Apr-2007 OpenSUSE@wotan
Installed OpenSUSE 10.2 on my former development system and migrated my Intranet web server and TWiki wiki from my old HP-UX workstation.
22-Jan-2007 page aging
JavaScript function to progressively dim a web page background based on the current date. Includes demo page and JsUnit tests.

2006

15-Dec-2006 download Dilbert
Windows shell script that scrapes an RSS feed to download today's comic from the Dilbert website.
12-Dec-2006 unisync
VBScript that performs a unidirectional synchronization of a directory tree, copying only files that have changed since the last sync; files and directories that have been removed in source will also be removed in destination. Differences are propagated only from source to destination, not the other way around.
24-Nov-2006 unison-tools, -unixhome, -public stuff
Set of Windows shell scripts that employ the Unison tool to sync various directories across multiple systems or portable media.
22-Sep-2006 autosource
Bash and Korn shell configuration scripts split off of my growing .bashrc, making my configuration modular and pluggable: aliases, history, additional commands, paths and prompt settings are defined in separate files.
27-Jul-2006 settings
Set of Windows shell scripts and registry fragments that configure a (test-)system with one click: Internet Explorer security settings and start page, proxy, virus scan config, eventlog, screensaver, Windows console properties.
2006 website utilities
Extension and refactoring of VBScripts to process and automate my private web sites:

2005

03-Nov-2005 makeMeAdmin
Enhancements to a Windows shell script by Aaron Margosis, then Johannes Endres. I'm now running Windows XP without administrative rights and use this script to temporarily gain Admin rights for my account.
15-Aug-2005 SendToGVIM
Shell alias to send files to an existing instance of the Vim editor. Unix port of the VBScript I had implemented earlier.
03-Aug-2005 cteclipse, ctfindbranch, ctfindchange, ctfindsince, ctsetcs, ctWaitForSync
Various Korn shell tools to simplify the handling of the Rational ClearCase revision control system.
Jul-2005 CppUnit libraries
Built libraries for the CppUnit unit testing framework on a variety of platforms: Win32, HP-UX 11.00 PA-RISC, HP-UX 11.23 Itanium, Solaris 8, AIX 5.1; all 32 and 64 bit. Learned a lot about C compiler and linker settings and each platform's peculiarities.
25-Jan-2005 CopyM3U.cmd
Windows shell script that numbers MP3 files based on a Winamp playlist, so that the tracks are played in the defined order on my MP3-player.

2004

22-Sep-2004 automatic updater
Windows shell script that implements a simple automatic software updater for directories using Windows file shares as installation sources. Allows bootstrapping, check via autostart, pre- and postinstall commands. Used to automatically update desktop background and system tools on a variety of test systems.
07-Sep-2004 lsEx
Unix aliases for many common and advanced uses of the ls command. Features include recursive flattened listing and omission of temporary and backup files.
05-Sep-2004 extedit
C++ tool to synchronize text from any Windows application to an external editor and back.
Many applications prompt the user to edit text via the standard Windows edit control. Most of the interaction with web applications (think about Internet forums, Internet mail composing) is done via a browser. Almost all browsers do not allow to configure another editor for text area editing (except for the text-only Lynx and a patch to Mozilla). The extedit tool allows more comfortable editing of edit boxes in Windows applications. It uses simulated keyboard input to capture the edit box content in the Clipboard, saves the Clipboard contents to a temporary file, and opens that file in an external editor. When editing is done, the modified contents are synchronized back to the edit box. With extedit, you can use your favorite text editor for increased productivity. In addition, you get a backup of the volatile edit box contents (have you ever pressed <Esc> in an Internet Explorer text area?) for free.

2003

11-Sep-2003 SendToGVIM
VBScript tool to send files to an existing instance of the Vim editor through its OLE interface. I call this from Total Commander, to quickly pass file(s) to my running Vim instance.
10-Jul-2003 writebackup
Korn shell script to write subsequent backups of the passed file(s) with a '.YYYYMMDD[a-z]' file extension. Unix port of the VBScript I had implemented earlier.
17-Jun-2003 TotalCmd dirstack
VBScript that implements an directory stack for the Total Commander alternative file explorer.
25-Mar-2003 site maintenance
CGI scripts for maintenance of my web server: List log files, backup website and Wiki, run link check.
05-Feb-2003 HP-UX @ wotan
Configured HP-UX 11.0 64bit on my HP B2600 workstation: Compiled Vim 6.1, Midnight Commander and various other tools; set up Samba, Apache and the TWiki Wiki.
31-Jan-2003 prepareForJava
Korn shell script wrapper to run and compile Java applications through various JDK versions, which are fetched from revision control. Later implemented a caching mechanism to speed up startup.
24-Jan-2003 writebackup
VBScript to write subsequent backups of the passed file(s) with a '.YYYYMMDD[a-z]' file extension.

2002

10-Oct-2002 prepareForImake
Korn shell script wrapper to simplify running Unix builds via the Imake tool. Detects and sets up the environment on various Unix dialects and even MS Windows (under the MKS Unix emulation).
08-May-2002 bginfo
Using the bginfo tool from sysinternals.com, I created a custom configuration for our internal development and test environment, wrote a simple installer, and embedded my automatic updater mechanism. The bginfo tool runs automatically during login, gathers the system information, checks for updates to itself, writes the information to the desktop background, then quits.
25-Apr-2002 vmware-scripts
Suite of Windows shell and VBScript to automate system customization after cloning a Windows VMware virtual system: Choose hostname / IP from a pool, create unique system ID, update DNS settings.
22-Feb-2002 default application startup
VBScript that starts my default applications in the correct order, waits until the app window appears, and can change the window size.
Also includes a couple of wrapper scripts that can be launched from the configurable extended keys on my Microsoft Natural keyboard.
01-Feb-2002 Simple Java WebServer
Extended a simple demo implementation of a Java web server from Sun with security bugfixes, external configuration, virtual directory directives, response delay, etc. Using this as an on-demand web server in my toolbox.

2001

01-Jun-2001 general.css
Complex Cascading Style Sheet to apply a consistent look and identity to my personal homepage, Intranet site, and portals on various systems. The styling is based on my Microsoft Word template ingodoku.dot; much of the complexity comes from the distinctive large left page margin. Being a strong believer in the separation of content and styling, this stylesheet has grown and expanded together with the CSS capabilities of the leading web browsers. (The CSS support of Netscape 4 was really bad; IE 5.5 offered much better support, in 2009 everyone derides IE 6's non-standard quirks.)
19-Apr-2001 CommandLineForwarder
This C++ tool calls a secondary file with the same name as the actual tool's name; only the .exe extension is stripped. All command-line arguments are passed AS-IS, i.e. exactly as they were provided to the tool. This tool is handy to forward '/RegServer' command-line arguments to a batch file.
2001 website utilities
VBScripts to process and automate my private web sites:

1999

03-Dec-1999 cdEx
Unix alias for the cd command that updates the prompt and has many advanced DWIM features.
28-Jan-1999 FileShuffle
Successor to Nextbmp: C++ tool to copy one randomly selected file from a specified directory to a destination, while changing the filename to a standardized one. Used in Windows autostart to change the desktop background on each login.

1998

Dec-1998 Style Guide: Definitions for C/C++ coding
This document grew from a couple of pages about C++ coding style (identifier naming, function prototype documentation comments) based on suggestions by my assistant project's mentor to a 50-page collection of various software engineering topics (project management, analysis, design, layout, coding). Various Internet sources and excerpts from software engineering books have been incorporated into it. It has spawned off comments templates for many programming languages.
09-Sep-1998 ingodoku.dot
Microsoft Word template for use in project documentation, specifications and design documents. Offers consistent styling for headings, paragraphs and lists with a landmark large left page margin that contains some minor headings and annotations. The file contains templates and copy-and-paste fragments for cover page, table of contents and tables.

1997

01-Oct-1997 Nextbmp
C tool to copy random bitmap file selected from 1..9, a..z files to Windows folder to use as background picture.

historic

27-Jun-1996 Vocabulary
A vocubulary trainer written as a Turbo Vision GUI application. Reads (English) vocabulary data from an imported Word document. I used this to improve my English.
1996 Normal.dot
Microsoft Word 6.0 document template with completely customized keyboard shortcuts and macros. I had a keyboard stencil for all CTRL-ALT-SHIFT-Function-key combinations, and a cheat sheet for all other keys and custom formats. Word was my main text processing system for many years, and I only broke up and left it completely (in 2002) when I discovered an editor and text systems that were more universal, portable and configurable (viz. Vim and HTML+CSS).
1993 - 1995 Mathematics & Chemistry applications
Various small Turbo Pascal applications for homework or additional investigations in my mathematics and chemistry classes.
25-Apr-1993 Backpropagation-Mustererkennung
Extension of a neural network sample application into a Turbo Vision GUI application which allows loading, training and recognition runs of patterns.
12-Oct-1992 Fractals
Various Turbo Pascal applications to display Mandelbrot and Julia sets in graphics mode through the Borland graphics interface.

Before personal computers, I played, programmed, and built electronics on my home computer.

Ingo Karkat, 24-Jun-2010, last update 28-Mar-2022