planet-cli tag cloud

Sponsored by
HostEurope Logo

planet-cli tag cloud

All 1 2 3 4

The things you find in upstream code…

13.02.2016 by tg@
Tags: archaeology bug pcli rant security snapshot

I had just gotten an eMail from the nightly /etc/security cronjob that the mailbox from the user foo.lock belongs to the user foo (name changed to protect the… innocent? well, I know that guy from #OpenBSD on IRC, so… YMMV… anyway). Of course, I wanted to change that to exclude mbox lockfiles…

	# Mailboxes should be owned by user and unreadable.
	ls -l /var/mail | sed 1d | \
	awk '$3 != $9 \
		{ print "user " $9 " mailbox is owned by " $3 }

… oh wow. Needless to say I fixed that, although you must update your stat(1) first; it now has a possibility to generate NUL-terminated output (or any separator, really) which I used for this. (And no, Schily, I’m still of the opinion that NUL termination, even when one has to add it to each utility separately, is the better way to go.)

Dear OpenBSD developers, repeat after me:
Do n̲o̲t̲ parse ls(1) output!
Or write 100 lines of it, or something, until it sinks in.

(It can take some writing for it to sink in… just yesternight the fanfiction I was reading was at the point where Dolores Umbridge uses her Blood Quill on the students. Coincidence.)

Our PDF manpages will, starting from now, be generated with Inconsolata instead of Bitstream Vera Mono as monospace font. The body font is still Gentium, of course.

To be more exact: the Teχ flavour of Inconsolata Regular and Bold, with the varl and varqu flags, is used, and because GNU groff also requires an Italic or at least Oblique font (also in its bold variant, which the mksh(1) manpage doesn’t use though), Inconsolata LGC (both Italic and Bold Italic) are plugged in there. I added them as PFA Type 1 fonts to GNU groff, so I had to make some fixes in FontForge (merging the variants into the main font, removing unused glyphs (not for LGC), fixing the validation (mostly, and not so much for LGC), autohinting where FontForge expressed a need for that, renaming glyphs to the names expected by afmtodit, …), but it works.

I’m not regenerating older PDF manpages though.

Inconsolata is also not all I wish for a monospaced font (and even bsiegert@ says nothing goes over FixedMisc) but it has, at least, a 0 (digit zero) with a correct stroke through it ☺

expect turmoil

08.02.2016 by tg@
Tags: archaeology bug hardware news pcli personal plan rant

My network at home is unstable. NetCologne suggests to switch to fibre network, but that only comes with a dynamic IPv6 address and NAT64; completely unsuitable to running a server. (I could arguably tunnel a static IPv4 address from a dedicated server to home, but that would completely foil my plans for redundancy.) So I may need an ISP (phone isn’t important) that provides me with connectivity where a static IPv4 (and, ideally, a static IPv6 /64 or /48 – but only if the reverse DNS gets delegated to me, otherwise that’s unusable) ends up at a device of my choosing (and not a plastic router which can then “forward ports”; I require full internet to end up at my own device).

HostEurope is relocating the other server, both physically and network-wise. Their plan seems fool-proof so far, though.

gecko2@ is decommissioning the server on which eurynome is hosted, shortly. This will also be no small amount of fun for everyone involved. Expect old links, SSH host keys, etc. to break. This explicitly includes /etc/ssh/*known_hosts.

During all those moves, I will downsize my DNS zones and change some entries, so that old or duplicate records will be gone.

I’ll likely generate and publish completely new hostkeys (both gzsig(1) and PGP clearsigned) once this is all over. The current gzsig(1) key is at the end of /usr/share/doc/README in any installed system. (Do note MD5 is considered insecure.) My current PGP key is 9031955E7A97A4FDA32B2B8676B534B2E99007E0 but this requires GnuPG, so check both.

My seeming inability to remember rarely-used “secure” passwords, i.e. those not fitting into my normal schemata, led to me not attempting to run a CA myself any more. While, thanks to rsc, we have an official certificate for now, I probably will get StartSSL for “all” other systems (i.e. herc, as I appear to be downsizing), despite it lacking the SSL client purpose (important e.g. to SMTP). This shouldn’t affect anyone.

PS: I still hate Karneval!

I just published the first version of git find on gh/mirabilos/git-find for easy collaboration. The repository deliberately only contains the script and the manual page so it can easily be merged into git.git with complete history later, should they accept it. git find is MirOS licenced. It does require a recent mksh (Update: I did start it in POSIX sh first, but it eventually turned out to require arrays, and I don’t know perl(1) and am not going to rewrite it in C) and some common utility extensions to deal with NUL-separated lines (sort -z, grep -z, git ls-tree -z); also, support for '\0' in tr(1) and a comm(1) that does not choke on embedded NULs in lines.

To install or uninstall it, run…

	$ git clone
	$ cd git-find
	$ sudo ln -sf $PWD/git-find /usr/lib/git-core/
	$ sudo cp git-find.1 /usr/local/share/man/man1/
	… hack …
	$ sudo rm /usr/lib/git-core/git-find \

… then you can call it as “git find” and look at the documentation with “git help find”, as is customary.

The idea behind this utility is to have a tool like “git grep” that acts on the list of files known to git (and not e.g. ignored files) to quickly search for, say, all PNG files in the repository (but not the generated ones). “git find” acts on the index for the HEAD, i.e. whatever commit is currently checked-out (unlike “git grep” which also knows about “git add”ed files; fix welcome) and then offers a filter syntax similar to find(1) to follow up: parenthesēs, ! for negation, -a and -o for boolean are supported, as well as -name, -regex and -wholename and their case-insensitive variants, although regex uses grep(1) without (or, if the global option -E is given, with) -E, and the pattern matches use mksh(1)’s, which ignores the locale and doesn’t do [[:alpha:]] character classes yet. On the plus side, the output is guaranteed to be sorted; on the minus side, it is rather wastefully using temporary files (under $TMPDIR of course, so use of tmpfs is recommended). -print0 is the only output option (-print being the default).

Another mode “forwards” the file list to the system find; since it doesn’t support DOS-style response files, this only works if the amount of files is smaller than the operating system’s limit; this mode supports the full range (except -maxdepth) of the system find(1) filters, e.g. -mmin -1 and -ls, but it occurs filesystem access penalty for the entire tree and doesn’t sort the output, but can do -ls or even -exec.

The idea here is that it can collaboratively be improved, reviewed, fixed, etc. and then, should they agree, with the entire history, subtree-merged into git.git and shipped to the world.

Part of the development was sponsored by tarent solutions GmbH, the rest and the entire manual page were done in my vacation.

mksh R52 released

12.12.2015 by tg@
Tags: mksh pcli

The MirBSD Korn Shell R52 was published today. While there are still several known bugs, this is a release that primarily fixes lots of these, and, as with R51, we have no known regressions. Some of the itinerary for R52 has moved to R53 instead, as some bugfixes change the shell language and thus warrant a new major version, which is why this is not R51b, and they accumulated and could use some testing ;-)

This release has a nōnzero chance to break existing scripts that use some extension features – I had to quote some tildes in dot.mkshrc and a variable expansion in ${x/y"$z"} in MirWebseite (the $z) – twice!. As usual, test!

In less related news, a new release of the FixedMisc [MirOS] font is available (in BDF form and no conflict with the system Fixed [Misc] font); our CVS has the sources in bdfctool(1) format.

Go enjoy shell

27.08.2015 by tg@
Tags: debian fun pcli

Dimitri, I personally enjoy shell…

tglase@tglase:~ $ x=車賈滑豈更串句龜龜契金喇車賈滑豈更串句龜龜契金喇
tglase@tglase:~ $ echo ${x::12}
tglase@tglase:~ $ printf '%s\n' 'import sys' 'print(sys.argv[1][:12])' >
tglase@tglase:~ $ python $x

… much more than Python, actually. (Python is the language in which you do not want to write code dealing with strings, due to UnicodeDecodeError and all; even py3k is not much better.)

I would have commented on your post if it allowed doing so without getting a proprietary Google+ account.

The MirBSD Korn Shell R51 was published today. This is a feature release clearly, but still something a lot of people would wish to use. It contains several known severe bugs, but they all are no regressions, i.e. they exist in R50f already.

This one is kinda an early release, as I wish to have those known issues all fixed, but the changes – both deep down and enduser-visible – already warrant people looking for breakages, plus it makes synchronisation with mksh-os2 easier.

mksh R52 will follow, as bugfix release, pretty soon. Itinerary:

  • Fixes for as much of these known bugs as possible (code rewrites)
  • Unicode 8
  • New feature: print -a
  • Fixes for bugs reported against R51
  • Possibly more EBCDIC and OS/2 code synchronisation
  • Maybe a dead useful debug tool…

Once that’s out, I’ll roll up the fixes into R50g, so that particular code branch is not dead yet either ☺

And afterwards, at least mksh(1)-wise – I have got a lot of other things on my plate after all – we can attempt getting EBCDIC and maybe OS/2 to a state where the code is included in CVS.

carstenh asked in IRC how to make a shebang for mksh(1) scripts that works on both regular Unix and Android.

This is not as easy as it looks, though. Most Unicēs will have mksh installed, either manually or by means of the native package system, as /bin/mksh. Some put it into package manager-specific directories; I saw /sw/bin/mksh, /usr/local/bin/mksh and /usr/pkg/bin/mksh so far. Some systems have it as /usr/bin/mksh but these are usually those who got poettering’d and have /bin a symlink anyway. Most of these systems also have env(1) as /usr/bin/env.

Android, on the contrary, ships with precisely one shell. This has been mksh for a while, thankfully. There is, however, neither a /bin nor a /usr directory. mksh usually lives as /system/bin/mksh, with /system/bin/sh a symlink(7) to the former location. Some broken Android versions ship the binary in the latter location instead and do not ship anything that matches mksh on the $PATH, but I hope they merge my AOSP patch to revert this bad change (especially as some third-party Android toolkits overwrite /system/bin/sh with busybox sh or GNU bash and you’d lose mksh in the progress). However, on all official Android systems, mksh is the system shell. This will be important later.

The obvious and correct fix is, of course, to chmod -x the scripts and call them explicitly as mksh scriptname. This is not always possible or desirable; sometimes, people will wish it to be in the $PATH and executable, so we need a different solution.

There’s a neat trick with shebangs – the absence of one is handled specifically by most systems in various ways. I remember reading about it, but don’t remember where; I can’t find it on Sven Mascheck’s excellent pages… but: the C shell variants run a script with the Bourne Shell if its first line is a sole colon (‘:’), the Bourne family shells run it with themselves or ${EXECSHELL:-/bin/sh} in those cases, and the kernel with the system shell, AFAIK. So we have a way to get most things that could call the script to interpret it as Bourne/POSIX shell script on most systems. Then we just have to add a Bourne shell scriptlet that switches to mksh iff the current shell isn’t it (lksh, or something totally different). On Android, there is only ever one shell (or the toolkit installer better preserve mksh as mksh), so this doesn’t do anything (I hope – but did not test – that the kernel invokes the system shell correctly despite it not lying under /bin/sh) nor does it need to.

This leaves us with the following “shebang”:

	case ${KSH_VERSION-} in
	*MIRBSD\ KSH*) ;;
	*)	# re-run with The MirBSD Korn Shell, this is an mksh-specific script
		test "${ZSH_VERSION+set}" = set && alias -g '${1+"$@"}'='"$@"'
		exec mksh "$0" ${1+"$@"}
		echo >&2 E: mksh re-exec failed, should not happen
		exit 127 ;;

The case argument not only does not need to, but actually should not be quoted; the expansion is a set -u guard; the entire scriptlet is set -e safe as well; comments and expansions are safe. exec shall not return, but if it does (GNU bash violates POSIX that way, for example), we use POSIX’ appropriate errorlevel. zsh is funny with the Bourne shell’s way of using "$@" properly. But this should really be portable. The snippet is both too short and too obvious (“only way to do it”) to be protected by copyright law.

Thanks to carstenh and Ypnose for discussing things like this with us in IRC, sending in bugfixes (and changes we decline, with reason), etc. – it feels like we have a real community, not just consuments ☺

mksh R50f coming soon

11.04.2015 by tg@
Tags: mksh pcli

Please test mksh-current from CVS (or the inofficial git mirror)! There are security-related fixes I’ll MFC in a few days, for which I’d prefer for them (and the other changes) to not introduce any regressions. Thanks!

WTF is Jessie; PA4 paper size

12.12.2014 by tg@
Tags: debian pcli rant

My personal APT repository now has a jessie suite – currently just a clone of the sid suite, but so, people can get on the correct “upgrade channel” already.

Besides that, the usual small updates to my metapackages, bugfixes, etc. – You might have noticed that it’s now on a (hopefully permanent) location. I’ve put a donated eee-pc from my father to good use and am now running a Debian system at home. (Fun, as I’m emeritus now, officially, and haven’t had one during my time as active uploading DD.) I’ve created a couple of cowbuilder chroots (pbuilderrc to achieve that included in the repo) and can build packages, but for i386 only (amd64 is still done on the x32 desktop at work), but, more importantly, I can build, sign and publish the repo, so it may grow. (popcon data is interesting. More than double the amount of machines I have installed that stuff on.)

Update: I’ve started writing a NEWS file and cobbled together an RSS 2.0 feed from that… still plaintext content, but at least signalling in feedreaders upon updates.

Installing gimp and inkscape, I’m asked for a default paper size by libpaper1. PA4 is still not an option, I wonder why. I also haven’t managed to get MirPorts GNU groff and Artifex Ghostscript to use that paper size, so the various PDF manpages I produce are still using DIN ISO A4, rendering e.g. Mexicans unable to print them. Help welcome.

Note, for arngc, you need a server component (MirBSD-current, of course; we’re rolling release nowadays). Config included, but I’m willing to open my firewall to people I know, provided they won’t use “too much” traffic (running a couple of arngc instances is fine, according to what I estimated).

A largish article about how to use some other packages in the repo, such as dash-mksh, is yet to come. In the meantime, I wrote a bit more in README.Debian in mirabilos-support.

Bernhard’s article on Plänet Debian about the “colon” command in the shell could use a clarification and a security-relevant correcture.

There is, indeed, no difference between the : and true built-in commands.

Stéphane Chazelas points out that writing : ${VARNAME:=default} is bad, : "${VARNAME:=default}" is correct. Reason: someone could preset $VARNAME with, for example, /*/*/*/*/../../../../*/*/*/*/../../../../*/*/*/* which will exhaust during globbing.

Besides that, the article is good. Thanks Bernhard for posting it!

PS: I sometimes use the colon as comment leader in the last line of a script or function, because it, unlike the octothorpe, sets $? to 0, which can be useful.

Update: As jilles pointed out in IRC, “colon” (‘:’) is a POSIX special built-in (most importantly, it keeps assignments), whereas “true” is a regular built-in utility.

mksh R50d released

07.10.2014 by tg@
Tags: bug debian mksh news pcli

The last MirBSD Korn Shell update broke update-initramfs because I accidentally introduced a regression in field splitting while fixing other bugs – sorry!

mksh R50d was just released to fix that, and a small NULL pointer dereference found by Goodbox on IRC. Thanks to my employer tarent for a bit of time to work on it.

All 1 2 3 4

MirOS Logo