planet-cli tag cloud

Sponsored by
HostEurope Logo

planet-cli tag cloud

All 1 2 3 4

FrOSCon

16.08.2017 by tg@
Tags: event fun pcli personal

I’ll not respond, much, until next Monday. We have FrOSCon.

[PSA] Fixing CVE-2017-12836 (Debian #871810) in GNU cvs

11.08.2017 by tg@
Tags: archaeology bug debian pcli security

Considering I’ve become the de-facto upstream of cvs(GNU) even if not yet formally the de-iure upstream maintainer, fixing this bug obviously falls to me — not quite the way I had planned passing this evening after coming home from work and a decent and, worse, very filling meal at the local Croatian restaurant. But, so’s life.

The problem here is basically that CVS invokes ssh(1) (well, rsh originally…) but doesn’t add the argument separator “--” before the (user-provided) hostname, which when starting with a hyphen-minus will be interpreted by ssh as an argument. (Apparently the other VCSes also had additional vulnerabilities such as not properly escaping semicoloi or pipes from the shell or unescaping percent-escaped fun characters, but that doesn’t affect us.)

The obvious fix and the one I implemented first is to simply add the dashes. This will also be backported to Debian {,{,old}old}stable-security.

Then I looked at other VCSes out of which only one did this, but they all added extra paranoia hostname checks (some of them passing invalid hostnames, such as those with underscores in them). OK, I thought, then also let’s add extra checks to CVS’ repository reference handling. This will end up in Debian sid and MirBSD, pending passing the regression tests of course… hah, while writing this article I had to fixup because a test failed. Anyway, it’s not strictly necessary AFAICT to fix the issue.

Update, about 2⅕ hours past midnight (the testsuite runs for several hours): of course, the “sanity” testsuite (which itself is rather insane…) also needs adjustments, plus a bonus fix (for something that got broken when the recent allow-root-regex patch was merged and got fixed in the same go to…night).

tl;dr: a fix will end up in Debian *stable-security and can be taken out of my mail to the bugreport; another few changes for robustness are being tested and then added to both MirBSD and Debian sid. The impact is likely small, as it’s hard to get a user (if you find one, in the first place) to use a crafted CVSROOT string, which is easy to spot as well.

Update, Monday: apparently someone took care of the DSA and DLA yesterday after ACCEPTing the uploads — thanks, I was outside during the day.

Update 2017-08-25: It was noted that ssh(1) does not parse its command line correctly, and therefore the patch above might not be enough in the general case. However, I still think it’s good enough for CVS because it constructs its command line in a way that doesn’t let users exploit that bug.

mksh R56 was released with experimental fixes for the “history no longer persisted when HISTFILE near-full” and interactive shell cannot wait on coprocess by PID issues (I hope they do not introduce any regressioins) and otherwise as a bugfix release. You might wish to know the $EDITOR selection mechanism in dot.mkshrc changed. Some more alias characters are allowed again, and POSIX character classes (for ASCII, and EBCDIC, only) appeared by popular vote.

mksh now has a FAQ; enjoy. Do feel free to contribute (answers, too, of course).

The jupp text editor has also received a new release; asides from being much smaller, and updated (mksh too, btw) to Unicode 10, and some segfault fixes, it features falling back to using /dev/tty if stdin or stdout is not a terminal (for use on GNU with find | xargs jupp, since they don’t have our xargs(1) -o option yet), a new command to exit nonzero (sometimes, utilities invoking the generic visual editor need this), and “presentation mode”.

Presentation mode, crediting Natureshadow, is basically putting your slides as (UTF-8, with fancy stuff inside) plaintext files into one directory, with sorting names (so e.g. zero-padded slide numbers as filenames), presenting them with jupp * in a fullscreen xterm. You’d hit F6 to switch to one-file view first, then present by using F8 to go forward (F7 to go backward), and, for demonstrations, F9 to pipe the entire slide through an external command (could be just “sh”) offering the previous one as default. Simple yet powerful; I imagine Sven Guckes would love it, were he not such a vim user.

The new release is offered as source tarball (as usual) and in distribution packages, but also, again, a Win32 version as PKZIP archive (right-click on setup.inf and hit I̲nstall to install it). Note that this comes with its own (thankfully local) version of the Cygwin32 library (compatible down to Windows 95, apparently), so if you have Cygwin installed yourself you’re better off compiling it there and using your own version instead.

I’ve also released a new DOS version of 2.8 with no code patches but an updated jupprc; the binary (self-extracting LHarc archive) this time comes with all resource files, not just jupp’s.

Today, the jupprc drop-in file for JOE 3.7 got a matching update (and some fixes for bugs discovered during that) and I added a new one for JOE 4.4 (the former being in Debian wheezy, the latter in jessie, stretch and buster/sid). It’s a bit rudimentary (the new shell window functionality is absent) but, mostly, gives the desired jupp feeling, more so than just using stock jstar would.

CVS’ ability to commit to multiple branches of a file at the same time, therefore grouping the commit (by commitid at least, unsure if cvsps et al. can be persuaded to recognise it). If you don’t know what cvs(GNU) is: it is a proper (although not distributed) version control system and the best for centralised tasks. (For decentral tasks, abusing git as pseudo-VCS has won by popularity vote; take this as a comparison.)

If desired, I can make these new versions available in my “WTF” APT repository on request. (Debian buster/sid users: please change “https” to “http” there, the site is only available with TLSv1.0 as it doesn’t require bank-level security.)

I’d welcome it very much if people using an OS which does not yet carry either to package it there. Message me when one more is added, too ☺

In unrelated news I uploaded MuseScore 2.1 to Debian unstable, mostly because the maintainers are busy (though I could comaintain it if needed, I’d just need help with the C++ and CMake details). Bonus side effect is that I can now build 2.2~ test versions with patches of mine added I plan to produce to fix some issues (and submit upstream) ☻

In other news, I’m working on a new i386+sparc MirBSD snapshot more than ever. Mostly to get everything old out from under my feet before tackling the LibreSSL import (to get TLSv1.2 support, due to the aforementioned idio…decision). I’ve yet to see whether our G++ port works on sparc, and I’ve yet to create ports for libGLU and xlock which used to be in the base X system but had to go away for being written in an unmaintainable language (plus a system is only reliable if it has only one libstdc++), but it’ll be a good stepping stone (plus mfny asked for a sparc snapshot on IRC). I was considering distributing ISOs at FrOSCon but, with an installed user base in the single digits (likely), you can imagine how useful that’d be. (Fun side idea: distribute ISOs with a boot menu where you can choose not only MirBSD installer or live system but also “minimal Debian system directly booting into the MirBSD live system running under qemu-kvm”. But I’ve got not enough spare time right now.)

Hurried snapshot and known issues

07.08.2017 by tg@
Tags: mksh pcli plan snapshot

As already mentioned I planned creating a new snapshot. Well, it will be out shortly, albeit in a hurried manner and not with everything I had planned for it, and with lagging sparc (as if that were new, though…). A hurried mksh release will there be as well. The reason for this is the top #1 known issue:

  • Debian OpenSSL now excludes TLS < 1.2 from communication
    ⇒ there will be some followup release with LibreSSL, I think
  • There’s still no port for libGLU and xlock
  • We didn’t import lzlib into base yet, nor recent fixes to pax(1) from OpenBSD necessary
  • The new Unicode property code is not written yet (although I fixed the data shipped so it matches, at least)
  • I didn’t test g++ from ports on sparc yet, we’ll see how that goes

That being said, you’ll be able to work with what I’ve got, like in olden times when MirBSD was defined as “the contents of my /usr/src and /usr/ports” and be assured that, besides working on things like MuseScore in the meantime, I’m on it.

An unrelated minor update to another recent post; apparently I managed to make the GitHub Legal people aware enough of the problems that they are working on fixing their ToS; I admit there’s been an update since August 1ˢᵗ/2ⁿᵈ which I haven’t yet gotten around to reading at all.

wtf rocks; Eugen is working on an iOS äpp and already has a beta version which just needs bugfixing.

Beltane Snapshot and Mainframe Korn Shell…ish

05.05.2017 by tg@
Tags: mksh pcli plan snapshot

I was planning to do an mksh R56 release and then a full MirBSD snapshot (i386, sparc — due to actual user request — and possibly even a Live CD or at least baselive) but this got stones on my way.

I’m not quite finished with what I originally had planned for R56 (basically, the Debian postfix package’s maintainer scripts started using character classes in bracket expressions, and this required not only careful planning and design but also quite some rewriting and thinking, fixing other bugs, reading the specs, and considering EBCDIC) which led to me asking the EBCDIC porter some things again, which led to trying to merge his outstanding patches and make R56 the Mainframe Korn Shell release (also mksh ;-) but we’re not quite there yet.

The MirBSD snapshot was planned to be started from CVS as of Beltane (Walpurgis) 2017 except the latest and greatest mksh is also kinda a requirement, and CVE fixes are tricking in, to add insult to injury for stuff I had just updated. I’d also love to have the latest sendmail and lynx in it but that’ll have to wait.

I’ll also do a new CVS snapshot tarball at the same time, so keep your eyes open for the new rolling MirBSD snapshot.

Somehow, spring weather does not agree with me anyway. I didn’t get much done, nor much good sleep. Private life’s looking up but also more busy. I do manage to still help out others even in code I don’t really understand… must be good karma. Vacation would be nice… but then, I know I wouldn’t get all out of it.

Updates to the last two posts

16.03.2017 by tg@
Tags: bug debian grml news pcli rant snippet tip work

Someone from the FSF’s licencing department posted an official-looking thing saying they don’t believe GitHub’s new ToS to be problematic with copyleft. Well, my lawyer (not my personal one, nor for The MirOS Project, but related to another association, informally) does agree with my reading of the new ToS, and I can point out at least a clause in the GPLv1 (I really don’t have time right now) which says contrary (but does this mean the FSF generally waives the restrictions of the GPL for anything on GitHub?). I’ll eMail GitHub Legal directly and will try to continue getting this fixed (as soon as I have enough time for it) as I’ll otherwise be forced to force GitHub to remove stuff from me (but with someone else as original author) under GPL, such as… tinyirc and e3.

My dbconfig-common Debian packaging example got a rather hefty upgrade because dbconfig-common (unlike any other DB schema framework I know of) doesn’t apply the upgrades on a fresh install (and doesn’t automatically put the upgrades into a transaction either) but only upgrades between Debian package versions (which can be funny with backports, but AFAICT that part is handled correctly). I now append the upgrades to the initial-version-as-seen-in-the-source to generate the initial-version-as-shipped-in-the-binary-package (optionally, only if it’s named .in) removing all transaction stuff from the upgrade files and wrapping the whole shit in BEGIN; and COMMIT; after merging. (This should at least not break nōn-PostgreSQL databases and… well, database-like-ish things I cannot test for obvious (SQLite is illegal, at least in Germany, but potentially worldwide, and then PostgreSQL is the only remaining Open Source database left ;) reasons.)

Update: Yes, this does mean that maintainers of databases and webservers should send me patches to make this work with not-PostgreSQL (new install/name.in, upgrade files) and not-Apache-2.2/2.4 (new debian/*/*.conf snippets) to make this packaging example even more generally usable.

Natureshadow already forked this and made a Python/Flask package from it, so I’ll prod him to provide a similarily versatile hello-python-world example package.

Since I use this as base for other PHP packages like SimKolab, I’ve updated my packaging example with:

  • PHP 7 support (untested, as I need libapache2-mod-php5)
  • tons more utility code for you to use
  • a class autoloader, with example (build time, for now)
  • (at build time) running a PHPUnit testsuite (unless nocheck)

The old features (Apache 2.2 and 2.4 support, dbconfig-common, etc.) are, of course, still there. Support for other webservers could be contributed by you, and I could extend the autoloader to work at runtime (using dpkg triggers) to include dependencies as packaged in other Debian packages. See, nobody needs “composer”! ☻

Feel free to check it out, play around with it, install it, test it, send me improvement patches and feature requests, etc. — it’s here with a mirror at GitHub (since I wrote it myself and the licence is permissive enough anyway).

This posting and the code behind it are sponsored by my employer ⮡ tarent.

Please use the correct (perma)link to bookmark this article, not the page listing all wlog entries of the last decade. Thank you.</update>

Some updates inline and at the bottom.

The new Terms of Service of GitHub became effective today, which is quite problematic — there was a review phase, but my reviews pointing out the problems were not answered, and, while the language is somewhat changed from the draft, they became effective immediately.

Now, the new ToS are not so bad that one immediately must stop using their service for disagreement, but it’s important that certain content may no longer legally be pushed to GitHub. I’ll try to explain which is affected, and why.

I’m mostly working my way backwards through section D, as that’s where the problems I identified lie, and because this is from easier to harder.

Note that using a private repository does not help, as the same terms apply.

Anything requiring attribution (e.g. CC-BY, but also BSD, …)

Section D.7 requires the person uploading content to waive any and all attribution rights. Ostensibly “to allow basic functions like search to work”, which I can even believe, but, for a work the uploader did not create completely by themselves, they can’t grant this licence.

The CC licences are notably bad because they don’t permit sublicencing, but even so, anything requiring attribution can, in almost all cases, not “written or otherwise, created or uploaded by our Users”. This is fact, and the exceptions are few.

Anything putting conditions on the right to “use, display and perform” the work and, worse, “reproduce” (all Copyleft)

Section D.5 requires the uploader to grant all other GitHub users…

  • the right to “use, display and perform” the work (with no further restrictions attached to it) — while this (likely — I didn’t check) does not exclude the GPL, many others (I believe CC-*-SA) are affected, and…
  • the right to “reproduce your Content solely on GitHub as permitted through GitHub's functionality”, with no further restructions attached; this is a killer for, I believe, any and all licences falling into the “copyleft” category.

Note that section D.4 is similar, but granting the licence to GitHub (and their successors); while this is worded much more friendly than in the draft, this fact only makes it harder to see if it affects works in a similar way. But that doesn’t matter since D.5 is clear enough. (This doesn’t mean it’s not a problem, just that I don’t want to go there and analyse D.4 as D.5 points out the same problems but is easier.)

This means that any and all content under copyleft licences is also no longer welcome on GitHub.

Anything requiring integrity of the author’s source (e.g. LPPL)

Some licences are famous for requiring people to keep the original intact while permitting patches to be piled on top; this is actually permissible for Open Source, even though annoying, and the most common LaTeX licence is rather close to that. Section D.3 says any (partial) content can be removed — though keeping a PKZIP archive of the original is a likely workaround.

Affected licences

Anything copyleft (GPL, AGPL, LGPL, CC-*-SA) or requiring attribution (CC-BY-*, but also 4-clause BSD, Apache 2 with NOTICE text file, …) are affected. BSD-style licences without advertising clause (MIT/Expat, MirOS, etc.) are probably not affected… if GitHub doesn’t go too far and dissociates excerpts from their context and legal info, but then nobody would be able to distribute it, so that’d be useless.

But what if I just fork something under such a licence?

Only “continuing to use GitHub” constitutes accepting the new terms. This means that repositories from people who last used GitHub before March 2017 are excluded.

Even then, the new terms likely only apply to content uploaded in March 2017 or later (note that git commit dates are unreliable, you have to actually check whether the contribution dates March 2017 or later).

And then, most people are likely unaware of the new terms. If they upload content they themselves don’t have the appropriate rights (waivers to attribution and copyleft/share-alike clauses), it’s plain illegal and also makes your upload of them or a derivate thereof no more legal.

Granted, people who, in full knowledge of the new ToS, share any “User-Generated Content” with GitHub on or after 1ˢᵗ March, 2017, and actually have the appropriate rights to do that, can do that; and if you encounter such a repository, you can fork, modify and upload that iff you also waive attribution and copyleft/share-alike rights for your portion of the upload. But — especially in the beginning — these will be few and far between (even more so taking into account that GitHub is, legally spoken, a mess, and they don’t even care about hosting only OSS / Free works).

Conclusion (Fazit)

I’ll be starting to remove any such content of mine, such as the source code mirrors of jupp, which is under the GNU GPLv1, now and will be requesting people who forked such repositories on GitHub to also remove them. This is not something I like to do but something I am required to do in order to comply with the licence granted to me by my upstream. Anything you’ve found contributed by me in the meantime is up for review; ping me if I forgot something. (mksh is likely safe, even if I hereby remind you that the attribution requirement of the BSD-style licences still applies outside of GitHub.)

(Pet peeve: why can’t I “adopt a licence” with British spelling? They seem to require oversea barbarian spelling.)

The others

Atlassian Bitbucket has similar terms (even worse actually; I looked at them to see whether I could mirror mksh there, and turns out, I can’t if I don’t want to lose most of what few rights I retain when publishing under a permissive licence). Gitlab seems to not have such, but requires you to indemnify them… YMMV. I think I’ll self-host the removed content.

And now?

I’m in contact with someone from GitHub Legal (not explicitly in the official capacity though) and will try to explain the sheer magnitude of the problem and ways to solve this (leaving the technical issues to technical solutions and requiring legal solutions only where strictly necessary), but for now, the ToS are enacted (another point of my criticism of this move) and thus, the aforementioned works must go off GitHub right now.

That’s not to say they may not come back later once this all has been addressed, if it will be addressed to allow that. The new ToS do have some good; for example, the old ToS said “you allow every GitHub user to fork your repositories” without ever specifying what that means. It’s just that the people over at GitHub need to understand that, both legally and technically¹, any and all OSS licences² grant enough to run a hosting platform already³, and separate explicit grants are only needed if a repository contains content not under an OSI/OKFN/Copyfree/FSF/DFSG-free licence. I have been told that “these are important issues” and been thanked for my feedback; we’ll see what comes from this.

① maybe with a little more effort on the coders’ side³

② All licences on one of those lists or conformant to the DFSG, OSD or OKD should do⁴.

③ e.g. when displaying search results, add a note “this is an excerpt, click HERE to get to the original work in its context, with licence and attribution” where “HERE” is a backlink to the file in the repository

④ It is understood those organisations never un-approve any licence that rightfully conforms to those definitions (also in cases like a grant saying “just use any OSS² licence” which is occasionally used)

Update: In the meantime, joeyh has written not one but two insightful articles (although I disagree in some details; the new licence is only to GitHub users (D.5) and GitHub (D.4) and only within their system, so, while uploaders would violate the ToS (they cannot grant the licence) and (probably) the upstream-granted copyleft licence, this would not mean that everyone else wasn’t bound by the copyleft licence in, well, enough cases to count (yes it’s possible to construct situations in which this hurts the copyleft fraction, but no, they’re nowhere near 100%).

How to use the subtree git merge strategy

20.12.2016 by tg@
Tags: debian grml pcli tip work

This article might be perceived as a blatant ripoff of this Linux kernel document, but, on the contrary, it’s intended as add-on, showing how to do a subtree merge (the multi-project merge strategy that’s actually doable in a heterogenous group of developers, as opposed to subprojects, which many just can’t wrap their heads around) with contemporary git (“stupid content tracker”). Furthermore, the commands are reformatted to be easier to copy/paste.

To summarise: you’re on the top level of a checkout of the project into which the “other” project (Bproject) is to be merged. We wish to merge the top level of Bproject’s “master” branch as (newly created) subdirectory “dir-B” under the current project’s top level.

	$ git remote add --no-tags -f Bproject /path/to/B/.git
	$ git merge -s ours --allow-unrelated-histories --no-commit Bproject/master
	$ git read-tree -u --prefix=dir-B/ Bproject/master
	$ git commit -m 'Merge B project as our subdirectory dir-B'

	Later updates are easy:
	$ git pull -s subtree Bproject master
 

(mind the trailing slash after dir-B/ on the read-tree command!)

Besides reformatting, the use of --allow-unrelated-histories recently became necessary. --no-tags is also usually what you want, because tags are not namespaced like branches.

Another command you might find relevant is how to clean up orphaned remote branches:

	$ for x in $(git remote); do git remote prune "$x"; done
 

This command locally deletes all remote branches (those named “origin/foo”) that have been deleted on the remote side.

Update: Natureshadow wishes you to know that there is such a command as git subtree which can do similar things to the subtree merge strategy explained above, and several more related things. It does, however, need the præfix on every subsequent pull.

“I don’t like computers”

13.11.2016 by tg@
Tags: debian pcli personal rant tip

cnuke@ spotted something on the internet, and shared. Do read this, including the comments. It’s so true. (My car is 30 years old, I use computers mostly for sirc, lynx and ssh, and I especially do not buy any product that needs to be “online” to work.)

Nice parts of the internet, to offset this, though, do exist. IRC as a way of cheap (affordable), mostly reliant, communication that’s easy enough to do with TELNET.EXE if necessary. Fanfiction; easy proliferation of people’s art (literature, in this case). Fast access to documentation and source code; OpenBSD’s AnonCVS was a first, nowadays almost everything (not Tom Dickey’s projects (lynx, ncurses, xterm, cdk, …), nor GNU bash, though) is on a public version control system repository. (Now people need to learn to not rewrite history, just commit whatever shit they do, to record thought process, not produce the perfect-looking patch.) Livestreams too, I guess, but ever since live365.com went dead due to a USA law change on 2016-01-02, it got bad.

Please save GMane!

28.07.2016 by tg@
Tags: debian news pcli rant

GMane has been down for a day or two, and flakey for a day before that. MidnightBSD’s laffer1 just linked the reason, which made me cry out loud.

GMane is really great, and I rely on the NNTP interface a lot, both posting and especially reading — it gives me the ability to download messages from mailing lists I don’t receive in order to be able to compose replies with (mostly) correct References and In-Reply-To headers. Its web interface, especially the article permalinks, are also extremely helpful.

This is a request for a petition to save GMane. Please, someone, do something! Thanks in advance!

The MirBSD Korn Shell R52c was published today as bugfix-accumulating release of low upto medium importance. Thanks to everyone who helped squashing all those bugs; this includes our bug reporters who always include reproducer testcases; you’re wonderful!

MirCPIO was also resynchronised from OpenBSD, to address the CVE-2015-{1193,1194} test cases, after a downstream (wow there are so many?) reminded us of it; thanks!
This is mostly to prevent extracting ../foo – either directly or from a symlink(7) – from actually ending up being placed in the parent directory. As such the severity is medium-high. And it has a page now – initially just a landing page / stub; will be fleshed out later.

Uploads for both should make their way into Debian very soon (these are the packages mksh and pax). Uploading backports for mksh (jessie and wheezy-sloppy) have been requested by several users, but none of the four(?) DDs asked about sponsoring them even answered at all, and the regular (current) sponsors don’t have experience with bpo, so… SOL ☹

I’ve also tweaked a bug in sed(1), in MirBSD. Unfortunately, this means it now comes with the GNUism -i too: don’t use it, use ed(1) (much nicer anyway) or perlrun(1) -p/-n…

Finally, our PDF manpages now use the PA4 paper size instead of DIN ISO A4, meaning they can be printed without cropping or scaling on both A4 and US-american “letter” paper. And a Бодун from the last announcement: we now use Gentium and Inconsolata as body text and monospace fonts, respectively. (And à propos, the website ought to be more legible due to text justification and better line spacing now.) I managed to hack this up in GNU groff and Ghostscript, thankfully. (LaTeX too) Currently there are PDF manpages for joe (jupp), mksh, and cpio/pax/tar.

And we had Grünkohl today!

Also, new console-setup package in the “WTF” APT repository since upstream managed to do actual work on it (even fixed some bugs). Read its feed if interested, as its news will not be repeated here usually. (That means, subscribe as there won’t be many future reminders in this place.)

The netboot.me service appears to be gone. I’ll not remove our images, but if someone knows what became of it drop us a message (IRC or mailing list will work just fine).

PS: This was originally written on 20160304 but opax refused to be merged in time… Happy Birthday, gecko2! In the meantime, the Street Food festival weekend provided wonderful food at BaseCamp, and headache prevented this from being finished on the fifth.

Update 06.03.2016: The pax changes were too intrusive, so I decided to only backport the fixes OpenBSD did (both those they mentioned and those silently included), well, the applicable parts of them, anyway, instead. There will be a MirCPIO release completely rebased later after all changes are merged and, more importantly, tested. Another release although not set for immediate future should bring a more sensible (and mksh-like) buildsystem for improved portability (and thus some more changes we had to exclude at first).

I’ve also cloned the halfwidth part of the FixedMisc [MirOS] font as FixedMiscHW for use with Qt5 applications, xfonts-base in the “WTF” APT repo. (Debian #809979)

tl;dr: mksh R52c (bugfix-only, low-medium); mircpio 20160306 (security backport; high) with future complete rebase (medium) upstream and in Debian. No mksh backports due to lacking a bpo capable sponsor. New console-setup in “WTF” APT repo, and mksh there as usual. xfonts-base too. netboot.me gone?

The things you find in upstream code…

13.02.2016 by tg@
Tags: archaeology bug pcli rant security snapshot

I had just gotten an eMail from the nightly /etc/security cronjob that the mailbox from the user foo.lock belongs to the user foo (name changed to protect the… innocent? well, I know that guy from #OpenBSD on IRC, so… YMMV… anyway). Of course, I wanted to change that to exclude mbox lockfiles…

	# Mailboxes should be owned by user and unreadable.
	ls -l /var/mail | sed 1d | \
	awk '$3 != $9 \
		{ print "user " $9 " mailbox is owned by " $3 }
 

… oh wow. Needless to say I fixed that, although you must update your stat(1) first; it now has a possibility to generate NUL-terminated output (or any separator, really) which I used for this. (And no, Schily, I’m still of the opinion that NUL termination, even when one has to add it to each utility separately, is the better way to go.)

Dear OpenBSD developers, repeat after me:
Do n̲o̲t̲ parse ls(1) output!
Or write 100 lines of it, or something, until it sinks in.

(It can take some writing for it to sink in… just yesternight the fanfiction I was reading was at the point where Dolores Umbridge uses her Blood Quill on the students. Coincidence.)

Our PDF manpages will, starting from now, be generated with Inconsolata instead of Bitstream Vera Mono as monospace font. The body font is still Gentium, of course.

To be more exact: the Teχ flavour of Inconsolata Regular and Bold, with the varl and varqu flags, is used, and because GNU groff also requires an Italic or at least Oblique font (also in its bold variant, which the mksh(1) manpage doesn’t use though), Inconsolata LGC (both Italic and Bold Italic) are plugged in there. I added them as PFA Type 1 fonts to GNU groff, so I had to make some fixes in FontForge (merging the variants into the main font, removing unused glyphs (not for LGC), fixing the validation (mostly, and not so much for LGC), autohinting where FontForge expressed a need for that, renaming glyphs to the names expected by afmtodit, …), but it works.

I’m not regenerating older PDF manpages though.

Inconsolata is also not all I wish for a monospaced font (and even bsiegert@ says nothing goes over FixedMisc) but it has, at least, a 0 (digit zero) with a correct stroke through it ☺

expect turmoil

08.02.2016 by tg@
Tags: archaeology bug hardware news pcli personal plan rant

My network at home is unstable. NetCologne suggests to switch to fibre network, but that only comes with a dynamic IPv6 address and NAT64; completely unsuitable to running a server. (I could arguably tunnel a static IPv4 address from a dedicated server to home, but that would completely foil my plans for redundancy.) So I may need an ISP (phone isn’t important) that provides me with connectivity where a static IPv4 (and, ideally, a static IPv6 /64 or /48 – but only if the reverse DNS gets delegated to me, otherwise that’s unusable) ends up at a device of my choosing (and not a plastic router which can then “forward ports”; I require full internet to end up at my own device).

HostEurope is relocating the other server, both physically and network-wise. Their plan seems fool-proof so far, though.

gecko2@ is decommissioning the server on which eurynome is hosted, shortly. This will also be no small amount of fun for everyone involved. Expect old links, SSH host keys, etc. to break. This explicitly includes /etc/ssh/*known_hosts.

During all those moves, I will downsize my DNS zones and change some entries, so that old or duplicate records will be gone.

I’ll likely generate and publish completely new hostkeys (both gzsig(1) and PGP clearsigned) once this is all over. The current gzsig(1) key is at the end of /usr/share/doc/README in any installed system. (Do note MD5 is considered insecure.) My current PGP key is 9031955E7A97A4FDA32B2B8676B534B2E99007E0 but this requires GnuPG, so check both.

My seeming inability to remember rarely-used “secure” passwords, i.e. those not fitting into my normal schemata, led to me not attempting to run a CA myself any more. While, thanks to rsc, we have an official certificate for www.mirbsd.org now, I probably will get StartSSL for “all” other systems (i.e. herc, as I appear to be downsizing), despite it lacking the SSL client purpose (important e.g. to SMTP). This shouldn’t affect anyone.

PS: I still hate Karneval!

I just published the first version of git find on gh/mirabilos/git-find for easy collaboration. The repository deliberately only contains the script and the manual page so it can easily be merged into git.git with complete history later, should they accept it. git find is MirOS licenced. It does require a recent mksh (Update: I did start it in POSIX sh first, but it eventually turned out to require arrays, and I don’t know perl(1) and am not going to rewrite it in C) and some common utility extensions to deal with NUL-separated lines (sort -z, grep -z, git ls-tree -z); also, support for '\0' in tr(1) and a comm(1) that does not choke on embedded NULs in lines.

To install or uninstall it, run…

	$ git clone git@github.com:mirabilos/git-find.git
	$ cd git-find
	$ sudo ln -sf $PWD/git-find /usr/lib/git-core/
	$ sudo cp git-find.1 /usr/local/share/man/man1/
	… hack …
	$ sudo rm /usr/lib/git-core/git-find \
	    /usr/local/share/man/man1/git-find.1

… then you can call it as “git find” and look at the documentation with “git help find”, as is customary.

The idea behind this utility is to have a tool like “git grep” that acts on the list of files known to git (and not e.g. ignored files) to quickly search for, say, all PNG files in the repository (but not the generated ones). “git find” acts on the index for the HEAD, i.e. whatever commit is currently checked-out (unlike “git grep” which also knows about “git add”ed files; fix welcome) and then offers a filter syntax similar to find(1) to follow up: parenthesēs, ! for negation, -a and -o for boolean are supported, as well as -name, -regex and -wholename and their case-insensitive variants, although regex uses grep(1) without (or, if the global option -E is given, with) -E, and the pattern matches use mksh(1)’s, which ignores the locale and doesn’t do [[:alpha:]] character classes yet. On the plus side, the output is guaranteed to be sorted; on the minus side, it is rather wastefully using temporary files (under $TMPDIR of course, so use of tmpfs is recommended). -print0 is the only output option (-print being the default).

Another mode “forwards” the file list to the system find; since it doesn’t support DOS-style response files, this only works if the amount of files is smaller than the operating system’s limit; this mode supports the full range (except -maxdepth) of the system find(1) filters, e.g. -mmin -1 and -ls, but it occurs filesystem access penalty for the entire tree and doesn’t sort the output, but can do -ls or even -exec.

The idea here is that it can collaboratively be improved, reviewed, fixed, etc. and then, should they agree, with the entire history, subtree-merged into git.git and shipped to the world.

Part of the development was sponsored by tarent solutions GmbH, the rest and the entire manual page were done in my vacation.

mksh R52 released

12.12.2015 by tg@
Tags: mksh pcli

The MirBSD Korn Shell R52 was published today. While there are still several known bugs, this is a release that primarily fixes lots of these, and, as with R51, we have no known regressions. Some of the itinerary for R52 has moved to R53 instead, as some bugfixes change the shell language and thus warrant a new major version, which is why this is not R51b, and they accumulated and could use some testing ;-)

This release has a nōnzero chance to break existing scripts that use some extension features – I had to quote some tildes in dot.mkshrc and a variable expansion in ${x/y"$z"} in MirWebseite (the $z) – twice!. As usual, test!

In less related news, a new release of the FixedMisc [MirOS] font is available (in BDF form and no conflict with the system Fixed [Misc] font); our CVS has the sources in bdfctool(1) format.

Go enjoy shell

27.08.2015 by tg@
Tags: debian fun pcli

Dimitri, I personally enjoy shell…

tglase@tglase:~ $ x=車賈滑豈更串句龜龜契金喇車賈滑豈更串句龜龜契金喇
tglase@tglase:~ $ echo ${x::12}
車賈滑豈更串句龜龜契金喇
tglase@tglase:~ $ printf '%s\n' 'import sys' 'print(sys.argv[1][:12])' >x.py
tglase@tglase:~ $ python x.py $x
車賈滑豈
 

… much more than Python, actually. (Python is the language in which you do not want to write code dealing with strings, due to UnicodeDecodeError and all; even py3k is not much better.)

I would have commented on your post if it allowed doing so without getting a proprietary Google+ account.

The MirBSD Korn Shell R51 was published today. This is a feature release clearly, but still something a lot of people would wish to use. It contains several known severe bugs, but they all are no regressions, i.e. they exist in R50f already.

This one is kinda an early release, as I wish to have those known issues all fixed, but the changes – both deep down and enduser-visible – already warrant people looking for breakages, plus it makes synchronisation with mksh-os2 easier.

mksh R52 will follow, as bugfix release, pretty soon. Itinerary:

  • Fixes for as much of these known bugs as possible (code rewrites)
  • Unicode 8
  • New feature: print -a
  • Fixes for bugs reported against R51
  • Possibly more EBCDIC and OS/2 code synchronisation
  • Maybe a dead useful debug tool…

Once that’s out, I’ll roll up the fixes into R50g, so that particular code branch is not dead yet either ☺

And afterwards, at least mksh(1)-wise – I have got a lot of other things on my plate after all – we can attempt getting EBCDIC and maybe OS/2 to a state where the code is included in CVS.

carstenh asked in IRC how to make a shebang for mksh(1) scripts that works on both regular Unix and Android.

This is not as easy as it looks, though. Most Unicēs will have mksh installed, either manually or by means of the native package system, as /bin/mksh. Some put it into package manager-specific directories; I saw /sw/bin/mksh, /usr/local/bin/mksh and /usr/pkg/bin/mksh so far. Some systems have it as /usr/bin/mksh but these are usually those who got poettering’d and have /bin a symlink anyway. Most of these systems also have env(1) as /usr/bin/env.

Android, on the contrary, ships with precisely one shell. This has been mksh for a while, thankfully. There is, however, neither a /bin nor a /usr directory. mksh usually lives as /system/bin/mksh, with /system/bin/sh a symlink(7) to the former location. Some broken Android versions ship the binary in the latter location instead and do not ship anything that matches mksh on the $PATH, but I hope they merge my AOSP patch to revert this bad change (especially as some third-party Android toolkits overwrite /system/bin/sh with busybox sh or GNU bash and you’d lose mksh in the progress). However, on all official Android systems, mksh is the system shell. This will be important later.

The obvious and correct fix is, of course, to chmod -x the scripts and call them explicitly as mksh scriptname. This is not always possible or desirable; sometimes, people will wish it to be in the $PATH and executable, so we need a different solution.

There’s a neat trick with shebangs – the absence of one is handled specifically by most systems in various ways. I remember reading about it, but don’t remember where; I can’t find it on Sven Mascheck’s excellent pages… but: the C shell variants run a script with the Bourne Shell if its first line is a sole colon (‘:’), the Bourne family shells run it with themselves or ${EXECSHELL:-/bin/sh} in those cases, and the kernel with the system shell, AFAIK. So we have a way to get most things that could call the script to interpret it as Bourne/POSIX shell script on most systems. Then we just have to add a Bourne shell scriptlet that switches to mksh iff the current shell isn’t it (lksh, or something totally different). On Android, there is only ever one shell (or the toolkit installer better preserve mksh as mksh), so this doesn’t do anything (I hope – but did not test – that the kernel invokes the system shell correctly despite it not lying under /bin/sh) nor does it need to.

This leaves us with the following “shebang”:

	:
	case ${KSH_VERSION-} in
	*MIRBSD\ KSH*) ;;
	*)	# re-run with The MirBSD Korn Shell, this is an mksh-specific script
		test "${ZSH_VERSION+set}" = set && alias -g '${1+"$@"}'='"$@"'
		exec mksh "$0" ${1+"$@"}
		echo >&2 E: mksh re-exec failed, should not happen
		exit 127 ;;
	esac
 

The case argument not only does not need to, but actually should not be quoted; the expansion is a set -u guard; the entire scriptlet is set -e safe as well; comments and expansions are safe. exec shall not return, but if it does (GNU bash violates POSIX that way, for example), we use POSIX’ appropriate errorlevel. zsh is funny with the Bourne shell’s way of using "$@" properly. But this should really be portable. The snippet is both too short and too obvious (“only way to do it”) to be protected by copyright law.

Thanks to carstenh and Ypnose for discussing things like this with us in IRC, sending in bugfixes (and changes we decline, with reason), etc. – it feels like we have a real community, not just consuments ☺

mksh R50f coming soon

11.04.2015 by tg@
Tags: mksh pcli

Please test mksh-current from CVS (or the inofficial git mirror)! There are security-related fixes I’ll MFC in a few days, for which I’d prefer for them (and the other changes) to not introduce any regressions. Thanks!

WTF is Jessie; PA4 paper size

12.12.2014 by tg@
Tags: debian pcli rant

My personal APT repository now has a jessie suite – currently just a clone of the sid suite, but so, people can get on the correct “upgrade channel” already.

Besides that, the usual small updates to my metapackages, bugfixes, etc. – You might have noticed that it’s now on a (hopefully permanent) location. I’ve put a donated eee-pc from my father to good use and am now running a Debian system at home. (Fun, as I’m emeritus now, officially, and haven’t had one during my time as active uploading DD.) I’ve created a couple of cowbuilder chroots (pbuilderrc to achieve that included in the repo) and can build packages, but for i386 only (amd64 is still done on the x32 desktop at work), but, more importantly, I can build, sign and publish the repo, so it may grow. (popcon data is interesting. More than double the amount of machines I have installed that stuff on.)

Update: I’ve started writing a NEWS file and cobbled together an RSS 2.0 feed from that… still plaintext content, but at least signalling in feedreaders upon updates.


Installing gimp and inkscape, I’m asked for a default paper size by libpaper1. PA4 is still not an option, I wonder why. I also haven’t managed to get MirPorts GNU groff and Artifex Ghostscript to use that paper size, so the various PDF manpages I produce are still using DIN ISO A4, rendering e.g. Mexicans unable to print them. Help welcome.


Note, for arngc, you need a server component (MirBSD-current, of course; we’re rolling release nowadays). Config included, but I’m willing to open my firewall to people I know, provided they won’t use “too much” traffic (running a couple of arngc instances is fine, according to what I estimated).

A largish article about how to use some other packages in the repo, such as dash-mksh, is yet to come. In the meantime, I wrote a bit more in README.Debian in mirabilos-support.

Bernhard’s article on Plänet Debian about the “colon” command in the shell could use a clarification and a security-relevant correcture.

There is, indeed, no difference between the : and true built-in commands.

Stéphane Chazelas points out that writing : ${VARNAME:=default} is bad, : "${VARNAME:=default}" is correct. Reason: someone could preset $VARNAME with, for example, /*/*/*/*/../../../../*/*/*/*/../../../../*/*/*/* which will exhaust during globbing.

Besides that, the article is good. Thanks Bernhard for posting it!

PS: I sometimes use the colon as comment leader in the last line of a script or function, because it, unlike the octothorpe, sets $? to 0, which can be useful.

Update: As jilles pointed out in IRC, “colon” (‘:’) is a POSIX special built-in (most importantly, it keeps assignments), whereas “true” is a regular built-in utility.

mksh R50d released

07.10.2014 by tg@
Tags: bug debian mksh news pcli

The last MirBSD Korn Shell update broke update-initramfs because I accidentally introduced a regression in field splitting while fixing other bugs – sorry!

mksh R50d was just released to fix that, and a small NULL pointer dereference found by Goodbox on IRC. Thanks to my employer tarent for a bit of time to work on it.

mksh R50c released, security fix

03.10.2014 by tg@
Tags: android bug debian mksh news pcli release security

The MirBSD Korn Shell has got a new security and maintenance release.

This release fixes one mksh(1)-specific issue when importing values from the environment. The issue has been detected by the main developer during careful code review, looking at whether the shell is affected by the recent “shellshock” bugs in GNU bash, many of which also affect AT&T ksh93. (The answer is: no, none of these bugs affects mksh.) Stephane Chanzelas kindly provided me with an in-depth look at how this can be exploited. The issue has not got a CVE identifier because it was identified as low-risk. The problem here is that the environment import filter mistakenly accepted variables named “FOO+” (for any FOO), which are, by general environ(7) syntax, distinct from “FOO”, and treated them as appending to the value of “FOO”. An attacker who already had access to the environment could so append values to parameters passed through programs (including sudo(8) or setuid) to shell scripts, including indirectly, after those programs intended to sanitise the environment, e.g. invalidating the last $PATH component. It could also be used to circumvent sudo’s environment filter which protected against the vulnerability of an unpatched GNU bash being exploited.

tl;dr: mksh not affected by any shellshock bugs, but we found a bug of our own, with low impact, which does not affect any other shell, during careful code review. Please do update to mksh R50c quickly.

mksh R50b released

03.09.2014 by tg@
Tags: mksh news pcli

The MirBSD Korn Shell has got a new bugfix release. Thought you’d want to know ☺

mksh R50, jupp 27 released

29.06.2014 by tg@
Tags: jupp mksh news pcli

Both the MirBSD Korn Shell and jupp – the editor which sucks less have seen new releases today. Please test them, report all bugs, and otherwise enjoy all the bugfixes.

Other subprojects will also have new releases… once I get around doing so after hacking them…

Update 03.07.2014: New release for MirCPIO, that is, cpio(1) and pax(1) and tar(1) in a somewhat portable package.

-r--r--r-- 4 tg miros-cvssrc 141973 Jul 3 19:56 /MirOS/dist/mir/cpio/paxmirabilis-20140703.cpio.gz

Dear FSF, stop recommending Enigmail.

05.06.2014 by tg@
Tags: debian pcli rant security tip work

Dear FSF, stop recommending Enigmail, please. It is broken, simple as that. Even if you switch everything HTML-related off, it still defaults to the latin9 (ISO-8859-15) encoding instead of UTF-8, and possibly some other nasties. Worse, it’s based upon obsolete Thunderbird/Icedove technology, which is dead since the release of Firefox® 17 and will only degrate over time.

Side note: I was asked recently how much entropy is used while generating a PGP key using GnuPG on Windows®, after having done the same for OpenSSL on Debian (and possibly almost all other OSes). I had to try to find out which was the actual code (GnuPG 2 with libgcrypt, it turns out), and it was not pretty. (You are hereby adviced to create a 600-byte file ${GNUPGHOME:-~/.gnupg}/random_seed from a good source before even attempting to use GnuPG 2 for the first time. OK, you can run gpg -k once, to create the GNUPGHOME directory from a skeleton.)

Stay off my computer, puppet!

18.04.2014 by tg@
Tags: bug debian fun geocache pcli rant tip work

I was out, seeing something that wasn’t there yet when I was at school (the “web” was not ubiquitous, back then), and decided to have a look:

pageok

Ugh. Oh well, PocketIE doesn’t provide a “View Source” thingy, so I asked Natureshadow (who got the same result on his Android, and had no “View Source” either apparently, so he used cURL to see it). We saw (here, re-enacted using ftp(1)):

	tg@blau:~ $ ftp -Vo - http://www.draitschbrunnen.de/
	<!-- pageok -->
	<!-- managed by puppet -->
	<html>
	<pre>pageok</pre>
	</html>
 

This is the final straw… after puppet managed to trash a sudoers(5) at work (I warned people to not introduce it) now it breaks websites. ☺

(Of course, tools are useful, but at best to the skill of their users. Merely dumbly copying recipes from “the ’net” without any understanding just makes debugging harder for those of us with skills.)

ObQuestion: Does anyone have ⓐ a transcript (into UTF-8) and ⓑ a translation for the other half of the OpenBSD 2.8 poster? (I get asked this regularily.)
Update: One person sent me the Kanji and Kana for it in UTF-8 「俺のマシンに手を出すな!」, and they and one more person told me it’s “Hands off my machine!” or “Don’t lay a hand on my machine!”. Now I’m not studying Japanese, but it LGTM in FixedMisc [MirOS], and JMdict from MirPorts says: ore no mashin ni te (w)o dasu na (roughly: my machine; particle; hands; particle; put out; prohibition) ☺ Thanks all, now I know what to tell visitors who wonder about that poster on my wall.

ObTip: I can install a few hundred Debian VMs at work manually before the effort needed to automate d-i would amortise. So I decided not to. Coworkers are shocked. I keep flexibility (can decide to have machines differ), and the boss accepts my explanations. Think before doing automation just for the sake of automation!

FreeWRT Archive

30.03.2014 by tg@
Tags: archaeology freewrt news pcli snapshot

As previously announced, the FreeWRT Project has been archived. You can access the content at the FreeWRT Archive Site on the MirWebseite.

ObRant: DST (Sommerzeit) sucks!

KISS

06.02.2014 by tg@
Tags: archaeology debian fun jupp pcli

Just saw this in my INBOX:

    B. The default init system for jessie will be a single /etc/rc script
 

I’d certainly vote that❣


In unrelated news, jupp 2.8 for DOS runs on cable3, which means it’ll still run on an original 8088/8086 ☻

Update 10.02.2014: The unobfuscated version of cable3 is called 8086tiny under the MIT licence. Thanks to the author for doing that (and not just dumping the IOCCC code) and to RT from the mksh(1) IRC channel for finding it on the ’net!

Thanks to Robert Scheck, jupp – the Editor which sucks less (a WordStar™-compatible Unix editor with lots of features, including a hex editor) is currently on its way to Fedora and EPEL (RHEL/CentOS 5 and 6).

Depending on your distribution, you will have it available within one to two weeks, I’m being told.

This adds another distribution to the list; jupp has been available in Debian and its derivates (some of which may not be named) for some time (due to user request), and the webpage contains Win32 binaries (made with Cygwin, an oldish version to be compatible to Win9x).

jupp is especially useful as programmers’ editor, but also used in teaching school-aged kids the joys of IT; Natureshadow has prepared a cheat sheet, which we will internationalise and localise, then link from the jupp homepage – so stay tuned! (I guess we’ll also need a concise list of jupp features, in lieu of advertising.)

mksh/Win32

18.07.2013 by tg@

Michael Langguth and Scalaris AG asked me to publish the mksh/Win32 Beta 14 source and binary archive, and it is with joy I’m doing this.

Checksums and Hashes

  • RMD160 (ports/mksh-w32-beta14.zip) = 0dc8ef6e95592bd132f701ca77c4e0a3afe46f24
  • TIGER (ports/mksh-w32-beta14.zip) = 966e548f9e9c1d5b137ae3ec48e60db4a57c9a0ed15720fb
  • 1181543005 517402 /MirOS/dist/mir/mksh/ports/mksh-w32-beta14.zip
  • MD5 (ports/mksh-w32-beta14.zip) = b57367b0710bf76a972b493562e2b6b5

Just a few words on it (more in the README.1st file included): this is a port of The MirBSD Korn Shell R39 to the native WinAPI; it’s not quite got the full Unix feel (especially as it targets the Weihenstephan unxutils instead of a full Interix or Cygwin environment) but doesn’t need a full POSIX emulation layer either. It’s intended to replace MKS ksh and the MKS Toolkit. Source for the compatibility library is also included under The MirOS Licence; we aim at publishing it as OSI Certified Open Source Software like mksh itself. (There is a situation with dlmalloc/nedmalloc being resolved, and the icon is derived from the BSD dæmon which is a protected unregistered trademark, but we’re not Mozilla and allow distro packages to keep using it ☺) Rebasing it on a newer mksh(1) followed by (partial) integration into the main source code is a goal.

Have fun trying it out and hacking on it. It’s currently built with -DMKSH_NOPROSPECTOFWORK (so coprocesses and a few other minor things won’t work), but a SIGCHLD emulation is being worked on – but if you want to help out, I’m sure it’s welcome, just come on IRC or post on the mailing list, and I’ll forward things to Michael as needed. Reports on testing with other toolchain and OS versions are also welcome.

Originally posted by bubulle on Planet Debian, a shell prompt that displays the current git branch, in colour on some terminals, after the current working directory. The following snippet does similar things for mksh users, except it doesn’t redefine your prompt but amend it – just throw it at the bottom of your ~/.mkshrc before that last line beginning with a colon (copy from /etc/skel/.mkshrc if you haven’t done that yet):

	function parse_git_branch {
		git branch 2>/dev/null | sed -n '/^\* \(.*\)/s//(\1)/p'
	}

	function amend_prompt_with_git {
		local p q='$(parse_git_branch)' r

		if [[ $TERM = @(xterm-color|xterm|screen*) ]]; then
			if [[ ${PS1:1:1} = $'\r' ]]; then
				p=${PS1:0:1}
			else
				p=$'\001'
				PS1=$p$'\r'$PS1
			fi
			q=$p$'\e[1;33m'$p$q$p$'\e[0m'$p
		fi

		p=${PS1%%*( )[#$]*( )}
		if [[ $p != "$PS1" ]]; then
			# prompt ends with space + #-or-$ + space, we can amend
			r=${PS1: ${#p}}
			PS1=$p$q$r
		fi
	}
	amend_prompt_with_git
	unset -f amend_prompt_with_git
 

The indirection by use of a function is not strictly necessary but allows the use of locals. I took the liberty of adding an asterisk after “screen” to match the GNU/Linux nonsense of having TERM=screen.xterm or somesuch.

KiBi is my hero of the day. I’ve long wondered why I couldn’t select fixed-misc as font on my workstation at the dayjob, which is running K?buntu Hardon Heroin. (Luckily, I managed to avoid upgrading to Prolonged Pain.) Now I guess that’ll work again.

My work laptop (running testing) also has got this X.org thingy. My keyboard layout now has got a grml branch (named after the person who first cursed about the insane idea of those toy-breaking boys to rearrange the keycodes) that works with it. Since Debian is marginally more sane than K?buntu, in contrast to the gnu branch I use on my orkstation, the grml branch still has Meta on the left Alt key, not Mode_switch, as it still works in uxterm, which reduces the diff between the MAIN branch (HEAD) on XFree86® and this beast.

And finally: X.org defaults to a black screen and disabled mouse pointer until an application first requests it. Totally unacceptable for evilwm(1) users, and letting people think it crashed, to boot. The Arch Linux guys found this, among others; the fix is: startx(1) users edit /etc/X11/xinit/xserverrc to add -retro behind the X, or copy the file to ~/.xserverrc and change it there:

	#!/bin/sh

	exec /usr/bin/X -retro -nolisten tcp "$@"
 

For display managers, similar files exist in /etc/kde4/kdm and related places.

Update: Also, newer xterm(1) justify an update to ~/.Xresources for we can finally get rid of cut buffers, and get a blinking underline cursor to boot!

On the other front, worked on Debian packaging, and upstream on pax(1) and jupp, with more things to follow (especially in mksh). Also fixed about ⅔ Linux klibc architectures and learned why I’m a BSD developer despite all the bad parts of it ☺ and fixed fakeroot with pax(1) on Hurd… incidentally in code originally designed to support the Linux pax. My dayjob’s keeping me busy, but I’ve got plans to run mksh(1) through Sonar, in addition to the static code analysēs done by (once again, thanks!) Coverity (commits to mksh pending) and Clang/LLVM scan-build. Uhm, what can I say more, grab me in IRC if you need it. Ah, and some other mksh things coming up that may be of interest to people needing to support legacy scripts.

While wtf(1) always has been a bit central to MirBSD, and the acronym database has been accessible by CVSweb, what we never had was a DAU compatible (and shellsnippets compatible) lookup. This has now changed: the above link to the acronyms file is a persistent link to its latest version (well, latest when the website was last recompiled), tooltips may very well follow soon, and we’ve got an online WTF lookup service.
Contributions to the acronym database are welcome, of course; just eMail them to tg@mirbsd.org.

Not to stop there, our online HTML manpage search is also new, shiny, and should replace the “!mbsdman” DuckDuckGo hash-bang shortly. (Both of these services offer a DDG search as fallback. Note that DDG is an external service included herein by linking, under their request to spread it, and not affiliated with The MirOS Project. They do, however, donate some advertising money to Debian.)
For all those who didn’t know: only manpages for software in the MirOS BSD base system and for the MirPorts Framework package tools are listed, not for third-party applications installable using ports or, recently, pkgsrc®. Still, if you want to have a peek at a modern classic BSD’s documentation, you’re welcome. (Not to mention content like re_format(7) and style(9) and that some of our documentation is much more legible than others.)

And because writing all that perl(1) made me ill, not to mention I don’t even know that language, I’ve hacked a bit more in the mirmake(1) and mksh(1) parts of the MirWebsite, finally implementing pointing out where in the navigation sidebar the visitor currently is.

We also have exciting mksh porting news involving RT trying a larger number of ancient platforms than I dare count, me fixing bugs in Linux klibc and diving into other things, learning more about why I consider me lucky for hacking a BSD operating system… sorry, I want to keep this short as it’s mostly an announcement.

The MirWebsite source code is, of course, also available. Improvements welcome. Except for these three CGIs, our website is fully statically precompiled, and that’s a good thing. Please help in making the CGIs secure.

On MirBSD and other sane OSes, you can just press ^T (Ctrl-T) when dd(1) runs; this sends it a SIGINFO (cf. sigaction(2)) which asks it to display (progress) information to the tty. This includes kFreeBSD, btw.

Update 07.01.2012 – this also works on Hurd. Linux neither has SIGINFO nor (cooked mode tty) support for it.

There’s also pv:

	dd if=/dev/mapper/vg01-${customername}--hudson bs=1048576 | \
	    pv -pter -B 1048576 -s 85899345920 | \
	    xz -0 >/mnt/ci-${customername}-snap-20120105-lenny.img.xz
 

I used this At wOrk today to back up a Jenkins VM before upgrading its underlying operating system for evaluation. Here, the -s flag is the total size (in bytes; don’t forget to multiply by 1024 when reading from Linux’ /proc/partitions) so pv can calculate a total and an ETA; -B is the same as bs; and xz is the currently best compressor to use, in any situation, unless you must stay compatible to gzip(1)-only systems. (Except that it’s not under an Open Source licence.)

clpbar might also be worth looking into. XTaran points out sid has this as bar.

PSA: Last of June, 2012, will be a leap second.

This is both a release announcement for the next installment of The MirBSD Korn Shell, mksh R40b, and a follow-up to Sune’s article about small tools of various degrees of usefulness.

I hope I don’t need to say too much about the first part; mksh(1) is packaged in a gazillion of operating environments (dear Planet readers, that of course includes Debian, which occasionally gets a development snapshot; I’ll wait uploading R40c until that two month fixed gcc bug will finally find its way into the packages for armel and armhf). Ah, we’re getting Arch Linux (after years) to include mksh now. (Probably because they couldn’t stand the teasing that Arch Hurd included it one day after having been told about its existence, wondering why it built without needing patches on Hurd…) MSYS is a supposedly supported target now, people are working on WinAPI and DJGPP in their spare time, and Cygwin and Debian packagers have deprecated pdksh in favour of mksh (thanks!). So, everything looking well on that front.

I’ve started a collection of shell snippets some time ago, where most of “those small things” of mine ends up. Even stuff I write at work – we’re an Open Source company and can generally publish under (currently) AGPLv3 or (if extending existing code) that code’s licence. I chose git as SCM in that FusionForge instance so that people would hopefully use it and contribute to it without fear, as it’s hosted on my current money source’s servers. (Can just clone it.) Feel free to register and ask for membership, to extend it (only if your shell-fu is up to the task, KNOPPIX-style scripts would be a bad style(9) example as the primary goal of the project is to give good examples to people who learn shell coding by looking at other peoples’ code).

Maybe you like my editor, too? At OpenRheinRuhr, the Atari people sure liked it as it uses WordStar® like key combinations, standardised across a lot of platforms and vendors (DR DOS Editor, Turbo Pascal, Borland C++ for Windows, …)

ObPromise: a posting to raise the level of ferrophility on the Planet aggregators this wlog reaches (got pix)

eMail

06.10.2011 by tg@
Tags: debian pcli rant tip

Would MTAs please stop sending hi-bit7 messages to other MTAs which do not advertise 8BITMIME! Recode it to QP or BASE64, damnit! The receiving MTA is entitled to strip the set bit7, which kinda makes things hard to read (while I know how to deal with blvde Stra_e, the advent of UTF-8 makes that blC6de StraC?e, introduces C0 control characters and makes typographic quotation marks into NUL-containing octet sequences (as their UTF-8 representation contains 0x80 octets) which let every sensible MDA terminate the line there). I even filed in the Debian BTS against the BTS (might be Drexim's fault, though).

Would MUAs please default to Quoted-Printable!

And mail hosters should use the same server when retrying delivery, to benefit greylisting. Or at least publish a list of outgoing IPv4 addresses they use for sending. Or use IPv6. Oh, and STARTTLS, while we are on my wishlist.

It's a sad day when the percentage of correctly encoded eMail messages in my INBOX is smaller than that of my Spambox...

Our MirBSD online manual pages and other assorted BSD documentation (except of course the merely copied ncurses, lynx etc. documentation and the texinfo generated HTML pages) has just gained a major facelift. They look alike in lynx(1) – best web browser ever – and less(1)/man(1) now, and remind of a DEC VT420 on a CSS capable Buntbrause.

Thanks to our contributor XTaran for aid with the colour scheme!

Since these are generated from catmanpages, heuristics are used for things like where should bold/underline begin/end (since nroff(1) is not always the brightest… but working on that), and hyperlinks can only be generated for other manpage references (whose targets may or may not exist, for example if they aren’t part of MirOS base/XFree86®). But on the other hand, Valid XHTML/1.1 and CSS speaks for itself ☻☺

	14:31⎜*<* Signoff: XTaran (*.net *.split)

… doesn’t prevent me from telling him…

	14:39⎜<mira|AO> XTaran: n̲i̲e̲ n̲i̲e̲ n̲i̲e̲ n̲i̲e̲ n̲i̲e̲ n̲i̲e̲ n̲i̲e̲
	     ⎜empfiehlt man k̶i̶l̶l̶a̶l̶l̶, i̲m̲m̲e̲r̲ nur p̲k̲i̲l̲l̲!

“Now playing: Monzy — kill dash” ⇒ good idea… ☺


By the way, you were probably looking for this…

     -x      Require an exact match of the process name, or argument list if
             -f is given. The default is to match any substring.
 

… excerpt from the pkill(1) manual page, where you can see it stems from grep(1) clearly.


Yes, this website (and thus the RSS export) is Lynx on uxterm -fn -misc-fixed-medium-r-normal--18-120-100-100-c-90-iso10646-1 -fw -misc-fixed-medium-r-normal-ko-18-120-100-100-c-180-iso10646-1 on XFree86® optimised. Your browser might not do combining.

mksh R40b (nowadays with filled in user’s caveats (for R40, too!) and packager’s upgrade hints) has just been released. This is a should-have upgrade, fixing a number of – admittedly some obscure – bugs, changing things begun in R40, improving upon others. Thanks to the PLD Linux guys for spotting all these errors; thanks to them and phpnet.org both for adopting mksh so well.

I have also fixed a bug in nroff(1) which will lead to an even nicer looking HTML manpage mksh(1) (after the next rebuild and upload of a MirBSD snapshot – scheduled RSN).

jupp 3.1.16 took on the task of merging Debian joe changes (aiming at an upload). I also split the jupprc file into three versions (2.8 generic/DOS, 3.1+jupp and 3.7/Unix) because of the differences in the baseline executables making rc files partially mutually incompatible (think Insert key), annoyingly warning (think syntax, hmsg), or less usable (joe’s new menu system).

jupp 2.8.2 is a companion to jupp 3.1.16 – mostly because of the new help window “character map” ☺

Binaries for jupp should be updated RSN too.

Considering Banja Luka is arriving quickly, the “r” in RSN should be taken with a few grains of salt. I’ve also scheduled working on the pcc Debian package for the next future; updating lynx and maybe others like OpenSSH in MirBSD is also due; cvs(1) will receive more of my time, but before the next Upload I’d like to fix LP#12230 once verified.

Builds for Debian/m68k are also still running. I note I did in fact not manage to make a new base image, yet (but 2.6.39 kernels miss a patch, anyway, so waiting for 3.0 is ok). It’s still using gcc-4.4 because nobody tests gcc-4.6 and gcj-4.6 FTBFS due to SIGSEGV, but that’s ok in my books. rsyslog is broken but sysklogd works.

The #ksh|Freenode page finally got a well-deserved link to Planet Commandline. Throw more my way!

Acronyms and translations, too. (Got Norwegian and Rumanian covered in the meantime. No idea whether any RTL languages will work in that beast. But I’m young and need the money)

Since I’m writing a wlog entry anyway… let me thank Gunnar for a nice summary on the current Free Culture discussion; my comments on Nina’s site seem to be eaten, but let me support it fully, although, of course, I normally use a copycenter style licence, which is specifically written for general works of authorship under copyright law, not limited to software. I did in fact have that in mind. Maybe some people will like it (it’s less than one Kibibyte long) either generally or just for their everyday random musings (they can then keep CC-BY-SA for the “big works” if they so desire).

Wouter, grass background makes green headlines illegible. I’ve never liked, and never installed manually, cups either. (Benny tells me that Apple’s new version refuses to talk with a non-Apple cups, kinda defeating the whole idea I think.) Port 9100 is JetDirect (probably with an HP in front and some subset of ©®™ trailing) and just nice. (Being able to talk ESC/P with your printer like print '\033K\x07\0\x3E\x81\x99\xA5\xA5\x81\x3E' >/dev/lpa too rocks though, IMHO. Yes, mine can, and I still can. /dev/lpa is BSD.)

Kai, thanks for your vimrc lines:

	:highlight TrailWhitespace ctermbg=red guibg=red
	:match TrailWhitespace /\s\+$\| \+\ze\t/
 

Automatic removal is harmful, though – I just fell into the trap since jupprc contains needed whitespace at EOL… but manual removal (bound to ^K] in jupp) rocks. And I like that your solution uses such strong a colour – vim users are the single most represented offender group for actually leaving the redundant whitespace at EOL there, and it should hurt their eyes. (Sadly there is some vehement disagreement preventing them from inclusion in grml-etc-core – but that’s why I re-post them here.) Ah, and jupp can of course display whitespace visibly (although it uses ‘·’/‘→’, replacing the arrow with ‘¬’ if no UTF-8, not ‘»’), accessible with ^Ov.

Steve, want to put up a checklist for sites? We can “crowdsource” the… testing… to maybe get some interesting results…

Some other people would get more comments if they were idling in IRC (Freenode) or allow comments on their blog, specifically without too high an entrance barrier – OpenID is ok, but many other things, and ECMAscript, are not; but I can’t really say that loud because our wlog is static HTML compiled from a flat plaintext data source so it doesn’t allow such either. I often forget what I wanted to add if I can’t get it out quickly enough (especially at work). Sowwy…

Me like the cat picture postings (Amayita, Tiago, ¡Gracias!).

(First posting to Plänet Commandline! Tag: pcli)

Vutral asked in IRC how to synchronise two shells’ environment while they’re running. As you may know, POSIX systems cannot change a process’ environment vector after it has been started, only the process itself can. Well, the shell can, and we’ll use a variety of things for this.

This trick assumes you have $HISTFILE set to the same pathname in both shells (obviously, they run under the same user). It uses export -p to render the current list of exported variables, then transforms the list from newline-separated to a single big one-line export statement.
Then it transforms all remaining newlines (which will be part of a single-quoted string, since that’s mksh(1)’s export format) into the sequence '$'\n'' which means: terminate current single-quoted string, append $'\n' and open up a new single-quoted string immediately; concatenate these three.
Now, $'\n' is just a fancy way of saying newline, and part of mksh because David Korn (yes, the Korn in Korn Shell) strongly suggested to me that this functionality be included – but, as we can see here, it pays off.
Finally, the so transformed string is prepended by unset \$(export); which, when executed, will cause the shell to unset (and unexport) all currently exported variables. The shell parameters that are not exported, i.e. not in the environment, are not affected by this code (except for $x and $nl, but… whatever).
This string is then passed to read -s (plus -r and clearing IFS to enable raw mode), which means, read into the parameter $REPLY (which we conveniently don’t use – but it’s trashed too, thus) but store into history at the same time.

Ah hah! Now, the persistent history feature comes into effect! After running the below statement in the “source” shell, switch into the terminal running the “destination” shell, press Enter once on the empty line (Ctrl-U to empty it if it wasn’t), then Cursor-Up (↑) to recall… voilà, an insanely large line with the previously created string sorta expanded… and press Enter again to run it. Now your set of exported parameters is the exact same (minus if you exported IFS, nl, x or REPLY) as in the “source” shell.

I’ve added extra spaces and a linewrap below, this is really just one big line:

nl=$'\n'; x=$(export -p); x=${x//${nl}export/}; IFS= read -rs <<<"unset \\\$(export);${x//$nl/\'\$\'\\\\n\'\'}"

Of course, this makes a nice function, for your ~/.mkshrc or somesuch.

Some things are ugly.

Waldi’s suggestion fails.

db4.6_upgrade: Program version 4.6 doesn’t match environment version 4.4
db4.6_upgrade: DB_ENV->open: DB_VERSION_MISMATCH: Database environment version mismatch

Can’t start it manually.

debian-sks@dev:~$ /usr/sbin/sks recon
Fatal error: exception Bdb.DBError(“Program version 4.6 doesn’t match environment version 4.4″)

The log only shows:

2010-05-09 16:59:29 Opening log
2010-05-09 16:59:29 sks_db, SKS version 1.1.0
2010-05-09 16:59:29 Copyright Yaron Minsky 2002, 2003, 2004
2010-05-09 16:59:29 Licensed under GPL. See COPYING file for details
2010-05-09 16:59:29 http port: 11371
2010-05-09 16:59:29 Malformed entry
2010-05-09 16:59:29 Malformed entry
2010-05-09 16:59:29 Malformed entry
2010-05-09 16:59:29 Malformed entry
2010-05-09 16:59:29 Malformed entry
2010-05-09 16:59:29 Malformed entry
2010-05-09 16:59:29 Malformed entry
2010-05-09 16:59:29 Malformed entry
2010-05-09 16:59:29 Malformed entry
2010-05-09 16:59:29 Malformed entry
2010-05-09 16:59:29 Malformed entry
2010-05-09 16:59:29 Malformed entry
2010-05-09 16:59:29 Malformed entry
2010-05-09 16:59:29 Opening KeyDB database
2010-05-09 16:59:29 Shutting down database

The solution is ugly as hell, too:

root@dev:/ # su – debian-sks
debian-sks@dev:~$ cd DB
debian-sks@dev:~/DB$ db4.4_checkpoint -1
debian-sks@dev:~/DB$ db4.4_recover
debian-sks@dev:~/DB$ db4.4_archive
log.0000002839
log.0000002840
log.0000002841
log.0000002842
log.0000002843
log.0000002844
log.0000002845
debian-sks@dev:~/DB$ db4.6_archive -d
debian-sks@dev:~/dump$ cd ../PTree/
debian-sks@dev:~/PTree$ db4.4_checkpoint -1
debian-sks@dev:~/PTree$ db4.4_recover
debian-sks@dev:~/PTree$ db4.4_archive
debian-sks@dev:~/PTree$ db4.6_archive -d
debian-sks@dev:~/PTree$ logout
root@dev:/ # /etc/init.d/sks start
Starting sks daemons: sksdb.. sksrecon.. done.

Wow, our internal keyserver works again. Thank you, Debian…

This solution courtesy of Uwe Hermann, although it was for Suckwürstchen.

All 1 2 3 4

MirOS Logo