Critical Infrastructure and the Internet (and by extensions, the Cloud)

September 1, 2013

Dan Geer has written a nice article ( about putting critical infrastructure (like power grids) online, unfortunately it seems to ignore the economics involved, using the Internet is vastly cheaper than anything else (e.g. dedicated physical communications infrastructure, imagine the cost of laying your own network to connect a few power plants and control centers up vs. just buying Internet access from a provider). I suspect this pattern of “ZOMG! critical infrastructure it attached to the Internet” will be repeated for the Cloud, with the same economic drivers resulting in stuff being shoved into the Cloud anyways.


The problem with “X”

September 1, 2013

I’m noticing that most of my kids books have three approaches to the latter X:

  1. Use “Xylophone” or “X-ray”
  2. Use something utterly insane like “Xoloitzcuintli” (I dare you to try and pronounce it)
  3. Ignore it:


The ignore it option is obviously not ideal, in fact there is a great 3 letter word starting with x, “Xis”, which is the 14th letter of the Greek alphabet (which means you can then explain to your kids that there are other languages, and some are root languages and so on, a teachable moment!).



Your Logical Fallacy Is

September 1, 2013

I can’t wait to teach my kids this stuff and send them to school:

And in case the site ever does down here’s a local high res copy of the poster: FallaciesPoster24x36


Why I love OpenSource – adding CSV export capability to Mailman

August 17, 2013

So mailman is by far one of the best mailing list management packages available. One thing I love is the command line access, Because of this I can write simple scripts like:

for list in `/usr/lib/mailman/bin/list_lists -b`
/usr/lib/mailman/bin/list_admins $list > /root/mailman-export/$i-admins.txt
/usr/lib/mailman/bin/list_owners $list > /root/mailman-export/$i-owners.txt
/usr/lib/mailman/bin/list_members $list > /root/mailman-export/$i-members.txt

And get a nice list of all the admins, owners and members for each list hosted on that server. I can then do things like grep for a specific user or domain across all the lists and then put that data into further scripts (e.g. to remove every account from a specific domain from every list). But not everyone wants a big list of text files. I had one administrative user that wants to run some analytics, they wanted a CSV file for every list in the form:

"list name","email address","user name","access level"

so for example:

"announcements","[email protected]","Kurt Seifried","member"

and so on. Now I could take the above bash script and modify it sufficiently to take the output and turn it into a CSV file, but there is probably a more elegant way. If you look at the “/usr/lib/mailman/bin/list_members” script you’ll see it’s pretty simple. In order to add CSV export capability I first copied it to “/usr/lib/mailman/bin/list_members-csv” and then opened it up in an editor.

First we’ll need support for CSV and datetime (so we can timestamp the output), just go to the import statements and add:

import csv
import datetime

Then you’ll want to create a CSV file to write to, I want my files in “/root/list-exports/” in a file name  that has the list name, membership level and date (which is redundant but makes dumping all the outputs into the same dir easy and safe). Simply go into “__main_-” and find the line:

listname = args[0].lower().strip()

Then add something like:

datestring = str(
cvsoutputdir = "/root/list-output/"
cvsoutputfilename = cvsoutputdir + listname + "-members-" + datestring + ".csv"
csvoutputfile = open(cvsoutputfilename, "wb")
csvwriter = csv.writer(csvoutputfile, dialect='excel', quoting=csv.QUOTE_ALL)

you now have a file name in the form “listname-members-date.csv”, all the data will always be quoted and the output will be in the format preferred by Excel (so for escaping/etc. it’ll use the characters Excel is expecting). I could have integrated this with the “–output file” command line option,  but then I need logic to handle the datetime and membership level and list name in the wrapper using my modified version of list_members, so it’s easier (for me) to just stick that logic into my modified list_members.

Now you simply look for the lines where the output is actually handled and replace:

print >> fp, formataddr((safe(name), addr))


csvwriter.writerow([listname, addr, safe(name), "member"])

And you’re done.  Now you could get fancy and make it an actual option (–csv?) in the existing program, and add some switch logic for output, but honestly, I couldn’t be bothered, this is simple enough and it works reliably.



Fedora 16 with SELinux running WordPress with Akismet

June 17, 2013

So you have WordPress and Akismet to get rid of spam. But for some reason Akismet is not working:

WordPress with Akismet failing

You can test if you have a valid key and connectivity from the command line with either wget:

wget --post-data 'key=YOURKEYGOESHERE&blog='\

or using curl:

curl -d 'key=YOURKEYGOESHERE' -d 'blog=' \

If it works you should receieve a file called “verify-key” containing the word “valid”

If that doesn’t work then you have problems outside the scope of this article.

But if you can retrieve the key than chances are your SELinux configuration is limiting what the httpd server can do.

Luckily the fix is simple: allow httpd to make outgoing connections:

setsebool -P httpd_can_network_connect on

But wait a minute you say. Now my httpd server can connect to anything, attackers can use it to attack other systems potentially (especially if you allow CGI scripts and arbitrary WordPress plugins or themes which can contain PHP code).

So we need to limit what systems the httpd server can connect to. The good news here is that IPTables supports this.

In the case of Akissmet you’d want to add something like this to your /etc/sysconfig/iptables file:

-A OUTPUT -m owner --uid-owner apache -m tcp -p tcp --dport 80 \
--dest -j ACCEPT
-A OUTPUT -m owner --uid-owner apache -m tcp -p tcp --dport 80 \
--dest -j ACCEPT 
-A OUTPUT -m owner --uid-owner apache -m tcp -p tcp --dport 80 \
--dest -j ACCEPT
-A OUTPUT -m owner --uid-owner apache -m tcp -p tcp --dport 80 \
--dest -j ACCEPT
-A OUTPUT -m owner --uid-owner apache -j REJECT

This should allow only existing inbound connections (e.g. web clients) and outgoing connections to Akismet (you may want to add any other services you use of course).

Is Microsoft spamming anyone else about robots.txt blocking Bing?

June 5, 2013

So Microsoft spammed me about my robots.txt again:

from: Jyoti Bhagavatula (HCL America Inc) <[email protected]>

to: “[email protected]” <[email protected]>
date: Wed, Jun 5, 2013 at 11:11 AM

subject: Robots.txt blocking Bing crawler:


I am contacting you on behalf of the Bing Search engine ( in regards to your robots.txt file:

Our customers have alerted us that your website was partially absent from our results and we have discovered that you are blocking our crawler, named BingBot, via a disallow directive in your robots.txt file:

User-agent: msnbot

Disallow: /

User-agent: bingbot

Disallow: /

We would be pleased if you could edit your robots.txt file to allow our crawler to fetch and index your content properly, which will in turn increase traffic to your site via our search results, by including the following section:

User-agent: Bingbot


I find this pretty annoying, they don’t take the time to even look at my website and see that the contact email is pretty obviously [email protected], instead they spam my DNS WHOIS email contact. Secondly, I’ve already told them several times I don’t want to let them index my site, first via robots.txt, and secondly in email replies to the spam they send me.

Is anyone else getting this emails?

Why Domain Specific Knowledge (DSK) is critical to writing good software – an example

May 27, 2013

Software for the police written by someone trained to be a policeman:

Look for a genuine problem to be solved, not a way to leverage some technology. Really understand the problem that you seek to solve. For example, we have one of the ugliest UIs in the world, and unless you are a cop it’s very difficult to understand why. On the left hand side is what we call the pedigree: the person’s image and an image of their car model. This is designed to be used on a portable computer in a moving unmarked car, and all we want to give you is the ability to see that person on the street or the car in the driveway. Some of this information looks redundant but it isn’t, e.g., age and date of birth. The age is so I know how old the person who I am going to encounter is. The date of birth is to confirm her identity once I see her. The area on the right changes. It shows home address and home street view when you are still driving. We require almost no keystrokes. Cops hate typing, and they are in a car so they are not allowed to type while they drive. Everything is one click away except when you are parked. We are anticipating the workflow in the order in which we believe a cop will do it but we don’t require them to follow our workflow order.

I would have put a pic of the front of the car (wrong) and just the DOB (making the cop do more work to mentally figure out the age). Good thing I don’t write software for the police.

VMware update insecurity

April 16, 2013

So the good news is VMware has a built in updater, and even better the server it uses to query for information is SSL/TLS enabled. The bad news is they are using Akamai with an insecure configuration: vmware-update-overall-grade A quick note: Akamai is a huge content delivery network (CDN) used by many vendors to deliver content and software updates to end customers. Some pretty insecure configurations of SSL and TLS are supported by Akamai for the simple fact that there is still a lot of old software/clients out there, which Akamai’s customers want to support. As you can see here if you query they use an Akamai certificate: vmware-update-akamai Nothing wrong so far, but when you look at the versions of the SSL protocol that are supported they have left SSL 2.0 enabled, and they have also left weak ciphers including several 40 bit ones enabled: vmware-update-weak-ciphers This makes an SSL downgrade attack trivial for an attacker as 40 bit keys can be broken in near real time now with even a remotely powerful computer that has a GPU.  Using this an attacker can man in the middle your upgrade session and tell the client no updates are available (forcing you to remain on an older version with security issues), or potentially send you a malicious update. I haven’t looked at the guts of the VMware updater, for all I know they may use additional protections like secure signing of the update software itself, but let’s hope they do, because the communication channels it is sent over are not very secure. The good news is that fixing this is easy, VMware just needs to make sure their updater software supports TLS and secure ciphers (which if they’re using an even moderately up to date library won’t be a problem), and they need to update their Akamai configuration to exclude SSL 2.0 and the weak ciphers.

SSD Failure modes

March 30, 2013

So everyone constantly talks about SSD failure modes and how they die suddenly with no warning and blah blah blah (honestly: everything dies, and sometimes the building burns down, so make sure you have off site backups). But here is an interesting failure mode I found this week. I have two older Intel 80GB SSD’s in a RAID 0 configuration I use for scratch space (e.g. unpacking the Linux Kernel and grepping for stuff). But the RAID 0 array was suddenly very very slow. Since I was retiring that machine anyways I had to wipe it, so I popped in the DBAN disk and started wiping all the drives. It said it would take 79 hours, which seemed a bit excessive:

DBAN screen showing two identical SSDs being wiped with very different results.

DBAN screen showing two identical SSDs being wiped with very different results.

That’s right, you may have missed it, I did the first time, there’s no “K” in the speed listing for the first SSD. It’s writing at 559240 BYTES/second, not Kilobytes. The second drive is writing at a reasonable ~56.5 Megabytes/sec but the first one is writing at 0.53 Megabytes/sec, about 100 times slower. This certainly explains why my RAID 0 was behaving so badly. The kicker is there are not write or read errors off of that first drive, it’s just really, really slow. So I guess the lesson to be learned is you should check if your RAID card/software can tell you drive specific statistics or periodically read/write test your SSD’s individually if things start behaving oddly.

Make Money Fast with BitCoin!

March 20, 2013

So I noticed these ads at these are high speed crypto devices designed to mine BotCoins. They literally make money, fast. But if they are actually capable of mining BitCoins cost effectively why is the company selling them? Wouldn’t it make more sense to simply run them and harvest the BitCoins themselves?

I did some back of the envelope math and they don’t look that cost effective (unless the price of BitCoins rise, but of course then all bets are off and in theory an Arduino would be cost effective). So I can only conclude that while this is some potentially cool technology it is not cost effective. If was going to try and make money with BitCoins, I would buy a ton of BitCoins, then sell them rapidly to crash the price (since the BitCoin market is still not terribly liquid) and then buy as the price bottoms out. Or I’d hack an exchange and steal a ton of BitCoins. Much like a Casino the only way to reliably mask money with BitCoins is to cheat.

Also if you want to get into BitCoins (beyond cheating/speculating) I suggest you read about deflationary spirals:

All in all considering that BitCoins are wholly unregulated, the exchanges keep getting compromised, and the long term deflationary issues I would imagine that most of us are better off investing in pretty much anything other than BitCoins.