MonthJuly 2013

Sprinkling some Douchebaginess in code

After being frustrated at Java's lack of a generic 'callback' type, I created this interface:

    public interface ContributionUploadProgress {
        void onUploadStarted(Contribution contribution);
        boolean isJavaAPieceOfShit();

And randomly throw around (with onComplete implementing ContributionUploadProgress)

    assert onComplete.isJavaAPieceOfShit();

This, of course, is trivial to fix with an IDE. Should be more fun with a dynamic language :)

(And yes, I removed that code before committing)

Nitpicking about the Order of Cards

Someone pointed out this happening on Gerrit…

GitHub Streaks

I'm on my longest GitHub Streak this year – currently 20 days (was previously 18 days). That means I've had at least one commit (though usually more!) each day for the last 20 days. Sounds much easier than it actually is. Wanting to keep my streak unbroken is also making me write more diverse code – I think those 20 days feature code in Javascript, Python, Java, PHP and Python. It's also distributed across a lot of projects – at least 8-9 different ones on last count. Streaks are helping me a fair bit! Getting the first commit of the day is the hardest – seems to get easier to keep things after.

So I've beat my previous streak length for this year (18 days). Next step would be to get over the awesome/scarily productive legoktm's longest streak (29 days). Eventual goal would be to match ori-l at some point in productivity.

Game of Life in APL

Ran across a video explaining building Game of Life in APL. Pretty awesome, and surprisingly – quite understandable too!

I should learn J at some point, I think. My problem with trying to learn such languages is that I find it hard to find something to build that's not a mere academic exercise, and I still do not have a solution as such. Just 'try harder' doesn't really work. Perhaps invent some sort of wonderful number crunching idea and then implement it?

Can’t take the India out of the W3C…

Presenting, the W3C India website, apparently from 2013. Funniest bit was probably:

Best view in IE 9+, latest version of chrome, mozilla, opera and safari.

Number of interactions with CDAC that have given me faith in any semblance of their competence: 0

First major feature to Commons deployed

My UploadWizard work has just been deployed to Wikimedia Commons and my name is listed in the on site credits. Special thanks to Mark Traceur for code review!

This feels very, very nice :) Now to sneak more code into other extensions…

Bikeshedding on Gerrit #1

First there was this.

I was bored, and so there was this. That of course, brought a swift end to the bikeshed. I'm only slightly ashamed.

God, biological engineering and friendly AIs

Talking to God is an interesting piece about apparent 'dangers' of knowledge and the necessary side effects of evolution. Thought provoking.

Attempting to Secure Redis in a Multi Tenant environment

When running Redis in a shared cluster/hosting environment (such as Wikimedia Tool Labs, on which I've been having fun doing a lot of work on), you would want to try to provide at least some guarantee of isolation for your keys from everyone else's keys. Since Redis doesn't do ACLs, this is problematic.

This can be solved in a couple of ways

Run a Redis instance for each user

This is simple enough to do – each user runs their own Redis instance, and has full access to it. Security is handled by setting a secret password, and running redis-server as the user in question. Boom, secure!

This doesn't really scale with a large number of users, because they each have lesser memory to work with now. Also having users who just want to run their tools have to deal with making sure their Redis instance is up and running fine isn't really good. Having the sysadmins be responsible for users' Redis instance is… not going to work :) This would also require all the redis instances to run on one box and/or have a separate cluster just for them, which isn't good either.

Add ACL support to Redis

Not happening, because I'm not good enough to do that yet :P But more realistically, it won't ever happen, since this will probably add a lot of overhead for what is arguably an edge case.

Build a small server that sits in front of Redis

Such a server would simply authenticate incoming requests via some mechanism (Keystone perhaps), and then enforce ACLs. It will have to speak the exact same protocol as Redis, since users should be able to use any library that connects to Redis. This isn't too hard – just replace the password functionality of Redis protocol to take in both a username and password (or token, or some other method of auth).

This also I discarded – it will still affect performance, which will now be limited by how fast this server runs, and that is definitely not faster than Redis. And it will also be hard to maintain, since I'll have to completely mimic Redis' protocol and make sure it is kept up to date. Debugging protocol issues with random client libraries is not my idea of fun. Another major disadvantage is that I would now be writing auth code, and I don't think handrolling auth code is a good idea, ever.

Security By Obscurity!

This is what we finally settled on :D It sounds horrible by the title, but I think it is Good EnoughTM.

Since Redis is a key value store at heart, you can do anything once you know the key. So, if an 'attacker' doesn't know the key, there isn't much they can do. So it can be considered SecureEnough for our purposes if we can make it so that other users can not find out or guess your keys.

We essentially did so with the following:

  1. Disable all Redis commands that let users list all / many keys.
  2. Have users use a random and long key prefix for all their keys.

(1) prevents someone from just listing all keys to find something interesting. (2) prevents people from brute-forcing or guessing keys. Since all code run on Tool Labs must be open source, guessing keys is super easy. By having a 'secret' prefix, having the actual keys is useless. This also prevents accidental key overwrites from different tools using a common key name.

Disabling commands is easy to do by using Redis' RENAME COMMAND config feature. I added support for RENAME COMMAND to Wikimedia's Redis puppet module, and then it was simple enough to configure a specific instance to disable 'list keys' type commands. That's the following commands:

  4. KEYS
  9. DEBUG.

After going through the list of Redis commands, I am guessing this is going to be GoodEnough to prevent key listing. (Note: if there's more that I'm missing, please, please let me know).

We also tell people to use a secure prefix that's at least 64bytes long, saved in a file that is only user readable. Generating that is as simple as:

openssl rand -base64 64

That should be long enough to be hard to brute force, even with Redis being as fast as it is.


The major problem with this is, of course – the fact that humans are involved :) I've heard "I do not care about my keys, do not need security" a fair amount of times already. The fact that the prefix generation is optional means that there will be people who do not use prefixes, and it will work for them for a (probably) long time – until it doesn't, and they have no idea why. This is personally acceptable to me, since they have been made aware of the risks beforehand.


This has now been deployed on toollabs for a month or so, and I've a couple of fun tools already written using it (and other people too). We had a patched memcached server we had that we'll kill in a few weeks, so people who used memcached before are also migrating to redis. And I was able to do all this without even having root! This is mostly thanks to the fact that we try to keep all our configuration in puppet (Wikimedia's Puppet repository) – for both our production cluster and for everything else. So I could re-use our production redis module, make changes to it, and build the new solution – all while being vetted by 'proper' ops people (whom I dearly love and respect). Building infrastructure in such a collaborative manner is a lot of fun, and I think I'm hooked. It's fun!

Testing Platen Blog Editor

Ever since the death of Windows Live Writer, I’ve never managed to find a nice blog editor that I really liked and also worked fine. I had a nice Vim plugin that posted to wordpress for a while, but that seems to have stopped working. The WordPress interface itself is mostly fine, but still a fair bit ‘meh’.

Platen is a markdown based editor that’s a Chrome Web Store app that looks cool so far. Let me see if it actually works.

© 2017 Yuvi Panda

Theme by Anders NorenUp ↑