The Lua Paper is a pretty nice read!

DNS servers, localhost and asynchronous code

localhost is always, right? Nope, can also be ::1 if your system only has IPV6 (apparently).

Asking a DNS server for an A record for localhost should give you back right? Nope – it varies wildly! gives me an NXDOMAIN which means it tells you straight up THIS DOMAIN DOES NOT EXIST! Which is true, since localhost isn’t a domain. But if you ask the same thing of any dnsmasq server, it’ll tell you localhost is Other servers vary wildly – I found one that returned an NXDOMAIN for AAAA but for A (which is pretty wild, since NXDOMAIN makes most software treat the domain as not existing and not attempt other lookups). So localhost and DNS servers don’t mix very well.

But why is this a problem, in general? Most DNS resolution happens via gethostbyname libc call, which reads /etc/hosts properly, right? Problem there is that there is popular software that’s completely asynchronous (coughnginxcough) that does not use gethostbyname (since that’s synchronous) and directly queries DNS servers (asynchronously). This works perfectly well until you try to hit localhost and it tells you ‘no such thing!’.

I should probably file a bug with nginx to have them read /etc/hosts as well, and in the mean-time work around by sending to nginx rather than localhost.

How did your thursday go?

DevLog for 21-30 Dec 2015

Clearly I missed an entire week. I need to build a better system to make this easier…

Random notes.

  • Kicked out NFS from the Tool Labs proxies (with 1). Yay! This hopefully explains the lockup of tools-proxy-01 yesterday night, maybe? It’s been restarted since, and I hope to no longer have instances randomly locoking on me. Infrastructure standards of 2009, here we come! :D I’ve also removed NFS from tools-redis, and migrated them to Jessie as well.
  • Fixed up all the races in how kubernetes workers are setup with 2
  • Another instance is ‘stuck’ again. Sigh. AAAAAAAAAAAAAAAAAAAAAAAAA. Paravoid helped debug this, tracking it down to NFS client issues in the 4.2 kernel (See phab). I moved k8s nodes back to a working 3.19 kernel (after filing issue about the other 3.19 kernel package I tried that didn’t work).
  • Moved the tools proxies to 4.2 (lol?) after finding out huge ksoftirqd spikes in them. Let’s see if that improves things
  • I split up the individual components of PAWS, and have a working nbserve in there now! Exciting times. Need to fixup nbserve to use traitlets for config
  • Big Tool Labs outage (again!). Some tool accidentally sent about 12million job requests and crashed gridengine’s underlying backing store (BerkeleyDB). Reset to a clean slate after many hours (Thanks Coren!) and mostly things are back up now. I’m reading through the Berkeley DB reference manual now.
  • Persistance failed for ores’s redises again, mostly because vm_overcommit was turned off. Fixed in the core redis module so it does not happen again.

DevLog for 2015 Dec 20

Probably going to take it easy and chill. Already sent a trivial PR up though.

Also saw the old devlog mailing list and feeling happy memories. Clearly need to bring something back like that, but I don’t know / think mailing lists are the best medium. More thinking!

Ended up learning some Tornad and wrote nbserve to serve rendered notebooks and static files in a configurable way. I should refactor PAWS to have separate jupyterhub, proxy and nbserve pods tomorrow. Also need to test for path traversal attacks and whatnot. Also coroutine based programming is a lot easier than I had originally thought! woo!

DevLog for 2015 Dec 19

  • Today looks like a day of finding, reporting and fixing bugs in WikiChatter. I had made a stupid mistake yesterday that meant not all of the Teahouse pages were being parsed, and I immediately started running into bugs. Have reported (and fixed!) two (1 and 2), and ran into another bug in mediawikiparserfromhell itself. I’m sure I’ll find more.
  • Another bug in WikiChatter! 3
  • Perils of digging through archives – you find Wikipedians giving relationship advice.
  • Should work on in some form over the next few weeks.

DevLog for 2015 Dec 18

Haven’t done these in a while, let’s see if I can get this back on the wagon!

  • Discovered the WikiChatter library (thanks to @halfak!), and using that in my Teahouse analysis notebook. Far better than writing my own parser and fighting with that. Lets me get on with the actual fun stuff I wanted to do (which is the actual analysis)
  • Learning about pandas, checking out matplotlib, bokeh and wordcloud libraries to use in the analysis. Have included matplotlib and bokeh (and with it, pandas and numpy) in the default libraries list for PAWS, and also fixed permissions so users can pip install stuff themselves too.
  • For context, I’m trying to do an analysis of the English Wikipedia Teahouse questions archive, mostly as a way of showing off what PAWS makes possible.
  • Also spent a good chunk of the day regretting previous life decisions. All temporary however – nothing irreversible was done, which is wonderful. Should figure out how to reduce likeliness of similar events happening in the future.
  • Docker build times on my machine are pathetic, both because of slow network (USA! USA! USA!) and cheap laptop. Need to find a proper solution to this soon.


I’m writing this post in an attempt to catalog the list of things I own so I can evaluate if I really need them and get rid of them.

  1. 15″ rMBP
  2. Kinesis Keyboard
  3. Apple Trackpad
  4. Moto X
  5. Kindle
  6. Nexus 7 (To be returned to the WMF)
  7. Broken Nexus 4 (To be backed up and then… something)
  8. iPod Touch
  9. Earphones (Soundmagic E10)
  10. Headphones (AudioTechnica ATH M50)
  11. Battery Pack #1
  12. Battery Pack #2
  13. MiFi (US Only)
  14. Multi USB Charger + 6 USB Cables
  15. Bluetooth Speaker (JBL Flip 2)
  16. Assorted Medication (in several loose covers, need to consolidate)
  17. Toiletries (Emergency Soap, Toothbrush, Toothpaste, Hair Gel, Shampoo, Conditioner)
  18. Velcro Rolls
  19. Box of Leaves
  20. Octopus
  21. Letter in Envelope
  22. Universal plug convertor
  23. Physical paper notebook
  24. Nailcolor
  25. Glasses + backup glasses
  26. Small Green Foldable Bag
  27. Pens
  28. Wallet with assorted currencies and cards
  29. Raspberry PI
  30. Assorted USB chargers (accumulated from various devices)
  31. Wrist Straps (left and right)
  32. Beard Trimmer


  1. 14 Underpants
  2. 23 Socks (not 19 pairs – I had given up on pairing socks a long long time ago)
  3. 8 T Shirts
  4. 3 pairs of cargo shorts
  5. 1 pair of jeans
  6. 3 Jackets of varying thickness
  7. 1 Down Jacket
  8. 1 Scarf
  9. 1 pair of thermal underclothes
  10. 2 Towels (1 slightly fluffy, 1 microfiber)

I’ll try and keep this list updated.

Decluttering actions:

  1. I gave away assorted USB Power chargers and Plug convertors – I have now a universal plug that should be good enough, and 3 USB power adaptors of various sizes. I should probably trim down the number of cables I have

“Green Stuff”

“Green Stuff”

TIL that Gregory Kohs was actually allowed to post on wikimedia-l at some point in the past.

Global merge of accounts at Wikimedia

is now slowly happening, and I am the test subject!

Best talk ever?

(Funniest, at least!). Well worth the 27min watch.

© 2016 Yuvi Panda

Theme by Anders NorenUp ↑