MonthDecember 2015

DevLog for 21-30 Dec 2015

Clearly I missed an entire week. I need to build a better system to make this easier…

Random notes.

  • Kicked out NFS from the Tool Labs proxies (with 1). Yay! This hopefully explains the lockup of tools-proxy-01 yesterday night, maybe? It’s been restarted since, and I hope to no longer have instances randomly locoking on me. Infrastructure standards of 2009, here we come! :D I’ve also removed NFS from tools-redis, and migrated them to Jessie as well.
  • Fixed up all the races in how kubernetes workers are setup with 2
  • Another instance is ‘stuck’ again. Sigh. AAAAAAAAAAAAAAAAAAAAAAAAA. Paravoid helped debug this, tracking it down to NFS client issues in the 4.2 kernel (See phab). I moved k8s nodes back to a working 3.19 kernel (after filing issue about the other 3.19 kernel package I tried that didn’t work).
  • Moved the tools proxies to 4.2 (lol?) after finding out huge ksoftirqd spikes in them. Let’s see if that improves things
  • I split up the individual components of PAWS, and have a working nbserve in there now! Exciting times. Need to fixup nbserve to use traitlets for config
  • Big Tool Labs outage (again!). Some tool accidentally sent about 12million job requests and crashed gridengine’s underlying backing store (BerkeleyDB). Reset to a clean slate after many hours (Thanks Coren!) and mostly things are back up now. I’m reading through the Berkeley DB reference manual now.
  • Persistance failed for ores’s redises again, mostly because vm_overcommit was turned off. Fixed in the core redis module so it does not happen again.

DevLog for 2015 Dec 20

Probably going to take it easy and chill. Already sent a trivial PR up though.

Also saw the old devlog mailing list and feeling happy memories. Clearly need to bring something back like that, but I don’t know / think mailing lists are the best medium. More thinking!

Ended up learning some Tornad and wrote nbserve to serve rendered notebooks and static files in a configurable way. I should refactor PAWS to have separate jupyterhub, proxy and nbserve pods tomorrow. Also need to test for path traversal attacks and whatnot. Also coroutine based programming is a lot easier than I had originally thought! woo!

DevLog for 2015 Dec 19

  • Today looks like a day of finding, reporting and fixing bugs in WikiChatter. I had made a stupid mistake yesterday that meant not all of the Teahouse pages were being parsed, and I immediately started running into bugs. Have reported (and fixed!) two (1 and 2), and ran into another bug in mediawikiparserfromhell itself. I’m sure I’ll find more.
  • Another bug in WikiChatter! 3
  • Perils of digging through archives – you find Wikipedians giving relationship advice.
  • Should work on https://github.com/jupyter/jupyterhub/issues/17 in some form over the next few weeks.

DevLog for 2015 Dec 18

Haven’t done these in a while, let’s see if I can get this back on the wagon!

  • Discovered the WikiChatter library (thanks to @halfak!), and using that in my Teahouse analysis notebook. Far better than writing my own parser and fighting with that. Lets me get on with the actual fun stuff I wanted to do (which is the actual analysis)
  • Learning about pandas, checking out matplotlib, bokeh and wordcloud libraries to use in the analysis. Have included matplotlib and bokeh (and with it, pandas and numpy) in the default libraries list for PAWS, and also fixed permissions so users can pip install stuff themselves too.
  • For context, I’m trying to do an analysis of the English Wikipedia Teahouse questions archive, mostly as a way of showing off what PAWS makes possible.
  • Also spent a good chunk of the day regretting previous life decisions. All temporary however – nothing irreversible was done, which is wonderful. Should figure out how to reduce likeliness of similar events happening in the future.
  • Docker build times on my machine are pathetic, both because of slow network (USA! USA! USA!) and cheap laptop. Need to find a proper solution to this soon.

© 2017 Yuvi Panda

Theme by Anders NorenUp ↑