Docker Tips

Here are a few tips that I've found useful while delving in to Docker, enjoy!

Search private registry.

sudo docker search <private domain>/<term>

Removing images and containers in bulk.

# remove all containers
sudo docker rm $(sudo docker ps -a -q)

# remove all images
sudo docker rmi $(sudo docker images -q)

# remove image containing TERM
sudo docker rmi $(sudo docker images | grep TERM | awk '{ print $3 }')

# remove all non-running containers
sudo docker ps -a | grep Exited | awk '{ print $NF }' | xargs sudo docker rm

Interacting with the most recent continer started.

# view last container
sudo docker ps -l 

# view last container sha only
sudo docker ps -lq

# stop, start, attach, logs, etc. last container
#
# $ sudo docker <action> $(sudo docker ps -lq)
sudo docker start $(sudo docker ps -lq)
sudo docker stop $(sudo docker ps -lq)
sudo docker logs $(sudo docker ps -lq)
sudo docker attach $(sudo docker ps -lq)

Pushing to a private registry.

# assuming image 'jmervine/centos6-nodejs'
#
#               <current image name>    <private registry>:<port>/<image name>
sudo docker tag jmervine/centos6-nodejs docker.myregstry.com:5000/jmervine/centos6-nodejs
sudo docker push docker.myregstry.com:5000/jmervine/centos6-nodejs

# I then recommend removing your old image to avoid accidentally pushing it to the public registry.
sudo docker rmi jmervine/centos6-nodejs

Ports

# running randomly assigning host port
sudo docker run -d -p 3000 image/name

# running with exposed ports randomly assigned on host
sudo docker run -d -P image/name

# printing randomly assigned ports (only)
sudo docker port image_name | awk -F':' '{ print $NF }'

Ruby / RoR... Why not?

In response to common question: "I'd be curious as to why your friends want to move away from the RoR/Ruby space..."

Continue reading

Published on in Ruby

Notes on Performance Testing

A couple weeks ago I did an ad hoc talk at the LAWebSpeed Meetup, hosted by MaxCDN, on general performance testing in the web world. I was asked to put together a list of the tools I spoke about and some brief notes on them...

Continue reading

Node.js Hello Framework

A far from complete collection of Hello World examples for various Node.js web frameworks.

Continue reading

YSlow.js: Release 0.3.1

Links: readme | package | source | tests

0.3.2

Previous Version

  • 0.3.1
  • 0.3.0
    • Fixing error handling in run. Now passes both error and results to callback. See README examples for details. This update is not backwards compatabile with previous releases.
  • 0.2.1
    • Removing unused dependainces.
  • 0.2.0
    • Top down refactor using updated Phapper.
    • Includes better pathing support for finding included yslow.js.
    • Downloads and install yslow.js if it can't be found, which should never happen.
    • Adding limited support for Windows.
  • 0.1.2
    • Fixing critical issue in NODE_PATH search when working with global installations.
  • 0.1.1
    • Refactored to use Phapper, way cleaner and less code.
    • Refactored tests for change to Phapper.
    • Refactored stubs..
    • Adding functional tests.
  • 0.0.1
    • Initial release.

Phapper.js: Release 0.1.9

Links: readme | package | source | tests

0.1.9

  • Fixing a minor bug with the install script.

Previous Versions

  • 0.1.8
    • Updating PhantomJS version to 1.9.7.
  • 0.1.6
    • Replacing exec-sync with execSync for easier Mac installation.
  • 0.1.5
    • Removing unused dependancies.
  • 0.1.4
    • Fixing small issue with passed in arguments on init.
    • Added ability to pass exec object, see readme examples.
    • Cleaned up tests, added more.
    • Cleaned up make test / npm test.
    • Allowing for passing of cwd to sync function.
  • 0.1.3
    • Adding windows handling and phantomjs version overide.
    • Updating readme.
  • 0.1.2
    • Adding phantomjs install.
    • Adding better phantomjs path support.
  • 0.1.1
    • Refactored to not require JSON stdout parse.
    • Refactored run and runSync return values, see readme.
  • 0.0.1
    • Initial release.

Github Webhooks with git-fish

I wrote git-fish – a Github Webhook listener – to provide a simple and modular method for executing an autodeployment on mervine.net when adding or updating a post. I designed it to as simple and as modular as possible. While written in Node.js, I tend to use it execute simple bash scripts, like the mervine.net deployment script:

#!/bin/bash

cd /home/jmervine/mervine.net
make deploy/soft

With this combination, I can use [Github] as my psudo-CMS, to create and update posts and when I save an addition or change, it becomes visable on the site in seconds (including, updating code and purging cache).

For detailed information on setting up and using git-fish or my other see my git-fish project page.

Enjoy!

HTTPerf.js: Release 0.1.0

Removing runSync. Refactoring run to support sending spawned process SIGINT to capture current report from httperf and exit.

Continue reading

Forking in Node.js / Threading HTTPerf with HTTPerf.js

Occasionally, we want to generate load beyond what a single httperf thread can handle, especially when working in Node.js, where the connection limits can get very high. The code sample below does that, but also serves as an example of how to use the cluster module to fork actions and collect the resulting data. Enjoy!

Continue reading

Simple Timing in Node.js

I just stumbled upon a cool feature of Node.js for adding timing to applications using console.time and console.timeEnd`.

// pointless example that show all parts
console.time('timer');
setTimeout(function() {
    console.timeEnd('timer');
}, 500);

// => timer: 511ms

Note: I've heard (and in somecases proven) that in most cases console. method are not asynchronous (i.e. blocking) and therefore should never be used in production code. Notice that in the above example, console.time and console.timeEnd appear to have about 11ms of overhead on my machine.

NPM Registry's

I'm starting this list with the plan on adding as many as I can find. Please shoot me any known public registry's in the comments below.

http://registry.npmjs.vitecho.com/
http://npm.nodejs.org.au:5984/registry/_design/app/_rewrite (AUS)

Usage:

npm install --registry http://registry.npmjs.vitecho.com/ npm-foo

Published on in Node.js

Jade Bootstrap Layout Template

After using the Express command line generation untility, you get a very basic layout.jade. Here's the standard modifications I make for use with BootstrapCDN.

Continue reading

Notes: Mosh IPTable Rules

I recently picked up a Note 3 and with the larger screen I found myself want to use it for shelling in to my machines. So I started playing with Mosh on one of my boxes. I (like hopefully most of you) set strick IPTable rules to keep things locked down as much as possible. I quickly found that (obviously) things weren't working due to this.

To make things work, I added this line to /etc/sysconfig/iptables:

-A INPUT -p udp -m udp --dport 60001:61000 -j ACCEPT

Here's the diff:

diff --git a/tmp/iptables.old b/etc/sysconfig/iptables
index d4229ca..b950f1f 100644
--- a/tmp/iptables.old
+++ b/etc/sysconfig/iptables
@@ -8,6 +8,7 @@
 -A INPUT -p icmp -j ACCEPT 
 -A INPUT -i lo -j ACCEPT 
 -A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT 
+-A INPUT -p udp -m udp --dport 60001:61000 -j ACCEPT
 -A INPUT -j REJECT --reject-with icmp-host-prohibited 
 -A FORWARD -j REJECT --reject-with icmp-host-prohibited 
 COMMIT

Once you've added the line, simply restart IPTables like so:

sudo /etc/init.d/iptables condrestart 

Enjoy!

Bundle Faster, Save Time

Bundler just annouced 1.4.0.pre.1 with --jobs support, which allows for multithreading gem installs -- I haven't looked at the code, but my guess is it's making use of the JOBS flag support in gmake (which multithreads C compilation) for navtive libs.

Anyway, here's my quick timing comparison on bundling a very large project with hundrends of gems:

rm -rf vendor/bundle
bundle install --path vendor/bundle
# time: 5:31.40 total

rm -rf vendor/bundle
gem install bundler -v 1.4.0.pre.1 --pre
bundle install --jobs=4 --path vendor/bundle
# time: 3:10.38 total

Enjoy!

Published on in Ruby

Tweet: Node.js, Eating Crow

Published on

Pretty Sleep using Node.js

$ node -e 'n=0; setInterval(function(){n++;if(n>=20){console.log(".");process.exit(0);}process.stdout.write(".");},1000);'

Twitter Bootstrap Theme for Hexo

Just finished building a Twitter Bootstrap theme for Hexo.

Published on

Command-line Google Search via `find`

Just for fun...

$ find -type m -name "glendale, ca coffee"
$ find -type m -name "glendale, ca smog check"
$ find -type g -name "cool bash commands"

The code:

# "Install" (I use that term loosely)
# - Paste the function below in your .bashrc / .profile / .zshrc / etc.
# Usage: find /usr/local -type [m|g] -name [KEYWORD]
# * -type m : google maps search
# * -type g : google search
# * all other types pass through to find
# Notes: 
# Tested on Ubuntu with ZSH. Comment's, suggestions, etc. welcome.
function find {
  if [ `uname -s` = "Darwin" ]; then
    $browser="open"
  fi
  test "$browser" || browser=`which chromium-browser`
  test "$browser" || browser=`which google-chrome`
  test "$browser" || browser=`which firefox`
  query="`echo "$@" | sed -e 's:^[a-z\/\~\.]* ::' -e 's/-type [mg]//' -e 's/-name//'`"
  if [[ $@ =~ "-type m" ]]; then
    $browser "http://maps.google.com/?q=$query" 2>&1 > /dev/null &
  elif [[ $@ =~ "-type g" ]]; then
    $browser "http://www.google.com/search?q=$query" 2>&1 > /dev/null &
  else
    /usr/bin/find $@
  fi
}

get the gist

Published on

Benchmarking with YSlow.js on Node.js

In my last post on this topic (Benchmarking with HTTPerf.js and NodeUnit) I covered benchmarking application render times from the server to first byte. In this post, I'm going cover basic client benchmarking using YSlow and PhantomJS via YSlow.js on Node.js.

Continue reading

RT: Benchmarking with HTTPerf.js and NodeUnit

Published on

Nginx Build Script

A simple (no frills) Nginx build script.

#!/bin/bash
#
# This script has been tested on CentOS 5 && Ubuntu 12.04.3
#
# By: Joshua Mervine <joshua at mervine dot net> (http://mervine.net)
#
# Note: Does not run `make install` unless run with `INSTALL=true`.

set -e
test "$DEBUG" && set -x

BUILD_ROOT=`pwd`

PAGESPEED=true # set to false to disable pagespeed

VERSION=1.4.2
PSVER=1.6.29.5-beta
PSOLVER=1.6.29.5
PCREVER=8.32
ZLIBVER=1.2.8

# parallelize builds
export JOBS=8

# We don't want to compile against random artifacts
export PATH=/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin

# bail if no yup or apt-get
(which apt-get || which yum) || exit 1
which apt-get && sudo apt-get install build-essential zlib1g-dev libpcre3 libpcre3-dev libssl-dev wget
which yum && sudo yum install gcc-c++ pcre-dev pcre-devel zlib-devel make openssl-devel wget

# delete libs is rebuild
rm -rf nginx-* pcre-* zlib-* release-* *.tar.gz ngx_pagespeed-release-*

if $PAGESPEED; then
  wget https://github.com/pagespeed/ngx_pagespeed/archive/release-$PSVER.zip -O release-$PSVER.zip
  unzip release-$PSVER.zip
  cd ngx_pagespeed-release-$PSVER/
  wget https://dl.google.com/dl/page-speed/psol/$PSOLVER.tar.gz
  tar -xzvf $PSOLVER.tar.gz
  cd ..
fi

wget http://nginx.org/download/nginx-$VERSION.tar.gz
tar xzf nginx-$VERSION.tar.gz

# for rewrite_module
wget http://sourceforge.net/projects/pcre/files/pcre/$PCREVER/pcre-$PCREVER.tar.gz/download \
        -O pcre-$PCREVER.tar.gz
tar xzf pcre-$PCREVER.tar.gz

# for gzip module
wget http://zlib.net/zlib-$ZLIBVER.tar.gz
tar xzf zlib-$ZLIBVER.tar.gz

cd nginx-$VERSION

OPTIONS=" --with-pcre=../pcre-$PCREVER
 --with-zlib=../zlib-$ZLIBVER
 --with-http_ssl_module
 --without-http_uwsgi_module
 --without-http_scgi_module
 --without-http_fastcgi_module
 --without-http_autoindex_module
 --without-http_auth_basic_module"

if $PAGESPEED; then
        OPTIONS+=" --add-module=$BUILD_ROOT/ngx_pagespeed-release-$PSVER"
fi

./configure $OPTIONS

make
test $INSTALL && sudo make install

get the gist

Published on

Benchmarking with HTTPerf.js and NodeUnit

I covered this in the HTTPerf.js README a bit, but wanted to take a deer look at how I'm using HTTPerf.js to benchmark web applications.

Continue reading

Working with Events and Strings in Node.js

Being new to Node.js, it took me an amazing amount of time to figure out how to assgin event stream chunks (foo.on('data', ...)) to a variable and work with it. An amazing amount of time, because it's actually really easy if you understand callbacks (which I didn't) and because all the examples I could find worked with console.log. In this case, I didn't want to log the result, but pass it, process it, etc. So here's an example from GitMD with some additional comments to help you along the way.

Continue reading

Published on in Node.js

Notes: JavaScript `median(Array)`

function median(values) {
    values.sort( function(a,b) {return a - b;} );
    var half = Math.floor(values.length/2);
    if (values.length % 2) {
        return values[half];
    } else {
        return (values[half-1] + values[half]) / 2.0;
    }
}

GG DevOps

Published on