A Programmers Toolbox

I spend most of my time in Xcode working on iOS apps. I just finished updating them to iOS7 and thought it would be fun to log all of the different programs that I use in the next week or so.

I’ll start by backtracking a bit. When proofing one app, a blank spot appeared where a picture should be. The most likely cause of that would be a mis-named picture or one that was completely missing. The names of the pictures are read from a database so I needed to check the names in the database with the names of the pictures. The database is on my server so I could open phpMyAdmin and check the names there, but in case the problem was in the PHP script that I used to create the app database, I decided to open Firefox and use the SQLite Manager plugin to look at the file that I actually use in the app. It turns out that one of the pictures was missing. I had an extra picture in my app so the total number was correct. I swapped the picture and the sounds and the app worked correctly.

We put a frame around the drawings that show up on the screen in the apps. Most of the drawings are floating in space, so the size of the frame doesn’t matter. Sometimes the drawing comes to the edge of the frame and the size of the frame is slightly different than the size of the picture. There is a couple of pixel wide band of whitespace that is out of place. We decided to extend the drawing to get rid of the white space between the picture and the frame. Our drawings are done in Adobe Flash and exported as .png’s. We usually have hundreds of drawings in our apps, so we reduce the size of them by using ImageOptim. I processed the new drawings and added them to the game.

I had a minor issue in the app that I thought might be related to a change I had made earlier. I use Git for version control, so in Xcode I checked previous versions of the file to see if that was where the problem occurred.

Once everything was proofed and submitted to Apple for review, I opened GitX to commit any files that Xcode missed. Xcode often misses deleted files. Then I opened Terminal and ran the git push command to synchronize my copy of the apps with the copy on the server.

I use Apple’s iTunes Connect website to manage my apps and check on daily sales. I checked to make sure all of the apps were waiting for review and I hadn’t missed any uploads. App Annie lets me track revenue by product and over arbitrary time periods. I spent a couple of hours reviewing my sales and checking reviews.

Checking the App Annie website doesn’t require a lot of concentration so I decided to listen to some music. I’ve been copying some of my LPs to disk so I can listen to them on my iPod or stream them with iTunes on my AirTunes network. I have a turntable hooked up to my Mac Mini using an old Roland UA-30 USB interface. I opened Felt Tip’s Sound Studio and recorded a few LPs. Sound Studio is great for recording and cleaning up voice recordings and we use it all the time. It doesn’t have filters for cleaning pops on LPs, so I opened Audacity to get rid of the pops and noise. Audacity is a good recording tool, but its workflow is arranged around projects and doesn’t work well for cutting hundreds of sounds out of a recording session and saving them as .aiff’s. Sound Studio works well for our process. It’s also great for marking and exporting songs from an LP.

I often help other people with their websites and today I got a request to update some data in an SQL file on another persons server. I usually use phpMyAdmin to work with MySQL files, but they don’t have it installed on their server, so I needed to either work on the command line or find another way to work with the database. I can work on the command line, but I really don’t like to. Oracle now owns MySQL so I went to their website and downloaded MySQLWorkbench. It took a while to figure out how to use it but it works fine. The person giving me the data didn’t know how to get it out of his spreadsheet and into a flat file, so I opened the spreadsheet in LibreOffice and exported the worksheets as comma-delimited text. The format of the data wasn’t the same as the format in the database so I used BBEdit to clean up the data. BBEdit has a search and replace feature that uses grep and it only took a few minutes to rearrange the file into something that could be imported into MySQL. I documented that process in a previous post. Since I hadn’t used MySQLWorkbench before, I opened a database on a Digital Ocean droplet that my nephew is using for learning web development. That way if I did something really dumb to the database, it wouldn’t affect a production server. Once I got the update working properly—the handoff had some issues that needed to be fixed was ready to update the database. I always make a copy of every table before I update it and phpMyAdmin has a button to push to do that. I couldn’t find a menu item or button in MySQLWorkbench, so I duplicated my test table on the Digital Ocean server using phpMyAdmin and looked for the sql code that it used.


CREATE TABLE  `Customer1`.`values_Air_Temp_BAK` (
`id` INT( 8 ) NOT NULL AUTO_INCREMENT ,
 `datetime` DATETIME DEFAULT NULL ,
 `value` FLOAT DEFAULT NULL ,
 `source_type_id` INT( 11 ) DEFAULT NULL ,
PRIMARY KEY (  `id` ) ,
UNIQUE KEY  `datetime` (  `datetime` )
) ENGINE = MYISAM DEFAULT CHARSET = utf8;

SET SQL_MODE =  'NO_AUTO_VALUE_ON_ZERO';

INSERT INTO  `Customer1`.`values_Air_Temp_176_BAK` 
SELECT * 
FROM  `Customer1`.`values_Air_Temp_176` ;

I pasted this code into MySQLWorkbench on the production server and had my backup.

I updated the data in the backup and had the user check it for egregious errors. Once it was validated I again used the test server to get the code to rename the backup.


RENAME TABLE  `Customer1 `.`values_Air_Temp` TO  `Customer1 `.`values_Air_Temp_OLD`;
RENAME TABLE  `Customer1 `.`values_Air_Temp_BAK` TO  `Customer1 `.`values_Air_Temp`;

I had the user double check that everything looked right and I was done with that task.

By the way, this blog is hosted on my server using WordPress. I host several blogs and they need updated from time to time. I can never remember the exact steps to do it (I should write a bash script to update them) so I keep the instructions and code in a wiki on our server. Trac is good for keeping notes on procedures and processes and we use it for bug reports when we are testing the software.

I’m managing four servers now and I’ve been meaning to automate some of the process. Since I had some down time, I decided to refresh my bash skills and write some scripts. I especially wanted to automate the login process to different servers and color code the prompts so it was clear where I was. I’ve also been trying out iTerm2 as a terminal.

I just got a call from my nephew about his server. He’s been playing with PHP, CSS, and JavaScript and messed up a couple of files. I have the originals on my machine, so I opened Cyberduck and uploaded them to his server. Cyberduck is an OSX only program, so he uses Filezilla on his Ubuntu Linux machine. It works on OSX, but the Cyberduck interface is more “Mac like”. I showed him how to use the W3C validator tools to check his code for errors. It’s a lot easier to figure out why your HTML and CSS isn’t doing what you want if you don’t have any errors in it.

After I wrote the script to log me in to all the machines I use, I set up SSH authorized keys so that I don’t have to type the password each time. I’m not sure that I would do this with a laptop that I carry around, but I’m using a Mac Mini and it rarely leaves the house. I need to copy my public key to a file called authorized_keys in my .ssh directory on each machine. I could use Cyberduck, but the scp command works better.


scp ~/.ssh/RSA_KEY.pub me@192.168.102.113:/home/me/.ssh/authorized_keys

I updated my .bash_config script so that I have a function that copies my config files to all the machines I use. It uses the same scp command, except that it copies the file from my mini to all of the servers. Since I just updated all of them to use secure authorization, no passwords are required for it to work.


id=$(who am i | cut -d\  -f 1)
server='192.168.194.220'
dave='192.168.201.11'
don='192.168.102.113'
purple='purple.someserver.com'

# Update the list of servers if you add one. It is used on connect and scp functions
servers=(server dave don purple)

# Share .bash_config. Requires . ~/.bash_config in .bash_profile
share() {
    
    # This works if you want to list out each server
    # But if you add new servers, you need to remember to update this list
    # scp .bash_config ${id}@${server}:.bash_config
    # scp .bash_config ${id}@${dave}:.bash_config
    # scp .bash_config ${id}@${don}:.bash_config
    
    # This is better because the list of servers isn’t hard coded in the script 
    for serv in "${servers[@]}"; do
        echo "Connecting to $serv"
        scp .bash_config ${id}@${!serv}:.bash_config
    done
}

Every morning, I open my php_error.log file with BBEdit to see if I have any php or MySQL errors. I started getting lots of weird errors a while ago and it turns out they are MySQL injection attacks. I fixed the code and tracked the attempts for a while, but since it seems to be working now, I no longer log them.

I just took some pictures of unusual cloud formations, and posted them on a flying blog that I run. I like to use Image Capture to grab the images from the camera and then crop and resize them in Acorn. Though it’s not related to work right now, we often we often use the same process to get images for apps. Some of the members of the team use Adobe PhotoShop to fix the images, but my version doesn’t run on OSX Mavericks, so I only use it when I’m using my laptop.

I also use some apps that are not directly related to programming. I just got a couple of orders for CDs so I opened the FileMaker Pro database on my laptop to process the order. This is a really old version of FileMaker that won’t run on Mavericks. But it still works fine and I don’t need any of the newer features. We used to use FileMaker more for developing the wordlists and text for our games, but we moved most of that to the web using PHP, MySQL, and a bit of JavaScript. That way people can work on the database without having to use sneaker-net to get the correct database. One of my goals for the year is to update the database so that it runs in LibreOffice. I use Square on my iPod to process credit cards and Stripe to process cards from international sales.

Our CD burning robot is also very old and runs on a PPC Mac Mini using PTPublisher.

So that’s about it for a fairly normal week.

Using grep to convert date and time.

I was given a flat file that had the date in this format


Date,Hour,Min,Seconds,Temperature,Unused field
11/25/2013,14,0,0,59.69,
11/25/2013,14,15,0,60.38,
11/25/2013,14,30,0,61.85,
11/25/2013,14,45,0,62.57,

but the table that I need to put it into is formatted as

(64349,'2014-01-06 20:45:00',44.258,NULL),
(64350,'2014-01-06 21:00:00',44.978,NULL),

In BBEdit I used this grep to convert the date and separate out the temp into a separate field.


Find
([0-9]+)/([0-9]+)/([0-9]+),([0-9]+),([0-9]+),([0-9]+),([0-9.]+),
Replace
'\3-\2-\1 \4:\5:\6',\7,NULL),

What I’ve done is separate out the different parts of the date with parentheses. Then I rearranged the month, day, year with \3-\2-\1 and added the dash separator. The time is in the correct order so all I need to do is put in the right separator. I want an SQL file when I’m done so I added the ‘ field separators and the NULL field and line terminator.

I’ll use the numbering feature of BBEdit to add the row numbers and then add a paren at the beginning of the line. I also need to remember to change the last comma to a semi-colon. Then I can import the data to the MySQL table.

After I rearranged the date, I did need to fix months and days that were only one digit—they need a 0 before the digit. I did this manually with a find and replace. Same with the time. If you did this a lot, you could write a grep that it looked for records with only one digit and prefix it with a zero. A ? will look for exactly one occurrence of a digit. Then do a second pass to get the rest. Or you could use the # to look for one or two digits.

bash

I’m playing around with bash and trying to understand how it thinks. I manage a bunch of servers so I wrote some aliases and functions to help me manage them. In my .profile or my .bash_profile I added a line to run the shell script that helps me do things. The last line is:


    . ~/.bash_config

If you make changes to this file, you can reload your profile and see the changes by running:


    source ~/.profile

I found a bunch of .bash_profile’s online and used some of the ideas from them. The best one is from Nathaniel Landau. I kept his basic structure and lots of his aliases. I also added a bunch of stuff people might be interested in. I left in a lot of stuff that I later rewrote so people learning bash can see my thought process as I developed the script.


#  ---------------------------------------------------------------------------
#  .bash_config
#  Description:  BASH configurations and aliases
#  Called from .profile or .bash_profile with . ~/.bash_config
#  Sections:
#  1.   Environment Configuration
#  2.   Make Terminal Better (remapping defaults and adding functionality)
#  3.   File and Folder Management
#  4.   Searching
#  5.   Process Management
#  6.   Networking
#  7.   System Operations & Information
#  8.   Web Development
#  9.   Reminders & Notes
#
# Source: http://natelandau.com/my-mac-osx-bash_profile/
# Modified extensively - January 2014
#  ---------------------------------------------------------------------------

#   -------------------------------
#   1.  ENVIRONMENT CONFIGURATION
#   -------------------------------

#   Change Prompt
#   ------------------------------------------------------------
#   export PS1="________________________________________________________________________________
     \n| \w @ \h (\u) \n| => "
#   export PS2="| => "
#  \e[ - Indicates the beginning of color prompt
#  x;ym - Indicates color code. Use the color code values mentioned below.
#  \e[m - indicates the end of color prompt
#   \e works on Ubuntu and in the OSX prompt, but not in echo commands on OSX use \033 instead
# Normal Colors
BLACK='\033[0;30m'        # Black
RED='\033[0;31m'          # Red
GREEN='\033[0;32m'        # Green
YELLOW='\033[0;33m'       # Yellow
BLUE='\033[0;34m'         # Blue
PURPLE='\033[0;35m'       # Purple
CYAN='\033[0;36m'         # Cyan
WHITE='\033[0;37m'        # White

# Bold - Change 0; to 1;
B_BLACK='\033[1;30m'      # Black
B_RED='\033[1;31m'        # Red
B_GREEN='\033[1;32m'      # Green
B_YELLOW='\033[1;33m'     # Yellow
B_BLUE='\033[1;34m'       # Blue
B_PURPLE='\033[1;35m'     # Purple
B_CYAN='\033[1;36m'       # Cyan
B_WHITE='\033[1;37m'      # White

NC='\033[m'               # No Color - Reset color prompt

#   Make it clear in the prompt which system I’m on
#   localName=$(echo $HOSTNAME | cut -d.  -f 2); # Pull out the local part of the HOSTNAME
if [ "$HOSTNAME" = "server" ]; then
    hostColor=$B_RED;
elif [ "$HOSTNAME" = "dave" ]; then
    hostColor=$B_PURPLE;
elif [ "$HOSTNAME" = "don" ]; then
    hostColor=$B_CYAN;
# Mac $HOSTNAME has .local appended, cut it out
elif [ $(echo $HOSTNAME | cut -d.  -f 2) = "local" ]; then
    hostColor=$B_BLACK;
else
    hostColor=$B_GREEN;
fi
    # Colors are in {} just to be clear, even though \ is not a valid variable character.
    # NC doesn’t need to be in {} since it is followed by a space.
    export PS1="${B_BLUE}\u@${hostColor}\h: ${B_GREEN}\w$NC $ "

#   Set Paths
#   ------------------------------------------------------------
#    export PATH="$PATH:/usr/local/bin/"
#    export PATH="$PATH:/usr/local/git/bin:/sw/bin/:/usr/local/bin:/usr/local/:/usr/local/sbin:/usr/local/mysql/bin"
    
#   Set Default Editor (change 'Nano' to the editor of your choice)
#   ------------------------------------------------------------
    export EDITOR=/usr/bin/vim
    if [ "$HOSTNAME" = "Dave" ]
    then
        export EDITOR=/usr/bin/nano
    fi

#   Keep the history file around forever
    export HISTSIZE=10000
    export HISTFILESIZE=1000000000

#   Set default blocksize for ls, df, du
#   from this: http://hints.macworld.com/comment.php?mode=view&cid=24491
#   ------------------------------------------------------------
    export BLOCKSIZE=1k

#   Add color to terminal
#   from http://osxdaily.com/2012/02/21/add-color-to-the-terminal-in-mac-os-x/
#   ------------------------------------------------------------
    export CLICOLOR=1
#   export LSCOLORS=ExFxBxDxCxegedabagacad

    export TZ=America/Los_Angeles
    
#   -----------------------------
#   2.  MAKE TERMINAL BETTER
#   -----------------------------
#   Override an alias with command e.g. command cp file1 file2
#   -i is the interactive flag, -v is the verbose flag
alias cp='cp -iv'                           # -i warns before overwriting
alias mv='mv -iv'                           # -i warns before overwriting
alias mkdir='mkdir -pv'                     # -p Create intermediate directories as required
mcd () { mkdir -p "$1" && cd "$1"; }        # Makes new Dir and jumps inside

alias ll='ls -FGlAhp'                       # 
alias less='less -FSRXc'                    # Preferred 'less' implementation

cd() { builtin cd "$@"; ll; }               # Always list directory contents upon 'cd'
alias cd..='cd ../'                         # Go back 1 directory level (for fast typers)
alias ..='cd ../'                           # Go back 1 directory level
alias ...='cd ../../'                       # Go back 2 directory levels
alias .3='cd ../../../'                     # Go back 3 directory levels
alias .4='cd ../../../../'                  # Go back 4 directory levels
alias .5='cd ../../../../../'               # Go back 5 directory levels
alias .6='cd ../../../../../../'            # Go back 6 directory levels
alias ~="cd ~"                              # Go Home

alias which='type -all'                     # Find executables
alias path='echo -e ${PATH//:/\\n}'         # Echo all executable Paths

#
#   OSX specific aliases and commands
#
alias f='open -a Finder ./'                 # Opens current directory in OSX Finder

trash () { command mv "$@" ~/.Trash ; }     # Moves a file to the OSX trash
ql () { qlmanage -p "$*" >& /dev/null; }    # Opens any file in OSX Quicklook Preview
alias DT='tee ~/Desktop/terminalOut.txt'    # Pipe content to file on OSX Desktop

#   lr:  Full Recursive Directory Listing
#   ------------------------------------------
alias lr='ls -R | grep ":$" | sed -e '\''s/:$//'\'' -e '\''s/[^-][^\/]*\//--/g'\'' -e '\''s/^/   /'\'' -e '\''s/-/|/'\'' | less'

#   mans:   Search manpage given in agument '1' for term given in argument '2' (case insensitive)
#           displays paginated result with colored search terms and two lines surrounding each hit. 
#           Example: mans mplayer codec
#   --------------------------------------------------------------------
    mans () {
        man $1 | grep -iC2 --color=always $2 | less
    }
alias mansearch='mans'
#   showa: to remind yourself of an alias (given some part of it)
#   ------------------------------------------------------------
    showa () { /usr/bin/grep --color=always -i -a1 $@ ~/Library/init/bash/aliases.bash | grep -v '^\s*$' | less -FSRXc ; }

#   -------------------------------
#   3.  FILE AND FOLDER MANAGEMENT
#   -------------------------------

zipf () { zip -r "$1".zip "$1" ; }          # Create a ZIP archive of a folder
alias numFiles='echo $(ls -1 | wc -l)'      # Count of non-hidden files in current dir
alias make1mb='mkfile 1m ./1MB.dat'         # Creates a file of 1mb size (all zeros)
alias make5mb='mkfile 5m ./5MB.dat'         # Creates a file of 5mb size (all zeros)
alias make10mb='mkfile 10m ./10MB.dat'      # Creates a file of 10mb size (all zeros)

#   extract:  Extract most known archives with one command
#   ---------------------------------------------------------
    extract () {
        if [ -f $1 ] ; then
          case $1 in
            *.tar.bz2)   tar xjf $1     ;;
            *.tar.gz)    tar xzf $1     ;;
            *.bz2)       bunzip2 $1     ;;
            *.rar)       unrar e $1     ;;
            *.gz)        gunzip $1      ;;
            *.tar)       tar xf $1      ;;
            *.tbz2)      tar xjf $1     ;;
            *.tgz)       tar xzf $1     ;;
            *.zip)       unzip $1       ;;
            *.Z)         uncompress $1  ;;
            *.7z)        7z x $1        ;;
            *)     echo "'$1' cannot be extracted via extract()" ;;
             esac
         else
             echo "'$1' is not a valid file"
         fi
    }

#   ---------------------------
#   4.  SEARCHING
#   ---------------------------

alias qfind="find . -name "                 # qfind:    Quickly search for file
ff () { /usr/bin/find . -name "$@" ; }      # ff:       Find file under the current directory
ffs () { /usr/bin/find . -name "$@"'*' ; }  # ffs:      Find file whose name starts with a given string
ffe () { /usr/bin/find . -name '*'"$@" ; }  # ffe:      Find file whose name ends with a given string

#   spotlight: Search for a file using MacOS Spotlight's metadata
#   -----------------------------------------------------------
    spotlight () { mdfind "kMDItemDisplayName == '$@'wc"; }

#   
alias h='history'                           # Shortcut for history
hist() { history | grep $1; }               # Find a command in the history, e.g hist man

#   ---------------------------
#   5.  PROCESS MANAGEMENT
#   ---------------------------

#   findPid: find out the pid of a specified process
#   -----------------------------------------------------
#       Note that the command name can be specified via a regex
#       E.g. findPid '/d$/' finds pids of all processes with names ending in 'd'
#       Without the 'sudo' it will only find processes of the current user
#   -----------------------------------------------------
    findPid () { lsof -t -c "$@" ; }

#   memHogsTop, memHogsPs:  Find memory hogs
#   -----------------------------------------------------
    alias memHogsTop='top -l 1 -o rsize | head -20'
    alias memHogsPs='ps wwaxm -o pid,stat,vsize,rss,time,command | head -10'

#   cpuHogs:  Find CPU hogs
#   -----------------------------------------------------
    alias cpu_hogs='ps wwaxr -o pid,stat,%cpu,time,command | head -10'

#   topForever:  Continual 'top' listing (every 10 seconds)
#   -----------------------------------------------------
    alias topForever='top -l 9999999 -s 10 -o cpu'

#   ttop:  Recommended 'top' invocation to minimize resources
#   ------------------------------------------------------------
#       Taken from this macosxhints article
#       http://www.macosxhints.com/article.php?story=20060816123853639
#   ------------------------------------------------------------
    alias ttop="top -R -F -s 10 -o rsize"

#   my_ps: List processes owned by my user:
#   ------------------------------------------------------------
    my_ps() { ps $@ -u $USER -o pid,%cpu,%mem,start,time,bsdtime,command ; }

#   ---------------------------
#   6.  NETWORKING
#   ---------------------------

#   Log in to other systems

id=$(who am i | cut -d\  -f 1)
server='192.168.194.220'
dave='192.168.201.11'
don='192.168.102.113'
purple='purple.aserver.com'

# Update the list of servers if you add one. It is used on connect and scp functions
servers=(server dave don purple)

#   The commented out aliases and functions are experiments with different ways to use aliases and functions

#   With the id placed in the alias, not generalizable
#   alias server='ssh myloginid@192.168.194.220'

#   Find the userID from login info
#   alias server='ssh $(who am i | cut -d\  -f 1)@192.168.194.220'

#   Make the userID an alias
#   alias myID='who am i | cut -d\  -f 1'
#   alias server='ssh "echo $(myID)"@192.168.194.220'
#   alias server='ssh $(myID)@192.168.194.220'

#   Make the userID a variable and use it later in the function
#   server() { id=$(who am i | cut -d\  -f 1); ssh $id@192.168.194.220; }

#   Make the userID a variable in the shell script and use it in an alias or function
#   $id can’t be local since isn’t called until the alias or function is invoked.
#   id=$(who am i | cut -d\  -f 1)
#   alias server='ssh $id@192.168.194.220'
#   server()  { ssh $(myID)@192.168.194.220 ; }

# Using the select conditional to pick a server
# This example shows how a case statement and $REPLY can be used
# It can be generalized so that answers aren’t hard coded
whichserver() {
    # We don’t need the list outside this function. Making it local assures that we don’t clobber a previous alias with the same name
    local serverlist=(server dave don purple)              
    PS3='Select a server. '
    
    select srv in ${serverlist[@]} "Quit"; do     # the @ expands all the elements of the array into separate words

        case "$REPLY" in
            # It would be nice if you could do this, but you can’t put anything in a case statement that isn’t one of the cases
            # arrayitem=$(( ${#serverlist[@]}+1 ))   
            1 ) 
                echo "Connecting to ${serverlist[$REPLY-1]}"; ssh "$id@$server" ;;
            2 ) 
                echo "Connecting to ${serverlist[$REPLY-1]}"; ssh "$id@$dane" ;;
            3 ) 
                echo "Connecting to ${serverlist[$REPLY-1]}"; ssh "$id@$dane" ;;
            
            # The last option is quit. (( )) is arithmetic evaluation. # counts the number of items in the array. 
            $(( ${#serverlist[@]}+1 )) )       
                echo "No server selected.";
        esac
        
        break

    done
}

#   Here's the same code using an eval to run the alias

connect() {
    # Requires an array called servers that has IP addresses or domain names.
    # It is defined when aliases are defined at the beginning of this script.
    if [ -z "$1" ]; then
        
        PS3='Select a server. '
        select srv in ${servers[@]} "Quit"; do     # the @ expands all the elements of the array into separate words

            if [ "$REPLY" -lt $(( ${#servers[@]}+1 )) ]; then
                    echo "Connecting to ${servers[$REPLY-1]}"
                    ssh ${!servers[$REPLY-1]}
             else
                    echo "No server selected.";
            fi
        
            break

        done
    else
        echo "Connecting to $1"
        ssh ${!1}
    fi
}

# Share .bash_config. Requires . ~/.bash_config in .bash_profile
share() {
    
    # This works if you want to list out each server
    # scp .bash_config ${id}@${server}:.bash_config
    # scp .bash_config ${id}@${dane}:.bash_config
    # scp .bash_config ${id}@${dan}:.bash_config
    
    for serv in "${servers[@]}"; do
        echo "Connecting to $serv"
        scp .bash_config ${id}@${!serv}:.bash_config
    done
}

alias myip='curl ip.appspot.com'                    # Use Google’s AppSpot site to find this machine’s public facing IP Address
alias netCons='lsof -i'                             # Show all open TCP/IP sockets
alias flushDNS='dscacheutil -flushcache'            # Flush out the DNS Cache
alias lsock='sudo /usr/sbin/lsof -i -P'             # Display open sockets
alias lsockU='sudo /usr/sbin/lsof -nP | grep UDP'   # Display only open UDP sockets
alias lsockT='sudo /usr/sbin/lsof -nP | grep TCP'   # Display only open TCP sockets
alias ipInfo0='ipconfig getpacket en0'              # Get info on connections for en0
alias ipInfo1='ipconfig getpacket en1'              # Get info on connections for en1
alias openPorts='sudo lsof -i | grep LISTEN'        # All listening connections
alias showBlocked='sudo ipfw list'                  # All ipfw rules inc/ blocked IPs

#   ii:  display useful host related informaton
#   -------------------------------------------------------------------
#   -e     enable interpretation of backslash escapes

    ii() {
        ## \e works on Ubuntu but not recognized on OSX, must use \033
        ## Didn’t work since colors weren’t defined. Now they are.
        ## local RED='\033[0;31m'          # Red
        ## local NC="\033[0m"              # Color Reset
        echo -e "\nYou are logged on to $HOST"
        echo -e "\nAdditional information: " ; uname -a
        echo -e "\n${RED}Users logged on:${NC} " ; w -h
        echo -e "\n${RED}Current date :${NC} " ; date
        echo -e "\n${RED}Machine stats :${NC} " ; uptime
        echo -e "\n${RED}Current network location :${NC} " ; scselect
        echo -e "\n${RED}Public facing IP Address :${NC} " ;myip
        #echo -e "\n${RED}DNS Configuration:${NC} " ; scutil --dns
        echo
    }

#   ---------------------------------------
#   7.  SYSTEMS OPERATIONS & INFORMATION
#   ---------------------------------------

alias mountReadWrite='/sbin/mount -uw /'    # mountReadWrite:   For use when booted into single-user

#   cleanupDS:  Recursively delete .DS_Store files
#   -------------------------------------------------------------------
    alias cleanupDS="find . -type f -name '*.DS_Store' -ls -delete"

#   finderShowHidden:   Show hidden files in Finder
#   finderHideHidden:   Hide hidden files in Finder
#   -------------------------------------------------------------------
    alias finderShowHidden='defaults write com.apple.finder ShowAllFiles TRUE'
    alias finderHideHidden='defaults write com.apple.finder ShowAllFiles FALSE'

#   cleanupLS:  Clean up LaunchServices to remove duplicates in the "Open With" menu
#   -----------------------------------------------------------------------------------
    alias cleanupLS="/System/Library/Frameworks/CoreServices.framework/Frameworks/LaunchServices.framework/Support/lsregister -kill -r -domain local -domain system -domain user && killall Finder"

#    screensaverDesktop: Run a screensaver on the Desktop
#   -----------------------------------------------------------------------------------
    alias screensaverDesktop='/System/Library/Frameworks/ScreenSaver.framework/Resources/ScreenSaverEngine.app/Contents/MacOS/ScreenSaverEngine -background'

#   ---------------------------------------
#   8.  WEB DEVELOPMENT
#   ---------------------------------------

alias apacheEdit='sudo $EDITOR /etc/httpd/httpd.conf'   # Edit httpd.conf
alias apacheRestart='sudo service apache2 restart'      # Restart Apache
alias mySQLRestart='sudo service mySQL restart'         # Restart mySQL
alias editHosts='sudo $EDITOR /etc/hosts'               # Edit /etc/hosts file
alias taile='tail -f /var/log/php_error.log'            # Tails PHP error logs
alias apacheLogs="less +F /var/log/apache2/error_log"   # Shows apache error logs
httpHeaders () { /usr/bin/curl -I -L $@ ; }             # httpHeaders:      Grabs headers from web page

#   httpDebug:  Download a web page and show info on what took time
#   -------------------------------------------------------------------
    httpDebug () { /usr/bin/curl $@ -o /dev/null -w "dns: %{time_namelookup} connect: %{time_connect} pretransfer: %{time_pretransfer} starttransfer: %{time_starttransfer} total: %{time_total}\n" ; }

#   ---------------------------------------
#   9.  REMINDERS & NOTES
#   ---------------------------------------

# Things I Can’t Remember
ticr() {
    echo -e "\n${B_PURPLE}Things I Can’t Remember$NC"
#   Ubuntu update commands
    echo -e "\n${B_BLACK}safe-upgrade is preferred$NC"
    echo -e "sudo aptitude safe-upgrade"

    echo -e "\n${B_BLACK}If safe-upgrade doesn’t work you can do it manually$NC"
    echo -e "sudo apt-get update"
    echo -e "sudo apt-get upgrade"

    echo -e "\n${B_BLACK}Sometimes a reboot is required after an update$NC"
    echo -e "sudo reboot"
    
    echo -e "\n${B_BLACK}scp a file e.g public key$NC"
    echo -e "scp ./.ssh/RSA_KEY.pub myloginid@198.199.102.113:/home/myloginid/.ssh/authorized_keys"
}

Bash: Using an alias in an alias

I’ve been helping set up some VPSs (Virtual Private Server) and at the moment they don’t have domain names assigned. I can use the history command to find the right IP address to ssh in but that can be cumbersome. So I decided to write an alias for each server. These lines are in my extended bash config file that is called when my .profile runs.

My login is the same on each server so I could hard code this, but I made it general so I can share it with other users. I don’t like to hard code things so I thought I’d let bash find my user id and then use it in the ssh command. My first thought was to just create an alias for my login and use it in an alias for each server. But I couldn’t get it to work. I tried using just the alias since examples of creating aliases seem to work this way with builtin commands. Then I thought that maybe I need to escape the alias so that bash knows that it should process it.


alias myID='who am i | cut -d\  -f 1'
alias myserver='ssh myID@192.245.184.221'      #Doesn’t work
alias myserver='ssh $(myID)@192.245.184.221'   #Works fine

It turns out that the second line works, but I did had a problem with aliases being set and it didn’t work for me. So I started playing around with things that did work and making small adjustments. FYI, One thing you need to do if you redefine aliases and functions is to make sure that you clear out the old ones. If you don’t then you may not be running the function you thought you were or you could be running an alias when you thought you were running a function.

You can use unset to remove them from memory. Use the -f flag for functions.


unset myID
unset -f dave

For myserver, I put the alias command in parens and prefixed it with a $. Bash executes this first and then uses it in the ssh command. If you don’t do this, bash thinks that the first argument for ssh is ‘who’ and fails. That worked, but I don’t like the idea of having duplicate code in login alias. So I thought I’d try a function. In the first function, I define a variable and then use it. The third function looks an awful lot like a failed alias in the example above, but since it is a function, it works. I think what is happening is that when bash sees $(myID) it thinks, I should process whatever is inside the parens. So it finds the alias and runs it.


#   Log in to other systems
alias myID='who am i | cut -d\  -f 1'

alias myserver='ssh $(who am i | cut -d\  -f 1)@192.245.184.221'
dave() { id=$(who am i | cut -d\  -f 1); ssh $id@192.233.221.11; }
dana()  { ssh $(myID)@192.149.142.163 ; }

alias bigserver='ssh "echo $(myID)"@192.255.174.220'

It finally occurred to me that the .profile script is a full blown shell script. So that means I can use variables in it. I define an id variable that is the result of the shell executing the commands that are in it. When the .profile is run, bash assigns id to my login. Then I can use it anywhere I want with $id. I don’t need to use ${id} since my login is one word and since the @ cannot be in a variable name, bash knows to stop processing when it hits the @. This is what I wanted to do with aliases. I thought about making it a local variable since I don’t need it to hang around outside of this script. But the alias isn’t resolved until it is used, so it needs to resolve the variable when it is called.


#   Log in to other systems
id=$(who am i | cut -d\  -f 1)
alias myserver='ssh $id@192.245.184.221'
alias dave='ssh $id@192.233.221.11'
alias dana='ssh $id@192.149.142.163'

I started playing around with aliases again and got strictly aliases to work as well.


#   Log in to other systems
alias myID='who am i | cut -d\  -f 1'
alias myserver='ssh $(myID)@192.245.184.221'
alias dave='ssh $(myID)@192.233.221.11'
alias dana='ssh $(myID)@192.149.142.163'

Update: I was playing with printenv and it turns out that one of the environment variables is LOGNAME. So you can replace the $id with $LOGNAME in the code above.

Notes on setting up a server.

I’m setting up a new Ubuntu server and while most of the defaults are fine, there are some things that I need to adjust. I have a very shallow understanding of this stuff, so there could be better and more secure ways to do this, but this works for me.

Disallow access to PHP include files

There isn’t any reason that people need to see the include files that I use in my websites. You could name them .inc.php so that the raw code isn’t available, but that’s not very elegant, and outsiders can still access the file. There isn’t anything particularly sensitive in them, but by themselves, they don’t display correctly. So I added a few lines to my /etc/apache2/apache2.conf file. Just below the section that disallows viewing .htacess files.


#
# The following lines prevent .htaccess and .htpasswd files from being 
# viewed by Web clients. 
#
<Files ~ "^\.ht">
    Order allow,deny
    Deny from all
    Satisfy all
</Files>

# The following lines prevent .inc files from being
# viewed by Web clients.
#
<Files ~ "\.inc$">
    Order allow,deny
    Deny from all
</Files>
#

Prevent directory browsing

If you have a bunch of images in a directory, then anyone who wants can view all of them just by looking at the web page source and putting the directory name after your URL. I’d rather they not do that, so I restrict listing of the files by adding this line to my /etc/apache2/httpd.conf file. On my default Ubuntu install this file is empty.


Options Includes FollowSymLinks MultiViews

Restart Apache for the changes to take effect.

Alternate method to prevent directory browsing

If you want to prevent directory browsing in just one directory and either don’t want to change the whole site or don’t have access to the files named above, add this line to your .htaccess file.


Options -Indexes

Probably don’t have to restart Apache for changes to take effect.

Prevent Directory Browsing on a Per Site Basis

Changing the httpd.conf file will change the behavior of all sites on your server. If you want to change the behavior of just one site, edit its file in /etc/apache2/sites-avalilable. Find the line that has Options FollowSymLinks in it and if it has Indexes in it, delete it. This is what the default Ubuntu install has.


  <Directory /var/www/>
    Options Indexes FollowSymLinks MultiViews
    AllowOverride None
    Order allow,deny
    allow from all
  </Directory>

Probably do have to restart Apache for changes to take effect.

Prevent access to your include directory

Add this to your site’s file in /etc/apache2/sites-avalilable.


#<Directory /www/MySite/include.php>
#    Deny from all
#</Directory>

Show an error document instead of the default 404 error

Create a normal php document with your sites navigation and a message that says the file can’t be found and maybe you can find it with the nav menus. Add this to your site’s file in /etc/apache2/sites-avalilable. And while you are at it, there is no reason you need to tell anyone that they don’t have permission to see a particular file, just tell them it’s not found, so add the same line for a 403 error. I take them back to the main page and display the missing file in the main page.


ErrorDocument 404 /index.php?p=missing
ErrorDocument 403 /index.php?p=missing