Howto: Unpack/Extract Files from Multiple Folders in One Go

Just recently I had a situation where I needed to extract multiple archives, roughly 100 in fact, with each being in a sub folder.

Not particularly wanting to sit clicking into each folder, right click the archive and extract each one, then move the file (avi in this case) up into the parent folder I got busy on the terminal. Luckily this is a perfect job for Bash and instead of the job taking around 2hrs of tedious clicking and waiting to extract each I was able to leave the command processing the files and get on with some more interesting work while the job was done.

So here is the command line that go it all sorted.

  cur="`pwd`";for i in `"ls"`; do echo "$i";cd "$i";unrar e "*.rar";mv "*.avi" "$cur"/ ;cd "$cur";done


So its not very elegant, I was going to use "find" but as I wanted to move the avi at the same time, thought this way would be better.

I would be happy for anyone to leave a comment and show me a better way, always eager to learn :]

Howto: Sync Bash History Between Terminals

Do you want to be able to use the history from another terminal quickly and simply in a new terminal, or to be able to use any one of the terminals on your desktops to continue working with a current history?

Me too :]

This is a very easy tweak and makes a lot of sense to most people. So we need to edit the file ~/.bashrc

gedit ~/.bashrc

At the bottom paste in the following:

shopt -s histappend
PROMPT_COMMAND="history -n; history -a"
unset HISTFILESIZE
HISTSIZE=2000

Unsetting HISTFILESIZE just gets rid of the filesize limitation, and we rely on HISTSIZE which allows 2000 lines in the file, both normally default to 500.

PROMPT_COMMAND tells bash to reload history and append to history each time it draws the prompt in the terminal.

The only thing to note is if you jump from one terminal to another and wish to recall the last history line from the former terminal, press enter once with a blank line, this causes bash to draw the prompt and sync, if you only just opened the second terminal then you don't have to. The sync only occurs when the prompt is drawn, nothing else to note.

A script to copy clipboard to a file.

While mucking about with some scripts I got a little bored of copying them into files and chmod +x and copying them into the path, so I just knocked up this little scriptlet. It will optionally set the execute flag on the file if it is a script you are pasting, or leave it as a normal txt file for just saving some text.


First install the package xclip with either Synaptic or:
sudo apt-get install xclip

This isn't a great script, there is very little error checking and it won't even go out of its way to tell you if the syntax is wrong, luckily its easy enough and does what I wanted simply.

Syntax is :
clip2file -x filename
Copies the clipboard into a file and sets it to execute with chmod +x then movies it into ~/bin
clip2file -a filename
Copies the clipboard into a file and nothing more.

You should have already made a bin/ folder in your home (~/bin/) this I believe is already defined as in the $PATH on Ubuntu install.

make a new file in that bin folder called "clip2file" and open it in your favourite text editor such as nano, vim or gedit and past the following script into it.

#!/bin/bash
#
# Copy the contents of the X clipboard into
# the specified file.
#
# This makes it crap loads easier to make new commands from scripts.
#
# Perhaps make it automagically +x if the first script line is
# #/bin/*
#
#

hflag=
aflag=
xflag=

while getopts 'ha:x:' OPTION
do
case $OPTION in
h) hflag=1 ;;

a) aflag=1
aval="$OPTARG" ;;

x) xflag=1
xval="$OPTARG" ;;

?) printf "Usage: %s: [-x filename] [-a filename] or use -h to recieve help\n"
exit 1 ;;
esac
done

shift $(($OPTIND - 1))

if [ "$hflag" ]
then
printf "Clip2file provides an easy method to create a file from the X clipboard\n"
printf "as either a text file, or make the file executable as a bash script.\n\n"
printf "-x filename ......... Create the file, then set the execute bit.\n"
printf "-a filename ......... Create the file but do not set execute.\n\n"
exit 1
fi

if [ "$xflag" ]
then
xclip -selection clipboard -o > "$xval"
chmod +x "$xval"
mv "$xval" $HOME/bin/
fi

if [ "$aflag" ]
then
xclip -selection clipboard -o > "$aval"
fi

exit 0

Set the file to allow executing, with with chmod +x clipt2file or by right clicking it in nautilus and choosing Proerties then going to the Permissions tab and setting it to allow executing.

Now if you want to add a new script off a website, simply copy the text into the clipboard and in a terminal or run dialog (alt+F2) enter:

clip2file -x filename

Filename of course being the name to save the file too, you can then run your new script right away.

Please remeber this script isn't perfect but its good enough for what it does, if someone would like to expand on it then I would happily post the improved version with credit.

Howto: SSH Public Key Authorisation Login (AKA Passwordless)

To be clear, this is how to login using SSH to another machine without having to enter your password. It is not an all encompassing solution but a "good enough" set up. I have to administer a couple of machines and a shell which I have use of, my own machine is kept up to date and I feel is secure enough with only myself as sole user.

Firstly we should make a key pair if you haven't already done so, these are deposited into into ~/.ssh

ssh-keygen -t rsa

Next we need to get the contents of ~/.ssh/id_rsa.pub onto our remote machine and into ~/.ssh/authorized_keys this can be done a couple ways, I found it as easy to just ssh into the remote machine and paste the contents directly into the file just note that if you paste it in you may have to correct the formatting so it is all on one line, I noticed a couple of line breaks sneak in. You could use something like the following however:

scp -P 20060 ~/.ssh/id_rsa.pub remoteaccount@remotesserver.com

Then you can just append it automatically with the following (we will touch the file in case it doesn't already exist on your machine:

touch ~/.ssh/authorized_keys
cat id_rsa.pub >> ~/.ssh/authorized_keys

We should at this point make the .ssh folder only readable by the owner:

chmod 700 ~/.ssh

You can optionally delete the id_rsa.pub key file if it was in your home folder, its the public key part so its not really necessary in my opinion but it is nice to tidy up after yourself ;)

rm id_rsa.pub

If you try and login now to the remote machine with SSH you should with any luck find you do not have to enter your password at all and can just get right on with your work.

How to use OTF fonts in Ubuntu

It isn't possible to use OTF fonts directly in Ubuntu without converting them into TTF's.

If we spend just a few minutes and install FontForge and make a very small script it becomes almost trivial to do.

sudo apt-get install fontforge

Now we should make the scripts, well it is two actually but one is a small config script for FontForge and the other is just a script to convert many OTF files in one go, rather than individually. First the FontForge settings file.

cd ~/bin
touch otf2ttf.conf

Then open otf2ttf.conf in your favourite text editor and paste the following into it.

#!/usr/local/bin/fontforge
# Quick and dirty hack: converts a font to truetype (.ttf)
Print("Opening "+$1);
Open($1);
Print("Saving "+$1:r+".ttf");
Generate($1:r+".ttf");
Quit(0);

Save the file, and now lets make the script to convert many fonts in one swoop.

cd ~/bin
touch otf2ttf
chmod +x otf2ttf

Again, open this file with your prefered text editor and paste the following in.

#!/bin/bash
#
# had to enable extglob
shopt -s extglob

# this uses globbing to match fiels ending in otf/OTF
for i in +(*.otf|*.OTF)

do fontforge -script /home/subbass/bin/otf2ttf.conf $i

done

To use this now (provided that your ~/bin folder is in your path) just open a terminal where your OTF files are, and issue the command otf2ttf. It will convert 20 files in just a couple of seconds to give you an idea of speed, you can then move all the resulting TTF files into ~/.fonts


Credit to http://www.stuermer.ch/blog/convert-otf-to-ttf-font-on-ubuntu.html where I found the script before re-typing the instructions. I put it here so I don't lose it next time I need it.

Howto Create Split RAR Files

Lets learn how to create split Rar files

If you have a large amount of data to backup or especially to transfer it can be very prudent to use an archive that can be split into smaller parts, if one section becomes corrupted during transfer, it is far better to transfer a single 15mb file again, than an entire 10GB file, right?

Lets grab the rar program:

sudo apt-get install rar

Ok lets compress our directory of files:

To compress file(s) to split rar archive know which directory you want to compress, I'll use a fictional DVD Image folder in the home folder.

rar a -m5 -v5M -R myarchive /home/yourname/dvdimage

Let me break the above command down

rar – starts the program
a – tells program to add files to the archive
-m5 – determine the compression level (0-store (fast)…3-default…5-maximum(slow))
-v15M – determine the size of each file in split archive, in this example you get files with size 15MB (if you wanted files of 512kB size you would write -v512k)
myarchive – name of the archive you are creating
/home/yourname/dvdimage – is folder of the files you wish to add to the archive

You can also add -p to the command after a and it will prompt you for a password.

You can read the manual for more options with man rar (Press q to exit and arrows to scroll up/down)

To uncompress the archive type:

rar x myarchive.part01.rar

Or right click on file myarchive.part01.rar in Nautilus and choose Extract Here.

Set up rxvt terminal with unicode and clickable URL's

Rxvt is a super fast unicode terminal which I have recently switched to away from Eterm for my transparent desktop terminal.

sudo apt-get install rxvt-unicode-ml

Next is to create a configuration file for it in your home folder.

touch .Xdefaults

Now paste this lot in...

URxvt*termName: rxvt

## borderless and no scrollbar
URxvt*scrollBar_right: false
URxvt*scrollBar: false
URxvt*borderLess: false

## teh transparency stuff
URxvt*inheritPixmap: true
URxvt*tintColor: white
URxvt*shading: 100

##
## These 3 lines make links clickable
##
URxvt.urlLauncher: firefox
URxvt.matcher.button: 1
URxvt.perl-ext-common: matcher,readline

## geometry and font
URxvt*geometry: 80×15
URxvt*font: xft:Terminus:pixelsize=12

## change default colors
URxvt*background: #000000
URxvt*foreground: #A8A8A8
URxvt*color0: #000000
URxvt*color1: #A80000

## URxvt*color2: #00A800
URxvt*color2: #ED254F

URxvt*color3: #A85400
## main bars in irssi
URxvt*color4: #020202
URxvt*color5: #A800A8
URxvt*color6: #00A8A8
URxvt*color7: #A8A8A8
URxvt*color8: #545054
URxvt*color9: #F85450

## Time in Irssi
URxvt*color10: #ED254F
## URxvt*color10: #50FC50

## URxvt*color11: #F2FC50
URxvt*color11: #ED254F

URxvt*color12: #5054F8

URxvt*color13: #ED254F
## URxvt*color13: #F854F8

URxvt*color14: #50FCF8
URxvt*color15: #F8FCF8

Note that some of the colours have been changed to coordinate with my current desktop theme, the original colours are commented out so you can easily revert. Also the line defining the window geometry seems to have no effect, I shall look into that and correct it when I get some time and inclination. At the moment I don't mind because the default window size is fine, and the launcher I use to start the transparent terminal I use on the desktop mainly for my screen with irssi, hellanzb and mediatomb in has the geometry set on it. the launcher command is:

rxvt -g 80x15

Last job is to tell compiz to not decorate this window, this will get rid of the title bar, borders and shadow etc.

Hopefully you have installed the advanced compiz settings (ccsm), so just open that up and go to the section for window decorations in Effects, Window Decorations.

Add the rxvt window to be excluded from shadows and decorations, and that should see you done.

How to read 20 magazines for free.

You will need Firefox and an addon called User agent Switcher and configure it with an iPhone setting :

Mozilla/5.0 (iPhone; U; CPU like Mac OS X; en) AppleWebKit/420.1 (KHTML, like Gecko) Version/3.0 Mobile/4A102 Safari/419 (United States)

You can then visit http://zinio.com/iphone/ and read 20 magazines each month for free.

It gets better though, it is very simple to download these pages onto your computer using Linux, aren't you glad you dumped windows ;)

We now need the URL to the image so "copy image location" from the right click menu then using this command:

curl -O -A "Mozilla/5.0 (iPhone; U; CPU like Mac OS X; en) AppleWebKit/420.1 (KHTML, like Gecko) Version/3.0 Mobile/4A93 Safari/419.3" INSERTURL

Replace INSERTURL with the copied link you got to page 1, then delete back to the underscore and append [1-300].jpg this is a range option to grab pages 1 to 300 and can be adjusted, so the final command will look like this:

curl -O -A "Mozilla/5.0 (iPhone; U; CPU like Mac OS X; en) AppleWebKit/420.1 (KHTML, like Gecko) Version/3.0 Mobile/4A93 Safari/419.3" http://imgs.zinio.com/iphone/issues/416078746/[1-300].jpg

The files will need to be renamed, as sadly they won't sort correctly if you are going to do the next step of combining them into a single pdf. We need the jpgs numbering from 001.jpg so they order correctly. this can be achieved in a variety of ways, using Thunar rename to insert the necessary 0's in two stages is one simple way, insert 00 at position 10 for number 1-9, and insert 0 into postion 10 for pages 10-99. it can also be scripted in a couple of steps with :

for i in `seq 1 9`; do mv *_$i.jpg 00$i.jpg ;done

for i in `seq 10 99`; do mv *_$i.jpg 0$i.jpg ;done

This could of course be scripted into one single command using a bash script.

Then we can simply issue the command:

convert *.jpg magazine.pdf

Thats it, you now have a local copy of the magazine in a handy PDF format relatively simply, it lokos more complex than it really is once you get your head around it.

Bash script to act as a thesaurus

I just stumbled across this excellent post at The Linux and Unix Menagerie which lists a brilliant little script to add a thesaurus command to your shell, this just perfectly rounds off my recent discovery of the dictionary word lookup commands!

This script is very small, in fact surprisingly so. Just be careful copying it into your text editor as the formatting isn't 100% and needs some tweaking, nothing major and no one should have any problems (yeah I know, fatal last words ;))

Use exiftran to transform jpeg images

Exiftran is a command line utility to transform digital image jpeg images. It can do lossless rotations like jpegtran, but unlike jpegtran it cares about the EXIF data: It can rotate images automatically by checking the exif orientation tag, it updates the exif informaton if needed (image dimension, orientation), it also rotates the exif thumbnail. It can process multiple images at once.

exiftran -ai *

This will transform all the images in a folder, rotating them to the correct orientation as defined by the exif data from the camera and replace the original, assuming your camera can sense and write the orientation into exif data.

exiftran -i9 *

Transform the image through 90degrees (to the right) and overwrite the original.

exiftran -i2

Transforms the image through 270degrees (to the left) and overwrite the original.

I have adapted this into two scripts in my nautilus-scripts folder "/home/subbass/.gnome2/nautilus-scripts/" you can make two scripts in this place using the following code and just changing the exiftran command to rotate left or right in each of the scripts.

#!/bin/sh
for arg; do
filetype=`file -i "$arg"`
if [ -n "`echo $filetype | grep -i '.jpg' `" ]; then

exiftran -i9 "$arg"

else
convert -rotate 90 "$arg" "$arg"
fi
done

The convert at the end is to convert other image formats without me requiring another set of scripts, feel free to omit this if desired. Always do a small test before setting any script to work on large numbers of images, I don't want to be held responsible if your exif data gets nuked.

rTorrent, light and fast bittorrent client.

I use two torrent clients almost every day, Transmission on my own box with a gnome desktop, and for a long time now rTorrent in a shell via SSH to a spare machine I have use of on a separate connection. Setting up rTorrent on an Ubuntu machine is what I am going to discuss here.

First step is to install the program, open a terminal and enter:

sudo apt-get install rtorrent

Or use Synaptic and search for 'rtorrent', install it. A basic config file is located /usr/share/doc/rtorrent/examples for you to tweak to your own liking, at its simplest you may want to change the download folder in it and the 'watch' folder. The watch folder is a place that rTorrent will monitor for *.torrent files coming into so it can automatically start, and when the download has completed and achieved a ratio you are happy with, you can just delete the torrent file from this folder to remove it from rTorrent. Dead simple stuff :]

cp /usr/share/doc/rtorrent/examples/rtorrent.rc ~/.rtorrent.rc

then edit this file with your prefered text editor, mine is usually nano, use gedit if you prefer a GUI editor:.

cd ~/
nano .rtorrent.rc

Now we can make the most basic of changes, first up is the download folder, look for the line:

# Default directory to save the downloaded torrents.

change the line after it to the folder you want to be your download folder and remove the # so you have something like this:

# Default directory to save the downloaded torrents.
directory = ~/torrents

This makes my download folder /torrents/ in my home, the default without editting is simply your home folder which you may be happy with.

Next lets tell it what folder to watch for torrent files. I have defined mine to be this same ~/torrents/ folder, you may prefer to leave it as your home folder. Either way find these lines just slightly down from the previous:

# Watch a directory for new torrents, and stop those that have been
# deleted.
schedule = watch_directory,5,5,load_start=~/torrents/*.torrent
schedule = untied_directory,5,5,stop_untied=

Yours will look very slightly different as I have shown how I changed mine to watch the ~/torrents/ folder. You should now save the file and close the editor, ctrl+o to save in nano if you used my prefered editor from above and don't know it, press enter to save wit hthe same name, then ctrl+x to close it.

You should now be free to start rTorrent with the command:

rtorrent

Not very impressive at first look is it, but download a torrent file to the watch folder and see rtorrent leap into action all by itself. You can adjust the upload and download speeds using the keys

a/s/dIncrease the upload throttle by 1/5/50 KB.
z/x/cDecrease the upload throttle by 1/5/50 KB.
A/S/DIncrease the download throttle by 1/5/50 KB.
Z/X/CDecrease the download throttle by 1/5/50 KB.

There are other keys to pause and resume torrents and many other facilities that you may require, for that I would point you at the User Guide page at http://libtorrent.rakshasa.no/wiki/RTorrentUserGuide

As I use this remotely I also run it inside a screen so I can safely detach from it and break the connection leaving it running on the remote machine, to do this enter in a terminal:

screen

then you will just see another command prompt on a clear terminal at which point enter:

rtorrent

rTorrent will start as normal but if you use the keys ctrl+a then d it will detach, that is it will drop you back at your original shell, leaving rtorrent running in the backgroun still. You can attach to it again with:

screen -r

You can then see rTorrent again to check your downloads or ratios. Check 'man screen' for more help on using screen, there are a *lot* of things you can do with it, and that might be a worthwhile post for me some time soon.

Recurse to find files and move them to a location

I just had to move a bunch of files out of individual sub-folders into the parent folder, time consuming to do that 50 or so times so a quick check of the "find" command and the solution is here:

find . -iname '*.avi' -exec mv {} /home/subbass \;

Command breakdown looks like this:

find .
find "here"
-iname '*.avi'
case insensitive name match on *.avi
-exec mv
execute the move command on the matches
{}
the match result from find
/home/subbass/
path to move the files to
\;
end of the -exec, each match runs as a new command

A quick command line, fairly simple syntax and it should prove a great time saver. You could easier of course cp or rm files instead of mv or many other possibilities.

Use convert to add a border to multiple images in the command line

Ok, so you you have taken a bunch of photos with your new camera and what do you know, some of them are half decent. I use this after I have copied photo's I want to print or photo's that I am sending to flickr or deviant art to a seperate folder as it really adds a finishing touch, I copy them to a seperate location so I know exactly which ones I am printing/uploading and don't omit any.

Now you could easily do this in The Gimp, load each image and enlarge the canvas followed by a fill and then resave the image. This is linux though and we can do things a heap simpler than that!
First install Imagemagick:

sudo apt-get install imagemagick

Or install it via Synaptic or your package manager.

Then use this simple script,

be aware that you should operate on copies as this does overwrite the original, as noted above I only do this on photo's I have copied for printing/uploading.

for img in `ls *.jpg`
do
convert -bordercolor white -border 50x50 $img $img
done

Easiest to pop it all into a bash script, or just close it up separating the commands with semicolons into a one liner eg.

for img in `ls *.jpg` ; do ; convert -bordercolor white -border 50x50 $img $img ; done

If your camera uses a different extension then adjust the .jpg accordingly.

Adjust "50x50" to whatever you feel adds a nicely sized border, and feel free to change the border colour (black for instance). If you want to get creative stack up the converts to add a thin black border before a larger white border or whatever you fancy.

Unpack multiple zip files easily

So you have just completed a download and it is in multiple zip files, rather than unpack them in the gui one at a time try this.

Open a terminal and 'cd' to the folder containing the zips, then enter this command:

for FILE in *.zip; do unzip -jo $FILE; done

I would suggest having the zip files in a separate folder to anything else to avoid confusion over the files that get unpacked.