Grep for a tab Character

Need to grep for a tab in a file, but don’t know how to do it. Here’s one plan, say I wanted to grep for the number 1 followed by a tab…

grep -E "1`echo -e '\t'`" file.txt

So I’m actually using echo to print the tab special character into my grep string before the shell passes it on to grep. If this makes sense to you it’s time to start worrying…

Free NEF to jpg Conversion

As in free software of course, but coincidentally it doesn’t cost you anything either. This conversion takes care of a few things,
1) raw -> jpg, the most basic aspect, we want to turn our nef files into jpg files
2) noise correction – as a function of iso, I haven’t played with the amount of noise correction yet, I just went with a function that starts at 100 at iso 100 and ends up at 1000 at iso 1600, which is the range of my camera, and according to the documentation of dcraw a normal range of noise corrections
3) it crops some pixels, my camera has some goofy edge effects, so I just cut them off
4) it preserves exif data, to me this is a pretty big one
5) it’s embarrassingly parallelized, i.e. it processes each image individually. And it will use one fewer cpu than your machine has. For instance I have an i7, with 8 cpus (after hyperthreading) so it uses 7, leaving me one to have a responsive computer during the time it’s processing images. This has the nice upside of significantly speeding up conversion, as I get approximately a 7x speedup, so processing a few hundred photos takes on the order of minutes instead of on the order of hours (1 hours 10 minutes -> 10 minutes).
6) white balance – I’m not sold on the white balance correction in dcraw, but I think it’s better than nothing. I sort of like that of the gimp, but I find that it sometimes introduces color casts. I think long term I will be moving towards camera white balance.

What it doesn’t do (yet)
1) geotagging – I think I’ve got this figured out, detailed a few posts back.
2) auto vignette correction by lens and focal length – seems like it should be pretty possible to do, I just haven’t done it yet. It would involve making a directory of averaged images at each focal length and then doing some sort of image magic compose divide. The biggest hassle would invariably be making the correction images.
3) chromatic aberration correction by lens and focal length. In a way easier, as one only needs to determine the multipliers for the different channels at different focal lengths and lenses. As opposed to having a collection of correction masks as for vignette correction. But finding the multipliers might be more difficult. I think it’s going to involve taking a picture of the sky with bars (buildings perhaps) on the edges of the frame. Essentially I’m looking for an instance that would create large chromatic aberrations. Then I can do a decompose in gimp and count pixels. Though I’m expecting the correction to be only 1-4 pixels over nearly 4000 pixels horizontal. Anyways, eventually I may try it and see what effect it has.

And finally the scripts:

#!/usr/bin/perl
#image_converter.pl
use strict;
my $numcpus = `grep -c processor /proc/cpuinfo`;
chomp($numcpus);
$numcpus -= 1; #save one processor for doing other stuffs
print "using $numcpus processors\n";
system("ls *.NEF *.nef 2>/dev/null|xargs -n 1 -P $numcpus rename 'y/A-Z/a-z/'");
system("ls *.nef|xargs -n 1 -P $numcpus chmod -x");
system("ls *.nef|xargs -n 1 -P $numcpus ~/bin/image_converter_inner.pl");
exit(0);

#!/usr/bin/perl
use strict;
my $numcpus = `grep -c processor /proc/cpuinfo`;
chomp($numcpus);
$numcpus -= 1; #save one processor for doing other stuffs
my $image = $ARGV[0];
$image =~ s/.nef//;
my $iso = `exiftool -ISO $image.nef|awk '{print \$3}'`;
chomp($iso);
my $waveletcorrect = log($iso/100)/log(2);
$waveletcorrect = 100*3.162^$waveletcorrect;
system("dcraw -t 0 -a -n $waveletcorrect $image.nef");
system("convert $image.ppm $image.jpg");
system("/bin/rm $image.ppm");
system("exiftool -q -overwrite_original -tagsFromFile $image.nef $image.jpg");
my $width = `identify -format "%[fx:w]" $image.jpg`;
my $height = `identify -format "%[fx:h]" $image.jpg`;
chomp($width);
chomp($height);
$width -= 4;
$height -= 4;
my $cmd = "convert -crop $width\\x$height+0+0 +repage $image.jpg $image.jpg.tmp; mv $image.jpg.tmp $image.jpg;";
system("$cmd");
exit(0);

Want time to behave like bash builtin

Try this: /usr/bin/time -f “real\t%es\nuser\t%Us\nsys\t%Ss”
You can find all the things that should be included in the man page but aren’t here.

Useful Linux Commands

List files not containing “string”. I know this one is simple, but I always end up using grep -c|grep “:0″|sed
grep -L string filelist

Linux list files only (or directories only)
So I had solved this problem multiple different ways, and always seemed to be foiled by different versions of ls, awk, grep, or whatever else I was using to solve what I think is a pretty simple problem. This is the simplest solution I’ve come up with, it’s great for piping output to whatever else you want.

#ldir list directories
alias ldir=’ls -p | egrep “/”‘
#lfo list files only
alias lfo=’ls -p | egrep -v “/”‘