Posts Tagged bash

sl: use this function all the time…

function sl() { if [ 1 = ${#} ] ; then less -i "${1}" ; else ls ${*} ; fi }

Leave a Comment

making m2 salad

Irrelevant Silliness

So… I’m a vim hold out. I still care deeply about emacs and it’s spawn being defeated once and for all.

Which means I don’t really like bloated IDE’s like Eclipse…. which makes me a pariah… which means people whisper and plot against me behind my back…

And… they’re always out to get me… and…

Ahem… sorry, about that… Like I was saying….

Which means I prefer to write a lot of tools and utilities to provide the sort of out-of-the-box functionality other people are perfectly happy to point and click thru with their brains in the neutral gear.

It’s part of a sickness I have… and I am going to confess it here on the internet: I love programming computers.

I know, I know… appalling! The gall! But there it is! My dirty little secret!

So what have I brewed up this time?

You know that functionality which lets you see right click and see methods, etc in Eclipse?

I thought it would be nice to have something like it for the CLI where I can use grep the way the Lawd intended. Here is the soup-to-nuts for nuts.

Finally he gets to the point

% m2_salad.sh 
m2_cp.txt is older than pom.xml... rebuilding m2_cp.txt
m2_soup.txt is older than m2_cp.txt... rebuilding m2_soup.txt
m2_salad.txt is older than m2_soup.txt... rebuilding m2_salad.txt

Zounds! So I wrote 2 new scripts: m2_soup.sh and m2_salad.sh.

Like m2_cp.sh, each produces a file named X.txt with some junk in it.

Here is the breakdown:

m2_cp.txt      colon separated list of the jars maven uses
m2_soup.txt    list of all the classes defined in the m2_cp.txt jars
m2_salad.txt   javap of all the classes listed in m2_soup.txt

These along with the standard UNIX CLI utils provides the same sort of handy lookup funtionality, where it belongs: in the CLI, not in the editor.

And?

Well… OK, maybe it could live in the editor too if it was slim-and-fast and minimalist…

I’ve really enjoyed the vim plugin supertab for tab completion.

Part of the challenge of integrating it with the output from the scripts is to make the completion be contextual… Which means a plugin would have to have some notion of the program semantics.

For example, it would need to know that a piece of text was a variable and what it’s type was so it could lookup the appropriate methods.

Not easy, but I’m a big believer in the 90/10 rule.

Alternatively… what other sort of useful development application could be written leveraging this info?

Good times… good times…

Leave a Comment

bash: string to integer conversion

Bash arrays are odd, globally scoped critters which can only be indexed by integer values… but the values don’t have to be contiguous so they are sort of like hash maps…

All you need is something to convert strings to integers, for example:

_map_string_to_integer() {
	echo ${*} | od -t u1 | sed 's,[^ ]*,,;s, ,,g;' | tr -d '\n'
	echo
}

And you can use it like this:

% fun[$(_map_string_to_integer this)]="is neat"
% echo ${fun[$(_map_string_to_integer this)]}
is neat

if that is of any use to anyone….

Leave a Comment

poll directory for changes in bash

Here is some quick bash to watch all the files under a directory and do something when one of them changes:

% directory_sum() { stat $( find ${*} -type f | grep -vw .swp ) | sha1sum  | awk '{print $1}' ; };
% directory_watcher() { local dir=${1} ; shift ; local delay=${1} ; shift ; local action=${*} ; local current="" ; while true ; do next=$( directory_sum ${dir} ) ; if [ "${current}" != "${next}" ] ; then echo modification: ${action} ; current=${next} ; fi ; % echo -n . ; sleep ${delay} ; done; };
% reset ; directory_watcher us 5 haxe compile.hxml

So… this has some trickz:

  • define functions in the shell which can then used like any other command
  • the syntax for jamming all that on one line is nutty!
  • using find with stat and sha1sum is a cheap (to write, not necessarily to run!) way to check the tree for mods
  • the shift command is a nice way to deal with arguments to functions

In the interests of sanity, I will bust that out somewhat for legibility:

directory_sum() { 
    stat $( find ${*} -type f | grep -vw .swp ) | sha1sum  | awk '{print $1}' 
}
directory_watcher() { 
    local dir=${1}
    shift
    local delay=${1}
    shift 
    local action=${*} 
    local current="" 
    while true ; do 
        next=$( directory_sum ${dir} ) 
        if [ "${current}" != "${next}" ] ; then 
                echo modification: ${action}
                ${action}
                 current=${next} 
        fi
        echo -n .
        sleep ${delay}
    done
}

If you want to use it as stands:

% wget http://brianin3d.googlepages.com/directory_watcher.sh
% source directory_watcher.sh
% directory_watcher 5 haxe compile.hxml

Once the file is source suddenly I can use these functions in other scripts, on the cli or whatever just like any other command. They are a nice middle-ground between copy-pasta and creating a different script for each discrete piece of functionality (that’s my definition of a function in case you are wondering).

Leave a Comment

Scripting with ANSI Color Codes

Oh the console! Love it or loath it, if you engage in computer actions you probably end up spending a lot of time looking at a console.

Of course not all consoles are created equal. Some consoles are really just the equivalent of

while line=nextLine() ; do print line ; end

Typically this is reserved to crappy IDE’s (yes, I know that’s redundant, I’m making a hah-hah).

Some deluded souls think cygwin is a real console, only to discover half-way into trying to actually get something done that it ends up going nutso and spewing text in the most unlikely of locations. Infrequently this can lead to a series of awkward questions in the ER.

Gnome’s terminal is pretty adequate, and of course you can’t go wrong with xterm or a host of other well written, robust terminals that oddly enuff seem to show up only under flavors of UNIX (yes, netBSD… err.. Mac counts too).

Why is writing a solid console implementation so difficult?

Because a console is not about printing lines of text. A terminal is more like a pixel-based display where ever pixel is a character + presentation information.

It is a highly sophisticated user interface. Unlike the farktarded point-and-click UI’s popular today for no good reason which use a maximum of say 8 buttons, a console typically has 108+ buttons.

Hah-hah, indeed. More importantly a console is really an interpreter.

No, I don’t mean the shell, I mean the console is an interpreter. Just like with shells we have bash, tcsh and crapsh, terminals come in flavors like VT100, VT110, VT220, 3270, 5270 and plethora of others.

Since a terminals has to accept all sorts of input, the language for controlling the terminal is a done via “control codes.” Typically a control code is just a string which is difficult to type.

For example: echo <ESC>[33mCheese<ESC>[0m

What is that <ESC>? Typically it is character 27. How do you type it? On a lot of consoles, you type it like this:

  1. hold down the control key
  2. press the ‘v’ key
  3. let go of the control and ‘v’ key
  4. press the escape key

Pretty neat, huh? That works for ANSI/VT100-based terminals. Not for all terminals. Luckily for us, that’s just about all we really care about.

Of course a real console application (eg: vim) cares about all terminals and you start to move away from writing control codes directly and towards using terminal apis like curses/ncurses PDQ.

For hacking purposes, VT100 terminal codes will do just fine.

I know.. what a load of historic irrelevant blather and so what and quit wasting my time! What is it good for?!

When was the last time you looked at a log file? Let me put it a different way, when did you stop looking at log files? Chances are you are watching a log file right now! I know I am!

What percentage of log output do you care about for a given task? 10%? 2%? Isn’t it a drag that it just blends in with the rest?

Of course you could grep it out, but then you lose the context… Or you might forget a term and have to reproduce the problem again (again [again]).

IMHO, a kick-aspirin use for VT100 color codes is for log highlighting, and it is some simple, it can be easily scripting into a general use script.

Here is the kind of usage I would like:

usage: cat foo | line_lite.sh ( pattern highlighting )+

pattern         is a ala grep
highlighting    is a colon separated list of colors

the colors are 

    reset        bright      dim         underline    blink
    reverse      hidden      black       red          green
    yellow       blue        magenta     cyan         white
    black_bg     red_bg      green_bg    yellow_bg    blue_bg
    magenta_bg   cyan_bg     white_bg

That way I can use it like: tail -f some.log | line_lite.sh Exception red “some other text” green:underline

In order to make this happen, I need to convert that red, green:underline crap to the appropriate VT100 code.

Natually, I do this with sed:

_line_lite_color_to_code() {
    echo ${*} | sed "s,reset,0,g;s,bright,1,g;s,dim,2,g;s,underline,4,g;s,blink,5,g;s,reverse,7,g;s,hidden,8,g;s,black,30,g;s,red,31,g;s,green,32,g;s,yellow,33,g;s,blue,34,g;s,magenta,35,g;s,cyan,36,g;s,white,37,g;s,black_bg,40,g;s,red_bg,41,g;s,green_bg,42,g;s,yellow_bg,43,g;s,blue_bg,44,g;s,magenta_bg,45,g;s,cyan_bg,46,g;s,white_bg,47,g;s,:,;,g"
}

_line_lite_code() {
    _line_lite_color_to_code ${*} | sed 's,.*,ESC[&m,'
}

This will convert “green:underline” to [32;4m

Now we just need some way to do our matching and add the color code. Once again, it’s sed time.

For each pair of pattern/color, we create a sed expression like: s!.*.*!&!g;

In this case we’d have s!.*some other text.*![32;4m&[0m!g;

All that we have to do is concatenate our sed expressions for each pattern/color pair and then call sed.

Viola! Arbitrarily colored logs!

Here is sample output from something that’s not a log:

% jar_minder_javap sun.security.provider.PolicyFile | line_lite.sh 'static'  red:bright
Compiled from "PolicyFile.java"
public class sun.security.provider.PolicyFile extends java.security.Policy{
    public sun.security.provider.PolicyFile();
    public sun.security.provider.PolicyFile(boolean);
    public void refresh();
    public boolean implies(java.security.ProtectionDomain, java.security.Permission);
    public java.security.PermissionCollection getPermissions(java.security.ProtectionDomain);
    public java.security.PermissionCollection getPermissions(java.security.CodeSource);
    protected java.security.cert.Certificate[] getSignerCertificates(java.security.CodeSource);
    static boolean access$002(sun.security.provider.PolicyFile, boolean);
    static boolean access$102(sun.security.provider.PolicyFile, boolean);
    static boolean access$202(sun.security.provider.PolicyFile, boolean);
    static boolean access$302(sun.security.provider.PolicyFile, boolean);
    static boolean access$200(sun.security.provider.PolicyFile);
    static sun.security.util.Debug access$400();
    static boolean access$500(sun.security.provider.PolicyFile, java.net.URL, sun.security.provider.PolicyInfo);
    static java.security.CodeSource access$600(sun.security.provider.PolicyFile, java.security.CodeSource, boolean);
    static java.lang.String access$700(sun.security.provider.PolicyFile, java.security.ProtectionDomain);
    static java.security.IdentityScope access$800();
    static {};
}

Sadly, my olde ascii2html script is not quite it… but I think you get the idea.

Go forth and dig them control codes, my friend! They exist only to serve your dread will.

Here is a link to a copy of the script just in case: http://brianin3d.googlepages.com/line_lite.sh


Dude has had this great link for a jillion years! http://www.fh-jena.de/~gmueller/Kurs_halle/esc_vt100.html, what a hero!

P.S.: Yes, I advocate EMACS-hateration. set -o vi or fight!

Comments (11)