Posts Tagged scripting

making m2 salad

March 25, 2010 at 8:00 pm · Filed under java, scripting ·Tagged , , ,

Irrelevant Silliness

So… I’m a vim hold out. I still care deeply about emacs and it’s spawn being defeated once and for all.

Which means I don’t really like bloated IDE’s like Eclipse…. which makes me a pariah… which means people whisper and plot against me behind my back…

And… they’re always out to get me… and…

Ahem… sorry, about that… Like I was saying….

Which means I prefer to write a lot of tools and utilities to provide the sort of out-of-the-box functionality other people are perfectly happy to point and click thru with their brains in the neutral gear.

It’s part of a sickness I have… and I am going to confess it here on the internet: I love programming computers.

I know, I know… appalling! The gall! But there it is! My dirty little secret!

So what have I brewed up this time?

You know that functionality which lets you see right click and see methods, etc in Eclipse?

I thought it would be nice to have something like it for the CLI where I can use grep the way the Lawd intended. Here is the soup-to-nuts for nuts.

Finally he gets to the point

% m2_salad.sh 
m2_cp.txt is older than pom.xml... rebuilding m2_cp.txt
m2_soup.txt is older than m2_cp.txt... rebuilding m2_soup.txt
m2_salad.txt is older than m2_soup.txt... rebuilding m2_salad.txt

Zounds! So I wrote 2 new scripts: m2_soup.sh and m2_salad.sh.

Like m2_cp.sh, each produces a file named X.txt with some junk in it.

Here is the breakdown:

m2_cp.txt      colon separated list of the jars maven uses
m2_soup.txt    list of all the classes defined in the m2_cp.txt jars
m2_salad.txt   javap of all the classes listed in m2_soup.txt

These along with the standard UNIX CLI utils provides the same sort of handy lookup funtionality, where it belongs: in the CLI, not in the editor.

And?

Well… OK, maybe it could live in the editor too if it was slim-and-fast and minimalist…

I’ve really enjoyed the vim plugin supertab for tab completion.

Part of the challenge of integrating it with the output from the scripts is to make the completion be contextual… Which means a plugin would have to have some notion of the program semantics.

For example, it would need to know that a piece of text was a variable and what it’s type was so it could lookup the appropriate methods.

Not easy, but I’m a big believer in the 90/10 rule.

Alternatively… what other sort of useful development application could be written leveraging this info?

Good times… good times…

Leave a Comment

Print the name of any class in a jar that has a main method:

December 23, 2009 at 5:14 pm · Filed under java, scripting ·Tagged , , ,

Print the name of any class in a jar that has a main method:

jar="output/opennlp-tools-1.4.3.jar"
javap -classpath ${jar} $(     
    jar tf ${jar}      \
    | grep ass$        \
    | cut -f1 -d.      \
    | tr '[$/]' .      
) \ 
| tr '\n' '@'          \
| sed 's,}@,}=,g'      \
| tr '=' '\n'          \
| grep -w main         \
| tr '@' '\n'          \
| grep '^public class' \
| awk '{print $3}'

Leave a Comment

pull analytics “site search terms” with google data api

December 10, 2009 at 1:57 pm · Filed under scripting ·Tagged , , , ,

I’d heard about the google data api, but up until recently hadn’t found much practical use for it.

Suddenly I needed to pull back the top search terms for a site monitored by google analytics. The default view only shows 500 at a time. Granted this accounted for 30% of the “unique searches’, but I wanted to see a bit more.

Though it was sort of hard to find, I used the example script dataFeed.sh to retrieve the top 10k search terms.

The script wants you to set the variables for username/password and “PROFILE_ID” The value for this last is the id=xxxx in the url when you cruise around the analytics site.

Rather than read thru the quirky documentation as to what dimensions and metrics were available, I used the Data Feed Query Explorer to find the values instead:

feedUri="https://www.google.com/analytics/feeds/data\
?start-date=2009-08-01\
&end-date=2009-12-04\
&dimensions=ga:searchKeyword\
&metrics=ga:searchDepth,ga:searchDuration,ga:searchExits,ga:searchRefinements,ga:searchUniques,ga:searchVisits\
&sort=-ga:searchUniques\
&max-results=40000\
&ids=ga:$PROFILE_ID\
&prettyprint=true"

curl $feedUri -s --header "Authorization: GoogleLogin $googleAuth"

Despite the max-results of 40k, it looks like there is a max of 10k, which is fine.

All in all, pretty ez. Digging it all up.. sort of sucked.

Hence the write up… hope it helps some buddy!

Leave a Comment

4 useful shell scripts for maven

November 2, 2009 at 8:22 pm · Filed under java, scripting ·Tagged ,

yeah maven

Maven (2 of course) is the jam when it comes to Java build systems.

Ant is some old timey donkey-mess madness which doesn’t even address the issue of dynamic build dependency resolution.

There are other build systems like: ivy, maven1, and made-up-mess, but the real goodness is the m2 universe.

In addition, to it’s main duties, maven produces a really neat resource in the form of your local repository (mine lives in ${HOME}/.m2/repository).

The dependencies your project pulls in also pull in dependencies and some of them are cool things you didn’t even know you couldn’t live without!

Here are a few scripts I use to leverage a little bit of that powerful force for funtimes!

m2_cp.sh : maven classpath

Maven compiles all your code and runs all your test cases. To do that it has to use an olde timey classpath just like duh, but …. how can you get at it when all you see is your pom.xml?

You can use the olde school trick: mvn -X test -Dmaven.test.skip=true ; and scrap it out of there

Or… you can use my little m2_cp.sh and it will parse it out for you into a file called “m2_cp.txt” that you can use to play more reindeer games.

Like what?

Like packaging up some tarball+script to run your stuff, or some uberJar (though the assembly plugin is a better solution for that), or maybe some…

m2_javap.sh : all your class are… ok…

Almost every Java developer uses an IDE. They pretty much all use eclipse.

I’m in that oddball 0.01% that just uses eclipse as a remote debugger… so… I use a lot of command line tools instead, for example: javap

The javap command will print the method signatures given a list of class names. But… you have to feed it a classpath.

In tricky eclipse, this happens like magic! Neat! But… as a magician, I want to be the magical man… so…

When I’m working on say, a portlet project (from maven-archetype-portlet), and I’m wanting to see some good times, I type: m2_javap.sh javax.portlet.GenericPortlet

Even that too much bother? How ’bout: m2_javap.sh GenericPortlet

m2_latest.sh : to the greatest

Ever use log4j in a project? Ha! Yeah… so now you have a new project and you want to pull it in again… what as the most recent version: m2_latest.sh log4j ; tells you the jar

m2_dep.sh : jar file to dependency

OK, so you need log4j, and the latest is ${HOME}/.m2/repository/log4j/log4j/1.2.14/log4j-1.2.14.jar ; time to start typing that annoying stanza… but wait:

m2_latest.sh log4j | m2_dep.sh 

% m2_latest.sh log4j | m2_dep.sh

<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.14</version>
</dependency>

Script it up

The local repo is really neato! Perfect scripting fodder! Good times…

Leave a Comment

scripting essentials: fullpath

October 30, 2009 at 2:48 pm · Filed under scripting ·Tagged ,

One of the things you end up doing in cli world a lot is jacking around with files.

Usually you are doing some junk with files in some (collection of) terminal(s) with a given working directory and referring to files with a local path eg: src/main/webapp/WEB-INF/web.xml

Then… for some reason, you want to manipulate them from another context, maybe a browser or some shells doing stuff in another working directory. In some cases you could change working directories, but is that really what you want?

Kind of breaks the flow, doesn’t it? “Now let me hop over here…”

Why are you hopping? You want to grep src/main/webapp/WEB-INF/web.xml? Just use the fullpath name, eg: /home/jameson/src/devilish/src/main/webapp/WEB-INF/web.xml !

But how? EZ: echo ${PWD}/src/main/webapp/WEB-INF/web.xml

By jamming around the fullname instead of hopping around the filesystem like a bunny rabbit, you will be better able to maintain the separation between the different contexts of your activities.

Ya dig it? If not, try this approach out for a while and see if you can feel the difference.

To make this easier, I prefer to use a bash function like this:

fullpath() { 
    local f
    for f in ${*}; do
        ( 
		 	( 
			 	cd ${f} && echo ${PWD} 
			) || ( 
			 	cd $(dirname ${f} ) && echo ${PWD}/$( basename ${f} ) 
			) 
		) 2> /dev/null
    done
}

This takes care of some annoying corner cases that I can’t remember at the moment… but feel free to shorten it

This tip says: move the world around your mind, not your mind around the world.

Leave a Comment

Older Posts »
Follow

Get every new post delivered to your Inbox.