Wednesday, November 30, 2011

nesting commands in xargs

cat links.txt | xargs -I {} sh -c "wget {} | tar czvf {}.tar.gz {}"

or write a for loop

or write xargs command in a separate script

cat links.txt | xargs -I {} myscript.sh {}

http://en.wikipedia.org/wiki/Xargs

http://www.andyd.net/2009/escaping-a-pipe-inside-xargs/


---getAndZip.sh----
URL=$1
FN=`basename $1`

echo URL = $1
echo FN = $FN

wget $URL
tar cvzf $FN.tar.gz $FN
rm $FN

1 comment:

Anonymous said...

cat links.txt | parallel wget {}\; tar czvf {/}.tar.gz {/}

You can install GNU Parallel simply by:

wget http://git.savannah.gnu.org/cgit/parallel.git/plain/src/parallel
chmod 755 parallel
cp parallel sem

Watch the intro videos for GNU Parallel to learn more:
https://www.youtube.com/playlist?list=PL284C9FF2488BC6D1