Percona Bash and GNU Tools

http://www.percona.com/blog/2015/03/17/mysql-qa-linux-upskill-bash-gnu-tools-scripting-fun/?mkt_tok=3RkMMJWWfF9wsRojuqnBZKXonjHpfsX%2F6O0oX6K3lMI%2F0ER3fOvrPUfGjI4CSMBjI%2BSLDwEYGJlv6SgFQrLNMadt3rgNWxI%3D

gnuwin32.sourceforge.net/

batch convert pictures

for i in `ls *.JPG *.jpg`; do convert -resize 50% $i resized_$i; done

Python, domain-domain interaction

Python code to take in two files, a pfam lookup table and a domain-domain interaction file.

#!/usr/bin/python

# loadin loopkup table
f = open("pfam-table.txt", "r")
pfam = {}
while True:
line = f.readline()
if (not line):
break
line = line.rstrip()
cols = line.split()
if (len(cols) < 2):
break
pfam[cols[0]] = cols[1]
f.close()

# load data to normalize
f = open("tmp2-clean.txt", "r")
dict = {}
while True:
line = f.readline()
if (not line):
break
line = line.rstrip()
cols = line.split()
if (len(cols) < 5):
break

d1 = pfam[cols[0]] #domain1
p1 = cols[1] #protein1
int = cols[2]
d2 = pfam[cols[3]]
p2 = cols[4]

key = p1+'-'+p2
key2 = p2+'-'+p1
if dict.has_key(key):
dict[key]=dict[key]+d1+'-'+d2+';'
elif dict.has_key(key2):
dict[key2]=dict[key2]+d1+'-'+d2+';'
else:
dict[key]=d1+'-'+d2+';'
f.close()

for e in dict.items():
srcdest = e[0].split('-')
print srcdest[0], srcdest[1], len(e[1].split(';'))-1, e[1]

web idea

Woke up today with this crazy idea of letting someone know secretly that you like them. Probably inspired by the movie `Let the Love Begin`. The concept of a secret admirer, secret saviour you name it, is very enticing. Now there's already something similar called `Send Secret Thoughts ♥` but some people complains that it shows up when you post something, so we'll try to address that issue if possible in our own implementation muwhahahaha.

expert finder, contact, location, fields, etc

Bioperl in action

Reads a FASTA file with multiple sequences (delimited by >) then print seq of length greater than 6000.

1 #!/usr/bin/perl
2 #
3 # print seq of length greater than 6000
4 #
5 #
6
7 use Bio::SeqIO;
8
9 unless ($#ARGV+1 == 1) { die "Need one argument\n"; }
10
11 my $infile = shift;
12
13 # !!!! use => not ->
14 my $file_obj = Bio::SeqIO->new(-file=>$infile,-formatr=>"Fasta");
15 my $cutoff = 6000;
16
17 while (my $seq_obj = $file_obj->next_seq() ) {
18 my $id = $seq_obj->id();
19 my $desc = $seq_obj->desc();
20 my $len = $seq_obj->length();
21
22 if ($len > $cutoff) {
23 # !!!!!!!!!!!! note: this prints reference
24 #print "$seq_obj->id()\t$seq_obj->desc()\n";
25
26 print "ID=$id\tDESC=$desc\tLEN=$len\n";
27 }
28 }
29
30

Imagemaps in HTML

Nifty trick to use image maps in HTML


Move the mouse over the image, and look at the status bar to see how the
coordinates change.

img src="planets.gif"
ismap width="146" height="126"

And

img src="planets.gif"
width="145" height="126"
usemap="#planetmap"

map id="planetmap" name="planetmap"

area shape="rect"
coords="0,0,82,126"
alt="Sun"
href="sun.htm"

Code from W3Schools

Web programming

Spent the last two days working on a site 20% writing code and another 80% uploading/refreshing/testing/googling, learned some more javascript/php/css along the way. Fun stuff. Had this crazy idea of doing a web-based sequence viewer too.

PyMol script
# show all surfaces with 50% transparency.
set transparency, 0.5
set ray_opaque_background, off
png ~/Videos/dna1.png, dpi=300

PHP DB
http://www.designdetector.com/archives/04/10/FlatFileDatabaseDemo.php
http://www.designdetector.com/archives/04/12/FlatFileDatabaseDemo2.php

IFrame Height Fix
http://guymal.com/mycode/100_percent_iframe/

XSLT
http://xslt.sitesfree.com/

CSS
http://www.webreference.com/programming/css_frames/



Now on to homework ...

Perl chop-sequence.pl

Perl code that takes a sequence and a read length (N) and outputs reads of size N that overlaps by two nucleotides.

Sample run
[ppt@bioinf tut09]$ ./chop-sequence.pl aacgtacgcttt 5 ; cat ./chopped-sequence.txt
aacgt
gtacg
cgctt
ttt

Code
1 #!/usr/bin/perl
2 #
3 # 1. Takes as input two parameters: (i) a DNA sequence and (ii) the size N of the chopped fragments.
4 # 2. Generates fragments of size N that are such that they overlap in two nucleotides with another fragment at its right side. Each fragment is writte n in a separate line in a file called chopped-sequence.txt.
5 # file: chop-sequences.pl
6
7 my $arg_count = $#ARGV + 1;
8 unless ($arg_count == 2) { die "Need two arguments $!\n"; }
9
10 my $out_file = "chopped-sequence.txt";
11 my $seq = shift;
12 my $chop_len = shift;
13 my $seq_len = length $seq;
14 my $overlap = 2;
15
16 open OUT, ">$out_file" or die "Can't create output file $out_file $!\n";
17
18 if ($seq_len <= $chop_len) {
19 #print "$seq\n";
20 print OUT "$seq\n";
21 close OUT;
22 exit;
23 }
24
25 for (my $i = 0; $i < $seq_len; $i += $chop_len - $overlap) {
26 $contig = substr $seq, $i, $chop_len;
27
28 # for this case, don't output tc on second line
29 # acctcacctcc 5
30 if ((length $contig) == $overlap) {
31 last;
32 }
33
34 #print "$contig $seq $i $chop_len $seq_len \n";
35 print OUT "$contig\n";
36
37 # handle boundary conditions, two case
38 # acctcc 5
39 if ( (length $contig) < $chop_len ) {
40 # reached the end
41 last;
42 }
43 }
44 close OUT;
45

Trivia:
Q. How do you copy-n-paste this code?
A. One way is to use a 1-liner awk script to filter out the first two columns.

url incrementer firefox extension

So I go watch stuff at Viikii.net and noticed that I kept on having to go back a page, click on the next part of the episode and do it all over again. So then I thought, since the video id seems to increase by 1 for each part (for the most part, not always the case), why don't I create a Firefox extension that will do this for me? This way, it'll save me from clicking back and remembering which part I just saw (since I have a very very short term memory).

Here's some links I used for my development:

• http://kb.mozillazine.org/Getting_started_with_extension_development
• https://developer.mozilla.org/en/Building_an_Extension
• http://ted.mielczarek.org/code/mozilla/extensionwiz/
• https://developer.mozilla.org/en/Working_with_windows_in_chrome_code#Accessing_content_documents
• http://www.viikii.net/videos/watch/2061
• http://www.viikii.net/viewer/viikiiplayer2.swf?video_id=994
• https://developer.mozilla.org/En/DOM/Window

I would recommend starting off with the extensionwiz link to create the stubs for you, then modify the icon.png and overlay.js to suit your needs. Took me about 1.5 days to learn it and get it up and running.

By the way, you put the link file in the profile folder that looks something like (Linux):
~/.mozilla/firefox/17jmqswm.default/extensions

Tried to upload the file, no luck and I'm too lazy to put it elsewhere :(

but here's the code

var newURL = content.document.URL;
var separator = "=";
var urlArray = newURL.split(separator);
var isFirstURL = (urlArray.length == 2) ? true : false;
var vidId = urlArray[urlArray.length-1];
var isFound = /^-?\d+$/.test(vidId);
if (isFound) {
++vidId;

// build the new URL, case 2
newURL = "";
for (var i = 0; i <>
newURL += urlArray[i] + separator;
}
newURL += vidId;
//promptService.alert(window, this.strings.getString("errorTitle"),"url="+newURL);

var strWindowFeatures = "toolbar=yes,menubar=yes,location=yes,resizable=yes,scrollbars=yes,status=yes,fullscreen=yes";
var windowObjectReference = window.open(newURL, "Viikii.net " + vidId, strWindowFeatures);
window.close();
} else {
promptService.alert(window, this.strings.getString("errorTitle"),
this.strings.getString("errorMessage") +
" '" + newURL + "' ");
}

mencoder stuff

# copy first 100 seconds
$ ffmpeg -i input.avi output.avi -t 100

http://forum.videohelp.com/topic329701.html

http://videotranscoding.wikispaces.com/mencoder

slower but with better quality and slightly better compression
$ mencoder ./in.MOV -o ./out.avi -oac pcm -ovc xvid -xvidencopts bitrate=740:par=pal43:qpel -vf scale=480:360,tfields -mc 0 -vf harddup

$ mencoder vid.wmv -o vid.avi -oac mp3lame -ovc xvid -xvidencopts bitrate=3000:pass=1 -vf scale=640:480

$ mencoder -ovc lavc -oac pcm -of lavf -lavfopts format=av ./input.MOV -o ./output.avi

Split
mencoder -ovc xvid -xvidencopts bitrate=740:par=pal43:qpel -vf scale=480:360,tfields -fps 50 -oac mp3lame -lameopts mode=1:cbr:aq=1:br=192 -mc 0 -vf harddup -ss 01:00:00 0 -endpos 02:00:00 -oac copy -ovc copy movie.avi -o second_half.avi

This will grab the segment between the first and second hour marks of the video.

-ss indicates where you want to start the encoding (1 hr from the start of the video 00:00:00).

-endpos indicates where you want to end (duration of 2 hours from -ss).


http://ubuntuliving.blogspot.com/2008/03/splitting-avi-file-into-smaller-parts.html


Convert
#!/bin/bash

#w=640; h=480; br=1600
w=320; h=240; br=800
#r="-vop mirror,rotate=x" [x=0..3]
#-ffourcc DX50 $r -o "$1".new "$1"

for i in 1 2
do
mencoder -vf scale=$w:$h -ofps 30000/1001 -channels 1 -srate 11025 -oac mp3lame -lameopts mode=3:abr:br=16 -ovc lavc -lavcopts vcodec=mpeg4:vhq:vbitrate=$br:vpass=$i -ffourcc DX50 -o "$1".new "$1"
done

Got the code from the link below and customized it a bit. Converted a 700mb wmv 640x480 file to 500mb mpg 320x480 with this script. Couldn't hear anything from the source wmv though.

http://marc.merlins.org/linux/technotes/mencoder_camera_encoding/README.html

RM to FLV
mencoder input.rm -o output.flv -of lavf -oac mp3lame -lameopts abr:br=56 -srate 22050 -ovc lavc -lavcopts vcodec=flv:vbitrate=500:mbd=2:mv0:trell:v4mv:cbp:last_pred=3


-----------

ext='avi'

mkdir flv_dir

for i in `ls *.$ext`
do
echo $i
ofile=`basename $i $ext`flv
mencoder -o $ofile "$i" -of lavf -oac mp3lame -lameopts abr:br=56 -srate 22050 -ovc lavc -lavcopts vcodec=flv:vbitrate=500:mbd=1:mv0:trell:v4mv:cbp:last_pred=3 -mc 0 -vf harddup > out.log 2> err.log
mv $ofile flv_dir
done

Beautiful Script

This is one beautiful code http://snipplr.com/view/5084/downloader-y-conversor-de-videos-de-youtube/

`cut` works like str.split(), -d is delimiter, -f is field number

#!/bin/bash

if [ $# -lt 1 ]; then
echo "Uso: $0 "
exit 1
fi

ID=`echo $1 | cut -d= -f2 | cut -d\& -f1`
FILE="youtube-${ID}"
BASE_URL="http://youtube.com/get_video.php"

wget -O /tmp/${FILE} $1

if [ $? == 0 ]; then
T_PARAM=`grep '&t=' /tmp/${FILE} | head -n 1 | awk -F'&t=' '{print $2}' | cut -d\& -f 1`
VIDEO_URL="${BASE_URL}?video_id=${ID}&t=${T_PARAM}"
wget -O ${FILE}.flv $VIDEO_URL

if [ $? != 0 ]; then
rm -f ${FILE}.flv
exit 1
else
echo "Formato (avi , mpg o wmv): "
read formato
ffmpeg -i ${FILE}.flv ${FILE}.$formato
fi
fi

rm -f /tmp/${FILE}