time - record time elapsed, memory

Some useful tools for figuring out how much memory an application is using.

ps aux

top

cat /proc/<PID>/status

alias time='/usr/bin/time -v'

/usr/bin/time -v ls

        Command being timed: "ls"
        User time (seconds): 0.00
        System time (seconds): 0.00
        Percent of CPU this job got: 0%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.00
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 3728
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 0
        Minor (reclaiming a frame) page faults: 277
        Voluntary context switches: 15
        Involuntary context switches: 1
        Swaps: 0
        File system inputs: 0
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0

Drew Dudley Everyday Leadership

http://www.ted.com/talks/drew_dudley_everyday_leadership.html?goback=%2Egde_4217370_member_5833707797237219331#%21

LabKey Software

http://medcitynews.com/2014/01/open-sourced-seattle-fated-collision-software-talent-life-science-researchers/#ixzz2rLZYJd7B

LabKey Server is open source software that helps scientists organize, analyze, and share biomedical research data.

Read more: http://medcitynews.com/2014/01/open-sourced-seattle-fated-collision-software-talent-life-science-researchers/#ixzz2rLvIbCgM

https://labkey.org/

Software carpentry

http://software-carpentry.org/v4/index.html

Who We Are

Our volunteers teach basic software skills to researchers in science, engineering, and medicine. Founded in 1998, we are now part of the Mozilla Science Lab.

What We Do

We run bootcamps all over the world, and provide open access material for self-paced instruction. We also run a training program for people who'd like to help us teach.

How To Help

Like all volunteer organizations, we depend on you to help us help others. You can host a bootcamp, help create new teaching materials, or improve the tools we use.

The gut–microbiome–brain connection

http://www.nature.com/nrn/journal/v15/n2/full/nrn3669.html?WT.ec_id=NRN-201402

Nature Reviews Neuroscience

 

15,

 

65

 

(2014)

 

doi:10.1038/nrn3669

Published online

 

27 December 2013

 

Studies in mice and human patients suggest that microbial imbalances in the gut influence various social, emotional and anxiety-like behaviours. Interestingly, gastrointestinal (GI) problems are common in autism spectrum disorders (ASDs), which are associated with social interaction deficits and stereotypical behaviours. A new study in Cell provides evidence that alterations in commensal gut bacteria contribute to certain behavioural abnormalities in a mouse model of ASD, highlighting the potential benefit of probiotic therapies for some of the features of such disorders.

1. Hsiao, E. Y. et al. Microbiota modulate behavioral and physiological abnormalities associated with neurodevelopmental disorders. Cellhttp://dx.doi.org/10.1016/j.cell.2013.11.024 (2013)

Taking R to the Limit (High Performance Computing in R)

http://www.slideshare.net/bytemining/r-hpc

bigmemory - it is ideal for problems involving the analysis in R for manageable subsets of the data, or when an analysis is conducted mostly in C++

the "big" family

biganalytics, synchronicity, bigtabulate, big.matrix, bigalgebra, bigvideo, shared.big.matrix, filebacked.big.matrix, bigsplit

linear models: biglm.big.matrix

mwhich

ff - "fast access files" - file-based access to datasets that cannot fit in memory

data.table

mapReduce - apply(map(data), reduce)

HadoopStreaming

SQL EXPLAIN to analyse and improve SQL queries

Database SQL query is slow?

You can use MySQL's EXPLAIN to analyse the query, maybe the column of the WHERE clause needs to be indexed?

EXPLAIN SELECT tt.TicketNumber, tt.TimeIn,
               tt.ProjectReference, tt.EstimatedShipDate,
               tt.ActualShipDate, tt.ClientID,
               tt.ServiceCodes, tt.RepetitiveID,
               tt.CurrentProcess, tt.CurrentDPPerson,
               tt.RecordVolume, tt.DPPrinted, et.COUNTRY,
               et_1.COUNTRY, do.CUSTNAME
        FROM tt, et, et AS et_1, do
        WHERE tt.SubmitTime IS NULL
          AND tt.ActualPC = et.EMPLOYID
          AND tt.AssignedPC = et_1.EMPLOYID
          AND tt.ClientID = do.CUSTNMBR;

http://dev.mysql.com/doc/refman/5.0/en/explain-output.html


mysql> alter table MY_TABLE add INDEX IDX_F1_F2 (F1,F2);
mysql> SHOW INDEX FROM MY_TABLE;

http://devzone.advantagedatabase.com/dz/webhelp/advantage9.1/how_indexes_are_used_by_the_where_clause.htm

http://www.experts-exchange.com/Database/MySQL/A_1250-3-Ways-to-Speed-Up-MySQL.html 

data.table: Extension of data.frame for fast indexing, fast ordered joins, fast assignment, fast grouping and list columns

http://cran.r-project.org/web/packages/data.table/index.html

Enhanced data.frame. Fast indexing, fast ordered joins, fast assignment by reference, fast grouping and list columns in a short and flexible syntax. i and j may be expressions of column names directly, for faster development. Example: X[Y] is a fast join for large data.

http://datatable.r-forge.r-project.org/ 

Science enters $1,000 genome era

http://www.bbc.co.uk/news/science-environment-25751958

The ability to sequence a human genome for just $1,000 has arrived, a US genetic technology company has announced.

The HiSeq X Ten high throughput genetic sequencing machine was announced at the annual JP Morgan Healthcare Conference in San Francisco this week. The company said it had broken the "sound barrier" of human genomics.

The Toxins of William B. Coley and the Treatment of Bone and Soft-Tissue Sarcomas

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1888599/

In 1891, William B. Coley injected streptococcal organisms into a patient with inoperable cancer. He thought that the infection he produced would have the side effect of shrinking the malignant tumor. He was successful, and this was one of the first examples of immunotherapy. Over the next forty years, as head of the Bone Tumor Service at Memorial Hospital in New York, Coley injected more than 1000 cancer patients with bacteria or bacterial products. These products became known as Coley's Toxins. He and other doctors who used them reported excellent results, especially in bone and soft-tissue sarcomas.

Despite his reported good results, Coley's Toxins came under a great deal of criticism because many doctors did not believe his results. This criticism, along with the development of radiation therapy and chemotherapy, caused Coley's Toxins to gradually disappear from use. However, the modern science of immunology has shown that Coley's principles were correct and that some cancers are sensitive to an enhanced immune system. Because research is very active in this field, William B. Coley, a bone sarcoma surgeon, deserves the title "Father of Immunotherapy."

A Field Guide to Genomics Research

http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1001744

High-throughput technologies are enabling scientists to profile genomes, transcriptomes, proteomes, and metabolomes at an unprecedented scale[1]. All this “-omics” research (let's call it genomics for simplicity) is exciting—and game changing—but it's also fraught with dangers for the tenderfoot. Here, we've put together a brief “field guide” for those wishing to visit the genomics frontier, in which we use caricatures to illustrate various pitfalls that can beset those who inhabit this new territory. By documenting the behaviors of these common types, we hope to guide researchers in their quest to apply sound practices when designing genomics experiments and analyzing the resulting data. Many of the tendencies we have encountered are not specific to genomics research, but they are particularly acute in this field due to its interdisciplinary nature and the complexity of the data it produces.

The Farmer

“Let's harvest a bunch of data and design fancy tools, and then we'll figure out what to do with them.”

...

Scientists tell the truth…

http://themetapicture.com/scientists-tell-the-truth/

JavaScript Code quality tool

http://www.jslint.com/lint.html

JSLint is a JavaScript program that looks for problems in JavaScript programs. It is a code quality tool.

When C was a young programming language, there were several common programming errors that were not caught by the primitive compilers, so an accessory program called lint was developed that would scan a source file, looking for problems.

As the language matured, the definition of the language was strengthened to eliminate some insecurities, and compilers got better at issuing warnings.lint is no longer needed.

IBM ThinkPad Laptop Keyboard not working

Problem: When you type "p" a "/" shows up, "U" becomes "4", etc.

Fix: Fn + ScrLk

http://en.kioskea.net/forum/affich-101274-laptop-keyboard-problem