Tract-Based Spatial Statistics (TBSS)

TBSS is a set of scripts in FSL to do a whole-brain voxelwise analysis of DTI data. There are three main components:

  1. Nonlinear registration of DTI volumes into template space (using fnirt).
  2. Local maximum intensity projection of DTI metrics onto white matter tract “skeleton”.
  3. Model fitting and inference (using randomise).

Processing instructions are available at: http://www.fmrib.ox.ac.uk/fsl/tbss

Processing on the UCLA LONI cluster

Here are some notes that I've collected about particular issues with running TBSS at LONI.

General information about the LONI cluster is available at http://www.loni.ucla.edu/twiki/bin/view/Infrastructure/GridComputing.

Memory

The default memory limit on the cluster is 2G. Some of the TBSS scripts often need more than that. To get around this limitation, I tweaked the fsl_sub submission script to take in an extra input argument -r to specify how much memory you want (up to 8G).

My version is at: /ifshome/jcolby/bin/fsl_sub

With this version, the usual TBSS commands look like this (note the new -r 8G argument):

fsl_sub -q short.q tbss_1_preproc *.nii.gz
tbss_2_reg -T
fsl_sub -r 8G tbss_3_postreg -S
fsl_sub -r 8G tbss_4_prestats 0.2

In general, you can tell if your jobs are failing due to hitting this soft 2G memory limit by running qacct -j <jobID>, where <jobID> is the SGE job ID of interest. Then look at the maxvmem row and see if it's around 2G. Here is the example output from a job that failed because of this:

>> qacct -j 9599700
==============================================================
qname        short.q             
hostname     ...
group        ...              
owner        ...              
project      NONE                
department   edevel              
jobname      matlab              
jobnumber    9599700             
taskid       undefined
account      sge                 
priority     12                  
qsub_time    Mon Mar 21 14:48:26 2011
start_time   Mon Mar 21 14:53:46 2011
end_time     Mon Mar 21 14:54:06 2011
granted_pe   NONE                
slots        1                   
failed       100 : assumedly after job
exit_status  137                 
ru_wallclock 20           
ru_utime     1.976        
ru_stime     0.317        
ru_maxrss    0                   
ru_ixrss     0                   
ru_ismrss    0                   
ru_idrss     0                   
ru_isrss     0                   
ru_minflt    33386               
ru_majflt    438                 
ru_nswap     0                   
ru_inblock   0                   
ru_oublock   0                   
ru_msgsnd    0                   
ru_msgrcv    0                   
ru_nsignals  0                   
ru_nvcsw     2709                
ru_nivcsw    203                 
cpu          2.690        
mem          3.431             
io           0.000             
iow          0.000             
maxvmem      2.062G
arid         undefined

NIfTI header errors

Some of the versions of FSL on LONI return errors when trying to read data that would otherwise need byte-swapping (ex: certain built-in standard files and some of our older data). The error looks like this:

/usr/local/fsl-4.1.0-1_64bit/bin/fslhd /usr/local/fsl-4.1.0-1_64bit/data/standard/tissuepriors/avg152T1_brain.nii.gz
** ERROR: nifti_convert_nhdr2nim: bad dim[0]
** ERROR (nifti_image_read): cannot create nifti image from header '/usr/local/fsl-4.1.0-1_64bit/data/standard/tissuepriors/avg152T1_brain.hdr'
** ERROR: nifti_image_open(/usr/local/fsl-4.1.0-1_64bit/data/standard/tissuepriors/avg152T1_brain): bad header info
Error: failed to open file /usr/local/fsl-4.1.0-1_64bit/data/standard/tissuepriors/avg152T1_brain
ERROR: Could not open file

Versions that show this are:

  • fsl-4.1.0-1_64bit

To work around this, the files must be opened/saved with another version of FSL that doesn't have this problem (ex: fsl-4.1.2_64bit, FSLView on your mac)

SGE Environment and job arrays

If a job array is submitted to the cluster through fsl_sub (ex: tbss_2_reg or randomise_parallel), the calling environment gets passed to the first bunch of jobs fine, but not for the remaining bunches that go through. Instead, a new bash shell is initiated. This means that if your normal login shell is tcsh, these jobs won't have the correct environment set up for FSL and will crash. I'm not sure if this is something to do with how the SGE is set up at LONI, or something with fsl_sub, itself.

Fortunately, the fix is easy. Just add the normal FSL environment setup commands for bash to your ~/.bash_profile or similar.

FSLDIR=/usr/local/fsl
. ${FSLDIR}/etc/fslconf/fsl.sh
PATH=${FSLDIR}/bin:${PATH}
export FSLDIR PATH

References

<bibtex> @article{smith_acquisition_2007,

title = {Acquisition and voxelwise analysis of multi-subject diffusion data with tract-based spatial statistics},
volume = {2},
url = {http://www.nature.com/nprot/journal/v2/n3/abs/nprot.2007.45.html},
doi = {10.1038/nprot.2007.45},
number = {3},
journal = {Nature Protocols},
author = {Stephen M Smith and Heidi {Johansen-Berg} and Mark Jenkinson and Daniel Rueckert and Thomas E Nichols and Karla L Miller and Matthew D Robson and Derek K Jones and Johannes C Klein and Andreas J Bartsch and Timothy E J Behrens},
month = jan,
year = {2007},
keywords = {Brain, Data Interpretation: Statistical, Diffusion, Magnetic Resonance Imaging, Models: Neurological, Water},
pages = {499-503}

},

@article{smith_tract-based_2006,

title = {Tract-based spatial statistics: voxelwise analysis of multi-subject diffusion data},
volume = {31},
url = {http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6WNP-4JS1MWF-D&_user=4423&_coverDate=07%252F15%252F2006&_rdoc=1&_fmt=&_orig=search&_sort=d&view=c&_acct=C000059605&_version=1&_urlVersion=0&_userid=4423&md5=7bc785f5e5a80b6da7199a245154b7b7},
doi = {10.1016/j.neuroimage.2006.02.024},
number = {4},
journal = {Neuroimage},
author = {Stephen M Smith and Mark Jenkinson and Heidi {Johansen-Berg} and Daniel Rueckert and Thomas E Nichols and Clare E Mackay and Kate E Watkins and Olga Ciccarelli and M Zaheer Cader and Paul M Matthews and Timothy E J Behrens},
month = jul,
year = {2006},
keywords = {Amyotrophic Lateral Sclerosis, {Computer-Assisted,} Diffusion Magnetic Resonance Imaging, Humans, Image Processing, Multiple Sclerosis, Nervous System Diseases, Neural Pathways, Nonlinear Dynamics, Normal Distribution, Reproducibility of Results, Schizophrenia},
pages = {1487-505}

} </bibtex>

neuroimaging/tbss.txt · Last modified: 2011/03/23 2:42 pm PDT by John Colby
 
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Noncommercial-Share Alike 4.0 International
Recent changes RSS feed Donate Powered by PHP Valid XHTML 1.0 Valid CSS Driven by DokuWiki