http://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/api.php?action=feedcontributions&user=JacobNewman&feedformat=atomBanghamLab - User contributions [en]2024-03-28T22:31:20ZUser contributionsMediaWiki 1.35.5http://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Software&diff=6942Software2016-08-24T10:58:31Z<p>JacobNewman: </p>
<hr />
<div>__TOC__<br />
<!-- ===[[BanghamLabSVN|<span style="color: Navy">Copy of BanghamLab source code</span>]]=== --><br />
<br />
<span style="color: DarkGreen">'''Current activity: a collaboration''' with the [http://rico-coen.jic.ac.uk/index.php/Main_Page CoenLab] with the aim of understanding how patterns of gene activity in biological organs influence the developing shape. The BanghamLab is focussed on the conceptual underpinning: concepts captured in computational growth models, experimental data visualisation and analysis.</span><br />
<br />
===<span style="color:DarkGreen;">Notes on documenting our software</span>=== <br />
<br />
[[Tricks for documenting software|Notes for Lab members on how to contribute to this Wiki and where to put downloads. ]]<br />
<br>Matlab tip: searching a large data structure for a particular field. Clear the command window. Evaluate the structure to list all the fields, then use the usual control-f search tool on the command window.<br />
<br />
=<span style="color:DarkGreen;">'''Computational biology'''</span>=<br />
==<span style="color:DarkGreen;">Quantitative understanding of growing shapes: '''GFtbox'''</span>==<br />
====<span style="color:#A52A2A;">We developed ''GFtbox'' to allow us to model the growth of complex shapes with the ultimate goal: to understand the relationships between genes, growth and form.</span>====<br />
{| border="0" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="400"| [[Image:Journal.pbio.1000537.g009.png|350px|Growth of a flower]] Example of a growing snapdragon flower and some mutants ( [http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1000537 Green et al 2011]). Growth is specified by factors (genes) according to the Growing Polarised Tissue Framework. Colours represent putative gene activity, arrows the polariser gradient and spots clones.<br />
|<wikiflv width="300" height="313" logo="false" loop="true" background="white" position="right" autoplay='true'>GPT_Snapdragon_2010_Green_et_al-0002_1.flv|GPT_Snapdragon_2010_Green_et_al-0002.png</wikiflv><br />
<!--|<wikiflv width="280" height="300" logo="false" loop="true" background="white">Journal.pbio.1000537.s025_1.flv|Journal.pbio.1000537.g009.png</wikiflv>--><br />
|}<br />
<br />
{| border="0" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="10%"| <imgicon>GPT_thumbnail2.png|120px|GFtbox</imgicon> <br />
|width="40%"|<br />
For modelling the growth of shapes. <br><br><br />
<br />
[[Ready Reference Manual|'''''Ready Reference''''' Manual]]<br><br><br />
(PC, Mac, Linux, uses Matlab<br>no Mathworks toolboxes needed<br>[http://www.mathworks.com/products/matlab/tryit.html Matlab 30 day free trial] and <br>[http://www.mathworks.com/academia/student_version/?s_cid=global_nav student edition])<br><br><br />
<br />
<br />
[[Ready Reference Manual ancillary functions|'''''Ready Reference'for ancillary functions: in '''interaction function''' and external functions'''' Manual]]<br><br><br />
<br />
[[GFtbox|'''What? How? Where?''']]<span style="color: Green"> Background</span><br><br><br />
[[GFtbox Tutorial pages|'''''Tutorials''''': from the beginning]]<span style="color: Green"> Start here</span><br><br><br />
[[GFtbox Example pages|'''''Examples''''': from publications]]<br><br><br />
<br />
[https://sourceforge.net/p/gftbox/ <span style="color: Gray">'''''Download GFTbox''''' from SourceForge</span>]<br><br><br />
<span style="color: Gray">'''''Download GFTbox project files:''''' </span><br><br><br />
<br />
[http://cmpdartsvr1.cmp.uea.ac.uk/downloads/software/OpenSourceDownload_Dev_Eldridge_2016/Fruit_Shape_Brassicaceae.zip <span style="color: Gray">'''''Fruits''''' Eldridge et al 2016</span>]<br><br><br />
<br />
[http://cmpdartsvr1.cmp.uea.ac.uk/downloads/software/OpenSourceDownload_PLoS_SauretGueto_2013/GPT_Petal_PLoS_20130502.zip <span style="color: Gray">'''''Petals''''' Sauret-Güeto et al 2013</span>]<br><br><br />
<br />
[http://cmpdartsvr1.cmp.uea.ac.uk/downloads/software/OpenSourceDownload_Science_Paper_2012/GPT_ArabidopsisLeafModel_20120207.zip <span style="color: Gray">'''''Leaves''''' Kuchen et al 2012</span>]<br><br><br />
<br />
[http://cmpdartsvr1.cmp.uea.ac.uk/downloads/software/OpenSourceDownload_PLoS_Kennaway_2011/Kennaway-etal-2011.zip <span style="color: Gray">'''''Principles and concepts''''' Kennaway et al 2011</span>]<br><br><br />
<br />
[http://cmpdartsvr1.cmp.uea.ac.uk/downloads/software/OpenSourceDownload_PLoS_Green_2011/Green-etal-2010.zip <span style="color: Gray">'''''Snapdragon''''' Green et al 2011, Cui et al 2010</span>]<br><br><br />
<br />
|width="50%"| <span style="color:#AF002A;"> ''GFtbox'' is an implementation of the Growing Polarised Tissue Framework (GPT-framework) for understanding and modelling the relationship between gene activity and the growth of shapes such leaves, flowers and animal embryos ([http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1002071 Kennaway et al 2011]). </span><br><br><br />
<br />
<span style="color:#AF002A;">The GPT-framework was used to capture an understanding of (to model) the growing petal ([http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1001550 Sauret-Güeto et al 2013]), leaf ([http://www.sciencemag.org/content/335/6072/1092.abstract Kuchen et al 2012]) and Snapdragon flower [http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1000537 Green et al 2011]. The Snapdragon model was validated by comparing the results with other mutant and transgenic flowers [http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1000538 Cui et al 2010.]</span><br><br><br />
<br />
The key point is how '''outgrowths can be specified by genes'''. The icon shows an asymmetrical outgrowth. Conceptually, it is specifed by two independent patterns under genetic control: a pattern of growth and a pattern of organisers. The outgrowth arises from a region of extra overall growth. Growth is aligned along axes set by two interacting systems. Organisers at the ends of the mesh create a lengthwise gradient. This gradient interacts with the second due to putative organisers that generate polariser sinks in the region that becomes the tips of the palette outgrowth. ([http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1002071 Kennaway et al 2011]). These hypotheses need to be tested in biological systems.<br />
|}<br />
<br />
==<span style="color:DarkGreen;">Viewing and measuring volume images: '''VolViewer'''==<br />
{| border="0" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="10%"| <imgicon>VolViewer-logo.png|120px|VolViewer</imgicon> <br />
|width="40%"|For viewing and measuring '''volume images''' on both normal and '''stereo''' screens. Typical images from: confocal microscope and Optical Projection Tomography (OPT) images<br><br><br />
[[VolViewer#Description|'''What? How? Where?''']]<br><br><br />
[[VolViewer#User Documentation|'''''Tutorials''''': from the beginning]]<br><br><br />
[[VolViewer#Download| '''''Download''''']]<br><br><br />
(Windows, Mac, Linux)<br><br><br />
Output from VolViewer has appeared in:<br><br />
<br />
[http://www.cell.com/cell_picture_show-plantbio Cell: Online Gallery] | [http://www.amazon.co.uk/Handbook-Plant-Science-Keith-Roberts/dp/0470057238/ref=sr_1_19?s=books&ie=UTF8&qid=1289321357&sr=1-19 Front cover: Handbook of Plant Science] | [http://www.plantcell.org/content/18/9.toc Front cover: The Plant Cell] | [http://www.americanscientist.org/issues/pub/2013/1/3d-carnivorous-plants American Scientist] | [http://www.rms.org.uk/Resources/Royal%20Microscopical%20Society/infocus/Edgar%20article.pdf Royal Microscopical Society: Infocus Magazine] | [http://www.bioptonics.com/Home.htm Bundled with the Bioptonic 3001 scanner: Bioptonics Viewer] | [http://www.dailymail.co.uk/sciencetech/article-2215052/The-complexity-intricacy-Mother-Nature-revealed-incredible-pictures-plants--seen-inside.html The Daily Mail] | [http://www.guardian.co.uk/science/gallery/2007/sep/04/fruitflybrain#/?picture=330675671&index=1 The Guardian newspaper: 3D Fruit fly] | [http://qt.nokia.com/qt-in-use/ambassadors/project?id=a0F20000006LZ2pEAG Qt Ambassador program] | [http://www.triffidnurseries.co.uk/special3.php Triffid Nurseries website]<br />
<br />
<br><br><br />
|width="50%"| VolViewer is used as a '''stand-alone''' app. or as a '''viewport for other systems''', e.g. Matlab programs. VolViewer uses [http://www.opengl.org/ OpenGL] and [http://qt.nokia.com/products/ Qt] to provide a user friendly application to interactively explore and quantify multi-dimensional biological images. It has been successfully used in our lab to explore and quantify confocal microscopy and optical projection tomography images. Written by Jerome Avondo it is open-source and is also compatible with the Open Microscopy Environment ([http://openmicroscopy.org/site OME]) (Chris Allen and Avondo, et. al. ''OMERO: flexible, model-driven data management for experimental biology'' Nature Methods 9, 245–253 (2012))<br> [[image:Silique.PNG|360px]]).<br />
|}<br />
<br />
==<span style="color:DarkGreen;">Analysing shapes in 2D and 3D: '''AAMToolbox'''==<br />
{| border="0" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="10%"| <imgicon>AAMToolbox_logo.jpg|120px|AAMToolbox</imgicon> <br />
|width="40%"|For analysing populations of shapes and colours within the shapes using principal component analysis. <br><br><br />
[[AAMToolbox Details|'''What? How? Where?''']]<br><br><br />
[[Tutorials on the Shape modelling toolbox|'''''Tutorials''''': from the beginning]]<br><br><br />
<br />
[[AAMToolbox Download|<span style="color: Gray">'''''Download revised Nov2012''''' </span>]]<br><br><br />
<br />
<br />
(PC, Mac, Linux, uses Matlab<br>no Mathworks toolboxes needed<br>[http://www.mathworks.com/products/matlab/tryit.html Matlab 30 day free trial] and <br>[http://www.mathworks.com/academia/student_version/?s_cid=global_nav student edition])<br><br><br />
|width="50%"| The AAMToolbox enables the user analyse the shape and colour of collections of similar objects. Originally developed to analyse face shapes for lipreading ([http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=982900 Matthews ''et al''. 2002][http://www2.cmp.uea.ac.uk/~sjc/matthews-pami-01.pdf version of pdf]), we have used it extensively for analysing the shapes of leaves ([http://www.pnas.org/content/102/29/10221.short Langlade ''et al'' 2005.],[http://www.tandfonline.com/doi/abs/10.2976/1.2836738 Bensmihen ''et al.'' 2010]) and petals ([http://www.sciencemag.org/content/313/5789/963.short Whibley ''et al'' 2006],[http://www.mssaleshops.info/content/21/10/2999.short Feng ''et al''. 2010]). The analysis can be applied to art, for example, finding systematic differences between portraits by Rembrandt and Modigliani.<br />
|}<br />
==<span style="color:DarkGreen;">Analysing the shapes of clones: '''SectorAnalysisToolbox'''==<br />
{| border="0" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="10%"| <imgicon>Sector analysis icon.jpg|120px|SectorAnalysisToolbox</imgicon> <br />
|width="40%"|For analysing the shapes of marked cell clones. <br><br><br />
[[SectorAnalysisToolbox Details|'''What? How? Where?''']]<br><br><br />
[[SectorAnalysisToolbox Documentation|'''''Tutorials''''': from the beginning]]<br><br><br />
[http://cmpdartsvr1.cmp.uea.ac.uk/downloads/software/OpenSourceDownload_Science_Paper_2012/SectorAnalysisToolbox.zip <span style="color: Gray">'''''Download''''' </span>]<br><br><br />
(PC, Mac, Linux, uses Matlab<br>no Mathworks toolboxes needed<br>[http://www.mathworks.com/products/matlab/tryit.html Matlab 30 day free trial] and <br>[http://www.mathworks.com/academia/student_version/?s_cid=global_nav student edition])<br><br><br />
|width="50%"| The SectorAnalysisToolbox enables the user analyse the shapes of marked clones in a sheet of tissue.<br />
|}<br />
<br />
=<span style="color:Navy;">'''Algorithms'''=<br />
==<span style="color:Chocolate;">MSERs, extrema, connected-set filters and sieves==<br />
====<span style="color:Chocolate;">The algorithm finding MSER's starts with a connected-set opening or 'o' sieve</span>====<br />
[[Comparison of Matlab MSER's and 'o' sieve|Comparison of Matlab MSER's and 'o' sieve]] Essentially, no difference.<br />
{| border="0" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="50%"| [[Image:Cameraman_iso_topview.jpg|300px|link=AAMToolbox Details|MSERs]] Cameraman image. Superimposed red spots are maximal extrema and blue spots are minima. Irregular cyan, blue and yellow regions illustrate regions associated with maxima and the magenta region is a minimum.<br />
|[[Image:cameraman_iso_tree.jpg|300px|link=AAMToolbox Details|MSERs over scale-space]]<br>Isometric view of the cameraman image with superimposed maxima (red) and minima (blue). The trees trace the maxima through increasing scale-space. Large spots have been identified as stable extrema.<br />
|}<br />
<br />
===<span style="color: #C364C5;">Finding interest points, features and segmenting images. </span>===<br />
#[[MSER and Sieve Details|<span style="color:Chocolate;">'''Technical briefing'''</span>]] <span style="color: #C364C5;">MSER's incorporate 'o' sieves.<br />
#[[MSER's and Connected sets|<span style="color:Chocolate;">'''The twist''': from restricted median filters to sieves and MSER's</span>]]<br />
#One dimensional sieves (measure length)<br />
##[[Types of 1D sieve|Types of 1D sieve]]<br />
##[[First applied to hydrophobicity plots|First applied to hydrophobicity plots]] but lets exploit their idempotency.<br />
#Two dimensional sieves (measure areas)<br />
##Properties<br />
##Relation to MSER's<br />
#Three dimensional sieves (measure volumes)<br />
##[[Segment by volume|Segment by volume]] instant gratification.<br />
<!--[[siv Download|<span style="color: Gray">'''''Download''''' </span>]]<br><br>--><br />
<br />
<!-- [[Software#MSERs extrema connected-set filters and sieves|<span style="color:Green;">'''MORE'''</span>]] --><br />
<br />
===<span style="color:Chocolate;">Art, extrema of light and shade: '''''PhotoArtMaster'''''===<br />
<br />
<br />
Art created using ArtMaster, and ArtMaster itself was featured in an exhibit at the London Victoria and Albert (V&A) Museum exhibition <span style="color:Chocolate;">'Cheating? How to make Perfect Work of Art' (2003).</span> The exhibition centered on the idea of [http://en.wikipedia.org/wiki/Hockney%E2%80%93Falco_thesis Hockney's] that advances in realism and accuracy in the history of Western art since the Renaissance were primarily the result of optical aids such as the camera obscura, camera lucida, and curved mirrors. My exhibit used a touch screen (rare in those days) and ArtMaster to help visitors create 'paintings' from photographs. [http://www.sciencemuseum.org.uk/visitmuseum/galleries/turing.aspx finding its name]. (It is entirely different in principle from the software more recently used by Hockney to paint with an iPad.)<br><br />
<br />
{| border="0" width=100% style="background-color:#ffffff;"<br />
|-<br />
|align="center"|[[Image:DegasLightAndShade.jpg|400px]][[Image:Emma_face_Art_C.jpg|300px]]<br />
<br><br><br />
Illustration of PhotoArtMaster used to find and 'paint' with regions of light and shade crisply segmented from a photograph. Likewise, on the right, edges.<br />
|}<br />
=====PhotoArtMaster=====<br />
Saturday 07/06/2014: Inspired Photographer of the Year 2013 Tony Bennett when asked whether his photograph [http://www.bbc.co.uk/programmes/galleries/p020hd8s Mists and Reflections] had been Photoshopped replied something like <br><br />
"A digital camera delivers an unemotional ''raw'' image of pixels that you have to manipulate to ''create your photograph''" Photographers manipulate as little as possible. <br><br />
However there is '''another path one that creates pictures'''. For this you need another piece of software: PhotoArtMaster (ArtMaster). Professional Photographer said ''"'''Forget any comparison whatsoever with the art filters in Photoshop - this software reaches out and enters different stratospheres'''"'' [[Professional Photographer]].<br><br><br />
Early versions of PhotoArtMaster are still '''available from Amazon''' at low prices (I'm not sure where they come from.)<br />
[http://www.amazon.co.uk/s/ref=nb_sb_noss?url=search-alias%3Daps&field-keywords=photoartmaster] . Some help for both the early versions and the latest version can be found in [http://cmpdartsvr1.cmp.uea.ac.uk/downloads/software/SieveWebPages/a4a_2_screensize.pdf <span style="color: Chocolate">''''this document''''' </span>]).<br />
=====Links to third party PhotoArtMastered pictures=====<br />
*[https://picasaweb.google.com/113257474829608374943/InTheStyleOf Oliver Bangham] Colouful rounded shapes from, yes, my brother.<br />
*[http://en.wikipedia.org/wiki/Wimbledon_%28film%29 The entry sequence of the comedy film 'Wimbledon']<br />
<br />
====The final version of the Windows ArtMaster2.0 [http://cmpdartsvr1.cmp.uea.ac.uk/downloads/software/OpenSource_ArtMaster/ArtMaster2.0Release.zip <span style="color: #B31B1B">'''''is downloadable here'''''</span>] with no support.====<br />
Unzip into (for example) the "Program Files" directory then set your system environment to include: ''C:\Program Files\Pam2.0 Release\jre\bin;C:\Program Files\Pam2.0 Release\bin;'' (You may need help for this. I right clicked 'computer' from the 'Start' menu, then selected 'Advanced system settings', then 'Environment Variables' and finally slid through the System variables until I found and selected 'Path'. This allowed me to edit the path by adding ';C:\Program Files\Pam2.0 Release\jre\bin;C:\Program Files\Pam2.0 Release\bin' to the end). Rather detailed help using the software is available in [http://cmpdartsvr1.cmp.uea.ac.uk/downloads/software/SieveWebPages/a4a_2_screensize.pdf <span style="color: Chocolate">''''this essay''''' </span>].<br><br><br />
[http://cmpdartsvr1.cmp.uea.ac.uk/downloads/software/SieveWebPages/Art_For_All_a4a_3_web2.pdf <span style="color: Chocolate">'''''The sieve algorithms underpinning PhotoArtMaster software''''' </span>] are described in an extract of the <br />
[http://cmpdartsvr1.cmp.uea.ac.uk/downloads/software/SieveWebPages/a4a_2_screensize.pdf <span style="color: Chocolate">''''essay''''' </span>]. These documents were written to support our Fo2Pix company. PhotoArtMaster originally sold >65,000 licences but ill health forced the closure of Fo2Pix.<br />
<br />
{| border="0" width=100% style="background-color:#ffffff;"<br />
|-<br />
|align="center"|[[Image:Trees_1.jpg|600px]]<br />
<br><br><br />
Illustration of PhotoArtMaster used to find and 'paint' with regions of colour crisply segmented from a photograph. <br />
|}<br />
<br />
<!-- [[Documentation_of_Connected_Set_Filters_or_Sieves|Art test page]]<br><br> <br />
[[Documentation_of_Connected_Set_Filters_or_Sieves]] --><br />
<br />
==<span style="color:Navy;">Reaction-diffusion and morphogenesis==<br />
{| border="0" width=100% style="background-color:#000000;"<br />
|-<br />
|align="center"|<br />
[[Image:tentacles_morphogenesis.png|600px]]<br />
|}<br />
Illustration of morphogenesis inspired by Turing's paper. <br><br><br />
[http://cmpdartsvr1.cmp.uea.ac.uk/downloads/software/GPT_ReactionDiffusionTentacles_20121211.zip <span style="color: Navy">Example using growth toolbox GPT_ReactionDiffusionTentacles_20121211.zip</span>]<br />
<br><br><br />
===1 A===<br />
<br />
=<span style="color:DarkGreen;">Open source systems to which we have contributed=<br />
==<span style="color:DarkGreen;">OMERO==<br />
{| border="0" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="10%"| <imgicon>OMERO_DIAGRAM.jpg|100px|OMERO</imgicon><br />
|width="40%"|For working with the OME image database. <br><br>See [http://www.openmicroscopy.org/site/products/omero ''Details''], [http://www.openmicroscopy.org/site/support/omero4/downloads ''Download'']<br> [http://dmbi.nbi.bbsrc.ac.uk/index.php/OMEROWorkshop OMERO Workshop] <br><br><br />
(Windows, Mac, Linux)<br />
|width="50%"| [http://openmicroscopy.org/site/support/omero4 Open Microscopy Environment Remote Objects (OMERO).] for visualising, managing, and annotating scientific image data. See also our [http://dmbi.nbi.bbsrc.ac.uk/index.php/OMEROWorkshop OMERO Workshop] training course we ran in April 2011.<br />
|}<br />
<br />
=<span style="color:DarkGreen;">Tools and Utilities=<br />
==<span style="color:DarkGreen;">BioformatsConverter==<br />
{| border="0" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="10%"| <imgicon>BioformatsConverterZip.png|100px|BioformatsConverter</imgicon><br />
|width="40%"|For converting microscope manufacturer proprietary file formats. <br><br>See [[BioformatsConverter|''Details'']]<br><br><br />
(Windows, Mac, Linux)<br />
|width="50%"| This tool allows for the batch conversion of microscope manufacturer proprietary file formats, to the open source OME-TIFF standard. Uses the [http://www.loci.wisc.edu/software/bio-formats Bioformats] library.<br />
|}<br />
==<span style="color:DarkGreen;">Dependency Checking Tool==<br />
<br />
Tool for recursively finding what further functions a function depends on. See [[myDepFun|''Details'']]<br />
<br />
=<span style="color:DarkGreen;">In development</span>=<br />
==<span style="color:DarkGreen;">MTtbox</span>==<br />
{| border="0" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="10%"| <imgicon>MTtboxA.jpg|100px|BioformatsConverter</imgicon><br />
|width="40%"|For modelling the behaviour of microtubules within a cell. <br><br><br />
See [[MTtbox documentation|''Details'']]<br><br><br />
(Windows, Mac, Linux)<br />
|width="50%"| In development. The idea is to be able to model the behaviour of growing microtubules and factors as they react chemically and diffuse within the different cell compartments.<br><br><br />
The icon shows a spherical cell sliced open to show concentric components: cell wall (magenta), plasma-membrane (yellow), cytoplasm (green) and vacuole (yellow). Microtubules (blue) grow in 3D within the cytoplasm.<br />
|}<br />
=<span style="color:Navy;">Historical</span>=<br />
==<span style="color:Navy;">Robot arm: still in production after 30 years (serving local industry)</span>==<br />
{| border="0" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="50%"|[[Image:2014-06-10 14.22.20 small robot.jpg|400px|]] <br />
|width="40%"|For teaching '''production control''' and '''interrupt programming'''. <br><br><br />
1983 and we are in a world of '''Apple II and BBC B computers''' - the ''6502'' processor reigns. Particularly good for real-time control it responded very fast to hardware interrupts from, for example, the timer. To illustrate timer interrupts what better than digital servo-motors? Set up the on-board timers to produce a stream of 'heartbeat' of pulses, one every 20 ms out of the parallel port and control up to eight motors. Pulse widths, from 1 to 2 ms, control the position of each motor arm. Derek Fulton and I made some loose lab. money by writing a series of articles showing exactly how to build and, in particular, control this robot arm.[[Publications#.28G.29_Computer_control.2C_measurement_and_commercial_software |Bangham et al.]]. <br />
Our copyright, I took it to a local company LJ-Electronics ([http://www.ljcreate.com/products/product.asp?id=314&program=195&curr=2 now LJ-Creative]) who incorporated it detail for detail into their product line. Originally, they called it the Emu. Still in production: '''Lovely outcome.'''<br />
|}<br />
<br />
=Historical=</div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Software&diff=6941Software2016-08-24T10:47:02Z<p>JacobNewman: </p>
<hr />
<div>__TOC__<br />
<!-- ===[[BanghamLabSVN|<span style="color: Navy">Copy of BanghamLab source code</span>]]=== --><br />
<br />
<span style="color: DarkGreen">'''Current activity: a collaboration''' with the [http://rico-coen.jic.ac.uk/index.php/Main_Page CoenLab] with the aim of understanding how patterns of gene activity in biological organs influence the developing shape. The BanghamLab is focussed on the conceptual underpinning: concepts captured in computational growth models, experimental data visualisation and analysis.</span><br />
<br />
===<span style="color:DarkGreen;">Notes on documenting our software</span>=== <br />
<br />
[[Tricks for documenting software|Notes for Lab members on how to contribute to this Wiki and where to put downloads. ]]<br />
<br>Matlab tip: searching a large data structure for a particular field. Clear the command window. Evaluate the structure to list all the fields, then use the usual control-f search tool on the command window.<br />
<br />
=<span style="color:DarkGreen;">'''Computational biology'''</span>=<br />
==<span style="color:DarkGreen;">Quantitative understanding of growing shapes: '''GFtbox'''</span>==<br />
====<span style="color:#A52A2A;">We developed ''GFtbox'' to allow us to model the growth of complex shapes with the ultimate goal: to understand the relationships between genes, growth and form.</span>====<br />
{| border="0" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="400"| [[Image:Journal.pbio.1000537.g009.png|350px|Growth of a flower]] Example of a growing snapdragon flower and some mutants ( [http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1000537 Green et al 2011]). Growth is specified by factors (genes) according to the Growing Polarised Tissue Framework. Colours represent putative gene activity, arrows the polariser gradient and spots clones.<br />
|<wikiflv width="300" height="313" logo="false" loop="true" background="white" position="right" autoplay='true'>GPT_Snapdragon_2010_Green_et_al-0002_1.flv|GPT_Snapdragon_2010_Green_et_al-0002.png</wikiflv><br />
<!--|<wikiflv width="280" height="300" logo="false" loop="true" background="white">Journal.pbio.1000537.s025_1.flv|Journal.pbio.1000537.g009.png</wikiflv>--><br />
|}<br />
<br />
{| border="0" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="10%"| <imgicon>GPT_thumbnail2.png|120px|GFtbox</imgicon> <br />
|width="40%"|<br />
For modelling the growth of shapes. <br><br><br />
<br />
[[Ready Reference Manual|'''''Ready Reference''''' Manual]]<br><br><br />
(PC, Mac, Linux, uses Matlab<br>no Mathworks toolboxes needed<br>[http://www.mathworks.com/products/matlab/tryit.html Matlab 30 day free trial] and <br>[http://www.mathworks.com/academia/student_version/?s_cid=global_nav student edition])<br><br><br />
<br />
<br />
[[Ready Reference Manual ancillary functions|'''''Ready Reference'for ancillary functions: in '''interaction function''' and external functions'''' Manual]]<br><br><br />
<br />
[[GFtbox|'''What? How? Where?''']]<span style="color: Green"> Background</span><br><br><br />
[[GFtbox Tutorial pages|'''''Tutorials''''': from the beginning]]<span style="color: Green"> Start here</span><br><br><br />
[[GFtbox Example pages|'''''Examples''''': from publications]]<br><br><br />
<br />
[https://sourceforge.net/p/gftbox/ <span style="color: Gray">'''''Download GFTbox''''' from SourceForge</span>]<br><br><br />
<span style="color: Gray">'''''Download GFTbox project files:''''' </span><br><br><br />
<br />
[http://cmpdartsvr1.cmp.uea.ac.uk/downloads/software/OpenSourceDownload_PAPER_Eldridge_2016/FILENAME.zip <span style="color: Gray">'''''Fruits''''' Eldridge et al 2016</span>]<br><br><br />
<br />
[http://cmpdartsvr1.cmp.uea.ac.uk/downloads/software/OpenSourceDownload_PLoS_SauretGueto_2013/GPT_Petal_PLoS_20130502.zip <span style="color: Gray">'''''Petals''''' Sauret-Güeto et al 2013</span>]<br><br><br />
<br />
[http://cmpdartsvr1.cmp.uea.ac.uk/downloads/software/OpenSourceDownload_Science_Paper_2012/GPT_ArabidopsisLeafModel_20120207.zip <span style="color: Gray">'''''Leaves''''' Kuchen et al 2012</span>]<br><br><br />
<br />
[http://cmpdartsvr1.cmp.uea.ac.uk/downloads/software/OpenSourceDownload_PLoS_Kennaway_2011/Kennaway-etal-2011.zip <span style="color: Gray">'''''Principles and concepts''''' Kennaway et al 2011</span>]<br><br><br />
<br />
[http://cmpdartsvr1.cmp.uea.ac.uk/downloads/software/OpenSourceDownload_PLoS_Green_2011/Green-etal-2010.zip <span style="color: Gray">'''''Snapdragon''''' Green et al 2011, Cui et al 2010</span>]<br><br><br />
<br />
|width="50%"| <span style="color:#AF002A;"> ''GFtbox'' is an implementation of the Growing Polarised Tissue Framework (GPT-framework) for understanding and modelling the relationship between gene activity and the growth of shapes such leaves, flowers and animal embryos ([http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1002071 Kennaway et al 2011]). </span><br><br><br />
<br />
<span style="color:#AF002A;">The GPT-framework was used to capture an understanding of (to model) the growing petal ([http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1001550 Sauret-Güeto et al 2013]), leaf ([http://www.sciencemag.org/content/335/6072/1092.abstract Kuchen et al 2012]) and Snapdragon flower [http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1000537 Green et al 2011]. The Snapdragon model was validated by comparing the results with other mutant and transgenic flowers [http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1000538 Cui et al 2010.]</span><br><br><br />
<br />
The key point is how '''outgrowths can be specified by genes'''. The icon shows an asymmetrical outgrowth. Conceptually, it is specifed by two independent patterns under genetic control: a pattern of growth and a pattern of organisers. The outgrowth arises from a region of extra overall growth. Growth is aligned along axes set by two interacting systems. Organisers at the ends of the mesh create a lengthwise gradient. This gradient interacts with the second due to putative organisers that generate polariser sinks in the region that becomes the tips of the palette outgrowth. ([http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1002071 Kennaway et al 2011]). These hypotheses need to be tested in biological systems.<br />
|}<br />
<br />
==<span style="color:DarkGreen;">Viewing and measuring volume images: '''VolViewer'''==<br />
{| border="0" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="10%"| <imgicon>VolViewer-logo.png|120px|VolViewer</imgicon> <br />
|width="40%"|For viewing and measuring '''volume images''' on both normal and '''stereo''' screens. Typical images from: confocal microscope and Optical Projection Tomography (OPT) images<br><br><br />
[[VolViewer#Description|'''What? How? Where?''']]<br><br><br />
[[VolViewer#User Documentation|'''''Tutorials''''': from the beginning]]<br><br><br />
[[VolViewer#Download| '''''Download''''']]<br><br><br />
(Windows, Mac, Linux)<br><br><br />
Output from VolViewer has appeared in:<br><br />
<br />
[http://www.cell.com/cell_picture_show-plantbio Cell: Online Gallery] | [http://www.amazon.co.uk/Handbook-Plant-Science-Keith-Roberts/dp/0470057238/ref=sr_1_19?s=books&ie=UTF8&qid=1289321357&sr=1-19 Front cover: Handbook of Plant Science] | [http://www.plantcell.org/content/18/9.toc Front cover: The Plant Cell] | [http://www.americanscientist.org/issues/pub/2013/1/3d-carnivorous-plants American Scientist] | [http://www.rms.org.uk/Resources/Royal%20Microscopical%20Society/infocus/Edgar%20article.pdf Royal Microscopical Society: Infocus Magazine] | [http://www.bioptonics.com/Home.htm Bundled with the Bioptonic 3001 scanner: Bioptonics Viewer] | [http://www.dailymail.co.uk/sciencetech/article-2215052/The-complexity-intricacy-Mother-Nature-revealed-incredible-pictures-plants--seen-inside.html The Daily Mail] | [http://www.guardian.co.uk/science/gallery/2007/sep/04/fruitflybrain#/?picture=330675671&index=1 The Guardian newspaper: 3D Fruit fly] | [http://qt.nokia.com/qt-in-use/ambassadors/project?id=a0F20000006LZ2pEAG Qt Ambassador program] | [http://www.triffidnurseries.co.uk/special3.php Triffid Nurseries website]<br />
<br />
<br><br><br />
|width="50%"| VolViewer is used as a '''stand-alone''' app. or as a '''viewport for other systems''', e.g. Matlab programs. VolViewer uses [http://www.opengl.org/ OpenGL] and [http://qt.nokia.com/products/ Qt] to provide a user friendly application to interactively explore and quantify multi-dimensional biological images. It has been successfully used in our lab to explore and quantify confocal microscopy and optical projection tomography images. Written by Jerome Avondo it is open-source and is also compatible with the Open Microscopy Environment ([http://openmicroscopy.org/site OME]) (Chris Allen and Avondo, et. al. ''OMERO: flexible, model-driven data management for experimental biology'' Nature Methods 9, 245–253 (2012))<br> [[image:Silique.PNG|360px]]).<br />
|}<br />
<br />
==<span style="color:DarkGreen;">Analysing shapes in 2D and 3D: '''AAMToolbox'''==<br />
{| border="0" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="10%"| <imgicon>AAMToolbox_logo.jpg|120px|AAMToolbox</imgicon> <br />
|width="40%"|For analysing populations of shapes and colours within the shapes using principal component analysis. <br><br><br />
[[AAMToolbox Details|'''What? How? Where?''']]<br><br><br />
[[Tutorials on the Shape modelling toolbox|'''''Tutorials''''': from the beginning]]<br><br><br />
<br />
[[AAMToolbox Download|<span style="color: Gray">'''''Download revised Nov2012''''' </span>]]<br><br><br />
<br />
<br />
(PC, Mac, Linux, uses Matlab<br>no Mathworks toolboxes needed<br>[http://www.mathworks.com/products/matlab/tryit.html Matlab 30 day free trial] and <br>[http://www.mathworks.com/academia/student_version/?s_cid=global_nav student edition])<br><br><br />
|width="50%"| The AAMToolbox enables the user analyse the shape and colour of collections of similar objects. Originally developed to analyse face shapes for lipreading ([http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=982900 Matthews ''et al''. 2002][http://www2.cmp.uea.ac.uk/~sjc/matthews-pami-01.pdf version of pdf]), we have used it extensively for analysing the shapes of leaves ([http://www.pnas.org/content/102/29/10221.short Langlade ''et al'' 2005.],[http://www.tandfonline.com/doi/abs/10.2976/1.2836738 Bensmihen ''et al.'' 2010]) and petals ([http://www.sciencemag.org/content/313/5789/963.short Whibley ''et al'' 2006],[http://www.mssaleshops.info/content/21/10/2999.short Feng ''et al''. 2010]). The analysis can be applied to art, for example, finding systematic differences between portraits by Rembrandt and Modigliani.<br />
|}<br />
==<span style="color:DarkGreen;">Analysing the shapes of clones: '''SectorAnalysisToolbox'''==<br />
{| border="0" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="10%"| <imgicon>Sector analysis icon.jpg|120px|SectorAnalysisToolbox</imgicon> <br />
|width="40%"|For analysing the shapes of marked cell clones. <br><br><br />
[[SectorAnalysisToolbox Details|'''What? How? Where?''']]<br><br><br />
[[SectorAnalysisToolbox Documentation|'''''Tutorials''''': from the beginning]]<br><br><br />
[http://cmpdartsvr1.cmp.uea.ac.uk/downloads/software/OpenSourceDownload_Science_Paper_2012/SectorAnalysisToolbox.zip <span style="color: Gray">'''''Download''''' </span>]<br><br><br />
(PC, Mac, Linux, uses Matlab<br>no Mathworks toolboxes needed<br>[http://www.mathworks.com/products/matlab/tryit.html Matlab 30 day free trial] and <br>[http://www.mathworks.com/academia/student_version/?s_cid=global_nav student edition])<br><br><br />
|width="50%"| The SectorAnalysisToolbox enables the user analyse the shapes of marked clones in a sheet of tissue.<br />
|}<br />
<br />
=<span style="color:Navy;">'''Algorithms'''=<br />
==<span style="color:Chocolate;">MSERs, extrema, connected-set filters and sieves==<br />
====<span style="color:Chocolate;">The algorithm finding MSER's starts with a connected-set opening or 'o' sieve</span>====<br />
[[Comparison of Matlab MSER's and 'o' sieve|Comparison of Matlab MSER's and 'o' sieve]] Essentially, no difference.<br />
{| border="0" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="50%"| [[Image:Cameraman_iso_topview.jpg|300px|link=AAMToolbox Details|MSERs]] Cameraman image. Superimposed red spots are maximal extrema and blue spots are minima. Irregular cyan, blue and yellow regions illustrate regions associated with maxima and the magenta region is a minimum.<br />
|[[Image:cameraman_iso_tree.jpg|300px|link=AAMToolbox Details|MSERs over scale-space]]<br>Isometric view of the cameraman image with superimposed maxima (red) and minima (blue). The trees trace the maxima through increasing scale-space. Large spots have been identified as stable extrema.<br />
|}<br />
<br />
===<span style="color: #C364C5;">Finding interest points, features and segmenting images. </span>===<br />
#[[MSER and Sieve Details|<span style="color:Chocolate;">'''Technical briefing'''</span>]] <span style="color: #C364C5;">MSER's incorporate 'o' sieves.<br />
#[[MSER's and Connected sets|<span style="color:Chocolate;">'''The twist''': from restricted median filters to sieves and MSER's</span>]]<br />
#One dimensional sieves (measure length)<br />
##[[Types of 1D sieve|Types of 1D sieve]]<br />
##[[First applied to hydrophobicity plots|First applied to hydrophobicity plots]] but lets exploit their idempotency.<br />
#Two dimensional sieves (measure areas)<br />
##Properties<br />
##Relation to MSER's<br />
#Three dimensional sieves (measure volumes)<br />
##[[Segment by volume|Segment by volume]] instant gratification.<br />
<!--[[siv Download|<span style="color: Gray">'''''Download''''' </span>]]<br><br>--><br />
<br />
<!-- [[Software#MSERs extrema connected-set filters and sieves|<span style="color:Green;">'''MORE'''</span>]] --><br />
<br />
===<span style="color:Chocolate;">Art, extrema of light and shade: '''''PhotoArtMaster'''''===<br />
<br />
<br />
Art created using ArtMaster, and ArtMaster itself was featured in an exhibit at the London Victoria and Albert (V&A) Museum exhibition <span style="color:Chocolate;">'Cheating? How to make Perfect Work of Art' (2003).</span> The exhibition centered on the idea of [http://en.wikipedia.org/wiki/Hockney%E2%80%93Falco_thesis Hockney's] that advances in realism and accuracy in the history of Western art since the Renaissance were primarily the result of optical aids such as the camera obscura, camera lucida, and curved mirrors. My exhibit used a touch screen (rare in those days) and ArtMaster to help visitors create 'paintings' from photographs. [http://www.sciencemuseum.org.uk/visitmuseum/galleries/turing.aspx finding its name]. (It is entirely different in principle from the software more recently used by Hockney to paint with an iPad.)<br><br />
<br />
{| border="0" width=100% style="background-color:#ffffff;"<br />
|-<br />
|align="center"|[[Image:DegasLightAndShade.jpg|400px]][[Image:Emma_face_Art_C.jpg|300px]]<br />
<br><br><br />
Illustration of PhotoArtMaster used to find and 'paint' with regions of light and shade crisply segmented from a photograph. Likewise, on the right, edges.<br />
|}<br />
=====PhotoArtMaster=====<br />
Saturday 07/06/2014: Inspired Photographer of the Year 2013 Tony Bennett when asked whether his photograph [http://www.bbc.co.uk/programmes/galleries/p020hd8s Mists and Reflections] had been Photoshopped replied something like <br><br />
"A digital camera delivers an unemotional ''raw'' image of pixels that you have to manipulate to ''create your photograph''" Photographers manipulate as little as possible. <br><br />
However there is '''another path one that creates pictures'''. For this you need another piece of software: PhotoArtMaster (ArtMaster). Professional Photographer said ''"'''Forget any comparison whatsoever with the art filters in Photoshop - this software reaches out and enters different stratospheres'''"'' [[Professional Photographer]].<br><br><br />
Early versions of PhotoArtMaster are still '''available from Amazon''' at low prices (I'm not sure where they come from.)<br />
[http://www.amazon.co.uk/s/ref=nb_sb_noss?url=search-alias%3Daps&field-keywords=photoartmaster] . Some help for both the early versions and the latest version can be found in [http://cmpdartsvr1.cmp.uea.ac.uk/downloads/software/SieveWebPages/a4a_2_screensize.pdf <span style="color: Chocolate">''''this document''''' </span>]).<br />
=====Links to third party PhotoArtMastered pictures=====<br />
*[https://picasaweb.google.com/113257474829608374943/InTheStyleOf Oliver Bangham] Colouful rounded shapes from, yes, my brother.<br />
*[http://en.wikipedia.org/wiki/Wimbledon_%28film%29 The entry sequence of the comedy film 'Wimbledon']<br />
<br />
====The final version of the Windows ArtMaster2.0 [http://cmpdartsvr1.cmp.uea.ac.uk/downloads/software/OpenSource_ArtMaster/ArtMaster2.0Release.zip <span style="color: #B31B1B">'''''is downloadable here'''''</span>] with no support.====<br />
Unzip into (for example) the "Program Files" directory then set your system environment to include: ''C:\Program Files\Pam2.0 Release\jre\bin;C:\Program Files\Pam2.0 Release\bin;'' (You may need help for this. I right clicked 'computer' from the 'Start' menu, then selected 'Advanced system settings', then 'Environment Variables' and finally slid through the System variables until I found and selected 'Path'. This allowed me to edit the path by adding ';C:\Program Files\Pam2.0 Release\jre\bin;C:\Program Files\Pam2.0 Release\bin' to the end). Rather detailed help using the software is available in [http://cmpdartsvr1.cmp.uea.ac.uk/downloads/software/SieveWebPages/a4a_2_screensize.pdf <span style="color: Chocolate">''''this essay''''' </span>].<br><br><br />
[http://cmpdartsvr1.cmp.uea.ac.uk/downloads/software/SieveWebPages/Art_For_All_a4a_3_web2.pdf <span style="color: Chocolate">'''''The sieve algorithms underpinning PhotoArtMaster software''''' </span>] are described in an extract of the <br />
[http://cmpdartsvr1.cmp.uea.ac.uk/downloads/software/SieveWebPages/a4a_2_screensize.pdf <span style="color: Chocolate">''''essay''''' </span>]. These documents were written to support our Fo2Pix company. PhotoArtMaster originally sold >65,000 licences but ill health forced the closure of Fo2Pix.<br />
<br />
{| border="0" width=100% style="background-color:#ffffff;"<br />
|-<br />
|align="center"|[[Image:Trees_1.jpg|600px]]<br />
<br><br><br />
Illustration of PhotoArtMaster used to find and 'paint' with regions of colour crisply segmented from a photograph. <br />
|}<br />
<br />
<!-- [[Documentation_of_Connected_Set_Filters_or_Sieves|Art test page]]<br><br> <br />
[[Documentation_of_Connected_Set_Filters_or_Sieves]] --><br />
<br />
==<span style="color:Navy;">Reaction-diffusion and morphogenesis==<br />
{| border="0" width=100% style="background-color:#000000;"<br />
|-<br />
|align="center"|<br />
[[Image:tentacles_morphogenesis.png|600px]]<br />
|}<br />
Illustration of morphogenesis inspired by Turing's paper. <br><br><br />
[http://cmpdartsvr1.cmp.uea.ac.uk/downloads/software/GPT_ReactionDiffusionTentacles_20121211.zip <span style="color: Navy">Example using growth toolbox GPT_ReactionDiffusionTentacles_20121211.zip</span>]<br />
<br><br><br />
===1 A===<br />
<br />
=<span style="color:DarkGreen;">Open source systems to which we have contributed=<br />
==<span style="color:DarkGreen;">OMERO==<br />
{| border="0" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="10%"| <imgicon>OMERO_DIAGRAM.jpg|100px|OMERO</imgicon><br />
|width="40%"|For working with the OME image database. <br><br>See [http://www.openmicroscopy.org/site/products/omero ''Details''], [http://www.openmicroscopy.org/site/support/omero4/downloads ''Download'']<br> [http://dmbi.nbi.bbsrc.ac.uk/index.php/OMEROWorkshop OMERO Workshop] <br><br><br />
(Windows, Mac, Linux)<br />
|width="50%"| [http://openmicroscopy.org/site/support/omero4 Open Microscopy Environment Remote Objects (OMERO).] for visualising, managing, and annotating scientific image data. See also our [http://dmbi.nbi.bbsrc.ac.uk/index.php/OMEROWorkshop OMERO Workshop] training course we ran in April 2011.<br />
|}<br />
<br />
=<span style="color:DarkGreen;">Tools and Utilities=<br />
==<span style="color:DarkGreen;">BioformatsConverter==<br />
{| border="0" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="10%"| <imgicon>BioformatsConverterZip.png|100px|BioformatsConverter</imgicon><br />
|width="40%"|For converting microscope manufacturer proprietary file formats. <br><br>See [[BioformatsConverter|''Details'']]<br><br><br />
(Windows, Mac, Linux)<br />
|width="50%"| This tool allows for the batch conversion of microscope manufacturer proprietary file formats, to the open source OME-TIFF standard. Uses the [http://www.loci.wisc.edu/software/bio-formats Bioformats] library.<br />
|}<br />
==<span style="color:DarkGreen;">Dependency Checking Tool==<br />
<br />
Tool for recursively finding what further functions a function depends on. See [[myDepFun|''Details'']]<br />
<br />
=<span style="color:DarkGreen;">In development</span>=<br />
==<span style="color:DarkGreen;">MTtbox</span>==<br />
{| border="0" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="10%"| <imgicon>MTtboxA.jpg|100px|BioformatsConverter</imgicon><br />
|width="40%"|For modelling the behaviour of microtubules within a cell. <br><br><br />
See [[MTtbox documentation|''Details'']]<br><br><br />
(Windows, Mac, Linux)<br />
|width="50%"| In development. The idea is to be able to model the behaviour of growing microtubules and factors as they react chemically and diffuse within the different cell compartments.<br><br><br />
The icon shows a spherical cell sliced open to show concentric components: cell wall (magenta), plasma-membrane (yellow), cytoplasm (green) and vacuole (yellow). Microtubules (blue) grow in 3D within the cytoplasm.<br />
|}<br />
=<span style="color:Navy;">Historical</span>=<br />
==<span style="color:Navy;">Robot arm: still in production after 30 years (serving local industry)</span>==<br />
{| border="0" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="50%"|[[Image:2014-06-10 14.22.20 small robot.jpg|400px|]] <br />
|width="40%"|For teaching '''production control''' and '''interrupt programming'''. <br><br><br />
1983 and we are in a world of '''Apple II and BBC B computers''' - the ''6502'' processor reigns. Particularly good for real-time control it responded very fast to hardware interrupts from, for example, the timer. To illustrate timer interrupts what better than digital servo-motors? Set up the on-board timers to produce a stream of 'heartbeat' of pulses, one every 20 ms out of the parallel port and control up to eight motors. Pulse widths, from 1 to 2 ms, control the position of each motor arm. Derek Fulton and I made some loose lab. money by writing a series of articles showing exactly how to build and, in particular, control this robot arm.[[Publications#.28G.29_Computer_control.2C_measurement_and_commercial_software |Bangham et al.]]. <br />
Our copyright, I took it to a local company LJ-Electronics ([http://www.ljcreate.com/products/product.asp?id=314&program=195&curr=2 now LJ-Creative]) who incorporated it detail for detail into their product line. Originally, they called it the Emu. Still in production: '''Lovely outcome.'''<br />
|}<br />
<br />
=Historical=</div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Software&diff=6940Software2016-08-24T10:27:00Z<p>JacobNewman: </p>
<hr />
<div>__TOC__<br />
<!-- ===[[BanghamLabSVN|<span style="color: Navy">Copy of BanghamLab source code</span>]]=== --><br />
<br />
<span style="color: DarkGreen">'''Current activity: a collaboration''' with the [http://rico-coen.jic.ac.uk/index.php/Main_Page CoenLab] with the aim of understanding how patterns of gene activity in biological organs influence the developing shape. The BanghamLab is focussed on the conceptual underpinning: concepts captured in computational growth models, experimental data visualisation and analysis.</span><br />
<br />
===<span style="color:DarkGreen;">Notes on documenting our software</span>=== <br />
<br />
[[Tricks for documenting software|Notes for Lab members on how to contribute to this Wiki and where to put downloads. ]]<br />
<br>Matlab tip: searching a large data structure for a particular field. Clear the command window. Evaluate the structure to list all the fields, then use the usual control-f search tool on the command window.<br />
<br />
=<span style="color:DarkGreen;">'''Computational biology'''</span>=<br />
==<span style="color:DarkGreen;">Quantitative understanding of growing shapes: '''GFtbox'''</span>==<br />
====<span style="color:#A52A2A;">We developed ''GFtbox'' to allow us to model the growth of complex shapes with the ultimate goal: to understand the relationships between genes, growth and form.</span>====<br />
{| border="0" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="400"| [[Image:Journal.pbio.1000537.g009.png|350px|Growth of a flower]] Example of a growing snapdragon flower and some mutants ( [http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1000537 Green et al 2011]). Growth is specified by factors (genes) according to the Growing Polarised Tissue Framework. Colours represent putative gene activity, arrows the polariser gradient and spots clones.<br />
|<wikiflv width="300" height="313" logo="false" loop="true" background="white" position="right" autoplay='true'>GPT_Snapdragon_2010_Green_et_al-0002_1.flv|GPT_Snapdragon_2010_Green_et_al-0002.png</wikiflv><br />
<!--|<wikiflv width="280" height="300" logo="false" loop="true" background="white">Journal.pbio.1000537.s025_1.flv|Journal.pbio.1000537.g009.png</wikiflv>--><br />
|}<br />
<br />
{| border="0" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="10%"| <imgicon>GPT_thumbnail2.png|120px|GFtbox</imgicon> <br />
|width="40%"|<br />
For modelling the growth of shapes. <br><br><br />
<br />
[[Ready Reference Manual|'''''Ready Reference''''' Manual]]<br><br><br />
(PC, Mac, Linux, uses Matlab<br>no Mathworks toolboxes needed<br>[http://www.mathworks.com/products/matlab/tryit.html Matlab 30 day free trial] and <br>[http://www.mathworks.com/academia/student_version/?s_cid=global_nav student edition])<br><br><br />
<br />
<br />
[[Ready Reference Manual ancillary functions|'''''Ready Reference'for ancillary functions: in '''interaction function''' and external functions'''' Manual]]<br><br><br />
<br />
[[GFtbox|'''What? How? Where?''']]<span style="color: Green"> Background</span><br><br><br />
[[GFtbox Tutorial pages|'''''Tutorials''''': from the beginning]]<span style="color: Green"> Start here</span><br><br><br />
[[GFtbox Example pages|'''''Examples''''': from publications]]<br><br><br />
<br />
[https://sourceforge.net/p/gftbox/ <span style="color: Gray">'''''Download GFTbox''''' from SourceForge</span>]<br><br><br />
<span style="color: Gray">'''''Download GFTbox project files:''''' </span><br><br><br />
<br />
[http://cmpdartsvr1.cmp.uea.ac.uk/downloads/software/OpenSourceDownload_PAPER_Eldridge_2016/FILENAME.zip <span style="color: Gray">'''''Capsella''''' Eldridge et al 2016</span>]<br><br><br />
<br />
[http://cmpdartsvr1.cmp.uea.ac.uk/downloads/software/OpenSourceDownload_PLoS_SauretGueto_2013/GPT_Petal_PLoS_20130502.zip <span style="color: Gray">'''''Petals''''' Sauret-Güeto et al 2013</span>]<br><br><br />
<br />
[http://cmpdartsvr1.cmp.uea.ac.uk/downloads/software/OpenSourceDownload_Science_Paper_2012/GPT_ArabidopsisLeafModel_20120207.zip <span style="color: Gray">'''''Leaves''''' Kuchen et al 2012</span>]<br><br><br />
<br />
[http://cmpdartsvr1.cmp.uea.ac.uk/downloads/software/OpenSourceDownload_PLoS_Kennaway_2011/Kennaway-etal-2011.zip <span style="color: Gray">'''''Principles and concepts''''' Kennaway et al 2011</span>]<br><br><br />
<br />
[http://cmpdartsvr1.cmp.uea.ac.uk/downloads/software/OpenSourceDownload_PLoS_Green_2011/Green-etal-2010.zip <span style="color: Gray">'''''Snapdragon''''' Green et al 2011, Cui et al 2010</span>]<br><br><br />
<br />
|width="50%"| <span style="color:#AF002A;"> ''GFtbox'' is an implementation of the Growing Polarised Tissue Framework (GPT-framework) for understanding and modelling the relationship between gene activity and the growth of shapes such leaves, flowers and animal embryos ([http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1002071 Kennaway et al 2011]). </span><br><br><br />
<br />
<span style="color:#AF002A;">The GPT-framework was used to capture an understanding of (to model) the growing petal ([http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1001550 Sauret-Güeto et al 2013]), leaf ([http://www.sciencemag.org/content/335/6072/1092.abstract Kuchen et al 2012]) and Snapdragon flower [http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1000537 Green et al 2011]. The Snapdragon model was validated by comparing the results with other mutant and transgenic flowers [http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1000538 Cui et al 2010.]</span><br><br><br />
<br />
The key point is how '''outgrowths can be specified by genes'''. The icon shows an asymmetrical outgrowth. Conceptually, it is specifed by two independent patterns under genetic control: a pattern of growth and a pattern of organisers. The outgrowth arises from a region of extra overall growth. Growth is aligned along axes set by two interacting systems. Organisers at the ends of the mesh create a lengthwise gradient. This gradient interacts with the second due to putative organisers that generate polariser sinks in the region that becomes the tips of the palette outgrowth. ([http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1002071 Kennaway et al 2011]). These hypotheses need to be tested in biological systems.<br />
|}<br />
<br />
==<span style="color:DarkGreen;">Viewing and measuring volume images: '''VolViewer'''==<br />
{| border="0" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="10%"| <imgicon>VolViewer-logo.png|120px|VolViewer</imgicon> <br />
|width="40%"|For viewing and measuring '''volume images''' on both normal and '''stereo''' screens. Typical images from: confocal microscope and Optical Projection Tomography (OPT) images<br><br><br />
[[VolViewer#Description|'''What? How? Where?''']]<br><br><br />
[[VolViewer#User Documentation|'''''Tutorials''''': from the beginning]]<br><br><br />
[[VolViewer#Download| '''''Download''''']]<br><br><br />
(Windows, Mac, Linux)<br><br><br />
Output from VolViewer has appeared in:<br><br />
<br />
[http://www.cell.com/cell_picture_show-plantbio Cell: Online Gallery] | [http://www.amazon.co.uk/Handbook-Plant-Science-Keith-Roberts/dp/0470057238/ref=sr_1_19?s=books&ie=UTF8&qid=1289321357&sr=1-19 Front cover: Handbook of Plant Science] | [http://www.plantcell.org/content/18/9.toc Front cover: The Plant Cell] | [http://www.americanscientist.org/issues/pub/2013/1/3d-carnivorous-plants American Scientist] | [http://www.rms.org.uk/Resources/Royal%20Microscopical%20Society/infocus/Edgar%20article.pdf Royal Microscopical Society: Infocus Magazine] | [http://www.bioptonics.com/Home.htm Bundled with the Bioptonic 3001 scanner: Bioptonics Viewer] | [http://www.dailymail.co.uk/sciencetech/article-2215052/The-complexity-intricacy-Mother-Nature-revealed-incredible-pictures-plants--seen-inside.html The Daily Mail] | [http://www.guardian.co.uk/science/gallery/2007/sep/04/fruitflybrain#/?picture=330675671&index=1 The Guardian newspaper: 3D Fruit fly] | [http://qt.nokia.com/qt-in-use/ambassadors/project?id=a0F20000006LZ2pEAG Qt Ambassador program] | [http://www.triffidnurseries.co.uk/special3.php Triffid Nurseries website]<br />
<br />
<br><br><br />
|width="50%"| VolViewer is used as a '''stand-alone''' app. or as a '''viewport for other systems''', e.g. Matlab programs. VolViewer uses [http://www.opengl.org/ OpenGL] and [http://qt.nokia.com/products/ Qt] to provide a user friendly application to interactively explore and quantify multi-dimensional biological images. It has been successfully used in our lab to explore and quantify confocal microscopy and optical projection tomography images. Written by Jerome Avondo it is open-source and is also compatible with the Open Microscopy Environment ([http://openmicroscopy.org/site OME]) (Chris Allen and Avondo, et. al. ''OMERO: flexible, model-driven data management for experimental biology'' Nature Methods 9, 245–253 (2012))<br> [[image:Silique.PNG|360px]]).<br />
|}<br />
<br />
==<span style="color:DarkGreen;">Analysing shapes in 2D and 3D: '''AAMToolbox'''==<br />
{| border="0" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="10%"| <imgicon>AAMToolbox_logo.jpg|120px|AAMToolbox</imgicon> <br />
|width="40%"|For analysing populations of shapes and colours within the shapes using principal component analysis. <br><br><br />
[[AAMToolbox Details|'''What? How? Where?''']]<br><br><br />
[[Tutorials on the Shape modelling toolbox|'''''Tutorials''''': from the beginning]]<br><br><br />
<br />
[[AAMToolbox Download|<span style="color: Gray">'''''Download revised Nov2012''''' </span>]]<br><br><br />
<br />
<br />
(PC, Mac, Linux, uses Matlab<br>no Mathworks toolboxes needed<br>[http://www.mathworks.com/products/matlab/tryit.html Matlab 30 day free trial] and <br>[http://www.mathworks.com/academia/student_version/?s_cid=global_nav student edition])<br><br><br />
|width="50%"| The AAMToolbox enables the user analyse the shape and colour of collections of similar objects. Originally developed to analyse face shapes for lipreading ([http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=982900 Matthews ''et al''. 2002][http://www2.cmp.uea.ac.uk/~sjc/matthews-pami-01.pdf version of pdf]), we have used it extensively for analysing the shapes of leaves ([http://www.pnas.org/content/102/29/10221.short Langlade ''et al'' 2005.],[http://www.tandfonline.com/doi/abs/10.2976/1.2836738 Bensmihen ''et al.'' 2010]) and petals ([http://www.sciencemag.org/content/313/5789/963.short Whibley ''et al'' 2006],[http://www.mssaleshops.info/content/21/10/2999.short Feng ''et al''. 2010]). The analysis can be applied to art, for example, finding systematic differences between portraits by Rembrandt and Modigliani.<br />
|}<br />
==<span style="color:DarkGreen;">Analysing the shapes of clones: '''SectorAnalysisToolbox'''==<br />
{| border="0" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="10%"| <imgicon>Sector analysis icon.jpg|120px|SectorAnalysisToolbox</imgicon> <br />
|width="40%"|For analysing the shapes of marked cell clones. <br><br><br />
[[SectorAnalysisToolbox Details|'''What? How? Where?''']]<br><br><br />
[[SectorAnalysisToolbox Documentation|'''''Tutorials''''': from the beginning]]<br><br><br />
[http://cmpdartsvr1.cmp.uea.ac.uk/downloads/software/OpenSourceDownload_Science_Paper_2012/SectorAnalysisToolbox.zip <span style="color: Gray">'''''Download''''' </span>]<br><br><br />
(PC, Mac, Linux, uses Matlab<br>no Mathworks toolboxes needed<br>[http://www.mathworks.com/products/matlab/tryit.html Matlab 30 day free trial] and <br>[http://www.mathworks.com/academia/student_version/?s_cid=global_nav student edition])<br><br><br />
|width="50%"| The SectorAnalysisToolbox enables the user analyse the shapes of marked clones in a sheet of tissue.<br />
|}<br />
<br />
=<span style="color:Navy;">'''Algorithms'''=<br />
==<span style="color:Chocolate;">MSERs, extrema, connected-set filters and sieves==<br />
====<span style="color:Chocolate;">The algorithm finding MSER's starts with a connected-set opening or 'o' sieve</span>====<br />
[[Comparison of Matlab MSER's and 'o' sieve|Comparison of Matlab MSER's and 'o' sieve]] Essentially, no difference.<br />
{| border="0" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="50%"| [[Image:Cameraman_iso_topview.jpg|300px|link=AAMToolbox Details|MSERs]] Cameraman image. Superimposed red spots are maximal extrema and blue spots are minima. Irregular cyan, blue and yellow regions illustrate regions associated with maxima and the magenta region is a minimum.<br />
|[[Image:cameraman_iso_tree.jpg|300px|link=AAMToolbox Details|MSERs over scale-space]]<br>Isometric view of the cameraman image with superimposed maxima (red) and minima (blue). The trees trace the maxima through increasing scale-space. Large spots have been identified as stable extrema.<br />
|}<br />
<br />
===<span style="color: #C364C5;">Finding interest points, features and segmenting images. </span>===<br />
#[[MSER and Sieve Details|<span style="color:Chocolate;">'''Technical briefing'''</span>]] <span style="color: #C364C5;">MSER's incorporate 'o' sieves.<br />
#[[MSER's and Connected sets|<span style="color:Chocolate;">'''The twist''': from restricted median filters to sieves and MSER's</span>]]<br />
#One dimensional sieves (measure length)<br />
##[[Types of 1D sieve|Types of 1D sieve]]<br />
##[[First applied to hydrophobicity plots|First applied to hydrophobicity plots]] but lets exploit their idempotency.<br />
#Two dimensional sieves (measure areas)<br />
##Properties<br />
##Relation to MSER's<br />
#Three dimensional sieves (measure volumes)<br />
##[[Segment by volume|Segment by volume]] instant gratification.<br />
<!--[[siv Download|<span style="color: Gray">'''''Download''''' </span>]]<br><br>--><br />
<br />
<!-- [[Software#MSERs extrema connected-set filters and sieves|<span style="color:Green;">'''MORE'''</span>]] --><br />
<br />
===<span style="color:Chocolate;">Art, extrema of light and shade: '''''PhotoArtMaster'''''===<br />
<br />
<br />
Art created using ArtMaster, and ArtMaster itself was featured in an exhibit at the London Victoria and Albert (V&A) Museum exhibition <span style="color:Chocolate;">'Cheating? How to make Perfect Work of Art' (2003).</span> The exhibition centered on the idea of [http://en.wikipedia.org/wiki/Hockney%E2%80%93Falco_thesis Hockney's] that advances in realism and accuracy in the history of Western art since the Renaissance were primarily the result of optical aids such as the camera obscura, camera lucida, and curved mirrors. My exhibit used a touch screen (rare in those days) and ArtMaster to help visitors create 'paintings' from photographs. [http://www.sciencemuseum.org.uk/visitmuseum/galleries/turing.aspx finding its name]. (It is entirely different in principle from the software more recently used by Hockney to paint with an iPad.)<br><br />
<br />
{| border="0" width=100% style="background-color:#ffffff;"<br />
|-<br />
|align="center"|[[Image:DegasLightAndShade.jpg|400px]][[Image:Emma_face_Art_C.jpg|300px]]<br />
<br><br><br />
Illustration of PhotoArtMaster used to find and 'paint' with regions of light and shade crisply segmented from a photograph. Likewise, on the right, edges.<br />
|}<br />
=====PhotoArtMaster=====<br />
Saturday 07/06/2014: Inspired Photographer of the Year 2013 Tony Bennett when asked whether his photograph [http://www.bbc.co.uk/programmes/galleries/p020hd8s Mists and Reflections] had been Photoshopped replied something like <br><br />
"A digital camera delivers an unemotional ''raw'' image of pixels that you have to manipulate to ''create your photograph''" Photographers manipulate as little as possible. <br><br />
However there is '''another path one that creates pictures'''. For this you need another piece of software: PhotoArtMaster (ArtMaster). Professional Photographer said ''"'''Forget any comparison whatsoever with the art filters in Photoshop - this software reaches out and enters different stratospheres'''"'' [[Professional Photographer]].<br><br><br />
Early versions of PhotoArtMaster are still '''available from Amazon''' at low prices (I'm not sure where they come from.)<br />
[http://www.amazon.co.uk/s/ref=nb_sb_noss?url=search-alias%3Daps&field-keywords=photoartmaster] . Some help for both the early versions and the latest version can be found in [http://cmpdartsvr1.cmp.uea.ac.uk/downloads/software/SieveWebPages/a4a_2_screensize.pdf <span style="color: Chocolate">''''this document''''' </span>]).<br />
=====Links to third party PhotoArtMastered pictures=====<br />
*[https://picasaweb.google.com/113257474829608374943/InTheStyleOf Oliver Bangham] Colouful rounded shapes from, yes, my brother.<br />
*[http://en.wikipedia.org/wiki/Wimbledon_%28film%29 The entry sequence of the comedy film 'Wimbledon']<br />
<br />
====The final version of the Windows ArtMaster2.0 [http://cmpdartsvr1.cmp.uea.ac.uk/downloads/software/OpenSource_ArtMaster/ArtMaster2.0Release.zip <span style="color: #B31B1B">'''''is downloadable here'''''</span>] with no support.====<br />
Unzip into (for example) the "Program Files" directory then set your system environment to include: ''C:\Program Files\Pam2.0 Release\jre\bin;C:\Program Files\Pam2.0 Release\bin;'' (You may need help for this. I right clicked 'computer' from the 'Start' menu, then selected 'Advanced system settings', then 'Environment Variables' and finally slid through the System variables until I found and selected 'Path'. This allowed me to edit the path by adding ';C:\Program Files\Pam2.0 Release\jre\bin;C:\Program Files\Pam2.0 Release\bin' to the end). Rather detailed help using the software is available in [http://cmpdartsvr1.cmp.uea.ac.uk/downloads/software/SieveWebPages/a4a_2_screensize.pdf <span style="color: Chocolate">''''this essay''''' </span>].<br><br><br />
[http://cmpdartsvr1.cmp.uea.ac.uk/downloads/software/SieveWebPages/Art_For_All_a4a_3_web2.pdf <span style="color: Chocolate">'''''The sieve algorithms underpinning PhotoArtMaster software''''' </span>] are described in an extract of the <br />
[http://cmpdartsvr1.cmp.uea.ac.uk/downloads/software/SieveWebPages/a4a_2_screensize.pdf <span style="color: Chocolate">''''essay''''' </span>]. These documents were written to support our Fo2Pix company. PhotoArtMaster originally sold >65,000 licences but ill health forced the closure of Fo2Pix.<br />
<br />
{| border="0" width=100% style="background-color:#ffffff;"<br />
|-<br />
|align="center"|[[Image:Trees_1.jpg|600px]]<br />
<br><br><br />
Illustration of PhotoArtMaster used to find and 'paint' with regions of colour crisply segmented from a photograph. <br />
|}<br />
<br />
<!-- [[Documentation_of_Connected_Set_Filters_or_Sieves|Art test page]]<br><br> <br />
[[Documentation_of_Connected_Set_Filters_or_Sieves]] --><br />
<br />
==<span style="color:Navy;">Reaction-diffusion and morphogenesis==<br />
{| border="0" width=100% style="background-color:#000000;"<br />
|-<br />
|align="center"|<br />
[[Image:tentacles_morphogenesis.png|600px]]<br />
|}<br />
Illustration of morphogenesis inspired by Turing's paper. <br><br><br />
[http://cmpdartsvr1.cmp.uea.ac.uk/downloads/software/GPT_ReactionDiffusionTentacles_20121211.zip <span style="color: Navy">Example using growth toolbox GPT_ReactionDiffusionTentacles_20121211.zip</span>]<br />
<br><br><br />
===1 A===<br />
<br />
=<span style="color:DarkGreen;">Open source systems to which we have contributed=<br />
==<span style="color:DarkGreen;">OMERO==<br />
{| border="0" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="10%"| <imgicon>OMERO_DIAGRAM.jpg|100px|OMERO</imgicon><br />
|width="40%"|For working with the OME image database. <br><br>See [http://www.openmicroscopy.org/site/products/omero ''Details''], [http://www.openmicroscopy.org/site/support/omero4/downloads ''Download'']<br> [http://dmbi.nbi.bbsrc.ac.uk/index.php/OMEROWorkshop OMERO Workshop] <br><br><br />
(Windows, Mac, Linux)<br />
|width="50%"| [http://openmicroscopy.org/site/support/omero4 Open Microscopy Environment Remote Objects (OMERO).] for visualising, managing, and annotating scientific image data. See also our [http://dmbi.nbi.bbsrc.ac.uk/index.php/OMEROWorkshop OMERO Workshop] training course we ran in April 2011.<br />
|}<br />
<br />
=<span style="color:DarkGreen;">Tools and Utilities=<br />
==<span style="color:DarkGreen;">BioformatsConverter==<br />
{| border="0" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="10%"| <imgicon>BioformatsConverterZip.png|100px|BioformatsConverter</imgicon><br />
|width="40%"|For converting microscope manufacturer proprietary file formats. <br><br>See [[BioformatsConverter|''Details'']]<br><br><br />
(Windows, Mac, Linux)<br />
|width="50%"| This tool allows for the batch conversion of microscope manufacturer proprietary file formats, to the open source OME-TIFF standard. Uses the [http://www.loci.wisc.edu/software/bio-formats Bioformats] library.<br />
|}<br />
==<span style="color:DarkGreen;">Dependency Checking Tool==<br />
<br />
Tool for recursively finding what further functions a function depends on. See [[myDepFun|''Details'']]<br />
<br />
=<span style="color:DarkGreen;">In development</span>=<br />
==<span style="color:DarkGreen;">MTtbox</span>==<br />
{| border="0" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="10%"| <imgicon>MTtboxA.jpg|100px|BioformatsConverter</imgicon><br />
|width="40%"|For modelling the behaviour of microtubules within a cell. <br><br><br />
See [[MTtbox documentation|''Details'']]<br><br><br />
(Windows, Mac, Linux)<br />
|width="50%"| In development. The idea is to be able to model the behaviour of growing microtubules and factors as they react chemically and diffuse within the different cell compartments.<br><br><br />
The icon shows a spherical cell sliced open to show concentric components: cell wall (magenta), plasma-membrane (yellow), cytoplasm (green) and vacuole (yellow). Microtubules (blue) grow in 3D within the cytoplasm.<br />
|}<br />
=<span style="color:Navy;">Historical</span>=<br />
==<span style="color:Navy;">Robot arm: still in production after 30 years (serving local industry)</span>==<br />
{| border="0" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="50%"|[[Image:2014-06-10 14.22.20 small robot.jpg|400px|]] <br />
|width="40%"|For teaching '''production control''' and '''interrupt programming'''. <br><br><br />
1983 and we are in a world of '''Apple II and BBC B computers''' - the ''6502'' processor reigns. Particularly good for real-time control it responded very fast to hardware interrupts from, for example, the timer. To illustrate timer interrupts what better than digital servo-motors? Set up the on-board timers to produce a stream of 'heartbeat' of pulses, one every 20 ms out of the parallel port and control up to eight motors. Pulse widths, from 1 to 2 ms, control the position of each motor arm. Derek Fulton and I made some loose lab. money by writing a series of articles showing exactly how to build and, in particular, control this robot arm.[[Publications#.28G.29_Computer_control.2C_measurement_and_commercial_software |Bangham et al.]]. <br />
Our copyright, I took it to a local company LJ-Electronics ([http://www.ljcreate.com/products/product.asp?id=314&program=195&curr=2 now LJ-Creative]) who incorporated it detail for detail into their product line. Originally, they called it the Emu. Still in production: '''Lovely outcome.'''<br />
|}<br />
<br />
=Historical=</div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Running_example_models_and_using_a_cluster&diff=5410Running example models and using a cluster2011-06-24T10:21:37Z<p>JacobNewman: </p>
<hr />
<div>[[GFtbox Tutorial pages | Back to GFtbox Tutorial pages]]<br />
<br />
The purpose of these tutorials is to learn how to run the example growth simulations included in GFtbox. We will describe methods for running simulations locally (on your own computer) and remotely (on a computing cluster). It is assumed that you have already downloaded the GFtbox software and have Matlab installed.<br />
<br />
=Getting Started=<br />
<br />
The remainder of this page is split into five sub-tutorials, each building on the preceding parts.<br />
<br />
1) '''Explaining the tools'''. In these tutorials, we will be using ''GFtboxCommand'' and ''ClusterMonitor''. This section explains the purpose of these tools. <br />
<br />
2) '''Computer or cluster?''' Here, we illustrate when and why you should use a computing cluster for your growth simulations, and conversely, when you should use your desktop computer.<br />
<br />
3) '''Running a growth simulation for an example model'''. This section demonstrates how to use ''GFtboxCommand'' to run a growth simulation on your computer. The model used for the simulation is an example model included with the GFtbox.<br />
<br />
4) '''Altering the simulation parameters'''. Following on from part 3, here we show how to adjust a simulation parameter within GFtboxCommand. Specifically, we alter the value of ''dt'', the temporal resolution of the simulation, and show how it can be used to verify that the value specified in the published literature is reasonable.<br />
<br />
5) '''Altering the model parameters on the cluster'''. Finally, we demonstrate how a number of model parameters can be varied by specifying a range of values for each model argument. We show how the computationally expensive task of simulating all combinations of specified ranges can be processed more efficiently if a computer cluster is used via the ''ClusterMonitor'' tool.<br />
<br />
='''1''' Explaining the tools=<br />
==GFtboxCommand==<br />
This is a command line version of the GFtbox. By command line, we mean that all program '''functions are operated via typed commands, without the GUI'''. Like GFtbox, GFtboxCommand is capable of running growth simulations of an interaction function, and allows the user to specify model and simulation parameters. Unlike GFtbox, this also allows the user to select '''ranges of values for a number of input parameters''', and will automatically spawn multiple simulations which explore the various combinations of those parameters. This can be used to evaluate the effect of various parameters on the growth of a given model.<br />
<br />
==ClusterMonitor==<br />
Provides a graphical user interface for managing simulations '''running remotely on a computer cluster'''. Specifically, it allows you to see which projects are present and running on the cluster, to retrieve the completed projects, to generate images of the simulations at specified stages of growth, and to remove projects from the cluster. If you do not intend to use a computer cluster, then you will not need to use ClusterMonitor.<br />
<br />
='''2''' Computer or cluster?=<br />
<br />
In basic terms, '''a computing cluster is effectively a network of many computer processors''' (often hundreds), centrally managed by a queuing system. When a job is submitted to a cluster, the job is sent to a processor that is not being used, or queued until one becomes available. In contrast, a typical desktop computer will contain one processor, limiting the number of tasks that can be performed at any one time. Jobs which must be run independently and sequentially on a desktop computer can be executed in parallel on a cluster, '''greatly reducing the total time required to complete all of the jobs'''. Although the exact details of your cluster might vary from those described in the rest of these tutorials, we aim to illustrate the generic processes involved in using GFtbox on a cluster. We are happy to offer assistance, where possible, to setup GFtbox for your cluster, so please contact us if you have any queries.<br />
<br />
Whilst the time savings offered by using a cluster can be significant, there is an overhead associated with returning the results to your personal computer. The total time to run 2 or more simulations on a cluster and to return the results will be less than running those simulations sequentially on one computer. Therefore, using a '''cluster is ideal for situations where you would like to run several simulations''', such as to evaluate the effect of a range of parameters on a growth model. You are not advised to use a cluster for running single simulations, or where you would like to step through a simulation and change parameters in a more interactive fashion. In such circumstances, you are advised to run GFtbox or GFtboxCommand on your own computer.<br />
<br />
The '''GFtbox GUI provides quick feedback''' about how the changes you have made to an interaction function have affected the growth simulation, as '''you can see the result of each simulation iteration as it completes'''. It is quick and easy therefore to see if you have dramatically changed the course of the growth simulation, and then to adjust the parameters according to your observations. This approach is well suited to the early stages of design, where you might wish to '''tweak some parameters to gauge whether or not they have had the desired effect'''. Or in the final stages, where you think your model is almost finalised, but where small adjustments are required. '''An alternative approach is to start many simulations''', based upon your initial model and by making intelligent choices about which parameters to explore. This allows you to '''harvest many results, to quickly and easily overview them''' in their finalised form, or to select interesting-looking simulations and examine them more closely on your desktop computer, using GFtbox.<br />
<br />
='''3''' Running a growth simulation for an example model=<br />
This tutorial is aimed at running a '''growth simulation for one of the example interaction functions''' included with the GFtbox. The purpose of this exercise is to firstly demonstrate how simulations can be invoked using GFtboxCommand, and secondly to show how to reproduce experimental results (specifically, those published in Kennaway et al (2011)) given an interaction function.<br />
<br />
Assuming Matlab is installed on your computer, and the latest GFtbox has been downloaded, you can add the GFtbox directory to Matlab's search path, which makes the toolbox accessible from any other path that you choose to work from. For a short tutorial on how to do this, please click [[Adding GFtbox to Matlab's search path | here]].<br />
<br />
Once the GFtbox is added to Matlab, you are ready to run a growth simulation using GFtboxCommand. In this example, '''the model that we will simulate is called GPT_CASE_RST'''. Results generated using this model are published in Kennaway et al (2011). By running this simulation, we can confirm the results in the published literature and investigate the suitability of the various parameters.<br />
<br />
===3A - Running a simulation on your computer===<br />
<br />
The following command can be typed into Matlab to run a simulation of the GPT_CASE_RST interaction function, which contains three growth models: R, S and T. '''Three separate simulations are run sequentially on your computer''', one for each model, each producing results corresponding to five intervals in the growth simulation.<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',[1:3]);<br />
<br />
GFtboxCommand accepts input arguments as name and value pairs, e.g. 'modelname', [1:3] or 'Use','Cluster'. The argument names entered in the example above are: ''Path'', ''Name'', ''Stages'' and ''modelname''.<br />
<br />
The optional '''''Path'' argument name refers to the location of the folder''' (or directory) on your computer where the model interaction function you wish to simulate is stored. '''''Name'', is the name of the folder itself'''. In this case, we are using the GPT_CASE_RST folder which is included in the GrowthToolbox. You may wish to copy this folder elsewhere, if you intend to make changes to the interaction function.<br />
<br />
During a growth simulation, a mesh can be generated at each time step of the simulation, which provides a visual representation of the growth of the biological tissue, given the various parameters of growth specified by the interaction function and how they have changed over time. Put another way, the mesh shows exactly what the growing tissue actually looks like. ''Stages'' refers to the points in the simulation (measured in hours) at which meshes should be generated and saved. In this example, '''five stages of growth will be written to disk'''. These values are chosen to best capture the appearance of the tissue at important stages of the tissue growth.<br />
<br />
The final argument name listed here is ''modelname''. This is a model-specific argument, and in this case the GPT_CASE_RST interaction function contains '''three separate models for plant growth''', allowing the desired model to be a selected. Here, the value [1:3] is specified, which is evaluated in the same way as entering [1 2 3]. This instructs GFtboxCommand to run three separate simulations, one for each of the growth models contained in the interaction function.<br />
<br />
The function of every permissible argument is given by keying the following command into Matlab:<br />
<br />
help GFtboxCommand<br />
<br />
===3B - Getting results from a completed simulation===<br />
<br />
'''1) - Generating images from a simulation you ran on your computer'''<br />
<br />
Once a growth simulation is completed, the project folder (GPT_CASE_RST, in our example) will contain another folder named "movies". Within movies, there are folders which contain results for the executed simulations. As was instructed in 3A, here we can see three folders corresponding to simulations for the three separate models. Within the first folder there are three items:<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| ''CommandLine.txt'' - This file contains the Matlab command which was used to generate the results that this sub-directory contains.<br />
<br />
''gpt_case_rst.txt'' - This file, named according to the project name, contains a copy of the interaction function which was used to produce these results.<br />
<br />
''meshes'' - This folder contains the mesh files corresponding to the stages of growth specified by the value of the Stages argument, described in 3A.<br />
|width="500px" align="center"|[[File:resultsdir.png| Directory structure of a project containing results]]<br />
|}<br />
<br />
<br />
The mesh files contain vertices information regarding the shape of the growth model at a particular stage of growth, but are not visualisable in that form. In order to '''convert these mesh files into viewable images''', we can execute the following command in Matlab:<br />
<br />
VMSreport('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Project','GPT_CASE_RST',...<br />
'Experiment','All','flattentime',572.5,'morphogen','KPAR','SNAPFIG',true);<br />
<br />
Where the ''Path'' and ''Project'' arguments have the same function as ''Path'' and ''Name'' in 3A, i.e. ''Path'' refers to the location of the project folder and ''Project'' is the name of the project folder itself. The images that are created will be stored in the ''meshes'' directory within the project folder.<br />
<br />
Here we can see the generated images, including the command line arguments, for one of the simulations performed in 3A. Click on a thumbnail to view a larger image.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="125px"| <small>001commandline.png</small><br> [[File:commandlineexample.png | 100px]]<br />
|width="125px"| <small>001GPT_CASE_RST_2 wild_.png</small><br>[[File:GPT_CASE_RST_example1.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_3 wild_.png</small><br>[[File:GPT_CASE_RST_example2.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_4 wild_.png</small><br>[[File:GPT_CASE_RST_example3.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_5 wild_.png</small><br>[[File:GPT_CASE_RST_example4.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_6 wild_.png</small><br>[[File:GPT_CASE_RST_example5.png | 125px]]<br />
|}<br />
<br />
NB. The results produced may not be visibly identical to those in the published literature. This is because of small, random perturbations which are applied to the initial model meshes to stop them from containing surfaces which are perfectly flat, and therefore biologically unrealistic. The results produced should be '''qualitatively but not quantitatively the same'''.<br />
<br />
'''2) - Generating images from a simulation you ran on the cluster'''<br />
<br />
Another way to generate images from results is by using the ClusterMonitor tool. When a cluster is used to run simulations, the result files are stored remotely, on the cluster. These files must be retrieved, and then the 'Make project pngs' button can be pressed to generate images, which performs the same function as ''VMSReport'' in part 3A1. This process is described fully in part 5.<br />
<br />
'''3) - Interacting with your results using the GFtbox GUI'''<br />
<br />
Once result mesh files have been created, either on your local computer (using ''GFtboxCommand'') or from retrieved files that were generated on a cluster, it is also possible to visualise the results in the GFtbox GUI.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| <br />
|width="600px"| [[File:examplegftboxgui.png| 200px]]<br />
|}<br />
<br />
='''4''' Altering the simulation parameters=<br />
<br />
One such simulation parameter is ''dt'', which is the time in seconds between iterations in the growth simulation. Large values of ''dt'' mean that fewer steps and therefore fewer calculations are required to complete a simulation. Whereas smaller values of ''dt'' mean that more steps are required, and therefore more processing time too. Therefore, a value of ''dt'' must be selected which is not so small that it is computationally unmanageable, but not so large that the observed growth is an artifact of the value of ''dt'', rather than the underlying growth model. It is necessary therefore to test a range of values for ''dt'' to ensure that the patterns of growth observed in a simulation are consistent across the range, and to find a value which is sufficient to demonstrate the model of growth and computationally efficient.<br />
<br />
'''Testing a range of ''dt''s can be achieved in several ways.''' One way is to use the ''dt'' argument when calling GFtboxCommand. This allows a single value to be tested. Another way is to make a batch of jobs, each using a different value for ''dt'', by using the 'State' argument. Lastly, a range of ''dt''s can be specified in GFtboxCommand, by using the dt argument where the values within the square brackets are the ''dt''s to simulate:<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5]);<br />
<br />
Here, we have specified '''four separate values for ''dt''''', and this means that '''four separate simulations will be run''' sequentially on your computer. As in 3B, images can then be generated for the mesh files produced, and compared to ensure consistency across the range of ''dt''s. In this figure, we can see the same model, at the same stage of growth, generated using the four values of ''dt''. Though quantitatively different, they are qualitatively the same, illustrating the suitability of the default ''dt'' value of 5.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="190px"| [[File:GPT_CASE_RST_dt0.5.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt1.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt2.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt5.png| 190px]]<br />
|}<br />
<br />
='''5''' Altering the model parameters on the cluster=<br />
It is easy to see that when even a small number of range variables, mutations or ''dt''s are specified, the '''total number of simulations increases quickly'''. If many ranges are specified, then the amount of processing time becomes unmanageable on a single computer. Via ''GFtboxCommand'' and the ''ClusterMonitor'' tool, we can remotely''' run a number of simulations, in parallel, on a computing cluster'''. It is assumed that the GrowthToolbox and Matlab are installed on your cluster, and PuTTY is required on your local computer for the tools ''pscp'' (for transferring files from your computer to a cluster) and ''plink'' (for remotely executing commands on a cluster). Using the ''dt'' range argument from 4 as an example, here we add the 'Use' name argument with the value 'Cluster':<br />
<br />
GFtboxCommand('State','Start','Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5],'Use','Cluster');<br />
<br />
The 'Cluster' option for the 'Use' name argument instructs GFtboxCommand to upload the required project directory to a remote Linux server. Instead of running the simulation on your own computer, GFtboxCommand then works out how many individual simulations are specified by the command that invoked it. In this example, the only argument which will generate multiple simulations is ''dt'', of which there will be 4. Separate commands for each of these jobs are then automatically generated, each with an accompanying unique ID (The value of the ''ExpID'' name argument), and '''these are submitted as individual jobs to the cluster'''.<br />
<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.1,'ExpID','GPT_CASE_RST_1');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.5,'ExpID','GPT_CASE_RST_2');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',1,'ExpID','GPT_CASE_RST_3');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',5,'ExpID','GPT_CASE_RST_4');<br />
<br />
Once these jobs have been submitted, the ''ClusterMonitor'' tool opens and the new job batch ID is added to the list of jobs. Here, we will concentrate on three of the functions of ''ClusterMonitor'': ''Queue?'', ''Get project results'' and ''Make project pngs''. As in part 4 of this page, these functions will enable us to visualise the growth simulation results.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| This is an example of what you will see when the ClusterMonitor tool launches after submitting a job using the 'Use','Cluster' argument pair in GFtboxCommand.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="middle"<br />
|width="200px"| Clicking the 'Queue?' button tells ClusterMonitor to check the status of your jobs on the cluster. It then prints the status of the current jobs in Matlab's command window. In the case of our cluster, this is the equivalent of the ''qstatme'' command. This screenshot shows an example job list, where two jobs are currently running.<br />
|width="600px"| [[File:queue.png| 600px]]<br />
|}<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Once your jobs have completed, the 'Queue?' button will show that there are no jobs running. Now you are ready to retrieve the project files from the cluster. This is achieved using the 'Get project results' button. Once pressed, the Matlab command window will show trace information to indicate which files are currently being downloaded. This is the equivalent of downloading the files manually, using an FTP program.<br />
|width="600px" align="center"| [[File:downloadingprojectfiles.png]]<br />
|}<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Once you have successfully downloaded your project results, you are ready to generate result images, as in part 3B1. Instead of entering a ''VMSReport'' command, you can press the 'Make project pngs' button, which will perform the same command. The images that are created will be in the ''meshes'' directory within the project folder.<br />
|width="600px" align="center"| [[File:meshesfolder.png]]<br />
|}<br />
<br />
<br />
Whilst ''ClusterMonitor'' is a useful tool for managing your cluster jobs, it is advisable to '''have at least a rudimentary understanding of and ability to use a Unix-based computer system'''. In particular, the abilities to list the contents of a folder, view the contents of a file, change the current working folder, delete, and to copy or move files or folders, are essential Unix skills for making sure everything is working as intended. It is beyond the scope of these tutorials to provide an in depth Unix tutorial (many good and simple tutorials exist on the web), but here is a short description of the Unix commands (which may be specific to our Unix-based system) that we believe to be important.<br />
<br />
''ls'' - list the files and folders in a directory. Use the -al option (e.g. ls -al) to see modification dates, file permissions, sizes, etc.<br />
<br />
''cd X'' - Change the current directory to directory X.<br />
<br />
''rm X'' - Delete the file X.<br />
<br />
''cp X NewPath/X2'' - Copy file X to the NewPath directory and name the copy X2.<br />
<br />
''mv X NewPath/X2'' - Move file X to the NewPath directory and rename it to X2.<br />
<br />
''cat test.txt'' - Print the contents of the text file test.txt onto the screen.<br />
<br />
''pwd'' - Print the present working directory onto the screen.<br />
<br />
''more test.txt'' - Print the file test.txt to the screen, a screenful at a time, scrolling each time you hit the enter key.</div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=File:Examplegftboxgui.png&diff=5517File:Examplegftboxgui.png2011-06-24T10:20:06Z<p>JacobNewman: </p>
<hr />
<div></div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Running_example_models_and_using_a_cluster&diff=5400Running example models and using a cluster2011-06-24T09:54:27Z<p>JacobNewman: </p>
<hr />
<div>[[GFtbox Tutorial pages | Back to GFtbox Tutorial pages]]<br />
<br />
The purpose of these tutorials is to learn how to run the example growth simulations included in GFtbox. We will describe methods for running simulations locally (on your own computer) and remotely (on a computing cluster). It is assumed that you have already downloaded the GFtbox software and have Matlab installed.<br />
<br />
=Getting Started=<br />
<br />
The remainder of this page is split into five sub-tutorials, each building on the preceding parts.<br />
<br />
1) '''Explaining the tools'''. In these tutorials, we will be using ''GFtboxCommand'' and ''ClusterMonitor''. This section explains the purpose of these tools. <br />
<br />
2) '''Computer or cluster?''' Here, we illustrate when and why you should use a computing cluster for your growth simulations, and conversely, when you should use your desktop computer.<br />
<br />
3) '''Running a growth simulation for an example model'''. This section demonstrates how to use ''GFtboxCommand'' to run a growth simulation on your computer. The model used for the simulation is an example model included with the GFtbox.<br />
<br />
4) '''Altering the simulation parameters'''. Following on from part 3, here we show how to adjust a simulation parameter within GFtboxCommand. Specifically, we alter the value of ''dt'', the temporal resolution of the simulation, and show how it can be used to verify that the value specified in the published literature is reasonable.<br />
<br />
5) '''Altering the model parameters on the cluster'''. Finally, we demonstrate how a number of model parameters can be varied by specifying a range of values for each model argument. We show how the computationally expensive task of simulating all combinations of specified ranges can be processed more efficiently if a computer cluster is used via the ''ClusterMonitor'' tool.<br />
<br />
='''1''' Explaining the tools=<br />
==GFtboxCommand==<br />
This is a command line version of the GFtbox. By command line, we mean that all program '''functions are operated via typed commands, without the GUI'''. Like GFtbox, GFtboxCommand is capable of running growth simulations of an interaction function, and allows the user to specify model and simulation parameters. Unlike GFtbox, this also allows the user to select '''ranges of values for a number of input parameters''', and will automatically spawn multiple simulations which explore the various combinations of those parameters. This can be used to evaluate the effect of various parameters on the growth of a given model.<br />
<br />
==ClusterMonitor==<br />
Provides a graphical user interface for managing simulations '''running remotely on a computer cluster'''. Specifically, it allows you to see which projects are present and running on the cluster, to retrieve the completed projects, to generate images of the simulations at specified stages of growth, and to remove projects from the cluster. If you do not intend to use a computer cluster, then you will not need to use ClusterMonitor.<br />
<br />
='''2''' Computer or cluster?=<br />
<br />
In basic terms, '''a computing cluster is effectively a network of many computer processors''' (often hundreds), centrally managed by a queuing system. When a job is submitted to a cluster, the job is sent to a processor that is not being used, or queued until one becomes available. In contrast, a typical desktop computer will contain one processor, limiting the number of tasks that can be performed at any one time. Jobs which must be run independently and sequentially on a desktop computer can be executed in parallel on a cluster, '''greatly reducing the total time required to complete all of the jobs'''. Although the exact details of your cluster might vary from those described in the rest of these tutorials, we aim to illustrate the generic processes involved in using GFtbox on a cluster. We are happy to offer assistance, where possible, to setup GFtbox for your cluster, so please contact us if you have any queries.<br />
<br />
Whilst the time savings offered by using a cluster can be significant, there is an overhead associated with returning the results to your personal computer. The total time to run 2 or more simulations on a cluster and to return the results will be less than running those simulations sequentially on one computer. Therefore, using a '''cluster is ideal for situations where you would like to run several simulations''', such as to evaluate the effect of a range of parameters on a growth model. You are not advised to use a cluster for running single simulations, or where you would like to step through a simulation and change parameters in a more interactive fashion. In such circumstances, you are advised to run GFtbox or GFtboxCommand on your own computer.<br />
<br />
The '''GFtbox GUI provides quick feedback''' about how the changes you have made to an interaction function have affected the growth simulation, as '''you can see the result of each simulation iteration as it completes'''. It is quick and easy therefore to see if you have dramatically changed the course of the growth simulation, and then to adjust the parameters according to your observations. This approach is well suited to the early stages of design, where you might wish to '''tweak some parameters to gauge whether or not they have had the desired effect'''. Or in the final stages, where you think your model is almost finalised, but where small adjustments are required. '''An alternative approach is to start many simulations''', based upon your initial model and by making intelligent choices about which parameters to explore. This allows you to '''harvest many results, to quickly and easily overview them''' in their finalised form, or to select interesting-looking simulations and examine them more closely on your desktop computer, using GFtbox.<br />
<br />
='''3''' Running a growth simulation for an example model=<br />
This tutorial is aimed at running a '''growth simulation for one of the example interaction functions''' included with the GFtbox. The purpose of this exercise is to firstly demonstrate how simulations can be invoked using GFtboxCommand, and secondly to show how to reproduce experimental results (specifically, those published in Kennaway et al (2011)) given an interaction function.<br />
<br />
Assuming Matlab is installed on your computer, and the latest GFtbox has been downloaded, you can add the GFtbox directory to Matlab's search path, which makes the toolbox accessible from any other path that you choose to work from. For a short tutorial on how to do this, please click [[Adding GFtbox to Matlab's search path | here]].<br />
<br />
Once the GFtbox is added to Matlab, you are ready to run a growth simulation using GFtboxCommand. In this example, '''the model that we will simulate is called GPT_CASE_RST'''. Results generated using this model are published in Kennaway et al (2011). By running this simulation, we can confirm the results in the published literature and investigate the suitability of the various parameters.<br />
<br />
===3A - Running a simulation on your computer===<br />
<br />
The following command can be typed into Matlab to run a simulation of the GPT_CASE_RST interaction function, which contains three growth models: R, S and T. '''Three separate simulations are run sequentially on your computer''', one for each model, each producing results corresponding to five intervals in the growth simulation.<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',[1:3]);<br />
<br />
GFtboxCommand accepts input arguments as name and value pairs, e.g. 'modelname', [1:3] or 'Use','Cluster'. The argument names entered in the example above are: ''Path'', ''Name'', ''Stages'' and ''modelname''.<br />
<br />
The optional '''''Path'' argument name refers to the location of the folder''' (or directory) on your computer where the model interaction function you wish to simulate is stored. '''''Name'', is the name of the folder itself'''. In this case, we are using the GPT_CASE_RST folder which is included in the GrowthToolbox. You may wish to copy this folder elsewhere, if you intend to make changes to the interaction function.<br />
<br />
During a growth simulation, a mesh can be generated at each time step of the simulation, which provides a visual representation of the growth of the biological tissue, given the various parameters of growth specified by the interaction function and how they have changed over time. Put another way, the mesh shows exactly what the growing tissue actually looks like. ''Stages'' refers to the points in the simulation (measured in hours) at which meshes should be generated and saved. In this example, '''five stages of growth will be written to disk'''. These values are chosen to best capture the appearance of the tissue at important stages of the tissue growth.<br />
<br />
The final argument name listed here is ''modelname''. This is a model-specific argument, and in this case the GPT_CASE_RST interaction function contains '''three separate models for plant growth''', allowing the desired model to be a selected. Here, the value [1:3] is specified, which is evaluated in the same way as entering [1 2 3]. This instructs GFtboxCommand to run three separate simulations, one for each of the growth models contained in the interaction function.<br />
<br />
The function of every permissible argument is given by keying the following command into Matlab:<br />
<br />
help GFtboxCommand<br />
<br />
===3B - Getting results from a completed simulation===<br />
<br />
'''1) - Generating images from a simulation you ran on your computer'''<br />
<br />
Once a growth simulation is completed, the project folder (GPT_CASE_RST, in our example) will contain another folder named "movies". Within movies, there are folders which contain results for the executed simulations. As was instructed in 3A, here we can see three folders corresponding to simulations for the three separate models. Within the first folder there are three items:<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| ''CommandLine.txt'' - This file contains the Matlab command which was used to generate the results that this sub-directory contains.<br />
<br />
''gpt_case_rst.txt'' - This file, named according to the project name, contains a copy of the interaction function which was used to produce these results.<br />
<br />
''meshes'' - This folder contains the mesh files corresponding to the stages of growth specified by the value of the Stages argument, described in 3A.<br />
|width="500px" align="center"|[[File:resultsdir.png| Directory structure of a project containing results]]<br />
|}<br />
<br />
<br />
The mesh files contain vertices information regarding the shape of the growth model at a particular stage of growth, but are not visualisable in that form. In order to '''convert these mesh files into viewable images''', we can execute the following command in Matlab:<br />
<br />
VMSreport('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Project','GPT_CASE_RST',...<br />
'Experiment','All','flattentime',572.5,'morphogen','KPAR','SNAPFIG',true);<br />
<br />
Where the ''Path'' and ''Project'' arguments have the same function as ''Path'' and ''Name'' in 3A, i.e. ''Path'' refers to the location of the project folder and ''Project'' is the name of the project folder itself. The images that are created will be stored in the ''meshes'' directory within the project folder.<br />
<br />
Here we can see the generated images, including the command line arguments, for one of the simulations performed in 3A. Click on a thumbnail to view a larger image.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="125px"| <small>001commandline.png</small><br> [[File:commandlineexample.png | 100px]]<br />
|width="125px"| <small>001GPT_CASE_RST_2 wild_.png</small><br>[[File:GPT_CASE_RST_example1.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_3 wild_.png</small><br>[[File:GPT_CASE_RST_example2.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_4 wild_.png</small><br>[[File:GPT_CASE_RST_example3.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_5 wild_.png</small><br>[[File:GPT_CASE_RST_example4.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_6 wild_.png</small><br>[[File:GPT_CASE_RST_example5.png | 125px]]<br />
|}<br />
<br />
NB. The results produced may not be visibly identical to those in the published literature. This is because of small, random perturbations which are applied to the initial model meshes to stop them from containing surfaces which are perfectly flat, and therefore biologically unrealistic. The results produced should be '''qualitatively but not quantitatively the same'''.<br />
<br />
'''2) - Generating images from a simulation you ran on the cluster'''<br />
<br />
Another way to generate images from results is by using the ClusterMonitor tool. When a cluster is used to run simulations, the result files are stored remotely, on the cluster. These files must be retrieved, and then the 'Make project pngs' button can be pressed to generate images, which performs the same function as ''VMSReport'' in part 3A1. This process is described fully in part 5.<br />
<br />
'''3) - Interacting with your results using the GFtbox GUI'''<br />
<br />
Once result mesh files have been created, either on your local computer (using ''GFtboxCommand'') or from retrieved files that were generated on a cluster, it is also possible to visualise the results in the GFtbox GUI.<br />
<br />
='''4''' Altering the simulation parameters=<br />
<br />
One such simulation parameter is ''dt'', which is the time in seconds between iterations in the growth simulation. Large values of ''dt'' mean that fewer steps and therefore fewer calculations are required to complete a simulation. Whereas smaller values of ''dt'' mean that more steps are required, and therefore more processing time too. Therefore, a value of ''dt'' must be selected which is not so small that it is computationally unmanageable, but not so large that the observed growth is an artifact of the value of ''dt'', rather than the underlying growth model. It is necessary therefore to test a range of values for ''dt'' to ensure that the patterns of growth observed in a simulation are consistent across the range, and to find a value which is sufficient to demonstrate the model of growth and computationally efficient.<br />
<br />
'''Testing a range of ''dt''s can be achieved in several ways.''' One way is to use the ''dt'' argument when calling GFtboxCommand. This allows a single value to be tested. Another way is to make a batch of jobs, each using a different value for ''dt'', by using the 'State' argument. Lastly, a range of ''dt''s can be specified in GFtboxCommand, by using the dt argument where the values within the square brackets are the ''dt''s to simulate:<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5]);<br />
<br />
Here, we have specified '''four separate values for ''dt''''', and this means that '''four separate simulations will be run''' sequentially on your computer. As in 3B, images can then be generated for the mesh files produced, and compared to ensure consistency across the range of ''dt''s. In this figure, we can see the same model, at the same stage of growth, generated using the four values of ''dt''. Though quantitatively different, they are qualitatively the same, illustrating the suitability of the default ''dt'' value of 5.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="190px"| [[File:GPT_CASE_RST_dt0.5.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt1.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt2.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt5.png| 190px]]<br />
|}<br />
<br />
='''5''' Altering the model parameters on the cluster=<br />
It is easy to see that when even a small number of range variables, mutations or ''dt''s are specified, the '''total number of simulations increases quickly'''. If many ranges are specified, then the amount of processing time becomes unmanageable on a single computer. Via ''GFtboxCommand'' and the ''ClusterMonitor'' tool, we can remotely''' run a number of simulations, in parallel, on a computing cluster'''. It is assumed that the GrowthToolbox and Matlab are installed on your cluster, and PuTTY is required on your local computer for the tools ''pscp'' (for transferring files from your computer to a cluster) and ''plink'' (for remotely executing commands on a cluster). Using the ''dt'' range argument from 4 as an example, here we add the 'Use' name argument with the value 'Cluster':<br />
<br />
GFtboxCommand('State','Start','Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5],'Use','Cluster');<br />
<br />
The 'Cluster' option for the 'Use' name argument instructs GFtboxCommand to upload the required project directory to a remote Linux server. Instead of running the simulation on your own computer, GFtboxCommand then works out how many individual simulations are specified by the command that invoked it. In this example, the only argument which will generate multiple simulations is ''dt'', of which there will be 4. Separate commands for each of these jobs are then automatically generated, each with an accompanying unique ID (The value of the ''ExpID'' name argument), and '''these are submitted as individual jobs to the cluster'''.<br />
<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.1,'ExpID','GPT_CASE_RST_1');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.5,'ExpID','GPT_CASE_RST_2');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',1,'ExpID','GPT_CASE_RST_3');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',5,'ExpID','GPT_CASE_RST_4');<br />
<br />
Once these jobs have been submitted, the ''ClusterMonitor'' tool opens and the new job batch ID is added to the list of jobs. Here, we will concentrate on three of the functions of ''ClusterMonitor'': ''Queue?'', ''Get project results'' and ''Make project pngs''. As in part 4 of this page, these functions will enable us to visualise the growth simulation results.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| This is an example of what you will see when the ClusterMonitor tool launches after submitting a job using the 'Use','Cluster' argument pair in GFtboxCommand.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="middle"<br />
|width="200px"| Clicking the 'Queue?' button tells ClusterMonitor to check the status of your jobs on the cluster. It then prints the status of the current jobs in Matlab's command window. In the case of our cluster, this is the equivalent of the ''qstatme'' command. This screenshot shows an example job list, where two jobs are currently running.<br />
|width="600px"| [[File:queue.png| 600px]]<br />
|}<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Once your jobs have completed, the 'Queue?' button will show that there are no jobs running. Now you are ready to retrieve the project files from the cluster. This is achieved using the 'Get project results' button. Once pressed, the Matlab command window will show trace information to indicate which files are currently being downloaded. This is the equivalent of downloading the files manually, using an FTP program.<br />
|width="600px" align="center"| [[File:downloadingprojectfiles.png]]<br />
|}<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Once you have successfully downloaded your project results, you are ready to generate result images, as in part 3B1. Instead of entering a ''VMSReport'' command, you can press the 'Make project pngs' button, which will perform the same command. The images that are created will be in the ''meshes'' directory within the project folder.<br />
|width="600px" align="center"| [[File:meshesfolder.png]]<br />
|}<br />
<br />
<br />
Whilst ''ClusterMonitor'' is a useful tool for managing your cluster jobs, it is advisable to '''have at least a rudimentary understanding of and ability to use a Unix-based computer system'''. In particular, the abilities to list the contents of a folder, view the contents of a file, change the current working folder, delete, and to copy or move files or folders, are essential Unix skills for making sure everything is working as intended. It is beyond the scope of these tutorials to provide an in depth Unix tutorial (many good and simple tutorials exist on the web), but here is a short description of the Unix commands (which may be specific to our Unix-based system) that we believe to be important.<br />
<br />
''ls'' - list the files and folders in a directory. Use the -al option (e.g. ls -al) to see modification dates, file permissions, sizes, etc.<br />
<br />
''cd X'' - Change the current directory to directory X.<br />
<br />
''rm X'' - Delete the file X.<br />
<br />
''cp X NewPath/X2'' - Copy file X to the NewPath directory and name the copy X2.<br />
<br />
''mv X NewPath/X2'' - Move file X to the NewPath directory and rename it to X2.<br />
<br />
''cat test.txt'' - Print the contents of the text file test.txt onto the screen.<br />
<br />
''pwd'' - Print the present working directory onto the screen.<br />
<br />
''more test.txt'' - Print the file test.txt to the screen, a screenful at a time, scrolling each time you hit the enter key.</div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Running_example_models_and_using_a_cluster&diff=5399Running example models and using a cluster2011-06-24T09:43:54Z<p>JacobNewman: </p>
<hr />
<div>[[GFtbox Tutorial pages | Back to GFtbox Tutorial pages]]<br />
<br />
The purpose of these tutorials is to learn how to run the example growth simulations included in GFtbox. We will describe methods for running simulations locally (on your own computer) and remotely (on a computing cluster). It is assumed that you have already downloaded the GFtbox software and have Matlab installed.<br />
<br />
=Getting Started=<br />
<br />
The remainder of this page is split into five sub-tutorials, each building on the preceding parts.<br />
<br />
1) '''Explaining the tools'''. In these tutorials, we will be using ''GFtboxCommand'' and ''ClusterMonitor''. This section explains the purpose of these tools. <br />
<br />
2) '''Computer or cluster?''' Here, we illustrate when and why you should use a computing cluster for your growth simulations, and conversely, when you should use your desktop computer.<br />
<br />
3) '''Running a growth simulation for an example model'''. This section demonstrates how to use ''GFtboxCommand'' to run a growth simulation on your computer. The model used for the simulation is an example model included with the GFtbox.<br />
<br />
4) '''Altering the simulation parameters'''. Following on from part 3, here we show how to adjust a simulation parameter within GFtboxCommand. Specifically, we alter the value of ''dt'', the temporal resolution of the simulation, and show how it can be used to verify that the value specified in the published literature is reasonable.<br />
<br />
5) '''Altering the model parameters on the cluster'''. Finally, we demonstrate how a number of model parameters can be varied by specifying a range of values for each model argument. We show how the computationally expensive task of simulating all combinations of specified ranges can be processed more efficiently if a computer cluster is used via the ''ClusterMonitor'' tool.<br />
<br />
='''1''' Explaining the tools=<br />
==GFtboxCommand==<br />
This is a command line version of the GFtbox. By command line, we mean that all program '''functions are operated via typed commands, without the GUI'''. Like GFtbox, GFtboxCommand is capable of running growth simulations of an interaction function, and allows the user to specify model and simulation parameters. Unlike GFtbox, this also allows the user to select '''ranges of values for a number of input parameters''', and will automatically spawn multiple simulations which explore the various combinations of those parameters. This can be used to evaluate the effect of various parameters on the growth of a given model.<br />
<br />
==ClusterMonitor==<br />
Provides a graphical user interface for managing simulations '''running remotely on a computer cluster'''. Specifically, it allows you to see which projects are present and running on the cluster, to retrieve the completed projects, to generate images of the simulations at specified stages of growth, and to remove projects from the cluster. If you do not intend to use a computer cluster, then you will not need to use ClusterMonitor.<br />
<br />
='''2''' Computer or cluster?=<br />
<br />
In basic terms, '''a computing cluster is effectively a network of many computer processors''' (often hundreds), centrally managed by a queuing system. When a job is submitted to a cluster, the job is sent to a processor that is not being used, or queued until one becomes available. In contrast, a typical desktop computer will contain one processor, limiting the number of tasks that can be performed at any one time. Jobs which must be run independently and sequentially on a desktop computer can be executed in parallel on a cluster, '''greatly reducing the total time required to complete all of the jobs'''. Although the exact details of your cluster might vary from those described in the rest of these tutorials, we aim to illustrate the generic processes involved in using GFtbox on a cluster. We are happy to offer assistance, where possible, to setup GFtbox for your cluster, so please contact us if you have any queries.<br />
<br />
Whilst the time savings offered by using a cluster can be significant, there is an overhead associated with returning the results to your personal computer. The total time to run 2 or more simulations on a cluster and to return the results will be less than running those simulations sequentially on one computer. Therefore, using a '''cluster is ideal for situations where you would like to run several simulations''', such as to evaluate the effect of a range of parameters on a growth model. You are not advised to use a cluster for running single simulations, or where you would like to step through a simulation and change parameters in a more interactive fashion. In such circumstances, you are advised to run GFtbox or GFtboxCommand on your own computer.<br />
<br />
The '''GFtbox GUI provides quick feedback''' about how the changes you have made to an interaction function have affected the growth simulation, as '''you can see the result of each simulation iteration as it completes'''. It is quick and easy therefore to see if you have dramatically changed the course of the growth simulation, and then to adjust the parameters according to your observations. This approach is well suited to the early stages of design, where you might wish to '''tweak some parameters to gauge whether or not they have had the desired effect'''. Or in the final stages, where you think your model is almost finalised, but where small adjustments are required. '''An alternative approach is to start many simulations''', based upon your initial model and by making intelligent choices about which parameters to explore. This allows you to '''harvest many results, to quickly and easily overview them''' in their finalised form, or to select interesting-looking simulations and examine them more closely on your desktop computer, using GFtbox.<br />
<br />
='''3''' Running a growth simulation for an example model=<br />
This tutorial is aimed at running a '''growth simulation for one of the example interaction functions''' included with the GFtbox. The purpose of this exercise is to firstly demonstrate how simulations can be invoked using GFtboxCommand, and secondly to show how to reproduce experimental results (specifically, those published in Kennaway et al (2011)) given an interaction function.<br />
<br />
Assuming Matlab is installed on your computer, and the latest GFtbox has been downloaded, you can add the GFtbox directory to Matlab's search path, which makes the toolbox accessible from any other path that you choose to work from. For a short tutorial on how to do this, please click [[Adding GFtbox to Matlab's search path | here]].<br />
<br />
Once the GFtbox is added to Matlab, you are ready to run a growth simulation using GFtboxCommand. In this example, '''the model that we will simulate is called GPT_CASE_RST'''. Results generated using this model are published in Kennaway et al (2011). By running this simulation, we can confirm the results in the published literature and investigate the suitability of the various parameters.<br />
<br />
===3A - Running a simulation on your computer===<br />
<br />
The following command can be typed into Matlab to run a simulation of the GPT_CASE_RST interaction function, which contains three growth models: R, S and T. '''Three separate simulations are run sequentially on your computer''', one for each model, each producing results corresponding to five intervals in the growth simulation.<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',[1:3]);<br />
<br />
GFtboxCommand accepts input arguments as name and value pairs, e.g. 'modelname', [1:3] or 'Use','Cluster'. The argument names entered in the example above are: ''Path'', ''Name'', ''Stages'' and ''modelname''.<br />
<br />
The optional '''''Path'' argument name refers to the location of the folder''' (or directory) on your computer where the model interaction function you wish to simulate is stored. '''''Name'', is the name of the folder itself'''. In this case, we are using the GPT_CASE_RST folder which is included in the GrowthToolbox. You may wish to copy this folder elsewhere, if you intend to make changes to the interaction function.<br />
<br />
During a growth simulation, a mesh can be generated at each time step of the simulation, which provides a visual representation of the growth of the biological tissue, given the various parameters of growth specified by the interaction function and how they have changed over time. Put another way, the mesh shows exactly what the growing tissue actually looks like. ''Stages'' refers to the points in the simulation (measured in hours) at which meshes should be generated and saved. In this example, '''five stages of growth will be written to disk'''. These values are chosen to best capture the appearance of the tissue at important stages of the tissue growth.<br />
<br />
The final argument name listed here is ''modelname''. This is a model-specific argument, and in this case the GPT_CASE_RST interaction function contains '''three separate models for plant growth''', allowing the desired model to be a selected. Here, the value [1:3] is specified, which is evaluated in the same way as entering [1 2 3]. This instructs GFtboxCommand to run three separate simulations, one for each of the growth models contained in the interaction function.<br />
<br />
The function of every permissible argument is given by keying the following command into Matlab:<br />
<br />
help GFtboxCommand<br />
<br />
===3B - Getting results from a completed simulation===<br />
<br />
'''1) - Generating images from a simulation you ran on your computer'''<br />
<br />
Once a growth simulation is completed, the project folder (GPT_CASE_RST, in our example) will contain another folder named "movies". Within movies, there are folders which contain results for the executed simulations. As was instructed in 3A, here we can see three folders corresponding to simulations for the three separate models. Within the first folder there are three items:<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| ''CommandLine.txt'' - This file contains the Matlab command which was used to generate the results that this sub-directory contains.<br />
<br />
''gpt_case_rst.txt'' - This file, named according to the project name, contains a copy of the interaction function which was used to produce these results.<br />
<br />
''meshes'' - This folder contains the mesh files corresponding to the stages of growth specified by the value of the Stages argument, described in 3A.<br />
|width="500px" align="center"|[[File:resultsdir.png| Directory structure of a project containing results]]<br />
|}<br />
<br />
<br />
The mesh files contain vertices information regarding the shape of the growth model at a particular stage of growth, but are not visualisable in that form. In order to '''convert these mesh files into viewable images''', we can execute the following command in Matlab:<br />
<br />
VMSreport('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Project','GPT_CASE_RST',...<br />
'Experiment','All','flattentime',572.5,'morphogen','KPAR','SNAPFIG',true);<br />
<br />
Where the ''Path'' and ''Project'' arguments have the same function as ''Path'' and ''Name'' in 3A, i.e. ''Path'' refers to the location of the project folder and ''Project'' is the name of the project folder itself. The images that are created will be stored in the ''meshes'' directory within the project folder.<br />
<br />
Here we can see the generated images, including the command line arguments, for one of the simulations performed in 3A. Click on a thumbnail to view a larger image.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="125px"| <small>001commandline.png</small><br> [[File:commandlineexample.png | 100px]]<br />
|width="125px"| <small>001GPT_CASE_RST_2 wild_.png</small><br>[[File:GPT_CASE_RST_example1.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_3 wild_.png</small><br>[[File:GPT_CASE_RST_example2.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_4 wild_.png</small><br>[[File:GPT_CASE_RST_example3.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_5 wild_.png</small><br>[[File:GPT_CASE_RST_example4.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_6 wild_.png</small><br>[[File:GPT_CASE_RST_example5.png | 125px]]<br />
|}<br />
<br />
NB. The results produced may not be visibly identical to those in the published literature. This is because of small, random perturbations which are applied to the initial model meshes to stop them from containing surfaces which are perfectly flat, and therefore biologically unrealistic. The results produced should be '''qualitatively but not quantitatively the same'''.<br />
<br />
'''2) - Generating images from a simulation you ran on the cluster'''<br />
<br />
Another way to generate images from results is by using the ClusterMonitor tool. When a cluster is used to run simulations, the result files are stored remotely, on the cluster. These files must be retrieved, and then the 'Make project pngs' button can be pressed to generate images, which performs the same function as ''VMSReport'' in part 3A1. This process is described fully in part 5.<br />
<br />
'''3) - Interacting with your results using the GFtbox GUI'''<br />
<br />
='''4''' Altering the simulation parameters=<br />
<br />
One such simulation parameter is ''dt'', which is the time in seconds between iterations in the growth simulation. Large values of ''dt'' mean that fewer steps and therefore fewer calculations are required to complete a simulation. Whereas smaller values of ''dt'' mean that more steps are required, and therefore more processing time too. Therefore, a value of ''dt'' must be selected which is not so small that it is computationally unmanageable, but not so large that the observed growth is an artifact of the value of ''dt'', rather than the underlying growth model. It is necessary therefore to test a range of values for ''dt'' to ensure that the patterns of growth observed in a simulation are consistent across the range, and to find a value which is sufficient to demonstrate the model of growth and computationally efficient.<br />
<br />
'''Testing a range of ''dt''s can be achieved in several ways.''' One way is to use the ''dt'' argument when calling GFtboxCommand. This allows a single value to be tested. Another way is to make a batch of jobs, each using a different value for ''dt'', by using the 'State' argument. Lastly, a range of ''dt''s can be specified in GFtboxCommand, by using the dt argument where the values within the square brackets are the ''dt''s to simulate:<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5]);<br />
<br />
Here, we have specified '''four separate values for ''dt''''', and this means that '''four separate simulations will be run''' sequentially on your computer. As in 3B, images can then be generated for the mesh files produced, and compared to ensure consistency across the range of ''dt''s. In this figure, we can see the same model, at the same stage of growth, generated using the four values of ''dt''. Though quantitatively different, they are qualitatively the same, illustrating the suitability of the default ''dt'' value of 5.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="190px"| [[File:GPT_CASE_RST_dt0.5.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt1.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt2.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt5.png| 190px]]<br />
|}<br />
<br />
='''5''' Altering the model parameters on the cluster=<br />
It is easy to see that when even a small number of range variables, mutations or ''dt''s are specified, the '''total number of simulations increases quickly'''. If many ranges are specified, then the amount of processing time becomes unmanageable on a single computer. Via ''GFtboxCommand'' and the ''ClusterMonitor'' tool, we can remotely''' run a number of simulations, in parallel, on a computing cluster'''. It is assumed that the GrowthToolbox and Matlab are installed on your cluster, and PuTTY is required on your local computer for the tools ''pscp'' (for transferring files from your computer to a cluster) and ''plink'' (for remotely executing commands on a cluster). Using the ''dt'' range argument from 4 as an example, here we add the 'Use' name argument with the value 'Cluster':<br />
<br />
GFtboxCommand('State','Start','Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5],'Use','Cluster');<br />
<br />
The 'Cluster' option for the 'Use' name argument instructs GFtboxCommand to upload the required project directory to a remote Linux server. Instead of running the simulation on your own computer, GFtboxCommand then works out how many individual simulations are specified by the command that invoked it. In this example, the only argument which will generate multiple simulations is ''dt'', of which there will be 4. Separate commands for each of these jobs are then automatically generated, each with an accompanying unique ID (The value of the ''ExpID'' name argument), and '''these are submitted as individual jobs to the cluster'''.<br />
<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.1,'ExpID','GPT_CASE_RST_1');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.5,'ExpID','GPT_CASE_RST_2');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',1,'ExpID','GPT_CASE_RST_3');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',5,'ExpID','GPT_CASE_RST_4');<br />
<br />
Once these jobs have been submitted, the ''ClusterMonitor'' tool opens and the new job batch ID is added to the list of jobs. Here, we will concentrate on three of the functions of ''ClusterMonitor'': ''Queue?'', ''Get project results'' and ''Make project pngs''. As in part 4 of this page, these functions will enable us to visualise the growth simulation results.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| This is an example of what you will see when the ClusterMonitor tool launches after submitting a job using the 'Use','Cluster' argument pair in GFtboxCommand.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="middle"<br />
|width="200px"| Clicking the 'Queue?' button tells ClusterMonitor to check the status of your jobs on the cluster. It then prints the status of the current jobs in Matlab's command window. In the case of our cluster, this is the equivalent of the ''qstatme'' command. This screenshot shows an example job list, where two jobs are currently running.<br />
|width="600px"| [[File:queue.png| 600px]]<br />
|}<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Once your jobs have completed, the 'Queue?' button will show that there are no jobs running. Now you are ready to retrieve the project files from the cluster. This is achieved using the 'Get project results' button. Once pressed, the Matlab command window will show trace information to indicate which files are currently being downloaded. This is the equivalent of downloading the files manually, using an FTP program.<br />
|width="600px" align="center"| [[File:downloadingprojectfiles.png]]<br />
|}<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Once you have successfully downloaded your project results, you are ready to generate result images, as in part 3B1. Instead of entering a ''VMSReport'' command, you can press the 'Make project pngs' button, which will perform the same command. The images that are created will be in the ''meshes'' directory within the project folder.<br />
|width="600px" align="center"| [[File:meshesfolder.png]]<br />
|}<br />
<br />
<br />
Whilst ''ClusterMonitor'' is a useful tool for managing your cluster jobs, it is advisable to '''have at least a rudimentary understanding of and ability to use a Unix-based computer system'''. In particular, the abilities to list the contents of a folder, view the contents of a file, change the current working folder, delete, and to copy or move files or folders, are essential Unix skills for making sure everything is working as intended. It is beyond the scope of these tutorials to provide an in depth Unix tutorial (many good and simple tutorials exist on the web), but here is a short description of the Unix commands (which may be specific to our Unix-based system) that we believe to be important.<br />
<br />
''ls'' - list the files and folders in a directory. Use the -al option (e.g. ls -al) to see modification dates, file permissions, sizes, etc.<br />
<br />
''cd X'' - Change the current directory to directory X.<br />
<br />
''rm X'' - Delete the file X.<br />
<br />
''cp X NewPath/X2'' - Copy file X to the NewPath directory and name the copy X2.<br />
<br />
''mv X NewPath/X2'' - Move file X to the NewPath directory and rename it to X2.<br />
<br />
''cat test.txt'' - Print the contents of the text file test.txt onto the screen.<br />
<br />
''pwd'' - Print the present working directory onto the screen.<br />
<br />
''more test.txt'' - Print the file test.txt to the screen, a screenful at a time, scrolling each time you hit the enter key.</div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Running_example_models_and_using_a_cluster&diff=5398Running example models and using a cluster2011-06-24T09:38:17Z<p>JacobNewman: </p>
<hr />
<div>[[GFtbox Tutorial pages | Back to GFtbox Tutorial pages]]<br />
<br />
The purpose of these tutorials is to learn how to run the example growth simulations included in GFtbox. We will describe methods for running simulations locally (on your own computer) and remotely (on a computing cluster). It is assumed that you have already downloaded the GFtbox software and have Matlab installed.<br />
<br />
=Getting Started=<br />
<br />
The remainder of this page is split into five sub-tutorials, each building on the preceding parts.<br />
<br />
1) '''Explaining the tools'''. In these tutorials, we will be using ''GFtboxCommand'' and ''ClusterMonitor''. This section explains the purpose of these tools. <br />
<br />
2) '''Computer or cluster?''' Here, we illustrate when and why you should use a computing cluster for your growth simulations, and conversely, when you should use your desktop computer.<br />
<br />
3) '''Running a growth simulation for an example model'''. This section demonstrates how to use ''GFtboxCommand'' to run a growth simulation on your computer. The model used for the simulation is an example model included with the GFtbox.<br />
<br />
4) '''Altering the simulation parameters'''. Following on from part 3, here we show how to adjust a simulation parameter within GFtboxCommand. Specifically, we alter the value of ''dt'', the temporal resolution of the simulation, and show how it can be used to verify that the value specified in the published literature is reasonable.<br />
<br />
5) '''Altering the model parameters on the cluster'''. Finally, we demonstrate how a number of model parameters can be varied by specifying a range of values for each model argument. We show how the computationally expensive task of simulating all combinations of specified ranges can be processed more efficiently if a computer cluster is used via the ''ClusterMonitor'' tool.<br />
<br />
='''1''' Explaining the tools=<br />
==GFtboxCommand==<br />
This is a command line version of the GFtbox. By command line, we mean that all program '''functions are operated via typed commands, without the GUI'''. Like GFtbox, GFtboxCommand is capable of running growth simulations of an interaction function, and allows the user to specify model and simulation parameters. Unlike GFtbox, this also allows the user to select '''ranges of values for a number of input parameters''', and will automatically spawn multiple simulations which explore the various combinations of those parameters. This can be used to evaluate the effect of various parameters on the growth of a given model.<br />
<br />
==ClusterMonitor==<br />
Provides a graphical user interface for managing simulations '''running remotely on a computer cluster'''. Specifically, it allows you to see which projects are present and running on the cluster, to retrieve the completed projects, to generate images of the simulations at specified stages of growth, and to remove projects from the cluster. If you do not intend to use a computer cluster, then you will not need to use ClusterMonitor.<br />
<br />
='''2''' Computer or cluster?=<br />
<br />
In basic terms, '''a computing cluster is effectively a network of many computer processors''' (often hundreds), centrally managed by a queuing system. When a job is submitted to a cluster, the job is sent to a processor that is not being used, or queued until one becomes available. In contrast, a typical desktop computer will contain one processor, limiting the number of tasks that can be performed at any one time. Jobs which must be run independently and sequentially on a desktop computer can be executed in parallel on a cluster, '''greatly reducing the total time required to complete all of the jobs'''. Although the exact details of your cluster might vary from those described in the rest of these tutorials, we aim to illustrate the generic processes involved in using GFtbox on a cluster. We are happy to offer assistance, where possible, to setup GFtbox for your cluster, so please contact us if you have any queries.<br />
<br />
Whilst the time savings offered by using a cluster can be significant, there is an overhead associated with returning the results to your personal computer. The total time to run 2 or more simulations on a cluster and to return the results will be less than running those simulations sequentially on one computer. Therefore, using a '''cluster is ideal for situations where you would like to run several simulations''', such as to evaluate the effect of a range of parameters on a growth model. You are not advised to use a cluster for running single simulations, or where you would like to step through a simulation and change parameters in a more interactive fashion. In such circumstances, you are advised to run GFtbox or GFtboxCommand on your own computer.<br />
<br />
The '''GFtbox GUI provides quick feedback''' about how the changes you have made to an interaction function have affected the growth simulation, as '''you can see the result of each simulation iteration as it completes'''. It is quick and easy therefore to see if you have dramatically changed the course of the growth simulation, and then to adjust the parameters according to your observations. This approach is well suited to the early stages of design, where you might wish to '''tweak some parameters to gauge whether or not they have had the desired effect'''. Or in the final stages, where you think your model is almost finalised, but where small adjustments are required. '''An alternative approach is to start many simulations''', based upon your initial model and by making intelligent choices about which parameters to explore. This allows you to '''harvest many results, to quickly and easily overview them''' in their finalised form, or to select interesting-looking simulations and examine them more closely on your desktop computer, using GFtbox.<br />
<br />
='''3''' Running a growth simulation for an example model=<br />
This tutorial is aimed at running a '''growth simulation for one of the example interaction functions''' included with the GFtbox. The purpose of this exercise is to firstly demonstrate how simulations can be invoked using GFtboxCommand, and secondly to show how to reproduce experimental results (specifically, those published in Kennaway et al (2011)) given an interaction function.<br />
<br />
Assuming Matlab is installed on your computer, and the latest GFtbox has been downloaded, you can add the GFtbox directory to Matlab's search path, which makes the toolbox accessible from any other path that you choose to work from. For a short tutorial on how to do this, please click [[Adding GFtbox to Matlab's search path | here]].<br />
<br />
Once the GFtbox is added to Matlab, you are ready to run a growth simulation using GFtboxCommand. In this example, '''the model that we will simulate is called GPT_CASE_RST'''. Results generated using this model are published in Kennaway et al (2011). By running this simulation, we can confirm the results in the published literature and investigate the suitability of the various parameters.<br />
<br />
===3A - Running a simulation on your computer===<br />
<br />
The following command can be typed into Matlab to run a simulation of the GPT_CASE_RST interaction function, which contains three growth models: R, S and T. '''Three separate simulations are run sequentially on your computer''', one for each model, each producing results corresponding to five intervals in the growth simulation.<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',[1:3]);<br />
<br />
GFtboxCommand accepts input arguments as name and value pairs, e.g. 'modelname', [1:3] or 'Use','Cluster'. The argument names entered in the example above are: ''Path'', ''Name'', ''Stages'' and ''modelname''.<br />
<br />
The optional '''''Path'' argument name refers to the location of the folder''' (or directory) on your computer where the model interaction function you wish to simulate is stored. '''''Name'', is the name of the folder itself'''. In this case, we are using the GPT_CASE_RST folder which is included in the GrowthToolbox. You may wish to copy this folder elsewhere, if you intend to make changes to the interaction function.<br />
<br />
During a growth simulation, a mesh can be generated at each time step of the simulation, which provides a visual representation of the growth of the biological tissue, given the various parameters of growth specified by the interaction function and how they have changed over time. Put another way, the mesh shows exactly what the growing tissue actually looks like. ''Stages'' refers to the points in the simulation (measured in hours) at which meshes should be generated and saved. In this example, '''five stages of growth will be written to disk'''. These values are chosen to best capture the appearance of the tissue at important stages of the tissue growth.<br />
<br />
The final argument name listed here is ''modelname''. This is a model-specific argument, and in this case the GPT_CASE_RST interaction function contains '''three separate models for plant growth''', allowing the desired model to be a selected. Here, the value [1:3] is specified, which is evaluated in the same way as entering [1 2 3]. This instructs GFtboxCommand to run three separate simulations, one for each of the growth models contained in the interaction function.<br />
<br />
The function of every permissible argument is given by keying the following command into Matlab:<br />
<br />
help GFtboxCommand<br />
<br />
===3B - Getting results from a completed simulation===<br />
<br />
'''1) - Generating images from a simulation you ran on your computer'''<br />
<br />
Once a growth simulation is completed, the project folder (GPT_CASE_RST, in our example) will contain another folder named "movies". Within movies, there are folders which contain results for the executed simulations. As was instructed in 3A, here we can see three folders corresponding to simulations for the three separate models. Within the first folder there are three items:<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| ''CommandLine.txt'' - This file contains the Matlab command which was used to generate the results that this sub-directory contains.<br />
<br />
''gpt_case_rst.txt'' - This file, named according to the project name, contains a copy of the interaction function which was used to produce these results.<br />
<br />
''meshes'' - This folder contains the mesh files corresponding to the stages of growth specified by the value of the Stages argument, described in 3A.<br />
|width="500px" align="center"|[[File:resultsdir.png| Directory structure of a project containing results]]<br />
|}<br />
<br />
<br />
The mesh files contain vertices information regarding the shape of the growth model at a particular stage of growth, but are not visualisable in that form. In order to '''convert these mesh files into viewable images''', we can execute the following command in Matlab:<br />
<br />
VMSreport('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Project','GPT_CASE_RST',...<br />
'Experiment','All','flattentime',572.5,'morphogen','KPAR','SNAPFIG',true);<br />
<br />
Where the ''Path'' and ''Project'' arguments have the same function as ''Path'' and ''Name'' in 3A, i.e. ''Path'' refers to the location of the project folder and ''Project'' is the name of the project folder itself. The images that are created will be stored in the ''meshes'' directory within the project folder.<br />
<br />
Here we can see the generated images, including the command line arguments, for one of the simulations performed in 3A. Click on a thumbnail to view a larger image.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="125px"| <small>001commandline.png</small><br> [[File:commandlineexample.png | 100px]]<br />
|width="125px"| <small>001GPT_CASE_RST_2 wild_.png</small><br>[[File:GPT_CASE_RST_example1.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_3 wild_.png</small><br>[[File:GPT_CASE_RST_example2.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_4 wild_.png</small><br>[[File:GPT_CASE_RST_example3.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_5 wild_.png</small><br>[[File:GPT_CASE_RST_example4.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_6 wild_.png</small><br>[[File:GPT_CASE_RST_example5.png | 125px]]<br />
|}<br />
<br />
<br />
'''2) - Generating images from a simulation you ran on the cluster'''<br />
<br />
NB. The results produced may not be visibly identical to those in the published literature. This is because of small, random perturbations which are applied to the initial model meshes to stop them from containing surfaces which are perfectly flat, and therefore biologically unrealistic. The results produced should be '''qualitatively but not quantitatively the same'''.<br />
<br />
'''3) - Interacting with your results using the GFtbox GUI'''<br />
<br />
='''4''' Altering the simulation parameters=<br />
<br />
One such simulation parameter is ''dt'', which is the time in seconds between iterations in the growth simulation. Large values of ''dt'' mean that fewer steps and therefore fewer calculations are required to complete a simulation. Whereas smaller values of ''dt'' mean that more steps are required, and therefore more processing time too. Therefore, a value of ''dt'' must be selected which is not so small that it is computationally unmanageable, but not so large that the observed growth is an artifact of the value of ''dt'', rather than the underlying growth model. It is necessary therefore to test a range of values for ''dt'' to ensure that the patterns of growth observed in a simulation are consistent across the range, and to find a value which is sufficient to demonstrate the model of growth and computationally efficient.<br />
<br />
'''Testing a range of ''dt''s can be achieved in several ways.''' One way is to use the ''dt'' argument when calling GFtboxCommand. This allows a single value to be tested. Another way is to make a batch of jobs, each using a different value for ''dt'', by using the 'State' argument. Lastly, a range of ''dt''s can be specified in GFtboxCommand, by using the dt argument where the values within the square brackets are the ''dt''s to simulate:<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5]);<br />
<br />
Here, we have specified '''four separate values for ''dt''''', and this means that '''four separate simulations will be run''' sequentially on your computer. As in 3B, images can then be generated for the mesh files produced, and compared to ensure consistency across the range of ''dt''s. In this figure, we can see the same model, at the same stage of growth, generated using the four values of ''dt''. Though quantitatively different, they are qualitatively the same, illustrating the suitability of the default ''dt'' value of 5.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="190px"| [[File:GPT_CASE_RST_dt0.5.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt1.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt2.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt5.png| 190px]]<br />
|}<br />
<br />
='''5''' Altering the model parameters on the cluster=<br />
It is easy to see that when even a small number of range variables, mutations or ''dt''s are specified, the '''total number of simulations increases quickly'''. If many ranges are specified, then the amount of processing time becomes unmanageable on a single computer. Via ''GFtboxCommand'' and the ''ClusterMonitor'' tool, we can remotely''' run a number of simulations, in parallel, on a computing cluster'''. It is assumed that the GrowthToolbox and Matlab are installed on your cluster, and PuTTY is required on your local computer for the tools ''pscp'' (for transferring files from your computer to a cluster) and ''plink'' (for remotely executing commands on a cluster). Using the ''dt'' range argument from 4 as an example, here we add the 'Use' name argument with the value 'Cluster':<br />
<br />
GFtboxCommand('State','Start','Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5],'Use','Cluster');<br />
<br />
The 'Cluster' option for the 'Use' name argument instructs GFtboxCommand to upload the required project directory to a remote Linux server. Instead of running the simulation on your own computer, GFtboxCommand then works out how many individual simulations are specified by the command that invoked it. In this example, the only argument which will generate multiple simulations is ''dt'', of which there will be 4. Separate commands for each of these jobs are then automatically generated, each with an accompanying unique ID (The value of the ''ExpID'' name argument), and '''these are submitted as individual jobs to the cluster'''.<br />
<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.1,'ExpID','GPT_CASE_RST_1');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.5,'ExpID','GPT_CASE_RST_2');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',1,'ExpID','GPT_CASE_RST_3');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',5,'ExpID','GPT_CASE_RST_4');<br />
<br />
Once these jobs have been submitted, the ''ClusterMonitor'' tool opens and the new job batch ID is added to the list of jobs. Here, we will concentrate on three of the functions of ''ClusterMonitor'': ''Queue?'', ''Get project results'' and ''Make project pngs''. As in part 4 of this page, these functions will enable us to visualise the growth simulation results.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| This is an example of what you will see when the ClusterMonitor tool launches after submitting a job using the 'Use','Cluster' argument pair in GFtboxCommand.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="middle"<br />
|width="200px"| Clicking the 'Queue?' button tells ClusterMonitor to check the status of your jobs on the cluster. It then prints the status of the current jobs in Matlab's command window. In the case of our cluster, this is the equivalent of the ''qstatme'' command. This screenshot shows an example job list, where two jobs are currently running.<br />
|width="600px"| [[File:queue.png| 600px]]<br />
|}<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Once your jobs have completed, the 'Queue?' button will show that there are no jobs running. Now you are ready to retrieve the project files from the cluster. This is achieved using the 'Get project results' button. Once pressed, the Matlab command window will show trace information to indicate which files are currently being downloaded. This is the equivalent of downloading the files manually, using an FTP program.<br />
|width="600px" align="center"| [[File:downloadingprojectfiles.png]]<br />
|}<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Once you have successfully downloaded your project results, you are ready to generate result images, as in part 3B1. Instead of entering a ''VMSReport'' command, you can press the 'Make project pngs' button, which will perform the same command. The images that are created will be in the ''meshes'' directory within the project folder.<br />
|width="600px" align="center"| [[File:meshesfolder.png]]<br />
|}<br />
<br />
<br />
Whilst ''ClusterMonitor'' is a useful tool for managing your cluster jobs, it is advisable to '''have at least a rudimentary understanding of and ability to use a Unix-based computer system'''. In particular, the abilities to list the contents of a folder, view the contents of a file, change the current working folder, delete, and to copy or move files or folders, are essential Unix skills for making sure everything is working as intended. It is beyond the scope of these tutorials to provide an in depth Unix tutorial (many good and simple tutorials exist on the web), but here is a short description of the Unix commands (which may be specific to our Unix-based system) that we believe to be important.<br />
<br />
''ls'' - list the files and folders in a directory. Use the -al option (e.g. ls -al) to see modification dates, file permissions, sizes, etc.<br />
<br />
''cd X'' - Change the current directory to directory X.<br />
<br />
''rm X'' - Delete the file X.<br />
<br />
''cp X NewPath/X2'' - Copy file X to the NewPath directory and name the copy X2.<br />
<br />
''mv X NewPath/X2'' - Move file X to the NewPath directory and rename it to X2.<br />
<br />
''cat test.txt'' - Print the contents of the text file test.txt onto the screen.<br />
<br />
''pwd'' - Print the present working directory onto the screen.<br />
<br />
''more test.txt'' - Print the file test.txt to the screen, a screenful at a time, scrolling each time you hit the enter key.</div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Running_example_models_and_using_a_cluster&diff=5396Running example models and using a cluster2011-06-24T09:36:26Z<p>JacobNewman: </p>
<hr />
<div>[[GFtbox Tutorial pages | Back to GFtbox Tutorial pages]]<br />
<br />
The purpose of these tutorials is to learn how to run the example growth simulations included in GFtbox. We will describe methods for running simulations locally (on your own computer) and remotely (on a computing cluster). It is assumed that you have already downloaded the GFtbox software and have Matlab installed.<br />
<br />
=Getting Started=<br />
<br />
The remainder of this page is split into five sub-tutorials, each building on the preceding parts.<br />
<br />
1) '''Explaining the tools'''. In these tutorials, we will be using ''GFtboxCommand'' and ''ClusterMonitor''. This section explains the purpose of these tools. <br />
<br />
2) '''Computer or cluster?''' Here, we illustrate when and why you should use a computing cluster for your growth simulations, and conversely, when you should use your desktop computer.<br />
<br />
3) '''Running a growth simulation for an example model'''. This section demonstrates how to use ''GFtboxCommand'' to run a growth simulation on your computer. The model used for the simulation is an example model included with the GFtbox.<br />
<br />
4) '''Altering the simulation parameters'''. Following on from part 3, here we show how to adjust a simulation parameter within GFtboxCommand. Specifically, we alter the value of ''dt'', the temporal resolution of the simulation, and show how it can be used to verify that the value specified in the published literature is reasonable.<br />
<br />
5) '''Altering the model parameters on the cluster'''. Finally, we demonstrate how a number of model parameters can be varied by specifying a range of values for each model argument. We show how the computationally expensive task of simulating all combinations of specified ranges can be processed more efficiently if a computer cluster is used via the ''ClusterMonitor'' tool.<br />
<br />
='''1''' Explaining the tools=<br />
==GFtboxCommand==<br />
This is a command line version of the GFtbox. By command line, we mean that all program '''functions are operated via typed commands, without the GUI'''. Like GFtbox, GFtboxCommand is capable of running growth simulations of an interaction function, and allows the user to specify model and simulation parameters. Unlike GFtbox, this also allows the user to select '''ranges of values for a number of input parameters''', and will automatically spawn multiple simulations which explore the various combinations of those parameters. This can be used to evaluate the effect of various parameters on the growth of a given model.<br />
<br />
==ClusterMonitor==<br />
Provides a graphical user interface for managing simulations '''running remotely on a computer cluster'''. Specifically, it allows you to see which projects are present and running on the cluster, to retrieve the completed projects, to generate images of the simulations at specified stages of growth, and to remove projects from the cluster. If you do not intend to use a computer cluster, then you will not need to use ClusterMonitor.<br />
<br />
='''2''' Computer or cluster?=<br />
<br />
In basic terms, '''a computing cluster is effectively a network of many computer processors''' (often hundreds), centrally managed by a queuing system. When a job is submitted to a cluster, the job is sent to a processor that is not being used, or queued until one becomes available. In contrast, a typical desktop computer will contain one processor, limiting the number of tasks that can be performed at any one time. Jobs which must be run independently and sequentially on a desktop computer can be executed in parallel on a cluster, '''greatly reducing the total time required to complete all of the jobs'''. Although the exact details of your cluster might vary from those described in the rest of these tutorials, we aim to illustrate the generic processes involved in using GFtbox on a cluster. We are happy to offer assistance, where possible, to setup GFtbox for your cluster, so please contact us if you have any queries.<br />
<br />
Whilst the time savings offered by using a cluster can be significant, there is an overhead associated with returning the results to your personal computer. The total time to run 2 or more simulations on a cluster and to return the results will be less than running those simulations sequentially on one computer. Therefore, using a '''cluster is ideal for situations where you would like to run several simulations''', such as to evaluate the effect of a range of parameters on a growth model. You are not advised to use a cluster for running single simulations, or where you would like to step through a simulation and change parameters in a more interactive fashion. In such circumstances, you are advised to run GFtbox or GFtboxCommand on your own computer.<br />
<br />
The '''GFtbox GUI provides quick feedback''' about how the changes you have made to an interaction function have affected the growth simulation, as '''you can see the result of each simulation iteration as it completes'''. It is quick and easy therefore to see if you have dramatically changed the course of the growth simulation, and then to adjust the parameters according to your observations. This approach is well suited to the early stages of design, where you might wish to '''tweak some parameters to gauge whether or not they have had the desired effect'''. Or in the final stages, where you think your model is almost finalised, but where small adjustments are required. '''An alternative approach is to start many simulations''', based upon your initial model and by making intelligent choices about which parameters to explore. This allows you to '''harvest many results, to quickly and easily overview them''' in their finalised form, or to select interesting-looking simulations and examine them more closely on your desktop computer, using GFtbox.<br />
<br />
='''3''' Running a growth simulation for an example model=<br />
This tutorial is aimed at running a '''growth simulation for one of the example interaction functions''' included with the GFtbox. The purpose of this exercise is to firstly demonstrate how simulations can be invoked using GFtboxCommand, and secondly to show how to reproduce experimental results (specifically, those published in Kennaway et al (2011)) given an interaction function.<br />
<br />
Assuming Matlab is installed on your computer, and the latest GFtbox has been downloaded, you can add the GFtbox directory to Matlab's search path, which makes the toolbox accessible from any other path that you choose to work from. For a short tutorial on how to do this, please click [[Adding GFtbox to Matlab's search path | here]].<br />
<br />
Once the GFtbox is added to Matlab, you are ready to run a growth simulation using GFtboxCommand. In this example, '''the model that we will simulate is called GPT_CASE_RST'''. Results generated using this model are published in Kennaway et al (2011). By running this simulation, we can confirm the results in the published literature and investigate the suitability of the various parameters.<br />
<br />
===3A - Running a simulation on your computer===<br />
<br />
The following command can be typed into Matlab to run a simulation of the GPT_CASE_RST interaction function, which contains three growth models: R, S and T. '''Three separate simulations are run sequentially on your computer''', one for each model, each producing results corresponding to five intervals in the growth simulation.<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',[1:3]);<br />
<br />
GFtboxCommand accepts input arguments as name and value pairs, e.g. 'modelname', [1:3] or 'Use','Cluster'. The argument names entered in the example above are: ''Path'', ''Name'', ''Stages'' and ''modelname''.<br />
<br />
The optional '''''Path'' argument name refers to the location of the folder''' (or directory) on your computer where the model interaction function you wish to simulate is stored. '''''Name'', is the name of the folder itself'''. In this case, we are using the GPT_CASE_RST folder which is included in the GrowthToolbox. You may wish to copy this folder elsewhere, if you intend to make changes to the interaction function.<br />
<br />
During a growth simulation, a mesh can be generated at each time step of the simulation, which provides a visual representation of the growth of the biological tissue, given the various parameters of growth specified by the interaction function and how they have changed over time. Put another way, the mesh shows exactly what the growing tissue actually looks like. ''Stages'' refers to the points in the simulation (measured in hours) at which meshes should be generated and saved. In this example, '''five stages of growth will be written to disk'''. These values are chosen to best capture the appearance of the tissue at important stages of the tissue growth.<br />
<br />
The final argument name listed here is ''modelname''. This is a model-specific argument, and in this case the GPT_CASE_RST interaction function contains '''three separate models for plant growth''', allowing the desired model to be a selected. Here, the value [1:3] is specified, which is evaluated in the same way as entering [1 2 3]. This instructs GFtboxCommand to run three separate simulations, one for each of the growth models contained in the interaction function.<br />
<br />
The function of every permissible argument is given by keying the following command into Matlab:<br />
<br />
help GFtboxCommand<br />
<br />
===3B - Getting results from a completed simulation===<br />
<br />
'''1) - Generating images from a simulation you ran on your computer'''<br />
<br />
Once a growth simulation is completed, the project folder (GPT_CASE_RST, in our example) will contain another folder named "movies". Within movies, there are folders which contain results for the executed simulations. As was instructed in 3A, here we can see three folders corresponding to simulations for the three separate models. Within the first folder there are three items:<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| ''CommandLine.txt'' - This file contains the Matlab command which was used to generate the results that this sub-directory contains.<br />
<br />
''gpt_case_rst.txt'' - This file, named according to the project name, contains a copy of the interaction function which was used to produce these results.<br />
<br />
''meshes'' - This folder contains the mesh files corresponding to the stages of growth specified by the value of the Stages argument, described in 3A.<br />
|width="500px" align="center"|[[File:resultsdir.png| Directory structure of a project containing results]]<br />
|}<br />
<br />
<br />
The mesh files contain vertices information regarding the shape of the growth model at a particular stage of growth, but are not visualisable in that form. In order to '''convert these mesh files into viewable images''', we can execute the following command in Matlab:<br />
<br />
VMSreport('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Project','GPT_CASE_RST',...<br />
'Experiment','All','flattentime',572.5,'morphogen','KPAR','SNAPFIG',true);<br />
<br />
Where the ''Path'' and ''Project'' arguments have the same function as ''Path'' and ''Name'' in 3A, i.e. ''Path'' refers to the location of the project folder and ''Project'' is the name of the project folder itself. The images that are created will be stored in the ''meshes'' directory within the project folder.<br />
<br />
Here we can see the generated images, including the command line arguments, for one of the simulations performed in 3A. Click on a thumbnail to view a larger image.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="125px"| <small>001commandline.png</small><br> [[File:commandlineexample.png | 100px]]<br />
|width="125px"| <small>001GPT_CASE_RST_2 wild_.png</small><br>[[File:GPT_CASE_RST_example1.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_3 wild_.png</small><br>[[File:GPT_CASE_RST_example2.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_4 wild_.png</small><br>[[File:GPT_CASE_RST_example3.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_5 wild_.png</small><br>[[File:GPT_CASE_RST_example4.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_6 wild_.png</small><br>[[File:GPT_CASE_RST_example5.png | 125px]]<br />
|}<br />
<br />
<br />
'''2) - Generating images from a simulation you ran on the cluster'''<br />
<br />
NB. The results produced may not be visibly identical to those in the published literature. This is because of small, random perturbations which are applied to the initial model meshes to stop them from containing surfaces which are perfectly flat, and therefore biologically unrealistic. The results produced should be '''qualitatively but not quantitatively the same'''.<br />
<br />
'''3) - Interacting with your results using the GFtbox GUI'''<br />
<br />
='''4''' Altering the simulation parameters=<br />
<br />
One such simulation parameter is ''dt'', which is the time in seconds between iterations in the growth simulation. Large values of ''dt'' mean that fewer steps and therefore fewer calculations are required to complete a simulation. Whereas smaller values of ''dt'' mean that more steps are required, and therefore more processing time too. Therefore, a value of ''dt'' must be selected which is not so small that it is computationally unmanageable, but not so large that the observed growth is an artifact of the value of ''dt'', rather than the underlying growth model. It is necessary therefore to test a range of values for ''dt'' to ensure that the patterns of growth observed in a simulation are consistent across the range, and to find a value which is sufficient to demonstrate the model of growth and computationally efficient.<br />
<br />
'''Testing a range of ''dt''s can be achieved in several ways.''' One way is to use the ''dt'' argument when calling GFtboxCommand. This allows a single value to be tested. Another way is to make a batch of jobs, each using a different value for ''dt'', by using the 'State' argument. Lastly, a range of ''dt''s can be specified in GFtboxCommand, by using the dt argument where the values within the square brackets are the ''dt''s to simulate:<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5]);<br />
<br />
Here, we have specified '''four separate values for ''dt''''', and this means that '''four separate simulations will be run''' sequentially on your computer. As in 3B, images can then be generated for the mesh files produced, and compared to ensure consistency across the range of ''dt''s. In this figure, we can see the same model, at the same stage of growth, generated using the four values of ''dt''. Though quantitatively different, they are qualitatively the same, illustrating the suitability of the default ''dt'' value of 5.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="190px"| [[File:GPT_CASE_RST_dt0.5.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt1.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt2.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt5.png| 190px]]<br />
|}<br />
<br />
='''5''' Altering the model parameters on the cluster=<br />
It is easy to see that when even a small number of range variables, mutations or ''dt''s are specified, the '''total number of simulations increases quickly'''. If many ranges are specified, then the amount of processing time becomes unmanageable on a single computer. Via ''GFtboxCommand'' and the ''ClusterMonitor'' tool, we can remotely''' run a number of simulations, in parallel, on a computing cluster'''. It is assumed that the GrowthToolbox and Matlab are installed on your cluster, and PuTTY is required on your local computer for the tools ''pscp'' (for transferring files from your computer to a cluster) and ''plink'' (for remotely executing commands on a cluster). Using the ''dt'' range argument from 4 as an example, here we add the 'Use' name argument with the value 'Cluster':<br />
<br />
GFtboxCommand('State','Start','Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5],'Use','Cluster');<br />
<br />
The 'Cluster' option for the 'Use' name argument instructs GFtboxCommand to upload the required project directory to a remote Linux server. Instead of running the simulation on your own computer, GFtboxCommand then works out how many individual simulations are specified by the command that invoked it. In this example, the only argument which will generate multiple simulations is ''dt'', of which there will be 4. Separate commands for each of these jobs are then automatically generated, each with an accompanying unique ID (The value of the ''ExpID'' name argument), and '''these are submitted as individual jobs to the cluster'''.<br />
<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.1,'ExpID','GPT_CASE_RST_1');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.5,'ExpID','GPT_CASE_RST_2');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',1,'ExpID','GPT_CASE_RST_3');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',5,'ExpID','GPT_CASE_RST_4');<br />
<br />
Once these jobs have been submitted, the ''ClusterMonitor'' tool opens and the new job batch ID is added to the list of jobs. Here, we will concentrate on three of the functions of ''ClusterMonitor'': ''Queue?'', ''Get project results'' and ''Make project pngs''. As in part 4 of this page, these functions will enable us to visualise the growth simulation results.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| This is an example of what you will see when the ClusterMonitor tool launches after submitting a job using the 'Use','Cluster' argument pair in GFtboxCommand.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="middle"<br />
|width="200px"| Clicking the 'Queue?' button tells ClusterMonitor to check the status of your jobs on the cluster. It then prints the status of the current jobs in Matlab's command window. In the case of our cluster, this is the equivalent of the ''qstatme'' command. This screenshot shows an example job list, where one job is currently running.<br />
<br />
This allows you too see whether or not your jobs have finished, and therefore whether or not their results are ready to retrieve.<br />
|width="600px"| [[File:queue.png| 600px]]<br />
|}<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Once your jobs have completed, the 'Queue?' button will show that there are no jobs running. Now you are ready to retrieve the project files from the cluster. This is achieved using the 'Get project results' button. Once pressed, the Matlab command window will show trace information to indicate which files are currently being downloaded. This is the equivalent of downloading the files manually, using an FTP program.<br />
|width="600px" align="center"| [[File:downloadingprojectfiles.png]]<br />
|}<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Once you have successfully downloaded your project results, you are ready to generate result images, as in part 3B1. Instead of entering a ''VMSReport'' command, you can press the 'Make project pngs' button, which will perform the same command. The images that are created will be in the ''meshes'' directory within the project folder.<br />
|width="600px" align="center"| [[File:meshesfolder.png]]<br />
|}<br />
<br />
<br />
Whilst ''ClusterMonitor'' is a useful tool for managing your cluster jobs, it is advisable to '''have at least a rudimentary understanding of and ability to use a Unix-based computer system'''. In particular, the abilities to list the contents of a folder, view the contents of a file, change the current working folder, delete, and to copy or move files or folders, are essential Unix skills for making sure everything is working as intended. It is beyond the scope of these tutorials to provide an in depth Unix tutorial (many good and simple tutorials exist on the web), but here is a short description of the Unix commands (which may be specific to our Unix-based system) that we believe to be important.<br />
<br />
''ls'' - list the files and folders in a directory. Use the -al option (e.g. ls -al) to see modification dates, file permissions, sizes, etc.<br />
<br />
''cd X'' - Change the current directory to directory X.<br />
<br />
''rm X'' - Delete the file X.<br />
<br />
''cp X NewPath/X2'' - Copy file X to the NewPath directory and name the copy X2.<br />
<br />
''mv X NewPath/X2'' - Move file X to the NewPath directory and rename it to X2.<br />
<br />
''cat test.txt'' - Print the contents of the text file test.txt onto the screen.<br />
<br />
''pwd'' - Print the present working directory onto the screen.<br />
<br />
''more test.txt'' - Print the file test.txt to the screen, a screenful at a time, scrolling each time you hit the enter key.</div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=File:Meshesfolder.png&diff=5516File:Meshesfolder.png2011-06-24T09:35:00Z<p>JacobNewman: </p>
<hr />
<div></div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Running_example_models_and_using_a_cluster&diff=5411Running example models and using a cluster2011-06-24T09:34:28Z<p>JacobNewman: </p>
<hr />
<div>[[GFtbox Tutorial pages | Back to GFtbox Tutorial pages]]<br />
<br />
The purpose of these tutorials is to learn how to run the example growth simulations included in GFtbox. We will describe methods for running simulations locally (on your own computer) and remotely (on a computing cluster). It is assumed that you have already downloaded the GFtbox software and have Matlab installed.<br />
<br />
=Getting Started=<br />
<br />
The remainder of this page is split into five sub-tutorials, each building on the preceding parts.<br />
<br />
1) '''Explaining the tools'''. In these tutorials, we will be using ''GFtboxCommand'' and ''ClusterMonitor''. This section explains the purpose of these tools. <br />
<br />
2) '''Computer or cluster?''' Here, we illustrate when and why you should use a computing cluster for your growth simulations, and conversely, when you should use your desktop computer.<br />
<br />
3) '''Running a growth simulation for an example model'''. This section demonstrates how to use ''GFtboxCommand'' to run a growth simulation on your computer. The model used for the simulation is an example model included with the GFtbox.<br />
<br />
4) '''Altering the simulation parameters'''. Following on from part 3, here we show how to adjust a simulation parameter within GFtboxCommand. Specifically, we alter the value of ''dt'', the temporal resolution of the simulation, and show how it can be used to verify that the value specified in the published literature is reasonable.<br />
<br />
5) '''Altering the model parameters on the cluster'''. Finally, we demonstrate how a number of model parameters can be varied by specifying a range of values for each model argument. We show how the computationally expensive task of simulating all combinations of specified ranges can be processed more efficiently if a computer cluster is used via the ''ClusterMonitor'' tool.<br />
<br />
='''1''' Explaining the tools=<br />
==GFtboxCommand==<br />
This is a command line version of the GFtbox. By command line, we mean that all program '''functions are operated via typed commands, without the GUI'''. Like GFtbox, GFtboxCommand is capable of running growth simulations of an interaction function, and allows the user to specify model and simulation parameters. Unlike GFtbox, this also allows the user to select '''ranges of values for a number of input parameters''', and will automatically spawn multiple simulations which explore the various combinations of those parameters. This can be used to evaluate the effect of various parameters on the growth of a given model.<br />
<br />
==ClusterMonitor==<br />
Provides a graphical user interface for managing simulations '''running remotely on a computer cluster'''. Specifically, it allows you to see which projects are present and running on the cluster, to retrieve the completed projects, to generate images of the simulations at specified stages of growth, and to remove projects from the cluster. If you do not intend to use a computer cluster, then you will not need to use ClusterMonitor.<br />
<br />
='''2''' Computer or cluster?=<br />
<br />
In basic terms, '''a computing cluster is effectively a network of many computer processors''' (often hundreds), centrally managed by a queuing system. When a job is submitted to a cluster, the job is sent to a processor that is not being used, or queued until one becomes available. In contrast, a typical desktop computer will contain one processor, limiting the number of tasks that can be performed at any one time. Jobs which must be run independently and sequentially on a desktop computer can be executed in parallel on a cluster, '''greatly reducing the total time required to complete all of the jobs'''. Although the exact details of your cluster might vary from those described in the rest of these tutorials, we aim to illustrate the generic processes involved in using GFtbox on a cluster. We are happy to offer assistance, where possible, to setup GFtbox for your cluster, so please contact us if you have any queries.<br />
<br />
Whilst the time savings offered by using a cluster can be significant, there is an overhead associated with returning the results to your personal computer. The total time to run 2 or more simulations on a cluster and to return the results will be less than running those simulations sequentially on one computer. Therefore, using a '''cluster is ideal for situations where you would like to run several simulations''', such as to evaluate the effect of a range of parameters on a growth model. You are not advised to use a cluster for running single simulations, or where you would like to step through a simulation and change parameters in a more interactive fashion. In such circumstances, you are advised to run GFtbox or GFtboxCommand on your own computer.<br />
<br />
The '''GFtbox GUI provides quick feedback''' about how the changes you have made to an interaction function have affected the growth simulation, as '''you can see the result of each simulation iteration as it completes'''. It is quick and easy therefore to see if you have dramatically changed the course of the growth simulation, and then to adjust the parameters according to your observations. This approach is well suited to the early stages of design, where you might wish to '''tweak some parameters to gauge whether or not they have had the desired effect'''. Or in the final stages, where you think your model is almost finalised, but where small adjustments are required. '''An alternative approach is to start many simulations''', based upon your initial model and by making intelligent choices about which parameters to explore. This allows you to '''harvest many results, to quickly and easily overview them''' in their finalised form, or to select interesting-looking simulations and examine them more closely on your desktop computer, using GFtbox.<br />
<br />
='''3''' Running a growth simulation for an example model=<br />
This tutorial is aimed at running a '''growth simulation for one of the example interaction functions''' included with the GFtbox. The purpose of this exercise is to firstly demonstrate how simulations can be invoked using GFtboxCommand, and secondly to show how to reproduce experimental results (specifically, those published in Kennaway et al (2011)) given an interaction function.<br />
<br />
Assuming Matlab is installed on your computer, and the latest GFtbox has been downloaded, you can add the GFtbox directory to Matlab's search path, which makes the toolbox accessible from any other path that you choose to work from. For a short tutorial on how to do this, please click [[Adding GFtbox to Matlab's search path | here]].<br />
<br />
Once the GFtbox is added to Matlab, you are ready to run a growth simulation using GFtboxCommand. In this example, '''the model that we will simulate is called GPT_CASE_RST'''. Results generated using this model are published in Kennaway et al (2011). By running this simulation, we can confirm the results in the published literature and investigate the suitability of the various parameters.<br />
<br />
===3A - Running a simulation on your computer===<br />
<br />
The following command can be typed into Matlab to run a simulation of the GPT_CASE_RST interaction function, which contains three growth models: R, S and T. '''Three separate simulations are run sequentially on your computer''', one for each model, each producing results corresponding to five intervals in the growth simulation.<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',[1:3]);<br />
<br />
GFtboxCommand accepts input arguments as name and value pairs, e.g. 'modelname', [1:3] or 'Use','Cluster'. The argument names entered in the example above are: ''Path'', ''Name'', ''Stages'' and ''modelname''.<br />
<br />
The optional '''''Path'' argument name refers to the location of the folder''' (or directory) on your computer where the model interaction function you wish to simulate is stored. '''''Name'', is the name of the folder itself'''. In this case, we are using the GPT_CASE_RST folder which is included in the GrowthToolbox. You may wish to copy this folder elsewhere, if you intend to make changes to the interaction function.<br />
<br />
During a growth simulation, a mesh can be generated at each time step of the simulation, which provides a visual representation of the growth of the biological tissue, given the various parameters of growth specified by the interaction function and how they have changed over time. Put another way, the mesh shows exactly what the growing tissue actually looks like. ''Stages'' refers to the points in the simulation (measured in hours) at which meshes should be generated and saved. In this example, '''five stages of growth will be written to disk'''. These values are chosen to best capture the appearance of the tissue at important stages of the tissue growth.<br />
<br />
The final argument name listed here is ''modelname''. This is a model-specific argument, and in this case the GPT_CASE_RST interaction function contains '''three separate models for plant growth''', allowing the desired model to be a selected. Here, the value [1:3] is specified, which is evaluated in the same way as entering [1 2 3]. This instructs GFtboxCommand to run three separate simulations, one for each of the growth models contained in the interaction function.<br />
<br />
The function of every permissible argument is given by keying the following command into Matlab:<br />
<br />
help GFtboxCommand<br />
<br />
===3B - Getting results from a completed simulation===<br />
<br />
'''1) - Generating images from a simulation you ran on your computer'''<br />
<br />
Once a growth simulation is completed, the project folder (GPT_CASE_RST, in our example) will contain another folder named "movies". Within movies, there are folders which contain results for the executed simulations. As was instructed in 3A, here we can see three folders corresponding to simulations for the three separate models. Within the first folder there are three items:<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| ''CommandLine.txt'' - This file contains the Matlab command which was used to generate the results that this sub-directory contains.<br />
<br />
''gpt_case_rst.txt'' - This file, named according to the project name, contains a copy of the interaction function which was used to produce these results.<br />
<br />
''meshes'' - This folder contains the mesh files corresponding to the stages of growth specified by the value of the Stages argument, described in 3A.<br />
|width="500px" align="center"|[[File:resultsdir.png| Directory structure of a project containing results]]<br />
|}<br />
<br />
<br />
The mesh files contain vertices information regarding the shape of the growth model at a particular stage of growth, but are not visualisable in that form. In order to '''convert these mesh files into viewable images''', we can execute the following command in Matlab:<br />
<br />
VMSreport('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Project','GPT_CASE_RST',...<br />
'Experiment','All','flattentime',572.5,'morphogen','KPAR','SNAPFIG',true);<br />
<br />
Where the ''Path'' and ''Project'' arguments have the same function as ''Path'' and ''Name'' in 3A, i.e. ''Path'' refers to the location of the project folder and ''Project'' is the name of the project folder itself. The result images will be stored in the ''meshes'' directory of the project.<br />
<br />
Here we can see the generated images, including the command line arguments, for one of the simulations performed in 3A. Click on a thumbnail to view a larger image.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="125px"| <small>001commandline.png</small><br> [[File:commandlineexample.png | 100px]]<br />
|width="125px"| <small>001GPT_CASE_RST_2 wild_.png</small><br>[[File:GPT_CASE_RST_example1.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_3 wild_.png</small><br>[[File:GPT_CASE_RST_example2.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_4 wild_.png</small><br>[[File:GPT_CASE_RST_example3.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_5 wild_.png</small><br>[[File:GPT_CASE_RST_example4.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_6 wild_.png</small><br>[[File:GPT_CASE_RST_example5.png | 125px]]<br />
|}<br />
<br />
<br />
'''2) - Generating images from a simulation you ran on the cluster'''<br />
<br />
NB. The results produced may not be visibly identical to those in the published literature. This is because of small, random perturbations which are applied to the initial model meshes to stop them from containing surfaces which are perfectly flat, and therefore biologically unrealistic. The results produced should be '''qualitatively but not quantitatively the same'''.<br />
<br />
'''3) - Interacting with your results using the GFtbox GUI'''<br />
<br />
='''4''' Altering the simulation parameters=<br />
<br />
One such simulation parameter is ''dt'', which is the time in seconds between iterations in the growth simulation. Large values of ''dt'' mean that fewer steps and therefore fewer calculations are required to complete a simulation. Whereas smaller values of ''dt'' mean that more steps are required, and therefore more processing time too. Therefore, a value of ''dt'' must be selected which is not so small that it is computationally unmanageable, but not so large that the observed growth is an artifact of the value of ''dt'', rather than the underlying growth model. It is necessary therefore to test a range of values for ''dt'' to ensure that the patterns of growth observed in a simulation are consistent across the range, and to find a value which is sufficient to demonstrate the model of growth and computationally efficient.<br />
<br />
'''Testing a range of ''dt''s can be achieved in several ways.''' One way is to use the ''dt'' argument when calling GFtboxCommand. This allows a single value to be tested. Another way is to make a batch of jobs, each using a different value for ''dt'', by using the 'State' argument. Lastly, a range of ''dt''s can be specified in GFtboxCommand, by using the dt argument where the values within the square brackets are the ''dt''s to simulate:<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5]);<br />
<br />
Here, we have specified '''four separate values for ''dt''''', and this means that '''four separate simulations will be run''' sequentially on your computer. As in 3B, images can then be generated for the mesh files produced, and compared to ensure consistency across the range of ''dt''s. In this figure, we can see the same model, at the same stage of growth, generated using the four values of ''dt''. Though quantitatively different, they are qualitatively the same, illustrating the suitability of the default ''dt'' value of 5.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="190px"| [[File:GPT_CASE_RST_dt0.5.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt1.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt2.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt5.png| 190px]]<br />
|}<br />
<br />
='''5''' Altering the model parameters on the cluster=<br />
It is easy to see that when even a small number of range variables, mutations or ''dt''s are specified, the '''total number of simulations increases quickly'''. If many ranges are specified, then the amount of processing time becomes unmanageable on a single computer. Via ''GFtboxCommand'' and the ''ClusterMonitor'' tool, we can remotely''' run a number of simulations, in parallel, on a computing cluster'''. It is assumed that the GrowthToolbox and Matlab are installed on your cluster, and PuTTY is required on your local computer for the tools ''pscp'' (for transferring files from your computer to a cluster) and ''plink'' (for remotely executing commands on a cluster). Using the ''dt'' range argument from 4 as an example, here we add the 'Use' name argument with the value 'Cluster':<br />
<br />
GFtboxCommand('State','Start','Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5],'Use','Cluster');<br />
<br />
The 'Cluster' option for the 'Use' name argument instructs GFtboxCommand to upload the required project directory to a remote Linux server. Instead of running the simulation on your own computer, GFtboxCommand then works out how many individual simulations are specified by the command that invoked it. In this example, the only argument which will generate multiple simulations is ''dt'', of which there will be 4. Separate commands for each of these jobs are then automatically generated, each with an accompanying unique ID (The value of the ''ExpID'' name argument), and '''these are submitted as individual jobs to the cluster'''.<br />
<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.1,'ExpID','GPT_CASE_RST_1');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.5,'ExpID','GPT_CASE_RST_2');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',1,'ExpID','GPT_CASE_RST_3');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',5,'ExpID','GPT_CASE_RST_4');<br />
<br />
Once these jobs have been submitted, the ''ClusterMonitor'' tool opens and the new job batch ID is added to the list of jobs. Here, we will concentrate on three of the functions of ''ClusterMonitor'': ''Queue?'', ''Get project results'' and ''Make project pngs''. As in part 4 of this page, these functions will enable us to visualise the growth simulation results.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| This is an example of what you will see when the ClusterMonitor tool launches after submitting a job using the 'Use','Cluster' argument pair in GFtboxCommand.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="middle"<br />
|width="200px"| Clicking the 'Queue?' button tells ClusterMonitor to check the status of your jobs on the cluster. It then prints the status of the current jobs in Matlab's command window. In the case of our cluster, this is the equivalent of the ''qstatme'' command. This screenshot shows an example job list, where one job is currently running.<br />
<br />
This allows you too see whether or not your jobs have finished, and therefore whether or not their results are ready to retrieve.<br />
|width="600px"| [[File:queue.png| 600px]]<br />
|}<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Once your jobs have completed, the 'Queue?' button will show that there are no jobs running. Now you are ready to retrieve the project files from the cluster. This is achieved using the 'Get project results' button. Once pressed, the Matlab command window will show trace information to indicate which files are currently being downloaded. This is the equivalent of downloading the files manually, using an FTP program.<br />
|width="600px" align="center"| [[File:downloadingprojectfiles.png]]<br />
|}<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Once you have successfully downloaded your project results, you are ready to generate result images, as in part 3B1. Instead of entering a ''VMSReport'' command, you can press the 'Make project pngs' button, which will perform the same command. The result images that are created will be in the ''meshes'' folder within your project.<br />
|width="600px" align="center"| [[File:meshesfolder.png]]<br />
|}<br />
<br />
<br />
Whilst ''ClusterMonitor'' is a useful tool for managing your cluster jobs, it is advisable to '''have at least a rudimentary understanding of and ability to use a Unix-based computer system'''. In particular, the abilities to list the contents of a folder, view the contents of a file, change the current working folder, delete, and to copy or move files or folders, are essential Unix skills for making sure everything is working as intended. It is beyond the scope of these tutorials to provide an in depth Unix tutorial (many good and simple tutorials exist on the web), but here is a short description of the Unix commands (which may be specific to our Unix-based system) that we believe to be important.<br />
<br />
''ls'' - list the files and folders in a directory. Use the -al option (e.g. ls -al) to see modification dates, file permissions, sizes, etc.<br />
<br />
''cd X'' - Change the current directory to directory X.<br />
<br />
''rm X'' - Delete the file X.<br />
<br />
''cp X NewPath/X2'' - Copy file X to the NewPath directory and name the copy X2.<br />
<br />
''mv X NewPath/X2'' - Move file X to the NewPath directory and rename it to X2.<br />
<br />
''cat test.txt'' - Print the contents of the text file test.txt onto the screen.<br />
<br />
''pwd'' - Print the present working directory onto the screen.<br />
<br />
''more test.txt'' - Print the file test.txt to the screen, a screenful at a time, scrolling each time you hit the enter key.</div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Running_example_models_and_using_a_cluster&diff=5395Running example models and using a cluster2011-06-24T09:33:03Z<p>JacobNewman: </p>
<hr />
<div>[[GFtbox Tutorial pages | Back to GFtbox Tutorial pages]]<br />
<br />
The purpose of these tutorials is to learn how to run the example growth simulations included in GFtbox. We will describe methods for running simulations locally (on your own computer) and remotely (on a computing cluster). It is assumed that you have already downloaded the GFtbox software and have Matlab installed.<br />
<br />
=Getting Started=<br />
<br />
The remainder of this page is split into five sub-tutorials, each building on the preceding parts.<br />
<br />
1) '''Explaining the tools'''. In these tutorials, we will be using ''GFtboxCommand'' and ''ClusterMonitor''. This section explains the purpose of these tools. <br />
<br />
2) '''Computer or cluster?''' Here, we illustrate when and why you should use a computing cluster for your growth simulations, and conversely, when you should use your desktop computer.<br />
<br />
3) '''Running a growth simulation for an example model'''. This section demonstrates how to use ''GFtboxCommand'' to run a growth simulation on your computer. The model used for the simulation is an example model included with the GFtbox.<br />
<br />
4) '''Altering the simulation parameters'''. Following on from part 3, here we show how to adjust a simulation parameter within GFtboxCommand. Specifically, we alter the value of ''dt'', the temporal resolution of the simulation, and show how it can be used to verify that the value specified in the published literature is reasonable.<br />
<br />
5) '''Altering the model parameters on the cluster'''. Finally, we demonstrate how a number of model parameters can be varied by specifying a range of values for each model argument. We show how the computationally expensive task of simulating all combinations of specified ranges can be processed more efficiently if a computer cluster is used via the ''ClusterMonitor'' tool.<br />
<br />
='''1''' Explaining the tools=<br />
==GFtboxCommand==<br />
This is a command line version of the GFtbox. By command line, we mean that all program '''functions are operated via typed commands, without the GUI'''. Like GFtbox, GFtboxCommand is capable of running growth simulations of an interaction function, and allows the user to specify model and simulation parameters. Unlike GFtbox, this also allows the user to select '''ranges of values for a number of input parameters''', and will automatically spawn multiple simulations which explore the various combinations of those parameters. This can be used to evaluate the effect of various parameters on the growth of a given model.<br />
<br />
==ClusterMonitor==<br />
Provides a graphical user interface for managing simulations '''running remotely on a computer cluster'''. Specifically, it allows you to see which projects are present and running on the cluster, to retrieve the completed projects, to generate images of the simulations at specified stages of growth, and to remove projects from the cluster. If you do not intend to use a computer cluster, then you will not need to use ClusterMonitor.<br />
<br />
='''2''' Computer or cluster?=<br />
<br />
In basic terms, '''a computing cluster is effectively a network of many computer processors''' (often hundreds), centrally managed by a queuing system. When a job is submitted to a cluster, the job is sent to a processor that is not being used, or queued until one becomes available. In contrast, a typical desktop computer will contain one processor, limiting the number of tasks that can be performed at any one time. Jobs which must be run independently and sequentially on a desktop computer can be executed in parallel on a cluster, '''greatly reducing the total time required to complete all of the jobs'''. Although the exact details of your cluster might vary from those described in the rest of these tutorials, we aim to illustrate the generic processes involved in using GFtbox on a cluster. We are happy to offer assistance, where possible, to setup GFtbox for your cluster, so please contact us if you have any queries.<br />
<br />
Whilst the time savings offered by using a cluster can be significant, there is an overhead associated with returning the results to your personal computer. The total time to run 2 or more simulations on a cluster and to return the results will be less than running those simulations sequentially on one computer. Therefore, using a '''cluster is ideal for situations where you would like to run several simulations''', such as to evaluate the effect of a range of parameters on a growth model. You are not advised to use a cluster for running single simulations, or where you would like to step through a simulation and change parameters in a more interactive fashion. In such circumstances, you are advised to run GFtbox or GFtboxCommand on your own computer.<br />
<br />
The '''GFtbox GUI provides quick feedback''' about how the changes you have made to an interaction function have affected the growth simulation, as '''you can see the result of each simulation iteration as it completes'''. It is quick and easy therefore to see if you have dramatically changed the course of the growth simulation, and then to adjust the parameters according to your observations. This approach is well suited to the early stages of design, where you might wish to '''tweak some parameters to gauge whether or not they have had the desired effect'''. Or in the final stages, where you think your model is almost finalised, but where small adjustments are required. '''An alternative approach is to start many simulations''', based upon your initial model and by making intelligent choices about which parameters to explore. This allows you to '''harvest many results, to quickly and easily overview them''' in their finalised form, or to select interesting-looking simulations and examine them more closely on your desktop computer, using GFtbox.<br />
<br />
='''3''' Running a growth simulation for an example model=<br />
This tutorial is aimed at running a '''growth simulation for one of the example interaction functions''' included with the GFtbox. The purpose of this exercise is to firstly demonstrate how simulations can be invoked using GFtboxCommand, and secondly to show how to reproduce experimental results (specifically, those published in Kennaway et al (2011)) given an interaction function.<br />
<br />
Assuming Matlab is installed on your computer, and the latest GFtbox has been downloaded, you can add the GFtbox directory to Matlab's search path, which makes the toolbox accessible from any other path that you choose to work from. For a short tutorial on how to do this, please click [[Adding GFtbox to Matlab's search path | here]].<br />
<br />
Once the GFtbox is added to Matlab, you are ready to run a growth simulation using GFtboxCommand. In this example, '''the model that we will simulate is called GPT_CASE_RST'''. Results generated using this model are published in Kennaway et al (2011). By running this simulation, we can confirm the results in the published literature and investigate the suitability of the various parameters.<br />
<br />
===3A - Running a simulation on your computer===<br />
<br />
The following command can be typed into Matlab to run a simulation of the GPT_CASE_RST interaction function, which contains three growth models: R, S and T. '''Three separate simulations are run sequentially on your computer''', one for each model, each producing results corresponding to five intervals in the growth simulation.<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',[1:3]);<br />
<br />
GFtboxCommand accepts input arguments as name and value pairs, e.g. 'modelname', [1:3] or 'Use','Cluster'. The argument names entered in the example above are: ''Path'', ''Name'', ''Stages'' and ''modelname''.<br />
<br />
The optional '''''Path'' argument name refers to the location of the folder''' (or directory) on your computer where the model interaction function you wish to simulate is stored. '''''Name'', is the name of the folder itself'''. In this case, we are using the GPT_CASE_RST folder which is included in the GrowthToolbox. You may wish to copy this folder elsewhere, if you intend to make changes to the interaction function.<br />
<br />
During a growth simulation, a mesh can be generated at each time step of the simulation, which provides a visual representation of the growth of the biological tissue, given the various parameters of growth specified by the interaction function and how they have changed over time. Put another way, the mesh shows exactly what the growing tissue actually looks like. ''Stages'' refers to the points in the simulation (measured in hours) at which meshes should be generated and saved. In this example, '''five stages of growth will be written to disk'''. These values are chosen to best capture the appearance of the tissue at important stages of the tissue growth.<br />
<br />
The final argument name listed here is ''modelname''. This is a model-specific argument, and in this case the GPT_CASE_RST interaction function contains '''three separate models for plant growth''', allowing the desired model to be a selected. Here, the value [1:3] is specified, which is evaluated in the same way as entering [1 2 3]. This instructs GFtboxCommand to run three separate simulations, one for each of the growth models contained in the interaction function.<br />
<br />
The function of every permissible argument is given by keying the following command into Matlab:<br />
<br />
help GFtboxCommand<br />
<br />
===3B - Getting results from a completed simulation===<br />
<br />
'''1) - Generating images from a simulation you ran on your computer'''<br />
<br />
Once a growth simulation is completed, the project folder (GPT_CASE_RST, in our example) will contain another folder named "movies". Within movies, there are folders which contain results for the executed simulations. As was instructed in 3A, here we can see three folders corresponding to simulations for the three separate models. Within the first folder there are three items:<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| ''CommandLine.txt'' - This file contains the Matlab command which was used to generate the results that this sub-directory contains.<br />
<br />
''gpt_case_rst.txt'' - This file, named according to the project name, contains a copy of the interaction function which was used to produce these results.<br />
<br />
''meshes'' - This folder contains the mesh files corresponding to the stages of growth specified by the value of the Stages argument, described in 3A.<br />
|width="500px" align="center"|[[File:resultsdir.png| Directory structure of a project containing results]]<br />
|}<br />
<br />
<br />
The mesh files contain vertices information regarding the shape of the growth model at a particular stage of growth, but are not visualisable in that form. In order to '''convert these mesh files into viewable images''', we can execute the following command in Matlab:<br />
<br />
VMSreport('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Project','GPT_CASE_RST',...<br />
'Experiment','All','flattentime',572.5,'morphogen','KPAR','SNAPFIG',true);<br />
<br />
Where the ''Path'' and ''Project'' arguments have the same function as ''Path'' and ''Name'' in 3A, i.e. ''Path'' refers to the location of the project folder and ''Project'' is the name of the project folder itself. The result images will be stored in the ''meshes'' directory of the project.<br />
<br />
Here we can see the generated images, including the command line arguments, for one of the simulations performed in 3A. Click on a thumbnail to view a larger image.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="125px"| <small>001commandline.png</small><br> [[File:commandlineexample.png | 100px]]<br />
|width="125px"| <small>001GPT_CASE_RST_2 wild_.png</small><br>[[File:GPT_CASE_RST_example1.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_3 wild_.png</small><br>[[File:GPT_CASE_RST_example2.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_4 wild_.png</small><br>[[File:GPT_CASE_RST_example3.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_5 wild_.png</small><br>[[File:GPT_CASE_RST_example4.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_6 wild_.png</small><br>[[File:GPT_CASE_RST_example5.png | 125px]]<br />
|}<br />
<br />
<br />
'''2) - Generating images from a simulation you ran on the cluster'''<br />
<br />
NB. The results produced may not be visibly identical to those in the published literature. This is because of small, random perturbations which are applied to the initial model meshes to stop them from containing surfaces which are perfectly flat, and therefore biologically unrealistic. The results produced should be '''qualitatively but not quantitatively the same'''.<br />
<br />
'''3) - Interacting with your results using the GFtbox GUI'''<br />
<br />
='''4''' Altering the simulation parameters=<br />
<br />
One such simulation parameter is ''dt'', which is the time in seconds between iterations in the growth simulation. Large values of ''dt'' mean that fewer steps and therefore fewer calculations are required to complete a simulation. Whereas smaller values of ''dt'' mean that more steps are required, and therefore more processing time too. Therefore, a value of ''dt'' must be selected which is not so small that it is computationally unmanageable, but not so large that the observed growth is an artifact of the value of ''dt'', rather than the underlying growth model. It is necessary therefore to test a range of values for ''dt'' to ensure that the patterns of growth observed in a simulation are consistent across the range, and to find a value which is sufficient to demonstrate the model of growth and computationally efficient.<br />
<br />
'''Testing a range of ''dt''s can be achieved in several ways.''' One way is to use the ''dt'' argument when calling GFtboxCommand. This allows a single value to be tested. Another way is to make a batch of jobs, each using a different value for ''dt'', by using the 'State' argument. Lastly, a range of ''dt''s can be specified in GFtboxCommand, by using the dt argument where the values within the square brackets are the ''dt''s to simulate:<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5]);<br />
<br />
Here, we have specified '''four separate values for ''dt''''', and this means that '''four separate simulations will be run''' sequentially on your computer. As in 3B, images can then be generated for the mesh files produced, and compared to ensure consistency across the range of ''dt''s. In this figure, we can see the same model, at the same stage of growth, generated using the four values of ''dt''. Though quantitatively different, they are qualitatively the same, illustrating the suitability of the default ''dt'' value of 5.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="190px"| [[File:GPT_CASE_RST_dt0.5.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt1.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt2.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt5.png| 190px]]<br />
|}<br />
<br />
='''5''' Altering the model parameters on the cluster=<br />
It is easy to see that when even a small number of range variables, mutations or ''dt''s are specified, the '''total number of simulations increases quickly'''. If many ranges are specified, then the amount of processing time becomes unmanageable on a single computer. Via ''GFtboxCommand'' and the ''ClusterMonitor'' tool, we can remotely''' run a number of simulations, in parallel, on a computing cluster'''. It is assumed that the GrowthToolbox and Matlab are installed on your cluster, and PuTTY is required on your local computer for the tools ''pscp'' (for transferring files from your computer to a cluster) and ''plink'' (for remotely executing commands on a cluster). Using the ''dt'' range argument from 4 as an example, here we add the 'Use' name argument with the value 'Cluster':<br />
<br />
GFtboxCommand('State','Start','Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5],'Use','Cluster');<br />
<br />
The 'Cluster' option for the 'Use' name argument instructs GFtboxCommand to upload the required project directory to a remote Linux server. Instead of running the simulation on your own computer, GFtboxCommand then works out how many individual simulations are specified by the command that invoked it. In this example, the only argument which will generate multiple simulations is ''dt'', of which there will be 4. Separate commands for each of these jobs are then automatically generated, each with an accompanying unique ID (The value of the ''ExpID'' name argument), and '''these are submitted as individual jobs to the cluster'''.<br />
<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.1,'ExpID','GPT_CASE_RST_1');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.5,'ExpID','GPT_CASE_RST_2');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',1,'ExpID','GPT_CASE_RST_3');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',5,'ExpID','GPT_CASE_RST_4');<br />
<br />
Once these jobs have been submitted, the ''ClusterMonitor'' tool opens and the new job batch ID is added to the list of jobs. Here, we will concentrate on three of the functions of ''ClusterMonitor'': ''Queue?'', ''Get project results'' and ''Make project pngs''. As in part 4 of this page, these functions will enable us to visualise the growth simulation results.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| This is an example of what you will see when the ClusterMonitor tool launches after submitting a job using the 'Use','Cluster' argument pair in GFtboxCommand.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="middle"<br />
|width="200px"| Clicking the 'Queue?' button tells ClusterMonitor to check the status of your jobs on the cluster. It then prints the status of the current jobs in Matlab's command window. In the case of our cluster, this is the equivalent of the ''qstatme'' command. This screenshot shows an example job list, where one job is currently running.<br />
<br />
This allows you too see whether or not your jobs have finished, and therefore whether or not their results are ready to retrieve.<br />
|width="600px"| [[File:queue.png| 600px]]<br />
|}<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Once your jobs have completed, the 'Queue?' button will show that there are no jobs running. Now you are ready to retrieve the project files from the cluster. This is achieved using the 'Get project results' button. Once pressed, the Matlab command window will show trace information to indicate which files are currently being downloaded. This is the equivalent of downloading the files manually, using an FTP program.<br />
|width="600px" align="center"| [[File:downloadingprojectfiles.png]]<br />
|}<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Once you have successfully downloaded your project results, you are ready to generate result images, as in part 3B1. Instead of entering a ''VMSReport'' command, you can press the 'Make project pngs' button, which will perform the same command.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
<br />
<br />
Whilst ''ClusterMonitor'' is a useful tool for managing your cluster jobs, it is advisable to '''have at least a rudimentary understanding of and ability to use a Unix-based computer system'''. In particular, the abilities to list the contents of a folder, view the contents of a file, change the current working folder, delete, and to copy or move files or folders, are essential Unix skills for making sure everything is working as intended. It is beyond the scope of these tutorials to provide an in depth Unix tutorial (many good and simple tutorials exist on the web), but here is a short description of the Unix commands (which may be specific to our Unix-based system) that we believe to be important.<br />
<br />
''ls'' - list the files and folders in a directory. Use the -al option (e.g. ls -al) to see modification dates, file permissions, sizes, etc.<br />
<br />
''cd X'' - Change the current directory to directory X.<br />
<br />
''rm X'' - Delete the file X.<br />
<br />
''cp X NewPath/X2'' - Copy file X to the NewPath directory and name the copy X2.<br />
<br />
''mv X NewPath/X2'' - Move file X to the NewPath directory and rename it to X2.<br />
<br />
''cat test.txt'' - Print the contents of the text file test.txt onto the screen.<br />
<br />
''pwd'' - Print the present working directory onto the screen.<br />
<br />
''more test.txt'' - Print the file test.txt to the screen, a screenful at a time, scrolling each time you hit the enter key.</div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Running_example_models_and_using_a_cluster&diff=5394Running example models and using a cluster2011-06-24T09:32:10Z<p>JacobNewman: </p>
<hr />
<div>[[GFtbox Tutorial pages | Back to GFtbox Tutorial pages]]<br />
<br />
The purpose of these tutorials is to learn how to run the example growth simulations included in GFtbox. We will describe methods for running simulations locally (on your own computer) and remotely (on a computing cluster). It is assumed that you have already downloaded the GFtbox software and have Matlab installed.<br />
<br />
=Getting Started=<br />
<br />
The remainder of this page is split into five sub-tutorials, each building on the preceding parts.<br />
<br />
1) '''Explaining the tools'''. In these tutorials, we will be using ''GFtboxCommand'' and ''ClusterMonitor''. This section explains the purpose of these tools. <br />
<br />
2) '''Computer or cluster?''' Here, we illustrate when and why you should use a computing cluster for your growth simulations, and conversely, when you should use your desktop computer.<br />
<br />
3) '''Running a growth simulation for an example model'''. This section demonstrates how to use ''GFtboxCommand'' to run a growth simulation on your computer. The model used for the simulation is an example model included with the GFtbox.<br />
<br />
4) '''Altering the simulation parameters'''. Following on from part 3, here we show how to adjust a simulation parameter within GFtboxCommand. Specifically, we alter the value of ''dt'', the temporal resolution of the simulation, and show how it can be used to verify that the value specified in the published literature is reasonable.<br />
<br />
5) '''Altering the model parameters on the cluster'''. Finally, we demonstrate how a number of model parameters can be varied by specifying a range of values for each model argument. We show how the computationally expensive task of simulating all combinations of specified ranges can be processed more efficiently if a computer cluster is used via the ''ClusterMonitor'' tool.<br />
<br />
='''1''' Explaining the tools=<br />
==GFtboxCommand==<br />
This is a command line version of the GFtbox. By command line, we mean that all program '''functions are operated via typed commands, without the GUI'''. Like GFtbox, GFtboxCommand is capable of running growth simulations of an interaction function, and allows the user to specify model and simulation parameters. Unlike GFtbox, this also allows the user to select '''ranges of values for a number of input parameters''', and will automatically spawn multiple simulations which explore the various combinations of those parameters. This can be used to evaluate the effect of various parameters on the growth of a given model.<br />
<br />
==ClusterMonitor==<br />
Provides a graphical user interface for managing simulations '''running remotely on a computer cluster'''. Specifically, it allows you to see which projects are present and running on the cluster, to retrieve the completed projects, to generate images of the simulations at specified stages of growth, and to remove projects from the cluster. If you do not intend to use a computer cluster, then you will not need to use ClusterMonitor.<br />
<br />
='''2''' Computer or cluster?=<br />
<br />
In basic terms, '''a computing cluster is effectively a network of many computer processors''' (often hundreds), centrally managed by a queuing system. When a job is submitted to a cluster, the job is sent to a processor that is not being used, or queued until one becomes available. In contrast, a typical desktop computer will contain one processor, limiting the number of tasks that can be performed at any one time. Jobs which must be run independently and sequentially on a desktop computer can be executed in parallel on a cluster, '''greatly reducing the total time required to complete all of the jobs'''. Although the exact details of your cluster might vary from those described in the rest of these tutorials, we aim to illustrate the generic processes involved in using GFtbox on a cluster. We are happy to offer assistance, where possible, to setup GFtbox for your cluster, so please contact us if you have any queries.<br />
<br />
Whilst the time savings offered by using a cluster can be significant, there is an overhead associated with returning the results to your personal computer. The total time to run 2 or more simulations on a cluster and to return the results will be less than running those simulations sequentially on one computer. Therefore, using a '''cluster is ideal for situations where you would like to run several simulations''', such as to evaluate the effect of a range of parameters on a growth model. You are not advised to use a cluster for running single simulations, or where you would like to step through a simulation and change parameters in a more interactive fashion. In such circumstances, you are advised to run GFtbox or GFtboxCommand on your own computer.<br />
<br />
The '''GFtbox GUI provides quick feedback''' about how the changes you have made to an interaction function have affected the growth simulation, as '''you can see the result of each simulation iteration as it completes'''. It is quick and easy therefore to see if you have dramatically changed the course of the growth simulation, and then to adjust the parameters according to your observations. This approach is well suited to the early stages of design, where you might wish to '''tweak some parameters to gauge whether or not they have had the desired effect'''. Or in the final stages, where you think your model is almost finalised, but where small adjustments are required. '''An alternative approach is to start many simulations''', based upon your initial model and by making intelligent choices about which parameters to explore. This allows you to '''harvest many results, to quickly and easily overview them''' in their finalised form, or to select interesting-looking simulations and examine them more closely on your desktop computer, using GFtbox.<br />
<br />
='''3''' Running a growth simulation for an example model=<br />
This tutorial is aimed at running a '''growth simulation for one of the example interaction functions''' included with the GFtbox. The purpose of this exercise is to firstly demonstrate how simulations can be invoked using GFtboxCommand, and secondly to show how to reproduce experimental results (specifically, those published in Kennaway et al (2011)) given an interaction function.<br />
<br />
Assuming Matlab is installed on your computer, and the latest GFtbox has been downloaded, you can add the GFtbox directory to Matlab's search path, which makes the toolbox accessible from any other path that you choose to work from. For a short tutorial on how to do this, please click [[Adding GFtbox to Matlab's search path | here]].<br />
<br />
Once the GFtbox is added to Matlab, you are ready to run a growth simulation using GFtboxCommand. In this example, '''the model that we will simulate is called GPT_CASE_RST'''. Results generated using this model are published in Kennaway et al (2011). By running this simulation, we can confirm the results in the published literature and investigate the suitability of the various parameters.<br />
<br />
===3A - Running a simulation on your computer===<br />
<br />
The following command can be typed into Matlab to run a simulation of the GPT_CASE_RST interaction function, which contains three growth models: R, S and T. '''Three separate simulations are run sequentially on your computer''', one for each model, each producing results corresponding to five intervals in the growth simulation.<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',[1:3]);<br />
<br />
GFtboxCommand accepts input arguments as name and value pairs, e.g. 'modelname', [1:3] or 'Use','Cluster'. The argument names entered in the example above are: ''Path'', ''Name'', ''Stages'' and ''modelname''.<br />
<br />
The optional '''''Path'' argument name refers to the location of the folder''' (or directory) on your computer where the model interaction function you wish to simulate is stored. '''''Name'', is the name of the folder itself'''. In this case, we are using the GPT_CASE_RST folder which is included in the GrowthToolbox. You may wish to copy this folder elsewhere, if you intend to make changes to the interaction function.<br />
<br />
During a growth simulation, a mesh can be generated at each time step of the simulation, which provides a visual representation of the growth of the biological tissue, given the various parameters of growth specified by the interaction function and how they have changed over time. Put another way, the mesh shows exactly what the growing tissue actually looks like. ''Stages'' refers to the points in the simulation (measured in hours) at which meshes should be generated and saved. In this example, '''five stages of growth will be written to disk'''. These values are chosen to best capture the appearance of the tissue at important stages of the tissue growth.<br />
<br />
The final argument name listed here is ''modelname''. This is a model-specific argument, and in this case the GPT_CASE_RST interaction function contains '''three separate models for plant growth''', allowing the desired model to be a selected. Here, the value [1:3] is specified, which is evaluated in the same way as entering [1 2 3]. This instructs GFtboxCommand to run three separate simulations, one for each of the growth models contained in the interaction function.<br />
<br />
The function of every permissible argument is given by keying the following command into Matlab:<br />
<br />
help GFtboxCommand<br />
<br />
===3B - Getting results from a completed simulation===<br />
<br />
'''1) - Generating images from a simulation you ran on your computer'''<br />
<br />
Once a growth simulation is completed, the project folder (GPT_CASE_RST, in our example) will contain another folder named "movies". Within movies, there are folders which contain results for the executed simulations. As was instructed in 3A, here we can see three folders corresponding to simulations for the three separate models. Within the first folder there are three items:<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| ''CommandLine.txt'' - This file contains the Matlab command which was used to generate the results that this sub-directory contains.<br />
<br />
''gpt_case_rst.txt'' - This file, named according to the project name, contains a copy of the interaction function which was used to produce these results.<br />
<br />
''meshes'' - This folder contains the mesh files corresponding to the stages of growth specified by the value of the Stages argument, described in 3A.<br />
|width="500px" align="center"|[[File:resultsdir.png| Directory structure of a project containing results]]<br />
|}<br />
<br />
<br />
The mesh files contain vertices information regarding the shape of the growth model at a particular stage of growth, but are not visualisable in that form. In order to '''convert these mesh files into viewable images''', we can execute the following command in Matlab:<br />
<br />
VMSreport('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Project','GPT_CASE_RST',...<br />
'Experiment','All','flattentime',572.5,'morphogen','KPAR','SNAPFIG',true);<br />
<br />
Where the ''Path'' and ''Project'' arguments have the same function as ''Path'' and ''Name'' in 3A, i.e. ''Path'' refers to the location of the project folder and ''Project'' is the name of the project folder itself. The result images will be stored in the ''meshes'' directory of the project.<br />
<br />
Here we can see the generated images, including the command line arguments, for one of the simulations performed in 3A. Click on a thumbnail to view a larger image.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="125px"| <small>001commandline.png</small><br> [[File:commandlineexample.png | 100px]]<br />
|width="125px"| <small>001GPT_CASE_RST_2 wild_.png</small><br>[[File:GPT_CASE_RST_example1.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_3 wild_.png</small><br>[[File:GPT_CASE_RST_example2.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_4 wild_.png</small><br>[[File:GPT_CASE_RST_example3.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_5 wild_.png</small><br>[[File:GPT_CASE_RST_example4.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_6 wild_.png</small><br>[[File:GPT_CASE_RST_example5.png | 125px]]<br />
|}<br />
<br />
<br />
'''2) - Generating images from a simulation you ran on the cluster'''<br />
<br />
NB. The results produced may not be visibly identical to those in the published literature. This is because of small, random perturbations which are applied to the initial model meshes to stop them from containing surfaces which are perfectly flat, and therefore biologically unrealistic. The results produced should be '''qualitatively but not quantitatively the same'''.<br />
<br />
'''3) - Interacting with your results using the GFtbox GUI'''<br />
<br />
='''4''' Altering the simulation parameters=<br />
<br />
One such simulation parameter is ''dt'', which is the time in seconds between iterations in the growth simulation. Large values of ''dt'' mean that fewer steps and therefore fewer calculations are required to complete a simulation. Whereas smaller values of ''dt'' mean that more steps are required, and therefore more processing time too. Therefore, a value of ''dt'' must be selected which is not so small that it is computationally unmanageable, but not so large that the observed growth is an artifact of the value of ''dt'', rather than the underlying growth model. It is necessary therefore to test a range of values for ''dt'' to ensure that the patterns of growth observed in a simulation are consistent across the range, and to find a value which is sufficient to demonstrate the model of growth and computationally efficient.<br />
<br />
'''Testing a range of ''dt''s can be achieved in several ways.''' One way is to use the ''dt'' argument when calling GFtboxCommand. This allows a single value to be tested. Another way is to make a batch of jobs, each using a different value for ''dt'', by using the 'State' argument. Lastly, a range of ''dt''s can be specified in GFtboxCommand, by using the dt argument where the values within the square brackets are the ''dt''s to simulate:<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5]);<br />
<br />
Here, we have specified '''four separate values for ''dt''''', and this means that '''four separate simulations will be run''' sequentially on your computer. As in 3B, images can then be generated for the mesh files produced, and compared to ensure consistency across the range of ''dt''s. In this figure, we can see the same model, at the same stage of growth, generated using the four values of ''dt''. Though quantitatively different, they are qualitatively the same, illustrating the suitability of the default ''dt'' value of 5.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="190px"| [[File:GPT_CASE_RST_dt0.5.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt1.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt2.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt5.png| 190px]]<br />
|}<br />
<br />
='''5''' Altering the model parameters on the cluster=<br />
It is easy to see that when even a small number of range variables, mutations or ''dt''s are specified, the '''total number of simulations increases quickly'''. If many ranges are specified, then the amount of processing time becomes unmanageable on a single computer. Via ''GFtboxCommand'' and the ''ClusterMonitor'' tool, we can remotely''' run a number of simulations, in parallel, on a computing cluster'''. It is assumed that the GrowthToolbox and Matlab are installed on your cluster, and PuTTY is required on your local computer for the tools ''pscp'' (for transferring files from your computer to a cluster) and ''plink'' (for remotely executing commands on a cluster). Using the ''dt'' range argument from 4 as an example, here we add the 'Use' name argument with the value 'Cluster':<br />
<br />
GFtboxCommand('State','Start','Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5],'Use','Cluster');<br />
<br />
The 'Cluster' option for the 'Use' name argument instructs GFtboxCommand to upload the required project directory to a remote Linux server. Instead of running the simulation on your own computer, GFtboxCommand then works out how many individual simulations are specified by the command that invoked it. In this example, the only argument which will generate multiple simulations is ''dt'', of which there will be 4. Separate commands for each of these jobs are then automatically generated, each with an accompanying unique ID (The value of the ''ExpID'' name argument), and '''these are submitted as individual jobs to the cluster'''.<br />
<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.1,'ExpID','GPT_CASE_RST_1');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.5,'ExpID','GPT_CASE_RST_2');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',1,'ExpID','GPT_CASE_RST_3');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',5,'ExpID','GPT_CASE_RST_4');<br />
<br />
Once these jobs have been submitted, the ''ClusterMonitor'' tool opens and the new job batch ID is added to the list of jobs. Here, we will concentrate on three of the functions of ''ClusterMonitor'': ''Queue?'', ''Get project results'' and ''Make project pngs''. As in part 4 of this page, these functions will enable us to visualise the growth simulation results.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| This is an example of what you will see when the ClusterMonitor tool launches after submitting a job using the 'Use','Cluster' argument pair in GFtboxCommand.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="middle"<br />
|width="200px"| Clicking the 'Queue?' button tells ClusterMonitor to check the status of your jobs on the cluster. It then prints the status of the current jobs in Matlab's command window. In the case of our cluster, this is the equivalent of the ''qstatme'' command. This screenshot shows an example job list, where one job is currently running.<br />
<br />
This allows you too see whether or not your jobs have finished, and therefore whether or not their results are ready to retrieve.<br />
|width="600px"| [[File:queue.png| 600px]]<br />
|}<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Once your jobs have completed, the 'Queue?' button will show that there are no jobs running. Now you are ready to retrieve the project files from the cluster. This is achieved using the 'Get project results' button. Once pressed, the Matlab command window will show trace information to indicate which files are currently being downloaded. This is the equivalent of downloading the files manually, using an FTP program.<br />
|width="600px"| [[File:downloadingprojectfiles.png]]<br />
|}<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Once you have successfully downloaded your project results, you are ready to generate result images, as in part 3B1. Instead of entering a ''VMSReport'' command, you can press the 'Make project pngs' button, which will perform the same command.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
<br />
<br />
Whilst ''ClusterMonitor'' is a useful tool for managing your cluster jobs, it is advisable to '''have at least a rudimentary understanding of and ability to use a Unix-based computer system'''. In particular, the abilities to list the contents of a folder, view the contents of a file, change the current working folder, delete, and to copy or move files or folders, are essential Unix skills for making sure everything is working as intended. It is beyond the scope of these tutorials to provide an in depth Unix tutorial (many good and simple tutorials exist on the web), but here is a short description of the Unix commands (which may be specific to our Unix-based system) that we believe to be important.<br />
<br />
''ls'' - list the files and folders in a directory. Use the -al option (e.g. ls -al) to see modification dates, file permissions, sizes, etc.<br />
<br />
''cd X'' - Change the current directory to directory X.<br />
<br />
''rm X'' - Delete the file X.<br />
<br />
''cp X NewPath/X2'' - Copy file X to the NewPath directory and name the copy X2.<br />
<br />
''mv X NewPath/X2'' - Move file X to the NewPath directory and rename it to X2.<br />
<br />
''cat test.txt'' - Print the contents of the text file test.txt onto the screen.<br />
<br />
''pwd'' - Print the present working directory onto the screen.<br />
<br />
''more test.txt'' - Print the file test.txt to the screen, a screenful at a time, scrolling each time you hit the enter key.</div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=File:Downloadingprojectfiles.png&diff=5515File:Downloadingprojectfiles.png2011-06-24T09:31:18Z<p>JacobNewman: </p>
<hr />
<div></div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Running_example_models_and_using_a_cluster&diff=5393Running example models and using a cluster2011-06-24T09:31:03Z<p>JacobNewman: </p>
<hr />
<div>[[GFtbox Tutorial pages | Back to GFtbox Tutorial pages]]<br />
<br />
The purpose of these tutorials is to learn how to run the example growth simulations included in GFtbox. We will describe methods for running simulations locally (on your own computer) and remotely (on a computing cluster). It is assumed that you have already downloaded the GFtbox software and have Matlab installed.<br />
<br />
=Getting Started=<br />
<br />
The remainder of this page is split into five sub-tutorials, each building on the preceding parts.<br />
<br />
1) '''Explaining the tools'''. In these tutorials, we will be using ''GFtboxCommand'' and ''ClusterMonitor''. This section explains the purpose of these tools. <br />
<br />
2) '''Computer or cluster?''' Here, we illustrate when and why you should use a computing cluster for your growth simulations, and conversely, when you should use your desktop computer.<br />
<br />
3) '''Running a growth simulation for an example model'''. This section demonstrates how to use ''GFtboxCommand'' to run a growth simulation on your computer. The model used for the simulation is an example model included with the GFtbox.<br />
<br />
4) '''Altering the simulation parameters'''. Following on from part 3, here we show how to adjust a simulation parameter within GFtboxCommand. Specifically, we alter the value of ''dt'', the temporal resolution of the simulation, and show how it can be used to verify that the value specified in the published literature is reasonable.<br />
<br />
5) '''Altering the model parameters on the cluster'''. Finally, we demonstrate how a number of model parameters can be varied by specifying a range of values for each model argument. We show how the computationally expensive task of simulating all combinations of specified ranges can be processed more efficiently if a computer cluster is used via the ''ClusterMonitor'' tool.<br />
<br />
='''1''' Explaining the tools=<br />
==GFtboxCommand==<br />
This is a command line version of the GFtbox. By command line, we mean that all program '''functions are operated via typed commands, without the GUI'''. Like GFtbox, GFtboxCommand is capable of running growth simulations of an interaction function, and allows the user to specify model and simulation parameters. Unlike GFtbox, this also allows the user to select '''ranges of values for a number of input parameters''', and will automatically spawn multiple simulations which explore the various combinations of those parameters. This can be used to evaluate the effect of various parameters on the growth of a given model.<br />
<br />
==ClusterMonitor==<br />
Provides a graphical user interface for managing simulations '''running remotely on a computer cluster'''. Specifically, it allows you to see which projects are present and running on the cluster, to retrieve the completed projects, to generate images of the simulations at specified stages of growth, and to remove projects from the cluster. If you do not intend to use a computer cluster, then you will not need to use ClusterMonitor.<br />
<br />
='''2''' Computer or cluster?=<br />
<br />
In basic terms, '''a computing cluster is effectively a network of many computer processors''' (often hundreds), centrally managed by a queuing system. When a job is submitted to a cluster, the job is sent to a processor that is not being used, or queued until one becomes available. In contrast, a typical desktop computer will contain one processor, limiting the number of tasks that can be performed at any one time. Jobs which must be run independently and sequentially on a desktop computer can be executed in parallel on a cluster, '''greatly reducing the total time required to complete all of the jobs'''. Although the exact details of your cluster might vary from those described in the rest of these tutorials, we aim to illustrate the generic processes involved in using GFtbox on a cluster. We are happy to offer assistance, where possible, to setup GFtbox for your cluster, so please contact us if you have any queries.<br />
<br />
Whilst the time savings offered by using a cluster can be significant, there is an overhead associated with returning the results to your personal computer. The total time to run 2 or more simulations on a cluster and to return the results will be less than running those simulations sequentially on one computer. Therefore, using a '''cluster is ideal for situations where you would like to run several simulations''', such as to evaluate the effect of a range of parameters on a growth model. You are not advised to use a cluster for running single simulations, or where you would like to step through a simulation and change parameters in a more interactive fashion. In such circumstances, you are advised to run GFtbox or GFtboxCommand on your own computer.<br />
<br />
The '''GFtbox GUI provides quick feedback''' about how the changes you have made to an interaction function have affected the growth simulation, as '''you can see the result of each simulation iteration as it completes'''. It is quick and easy therefore to see if you have dramatically changed the course of the growth simulation, and then to adjust the parameters according to your observations. This approach is well suited to the early stages of design, where you might wish to '''tweak some parameters to gauge whether or not they have had the desired effect'''. Or in the final stages, where you think your model is almost finalised, but where small adjustments are required. '''An alternative approach is to start many simulations''', based upon your initial model and by making intelligent choices about which parameters to explore. This allows you to '''harvest many results, to quickly and easily overview them''' in their finalised form, or to select interesting-looking simulations and examine them more closely on your desktop computer, using GFtbox.<br />
<br />
='''3''' Running a growth simulation for an example model=<br />
This tutorial is aimed at running a '''growth simulation for one of the example interaction functions''' included with the GFtbox. The purpose of this exercise is to firstly demonstrate how simulations can be invoked using GFtboxCommand, and secondly to show how to reproduce experimental results (specifically, those published in Kennaway et al (2011)) given an interaction function.<br />
<br />
Assuming Matlab is installed on your computer, and the latest GFtbox has been downloaded, you can add the GFtbox directory to Matlab's search path, which makes the toolbox accessible from any other path that you choose to work from. For a short tutorial on how to do this, please click [[Adding GFtbox to Matlab's search path | here]].<br />
<br />
Once the GFtbox is added to Matlab, you are ready to run a growth simulation using GFtboxCommand. In this example, '''the model that we will simulate is called GPT_CASE_RST'''. Results generated using this model are published in Kennaway et al (2011). By running this simulation, we can confirm the results in the published literature and investigate the suitability of the various parameters.<br />
<br />
===3A - Running a simulation on your computer===<br />
<br />
The following command can be typed into Matlab to run a simulation of the GPT_CASE_RST interaction function, which contains three growth models: R, S and T. '''Three separate simulations are run sequentially on your computer''', one for each model, each producing results corresponding to five intervals in the growth simulation.<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',[1:3]);<br />
<br />
GFtboxCommand accepts input arguments as name and value pairs, e.g. 'modelname', [1:3] or 'Use','Cluster'. The argument names entered in the example above are: ''Path'', ''Name'', ''Stages'' and ''modelname''.<br />
<br />
The optional '''''Path'' argument name refers to the location of the folder''' (or directory) on your computer where the model interaction function you wish to simulate is stored. '''''Name'', is the name of the folder itself'''. In this case, we are using the GPT_CASE_RST folder which is included in the GrowthToolbox. You may wish to copy this folder elsewhere, if you intend to make changes to the interaction function.<br />
<br />
During a growth simulation, a mesh can be generated at each time step of the simulation, which provides a visual representation of the growth of the biological tissue, given the various parameters of growth specified by the interaction function and how they have changed over time. Put another way, the mesh shows exactly what the growing tissue actually looks like. ''Stages'' refers to the points in the simulation (measured in hours) at which meshes should be generated and saved. In this example, '''five stages of growth will be written to disk'''. These values are chosen to best capture the appearance of the tissue at important stages of the tissue growth.<br />
<br />
The final argument name listed here is ''modelname''. This is a model-specific argument, and in this case the GPT_CASE_RST interaction function contains '''three separate models for plant growth''', allowing the desired model to be a selected. Here, the value [1:3] is specified, which is evaluated in the same way as entering [1 2 3]. This instructs GFtboxCommand to run three separate simulations, one for each of the growth models contained in the interaction function.<br />
<br />
The function of every permissible argument is given by keying the following command into Matlab:<br />
<br />
help GFtboxCommand<br />
<br />
===3B - Getting results from a completed simulation===<br />
<br />
'''1) - Generating images from a simulation you ran on your computer'''<br />
<br />
Once a growth simulation is completed, the project folder (GPT_CASE_RST, in our example) will contain another folder named "movies". Within movies, there are folders which contain results for the executed simulations. As was instructed in 3A, here we can see three folders corresponding to simulations for the three separate models. Within the first folder there are three items:<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| ''CommandLine.txt'' - This file contains the Matlab command which was used to generate the results that this sub-directory contains.<br />
<br />
''gpt_case_rst.txt'' - This file, named according to the project name, contains a copy of the interaction function which was used to produce these results.<br />
<br />
''meshes'' - This folder contains the mesh files corresponding to the stages of growth specified by the value of the Stages argument, described in 3A.<br />
|width="500px" align="center"|[[File:resultsdir.png| Directory structure of a project containing results]]<br />
|}<br />
<br />
<br />
The mesh files contain vertices information regarding the shape of the growth model at a particular stage of growth, but are not visualisable in that form. In order to '''convert these mesh files into viewable images''', we can execute the following command in Matlab:<br />
<br />
VMSreport('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Project','GPT_CASE_RST',...<br />
'Experiment','All','flattentime',572.5,'morphogen','KPAR','SNAPFIG',true);<br />
<br />
Where the ''Path'' and ''Project'' arguments have the same function as ''Path'' and ''Name'' in 3A, i.e. ''Path'' refers to the location of the project folder and ''Project'' is the name of the project folder itself. The result images will be stored in the ''meshes'' directory of the project.<br />
<br />
Here we can see the generated images, including the command line arguments, for one of the simulations performed in 3A. Click on a thumbnail to view a larger image.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="125px"| <small>001commandline.png</small><br> [[File:commandlineexample.png | 100px]]<br />
|width="125px"| <small>001GPT_CASE_RST_2 wild_.png</small><br>[[File:GPT_CASE_RST_example1.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_3 wild_.png</small><br>[[File:GPT_CASE_RST_example2.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_4 wild_.png</small><br>[[File:GPT_CASE_RST_example3.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_5 wild_.png</small><br>[[File:GPT_CASE_RST_example4.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_6 wild_.png</small><br>[[File:GPT_CASE_RST_example5.png | 125px]]<br />
|}<br />
<br />
<br />
'''2) - Generating images from a simulation you ran on the cluster'''<br />
<br />
NB. The results produced may not be visibly identical to those in the published literature. This is because of small, random perturbations which are applied to the initial model meshes to stop them from containing surfaces which are perfectly flat, and therefore biologically unrealistic. The results produced should be '''qualitatively but not quantitatively the same'''.<br />
<br />
'''3) - Interacting with your results using the GFtbox GUI'''<br />
<br />
='''4''' Altering the simulation parameters=<br />
<br />
One such simulation parameter is ''dt'', which is the time in seconds between iterations in the growth simulation. Large values of ''dt'' mean that fewer steps and therefore fewer calculations are required to complete a simulation. Whereas smaller values of ''dt'' mean that more steps are required, and therefore more processing time too. Therefore, a value of ''dt'' must be selected which is not so small that it is computationally unmanageable, but not so large that the observed growth is an artifact of the value of ''dt'', rather than the underlying growth model. It is necessary therefore to test a range of values for ''dt'' to ensure that the patterns of growth observed in a simulation are consistent across the range, and to find a value which is sufficient to demonstrate the model of growth and computationally efficient.<br />
<br />
'''Testing a range of ''dt''s can be achieved in several ways.''' One way is to use the ''dt'' argument when calling GFtboxCommand. This allows a single value to be tested. Another way is to make a batch of jobs, each using a different value for ''dt'', by using the 'State' argument. Lastly, a range of ''dt''s can be specified in GFtboxCommand, by using the dt argument where the values within the square brackets are the ''dt''s to simulate:<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5]);<br />
<br />
Here, we have specified '''four separate values for ''dt''''', and this means that '''four separate simulations will be run''' sequentially on your computer. As in 3B, images can then be generated for the mesh files produced, and compared to ensure consistency across the range of ''dt''s. In this figure, we can see the same model, at the same stage of growth, generated using the four values of ''dt''. Though quantitatively different, they are qualitatively the same, illustrating the suitability of the default ''dt'' value of 5.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="190px"| [[File:GPT_CASE_RST_dt0.5.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt1.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt2.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt5.png| 190px]]<br />
|}<br />
<br />
='''5''' Altering the model parameters on the cluster=<br />
It is easy to see that when even a small number of range variables, mutations or ''dt''s are specified, the '''total number of simulations increases quickly'''. If many ranges are specified, then the amount of processing time becomes unmanageable on a single computer. Via ''GFtboxCommand'' and the ''ClusterMonitor'' tool, we can remotely''' run a number of simulations, in parallel, on a computing cluster'''. It is assumed that the GrowthToolbox and Matlab are installed on your cluster, and PuTTY is required on your local computer for the tools ''pscp'' (for transferring files from your computer to a cluster) and ''plink'' (for remotely executing commands on a cluster). Using the ''dt'' range argument from 4 as an example, here we add the 'Use' name argument with the value 'Cluster':<br />
<br />
GFtboxCommand('State','Start','Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5],'Use','Cluster');<br />
<br />
The 'Cluster' option for the 'Use' name argument instructs GFtboxCommand to upload the required project directory to a remote Linux server. Instead of running the simulation on your own computer, GFtboxCommand then works out how many individual simulations are specified by the command that invoked it. In this example, the only argument which will generate multiple simulations is ''dt'', of which there will be 4. Separate commands for each of these jobs are then automatically generated, each with an accompanying unique ID (The value of the ''ExpID'' name argument), and '''these are submitted as individual jobs to the cluster'''.<br />
<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.1,'ExpID','GPT_CASE_RST_1');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.5,'ExpID','GPT_CASE_RST_2');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',1,'ExpID','GPT_CASE_RST_3');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',5,'ExpID','GPT_CASE_RST_4');<br />
<br />
Once these jobs have been submitted, the ''ClusterMonitor'' tool opens and the new job batch ID is added to the list of jobs. Here, we will concentrate on three of the functions of ''ClusterMonitor'': ''Queue?'', ''Get project results'' and ''Make project pngs''. As in part 4 of this page, these functions will enable us to visualise the growth simulation results.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| This is an example of what you will see when the ClusterMonitor tool launches after submitting a job using the 'Use','Cluster' argument pair in GFtboxCommand.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="middle"<br />
|width="200px"| Clicking the 'Queue?' button tells ClusterMonitor to check the status of your jobs on the cluster. It then prints the status of the current jobs in Matlab's command window. In the case of our cluster, this is the equivalent of the ''qstatme'' command. This screenshot shows an example job list, where one job is currently running.<br />
<br />
This allows you too see whether or not your jobs have finished, and therefore whether or not their results are ready to retrieve.<br />
|width="600px"| [[File:queue.png| 600px]]<br />
|}<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Once your jobs have completed, the 'Queue?' button will show that there are no jobs running. Now you are ready to retrieve the project files from the cluster. This is achieved using the 'Get project results' button. Once pressed, the Matlab command window will show trace information to indicate which files are currently being downloaded. This is the equivalent of downloading the files manually, using an FTP program.<br />
|width="600px"| [[File:downloadingprojectfiles.png| 600px]]<br />
|}<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Once you have successfully downloaded your project results, you are ready to generate result images, as in part 3B1. Instead of entering a ''VMSReport'' command, you can press the 'Make project pngs' button, which will perform the same command.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
<br />
<br />
Whilst ''ClusterMonitor'' is a useful tool for managing your cluster jobs, it is advisable to '''have at least a rudimentary understanding of and ability to use a Unix-based computer system'''. In particular, the abilities to list the contents of a folder, view the contents of a file, change the current working folder, delete, and to copy or move files or folders, are essential Unix skills for making sure everything is working as intended. It is beyond the scope of these tutorials to provide an in depth Unix tutorial (many good and simple tutorials exist on the web), but here is a short description of the Unix commands (which may be specific to our Unix-based system) that we believe to be important.<br />
<br />
''ls'' - list the files and folders in a directory. Use the -al option (e.g. ls -al) to see modification dates, file permissions, sizes, etc.<br />
<br />
''cd X'' - Change the current directory to directory X.<br />
<br />
''rm X'' - Delete the file X.<br />
<br />
''cp X NewPath/X2'' - Copy file X to the NewPath directory and name the copy X2.<br />
<br />
''mv X NewPath/X2'' - Move file X to the NewPath directory and rename it to X2.<br />
<br />
''cat test.txt'' - Print the contents of the text file test.txt onto the screen.<br />
<br />
''pwd'' - Print the present working directory onto the screen.<br />
<br />
''more test.txt'' - Print the file test.txt to the screen, a screenful at a time, scrolling each time you hit the enter key.</div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Running_example_models_and_using_a_cluster&diff=5392Running example models and using a cluster2011-06-24T09:29:54Z<p>JacobNewman: </p>
<hr />
<div>[[GFtbox Tutorial pages | Back to GFtbox Tutorial pages]]<br />
<br />
The purpose of these tutorials is to learn how to run the example growth simulations included in GFtbox. We will describe methods for running simulations locally (on your own computer) and remotely (on a computing cluster). It is assumed that you have already downloaded the GFtbox software and have Matlab installed.<br />
<br />
=Getting Started=<br />
<br />
The remainder of this page is split into five sub-tutorials, each building on the preceding parts.<br />
<br />
1) '''Explaining the tools'''. In these tutorials, we will be using ''GFtboxCommand'' and ''ClusterMonitor''. This section explains the purpose of these tools. <br />
<br />
2) '''Computer or cluster?''' Here, we illustrate when and why you should use a computing cluster for your growth simulations, and conversely, when you should use your desktop computer.<br />
<br />
3) '''Running a growth simulation for an example model'''. This section demonstrates how to use ''GFtboxCommand'' to run a growth simulation on your computer. The model used for the simulation is an example model included with the GFtbox.<br />
<br />
4) '''Altering the simulation parameters'''. Following on from part 3, here we show how to adjust a simulation parameter within GFtboxCommand. Specifically, we alter the value of ''dt'', the temporal resolution of the simulation, and show how it can be used to verify that the value specified in the published literature is reasonable.<br />
<br />
5) '''Altering the model parameters on the cluster'''. Finally, we demonstrate how a number of model parameters can be varied by specifying a range of values for each model argument. We show how the computationally expensive task of simulating all combinations of specified ranges can be processed more efficiently if a computer cluster is used via the ''ClusterMonitor'' tool.<br />
<br />
='''1''' Explaining the tools=<br />
==GFtboxCommand==<br />
This is a command line version of the GFtbox. By command line, we mean that all program '''functions are operated via typed commands, without the GUI'''. Like GFtbox, GFtboxCommand is capable of running growth simulations of an interaction function, and allows the user to specify model and simulation parameters. Unlike GFtbox, this also allows the user to select '''ranges of values for a number of input parameters''', and will automatically spawn multiple simulations which explore the various combinations of those parameters. This can be used to evaluate the effect of various parameters on the growth of a given model.<br />
<br />
==ClusterMonitor==<br />
Provides a graphical user interface for managing simulations '''running remotely on a computer cluster'''. Specifically, it allows you to see which projects are present and running on the cluster, to retrieve the completed projects, to generate images of the simulations at specified stages of growth, and to remove projects from the cluster. If you do not intend to use a computer cluster, then you will not need to use ClusterMonitor.<br />
<br />
='''2''' Computer or cluster?=<br />
<br />
In basic terms, '''a computing cluster is effectively a network of many computer processors''' (often hundreds), centrally managed by a queuing system. When a job is submitted to a cluster, the job is sent to a processor that is not being used, or queued until one becomes available. In contrast, a typical desktop computer will contain one processor, limiting the number of tasks that can be performed at any one time. Jobs which must be run independently and sequentially on a desktop computer can be executed in parallel on a cluster, '''greatly reducing the total time required to complete all of the jobs'''. Although the exact details of your cluster might vary from those described in the rest of these tutorials, we aim to illustrate the generic processes involved in using GFtbox on a cluster. We are happy to offer assistance, where possible, to setup GFtbox for your cluster, so please contact us if you have any queries.<br />
<br />
Whilst the time savings offered by using a cluster can be significant, there is an overhead associated with returning the results to your personal computer. The total time to run 2 or more simulations on a cluster and to return the results will be less than running those simulations sequentially on one computer. Therefore, using a '''cluster is ideal for situations where you would like to run several simulations''', such as to evaluate the effect of a range of parameters on a growth model. You are not advised to use a cluster for running single simulations, or where you would like to step through a simulation and change parameters in a more interactive fashion. In such circumstances, you are advised to run GFtbox or GFtboxCommand on your own computer.<br />
<br />
The '''GFtbox GUI provides quick feedback''' about how the changes you have made to an interaction function have affected the growth simulation, as '''you can see the result of each simulation iteration as it completes'''. It is quick and easy therefore to see if you have dramatically changed the course of the growth simulation, and then to adjust the parameters according to your observations. This approach is well suited to the early stages of design, where you might wish to '''tweak some parameters to gauge whether or not they have had the desired effect'''. Or in the final stages, where you think your model is almost finalised, but where small adjustments are required. '''An alternative approach is to start many simulations''', based upon your initial model and by making intelligent choices about which parameters to explore. This allows you to '''harvest many results, to quickly and easily overview them''' in their finalised form, or to select interesting-looking simulations and examine them more closely on your desktop computer, using GFtbox.<br />
<br />
='''3''' Running a growth simulation for an example model=<br />
This tutorial is aimed at running a '''growth simulation for one of the example interaction functions''' included with the GFtbox. The purpose of this exercise is to firstly demonstrate how simulations can be invoked using GFtboxCommand, and secondly to show how to reproduce experimental results (specifically, those published in Kennaway et al (2011)) given an interaction function.<br />
<br />
Assuming Matlab is installed on your computer, and the latest GFtbox has been downloaded, you can add the GFtbox directory to Matlab's search path, which makes the toolbox accessible from any other path that you choose to work from. For a short tutorial on how to do this, please click [[Adding GFtbox to Matlab's search path | here]].<br />
<br />
Once the GFtbox is added to Matlab, you are ready to run a growth simulation using GFtboxCommand. In this example, '''the model that we will simulate is called GPT_CASE_RST'''. Results generated using this model are published in Kennaway et al (2011). By running this simulation, we can confirm the results in the published literature and investigate the suitability of the various parameters.<br />
<br />
===3A - Running a simulation on your computer===<br />
<br />
The following command can be typed into Matlab to run a simulation of the GPT_CASE_RST interaction function, which contains three growth models: R, S and T. '''Three separate simulations are run sequentially on your computer''', one for each model, each producing results corresponding to five intervals in the growth simulation.<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',[1:3]);<br />
<br />
GFtboxCommand accepts input arguments as name and value pairs, e.g. 'modelname', [1:3] or 'Use','Cluster'. The argument names entered in the example above are: ''Path'', ''Name'', ''Stages'' and ''modelname''.<br />
<br />
The optional '''''Path'' argument name refers to the location of the folder''' (or directory) on your computer where the model interaction function you wish to simulate is stored. '''''Name'', is the name of the folder itself'''. In this case, we are using the GPT_CASE_RST folder which is included in the GrowthToolbox. You may wish to copy this folder elsewhere, if you intend to make changes to the interaction function.<br />
<br />
During a growth simulation, a mesh can be generated at each time step of the simulation, which provides a visual representation of the growth of the biological tissue, given the various parameters of growth specified by the interaction function and how they have changed over time. Put another way, the mesh shows exactly what the growing tissue actually looks like. ''Stages'' refers to the points in the simulation (measured in hours) at which meshes should be generated and saved. In this example, '''five stages of growth will be written to disk'''. These values are chosen to best capture the appearance of the tissue at important stages of the tissue growth.<br />
<br />
The final argument name listed here is ''modelname''. This is a model-specific argument, and in this case the GPT_CASE_RST interaction function contains '''three separate models for plant growth''', allowing the desired model to be a selected. Here, the value [1:3] is specified, which is evaluated in the same way as entering [1 2 3]. This instructs GFtboxCommand to run three separate simulations, one for each of the growth models contained in the interaction function.<br />
<br />
The function of every permissible argument is given by keying the following command into Matlab:<br />
<br />
help GFtboxCommand<br />
<br />
===3B - Getting results from a completed simulation===<br />
<br />
'''1) - Generating images from a simulation you ran on your computer'''<br />
<br />
Once a growth simulation is completed, the project folder (GPT_CASE_RST, in our example) will contain another folder named "movies". Within movies, there are folders which contain results for the executed simulations. As was instructed in 3A, here we can see three folders corresponding to simulations for the three separate models. Within the first folder there are three items:<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| ''CommandLine.txt'' - This file contains the Matlab command which was used to generate the results that this sub-directory contains.<br />
<br />
''gpt_case_rst.txt'' - This file, named according to the project name, contains a copy of the interaction function which was used to produce these results.<br />
<br />
''meshes'' - This folder contains the mesh files corresponding to the stages of growth specified by the value of the Stages argument, described in 3A.<br />
|width="500px" align="center"|[[File:resultsdir.png| Directory structure of a project containing results]]<br />
|}<br />
<br />
<br />
The mesh files contain vertices information regarding the shape of the growth model at a particular stage of growth, but are not visualisable in that form. In order to '''convert these mesh files into viewable images''', we can execute the following command in Matlab:<br />
<br />
VMSreport('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Project','GPT_CASE_RST',...<br />
'Experiment','All','flattentime',572.5,'morphogen','KPAR','SNAPFIG',true);<br />
<br />
Where the ''Path'' and ''Project'' arguments have the same function as ''Path'' and ''Name'' in 3A, i.e. ''Path'' refers to the location of the project folder and ''Project'' is the name of the project folder itself. The result images will be stored in the ''meshes'' directory of the project.<br />
<br />
Here we can see the generated images, including the command line arguments, for one of the simulations performed in 3A. Click on a thumbnail to view a larger image.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="125px"| <small>001commandline.png</small><br> [[File:commandlineexample.png | 100px]]<br />
|width="125px"| <small>001GPT_CASE_RST_2 wild_.png</small><br>[[File:GPT_CASE_RST_example1.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_3 wild_.png</small><br>[[File:GPT_CASE_RST_example2.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_4 wild_.png</small><br>[[File:GPT_CASE_RST_example3.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_5 wild_.png</small><br>[[File:GPT_CASE_RST_example4.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_6 wild_.png</small><br>[[File:GPT_CASE_RST_example5.png | 125px]]<br />
|}<br />
<br />
<br />
'''2) - Generating images from a simulation you ran on the cluster'''<br />
<br />
NB. The results produced may not be visibly identical to those in the published literature. This is because of small, random perturbations which are applied to the initial model meshes to stop them from containing surfaces which are perfectly flat, and therefore biologically unrealistic. The results produced should be '''qualitatively but not quantitatively the same'''.<br />
<br />
'''3) - Interacting with your results using the GFtbox GUI'''<br />
<br />
='''4''' Altering the simulation parameters=<br />
<br />
One such simulation parameter is ''dt'', which is the time in seconds between iterations in the growth simulation. Large values of ''dt'' mean that fewer steps and therefore fewer calculations are required to complete a simulation. Whereas smaller values of ''dt'' mean that more steps are required, and therefore more processing time too. Therefore, a value of ''dt'' must be selected which is not so small that it is computationally unmanageable, but not so large that the observed growth is an artifact of the value of ''dt'', rather than the underlying growth model. It is necessary therefore to test a range of values for ''dt'' to ensure that the patterns of growth observed in a simulation are consistent across the range, and to find a value which is sufficient to demonstrate the model of growth and computationally efficient.<br />
<br />
'''Testing a range of ''dt''s can be achieved in several ways.''' One way is to use the ''dt'' argument when calling GFtboxCommand. This allows a single value to be tested. Another way is to make a batch of jobs, each using a different value for ''dt'', by using the 'State' argument. Lastly, a range of ''dt''s can be specified in GFtboxCommand, by using the dt argument where the values within the square brackets are the ''dt''s to simulate:<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5]);<br />
<br />
Here, we have specified '''four separate values for ''dt''''', and this means that '''four separate simulations will be run''' sequentially on your computer. As in 3B, images can then be generated for the mesh files produced, and compared to ensure consistency across the range of ''dt''s. In this figure, we can see the same model, at the same stage of growth, generated using the four values of ''dt''. Though quantitatively different, they are qualitatively the same, illustrating the suitability of the default ''dt'' value of 5.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="190px"| [[File:GPT_CASE_RST_dt0.5.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt1.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt2.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt5.png| 190px]]<br />
|}<br />
<br />
='''5''' Altering the model parameters on the cluster=<br />
It is easy to see that when even a small number of range variables, mutations or ''dt''s are specified, the '''total number of simulations increases quickly'''. If many ranges are specified, then the amount of processing time becomes unmanageable on a single computer. Via ''GFtboxCommand'' and the ''ClusterMonitor'' tool, we can remotely''' run a number of simulations, in parallel, on a computing cluster'''. It is assumed that the GrowthToolbox and Matlab are installed on your cluster, and PuTTY is required on your local computer for the tools ''pscp'' (for transferring files from your computer to a cluster) and ''plink'' (for remotely executing commands on a cluster). Using the ''dt'' range argument from 4 as an example, here we add the 'Use' name argument with the value 'Cluster':<br />
<br />
GFtboxCommand('State','Start','Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5],'Use','Cluster');<br />
<br />
The 'Cluster' option for the 'Use' name argument instructs GFtboxCommand to upload the required project directory to a remote Linux server. Instead of running the simulation on your own computer, GFtboxCommand then works out how many individual simulations are specified by the command that invoked it. In this example, the only argument which will generate multiple simulations is ''dt'', of which there will be 4. Separate commands for each of these jobs are then automatically generated, each with an accompanying unique ID (The value of the ''ExpID'' name argument), and '''these are submitted as individual jobs to the cluster'''.<br />
<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.1,'ExpID','GPT_CASE_RST_1');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.5,'ExpID','GPT_CASE_RST_2');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',1,'ExpID','GPT_CASE_RST_3');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',5,'ExpID','GPT_CASE_RST_4');<br />
<br />
Once these jobs have been submitted, the ''ClusterMonitor'' tool opens and the new job batch ID is added to the list of jobs. Here, we will concentrate on three of the functions of ''ClusterMonitor'': ''Queue?'', ''Get project results'' and ''Make project pngs''. As in part 4 of this page, these functions will enable us to visualise the growth simulation results.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| This is an example of what you will see when the ClusterMonitor tool launches after submitting a job using the 'Use','Cluster' argument pair in GFtboxCommand.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Clicking the 'Queue?' button tells ClusterMonitor to check the status of your jobs on the cluster. It then prints the status of the current jobs in Matlab's command window. In the case of our cluster, this is the equivalent of the ''qstatme'' command. This screenshot shows an example job list, where one job is currently running.<br />
<br />
This allows you too see whether or not your jobs have finished, and therefore whether or not their results are ready to retrieve.<br />
|width="600px"| [[File:queue.png| 600px]]<br />
|}<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Once your jobs have completed, the 'Queue?' button will show that there are no jobs running. Now you are ready to retrieve the project files from the cluster. This is achieved using the 'Get project results' button. Once pressed, the Matlab command window will show trace information to indicate which files are currently being downloaded. This is the equivalent of downloading the files manually, using an FTP program.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Once you have successfully downloaded your project results, you are ready to generate result images, as in part 3B1. Instead of entering a ''VMSReport'' command, you can press the 'Make project pngs' button, which will perform the same command.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
<br />
<br />
Whilst ''ClusterMonitor'' is a useful tool for managing your cluster jobs, it is advisable to '''have at least a rudimentary understanding of and ability to use a Unix-based computer system'''. In particular, the abilities to list the contents of a folder, view the contents of a file, change the current working folder, delete, and to copy or move files or folders, are essential Unix skills for making sure everything is working as intended. It is beyond the scope of these tutorials to provide an in depth Unix tutorial (many good and simple tutorials exist on the web), but here is a short description of the Unix commands (which may be specific to our Unix-based system) that we believe to be important.<br />
<br />
''ls'' - list the files and folders in a directory. Use the -al option (e.g. ls -al) to see modification dates, file permissions, sizes, etc.<br />
<br />
''cd X'' - Change the current directory to directory X.<br />
<br />
''rm X'' - Delete the file X.<br />
<br />
''cp X NewPath/X2'' - Copy file X to the NewPath directory and name the copy X2.<br />
<br />
''mv X NewPath/X2'' - Move file X to the NewPath directory and rename it to X2.<br />
<br />
''cat test.txt'' - Print the contents of the text file test.txt onto the screen.<br />
<br />
''pwd'' - Print the present working directory onto the screen.<br />
<br />
''more test.txt'' - Print the file test.txt to the screen, a screenful at a time, scrolling each time you hit the enter key.</div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=File:Queue.png&diff=5514File:Queue.png2011-06-24T09:29:08Z<p>JacobNewman: </p>
<hr />
<div></div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Running_example_models_and_using_a_cluster&diff=5391Running example models and using a cluster2011-06-24T09:13:27Z<p>JacobNewman: </p>
<hr />
<div>[[GFtbox Tutorial pages | Back to GFtbox Tutorial pages]]<br />
<br />
The purpose of these tutorials is to learn how to run the example growth simulations included in GFtbox. We will describe methods for running simulations locally (on your own computer) and remotely (on a computing cluster). It is assumed that you have already downloaded the GFtbox software and have Matlab installed.<br />
<br />
=Getting Started=<br />
<br />
The remainder of this page is split into five sub-tutorials, each building on the preceding parts.<br />
<br />
1) '''Explaining the tools'''. In these tutorials, we will be using ''GFtboxCommand'' and ''ClusterMonitor''. This section explains the purpose of these tools. <br />
<br />
2) '''Computer or cluster?''' Here, we illustrate when and why you should use a computing cluster for your growth simulations, and conversely, when you should use your desktop computer.<br />
<br />
3) '''Running a growth simulation for an example model'''. This section demonstrates how to use ''GFtboxCommand'' to run a growth simulation on your computer. The model used for the simulation is an example model included with the GFtbox.<br />
<br />
4) '''Altering the simulation parameters'''. Following on from part 3, here we show how to adjust a simulation parameter within GFtboxCommand. Specifically, we alter the value of ''dt'', the temporal resolution of the simulation, and show how it can be used to verify that the value specified in the published literature is reasonable.<br />
<br />
5) '''Altering the model parameters on the cluster'''. Finally, we demonstrate how a number of model parameters can be varied by specifying a range of values for each model argument. We show how the computationally expensive task of simulating all combinations of specified ranges can be processed more efficiently if a computer cluster is used via the ''ClusterMonitor'' tool.<br />
<br />
='''1''' Explaining the tools=<br />
==GFtboxCommand==<br />
This is a command line version of the GFtbox. By command line, we mean that all program '''functions are operated via typed commands, without the GUI'''. Like GFtbox, GFtboxCommand is capable of running growth simulations of an interaction function, and allows the user to specify model and simulation parameters. Unlike GFtbox, this also allows the user to select '''ranges of values for a number of input parameters''', and will automatically spawn multiple simulations which explore the various combinations of those parameters. This can be used to evaluate the effect of various parameters on the growth of a given model.<br />
<br />
==ClusterMonitor==<br />
Provides a graphical user interface for managing simulations '''running remotely on a computer cluster'''. Specifically, it allows you to see which projects are present and running on the cluster, to retrieve the completed projects, to generate images of the simulations at specified stages of growth, and to remove projects from the cluster. If you do not intend to use a computer cluster, then you will not need to use ClusterMonitor.<br />
<br />
='''2''' Computer or cluster?=<br />
<br />
In basic terms, '''a computing cluster is effectively a network of many computer processors''' (often hundreds), centrally managed by a queuing system. When a job is submitted to a cluster, the job is sent to a processor that is not being used, or queued until one becomes available. In contrast, a typical desktop computer will contain one processor, limiting the number of tasks that can be performed at any one time. Jobs which must be run independently and sequentially on a desktop computer can be executed in parallel on a cluster, '''greatly reducing the total time required to complete all of the jobs'''. Although the exact details of your cluster might vary from those described in the rest of these tutorials, we aim to illustrate the generic processes involved in using GFtbox on a cluster. We are happy to offer assistance, where possible, to setup GFtbox for your cluster, so please contact us if you have any queries.<br />
<br />
Whilst the time savings offered by using a cluster can be significant, there is an overhead associated with returning the results to your personal computer. The total time to run 2 or more simulations on a cluster and to return the results will be less than running those simulations sequentially on one computer. Therefore, using a '''cluster is ideal for situations where you would like to run several simulations''', such as to evaluate the effect of a range of parameters on a growth model. You are not advised to use a cluster for running single simulations, or where you would like to step through a simulation and change parameters in a more interactive fashion. In such circumstances, you are advised to run GFtbox or GFtboxCommand on your own computer.<br />
<br />
The '''GFtbox GUI provides quick feedback''' about how the changes you have made to an interaction function have affected the growth simulation, as '''you can see the result of each simulation iteration as it completes'''. It is quick and easy therefore to see if you have dramatically changed the course of the growth simulation, and then to adjust the parameters according to your observations. This approach is well suited to the early stages of design, where you might wish to '''tweak some parameters to gauge whether or not they have had the desired effect'''. Or in the final stages, where you think your model is almost finalised, but where small adjustments are required. '''An alternative approach is to start many simulations''', based upon your initial model and by making intelligent choices about which parameters to explore. This allows you to '''harvest many results, to quickly and easily overview them''' in their finalised form, or to select interesting-looking simulations and examine them more closely on your desktop computer, using GFtbox.<br />
<br />
='''3''' Running a growth simulation for an example model=<br />
This tutorial is aimed at running a '''growth simulation for one of the example interaction functions''' included with the GFtbox. The purpose of this exercise is to firstly demonstrate how simulations can be invoked using GFtboxCommand, and secondly to show how to reproduce experimental results (specifically, those published in Kennaway et al (2011)) given an interaction function.<br />
<br />
Assuming Matlab is installed on your computer, and the latest GFtbox has been downloaded, you can add the GFtbox directory to Matlab's search path, which makes the toolbox accessible from any other path that you choose to work from. For a short tutorial on how to do this, please click [[Adding GFtbox to Matlab's search path | here]].<br />
<br />
Once the GFtbox is added to Matlab, you are ready to run a growth simulation using GFtboxCommand. In this example, '''the model that we will simulate is called GPT_CASE_RST'''. Results generated using this model are published in Kennaway et al (2011). By running this simulation, we can confirm the results in the published literature and investigate the suitability of the various parameters.<br />
<br />
===3A - Running a simulation on your computer===<br />
<br />
The following command can be typed into Matlab to run a simulation of the GPT_CASE_RST interaction function, which contains three growth models: R, S and T. '''Three separate simulations are run sequentially on your computer''', one for each model, each producing results corresponding to five intervals in the growth simulation.<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',[1:3]);<br />
<br />
GFtboxCommand accepts input arguments as name and value pairs, e.g. 'modelname', [1:3] or 'Use','Cluster'. The argument names entered in the example above are: ''Path'', ''Name'', ''Stages'' and ''modelname''.<br />
<br />
The optional '''''Path'' argument name refers to the location of the folder''' (or directory) on your computer where the model interaction function you wish to simulate is stored. '''''Name'', is the name of the folder itself'''. In this case, we are using the GPT_CASE_RST folder which is included in the GrowthToolbox. You may wish to copy this folder elsewhere, if you intend to make changes to the interaction function.<br />
<br />
During a growth simulation, a mesh can be generated at each time step of the simulation, which provides a visual representation of the growth of the biological tissue, given the various parameters of growth specified by the interaction function and how they have changed over time. Put another way, the mesh shows exactly what the growing tissue actually looks like. ''Stages'' refers to the points in the simulation (measured in hours) at which meshes should be generated and saved. In this example, '''five stages of growth will be written to disk'''. These values are chosen to best capture the appearance of the tissue at important stages of the tissue growth.<br />
<br />
The final argument name listed here is ''modelname''. This is a model-specific argument, and in this case the GPT_CASE_RST interaction function contains '''three separate models for plant growth''', allowing the desired model to be a selected. Here, the value [1:3] is specified, which is evaluated in the same way as entering [1 2 3]. This instructs GFtboxCommand to run three separate simulations, one for each of the growth models contained in the interaction function.<br />
<br />
The function of every permissible argument is given by keying the following command into Matlab:<br />
<br />
help GFtboxCommand<br />
<br />
===3B - Getting results from a completed simulation===<br />
<br />
'''1) - Generating images from a simulation you ran on your computer'''<br />
<br />
Once a growth simulation is completed, the project folder (GPT_CASE_RST, in our example) will contain another folder named "movies". Within movies, there are folders which contain results for the executed simulations. As was instructed in 3A, here we can see three folders corresponding to simulations for the three separate models. Within the first folder there are three items:<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| ''CommandLine.txt'' - This file contains the Matlab command which was used to generate the results that this sub-directory contains.<br />
<br />
''gpt_case_rst.txt'' - This file, named according to the project name, contains a copy of the interaction function which was used to produce these results.<br />
<br />
''meshes'' - This folder contains the mesh files corresponding to the stages of growth specified by the value of the Stages argument, described in 3A.<br />
|width="500px" align="center"|[[File:resultsdir.png| Directory structure of a project containing results]]<br />
|}<br />
<br />
<br />
The mesh files contain vertices information regarding the shape of the growth model at a particular stage of growth, but are not visualisable in that form. In order to '''convert these mesh files into viewable images''', we can execute the following command in Matlab:<br />
<br />
VMSreport('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Project','GPT_CASE_RST',...<br />
'Experiment','All','flattentime',572.5,'morphogen','KPAR','SNAPFIG',true);<br />
<br />
Where the ''Path'' and ''Project'' arguments have the same function as ''Path'' and ''Name'' in 3A, i.e. ''Path'' refers to the location of the project folder and ''Project'' is the name of the project folder itself. The result images will be stored in the ''meshes'' directory of the project.<br />
<br />
Here we can see the generated images, including the command line arguments, for one of the simulations performed in 3A. Click on a thumbnail to view a larger image.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="125px"| <small>001commandline.png</small><br> [[File:commandlineexample.png | 100px]]<br />
|width="125px"| <small>001GPT_CASE_RST_2 wild_.png</small><br>[[File:GPT_CASE_RST_example1.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_3 wild_.png</small><br>[[File:GPT_CASE_RST_example2.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_4 wild_.png</small><br>[[File:GPT_CASE_RST_example3.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_5 wild_.png</small><br>[[File:GPT_CASE_RST_example4.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_6 wild_.png</small><br>[[File:GPT_CASE_RST_example5.png | 125px]]<br />
|}<br />
<br />
<br />
'''2) - Generating images from a simulation you ran on the cluster'''<br />
<br />
NB. The results produced may not be visibly identical to those in the published literature. This is because of small, random perturbations which are applied to the initial model meshes to stop them from containing surfaces which are perfectly flat, and therefore biologically unrealistic. The results produced should be '''qualitatively but not quantitatively the same'''.<br />
<br />
'''3) - Interacting with your results using the GFtbox GUI'''<br />
<br />
='''4''' Altering the simulation parameters=<br />
<br />
One such simulation parameter is ''dt'', which is the time in seconds between iterations in the growth simulation. Large values of ''dt'' mean that fewer steps and therefore fewer calculations are required to complete a simulation. Whereas smaller values of ''dt'' mean that more steps are required, and therefore more processing time too. Therefore, a value of ''dt'' must be selected which is not so small that it is computationally unmanageable, but not so large that the observed growth is an artifact of the value of ''dt'', rather than the underlying growth model. It is necessary therefore to test a range of values for ''dt'' to ensure that the patterns of growth observed in a simulation are consistent across the range, and to find a value which is sufficient to demonstrate the model of growth and computationally efficient.<br />
<br />
'''Testing a range of ''dt''s can be achieved in several ways.''' One way is to use the ''dt'' argument when calling GFtboxCommand. This allows a single value to be tested. Another way is to make a batch of jobs, each using a different value for ''dt'', by using the 'State' argument. Lastly, a range of ''dt''s can be specified in GFtboxCommand, by using the dt argument where the values within the square brackets are the ''dt''s to simulate:<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5]);<br />
<br />
Here, we have specified '''four separate values for ''dt''''', and this means that '''four separate simulations will be run''' sequentially on your computer. As in 3B, images can then be generated for the mesh files produced, and compared to ensure consistency across the range of ''dt''s. In this figure, we can see the same model, at the same stage of growth, generated using the four values of ''dt''. Though quantitatively different, they are qualitatively the same, illustrating the suitability of the default ''dt'' value of 5.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="190px"| [[File:GPT_CASE_RST_dt0.5.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt1.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt2.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt5.png| 190px]]<br />
|}<br />
<br />
='''5''' Altering the model parameters on the cluster=<br />
It is easy to see that when even a small number of range variables, mutations or ''dt''s are specified, the '''total number of simulations increases quickly'''. If many ranges are specified, then the amount of processing time becomes unmanageable on a single computer. Via ''GFtboxCommand'' and the ''ClusterMonitor'' tool, we can remotely''' run a number of simulations, in parallel, on a computing cluster'''. It is assumed that the GrowthToolbox and Matlab are installed on your cluster, and PuTTY is required on your local computer for the tools ''pscp'' (for transferring files from your computer to a cluster) and ''plink'' (for remotely executing commands on a cluster). Using the ''dt'' range argument from 4 as an example, here we add the 'Use' name argument with the value 'Cluster':<br />
<br />
GFtboxCommand('State','Start','Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5],'Use','Cluster');<br />
<br />
The 'Cluster' option for the 'Use' name argument instructs GFtboxCommand to upload the required project directory to a remote Linux server. Instead of running the simulation on your own computer, GFtboxCommand then works out how many individual simulations are specified by the command that invoked it. In this example, the only argument which will generate multiple simulations is ''dt'', of which there will be 4. Separate commands for each of these jobs are then automatically generated, each with an accompanying unique ID (The value of the ''ExpID'' name argument), and '''these are submitted as individual jobs to the cluster'''.<br />
<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.1,'ExpID','GPT_CASE_RST_1');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.5,'ExpID','GPT_CASE_RST_2');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',1,'ExpID','GPT_CASE_RST_3');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',5,'ExpID','GPT_CASE_RST_4');<br />
<br />
Once these jobs have been submitted, the ''ClusterMonitor'' tool opens and the new job batch ID is added to the list of jobs. Here, we will concentrate on three of the functions of ''ClusterMonitor'': ''Queue?'', ''Get project results'' and ''Make project pngs''. As in part 4 of this page, these functions will enable us to visualise the growth simulation results.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| This is an example of what you will see when the ClusterMonitor tool launches after submitting a job using the 'Use','Cluster' argument pair in GFtboxCommand.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Clicking the 'Queue?' button tells ClusterMonitor to check the status of your jobs on the cluster. It then prints the status of the current jobs in Matlab's command window. In the case of our cluster, this is the equivalent of the ''qstatme'' command. This screenshot shows an example job list, where one job is currently running.<br />
<br />
This allows you too see whether or not your jobs have finished, and therefore whether or not their results are ready to retrieve.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Once your jobs have completed, the 'Queue?' button will show that there are no jobs running. Now you are ready to retrieve the project files from the cluster. This is achieved using the 'Get project results' button. Once pressed, the Matlab command window will show trace information to indicate which files are currently being downloaded. This is the equivalent of downloading the files manually, using an FTP program.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Once you have successfully downloaded your project results, you are ready to generate result images, as in part 3B1. Instead of entering a ''VMSReport'' command, you can press the 'Make project pngs' button, which will perform the same command.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
<br />
<br />
Whilst ''ClusterMonitor'' is a useful tool for managing your cluster jobs, it is advisable to '''have at least a rudimentary understanding of and ability to use a Unix-based computer system'''. In particular, the abilities to list the contents of a folder, view the contents of a file, change the current working folder, delete, and to copy or move files or folders, are essential Unix skills for making sure everything is working as intended. It is beyond the scope of these tutorials to provide an in depth Unix tutorial (many good and simple tutorials exist on the web), but here is a short description of the Unix commands (which may be specific to our Unix-based system) that we believe to be important.<br />
<br />
''ls'' - list the files and folders in a directory. Use the -al option (e.g. ls -al) to see modification dates, file permissions, sizes, etc.<br />
<br />
''cd X'' - Change the current directory to directory X.<br />
<br />
''rm X'' - Delete the file X.<br />
<br />
''cp X NewPath/X2'' - Copy file X to the NewPath directory and name the copy X2.<br />
<br />
''mv X NewPath/X2'' - Move file X to the NewPath directory and rename it to X2.<br />
<br />
''cat test.txt'' - Print the contents of the text file test.txt onto the screen.<br />
<br />
''pwd'' - Print the present working directory onto the screen.<br />
<br />
''more test.txt'' - Print the file test.txt to the screen, a screenful at a time, scrolling each time you hit the enter key.</div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Running_example_models_and_using_a_cluster&diff=5390Running example models and using a cluster2011-06-24T09:07:44Z<p>JacobNewman: </p>
<hr />
<div>[[GFtbox Tutorial pages | Back to GFtbox Tutorial pages]]<br />
<br />
The purpose of these tutorials is to learn how to run the example growth simulations included in GFtbox. We will describe methods for running simulations locally (on your own computer) and remotely (on a computing cluster). It is assumed that you have already downloaded the GFtbox software and have Matlab installed.<br />
<br />
=Getting Started=<br />
<br />
The remainder of this page is split into five sub-tutorials, each building on the preceding parts.<br />
<br />
1) '''Explaining the tools'''. In these tutorials, we will be using ''GFtboxCommand'' and ''ClusterMonitor''. This section explains the purpose of these tools. <br />
<br />
2) '''Computer or cluster?''' Here, we illustrate when and why you should use a computing cluster for your growth simulations, and conversely, when you should use your desktop computer.<br />
<br />
3) '''Running a growth simulation for an example model'''. This section demonstrates how to use ''GFtboxCommand'' to run a growth simulation on your computer. The model used for the simulation is an example model included with the GFtbox.<br />
<br />
4) '''Altering the simulation parameters'''. Following on from part 3, here we show how to adjust a simulation parameter within GFtboxCommand. Specifically, we alter the value of ''dt'', the temporal resolution of the simulation, and show how it can be used to verify that the value specified in the published literature is reasonable.<br />
<br />
5) '''Altering the model parameters on the cluster'''. Finally, we demonstrate how a number of model parameters can be varied by specifying a range of values for each model argument. We show how the computationally expensive task of simulating all combinations of specified ranges can be processed more efficiently if a computer cluster is used via the ''ClusterMonitor'' tool.<br />
<br />
='''1''' Explaining the tools=<br />
==GFtboxCommand==<br />
This is a command line version of the GFtbox. By command line, we mean that all program '''functions are operated via typed commands, without the GUI'''. Like GFtbox, GFtboxCommand is capable of running growth simulations of an interaction function, and allows the user to specify model and simulation parameters. Unlike GFtbox, this also allows the user to select '''ranges of values for a number of input parameters''', and will automatically spawn multiple simulations which explore the various combinations of those parameters. This can be used to evaluate the effect of various parameters on the growth of a given model.<br />
<br />
==ClusterMonitor==<br />
Provides a graphical user interface for managing simulations '''running remotely on a computer cluster'''. Specifically, it allows you to see which projects are present and running on the cluster, to retrieve the completed projects, to generate images of the simulations at specified stages of growth, and to remove projects from the cluster. If you do not intend to use a computer cluster, then you will not need to use ClusterMonitor.<br />
<br />
='''2''' Computer or cluster?=<br />
<br />
In basic terms, '''a computing cluster is effectively a network of many computer processors''' (often hundreds), centrally managed by a queuing system. When a job is submitted to a cluster, the job is sent to a processor that is not being used, or queued until one becomes available. In contrast, a typical desktop computer will contain one processor, limiting the number of tasks that can be performed at any one time. Jobs which must be run independently and sequentially on a desktop computer can be executed in parallel on a cluster, '''greatly reducing the total time required to complete all of the jobs'''. Although the exact details of your cluster might vary from those described in the rest of these tutorials, we aim to illustrate the generic processes involved in using GFtbox on a cluster. We are happy to offer assistance, where possible, to setup GFtbox for your cluster, so please contact us if you have any queries.<br />
<br />
Whilst the time savings offered by using a cluster can be significant, there is an overhead associated with returning the results to your personal computer. The total time to run 2 or more simulations on a cluster and to return the results will be less than running those simulations sequentially on one computer. Therefore, using a '''cluster is ideal for situations where you would like to run several simulations''', such as to evaluate the effect of a range of parameters on a growth model. You are not advised to use a cluster for running single simulations, or where you would like to step through a simulation and change parameters in a more interactive fashion. In such circumstances, you are advised to run GFtbox or GFtboxCommand on your own computer.<br />
<br />
The '''GFtbox GUI provides quick feedback''' about how the changes you have made to an interaction function have affected the growth simulation, as '''you can see the result of each simulation iteration as it completes'''. It is quick and easy therefore to see if you have dramatically changed the course of the growth simulation, and then to adjust the parameters according to your observations. This approach is well suited to the early stages of design, where you might wish to '''tweak some parameters to gauge whether or not they have had the desired effect'''. Or in the final stages, where you think your model is almost finalised, but where small adjustments are required. '''An alternative approach is to start many simulations''', based upon your initial model and by making intelligent choices about which parameters to explore. This allows you to '''harvest many results, to quickly and easily overview them''' in their finalised form, or to select interesting-looking simulations and examine them more closely on your desktop computer, using GFtbox.<br />
<br />
='''3''' Running a growth simulation for an example model=<br />
This tutorial is aimed at running a '''growth simulation for one of the example interaction functions''' included with the GFtbox. The purpose of this exercise is to firstly demonstrate how simulations can be invoked using GFtboxCommand, and secondly to show how to reproduce experimental results (specifically, those published in Kennaway et al (2011)) given an interaction function.<br />
<br />
Assuming Matlab is installed on your computer, and the latest GFtbox has been downloaded, you can add the GFtbox directory to Matlab's search path, which makes the toolbox accessible from any other path that you choose to work from. For a short tutorial on how to do this, please click [[Adding GFtbox to Matlab's search path | here]].<br />
<br />
Once the GFtbox is added to Matlab, you are ready to run a growth simulation using GFtboxCommand. In this example, '''the model that we will simulate is called GPT_CASE_RST'''. Results generated using this model are published in Kennaway et al (2011). By running this simulation, we can confirm the results in the published literature and investigate the suitability of the various parameters.<br />
<br />
===3A - Running a simulation on your computer===<br />
<br />
The following command can be typed into Matlab to run a simulation of the GPT_CASE_RST interaction function, which contains three growth models: R, S and T. '''Three separate simulations are run sequentially on your computer''', one for each model, each producing results corresponding to five intervals in the growth simulation.<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',[1:3]);<br />
<br />
GFtboxCommand accepts input arguments as name and value pairs, e.g. 'modelname', [1:3] or 'Use','Cluster'. The argument names entered in the example above are: ''Path'', ''Name'', ''Stages'' and ''modelname''.<br />
<br />
The optional '''''Path'' argument name refers to the location of the folder''' (or directory) on your computer where the model interaction function you wish to simulate is stored. '''''Name'', is the name of the folder itself'''. In this case, we are using the GPT_CASE_RST folder which is included in the GrowthToolbox. You may wish to copy this folder elsewhere, if you intend to make changes to the interaction function.<br />
<br />
During a growth simulation, a mesh can be generated at each time step of the simulation, which provides a visual representation of the growth of the biological tissue, given the various parameters of growth specified by the interaction function and how they have changed over time. Put another way, the mesh shows exactly what the growing tissue actually looks like. ''Stages'' refers to the points in the simulation (measured in hours) at which meshes should be generated and saved. In this example, '''five stages of growth will be written to disk'''. These values are chosen to best capture the appearance of the tissue at important stages of the tissue growth.<br />
<br />
The final argument name listed here is ''modelname''. This is a model-specific argument, and in this case the GPT_CASE_RST interaction function contains '''three separate models for plant growth''', allowing the desired model to be a selected. Here, the value [1:3] is specified, which is evaluated in the same way as entering [1 2 3]. This instructs GFtboxCommand to run three separate simulations, one for each of the growth models contained in the interaction function.<br />
<br />
The function of every permissible argument is given by keying the following command into Matlab:<br />
<br />
help GFtboxCommand<br />
<br />
===3B - Getting results from a completed simulation===<br />
<br />
'''1) - Generating images from a simulation you ran on your computer'''<br />
<br />
Once a growth simulation is completed, the project folder (GPT_CASE_RST, in our example) will contain another folder named "movies". Within movies, there are folders which contain results for the executed simulations. As was instructed in 3A, here we can see three folders corresponding to simulations for the three separate models. Within the first folder there are three items:<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| ''CommandLine.txt'' - This file contains the Matlab command which was used to generate the results that this sub-directory contains.<br />
<br />
''gpt_case_rst.txt'' - This file, named according to the project name, contains a copy of the interaction function which was used to produce these results.<br />
<br />
''meshes'' - This folder contains the mesh files corresponding to the stages of growth specified by the value of the Stages argument, described in 3A.<br />
|width="500px" align="center"|[[File:resultsdir.png| Directory structure of a project containing results]]<br />
|}<br />
<br />
<br />
The mesh files contain vertices information regarding the shape of the growth model at a particular stage of growth, but are not visualisable in that form. In order to '''convert these mesh files into viewable images''', we can execute the following command in Matlab:<br />
<br />
VMSreport('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Project','GPT_CASE_RST',...<br />
'Experiment','All','flattentime',572.5,'morphogen','KPAR','SNAPFIG',true);<br />
<br />
Where the ''Path'' and ''Project'' arguments have the same function as ''Path'' and ''Name'' in 3A, i.e. ''Path'' refers to the location of the project folder and ''Project'' is the name of the project folder itself.<br />
<br />
Here we can see the generated images, including the command line arguments, for one of the simulations performed in 3A. Click on a thumbnail to view a larger image.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="125px"| <small>001commandline.png</small><br> [[File:commandlineexample.png | 100px]]<br />
|width="125px"| <small>001GPT_CASE_RST_2 wild_.png</small><br>[[File:GPT_CASE_RST_example1.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_3 wild_.png</small><br>[[File:GPT_CASE_RST_example2.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_4 wild_.png</small><br>[[File:GPT_CASE_RST_example3.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_5 wild_.png</small><br>[[File:GPT_CASE_RST_example4.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_6 wild_.png</small><br>[[File:GPT_CASE_RST_example5.png | 125px]]<br />
|}<br />
<br />
<br />
'''2) - Generating images from a simulation you ran on the cluster'''<br />
<br />
NB. The results produced may not be visibly identical to those in the published literature. This is because of small, random perturbations which are applied to the initial model meshes to stop them from containing surfaces which are perfectly flat, and therefore biologically unrealistic. The results produced should be '''qualitatively but not quantitatively the same'''.<br />
<br />
'''3) - Interacting with your results using the GFtbox GUI'''<br />
<br />
='''4''' Altering the simulation parameters=<br />
<br />
One such simulation parameter is ''dt'', which is the time in seconds between iterations in the growth simulation. Large values of ''dt'' mean that fewer steps and therefore fewer calculations are required to complete a simulation. Whereas smaller values of ''dt'' mean that more steps are required, and therefore more processing time too. Therefore, a value of ''dt'' must be selected which is not so small that it is computationally unmanageable, but not so large that the observed growth is an artifact of the value of ''dt'', rather than the underlying growth model. It is necessary therefore to test a range of values for ''dt'' to ensure that the patterns of growth observed in a simulation are consistent across the range, and to find a value which is sufficient to demonstrate the model of growth and computationally efficient.<br />
<br />
'''Testing a range of ''dt''s can be achieved in several ways.''' One way is to use the ''dt'' argument when calling GFtboxCommand. This allows a single value to be tested. Another way is to make a batch of jobs, each using a different value for ''dt'', by using the 'State' argument. Lastly, a range of ''dt''s can be specified in GFtboxCommand, by using the dt argument where the values within the square brackets are the ''dt''s to simulate:<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5]);<br />
<br />
Here, we have specified '''four separate values for ''dt''''', and this means that '''four separate simulations will be run''' sequentially on your computer. As in 3B, images can then be generated for the mesh files produced, and compared to ensure consistency across the range of ''dt''s. In this figure, we can see the same model, at the same stage of growth, generated using the four values of ''dt''. Though quantitatively different, they are qualitatively the same, illustrating the suitability of the default ''dt'' value of 5.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="190px"| [[File:GPT_CASE_RST_dt0.5.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt1.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt2.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt5.png| 190px]]<br />
|}<br />
<br />
='''5''' Altering the model parameters on the cluster=<br />
It is easy to see that when even a small number of range variables, mutations or ''dt''s are specified, the '''total number of simulations increases quickly'''. If many ranges are specified, then the amount of processing time becomes unmanageable on a single computer. Via ''GFtboxCommand'' and the ''ClusterMonitor'' tool, we can remotely''' run a number of simulations, in parallel, on a computing cluster'''. It is assumed that the GrowthToolbox and Matlab are installed on your cluster, and PuTTY is required on your local computer for the tools ''pscp'' (for transferring files from your computer to a cluster) and ''plink'' (for remotely executing commands on a cluster). Using the ''dt'' range argument from 4 as an example, here we add the 'Use' name argument with the value 'Cluster':<br />
<br />
GFtboxCommand('State','Start','Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5],'Use','Cluster');<br />
<br />
The 'Cluster' option for the 'Use' name argument instructs GFtboxCommand to upload the required project directory to a remote Linux server. Instead of running the simulation on your own computer, GFtboxCommand then works out how many individual simulations are specified by the command that invoked it. In this example, the only argument which will generate multiple simulations is ''dt'', of which there will be 4. Separate commands for each of these jobs are then automatically generated, each with an accompanying unique ID (The value of the ''ExpID'' name argument), and '''these are submitted as individual jobs to the cluster'''.<br />
<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.1,'ExpID','GPT_CASE_RST_1');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.5,'ExpID','GPT_CASE_RST_2');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',1,'ExpID','GPT_CASE_RST_3');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',5,'ExpID','GPT_CASE_RST_4');<br />
<br />
Once these jobs have been submitted, the ''ClusterMonitor'' tool opens and the new job batch ID is added to the list of jobs. Here, we will concentrate on three of the functions of ''ClusterMonitor'': ''Queue?'', ''Get project results'' and ''Make project pngs''. As in part 4 of this page, these functions will enable us to visualise the growth simulation results.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| This is an example of what you will see when the ClusterMonitor tool launches after submitting a job using the 'Use','Cluster' argument pair in GFtboxCommand.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Clicking the 'Queue?' button tells ClusterMonitor to check the status of your jobs on the cluster. It then prints the status of the current jobs in Matlab's command window. In the case of our cluster, this is the equivalent of the ''qstatme'' command. This screenshot shows an example job list, where one job is currently running.<br />
<br />
This allows you too see whether or not your jobs have finished, and therefore whether or not their results are ready to retrieve.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Once your jobs have completed, the 'Queue?' button will show that there are no jobs running. Now you are ready to retrieve the project files from the cluster. This is achieved using the 'Get project results' button. Once pressed, the Matlab command window will show trace information to indicate which files are currently being downloaded. This is the equivalent of downloading the files manually, using an FTP program.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Once you have successfully downloaded your project results, you are ready to generate result images, as in part 3B1. Instead of entering a ''VMSReport'' command, you can press the 'Make project pngs' button, which will perform the same command.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
<br />
<br />
Whilst ''ClusterMonitor'' is a useful tool for managing your cluster jobs, it is advisable to '''have at least a rudimentary understanding of and ability to use a Unix-based computer system'''. In particular, the abilities to list the contents of a folder, view the contents of a file, change the current working folder, delete, and to copy or move files or folders, are essential Unix skills for making sure everything is working as intended. It is beyond the scope of these tutorials to provide an in depth Unix tutorial (many good and simple tutorials exist on the web), but here is a short description of the Unix commands (which may be specific to our Unix-based system) that we believe to be important.<br />
<br />
''ls'' - list the files and folders in a directory. Use the -al option (e.g. ls -al) to see modification dates, file permissions, sizes, etc.<br />
<br />
''cd X'' - Change the current directory to directory X.<br />
<br />
''rm X'' - Delete the file X.<br />
<br />
''cp X NewPath/X2'' - Copy file X to the NewPath directory and name the copy X2.<br />
<br />
''mv X NewPath/X2'' - Move file X to the NewPath directory and rename it to X2.<br />
<br />
''cat test.txt'' - Print the contents of the text file test.txt onto the screen.<br />
<br />
''pwd'' - Print the present working directory onto the screen.<br />
<br />
''more test.txt'' - Print the file test.txt to the screen, a screenful at a time, scrolling each time you hit the enter key.</div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Running_example_models_and_using_a_cluster&diff=5389Running example models and using a cluster2011-06-24T09:06:34Z<p>JacobNewman: </p>
<hr />
<div>[[GFtbox Tutorial pages | Back to GFtbox Tutorial pages]]<br />
<br />
The purpose of these tutorials is to learn how to run the example growth simulations included in GFtbox. We will describe methods for running simulations locally (on your own computer) and remotely (on a computing cluster). It is assumed that you have already downloaded the GFtbox software and have Matlab installed.<br />
<br />
=Getting Started=<br />
<br />
The remainder of this page is split into five sub-tutorials, each building on the preceding parts.<br />
<br />
1) '''Explaining the tools'''. In these tutorials, we will be using ''GFtboxCommand'' and ''ClusterMonitor''. This section explains the purpose of these tools. <br />
<br />
2) '''Computer or cluster?''' Here, we illustrate when and why you should use a computing cluster for your growth simulations, and conversely, when you should use your desktop computer.<br />
<br />
3) '''Running a growth simulation for an example model'''. This section demonstrates how to use ''GFtboxCommand'' to run a growth simulation on your computer. The model used for the simulation is an example model included with the GFtbox.<br />
<br />
4) '''Altering the simulation parameters'''. Following on from part 3, here we show how to adjust a simulation parameter within GFtboxCommand. Specifically, we alter the value of ''dt'', the temporal resolution of the simulation, and show how it can be used to verify that the value specified in the published literature is reasonable.<br />
<br />
5) '''Altering the model parameters on the cluster'''. Finally, we demonstrate how a number of model parameters can be varied by specifying a range of values for each model argument. We show how the computationally expensive task of simulating all combinations of specified ranges can be processed more efficiently if a computer cluster is used via the ''ClusterMonitor'' tool.<br />
<br />
='''1''' Explaining the tools=<br />
==GFtboxCommand==<br />
This is a command line version of the GFtbox. By command line, we mean that all program '''functions are operated via typed commands, without the GUI'''. Like GFtbox, GFtboxCommand is capable of running growth simulations of an interaction function, and allows the user to specify model and simulation parameters. Unlike GFtbox, this also allows the user to select '''ranges of values for a number of input parameters''', and will automatically spawn multiple simulations which explore the various combinations of those parameters. This can be used to evaluate the effect of various parameters on the growth of a given model.<br />
<br />
==ClusterMonitor==<br />
Provides a graphical user interface for managing simulations '''running remotely on a computer cluster'''. Specifically, it allows you to see which projects are present and running on the cluster, to retrieve the completed projects, to generate images of the simulations at specified stages of growth, and to remove projects from the cluster. If you do not intend to use a computer cluster, then you will not need to use ClusterMonitor.<br />
<br />
='''2''' Computer or cluster?=<br />
<br />
In basic terms, '''a computing cluster is effectively a network of many computer processors''' (often hundreds), centrally managed by a queuing system. When a job is submitted to a cluster, the job is sent to a processor that is not being used, or queued until one becomes available. In contrast, a typical desktop computer will contain one processor, limiting the number of tasks that can be performed at any one time. Jobs which must be run independently and sequentially on a desktop computer can be executed in parallel on a cluster, '''greatly reducing the total time required to complete all of the jobs'''. Although the exact details of your cluster might vary from those described in the rest of these tutorials, we aim to illustrate the generic processes involved in using GFtbox on a cluster. We are happy to offer assistance, where possible, to setup GFtbox for your cluster, so please contact us if you have any queries.<br />
<br />
Whilst the time savings offered by using a cluster can be significant, there is an overhead associated with returning the results to your personal computer. The total time to run 2 or more simulations on a cluster and to return the results will be less than running those simulations sequentially on one computer. Therefore, using a '''cluster is ideal for situations where you would like to run several simulations''', such as to evaluate the effect of a range of parameters on a growth model. You are not advised to use a cluster for running single simulations, or where you would like to step through a simulation and change parameters in a more interactive fashion. In such circumstances, you are advised to run GFtbox or GFtboxCommand on your own computer.<br />
<br />
The '''GFtbox GUI provides quick feedback''' about how the changes you have made to an interaction function have affected the growth simulation, as '''you can see the result of each simulation iteration as it completes'''. It is quick and easy therefore to see if you have dramatically changed the course of the growth simulation, and then to adjust the parameters according to your observations. This approach is well suited to the early stages of design, where you might wish to '''tweak some parameters to gauge whether or not they have had the desired effect'''. Or in the final stages, where you think your model is almost finalised, but where small adjustments are required. '''An alternative approach is to start many simulations''', based upon your initial model and by making intelligent choices about which parameters to explore. This allows you to '''harvest many results, to quickly and easily overview them''' in their finalised form, or to select interesting-looking simulations and examine them more closely on your desktop computer, using GFtbox.<br />
<br />
='''3''' Running a growth simulation for an example model=<br />
This tutorial is aimed at running a '''growth simulation for one of the example interaction functions''' included with the GFtbox. The purpose of this exercise is to firstly demonstrate how simulations can be invoked using GFtboxCommand, and secondly to show how to reproduce experimental results (specifically, those published in Kennaway et al (2011)) given an interaction function.<br />
<br />
Assuming Matlab is installed on your computer, and the latest GFtbox has been downloaded, you can add the GFtbox directory to Matlab's search path, which makes the toolbox accessible from any other path that you choose to work from. For a short tutorial on how to do this, please click [[Adding GFtbox to Matlab's search path | here]].<br />
<br />
Once the GFtbox is added to Matlab, you are ready to run a growth simulation using GFtboxCommand. In this example, '''the model that we will simulate is called GPT_CASE_RST'''. Results generated using this model are published in Kennaway et al (2011). By running this simulation, we can confirm the results in the published literature and investigate the suitability of the various parameters.<br />
<br />
===3A - Running a simulation on your computer===<br />
<br />
The following command can be typed into Matlab to run a simulation of the GPT_CASE_RST interaction function, which contains three growth models: R, S and T. '''Three separate simulations are run sequentially on your computer''', one for each model, each producing results corresponding to five intervals in the growth simulation.<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',[1:3]);<br />
<br />
GFtboxCommand accepts input arguments as name and value pairs, e.g. 'modelname', [1:3] or 'Use','Cluster'. The argument names entered in the example above are: ''Path'', ''Name'', ''Stages'' and ''modelname''.<br />
<br />
The optional '''''Path'' argument name refers to the location of the folder''' (or directory) on your computer where the model interaction function you wish to simulate is stored. '''''Name'', is the name of the folder itself'''. In this case, we are using the GPT_CASE_RST folder which is included in the GrowthToolbox. You may wish to copy this folder elsewhere, if you intend to make changes to the interaction function.<br />
<br />
During a growth simulation, a mesh can be generated at each time step of the simulation, which provides a visual representation of the growth of the biological tissue, given the various parameters of growth specified by the interaction function and how they have changed over time. Put another way, the mesh shows exactly what the growing tissue actually looks like. ''Stages'' refers to the points in the simulation (measured in hours) at which meshes should be generated and saved. In this example, '''five stages of growth will be written to disk'''. These values are chosen to best capture the appearance of the tissue at important stages of the tissue growth.<br />
<br />
The final argument name listed here is ''modelname''. This is a model-specific argument, and in this case the GPT_CASE_RST interaction function contains '''three separate models for plant growth''', allowing the desired model to be a selected. Here, the value [1:3] is specified, which is evaluated in the same way as entering [1 2 3]. This instructs GFtboxCommand to run three separate simulations, one for each of the growth models contained in the interaction function.<br />
<br />
The function of every permissible argument is given by keying the following command into Matlab:<br />
<br />
help GFtboxCommand<br />
<br />
===3B - Getting results from a completed simulation===<br />
<br />
'''1) - Generating images from a simulation you ran on your computer'''<br />
<br />
Once a growth simulation is completed, the project folder (GPT_CASE_RST, in our example) will contain another folder named "movies". Within movies, there are folders which contain results for the executed simulations. As was instructed in 3A, here we can see three folders corresponding to simulations for the three separate models. Within the first folder there are three items:<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| ''CommandLine.txt'' - This file contains the Matlab command which was used to generate the results that this sub-directory contains.<br />
<br />
''gpt_case_rst.txt'' - This file, named according to the project name, contains a copy of the interaction function which was used to produce these results.<br />
<br />
''meshes'' - This folder contains the mesh files corresponding to the stages of growth specified by the value of the Stages argument, described in 3A.<br />
|width="500px" align="center"|[[File:resultsdir.png| Directory structure of a project containing results]]<br />
|}<br />
<br />
<br />
The mesh files contain vertices information regarding the shape of the growth model at a particular stage of growth, but are not visualisable in that form. In order to '''convert these mesh files into viewable images''', we can execute the following command in Matlab:<br />
<br />
VMSreport('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Project','GPT_CASE_RST',...<br />
'Experiment','All','flattentime',572.5,'morphogen','KPAR','SNAPFIG',true);<br />
<br />
Where the ''Path'' and ''Project'' arguments have the same function as ''Path'' and ''Name'' in 3A, i.e. ''Path'' refers to the location of the project folder and ''Project'' is the name of the project folder itself.<br />
<br />
Here we can see the generated images, including the command line arguments, for one of the simulations performed in 3A. Click on a thumbnail to view a larger image.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="125px"| <small>001commandline.png</small><br> [[File:commandlineexample.png | 100px]]<br />
|width="125px"| <small>001GPT_CASE_RST_2 wild_.png</small><br>[[File:GPT_CASE_RST_example1.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_3 wild_.png</small><br>[[File:GPT_CASE_RST_example2.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_4 wild_.png</small><br>[[File:GPT_CASE_RST_example3.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_5 wild_.png</small><br>[[File:GPT_CASE_RST_example4.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_6 wild_.png</small><br>[[File:GPT_CASE_RST_example5.png | 125px]]<br />
|}<br />
<br />
<br />
'''2) - Generating images from a simulation you ran on the cluster'''<br />
<br />
NB. The results produced may not be visibly identical to those in the published literature. This is because of small, random perturbations which are applied to the initial model meshes to stop them from containing surfaces which are perfectly flat, and therefore biologically unrealistic. The results produced should be '''qualitatively but not quantitatively the same'''.<br />
<br />
'''3) - Interacting with your results using the GFtbox GUI'''<br />
<br />
='''4''' Altering the simulation parameters=<br />
<br />
One such simulation parameter is ''dt'', which is the time in seconds between iterations in the growth simulation. Large values of ''dt'' mean that fewer steps and therefore fewer calculations are required to complete a simulation. Whereas smaller values of ''dt'' mean that more steps are required, and therefore more processing time too. Therefore, a value of ''dt'' must be selected which is not so small that it is computationally unmanageable, but not so large that the observed growth is an artifact of the value of ''dt'', rather than the underlying growth model. It is necessary therefore to test a range of values for ''dt'' to ensure that the patterns of growth observed in a simulation are consistent across the range, and to find a value which is sufficient to demonstrate the model of growth and computationally efficient.<br />
<br />
'''Testing a range of ''dt''s can be achieved in several ways.''' One way is to use the ''dt'' argument when calling GFtboxCommand. This allows a single value to be tested. Another way is to make a batch of jobs, each using a different value for ''dt'', by using the 'State' argument. Lastly, a range of ''dt''s can be specified in GFtboxCommand, by using the dt argument where the values within the square brackets are the ''dt''s to simulate:<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5]);<br />
<br />
Here, we have specified '''four separate values for ''dt''''', and this means that '''four separate simulations will be run''' sequentially on your computer. As in 3B, images can then be generated for the mesh files produced, and compared to ensure consistency across the range of ''dt''s. In this figure, we can see the same model, at the same stage of growth, generated using the four values of ''dt''. Though quantitatively different, they are qualitatively the same, illustrating the suitability of the default ''dt'' value of 5.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="190px"| [[File:GPT_CASE_RST_dt0.5.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt1.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt2.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt5.png| 190px]]<br />
|}<br />
<br />
='''5''' Altering the model parameters on the cluster=<br />
It is easy to see that when even a small number of range variables, mutations or ''dt''s are specified, the '''total number of simulations increases quickly'''. If many ranges are specified, then the amount of processing time becomes unmanageable on a single computer. Via ''GFtboxCommand'' and the ''ClusterMonitor'' tool, we can remotely''' run a number of simulations, in parallel, on a computing cluster'''. It is assumed that the GrowthToolbox and Matlab are installed on your cluster, and PuTTY is required on your local computer for the tools ''pscp'' (for transferring files from your computer to a cluster) and ''plink'' (for remotely executing commands on a cluster). Using the ''dt'' range argument from 4 as an example, here we add the 'Use' name argument with the value 'Cluster':<br />
<br />
GFtboxCommand('State','Start','Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5],'Use','Cluster');<br />
<br />
The 'Cluster' option for the 'Use' name argument instructs GFtboxCommand to upload the required project directory to a remote Linux server. Instead of running the simulation on your own computer, GFtboxCommand then works out how many individual simulations are specified by the command that invoked it. In this example, the only argument which will generate multiple simulations is ''dt'', of which there will be 4. Separate commands for each of these jobs are then automatically generated, each with an accompanying unique ID (The value of the ''ExpID'' name argument), and '''these are submitted as individual jobs to the cluster'''.<br />
<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.1,'ExpID','GPT_CASE_RST_1');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.5,'ExpID','GPT_CASE_RST_2');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',1,'ExpID','GPT_CASE_RST_3');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',5,'ExpID','GPT_CASE_RST_4');<br />
<br />
Once these jobs have been submitted, the ''ClusterMonitor'' tool opens and the new job batch ID is added to the list of jobs. Here, we will concentrate on three of the functions of ''ClusterMonitor'': ''Queue?'', ''Get project results'' and ''Make project pngs''. As in part 4 of this page, these functions will enable us to visualise the growth simulation results.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| This is an example of what you will see when the ClusterMonitor tool launches after submitting a job using the 'Use','Cluster' argument pair in GFtboxCommand.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Clicking the 'Queue?' button tells ClusterMonitor to check the status of your jobs on the cluster. It then prints the status of the current jobs in Matlab's command window. In the case of our cluster, this is the equivalent of the ''qstatme'' command. This screenshot shows an example job list, where one job is currently running.<br />
<br />
This allows you too see whether or not your jobs have finished, and therefore whether or not their results are ready to retrieve.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| This is an example of what you will see when the ClusterMonitor tool launches after submitting a job using the 'Use','Cluster' argument pair in GFtboxCommand.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Once your jobs have completed, the 'Queue?' button will show that there are no jobs running. Now you are ready to retrieve the project files from the cluster. This is achieved using the 'Get project results' button. Once pressed, the Matlab command window will show trace information to indicate which files are currently being downloaded. This is the equivalent of downloading the files manually, using an FTP program.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Once you have successfully downloaded your project results, you are ready to generate result images, as in part 3B1. Instead of entering a ''VMSReport'' command, you can press the 'Make project pngs' button, which will perform the same command.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
<br />
<br />
Whilst ''ClusterMonitor'' is a useful tool for managing your cluster jobs, it is advisable to '''have at least a rudimentary understanding of and ability to use a Unix-based computer system'''. In particular, the abilities to list the contents of a folder, view the contents of a file, change the current working folder, delete, and to copy or move files or folders, are essential Unix skills for making sure everything is working as intended. It is beyond the scope of these tutorials to provide an in depth Unix tutorial (many good and simple tutorials exist on the web), but here is a short description of the Unix commands (which may be specific to our Unix-based system) that we believe to be important.<br />
<br />
''ls'' - list the files and folders in a directory. Use the -al option (e.g. ls -al) to see modification dates, file permissions, sizes, etc.<br />
<br />
''cd X'' - Change the current directory to directory X.<br />
<br />
''rm X'' - Delete the file X.<br />
<br />
''cp X NewPath/X2'' - Copy file X to the NewPath directory and name the copy X2.<br />
<br />
''mv X NewPath/X2'' - Move file X to the NewPath directory and rename it to X2.<br />
<br />
''cat test.txt'' - Print the contents of the text file test.txt onto the screen.<br />
<br />
''pwd'' - Print the present working directory onto the screen.<br />
<br />
''more test.txt'' - Print the file test.txt to the screen, a screenful at a time, scrolling each time you hit the enter key.</div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Running_example_models_and_using_a_cluster&diff=5414Running example models and using a cluster2011-06-24T09:02:46Z<p>JacobNewman: </p>
<hr />
<div>[[GFtbox Tutorial pages | Back to GFtbox Tutorial pages]]<br />
<br />
The purpose of these tutorials is to learn how to run the example growth simulations included in GFtbox. We will describe methods for running simulations locally (on your own computer) and remotely (on a computing cluster). It is assumed that you have already downloaded the GFtbox software and have Matlab installed.<br />
<br />
=Getting Started=<br />
<br />
The remainder of this page is split into five sub-tutorials, each building on the preceding parts.<br />
<br />
1) '''Explaining the tools'''. In these tutorials, we will be using ''GFtboxCommand'' and ''ClusterMonitor''. This section explains the purpose of these tools. <br />
<br />
2) '''Computer or cluster?''' Here, we illustrate when and why you should use a computing cluster for your growth simulations, and conversely, when you should use your desktop computer.<br />
<br />
3) '''Running a growth simulation for an example model'''. This section demonstrates how to use ''GFtboxCommand'' to run a growth simulation on your computer. The model used for the simulation is an example model included with the GFtbox.<br />
<br />
4) '''Altering the simulation parameters'''. Following on from part 3, here we show how to adjust a simulation parameter within GFtboxCommand. Specifically, we alter the value of ''dt'', the temporal resolution of the simulation, and show how it can be used to verify that the value specified in the published literature is reasonable.<br />
<br />
5) '''Altering the model parameters on the cluster'''. Finally, we demonstrate how a number of model parameters can be varied by specifying a range of values for each model argument. We show how the computationally expensive task of simulating all combinations of specified ranges can be processed more efficiently if a computer cluster is used via the ''ClusterMonitor'' tool.<br />
<br />
='''1''' Explaining the tools=<br />
==GFtboxCommand==<br />
This is a command line version of the GFtbox. By command line, we mean that all program '''functions are operated via typed commands, without the GUI'''. Like GFtbox, GFtboxCommand is capable of running growth simulations of an interaction function, and allows the user to specify model and simulation parameters. Unlike GFtbox, this also allows the user to select '''ranges of values for a number of input parameters''', and will automatically spawn multiple simulations which explore the various combinations of those parameters. This can be used to evaluate the effect of various parameters on the growth of a given model.<br />
<br />
==ClusterMonitor==<br />
Provides a graphical user interface for managing simulations '''running remotely on a computer cluster'''. Specifically, it allows you to see which projects are present and running on the cluster, to retrieve the completed projects, to generate images of the simulations at specified stages of growth, and to remove projects from the cluster. If you do not intend to use a computer cluster, then you will not need to use ClusterMonitor.<br />
<br />
='''2''' Computer or cluster?=<br />
<br />
In basic terms, '''a computing cluster is effectively a network of many computer processors''' (often hundreds), centrally managed by a queuing system. When a job is submitted to a cluster, the job is sent to a processor that is not being used, or queued until one becomes available. In contrast, a typical desktop computer will contain one processor, limiting the number of tasks that can be performed at any one time. Jobs which must be run independently and sequentially on a desktop computer can be executed in parallel on a cluster, '''greatly reducing the total time required to complete all of the jobs'''. Although the exact details of your cluster might vary from those described in the rest of these tutorials, we aim to illustrate the generic processes involved in using GFtbox on a cluster. We are happy to offer assistance, where possible, to setup GFtbox for your cluster, so please contact us if you have any queries.<br />
<br />
Whilst the time savings offered by using a cluster can be significant, there is an overhead associated with returning the results to your personal computer. The total time to run 2 or more simulations on a cluster and to return the results will be less than running those simulations sequentially on one computer. Therefore, using a '''cluster is ideal for situations where you would like to run several simulations''', such as to evaluate the effect of a range of parameters on a growth model. You are not advised to use a cluster for running single simulations, or where you would like to step through a simulation and change parameters in a more interactive fashion. In such circumstances, you are advised to run GFtbox or GFtboxCommand on your own computer.<br />
<br />
The '''GFtbox GUI provides quick feedback''' about how the changes you have made to an interaction function have affected the growth simulation, as '''you can see the result of each simulation iteration as it completes'''. It is quick and easy therefore to see if you have dramatically changed the course of the growth simulation, and then to adjust the parameters according to your observations. This approach is well suited to the early stages of design, where you might wish to '''tweak some parameters to gauge whether or not they have had the desired effect'''. Or in the final stages, where you think your model is almost finalised, but where small adjustments are required. '''An alternative approach is to start many simulations''', based upon your initial model and by making intelligent choices about which parameters to explore. This allows you to '''harvest many results, to quickly and easily overview them''' in their finalised form, or to select interesting-looking simulations and examine them more closely on your desktop computer, using GFtbox.<br />
<br />
='''3''' Running a growth simulation for an example model=<br />
This tutorial is aimed at running a '''growth simulation for one of the example interaction functions''' included with the GFtbox. The purpose of this exercise is to firstly demonstrate how simulations can be invoked using GFtboxCommand, and secondly to show how to reproduce experimental results (specifically, those published in Kennaway et al (2011)) given an interaction function.<br />
<br />
Assuming Matlab is installed on your computer, and the latest GFtbox has been downloaded, you can add the GFtbox directory to Matlab's search path, which makes the toolbox accessible from any other path that you choose to work from. For a short tutorial on how to do this, please click [[Adding GFtbox to Matlab's search path | here]].<br />
<br />
Once the GFtbox is added to Matlab, you are ready to run a growth simulation using GFtboxCommand. In this example, '''the model that we will simulate is called GPT_CASE_RST'''. Results generated using this model are published in Kennaway et al (2011). By running this simulation, we can confirm the results in the published literature and investigate the suitability of the various parameters.<br />
<br />
===3A - Running a simulation on your computer===<br />
<br />
The following command can be typed into Matlab to run a simulation of the GPT_CASE_RST interaction function, which contains three growth models: R, S and T. '''Three separate simulations are run sequentially on your computer''', one for each model, each producing results corresponding to five intervals in the growth simulation.<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',[1:3]);<br />
<br />
GFtboxCommand accepts input arguments as name and value pairs, e.g. 'modelname', [1:3] or 'Use','Cluster'. The argument names entered in the example above are: ''Path'', ''Name'', ''Stages'' and ''modelname''.<br />
<br />
The optional '''''Path'' argument name refers to the location of the folder''' (or directory) on your computer where the model interaction function you wish to simulate is stored. '''''Name'', is the name of the folder itself'''. In this case, we are using the GPT_CASE_RST folder which is included in the GrowthToolbox. You may wish to copy this folder elsewhere, if you intend to make changes to the interaction function.<br />
<br />
During a growth simulation, a mesh can be generated at each time step of the simulation, which provides a visual representation of the growth of the biological tissue, given the various parameters of growth specified by the interaction function and how they have changed over time. Put another way, the mesh shows exactly what the growing tissue actually looks like. ''Stages'' refers to the points in the simulation (measured in hours) at which meshes should be generated and saved. In this example, '''five stages of growth will be written to disk'''. These values are chosen to best capture the appearance of the tissue at important stages of the tissue growth.<br />
<br />
The final argument name listed here is ''modelname''. This is a model-specific argument, and in this case the GPT_CASE_RST interaction function contains '''three separate models for plant growth''', allowing the desired model to be a selected. Here, the value [1:3] is specified, which is evaluated in the same way as entering [1 2 3]. This instructs GFtboxCommand to run three separate simulations, one for each of the growth models contained in the interaction function.<br />
<br />
The function of every permissible argument is given by keying the following command into Matlab:<br />
<br />
help GFtboxCommand<br />
<br />
===3B - Getting results from a completed simulation===<br />
<br />
'''1) - Generating images from a simulation you ran on your computer'''<br />
<br />
Once a growth simulation is completed, the project folder (GPT_CASE_RST, in our example) will contain another folder named "movies". Within movies, there are folders which contain results for the executed simulations. As was instructed in 3A, here we can see three folders corresponding to simulations for the three separate models. Within the first folder there are three items:<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| ''CommandLine.txt'' - This file contains the Matlab command which was used to generate the results that this sub-directory contains.<br />
<br />
''gpt_case_rst.txt'' - This file, named according to the project name, contains a copy of the interaction function which was used to produce these results.<br />
<br />
''meshes'' - This folder contains the mesh files corresponding to the stages of growth specified by the value of the Stages argument, described in 3A.<br />
|width="500px" align="center"|[[File:resultsdir.png| Directory structure of a project containing results]]<br />
|}<br />
<br />
<br />
The mesh files contain vertices information regarding the shape of the growth model at a particular stage of growth, but are not visualisable in that form. In order to '''convert these mesh files into viewable images''', we can execute the following command in Matlab:<br />
<br />
VMSreport('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Project','GPT_CASE_RST',...<br />
'Experiment','All','flattentime',572.5,'morphogen','KPAR','SNAPFIG',true);<br />
<br />
Where the ''Path'' and ''Project'' arguments have the same function as ''Path'' and ''Name'' in 3A, i.e. ''Path'' refers to the location of the project folder and ''Project'' is the name of the project folder itself.<br />
<br />
Here we can see the generated images, including the command line arguments, for one of the simulations performed in 3A. Click on a thumbnail to view a larger image.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="125px"| <small>001commandline.png</small><br> [[File:commandlineexample.png | 100px]]<br />
|width="125px"| <small>001GPT_CASE_RST_2 wild_.png</small><br>[[File:GPT_CASE_RST_example1.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_3 wild_.png</small><br>[[File:GPT_CASE_RST_example2.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_4 wild_.png</small><br>[[File:GPT_CASE_RST_example3.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_5 wild_.png</small><br>[[File:GPT_CASE_RST_example4.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_6 wild_.png</small><br>[[File:GPT_CASE_RST_example5.png | 125px]]<br />
|}<br />
<br />
<br />
'''2) - Generating images from a simulation you ran on the cluster'''<br />
<br />
NB. The results produced may not be visibly identical to those in the published literature. This is because of small, random perturbations which are applied to the initial model meshes to stop them from containing surfaces which are perfectly flat, and therefore biologically unrealistic. The results produced should be '''qualitatively but not quantitatively the same'''.<br />
<br />
'''3) - Interacting with your results using the GFtbox GUI'''<br />
<br />
='''4''' Altering the simulation parameters=<br />
<br />
One such simulation parameter is ''dt'', which is the time in seconds between iterations in the growth simulation. Large values of ''dt'' mean that fewer steps and therefore fewer calculations are required to complete a simulation. Whereas smaller values of ''dt'' mean that more steps are required, and therefore more processing time too. Therefore, a value of ''dt'' must be selected which is not so small that it is computationally unmanageable, but not so large that the observed growth is an artifact of the value of ''dt'', rather than the underlying growth model. It is necessary therefore to test a range of values for ''dt'' to ensure that the patterns of growth observed in a simulation are consistent across the range, and to find a value which is sufficient to demonstrate the model of growth and computationally efficient.<br />
<br />
'''Testing a range of ''dt''s can be achieved in several ways.''' One way is to use the ''dt'' argument when calling GFtboxCommand. This allows a single value to be tested. Another way is to make a batch of jobs, each using a different value for ''dt'', by using the 'State' argument. Lastly, a range of ''dt''s can be specified in GFtboxCommand, by using the dt argument where the values within the square brackets are the ''dt''s to simulate:<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5]);<br />
<br />
Here, we have specified '''four separate values for ''dt''''', and this means that '''four separate simulations will be run''' sequentially on your computer. As in 3B, images can then be generated for the mesh files produced, and compared to ensure consistency across the range of ''dt''s. In this figure, we can see the same model, at the same stage of growth, generated using the four values of ''dt''. Though quantitatively different, they are qualitatively the same, illustrating the suitability of the default ''dt'' value of 5.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="190px"| [[File:GPT_CASE_RST_dt0.5.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt1.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt2.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt5.png| 190px]]<br />
|}<br />
<br />
='''5''' Altering the model parameters on the cluster=<br />
It is easy to see that when even a small number of range variables, mutations or ''dt''s are specified, the '''total number of simulations increases quickly'''. If many ranges are specified, then the amount of processing time becomes unmanageable on a single computer. Via ''GFtboxCommand'' and the ''ClusterMonitor'' tool, we can remotely''' run a number of simulations, in parallel, on a computing cluster'''. It is assumed that the GrowthToolbox and Matlab are installed on your cluster, and PuTTY is required on your local computer for the tools ''pscp'' (for transferring files from your computer to a cluster) and ''plink'' (for remotely executing commands on a cluster). Using the ''dt'' range argument from 4 as an example, here we add the 'Use' name argument with the value 'Cluster':<br />
<br />
GFtboxCommand('State','Start','Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5],'Use','Cluster');<br />
<br />
The 'Cluster' option for the 'Use' name argument instructs GFtboxCommand to upload the required project directory to a remote Linux server. Instead of running the simulation on your own computer, GFtboxCommand then works out how many individual simulations are specified by the command that invoked it. In this example, the only argument which will generate multiple simulations is ''dt'', of which there will be 4. Separate commands for each of these jobs are then automatically generated, each with an accompanying unique ID (The value of the ''ExpID'' name argument), and '''these are submitted as individual jobs to the cluster'''.<br />
<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.1,'ExpID','GPT_CASE_RST_1');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.5,'ExpID','GPT_CASE_RST_2');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',1,'ExpID','GPT_CASE_RST_3');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',5,'ExpID','GPT_CASE_RST_4');<br />
<br />
Once these jobs have been submitted, the ''ClusterMonitor'' tool opens and the new job batch ID is added to the list of jobs. Here, we will concentrate on three of the functions of ''ClusterMonitor'': ''Queue?'', ''Get project results'' and ''Make project pngs''. As in part 4 of this page, these functions will enable us to visualise the growth simulation results.<br />
Queue <br> Get project results <br> Make project pngs<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| This is an example of what you will see when the ClusterMonitor tool launches after submitting a job using the 'Use','Cluster' argument pair in GFtboxCommand.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Clicking the 'Queue?' button tells ClusterMonitor to check the status of your jobs on the cluster. It then prints the status of the current jobs in Matlab's command window. In the case of our cluster, this is the equivalent of the ''qstatme'' command. This screenshot shows an example job list, where one job is currently running.<br />
<br />
This allows you too see whether or not your jobs have finished, and therefore whether or not their results are ready to retrieve.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| This is an example of what you will see when the ClusterMonitor tool launches after submitting a job using the 'Use','Cluster' argument pair in GFtboxCommand.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Once your jobs have completed, the 'Queue?' button will show that there are no jobs running. Now you are ready to retrieve the project files from the cluster. This is achieved using the 'Get project results' button. Once pressed, the Matlab command window will show trace information to indicate which files are currently being downloaded. This is the equivalent of downloading the files manually, using an FTP program.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Once you have successfully downloaded your project results, you are ready to generate result images.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
<br />
<br />
Whilst ''ClusterMonitor'' is a useful tool for managing your cluster jobs, it is advisable to '''have at least a rudimentary understanding of and ability to use a Unix-based computer system'''. In particular, the abilities to list the contents of a folder, view the contents of a file, change the current working folder, delete, and to copy or move files or folders, are essential Unix skills for making sure everything is working as intended. It is beyond the scope of these tutorials to provide an in depth Unix tutorial (many good and simple tutorials exist on the web), but here is a short description of the Unix commands (which may be specific to our Unix-based system) that we believe to be important.<br />
<br />
''ls'' - list the files and folders in a directory. Use the -al option (e.g. ls -al) to see modification dates, file permissions, sizes, etc.<br />
<br />
''cd X'' - Change the current directory to directory X.<br />
<br />
''rm X'' - Delete the file X.<br />
<br />
''cp X NewPath/X2'' - Copy file X to the NewPath directory and name the copy X2.<br />
<br />
''mv X NewPath/X2'' - Move file X to the NewPath directory and rename it to X2.<br />
<br />
''cat test.txt'' - Print the contents of the text file test.txt onto the screen.<br />
<br />
''pwd'' - Print the present working directory onto the screen.<br />
<br />
''more test.txt'' - Print the file test.txt to the screen, a screenful at a time, scrolling each time you hit the enter key.</div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Running_example_models_and_using_a_cluster&diff=5388Running example models and using a cluster2011-06-24T08:24:47Z<p>JacobNewman: </p>
<hr />
<div>[[GFtbox Tutorial pages | Back to GFtbox Tutorial pages]]<br />
<br />
The purpose of these tutorials is to learn how to run the example growth simulations included in GFtbox. We will describe methods for running simulations locally (on your own computer) and remotely (on a computing cluster). It is assumed that you have already downloaded the GFtbox software and have Matlab installed.<br />
<br />
=Getting Started=<br />
<br />
The remainder of this page is split into five sub-tutorials, each building on the preceding parts.<br />
<br />
1) '''Explaining the tools'''. In these tutorials, we will be using ''GFtboxCommand'' and ''ClusterMonitor''. This section explains the purpose of these tools. <br />
<br />
2) '''Computer or cluster?''' Here, we illustrate when and why you should use a computing cluster for your growth simulations, and conversely, when you should use your desktop computer.<br />
<br />
3) '''Running a growth simulation for an example model'''. This section demonstrates how to use ''GFtboxCommand'' to run a growth simulation on your computer. The model used for the simulation is an example model included with the GFtbox.<br />
<br />
4) '''Altering the simulation parameters'''. Following on from part 3, here we show how to adjust a simulation parameter within GFtboxCommand. Specifically, we alter the value of ''dt'', the temporal resolution of the simulation, and show how it can be used to verify that the value specified in the published literature is reasonable.<br />
<br />
5) '''Altering the model parameters on the cluster'''. Finally, we demonstrate how a number of model parameters can be varied by specifying a range of values for each model argument. We show how the computationally expensive task of simulating all combinations of specified ranges can be processed more efficiently if a computer cluster is used via the ''ClusterMonitor'' tool.<br />
<br />
='''1''' Explaining the tools=<br />
==GFtboxCommand==<br />
This is a command line version of the GFtbox. By command line, we mean that all program '''functions are operated via typed commands, without the GUI'''. Like GFtbox, GFtboxCommand is capable of running growth simulations of an interaction function, and allows the user to specify model and simulation parameters. Unlike GFtbox, this also allows the user to select '''ranges of values for a number of input parameters''', and will automatically spawn multiple simulations which explore the various combinations of those parameters. This can be used to evaluate the effect of various parameters on the growth of a given model.<br />
<br />
==ClusterMonitor==<br />
Provides a graphical user interface for managing simulations '''running remotely on a computer cluster'''. Specifically, it allows you to see which projects are present and running on the cluster, to retrieve the completed projects, to generate images of the simulations at specified stages of growth, and to remove projects from the cluster. If you do not intend to use a computer cluster, then you will not need to use ClusterMonitor.<br />
<br />
='''2''' Computer or cluster?=<br />
<br />
In basic terms, '''a computing cluster is effectively a network of many computer processors''' (often hundreds), centrally managed by a queuing system. When a job is submitted to a cluster, the job is sent to a processor that is not being used, or queued until one becomes available. In contrast, a typical desktop computer will contain one processor, limiting the number of tasks that can be performed at any one time. Jobs which must be run independently and sequentially on a desktop computer can be executed in parallel on a cluster, '''greatly reducing the total time required to complete all of the jobs'''. Although the exact details of your cluster might vary from those described in the rest of these tutorials, we aim to illustrate the generic processes involved in using GFtbox on a cluster. We are happy to offer assistance, where possible, to setup GFtbox for your cluster, so please contact us if you have any queries.<br />
<br />
Whilst the time savings offered by using a cluster can be significant, there is an overhead associated with returning the results to your personal computer. The total time to run 2 or more simulations on a cluster and to return the results will be less than running those simulations sequentially on one computer. Therefore, using a '''cluster is ideal for situations where you would like to run several simulations''', such as to evaluate the effect of a range of parameters on a growth model. You are not advised to use a cluster for running single simulations, or where you would like to step through a simulation and change parameters in a more interactive fashion. In such circumstances, you are advised to run GFtbox or GFtboxCommand on your own computer.<br />
<br />
The '''GFtbox GUI provides quick feedback''' about how the changes you have made to an interaction function have affected the growth simulation, as '''you can see the result of each simulation iteration as it completes'''. It is quick and easy therefore to see if you have dramatically changed the course of the growth simulation, and then to adjust the parameters according to your observations. This approach is well suited to the early stages of design, where you might wish to '''tweak some parameters to gauge whether or not they have had the desired effect'''. Or in the final stages, where you think your model is almost finalised, but where small adjustments are required. '''An alternative approach is to start many simulations''', based upon your initial model and by making intelligent choices about which parameters to explore. This allows you to '''harvest many results, to quickly and easily overview them''' in their finalised form, or to select interesting-looking simulations and examine them more closely on your desktop computer, using GFtbox.<br />
<br />
='''3''' Running a growth simulation for an example model=<br />
This tutorial is aimed at running a '''growth simulation for one of the example interaction functions''' included with the GFtbox. The purpose of this exercise is to firstly demonstrate how simulations can be invoked using GFtboxCommand, and secondly to show how to reproduce experimental results (specifically, those published in Kennaway et al (2011)) given an interaction function.<br />
<br />
Assuming Matlab is installed on your computer, and the latest GFtbox has been downloaded, you can add the GFtbox directory to Matlab's search path, which makes the toolbox accessible from any other path that you choose to work from. For a short tutorial on how to do this, please click [[Adding GFtbox to Matlab's search path | here]].<br />
<br />
Once the GFtbox is added to Matlab, you are ready to run a growth simulation using GFtboxCommand. In this example, '''the model that we will simulate is called GPT_CASE_RST'''. Results generated using this model are published in Kennaway et al (2011). By running this simulation, we can confirm the results in the published literature and investigate the suitability of the various parameters.<br />
<br />
===3A - Running a simulation on your computer===<br />
<br />
The following command can be typed into Matlab to run a simulation of the GPT_CASE_RST interaction function, which contains three growth models: R, S and T. '''Three separate simulations are run sequentially on your computer''', one for each model, each producing results corresponding to five intervals in the growth simulation.<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',[1:3]);<br />
<br />
GFtboxCommand accepts input arguments as name and value pairs, e.g. 'modelname', [1:3] or 'Use','Cluster'. The argument names entered in the example above are: ''Path'', ''Name'', ''Stages'' and ''modelname''.<br />
<br />
The optional '''''Path'' argument name refers to the location of the folder''' (or directory) on your computer where the model interaction function you wish to simulate is stored. '''''Name'', is the name of the folder itself'''. In this case, we are using the GPT_CASE_RST folder which is included in the GrowthToolbox. You may wish to copy this folder elsewhere, if you intend to make changes to the interaction function.<br />
<br />
During a growth simulation, a mesh can be generated at each time step of the simulation, which provides a visual representation of the growth of the biological tissue, given the various parameters of growth specified by the interaction function and how they have changed over time. Put another way, the mesh shows exactly what the growing tissue actually looks like. ''Stages'' refers to the points in the simulation (measured in hours) at which meshes should be generated and saved. In this example, '''five stages of growth will be written to disk'''. These values are chosen to best capture the appearance of the tissue at important stages of the tissue growth.<br />
<br />
The final argument name listed here is ''modelname''. This is a model-specific argument, and in this case the GPT_CASE_RST interaction function contains '''three separate models for plant growth''', allowing the desired model to be a selected. Here, the value [1:3] is specified, which is evaluated in the same way as entering [1 2 3]. This instructs GFtboxCommand to run three separate simulations, one for each of the growth models contained in the interaction function.<br />
<br />
The function of every permissible argument is given by keying the following command into Matlab:<br />
<br />
help GFtboxCommand<br />
<br />
===3B - Getting results from a completed simulation===<br />
<br />
'''1) - Generating images from a simulation you ran on your computer'''<br />
<br />
Once a growth simulation is completed, the project folder (GPT_CASE_RST, in our example) will contain another folder named "movies". Within movies, there are folders which contain results for the executed simulations. As was instructed in 3A, here we can see three folders corresponding to simulations for the three separate models. Within the first folder there are three items:<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| ''CommandLine.txt'' - This file contains the Matlab command which was used to generate the results that this sub-directory contains.<br />
<br />
''gpt_case_rst.txt'' - This file, named according to the project name, contains a copy of the interaction function which was used to produce these results.<br />
<br />
''meshes'' - This folder contains the mesh files corresponding to the stages of growth specified by the value of the Stages argument, described in 3A.<br />
|width="500px" align="center"|[[File:resultsdir.png| Directory structure of a project containing results]]<br />
|}<br />
<br />
<br />
The mesh files contain vertices information regarding the shape of the growth model at a particular stage of growth, but are not visualisable in that form. In order to '''convert these mesh files into viewable images''', we can execute the following command in Matlab:<br />
<br />
VMSreport('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Project','GPT_CASE_RST',...<br />
'Experiment','All','flattentime',572.5,'morphogen','KPAR','SNAPFIG',true);<br />
<br />
Where the ''Path'' and ''Project'' arguments have the same function as ''Path'' and ''Name'' in 3A, i.e. ''Path'' refers to the location of the project folder and ''Project'' is the name of the project folder itself.<br />
<br />
Here we can see the generated images, including the command line arguments, for one of the simulations performed in 3A. Click on a thumbnail to view a larger image.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="125px"| <small>001commandline.png</small><br> [[File:commandlineexample.png | 100px]]<br />
|width="125px"| <small>001GPT_CASE_RST_2 wild_.png</small><br>[[File:GPT_CASE_RST_example1.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_3 wild_.png</small><br>[[File:GPT_CASE_RST_example2.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_4 wild_.png</small><br>[[File:GPT_CASE_RST_example3.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_5 wild_.png</small><br>[[File:GPT_CASE_RST_example4.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_6 wild_.png</small><br>[[File:GPT_CASE_RST_example5.png | 125px]]<br />
|}<br />
<br />
<br />
'''2) - Generating images from a simulation you ran on the cluster'''<br />
<br />
NB. The results produced may not be visibly identical to those in the published literature. This is because of small, random perturbations which are applied to the initial model meshes to stop them from containing surfaces which are perfectly flat, and therefore biologically unrealistic. The results produced should be '''qualitatively but not quantitatively the same'''.<br />
<br />
'''3) - Interacting with your results using the GFtbox GUI'''<br />
<br />
='''4''' Altering the simulation parameters=<br />
<br />
One such simulation parameter is ''dt'', which is the time in seconds between iterations in the growth simulation. Large values of ''dt'' mean that fewer steps and therefore fewer calculations are required to complete a simulation. Whereas smaller values of ''dt'' mean that more steps are required, and therefore more processing time too. Therefore, a value of ''dt'' must be selected which is not so small that it is computationally unmanageable, but not so large that the observed growth is an artifact of the value of ''dt'', rather than the underlying growth model. It is necessary therefore to test a range of values for ''dt'' to ensure that the patterns of growth observed in a simulation are consistent across the range, and to find a value which is sufficient to demonstrate the model of growth and computationally efficient.<br />
<br />
'''Testing a range of ''dt''s can be achieved in several ways.''' One way is to use the ''dt'' argument when calling GFtboxCommand. This allows a single value to be tested. Another way is to make a batch of jobs, each using a different value for ''dt'', by using the 'State' argument. Lastly, a range of ''dt''s can be specified in GFtboxCommand, by using the dt argument where the values within the square brackets are the ''dt''s to simulate:<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5]);<br />
<br />
Here, we have specified '''four separate values for ''dt''''', and this means that '''four separate simulations will be run''' sequentially on your computer. As in 3B, images can then be generated for the mesh files produced, and compared to ensure consistency across the range of ''dt''s. In this figure, we can see the same model, at the same stage of growth, generated using the four values of ''dt''. Though quantitatively different, they are qualitatively the same, illustrating the suitability of the default ''dt'' value of 5.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="190px"| [[File:GPT_CASE_RST_dt0.5.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt1.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt2.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt5.png| 190px]]<br />
|}<br />
<br />
='''5''' Altering the model parameters on the cluster=<br />
It is easy to see that when even a small number of range variables, mutations or ''dt''s are specified, the '''total number of simulations increases quickly'''. If many ranges are specified, then the amount of processing time becomes unmanageable on a single computer. Via ''GFtboxCommand'' and the ''ClusterMonitor'' tool, we can remotely''' run a number of simulations, in parallel, on a computing cluster'''. It is assumed that the GrowthToolbox and Matlab are installed on your cluster, and PuTTY is required on your local computer for the tools ''pscp'' (for transferring files from your computer to a cluster) and ''plink'' (for remotely executing commands on a cluster). Using the ''dt'' range argument from 4 as an example, here we add the 'Use' name argument with the value 'Cluster':<br />
<br />
GFtboxCommand('State','Start','Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5],'Use','Cluster');<br />
<br />
The 'Cluster' option for the 'Use' name argument instructs GFtboxCommand to upload the required project directory to a remote Linux server. Instead of running the simulation on your own computer, GFtboxCommand then works out how many individual simulations are specified by the command that invoked it. In this example, the only argument which will generate multiple simulations is ''dt'', of which there will be 4. Separate commands for each of these jobs are then automatically generated, each with an accompanying unique ID (The value of the ''ExpID'' name argument), and '''these are submitted as individual jobs to the cluster'''.<br />
<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.1,'ExpID','GPT_CASE_RST_1');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.5,'ExpID','GPT_CASE_RST_2');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',1,'ExpID','GPT_CASE_RST_3');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',5,'ExpID','GPT_CASE_RST_4');<br />
<br />
Once these jobs have been submitted, the ''ClusterMonitor'' tool opens and the new job batch ID is added to the list of jobs. Here, we will concentrate on three of the functions of ''ClusterMonitor'': ''Queue?'', ''Get project results'' and ''Make project pngs''. As in part 4 of this page, these functions will enable us to visualise the growth simulation results.<br />
Queue <br> Get project results <br> Make project pngs<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| This is an example of what you will see when the ClusterMonitor tool launches after submitting a job using the 'Use','Cluster' argument pair in GFtboxCommand.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Clicking the 'Queue?' button tells ClusterMonitor to check the status of your jobs on the cluster. It then prints the status of the current jobs in Matlab's command window. In the case of our cluster, this is the equivalent of the ''qstatme'' command. This screenshot shows an example job list, where one job is currently running.<br />
<br />
This allows you too see whether or not your jobs have finished, and therefore whether or not their results are ready to retrieve.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
<br />
Whilst ''ClusterMonitor'' is a useful tool for managing your cluster jobs, it is advisable to '''have at least a rudimentary understanding of and ability to use a Unix-based computer system'''. In particular, the abilities to list the contents of a folder, view the contents of a file, change the current working folder, delete, and to copy or move files or folders, are essential Unix skills for making sure everything is working as intended. It is beyond the scope of these tutorials to provide an in depth Unix tutorial (many good and simple tutorials exist on the web), but here is a short description of the Unix commands (which may be specific to our Unix-based system) that we believe to be important.<br />
<br />
''ls'' - list the files and folders in a directory. Use the -al option (e.g. ls -al) to see modification dates, file permissions, sizes, etc.<br />
<br />
''cd X'' - Change the current directory to directory X.<br />
<br />
''rm X'' - Delete the file X.<br />
<br />
''cp X NewPath/X2'' - Copy file X to the NewPath directory and name the copy X2.<br />
<br />
''mv X NewPath/X2'' - Move file X to the NewPath directory and rename it to X2.<br />
<br />
''cat test.txt'' - Print the contents of the text file test.txt onto the screen.<br />
<br />
''pwd'' - Print the present working directory onto the screen.<br />
<br />
''more test.txt'' - Print the file test.txt to the screen, a screenful at a time, scrolling each time you hit the enter key.</div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Running_example_models_and_using_a_cluster&diff=5415Running example models and using a cluster2011-06-24T08:22:41Z<p>JacobNewman: </p>
<hr />
<div>[[GFtbox Tutorial pages | Back to GFtbox Tutorial pages]]<br />
<br />
The purpose of these tutorials is to learn how to run the example growth simulations included in GFtbox. We will describe methods for running simulations locally (on your own computer) and remotely (on a computing cluster). It is assumed that you have already downloaded the GFtbox software and have Matlab installed.<br />
<br />
=Getting Started=<br />
<br />
The remainder of this page is split into five sub-tutorials, each building on the preceding parts.<br />
<br />
1) '''Explaining the tools'''. In these tutorials, we will be using ''GFtboxCommand'' and ''ClusterMonitor''. This section explains the purpose of these tools. <br />
<br />
2) '''Computer or cluster?''' Here, we illustrate when and why you should use a computing cluster for your growth simulations, and conversely, when you should use your desktop computer.<br />
<br />
3) '''Running a growth simulation for an example model'''. This section demonstrates how to use ''GFtboxCommand'' to run a growth simulation on your computer. The model used for the simulation is an example model included with the GFtbox.<br />
<br />
4) '''Altering the simulation parameters'''. Following on from part 3, here we show how to adjust a simulation parameter within GFtboxCommand. Specifically, we alter the value of ''dt'', the temporal resolution of the simulation, and show how it can be used to verify that the value specified in the published literature is reasonable.<br />
<br />
5) '''Altering the model parameters on the cluster'''. Finally, we demonstrate how a number of model parameters can be varied by specifying a range of values for each model argument. We show how the computationally expensive task of simulating all combinations of specified ranges can be processed more efficiently if a computer cluster is used via the ''ClusterMonitor'' tool.<br />
<br />
='''1''' Explaining the tools=<br />
==GFtboxCommand==<br />
This is a command line version of the GFtbox. By command line, we mean that all program '''functions are operated via typed commands, without the GUI'''. Like GFtbox, GFtboxCommand is capable of running growth simulations of an interaction function, and allows the user to specify model and simulation parameters. Unlike GFtbox, this also allows the user to select '''ranges of values for a number of input parameters''', and will automatically spawn multiple simulations which explore the various combinations of those parameters. This can be used to evaluate the effect of various parameters on the growth of a given model.<br />
<br />
==ClusterMonitor==<br />
Provides a graphical user interface for managing simulations '''running remotely on a computer cluster'''. Specifically, it allows you to see which projects are present and running on the cluster, to retrieve the completed projects, to generate images of the simulations at specified stages of growth, and to remove projects from the cluster. If you do not intend to use a computer cluster, then you will not need to use ClusterMonitor.<br />
<br />
='''2''' Computer or cluster?=<br />
<br />
In basic terms, '''a computing cluster is effectively a network of many computer processors''' (often hundreds), centrally managed by a queuing system. When a job is submitted to a cluster, the job is sent to a processor that is not being used, or queued until one becomes available. In contrast, a typical desktop computer will contain one processor, limiting the number of tasks that can be performed at any one time. Jobs which must be run independently and sequentially on a desktop computer can be executed in parallel on a cluster, '''greatly reducing the total time required to complete all of the jobs'''. Although the exact details of your cluster might vary from those described in the rest of these tutorials, we aim to illustrate the generic processes involved in using GFtbox on a cluster. We are happy to offer assistance, where possible, to setup GFtbox for your cluster, so please contact us if you have any queries.<br />
<br />
Whilst the time savings offered by using a cluster can be significant, there is an overhead associated with returning the results to your personal computer. The total time to run 2 or more simulations on a cluster and to return the results will be less than running those simulations sequentially on one computer. Therefore, using a '''cluster is ideal for situations where you would like to run several simulations''', such as to evaluate the effect of a range of parameters on a growth model. You are not advised to use a cluster for running single simulations, or where you would like to step through a simulation and change parameters in a more interactive fashion. In such circumstances, you are advised to run GFtbox or GFtboxCommand on your own computer.<br />
<br />
The '''GFtbox GUI provides quick feedback''' about how the changes you have made to an interaction function have affected the growth simulation, as '''you can see the result of each simulation iteration as it completes'''. It is quick and easy therefore to see if you have dramatically changed the course of the growth simulation, and then to adjust the parameters according to your observations. This approach is well suited to the early stages of design, where you might wish to '''tweak some parameters to gauge whether or not they have had the desired effect'''. Or in the final stages, where you think your model is almost finalised, but where small adjustments are required. '''An alternative approach is to start many simulations''', based upon your initial model and by making intelligent choices about which parameters to explore. This allows you to '''harvest many results, to quickly and easily overview them''' in their finalised form, or to select interesting-looking simulations and examine them more closely on your desktop computer, using GFtbox.<br />
<br />
='''3''' Running a growth simulation for an example model=<br />
This tutorial is aimed at running a '''growth simulation for one of the example interaction functions''' included with the GFtbox. The purpose of this exercise is to firstly demonstrate how simulations can be invoked using GFtboxCommand, and secondly to show how to reproduce experimental results (specifically, those published in Kennaway et al (2011)) given an interaction function.<br />
<br />
Assuming Matlab is installed on your computer, and the latest GFtbox has been downloaded, you can add the GFtbox directory to Matlab's search path, which makes the toolbox accessible from any other path that you choose to work from. For a short tutorial on how to do this, please click [[Adding GFtbox to Matlab's search path | here]].<br />
<br />
Once the GFtbox is added to Matlab, you are ready to run a growth simulation using GFtboxCommand. In this example, '''the model that we will simulate is called GPT_CASE_RST'''. Results generated using this model are published in Kennaway et al (2011). By running this simulation, we can confirm the results in the published literature and investigate the suitability of the various parameters.<br />
<br />
===3A - Running a simulation on your computer===<br />
<br />
The following command can be typed into Matlab to run a simulation of the GPT_CASE_RST interaction function, which contains three growth models: R, S and T. '''Three separate simulations are run sequentially on your computer''', one for each model, each producing results corresponding to five intervals in the growth simulation.<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',[1:3]);<br />
<br />
GFtboxCommand accepts input arguments as name and value pairs, e.g. 'modelname', [1:3] or 'Use','Cluster'. The argument names entered in the example above are: ''Path'', ''Name'', ''Stages'' and ''modelname''.<br />
<br />
The optional '''''Path'' argument name refers to the location of the folder''' (or directory) on your computer where the model interaction function you wish to simulate is stored. '''''Name'', is the name of the folder itself'''. In this case, we are using the GPT_CASE_RST folder which is included in the GrowthToolbox. You may wish to copy this folder elsewhere, if you intend to make changes to the interaction function.<br />
<br />
During a growth simulation, a mesh can be generated at each time step of the simulation, which provides a visual representation of the growth of the biological tissue, given the various parameters of growth specified by the interaction function and how they have changed over time. Put another way, the mesh shows exactly what the growing tissue actually looks like. ''Stages'' refers to the points in the simulation (measured in hours) at which meshes should be generated and saved. In this example, '''five stages of growth will be written to disk'''. These values are chosen to best capture the appearance of the tissue at important stages of the tissue growth.<br />
<br />
The final argument name listed here is ''modelname''. This is a model-specific argument, and in this case the GPT_CASE_RST interaction function contains '''three separate models for plant growth''', allowing the desired model to be a selected. Here, the value [1:3] is specified, which is evaluated in the same way as entering [1 2 3]. This instructs GFtboxCommand to run three separate simulations, one for each of the growth models contained in the interaction function.<br />
<br />
The function of every permissible argument is given by keying the following command into Matlab:<br />
<br />
help GFtboxCommand<br />
<br />
===3B - Getting results from a completed simulation===<br />
<br />
'''1) - Generating images from a simulation you ran on your computer'''<br />
<br />
Once a growth simulation is completed, the project folder (GPT_CASE_RST, in our example) will contain another folder named "movies". Within movies, there are folders which contain results for the executed simulations. As was instructed in 3A, here we can see three folders corresponding to simulations for the three separate models. Within the first folder there are three items:<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| ''CommandLine.txt'' - This file contains the Matlab command which was used to generate the results that this sub-directory contains.<br />
<br />
''gpt_case_rst.txt'' - This file, named according to the project name, contains a copy of the interaction function which was used to produce these results.<br />
<br />
''meshes'' - This folder contains the mesh files corresponding to the stages of growth specified by the value of the Stages argument, described in 3A.<br />
|width="500px" align="center"|[[File:resultsdir.png| Directory structure of a project containing results]]<br />
|}<br />
<br />
<br />
The mesh files contain vertices information regarding the shape of the growth model at a particular stage of growth, but are not visualisable in that form. In order to '''convert these mesh files into viewable images''', we can execute the following command in Matlab:<br />
<br />
VMSreport('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Project','GPT_CASE_RST',...<br />
'Experiment','All','flattentime',572.5,'morphogen','KPAR','SNAPFIG',true);<br />
<br />
Where the ''Path'' and ''Project'' arguments have the same function as ''Path'' and ''Name'' in 3A, i.e. ''Path'' refers to the location of the project folder and ''Project'' is the name of the project folder itself.<br />
<br />
Here we can see the generated images, including the command line arguments, for one of the simulations performed in 3A. Click on a thumbnail to view a larger image.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="125px"| <small>001commandline.png</small><br> [[File:commandlineexample.png | 100px]]<br />
|width="125px"| <small>001GPT_CASE_RST_2 wild_.png</small><br>[[File:GPT_CASE_RST_example1.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_3 wild_.png</small><br>[[File:GPT_CASE_RST_example2.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_4 wild_.png</small><br>[[File:GPT_CASE_RST_example3.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_5 wild_.png</small><br>[[File:GPT_CASE_RST_example4.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_6 wild_.png</small><br>[[File:GPT_CASE_RST_example5.png | 125px]]<br />
|}<br />
<br />
<br />
'''2) - Generating images from a simulation you ran on the cluster'''<br />
<br />
NB. The results produced may not be visibly identical to those in the published literature. This is because of small, random perturbations which are applied to the initial model meshes to stop them from containing surfaces which are perfectly flat, and therefore biologically unrealistic. The results produced should be '''qualitatively but not quantitatively the same'''.<br />
<br />
'''3) - Interacting with your results using the GFtbox GUI'''<br />
<br />
='''4''' Altering the simulation parameters=<br />
<br />
One such simulation parameter is ''dt'', which is the time in seconds between iterations in the growth simulation. Large values of ''dt'' mean that fewer steps and therefore fewer calculations are required to complete a simulation. Whereas smaller values of ''dt'' mean that more steps are required, and therefore more processing time too. Therefore, a value of ''dt'' must be selected which is not so small that it is computationally unmanageable, but not so large that the observed growth is an artifact of the value of ''dt'', rather than the underlying growth model. It is necessary therefore to test a range of values for ''dt'' to ensure that the patterns of growth observed in a simulation are consistent across the range, and to find a value which is sufficient to demonstrate the model of growth and computationally efficient.<br />
<br />
'''Testing a range of ''dt''s can be achieved in several ways.''' One way is to use the ''dt'' argument when calling GFtboxCommand. This allows a single value to be tested. Another way is to make a batch of jobs, each using a different value for ''dt'', by using the 'State' argument. Lastly, a range of ''dt''s can be specified in GFtboxCommand, by using the dt argument where the values within the square brackets are the ''dt''s to simulate:<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5]);<br />
<br />
Here, we have specified '''four separate values for ''dt''''', and this means that '''four separate simulations will be run''' sequentially on your computer. As in 3B, images can then be generated for the mesh files produced, and compared to ensure consistency across the range of ''dt''s. In this figure, we can see the same model, at the same stage of growth, generated using the four values of ''dt''. Though quantitatively different, they are qualitatively the same, illustrating the suitability of the default ''dt'' value of 5.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="190px"| [[File:GPT_CASE_RST_dt0.5.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt1.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt2.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt5.png| 190px]]<br />
|}<br />
<br />
='''5''' Altering the model parameters on the cluster=<br />
It is easy to see that when even a small number of range variables, mutations or ''dt''s are specified, the '''total number of simulations increases quickly'''. If many ranges are specified, then the amount of processing time becomes unmanageable on a single computer. Via ''GFtboxCommand'' and the ''ClusterMonitor'' tool, we can remotely''' run a number of simulations, in parallel, on a computing cluster'''. It is assumed that the GrowthToolbox and Matlab are installed on your cluster, and PuTTY is required on your local computer for the tools ''pscp'' (for transferring files from your computer to a cluster) and ''plink'' (for remotely executing commands on a cluster). Using the ''dt'' range argument from 4 as an example, here we add the 'Use' name argument with the value 'Cluster':<br />
<br />
GFtboxCommand('State','Start','Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5],'Use','Cluster');<br />
<br />
The 'Cluster' option for the 'Use' name argument instructs GFtboxCommand to upload the required project directory to a remote Linux server. Instead of running the simulation on your own computer, GFtboxCommand then works out how many individual simulations are specified by the command that invoked it. In this example, the only argument which will generate multiple simulations is ''dt'', of which there will be 4. Separate commands for each of these jobs are then automatically generated, each with an accompanying unique ID (The value of the ''ExpID'' name argument), and '''these are submitted as individual jobs to the cluster'''.<br />
<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.1,'ExpID','GPT_CASE_RST_1');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.5,'ExpID','GPT_CASE_RST_2');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',1,'ExpID','GPT_CASE_RST_3');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',5,'ExpID','GPT_CASE_RST_4');<br />
<br />
Once these jobs have been submitted, the ''ClusterMonitor'' tool opens and the new job batch ID is added to the list of jobs. Here, we will concentrate on three of the functions of ''ClusterMonitor'': ''Queue?'', ''Get project results'' and ''Make project pngs''. As in part 4 of this page, these functions will enable us to visualise the growth simulation results.<br />
Queue <br> Get project results <br> Make project pngs<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| This is an example of what you will see when the ClusterMonitor tool launches after submitting a job using the 'Use','Cluster' argument pair in GFtboxCommand.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Clicking the 'Queue?' button tells ClusterMonitor to check the status of your jobs on the cluster. It then prints the status of the current jobs in Matlab's command window. In the case of our cluster, this is the equivalent of the qstatme command.<br />
<br />
This allows you too see whether or not your jobs have finished, and therefore whether or not you can move onto the next step.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
<br />
Whilst ''ClusterMonitor'' is a useful tool for managing your cluster jobs, it is advisable to '''have at least a rudimentary understanding of and ability to use a Unix-based computer system'''. In particular, the abilities to list the contents of a folder, view the contents of a file, change the current working folder, delete, and to copy or move files or folders, are essential Unix skills for making sure everything is working as intended. It is beyond the scope of these tutorials to provide an in depth Unix tutorial (many good and simple tutorials exist on the web), but here is a short description of the Unix commands (which may be specific to our Unix-based system) that we believe to be important.<br />
<br />
''ls'' - list the files and folders in a directory. Use the -al option (e.g. ls -al) to see modification dates, file permissions, sizes, etc.<br />
<br />
''cd X'' - Change the current directory to directory X.<br />
<br />
''rm X'' - Delete the file X.<br />
<br />
''cp X NewPath/X2'' - Copy file X to the NewPath directory and name the copy X2.<br />
<br />
''mv X NewPath/X2'' - Move file X to the NewPath directory and rename it to X2.<br />
<br />
''cat test.txt'' - Print the contents of the text file test.txt onto the screen.<br />
<br />
''pwd'' - Print the present working directory onto the screen.<br />
<br />
''more test.txt'' - Print the file test.txt to the screen, a screenful at a time, scrolling each time you hit the enter key.</div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Running_example_models_and_using_a_cluster&diff=5387Running example models and using a cluster2011-06-24T08:22:13Z<p>JacobNewman: </p>
<hr />
<div>[[GFtbox Tutorial pages | Back to GFtbox Tutorial pages]]<br />
<br />
The purpose of these tutorials is to learn how to run the example growth simulations included in GFtbox. We will describe methods for running simulations locally (on your own computer) and remotely (on a computing cluster). It is assumed that you have already downloaded the GFtbox software and have Matlab installed.<br />
<br />
=Getting Started=<br />
<br />
The remainder of this page is split into five sub-tutorials, each building on the preceding parts.<br />
<br />
1) '''Explaining the tools'''. In these tutorials, we will be using ''GFtboxCommand'' and ''ClusterMonitor''. This section explains the purpose of these tools. <br />
<br />
2) '''Computer or cluster?''' Here, we illustrate when and why you should use a computing cluster for your growth simulations, and conversely, when you should use your desktop computer.<br />
<br />
3) '''Running a growth simulation for an example model'''. This section demonstrates how to use ''GFtboxCommand'' to run a growth simulation on your computer. The model used for the simulation is an example model included with the GFtbox.<br />
<br />
4) '''Altering the simulation parameters on the cluster'''. Following on from part 3, here we show how to adjust a simulation parameter within GFtboxCommand. Specifically, we alter the value of ''dt'', the temporal resolution of the simulation, and show how it can be used to verify that the value specified in the published literature is reasonable.<br />
<br />
5) '''Altering the model parameters'''. Finally, we demonstrate how a number of model parameters can be varied by specifying a range of values for each model argument. We show how the computationally expensive task of simulating all combinations of specified ranges can be processed more efficiently if a computer cluster is used via the ''ClusterMonitor'' tool.<br />
<br />
='''1''' Explaining the tools=<br />
==GFtboxCommand==<br />
This is a command line version of the GFtbox. By command line, we mean that all program '''functions are operated via typed commands, without the GUI'''. Like GFtbox, GFtboxCommand is capable of running growth simulations of an interaction function, and allows the user to specify model and simulation parameters. Unlike GFtbox, this also allows the user to select '''ranges of values for a number of input parameters''', and will automatically spawn multiple simulations which explore the various combinations of those parameters. This can be used to evaluate the effect of various parameters on the growth of a given model.<br />
<br />
==ClusterMonitor==<br />
Provides a graphical user interface for managing simulations '''running remotely on a computer cluster'''. Specifically, it allows you to see which projects are present and running on the cluster, to retrieve the completed projects, to generate images of the simulations at specified stages of growth, and to remove projects from the cluster. If you do not intend to use a computer cluster, then you will not need to use ClusterMonitor.<br />
<br />
='''2''' Computer or cluster?=<br />
<br />
In basic terms, '''a computing cluster is effectively a network of many computer processors''' (often hundreds), centrally managed by a queuing system. When a job is submitted to a cluster, the job is sent to a processor that is not being used, or queued until one becomes available. In contrast, a typical desktop computer will contain one processor, limiting the number of tasks that can be performed at any one time. Jobs which must be run independently and sequentially on a desktop computer can be executed in parallel on a cluster, '''greatly reducing the total time required to complete all of the jobs'''. Although the exact details of your cluster might vary from those described in the rest of these tutorials, we aim to illustrate the generic processes involved in using GFtbox on a cluster. We are happy to offer assistance, where possible, to setup GFtbox for your cluster, so please contact us if you have any queries.<br />
<br />
Whilst the time savings offered by using a cluster can be significant, there is an overhead associated with returning the results to your personal computer. The total time to run 2 or more simulations on a cluster and to return the results will be less than running those simulations sequentially on one computer. Therefore, using a '''cluster is ideal for situations where you would like to run several simulations''', such as to evaluate the effect of a range of parameters on a growth model. You are not advised to use a cluster for running single simulations, or where you would like to step through a simulation and change parameters in a more interactive fashion. In such circumstances, you are advised to run GFtbox or GFtboxCommand on your own computer.<br />
<br />
The '''GFtbox GUI provides quick feedback''' about how the changes you have made to an interaction function have affected the growth simulation, as '''you can see the result of each simulation iteration as it completes'''. It is quick and easy therefore to see if you have dramatically changed the course of the growth simulation, and then to adjust the parameters according to your observations. This approach is well suited to the early stages of design, where you might wish to '''tweak some parameters to gauge whether or not they have had the desired effect'''. Or in the final stages, where you think your model is almost finalised, but where small adjustments are required. '''An alternative approach is to start many simulations''', based upon your initial model and by making intelligent choices about which parameters to explore. This allows you to '''harvest many results, to quickly and easily overview them''' in their finalised form, or to select interesting-looking simulations and examine them more closely on your desktop computer, using GFtbox.<br />
<br />
='''3''' Running a growth simulation for an example model=<br />
This tutorial is aimed at running a '''growth simulation for one of the example interaction functions''' included with the GFtbox. The purpose of this exercise is to firstly demonstrate how simulations can be invoked using GFtboxCommand, and secondly to show how to reproduce experimental results (specifically, those published in Kennaway et al (2011)) given an interaction function.<br />
<br />
Assuming Matlab is installed on your computer, and the latest GFtbox has been downloaded, you can add the GFtbox directory to Matlab's search path, which makes the toolbox accessible from any other path that you choose to work from. For a short tutorial on how to do this, please click [[Adding GFtbox to Matlab's search path | here]].<br />
<br />
Once the GFtbox is added to Matlab, you are ready to run a growth simulation using GFtboxCommand. In this example, '''the model that we will simulate is called GPT_CASE_RST'''. Results generated using this model are published in Kennaway et al (2011). By running this simulation, we can confirm the results in the published literature and investigate the suitability of the various parameters.<br />
<br />
===3A - Running a simulation on your computer===<br />
<br />
The following command can be typed into Matlab to run a simulation of the GPT_CASE_RST interaction function, which contains three growth models: R, S and T. '''Three separate simulations are run sequentially on your computer''', one for each model, each producing results corresponding to five intervals in the growth simulation.<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',[1:3]);<br />
<br />
GFtboxCommand accepts input arguments as name and value pairs, e.g. 'modelname', [1:3] or 'Use','Cluster'. The argument names entered in the example above are: ''Path'', ''Name'', ''Stages'' and ''modelname''.<br />
<br />
The optional '''''Path'' argument name refers to the location of the folder''' (or directory) on your computer where the model interaction function you wish to simulate is stored. '''''Name'', is the name of the folder itself'''. In this case, we are using the GPT_CASE_RST folder which is included in the GrowthToolbox. You may wish to copy this folder elsewhere, if you intend to make changes to the interaction function.<br />
<br />
During a growth simulation, a mesh can be generated at each time step of the simulation, which provides a visual representation of the growth of the biological tissue, given the various parameters of growth specified by the interaction function and how they have changed over time. Put another way, the mesh shows exactly what the growing tissue actually looks like. ''Stages'' refers to the points in the simulation (measured in hours) at which meshes should be generated and saved. In this example, '''five stages of growth will be written to disk'''. These values are chosen to best capture the appearance of the tissue at important stages of the tissue growth.<br />
<br />
The final argument name listed here is ''modelname''. This is a model-specific argument, and in this case the GPT_CASE_RST interaction function contains '''three separate models for plant growth''', allowing the desired model to be a selected. Here, the value [1:3] is specified, which is evaluated in the same way as entering [1 2 3]. This instructs GFtboxCommand to run three separate simulations, one for each of the growth models contained in the interaction function.<br />
<br />
The function of every permissible argument is given by keying the following command into Matlab:<br />
<br />
help GFtboxCommand<br />
<br />
===3B - Getting results from a completed simulation===<br />
<br />
'''1) - Generating images from a simulation you ran on your computer'''<br />
<br />
Once a growth simulation is completed, the project folder (GPT_CASE_RST, in our example) will contain another folder named "movies". Within movies, there are folders which contain results for the executed simulations. As was instructed in 3A, here we can see three folders corresponding to simulations for the three separate models. Within the first folder there are three items:<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| ''CommandLine.txt'' - This file contains the Matlab command which was used to generate the results that this sub-directory contains.<br />
<br />
''gpt_case_rst.txt'' - This file, named according to the project name, contains a copy of the interaction function which was used to produce these results.<br />
<br />
''meshes'' - This folder contains the mesh files corresponding to the stages of growth specified by the value of the Stages argument, described in 3A.<br />
|width="500px" align="center"|[[File:resultsdir.png| Directory structure of a project containing results]]<br />
|}<br />
<br />
<br />
The mesh files contain vertices information regarding the shape of the growth model at a particular stage of growth, but are not visualisable in that form. In order to '''convert these mesh files into viewable images''', we can execute the following command in Matlab:<br />
<br />
VMSreport('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Project','GPT_CASE_RST',...<br />
'Experiment','All','flattentime',572.5,'morphogen','KPAR','SNAPFIG',true);<br />
<br />
Where the ''Path'' and ''Project'' arguments have the same function as ''Path'' and ''Name'' in 3A, i.e. ''Path'' refers to the location of the project folder and ''Project'' is the name of the project folder itself.<br />
<br />
Here we can see the generated images, including the command line arguments, for one of the simulations performed in 3A. Click on a thumbnail to view a larger image.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="125px"| <small>001commandline.png</small><br> [[File:commandlineexample.png | 100px]]<br />
|width="125px"| <small>001GPT_CASE_RST_2 wild_.png</small><br>[[File:GPT_CASE_RST_example1.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_3 wild_.png</small><br>[[File:GPT_CASE_RST_example2.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_4 wild_.png</small><br>[[File:GPT_CASE_RST_example3.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_5 wild_.png</small><br>[[File:GPT_CASE_RST_example4.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_6 wild_.png</small><br>[[File:GPT_CASE_RST_example5.png | 125px]]<br />
|}<br />
<br />
<br />
'''2) - Generating images from a simulation you ran on the cluster'''<br />
<br />
NB. The results produced may not be visibly identical to those in the published literature. This is because of small, random perturbations which are applied to the initial model meshes to stop them from containing surfaces which are perfectly flat, and therefore biologically unrealistic. The results produced should be '''qualitatively but not quantitatively the same'''.<br />
<br />
'''3) - Interacting with your results using the GFtbox GUI'''<br />
<br />
='''4''' Altering the simulation parameters=<br />
<br />
One such simulation parameter is ''dt'', which is the time in seconds between iterations in the growth simulation. Large values of ''dt'' mean that fewer steps and therefore fewer calculations are required to complete a simulation. Whereas smaller values of ''dt'' mean that more steps are required, and therefore more processing time too. Therefore, a value of ''dt'' must be selected which is not so small that it is computationally unmanageable, but not so large that the observed growth is an artifact of the value of ''dt'', rather than the underlying growth model. It is necessary therefore to test a range of values for ''dt'' to ensure that the patterns of growth observed in a simulation are consistent across the range, and to find a value which is sufficient to demonstrate the model of growth and computationally efficient.<br />
<br />
'''Testing a range of ''dt''s can be achieved in several ways.''' One way is to use the ''dt'' argument when calling GFtboxCommand. This allows a single value to be tested. Another way is to make a batch of jobs, each using a different value for ''dt'', by using the 'State' argument. Lastly, a range of ''dt''s can be specified in GFtboxCommand, by using the dt argument where the values within the square brackets are the ''dt''s to simulate:<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5]);<br />
<br />
Here, we have specified '''four separate values for ''dt''''', and this means that '''four separate simulations will be run''' sequentially on your computer. As in 3B, images can then be generated for the mesh files produced, and compared to ensure consistency across the range of ''dt''s. In this figure, we can see the same model, at the same stage of growth, generated using the four values of ''dt''. Though quantitatively different, they are qualitatively the same, illustrating the suitability of the default ''dt'' value of 5.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="190px"| [[File:GPT_CASE_RST_dt0.5.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt1.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt2.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt5.png| 190px]]<br />
|}<br />
<br />
='''5''' Altering the model parameters on the cluster=<br />
It is easy to see that when even a small number of range variables, mutations or ''dt''s are specified, the '''total number of simulations increases quickly'''. If many ranges are specified, then the amount of processing time becomes unmanageable on a single computer. Via ''GFtboxCommand'' and the ''ClusterMonitor'' tool, we can remotely''' run a number of simulations, in parallel, on a computing cluster'''. It is assumed that the GrowthToolbox and Matlab are installed on your cluster, and PuTTY is required on your local computer for the tools ''pscp'' (for transferring files from your computer to a cluster) and ''plink'' (for remotely executing commands on a cluster). Using the ''dt'' range argument from 4 as an example, here we add the 'Use' name argument with the value 'Cluster':<br />
<br />
GFtboxCommand('State','Start','Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5],'Use','Cluster');<br />
<br />
The 'Cluster' option for the 'Use' name argument instructs GFtboxCommand to upload the required project directory to a remote Linux server. Instead of running the simulation on your own computer, GFtboxCommand then works out how many individual simulations are specified by the command that invoked it. In this example, the only argument which will generate multiple simulations is ''dt'', of which there will be 4. Separate commands for each of these jobs are then automatically generated, each with an accompanying unique ID (The value of the ''ExpID'' name argument), and '''these are submitted as individual jobs to the cluster'''.<br />
<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.1,'ExpID','GPT_CASE_RST_1');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.5,'ExpID','GPT_CASE_RST_2');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',1,'ExpID','GPT_CASE_RST_3');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',5,'ExpID','GPT_CASE_RST_4');<br />
<br />
Once these jobs have been submitted, the ''ClusterMonitor'' tool opens and the new job batch ID is added to the list of jobs. Here, we will concentrate on three of the functions of ''ClusterMonitor'': ''Queue?'', ''Get project results'' and ''Make project pngs''. As in part 4 of this page, these functions will enable us to visualise the growth simulation results.<br />
Queue <br> Get project results <br> Make project pngs<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| This is an example of what you will see when the ClusterMonitor tool launches after submitting a job using the 'Use','Cluster' argument pair in GFtboxCommand.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Clicking the 'Queue?' button tells ClusterMonitor to check the status of your jobs on the cluster. It then prints the status of the current jobs in Matlab's command window. In the case of our cluster, this is the equivalent of the qstatme command.<br />
<br />
This allows you too see whether or not your jobs have finished, and therefore whether or not you can move onto the next step.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
<br />
Whilst ''ClusterMonitor'' is a useful tool for managing your cluster jobs, it is advisable to '''have at least a rudimentary understanding of and ability to use a Unix-based computer system'''. In particular, the abilities to list the contents of a folder, view the contents of a file, change the current working folder, delete, and to copy or move files or folders, are essential Unix skills for making sure everything is working as intended. It is beyond the scope of these tutorials to provide an in depth Unix tutorial (many good and simple tutorials exist on the web), but here is a short description of the Unix commands (which may be specific to our Unix-based system) that we believe to be important.<br />
<br />
''ls'' - list the files and folders in a directory. Use the -al option (e.g. ls -al) to see modification dates, file permissions, sizes, etc.<br />
<br />
''cd X'' - Change the current directory to directory X.<br />
<br />
''rm X'' - Delete the file X.<br />
<br />
''cp X NewPath/X2'' - Copy file X to the NewPath directory and name the copy X2.<br />
<br />
''mv X NewPath/X2'' - Move file X to the NewPath directory and rename it to X2.<br />
<br />
''cat test.txt'' - Print the contents of the text file test.txt onto the screen.<br />
<br />
''pwd'' - Print the present working directory onto the screen.<br />
<br />
''more test.txt'' - Print the file test.txt to the screen, a screenful at a time, scrolling each time you hit the enter key.</div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Running_example_models_and_using_a_cluster&diff=5413Running example models and using a cluster2011-06-24T08:20:53Z<p>JacobNewman: </p>
<hr />
<div>[[GFtbox Tutorial pages | Back to GFtbox Tutorial pages]]<br />
<br />
The purpose of these tutorials is to learn how to run the example growth simulations included in GFtbox. We will describe methods for running simulations locally (on your own computer) and remotely (on a computing cluster). It is assumed that you have already downloaded the GFtbox software and have Matlab installed.<br />
<br />
=Getting Started=<br />
<br />
The remainder of this page is split into five sub-tutorials, each building on the preceding parts.<br />
<br />
1) '''Explaining the tools'''. In these tutorials, we will be using ''GFtboxCommand'' and ''ClusterMonitor''. This section explains the purpose of these tools. <br />
<br />
2) '''Computer or cluster?''' Here, we illustrate when and why you should use a computing cluster for your growth simulations, and conversely, when you should use your desktop computer.<br />
<br />
3) '''Running a growth simulation for an example model'''. This section demonstrates how to use ''GFtboxCommand'' to run a growth simulation on your computer. The model used for the simulation is an example model included with the GFtbox.<br />
<br />
4) '''Altering the simulation parameters'''. Following on from part 3, here we show how to adjust a simulation parameter within GFtboxCommand. Specifically, we alter the value of ''dt'', the temporal resolution of the simulation, and show how it can be used to verify that the value specified in the published literature is reasonable.<br />
<br />
5) '''Altering the model parameters'''. Finally, we demonstrate how a number of model parameters can be varied by specifying a range of values for each model argument. We show how the computationally expensive task of simulating all combinations of specified ranges can be processed more efficiently if a computer cluster is used via the ''ClusterMonitor'' tool.<br />
<br />
='''1''' Explaining the tools=<br />
==GFtboxCommand==<br />
This is a command line version of the GFtbox. By command line, we mean that all program '''functions are operated via typed commands, without the GUI'''. Like GFtbox, GFtboxCommand is capable of running growth simulations of an interaction function, and allows the user to specify model and simulation parameters. Unlike GFtbox, this also allows the user to select '''ranges of values for a number of input parameters''', and will automatically spawn multiple simulations which explore the various combinations of those parameters. This can be used to evaluate the effect of various parameters on the growth of a given model.<br />
<br />
==ClusterMonitor==<br />
Provides a graphical user interface for managing simulations '''running remotely on a computer cluster'''. Specifically, it allows you to see which projects are present and running on the cluster, to retrieve the completed projects, to generate images of the simulations at specified stages of growth, and to remove projects from the cluster. If you do not intend to use a computer cluster, then you will not need to use ClusterMonitor.<br />
<br />
='''2''' Computer or cluster?=<br />
<br />
In basic terms, '''a computing cluster is effectively a network of many computer processors''' (often hundreds), centrally managed by a queuing system. When a job is submitted to a cluster, the job is sent to a processor that is not being used, or queued until one becomes available. In contrast, a typical desktop computer will contain one processor, limiting the number of tasks that can be performed at any one time. Jobs which must be run independently and sequentially on a desktop computer can be executed in parallel on a cluster, '''greatly reducing the total time required to complete all of the jobs'''. Although the exact details of your cluster might vary from those described in the rest of these tutorials, we aim to illustrate the generic processes involved in using GFtbox on a cluster. We are happy to offer assistance, where possible, to setup GFtbox for your cluster, so please contact us if you have any queries.<br />
<br />
Whilst the time savings offered by using a cluster can be significant, there is an overhead associated with returning the results to your personal computer. The total time to run 2 or more simulations on a cluster and to return the results will be less than running those simulations sequentially on one computer. Therefore, using a '''cluster is ideal for situations where you would like to run several simulations''', such as to evaluate the effect of a range of parameters on a growth model. You are not advised to use a cluster for running single simulations, or where you would like to step through a simulation and change parameters in a more interactive fashion. In such circumstances, you are advised to run GFtbox or GFtboxCommand on your own computer.<br />
<br />
The '''GFtbox GUI provides quick feedback''' about how the changes you have made to an interaction function have affected the growth simulation, as '''you can see the result of each simulation iteration as it completes'''. It is quick and easy therefore to see if you have dramatically changed the course of the growth simulation, and then to adjust the parameters according to your observations. This approach is well suited to the early stages of design, where you might wish to '''tweak some parameters to gauge whether or not they have had the desired effect'''. Or in the final stages, where you think your model is almost finalised, but where small adjustments are required. '''An alternative approach is to start many simulations''', based upon your initial model and by making intelligent choices about which parameters to explore. This allows you to '''harvest many results, to quickly and easily overview them''' in their finalised form, or to select interesting-looking simulations and examine them more closely on your desktop computer, using GFtbox.<br />
<br />
='''3''' Running a growth simulation for an example model=<br />
This tutorial is aimed at running a '''growth simulation for one of the example interaction functions''' included with the GFtbox. The purpose of this exercise is to firstly demonstrate how simulations can be invoked using GFtboxCommand, and secondly to show how to reproduce experimental results (specifically, those published in Kennaway et al (2011)) given an interaction function.<br />
<br />
Assuming Matlab is installed on your computer, and the latest GFtbox has been downloaded, you can add the GFtbox directory to Matlab's search path, which makes the toolbox accessible from any other path that you choose to work from. For a short tutorial on how to do this, please click [[Adding GFtbox to Matlab's search path | here]].<br />
<br />
Once the GFtbox is added to Matlab, you are ready to run a growth simulation using GFtboxCommand. In this example, '''the model that we will simulate is called GPT_CASE_RST'''. Results generated using this model are published in Kennaway et al (2011). By running this simulation, we can confirm the results in the published literature and investigate the suitability of the various parameters.<br />
<br />
===3A - Running a simulation on your computer===<br />
<br />
The following command can be typed into Matlab to run a simulation of the GPT_CASE_RST interaction function, which contains three growth models: R, S and T. '''Three separate simulations are run sequentially on your computer''', one for each model, each producing results corresponding to five intervals in the growth simulation.<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',[1:3]);<br />
<br />
GFtboxCommand accepts input arguments as name and value pairs, e.g. 'modelname', [1:3] or 'Use','Cluster'. The argument names entered in the example above are: ''Path'', ''Name'', ''Stages'' and ''modelname''.<br />
<br />
The optional '''''Path'' argument name refers to the location of the folder''' (or directory) on your computer where the model interaction function you wish to simulate is stored. '''''Name'', is the name of the folder itself'''. In this case, we are using the GPT_CASE_RST folder which is included in the GrowthToolbox. You may wish to copy this folder elsewhere, if you intend to make changes to the interaction function.<br />
<br />
During a growth simulation, a mesh can be generated at each time step of the simulation, which provides a visual representation of the growth of the biological tissue, given the various parameters of growth specified by the interaction function and how they have changed over time. Put another way, the mesh shows exactly what the growing tissue actually looks like. ''Stages'' refers to the points in the simulation (measured in hours) at which meshes should be generated and saved. In this example, '''five stages of growth will be written to disk'''. These values are chosen to best capture the appearance of the tissue at important stages of the tissue growth.<br />
<br />
The final argument name listed here is ''modelname''. This is a model-specific argument, and in this case the GPT_CASE_RST interaction function contains '''three separate models for plant growth''', allowing the desired model to be a selected. Here, the value [1:3] is specified, which is evaluated in the same way as entering [1 2 3]. This instructs GFtboxCommand to run three separate simulations, one for each of the growth models contained in the interaction function.<br />
<br />
The function of every permissible argument is given by keying the following command into Matlab:<br />
<br />
help GFtboxCommand<br />
<br />
===3B - Getting results from a completed simulation===<br />
<br />
'''1) - Generating images from a simulation you ran on your computer'''<br />
<br />
Once a growth simulation is completed, the project folder (GPT_CASE_RST, in our example) will contain another folder named "movies". Within movies, there are folders which contain results for the executed simulations. As was instructed in 3A, here we can see three folders corresponding to simulations for the three separate models. Within the first folder there are three items:<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| ''CommandLine.txt'' - This file contains the Matlab command which was used to generate the results that this sub-directory contains.<br />
<br />
''gpt_case_rst.txt'' - This file, named according to the project name, contains a copy of the interaction function which was used to produce these results.<br />
<br />
''meshes'' - This folder contains the mesh files corresponding to the stages of growth specified by the value of the Stages argument, described in 3A.<br />
|width="500px" align="center"|[[File:resultsdir.png| Directory structure of a project containing results]]<br />
|}<br />
<br />
<br />
The mesh files contain vertices information regarding the shape of the growth model at a particular stage of growth, but are not visualisable in that form. In order to '''convert these mesh files into viewable images''', we can execute the following command in Matlab:<br />
<br />
VMSreport('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Project','GPT_CASE_RST',...<br />
'Experiment','All','flattentime',572.5,'morphogen','KPAR','SNAPFIG',true);<br />
<br />
Where the ''Path'' and ''Project'' arguments have the same function as ''Path'' and ''Name'' in 3A, i.e. ''Path'' refers to the location of the project folder and ''Project'' is the name of the project folder itself.<br />
<br />
Here we can see the generated images, including the command line arguments, for one of the simulations performed in 3A. Click on a thumbnail to view a larger image.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="125px"| <small>001commandline.png</small><br> [[File:commandlineexample.png | 100px]]<br />
|width="125px"| <small>001GPT_CASE_RST_2 wild_.png</small><br>[[File:GPT_CASE_RST_example1.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_3 wild_.png</small><br>[[File:GPT_CASE_RST_example2.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_4 wild_.png</small><br>[[File:GPT_CASE_RST_example3.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_5 wild_.png</small><br>[[File:GPT_CASE_RST_example4.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_6 wild_.png</small><br>[[File:GPT_CASE_RST_example5.png | 125px]]<br />
|}<br />
<br />
<br />
'''2) - Generating images from a simulation you ran on the cluster'''<br />
<br />
NB. The results produced may not be visibly identical to those in the published literature. This is because of small, random perturbations which are applied to the initial model meshes to stop them from containing surfaces which are perfectly flat, and therefore biologically unrealistic. The results produced should be '''qualitatively but not quantitatively the same'''.<br />
<br />
'''3) - Interacting with your results using the GFtbox GUI'''<br />
<br />
='''4''' Altering the simulation parameters=<br />
<br />
One such simulation parameter is ''dt'', which is the time in seconds between iterations in the growth simulation. Large values of ''dt'' mean that fewer steps and therefore fewer calculations are required to complete a simulation. Whereas smaller values of ''dt'' mean that more steps are required, and therefore more processing time too. Therefore, a value of ''dt'' must be selected which is not so small that it is computationally unmanageable, but not so large that the observed growth is an artifact of the value of ''dt'', rather than the underlying growth model. It is necessary therefore to test a range of values for ''dt'' to ensure that the patterns of growth observed in a simulation are consistent across the range, and to find a value which is sufficient to demonstrate the model of growth and computationally efficient.<br />
<br />
'''Testing a range of ''dt''s can be achieved in several ways.''' One way is to use the ''dt'' argument when calling GFtboxCommand. This allows a single value to be tested. Another way is to make a batch of jobs, each using a different value for ''dt'', by using the 'State' argument. Lastly, a range of ''dt''s can be specified in GFtboxCommand, by using the dt argument where the values within the square brackets are the ''dt''s to simulate:<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5]);<br />
<br />
Here, we have specified '''four separate values for ''dt''''', and this means that '''four separate simulations will be run''' sequentially on your computer. As in 3B, images can then be generated for the mesh files produced, and compared to ensure consistency across the range of ''dt''s. In this figure, we can see the same model, at the same stage of growth, generated using the four values of ''dt''. Though quantitatively different, they are qualitatively the same, illustrating the suitability of the default ''dt'' value of 5.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="190px"| [[File:GPT_CASE_RST_dt0.5.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt1.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt2.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt5.png| 190px]]<br />
|}<br />
<br />
='''5''' Altering the model parameters=<br />
It is easy to see that when even a small number of range variables, mutations or ''dt''s are specified, the '''total number of simulations increases quickly'''. If many ranges are specified, then the amount of processing time becomes unmanageable on a single computer. Via ''GFtboxCommand'' and the ''ClusterMonitor'' tool, we can remotely''' run a number of simulations, in parallel, on a computing cluster'''. It is assumed that the GrowthToolbox and Matlab are installed on your cluster, and PuTTY is required on your local computer for the tools ''pscp'' (for transferring files from your computer to a cluster) and ''plink'' (for remotely executing commands on a cluster). Using the ''dt'' range argument from 4 as an example, here we add the 'Use' name argument with the value 'Cluster':<br />
<br />
GFtboxCommand('State','Start','Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5],'Use','Cluster');<br />
<br />
The 'Cluster' option for the 'Use' name argument instructs GFtboxCommand to upload the required project directory to a remote Linux server. Instead of running the simulation on your own computer, GFtboxCommand then works out how many individual simulations are specified by the command that invoked it. In this example, the only argument which will generate multiple simulations is ''dt'', of which there will be 4. Separate commands for each of these jobs are then automatically generated, each with an accompanying unique ID (The value of the ''ExpID'' name argument), and '''these are submitted as individual jobs to the cluster'''.<br />
<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.1,'ExpID','GPT_CASE_RST_1');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.5,'ExpID','GPT_CASE_RST_2');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',1,'ExpID','GPT_CASE_RST_3');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',5,'ExpID','GPT_CASE_RST_4');<br />
<br />
Once these jobs have been submitted, the ''ClusterMonitor'' tool opens and the new job batch ID is added to the list of jobs. Here, we will concentrate on three of the functions of ''ClusterMonitor'': ''Queue?'', ''Get project results'' and ''Make project pngs''. As in part 4 of this page, these functions will enable us to visualise the growth simulation results.<br />
Queue <br> Get project results <br> Make project pngs<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| This is an example of what you will see when the ClusterMonitor tool launches after submitting a job using the 'Use','Cluster' argument pair in GFtboxCommand.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Clicking the 'Queue?' button tells ClusterMonitor to check the status of your jobs on the cluster. It then prints the status of the current jobs in Matlab's command window. In the case of our cluster, this is the equivalent of the qstatme command.<br />
<br />
This allows you too see whether or not your jobs have finished, and therefore whether or not you can move onto the next step.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
<br />
Whilst ''ClusterMonitor'' is a useful tool for managing your cluster jobs, it is advisable to '''have at least a rudimentary understanding of and ability to use a Unix-based computer system'''. In particular, the abilities to list the contents of a folder, view the contents of a file, change the current working folder, delete, and to copy or move files or folders, are essential Unix skills for making sure everything is working as intended. It is beyond the scope of these tutorials to provide an in depth Unix tutorial (many good and simple tutorials exist on the web), but here is a short description of the Unix commands (which may be specific to our Unix-based system) that we believe to be important.<br />
<br />
''ls'' - list the files and folders in a directory. Use the -al option (e.g. ls -al) to see modification dates, file permissions, sizes, etc.<br />
<br />
''cd X'' - Change the current directory to directory X.<br />
<br />
''rm X'' - Delete the file X.<br />
<br />
''cp X NewPath/X2'' - Copy file X to the NewPath directory and name the copy X2.<br />
<br />
''mv X NewPath/X2'' - Move file X to the NewPath directory and rename it to X2.<br />
<br />
''cat test.txt'' - Print the contents of the text file test.txt onto the screen.<br />
<br />
''pwd'' - Print the present working directory onto the screen.<br />
<br />
''more test.txt'' - Print the file test.txt to the screen, a screenful at a time, scrolling each time you hit the enter key.</div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Running_example_models_and_using_a_cluster&diff=5385Running example models and using a cluster2011-06-24T08:17:32Z<p>JacobNewman: </p>
<hr />
<div>[[GFtbox Tutorial pages | Back to GFtbox Tutorial pages]]<br />
<br />
The purpose of these tutorials is to learn how to run the example growth simulations included in GFtbox. We will describe methods for running simulations locally (on your own computer) and remotely (on a computing cluster). It is assumed that you have already downloaded the GFtbox software and have Matlab installed.<br />
<br />
=Getting Started=<br />
<br />
The remainder of this page is split into five sub-tutorials, each building on the preceding parts.<br />
<br />
1) '''Explaining the tools'''. In these tutorials, we will be using ''GFtboxCommand'' and ''ClusterMonitor''. This section explains the purpose of these tools. <br />
<br />
2) '''Computer or cluster?''' Here, we illustrate when and why you should use a computing cluster for your growth simulations, and conversely, when you should use your desktop computer.<br />
<br />
3) '''Running a growth simulation for an example model'''. This section demonstrates how to use ''GFtboxCommand'' to run a growth simulation on your computer. The model used for the simulation is an example model included with the GFtbox.<br />
<br />
4) '''Altering the simulation parameters'''. Following on from part 3, here we show how to adjust a simulation parameter within GFtboxCommand. Specifically, we alter the value of ''dt'', the temporal resolution of the simulation, and show how it can be used to verify that the value specified in the published literature is reasonable.<br />
<br />
5) '''Altering the model parameters'''. Finally, we demonstrate how a number of model parameters can be varied by specifying a range of values for each model argument. We show how the computationally expensive task of simulating all combinations of specified ranges can be processed more efficiently if a computer cluster is used via the ''ClusterMonitor'' tool.<br />
<br />
='''1''' Explaining the tools=<br />
==GFtboxCommand==<br />
This is a command line version of the GFtbox. By command line, we mean that all program '''functions are operated via typed commands, without the GUI'''. Like GFtbox, GFtboxCommand is capable of running growth simulations of an interaction function, and allows the user to specify model and simulation parameters. Unlike GFtbox, this also allows the user to select '''ranges of values for a number of input parameters''', and will automatically spawn multiple simulations which explore the various combinations of those parameters. This can be used to evaluate the effect of various parameters on the growth of a given model.<br />
<br />
==ClusterMonitor==<br />
Provides a graphical user interface for managing simulations '''running remotely on a computer cluster'''. Specifically, it allows you to see which projects are present and running on the cluster, to retrieve the completed projects, to generate images of the simulations at specified stages of growth, and to remove projects from the cluster. If you do not intend to use a computer cluster, then you will not need to use ClusterMonitor.<br />
<br />
='''2''' Computer or cluster?=<br />
<br />
In basic terms, '''a computing cluster is effectively a network of many computer processors''' (often hundreds), centrally managed by a queuing system. When a job is submitted to a cluster, the job is sent to a processor that is not being used, or queued until one becomes available. In contrast, a typical desktop computer will contain one processor, limiting the number of tasks that can be performed at any one time. Jobs which must be run independently and sequentially on a desktop computer can be executed in parallel on a cluster, '''greatly reducing the total time required to complete all of the jobs'''. Although the exact details of your cluster might vary from those described in the rest of these tutorials, we aim to illustrate the generic processes involved in using GFtbox on a cluster. We are happy to offer assistance, where possible, to setup GFtbox for your cluster, so please contact us if you have any queries.<br />
<br />
Whilst the time savings offered by using a cluster can be significant, there is an overhead associated with returning the results to your personal computer. The total time to run 2 or more simulations on a cluster and to return the results will be less than running those simulations sequentially on one computer. Therefore, using a '''cluster is ideal for situations where you would like to run several simulations''', such as to evaluate the effect of a range of parameters on a growth model. You are not advised to use a cluster for running single simulations, or where you would like to step through a simulation and change parameters in a more interactive fashion. In such circumstances, you are advised to run GFtbox or GFtboxCommand on your own computer.<br />
<br />
The '''GFtbox GUI provides quick feedback''' about how the changes you have made to an interaction function have affected the growth simulation, as '''you can see the result of each simulation iteration as it completes'''. It is quick and easy therefore to see if you have dramatically changed the course of the growth simulation, and then to adjust the parameters according to your observations. This approach is well suited to the early stages of design, where you might wish to '''tweak some parameters to gauge whether or not they have had the desired effect'''. Or in the final stages, where you think your model is almost finalised, but where small adjustments are required. '''An alternative approach is to start many simulations''', based upon your initial model and by making intelligent choices about which parameters to explore. This allows you to '''harvest many results, to quickly and easily overview them''' in their finalised form, or to select interesting-looking simulations and examine them more closely on your desktop computer, using GFtbox.<br />
<br />
='''3''' Running a growth simulation for an example model=<br />
This tutorial is aimed at running a '''growth simulation for one of the example interaction functions''' included with the GFtbox. The purpose of this exercise is to firstly demonstrate how simulations can be invoked using GFtboxCommand, and secondly to show how to reproduce experimental results (specifically, those published in Kennaway et al (2011)) given an interaction function.<br />
<br />
Assuming Matlab is installed on your computer, and the latest GFtbox has been downloaded, you can add the GFtbox directory to Matlab's search path, which makes the toolbox accessible from any other path that you choose to work from. For a short tutorial on how to do this, please click [[Adding GFtbox to Matlab's search path | here]].<br />
<br />
Once the GFtbox is added to Matlab, you are ready to run a growth simulation using GFtboxCommand. In this example, '''the model that we will simulate is called GPT_CASE_RST'''. Results generated using this model are published in Kennaway et al (2011). By running this simulation, we can confirm the results in the published literature and investigate the suitability of the various parameters.<br />
<br />
===3A - Running a simulation on your computer===<br />
<br />
The following command can be typed into Matlab to run a simulation of the GPT_CASE_RST interaction function, which contains three growth models: R, S and T. '''Three separate simulations are run sequentially on your computer''', one for each model, each producing results corresponding to five intervals in the growth simulation.<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',[1:3]);<br />
<br />
GFtboxCommand accepts input arguments as name and value pairs, e.g. 'modelname', [1:3] or 'Use','Cluster'. The argument names entered in the example above are: ''Path'', ''Name'', ''Stages'' and ''modelname''.<br />
<br />
The optional '''''Path'' argument name refers to the location of the folder''' (or directory) on your computer where the model interaction function you wish to simulate is stored. '''''Name'', is the name of the folder itself'''. In this case, we are using the GPT_CASE_RST folder which is included in the GrowthToolbox. You may wish to copy this folder elsewhere, if you intend to make changes to the interaction function.<br />
<br />
During a growth simulation, a mesh can be generated at each time step of the simulation, which provides a visual representation of the growth of the biological tissue, given the various parameters of growth specified by the interaction function and how they have changed over time. Put another way, the mesh shows exactly what the growing tissue actually looks like. ''Stages'' refers to the points in the simulation (measured in hours) at which meshes should be generated and saved. In this example, '''five stages of growth will be written to disk'''. These values are chosen to best capture the appearance of the tissue at important stages of the tissue growth.<br />
<br />
The final argument name listed here is ''modelname''. This is a model-specific argument, and in this case the GPT_CASE_RST interaction function contains '''three separate models for plant growth''', allowing the desired model to be a selected. Here, the value [1:3] is specified, which is evaluated in the same way as entering [1 2 3]. This instructs GFtboxCommand to run three separate simulations, one for each of the growth models contained in the interaction function.<br />
<br />
The function of every permissible argument is given by keying the following command into Matlab:<br />
<br />
help GFtboxCommand<br />
<br />
===3B - Getting results from a completed simulation===<br />
<br />
'''1) - Generating images from a simulation you ran on your computer'''<br />
<br />
Once a growth simulation is completed, the project folder (GPT_CASE_RST, in our example) will contain another folder named "movies". Within movies, there are folders which contain results for the executed simulations. As was instructed in 3A, here we can see three folders corresponding to simulations for the three separate models. Within the first folder there are three items:<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| ''CommandLine.txt'' - This file contains the Matlab command which was used to generate the results that this sub-directory contains.<br />
<br />
''gpt_case_rst.txt'' - This file, named according to the project name, contains a copy of the interaction function which was used to produce these results.<br />
<br />
''meshes'' - This folder contains the mesh files corresponding to the stages of growth specified by the value of the Stages argument, described in 3A.<br />
|width="500px" align="center"|[[File:resultsdir.png| Directory structure of a project containing results]]<br />
|}<br />
<br />
<br />
The mesh files contain vertices information regarding the shape of the growth model at a particular stage of growth, but are not visualisable in that form. In order to '''convert these mesh files into viewable images''', we can execute the following command in Matlab:<br />
<br />
VMSreport('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Project','GPT_CASE_RST',...<br />
'Experiment','All','flattentime',572.5,'morphogen','KPAR','SNAPFIG',true);<br />
<br />
Where the ''Path'' and ''Project'' arguments have the same function as ''Path'' and ''Name'' in 3A, i.e. ''Path'' refers to the location of the project folder and ''Project'' is the name of the project folder itself.<br />
<br />
Here we can see the generated images, including the command line arguments, for one of the simulations performed in 3A. Click on a thumbnail to view a larger image.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="125px"| <small>001commandline.png</small><br> [[File:commandlineexample.png | 100px]]<br />
|width="125px"| <small>001GPT_CASE_RST_2 wild_.png</small><br>[[File:GPT_CASE_RST_example1.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_3 wild_.png</small><br>[[File:GPT_CASE_RST_example2.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_4 wild_.png</small><br>[[File:GPT_CASE_RST_example3.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_5 wild_.png</small><br>[[File:GPT_CASE_RST_example4.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_6 wild_.png</small><br>[[File:GPT_CASE_RST_example5.png | 125px]]<br />
|}<br />
<br />
<br />
'''2) - Generating images from a simulation you ran on the cluster'''<br />
<br />
NB. The results produced may not be visibly identical to those in the published literature. This is because of small, random perturbations which are applied to the initial model meshes to stop them from containing surfaces which are perfectly flat, and therefore biologically unrealistic. The results produced should be '''qualitatively but not quantitatively the same'''.<br />
<br />
'''3) - Interacting with your results using the GFtbox GUI'''<br />
<br />
='''4''' Altering the simulation parameters=<br />
<br />
One such simulation parameter is ''dt'', which is the time in seconds between iterations in the growth simulation. Large values of ''dt'' mean that fewer steps and therefore fewer calculations are required to complete a simulation. Whereas smaller values of ''dt'' mean that more steps are required, and therefore more processing time too. Therefore, a value of ''dt'' must be selected which is not so small that it is computationally unmanageable, but not so large that the observed growth is an artifact of the value of ''dt'', rather than the underlying growth model. It is necessary therefore to test a range of values for ''dt'' to ensure that the patterns of growth observed in a simulation are consistent across the range, and to find a value which is sufficient to demonstrate the model of growth and computationally efficient.<br />
<br />
'''Testing a range of ''dt''s can be achieved in several ways.''' One way is to use the ''dt'' argument when calling GFtboxCommand. This allows a single value to be tested. Another way is to make a batch of jobs, each using a different value for ''dt'', by using the 'State' argument. Lastly, a range of ''dt''s can be specified in GFtboxCommand, by using the dt argument where the values within the square brackets are the ''dt''s to simulate:<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5]);<br />
<br />
Here, we have specified '''four separate values for ''dt''''', and this means that '''four separate simulations will be run''' sequentially on your computer. As in 3B, images can then be generated for the mesh files produced, and compared to ensure consistency across the range of ''dt''s. In this figure, we can see the same model, at the same stage of growth, generated using the four values of ''dt''. Though quantitatively different, they are qualitatively the same, illustrating the suitability of the default ''dt'' value of 5.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="190px"| [[File:GPT_CASE_RST_dt0.5.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt1.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt2.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt5.png| 190px]]<br />
|}<br />
<br />
='''5''' Altering the model parameters=<br />
It is easy to see that when even a small number of range variables, mutations or ''dt''s are specified, the '''total number of simulations increases quickly'''. If many ranges are specified, then the amount of processing time becomes unmanageable on a single computer. Via ''GFtboxCommand'' and the ''ClusterMonitor'' tool, we can remotely''' run a number of simulations, in parallel, on a computing cluster'''. It is assumed that the GrowthToolbox and Matlab are installed on your cluster, and PuTTY is required on your local computer for the tools ''pscp'' (for transferring files from your computer to a cluster) and ''plink'' (for remotely executing commands on a cluster). Using the ''dt'' range argument from 4 as an example, here we add the 'Use' name argument with the value 'Cluster':<br />
<br />
GFtboxCommand('State','Start','Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5],'Use','Cluster');<br />
<br />
The 'Cluster' option for the 'Use' name argument instructs GFtboxCommand to upload the required project directory to a remote Linux server. Instead of running the simulation on your own computer, GFtboxCommand then works out how many individual simulations are specified by the command that invoked it. In this example, the only argument which will generate multiple simulations is ''dt'', of which there will be 4. Separate commands for each of these jobs are then automatically generated, each with an accompanying unique ID (The value of the ''ExpID'' name argument), and '''these are submitted as individual jobs to the cluster'''.<br />
<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.1,'ExpID','GPT_CASE_RST_1');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.5,'ExpID','GPT_CASE_RST_2');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',1,'ExpID','GPT_CASE_RST_3');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',5,'ExpID','GPT_CASE_RST_4');<br />
<br />
Once these jobs have been submitted, the ''ClusterMonitor'' tool opens and the new job batch ID is added to the list of jobs. Here, we will concentrate on three of the functions of ''ClusterMonitor'': ''Queue?'', ''Get project results'' and ''Make project pngs''. As in part 4 of this page, these functions will enable us to visualise the growth simulation results.<br />
Queue <br> Get project results <br> Make project pngs<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| This is an example of what you will see when the ClusterMonitor tool launches after submitting a job using the 'Use','Cluster' argument pair in GFtboxCommand.<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
<br />
Whilst ''ClusterMonitor'' is a useful tool for managing your cluster jobs, it is advisable to '''have at least a rudimentary understanding of and ability to use a Unix-based computer system'''. In particular, the abilities to list the contents of a folder, view the contents of a file, change the current working folder, delete, and to copy or move files or folders, are essential Unix skills for making sure everything is working as intended. It is beyond the scope of these tutorials to provide an in depth Unix tutorial (many good and simple tutorials exist on the web), but here is a short description of the Unix commands (which may be specific to our Unix-based system) that we believe to be important.<br />
<br />
''ls'' - list the files and folders in a directory. Use the -al option (e.g. ls -al) to see modification dates, file permissions, sizes, etc.<br />
<br />
''cd X'' - Change the current directory to directory X.<br />
<br />
''rm X'' - Delete the file X.<br />
<br />
''cp X NewPath/X2'' - Copy file X to the NewPath directory and name the copy X2.<br />
<br />
''mv X NewPath/X2'' - Move file X to the NewPath directory and rename it to X2.<br />
<br />
''cat test.txt'' - Print the contents of the text file test.txt onto the screen.<br />
<br />
''pwd'' - Print the present working directory onto the screen.<br />
<br />
''more test.txt'' - Print the file test.txt to the screen, a screenful at a time, scrolling each time you hit the enter key.</div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Running_example_models_and_using_a_cluster&diff=5382Running example models and using a cluster2011-06-23T14:44:27Z<p>JacobNewman: </p>
<hr />
<div>[[GFtbox Tutorial pages | Back to GFtbox Tutorial pages]]<br />
<br />
The purpose of these tutorials is to learn how to run the example growth simulations included in GFtbox. We will describe methods for running simulations locally (on your own computer) and remotely (on a computing cluster). It is assumed that you have already downloaded the GFtbox software and have Matlab installed.<br />
<br />
=Getting Started=<br />
<br />
The remainder of this page is split into five sub-tutorials, each building on the preceding parts.<br />
<br />
1) '''Explaining the tools'''. In these tutorials, we will be using ''GFtboxCommand'' and ''ClusterMonitor''. This section explains the purpose of these tools. <br />
<br />
2) '''Computer or cluster?''' Here, we illustrate when and why you should use a computing cluster for your growth simulations, and conversely, when you should use your desktop computer.<br />
<br />
3) '''Running a growth simulation for an example model'''. This section demonstrates how to use ''GFtboxCommand'' to run a growth simulation on your computer. The model used for the simulation is an example model included with the GFtbox.<br />
<br />
4) '''Altering the simulation parameters'''. Following on from part 3, here we show how to adjust a simulation parameter within GFtboxCommand. Specifically, we alter the value of ''dt'', the temporal resolution of the simulation, and show how it can be used to verify that the value specified in the published literature is reasonable.<br />
<br />
5) '''Altering the model parameters'''. Finally, we demonstrate how a number of model parameters can be varied by specifying a range of values for each model argument. We show how the computationally expensive task of simulating all combinations of specified ranges can be processed more efficiently if a computer cluster is used via the ''ClusterMonitor'' tool.<br />
<br />
='''1''' Explaining the tools=<br />
==GFtboxCommand==<br />
This is a command line version of the GFtbox. By command line, we mean that all program '''functions are operated via typed commands, without the GUI'''. Like GFtbox, GFtboxCommand is capable of running growth simulations of an interaction function, and allows the user to specify model and simulation parameters. Unlike GFtbox, this also allows the user to select '''ranges of values for a number of input parameters''', and will automatically spawn multiple simulations which explore the various combinations of those parameters. This can be used to evaluate the effect of various parameters on the growth of a given model.<br />
<br />
==ClusterMonitor==<br />
Provides a graphical user interface for managing simulations '''running remotely on a computer cluster'''. Specifically, it allows you to see which projects are present and running on the cluster, to retrieve the completed projects, to generate images of the simulations at specified stages of growth, and to remove projects from the cluster. If you do not intend to use a computer cluster, then you will not need to use ClusterMonitor.<br />
<br />
='''2''' Computer or cluster?=<br />
<br />
In basic terms, '''a computing cluster is effectively a network of many computer processors''' (often hundreds), centrally managed by a queuing system. When a job is submitted to a cluster, the job is sent to a processor that is not being used, or queued until one becomes available. In contrast, a typical desktop computer will contain one processor, limiting the number of tasks that can be performed at any one time. Jobs which must be run independently and sequentially on a desktop computer can be executed in parallel on a cluster, '''greatly reducing the total time required to complete all of the jobs'''. Although the exact details of your cluster might vary from those described in the rest of these tutorials, we aim to illustrate the generic processes involved in using GFtbox on a cluster. We are happy to offer assistance, where possible, to setup GFtbox for your cluster, so please contact us if you have any queries.<br />
<br />
Whilst the time savings offered by using a cluster can be significant, there is an overhead associated with returning the results to your personal computer. The total time to run 2 or more simulations on a cluster and to return the results will be less than running those simulations sequentially on one computer. Therefore, using a '''cluster is ideal for situations where you would like to run several simulations''', such as to evaluate the effect of a range of parameters on a growth model. You are not advised to use a cluster for running single simulations, or where you would like to step through a simulation and change parameters in a more interactive fashion. In such circumstances, you are advised to run GFtbox or GFtboxCommand on your own computer.<br />
<br />
The '''GFtbox GUI provides quick feedback''' about how the changes you have made to an interaction function have affected the growth simulation, as '''you can see the result of each simulation iteration as it completes'''. It is quick and easy therefore to see if you have dramatically changed the course of the growth simulation, and then to adjust the parameters according to your observations. This approach is well suited to the early stages of design, where you might wish to '''tweak some parameters to gauge whether or not they have had the desired effect'''. Or in the final stages, where you think your model is almost finalised, but where small adjustments are required. '''An alternative approach is to start many simulations''', based upon your initial model and by making intelligent choices about which parameters to explore. This allows you to '''harvest many results, to quickly and easily overview them''' in their finalised form, or to select interesting-looking simulations and examine them more closely on your desktop computer, using GFtbox.<br />
<br />
='''3''' Running a growth simulation for an example model=<br />
This tutorial is aimed at running a '''growth simulation for one of the example interaction functions''' included with the GFtbox. The purpose of this exercise is to firstly demonstrate how simulations can be invoked using GFtboxCommand, and secondly to show how to reproduce experimental results (specifically, those published in Kennaway et al (2011)) given an interaction function.<br />
<br />
Assuming Matlab is installed on your computer, and the latest GFtbox has been downloaded, you can add the GFtbox directory to Matlab's search path, which makes the toolbox accessible from any other path that you choose to work from. For a short tutorial on how to do this, please click [[Adding GFtbox to Matlab's search path | here]].<br />
<br />
Once the GFtbox is added to Matlab, you are ready to run a growth simulation using GFtboxCommand. In this example, '''the model that we will simulate is called GPT_CASE_RST'''. Results generated using this model are published in Kennaway et al (2011). By running this simulation, we can confirm the results in the published literature and investigate the suitability of the various parameters.<br />
<br />
===3A - Running a simulation on your computer===<br />
<br />
The following command can be typed into Matlab to run a simulation of the GPT_CASE_RST interaction function, which contains three growth models: R, S and T. '''Three separate simulations are run sequentially on your computer''', one for each model, each producing results corresponding to five intervals in the growth simulation.<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',[1:3]);<br />
<br />
GFtboxCommand accepts input arguments as name and value pairs, e.g. 'modelname', [1:3] or 'Use','Cluster'. The argument names entered in the example above are: ''Path'', ''Name'', ''Stages'' and ''modelname''.<br />
<br />
The optional '''''Path'' argument name refers to the location of the folder''' (or directory) on your computer where the model interaction function you wish to simulate is stored. '''''Name'', is the name of the folder itself'''. In this case, we are using the GPT_CASE_RST folder which is included in the GrowthToolbox. You may wish to copy this folder elsewhere, if you intend to make changes to the interaction function.<br />
<br />
During a growth simulation, a mesh can be generated at each time step of the simulation, which provides a visual representation of the growth of the biological tissue, given the various parameters of growth specified by the interaction function and how they have changed over time. Put another way, the mesh shows exactly what the growing tissue actually looks like. ''Stages'' refers to the points in the simulation (measured in hours) at which meshes should be generated and saved. In this example, '''five stages of growth will be written to disk'''. These values are chosen to best capture the appearance of the tissue at important stages of the tissue growth.<br />
<br />
The final argument name listed here is ''modelname''. This is a model-specific argument, and in this case the GPT_CASE_RST interaction function contains '''three separate models for plant growth''', allowing the desired model to be a selected. Here, the value [1:3] is specified, which is evaluated in the same way as entering [1 2 3]. This instructs GFtboxCommand to run three separate simulations, one for each of the growth models contained in the interaction function.<br />
<br />
The function of every permissible argument is given by keying the following command into Matlab:<br />
<br />
help GFtboxCommand<br />
<br />
===3B - Getting results from a completed simulation===<br />
<br />
'''1) - Generating images from a simulation you ran on your computer'''<br />
<br />
Once a growth simulation is completed, the project folder (GPT_CASE_RST, in our example) will contain another folder named "movies". Within movies, there are folders which contain results for the executed simulations. As was instructed in 3A, here we can see three folders corresponding to simulations for the three separate models. Within the first folder there are three items:<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| ''CommandLine.txt'' - This file contains the Matlab command which was used to generate the results that this sub-directory contains.<br />
<br />
''gpt_case_rst.txt'' - This file, named according to the project name, contains a copy of the interaction function which was used to produce these results.<br />
<br />
''meshes'' - This folder contains the mesh files corresponding to the stages of growth specified by the value of the Stages argument, described in 3A.<br />
|width="500px" align="center"|[[File:resultsdir.png| Directory structure of a project containing results]]<br />
|}<br />
<br />
<br />
The mesh files contain vertices information regarding the shape of the growth model at a particular stage of growth, but are not visualisable in that form. In order to '''convert these mesh files into viewable images''', we can execute the following command in Matlab:<br />
<br />
VMSreport('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Project','GPT_CASE_RST',...<br />
'Experiment','All','flattentime',572.5,'morphogen','KPAR','SNAPFIG',true);<br />
<br />
Where the ''Path'' and ''Project'' arguments have the same function as ''Path'' and ''Name'' in 3A, i.e. ''Path'' refers to the location of the project folder and ''Project'' is the name of the project folder itself.<br />
<br />
Here we can see the generated images, including the command line arguments, for one of the simulations performed in 3A. Click on a thumbnail to view a larger image.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="125px"| <small>001commandline.png</small><br> [[File:commandlineexample.png | 100px]]<br />
|width="125px"| <small>001GPT_CASE_RST_2 wild_.png</small><br>[[File:GPT_CASE_RST_example1.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_3 wild_.png</small><br>[[File:GPT_CASE_RST_example2.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_4 wild_.png</small><br>[[File:GPT_CASE_RST_example3.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_5 wild_.png</small><br>[[File:GPT_CASE_RST_example4.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_6 wild_.png</small><br>[[File:GPT_CASE_RST_example5.png | 125px]]<br />
|}<br />
<br />
<br />
'''2) - Generating images from a simulation you ran on the cluster'''<br />
<br />
NB. The results produced may not be visibly identical to those in the published literature. This is because of small, random perturbations which are applied to the initial model meshes to stop them from containing surfaces which are perfectly flat, and therefore biologically unrealistic. The results produced should be '''qualitatively but not quantitatively the same'''.<br />
<br />
'''3) - Interacting with your results using the GFtbox GUI'''<br />
<br />
='''4''' Altering the simulation parameters=<br />
<br />
One such simulation parameter is ''dt'', which is the time in seconds between iterations in the growth simulation. Large values of ''dt'' mean that fewer steps and therefore fewer calculations are required to complete a simulation. Whereas smaller values of ''dt'' mean that more steps are required, and therefore more processing time too. Therefore, a value of ''dt'' must be selected which is not so small that it is computationally unmanageable, but not so large that the observed growth is an artifact of the value of ''dt'', rather than the underlying growth model. It is necessary therefore to test a range of values for ''dt'' to ensure that the patterns of growth observed in a simulation are consistent across the range, and to find a value which is sufficient to demonstrate the model of growth and computationally efficient.<br />
<br />
'''Testing a range of ''dt''s can be achieved in several ways.''' One way is to use the ''dt'' argument when calling GFtboxCommand. This allows a single value to be tested. Another way is to make a batch of jobs, each using a different value for ''dt'', by using the 'State' argument. Lastly, a range of ''dt''s can be specified in GFtboxCommand, by using the dt argument where the values within the square brackets are the ''dt''s to simulate:<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5]);<br />
<br />
Here, we have specified '''four separate values for ''dt''''', and this means that '''four separate simulations will be run''' sequentially on your computer. As in 3B, images can then be generated for the mesh files produced, and compared to ensure consistency across the range of ''dt''s. In this figure, we can see the same model, at the same stage of growth, generated using the four values of ''dt''. Though quantitatively different, they are qualitatively the same, illustrating the suitability of the default ''dt'' value of 5.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="190px"| [[File:GPT_CASE_RST_dt0.5.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt1.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt2.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt5.png| 190px]]<br />
|}<br />
<br />
='''5''' Altering the model parameters=<br />
It is easy to see that when even a small number of range variables, mutations or ''dt''s are specified, the '''total number of simulations increases quickly'''. If many ranges are specified, then the amount of processing time becomes unmanageable on a single computer. Via ''GFtboxCommand'' and the ''ClusterMonitor'' tool, we can remotely''' run a number of simulations, in parallel, on a computing cluster'''. It is assumed that the GrowthToolbox and Matlab are installed on your cluster, and PuTTY is required on your local computer for the tools ''pscp'' (for transferring files from your computer to a cluster) and ''plink'' (for remotely executing commands on a cluster). Using the ''dt'' range argument from 4 as an example, here we add the 'Use' name argument with the value 'Cluster':<br />
<br />
GFtboxCommand('State','Start','Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5],'Use','Cluster');<br />
<br />
The 'Cluster' option for the 'Use' name argument instructs GFtboxCommand to upload the required project directory to a remote Linux server. Instead of running the simulation on your own computer, GFtboxCommand then works out how many individual simulations are specified by the command that invoked it. In this example, the only argument which will generate multiple simulations is ''dt'', of which there will be 4. Separate commands for each of these jobs are then automatically generated, each with an accompanying unique ID (The value of the ''ExpID'' name argument), and '''these are submitted as individual jobs to the cluster'''.<br />
<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.1,'ExpID','GPT_CASE_RST_1');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.5,'ExpID','GPT_CASE_RST_2');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',1,'ExpID','GPT_CASE_RST_3');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',5,'ExpID','GPT_CASE_RST_4');<br />
<br />
Once these jobs have been submitted, the ''ClusterMonitor'' tool opens and the new job batch ID is added to the list of jobs. Here, we will concentrate on three of the functions of ''ClusterMonitor'': ''Queue?'', ''Get project results'' and ''Make project pngs''. As in part 4 of this page, these functions will enable us to visualise the growth simulation results.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Queue <br> Get project results <br> Make project pngs<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
<br />
Whilst ''ClusterMonitor'' is a useful tool for managing your cluster jobs, it is advisable to '''have at least a rudimentary understanding of and ability to use a Unix-based computer system'''. In particular, the abilities to list the contents of a folder, view the contents of a file, change the current working folder, delete, and to copy or move files or folders, are essential Unix skills for making sure everything is working as intended. It is beyond the scope of these tutorials to provide an in depth Unix tutorial (many good and simple tutorials exist on the web), but here is a short description of the Unix commands (which may be specific to our Unix-based system) that we believe to be important.<br />
<br />
''ls'' - list the files and folders in a directory. Use the -al option (e.g. ls -al) to see modification dates, file permissions, sizes, etc.<br />
<br />
''cd X'' - Change the current directory to directory X.<br />
<br />
''rm X'' - Delete the file X.<br />
<br />
''cp X NewPath/X2'' - Copy file X to the NewPath directory and name the copy X2.<br />
<br />
''mv X NewPath/X2'' - Move file X to the NewPath directory and rename it to X2.<br />
<br />
''cat test.txt'' - Print the contents of the text file test.txt onto the screen.<br />
<br />
''pwd'' - Print the present working directory onto the screen.<br />
<br />
''more test.txt'' - Print the file test.txt to the screen, a screenful at a time, scrolling each time you hit the enter key.</div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Running_example_models_and_using_a_cluster&diff=5381Running example models and using a cluster2011-06-23T14:36:15Z<p>JacobNewman: </p>
<hr />
<div>[[GFtbox Tutorial pages | Back to GFtbox Tutorial pages]]<br />
<br />
The purpose of these tutorials is to learn how to run the example growth simulations included in GFtbox. We will describe methods for running simulations locally (on your own computer) and remotely (on a computing cluster). It is assumed that you have already downloaded the GFtbox software and have Matlab installed.<br />
<br />
=Getting Started=<br />
<br />
The remainder of this page is split into five sub-tutorials, each building on the preceding parts.<br />
<br />
1) '''Explaining the tools'''. In these tutorials, we will be using ''GFtboxCommand'' and ''ClusterMonitor''. This section explains the purpose of these tools. <br />
<br />
2) '''Computer or cluster?''' Here, we illustrate when and why you should use a computing cluster for your growth simulations, and conversely, when you should use your desktop computer.<br />
<br />
3) '''Running a growth simulation for an example model'''. This section demonstrates how to use ''GFtboxCommand'' to run a growth simulation on your computer. The model used for the simulation is an example model included with the GFtbox.<br />
<br />
4) '''Altering the simulation parameters'''. Following on from part 3, here we show how to adjust a simulation parameter within GFtboxCommand. Specifically, we alter the value of ''dt'', the temporal resolution of the simulation, and show how it can be used to verify that the value specified in the published literature is reasonable.<br />
<br />
5) '''Altering the model parameters'''. Finally, we demonstrate how a number of model parameters can be varied by specifying a range of values for each model argument. We show how the computationally expensive task of simulating all combinations of specified ranges can be processed more efficiently if a computer cluster is used via the ''ClusterMonitor'' tool.<br />
<br />
='''1''' Explaining the tools=<br />
==GFtboxCommand==<br />
This is a command line version of the GFtbox. By command line, we mean that all program '''functions are operated via typed commands, without the GUI'''. Like GFtbox, GFtboxCommand is capable of running growth simulations of an interaction function, and allows the user to specify model and simulation parameters. Unlike GFtbox, this also allows the user to select '''ranges of values for a number of input parameters''', and will automatically spawn multiple simulations which explore the various combinations of those parameters. This can be used to evaluate the effect of various parameters on the growth of a given model.<br />
<br />
==ClusterMonitor==<br />
Provides a graphical user interface for managing simulations '''running remotely on a computer cluster'''. Specifically, it allows you to see which projects are present and running on the cluster, to retrieve the completed projects, to generate images of the simulations at specified stages of growth, and to remove projects from the cluster. If you do not intend to use a computer cluster, then you will not need to use ClusterMonitor.<br />
<br />
='''2''' Computer or cluster?=<br />
<br />
In basic terms, '''a computing cluster is effectively a network of many computer processors''' (often hundreds), centrally managed by a queuing system. When a job is submitted to a cluster, the job is sent to a processor that is not being used, or queued until one becomes available. In contrast, a typical desktop computer will contain one processor, limiting the number of tasks that can be performed at any one time. Jobs which must be run independently and sequentially on a desktop computer can be executed in parallel on a cluster, '''greatly reducing the total time required to complete all of the jobs'''. Although the exact details of your cluster might vary from those described in the rest of these tutorials, we aim to illustrate the generic processes involved in using GFtbox on a cluster. We are happy to offer assistance, where possible, to setup GFtbox for your cluster, so please contact us if you have any queries.<br />
<br />
Whilst the time savings offered by using a cluster can be significant, there is an overhead associated with returning the results to your personal computer. The total time to run 2 or more simulations on a cluster and to return the results will be less than running those simulations sequentially on one computer. Therefore, using a '''cluster is ideal for situations where you would like to run several simulations''', such as to evaluate the effect of a range of parameters on a growth model. You are not advised to use a cluster for running single simulations, or where you would like to step through a simulation and change parameters in a more interactive fashion. In such circumstances, you are advised to run GFtbox or GFtboxCommand on your own computer.<br />
<br />
The '''GFtbox GUI provides quick feedback''' about how the changes you have made to an interaction function have affected the growth simulation, as '''you can see the result of each simulation iteration as it completes'''. It is quick and easy therefore to see if you have dramatically changed the course of the growth simulation, and then to adjust the parameters according to your observations. This approach is well suited to the early stages of design, where you might wish to '''tweak some parameters to gauge whether or not they have had the desired effect'''. Or in the final stages, where you think your model is almost finalised, but where small adjustments are required. '''An alternative approach is to start many simulations''', based upon your initial model and by making intelligent choices about which parameters to explore. This allows you to '''harvest many results, to quickly and easily overview them''' in their finalised form, or to select interesting-looking simulations and examine them more closely on your desktop computer, using GFtbox.<br />
<br />
='''3''' Running a growth simulation for an example model=<br />
This tutorial is aimed at running a '''growth simulation for one of the example interaction functions''' included with the GFtbox. The purpose of this exercise is to firstly demonstrate how simulations can be invoked using GFtboxCommand, and secondly to show how to reproduce experimental results (specifically, those published in Kennaway et al (2011)) given an interaction function.<br />
<br />
Assuming Matlab is installed on your computer, and the latest GFtbox has been downloaded, you can add the GFtbox directory to Matlab's search path, which makes the toolbox accessible from any other path that you choose to work from. For a short tutorial on how to do this, please click [[Adding GFtbox to Matlab's search path | here]].<br />
<br />
Once the GFtbox is added to Matlab, you are ready to run a growth simulation using GFtboxCommand. In this example, '''the model that we will simulate is called GPT_CASE_RST'''. Results generated using this model are published in Kennaway et al (2011). By running this simulation, we can confirm the results in the published literature and investigate the suitability of the various parameters.<br />
<br />
===3A - Running a simulation on your computer===<br />
<br />
The following command can be typed into Matlab to run a simulation of the GPT_CASE_RST interaction function, which contains three growth models: R, S and T. '''Three separate simulations are run sequentially on your computer''', one for each model, each producing results corresponding to five intervals in the growth simulation.<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',[1:3]);<br />
<br />
GFtboxCommand accepts input arguments as name and value pairs, e.g. 'modelname', [1:3] or 'Use','Cluster'. The argument names entered in the example above are: ''Path'', ''Name'', ''Stages'' and ''modelname''.<br />
<br />
The optional '''''Path'' argument name refers to the location of the folder''' (or directory) on your computer where the model interaction function you wish to simulate is stored. '''''Name'', is the name of the folder itself'''. In this case, we are using the GPT_CASE_RST folder which is included in the GrowthToolbox. You may wish to copy this folder elsewhere, if you intend to make changes to the interaction function.<br />
<br />
During a growth simulation, a mesh can be generated at each time step of the simulation, which provides a visual representation of the growth of the biological tissue, given the various parameters of growth specified by the interaction function and how they have changed over time. Put another way, the mesh shows exactly what the growing tissue actually looks like. ''Stages'' refers to the points in the simulation (measured in hours) at which meshes should be generated and saved. In this example, '''five stages of growth will be written to disk'''. These values are chosen to best capture the appearance of the tissue at important stages of the tissue growth.<br />
<br />
The final argument name listed here is ''modelname''. This is a model-specific argument, and in this case the GPT_CASE_RST interaction function contains '''three separate models for plant growth''', allowing the desired model to be a selected. Here, the value [1:3] is specified, which is evaluated in the same way as entering [1 2 3]. This instructs GFtboxCommand to run three separate simulations, one for each of the growth models contained in the interaction function.<br />
<br />
The function of every permissible argument is given by keying the following command into Matlab:<br />
<br />
help GFtboxCommand<br />
<br />
===3B - Getting results from a completed simulation===<br />
<br />
'''1) - Generating images from a simulation you ran on your computer'''<br />
<br />
Once a growth simulation is completed, the project folder (GPT_CASE_RST, in our example) will contain another folder named "movies". Within movies, there are folders which contain results for the executed simulations. As was instructed in 3A, here we can see three folders corresponding to simulations for the three separate models. Within the first folder there are three items:<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| ''CommandLine.txt'' - This file contains the Matlab command which was used to generate the results that this sub-directory contains.<br />
<br />
''gpt_case_rst.txt'' - This file, named according to the project name, contains a copy of the interaction function which was used to produce these results.<br />
<br />
''meshes'' - This folder contains the mesh files corresponding to the stages of growth specified by the value of the Stages argument, described in 3A.<br />
|width="500px" align="center"|[[File:resultsdir.png| Directory structure of a project containing results]]<br />
|}<br />
<br />
<br />
The mesh files contain vertices information regarding the shape of the growth model at a particular stage of growth, but are not visualisable in that form. In order to '''convert these mesh files into viewable images''', we can execute the following command in Matlab:<br />
<br />
VMSreport('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Project','GPT_CASE_RST',...<br />
'Experiment','All','flattentime',572.5,'morphogen','KPAR','SNAPFIG',true);<br />
<br />
Where the ''Path'' and ''Project'' arguments have the same function as ''Path'' and ''Name'' in 3A, i.e. ''Path'' refers to the location of the project folder and ''Project'' is the name of the project folder itself.<br />
<br />
Here we can see the generated images, including the command line arguments, for one of the simulations performed in 3A. Click on a thumbnail to view a larger image.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="125px"| <small>001commandline.png</small><br> [[File:commandlineexample.png | 100px]]<br />
|width="125px"| <small>001GPT_CASE_RST_2 wild_.png</small><br>[[File:GPT_CASE_RST_example1.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_3 wild_.png</small><br>[[File:GPT_CASE_RST_example2.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_4 wild_.png</small><br>[[File:GPT_CASE_RST_example3.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_5 wild_.png</small><br>[[File:GPT_CASE_RST_example4.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_6 wild_.png</small><br>[[File:GPT_CASE_RST_example5.png | 125px]]<br />
|}<br />
<br />
<br />
'''2) - Generating images from a simulation you ran on the cluster'''<br />
<br />
NB. The results produced may not be visibly identical to those in the published literature. This is because of small, random perturbations which are applied to the initial model meshes to stop them from containing surfaces which are perfectly flat, and therefore biologically unrealistic. The results produced should be '''qualitatively but not quantitatively the same'''.<br />
<br />
'''3) - Interacting with your results using the GFtbox GUI'''<br />
<br />
='''4''' Altering the simulation parameters=<br />
<br />
One such simulation parameter is ''dt'', which is the time in seconds between iterations in the growth simulation. Large values of ''dt'' mean that fewer steps and therefore fewer calculations are required to complete a simulation. Whereas smaller values of ''dt'' mean that more steps are required, and therefore more processing time too. Therefore, a value of ''dt'' must be selected which is not so small that it is computationally unmanageable, but not so large that the observed growth is an artifact of the value of ''dt'', rather than the underlying growth model. It is necessary therefore to test a range of values for ''dt'' to ensure that the patterns of growth observed in a simulation are consistent across the range, and to find a value which is sufficient to demonstrate the model of growth and computationally efficient.<br />
<br />
'''Testing a range of ''dt''s can be achieved in several ways.''' One way is to use the ''dt'' argument when calling GFtboxCommand. This allows a single value to be tested. Another way is to make a batch of jobs, each using a different value for ''dt'', by using the 'State' argument. Lastly, a range of ''dt''s can be specified in GFtboxCommand, by using the dt argument where the values within the square brackets are the ''dt''s to simulate:<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5]);<br />
<br />
Here, we have specified '''four separate values for ''dt''''', and this means that '''four separate simulations will be run''' sequentially on your computer. As in 3B, images can then be generated for the mesh files produced, and compared to ensure consistency across the range of ''dt''s. In this figure, we can see the same model, at the same stage of growth, generated using the four values of ''dt''. Though quantitatively different, they are qualitatively the same, illustrating the suitability of the default ''dt'' value of 5.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="190px"| [[File:GPT_CASE_RST_dt0.5.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt1.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt2.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt5.png| 190px]]<br />
|}<br />
<br />
='''5''' Altering the model parameters=<br />
It is easy to see that when even a small number of range variables, mutations or ''dt''s are specified, the '''total number of simulations increases quickly'''. If many ranges are specified, then the amount of processing time becomes unmanageable on a single computer. Via ''GFtboxCommand'' and the ''ClusterMonitor'' tool, we can remotely''' run a number of simulations, in parallel, on a computing cluster'''. It is assumed that the GrowthToolbox and Matlab are installed on your cluster, and PuTTY is required on your local computer for the tools ''pscp'' (for transferring files from your computer to a cluster) and ''plink'' (for remotely executing commands on a cluster). Using the ''dt'' range argument from 4 as an example, here we add the 'Use' name argument with the value 'Cluster':<br />
<br />
GFtboxCommand('State','Start','Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5],'Use','Cluster');<br />
<br />
The 'Cluster' option for the 'Use' name argument instructs GFtboxCommand to upload the required project directory to a remote Linux server. Instead of running the simulation on your own computer, GFtboxCommand then works out how many individual simulations are specified by the command that invoked it. In this example, the only argument which will generate multiple simulations is ''dt'', of which there will be 4. Separate commands for each of these jobs are then automatically generated, each with an accompanying unique ID (The value of the ''ExpID'' name argument), and '''these are submitted as individual jobs to the cluster'''.<br />
<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.1,'ExpID','GPT_CASE_RST_1');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.5,'ExpID','GPT_CASE_RST_2');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',1,'ExpID','GPT_CASE_RST_3');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',5,'ExpID','GPT_CASE_RST_4');<br />
<br />
Once these jobs have been submitted, the ''ClusterMonitor'' tool opens and the new job batch ID is added to the list of jobs. Here, we will concentrate on three of the functions of ''ClusterMonitor'': ''Queue?'', ''Get project results'' and ''Make project pngs''. As in part 4 of this page, these functions will enable us to visualise the growth simulation results.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Queue <br> Get project results <br> Make project pngs<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
<br />
Whilst ''ClusterMonitor'' is a useful tool for managing your cluster jobs, it is advisable to '''have at least a rudimentary understanding of and ability to use a Unix-based computer system'''. In particular, the abilities to list the contents of a folder, view the contents of a file, change the current working folder, delete, and to copy or move files or folders, are essential Unix skills for making sure everything is working as intended. It is beyond the scope of these tutorials to provide an in depth Unix tutorial (many good and simple tutorials exist on the web), but here is a short description of the Unix commands (which may be specific to our Unix-based system) that we believe to be important.<br />
<br />
''ls'' - list the files and folders in a directory<br />
<br />
''cd X'' - Change the current directory to directory X<br />
<br />
''rm X'' - Delete the file X<br />
<br />
''mv X NewPath/X2'' - Move file X to the NewPath directory and rename it to X2</div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Running_example_models_and_using_a_cluster&diff=5380Running example models and using a cluster2011-06-23T14:35:48Z<p>JacobNewman: </p>
<hr />
<div>[[GFtbox Tutorial pages | Back to GFtbox Tutorial pages]]<br />
<br />
The purpose of these tutorials is to learn how to run the example growth simulations included in GFtbox. We will describe methods for running simulations locally (on your own computer) and remotely (on a computing cluster). It is assumed that you have already downloaded the GFtbox software and have Matlab installed.<br />
<br />
=Getting Started=<br />
<br />
The remainder of this page is split into five sub-tutorials, each building on the preceding parts.<br />
<br />
1) '''Explaining the tools'''. In these tutorials, we will be using ''GFtboxCommand'' and ''ClusterMonitor''. This section explains the purpose of these tools. <br />
<br />
2) '''Computer or cluster?''' Here, we illustrate when and why you should use a computing cluster for your growth simulations, and conversely, when you should use your desktop computer.<br />
<br />
3) '''Running a growth simulation for an example model'''. This section demonstrates how to use ''GFtboxCommand'' to run a growth simulation on your computer. The model used for the simulation is an example model included with the GFtbox.<br />
<br />
4) '''Altering the simulation parameters'''. Following on from part 3, here we show how to adjust a simulation parameter within GFtboxCommand. Specifically, we alter the value of ''dt'', the temporal resolution of the simulation, and show how it can be used to verify that the value specified in the published literature is reasonable.<br />
<br />
5) '''Altering the model parameters'''. Finally, we demonstrate how a number of model parameters can be varied by specifying a range of values for each model argument. We show how the computationally expensive task of simulating all combinations of specified ranges can be processed more efficiently if a computer cluster is used via the ''ClusterMonitor'' tool.<br />
<br />
='''1''' Explaining the tools=<br />
==GFtboxCommand==<br />
This is a command line version of the GFtbox. By command line, we mean that all program '''functions are operated via typed commands, without the GUI'''. Like GFtbox, GFtboxCommand is capable of running growth simulations of an interaction function, and allows the user to specify model and simulation parameters. Unlike GFtbox, this also allows the user to select '''ranges of values for a number of input parameters''', and will automatically spawn multiple simulations which explore the various combinations of those parameters. This can be used to evaluate the effect of various parameters on the growth of a given model.<br />
<br />
==ClusterMonitor==<br />
Provides a graphical user interface for managing simulations '''running remotely on a computer cluster'''. Specifically, it allows you to see which projects are present and running on the cluster, to retrieve the completed projects, to generate images of the simulations at specified stages of growth, and to remove projects from the cluster. If you do not intend to use a computer cluster, then you will not need to use ClusterMonitor.<br />
<br />
='''2''' Computer or cluster?=<br />
<br />
In basic terms, '''a computing cluster is effectively a network of many computer processors''' (often hundreds), centrally managed by a queuing system. When a job is submitted to a cluster, the job is sent to a processor that is not being used, or queued until one becomes available. In contrast, a typical desktop computer will contain one processor, limiting the number of tasks that can be performed at any one time. Jobs which must be run independently and sequentially on a desktop computer can be executed in parallel on a cluster, '''greatly reducing the total time required to complete all of the jobs'''. Although the exact details of your cluster might vary from those described in the rest of these tutorials, we aim to illustrate the generic processes involved in using GFtbox on a cluster. We are happy to offer assistance, where possible, to setup GFtbox for your cluster, so please contact us if you have any queries.<br />
<br />
Whilst the time savings offered by using a cluster can be significant, there is an overhead associated with returning the results to your personal computer. The total time to run 2 or more simulations on a cluster and to return the results will be less than running those simulations sequentially on one computer. Therefore, using a '''cluster is ideal for situations where you would like to run several simulations''', such as to evaluate the effect of a range of parameters on a growth model. You are not advised to use a cluster for running single simulations, or where you would like to step through a simulation and change parameters in a more interactive fashion. In such circumstances, you are advised to run GFtbox or GFtboxCommand on your own computer.<br />
<br />
The '''GFtbox GUI provides quick feedback''' about how the changes you have made to an interaction function have affected the growth simulation, as '''you can see the result of each simulation iteration as it completes'''. It is quick and easy therefore to see if you have dramatically changed the course of the growth simulation, and then to adjust the parameters according to your observations. This approach is well suited to the early stages of design, where you might wish to '''tweak some parameters to gauge whether or not they have had the desired effect'''. Or in the final stages, where you think your model is almost finalised, but where small adjustments are required. '''An alternative approach is to start many simulations''', based upon your initial model and by making intelligent choices about which parameters to explore. This allows you to '''harvest many results, to quickly and easily overview them''' in their finalised form, or to select interesting-looking simulations and examine them more closely on your desktop computer, using GFtbox.<br />
<br />
='''3''' Running a growth simulation for an example model=<br />
This tutorial is aimed at running a '''growth simulation for one of the example interaction functions''' included with the GFtbox. The purpose of this exercise is to firstly demonstrate how simulations can be invoked using GFtboxCommand, and secondly to show how to reproduce experimental results (specifically, those published in Kennaway et al (2011)) given an interaction function.<br />
<br />
Assuming Matlab is installed on your computer, and the latest GFtbox has been downloaded, you can add the GFtbox directory to Matlab's search path, which makes the toolbox accessible from any other path that you choose to work from. For a short tutorial on how to do this, please click [[Adding GFtbox to Matlab's search path | here]].<br />
<br />
Once the GFtbox is added to Matlab, you are ready to run a growth simulation using GFtboxCommand. In this example, '''the model that we will simulate is called GPT_CASE_RST'''. Results generated using this model are published in Kennaway et al (2011). By running this simulation, we can confirm the results in the published literature and investigate the suitability of the various parameters.<br />
<br />
===3A - Running a simulation on your computer===<br />
<br />
The following command can be typed into Matlab to run a simulation of the GPT_CASE_RST interaction function, which contains three growth models: R, S and T. '''Three separate simulations are run sequentially on your computer''', one for each model, each producing results corresponding to five intervals in the growth simulation.<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',[1:3]);<br />
<br />
GFtboxCommand accepts input arguments as name and value pairs, e.g. 'modelname', [1:3] or 'Use','Cluster'. The argument names entered in the example above are: ''Path'', ''Name'', ''Stages'' and ''modelname''.<br />
<br />
The optional '''''Path'' argument name refers to the location of the folder''' (or directory) on your computer where the model interaction function you wish to simulate is stored. '''''Name'', is the name of the folder itself'''. In this case, we are using the GPT_CASE_RST folder which is included in the GrowthToolbox. You may wish to copy this folder elsewhere, if you intend to make changes to the interaction function.<br />
<br />
During a growth simulation, a mesh can be generated at each time step of the simulation, which provides a visual representation of the growth of the biological tissue, given the various parameters of growth specified by the interaction function and how they have changed over time. Put another way, the mesh shows exactly what the growing tissue actually looks like. ''Stages'' refers to the points in the simulation (measured in hours) at which meshes should be generated and saved. In this example, '''five stages of growth will be written to disk'''. These values are chosen to best capture the appearance of the tissue at important stages of the tissue growth.<br />
<br />
The final argument name listed here is ''modelname''. This is a model-specific argument, and in this case the GPT_CASE_RST interaction function contains '''three separate models for plant growth''', allowing the desired model to be a selected. Here, the value [1:3] is specified, which is evaluated in the same way as entering [1 2 3]. This instructs GFtboxCommand to run three separate simulations, one for each of the growth models contained in the interaction function.<br />
<br />
The function of every permissible argument is given by keying the following command into Matlab:<br />
<br />
help GFtboxCommand<br />
<br />
===3B - Getting results from a completed simulation===<br />
<br />
'''1) - Generating images from a simulation you ran on your computer'''<br />
<br />
Once a growth simulation is completed, the project folder (GPT_CASE_RST, in our example) will contain another folder named "movies". Within movies, there are folders which contain results for the executed simulations. As was instructed in 3A, here we can see three folders corresponding to simulations for the three separate models. Within the first folder there are three items:<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| ''CommandLine.txt'' - This file contains the Matlab command which was used to generate the results that this sub-directory contains.<br />
<br />
''gpt_case_rst.txt'' - This file, named according to the project name, contains a copy of the interaction function which was used to produce these results.<br />
<br />
''meshes'' - This folder contains the mesh files corresponding to the stages of growth specified by the value of the Stages argument, described in 3A.<br />
|width="500px" align="center"|[[File:resultsdir.png| Directory structure of a project containing results]]<br />
|}<br />
<br />
<br />
The mesh files contain vertices information regarding the shape of the growth model at a particular stage of growth, but are not visualisable in that form. In order to '''convert these mesh files into viewable images''', we can execute the following command in Matlab:<br />
<br />
VMSreport('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Project','GPT_CASE_RST',...<br />
'Experiment','All','flattentime',572.5,'morphogen','KPAR','SNAPFIG',true);<br />
<br />
Where the ''Path'' and ''Project'' arguments have the same function as ''Path'' and ''Name'' in 3A, i.e. ''Path'' refers to the location of the project folder and ''Project'' is the name of the project folder itself.<br />
<br />
Here we can see the generated images, including the command line arguments, for one of the simulations performed in 3A. Click on a thumbnail to view a larger image.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="125px"| <small>001commandline.png</small><br> [[File:commandlineexample.png | 100px]]<br />
|width="125px"| <small>001GPT_CASE_RST_2 wild_.png</small><br>[[File:GPT_CASE_RST_example1.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_3 wild_.png</small><br>[[File:GPT_CASE_RST_example2.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_4 wild_.png</small><br>[[File:GPT_CASE_RST_example3.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_5 wild_.png</small><br>[[File:GPT_CASE_RST_example4.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_6 wild_.png</small><br>[[File:GPT_CASE_RST_example5.png | 125px]]<br />
|}<br />
<br />
<br />
'''2) - Generating images from a simulation you ran on the cluster'''<br />
<br />
NB. The results produced may not be visibly identical to those in the published literature. This is because of small, random perturbations which are applied to the initial model meshes to stop them from containing surfaces which are perfectly flat, and therefore biologically unrealistic. The results produced should be '''qualitatively but not quantitatively the same'''.<br />
<br />
'''3) - Interacting with your results using the GFtbox GUI'''<br />
<br />
='''4''' Altering the simulation parameters=<br />
<br />
One such simulation parameter is ''dt'', which is the time in seconds between iterations in the growth simulation. Large values of ''dt'' mean that fewer steps and therefore fewer calculations are required to complete a simulation. Whereas smaller values of ''dt'' mean that more steps are required, and therefore more processing time too. Therefore, a value of ''dt'' must be selected which is not so small that it is computationally unmanageable, but not so large that the observed growth is an artifact of the value of ''dt'', rather than the underlying growth model. It is necessary therefore to test a range of values for ''dt'' to ensure that the patterns of growth observed in a simulation are consistent across the range, and to find a value which is sufficient to demonstrate the model of growth and computationally efficient.<br />
<br />
'''Testing a range of ''dt''s can be achieved in several ways.''' One way is to use the ''dt'' argument when calling GFtboxCommand. This allows a single value to be tested. Another way is to make a batch of jobs, each using a different value for ''dt'', by using the 'State' argument. Lastly, a range of ''dt''s can be specified in GFtboxCommand, by using the dt argument where the values within the square brackets are the ''dt''s to simulate:<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5]);<br />
<br />
Here, we have specified '''four separate values for ''dt''''', and this means that '''four separate simulations will be run''' sequentially on your computer. As in 3B, images can then be generated for the mesh files produced, and compared to ensure consistency across the range of ''dt''s. In this figure, we can see the same model, at the same stage of growth, generated using the four values of ''dt''. Though quantitatively different, they are qualitatively the same, illustrating the suitability of the default ''dt'' value of 5.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="190px"| [[File:GPT_CASE_RST_dt0.5.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt1.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt2.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt5.png| 190px]]<br />
|}<br />
<br />
='''5''' Altering the model parameters=<br />
It is easy to see that when even a small number of range variables, mutations or ''dt''s are specified, the '''total number of simulations increases quickly'''. If many ranges are specified, then the amount of processing time becomes unmanageable on a single computer. Via ''GFtboxCommand'' and the ''ClusterMonitor'' tool, we can remotely''' run a number of simulations, in parallel, on a computing cluster'''. It is assumed that the GrowthToolbox and Matlab are installed on your cluster, and PuTTY is required on your local computer for the tools ''pscp'' (for transferring files from your computer to a cluster) and ''plink'' (for remotely executing commands on a cluster). Using the ''dt'' range argument from 4 as an example, here we add the 'Use' name argument with the value 'Cluster':<br />
<br />
GFtboxCommand('State','Start','Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5],'Use','Cluster');<br />
<br />
The 'Cluster' option for the 'Use' name argument instructs GFtboxCommand to upload the required project directory to a remote Linux server. Instead of running the simulation on your own computer, GFtboxCommand then works out how many individual simulations are specified by the command that invoked it. In this example, the only argument which will generate multiple simulations is ''dt'', of which there will be 4. Separate commands for each of these jobs are then automatically generated, each with an accompanying unique ID (The value of the ''ExpID'' name argument), and '''these are submitted as individual jobs to the cluster'''.<br />
<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.1,'ExpID','GPT_CASE_RST_1');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.5,'ExpID','GPT_CASE_RST_2');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',1,'ExpID','GPT_CASE_RST_3');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',5,'ExpID','GPT_CASE_RST_4');<br />
<br />
Once these jobs have been submitted, the ''ClusterMonitor'' tool opens and the new job batch ID is added to the list of jobs. Here, we will concentrate on three of the functions of ''ClusterMonitor'': ''Queue?'', ''Get project results'' and ''Make project pngs''. As in part 4 of this page, these functions will enable us to visualise the growth simulation results.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Queue <br> Get project results <br> Make project pngs<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
<br />
Whilst ''ClusterMonitor'' is a useful tool for managing your cluster jobs, it is advisable to '''have at least a rudimentary understanding of and ability to use a Unix-based computer system'''. In particular, the abilities to list the contents of a folder, view the contents of a file, change the current working folder, delete, and to copy or move files or folders, are essential Unix skills for making sure everything is working as intended. It is beyond the scope of these tutorials to provide an in depth Unix tutorial (many good and simple tutorials exist on the web), but here is a short description of the Unix commands (which may be specific to our Unix-based system) that we believe to be important.<br />
<br />
''ls'' - list the files and folders in a directory<br />
''cd X'' - Change the current directory to directory X<br />
''rm X'' - Delete the file X<br />
''mv X NewPath/X2'' - Move file X to the NewPath directory and rename it to X2</div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Running_example_models_and_using_a_cluster&diff=5379Running example models and using a cluster2011-06-23T14:18:06Z<p>JacobNewman: </p>
<hr />
<div>[[GFtbox Tutorial pages | Back to GFtbox Tutorial pages]]<br />
<br />
The purpose of these tutorials is to learn how to run the example growth simulations included in GFtbox. We will describe methods for running simulations locally (on your own computer) and remotely (on a computing cluster). It is assumed that you have already downloaded the GFtbox software and have Matlab installed.<br />
<br />
=Getting Started=<br />
<br />
The remainder of this page is split into five sub-tutorials, each building on the preceding parts.<br />
<br />
1) '''Explaining the tools'''. In these tutorials, we will be using ''GFtboxCommand'' and ''ClusterMonitor''. This section explains the purpose of these tools. <br />
<br />
2) '''Computer or cluster?''' Here, we illustrate when and why you should use a computing cluster for your growth simulations, and conversely, when you should use your desktop computer.<br />
<br />
3) '''Running a growth simulation for an example model'''. This section demonstrates how to use ''GFtboxCommand'' to run a growth simulation on your computer. The model used for the simulation is an example model included with the GFtbox.<br />
<br />
4) '''Altering the simulation parameters'''. Following on from part 3, here we show how to adjust a simulation parameter within GFtboxCommand. Specifically, we alter the value of ''dt'', the temporal resolution of the simulation, and show how it can be used to verify that the value specified in the published literature is reasonable.<br />
<br />
5) '''Altering the model parameters'''. Finally, we demonstrate how a number of model parameters can be varied by specifying a range of values for each model argument. We show how the computationally expensive task of simulating all combinations of specified ranges can be processed more efficiently if a computer cluster is used via the ''ClusterMonitor'' tool.<br />
<br />
='''1''' Explaining the tools=<br />
==GFtboxCommand==<br />
This is a command line version of the GFtbox. By command line, we mean that all program '''functions are operated via typed commands, without the GUI'''. Like GFtbox, GFtboxCommand is capable of running growth simulations of an interaction function, and allows the user to specify model and simulation parameters. Unlike GFtbox, this also allows the user to select '''ranges of values for a number of input parameters''', and will automatically spawn multiple simulations which explore the various combinations of those parameters. This can be used to evaluate the effect of various parameters on the growth of a given model.<br />
<br />
==ClusterMonitor==<br />
Provides a graphical user interface for managing simulations '''running remotely on a computer cluster'''. Specifically, it allows you to see which projects are present and running on the cluster, to retrieve the completed projects, to generate images of the simulations at specified stages of growth, and to remove projects from the cluster. If you do not intend to use a computer cluster, then you will not need to use ClusterMonitor.<br />
<br />
='''2''' Computer or cluster?=<br />
<br />
In basic terms, '''a computing cluster is effectively a network of many computer processors''' (often hundreds), centrally managed by a queuing system. When a job is submitted to a cluster, the job is sent to a processor that is not being used, or queued until one becomes available. In contrast, a typical desktop computer will contain one processor, limiting the number of tasks that can be performed at any one time. Jobs which must be run independently and sequentially on a desktop computer can be executed in parallel on a cluster, '''greatly reducing the total time required to complete all of the jobs'''. Although the exact details of your cluster might vary from those described in the rest of these tutorials, we aim to illustrate the generic processes involved in using GFtbox on a cluster. We are happy to offer assistance, where possible, to setup GFtbox for your cluster, so please contact us if you have any queries.<br />
<br />
Whilst the time savings offered by using a cluster can be significant, there is an overhead associated with returning the results to your personal computer. The total time to run 2 or more simulations on a cluster and to return the results will be less than running those simulations sequentially on one computer. Therefore, using a '''cluster is ideal for situations where you would like to run several simulations''', such as to evaluate the effect of a range of parameters on a growth model. You are not advised to use a cluster for running single simulations, or where you would like to step through a simulation and change parameters in a more interactive fashion. In such circumstances, you are advised to run GFtbox or GFtboxCommand on your own computer.<br />
<br />
The '''GFtbox GUI provides quick feedback''' about how the changes you have made to an interaction function have affected the growth simulation, as '''you can see the result of each simulation iteration as it completes'''. It is quick and easy therefore to see if you have dramatically changed the course of the growth simulation, and then to adjust the parameters according to your observations. This approach is well suited to the early stages of design, where you might wish to '''tweak some parameters to gauge whether or not they have had the desired effect'''. Or in the final stages, where you think your model is almost finalised, but where small adjustments are required. '''An alternative approach is to start many simulations''', based upon your initial model and by making intelligent choices about which parameters to explore. This allows you to '''harvest many results, to quickly and easily overview them''' in their finalised form, or to select interesting-looking simulations and examine them more closely on your desktop computer, using GFtbox.<br />
<br />
='''3''' Running a growth simulation for an example model=<br />
This tutorial is aimed at running a '''growth simulation for one of the example interaction functions''' included with the GFtbox. The purpose of this exercise is to firstly demonstrate how simulations can be invoked using GFtboxCommand, and secondly to show how to reproduce experimental results (specifically, those published in Kennaway et al (2011)) given an interaction function.<br />
<br />
Assuming Matlab is installed on your computer, and the latest GFtbox has been downloaded, you can add the GFtbox directory to Matlab's search path, which makes the toolbox accessible from any other path that you choose to work from. For a short tutorial on how to do this, please click [[Adding GFtbox to Matlab's search path | here]].<br />
<br />
Once the GFtbox is added to Matlab, you are ready to run a growth simulation using GFtboxCommand. In this example, '''the model that we will simulate is called GPT_CASE_RST'''. Results generated using this model are published in Kennaway et al (2011). By running this simulation, we can confirm the results in the published literature and investigate the suitability of the various parameters.<br />
<br />
===3A - Running a simulation on your computer===<br />
<br />
The following command can be typed into Matlab to run a simulation of the GPT_CASE_RST interaction function, which contains three growth models: R, S and T. '''Three separate simulations are run sequentially on your computer''', one for each model, each producing results corresponding to five intervals in the growth simulation.<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',[1:3]);<br />
<br />
GFtboxCommand accepts input arguments as name and value pairs, e.g. 'modelname', [1:3] or 'Use','Cluster'. The argument names entered in the example above are: ''Path'', ''Name'', ''Stages'' and ''modelname''.<br />
<br />
The optional '''''Path'' argument name refers to the location of the folder''' (or directory) on your computer where the model interaction function you wish to simulate is stored. '''''Name'', is the name of the folder itself'''. In this case, we are using the GPT_CASE_RST folder which is included in the GrowthToolbox. You may wish to copy this folder elsewhere, if you intend to make changes to the interaction function.<br />
<br />
During a growth simulation, a mesh can be generated at each time step of the simulation, which provides a visual representation of the growth of the biological tissue, given the various parameters of growth specified by the interaction function and how they have changed over time. Put another way, the mesh shows exactly what the growing tissue actually looks like. ''Stages'' refers to the points in the simulation (measured in hours) at which meshes should be generated and saved. In this example, '''five stages of growth will be written to disk'''. These values are chosen to best capture the appearance of the tissue at important stages of the tissue growth.<br />
<br />
The final argument name listed here is ''modelname''. This is a model-specific argument, and in this case the GPT_CASE_RST interaction function contains '''three separate models for plant growth''', allowing the desired model to be a selected. Here, the value [1:3] is specified, which is evaluated in the same way as entering [1 2 3]. This instructs GFtboxCommand to run three separate simulations, one for each of the growth models contained in the interaction function.<br />
<br />
The function of every permissible argument is given by keying the following command into Matlab:<br />
<br />
help GFtboxCommand<br />
<br />
===3B - Getting results from a completed simulation===<br />
<br />
'''1) - Generating images from a simulation you ran on your computer'''<br />
<br />
Once a growth simulation is completed, the project folder (GPT_CASE_RST, in our example) will contain another folder named "movies". Within movies, there are folders which contain results for the executed simulations. As was instructed in 3A, here we can see three folders corresponding to simulations for the three separate models. Within the first folder there are three items:<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| ''CommandLine.txt'' - This file contains the Matlab command which was used to generate the results that this sub-directory contains.<br />
<br />
''gpt_case_rst.txt'' - This file, named according to the project name, contains a copy of the interaction function which was used to produce these results.<br />
<br />
''meshes'' - This folder contains the mesh files corresponding to the stages of growth specified by the value of the Stages argument, described in 3A.<br />
|width="500px" align="center"|[[File:resultsdir.png| Directory structure of a project containing results]]<br />
|}<br />
<br />
<br />
The mesh files contain vertices information regarding the shape of the growth model at a particular stage of growth, but are not visualisable in that form. In order to '''convert these mesh files into viewable images''', we can execute the following command in Matlab:<br />
<br />
VMSreport('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Project','GPT_CASE_RST',...<br />
'Experiment','All','flattentime',572.5,'morphogen','KPAR','SNAPFIG',true);<br />
<br />
Where the ''Path'' and ''Project'' arguments have the same function as ''Path'' and ''Name'' in 3A, i.e. ''Path'' refers to the location of the project folder and ''Project'' is the name of the project folder itself.<br />
<br />
Here we can see the generated images, including the command line arguments, for one of the simulations performed in 3A. Click on a thumbnail to view a larger image.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="125px"| <small>001commandline.png</small><br> [[File:commandlineexample.png | 100px]]<br />
|width="125px"| <small>001GPT_CASE_RST_2 wild_.png</small><br>[[File:GPT_CASE_RST_example1.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_3 wild_.png</small><br>[[File:GPT_CASE_RST_example2.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_4 wild_.png</small><br>[[File:GPT_CASE_RST_example3.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_5 wild_.png</small><br>[[File:GPT_CASE_RST_example4.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_6 wild_.png</small><br>[[File:GPT_CASE_RST_example5.png | 125px]]<br />
|}<br />
<br />
<br />
'''2) - Generating images from a simulation you ran on the cluster'''<br />
<br />
NB. The results produced may not be visibly identical to those in the published literature. This is because of small, random perturbations which are applied to the initial model meshes to stop them from containing surfaces which are perfectly flat, and therefore biologically unrealistic. The results produced should be '''qualitatively but not quantitatively the same'''.<br />
<br />
'''3) - Interacting with your results using the GFtbox GUI'''<br />
<br />
='''4''' Altering the simulation parameters=<br />
<br />
One such simulation parameter is ''dt'', which is the time in seconds between iterations in the growth simulation. Large values of ''dt'' mean that fewer steps and therefore fewer calculations are required to complete a simulation. Whereas smaller values of ''dt'' mean that more steps are required, and therefore more processing time too. Therefore, a value of ''dt'' must be selected which is not so small that it is computationally unmanageable, but not so large that the observed growth is an artifact of the value of ''dt'', rather than the underlying growth model. It is necessary therefore to test a range of values for ''dt'' to ensure that the patterns of growth observed in a simulation are consistent across the range, and to find a value which is sufficient to demonstrate the model of growth and computationally efficient.<br />
<br />
'''Testing a range of ''dt''s can be achieved in several ways.''' One way is to use the ''dt'' argument when calling GFtboxCommand. This allows a single value to be tested. Another way is to make a batch of jobs, each using a different value for ''dt'', by using the 'State' argument. Lastly, a range of ''dt''s can be specified in GFtboxCommand, by using the dt argument where the values within the square brackets are the ''dt''s to simulate:<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5]);<br />
<br />
Here, we have specified '''four separate values for ''dt''''', and this means that '''four separate simulations will be run''' sequentially on your computer. As in 3B, images can then be generated for the mesh files produced, and compared to ensure consistency across the range of ''dt''s. In this figure, we can see the same model, at the same stage of growth, generated using the four values of ''dt''. Though quantitatively different, they are qualitatively the same, illustrating the suitability of the default ''dt'' value of 5.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="190px"| [[File:GPT_CASE_RST_dt0.5.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt1.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt2.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt5.png| 190px]]<br />
|}<br />
<br />
='''5''' Altering the model parameters=<br />
It is easy to see that when even a small number of range variables, mutations or ''dt''s are specified, the '''total number of simulations increases quickly'''. If many ranges are specified, then the amount of processing time becomes unmanageable on a single computer. Via ''GFtboxCommand'' and the ''ClusterMonitor'' tool, we can remotely''' run a number of simulations, in parallel, on a computing cluster'''. It is assumed that the GrowthToolbox and Matlab are installed on your cluster, and PuTTY is required on your local computer for the tools ''pscp'' (for transferring files from your computer to a cluster) and ''plink'' (for remotely executing commands on a cluster). Using the ''dt'' range argument from 4 as an example, here we add the 'Use' name argument with the value 'Cluster':<br />
<br />
GFtboxCommand('State','Start','Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5],'Use','Cluster');<br />
<br />
The 'Cluster' option for the 'Use' name argument instructs GFtboxCommand to upload the required project directory to a remote Linux server. Instead of running the simulation on your own computer, GFtboxCommand then works out how many individual simulations are specified by the command that invoked it. In this example, the only argument which will generate multiple simulations is ''dt'', of which there will be 4. Separate commands for each of these jobs are then automatically generated, each with an accompanying unique ID (The value of the ''ExpID'' name argument), and '''these are submitted as individual jobs to the cluster'''.<br />
<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.1,'ExpID','GPT_CASE_RST_1');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.5,'ExpID','GPT_CASE_RST_2');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',1,'ExpID','GPT_CASE_RST_3');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',5,'ExpID','GPT_CASE_RST_4');<br />
<br />
Once these jobs have been submitted, the ''ClusterMonitor'' tool opens and the new job batch ID is added to the list of jobs. Here, we will concentrate on three of the functions of ''ClusterMonitor'': ''Queue?'', ''Get project results'' and ''Make project pngs''. As in part 4 of this page, these functions will enable us to visualise the growth simulation results.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| Queue <br> Get project results <br> Make project pngs<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
<br />
Whilst ''ClusterMonitor'' is a useful tool for managing your cluster jobs, it is advisable to '''have at least a rudimentary understanding of and ability to use a Unix-based computer system'''. In particular, the abilities to list the contents of a folder, view the contents of a file, change the current working folder, delete, and to copy or move files or folders, are essential Unix skills for making sure everything is working as intended. It is beyond the scope of these tutorials to provide an in depth Unix tutorial (many good and simple tutorials exist on the web), but here is a short description of the Unix commands (which may be specific to our Unix-based system) that we believe to be important.</div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Running_example_models_and_using_a_cluster&diff=5378Running example models and using a cluster2011-06-23T14:17:30Z<p>JacobNewman: </p>
<hr />
<div>[[GFtbox Tutorial pages | Back to GFtbox Tutorial pages]]<br />
<br />
The purpose of these tutorials is to learn how to run the example growth simulations included in GFtbox. We will describe methods for running simulations locally (on your own computer) and remotely (on a computing cluster). It is assumed that you have already downloaded the GFtbox software and have Matlab installed.<br />
<br />
=Getting Started=<br />
<br />
The remainder of this page is split into five sub-tutorials, each building on the preceding parts.<br />
<br />
1) '''Explaining the tools'''. In these tutorials, we will be using ''GFtboxCommand'' and ''ClusterMonitor''. This section explains the purpose of these tools. <br />
<br />
2) '''Computer or cluster?''' Here, we illustrate when and why you should use a computing cluster for your growth simulations, and conversely, when you should use your desktop computer.<br />
<br />
3) '''Running a growth simulation for an example model'''. This section demonstrates how to use ''GFtboxCommand'' to run a growth simulation on your computer. The model used for the simulation is an example model included with the GFtbox.<br />
<br />
4) '''Altering the simulation parameters'''. Following on from part 3, here we show how to adjust a simulation parameter within GFtboxCommand. Specifically, we alter the value of ''dt'', the temporal resolution of the simulation, and show how it can be used to verify that the value specified in the published literature is reasonable.<br />
<br />
5) '''Altering the model parameters'''. Finally, we demonstrate how a number of model parameters can be varied by specifying a range of values for each model argument. We show how the computationally expensive task of simulating all combinations of specified ranges can be processed more efficiently if a computer cluster is used via the ''ClusterMonitor'' tool.<br />
<br />
='''1''' Explaining the tools=<br />
==GFtboxCommand==<br />
This is a command line version of the GFtbox. By command line, we mean that all program '''functions are operated via typed commands, without the GUI'''. Like GFtbox, GFtboxCommand is capable of running growth simulations of an interaction function, and allows the user to specify model and simulation parameters. Unlike GFtbox, this also allows the user to select '''ranges of values for a number of input parameters''', and will automatically spawn multiple simulations which explore the various combinations of those parameters. This can be used to evaluate the effect of various parameters on the growth of a given model.<br />
<br />
==ClusterMonitor==<br />
Provides a graphical user interface for managing simulations '''running remotely on a computer cluster'''. Specifically, it allows you to see which projects are present and running on the cluster, to retrieve the completed projects, to generate images of the simulations at specified stages of growth, and to remove projects from the cluster. If you do not intend to use a computer cluster, then you will not need to use ClusterMonitor.<br />
<br />
='''2''' Computer or cluster?=<br />
<br />
In basic terms, '''a computing cluster is effectively a network of many computer processors''' (often hundreds), centrally managed by a queuing system. When a job is submitted to a cluster, the job is sent to a processor that is not being used, or queued until one becomes available. In contrast, a typical desktop computer will contain one processor, limiting the number of tasks that can be performed at any one time. Jobs which must be run independently and sequentially on a desktop computer can be executed in parallel on a cluster, '''greatly reducing the total time required to complete all of the jobs'''. Although the exact details of your cluster might vary from those described in the rest of these tutorials, we aim to illustrate the generic processes involved in using GFtbox on a cluster. We are happy to offer assistance, where possible, to setup GFtbox for your cluster, so please contact us if you have any queries.<br />
<br />
Whilst the time savings offered by using a cluster can be significant, there is an overhead associated with returning the results to your personal computer. The total time to run 2 or more simulations on a cluster and to return the results will be less than running those simulations sequentially on one computer. Therefore, using a '''cluster is ideal for situations where you would like to run several simulations''', such as to evaluate the effect of a range of parameters on a growth model. You are not advised to use a cluster for running single simulations, or where you would like to step through a simulation and change parameters in a more interactive fashion. In such circumstances, you are advised to run GFtbox or GFtboxCommand on your own computer.<br />
<br />
The '''GFtbox GUI provides quick feedback''' about how the changes you have made to an interaction function have affected the growth simulation, as '''you can see the result of each simulation iteration as it completes'''. It is quick and easy therefore to see if you have dramatically changed the course of the growth simulation, and then to adjust the parameters according to your observations. This approach is well suited to the early stages of design, where you might wish to '''tweak some parameters to gauge whether or not they have had the desired effect'''. Or in the final stages, where you think your model is almost finalised, but where small adjustments are required. '''An alternative approach is to start many simulations''', based upon your initial model and by making intelligent choices about which parameters to explore. This allows you to '''harvest many results, to quickly and easily overview them''' in their finalised form, or to select interesting-looking simulations and examine them more closely on your desktop computer, using GFtbox.<br />
<br />
='''3''' Running a growth simulation for an example model=<br />
This tutorial is aimed at running a '''growth simulation for one of the example interaction functions''' included with the GFtbox. The purpose of this exercise is to firstly demonstrate how simulations can be invoked using GFtboxCommand, and secondly to show how to reproduce experimental results (specifically, those published in Kennaway et al (2011)) given an interaction function.<br />
<br />
Assuming Matlab is installed on your computer, and the latest GFtbox has been downloaded, you can add the GFtbox directory to Matlab's search path, which makes the toolbox accessible from any other path that you choose to work from. For a short tutorial on how to do this, please click [[Adding GFtbox to Matlab's search path | here]].<br />
<br />
Once the GFtbox is added to Matlab, you are ready to run a growth simulation using GFtboxCommand. In this example, '''the model that we will simulate is called GPT_CASE_RST'''. Results generated using this model are published in Kennaway et al (2011). By running this simulation, we can confirm the results in the published literature and investigate the suitability of the various parameters.<br />
<br />
===3A - Running a simulation on your computer===<br />
<br />
The following command can be typed into Matlab to run a simulation of the GPT_CASE_RST interaction function, which contains three growth models: R, S and T. '''Three separate simulations are run sequentially on your computer''', one for each model, each producing results corresponding to five intervals in the growth simulation.<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',[1:3]);<br />
<br />
GFtboxCommand accepts input arguments as name and value pairs, e.g. 'modelname', [1:3] or 'Use','Cluster'. The argument names entered in the example above are: ''Path'', ''Name'', ''Stages'' and ''modelname''.<br />
<br />
The optional '''''Path'' argument name refers to the location of the folder''' (or directory) on your computer where the model interaction function you wish to simulate is stored. '''''Name'', is the name of the folder itself'''. In this case, we are using the GPT_CASE_RST folder which is included in the GrowthToolbox. You may wish to copy this folder elsewhere, if you intend to make changes to the interaction function.<br />
<br />
During a growth simulation, a mesh can be generated at each time step of the simulation, which provides a visual representation of the growth of the biological tissue, given the various parameters of growth specified by the interaction function and how they have changed over time. Put another way, the mesh shows exactly what the growing tissue actually looks like. ''Stages'' refers to the points in the simulation (measured in hours) at which meshes should be generated and saved. In this example, '''five stages of growth will be written to disk'''. These values are chosen to best capture the appearance of the tissue at important stages of the tissue growth.<br />
<br />
The final argument name listed here is ''modelname''. This is a model-specific argument, and in this case the GPT_CASE_RST interaction function contains '''three separate models for plant growth''', allowing the desired model to be a selected. Here, the value [1:3] is specified, which is evaluated in the same way as entering [1 2 3]. This instructs GFtboxCommand to run three separate simulations, one for each of the growth models contained in the interaction function.<br />
<br />
The function of every permissible argument is given by keying the following command into Matlab:<br />
<br />
help GFtboxCommand<br />
<br />
===3B - Getting results from a completed simulation===<br />
<br />
'''1) - Generating images from a simulation you ran on your computer'''<br />
<br />
Once a growth simulation is completed, the project folder (GPT_CASE_RST, in our example) will contain another folder named "movies". Within movies, there are folders which contain results for the executed simulations. As was instructed in 3A, here we can see three folders corresponding to simulations for the three separate models. Within the first folder there are three items:<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| ''CommandLine.txt'' - This file contains the Matlab command which was used to generate the results that this sub-directory contains.<br />
<br />
''gpt_case_rst.txt'' - This file, named according to the project name, contains a copy of the interaction function which was used to produce these results.<br />
<br />
''meshes'' - This folder contains the mesh files corresponding to the stages of growth specified by the value of the Stages argument, described in 3A.<br />
|width="500px" align="center"|[[File:resultsdir.png| Directory structure of a project containing results]]<br />
|}<br />
<br />
<br />
The mesh files contain vertices information regarding the shape of the growth model at a particular stage of growth, but are not visualisable in that form. In order to '''convert these mesh files into viewable images''', we can execute the following command in Matlab:<br />
<br />
VMSreport('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Project','GPT_CASE_RST',...<br />
'Experiment','All','flattentime',572.5,'morphogen','KPAR','SNAPFIG',true);<br />
<br />
Where the ''Path'' and ''Project'' arguments have the same function as ''Path'' and ''Name'' in 3A, i.e. ''Path'' refers to the location of the project folder and ''Project'' is the name of the project folder itself.<br />
<br />
Here we can see the generated images, including the command line arguments, for one of the simulations performed in 3A. Click on a thumbnail to view a larger image.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="125px"| <small>001commandline.png</small><br> [[File:commandlineexample.png | 100px]]<br />
|width="125px"| <small>001GPT_CASE_RST_2 wild_.png</small><br>[[File:GPT_CASE_RST_example1.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_3 wild_.png</small><br>[[File:GPT_CASE_RST_example2.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_4 wild_.png</small><br>[[File:GPT_CASE_RST_example3.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_5 wild_.png</small><br>[[File:GPT_CASE_RST_example4.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_6 wild_.png</small><br>[[File:GPT_CASE_RST_example5.png | 125px]]<br />
|}<br />
<br />
<br />
'''2) - Generating images from a simulation you ran on the cluster'''<br />
<br />
NB. The results produced may not be visibly identical to those in the published literature. This is because of small, random perturbations which are applied to the initial model meshes to stop them from containing surfaces which are perfectly flat, and therefore biologically unrealistic. The results produced should be '''qualitatively but not quantitatively the same'''.<br />
<br />
'''3) - Interacting with your results using the GFtbox GUI'''<br />
<br />
='''4''' Altering the simulation parameters=<br />
<br />
One such simulation parameter is ''dt'', which is the time in seconds between iterations in the growth simulation. Large values of ''dt'' mean that fewer steps and therefore fewer calculations are required to complete a simulation. Whereas smaller values of ''dt'' mean that more steps are required, and therefore more processing time too. Therefore, a value of ''dt'' must be selected which is not so small that it is computationally unmanageable, but not so large that the observed growth is an artifact of the value of ''dt'', rather than the underlying growth model. It is necessary therefore to test a range of values for ''dt'' to ensure that the patterns of growth observed in a simulation are consistent across the range, and to find a value which is sufficient to demonstrate the model of growth and computationally efficient.<br />
<br />
'''Testing a range of ''dt''s can be achieved in several ways.''' One way is to use the ''dt'' argument when calling GFtboxCommand. This allows a single value to be tested. Another way is to make a batch of jobs, each using a different value for ''dt'', by using the 'State' argument. Lastly, a range of ''dt''s can be specified in GFtboxCommand, by using the dt argument where the values within the square brackets are the ''dt''s to simulate:<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5]);<br />
<br />
Here, we have specified '''four separate values for ''dt''''', and this means that '''four separate simulations will be run''' sequentially on your computer. As in 3B, images can then be generated for the mesh files produced, and compared to ensure consistency across the range of ''dt''s. In this figure, we can see the same model, at the same stage of growth, generated using the four values of ''dt''. Though quantitatively different, they are qualitatively the same, illustrating the suitability of the default ''dt'' value of 5.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="190px"| [[File:GPT_CASE_RST_dt0.5.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt1.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt2.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt5.png| 190px]]<br />
|}<br />
<br />
='''5''' Altering the model parameters=<br />
It is easy to see that when even a small number of range variables, mutations or ''dt''s are specified, the '''total number of simulations increases quickly'''. If many ranges are specified, then the amount of processing time becomes unmanageable on a single computer. Via ''GFtboxCommand'' and the ''ClusterMonitor'' tool, we can remotely''' run a number of simulations, in parallel, on a computing cluster'''. It is assumed that the GrowthToolbox and Matlab are installed on your cluster, and PuTTY is required on your local computer for the tools ''pscp'' (for transferring files from your computer to a cluster) and ''plink'' (for remotely executing commands on a cluster). Using the ''dt'' range argument from 4 as an example, here we add the 'Use' name argument with the value 'Cluster':<br />
<br />
GFtboxCommand('State','Start','Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5],'Use','Cluster');<br />
<br />
The 'Cluster' option for the 'Use' name argument instructs GFtboxCommand to upload the required project directory to a remote Linux server. Instead of running the simulation on your own computer, GFtboxCommand then works out how many individual simulations are specified by the command that invoked it. In this example, the only argument which will generate multiple simulations is ''dt'', of which there will be 4. Separate commands for each of these jobs are then automatically generated, each with an accompanying unique ID (The value of the ''ExpID'' name argument), and '''these are submitted as individual jobs to the cluster'''.<br />
<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.1,'ExpID','GPT_CASE_RST_1');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.5,'ExpID','GPT_CASE_RST_2');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',1,'ExpID','GPT_CASE_RST_3');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',5,'ExpID','GPT_CASE_RST_4');<br />
<br />
Once these jobs have been submitted, the ''ClusterMonitor'' tool opens and the new job batch ID is added to the list of jobs. Here, we will concentrate on three of the functions of ''ClusterMonitor'': ''Queue?'', ''Get project results'' and ''Make project pngs''. As in part 4 of this page, these functions will enable us to visualise the growth simulation results.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="Queue <br> Get project results <br> Make project pngs"<br />
|width="200px"| test<br />
|width="600px"| [[File:clustermonitor.png| 600px]]<br />
|}<br />
<br />
Whilst ''ClusterMonitor'' is a useful tool for managing your cluster jobs, it is advisable to '''have at least a rudimentary understanding of and ability to use a Unix-based computer system'''. In particular, the abilities to list the contents of a folder, view the contents of a file, change the current working folder, delete, and to copy or move files or folders, are essential Unix skills for making sure everything is working as intended. It is beyond the scope of these tutorials to provide an in depth Unix tutorial (many good and simple tutorials exist on the web), but here is a short description of the Unix commands (which may be specific to our Unix-based system) that we believe to be important.</div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=File:Clustermonitor.png&diff=5513File:Clustermonitor.png2011-06-23T14:14:32Z<p>JacobNewman: </p>
<hr />
<div></div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Running_example_models_and_using_a_cluster&diff=5418Running example models and using a cluster2011-06-23T13:38:10Z<p>JacobNewman: </p>
<hr />
<div>[[GFtbox Tutorial pages | Back to GFtbox Tutorial pages]]<br />
<br />
The purpose of these tutorials is to learn how to run the example growth simulations included in GFtbox. We will describe methods for running simulations locally (on your own computer) and remotely (on a computing cluster). It is assumed that you have already downloaded the GFtbox software and have Matlab installed.<br />
<br />
=Getting Started=<br />
<br />
The remainder of this page is split into five sub-tutorials, each building on the preceding parts.<br />
<br />
1) '''Explaining the tools'''. In these tutorials, we will be using ''GFtboxCommand'' and ''ClusterMonitor''. This section explains the purpose of these tools. <br />
<br />
2) '''Computer or cluster?''' Here, we illustrate when and why you should use a computing cluster for your growth simulations, and conversely, when you should use your desktop computer.<br />
<br />
3) '''Running a growth simulation for an example model'''. This section demonstrates how to use ''GFtboxCommand'' to run a growth simulation on your computer. The model used for the simulation is an example model included with the GFtbox.<br />
<br />
4) '''Altering the simulation parameters'''. Following on from part 3, here we show how to adjust a simulation parameter within GFtboxCommand. Specifically, we alter the value of ''dt'', the temporal resolution of the simulation, and show how it can be used to verify that the value specified in the published literature is reasonable.<br />
<br />
5) '''Altering the model parameters'''. Finally, we demonstrate how a number of model parameters can be varied by specifying a range of values for each model argument. We show how the computationally expensive task of simulating all combinations of specified ranges can be processed more efficiently if a computer cluster is used via the ''ClusterMonitor'' tool.<br />
<br />
='''1''' Explaining the tools=<br />
==GFtboxCommand==<br />
This is a command line version of the GFtbox. By command line, we mean that all program '''functions are operated via typed commands, without the GUI'''. Like GFtbox, GFtboxCommand is capable of running growth simulations of an interaction function, and allows the user to specify model and simulation parameters. Unlike GFtbox, this also allows the user to select '''ranges of values for a number of input parameters''', and will automatically spawn multiple simulations which explore the various combinations of those parameters. This can be used to evaluate the effect of various parameters on the growth of a given model.<br />
<br />
==ClusterMonitor==<br />
Provides a graphical user interface for managing simulations '''running remotely on a computer cluster'''. Specifically, it allows you to see which projects are present and running on the cluster, to retrieve the completed projects, to generate images of the simulations at specified stages of growth, and to remove projects from the cluster. If you do not intend to use a computer cluster, then you will not need to use ClusterMonitor.<br />
<br />
='''2''' Computer or cluster?=<br />
<br />
In basic terms, '''a computing cluster is effectively a network of many computer processors''' (often hundreds), centrally managed by a queuing system. When a job is submitted to a cluster, the job is sent to a processor that is not being used, or queued until one becomes available. In contrast, a typical desktop computer will contain one processor, limiting the number of tasks that can be performed at any one time. Jobs which must be run independently and sequentially on a desktop computer can be executed in parallel on a cluster, '''greatly reducing the total time required to complete all of the jobs'''. Although the exact details of your cluster might vary from those described in the rest of these tutorials, we aim to illustrate the generic processes involved in using GFtbox on a cluster. We are happy to offer assistance, where possible, to setup GFtbox for your cluster, so please contact us if you have any queries.<br />
<br />
Whilst the time savings offered by using a cluster can be significant, there is an overhead associated with returning the results to your personal computer. The total time to run 2 or more simulations on a cluster and to return the results will be less than running those simulations sequentially on one computer. Therefore, using a '''cluster is ideal for situations where you would like to run several simulations''', such as to evaluate the effect of a range of parameters on a growth model. You are not advised to use a cluster for running single simulations, or where you would like to step through a simulation and change parameters in a more interactive fashion. In such circumstances, you are advised to run GFtbox or GFtboxCommand on your own computer.<br />
<br />
The '''GFtbox GUI provides quick feedback''' about how the changes you have made to an interaction function have affected the growth simulation, as '''you can see the result of each simulation iteration as it completes'''. It is quick and easy therefore to see if you have dramatically changed the course of the growth simulation, and then to adjust the parameters according to your observations. This approach is well suited to the early stages of design, where you might wish to '''tweak some parameters to gauge whether or not they have had the desired effect'''. Or in the final stages, where you think your model is almost finalised, but where small adjustments are required. '''An alternative approach is to start many simulations''', based upon your initial model and by making intelligent choices about which parameters to explore. This allows you to '''harvest many results, to quickly and easily overview them''' in their finalised form, or to select interesting-looking simulations and examine them more closely on your desktop computer, using GFtbox.<br />
<br />
='''3''' Running a growth simulation for an example model=<br />
This tutorial is aimed at running a '''growth simulation for one of the example interaction functions''' included with the GFtbox. The purpose of this exercise is to firstly demonstrate how simulations can be invoked using GFtboxCommand, and secondly to show how to reproduce experimental results (specifically, those published in Kennaway et al (2011)) given an interaction function.<br />
<br />
Assuming Matlab is installed on your computer, and the latest GFtbox has been downloaded, you can add the GFtbox directory to Matlab's search path, which makes the toolbox accessible from any other path that you choose to work from. For a short tutorial on how to do this, please click [[Adding GFtbox to Matlab's search path | here]].<br />
<br />
Once the GFtbox is added to Matlab, you are ready to run a growth simulation using GFtboxCommand. In this example, '''the model that we will simulate is called GPT_CASE_RST'''. Results generated using this model are published in Kennaway et al (2011). By running this simulation, we can confirm the results in the published literature and investigate the suitability of the various parameters.<br />
<br />
===3A - Running a simulation on your computer===<br />
<br />
The following command can be typed into Matlab to run a simulation of the GPT_CASE_RST interaction function, which contains three growth models: R, S and T. '''Three separate simulations are run sequentially on your computer''', one for each model, each producing results corresponding to five intervals in the growth simulation.<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',[1:3]);<br />
<br />
GFtboxCommand accepts input arguments as name and value pairs, e.g. 'modelname', [1:3] or 'Use','Cluster'. The argument names entered in the example above are: ''Path'', ''Name'', ''Stages'' and ''modelname''.<br />
<br />
The optional '''''Path'' argument name refers to the location of the folder''' (or directory) on your computer where the model interaction function you wish to simulate is stored. '''''Name'', is the name of the folder itself'''. In this case, we are using the GPT_CASE_RST folder which is included in the GrowthToolbox. You may wish to copy this folder elsewhere, if you intend to make changes to the interaction function.<br />
<br />
During a growth simulation, a mesh can be generated at each time step of the simulation, which provides a visual representation of the growth of the biological tissue, given the various parameters of growth specified by the interaction function and how they have changed over time. Put another way, the mesh shows exactly what the growing tissue actually looks like. ''Stages'' refers to the points in the simulation (measured in hours) at which meshes should be generated and saved. In this example, '''five stages of growth will be written to disk'''. These values are chosen to best capture the appearance of the tissue at important stages of the tissue growth.<br />
<br />
The final argument name listed here is ''modelname''. This is a model-specific argument, and in this case the GPT_CASE_RST interaction function contains '''three separate models for plant growth''', allowing the desired model to be a selected. Here, the value [1:3] is specified, which is evaluated in the same way as entering [1 2 3]. This instructs GFtboxCommand to run three separate simulations, one for each of the growth models contained in the interaction function.<br />
<br />
The function of every permissible argument is given by keying the following command into Matlab:<br />
<br />
help GFtboxCommand<br />
<br />
===3B - Getting results from a completed simulation===<br />
<br />
'''1) - Generating images from a simulation you ran on your computer'''<br />
<br />
Once a growth simulation is completed, the project folder (GPT_CASE_RST, in our example) will contain another folder named "movies". Within movies, there are folders which contain results for the executed simulations. As was instructed in 3A, here we can see three folders corresponding to simulations for the three separate models. Within the first folder there are three items:<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| ''CommandLine.txt'' - This file contains the Matlab command which was used to generate the results that this sub-directory contains.<br />
<br />
''gpt_case_rst.txt'' - This file, named according to the project name, contains a copy of the interaction function which was used to produce these results.<br />
<br />
''meshes'' - This folder contains the mesh files corresponding to the stages of growth specified by the value of the Stages argument, described in 3A.<br />
|width="500px" align="center"|[[File:resultsdir.png| Directory structure of a project containing results]]<br />
|}<br />
<br />
<br />
The mesh files contain vertices information regarding the shape of the growth model at a particular stage of growth, but are not visualisable in that form. In order to '''convert these mesh files into viewable images''', we can execute the following command in Matlab:<br />
<br />
VMSreport('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Project','GPT_CASE_RST',...<br />
'Experiment','All','flattentime',572.5,'morphogen','KPAR','SNAPFIG',true);<br />
<br />
Where the ''Path'' and ''Project'' arguments have the same function as ''Path'' and ''Name'' in 3A, i.e. ''Path'' refers to the location of the project folder and ''Project'' is the name of the project folder itself.<br />
<br />
Here we can see the generated images, including the command line arguments, for one of the simulations performed in 3A. Click on a thumbnail to view a larger image.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="125px"| <small>001commandline.png</small><br> [[File:commandlineexample.png | 100px]]<br />
|width="125px"| <small>001GPT_CASE_RST_2 wild_.png</small><br>[[File:GPT_CASE_RST_example1.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_3 wild_.png</small><br>[[File:GPT_CASE_RST_example2.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_4 wild_.png</small><br>[[File:GPT_CASE_RST_example3.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_5 wild_.png</small><br>[[File:GPT_CASE_RST_example4.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_6 wild_.png</small><br>[[File:GPT_CASE_RST_example5.png | 125px]]<br />
|}<br />
<br />
<br />
'''2) - Generating images from a simulation you ran on the cluster'''<br />
<br />
NB. The results produced may not be visibly identical to those in the published literature. This is because of small, random perturbations which are applied to the initial model meshes to stop them from containing surfaces which are perfectly flat, and therefore biologically unrealistic. The results produced should be '''qualitatively but not quantitatively the same'''.<br />
<br />
'''3) - Interacting with your results using the GFtbox GUI'''<br />
<br />
='''4''' Altering the simulation parameters=<br />
<br />
One such simulation parameter is ''dt'', which is the time in seconds between iterations in the growth simulation. Large values of ''dt'' mean that fewer steps and therefore fewer calculations are required to complete a simulation. Whereas smaller values of ''dt'' mean that more steps are required, and therefore more processing time too. Therefore, a value of ''dt'' must be selected which is not so small that it is computationally unmanageable, but not so large that the observed growth is an artifact of the value of ''dt'', rather than the underlying growth model. It is necessary therefore to test a range of values for ''dt'' to ensure that the patterns of growth observed in a simulation are consistent across the range, and to find a value which is sufficient to demonstrate the model of growth and computationally efficient.<br />
<br />
'''Testing a range of ''dt''s can be achieved in several ways.''' One way is to use the ''dt'' argument when calling GFtboxCommand. This allows a single value to be tested. Another way is to make a batch of jobs, each using a different value for ''dt'', by using the 'State' argument. Lastly, a range of ''dt''s can be specified in GFtboxCommand, by using the dt argument where the values within the square brackets are the ''dt''s to simulate:<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5]);<br />
<br />
Here, we have specified '''four separate values for ''dt''''', and this means that '''four separate simulations will be run''' sequentially on your computer. As in 3B, images can then be generated for the mesh files produced, and compared to ensure consistency across the range of ''dt''s. In this figure, we can see the same model, at the same stage of growth, generated using the four values of ''dt''. Though quantitatively different, they are qualitatively the same, illustrating the suitability of the default ''dt'' value of 5.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="190px"| [[File:GPT_CASE_RST_dt0.5.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt1.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt2.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt5.png| 190px]]<br />
|}<br />
<br />
='''5''' Altering the model parameters=<br />
It is easy to see that when even a small number of range variables, mutations or ''dt''s are specified, the '''total number of simulations increases quickly'''. If many ranges are specified, then the amount of processing time becomes unmanageable on a single computer. Via ''GFtboxCommand'' and the ''ClusterMonitor'' tool, we can remotely''' run a number of simulations, in parallel, on a computing cluster'''. It is assumed that the GrowthToolbox and Matlab are installed on your cluster, and PuTTY is required on your local computer for the tools ''pscp'' (for transferring files from your computer to a cluster) and ''plink'' (for remotely executing commands on a cluster). Using the ''dt'' range argument from 4 as an example, here we add the 'Use' name argument with the value 'Cluster':<br />
<br />
GFtboxCommand('State','Start','Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5],'Use','Cluster');<br />
<br />
The 'Cluster' option for the 'Use' name argument instructs GFtboxCommand to upload the required project directory to a remote Linux server. Instead of running the simulation on your own computer, GFtboxCommand then works out how many individual simulations are specified by the command that invoked it. In this example, the only argument which will generate multiple simulations is ''dt'', of which there will be 4. Separate commands for each of these jobs are then automatically generated, each with an accompanying unique ID (The value of the ''ExpID'' name argument), and '''these are submitted as individual jobs to the cluster'''.<br />
<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.1,'ExpID','GPT_CASE_RST_1');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.5,'ExpID','GPT_CASE_RST_2');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',1,'ExpID','GPT_CASE_RST_3');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',5,'ExpID','GPT_CASE_RST_4');<br />
<br />
Once these jobs have been submitted, the ''ClusterMonitor'' tool opens and the new job batch ID is added to the list of jobs. Here, we will concentrate on three of the functions of ''ClusterMonitor'': ''Queue?'', ''Get project results'' and ''Make project pngs''. As in part 4 of this page, these functions will enable us to visualise the growth simulation results.<br />
<br />
Whilst ''ClusterMonitor'' is a useful tool for managing your cluster jobs, it is advisable to '''have at least a rudimentary understanding of and ability to use a Unix-based computer system'''. In particular, the abilities to list the contents of a folder, view the contents of a file, change the current working folder, delete, and to copy or move files or folders, are essential Unix skills for making sure everything is working as intended. It is beyond the scope of these tutorials to provide an in depth Unix tutorial (many good and simple tutorials exist on the web), but here is a short description of the Unix commands (which may be specific to our Unix-based system) that we believe to be important.</div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Running_example_models_and_using_a_cluster&diff=5377Running example models and using a cluster2011-06-23T13:36:52Z<p>JacobNewman: </p>
<hr />
<div>[[GFtbox Tutorial pages | Back to GFtbox Tutorial pages]]<br />
<br />
The purpose of these tutorials is to learn how to run the example growth simulations included in GFtbox. We will describe methods for running simulations locally (on your own computer) and remotely (on a computing cluster). It is assumed that you have already downloaded the GFtbox software and have Matlab installed.<br />
<br />
=Getting Started=<br />
<br />
The remainder of this page is split into five sub-tutorials, each building on the preceding parts.<br />
<br />
1) '''Explaining the tools'''. In these tutorials, we will be using ''GFtboxCommand'' and ''ClusterMonitor''. This section explains the purpose of these tools. <br />
<br />
2) '''Computer or cluster?''' Here, we illustrate when and why you should use a computing cluster for your growth simulations, and conversely, when you should use your desktop computer.<br />
<br />
3) '''Running a growth simulation for an example model'''. This section demonstrates how to use ''GFtboxCommand'' to run a growth simulation on your computer. The model used for the simulation is an example model included with the GFtbox.<br />
<br />
4) '''Altering the simulation parameters'''. Following on from part 3, here we show how to adjust a simulation parameter within GFtboxCommand. Specifically, we alter the value of ''dt'', the temporal resolution of the simulation, and show how it can be used to verify that the value specified in the published literature is reasonable.<br />
<br />
5) '''Altering the model parameters'''. Finally, we demonstrate how a number of model parameters can be varied by specifying a range of values for each model argument. We show how the computationally expensive task of simulating all combinations of specified ranges can be processed more efficiently if a computer cluster is used via the ''ClusterMonitor'' tool.<br />
<br />
='''1''' Explaining the tools=<br />
==GFtboxCommand==<br />
This is a command line version of the GFtbox. By command line, we mean that all program '''functions are operated via typed commands, without the GUI'''. Like GFtbox, GFtboxCommand is capable of running growth simulations of an interaction function, and allows the user to specify model and simulation parameters. Unlike GFtbox, this also allows the user to select '''ranges of values for a number of input parameters''', and will automatically spawn multiple simulations which explore the various combinations of those parameters. This can be used to evaluate the effect of various parameters on the growth of a given model.<br />
<br />
==ClusterMonitor==<br />
Provides a graphical user interface for managing simulations '''running remotely on a computer cluster'''. Specifically, it allows you to see which projects are present and running on the cluster, to retrieve the completed projects, to generate images of the simulations at specified stages of growth, and to remove projects from the cluster. If you do not intend to use a computer cluster, then you will not need to use ClusterMonitor.<br />
<br />
='''2''' Computer or cluster?=<br />
<br />
In basic terms, '''a computing cluster is effectively a network of many computer processors''' (often hundreds), centrally managed by a queuing system. When a job is submitted to a cluster, the job is sent to a processor that is not being used, or queued until one becomes available. In contrast, a typical desktop computer will contain one processor, limiting the number of tasks that can be performed at any one time. Jobs which must be run independently and sequentially on a desktop computer can be executed in parallel on a cluster, '''greatly reducing the total time required to complete all of the jobs'''. Although the exact details of your cluster might vary from those described in the rest of these tutorials, we aim to illustrate the generic processes involved in using GFtbox on a cluster. We are happy to offer assistance, where possible, to setup GFtbox for your cluster, so please contact us if you have any queries.<br />
<br />
Whilst the time savings offered by using a cluster can be significant, there is an overhead associated with returning the results to your personal computer. The total time to run 2 or more simulations on a cluster and to return the results will be less than running those simulations sequentially on one computer. Therefore, using a '''cluster is ideal for situations where you would like to run several simulations''', such as to evaluate the effect of a range of parameters on a growth model. You are not advised to use a cluster for running single simulations, or where you would like to step through a simulation and change parameters in a more interactive fashion. In such circumstances, you are advised to run GFtbox or GFtboxCommand on your own computer.<br />
<br />
The '''GFtbox GUI provides quick feedback''' about how the changes you have made to an interaction function have affected the growth simulation, as '''you can see the result of each simulation iteration as it completes'''. It is quick and easy therefore to see if you have dramatically changed the course of the growth simulation, and then to adjust the parameters according to your observations. This approach is well suited to the early stages of design, where you might wish to '''tweak some parameters to gauge whether or not they have had the desired effect'''. Or in the final stages, where you think your model is almost finalised, but where small adjustments are required. '''An alternative approach is to start many simulations''', based upon your initial model and by making intelligent choices about which parameters to explore. This allows you to '''harvest many results, to quickly and easily overview them''' in their finalised form, or to select interesting-looking simulations and examine them more closely on your desktop computer, using GFtbox.<br />
<br />
='''3''' Running a growth simulation for an example model=<br />
This tutorial is aimed at running a '''growth simulation for one of the example interaction functions''' included with the GFtbox. The purpose of this exercise is to firstly demonstrate how simulations can be invoked using GFtboxCommand, and secondly to show how to reproduce experimental results (specifically, those published in Kennaway et al (2011)) given an interaction function.<br />
<br />
Assuming Matlab is installed on your computer, and the latest GFtbox has been downloaded, you can add the GFtbox directory to Matlab's search path, which makes the toolbox accessible from any other path that you choose to work from. For a short tutorial on how to do this, please click [[Adding GFtbox to Matlab's search path | here]].<br />
<br />
Once the GFtbox is added to Matlab, you are ready to run a growth simulation using GFtboxCommand. In this example, '''the model that we will simulate is called GPT_CASE_RST'''. Results generated using this model are published in Kennaway et al (2011). By running this simulation, we can confirm the results in the published literature and investigate the suitability of the various parameters.<br />
<br />
===3A - Running a simulation on your computer===<br />
<br />
The following command can be typed into Matlab to run a simulation of the GPT_CASE_RST interaction function, which contains three growth models: R, S and T. '''Three separate simulations are run sequentially on your computer''', one for each model, each producing results corresponding to five intervals in the growth simulation.<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',[1:3]);<br />
<br />
GFtboxCommand accepts input arguments as name and value pairs, e.g. 'modelname', [1:3] or 'Use','Cluster'. The argument names entered in the example above are: ''Path'', ''Name'', ''Stages'' and ''modelname''.<br />
<br />
The optional '''''Path'' argument name refers to the location of the folder''' (or directory) on your computer where the model interaction function you wish to simulate is stored. '''''Name'', is the name of the folder itself'''. In this case, we are using the GPT_CASE_RST folder which is included in the GrowthToolbox. You may wish to copy this folder elsewhere, if you intend to make changes to the interaction function.<br />
<br />
During a growth simulation, a mesh can be generated at each time step of the simulation, which provides a visual representation of the growth of the biological tissue, given the various parameters of growth specified by the interaction function and how they have changed over time. Put another way, the mesh shows exactly what the growing tissue actually looks like. ''Stages'' refers to the points in the simulation (measured in hours) at which meshes should be generated and saved. In this example, '''five stages of growth will be written to disk'''. These values are chosen to best capture the appearance of the tissue at important stages of the tissue growth.<br />
<br />
The final argument name listed here is ''modelname''. This is a model-specific argument, and in this case the GPT_CASE_RST interaction function contains '''three separate models for plant growth''', allowing the desired model to be a selected. Here, the value [1:3] is specified, which is evaluated in the same way as entering [1 2 3]. This instructs GFtboxCommand to run three separate simulations, one for each of the growth models contained in the interaction function.<br />
<br />
The function of every permissible argument is given by keying the following command into Matlab:<br />
<br />
help GFtboxCommand<br />
<br />
===3B - Getting results from a completed simulation===<br />
<br />
'''1) - Generating images from a simulation you ran on your computer'''<br />
<br />
Once a growth simulation is completed, the project folder (GPT_CASE_RST, in our example) will contain another folder named "movies". Within movies, there are folders which contain results for the executed simulations. As was instructed in 3A, here we can see three folders corresponding to simulations for the three separate models. Within the first folder there are three items:<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| ''CommandLine.txt'' - This file contains the Matlab command which was used to generate the results that this sub-directory contains.<br />
<br />
''gpt_case_rst.txt'' - This file, named according to the project name, contains a copy of the interaction function which was used to produce these results.<br />
<br />
''meshes'' - This folder contains the mesh files corresponding to the stages of growth specified by the value of the Stages argument, described in 3A.<br />
|width="500px" align="center"|[[File:resultsdir.png| Directory structure of a project containing results]]<br />
|}<br />
<br />
<br />
The mesh files contain vertices information regarding the shape of the growth model at a particular stage of growth, but are not visualisable in that form. In order to '''convert these mesh files into viewable images''', we can execute the following command in Matlab:<br />
<br />
VMSreport('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Project','GPT_CASE_RST',...<br />
'Experiment','All','flattentime',572.5,'morphogen','KPAR','SNAPFIG',true);<br />
<br />
Where the ''Path'' and ''Project'' arguments have the same function as ''Path'' and ''Name'' in 3A, i.e. ''Path'' refers to the location of the project folder and ''Project'' is the name of the project folder itself.<br />
<br />
Here we can see the generated images, including the command line arguments, for one of the simulations performed in 3A. Click on a thumbnail to view a larger image.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="125px"| <small>001commandline.png</small><br> [[File:commandlineexample.png | 100px]]<br />
|width="125px"| <small>001GPT_CASE_RST_2 wild_.png</small><br>[[File:GPT_CASE_RST_example1.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_3 wild_.png</small><br>[[File:GPT_CASE_RST_example2.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_4 wild_.png</small><br>[[File:GPT_CASE_RST_example3.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_5 wild_.png</small><br>[[File:GPT_CASE_RST_example4.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_6 wild_.png</small><br>[[File:GPT_CASE_RST_example5.png | 125px]]<br />
|}<br />
<br />
<br />
'''2) - Generating images from a simulation you ran on the cluster'''<br />
<br />
NB. The results produced may not be visibly identical to those in the published literature. This is because of small, random perturbations which are applied to the initial model meshes to stop them from containing surfaces which are perfectly flat, and therefore biologically unrealistic. The results produced should be '''qualitatively but not quantitatively the same'''.<br />
<br />
'''3) - Interacting with your results using the GFtbox GUI'''<br />
<br />
='''4''' Altering the simulation parameters=<br />
<br />
One such simulation parameter is ''dt'', which is the time in seconds between iterations in the growth simulation. Large values of ''dt'' mean that fewer steps and therefore fewer calculations are required to complete a simulation. Whereas smaller values of ''dt'' mean that more steps are required, and therefore more processing time too. Therefore, a value of ''dt'' must be selected which is not so small that it is computationally unmanageable, but not so large that the observed growth is an artifact of the value of ''dt'', rather than the underlying growth model. It is necessary therefore to test a range of values for ''dt'' to ensure that the patterns of growth observed in a simulation are consistent across the range, and to find a value which is sufficient to demonstrate the model of growth and computationally efficient.<br />
<br />
'''Testing a range of ''dt''s can be achieved in several ways.''' One way is to use the ''dt'' argument when calling GFtboxCommand. This allows a single value to be tested. Another way is to make a batch of jobs, each using a different value for ''dt'', by using the 'State' argument. Lastly, a range of ''dt''s can be specified in GFtboxCommand, by using the dt argument where the values within the square brackets are the ''dt''s to simulate:<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5]);<br />
<br />
Here, we have specified '''four separate values for ''dt''''', and this means that '''four separate simulations will be run''' sequentially on your computer. As in 3B, images can then be generated for the mesh files produced, and compared to ensure consistency across the range of ''dt''s. In this figure, we can see the same model, at the same stage of growth, generated using the four values of ''dt''. Though quantitatively different, they are qualitatively the same, illustrating the suitability of the default ''dt'' value of 5.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="190px"| [[File:GPT_CASE_RST_dt0.5.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt1.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt2.png| 190px]]<br />
|width="190px"| [[File:GPT_CASE_RST_dt5.png| 190px]]<br />
|}<br />
<br />
='''5''' Altering the model parameters=<br />
It is easy to see that when even a small number of range variables, mutations or ''dt''s are specified, the '''total number of simulations increases quickly'''. If many ranges are specified, then the amount of processing time becomes unmanageable on a single computer. Via ''GFtboxCommand'' and the ''ClusterMonitor'' tool, we can remotely''' run a number of simulations, in parallel, on a computing cluster'''. It is assumed that the GrowthToolbox and Matlab are installed on your cluster, and PuTTY is required on your local computer for the tools ''pscp'' (for transferring files from your computer to a cluster) and ''plink'' (for remotely executing commands on a cluster). Using the ''dt'' range argument from 3 as an example, here we add the 'Use' name argument with the value 'Cluster':<br />
<br />
GFtboxCommand('State','Start','Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5],'Use','Cluster');<br />
<br />
The 'Cluster' option for the 'Use' name argument instructs GFtboxCommand to upload the required project directory to a remote Linux server. Instead of running the simulation on your own computer, GFtboxCommand then works out how many individual simulations are specified by the command that invoked it. In this example, the only argument which will generate multiple simulations is ''dt'', of which there will be 4. Separate commands for each of these jobs are then automatically generated, each with an accompanying unique ID (The value of the ''ExpID'' name argument), and '''these are submitted as individual jobs to the cluster'''.<br />
<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.1,'ExpID','GPT_CASE_RST_1');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.5,'ExpID','GPT_CASE_RST_2');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',1,'ExpID','GPT_CASE_RST_3');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',5,'ExpID','GPT_CASE_RST_4');<br />
<br />
Once these jobs have been submitted, the ''ClusterMonitor'' tool opens and the new job batch ID is added to the list of jobs. Here, we will concentrate on three of the functions of ''ClusterMonitor'': ''Queue?'', ''Get project results'' and ''Make project pngs''. As in part 3 of this page, these functions will enable us to visualise the growth simulation results.<br />
<br />
Whilst ''ClusterMonitor'' is a useful tool for managing your cluster jobs, it is advisable to '''have at least a rudimentary understanding of and ability to use a Unix-based computer system'''. In particular, the abilities to list the contents of a folder, view the contents of a file, change the current working folder, delete, and to copy or move files or folders, are essential Unix skills for making sure everything is working as intended. It is beyond the scope of these tutorials to provide an in depth Unix tutorial (many good and simple tutorials exist on the web), but here is a short description of the Unix commands (which may be specific to our Unix-based system) that we believe to be important.</div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Running_example_models_and_using_a_cluster&diff=5376Running example models and using a cluster2011-06-23T13:35:03Z<p>JacobNewman: </p>
<hr />
<div>[[GFtbox Tutorial pages | Back to GFtbox Tutorial pages]]<br />
<br />
The purpose of these tutorials is to learn how to run the example growth simulations included in GFtbox. We will describe methods for running simulations locally (on your own computer) and remotely (on a computing cluster). It is assumed that you have already downloaded the GFtbox software and have Matlab installed.<br />
<br />
=Getting Started=<br />
<br />
The remainder of this page is split into five sub-tutorials, each building on the preceding parts.<br />
<br />
1) '''Explaining the tools'''. In these tutorials, we will be using ''GFtboxCommand'' and ''ClusterMonitor''. This section explains the purpose of these tools. <br />
<br />
2) '''Computer or cluster?''' Here, we illustrate when and why you should use a computing cluster for your growth simulations, and conversely, when you should use your desktop computer.<br />
<br />
3) '''Running a growth simulation for an example model'''. This section demonstrates how to use ''GFtboxCommand'' to run a growth simulation on your computer. The model used for the simulation is an example model included with the GFtbox.<br />
<br />
4) '''Altering the simulation parameters'''. Following on from part 3, here we show how to adjust a simulation parameter within GFtboxCommand. Specifically, we alter the value of ''dt'', the temporal resolution of the simulation, and show how it can be used to verify that the value specified in the published literature is reasonable.<br />
<br />
5) '''Altering the model parameters'''. Finally, we demonstrate how a number of model parameters can be varied by specifying a range of values for each model argument. We show how the computationally expensive task of simulating all combinations of specified ranges can be processed more efficiently if a computer cluster is used via the ''ClusterMonitor'' tool.<br />
<br />
='''1''' Explaining the tools=<br />
==GFtboxCommand==<br />
This is a command line version of the GFtbox. By command line, we mean that all program '''functions are operated via typed commands, without the GUI'''. Like GFtbox, GFtboxCommand is capable of running growth simulations of an interaction function, and allows the user to specify model and simulation parameters. Unlike GFtbox, this also allows the user to select '''ranges of values for a number of input parameters''', and will automatically spawn multiple simulations which explore the various combinations of those parameters. This can be used to evaluate the effect of various parameters on the growth of a given model.<br />
<br />
==ClusterMonitor==<br />
Provides a graphical user interface for managing simulations '''running remotely on a computer cluster'''. Specifically, it allows you to see which projects are present and running on the cluster, to retrieve the completed projects, to generate images of the simulations at specified stages of growth, and to remove projects from the cluster. If you do not intend to use a computer cluster, then you will not need to use ClusterMonitor.<br />
<br />
='''2''' Computer or cluster?=<br />
<br />
In basic terms, '''a computing cluster is effectively a network of many computer processors''' (often hundreds), centrally managed by a queuing system. When a job is submitted to a cluster, the job is sent to a processor that is not being used, or queued until one becomes available. In contrast, a typical desktop computer will contain one processor, limiting the number of tasks that can be performed at any one time. Jobs which must be run independently and sequentially on a desktop computer can be executed in parallel on a cluster, '''greatly reducing the total time required to complete all of the jobs'''. Although the exact details of your cluster might vary from those described in the rest of these tutorials, we aim to illustrate the generic processes involved in using GFtbox on a cluster. We are happy to offer assistance, where possible, to setup GFtbox for your cluster, so please contact us if you have any queries.<br />
<br />
Whilst the time savings offered by using a cluster can be significant, there is an overhead associated with returning the results to your personal computer. The total time to run 2 or more simulations on a cluster and to return the results will be less than running those simulations sequentially on one computer. Therefore, using a '''cluster is ideal for situations where you would like to run several simulations''', such as to evaluate the effect of a range of parameters on a growth model. You are not advised to use a cluster for running single simulations, or where you would like to step through a simulation and change parameters in a more interactive fashion. In such circumstances, you are advised to run GFtbox or GFtboxCommand on your own computer.<br />
<br />
The '''GFtbox GUI provides quick feedback''' about how the changes you have made to an interaction function have affected the growth simulation, as '''you can see the result of each simulation iteration as it completes'''. It is quick and easy therefore to see if you have dramatically changed the course of the growth simulation, and then to adjust the parameters according to your observations. This approach is well suited to the early stages of design, where you might wish to '''tweak some parameters to gauge whether or not they have had the desired effect'''. Or in the final stages, where you think your model is almost finalised, but where small adjustments are required. '''An alternative approach is to start many simulations''', based upon your initial model and by making intelligent choices about which parameters to explore. This allows you to '''harvest many results, to quickly and easily overview them''' in their finalised form, or to select interesting-looking simulations and examine them more closely on your desktop computer, using GFtbox.<br />
<br />
='''3''' Running a growth simulation for an example model=<br />
This tutorial is aimed at running a '''growth simulation for one of the example interaction functions''' included with the GFtbox. The purpose of this exercise is to firstly demonstrate how simulations can be invoked using GFtboxCommand, and secondly to show how to reproduce experimental results (specifically, those published in Kennaway et al (2011)) given an interaction function.<br />
<br />
Assuming Matlab is installed on your computer, and the latest GFtbox has been downloaded, you can add the GFtbox directory to Matlab's search path, which makes the toolbox accessible from any other path that you choose to work from. For a short tutorial on how to do this, please click [[Adding GFtbox to Matlab's search path | here]].<br />
<br />
Once the GFtbox is added to Matlab, you are ready to run a growth simulation using GFtboxCommand. In this example, '''the model that we will simulate is called GPT_CASE_RST'''. Results generated using this model are published in Kennaway et al (2011). By running this simulation, we can confirm the results in the published literature and investigate the suitability of the various parameters.<br />
<br />
===3A - Running a simulation on your computer===<br />
<br />
The following command can be typed into Matlab to run a simulation of the GPT_CASE_RST interaction function, which contains three growth models: R, S and T. '''Three separate simulations are run sequentially on your computer''', one for each model, each producing results corresponding to five intervals in the growth simulation.<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',[1:3]);<br />
<br />
GFtboxCommand accepts input arguments as name and value pairs, e.g. 'modelname', [1:3] or 'Use','Cluster'. The argument names entered in the example above are: ''Path'', ''Name'', ''Stages'' and ''modelname''.<br />
<br />
The optional '''''Path'' argument name refers to the location of the folder''' (or directory) on your computer where the model interaction function you wish to simulate is stored. '''''Name'', is the name of the folder itself'''. In this case, we are using the GPT_CASE_RST folder which is included in the GrowthToolbox. You may wish to copy this folder elsewhere, if you intend to make changes to the interaction function.<br />
<br />
During a growth simulation, a mesh can be generated at each time step of the simulation, which provides a visual representation of the growth of the biological tissue, given the various parameters of growth specified by the interaction function and how they have changed over time. Put another way, the mesh shows exactly what the growing tissue actually looks like. ''Stages'' refers to the points in the simulation (measured in hours) at which meshes should be generated and saved. In this example, '''five stages of growth will be written to disk'''. These values are chosen to best capture the appearance of the tissue at important stages of the tissue growth.<br />
<br />
The final argument name listed here is ''modelname''. This is a model-specific argument, and in this case the GPT_CASE_RST interaction function contains '''three separate models for plant growth''', allowing the desired model to be a selected. Here, the value [1:3] is specified, which is evaluated in the same way as entering [1 2 3]. This instructs GFtboxCommand to run three separate simulations, one for each of the growth models contained in the interaction function.<br />
<br />
The function of every permissible argument is given by keying the following command into Matlab:<br />
<br />
help GFtboxCommand<br />
<br />
===3B - Getting results from a completed simulation===<br />
<br />
'''1) - Generating images from a simulation you ran on your computer'''<br />
<br />
Once a growth simulation is completed, the project folder (GPT_CASE_RST, in our example) will contain another folder named "movies". Within movies, there are folders which contain results for the executed simulations. As was instructed in 3A, here we can see three folders corresponding to simulations for the three separate models. Within the first folder there are three items:<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| ''CommandLine.txt'' - This file contains the Matlab command which was used to generate the results that this sub-directory contains.<br />
<br />
''gpt_case_rst.txt'' - This file, named according to the project name, contains a copy of the interaction function which was used to produce these results.<br />
<br />
''meshes'' - This folder contains the mesh files corresponding to the stages of growth specified by the value of the Stages argument, described in 3A.<br />
|width="500px" align="center"|[[File:resultsdir.png| Directory structure of a project containing results]]<br />
|}<br />
<br />
<br />
The mesh files contain vertices information regarding the shape of the growth model at a particular stage of growth, but are not visualisable in that form. In order to '''convert these mesh files into viewable images''', we can execute the following command in Matlab:<br />
<br />
VMSreport('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Project','GPT_CASE_RST',...<br />
'Experiment','All','flattentime',572.5,'morphogen','KPAR','SNAPFIG',true);<br />
<br />
Where the ''Path'' and ''Project'' arguments have the same function as ''Path'' and ''Name'' in 3A, i.e. ''Path'' refers to the location of the project folder and ''Project'' is the name of the project folder itself.<br />
<br />
Here we can see the generated images, including the command line arguments, for one of the simulations performed in 3A. Click on a thumbnail to view a larger image.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="125px"| <small>001commandline.png</small><br> [[File:commandlineexample.png | 100px]]<br />
|width="125px"| <small>001GPT_CASE_RST_2 wild_.png</small><br>[[File:GPT_CASE_RST_example1.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_3 wild_.png</small><br>[[File:GPT_CASE_RST_example2.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_4 wild_.png</small><br>[[File:GPT_CASE_RST_example3.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_5 wild_.png</small><br>[[File:GPT_CASE_RST_example4.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_6 wild_.png</small><br>[[File:GPT_CASE_RST_example5.png | 125px]]<br />
|}<br />
<br />
<br />
'''2) - Generating images from a simulation you ran on the cluster'''<br />
<br />
NB. The results produced may not be visibly identical to those in the published literature. This is because of small, random perturbations which are applied to the initial model meshes to stop them from containing surfaces which are perfectly flat, and therefore biologically unrealistic. The results produced should be '''qualitatively but not quantitatively the same'''.<br />
<br />
'''3) - Interacting with your results using the GFtbox GUI'''<br />
<br />
='''4''' Altering the simulation parameters=<br />
<br />
One such simulation parameter is ''dt'', which is the time in seconds between iterations in the growth simulation. Large values of ''dt'' mean that fewer steps and therefore fewer calculations are required to complete a simulation. Whereas smaller values of ''dt'' mean that more steps are required, and therefore more processing time too. Therefore, a value of ''dt'' must be selected which is not so small that it is computationally unmanageable, but not so large that the observed growth is an artifact of the value of ''dt'', rather than the underlying growth model. It is necessary therefore to test a range of values for ''dt'' to ensure that the patterns of growth observed in a simulation are consistent across the range, and to find a value which is sufficient to demonstrate the model of growth and computationally efficient.<br />
<br />
'''Testing a range of ''dt''s can be achieved in several ways.''' One way is to use the ''dt'' argument when calling GFtboxCommand. This allows a single value to be tested. Another way is to make a batch of jobs, each using a different value for ''dt'', by using the 'State' argument. Lastly, a range of ''dt''s can be specified in GFtboxCommand, by using the dt argument where the values within the square brackets are the ''dt''s to simulate:<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5]);<br />
<br />
Here, we have specified '''four separate values for ''dt''''', and this means that '''four separate simulations will be run''' sequentially on your computer. As in 3B, images can then be generated for the mesh files produced, and compared to ensure consistency across the range of ''dt''s. In this figure, we can see the same model, at the same stage of growth, generated using the four values of ''dt''. Though quantitatively different, they are qualitatively the same, illustrating the suitability of the default ''dt'' value of 5.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| [[File:GPT_CASE_RST_dt0.5.png| 200px]]<br />
|width="200px"| [[File:GPT_CASE_RST_dt1.png| 200px]]<br />
|width="200px"| [[File:GPT_CASE_RST_dt2.png| 200px]]<br />
|width="200px"| [[File:GPT_CASE_RST_dt5.png| 200px]]<br />
|}<br />
<br />
='''5''' Altering the model parameters=<br />
It is easy to see that when even a small number of range variables, mutations or ''dt''s are specified, the '''total number of simulations increases quickly'''. If many ranges are specified, then the amount of processing time becomes unmanageable on a single computer. Via ''GFtboxCommand'' and the ''ClusterMonitor'' tool, we can remotely''' run a number of simulations, in parallel, on a computing cluster'''. It is assumed that the GrowthToolbox and Matlab are installed on your cluster, and PuTTY is required on your local computer for the tools ''pscp'' (for transferring files from your computer to a cluster) and ''plink'' (for remotely executing commands on a cluster). Using the ''dt'' range argument from 3 as an example, here we add the 'Use' name argument with the value 'Cluster':<br />
<br />
GFtboxCommand('State','Start','Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5],'Use','Cluster');<br />
<br />
The 'Cluster' option for the 'Use' name argument instructs GFtboxCommand to upload the required project directory to a remote Linux server. Instead of running the simulation on your own computer, GFtboxCommand then works out how many individual simulations are specified by the command that invoked it. In this example, the only argument which will generate multiple simulations is ''dt'', of which there will be 4. Separate commands for each of these jobs are then automatically generated, each with an accompanying unique ID (The value of the ''ExpID'' name argument), and '''these are submitted as individual jobs to the cluster'''.<br />
<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.1,'ExpID','GPT_CASE_RST_1');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.5,'ExpID','GPT_CASE_RST_2');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',1,'ExpID','GPT_CASE_RST_3');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',5,'ExpID','GPT_CASE_RST_4');<br />
<br />
Once these jobs have been submitted, the ''ClusterMonitor'' tool opens and the new job batch ID is added to the list of jobs. Here, we will concentrate on three of the functions of ''ClusterMonitor'': ''Queue?'', ''Get project results'' and ''Make project pngs''. As in part 3 of this page, these functions will enable us to visualise the growth simulation results.<br />
<br />
Whilst ''ClusterMonitor'' is a useful tool for managing your cluster jobs, it is advisable to '''have at least a rudimentary understanding of and ability to use a Unix-based computer system'''. In particular, the abilities to list the contents of a folder, view the contents of a file, change the current working folder, delete, and to copy or move files or folders, are essential Unix skills for making sure everything is working as intended. It is beyond the scope of these tutorials to provide an in depth Unix tutorial (many good and simple tutorials exist on the web), but here is a short description of the Unix commands (which may be specific to our Unix-based system) that we believe to be important.</div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Running_example_models_and_using_a_cluster&diff=5375Running example models and using a cluster2011-06-23T13:33:52Z<p>JacobNewman: </p>
<hr />
<div>[[GFtbox Tutorial pages | Back to GFtbox Tutorial pages]]<br />
<br />
The purpose of these tutorials is to learn how to run the example growth simulations included in GFtbox. We will describe methods for running simulations locally (on your own computer) and remotely (on a computing cluster). It is assumed that you have already downloaded the GFtbox software and have Matlab installed.<br />
<br />
=Getting Started=<br />
<br />
The remainder of this page is split into five sub-tutorials, each building on the preceding parts.<br />
<br />
1) '''Explaining the tools'''. In these tutorials, we will be using ''GFtboxCommand'' and ''ClusterMonitor''. This section explains the purpose of these tools. <br />
<br />
2) '''Computer or cluster?''' Here, we illustrate when and why you should use a computing cluster for your growth simulations, and conversely, when you should use your desktop computer.<br />
<br />
3) '''Running a growth simulation for an example model'''. This section demonstrates how to use ''GFtboxCommand'' to run a growth simulation on your computer. The model used for the simulation is an example model included with the GFtbox.<br />
<br />
4) '''Altering the simulation parameters'''. Following on from part 3, here we show how to adjust a simulation parameter within GFtboxCommand. Specifically, we alter the value of ''dt'', the temporal resolution of the simulation, and show how it can be used to verify that the value specified in the published literature is reasonable.<br />
<br />
5) '''Altering the model parameters'''. Finally, we demonstrate how a number of model parameters can be varied by specifying a range of values for each model argument. We show how the computationally expensive task of simulating all combinations of specified ranges can be processed more efficiently if a computer cluster is used via the ''ClusterMonitor'' tool.<br />
<br />
='''1''' Explaining the tools=<br />
==GFtboxCommand==<br />
This is a command line version of the GFtbox. By command line, we mean that all program '''functions are operated via typed commands, without the GUI'''. Like GFtbox, GFtboxCommand is capable of running growth simulations of an interaction function, and allows the user to specify model and simulation parameters. Unlike GFtbox, this also allows the user to select '''ranges of values for a number of input parameters''', and will automatically spawn multiple simulations which explore the various combinations of those parameters. This can be used to evaluate the effect of various parameters on the growth of a given model.<br />
<br />
==ClusterMonitor==<br />
Provides a graphical user interface for managing simulations '''running remotely on a computer cluster'''. Specifically, it allows you to see which projects are present and running on the cluster, to retrieve the completed projects, to generate images of the simulations at specified stages of growth, and to remove projects from the cluster. If you do not intend to use a computer cluster, then you will not need to use ClusterMonitor.<br />
<br />
='''2''' Computer or cluster?=<br />
<br />
In basic terms, '''a computing cluster is effectively a network of many computer processors''' (often hundreds), centrally managed by a queuing system. When a job is submitted to a cluster, the job is sent to a processor that is not being used, or queued until one becomes available. In contrast, a typical desktop computer will contain one processor, limiting the number of tasks that can be performed at any one time. Jobs which must be run independently and sequentially on a desktop computer can be executed in parallel on a cluster, '''greatly reducing the total time required to complete all of the jobs'''. Although the exact details of your cluster might vary from those described in the rest of these tutorials, we aim to illustrate the generic processes involved in using GFtbox on a cluster. We are happy to offer assistance, where possible, to setup GFtbox for your cluster, so please contact us if you have any queries.<br />
<br />
Whilst the time savings offered by using a cluster can be significant, there is an overhead associated with returning the results to your personal computer. The total time to run 2 or more simulations on a cluster and to return the results will be less than running those simulations sequentially on one computer. Therefore, using a '''cluster is ideal for situations where you would like to run several simulations''', such as to evaluate the effect of a range of parameters on a growth model. You are not advised to use a cluster for running single simulations, or where you would like to step through a simulation and change parameters in a more interactive fashion. In such circumstances, you are advised to run GFtbox or GFtboxCommand on your own computer.<br />
<br />
The '''GFtbox GUI provides quick feedback''' about how the changes you have made to an interaction function have affected the growth simulation, as '''you can see the result of each simulation iteration as it completes'''. It is quick and easy therefore to see if you have dramatically changed the course of the growth simulation, and then to adjust the parameters according to your observations. This approach is well suited to the early stages of design, where you might wish to '''tweak some parameters to gauge whether or not they have had the desired effect'''. Or in the final stages, where you think your model is almost finalised, but where small adjustments are required. '''An alternative approach is to start many simulations''', based upon your initial model and by making intelligent choices about which parameters to explore. This allows you to '''harvest many results, to quickly and easily overview them''' in their finalised form, or to select interesting-looking simulations and examine them more closely on your desktop computer, using GFtbox.<br />
<br />
='''3''' Running a growth simulation for an example model=<br />
This tutorial is aimed at running a '''growth simulation for one of the example interaction functions''' included with the GFtbox. The purpose of this exercise is to firstly demonstrate how simulations can be invoked using GFtboxCommand, and secondly to show how to reproduce experimental results (specifically, those published in Kennaway et al (2011)) given an interaction function.<br />
<br />
Assuming Matlab is installed on your computer, and the latest GFtbox has been downloaded, you can add the GFtbox directory to Matlab's search path, which makes the toolbox accessible from any other path that you choose to work from. For a short tutorial on how to do this, please click [[Adding GFtbox to Matlab's search path | here]].<br />
<br />
Once the GFtbox is added to Matlab, you are ready to run a growth simulation using GFtboxCommand. In this example, '''the model that we will simulate is called GPT_CASE_RST'''. Results generated using this model are published in Kennaway et al (2011). By running this simulation, we can confirm the results in the published literature and investigate the suitability of the various parameters.<br />
<br />
===3A - Running a simulation on your computer===<br />
<br />
The following command can be typed into Matlab to run a simulation of the GPT_CASE_RST interaction function, which contains three growth models: R, S and T. '''Three separate simulations are run sequentially on your computer''', one for each model, each producing results corresponding to five intervals in the growth simulation.<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',[1:3]);<br />
<br />
GFtboxCommand accepts input arguments as name and value pairs, e.g. 'modelname', [1:3] or 'Use','Cluster'. The argument names entered in the example above are: ''Path'', ''Name'', ''Stages'' and ''modelname''.<br />
<br />
The optional '''''Path'' argument name refers to the location of the folder''' (or directory) on your computer where the model interaction function you wish to simulate is stored. '''''Name'', is the name of the folder itself'''. In this case, we are using the GPT_CASE_RST folder which is included in the GrowthToolbox. You may wish to copy this folder elsewhere, if you intend to make changes to the interaction function.<br />
<br />
During a growth simulation, a mesh can be generated at each time step of the simulation, which provides a visual representation of the growth of the biological tissue, given the various parameters of growth specified by the interaction function and how they have changed over time. Put another way, the mesh shows exactly what the growing tissue actually looks like. ''Stages'' refers to the points in the simulation (measured in hours) at which meshes should be generated and saved. In this example, '''five stages of growth will be written to disk'''. These values are chosen to best capture the appearance of the tissue at important stages of the tissue growth.<br />
<br />
The final argument name listed here is ''modelname''. This is a model-specific argument, and in this case the GPT_CASE_RST interaction function contains '''three separate models for plant growth''', allowing the desired model to be a selected. Here, the value [1:3] is specified, which is evaluated in the same way as entering [1 2 3]. This instructs GFtboxCommand to run three separate simulations, one for each of the growth models contained in the interaction function.<br />
<br />
The function of every permissible argument is given by keying the following command into Matlab:<br />
<br />
help GFtboxCommand<br />
<br />
===3B - Getting results from a completed simulation===<br />
<br />
'''1) - Generating images from a simulation you ran on your computer'''<br />
<br />
Once a growth simulation is completed, the project folder (GPT_CASE_RST, in our example) will contain another folder named "movies". Within movies, there are folders which contain results for the executed simulations. As was instructed in 3A, here we can see three folders corresponding to simulations for the three separate models. Within the first folder there are three items:<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| ''CommandLine.txt'' - This file contains the Matlab command which was used to generate the results that this sub-directory contains.<br />
<br />
''gpt_case_rst.txt'' - This file, named according to the project name, contains a copy of the interaction function which was used to produce these results.<br />
<br />
''meshes'' - This folder contains the mesh files corresponding to the stages of growth specified by the value of the Stages argument, described in 3A.<br />
|width="500px" align="center"|[[File:resultsdir.png| Directory structure of a project containing results]]<br />
|}<br />
<br />
<br />
The mesh files contain vertices information regarding the shape of the growth model at a particular stage of growth, but are not visualisable in that form. In order to '''convert these mesh files into viewable images''', we can execute the following command in Matlab:<br />
<br />
VMSreport('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Project','GPT_CASE_RST',...<br />
'Experiment','All','flattentime',572.5,'morphogen','KPAR','SNAPFIG',true);<br />
<br />
Where the ''Path'' and ''Project'' arguments have the same function as ''Path'' and ''Name'' in 3A, i.e. ''Path'' refers to the location of the project folder and ''Project'' is the name of the project folder itself.<br />
<br />
Here we can see the generated images, including the command line arguments, for one of the simulations performed in 3A. Click on a thumbnail to view a larger image.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="125px"| <small>001commandline.png</small><br> [[File:commandlineexample.png | 100px]]<br />
|width="125px"| <small>001GPT_CASE_RST_2 wild_.png</small><br>[[File:GPT_CASE_RST_example1.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_3 wild_.png</small><br>[[File:GPT_CASE_RST_example2.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_4 wild_.png</small><br>[[File:GPT_CASE_RST_example3.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_5 wild_.png</small><br>[[File:GPT_CASE_RST_example4.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_6 wild_.png</small><br>[[File:GPT_CASE_RST_example5.png | 125px]]<br />
|}<br />
<br />
<br />
'''2) - Generating images from a simulation you ran on the cluster'''<br />
<br />
NB. The results produced may not be visibly identical to those in the published literature. This is because of small, random perturbations which are applied to the initial model meshes to stop them from containing surfaces which are perfectly flat, and therefore biologically unrealistic. The results produced should be '''qualitatively but not quantitatively the same'''.<br />
<br />
'''3) - Interacting with your results using the GFtbox GUI'''<br />
<br />
='''4''' Altering the simulation parameters=<br />
<br />
One such simulation parameter is ''dt'', which is the time in seconds between iterations in the growth simulation. Large values of ''dt'' mean that fewer steps and therefore fewer calculations are required to complete a simulation. Whereas smaller values of ''dt'' mean that more steps are required, and therefore more processing time too. Therefore, a value of ''dt'' must be selected which is not so small that it is computationally unmanageable, but not so large that the observed growth is an artifact of the value of ''dt'', rather than the underlying growth model. It is necessary therefore to test a range of values for ''dt'' to ensure that the patterns of growth observed in a simulation are consistent across the range, and to find a value which is sufficient to demonstrate the model of growth and computationally efficient.<br />
<br />
'''Testing a range of ''dt''s can be achieved in several ways.''' One way is to use the ''dt'' argument when calling GFtboxCommand. This allows a single value to be tested. Another way is to make a batch of jobs, each using a different value for ''dt'', by using the 'State' argument. Lastly, a range of ''dt''s can be specified in GFtboxCommand, by using the dt argument where the values within the square brackets are the ''dt''s to simulate:<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5]);<br />
<br />
Here, we have specified '''four separate values for ''dt''''', and this means that '''four separate simulations will be run''' sequentially on your computer. As in 3B, images can then be generated for the mesh files produced, and compared to ensure consistency across the range of ''dt''s. In this figure, we can see the same model, at the same stage of growth, generated using the four values of ''dt''. Though quantitatively different, they are qualitatively the same, illustrating the suitability of the default ''dt'' value of 5.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="175px"| [[File:GPT_CASE_RST_dt0.5.png| 175px]]<br />
|width="175px"| [[File:GPT_CASE_RST_dt1.png| 175px]]<br />
|width="175px"| [[File:GPT_CASE_RST_dt2.png| 175px]]<br />
|width="175px"| [[File:GPT_CASE_RST_dt5.png| 175px]]<br />
|}<br />
<br />
='''5''' Altering the model parameters=<br />
It is easy to see that when even a small number of range variables, mutations or ''dt''s are specified, the '''total number of simulations increases quickly'''. If many ranges are specified, then the amount of processing time becomes unmanageable on a single computer. Via ''GFtboxCommand'' and the ''ClusterMonitor'' tool, we can remotely''' run a number of simulations, in parallel, on a computing cluster'''. It is assumed that the GrowthToolbox and Matlab are installed on your cluster, and PuTTY is required on your local computer for the tools ''pscp'' (for transferring files from your computer to a cluster) and ''plink'' (for remotely executing commands on a cluster). Using the ''dt'' range argument from 3 as an example, here we add the 'Use' name argument with the value 'Cluster':<br />
<br />
GFtboxCommand('State','Start','Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5],'Use','Cluster');<br />
<br />
The 'Cluster' option for the 'Use' name argument instructs GFtboxCommand to upload the required project directory to a remote Linux server. Instead of running the simulation on your own computer, GFtboxCommand then works out how many individual simulations are specified by the command that invoked it. In this example, the only argument which will generate multiple simulations is ''dt'', of which there will be 4. Separate commands for each of these jobs are then automatically generated, each with an accompanying unique ID (The value of the ''ExpID'' name argument), and '''these are submitted as individual jobs to the cluster'''.<br />
<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.1,'ExpID','GPT_CASE_RST_1');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.5,'ExpID','GPT_CASE_RST_2');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',1,'ExpID','GPT_CASE_RST_3');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',5,'ExpID','GPT_CASE_RST_4');<br />
<br />
Once these jobs have been submitted, the ''ClusterMonitor'' tool opens and the new job batch ID is added to the list of jobs. Here, we will concentrate on three of the functions of ''ClusterMonitor'': ''Queue?'', ''Get project results'' and ''Make project pngs''. As in part 3 of this page, these functions will enable us to visualise the growth simulation results.<br />
<br />
Whilst ''ClusterMonitor'' is a useful tool for managing your cluster jobs, it is advisable to '''have at least a rudimentary understanding of and ability to use a Unix-based computer system'''. In particular, the abilities to list the contents of a folder, view the contents of a file, change the current working folder, delete, and to copy or move files or folders, are essential Unix skills for making sure everything is working as intended. It is beyond the scope of these tutorials to provide an in depth Unix tutorial (many good and simple tutorials exist on the web), but here is a short description of the Unix commands (which may be specific to our Unix-based system) that we believe to be important.</div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=File:GPT_CASE_RST_dt5.png&diff=5512File:GPT CASE RST dt5.png2011-06-23T13:32:28Z<p>JacobNewman: </p>
<hr />
<div></div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=File:GPT_CASE_RST_dt2.png&diff=5511File:GPT CASE RST dt2.png2011-06-23T13:32:16Z<p>JacobNewman: </p>
<hr />
<div></div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=File:GPT_CASE_RST_dt1.png&diff=5510File:GPT CASE RST dt1.png2011-06-23T13:31:57Z<p>JacobNewman: </p>
<hr />
<div></div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=File:GPT_CASE_RST_dt0.5.png&diff=5509File:GPT CASE RST dt0.5.png2011-06-23T13:31:42Z<p>JacobNewman: </p>
<hr />
<div></div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Running_example_models_and_using_a_cluster&diff=5419Running example models and using a cluster2011-06-23T13:31:27Z<p>JacobNewman: </p>
<hr />
<div>[[GFtbox Tutorial pages | Back to GFtbox Tutorial pages]]<br />
<br />
The purpose of these tutorials is to learn how to run the example growth simulations included in GFtbox. We will describe methods for running simulations locally (on your own computer) and remotely (on a computing cluster). It is assumed that you have already downloaded the GFtbox software and have Matlab installed.<br />
<br />
=Getting Started=<br />
<br />
The remainder of this page is split into five sub-tutorials, each building on the preceding parts.<br />
<br />
1) '''Explaining the tools'''. In these tutorials, we will be using ''GFtboxCommand'' and ''ClusterMonitor''. This section explains the purpose of these tools. <br />
<br />
2) '''Computer or cluster?''' Here, we illustrate when and why you should use a computing cluster for your growth simulations, and conversely, when you should use your desktop computer.<br />
<br />
3) '''Running a growth simulation for an example model'''. This section demonstrates how to use ''GFtboxCommand'' to run a growth simulation on your computer. The model used for the simulation is an example model included with the GFtbox.<br />
<br />
4) '''Altering the simulation parameters'''. Following on from part 3, here we show how to adjust a simulation parameter within GFtboxCommand. Specifically, we alter the value of ''dt'', the temporal resolution of the simulation, and show how it can be used to verify that the value specified in the published literature is reasonable.<br />
<br />
5) '''Altering the model parameters'''. Finally, we demonstrate how a number of model parameters can be varied by specifying a range of values for each model argument. We show how the computationally expensive task of simulating all combinations of specified ranges can be processed more efficiently if a computer cluster is used via the ''ClusterMonitor'' tool.<br />
<br />
='''1''' Explaining the tools=<br />
==GFtboxCommand==<br />
This is a command line version of the GFtbox. By command line, we mean that all program '''functions are operated via typed commands, without the GUI'''. Like GFtbox, GFtboxCommand is capable of running growth simulations of an interaction function, and allows the user to specify model and simulation parameters. Unlike GFtbox, this also allows the user to select '''ranges of values for a number of input parameters''', and will automatically spawn multiple simulations which explore the various combinations of those parameters. This can be used to evaluate the effect of various parameters on the growth of a given model.<br />
<br />
==ClusterMonitor==<br />
Provides a graphical user interface for managing simulations '''running remotely on a computer cluster'''. Specifically, it allows you to see which projects are present and running on the cluster, to retrieve the completed projects, to generate images of the simulations at specified stages of growth, and to remove projects from the cluster. If you do not intend to use a computer cluster, then you will not need to use ClusterMonitor.<br />
<br />
='''2''' Computer or cluster?=<br />
<br />
In basic terms, '''a computing cluster is effectively a network of many computer processors''' (often hundreds), centrally managed by a queuing system. When a job is submitted to a cluster, the job is sent to a processor that is not being used, or queued until one becomes available. In contrast, a typical desktop computer will contain one processor, limiting the number of tasks that can be performed at any one time. Jobs which must be run independently and sequentially on a desktop computer can be executed in parallel on a cluster, '''greatly reducing the total time required to complete all of the jobs'''. Although the exact details of your cluster might vary from those described in the rest of these tutorials, we aim to illustrate the generic processes involved in using GFtbox on a cluster. We are happy to offer assistance, where possible, to setup GFtbox for your cluster, so please contact us if you have any queries.<br />
<br />
Whilst the time savings offered by using a cluster can be significant, there is an overhead associated with returning the results to your personal computer. The total time to run 2 or more simulations on a cluster and to return the results will be less than running those simulations sequentially on one computer. Therefore, using a '''cluster is ideal for situations where you would like to run several simulations''', such as to evaluate the effect of a range of parameters on a growth model. You are not advised to use a cluster for running single simulations, or where you would like to step through a simulation and change parameters in a more interactive fashion. In such circumstances, you are advised to run GFtbox or GFtboxCommand on your own computer.<br />
<br />
The '''GFtbox GUI provides quick feedback''' about how the changes you have made to an interaction function have affected the growth simulation, as '''you can see the result of each simulation iteration as it completes'''. It is quick and easy therefore to see if you have dramatically changed the course of the growth simulation, and then to adjust the parameters according to your observations. This approach is well suited to the early stages of design, where you might wish to '''tweak some parameters to gauge whether or not they have had the desired effect'''. Or in the final stages, where you think your model is almost finalised, but where small adjustments are required. '''An alternative approach is to start many simulations''', based upon your initial model and by making intelligent choices about which parameters to explore. This allows you to '''harvest many results, to quickly and easily overview them''' in their finalised form, or to select interesting-looking simulations and examine them more closely on your desktop computer, using GFtbox.<br />
<br />
='''3''' Running a growth simulation for an example model=<br />
This tutorial is aimed at running a '''growth simulation for one of the example interaction functions''' included with the GFtbox. The purpose of this exercise is to firstly demonstrate how simulations can be invoked using GFtboxCommand, and secondly to show how to reproduce experimental results (specifically, those published in Kennaway et al (2011)) given an interaction function.<br />
<br />
Assuming Matlab is installed on your computer, and the latest GFtbox has been downloaded, you can add the GFtbox directory to Matlab's search path, which makes the toolbox accessible from any other path that you choose to work from. For a short tutorial on how to do this, please click [[Adding GFtbox to Matlab's search path | here]].<br />
<br />
Once the GFtbox is added to Matlab, you are ready to run a growth simulation using GFtboxCommand. In this example, '''the model that we will simulate is called GPT_CASE_RST'''. Results generated using this model are published in Kennaway et al (2011). By running this simulation, we can confirm the results in the published literature and investigate the suitability of the various parameters.<br />
<br />
===3A - Running a simulation on your computer===<br />
<br />
The following command can be typed into Matlab to run a simulation of the GPT_CASE_RST interaction function, which contains three growth models: R, S and T. '''Three separate simulations are run sequentially on your computer''', one for each model, each producing results corresponding to five intervals in the growth simulation.<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',[1:3]);<br />
<br />
GFtboxCommand accepts input arguments as name and value pairs, e.g. 'modelname', [1:3] or 'Use','Cluster'. The argument names entered in the example above are: ''Path'', ''Name'', ''Stages'' and ''modelname''.<br />
<br />
The optional '''''Path'' argument name refers to the location of the folder''' (or directory) on your computer where the model interaction function you wish to simulate is stored. '''''Name'', is the name of the folder itself'''. In this case, we are using the GPT_CASE_RST folder which is included in the GrowthToolbox. You may wish to copy this folder elsewhere, if you intend to make changes to the interaction function.<br />
<br />
During a growth simulation, a mesh can be generated at each time step of the simulation, which provides a visual representation of the growth of the biological tissue, given the various parameters of growth specified by the interaction function and how they have changed over time. Put another way, the mesh shows exactly what the growing tissue actually looks like. ''Stages'' refers to the points in the simulation (measured in hours) at which meshes should be generated and saved. In this example, '''five stages of growth will be written to disk'''. These values are chosen to best capture the appearance of the tissue at important stages of the tissue growth.<br />
<br />
The final argument name listed here is ''modelname''. This is a model-specific argument, and in this case the GPT_CASE_RST interaction function contains '''three separate models for plant growth''', allowing the desired model to be a selected. Here, the value [1:3] is specified, which is evaluated in the same way as entering [1 2 3]. This instructs GFtboxCommand to run three separate simulations, one for each of the growth models contained in the interaction function.<br />
<br />
The function of every permissible argument is given by keying the following command into Matlab:<br />
<br />
help GFtboxCommand<br />
<br />
===3B - Getting results from a completed simulation===<br />
<br />
'''1) - Generating images from a simulation you ran on your computer'''<br />
<br />
Once a growth simulation is completed, the project folder (GPT_CASE_RST, in our example) will contain another folder named "movies". Within movies, there are folders which contain results for the executed simulations. As was instructed in 3A, here we can see three folders corresponding to simulations for the three separate models. Within the first folder there are three items:<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| ''CommandLine.txt'' - This file contains the Matlab command which was used to generate the results that this sub-directory contains.<br />
<br />
''gpt_case_rst.txt'' - This file, named according to the project name, contains a copy of the interaction function which was used to produce these results.<br />
<br />
''meshes'' - This folder contains the mesh files corresponding to the stages of growth specified by the value of the Stages argument, described in 3A.<br />
|width="500px" align="center"|[[File:resultsdir.png| Directory structure of a project containing results]]<br />
|}<br />
<br />
<br />
The mesh files contain vertices information regarding the shape of the growth model at a particular stage of growth, but are not visualisable in that form. In order to '''convert these mesh files into viewable images''', we can execute the following command in Matlab:<br />
<br />
VMSreport('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Project','GPT_CASE_RST',...<br />
'Experiment','All','flattentime',572.5,'morphogen','KPAR','SNAPFIG',true);<br />
<br />
Where the ''Path'' and ''Project'' arguments have the same function as ''Path'' and ''Name'' in 3A, i.e. ''Path'' refers to the location of the project folder and ''Project'' is the name of the project folder itself.<br />
<br />
Here we can see the generated images, including the command line arguments, for one of the simulations performed in 3A. Click on a thumbnail to view a larger image.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="125px"| <small>001commandline.png</small><br> [[File:commandlineexample.png | 100px]]<br />
|width="125px"| <small>001GPT_CASE_RST_2 wild_.png</small><br>[[File:GPT_CASE_RST_example1.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_3 wild_.png</small><br>[[File:GPT_CASE_RST_example2.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_4 wild_.png</small><br>[[File:GPT_CASE_RST_example3.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_5 wild_.png</small><br>[[File:GPT_CASE_RST_example4.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_6 wild_.png</small><br>[[File:GPT_CASE_RST_example5.png | 125px]]<br />
|}<br />
<br />
<br />
'''2) - Generating images from a simulation you ran on the cluster'''<br />
<br />
NB. The results produced may not be visibly identical to those in the published literature. This is because of small, random perturbations which are applied to the initial model meshes to stop them from containing surfaces which are perfectly flat, and therefore biologically unrealistic. The results produced should be '''qualitatively but not quantitatively the same'''.<br />
<br />
'''3) - Interacting with your results using the GFtbox GUI'''<br />
<br />
='''4''' Altering the simulation parameters=<br />
<br />
One such simulation parameter is ''dt'', which is the time in seconds between iterations in the growth simulation. Large values of ''dt'' mean that fewer steps and therefore fewer calculations are required to complete a simulation. Whereas smaller values of ''dt'' mean that more steps are required, and therefore more processing time too. Therefore, a value of ''dt'' must be selected which is not so small that it is computationally unmanageable, but not so large that the observed growth is an artifact of the value of ''dt'', rather than the underlying growth model. It is necessary therefore to test a range of values for ''dt'' to ensure that the patterns of growth observed in a simulation are consistent across the range, and to find a value which is sufficient to demonstrate the model of growth and computationally efficient.<br />
<br />
'''Testing a range of ''dt''s can be achieved in several ways.''' One way is to use the ''dt'' argument when calling GFtboxCommand. This allows a single value to be tested. Another way is to make a batch of jobs, each using a different value for ''dt'', by using the 'State' argument. Lastly, a range of ''dt''s can be specified in GFtboxCommand, by using the dt argument where the values within the square brackets are the ''dt''s to simulate:<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5]);<br />
<br />
Here, we have specified '''four separate values for ''dt''''', and this means that '''four separate simulations will be run''' sequentially on your computer. As in 3B, images can then be generated for the mesh files produced, and compared to ensure consistency across the range of ''dt''s. In this figure, we can see the same model, at the same stage of growth, generated using the four values of ''dt''. Though quantitatively different, they are qualitatively the same, illustrating the suitability of the default ''dt'' value of 5.<br />
<br />
='''5''' Altering the model parameters=<br />
It is easy to see that when even a small number of range variables, mutations or ''dt''s are specified, the '''total number of simulations increases quickly'''. If many ranges are specified, then the amount of processing time becomes unmanageable on a single computer. Via ''GFtboxCommand'' and the ''ClusterMonitor'' tool, we can remotely''' run a number of simulations, in parallel, on a computing cluster'''. It is assumed that the GrowthToolbox and Matlab are installed on your cluster, and PuTTY is required on your local computer for the tools ''pscp'' (for transferring files from your computer to a cluster) and ''plink'' (for remotely executing commands on a cluster). Using the ''dt'' range argument from 3 as an example, here we add the 'Use' name argument with the value 'Cluster':<br />
<br />
GFtboxCommand('State','Start','Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5],'Use','Cluster');<br />
<br />
The 'Cluster' option for the 'Use' name argument instructs GFtboxCommand to upload the required project directory to a remote Linux server. Instead of running the simulation on your own computer, GFtboxCommand then works out how many individual simulations are specified by the command that invoked it. In this example, the only argument which will generate multiple simulations is ''dt'', of which there will be 4. Separate commands for each of these jobs are then automatically generated, each with an accompanying unique ID (The value of the ''ExpID'' name argument), and '''these are submitted as individual jobs to the cluster'''.<br />
<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.1,'ExpID','GPT_CASE_RST_1');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.5,'ExpID','GPT_CASE_RST_2');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',1,'ExpID','GPT_CASE_RST_3');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',5,'ExpID','GPT_CASE_RST_4');<br />
<br />
Once these jobs have been submitted, the ''ClusterMonitor'' tool opens and the new job batch ID is added to the list of jobs. Here, we will concentrate on three of the functions of ''ClusterMonitor'': ''Queue?'', ''Get project results'' and ''Make project pngs''. As in part 3 of this page, these functions will enable us to visualise the growth simulation results.<br />
<br />
Whilst ''ClusterMonitor'' is a useful tool for managing your cluster jobs, it is advisable to '''have at least a rudimentary understanding of and ability to use a Unix-based computer system'''. In particular, the abilities to list the contents of a folder, view the contents of a file, change the current working folder, delete, and to copy or move files or folders, are essential Unix skills for making sure everything is working as intended. It is beyond the scope of these tutorials to provide an in depth Unix tutorial (many good and simple tutorials exist on the web), but here is a short description of the Unix commands (which may be specific to our Unix-based system) that we believe to be important.</div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Running_example_models_and_using_a_cluster&diff=5374Running example models and using a cluster2011-06-23T13:27:42Z<p>JacobNewman: </p>
<hr />
<div>[[GFtbox Tutorial pages | Back to GFtbox Tutorial pages]]<br />
<br />
The purpose of these tutorials is to learn how to run the example growth simulations included in GFtbox. We will describe methods for running simulations locally (on your own computer) and remotely (on a computing cluster). It is assumed that you have already downloaded the GFtbox software and have Matlab installed.<br />
<br />
=Getting Started=<br />
<br />
The remainder of this page is split into five sub-tutorials, each building on the preceding parts.<br />
<br />
1) '''Explaining the tools'''. In these tutorials, we will be using ''GFtboxCommand'' and ''ClusterMonitor''. This section explains the purpose of these tools. <br />
<br />
2) '''Computer or cluster?''' Here, we illustrate when and why you should use a computing cluster for your growth simulations, and conversely, when you should use your desktop computer.<br />
<br />
3) '''Running a growth simulation for an example model'''. This section demonstrates how to use ''GFtboxCommand'' to run a growth simulation on your computer. The model used for the simulation is an example model included with the GFtbox.<br />
<br />
4) '''Altering the simulation parameters'''. Following on from part 3, here we show how to adjust a simulation parameter within GFtboxCommand. Specifically, we alter the value of ''dt'', the temporal resolution of the simulation, and show how it can be used to verify that the value specified in the published literature is reasonable.<br />
<br />
5) '''Altering the model parameters'''. Finally, we demonstrate how a number of model parameters can be varied by specifying a range of values for each model argument. We show how the computationally expensive task of simulating all combinations of specified ranges can be processed more efficiently if a computer cluster is used via the ''ClusterMonitor'' tool.<br />
<br />
='''1''' Explaining the tools=<br />
==GFtboxCommand==<br />
This is a command line version of the GFtbox. By command line, we mean that all program '''functions are operated via typed commands, without the GUI'''. Like GFtbox, GFtboxCommand is capable of running growth simulations of an interaction function, and allows the user to specify model and simulation parameters. Unlike GFtbox, this also allows the user to select '''ranges of values for a number of input parameters''', and will automatically spawn multiple simulations which explore the various combinations of those parameters. This can be used to evaluate the effect of various parameters on the growth of a given model.<br />
<br />
==ClusterMonitor==<br />
Provides a graphical user interface for managing simulations '''running remotely on a computer cluster'''. Specifically, it allows you to see which projects are present and running on the cluster, to retrieve the completed projects, to generate images of the simulations at specified stages of growth, and to remove projects from the cluster. If you do not intend to use a computer cluster, then you will not need to use ClusterMonitor.<br />
<br />
='''2''' Computer or cluster?=<br />
<br />
In basic terms, '''a computing cluster is effectively a network of many computer processors''' (often hundreds), centrally managed by a queuing system. When a job is submitted to a cluster, the job is sent to a processor that is not being used, or queued until one becomes available. In contrast, a typical desktop computer will contain one processor, limiting the number of tasks that can be performed at any one time. Jobs which must be run independently and sequentially on a desktop computer can be executed in parallel on a cluster, '''greatly reducing the total time required to complete all of the jobs'''. Although the exact details of your cluster might vary from those described in the rest of these tutorials, we aim to illustrate the generic processes involved in using GFtbox on a cluster. We are happy to offer assistance, where possible, to setup GFtbox for your cluster, so please contact us if you have any queries.<br />
<br />
Whilst the time savings offered by using a cluster can be significant, there is an overhead associated with returning the results to your personal computer. The total time to run 2 or more simulations on a cluster and to return the results will be less than running those simulations sequentially on one computer. Therefore, using a '''cluster is ideal for situations where you would like to run several simulations''', such as to evaluate the effect of a range of parameters on a growth model. You are not advised to use a cluster for running single simulations, or where you would like to step through a simulation and change parameters in a more interactive fashion. In such circumstances, you are advised to run GFtbox or GFtboxCommand on your own computer.<br />
<br />
The '''GFtbox GUI provides quick feedback''' about how the changes you have made to an interaction function have affected the growth simulation, as '''you can see the result of each simulation iteration as it completes'''. It is quick and easy therefore to see if you have dramatically changed the course of the growth simulation, and then to adjust the parameters according to your observations. This approach is well suited to the early stages of design, where you might wish to '''tweak some parameters to gauge whether or not they have had the desired effect'''. Or in the final stages, where you think your model is almost finalised, but where small adjustments are required. '''An alternative approach is to start many simulations''', based upon your initial model and by making intelligent choices about which parameters to explore. This allows you to '''harvest many results, to quickly and easily overview them''' in their finalised form, or to select interesting-looking simulations and examine them more closely on your desktop computer, using GFtbox.<br />
<br />
='''3''' Running a growth simulation for an example model=<br />
This tutorial is aimed at running a '''growth simulation for one of the example interaction functions''' included with the GFtbox. The purpose of this exercise is to firstly demonstrate how simulations can be invoked using GFtboxCommand, and secondly to show how to reproduce experimental results (specifically, those published in Kennaway et al (2011)) given an interaction function.<br />
<br />
Assuming Matlab is installed on your computer, and the latest GFtbox has been downloaded, you can add the GFtbox directory to Matlab's search path, which makes the toolbox accessible from any other path that you choose to work from. For a short tutorial on how to do this, please click [[Adding GFtbox to Matlab's search path | here]].<br />
<br />
Once the GFtbox is added to Matlab, you are ready to run a growth simulation using GFtboxCommand. In this example, '''the model that we will simulate is called GPT_CASE_RST'''. Results generated using this model are published in Kennaway et al (2011). By running this simulation, we can confirm the results in the published literature and investigate the suitability of the various parameters.<br />
<br />
===3A - Running a simulation on your computer===<br />
<br />
The following command can be typed into Matlab to run a simulation of the GPT_CASE_RST interaction function, which contains three growth models: R, S and T. '''Three separate simulations are run sequentially on your computer''', one for each model, each producing results corresponding to five intervals in the growth simulation.<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',[1:3]);<br />
<br />
GFtboxCommand accepts input arguments as name and value pairs, e.g. 'modelname', [1:3] or 'Use','Cluster'. The argument names entered in the example above are: ''Path'', ''Name'', ''Stages'' and ''modelname''.<br />
<br />
The optional '''''Path'' argument name refers to the location of the folder''' (or directory) on your computer where the model interaction function you wish to simulate is stored. '''''Name'', is the name of the folder itself'''. In this case, we are using the GPT_CASE_RST folder which is included in the GrowthToolbox. You may wish to copy this folder elsewhere, if you intend to make changes to the interaction function.<br />
<br />
During a growth simulation, a mesh can be generated at each time step of the simulation, which provides a visual representation of the growth of the biological tissue, given the various parameters of growth specified by the interaction function and how they have changed over time. Put another way, the mesh shows exactly what the growing tissue actually looks like. ''Stages'' refers to the points in the simulation (measured in hours) at which meshes should be generated and saved. In this example, '''five stages of growth will be written to disk'''. These values are chosen to best capture the appearance of the tissue at important stages of the tissue growth.<br />
<br />
The final argument name listed here is ''modelname''. This is a model-specific argument, and in this case the GPT_CASE_RST interaction function contains '''three separate models for plant growth''', allowing the desired model to be a selected. Here, the value [1:3] is specified, which is evaluated in the same way as entering [1 2 3]. This instructs GFtboxCommand to run three separate simulations, one for each of the growth models contained in the interaction function.<br />
<br />
The function of every permissible argument is given by keying the following command into Matlab:<br />
<br />
help GFtboxCommand<br />
<br />
===3B - Getting results from a completed simulation===<br />
<br />
'''1) - Generating images from a simulation you ran on your computer'''<br />
<br />
Once a growth simulation is completed, the project folder (GPT_CASE_RST, in our example) will contain another folder named "movies". Within movies, there are folders which contain results for the executed simulations. As was instructed in 3A, here we can see three folders corresponding to simulations for the three separate models. Within the first folder there are three items:<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| ''CommandLine.txt'' - This file contains the Matlab command which was used to generate the results that this sub-directory contains.<br />
<br />
''gpt_case_rst.txt'' - This file, named according to the project name, contains a copy of the interaction function which was used to produce these results.<br />
<br />
''meshes'' - This folder contains the mesh files corresponding to the stages of growth specified by the value of the Stages argument, described in 3A.<br />
|width="500px" align="center"|[[File:resultsdir.png| Directory structure of a project containing results]]<br />
|}<br />
<br />
<br />
The mesh files contain vertices information regarding the shape of the growth model at a particular stage of growth, but are not visualisable in that form. In order to '''convert these mesh files into viewable images''', we can execute the following command in Matlab:<br />
<br />
VMSreport('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Project','GPT_CASE_RST',...<br />
'Experiment','All','flattentime',572.5,'morphogen','KPAR','SNAPFIG',true);<br />
<br />
Where the ''Path'' and ''Project'' arguments have the same function as ''Path'' and ''Name'' in 3A, i.e. ''Path'' refers to the location of the project folder and ''Project'' is the name of the project folder itself.<br />
<br />
Here we can see the generated images, including the command line arguments, for one of the simulations performed in 3A. Click on a thumbnail to view a larger image.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="125px"| <small>001commandline.png</small><br> [[File:commandlineexample.png | 100px]]<br />
|width="125px"| <small>001GPT_CASE_RST_2 wild_.png</small><br>[[File:GPT_CASE_RST_example1.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_3 wild_.png</small><br>[[File:GPT_CASE_RST_example2.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_4 wild_.png</small><br>[[File:GPT_CASE_RST_example3.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_5 wild_.png</small><br>[[File:GPT_CASE_RST_example4.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_6 wild_.png</small><br>[[File:GPT_CASE_RST_example5.png | 125px]]<br />
|}<br />
<br />
<br />
'''2) - Generating images from a simulation you ran on the cluster'''<br />
<br />
NB. The results produced may not be visibly identical to those in the published literature. This is because of small, random perturbations which are applied to the initial model meshes to stop them from containing surfaces which are perfectly flat, and therefore biologically unrealistic. The results produced should be '''qualitatively but not quantitatively the same'''.<br />
<br />
'''3) - Interacting with your results using the GFtbox GUI'''<br />
<br />
='''4''' Altering the simulation parameters=<br />
<br />
One such simulation parameter is ''dt'', which is the time in seconds between iterations in the growth simulation. Large values of ''dt'' mean that fewer steps and therefore fewer calculations are required to complete a simulation. Whereas smaller values of ''dt'' mean that more steps are required, and therefore more processing time too. Therefore, a value of ''dt'' must be selected which is not so small that it is computationally unmanageable, but not so large that the observed growth is an artifact of the value of ''dt'', rather than the underlying growth model. It is necessary therefore to test a range of values for ''dt'' to ensure that the patterns of growth observed in a simulation are consistent across the range, and to find a value which is sufficient to demonstrate the model of growth and computationally efficient.<br />
<br />
'''Testing a range of ''dt''s can be achieved in several ways.''' One way is to use the ''dt'' argument when calling GFtboxCommand. This allows a single value to be tested. Another way is to make a batch of jobs, each using a different value for ''dt'', by using the 'State' argument. Lastly, a range of ''dt''s can be specified in GFtboxCommand, by using the dt argument where the values within the square brackets are the ''dt''s to simulate:<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5]);<br />
<br />
Here, we have specified '''four separate values for ''dt''''', and this means that '''four separate simulations will be run''' sequentially on your computer. As in 2B, images can then be generated for the mesh files produced, and compared to ensure consistency across the range of ''dt''s. In this figure, we can see the same model, at the same stage of growth, generated using the four values of ''dt''. Though quantitatively different, they are qualitatively the same, illustrating the suitability of the default ''dt'' value of 5.<br />
<br />
='''5''' Altering the model parameters=<br />
It is easy to see that when even a small number of range variables, mutations or ''dt''s are specified, the '''total number of simulations increases quickly'''. If many ranges are specified, then the amount of processing time becomes unmanageable on a single computer. Via ''GFtboxCommand'' and the ''ClusterMonitor'' tool, we can remotely''' run a number of simulations, in parallel, on a computing cluster'''. It is assumed that the GrowthToolbox and Matlab are installed on your cluster, and PuTTY is required on your local computer for the tools ''pscp'' (for transferring files from your computer to a cluster) and ''plink'' (for remotely executing commands on a cluster). Using the ''dt'' range argument from 3 as an example, here we add the 'Use' name argument with the value 'Cluster':<br />
<br />
GFtboxCommand('State','Start','Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5],'Use','Cluster');<br />
<br />
The 'Cluster' option for the 'Use' name argument instructs GFtboxCommand to upload the required project directory to a remote Linux server. Instead of running the simulation on your own computer, GFtboxCommand then works out how many individual simulations are specified by the command that invoked it. In this example, the only argument which will generate multiple simulations is ''dt'', of which there will be 4. Separate commands for each of these jobs are then automatically generated, each with an accompanying unique ID (The value of the ''ExpID'' name argument), and '''these are submitted as individual jobs to the cluster'''.<br />
<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.1,'ExpID','GPT_CASE_RST_1');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.5,'ExpID','GPT_CASE_RST_2');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',1,'ExpID','GPT_CASE_RST_3');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',5,'ExpID','GPT_CASE_RST_4');<br />
<br />
Once these jobs have been submitted, the ''ClusterMonitor'' tool opens and the new job batch ID is added to the list of jobs. Here, we will concentrate on three of the functions of ''ClusterMonitor'': ''Queue?'', ''Get project results'' and ''Make project pngs''. As in part 3 of this page, these functions will enable us to visualise the growth simulation results.<br />
<br />
Whilst ''ClusterMonitor'' is a useful tool for managing your cluster jobs, it is advisable to '''have at least a rudimentary understanding of and ability to use a Unix-based computer system'''. In particular, the abilities to list the contents of a folder, view the contents of a file, change the current working folder, delete, and to copy or move files or folders, are essential Unix skills for making sure everything is working as intended. It is beyond the scope of these tutorials to provide an in depth Unix tutorial (many good and simple tutorials exist on the web), but here is a short description of the Unix commands (which may be specific to our Unix-based system) that we believe to be important.</div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Running_example_models_and_using_a_cluster&diff=5373Running example models and using a cluster2011-06-23T13:24:51Z<p>JacobNewman: </p>
<hr />
<div>[[GFtbox Tutorial pages | Back to GFtbox Tutorial pages]]<br />
<br />
The purpose of these tutorials is to learn how to run the example growth simulations included in GFtbox. We will describe methods for running simulations locally (on your own computer) and remotely (on a computing cluster). It is assumed that you have already downloaded the GFtbox software and have Matlab installed.<br />
<br />
=Getting Started=<br />
<br />
The remainder of this page is split into five sub-tutorials, each building on the preceding parts.<br />
<br />
1) '''Explaining the tools'''. In these tutorials, we will be using ''GFtboxCommand'' and ''ClusterMonitor''. This section explains the purpose of these tools. <br />
<br />
2) '''Computer or cluster?''' Here, we illustrate when and why you should use a computing cluster for your growth simulations, and conversely, when you should use your desktop computer.<br />
<br />
3) '''Running a growth simulation for an example model'''. This section demonstrates how to use ''GFtboxCommand'' to run a growth simulation on your computer. The model used for the simulation is an example model included with the GFtbox.<br />
<br />
4) '''Altering the simulation parameters'''. Following on from part 3, here we show how to adjust a simulation parameter within GFtboxCommand. Specifically, we alter the value of ''dt'', the temporal resolution of the simulation, and show how it can be used to verify that the value specified in the published literature is reasonable.<br />
<br />
5) '''Altering the model parameters'''. Finally, we demonstrate how a number of model parameters can be varied by specifying a range of values for each model argument. We show how the computationally expensive task of simulating all combinations of specified ranges can be processed more efficiently if a computer cluster is used via the ''ClusterMonitor'' tool.<br />
<br />
='''1''' Explaining the tools=<br />
==GFtboxCommand==<br />
This is a command line version of the GFtbox. By command line, we mean that all program '''functions are operated via typed commands, without the GUI'''. Like GFtbox, GFtboxCommand is capable of running growth simulations of an interaction function, and allows the user to specify model and simulation parameters. Unlike GFtbox, this also allows the user to select '''ranges of values for a number of input parameters''', and will automatically spawn multiple simulations which explore the various combinations of those parameters. This can be used to evaluate the effect of various parameters on the growth of a given model.<br />
<br />
==ClusterMonitor==<br />
Provides a graphical user interface for managing simulations '''running remotely on a computer cluster'''. Specifically, it allows you to see which projects are present and running on the cluster, to retrieve the completed projects, to generate images of the simulations at specified stages of growth, and to remove projects from the cluster. If you do not intend to use a computer cluster, then you will not need to use ClusterMonitor.<br />
<br />
='''2''' Computer or cluster?=<br />
<br />
In basic terms, '''a computing cluster is effectively a network of many computer processors''' (often hundreds), centrally managed by a queuing system. When a job is submitted to a cluster, the job is sent to a processor that is not being used, or queued until one becomes available. In contrast, a typical desktop computer will contain one processor, limiting the number of tasks that can be performed at any one time. Jobs which must be run independently and sequentially on a desktop computer can be executed in parallel on a cluster, '''greatly reducing the total time required to complete all of the jobs'''. Although the exact details of your cluster might vary from those described in the rest of these tutorials, we aim to illustrate the generic processes involved in using GFtbox on a cluster. We are happy to offer assistance, where possible, to setup GFtbox for your cluster, so please contact us if you have any queries.<br />
<br />
Whilst the time savings offered by using a cluster can be significant, there is an overhead associated with returning the results to your personal computer. The total time to run 2 or more simulations on a cluster and to return the results will be less than running those simulations sequentially on one computer. Therefore, using a '''cluster is ideal for situations where you would like to run several simulations''', such as to evaluate the effect of a range of parameters on a growth model. You are not advised to use a cluster for running single simulations, or where you would like to step through a simulation and change parameters in a more interactive fashion. In such circumstances, you are advised to run GFtbox or GFtboxCommand on your own computer.<br />
<br />
The '''GFtbox GUI provides quick feedback''' about how the changes you have made to an interaction function have affected the growth simulation, as '''you can see the result of each simulation iteration as it completes'''. It is quick and easy therefore to see if you have dramatically changed the course of the growth simulation, and then to adjust the parameters according to your observations. This approach is well suited to the early stages of design, where you might wish to '''tweak some parameters to gauge whether or not they have had the desired effect'''. Or in the final stages, where you think your model is almost finalised, but where small adjustments are required. '''An alternative approach is to start many simulations''', based upon your initial model and by making intelligent choices about which parameters to explore. This allows you to '''harvest many results, to quickly and easily overview them''' in their finalised form, or to select interesting-looking simulations and examine them more closely on your desktop computer, using GFtbox.<br />
<br />
='''3''' Running a growth simulation for an example model=<br />
This tutorial is aimed at running a '''growth simulation for one of the example interaction functions''' included with the GFtbox. The purpose of this exercise is to firstly demonstrate how simulations can be invoked using GFtboxCommand, and secondly to show how to reproduce experimental results (specifically, those published in Kennaway et al (2011)) given an interaction function.<br />
<br />
Assuming Matlab is installed on your computer, and the latest GFtbox has been downloaded, you can add the GFtbox directory to Matlab's search path, which makes the toolbox accessible from any other path that you choose to work from. For a short tutorial on how to do this, please click [[Adding GFtbox to Matlab's search path | here]].<br />
<br />
Once the GFtbox is added to Matlab, you are ready to run a growth simulation using GFtboxCommand. In this example, '''the model that we will simulate is called GPT_CASE_RST'''. Results generated using this model are published in Kennaway et al (2011). By running this simulation, we can confirm the results in the published literature and investigate the suitability of the various parameters.<br />
<br />
===3A - Running a simulation on your computer===<br />
<br />
The following command can be typed into Matlab to run a simulation of the GPT_CASE_RST interaction function, which contains three growth models: R, S and T. '''Three separate simulations are run sequentially on your computer''', one for each model, each producing results corresponding to five intervals in the growth simulation.<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',[1:3]);<br />
<br />
GFtboxCommand accepts input arguments as name and value pairs, e.g. 'modelname', [1:3] or 'Use','Cluster'. The argument names entered in the example above are: ''Path'', ''Name'', ''Stages'' and ''modelname''.<br />
<br />
The optional '''''Path'' argument name refers to the location of the folder''' (or directory) on your computer where the model interaction function you wish to simulate is stored. '''''Name'', is the name of the folder itself'''. In this case, we are using the GPT_CASE_RST folder which is included in the GrowthToolbox. You may wish to copy this folder elsewhere, if you intend to make changes to the interaction function.<br />
<br />
During a growth simulation, a mesh can be generated at each time step of the simulation, which provides a visual representation of the growth of the biological tissue, given the various parameters of growth specified by the interaction function and how they have changed over time. Put another way, the mesh shows exactly what the growing tissue actually looks like. ''Stages'' refers to the points in the simulation (measured in hours) at which meshes should be generated and saved. In this example, '''five stages of growth will be written to disk'''. These values are chosen to best capture the appearance of the tissue at important stages of the tissue growth.<br />
<br />
The final argument name listed here is ''modelname''. This is a model-specific argument, and in this case the GPT_CASE_RST interaction function contains '''three separate models for plant growth''', allowing the desired model to be a selected. Here, the value [1:3] is specified, which is evaluated in the same way as entering [1 2 3]. This instructs GFtboxCommand to run three separate simulations, one for each of the growth models contained in the interaction function.<br />
<br />
The function of every permissible argument is given by keying the following command into Matlab:<br />
<br />
help GFtboxCommand<br />
<br />
===3B - Getting results from a completed simulation===<br />
<br />
'''1) - Generating images from a simulation you ran on your computer'''<br />
<br />
Once a growth simulation is completed, the project folder (GPT_CASE_RST, in our example) will contain another folder named "movies". Within movies, there are folders which contain results for the executed simulations. As was instructed in 3A, here we can see three folders corresponding to simulations for the three separate models. Within the first folder there are three items:<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| ''CommandLine.txt'' - This file contains the Matlab command which was used to generate the results that this sub-directory contains.<br />
<br />
''gpt_case_rst.txt'' - This file, named according to the project name, contains a copy of the interaction function which was used to produce these results.<br />
<br />
''meshes'' - This folder contains the mesh files corresponding to the stages of growth specified by the value of the Stages argument, described in 3A.<br />
|width="500px" align="center"|[[File:resultsdir.png| Directory structure of a project containing results]]<br />
|}<br />
<br />
<br />
The mesh files contain vertices information regarding the shape of the growth model at a particular stage of growth, but are not visualisable in that form. In order to '''convert these mesh files into viewable images''', we can execute the following command in Matlab:<br />
<br />
VMSreport('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Project','GPT_CASE_RST',...<br />
'Experiment','All','flattentime',572.5,'morphogen','KPAR','SNAPFIG',true);<br />
<br />
Where the ''Path'' and ''Project'' arguments have the same function as ''Path'' and ''Name'' in 3A, i.e. ''Path'' refers to the location of the project folder and ''Project'' is the name of the project folder itself.<br />
<br />
Here we can see the generated images, including the command line arguments, for one of the simulations performed in 3A. Click on a thumbnail to view a larger image.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="125px"| <small>001commandline.png</small><br> [[File:commandlineexample.png | 100px]]<br />
|width="125px"| <small>001GPT_CASE_RST_2 wild_.png</small><br>[[File:GPT_CASE_RST_example1.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_3 wild_.png</small><br>[[File:GPT_CASE_RST_example2.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_4 wild_.png</small><br>[[File:GPT_CASE_RST_example3.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_5 wild_.png</small><br>[[File:GPT_CASE_RST_example4.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_6 wild_.png</small><br>[[File:GPT_CASE_RST_example5.png | 125px]]<br />
|}<br />
<br />
<br />
'''2) - Generating images from a simulation you ran on the cluster'''<br />
<br />
NB. The results produced may not be visibly identical to those in the published literature. This is because of small, random perturbations which are applied to the initial model meshes to stop them from containing surfaces which are perfectly flat, and therefore biologically unrealistic. The results produced should be '''qualitatively but not quantitatively the same'''.<br />
<br />
'''3) - Interacting with your results using the GFtbox GUI'''<br />
<br />
='''4''' Altering the simulation parameters=<br />
<br />
One such simulation parameter is ''dt'', which is the time in seconds between iterations in the growth simulation. Large values of ''dt'' mean that fewer steps and therefore fewer calculations are required to complete a simulation. Whereas smaller values of ''dt'' mean that more steps are required, and therefore more processing time too. Therefore, a value of ''dt'' must be selected which is not so small that it is computationally unmanageable, but not so large that the observed growth is an artifact of the value of ''dt'', rather than the underlying growth model. It is necessary therefore to test a range of values for ''dt'' to ensure that the patterns of growth observed in a simulation are consistent across the range, and to find a value which is sufficient to demonstrate the model of growth and computationally efficient.<br />
<br />
'''Testing a range of ''dt''s can be achieved in several ways.''' One way is to use the ''dt'' argument when calling GFtboxCommand. This allows a single value to be tested. Another way is to make a batch of jobs, each using a different value for ''dt'', by using the 'State' argument, as described here. Lastly, a range of ''dt''s can be specified in GFtboxCommand, by using the dt argument where the values within the square brackets are the ''dt''s to simulate:<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5]);<br />
<br />
Here, we have specified '''four separate values for ''dt''''', and this means that '''four separate simulations will be run''' sequentially on your computer. As in 2B, images can then be generated for the mesh files produced, and compared to ensure consistency across the range of ''dt''s. In this figure, we can see the same model, at the same stage of growth, generated using the four values of ''dt''. Though quantitatively different, they are qualitatively the same, illustrating the suitability of the default ''dt'' value of 5.<br />
<br />
='''5''' Altering the model parameters=<br />
It is easy to see that when even a small number of range variables, mutations or ''dt''s are specified, the '''total number of simulations increases quickly'''. If many ranges are specified, then the amount of processing time becomes unmanageable on a single computer. Via ''GFtboxCommand'' and the ''ClusterMonitor'' tool, we can remotely''' run a number of simulations, in parallel, on a computing cluster'''. It is assumed that the GrowthToolbox and Matlab are installed on your cluster, and PuTTY is required on your local computer for the tools ''pscp'' (for transferring files from your computer to a cluster) and ''plink'' (for remotely executing commands on a cluster). Using the ''dt'' range argument from 3 as an example, here we add the 'Use' name argument with the value 'Cluster':<br />
<br />
GFtboxCommand('State','Start','Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5],'Use','Cluster');<br />
<br />
The 'Cluster' option for the 'Use' name argument instructs GFtboxCommand to upload the required project directory to a remote Linux server. Instead of running the simulation on your own computer, GFtboxCommand then works out how many individual simulations are specified by the command that invoked it. In this example, the only argument which will generate multiple simulations is ''dt'', of which there will be 4. Separate commands for each of these jobs are then automatically generated, each with an accompanying unique ID (The value of the ''ExpID'' name argument), and '''these are submitted as individual jobs to the cluster'''.<br />
<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.1,'ExpID','GPT_CASE_RST_1');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.5,'ExpID','GPT_CASE_RST_2');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',1,'ExpID','GPT_CASE_RST_3');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',5,'ExpID','GPT_CASE_RST_4');<br />
<br />
Once these jobs have been submitted, the ''ClusterMonitor'' tool opens and the new job batch ID is added to the list of jobs. Here, we will concentrate on three of the functions of ''ClusterMonitor'': ''Queue?'', ''Get project results'' and ''Make project pngs''. As in part 3 of this page, these functions will enable us to visualise the growth simulation results.<br />
<br />
Whilst ''ClusterMonitor'' is a useful tool for managing your cluster jobs, it is advisable to '''have at least a rudimentary understanding of and ability to use a Unix-based computer system'''. In particular, the abilities to list the contents of a folder, view the contents of a file, change the current working folder, delete, and to copy or move files or folders, are essential Unix skills for making sure everything is working as intended. It is beyond the scope of these tutorials to provide an in depth Unix tutorial (many good and simple tutorials exist on the web), but here is a short description of the Unix commands (which may be specific to our Unix-based system) that we believe to be important.</div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Running_example_models_and_using_a_cluster&diff=5372Running example models and using a cluster2011-06-23T13:23:54Z<p>JacobNewman: </p>
<hr />
<div>[[GFtbox Tutorial pages | Back to GFtbox Tutorial pages]]<br />
<br />
The purpose of these tutorials is to learn how to run the example growth simulations included in GFtbox. We will describe methods for running simulations locally (on your own computer) and remotely (on a computing cluster). It is assumed that you have already downloaded the GFtbox software and have Matlab installed.<br />
<br />
=Getting Started=<br />
<br />
The remainder of this page is split into five sub-tutorials, each building on the preceding parts.<br />
<br />
1) '''Explaining the tools'''. In these tutorials, we will be using ''GFtboxCommand'' and ''ClusterMonitor''. This section explains the purpose of these tools. <br />
<br />
2) '''Computer or cluster?''' Here, we illustrate when and why you should use a computing cluster for your growth simulations, and conversely, when you should use your desktop computer.<br />
<br />
3) '''Running a growth simulation for an example model'''. This section demonstrates how to use ''GFtboxCommand'' to run a growth simulation on your computer. The model used for the simulation is an example model included with the GFtbox.<br />
<br />
4) '''Altering the simulation parameters'''. Following on from part 3, here we show how to adjust a simulation parameter within GFtboxCommand. Specifically, we alter the value of ''dt'', the temporal resolution of the simulation, and show how it can be used to verify that the value specified in the published literature is reasonable.<br />
<br />
5) '''Altering the model parameters'''. Finally, we demonstrate how a number of model parameters can be varied by specifying a range of values for each model argument. We show how the computationally expensive task of simulating all combinations of specified ranges can be processed more efficiently if a computer cluster is used via the ''ClusterMonitor'' tool.<br />
<br />
='''1''' Explaining the tools=<br />
==GFtboxCommand==<br />
This is a command line version of the GFtbox. By command line, we mean that all program '''functions are operated via typed commands, without the GUI'''. Like GFtbox, GFtboxCommand is capable of running growth simulations of an interaction function, and allows the user to specify model and simulation parameters. Unlike GFtbox, this also allows the user to select '''ranges of values for a number of input parameters''', and will automatically spawn multiple simulations which explore the various combinations of those parameters. This can be used to evaluate the effect of various parameters on the growth of a given model.<br />
<br />
==ClusterMonitor==<br />
Provides a graphical user interface for managing simulations '''running remotely on a computer cluster'''. Specifically, it allows you to see which projects are present and running on the cluster, to retrieve the completed projects, to generate images of the simulations at specified stages of growth, and to remove projects from the cluster. If you do not intend to use a computer cluster, then you will not need to use ClusterMonitor.<br />
<br />
='''2''' Computer or cluster?=<br />
<br />
In basic terms, '''a computing cluster is effectively a network of many computer processors''' (often hundreds), centrally managed by a queuing system. When a job is submitted to a cluster, the job is sent to a processor that is not being used, or queued until one becomes available. In contrast, a typical desktop computer will contain one processor, limiting the number of tasks that can be performed at any one time. Jobs which must be run independently and sequentially on a desktop computer can be executed in parallel on a cluster, '''greatly reducing the total time required to complete all of the jobs'''. Although the exact details of your cluster might vary from those described in the rest of these tutorials, we aim to illustrate the generic processes involved in using GFtbox on a cluster. We are happy to offer assistance, where possible, to setup GFtbox for your cluster, so please contact us if you have any queries.<br />
<br />
Whilst the time savings offered by using a cluster can be significant, there is an overhead associated with returning the results to your personal computer. The total time to run 2 or more simulations on a cluster and to return the results will be less than running those simulations sequentially on one computer. Therefore, using a '''cluster is ideal for situations where you would like to run several simulations''', such as to evaluate the effect of a range of parameters on a growth model. You are not advised to use a cluster for running single simulations, or where you would like to step through a simulation and change parameters in a more interactive fashion. In such circumstances, you are advised to run GFtbox or GFtboxCommand on your own computer.<br />
<br />
The '''GFtbox GUI provides quick feedback''' about how the changes you have made to an interaction function have affected the growth simulation, as '''you can see the result of each simulation iteration as it completes'''. It is quick and easy therefore to see if you have dramatically changed the course of the growth simulation, and then to adjust the parameters according to your observations. This approach is well suited to the early stages of design, where you might wish to '''tweak some parameters to gauge whether or not they have had the desired effect'''. Or in the final stages, where you think your model is almost finalised, but where small adjustments are required. '''An alternative approach is to start many simulations''', based upon your initial model and by making intelligent choices about which parameters to explore. This allows you to '''harvest many results, to quickly and easily overview them''' in their finalised form, or to select interesting-looking simulations and examine them more closely on your desktop computer, using GFtbox.<br />
<br />
='''3''' Running a growth simulation for an example model=<br />
This tutorial is aimed at running a '''growth simulation for one of the example interaction functions''' included with the GFtbox. The purpose of this exercise is to firstly demonstrate how simulations can be invoked using GFtboxCommand, and secondly to show how to reproduce experimental results (specifically, those published in Kennaway et al (2011)) given an interaction function.<br />
<br />
Assuming Matlab is installed on your computer, and the latest GFtbox has been downloaded, you can add the GFtbox directory to Matlab's search path, which makes the toolbox accessible from any other path that you choose to work from. For a short tutorial on how to do this, please click [[Adding GFtbox to Matlab's search path | here]].<br />
<br />
Once the GFtbox is added to Matlab, you are ready to run a growth simulation using GFtboxCommand. In this example, '''the model that we will simulate is called GPT_CASE_RST'''. Results generated using this model are published in Kennaway et al (2011). By running this simulation, we can confirm the results in the published literature and investigate the suitability of the various parameters.<br />
<br />
===3A - Running a simulation on your computer===<br />
<br />
The following command can be typed into Matlab to run a simulation of the GPT_CASE_RST interaction function, which contains three growth models: R, S and T. '''Three separate simulations are run sequentially on your computer''', one for each model, each producing results corresponding to five intervals in the growth simulation.<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',[1:3]);<br />
<br />
GFtboxCommand accepts input arguments as name and value pairs, e.g. 'modelname', [1:3] or 'Use','Cluster'. The argument names entered in the example above are: ''Path'', ''Name'', ''Stages'' and ''modelname''.<br />
<br />
The optional '''''Path'' argument name refers to the location of the folder''' (or directory) on your computer where the model interaction function you wish to simulate is stored. '''''Name'', is the name of the folder itself'''. In this case, we are using the GPT_CASE_RST folder which is included in the GrowthToolbox. You may wish to copy this folder elsewhere, if you intend to make changes to the interaction function.<br />
<br />
During a growth simulation, a mesh can be generated at each time step of the simulation, which provides a visual representation of the growth of the biological tissue, given the various parameters of growth specified by the interaction function and how they have changed over time. Put another way, the mesh shows exactly what the growing tissue actually looks like. ''Stages'' refers to the points in the simulation (measured in hours) at which meshes should be generated and saved. In this example, '''five stages of growth will be written to disk'''. These values are chosen to best capture the appearance of the tissue at important stages of the tissue growth.<br />
<br />
The final argument name listed here is ''modelname''. This is a model-specific argument, and in this case the GPT_CASE_RST interaction function contains '''three separate models for plant growth''', allowing the desired model to be a selected. Here, the value [1:3] is specified, which is evaluated in the same way as entering [1 2 3]. This instructs GFtboxCommand to run three separate simulations, one for each of the growth models contained in the interaction function.<br />
<br />
The function of every permissible argument is given by keying the following command into Matlab:<br />
<br />
help GFtboxCommand<br />
<br />
===3B - Getting results from a completed simulation===<br />
<br />
'''1) - Generating images from a simulation you ran on your computer'''<br />
<br />
Once a growth simulation is completed, the project folder (GPT_CASE_RST, in our example) will contain another folder named "movies". Within movies, there are folders which contain results for the executed simulations. As was instructed in 3A, here we can see three folders corresponding to simulations for the three separate models. Within the first folder there are three items:<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| ''CommandLine.txt'' - This file contains the Matlab command which was used to generate the results that this sub-directory contains.<br />
<br />
''gpt_case_rst.txt'' - This file, named according to the project name, contains a copy of the interaction function which was used to produce these results.<br />
<br />
''meshes'' - This folder contains the mesh files corresponding to the stages of growth specified by the value of the Stages argument, described in 3A.<br />
|width="500px" align="center"|[[File:resultsdir.png| Directory structure of a project containing results]]<br />
|}<br />
<br />
<br />
The mesh files contain vertices information regarding the shape of the growth model at a particular stage of growth, but are not visualisable in that form. In order to '''convert these mesh files into viewable images''', we can execute the following command in Matlab:<br />
<br />
VMSreport('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Project','GPT_CASE_RST',...<br />
'Experiment','All','flattentime',572.5,'morphogen','KPAR','SNAPFIG',true);<br />
<br />
Where the ''Path'' and ''Project'' arguments have the same function as ''Path'' and ''Name'' in 3A, i.e. ''Path'' refers to the location of the project folder and ''Project'' is the name of the project folder itself.<br />
<br />
Here we can see the generated images, including the command line arguments, for one of the simulations performed in 3A. Click on a thumbnail to view a larger image.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="125px"| <small>001commandline.png</small><br> [[File:commandlineexample.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_2 wild_.png</small><br>[[File:GPT_CASE_RST_example1.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_3 wild_.png</small><br>[[File:GPT_CASE_RST_example2.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_4 wild_.png</small><br>[[File:GPT_CASE_RST_example3.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_5 wild_.png</small><br>[[File:GPT_CASE_RST_example4.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_6 wild_.png</small><br>[[File:GPT_CASE_RST_example5.png | 125px]]<br />
|}<br />
<br />
<br />
'''2) - Generating images from a simulation you ran on the cluster'''<br />
<br />
NB. The results produced may not be visibly identical to those in the published literature. This is because of small, random perturbations which are applied to the initial model meshes to stop them from containing surfaces which are perfectly flat, and therefore biologically unrealistic. The results produced should be '''qualitatively but not quantitatively the same'''.<br />
<br />
'''3) - Interacting with your results using the GFtbox GUI'''<br />
<br />
='''4''' Altering the simulation parameters=<br />
<br />
One such simulation parameter is ''dt'', which is the time in seconds between iterations in the growth simulation. Large values of ''dt'' mean that fewer steps and therefore fewer calculations are required to complete a simulation. Whereas smaller values of ''dt'' mean that more steps are required, and therefore more processing time too. Therefore, a value of ''dt'' must be selected which is not so small that it is computationally unmanageable, but not so large that the observed growth is an artifact of the value of ''dt'', rather than the underlying growth model. It is necessary therefore to test a range of values for ''dt'' to ensure that the patterns of growth observed in a simulation are consistent across the range, and to find a value which is sufficient to demonstrate the model of growth and computationally efficient.<br />
<br />
'''Testing a range of ''dt''s can be achieved in several ways.''' One way is to use the ''dt'' argument when calling GFtboxCommand. This allows a single value to be tested. Another way is to make a batch of jobs, each using a different value for ''dt'', by using the 'State' argument, as described here. Lastly, a range of ''dt''s can be specified in GFtboxCommand, by using the dt argument where the values within the square brackets are the ''dt''s to simulate:<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5]);<br />
<br />
Here, we have specified '''four separate values for ''dt''''', and this means that '''four separate simulations will be run''' sequentially on your computer. As in 2B, images can then be generated for the mesh files produced, and compared to ensure consistency across the range of ''dt''s. In this figure, we can see the same model, at the same stage of growth, generated using the four values of ''dt''. Though quantitatively different, they are qualitatively the same, illustrating the suitability of the default ''dt'' value of 5.<br />
<br />
='''5''' Altering the model parameters=<br />
It is easy to see that when even a small number of range variables, mutations or ''dt''s are specified, the '''total number of simulations increases quickly'''. If many ranges are specified, then the amount of processing time becomes unmanageable on a single computer. Via ''GFtboxCommand'' and the ''ClusterMonitor'' tool, we can remotely''' run a number of simulations, in parallel, on a computing cluster'''. It is assumed that the GrowthToolbox and Matlab are installed on your cluster, and PuTTY is required on your local computer for the tools ''pscp'' (for transferring files from your computer to a cluster) and ''plink'' (for remotely executing commands on a cluster). Using the ''dt'' range argument from 3 as an example, here we add the 'Use' name argument with the value 'Cluster':<br />
<br />
GFtboxCommand('State','Start','Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5],'Use','Cluster');<br />
<br />
The 'Cluster' option for the 'Use' name argument instructs GFtboxCommand to upload the required project directory to a remote Linux server. Instead of running the simulation on your own computer, GFtboxCommand then works out how many individual simulations are specified by the command that invoked it. In this example, the only argument which will generate multiple simulations is ''dt'', of which there will be 4. Separate commands for each of these jobs are then automatically generated, each with an accompanying unique ID (The value of the ''ExpID'' name argument), and '''these are submitted as individual jobs to the cluster'''.<br />
<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.1,'ExpID','GPT_CASE_RST_1');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.5,'ExpID','GPT_CASE_RST_2');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',1,'ExpID','GPT_CASE_RST_3');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',5,'ExpID','GPT_CASE_RST_4');<br />
<br />
Once these jobs have been submitted, the ''ClusterMonitor'' tool opens and the new job batch ID is added to the list of jobs. Here, we will concentrate on three of the functions of ''ClusterMonitor'': ''Queue?'', ''Get project results'' and ''Make project pngs''. As in part 3 of this page, these functions will enable us to visualise the growth simulation results.<br />
<br />
Whilst ''ClusterMonitor'' is a useful tool for managing your cluster jobs, it is advisable to '''have at least a rudimentary understanding of and ability to use a Unix-based computer system'''. In particular, the abilities to list the contents of a folder, view the contents of a file, change the current working folder, delete, and to copy or move files or folders, are essential Unix skills for making sure everything is working as intended. It is beyond the scope of these tutorials to provide an in depth Unix tutorial (many good and simple tutorials exist on the web), but here is a short description of the Unix commands (which may be specific to our Unix-based system) that we believe to be important.</div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Running_example_models_and_using_a_cluster&diff=5397Running example models and using a cluster2011-06-23T13:23:13Z<p>JacobNewman: </p>
<hr />
<div>[[GFtbox Tutorial pages | Back to GFtbox Tutorial pages]]<br />
<br />
The purpose of these tutorials is to learn how to run the example growth simulations included in GFtbox. We will describe methods for running simulations locally (on your own computer) and remotely (on a computing cluster). It is assumed that you have already downloaded the GFtbox software and have Matlab installed.<br />
<br />
=Getting Started=<br />
<br />
The remainder of this page is split into five sub-tutorials, each building on the preceding parts.<br />
<br />
1) '''Explaining the tools'''. In these tutorials, we will be using ''GFtboxCommand'' and ''ClusterMonitor''. This section explains the purpose of these tools. <br />
<br />
2) '''Computer or cluster?''' Here, we illustrate when and why you should use a computing cluster for your growth simulations, and conversely, when you should use your desktop computer.<br />
<br />
3) '''Running a growth simulation for an example model'''. This section demonstrates how to use ''GFtboxCommand'' to run a growth simulation on your computer. The model used for the simulation is an example model included with the GFtbox.<br />
<br />
4) '''Altering the simulation parameters'''. Following on from part 3, here we show how to adjust a simulation parameter within GFtboxCommand. Specifically, we alter the value of ''dt'', the temporal resolution of the simulation, and show how it can be used to verify that the value specified in the published literature is reasonable.<br />
<br />
5) '''Altering the model parameters'''. Finally, we demonstrate how a number of model parameters can be varied by specifying a range of values for each model argument. We show how the computationally expensive task of simulating all combinations of specified ranges can be processed more efficiently if a computer cluster is used via the ''ClusterMonitor'' tool.<br />
<br />
='''1''' Explaining the tools=<br />
==GFtboxCommand==<br />
This is a command line version of the GFtbox. By command line, we mean that all program '''functions are operated via typed commands, without the GUI'''. Like GFtbox, GFtboxCommand is capable of running growth simulations of an interaction function, and allows the user to specify model and simulation parameters. Unlike GFtbox, this also allows the user to select '''ranges of values for a number of input parameters''', and will automatically spawn multiple simulations which explore the various combinations of those parameters. This can be used to evaluate the effect of various parameters on the growth of a given model.<br />
<br />
==ClusterMonitor==<br />
Provides a graphical user interface for managing simulations '''running remotely on a computer cluster'''. Specifically, it allows you to see which projects are present and running on the cluster, to retrieve the completed projects, to generate images of the simulations at specified stages of growth, and to remove projects from the cluster. If you do not intend to use a computer cluster, then you will not need to use ClusterMonitor.<br />
<br />
='''2''' Computer or cluster?=<br />
<br />
In basic terms, '''a computing cluster is effectively a network of many computer processors''' (often hundreds), centrally managed by a queuing system. When a job is submitted to a cluster, the job is sent to a processor that is not being used, or queued until one becomes available. In contrast, a typical desktop computer will contain one processor, limiting the number of tasks that can be performed at any one time. Jobs which must be run independently and sequentially on a desktop computer can be executed in parallel on a cluster, '''greatly reducing the total time required to complete all of the jobs'''. Although the exact details of your cluster might vary from those described in the rest of these tutorials, we aim to illustrate the generic processes involved in using GFtbox on a cluster. We are happy to offer assistance, where possible, to setup GFtbox for your cluster, so please contact us if you have any queries.<br />
<br />
Whilst the time savings offered by using a cluster can be significant, there is an overhead associated with returning the results to your personal computer. The total time to run 2 or more simulations on a cluster and to return the results will be less than running those simulations sequentially on one computer. Therefore, using a '''cluster is ideal for situations where you would like to run several simulations''', such as to evaluate the effect of a range of parameters on a growth model. You are not advised to use a cluster for running single simulations, or where you would like to step through a simulation and change parameters in a more interactive fashion. In such circumstances, you are advised to run GFtbox or GFtboxCommand on your own computer.<br />
<br />
The '''GFtbox GUI provides quick feedback''' about how the changes you have made to an interaction function have affected the growth simulation, as '''you can see the result of each simulation iteration as it completes'''. It is quick and easy therefore to see if you have dramatically changed the course of the growth simulation, and then to adjust the parameters according to your observations. This approach is well suited to the early stages of design, where you might wish to '''tweak some parameters to gauge whether or not they have had the desired effect'''. Or in the final stages, where you think your model is almost finalised, but where small adjustments are required. '''An alternative approach is to start many simulations''', based upon your initial model and by making intelligent choices about which parameters to explore. This allows you to '''harvest many results, to quickly and easily overview them''' in their finalised form, or to select interesting-looking simulations and examine them more closely on your desktop computer, using GFtbox.<br />
<br />
='''3''' Running a growth simulation for an example model=<br />
This tutorial is aimed at running a '''growth simulation for one of the example interaction functions''' included with the GFtbox. The purpose of this exercise is to firstly demonstrate how simulations can be invoked using GFtboxCommand, and secondly to show how to reproduce experimental results (specifically, those published in Kennaway et al (2011)) given an interaction function.<br />
<br />
Assuming Matlab is installed on your computer, and the latest GFtbox has been downloaded, you can add the GFtbox directory to Matlab's search path, which makes the toolbox accessible from any other path that you choose to work from. For a short tutorial on how to do this, please click [[Adding GFtbox to Matlab's search path | here]].<br />
<br />
Once the GFtbox is added to Matlab, you are ready to run a growth simulation using GFtboxCommand. In this example, '''the model that we will simulate is called GPT_CASE_RST'''. Results generated using this model are published in Kennaway et al (2011). By running this simulation, we can confirm the results in the published literature and investigate the suitability of the various parameters.<br />
<br />
===3A - Running a simulation on your computer===<br />
<br />
The following command can be typed into Matlab to run a simulation of the GPT_CASE_RST interaction function, which contains three growth models: R, S and T. '''Three separate simulations are run sequentially on your computer''', one for each model, each producing results corresponding to five intervals in the growth simulation.<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',[1:3]);<br />
<br />
GFtboxCommand accepts input arguments as name and value pairs, e.g. 'modelname', [1:3] or 'Use','Cluster'. The argument names entered in the example above are: ''Path'', ''Name'', ''Stages'' and ''modelname''.<br />
<br />
The optional '''''Path'' argument name refers to the location of the folder''' (or directory) on your computer where the model interaction function you wish to simulate is stored. '''''Name'', is the name of the folder itself'''. In this case, we are using the GPT_CASE_RST folder which is included in the GrowthToolbox. You may wish to copy this folder elsewhere, if you intend to make changes to the interaction function.<br />
<br />
During a growth simulation, a mesh can be generated at each time step of the simulation, which provides a visual representation of the growth of the biological tissue, given the various parameters of growth specified by the interaction function and how they have changed over time. Put another way, the mesh shows exactly what the growing tissue actually looks like. ''Stages'' refers to the points in the simulation (measured in hours) at which meshes should be generated and saved. In this example, '''five stages of growth will be written to disk'''. These values are chosen to best capture the appearance of the tissue at important stages of the tissue growth.<br />
<br />
The final argument name listed here is ''modelname''. This is a model-specific argument, and in this case the GPT_CASE_RST interaction function contains '''three separate models for plant growth''', allowing the desired model to be a selected. Here, the value [1:3] is specified, which is evaluated in the same way as entering [1 2 3]. This instructs GFtboxCommand to run three separate simulations, one for each of the growth models contained in the interaction function.<br />
<br />
The function of every permissible argument is given by keying the following command into Matlab:<br />
<br />
help GFtboxCommand<br />
<br />
===3B - Getting results from a completed simulation===<br />
<br />
'''1) - Generating images from a simulation you ran on your computer'''<br />
<br />
Once a growth simulation is completed, the project folder (GPT_CASE_RST, in our example) will contain another folder named "movies". Within movies, there are folders which contain results for the executed simulations. As was instructed in 3A, here we can see three folders corresponding to simulations for the three separate models. Within the first folder there are three items:<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| ''CommandLine.txt'' - This file contains the Matlab command which was used to generate the results that this sub-directory contains.<br />
<br />
''gpt_case_rst.txt'' - This file, named according to the project name, contains a copy of the interaction function which was used to produce these results.<br />
<br />
''meshes'' - This folder contains the mesh files corresponding to the stages of growth specified by the value of the Stages argument, described in 3A.<br />
|width="500px" align="center"|[[File:resultsdir.png| Directory structure of a project containing results]]<br />
|}<br />
<br />
<br />
The mesh files contain vertices information regarding the shape of the growth model at a particular stage of growth, but are not visualisable in that form. In order to '''convert these mesh files into viewable images''', we can execute the following command in Matlab:<br />
<br />
VMSreport('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Project','GPT_CASE_RST',...<br />
'Experiment','All','flattentime',572.5,'morphogen','KPAR','SNAPFIG',true);<br />
<br />
Where the ''Path'' and ''Project'' arguments have the same function as ''Path'' and ''Name'' in 3A, i.e. ''Path'' refers to the location of the project folder and ''Project'' is the name of the project folder itself.<br />
<br />
Here we can see the generated images, including the command line arguments, for one of the simulations performed in 3A. Click on a thumbnail to view a larger image.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="middle"<br />
|width="125px"| <small>001commandline.png</small><br> [[File:commandlineexample.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_2 wild_.png</small><br>[[File:GPT_CASE_RST_example1.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_3 wild_.png</small><br>[[File:GPT_CASE_RST_example2.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_4 wild_.png</small><br>[[File:GPT_CASE_RST_example3.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_5 wild_.png</small><br>[[File:GPT_CASE_RST_example4.png | 125px]]<br />
|width="125px"| <small>001GPT_CASE_RST_6 wild_.png</small><br>[[File:GPT_CASE_RST_example5.png | 125px]]<br />
|}<br />
<br />
<br />
'''2) - Generating images from a simulation you ran on the cluster'''<br />
<br />
NB. The results produced may not be visibly identical to those in the published literature. This is because of small, random perturbations which are applied to the initial model meshes to stop them from containing surfaces which are perfectly flat, and therefore biologically unrealistic. The results produced should be '''qualitatively but not quantitatively the same'''.<br />
<br />
'''3) - Interacting with your results using the GFtbox GUI'''<br />
<br />
='''4''' Altering the simulation parameters=<br />
<br />
One such simulation parameter is ''dt'', which is the time in seconds between iterations in the growth simulation. Large values of ''dt'' mean that fewer steps and therefore fewer calculations are required to complete a simulation. Whereas smaller values of ''dt'' mean that more steps are required, and therefore more processing time too. Therefore, a value of ''dt'' must be selected which is not so small that it is computationally unmanageable, but not so large that the observed growth is an artifact of the value of ''dt'', rather than the underlying growth model. It is necessary therefore to test a range of values for ''dt'' to ensure that the patterns of growth observed in a simulation are consistent across the range, and to find a value which is sufficient to demonstrate the model of growth and computationally efficient.<br />
<br />
'''Testing a range of ''dt''s can be achieved in several ways.''' One way is to use the ''dt'' argument when calling GFtboxCommand. This allows a single value to be tested. Another way is to make a batch of jobs, each using a different value for ''dt'', by using the 'State' argument, as described here. Lastly, a range of ''dt''s can be specified in GFtboxCommand, by using the dt argument where the values within the square brackets are the ''dt''s to simulate:<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5]);<br />
<br />
Here, we have specified '''four separate values for ''dt''''', and this means that '''four separate simulations will be run''' sequentially on your computer. As in 2B, images can then be generated for the mesh files produced, and compared to ensure consistency across the range of ''dt''s. In this figure, we can see the same model, at the same stage of growth, generated using the four values of ''dt''. Though quantitatively different, they are qualitatively the same, illustrating the suitability of the default ''dt'' value of 5.<br />
<br />
='''5''' Altering the model parameters=<br />
It is easy to see that when even a small number of range variables, mutations or ''dt''s are specified, the '''total number of simulations increases quickly'''. If many ranges are specified, then the amount of processing time becomes unmanageable on a single computer. Via ''GFtboxCommand'' and the ''ClusterMonitor'' tool, we can remotely''' run a number of simulations, in parallel, on a computing cluster'''. It is assumed that the GrowthToolbox and Matlab are installed on your cluster, and PuTTY is required on your local computer for the tools ''pscp'' (for transferring files from your computer to a cluster) and ''plink'' (for remotely executing commands on a cluster). Using the ''dt'' range argument from 3 as an example, here we add the 'Use' name argument with the value 'Cluster':<br />
<br />
GFtboxCommand('State','Start','Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5],'Use','Cluster');<br />
<br />
The 'Cluster' option for the 'Use' name argument instructs GFtboxCommand to upload the required project directory to a remote Linux server. Instead of running the simulation on your own computer, GFtboxCommand then works out how many individual simulations are specified by the command that invoked it. In this example, the only argument which will generate multiple simulations is ''dt'', of which there will be 4. Separate commands for each of these jobs are then automatically generated, each with an accompanying unique ID (The value of the ''ExpID'' name argument), and '''these are submitted as individual jobs to the cluster'''.<br />
<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.1,'ExpID','GPT_CASE_RST_1');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.5,'ExpID','GPT_CASE_RST_2');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',1,'ExpID','GPT_CASE_RST_3');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',5,'ExpID','GPT_CASE_RST_4');<br />
<br />
Once these jobs have been submitted, the ''ClusterMonitor'' tool opens and the new job batch ID is added to the list of jobs. Here, we will concentrate on three of the functions of ''ClusterMonitor'': ''Queue?'', ''Get project results'' and ''Make project pngs''. As in part 3 of this page, these functions will enable us to visualise the growth simulation results.<br />
<br />
Whilst ''ClusterMonitor'' is a useful tool for managing your cluster jobs, it is advisable to '''have at least a rudimentary understanding of and ability to use a Unix-based computer system'''. In particular, the abilities to list the contents of a folder, view the contents of a file, change the current working folder, delete, and to copy or move files or folders, are essential Unix skills for making sure everything is working as intended. It is beyond the scope of these tutorials to provide an in depth Unix tutorial (many good and simple tutorials exist on the web), but here is a short description of the Unix commands (which may be specific to our Unix-based system) that we believe to be important.</div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Running_example_models_and_using_a_cluster&diff=5371Running example models and using a cluster2011-06-23T13:22:36Z<p>JacobNewman: </p>
<hr />
<div>[[GFtbox Tutorial pages | Back to GFtbox Tutorial pages]]<br />
<br />
The purpose of these tutorials is to learn how to run the example growth simulations included in GFtbox. We will describe methods for running simulations locally (on your own computer) and remotely (on a computing cluster). It is assumed that you have already downloaded the GFtbox software and have Matlab installed.<br />
<br />
=Getting Started=<br />
<br />
The remainder of this page is split into five sub-tutorials, each building on the preceding parts.<br />
<br />
1) '''Explaining the tools'''. In these tutorials, we will be using ''GFtboxCommand'' and ''ClusterMonitor''. This section explains the purpose of these tools. <br />
<br />
2) '''Computer or cluster?''' Here, we illustrate when and why you should use a computing cluster for your growth simulations, and conversely, when you should use your desktop computer.<br />
<br />
3) '''Running a growth simulation for an example model'''. This section demonstrates how to use ''GFtboxCommand'' to run a growth simulation on your computer. The model used for the simulation is an example model included with the GFtbox.<br />
<br />
4) '''Altering the simulation parameters'''. Following on from part 3, here we show how to adjust a simulation parameter within GFtboxCommand. Specifically, we alter the value of ''dt'', the temporal resolution of the simulation, and show how it can be used to verify that the value specified in the published literature is reasonable.<br />
<br />
5) '''Altering the model parameters'''. Finally, we demonstrate how a number of model parameters can be varied by specifying a range of values for each model argument. We show how the computationally expensive task of simulating all combinations of specified ranges can be processed more efficiently if a computer cluster is used via the ''ClusterMonitor'' tool.<br />
<br />
='''1''' Explaining the tools=<br />
==GFtboxCommand==<br />
This is a command line version of the GFtbox. By command line, we mean that all program '''functions are operated via typed commands, without the GUI'''. Like GFtbox, GFtboxCommand is capable of running growth simulations of an interaction function, and allows the user to specify model and simulation parameters. Unlike GFtbox, this also allows the user to select '''ranges of values for a number of input parameters''', and will automatically spawn multiple simulations which explore the various combinations of those parameters. This can be used to evaluate the effect of various parameters on the growth of a given model.<br />
<br />
==ClusterMonitor==<br />
Provides a graphical user interface for managing simulations '''running remotely on a computer cluster'''. Specifically, it allows you to see which projects are present and running on the cluster, to retrieve the completed projects, to generate images of the simulations at specified stages of growth, and to remove projects from the cluster. If you do not intend to use a computer cluster, then you will not need to use ClusterMonitor.<br />
<br />
='''2''' Computer or cluster?=<br />
<br />
In basic terms, '''a computing cluster is effectively a network of many computer processors''' (often hundreds), centrally managed by a queuing system. When a job is submitted to a cluster, the job is sent to a processor that is not being used, or queued until one becomes available. In contrast, a typical desktop computer will contain one processor, limiting the number of tasks that can be performed at any one time. Jobs which must be run independently and sequentially on a desktop computer can be executed in parallel on a cluster, '''greatly reducing the total time required to complete all of the jobs'''. Although the exact details of your cluster might vary from those described in the rest of these tutorials, we aim to illustrate the generic processes involved in using GFtbox on a cluster. We are happy to offer assistance, where possible, to setup GFtbox for your cluster, so please contact us if you have any queries.<br />
<br />
Whilst the time savings offered by using a cluster can be significant, there is an overhead associated with returning the results to your personal computer. The total time to run 2 or more simulations on a cluster and to return the results will be less than running those simulations sequentially on one computer. Therefore, using a '''cluster is ideal for situations where you would like to run several simulations''', such as to evaluate the effect of a range of parameters on a growth model. You are not advised to use a cluster for running single simulations, or where you would like to step through a simulation and change parameters in a more interactive fashion. In such circumstances, you are advised to run GFtbox or GFtboxCommand on your own computer.<br />
<br />
The '''GFtbox GUI provides quick feedback''' about how the changes you have made to an interaction function have affected the growth simulation, as '''you can see the result of each simulation iteration as it completes'''. It is quick and easy therefore to see if you have dramatically changed the course of the growth simulation, and then to adjust the parameters according to your observations. This approach is well suited to the early stages of design, where you might wish to '''tweak some parameters to gauge whether or not they have had the desired effect'''. Or in the final stages, where you think your model is almost finalised, but where small adjustments are required. '''An alternative approach is to start many simulations''', based upon your initial model and by making intelligent choices about which parameters to explore. This allows you to '''harvest many results, to quickly and easily overview them''' in their finalised form, or to select interesting-looking simulations and examine them more closely on your desktop computer, using GFtbox.<br />
<br />
='''3''' Running a growth simulation for an example model=<br />
This tutorial is aimed at running a '''growth simulation for one of the example interaction functions''' included with the GFtbox. The purpose of this exercise is to firstly demonstrate how simulations can be invoked using GFtboxCommand, and secondly to show how to reproduce experimental results (specifically, those published in Kennaway et al (2011)) given an interaction function.<br />
<br />
Assuming Matlab is installed on your computer, and the latest GFtbox has been downloaded, you can add the GFtbox directory to Matlab's search path, which makes the toolbox accessible from any other path that you choose to work from. For a short tutorial on how to do this, please click [[Adding GFtbox to Matlab's search path | here]].<br />
<br />
Once the GFtbox is added to Matlab, you are ready to run a growth simulation using GFtboxCommand. In this example, '''the model that we will simulate is called GPT_CASE_RST'''. Results generated using this model are published in Kennaway et al (2011). By running this simulation, we can confirm the results in the published literature and investigate the suitability of the various parameters.<br />
<br />
===3A - Running a simulation on your computer===<br />
<br />
The following command can be typed into Matlab to run a simulation of the GPT_CASE_RST interaction function, which contains three growth models: R, S and T. '''Three separate simulations are run sequentially on your computer''', one for each model, each producing results corresponding to five intervals in the growth simulation.<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',[1:3]);<br />
<br />
GFtboxCommand accepts input arguments as name and value pairs, e.g. 'modelname', [1:3] or 'Use','Cluster'. The argument names entered in the example above are: ''Path'', ''Name'', ''Stages'' and ''modelname''.<br />
<br />
The optional '''''Path'' argument name refers to the location of the folder''' (or directory) on your computer where the model interaction function you wish to simulate is stored. '''''Name'', is the name of the folder itself'''. In this case, we are using the GPT_CASE_RST folder which is included in the GrowthToolbox. You may wish to copy this folder elsewhere, if you intend to make changes to the interaction function.<br />
<br />
During a growth simulation, a mesh can be generated at each time step of the simulation, which provides a visual representation of the growth of the biological tissue, given the various parameters of growth specified by the interaction function and how they have changed over time. Put another way, the mesh shows exactly what the growing tissue actually looks like. ''Stages'' refers to the points in the simulation (measured in hours) at which meshes should be generated and saved. In this example, '''five stages of growth will be written to disk'''. These values are chosen to best capture the appearance of the tissue at important stages of the tissue growth.<br />
<br />
The final argument name listed here is ''modelname''. This is a model-specific argument, and in this case the GPT_CASE_RST interaction function contains '''three separate models for plant growth''', allowing the desired model to be a selected. Here, the value [1:3] is specified, which is evaluated in the same way as entering [1 2 3]. This instructs GFtboxCommand to run three separate simulations, one for each of the growth models contained in the interaction function.<br />
<br />
The function of every permissible argument is given by keying the following command into Matlab:<br />
<br />
help GFtboxCommand<br />
<br />
===3B - Getting results from a completed simulation===<br />
<br />
'''1) - Generating images from a simulation you ran on your computer'''<br />
<br />
Once a growth simulation is completed, the project folder (GPT_CASE_RST, in our example) will contain another folder named "movies". Within movies, there are folders which contain results for the executed simulations. As was instructed in 3A, here we can see three folders corresponding to simulations for the three separate models. Within the first folder there are three items:<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| ''CommandLine.txt'' - This file contains the Matlab command which was used to generate the results that this sub-directory contains.<br />
<br />
''gpt_case_rst.txt'' - This file, named according to the project name, contains a copy of the interaction function which was used to produce these results.<br />
<br />
''meshes'' - This folder contains the mesh files corresponding to the stages of growth specified by the value of the Stages argument, described in 3A.<br />
|width="500px" align="center"|[[File:resultsdir.png| Directory structure of a project containing results]]<br />
|}<br />
<br />
<br />
The mesh files contain vertices information regarding the shape of the growth model at a particular stage of growth, but are not visualisable in that form. In order to '''convert these mesh files into viewable images''', we can execute the following command in Matlab:<br />
<br />
VMSreport('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Project','GPT_CASE_RST',...<br />
'Experiment','All','flattentime',572.5,'morphogen','KPAR','SNAPFIG',true);<br />
<br />
Where the ''Path'' and ''Project'' arguments have the same function as ''Path'' and ''Name'' in 3A, i.e. ''Path'' refers to the location of the project folder and ''Project'' is the name of the project folder itself.<br />
<br />
Here we can see the generated images, including the command line arguments, for one of the simulations performed in 3A. Click on a thumbnail to view a larger image.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="middle"<br />
|width="125px"| <small>001commandline.png<br> [[File:commandlineexample.png | 125px]]<br />
|width="125px"| 001GPT_CASE_RST_2 wild_.png<br>[[File:GPT_CASE_RST_example1.png | 125px]]<br />
|width="125px"| 001GPT_CASE_RST_3 wild_.png<br>[[File:GPT_CASE_RST_example2.png | 125px]]<br />
|width="125px"| 001GPT_CASE_RST_4 wild_.png<br>[[File:GPT_CASE_RST_example3.png | 125px]]<br />
|width="125px"| 001GPT_CASE_RST_5 wild_.png<br>[[File:GPT_CASE_RST_example4.png | 125px]]<br />
|width="125px"| 001GPT_CASE_RST_6 wild_.png</small><br>[[File:GPT_CASE_RST_example5.png | 125px]]<br />
|}<br />
<br />
<br />
'''2) - Generating images from a simulation you ran on the cluster'''<br />
<br />
NB. The results produced may not be visibly identical to those in the published literature. This is because of small, random perturbations which are applied to the initial model meshes to stop them from containing surfaces which are perfectly flat, and therefore biologically unrealistic. The results produced should be '''qualitatively but not quantitatively the same'''.<br />
<br />
'''3) - Interacting with your results using the GFtbox GUI'''<br />
<br />
='''4''' Altering the simulation parameters=<br />
<br />
One such simulation parameter is ''dt'', which is the time in seconds between iterations in the growth simulation. Large values of ''dt'' mean that fewer steps and therefore fewer calculations are required to complete a simulation. Whereas smaller values of ''dt'' mean that more steps are required, and therefore more processing time too. Therefore, a value of ''dt'' must be selected which is not so small that it is computationally unmanageable, but not so large that the observed growth is an artifact of the value of ''dt'', rather than the underlying growth model. It is necessary therefore to test a range of values for ''dt'' to ensure that the patterns of growth observed in a simulation are consistent across the range, and to find a value which is sufficient to demonstrate the model of growth and computationally efficient.<br />
<br />
'''Testing a range of ''dt''s can be achieved in several ways.''' One way is to use the ''dt'' argument when calling GFtboxCommand. This allows a single value to be tested. Another way is to make a batch of jobs, each using a different value for ''dt'', by using the 'State' argument, as described here. Lastly, a range of ''dt''s can be specified in GFtboxCommand, by using the dt argument where the values within the square brackets are the ''dt''s to simulate:<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5]);<br />
<br />
Here, we have specified '''four separate values for ''dt''''', and this means that '''four separate simulations will be run''' sequentially on your computer. As in 2B, images can then be generated for the mesh files produced, and compared to ensure consistency across the range of ''dt''s. In this figure, we can see the same model, at the same stage of growth, generated using the four values of ''dt''. Though quantitatively different, they are qualitatively the same, illustrating the suitability of the default ''dt'' value of 5.<br />
<br />
='''5''' Altering the model parameters=<br />
It is easy to see that when even a small number of range variables, mutations or ''dt''s are specified, the '''total number of simulations increases quickly'''. If many ranges are specified, then the amount of processing time becomes unmanageable on a single computer. Via ''GFtboxCommand'' and the ''ClusterMonitor'' tool, we can remotely''' run a number of simulations, in parallel, on a computing cluster'''. It is assumed that the GrowthToolbox and Matlab are installed on your cluster, and PuTTY is required on your local computer for the tools ''pscp'' (for transferring files from your computer to a cluster) and ''plink'' (for remotely executing commands on a cluster). Using the ''dt'' range argument from 3 as an example, here we add the 'Use' name argument with the value 'Cluster':<br />
<br />
GFtboxCommand('State','Start','Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5],'Use','Cluster');<br />
<br />
The 'Cluster' option for the 'Use' name argument instructs GFtboxCommand to upload the required project directory to a remote Linux server. Instead of running the simulation on your own computer, GFtboxCommand then works out how many individual simulations are specified by the command that invoked it. In this example, the only argument which will generate multiple simulations is ''dt'', of which there will be 4. Separate commands for each of these jobs are then automatically generated, each with an accompanying unique ID (The value of the ''ExpID'' name argument), and '''these are submitted as individual jobs to the cluster'''.<br />
<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.1,'ExpID','GPT_CASE_RST_1');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.5,'ExpID','GPT_CASE_RST_2');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',1,'ExpID','GPT_CASE_RST_3');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',5,'ExpID','GPT_CASE_RST_4');<br />
<br />
Once these jobs have been submitted, the ''ClusterMonitor'' tool opens and the new job batch ID is added to the list of jobs. Here, we will concentrate on three of the functions of ''ClusterMonitor'': ''Queue?'', ''Get project results'' and ''Make project pngs''. As in part 3 of this page, these functions will enable us to visualise the growth simulation results.<br />
<br />
Whilst ''ClusterMonitor'' is a useful tool for managing your cluster jobs, it is advisable to '''have at least a rudimentary understanding of and ability to use a Unix-based computer system'''. In particular, the abilities to list the contents of a folder, view the contents of a file, change the current working folder, delete, and to copy or move files or folders, are essential Unix skills for making sure everything is working as intended. It is beyond the scope of these tutorials to provide an in depth Unix tutorial (many good and simple tutorials exist on the web), but here is a short description of the Unix commands (which may be specific to our Unix-based system) that we believe to be important.</div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Running_example_models_and_using_a_cluster&diff=5370Running example models and using a cluster2011-06-23T13:21:59Z<p>JacobNewman: </p>
<hr />
<div>[[GFtbox Tutorial pages | Back to GFtbox Tutorial pages]]<br />
<br />
The purpose of these tutorials is to learn how to run the example growth simulations included in GFtbox. We will describe methods for running simulations locally (on your own computer) and remotely (on a computing cluster). It is assumed that you have already downloaded the GFtbox software and have Matlab installed.<br />
<br />
=Getting Started=<br />
<br />
The remainder of this page is split into five sub-tutorials, each building on the preceding parts.<br />
<br />
1) '''Explaining the tools'''. In these tutorials, we will be using ''GFtboxCommand'' and ''ClusterMonitor''. This section explains the purpose of these tools. <br />
<br />
2) '''Computer or cluster?''' Here, we illustrate when and why you should use a computing cluster for your growth simulations, and conversely, when you should use your desktop computer.<br />
<br />
3) '''Running a growth simulation for an example model'''. This section demonstrates how to use ''GFtboxCommand'' to run a growth simulation on your computer. The model used for the simulation is an example model included with the GFtbox.<br />
<br />
4) '''Altering the simulation parameters'''. Following on from part 3, here we show how to adjust a simulation parameter within GFtboxCommand. Specifically, we alter the value of ''dt'', the temporal resolution of the simulation, and show how it can be used to verify that the value specified in the published literature is reasonable.<br />
<br />
5) '''Altering the model parameters'''. Finally, we demonstrate how a number of model parameters can be varied by specifying a range of values for each model argument. We show how the computationally expensive task of simulating all combinations of specified ranges can be processed more efficiently if a computer cluster is used via the ''ClusterMonitor'' tool.<br />
<br />
='''1''' Explaining the tools=<br />
==GFtboxCommand==<br />
This is a command line version of the GFtbox. By command line, we mean that all program '''functions are operated via typed commands, without the GUI'''. Like GFtbox, GFtboxCommand is capable of running growth simulations of an interaction function, and allows the user to specify model and simulation parameters. Unlike GFtbox, this also allows the user to select '''ranges of values for a number of input parameters''', and will automatically spawn multiple simulations which explore the various combinations of those parameters. This can be used to evaluate the effect of various parameters on the growth of a given model.<br />
<br />
==ClusterMonitor==<br />
Provides a graphical user interface for managing simulations '''running remotely on a computer cluster'''. Specifically, it allows you to see which projects are present and running on the cluster, to retrieve the completed projects, to generate images of the simulations at specified stages of growth, and to remove projects from the cluster. If you do not intend to use a computer cluster, then you will not need to use ClusterMonitor.<br />
<br />
='''2''' Computer or cluster?=<br />
<br />
In basic terms, '''a computing cluster is effectively a network of many computer processors''' (often hundreds), centrally managed by a queuing system. When a job is submitted to a cluster, the job is sent to a processor that is not being used, or queued until one becomes available. In contrast, a typical desktop computer will contain one processor, limiting the number of tasks that can be performed at any one time. Jobs which must be run independently and sequentially on a desktop computer can be executed in parallel on a cluster, '''greatly reducing the total time required to complete all of the jobs'''. Although the exact details of your cluster might vary from those described in the rest of these tutorials, we aim to illustrate the generic processes involved in using GFtbox on a cluster. We are happy to offer assistance, where possible, to setup GFtbox for your cluster, so please contact us if you have any queries.<br />
<br />
Whilst the time savings offered by using a cluster can be significant, there is an overhead associated with returning the results to your personal computer. The total time to run 2 or more simulations on a cluster and to return the results will be less than running those simulations sequentially on one computer. Therefore, using a '''cluster is ideal for situations where you would like to run several simulations''', such as to evaluate the effect of a range of parameters on a growth model. You are not advised to use a cluster for running single simulations, or where you would like to step through a simulation and change parameters in a more interactive fashion. In such circumstances, you are advised to run GFtbox or GFtboxCommand on your own computer.<br />
<br />
The '''GFtbox GUI provides quick feedback''' about how the changes you have made to an interaction function have affected the growth simulation, as '''you can see the result of each simulation iteration as it completes'''. It is quick and easy therefore to see if you have dramatically changed the course of the growth simulation, and then to adjust the parameters according to your observations. This approach is well suited to the early stages of design, where you might wish to '''tweak some parameters to gauge whether or not they have had the desired effect'''. Or in the final stages, where you think your model is almost finalised, but where small adjustments are required. '''An alternative approach is to start many simulations''', based upon your initial model and by making intelligent choices about which parameters to explore. This allows you to '''harvest many results, to quickly and easily overview them''' in their finalised form, or to select interesting-looking simulations and examine them more closely on your desktop computer, using GFtbox.<br />
<br />
='''3''' Running a growth simulation for an example model=<br />
This tutorial is aimed at running a '''growth simulation for one of the example interaction functions''' included with the GFtbox. The purpose of this exercise is to firstly demonstrate how simulations can be invoked using GFtboxCommand, and secondly to show how to reproduce experimental results (specifically, those published in Kennaway et al (2011)) given an interaction function.<br />
<br />
Assuming Matlab is installed on your computer, and the latest GFtbox has been downloaded, you can add the GFtbox directory to Matlab's search path, which makes the toolbox accessible from any other path that you choose to work from. For a short tutorial on how to do this, please click [[Adding GFtbox to Matlab's search path | here]].<br />
<br />
Once the GFtbox is added to Matlab, you are ready to run a growth simulation using GFtboxCommand. In this example, '''the model that we will simulate is called GPT_CASE_RST'''. Results generated using this model are published in Kennaway et al (2011). By running this simulation, we can confirm the results in the published literature and investigate the suitability of the various parameters.<br />
<br />
===3A - Running a simulation on your computer===<br />
<br />
The following command can be typed into Matlab to run a simulation of the GPT_CASE_RST interaction function, which contains three growth models: R, S and T. '''Three separate simulations are run sequentially on your computer''', one for each model, each producing results corresponding to five intervals in the growth simulation.<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',[1:3]);<br />
<br />
GFtboxCommand accepts input arguments as name and value pairs, e.g. 'modelname', [1:3] or 'Use','Cluster'. The argument names entered in the example above are: ''Path'', ''Name'', ''Stages'' and ''modelname''.<br />
<br />
The optional '''''Path'' argument name refers to the location of the folder''' (or directory) on your computer where the model interaction function you wish to simulate is stored. '''''Name'', is the name of the folder itself'''. In this case, we are using the GPT_CASE_RST folder which is included in the GrowthToolbox. You may wish to copy this folder elsewhere, if you intend to make changes to the interaction function.<br />
<br />
During a growth simulation, a mesh can be generated at each time step of the simulation, which provides a visual representation of the growth of the biological tissue, given the various parameters of growth specified by the interaction function and how they have changed over time. Put another way, the mesh shows exactly what the growing tissue actually looks like. ''Stages'' refers to the points in the simulation (measured in hours) at which meshes should be generated and saved. In this example, '''five stages of growth will be written to disk'''. These values are chosen to best capture the appearance of the tissue at important stages of the tissue growth.<br />
<br />
The final argument name listed here is ''modelname''. This is a model-specific argument, and in this case the GPT_CASE_RST interaction function contains '''three separate models for plant growth''', allowing the desired model to be a selected. Here, the value [1:3] is specified, which is evaluated in the same way as entering [1 2 3]. This instructs GFtboxCommand to run three separate simulations, one for each of the growth models contained in the interaction function.<br />
<br />
The function of every permissible argument is given by keying the following command into Matlab:<br />
<br />
help GFtboxCommand<br />
<br />
===3B - Getting results from a completed simulation===<br />
<br />
'''1) - Generating images from a simulation you ran on your computer'''<br />
<br />
Once a growth simulation is completed, the project folder (GPT_CASE_RST, in our example) will contain another folder named "movies". Within movies, there are folders which contain results for the executed simulations. As was instructed in 3A, here we can see three folders corresponding to simulations for the three separate models. Within the first folder there are three items:<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| ''CommandLine.txt'' - This file contains the Matlab command which was used to generate the results that this sub-directory contains.<br />
<br />
''gpt_case_rst.txt'' - This file, named according to the project name, contains a copy of the interaction function which was used to produce these results.<br />
<br />
''meshes'' - This folder contains the mesh files corresponding to the stages of growth specified by the value of the Stages argument, described in 3A.<br />
|width="500px" align="center"|[[File:resultsdir.png| Directory structure of a project containing results]]<br />
|}<br />
<br />
<br />
The mesh files contain vertices information regarding the shape of the growth model at a particular stage of growth, but are not visualisable in that form. In order to '''convert these mesh files into viewable images''', we can execute the following command in Matlab:<br />
<br />
VMSreport('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Project','GPT_CASE_RST',...<br />
'Experiment','All','flattentime',572.5,'morphogen','KPAR','SNAPFIG',true);<br />
<br />
Where the ''Path'' and ''Project'' arguments have the same function as ''Path'' and ''Name'' in 3A, i.e. ''Path'' refers to the location of the project folder and ''Project'' is the name of the project folder itself.<br />
<br />
Here we can see the generated images, including the command line arguments, for one of the simulations performed in 3A. Click on a thumbnail to view a larger image.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="middle"<br />
|width="125px"| <small>001commandline.png</small><br> [[File:commandlineexample.png | 125px]]<br />
|width="125px"| 001GPT_CASE_RST_2 wild_.png<br>[[File:GPT_CASE_RST_example1.png | 125px]]<br />
|width="125px"| 001GPT_CASE_RST_3 wild_.png<br>[[File:GPT_CASE_RST_example2.png | 125px]]<br />
|width="125px"| 001GPT_CASE_RST_4 wild_.png<br>[[File:GPT_CASE_RST_example3.png | 125px]]<br />
|width="125px"| 001GPT_CASE_RST_5 wild_.png<br>[[File:GPT_CASE_RST_example4.png | 125px]]<br />
|width="125px"| 001GPT_CASE_RST_6 wild_.png<br>[[File:GPT_CASE_RST_example5.png | 125px]]<br />
|}<br />
<br />
<br />
'''2) - Generating images from a simulation you ran on the cluster'''<br />
<br />
NB. The results produced may not be visibly identical to those in the published literature. This is because of small, random perturbations which are applied to the initial model meshes to stop them from containing surfaces which are perfectly flat, and therefore biologically unrealistic. The results produced should be '''qualitatively but not quantitatively the same'''.<br />
<br />
'''3) - Interacting with your results using the GFtbox GUI'''<br />
<br />
='''4''' Altering the simulation parameters=<br />
<br />
One such simulation parameter is ''dt'', which is the time in seconds between iterations in the growth simulation. Large values of ''dt'' mean that fewer steps and therefore fewer calculations are required to complete a simulation. Whereas smaller values of ''dt'' mean that more steps are required, and therefore more processing time too. Therefore, a value of ''dt'' must be selected which is not so small that it is computationally unmanageable, but not so large that the observed growth is an artifact of the value of ''dt'', rather than the underlying growth model. It is necessary therefore to test a range of values for ''dt'' to ensure that the patterns of growth observed in a simulation are consistent across the range, and to find a value which is sufficient to demonstrate the model of growth and computationally efficient.<br />
<br />
'''Testing a range of ''dt''s can be achieved in several ways.''' One way is to use the ''dt'' argument when calling GFtboxCommand. This allows a single value to be tested. Another way is to make a batch of jobs, each using a different value for ''dt'', by using the 'State' argument, as described here. Lastly, a range of ''dt''s can be specified in GFtboxCommand, by using the dt argument where the values within the square brackets are the ''dt''s to simulate:<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5]);<br />
<br />
Here, we have specified '''four separate values for ''dt''''', and this means that '''four separate simulations will be run''' sequentially on your computer. As in 2B, images can then be generated for the mesh files produced, and compared to ensure consistency across the range of ''dt''s. In this figure, we can see the same model, at the same stage of growth, generated using the four values of ''dt''. Though quantitatively different, they are qualitatively the same, illustrating the suitability of the default ''dt'' value of 5.<br />
<br />
='''5''' Altering the model parameters=<br />
It is easy to see that when even a small number of range variables, mutations or ''dt''s are specified, the '''total number of simulations increases quickly'''. If many ranges are specified, then the amount of processing time becomes unmanageable on a single computer. Via ''GFtboxCommand'' and the ''ClusterMonitor'' tool, we can remotely''' run a number of simulations, in parallel, on a computing cluster'''. It is assumed that the GrowthToolbox and Matlab are installed on your cluster, and PuTTY is required on your local computer for the tools ''pscp'' (for transferring files from your computer to a cluster) and ''plink'' (for remotely executing commands on a cluster). Using the ''dt'' range argument from 3 as an example, here we add the 'Use' name argument with the value 'Cluster':<br />
<br />
GFtboxCommand('State','Start','Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5],'Use','Cluster');<br />
<br />
The 'Cluster' option for the 'Use' name argument instructs GFtboxCommand to upload the required project directory to a remote Linux server. Instead of running the simulation on your own computer, GFtboxCommand then works out how many individual simulations are specified by the command that invoked it. In this example, the only argument which will generate multiple simulations is ''dt'', of which there will be 4. Separate commands for each of these jobs are then automatically generated, each with an accompanying unique ID (The value of the ''ExpID'' name argument), and '''these are submitted as individual jobs to the cluster'''.<br />
<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.1,'ExpID','GPT_CASE_RST_1');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.5,'ExpID','GPT_CASE_RST_2');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',1,'ExpID','GPT_CASE_RST_3');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',5,'ExpID','GPT_CASE_RST_4');<br />
<br />
Once these jobs have been submitted, the ''ClusterMonitor'' tool opens and the new job batch ID is added to the list of jobs. Here, we will concentrate on three of the functions of ''ClusterMonitor'': ''Queue?'', ''Get project results'' and ''Make project pngs''. As in part 3 of this page, these functions will enable us to visualise the growth simulation results.<br />
<br />
Whilst ''ClusterMonitor'' is a useful tool for managing your cluster jobs, it is advisable to '''have at least a rudimentary understanding of and ability to use a Unix-based computer system'''. In particular, the abilities to list the contents of a folder, view the contents of a file, change the current working folder, delete, and to copy or move files or folders, are essential Unix skills for making sure everything is working as intended. It is beyond the scope of these tutorials to provide an in depth Unix tutorial (many good and simple tutorials exist on the web), but here is a short description of the Unix commands (which may be specific to our Unix-based system) that we believe to be important.</div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Running_example_models_and_using_a_cluster&diff=5369Running example models and using a cluster2011-06-23T13:21:20Z<p>JacobNewman: </p>
<hr />
<div>[[GFtbox Tutorial pages | Back to GFtbox Tutorial pages]]<br />
<br />
The purpose of these tutorials is to learn how to run the example growth simulations included in GFtbox. We will describe methods for running simulations locally (on your own computer) and remotely (on a computing cluster). It is assumed that you have already downloaded the GFtbox software and have Matlab installed.<br />
<br />
=Getting Started=<br />
<br />
The remainder of this page is split into five sub-tutorials, each building on the preceding parts.<br />
<br />
1) '''Explaining the tools'''. In these tutorials, we will be using ''GFtboxCommand'' and ''ClusterMonitor''. This section explains the purpose of these tools. <br />
<br />
2) '''Computer or cluster?''' Here, we illustrate when and why you should use a computing cluster for your growth simulations, and conversely, when you should use your desktop computer.<br />
<br />
3) '''Running a growth simulation for an example model'''. This section demonstrates how to use ''GFtboxCommand'' to run a growth simulation on your computer. The model used for the simulation is an example model included with the GFtbox.<br />
<br />
4) '''Altering the simulation parameters'''. Following on from part 3, here we show how to adjust a simulation parameter within GFtboxCommand. Specifically, we alter the value of ''dt'', the temporal resolution of the simulation, and show how it can be used to verify that the value specified in the published literature is reasonable.<br />
<br />
5) '''Altering the model parameters'''. Finally, we demonstrate how a number of model parameters can be varied by specifying a range of values for each model argument. We show how the computationally expensive task of simulating all combinations of specified ranges can be processed more efficiently if a computer cluster is used via the ''ClusterMonitor'' tool.<br />
<br />
='''1''' Explaining the tools=<br />
==GFtboxCommand==<br />
This is a command line version of the GFtbox. By command line, we mean that all program '''functions are operated via typed commands, without the GUI'''. Like GFtbox, GFtboxCommand is capable of running growth simulations of an interaction function, and allows the user to specify model and simulation parameters. Unlike GFtbox, this also allows the user to select '''ranges of values for a number of input parameters''', and will automatically spawn multiple simulations which explore the various combinations of those parameters. This can be used to evaluate the effect of various parameters on the growth of a given model.<br />
<br />
==ClusterMonitor==<br />
Provides a graphical user interface for managing simulations '''running remotely on a computer cluster'''. Specifically, it allows you to see which projects are present and running on the cluster, to retrieve the completed projects, to generate images of the simulations at specified stages of growth, and to remove projects from the cluster. If you do not intend to use a computer cluster, then you will not need to use ClusterMonitor.<br />
<br />
='''2''' Computer or cluster?=<br />
<br />
In basic terms, '''a computing cluster is effectively a network of many computer processors''' (often hundreds), centrally managed by a queuing system. When a job is submitted to a cluster, the job is sent to a processor that is not being used, or queued until one becomes available. In contrast, a typical desktop computer will contain one processor, limiting the number of tasks that can be performed at any one time. Jobs which must be run independently and sequentially on a desktop computer can be executed in parallel on a cluster, '''greatly reducing the total time required to complete all of the jobs'''. Although the exact details of your cluster might vary from those described in the rest of these tutorials, we aim to illustrate the generic processes involved in using GFtbox on a cluster. We are happy to offer assistance, where possible, to setup GFtbox for your cluster, so please contact us if you have any queries.<br />
<br />
Whilst the time savings offered by using a cluster can be significant, there is an overhead associated with returning the results to your personal computer. The total time to run 2 or more simulations on a cluster and to return the results will be less than running those simulations sequentially on one computer. Therefore, using a '''cluster is ideal for situations where you would like to run several simulations''', such as to evaluate the effect of a range of parameters on a growth model. You are not advised to use a cluster for running single simulations, or where you would like to step through a simulation and change parameters in a more interactive fashion. In such circumstances, you are advised to run GFtbox or GFtboxCommand on your own computer.<br />
<br />
The '''GFtbox GUI provides quick feedback''' about how the changes you have made to an interaction function have affected the growth simulation, as '''you can see the result of each simulation iteration as it completes'''. It is quick and easy therefore to see if you have dramatically changed the course of the growth simulation, and then to adjust the parameters according to your observations. This approach is well suited to the early stages of design, where you might wish to '''tweak some parameters to gauge whether or not they have had the desired effect'''. Or in the final stages, where you think your model is almost finalised, but where small adjustments are required. '''An alternative approach is to start many simulations''', based upon your initial model and by making intelligent choices about which parameters to explore. This allows you to '''harvest many results, to quickly and easily overview them''' in their finalised form, or to select interesting-looking simulations and examine them more closely on your desktop computer, using GFtbox.<br />
<br />
='''3''' Running a growth simulation for an example model=<br />
This tutorial is aimed at running a '''growth simulation for one of the example interaction functions''' included with the GFtbox. The purpose of this exercise is to firstly demonstrate how simulations can be invoked using GFtboxCommand, and secondly to show how to reproduce experimental results (specifically, those published in Kennaway et al (2011)) given an interaction function.<br />
<br />
Assuming Matlab is installed on your computer, and the latest GFtbox has been downloaded, you can add the GFtbox directory to Matlab's search path, which makes the toolbox accessible from any other path that you choose to work from. For a short tutorial on how to do this, please click [[Adding GFtbox to Matlab's search path | here]].<br />
<br />
Once the GFtbox is added to Matlab, you are ready to run a growth simulation using GFtboxCommand. In this example, '''the model that we will simulate is called GPT_CASE_RST'''. Results generated using this model are published in Kennaway et al (2011). By running this simulation, we can confirm the results in the published literature and investigate the suitability of the various parameters.<br />
<br />
===3A - Running a simulation on your computer===<br />
<br />
The following command can be typed into Matlab to run a simulation of the GPT_CASE_RST interaction function, which contains three growth models: R, S and T. '''Three separate simulations are run sequentially on your computer''', one for each model, each producing results corresponding to five intervals in the growth simulation.<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',[1:3]);<br />
<br />
GFtboxCommand accepts input arguments as name and value pairs, e.g. 'modelname', [1:3] or 'Use','Cluster'. The argument names entered in the example above are: ''Path'', ''Name'', ''Stages'' and ''modelname''.<br />
<br />
The optional '''''Path'' argument name refers to the location of the folder''' (or directory) on your computer where the model interaction function you wish to simulate is stored. '''''Name'', is the name of the folder itself'''. In this case, we are using the GPT_CASE_RST folder which is included in the GrowthToolbox. You may wish to copy this folder elsewhere, if you intend to make changes to the interaction function.<br />
<br />
During a growth simulation, a mesh can be generated at each time step of the simulation, which provides a visual representation of the growth of the biological tissue, given the various parameters of growth specified by the interaction function and how they have changed over time. Put another way, the mesh shows exactly what the growing tissue actually looks like. ''Stages'' refers to the points in the simulation (measured in hours) at which meshes should be generated and saved. In this example, '''five stages of growth will be written to disk'''. These values are chosen to best capture the appearance of the tissue at important stages of the tissue growth.<br />
<br />
The final argument name listed here is ''modelname''. This is a model-specific argument, and in this case the GPT_CASE_RST interaction function contains '''three separate models for plant growth''', allowing the desired model to be a selected. Here, the value [1:3] is specified, which is evaluated in the same way as entering [1 2 3]. This instructs GFtboxCommand to run three separate simulations, one for each of the growth models contained in the interaction function.<br />
<br />
The function of every permissible argument is given by keying the following command into Matlab:<br />
<br />
help GFtboxCommand<br />
<br />
===3B - Getting results from a completed simulation===<br />
<br />
'''1) - Generating images from a simulation you ran on your computer'''<br />
<br />
Once a growth simulation is completed, the project folder (GPT_CASE_RST, in our example) will contain another folder named "movies". Within movies, there are folders which contain results for the executed simulations. As was instructed in 3A, here we can see three folders corresponding to simulations for the three separate models. Within the first folder there are three items:<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| ''CommandLine.txt'' - This file contains the Matlab command which was used to generate the results that this sub-directory contains.<br />
<br />
''gpt_case_rst.txt'' - This file, named according to the project name, contains a copy of the interaction function which was used to produce these results.<br />
<br />
''meshes'' - This folder contains the mesh files corresponding to the stages of growth specified by the value of the Stages argument, described in 3A.<br />
|width="500px" align="center"|[[File:resultsdir.png| Directory structure of a project containing results]]<br />
|}<br />
<br />
<br />
The mesh files contain vertices information regarding the shape of the growth model at a particular stage of growth, but are not visualisable in that form. In order to '''convert these mesh files into viewable images''', we can execute the following command in Matlab:<br />
<br />
VMSreport('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Project','GPT_CASE_RST',...<br />
'Experiment','All','flattentime',572.5,'morphogen','KPAR','SNAPFIG',true);<br />
<br />
Where the ''Path'' and ''Project'' arguments have the same function as ''Path'' and ''Name'' in 3A, i.e. ''Path'' refers to the location of the project folder and ''Project'' is the name of the project folder itself.<br />
<br />
Here we can see the generated images, including the command line arguments, for one of the simulations performed in 3A. Click on a thumbnail to view a larger image.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="middle"<br />
|width="125px"| 001commandline.png<br> [[File:commandlineexample.png | 125px]]<br />
|width="125px"| 001GPT_CASE_RST_2 wild_.png<br>[[File:GPT_CASE_RST_example1.png | 125px]]<br />
|width="125px"| 001GPT_CASE_RST_3 wild_.png<br>[[File:GPT_CASE_RST_example2.png | 125px]]<br />
|width="125px"| 001GPT_CASE_RST_4 wild_.png<br>[[File:GPT_CASE_RST_example3.png | 125px]]<br />
|width="125px"| 001GPT_CASE_RST_5 wild_.png<br>[[File:GPT_CASE_RST_example4.png | 125px]]<br />
|width="125px"| 001GPT_CASE_RST_6 wild_.png<br>[[File:GPT_CASE_RST_example5.png | 125px]]<br />
|}<br />
<br />
<br />
'''2) - Generating images from a simulation you ran on the cluster'''<br />
<br />
NB. The results produced may not be visibly identical to those in the published literature. This is because of small, random perturbations which are applied to the initial model meshes to stop them from containing surfaces which are perfectly flat, and therefore biologically unrealistic. The results produced should be '''qualitatively but not quantitatively the same'''.<br />
<br />
'''3) - Interacting with your results using the GFtbox GUI'''<br />
<br />
='''4''' Altering the simulation parameters=<br />
<br />
One such simulation parameter is ''dt'', which is the time in seconds between iterations in the growth simulation. Large values of ''dt'' mean that fewer steps and therefore fewer calculations are required to complete a simulation. Whereas smaller values of ''dt'' mean that more steps are required, and therefore more processing time too. Therefore, a value of ''dt'' must be selected which is not so small that it is computationally unmanageable, but not so large that the observed growth is an artifact of the value of ''dt'', rather than the underlying growth model. It is necessary therefore to test a range of values for ''dt'' to ensure that the patterns of growth observed in a simulation are consistent across the range, and to find a value which is sufficient to demonstrate the model of growth and computationally efficient.<br />
<br />
'''Testing a range of ''dt''s can be achieved in several ways.''' One way is to use the ''dt'' argument when calling GFtboxCommand. This allows a single value to be tested. Another way is to make a batch of jobs, each using a different value for ''dt'', by using the 'State' argument, as described here. Lastly, a range of ''dt''s can be specified in GFtboxCommand, by using the dt argument where the values within the square brackets are the ''dt''s to simulate:<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5]);<br />
<br />
Here, we have specified '''four separate values for ''dt''''', and this means that '''four separate simulations will be run''' sequentially on your computer. As in 2B, images can then be generated for the mesh files produced, and compared to ensure consistency across the range of ''dt''s. In this figure, we can see the same model, at the same stage of growth, generated using the four values of ''dt''. Though quantitatively different, they are qualitatively the same, illustrating the suitability of the default ''dt'' value of 5.<br />
<br />
='''5''' Altering the model parameters=<br />
It is easy to see that when even a small number of range variables, mutations or ''dt''s are specified, the '''total number of simulations increases quickly'''. If many ranges are specified, then the amount of processing time becomes unmanageable on a single computer. Via ''GFtboxCommand'' and the ''ClusterMonitor'' tool, we can remotely''' run a number of simulations, in parallel, on a computing cluster'''. It is assumed that the GrowthToolbox and Matlab are installed on your cluster, and PuTTY is required on your local computer for the tools ''pscp'' (for transferring files from your computer to a cluster) and ''plink'' (for remotely executing commands on a cluster). Using the ''dt'' range argument from 3 as an example, here we add the 'Use' name argument with the value 'Cluster':<br />
<br />
GFtboxCommand('State','Start','Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5],'Use','Cluster');<br />
<br />
The 'Cluster' option for the 'Use' name argument instructs GFtboxCommand to upload the required project directory to a remote Linux server. Instead of running the simulation on your own computer, GFtboxCommand then works out how many individual simulations are specified by the command that invoked it. In this example, the only argument which will generate multiple simulations is ''dt'', of which there will be 4. Separate commands for each of these jobs are then automatically generated, each with an accompanying unique ID (The value of the ''ExpID'' name argument), and '''these are submitted as individual jobs to the cluster'''.<br />
<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.1,'ExpID','GPT_CASE_RST_1');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.5,'ExpID','GPT_CASE_RST_2');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',1,'ExpID','GPT_CASE_RST_3');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',5,'ExpID','GPT_CASE_RST_4');<br />
<br />
Once these jobs have been submitted, the ''ClusterMonitor'' tool opens and the new job batch ID is added to the list of jobs. Here, we will concentrate on three of the functions of ''ClusterMonitor'': ''Queue?'', ''Get project results'' and ''Make project pngs''. As in part 3 of this page, these functions will enable us to visualise the growth simulation results.<br />
<br />
Whilst ''ClusterMonitor'' is a useful tool for managing your cluster jobs, it is advisable to '''have at least a rudimentary understanding of and ability to use a Unix-based computer system'''. In particular, the abilities to list the contents of a folder, view the contents of a file, change the current working folder, delete, and to copy or move files or folders, are essential Unix skills for making sure everything is working as intended. It is beyond the scope of these tutorials to provide an in depth Unix tutorial (many good and simple tutorials exist on the web), but here is a short description of the Unix commands (which may be specific to our Unix-based system) that we believe to be important.</div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Running_example_models_and_using_a_cluster&diff=5421Running example models and using a cluster2011-06-23T13:19:39Z<p>JacobNewman: </p>
<hr />
<div>[[GFtbox Tutorial pages | Back to GFtbox Tutorial pages]]<br />
<br />
The purpose of these tutorials is to learn how to run the example growth simulations included in GFtbox. We will describe methods for running simulations locally (on your own computer) and remotely (on a computing cluster). It is assumed that you have already downloaded the GFtbox software and have Matlab installed.<br />
<br />
=Getting Started=<br />
<br />
The remainder of this page is split into five sub-tutorials, each building on the preceding parts.<br />
<br />
1) '''Explaining the tools'''. In these tutorials, we will be using ''GFtboxCommand'' and ''ClusterMonitor''. This section explains the purpose of these tools. <br />
<br />
2) '''Computer or cluster?''' Here, we illustrate when and why you should use a computing cluster for your growth simulations, and conversely, when you should use your desktop computer.<br />
<br />
3) '''Running a growth simulation for an example model'''. This section demonstrates how to use ''GFtboxCommand'' to run a growth simulation on your computer. The model used for the simulation is an example model included with the GFtbox.<br />
<br />
4) '''Altering the simulation parameters'''. Following on from part 3, here we show how to adjust a simulation parameter within GFtboxCommand. Specifically, we alter the value of ''dt'', the temporal resolution of the simulation, and show how it can be used to verify that the value specified in the published literature is reasonable.<br />
<br />
5) '''Altering the model parameters'''. Finally, we demonstrate how a number of model parameters can be varied by specifying a range of values for each model argument. We show how the computationally expensive task of simulating all combinations of specified ranges can be processed more efficiently if a computer cluster is used via the ''ClusterMonitor'' tool.<br />
<br />
='''1''' Explaining the tools=<br />
==GFtboxCommand==<br />
This is a command line version of the GFtbox. By command line, we mean that all program '''functions are operated via typed commands, without the GUI'''. Like GFtbox, GFtboxCommand is capable of running growth simulations of an interaction function, and allows the user to specify model and simulation parameters. Unlike GFtbox, this also allows the user to select '''ranges of values for a number of input parameters''', and will automatically spawn multiple simulations which explore the various combinations of those parameters. This can be used to evaluate the effect of various parameters on the growth of a given model.<br />
<br />
==ClusterMonitor==<br />
Provides a graphical user interface for managing simulations '''running remotely on a computer cluster'''. Specifically, it allows you to see which projects are present and running on the cluster, to retrieve the completed projects, to generate images of the simulations at specified stages of growth, and to remove projects from the cluster. If you do not intend to use a computer cluster, then you will not need to use ClusterMonitor.<br />
<br />
='''2''' Computer or cluster?=<br />
<br />
In basic terms, '''a computing cluster is effectively a network of many computer processors''' (often hundreds), centrally managed by a queuing system. When a job is submitted to a cluster, the job is sent to a processor that is not being used, or queued until one becomes available. In contrast, a typical desktop computer will contain one processor, limiting the number of tasks that can be performed at any one time. Jobs which must be run independently and sequentially on a desktop computer can be executed in parallel on a cluster, '''greatly reducing the total time required to complete all of the jobs'''. Although the exact details of your cluster might vary from those described in the rest of these tutorials, we aim to illustrate the generic processes involved in using GFtbox on a cluster. We are happy to offer assistance, where possible, to setup GFtbox for your cluster, so please contact us if you have any queries.<br />
<br />
Whilst the time savings offered by using a cluster can be significant, there is an overhead associated with returning the results to your personal computer. The total time to run 2 or more simulations on a cluster and to return the results will be less than running those simulations sequentially on one computer. Therefore, using a '''cluster is ideal for situations where you would like to run several simulations''', such as to evaluate the effect of a range of parameters on a growth model. You are not advised to use a cluster for running single simulations, or where you would like to step through a simulation and change parameters in a more interactive fashion. In such circumstances, you are advised to run GFtbox or GFtboxCommand on your own computer.<br />
<br />
The '''GFtbox GUI provides quick feedback''' about how the changes you have made to an interaction function have affected the growth simulation, as '''you can see the result of each simulation iteration as it completes'''. It is quick and easy therefore to see if you have dramatically changed the course of the growth simulation, and then to adjust the parameters according to your observations. This approach is well suited to the early stages of design, where you might wish to '''tweak some parameters to gauge whether or not they have had the desired effect'''. Or in the final stages, where you think your model is almost finalised, but where small adjustments are required. '''An alternative approach is to start many simulations''', based upon your initial model and by making intelligent choices about which parameters to explore. This allows you to '''harvest many results, to quickly and easily overview them''' in their finalised form, or to select interesting-looking simulations and examine them more closely on your desktop computer, using GFtbox.<br />
<br />
='''3''' Running a growth simulation for an example model=<br />
This tutorial is aimed at running a '''growth simulation for one of the example interaction functions''' included with the GFtbox. The purpose of this exercise is to firstly demonstrate how simulations can be invoked using GFtboxCommand, and secondly to show how to reproduce experimental results (specifically, those published in Kennaway et al (2011)) given an interaction function.<br />
<br />
Assuming Matlab is installed on your computer, and the latest GFtbox has been downloaded, you can add the GFtbox directory to Matlab's search path, which makes the toolbox accessible from any other path that you choose to work from. For a short tutorial on how to do this, please click [[Adding GFtbox to Matlab's search path | here]].<br />
<br />
Once the GFtbox is added to Matlab, you are ready to run a growth simulation using GFtboxCommand. In this example, '''the model that we will simulate is called GPT_CASE_RST'''. Results generated using this model are published in Kennaway et al (2011). By running this simulation, we can confirm the results in the published literature and investigate the suitability of the various parameters.<br />
<br />
===3A - Running a simulation on your computer===<br />
<br />
The following command can be typed into Matlab to run a simulation of the GPT_CASE_RST interaction function, which contains three growth models: R, S and T. '''Three separate simulations are run sequentially on your computer''', one for each model, each producing results corresponding to five intervals in the growth simulation.<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',[1:3]);<br />
<br />
GFtboxCommand accepts input arguments as name and value pairs, e.g. 'modelname', [1:3] or 'Use','Cluster'. The argument names entered in the example above are: ''Path'', ''Name'', ''Stages'' and ''modelname''.<br />
<br />
The optional '''''Path'' argument name refers to the location of the folder''' (or directory) on your computer where the model interaction function you wish to simulate is stored. '''''Name'', is the name of the folder itself'''. In this case, we are using the GPT_CASE_RST folder which is included in the GrowthToolbox. You may wish to copy this folder elsewhere, if you intend to make changes to the interaction function.<br />
<br />
During a growth simulation, a mesh can be generated at each time step of the simulation, which provides a visual representation of the growth of the biological tissue, given the various parameters of growth specified by the interaction function and how they have changed over time. Put another way, the mesh shows exactly what the growing tissue actually looks like. ''Stages'' refers to the points in the simulation (measured in hours) at which meshes should be generated and saved. In this example, '''five stages of growth will be written to disk'''. These values are chosen to best capture the appearance of the tissue at important stages of the tissue growth.<br />
<br />
The final argument name listed here is ''modelname''. This is a model-specific argument, and in this case the GPT_CASE_RST interaction function contains '''three separate models for plant growth''', allowing the desired model to be a selected. Here, the value [1:3] is specified, which is evaluated in the same way as entering [1 2 3]. This instructs GFtboxCommand to run three separate simulations, one for each of the growth models contained in the interaction function.<br />
<br />
The function of every permissible argument is given by keying the following command into Matlab:<br />
<br />
help GFtboxCommand<br />
<br />
===3B - Getting results from a completed simulation===<br />
<br />
'''1) - Generating images from a simulation you ran on your computer'''<br />
<br />
Once a growth simulation is completed, the project folder (GPT_CASE_RST, in our example) will contain another folder named "movies". Within movies, there are folders which contain results for the executed simulations. As was instructed in 3A, here we can see three folders corresponding to simulations for the three separate models. Within the first folder there are three items:<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| ''CommandLine.txt'' - This file contains the Matlab command which was used to generate the results that this sub-directory contains.<br />
<br />
''gpt_case_rst.txt'' - This file, named according to the project name, contains a copy of the interaction function which was used to produce these results.<br />
<br />
''meshes'' - This folder contains the mesh files corresponding to the stages of growth specified by the value of the Stages argument, described in 3A.<br />
|width="500px" align="center"|[[File:resultsdir.png| Directory structure of a project containing results]]<br />
|}<br />
<br />
<br />
The mesh files contain vertices information regarding the shape of the growth model at a particular stage of growth, but are not visualisable in that form. In order to '''convert these mesh files into viewable images''', we can execute the following command in Matlab:<br />
<br />
VMSreport('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Project','GPT_CASE_RST',...<br />
'Experiment','All','flattentime',572.5,'morphogen','KPAR','SNAPFIG',true);<br />
<br />
Where the ''Path'' and ''Project'' arguments have the same function as ''Path'' and ''Name'' in 3A, i.e. ''Path'' refers to the location of the project folder and ''Project'' is the name of the project folder itself.<br />
<br />
Here we can see the generated images, including the command line arguments, for one of the simulations performed in 3A. Click on a thumbnail to view a larger image.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="middle"<br />
|width="125px"| 001commandline.png<br> [[File:commandlineexample.png | 125px]]<br />
|width="125px"| [[File:GPT_CASE_RST_example1.png | 125px]]<br />
|width="125px"| [[File:GPT_CASE_RST_example2.png | 125px]]<br />
|width="125px"| [[File:GPT_CASE_RST_example3.png | 125px]]<br />
|width="125px"| [[File:GPT_CASE_RST_example4.png | 125px]]<br />
|width="125px"| [[File:GPT_CASE_RST_example5.png | 125px]]<br />
|}<br />
<br />
<br />
'''2) - Generating images from a simulation you ran on the cluster'''<br />
<br />
NB. The results produced may not be visibly identical to those in the published literature. This is because of small, random perturbations which are applied to the initial model meshes to stop them from containing surfaces which are perfectly flat, and therefore biologically unrealistic. The results produced should be '''qualitatively but not quantitatively the same'''.<br />
<br />
'''3) - Interacting with your results using the GFtbox GUI'''<br />
<br />
='''4''' Altering the simulation parameters=<br />
<br />
One such simulation parameter is ''dt'', which is the time in seconds between iterations in the growth simulation. Large values of ''dt'' mean that fewer steps and therefore fewer calculations are required to complete a simulation. Whereas smaller values of ''dt'' mean that more steps are required, and therefore more processing time too. Therefore, a value of ''dt'' must be selected which is not so small that it is computationally unmanageable, but not so large that the observed growth is an artifact of the value of ''dt'', rather than the underlying growth model. It is necessary therefore to test a range of values for ''dt'' to ensure that the patterns of growth observed in a simulation are consistent across the range, and to find a value which is sufficient to demonstrate the model of growth and computationally efficient.<br />
<br />
'''Testing a range of ''dt''s can be achieved in several ways.''' One way is to use the ''dt'' argument when calling GFtboxCommand. This allows a single value to be tested. Another way is to make a batch of jobs, each using a different value for ''dt'', by using the 'State' argument, as described here. Lastly, a range of ''dt''s can be specified in GFtboxCommand, by using the dt argument where the values within the square brackets are the ''dt''s to simulate:<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5]);<br />
<br />
Here, we have specified '''four separate values for ''dt''''', and this means that '''four separate simulations will be run''' sequentially on your computer. As in 2B, images can then be generated for the mesh files produced, and compared to ensure consistency across the range of ''dt''s. In this figure, we can see the same model, at the same stage of growth, generated using the four values of ''dt''. Though quantitatively different, they are qualitatively the same, illustrating the suitability of the default ''dt'' value of 5.<br />
<br />
='''5''' Altering the model parameters=<br />
It is easy to see that when even a small number of range variables, mutations or ''dt''s are specified, the '''total number of simulations increases quickly'''. If many ranges are specified, then the amount of processing time becomes unmanageable on a single computer. Via ''GFtboxCommand'' and the ''ClusterMonitor'' tool, we can remotely''' run a number of simulations, in parallel, on a computing cluster'''. It is assumed that the GrowthToolbox and Matlab are installed on your cluster, and PuTTY is required on your local computer for the tools ''pscp'' (for transferring files from your computer to a cluster) and ''plink'' (for remotely executing commands on a cluster). Using the ''dt'' range argument from 3 as an example, here we add the 'Use' name argument with the value 'Cluster':<br />
<br />
GFtboxCommand('State','Start','Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5],'Use','Cluster');<br />
<br />
The 'Cluster' option for the 'Use' name argument instructs GFtboxCommand to upload the required project directory to a remote Linux server. Instead of running the simulation on your own computer, GFtboxCommand then works out how many individual simulations are specified by the command that invoked it. In this example, the only argument which will generate multiple simulations is ''dt'', of which there will be 4. Separate commands for each of these jobs are then automatically generated, each with an accompanying unique ID (The value of the ''ExpID'' name argument), and '''these are submitted as individual jobs to the cluster'''.<br />
<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.1,'ExpID','GPT_CASE_RST_1');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.5,'ExpID','GPT_CASE_RST_2');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',1,'ExpID','GPT_CASE_RST_3');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',5,'ExpID','GPT_CASE_RST_4');<br />
<br />
Once these jobs have been submitted, the ''ClusterMonitor'' tool opens and the new job batch ID is added to the list of jobs. Here, we will concentrate on three of the functions of ''ClusterMonitor'': ''Queue?'', ''Get project results'' and ''Make project pngs''. As in part 3 of this page, these functions will enable us to visualise the growth simulation results.<br />
<br />
Whilst ''ClusterMonitor'' is a useful tool for managing your cluster jobs, it is advisable to '''have at least a rudimentary understanding of and ability to use a Unix-based computer system'''. In particular, the abilities to list the contents of a folder, view the contents of a file, change the current working folder, delete, and to copy or move files or folders, are essential Unix skills for making sure everything is working as intended. It is beyond the scope of these tutorials to provide an in depth Unix tutorial (many good and simple tutorials exist on the web), but here is a short description of the Unix commands (which may be specific to our Unix-based system) that we believe to be important.</div>JacobNewmanhttp://cmpdartsvr3-v.uea.ac.uk/wiki/BanghamLab/index.php?title=Running_example_models_and_using_a_cluster&diff=5368Running example models and using a cluster2011-06-23T13:18:32Z<p>JacobNewman: </p>
<hr />
<div>[[GFtbox Tutorial pages | Back to GFtbox Tutorial pages]]<br />
<br />
The purpose of these tutorials is to learn how to run the example growth simulations included in GFtbox. We will describe methods for running simulations locally (on your own computer) and remotely (on a computing cluster). It is assumed that you have already downloaded the GFtbox software and have Matlab installed.<br />
<br />
=Getting Started=<br />
<br />
The remainder of this page is split into five sub-tutorials, each building on the preceding parts.<br />
<br />
1) '''Explaining the tools'''. In these tutorials, we will be using ''GFtboxCommand'' and ''ClusterMonitor''. This section explains the purpose of these tools. <br />
<br />
2) '''Computer or cluster?''' Here, we illustrate when and why you should use a computing cluster for your growth simulations, and conversely, when you should use your desktop computer.<br />
<br />
3) '''Running a growth simulation for an example model'''. This section demonstrates how to use ''GFtboxCommand'' to run a growth simulation on your computer. The model used for the simulation is an example model included with the GFtbox.<br />
<br />
4) '''Altering the simulation parameters'''. Following on from part 3, here we show how to adjust a simulation parameter within GFtboxCommand. Specifically, we alter the value of ''dt'', the temporal resolution of the simulation, and show how it can be used to verify that the value specified in the published literature is reasonable.<br />
<br />
5) '''Altering the model parameters'''. Finally, we demonstrate how a number of model parameters can be varied by specifying a range of values for each model argument. We show how the computationally expensive task of simulating all combinations of specified ranges can be processed more efficiently if a computer cluster is used via the ''ClusterMonitor'' tool.<br />
<br />
='''1''' Explaining the tools=<br />
==GFtboxCommand==<br />
This is a command line version of the GFtbox. By command line, we mean that all program '''functions are operated via typed commands, without the GUI'''. Like GFtbox, GFtboxCommand is capable of running growth simulations of an interaction function, and allows the user to specify model and simulation parameters. Unlike GFtbox, this also allows the user to select '''ranges of values for a number of input parameters''', and will automatically spawn multiple simulations which explore the various combinations of those parameters. This can be used to evaluate the effect of various parameters on the growth of a given model.<br />
<br />
==ClusterMonitor==<br />
Provides a graphical user interface for managing simulations '''running remotely on a computer cluster'''. Specifically, it allows you to see which projects are present and running on the cluster, to retrieve the completed projects, to generate images of the simulations at specified stages of growth, and to remove projects from the cluster. If you do not intend to use a computer cluster, then you will not need to use ClusterMonitor.<br />
<br />
='''2''' Computer or cluster?=<br />
<br />
In basic terms, '''a computing cluster is effectively a network of many computer processors''' (often hundreds), centrally managed by a queuing system. When a job is submitted to a cluster, the job is sent to a processor that is not being used, or queued until one becomes available. In contrast, a typical desktop computer will contain one processor, limiting the number of tasks that can be performed at any one time. Jobs which must be run independently and sequentially on a desktop computer can be executed in parallel on a cluster, '''greatly reducing the total time required to complete all of the jobs'''. Although the exact details of your cluster might vary from those described in the rest of these tutorials, we aim to illustrate the generic processes involved in using GFtbox on a cluster. We are happy to offer assistance, where possible, to setup GFtbox for your cluster, so please contact us if you have any queries.<br />
<br />
Whilst the time savings offered by using a cluster can be significant, there is an overhead associated with returning the results to your personal computer. The total time to run 2 or more simulations on a cluster and to return the results will be less than running those simulations sequentially on one computer. Therefore, using a '''cluster is ideal for situations where you would like to run several simulations''', such as to evaluate the effect of a range of parameters on a growth model. You are not advised to use a cluster for running single simulations, or where you would like to step through a simulation and change parameters in a more interactive fashion. In such circumstances, you are advised to run GFtbox or GFtboxCommand on your own computer.<br />
<br />
The '''GFtbox GUI provides quick feedback''' about how the changes you have made to an interaction function have affected the growth simulation, as '''you can see the result of each simulation iteration as it completes'''. It is quick and easy therefore to see if you have dramatically changed the course of the growth simulation, and then to adjust the parameters according to your observations. This approach is well suited to the early stages of design, where you might wish to '''tweak some parameters to gauge whether or not they have had the desired effect'''. Or in the final stages, where you think your model is almost finalised, but where small adjustments are required. '''An alternative approach is to start many simulations''', based upon your initial model and by making intelligent choices about which parameters to explore. This allows you to '''harvest many results, to quickly and easily overview them''' in their finalised form, or to select interesting-looking simulations and examine them more closely on your desktop computer, using GFtbox.<br />
<br />
='''3''' Running a growth simulation for an example model=<br />
This tutorial is aimed at running a '''growth simulation for one of the example interaction functions''' included with the GFtbox. The purpose of this exercise is to firstly demonstrate how simulations can be invoked using GFtboxCommand, and secondly to show how to reproduce experimental results (specifically, those published in Kennaway et al (2011)) given an interaction function.<br />
<br />
Assuming Matlab is installed on your computer, and the latest GFtbox has been downloaded, you can add the GFtbox directory to Matlab's search path, which makes the toolbox accessible from any other path that you choose to work from. For a short tutorial on how to do this, please click [[Adding GFtbox to Matlab's search path | here]].<br />
<br />
Once the GFtbox is added to Matlab, you are ready to run a growth simulation using GFtboxCommand. In this example, '''the model that we will simulate is called GPT_CASE_RST'''. Results generated using this model are published in Kennaway et al (2011). By running this simulation, we can confirm the results in the published literature and investigate the suitability of the various parameters.<br />
<br />
===3A - Running a simulation on your computer===<br />
<br />
The following command can be typed into Matlab to run a simulation of the GPT_CASE_RST interaction function, which contains three growth models: R, S and T. '''Three separate simulations are run sequentially on your computer''', one for each model, each producing results corresponding to five intervals in the growth simulation.<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',[1:3]);<br />
<br />
GFtboxCommand accepts input arguments as name and value pairs, e.g. 'modelname', [1:3] or 'Use','Cluster'. The argument names entered in the example above are: ''Path'', ''Name'', ''Stages'' and ''modelname''.<br />
<br />
The optional '''''Path'' argument name refers to the location of the folder''' (or directory) on your computer where the model interaction function you wish to simulate is stored. '''''Name'', is the name of the folder itself'''. In this case, we are using the GPT_CASE_RST folder which is included in the GrowthToolbox. You may wish to copy this folder elsewhere, if you intend to make changes to the interaction function.<br />
<br />
During a growth simulation, a mesh can be generated at each time step of the simulation, which provides a visual representation of the growth of the biological tissue, given the various parameters of growth specified by the interaction function and how they have changed over time. Put another way, the mesh shows exactly what the growing tissue actually looks like. ''Stages'' refers to the points in the simulation (measured in hours) at which meshes should be generated and saved. In this example, '''five stages of growth will be written to disk'''. These values are chosen to best capture the appearance of the tissue at important stages of the tissue growth.<br />
<br />
The final argument name listed here is ''modelname''. This is a model-specific argument, and in this case the GPT_CASE_RST interaction function contains '''three separate models for plant growth''', allowing the desired model to be a selected. Here, the value [1:3] is specified, which is evaluated in the same way as entering [1 2 3]. This instructs GFtboxCommand to run three separate simulations, one for each of the growth models contained in the interaction function.<br />
<br />
The function of every permissible argument is given by keying the following command into Matlab:<br />
<br />
help GFtboxCommand<br />
<br />
===3B - Getting results from a completed simulation===<br />
<br />
'''1) - Generating images from a simulation you ran on your computer'''<br />
<br />
Once a growth simulation is completed, the project folder (GPT_CASE_RST, in our example) will contain another folder named "movies". Within movies, there are folders which contain results for the executed simulations. As was instructed in 3A, here we can see three folders corresponding to simulations for the three separate models. Within the first folder there are three items:<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="top"<br />
|width="200px"| ''CommandLine.txt'' - This file contains the Matlab command which was used to generate the results that this sub-directory contains.<br />
<br />
''gpt_case_rst.txt'' - This file, named according to the project name, contains a copy of the interaction function which was used to produce these results.<br />
<br />
''meshes'' - This folder contains the mesh files corresponding to the stages of growth specified by the value of the Stages argument, described in 3A.<br />
|width="500px" align="center"|[[File:resultsdir.png| Directory structure of a project containing results]]<br />
|}<br />
<br />
The mesh files contain vertices information regarding the shape of the growth model at a particular stage of growth, but are not visualisable in that form. In order to '''convert these mesh files into viewable images''', we can execute the following command in Matlab:<br />
<br />
VMSreport('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Project','GPT_CASE_RST',...<br />
'Experiment','All','flattentime',572.5,'morphogen','KPAR','SNAPFIG',true);<br />
<br />
Where the ''Path'' and ''Project'' arguments have the same function as ''Path'' and ''Name'' in 3A, i.e. ''Path'' refers to the location of the project folder and ''Project'' is the name of the project folder itself.<br />
<br />
Here we can see the generated images, including the command line arguments, for one of the simulations performed in 3A. Click on a thumbnail to view a larger image.<br />
<br />
{| border="1" cellpadding="5" cellspacing="5"<br />
|- valign="middle"<br />
|width="125px"| [[File:commandlineexample.png | 125px]]<br><br />
|width="125px"| [[File:GPT_CASE_RST_example1.png | 125px]]<br />
|width="125px"| [[File:GPT_CASE_RST_example2.png | 125px]]<br />
|width="125px"| [[File:GPT_CASE_RST_example3.png | 125px]]<br />
|width="125px"| [[File:GPT_CASE_RST_example4.png | 125px]]<br />
|width="125px"| [[File:GPT_CASE_RST_example5.png | 125px]]<br />
|}<br />
<br />
'''2) - Generating images from a simulation you ran on the cluster'''<br />
<br />
NB. The results produced may not be visibly identical to those in the published literature. This is because of small, random perturbations which are applied to the initial model meshes to stop them from containing surfaces which are perfectly flat, and therefore biologically unrealistic. The results produced should be '''qualitatively but not quantitatively the same'''.<br />
<br />
'''3) - Interacting with your results using the GFtbox GUI'''<br />
<br />
='''4''' Altering the simulation parameters=<br />
<br />
One such simulation parameter is ''dt'', which is the time in seconds between iterations in the growth simulation. Large values of ''dt'' mean that fewer steps and therefore fewer calculations are required to complete a simulation. Whereas smaller values of ''dt'' mean that more steps are required, and therefore more processing time too. Therefore, a value of ''dt'' must be selected which is not so small that it is computationally unmanageable, but not so large that the observed growth is an artifact of the value of ''dt'', rather than the underlying growth model. It is necessary therefore to test a range of values for ''dt'' to ensure that the patterns of growth observed in a simulation are consistent across the range, and to find a value which is sufficient to demonstrate the model of growth and computationally efficient.<br />
<br />
'''Testing a range of ''dt''s can be achieved in several ways.''' One way is to use the ''dt'' argument when calling GFtboxCommand. This allows a single value to be tested. Another way is to make a batch of jobs, each using a different value for ''dt'', by using the 'State' argument, as described here. Lastly, a range of ''dt''s can be specified in GFtboxCommand, by using the dt argument where the values within the square brackets are the ''dt''s to simulate:<br />
<br />
GFtboxCommand('Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5]);<br />
<br />
Here, we have specified '''four separate values for ''dt''''', and this means that '''four separate simulations will be run''' sequentially on your computer. As in 2B, images can then be generated for the mesh files produced, and compared to ensure consistency across the range of ''dt''s. In this figure, we can see the same model, at the same stage of growth, generated using the four values of ''dt''. Though quantitatively different, they are qualitatively the same, illustrating the suitability of the default ''dt'' value of 5.<br />
<br />
='''5''' Altering the model parameters=<br />
It is easy to see that when even a small number of range variables, mutations or ''dt''s are specified, the '''total number of simulations increases quickly'''. If many ranges are specified, then the amount of processing time becomes unmanageable on a single computer. Via ''GFtboxCommand'' and the ''ClusterMonitor'' tool, we can remotely''' run a number of simulations, in parallel, on a computing cluster'''. It is assumed that the GrowthToolbox and Matlab are installed on your cluster, and PuTTY is required on your local computer for the tools ''pscp'' (for transferring files from your computer to a cluster) and ''plink'' (for remotely executing commands on a cluster). Using the ''dt'' range argument from 3 as an example, here we add the 'Use' name argument with the value 'Cluster':<br />
<br />
GFtboxCommand('State','Start','Path','/GrowthToolbox/Models/Published/Kennaway-etal-2011/','Name','GPT_CASE_RST',...<br />
'Stages',[20 100 140 180 200],'modelname',3,'dt',[0.1 0.5 1 5],'Use','Cluster');<br />
<br />
The 'Cluster' option for the 'Use' name argument instructs GFtboxCommand to upload the required project directory to a remote Linux server. Instead of running the simulation on your own computer, GFtboxCommand then works out how many individual simulations are specified by the command that invoked it. In this example, the only argument which will generate multiple simulations is ''dt'', of which there will be 4. Separate commands for each of these jobs are then automatically generated, each with an accompanying unique ID (The value of the ''ExpID'' name argument), and '''these are submitted as individual jobs to the cluster'''.<br />
<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.1,'ExpID','GPT_CASE_RST_1');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',0.5,'ExpID','GPT_CASE_RST_2');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',1,'ExpID','GPT_CASE_RST_3');<br />
GFtboxCommand('Name','GPT_CASE_RST','Stages',[20 100 140 180 200],'modelname',3,'dt',5,'ExpID','GPT_CASE_RST_4');<br />
<br />
Once these jobs have been submitted, the ''ClusterMonitor'' tool opens and the new job batch ID is added to the list of jobs. Here, we will concentrate on three of the functions of ''ClusterMonitor'': ''Queue?'', ''Get project results'' and ''Make project pngs''. As in part 3 of this page, these functions will enable us to visualise the growth simulation results.<br />
<br />
Whilst ''ClusterMonitor'' is a useful tool for managing your cluster jobs, it is advisable to '''have at least a rudimentary understanding of and ability to use a Unix-based computer system'''. In particular, the abilities to list the contents of a folder, view the contents of a file, change the current working folder, delete, and to copy or move files or folders, are essential Unix skills for making sure everything is working as intended. It is beyond the scope of these tutorials to provide an in depth Unix tutorial (many good and simple tutorials exist on the web), but here is a short description of the Unix commands (which may be specific to our Unix-based system) that we believe to be important.</div>JacobNewman