Wynton
@@ -668,8 +670,6 @@ checking on jobs)
nodes have
- The primary method to log in is to use an SSH client
application
-- The Wynton HPC is up to date with information on logging in: Access
-Cluster
Names:
log1, log2 and plog1 (for PHI users)
@@ -738,78 +738,6 @@ container
-
-The File System
-
-- A file system how information is stored and retrieved on a computer
-
-- Consists of files and directories
-
-- A local file system is function of the operating system and only
-accessible from a single computer
-- A shared file system is accessible from multiple computers
-
-
-
-BeeGFS
-
-- Wynton uses a parallel shared file system called BeeGFS
-
-- The files are stored as “chunks” spread across many different
-servers
-
-- BeeGFS has multiple services that work together to manage the file
-system
-
-- Storage (stores the chunks)
-- Metadata (tracks the chunks and information about their file)
-- Management (tracks all of the services)
-- Client (provides linux access to the file system)
-
-
-
-
-BeeGFS - Advantages
-
-- High throughput
-- Redundancy can be built in by mirroring services
-- Adding new storage is fast and does not require downtime
-
-
-
-BeeGFS - Caveats
-
-- For any client node, performance is limited by the network bandwidth
-of that node
-- Network latency becomes extremely important for all metadata
-requests
-- Certain input/output patterns can be problematic
-
-
-
-BeeGFS - I/O patterns
-
-- Anything that requires lots of metadata operations can feel slow
-
-- e.g: lots of writes to the same directory and lots of file lookups
-and directory searches (conda)
-
-- Keep the number of reads and writes to a single directory to a
-reasonable number
-
-
-
-BeeGFS - Takehome Message
-
-- Prefer fewer, large files over many small ones
-- Distribute reading and writing over several directories
-- Use local scratch (/scratch) when possible
-- Don’t include anything in /wynton in your default
-LD_LIBRARY_PATH
-- If using conda, putting the conda application inside a Apptainer
-(formerly singularity) container will result in better performance
-
Storage
@@ -1009,20 +937,36 @@ from source
[alice@dev1 ~]$ mkdir -p "/scratch/$USER"
[alice@dev1 ~]$ cd "/scratch/$USER"
-[alice@dev1 alice]$ wget https://github.com/samtools/samtools/releases/download/1.19.2/samtools-1.19.2.tar.bz2
-[alice@dev1 alice]$ tar -x -f samtools-1.19.2.tar.bz2
+[alice@dev1 alice]$ wget https://github.com/samtools/samtools/releases/download/1.21/samtools-1.21.tar.bz2
+[alice@dev1 alice]$ tar -x -f samtools-1.21.tar.bz2
- Create install location and configure
-[alice@dev1 ~]$ mkdir -p $HOME/software/samtools-1.14
-[alice@dev1 ~]$ cd samtools-1.19.2
-[alice@dev1 ~]$ ./configure --prefix=$HOME/software/samtools-1.14
+[alice@dev1 ~]$ mkdir -p $HOME/software/samtools-1.21
+[alice@dev1 ~]$ cd samtools-1.21
+[alice@dev1 ~]$ ./configure --prefix=$HOME/software/samtools-1.21
- Build and install
[alice@dev1 ~]$ make
[alice@dev1 ~]$ make install
+
Install Nextflow
@@ -1065,12 +1009,13 @@ Rocky 8 Linux
Definitions
-- Containers: An isolated environment for running
-software that is created from an image file, preventing
-conflicts with the host system.
-- Images: An ordered collection of root filesystem
-changes that contain all necessary dependencies, ensuring software run
-identically across various computing platforms.
+- Containers: An isolated environment for running
+software that avoids conflicts with the host system. Containers are
+stored, shared and executed as image files with a .sif
+extension.
+- Images: are built from definition files (or
+Dockerfiles) which are a set of instruction you specify for your
+environment.
@@ -1150,20 +1095,13 @@ run:
diff --git a/docs/Working_on_Wynton_Part_2.html b/docs/Working_on_Wynton_Part_2.html
index 1984afa..4022ae4 100644
--- a/docs/Working_on_Wynton_Part_2.html
+++ b/docs/Working_on_Wynton_Part_2.html
@@ -45,6 +45,69 @@ span.underline{text-decoration: underline;}
div.column{display: inline-block; vertical-align: top; width: 50%;}
div.hanging-indent{margin-left: 1.5em; text-indent: -1.5em;}
ul.task-list{list-style: none;}
+pre > code.sourceCode { white-space: pre; position: relative; }
+pre > code.sourceCode > span { line-height: 1.25; }
+pre > code.sourceCode > span:empty { height: 1.2em; }
+.sourceCode { overflow: visible; }
+code.sourceCode > span { color: inherit; text-decoration: inherit; }
+div.sourceCode { margin: 1em 0; }
+pre.sourceCode { margin: 0; }
+@media screen {
+div.sourceCode { overflow: auto; }
+}
+@media print {
+pre > code.sourceCode { white-space: pre-wrap; }
+pre > code.sourceCode > span { display: inline-block; text-indent: -5em; padding-left: 5em; }
+}
+pre.numberSource code
+{ counter-reset: source-line 0; }
+pre.numberSource code > span
+{ position: relative; left: -4em; counter-increment: source-line; }
+pre.numberSource code > span > a:first-child::before
+{ content: counter(source-line);
+position: relative; left: -1em; text-align: right; vertical-align: baseline;
+border: none; display: inline-block;
+-webkit-touch-callout: none; -webkit-user-select: none;
+-khtml-user-select: none; -moz-user-select: none;
+-ms-user-select: none; user-select: none;
+padding: 0 4px; width: 4em;
+color: #aaaaaa;
+}
+pre.numberSource { margin-left: 3em; border-left: 1px solid #aaaaaa; padding-left: 4px; }
+div.sourceCode
+{ }
+@media screen {
+pre > code.sourceCode > span > a:first-child::before { text-decoration: underline; }
+}
+code span.al { color: #ff0000; font-weight: bold; }
+code span.an { color: #60a0b0; font-weight: bold; font-style: italic; }
+code span.at { color: #7d9029; }
+code span.bn { color: #40a070; }
+code span.bu { color: #008000; }
+code span.cf { color: #007020; font-weight: bold; }
+code span.ch { color: #4070a0; }
+code span.cn { color: #880000; }
+code span.co { color: #60a0b0; font-style: italic; }
+code span.cv { color: #60a0b0; font-weight: bold; font-style: italic; }
+code span.do { color: #ba2121; font-style: italic; }
+code span.dt { color: #902000; }
+code span.dv { color: #40a070; }
+code span.er { color: #ff0000; font-weight: bold; }
+code span.ex { }
+code span.fl { color: #40a070; }
+code span.fu { color: #06287e; }
+code span.im { color: #008000; font-weight: bold; }
+code span.in { color: #60a0b0; font-weight: bold; font-style: italic; }
+code span.kw { color: #007020; font-weight: bold; }
+code span.op { color: #666666; }
+code span.ot { color: #007020; }
+code span.pp { color: #bc7a00; }
+code span.sc { color: #4070a0; }
+code span.ss { color: #bb6688; }
+code span.st { color: #4070a0; }
+code span.va { color: #19177c; }
+code span.vs { color: #4070a0; }
+code span.wa { color: #60a0b0; font-weight: bold; font-style: italic; }
