New to the Lab
Getting Work Done
Computer Resources
Please see FAQ: Shared Resources for in depth list of available resources.
Virtually all work in the lab is done in one of four ways:
- On a group resource lab computer
- On the hoffman2 cluster
- On the miles cluster
- On another lab member's computer
Let's cover each of these in detail and how you can gain access to that resource.
Group Resource Lab Computer
These are computers available for reserving by anyone in the lab. So far, we have approximately 5 of these. To gain an account on these computers, you must send a request email to: CCN Support with the following information
Email Subject: Lab Account Request Name: Your Name Email: Your preferred email Phone: Phone Number to reach you when out of the office AIM: your aim chat account if you don't mind being reached this way
You receive a confirmation email with any specific instructions once your account is created.
Backups
Every time you log out of or into a lab computer, your homes are sync'd to the main server *each time you log out*. So, if you have a computer assigned to you make a point of logging out (a safe and practical thing to do) to ensure all your data is backed up.
If you accidentally delete a file or folder, send an email to support and request that it be restored. Please provide
- the full path to the file or directory you want restored
- whether you'd like it copied back to your computer or emailed to you separately.
If you have a personal computer you would like backed up, please send an email to support requesting it be configured for you.
If you have a personal backup account, all backups can be accessed and schedules set at:
Accessing the backup server requires a lab account.
Hoffman2 Cluster
A very detailed, step by step walk through on obtaining an account on Hoffman2 is found on the Getting_a_Miles_or_Hoffman2_Account wiki page. Please remember that all support issues, lost passwords, forgotten accounts, and more regarding Hoffman2 should be directed to the ATS Support.
Read their FAQ first!
Once you've obtained an account, you may wish to look over the Every Day UNIX and Working with Hoffman page.
Another Lab Member's Comptuer
Although it may sound wild, the only person who can help you with this is the person who owns the computer. :)
Tools & Utilities
If you are working on Hoffman2, miles, or a lab computer all the utilities you need to get working are already available. If this is not the case, please send a request to CCN Support requesting the program be added.
Editors
A word on editors. Just about any editor is sufficient. However, some are less prone to user error than others when used for tasks such as programming or scripting.
TextEdit
TextEdit is the built in OSX editor. By default, the files it creates are Rich Text. These are entirely unworkable for scripting or programming use. It also has a bad habit of leaving off appropriate UNIX end of line characters (this means a file looks like one big line to the operating system).
Though a decent general purpose editor, it's highly discouraged to use this editor for scripting, programming, or editing of system files (like your bash_profile).
Vim
Vim is an old standby, it's the default editor on many, many unix based systems. It is a full featured editor with many plugins to extend functionality (if you need them). It's usable in the Terminal and has several GUI editors based on it as well. For the mac, the best one I've found is MacVim.
Vim is a modal editor. This method of editing puts off some users. If you're interested, you can access the built in tutorial called (ingeniously) vimtutor.
$ vimtutor
Emacs
Emacs is the other big player in editors. It's a first class editor with very powerful features, but easy enough to use for simple tasks as well. It also is usable in both the terminal and has several GUI options like AquaEmacs
You can access the emacs tutorial by
$ emacs
Once inside emacs, use the following key combo:
Control-h, t
Komodo Edit
From the site: Dynamic language expertise for Perl, PHP, Python, Ruby, and Tcl, plus JavaScript, CSS, HTML, and XML, and template languages like RHTML, Template-Toolkit, HTML-Smarty and Django.
Komodo Edit runs on every platform and supports both Vim and Emacs keybindings. It's really a first class editor by any definition of the word.
TextMate
TextMate is a very popular OSX editor. It has all the features you'd expect such as syntax highlighting, auto-indenting, etc. It's only draw back is that it does not have a terminal interface and does not work on other platforms besides OSX. It also costs money, which might turn away a starving student or two. For more information see TextMate.
Others
These are just a few and by no means meant to be exhaustive. However, there are some "essential features" you should look for when choosing your editor of choice. A few are:
- Syntax Coloring for source code and scripts
- Auto-indentation
- Plugable (e.g. others can extend it via plugins)
Though worthy of simple usage, the common basic editors Nano, Pico, and friends often do not provide the tools or functionality for writing error free, easy to read code or scripts. For anything beyond the ridiculously trivial the user should seriously consider a more feature rich editor.
Keeping "In the Loop"
Lab meetings, mail outs, notifications, where do they come from and how do you get them? Easy.
Mailing Lists
- The Cohen-Bookheimer Lab mailing list receives all email specific to our lab. This includes notifications of hardware issues, software updates, general notifications, meetings, filed trips, and picnics.
- The Cluster-Users mailing list receives notifications about various cluster issues and updates. These are mostly hardware related or may involve software specific to the cluster, e.g. a scheduled down time for updating FSL.
- Classes. If you're taking a course from Susan or Mark, there may be a mailing list specifically for that class. Please check with your professor for details.
Lab Meetings
| Lab | Time | Location |
|---|---|---|
| Bookheimer | 1pm Thursdays | C8-369 |
| Cohen | 1pm Every Other Thursday | C8-885 |
Blogs & Wikis
If you're here you found one of them! Here's the full list:
- CCN Wiki
- HowTo pages, pages for classes, and for image analysis.
- Lab Groups Blog/Wiki
- Honestly, this one isn't much used. But it may be worth wasting 5m to see if anything interesting is there.
- Lab Users Blog/Wii
- Except for the James Kyle blog, these aren't used much. If you're a lab member, you can create your own blog/wiki here if that interests you. Just sign in using the username and password that you were given for logging into lab computers.
Other Sources of Information (What You Should Know)
During the course of your work for the lab, you'll often need to perform various tasks that revolve around using image analysis software, scripting, shell scripting (bash, zsh), and use of clusters (qsub, qstat, and writing job submission scripts).
In addition to the above resources, here are some links to get started.
Sun Grid Engine (The Cluster)
For information on the most commonly used commands for managing and submitting cluster jobs, see the ATS SGE page.
Python
Python is the official scripting language of the lab. Anything beyond trivial work should use python to ensure homogeneity, interoperability, and future use of that work.
Experienced in Programming, New to Python
If you're already an experienced programmer, but new to Python. The free online book Dive Into Python is right up your alley. It only lightly focuses on basic controllers like iterators, loops, conditional branching (if/else), and common data structures (arrays, hashes, etc). It focuses on specific Python idioms and linguistic sugar to get you from 0 to coding in as short a time as possible.
Please note the book was written for Python 2.2, so some of the behavior may vary slightly from that given in the book. For example, dir() lists both built-ins and public methods rather than just public methods. However, the differences aren't that significant and it still servers well as a quickie introduction.
New to Programming
If you're new to programming in general, the Think Python free online book was written with you in mind. It's a gentle introduction to Object Oriented design principles...the python way. Don't worry, if you don't know what that means this manual will teach you.
Standard Documentation
Finally, there are the standard Python resources of Official Python Tutorial, The Semi-Official Python FAQ, and Python.org.
Shell Scripting
BASH is the shell you'll most likely be using in the lab and on our clusters. Knowing the shell and how to write simple scripts is absolutely essential to get your work done without suffering the ails of Repetitive Stress Syndrome. Ask around the office about Carpal Tunnel if you need inspiration to learn how to script. :)
New to UNIX and BASH
Here's a general list of useful sites to get you started on bash and unix
- Bash FAQ
- Bash Guide
- Bash Reference Manual
- Bash Hackers Wiki
- Learn to Quote Everything you'd want to know about how quotes work in bash....and probably a lot more you don't.
- Bourne Shell Idioms
- Shelldorado Collection of tutorials, guides, articles, and more about the Bash shell.
Familiar with UNIX and BASH, But No Expert
If you're familiar with the Terminal, Bash, and getting around, but are no scripting Guru there's one stop to get you from n00b to whiz.
Getting, Sending, and Transferring Files, Logging In, and All That Jazz
The main method of transferring files back and forth and logging into remote servers is OpenSSH.
Finding More Information
Man Pages
For further information there is one resource above all others. I'm going to repeat it 3 times (each repetition imbues the phrase with deeper meaning...I learned this from top 40 music):
- Man Pages
- Man Pages
- Man Pages
Man Pages are the be all end all of information for UNIX programs. Man is short for "Manual" or as Websters likes to chirp:
1. a book of instructions, esp. for operating a machine or learning a subject; a handbook : a computer manual | a training
And that's just what man pages do, they instruct you on all the ins and outs of how to use a specific program. Man pages are a formal language. They require a rigid format that is unambiguous and succinct in its interpretation. It's not as bad as it sounds, but may look confusing at first. Fear not! Our friendly O'Reilly members have written a tutorial on how to use the built in UNIX manuals and documentation to uncover the mystery:
- Read the F(riendly) Manpage or RTFM
It may sound funny, but Google is an amazing resource. Often simply typing in the error you receive in google is enough to find a solution.
Boolean, AND Searches
In my experience, one powerful feature of Google that many are not aware of is its Boolean search methods. By default, the AND operator is used. For example, a search for red dog is translated as "red AND dog". This means google looks for all pages that contain both the words Red and the word Dog, but not pages with only red or only dog.
Quoted Searches
Any set of words placed in quotes is taken as a literal search. For example, by quoting the above search "red dog" google finds all pages with the exact string "red dog", but ignores pages that may have red and dog at different parts of the page. The quoted string is very useful for finding information about errors.
Broadening and Narrowing Searches
The technique of broadening and narrowing searches is the most important for efficient Googling. Here are some tips:
- If you search for the word "analysis" and find nothing, try a variation of the word such as "analyze".
- Use the shortest variation of a word in your search that is still unique or pertinent to increase the number of results. For example, instead of searching for "programming" search for "program". The key here is that all pages that contain "programming" also contains "program", thus you have broadened the possible number of hits. Some other results:
- instead of scanning, search for scan
- instead of unfolding, search for unfold
- instead of boolean, search for bool
- Narrow your search with quotes
- For example, if you search for functional magnetic resonance and get far too many hits, you can narrow the results by searching for the exact string "functional magnetic resonance"
- Use acronym and synonyms liberally. e.g. MRI instead of magnetic resonance imaging
- Eliminate terms from your search that are obviously specific to your individual case.
- For example, say you get an error: "FSL Fubar Error 521: some_file_I_named.txt". Now, obviously not everyone in the world has a file named "some_file_I_named.txt". Thus, it is a frivolous search term that does little in helping google find a solution for us. Trimming this down to the terms that would be common to any error of this time produces much better results. e.g. "FSL Fubar Error 521"
AHHHHH HELP ME!!: How to Get Help
You've read the docs, you've searched google, you've read the man pages....and yet cannot find a solution. What do you do now?
First off, it's worth looking over this informative write up on the process of How_to_Ask_for_Help. Often in our frustration we overlook the obvious. How_to_Ask_for_Help serves as a nice guide on the steps to take in building a good help request (good for you and good for the person you're asking). It helps cut through the frustration and confusion in order to get to what we're really looking for. . . Answers!
After you've got a well formed support question, fire an email off to CCN Support. You'll receive updates on the status of your inquiry, notification by email when the issue is resolved, and links to possible solutions.