Hoffman2:Linux Tutorial: Difference between revisions

From Center for Cognitive Neuroscience
Jump to navigation Jump to search
m (6 revisions)
No edit summary
Line 1: Line 1:
[[Hoffman2|Back to all things Hoffman2]]
[[Hoffman2|Back to all things Hoffman2]]


Working with UNIX and UNIX like systems via the command line can be intimidating. The good news is a normal user only needs to have a firm understanding of some principle actions to efficiently use UNIX for their every day work.
== Lynda.com tutorials ==


# POSIX Permissions
UCLA employees and students have access to video tutorials through Lynda.com.
# File System Navigation
# File & Shell Management
# Environment variables


Each of these areas requires knowledge of a few small programs and how they behave. In the following sections, each program and utility is covered in detail. When appropriate, a link to the relevant wiki page on the topic is provided.  
One course that will be especially helpful to Hoffman2 users is "Learning Linux Command Line". This video series will help you get up to speed with commands like cd, ls, mkdir, cp, mv, find, grep, awk, sed, vi, tar, and so on. The whole series will take you about 2 hours to complete, but you are welcome to pick and choose from the individual tutorials that interest you (each video runs from about 2 to 6 minutes).


Each section is followed by an itemized list of the programs covered and some links to further information on that particular utility as well as the command to bring up the man page for the utility contained within parthesis.
To access these video tutorials:
# Start here: http://learnit.ucla.edu
# Click "Lynda.com"
# Log in with your UCLA Logon ID and password
# If this is the first time you are using Lynda.com, you may need to type in your name and preferred email address.
# Search for "Learning Linux Command Line"


Where appropriate, links to external tutorials deemed suitable and appropriate to the topic are provided rather than reproducing the work here.
Once you have sharpened your Linux command line skills, search around for other videos that interest you.  Hint: I really enjoyed the "iMovie Essential Training" course.
 
 
 
==POSIX Permissions==
POSIX permissions are what determines who and to what degree users can access a file.
 
The key terminology and function of the POSIX permission system is found here:
:<b>[[UNIX Permissions]]</b>
 
 
===List of Utilities Covered===
* [[UNIX Permissions#ls | ls]]
* [[UNIX Permissions#chmod | chmod]] (man chmod)
** [http://www.dartmouth.edu/~rc/help/faq/permissions.html Dartmouth Tutorial]
** Exhaustive tutorial including min-quizzes [http://www.catcode.com/teachmod/ CatCode Tutorial]
* [[UNIX Permissions#chgrp | chgrp]]
* [[UNIX Permissions#umask | umask]]
* [[UNIX Permissions#newgrp | newgrp]]
 
==File System Navigation==
The following series of tutorials provide a very basic introduction to file system navigation on unix like systems without any assumptions of prior knowledge on the topics.
 
* Listing files & directories, making directories, changing directories, the . and .. directories, pathnames, the "home" directories
** [http://www.ee.surrey.ac.uk/Teaching/Unix/unix1.html Tutorial One]
* Copying files, moving files, removing files and directories, displaying the contents of a file, searching the contents of a file
** [http://www.ee.surrey.ac.uk/Teaching/Unix/unix2.html Tutorial Two]
 
===List of Utilities Covered===
 
* ls
* mkdir
* cd
* pwd
* cp
* mv
* rm
* cat
* less
* head & tail
* grep
 
 
 
==File & Shell Management==
This is where your career on a UNIX type system can be made or crippled. Sure you know how to move around, list files, find out where you are, and display the contents of files. But now you have to ''do something'' with those files. And let's face it, there are a whole lot of files.
 
It is '''''highly recommended''''' that the reader look over [http://www.ee.surrey.ac.uk/Teaching/Unix/unix4.html Tutorial Four] on how to use wildcards for matching before preceding with this section.
 
The good news is, using the above utilties we just learned about we can accomplish almost anything we want to do using a very handy utility called '''''find'''''.
 
As the name might imply, find, well, finds things. What it finds is up to you. ''find'' has many, many options. All laid out in its man page. However, for most purposes only a few are needed. We'll cover those here.
 
A basic find command looks like
$ find /path/to/directory -name 'filename.txt'
This command looks at all files in /path/to/directory and in all directories therein for a file named 'filename.txt'.
 
===Common Options===
;-type
:Specifies the type of file we're looking for. e.g. text file, directory, link, etc.
;-name
:Specifies the name of the file. Case Sensitive
;-iname
:Specifies the name of the file. Case Insensitive
;-or
:Joins the precedeing and following terms by the boolean OR
;-and
:Joins the preceding and following terms by the boolean AND
;-not
:negates the next term. e.g. -not -empty means "is not empty"
;-exec
:excutes a shell command for each file found. The only 'trick' is to replace the actual file name with {} and end the command with a \;. This should become clear when reviewing the examples below.
;-empty
:The file or directory is empty.
 
We can combine the above options to preform complex searches on the file system and, even better, execute commands on those search terms. For a list of all options and their arguments, please see [http://unixhelp.ed.ac.uk/CGI/man-cgi?find Find Man Page].
 
===Examples===
#Find all directories named 'tsplot' in the current directory
#: <pre> $ find . -name tsplot -type d </pre>
#Find all empty directories in the directory /u/home9/foo/data
#: <pre> $ find /u/home9/foo/data -empty </pre>
#Find all empty files or directories named 'tsplot' in the current directory
#: <pre> $ find . -name tsplot -or -type d -empty </pre>
 
Using commands we've already learned to perform actions on the above.
#Find all directories named 'tsplot' in the current directory and delete them using ''rm''
#: <pre> $ find . -name tsplot -type d -exec rm -rf {} \; </pre>
#Find all empty directories in the directory /u/home9/foo/data and delete them
#: <pre> $ find . -type d -empty -exec rmdir {} \; # using rmdir for empty directories is safe as it will refuse to delete a directory with a file in it </pre>
#Find all empty files or directories named 'tsplot' in the current directory and delete them
#: <pre> $ find . -name tsplot -or -type d -empty -exec rm -rf {} \; </pre>
But we're not restricted to rm, of course, we can use any of the utilities we've learned about as the -exec argument.
#Find all files named design.fsf and look for a subject named 'foo'
#: <pre> $ find . -name design.fsf -type f -exec grep "foo" {} \; </pre>
 
With a little reading of the man page, we find a new option called ''-user'', which surprisingly finds all files that belong to the specified user
#Find all files owned by user 'foo' and change their permissions
#: <pre> $ find . -type f -user foo -exec chmod -R ug+rwX {} \; </pre>
 
 
 
==Environment Variables==
UNIX uses environment variables to pass information to various tools during a session.  These variables are named in all capitals by convention.  You can see all of the environment variables and their values by using the command
$ env
To simply see the value of one variable, you can ''echo'' its value
$ echo $VARIABLENAME
 
There are a few environment variables that you should be familiar with...
 
===USER===
This has the username of the current user, which should usually be you.  See what the value is
$ echo $USER
 
===HOME===
This is the path to the home directory of the current user.  The tilde symbol (''~'') is also recognized as shorthand for this home directory.
$ echo $HOME
 
===PATH===
This is a list of directories, separated by colons, in which the operating system should check for commands that you type.  For instance, when you were using the ''ls'', ''grep'', or ''find'' commands in previous tutorials, the operating system started looking through the directories in your PATH variable to find the '''first''' command that matched that name and tried executing it.  See the directories that the operating system will check for you by typing
$ echo $PATH
 
Making additions to this variable can be important.  Let's say you have a personal set of scripts you have created and you store them in a directory <code>~/scripts</code>.  Every time you want to use one of those scripts, you have to type out the full path to it
$ ~/scripts/my-first-script.sh
or
$ $HOME/scripts/my-first-script.sh
 
This can get tiring.  If you added your ''scripts'' directory to your path, you wouldn't have to type that extra bit every time.  The best way to do this would be to edit your Bash Profile with a [[Text Editors|text editor]]. e.g.
$ vim ~/.bash_profile
and add the line
export PATH=$PATH:~/scripts
to the end of the file and saved it.  This will "export" the variable named PATH to the environment and set its value equal to whatever was already in PATH plus the directory ''~/scripts''.  The next time you login, you can do
$ echo $PATH
and see that at the end of the list of directories to search, your directory ''~/scripts'' has been added.  Now you can be anywhere in the filesystem and call the command
$ my-first-script.sh
to run that same script from before.
 
====Collisions on the PATH====
If you get in the habit of naming your scripts the same thing (e.g. ''my-script.sh'') and placing them in different directories, you may run into a collision on your PATH.  This is a case where you think you are running one script, but the operating system is actually running another.  Let's look at an example.
 
Continuing from the previous example where we have the script ''~/scripts/my-first-script.sh'', let's say that we make another directory called ''analyze'' and we are working with some data there and make a processing script coincidentally called ''my-first-script.sh''.  So we have the files
~/scripts/my-first-script.sh
~/analyze/my-first-script.sh
~/analyze/data-file-1
~/analyze/data-file-2
...
And we have amended our ''.bash_profile'' so that ''~/scripts'' is at the end of our PATH environment variable.
 
If we change to the ''analyze'' directory
$ cd ~/analyze
and wish to run the processing script ''my-first-script.sh'' on the data, you may think we can execute
$ my-first-script.sh
and call it a day.  But this will actually run the file ''~/scripts/my-first-script.sh'' because it is the first file the operating system found in the directories of PATH that matched that name.  If you wanted to verify this, you can execute
$ which my-first-script.sh
This command will search your PATH variable for the first instance of ''my-first-script.sh'' and return the full path to it, something like this
~/scripts/my-first-script.sh
 
To run the script we had intended, we would need to execute
$ ./my-first-script.sh
The period and slash specify that the operating system should look in the current directory for this script.
 
'''If something seems weird or script isn't working, a good starting point is to check that you are running the script you think you are.  Use ''which'' to find out'''
 
 
 
==Learning to Fish or The Man Pages==
However, possibly even more important is being able to find information on one's own. In the spirit of this, the instructional is concluded with a detailed coverage of the built-in unix manual pages. Virtually every program installed on a UNIX like system provied a manual page for reference. Even the most proficient of UNIX users could never retain the dizzying array of programs and their options in their working memory and with the built-in manual pages, they'll never need to.
 
The man pages (for ''manual'') are the be all end all reference on UNIX systems. When reading a man page becomes natural, you know there is nothing you can't get done quickly and efficiently.
 
A [http://www.tfug.org/helpdesk/general/man.html Beginners Guide to man Pages] is an excellent introduction into how to move around a man page easily and understand what it's telling you.
 
 
 
==External Links==
*[http://www.ee.surrey.ac.uk/Teaching/Unix/ UNIX Tutorial]

Revision as of 18:19, 11 August 2017

Back to all things Hoffman2

Lynda.com tutorials

UCLA employees and students have access to video tutorials through Lynda.com.

One course that will be especially helpful to Hoffman2 users is "Learning Linux Command Line". This video series will help you get up to speed with commands like cd, ls, mkdir, cp, mv, find, grep, awk, sed, vi, tar, and so on. The whole series will take you about 2 hours to complete, but you are welcome to pick and choose from the individual tutorials that interest you (each video runs from about 2 to 6 minutes).

To access these video tutorials:

  1. Start here: http://learnit.ucla.edu
  2. Click "Lynda.com"
  3. Log in with your UCLA Logon ID and password
  4. If this is the first time you are using Lynda.com, you may need to type in your name and preferred email address.
  5. Search for "Learning Linux Command Line"

Once you have sharpened your Linux command line skills, search around for other videos that interest you. Hint: I really enjoyed the "iMovie Essential Training" course.

Retrieved from "https://www.ccn.ucla.edu/wiki/index.php?title=Hoffman2:Linux_Tutorial&oldid=3468"