courses.edx.org : tr, tee, wc, cut

The tr utility is used to translate specified characters into other characters or to delete them. The general syntax is as follows:

$ tr [options] set1 [set2]

The items in the square brackets are optional. tr requires at least one argument and accepts a maximum of two. The first, designated set1 in the example, lists the characters in the text to be replaced or removed. The second, set2, lists the characters that are to be substituted for the characters listed in the first argument. Sometimes these sets need to be surrounded by apostrophes (or single-quotes (‘)) in order to have the shell ignore that they mean something special to the shell. It is usually safe (and may be required) to use the single-quotes around each of the sets as you will see in the examples below.

For example, suppose you have a file named city containing several lines of text in mixed case. To translate all lower case characters to upper case, at the command prompt type cat city | tr a-z A-Z and press the Enter key.

Command Usage
$ tr abcdefghijklmnopqrstuvwxyz ABCDEFGHIJKLMNOPQRSTUVWXYZ Convert lower case to upper case
$ tr ‘{}’ ‘()’ < inputfile > outputfile Translate braces into parenthesis
$ echo “This is for testing” | tr [:space:] ‘t’ Translate white-space to tabs
$ echo “This   is   for    testing” | tr -s [:space:] Squeeze repetition of characters using -s
$ echo “the geek stuff” | tr -d ‘t’ Delete specified characters using -d option
$ echo “my username is 432234” | tr -cd [:digit:] Complement the sets using -c option
$ tr -cd [:print:] < file.txt Remove all non-printable character from a file
$ tr -s ‘n’ ‘ ‘ < file.txt Join all the lines in a file into a single line

tee:
tee takes the output from any command, and while sending it to standard output, it also saves it to a file. In other words, it “teesthe output stream from the command: one stream is displayed on the standard output and the other is saved to a file.

For example, to list the contents of a directory on the screen and save the output to a file, at the command prompt type ls -l | tee newfileand press the Enter key.

Typing cat newfile will then display the output of ls –l.

 

wc
wc (word count) counts the number of lines, words, and characters in a file or list of files. Options are given in the table below.

By default all three of these options are active.

For example, to print the number of lines contained in a file, at the command prompt type wc -l filename and press the Enter key.

Option Description
–l display the number of lines.
-c display the number of bytes.
-w display the number of words.

cut
cut is used for manipulating column-based files and is designed to extract specific columns. The default column separator is the tab character. A different delimiter can be given as a command option.

For example, to display the third column delimited by a blank space, at the command prompt type ls -l | cut -d” ” -f3 and press the Enter key.

 

courses.edx.org : grep

Command Usage
grep [pattern] <filename> Search for a pattern in a file and print all matching lines
grep -v [pattern] <filename> Print all lines that do not match the pattern
grep [0-9] <filename> Print the lines that contain the numbers 0 through 9
grep -C 3 [pattern] <filename> Print context of lines (specified number of lines above and below the pattern) for matching the pattern. Here the number of lines is specified as 3.

 

 

courses.edx.org : regex

Metacharacter Description
Specifies the next character as either a special character, a literal, a back reference, or an octal escape.
^ Matches the position at the beginning of the input string.
$ Matches the position at the end of the input string.
* Matches the preceding subexpression zero or more times.
+ Matches the preceding subexpression one or more times.
? Matches the preceding subexpression zero or one time.
{n} Matches exactly n times, where n is a non-negative integer.
{n,} Matches at least n times, n is a non-negative integer.
{n,m} Matches at least n and at most m times, where m and n are non-negative integers and n <= m.
. Matches any single character except “n”.
[xyz] A character set. Matches any one of the enclosed characters.
x|y Matches either x or y.
[^xyz] A negative character set. Matches any character not enclosed.
[a-z] A range of characters. Matches any character in the specified range.
[^a-z] A negative range characters. Matches any character not in the specified range.
b Matches a word boundary, that is, the position between a word and a space.
B Matches a nonword boundary. ‘erB’ matches the ‘er’ in “verb” but not the ‘er’ in “never”.
d Matches a digit character.
D Matches a non-digit character.
f Matches a form-feed character.
n Matches a newline character.
r Matches a carriage return character.
s Matches any whitespace character including space, tab, form-feed, etc.
S Matches any non-whitespace character.
t Matches a tab character.
v Matches a vertical tab character.
w Matches any word character including underscore.
W Matches any non-word character.
un Matches n, where n is a Unicode character expressed as four hexadecimal digits. For example, u00A9 matches the copyright symbol

Origin: http://www.hscripts.com/tutorials/regular-expression/metacharacter-list.php

 

Examples:

the quick brown fox jumped over the lazy dog

Some of the patterns that can be applied to this sentence are as follows:

Command Usage
a.. matches azy
b.|j. matches both br and ju
..$ matches og
l.* matches lazy dog
l.*y matches lazy
the.* matches the whole sentence

courses.edx.org : sort, uniq, paste, join

Syntax Usage
sort <filename> Sort the lines in the specified file
cat file1 file2 | sort Append the two files, then sort the lines and display the output on the terminal
sort -r <filename> Sort the lines in reverse order

To remove duplicate entries from some files, use the following command:

sort file1 file2 | uniq > file3

To count the number of duplicate entries, use the following command:

uniq -c filename

PASTE:
To paste contents from two files one can do:

$ paste file1 file2

The syntax to use a different delimiter is as follows:

$ paste -d, file1 file2

Common delimiters are ‘space’, ‘tab’, ‘|’, ‘comma’, etc.

JOIN:
To combine two files on a common field, at the command prompt type

join file1 file2

and press the Enter key.

SPLIT:
By default split breaks up a file into 1,000-line segments. The original file remains unchanged, and set of new files with the same name plus an added prefix is created. By default, the x prefix is added. To split a file into segments, use the command

$ split  

sed & awk – Linux Tools, courses.edx.org


Command
Usage
sed s/pattern/replace_string/ file Substitute first string occurrence in a line
sed s/pattern/replace_string/g file Substitute all string occurrences in a line
sed 1,3s/pattern/replace_string/g file Substitute all string occurrences in a range of lines
sed -i s/pattern/replace_string/g file Save changes for string substitution in the same file
Command Usage
awk ‘command’ var=value file Specify a command directly at the command line
awk -f scriptfile var=value file Specify a file that contains the script to be executed along with f
Command Usage
awk ‘{ print $0 }’ /etc/passwd Print entire file
awk -F: ‘{ print $1 }’ /etc/passwd Print first field (column) of every line, separated by a space
awk -F: ‘{ print $1 $6 }’ /etc/passwd Print first and sixth field

 

_

Networking in Linux

Networking Tools Description
ethtool Queries network interfaces and can also set various parameters such as the speed.
netstat Displays all active connections and routing tables. Useful for monitoring performance and troubleshooting.
nmap Scans open ports on a network. Important for security analysis
tcpdump Dumps network traffic for analysis.
iptraf Monitors network traffic in text mode.

 

route -n (display routing)

sudo ethtool eth0

netstat -r

sudo nmap -sP <network address> (scan for open ports on network address ie: 192.168.0.0)

emacs commands

Linux Foundation

Working with emacs

Key Usage
emacs myfile Start emacs and edit myfile
Ctrl-x i Insert prompted for file at current position
Ctrl-x s Write to the file keeping current name
Ctrl-x Ctrl-w Write to the file giving a new name when prompted
Ctrl-x Ctrl-s Write to all files currently being worked on and exit
Ctrl-x Ctrl-c Exit after being prompted if there any unwritten modified files

 

Changing Cursor Positions

Key Usage
arrow keys Use the arrow keys for up, down, left and right
Ctrl-n one line down
Ctrl-p one line up
Ctrl-f one character left
Ctrl-b one character right
Ctrl-a move to beginning of line
Ctrl-e move to end of line
E-f move to beginning of next word
E-b move back to beginning of preceding word
E-< move to beginning of file
E-x goto-line n move to line n
E-> move to end of file
Ctrl-v or Page Down move forward one page
E-v or Page Up move backward one page
Ctrl-l refresh and center screen

 

Searching for Text

 

Key Usage
Ctrl-s search forward for prompted for pattern, or for next pattern

 

Ctrl-r search backwards for prompted for pattern, or for next pattern

Working with Text

 

 

Key Usage
Ctrl-o Insert a blank line.
Ctrl-d Delete character at current position.
Ctrl-k Delete the rest of the current line.
Ctrl-_ Undo the previous operation.
Ctrl- space Mark the beginning of the selected region. The end will be at the cursor position.
Ctrl-w Delete the current marked text and write it to the buffer.
Ctrl-y Insert at current cursor location whatever was most recently deleted

 

Vi Basics

Linux Foundation

Commands Used to Start, Exit, Read, and Write Files in vi

Command Usage
vi myfile Start the vi editor and edit the m yfile file
vi -r myfile Start vi and edit m yfile in recovery mode from a system crash
:r file2<ret> Read in file2 and insert at current position
:w<ret> Write to the file
:w myfile<ret> Write out the file to m yfile
:w!    file2<ret> Overwrite file2
:x<ret> or :wq<ret> Exit vi and write out modified file
:q<ret> Quit vi
:q!<ret> Quit vi even though modifications have not been saved

 

Keystrokes Used When Changing Cursor Position in vi

Keys Usage
arrow keys To move up, down, left and right
j or <ret> To move one line down
k To move one line up
h or Backspace To move one character left
l or Space To move one character right
0 To move to beginning of line
$ To move to end of line
w To move to beginning of next word

 

 h  To move to top of file
b To move back to beginning of preceding word
:0 <ret> or (G To move to beginning of file
:n <ret> or nG To move to line n
:$ <ret> or G To move to last line in file
CTRL-F or Page Down To move forward one page
CTRL-B or Page Up To move backward one page
Al To refresh and center screen

 

Searching for Text in vi

Command Usage
/pattern<ret> Search forward for pattern
string<ret> Search backward for pattern

 

Key Usage
n Move to next occurrence of search pattern
N Move to previous occurrence of search pattern

 

Working with Text in vi

Key Usage
a Append text after cursor; stop upon Escape key
A Append text at end of current line; stop upon Escape key
i Insert text before cursor; stop upon Escape key
I Insert text at beginning of current line; stop upon Escape key
o Start a new line below current line, insert text there; stop upon Escape key
0 Start a new line above current line, insert text there; stop upon Escape key
r Replace character at current position
R Replace text starting with current position; stop upon Escape key

 

x Delete character at current position
Nx Delete N characters, starting at current position
dw Delete the word at the current position
D Delete the rest of the current line
dd Delete the current line
Ndd or dNd Delete N lines
u Undo the previous operation
yy Yank (cut) the current line and put it in buffer
Nyy or yNy Yank (cut) N lines and put it in buffer
P Paste at the current position the yanked line or lines from the buffer