Linux menu

Wednesday, September 24, 2014

linux command line reference for common operations


CommandDescription
apropos whatisShow commands pertinent to string. See also threadsafe
man -t ascii | ps2pdf - > ascii.pdfmake a pdf of a manual page
which commandShow full path name of command
time commandSee how long a command takes
time catStart stopwatch. Ctrl-d to stop. See also sw
dir navigation
cd -Go to previous directory
cdGo to $HOME directory
(cd dir && command)Go to dir, execute command and return to current dir
pushd .Put current dir on stack so you can popd back to it
alias l='ls -l --color=auto'quick dir listing
ls -lrtList files by date. See also newest and find_mm_yyyy
ls /usr/bin | pr -T9 -W$COLUMNSPrint in 9 columns to width of terminal
find -name '*.[ch]' | xargs grep -E 'expr'Search 'expr' in this dir and below. See also findrepo
find -type f -print0 | xargs -r0 grep -F 'example'Search all regular files for 'example' in this dir and below
find -maxdepth 1 -type f | xargs grep -F 'example'Search all regular files for 'example' in this dir
find -maxdepth 1 -type d | while read dir; do echo $dir; echo cmd2; doneProcess each item with multiple commands (in while loop)
find -type f ! -perm -444Find files not readable by all (useful for web site)
find -type d ! -perm -111Find dirs not accessible by all (useful for web site)
locate -r 'file[^/]*\.txt'Search cached index for names. This re is like glob *file*.txt
look referenceQuickly search (sorted) dictionary for prefix
grep --color reference /usr/share/dict/wordsHighlight occurances of regular expression in dictionary
archives and compression
gpg -c fileEncrypt file
gpg file.gpgDecrypt file
tar -c dir/ | bzip2 > dir.tar.bz2Make compressed archive of dir/
bzip2 -dc dir.tar.bz2 | tar -xExtract archive (use gzip instead of bzip2 for tar.gz files)
tar -c dir/ | gzip | gpg -c | ssh user@remote 'dd of=dir.tar.gz.gpg'Make encrypted archive of dir/ on remote machine
find dir/ -name '*.txt' | tar -c --files-from=- | bzip2 > dir_txt.tar.bz2Make archive of subset of dir/ and below
find dir/ -name '*.txt' | xargs cp -a --target-directory=dir_txt/ --parentsMake copy of subset of dir/ and below
( tar -c /dir/to/copy ) | ( cd /where/to/ && tar -x -p )Copy (with permissions) copy/ dir to /where/to/ dir
( cd /dir/to/copy && tar -c . ) | ( cd /where/to/ && tar -x -p )Copy (with permissions) contents of copy/ dir to /where/to/
( tar -c /dir/to/copy ) | ssh -C user@remote 'cd /where/to/ && tar -x -p'Copy (with permissions) copy/ dir to remote:/where/to/ dir
dd bs=1M if=/dev/sda | gzip | ssh user@remote 'dd of=sda.gz'Backup harddisk to remote machine
rsync (Network efficient file copier: Use the --dry-run option for testing)
rsync -P rsync://rsync.server.com/path/to/file fileOnly get diffs. Do multiple times for troublesome downloads
rsync --bwlimit=1000 fromfile tofileLocally copy with rate limit. It's like nice for I/O
rsync -az -e ssh --delete ~/public_html/ remote.com:'~/public_html'Mirror web site (using compression and encryption)
rsync -auz -e ssh remote:/dir/ . && rsync -auz -e ssh . remote:/dir/Synchronize current directory with remote one
ssh (Secure SHell)
ssh $USER@$HOST commandRun command on $HOST as $USER (default command=shell)
ssh -f -Y $USER@$HOSTNAME xeyesRun GUI command on $HOSTNAME as $USER
scp -p -r $USER@$HOST: file dir/Copy with permissions to $USER's home directory on $HOST
scp -c arcfour $USER@$LANHOST: bigfileUse faster crypto for local LAN. This might saturate GigE
ssh -g -L 8080:localhost:80 root@$HOSTForward connections to $HOSTNAME:8080 out to $HOST:80
ssh -R 1434:imap:143 root@$HOSTForward connections from $HOST:1434 in to imap:143
ssh-copy-id $USER@$HOSTInstall public key for $USER@$HOST for password-less log in
wget (multi purpose download tool)
(cd dir/ && wget -nd -pHEKk http://www.pixelbeat.org/cmdline.html)Store local browsable version of a page to the current dir
wget -c http://www.example.com/large.fileContinue downloading a partially downloaded file
wget -r -nd -np -l1 -A '*.jpg' http://www.example.com/dir/Download a set of files to the current directory
wget ftp://remote/file[1-9].iso/FTP supports globbing directly
wget -q -O- http://www.pixelbeat.org/timeline.html | grep 'a href' | headProcess output directly
echo 'wget url' | at 01:00Download url at 1AM to current dir
wget --limit-rate=20k urlDo a low priority download (limit to 20KB/s in this case)
wget -nv --spider --force-html -i bookmarks.htmlCheck links in a file
wget --mirror http://www.example.com/Efficiently update a local copy of a site (handy from cron)
networking (Note ifconfig, route, mii-tool, nslookup commands are obsolete)
ethtool eth0Show status of ethernet interface eth0
ethtool --change eth0 autoneg off speed 100 duplex fullManually set ethernet interface speed
iw dev wlan0 linkShow link status of wireless interface wlan0
iw dev wlan0 set bitrates legacy-2.4 1Manually set wireless interface speed
iw dev wlan0 scanList wireless networks in range
ip link showList network interfaces
ip link set dev eth0 name wanRename interface eth0 to wan
ip link set dev eth0 upBring interface eth0 up (or down)
ip addr showList addresses for interfaces
ip addr add 1.2.3.4/24 brd + dev eth0Add (or del) ip and mask (255.255.255.0)
ip route showList routing table
ip route add default via 1.2.3.254Set default gateway to 1.2.3.254
ss -tuplList internet services on a system
ss -tupList active connections to/from system
host pixelbeat.orgLookup DNS ip address for name or vice versa
hostname -iLookup local ip address (equivalent to host `hostname`)
whois pixelbeat.orgLookup whois info for hostname or ip address
windows networking (Note samba is the package that provides all this windows specific networking support)
smbtreeFind windows machines. See also findsmb
nmblookup -A 1.2.3.4Find the windows (netbios) name associated with ip address
smbclient -L windows_boxList shares on windows machine or samba server
mount -t smbfs -o fmask=666,guest //windows_box/share /mnt/shareMount a windows share
echo 'message' | smbclient -M windows_boxSend popup to windows machine (off by default in XP sp2)
text manipulation (Note sed uses stdin and stdout. Newer versions support inplace editing with the -i option)
sed 's/string1/string2/g'Replace string1 with string2
sed 's/\(.*\)1/\12/g'Modify anystring1 to anystring2
sed '/^ *#/d; /^ *$/d'Remove comments and blank lines
sed ':a; /\\$/N; s/\\\n//; ta'Concatenate lines with trailing \
sed 's/[ \t]*$//'Remove trailing spaces from lines
sed 's/\([`"$\]\)/\\\1/g'Escape shell metacharacters active within double quotes
seq 10 | sed "s/^/      /; s/ *\(.\{7,\}\)/\1/"Right align numbers
seq 10 | sed p | paste - -Duplicate a column
sed -n '1000{p;q}'Print 1000th line
sed -n '10,20p;20q'Print lines 10 to 20
sed -n 's/.*<title>\(.*\)<\/title>.*/\1/ip;T;q'Extract title from HTML web page
sed -i 42d ~/.ssh/known_hostsDelete a particular line
sort -t. -k1,1n -k2,2n -k3,3n -k4,4nSort IPV4 ip addresses
echo 'Test' | tr '[:lower:]' '[:upper:]'Case conversion
tr -dc '[:print:]' < /dev/urandomFilter non printable characters
tr -s '[:blank:]' '\t' </proc/diskstats | cut -f4cut fields separated by blanks
history | wc -lCount lines
seq 10 | paste -s -d ' 'Concatenate and separate line items to a single line
set operations (Note you can export LANG=C for speed. Also these assume no duplicate lines within a file)
sort file1 file2 | uniqUnion of unsorted files
sort file1 file2 | uniq -dIntersection of unsorted files
sort file1 file1 file2 | uniq -uDifference of unsorted files
sort file1 file2 | uniq -uSymmetric Difference of unsorted files
join -t'\0' -a1 -a2 file1 file2Union of sorted files
join -t'\0' file1 file2Intersection of sorted files
join -t'\0' -v2 file1 file2Difference of sorted files
join -t'\0' -v1 -v2 file1 file2Symmetric Difference of sorted files
math
echo '(1 + sqrt(5))/2' | bc -lQuick math (Calculate φ). See also bc
seq -f '4/%g' 1 2 99999 | paste -sd-+ | bc -lCalculate π the unix way
echo 'pad=20; min=64; (100*10^6)/((pad+min)*8)' | bcMore complex (int) e.g. This shows max FastE packet rate
echo 'pad=20; min=64; print (100E6)/((pad+min)*8)' | pythonPython handles scientific notation
echo 'pad=20; plot [64:1518] (100*10**6)/((pad+x)*8)' | gnuplot -persistPlot FastE packet rate vs packet size
echo 'obase=16; ibase=10; 64206' | bcBase conversion (decimal to hexadecimal)
echo $((0x2dec))Base conversion (hex to dec) ((shell arithmetic expansion))
units -t '100m/9.58s' 'miles/hour'Unit conversion (metric to imperial)
units -t '500GB' 'GiB'Unit conversion (SI to IEC prefixes)
units -t '1 googol'Definition lookup
seq 100 | (tr '\n' +; echo 0) | bcAdd a column of numbers. See also add and funcpy
calendar
cal -3Display a calendar
cal 9 1752Display a calendar for a particular month year
date -d friWhat date is it this friday. See also day
[ $(date -d '12:00 +1 day' +%d) = '01' ] || exitexit a script unless it's the last day of the month
date --date='25 Dec' +%AWhat day does xmas fall on, this year
date --date='@2147483647'Convert seconds since the epoch (1970-01-01 UTC) to date
TZ='America/Los_Angeles' dateWhat time is it on west coast of US (use tzselect to find TZ)
date --date='TZ="America/Los_Angeles" 09:00 next Fri'What's the local time for 9AM next Friday on west coast US
locales
printf "%'d\n" 1234Print number with thousands grouping appropriate to locale
BLOCK_SIZE=\'1 ls -lUse locale thousands grouping in ls. See also l
echo "I live in `locale territory`"Extract info from locale database
LANG=en_IE.utf8 locale int_prefixLookup locale info for specific country. See also ccodes
locale -kc $(locale | sed -n 's/\(LC_.\{4,\}\)=.*/\1/p') | lessList fields available in locale database
recode (Obsoletes iconv, dos2unix, unix2dos)
recode -l | lessShow available conversions (aliases on each line)
recode windows-1252.. file_to_change.txtWindows "ansi" to local charset (auto does CRLF conversion)
recode utf-8/CRLF.. file_to_change.txtWindows utf8 to local charset
recode iso-8859-15..utf8 file_to_change.txtLatin9 (western europe) to utf8
recode ../b64 < file.txt > file.b64Base64 encode
recode /qp.. < file.qp > file.txtQuoted printable decode
recode ..HTML < file.txt > file.htmlText to HTML
recode -lf windows-1252 | grep euroLookup table of characters
echo -n 0x80 | recode latin-9/x1..dumpShow what a code represents in latin-9 charmap
echo -n 0x20AC | recode ucs-2/x2..latin-9/xShow latin-9 encoding
echo -n 0x20AC | recode ucs-2/x2..utf-8/xShow utf-8 encoding
CDs
gzip < /dev/cdrom > cdrom.iso.gzSave copy of data cdrom
mkisofs -V LABEL -r dir | gzip > cdrom.iso.gzCreate cdrom image from contents of dir
mount -o loop cdrom.iso /mnt/dirMount the cdrom image at /mnt/dir (read only)
cdrecord -v dev=/dev/cdrom blank=fastClear a CDRW
gzip -dc cdrom.iso.gz | cdrecord -v dev=/dev/cdrom -Burn cdrom image (use dev=ATAPI -scanbus to confirm dev)
cdparanoia -BRip audio tracks from CD to wav files in current dir
cdrecord -v dev=/dev/cdrom -audio -pad *.wavMake audio CD from all wavs in current dir (see also cdrdao)
oggenc --tracknum=$track track.cdda.wav -o track.oggMake ogg file from wav file
disk space (See also FSlint)
ls -lSrShow files by size, biggest last
du -s * | sort -k1,1rn | headShow top disk users in current dir. See also dutop
du -hs /home/* | sort -k1,1hSort paths by easy to interpret disk usage
df -hShow free space on mounted filesystems
df -iShow free inodes on mounted filesystems
fdisk -lShow disks partitions sizes and types (run as root)
rpm -q -a --qf '%10{SIZE}\t%{NAME}\n' | sort -k1,1nList all packages by installed size (Bytes) on rpm distros
dpkg-query -W -f='${Installed-Size;10}\t${Package}\n' | sort -k1,1nList all packages by installed size (KBytes) on deb distros
dd bs=1 seek=2TB if=/dev/null of=ext3.testCreate a large test file (taking no space). See also truncate
> filetruncate data of file or create an empty file
monitoring/debugging
tail -f /var/log/messagesMonitor messages in a log file
strace -c ls >/dev/nullSummarise/profile system calls made by command
strace -f -e open ls >/dev/nullList system calls made by command
strace -f -e trace=write -e write=1,2 ls >/dev/nullMonitor what's written to stdout and stderr
ltrace -f -e getenv ls >/dev/nullList library calls made by command
lsof -p $$List paths that process id has open
lsof ~List processes that have specified path open
tcpdump not port 22Show network traffic except ssh. See also tcpdump_not_me
ps -e -o pid,args --forestList processes in a hierarchy
ps -e -o pcpu,cpu,nice,state,cputime,args --sort pcpu | sed '/^ 0.0 /d'List processes by % cpu usage
ps -e -orss=,args= | sort -b -k1,1n | pr -TW$COLUMNSList processes by mem (KB) usage. See also ps_mem.py
ps -C firefox-bin -L -o pid,tid,pcpu,stateList all threads for a particular process
ps -p 1,$$ -o etime=List elapsed wall time for particular process IDs
last rebootShow system reboot history
free -mShow amount of (remaining) RAM (-m displays in MB)
watch -n.1 'cat /proc/interrupts'Watch changeable data continuously
udevadm monitorMonitor udev events to help configure rules
system information (see also sysinfo) ('#' means root access is required)
uname -aShow kernel version and system architecture
head -n1 /etc/issueShow name and version of distribution
cat /proc/partitionsShow all partitions registered on the system
grep MemTotal /proc/meminfoShow RAM total seen by the system
grep "model name" /proc/cpuinfoShow CPU(s) info
lspci -tvShow PCI info
lsusb -tvShow USB info
mount | column -tList mounted filesystems on the system (and align output)
grep -F capacity: /proc/acpi/battery/BAT0/infoShow state of cells in laptop battery
#dmidecode -q | lessDisplay SMBIOS/DMI information
#smartctl -A /dev/sda | grep Power_On_HoursHow long has this disk (system) been powered on in total
#hdparm -i /dev/sdaShow info about disk sda
#hdparm -tT /dev/sdaDo a read speed test on disk sda
#badblocks -s /dev/sdaTest for unreadable blocks on disk sda
interactive (see also linux keyboard shortcuts)
readlineLine editor used by bash, python, bc, gnuplot, ...
screenVirtual terminals with detach capability, ...
mcPowerful file manager that can browse rpm, tar, ftp, ssh, ...
gnuplotInteractive/scriptable graphing
linksWeb browser
xdg-open .open a file or url with the registered desktop application



CommandDescription
grep . /proc/sys/net/ipv4/*List the contents of flag files
set | grep $USERSearch current environment
tr '\0' '\n' < /proc/$$/environDisplay the startup environment for any process
echo $PATH | tr : '\n'Display the $PATH one per line
kill -0 $$ && echo process exists and can accept signalsCheck for the existence of a process (pid)
find /etc -readable | xargs less -K -p'*ntp' -j $((${LINES:-25}/2))Search paths and data with full context. Use n to iterate
namei -l ~/.sshOutput attributes for all directories leading to a file name
Low impact admin
#apt-get install "package" -o Acquire::http::Dl-Limit=42 \
-o Acquire::Queue-mode=access
Rate limit apt-get to 42KB/s
echo 'wget url' | at 01:00Download url at 1AM to current dir
#apache2ctl configtest && apache2ctl gracefulRestart apache if config is OK
nice openssl speed sha1Run a low priority command (openssl benchmark)
chrt -i 0 openssl speed sha1Run a low priority command (more effective than nice)
renice 19 -p $$; ionice -c3 -p $$Make shell (script) low priority. Use for non interactive tasks
Interactive monitoring
watch -t -n1 uptimeClock with system load
htop -d 5Better top (scrollable, tree view, lsof/strace integration, ...)
iotopWhat's doing I/O
#watch -d -n30 "nice ps_mem.py | tail -n $((${LINES:-12}-2))"What's using RAM
#iftopWhat's using the network. See also iptraf
#mtr www.pixelbeat.orgping and traceroute combined
Useful utilities
pv < /dev/zero > /dev/nullProgress Viewer for data copying from files and pipes
wkhtml2pdf http://.../linux_commands.html linux_commands.pdfMake a pdf of a web page
timeout 1 sleep infrun a command with bounded time. See also timeout
Networking
python -m SimpleHTTPServerServe current directory tree at http://$HOSTNAME:8000/
openssl s_client -connect www.google.com:443 </dev/null 2>&0 |
openssl x509 -dates -noout
Display the date range for a site's certs
curl -I www.pixelbeat.orgDisplay the server headers for a web site
#lsof -i tcp:80What's using port 80
#httpd -SDisplay a list of apache virtual hosts
vim scp://user@remote//path/to/fileEdit remote file using local vim. Good for high latency links
curl -s http://www.pixelbeat.org/pixelbeat.asc | gpg --importImport a gpg key from the web
tc qdisc add dev lo root handle 1:0 netem delay 20msecAdd 20ms latency to loopback device (for testing)
tc qdisc del dev lo rootRemove latency added above
Notification
echo "DISPLAY=$DISPLAY xmessage cooker" | at "NOW +30min"Popup reminder
notify-send "subject" "message"Display a gnome popup notification
echo "mail -s 'go home' P@draigBrady.com < /dev/null" | at 17:30Email reminder
uuencode file name | mail -s subject P@draigBrady.comSend a file via email
ansi2html.sh | mail -a "Content-Type: text/html" P@draigBrady.comSend/Generate HTML email
Better default settings (useful in your .bashrc)
#tail -s.1 -f /var/log/messagesDisplay file additions more responsively
seq 100 | tail -n $((${LINES:-12}-2))Display as many lines as possible without scrolling
#tcpdump -s0Capture full network packets
Useful functions/aliases (useful in your .bashrc)
md () { mkdir -p "$1" && cd "$1"; }Change to a new directory
strerror() { python -c "import os; print os.strerror($1)"; }Display the meaning of an errno
plot() { { echo 'plot "-"' "$@"; cat; } | gnuplot -persist; }Plot stdin. (e.g: • seq 1000 | sed 's/.*/s(&)/' | bc -l | plot)
hili() { e="$1"; shift; grep --col=always -Eih "$e|$" "$@"; }highlight occurences of expr. (e.g: • env | hili $USER)
alias hd='od -Ax -tx1z -v'Hexdump. (usage e.g.: • hd /proc/self/cmdline | less)
alias realpath='readlink -f'Canonicalize path. (usage e.g.: • realpath ~/../$USER)
ord() { printf "0x%x\n" "'$1"; }shell version of the ord() function
chr() { printf $(printf '\\%03o\\n' "$1"); }shell version of the chr() function
Multimedia
DISPLAY=:0.0 import -window root orig.pngTake a (remote) screenshot
convert -filter catrom -resize '600x>' orig.png 600px_wide.pngShrink to width, computer gen images or screenshots
mplayer -ao pcm -vo null -vc dummy /tmp/Flash*Extract audio from flash video to audiodump.wav
ffmpeg -i filename.aviDisplay info about multimedia file
ffmpeg -f x11grab -s xga -r 25 -i :0 -sameq demo.mpgCapture video of an X display
DVD
for i in $(seq 9); do ffmpeg -i $i.avi -target pal-dvd $i.mpg; doneConvert video to the correct encoding and aspect for DVD
dvdauthor -odvd -t -v "pal,4:3,720xfull" *.mpg;dvdauthor -odvd -TBuild DVD file system. Use 16:9 for widescreen input
growisofs -dvd-compat -Z /dev/dvd -dvd-video dvdBurn DVD file system to disc
Unicode
python -c "import unicodedata as u; print u.name(unichr(0x2028))"Lookup a unicode character
uconv -f utf8 -t utf8 -x nfcNormalize combining characters
printf '\300\200' | iconv -futf8 -tutf8 >/dev/nullValidate UTF-8
printf 'ŨTF8\n' | LANG=C grep --color=always '[^ -~]\+'Highlight non printable ASCII chars in UTF-8
fc-match -s "sans:lang=zh"List font match order for language and style
Development
gcc -march=native -E -v -</dev/null 2>&1|sed -n 's/.*-mar/-mar/p'Show autodetected gcc tuning params. See also gcccpuopt
for i in $(seq 4); do { [ $i = 1 ] && wget http://url.ie/6lko -qO-||
./a.out; } | tee /dev/tty | gcc -xc - 2>/dev/null; done
Compile and execute C code from stdin
cpp -dM /dev/nullShow all predefined macros
echo "#include <features.h>" | cpp -dN | grep "#define __USE_"Show all glibc feature macros
gdb -tuiDebug showing source code context in separate windows
udev
udevadm info -a -p $(udevadm info -q path -n /dev/input/mouse0)List udev attributes of a device, for matching rules etc.
udevadm test /sys/class/input/mouse0See how udev rules are applied for a device
#udevadm control --reload-rulesReload udev rules after modification
Extended Attributes (Note you may need to (re)mount with "acl" or "user_xattr" options)
getfacl .Show ACLs for file
setfacl -m u:nobody:r .Allow a specific user to read file
setfacl -x u:nobody .Delete a specific user's rights to file
setfacl --default -m group:users:rw- dir/Set umask for a for a specific dir
getcap fileShow capabilities for a program
setcap cap_net_raw+ep your_gtk_progAllow gtk program raw access to network
stat -c%C .Show SELinux context for file
chcon ... fileSet SELinux context for file (see also restorecon)
getfattr -m- -d .Show all extended attributes (includes selinux,acls,...)
setfattr -n "user.foo" -v "bar" .Set arbitrary user attributes
BASH specific
echo 123 | tee >(tr 1 a) | tr 1 bSplit data to 2 commands (using process substitution)
meld local_file <(ssh host cat remote_file)Compare a local and remote file (using process substitution)
Multicore
taskset -c 0 nprocRestrict a command to certain processors
find -type f -print0 | xargs -r0 -P$(nproc) -n10 md5sumProcess files in parallel over available processors
sort -m <(sort data1) <(sort data2) >data.sortedSort separate data files over 2 processors

No comments: