Oracle Virtual Box VM:-How to create network setting and interface in Oracle Virtual Box to connect internet as well as connect from host

This note will describe the process to connect Oracle Virtual box VM directly from your host using ssh as well as from VM to internet connection.

I will use 2 types of VM interface.

a)NAT network for internet connection from VM

b)Host only adapter for ssh connection from host to the VM.

Step 1:-First I will go to File->Preferences and check/create NAT network with below settings in Oracle Virtual box :-

 

Step 2:-Create host-only network as well

DHCP server should not be enable here

Step 3:-Please create first network interface (NAT)

Step 4:-Please create host-only adapter interface

step 5:-Please check persistent rules under /etc/udev/rules.d and modify mac address if possible.The MC address should be same which was visible in step 3 and step 4

Step 6:-Please create ifcfg-eth1 and ifcfg-eth2 using below settings.ETH1 is for NAT and ETH2 is for host-only adapter.Please check HWADDR value as MAC address in previous steps.

Step 7:-Now reboot your VM or restart individual Network interface (eth1 and eth2) using ifup or ifdown command.

Step 8:-Please check ifconfig -a

step 9:-Please enable root login

enable  PermitRootLogin to yes in  /etc/ssh/sshd_config.

[root@rac1 yum.repos.d]# service sshd restart

Redirecting to /bin/systemctl restart sshd.service

Step 10:-Now you can check whether you connect to host using IP 169.254.156.100.You can also check ping to google.com after login to VM

Add additional swap space in RHEL 7

This document describes process to add additional swap space in RHEL 7.

1.First partition your newly added disk

[root@LinuxAcademy dev]# gdisk /dev/xvdf
GPT fdisk (gdisk) version 0.8.10

Partition table scan:
MBR: not present
BSD: not present
APM: not present
GPT: not present

Creating new GPT entries.

Command (? for help): n
Partition number (1-128, default 1):
First sector (34-41943006, default = 2048) or {+-}size{KMGTP}:
Last sector (2048-41943006, default = 41943006) or {+-}size{KMGTP}:
Current type is ‘Linux filesystem’
Hex code or GUID (L to show codes, Enter = 8300): 8e00
Changed type of partition to ‘Linux LVM’

Command (? for help): w

Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING
PARTITIONS!!

Do you want to proceed? (Y/N): y
OK; writing new GUID partition table (GPT) to /dev/xvdf.
The operation has completed successfully.

2.Please create physical volume group and logical volume group.

[root@LinuxAcademy dev]# pvdisplay
[root@LinuxAcademy dev]# vgcreate battlestar /dev/xvdf1
Physical volume “/dev/xvdf1” successfully created
Volume group “battlestar” successfully created
[root@LinuxAcademy dev]# lvcreate -n swap -L 2G battlestar
Logical volume “swap” created.

4.Please create swap file system now.

[root@LinuxAcademy dev]# mkswap /dev/battlestar/swap
mkswap: /dev/battlestar/swap: warning: don’t erase bootbits sectors
on whole disk. Use -f to force.
Setting up swapspace version 1, size = 2097148 KiB
no label, UUID=ffb912cb-a280-41ca-9558-d2cc70447469

swapon /dev/battlestar/swap

5.Please check free memory now

free -m

6.Please add below entry in fstab for auto mount after restart.

vi /etc/fstab
/dev/battlestar/swap swap swap 0 0

 

7.Check swap space now.

swapon -a
swapon -s

Extend LVM after adding new disk to the server Linux

This document describes the procedure to extend your existing mount point after adding the disk in RHEL 7 using LVM.

1.Please create partition on newly added disk

[root@linuxacademy1 ~]# gdisk /dev/xvdf
GPT fdisk (gdisk) version 0.8.6

Partition table scan:
MBR: not present
BSD: not present
APM: not present
GPT: not present

Creating new GPT entries.

Command (? for help): n
Partition number (1-128, default 1):
First sector (34-41943006, default = 2048) or {+-}size{KMGTP}:
Last sector (2048-41943006, default = 41943006) or {+-}size{KMGTP}:
Current type is ‘Linux filesystem’
Hex code or GUID (L to show codes, Enter = 8300): 008a
Exact type match not found for type code 008A; assigning type code for
‘Linux filesystem’
Changed type of partition to ‘Linux filesystem’

Command (? for help): w

Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING
PARTITIONS!!

Do you want to proceed? (Y/N): y
OK; writing new GUID partition table (GPT) to /dev/xvdf.
The operation has completed successfully.
[root@linuxacademy1 ~]# gdisk /dev/xvdg
GPT fdisk (gdisk) version 0.8.6

Partition table scan:
MBR: not present
BSD: not present
APM: not present
GPT: not present

Creating new GPT entries.

Command (? for help): n
Partition number (1-128, default 1): 1
First sector (34-41943006, default = 2048) or {+-}size{KMGTP}:
Last sector (2048-41943006, default = 41943006) or {+-}size{KMGTP}:
Current type is ‘Linux filesystem’
Hex code or GUID (L to show codes, Enter = 8300): 008a
Exact type match not found for type code 008A; assigning type code for
‘Linux filesystem’
Changed type of partition to ‘Linux filesystem’

Command (? for help): w

Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING
PARTITIONS!!

Do you want to proceed? (Y/N): y
OK; writing new GUID partition table (GPT) to /dev/xvdg.
The operation has completed successfully.

2.Now please create physical volume group

[root@linuxacademy1 ~]# pvcreate /dev/xvdf1 /dev/xvdg1
Physical volume “/dev/xvdf1” successfully created.
Physical volume “/dev/xvdg1” successfully created.
[root@linuxacademy1 ~]# pvdisplay

“/dev/xvdf1” is a new physical volume of “<20.00 GiB”
— NEW Physical volume —
PV Name /dev/xvdf1
VG Name
PV Size <20.00 GiB
Allocatable NO
PE Size 0
Total PE 0
Free PE 0
Allocated PE 0
PV UUID a52OwU-svuG-mR1O-wJ2E-pZ2i-HCFa-287P7I

“/dev/xvdg1” is a new physical volume of “<20.00 GiB”
— NEW Physical volume —
PV Name /dev/xvdg1
VG Name
PV Size <20.00 GiB
Allocatable NO
PE Size 0
Total PE 0
Free PE 0
Allocated PE 0
PV UUID cVCWqw-BSdi-FC00-e5JW-2dq9-QQH2-13q7eZ

[root@linuxacademy1 ~]# vgcreate battlestar /dev/xvdf1 /dev/xvdg1
Volume group “battlestar” successfully created
[root@linuxacademy1 ~]# vgdisplay
— Volume group —
VG Name battlestar
System ID
Format lvm2
Metadata Areas 2
Metadata Sequence No 1
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 0
Open LV 0
Max PV 0
Cur PV 2
Act PV 2
VG Size 39.99 GiB
PE Size 4.00 MiB
Total PE 10238
Alloc PE / Size 0 / 0
Free PE / Size 10238 / 39.99 GiB
VG UUID xtAtA6-4WZY-KZK8-wskY-QNcJ-YYKt-3XlSh1

3.Please create logical volume now

[root@linuxacademy1 ~]# lvcreate -n galactica -L 20G battlestar
Logical volume “galactica” created.
[root@linuxacademy1 ~]# mkfs -t xfs /dev/battlestar/galactica
meta-data=/dev/battlestar/galactica isize=512 agcount=4, agsize=1310720 blks
= sectsz=512 attr=2, projid32bit=1
= crc=1 finobt=0, sparse=0
data = bsize=4096 blocks=5242880, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0 ftype=1
log =internal log bsize=4096 blocks=2560, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0

4.Mount now.

[root@linuxacademy1 ~]# mkdir /mnt/mydir
[root@linuxacademy1 ~]# mount /dev/battlestar/galactica /mnt/mydir

5.Please add the extra disk to expand mount point

You can use following command to scan newly added disk online(No need to restart server)

echo “- – -” > /sys/class/scsi_host/host0/scan
 echo “- – -” > /sys/class/scsi_host/host1/scan

[root@linuxacademy1 ~]# gdisk /dev/xvdj
GPT fdisk (gdisk) version 0.8.6

Partition table scan:
MBR: not present
BSD: not present
APM: not present
GPT: not present

Creating new GPT entries.

Command (? for help): n
Partition number (1-128, default 1):
First sector (34-41943006, default = 2048) or {+-}size{KMGTP}:
Last sector (2048-41943006, default = 41943006) or {+-}size{KMGTP}:
Current type is ‘Linux filesystem’
Hex code or GUID (L to show codes, Enter = 8300): 008a
Exact type match not found for type code 008A; assigning type code for
‘Linux filesystem’
Changed type of partition to ‘Linux filesystem’

Command (? for help): w

Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING
PARTITIONS!!

Do you want to proceed? (Y/N): y
OK; writing new GUID partition table (GPT) to /dev/xvdj.
The operation has completed successfully.

[root@linuxacademy1 ~]# partprobe

6.Extending volume group now

[root@linuxacademy1 ~]# pvcreate /dev/xvdj1
Physical volume “/dev/xvdj1” successfully created.
[root@linuxacademy1 ~]# vgextend battlestar /dev/xvdj1
Volume group “battlestar” successfully extended
[root@linuxacademy1 ~]# vgdisplay
— Volume group —
VG Name battlestar
System ID
Format lvm2
Metadata Areas 3
Metadata Sequence No 3
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 1
Open LV 1
Max PV 0
Cur PV 3
Act PV 3
VG Size <59.99 GiB
PE Size 4.00 MiB
Total PE 15357
Alloc PE / Size 5120 / 20.00 GiB
Free PE / Size 10237 / <39.99 GiB
VG UUID xtAtA6-4WZY-KZK8-wskY-QNcJ-YYKt-3XlSh1

[root@linuxacademy1 ~]# lvextend -L 59G /dev/battlestar/galactica
Size of logical volume battlestar/galactica changed from 20.00 GiB (5120 extents) to 59.00 GiB (15104 extents).
Logical volume battlestar/galactica successfully resized.
[root@linuxacademy1 ~]# lvdisplay
— Logical volume —
LV Path /dev/battlestar/galactica
LV Name galactica
VG Name battlestar
LV UUID 0C8PRN-HsxV-mnne-9Awb-Net4-zNAN-kpJrM9
LV Write Access read/write
LV Creation host, time linuxacademy1, 2017-10-16 05:06:56 -0400
LV Status available
# open 1
LV Size 59.00 GiB
Current LE 15104
Segments 3
Allocation inherit
Read ahead sectors auto
– currently set to 8192
Block device 253:0

7.Please check disk space now and this has not reflected new size

[root@linuxacademy1 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/xvda2 10G 1.9G 8.2G 19% /
devtmpfs 477M 0 477M 0% /dev
tmpfs 496M 0 496M 0% /dev/shm
tmpfs 496M 13M 483M 3% /run
tmpfs 496M 0 496M 0% /sys/fs/cgroup
tmpfs 100M 0 100M 0% /run/user/1001
tmpfs 100M 0 100M 0% /run/user/0
/dev/mapper/battlestar-galactica 20G 33M 20G 1% /mnt/mydir

 

8.Please use following command to resize mount point

For XFS file system

[root@linuxacademy1 ~]# xfs_growfs /mnt/mydir
meta-data=/dev/mapper/battlestar-galactica isize=512 agcount=4, agsize=1310720 blks
= sectsz=512 attr=2, projid32bit=1
= crc=1 finobt=0 spinodes=0
data = bsize=4096 blocks=5242880, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0 ftype=1
log =internal bsize=4096 blocks=2560, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
data blocks changed from 5242880 to 15466496

For Ext4 file system

[root@linuxacademy1 ~]# resize2fs /dev/mapper/battlestar-galactica
resize2fs 1.42.9 (28-Dec-2013)
Filesystem at /dev/mapper/oemcc-oemcc is mounted on /oemcc; on-line resizing required
old_desc_blocks = 4, new_desc_blocks = 7
The filesystem on /dev/mapper/oemcc-oemcc is now 14680064 blocks long.

[root@linuxacademy1 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/xvda2 10G 1.9G 8.2G 19% /
devtmpfs 477M 0 477M 0% /dev
tmpfs 496M 0 496M 0% /dev/shm
tmpfs 496M 13M 483M 3% /run
tmpfs 496M 0 496M 0% /sys/fs/cgroup
tmpfs 100M 0 100M 0% /run/user/1001
tmpfs 100M 0 100M 0% /run/user/0
/dev/mapper/battlestar-galactica 59G 33M 59G 1% /mnt/mydir

Create new mount point using LVM in RHEL 7

First install gdisk utility (you may use old fdisk utility if LUN/drive size is less than 2 TB)

-bash-4.1$ yum install gdisk
Loaded plugins: fastestmirror, security
You need to be root to perform this command.
-bash-4.1$ su
Password:
[root@LinuxAcademy linuxacademy]# yum install gdisk
Loaded plugins: fastestmirror, security
Setting up Install Process
Determining fastest mirrors
* base: mirror.cs.pitt.edu
* extras: mirrors.advancedhosters.com
* updates: mirror.atlanticmetro.net
base | 3.7 kB 00:00
base/primary_db | 3.7 MB 00:00
extras | 3.3 kB 00:00
extras/primary_db | 21 kB 00:00
updates | 3.4 kB 00:00
updates/primary_db | 4.6 MB 00:00
Resolving Dependencies
–> Running transaction check
—> Package gdisk.i686 0:0.8.10-1.el6 will be installed
–> Finished Dependency Resolution

Dependencies Resolved

================================================================================
Package Arch Version Repository Size
================================================================================
Installing:
gdisk i686 0.8.10-1.el6 base 172 k

Transaction Summary
================================================================================
Install 1 Package(s)

Total download size: 172 k
Installed size: 645 k
Is this ok [y/N]: y
Downloading Packages:
gdisk-0.8.10-1.el6.i686.rpm | 172 kB 00:00
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : gdisk-0.8.10-1.el6.i686 1/1
Verifying : gdisk-0.8.10-1.el6.i686 1/1

Installed:
gdisk.i686 0:0.8.10-1.el6

Complete!

Please partition the new disk using gdisk

[root@LinuxAcademy linuxacademy]# gdisk /dev/xvdf

GPT fdisk (gdisk) version 0.8.10

Partition table scan:
MBR: not present
BSD: not present
APM: not present
GPT: not present

Creating new GPT entries.

Command (? for help): n
Partition number (1-128, default 1):
First sector (34-41943006, default = 2048) or {+-}size{KMGTP}:
Last sector (2048-41943006, default = 41943006) or {+-}size{KMGTP}:
Current type is ‘Linux filesystem’
Hex code or GUID (L to show codes, Enter = 8300): L
0700 Microsoft basic data 0c01 Microsoft reserved 2700 Windows RE
3000 ONIE boot 3001 ONIE config 4100 PowerPC PReP boot
4200 Windows LDM data 4201 Windows LDM metadata 7501 IBM GPFS
7f00 ChromeOS kernel 7f01 ChromeOS root 7f02 ChromeOS reserved
8200 Linux swap 8300 Linux filesystem 8301 Linux reserved
8302 Linux /home 8400 Intel Rapid Start 8e00 Linux LVM
a500 FreeBSD disklabel a501 FreeBSD boot a502 FreeBSD swap
a503 FreeBSD UFS a504 FreeBSD ZFS a505 FreeBSD Vinum/RAID
a580 Midnight BSD data a581 Midnight BSD boot a582 Midnight BSD swap
a583 Midnight BSD UFS a584 Midnight BSD ZFS a585 Midnight BSD Vinum
a800 Apple UFS a901 NetBSD swap a902 NetBSD FFS
a903 NetBSD LFS a904 NetBSD concatenated a905 NetBSD encrypted
a906 NetBSD RAID ab00 Apple boot af00 Apple HFS/HFS+
af01 Apple RAID af02 Apple RAID offline af03 Apple label
af04 AppleTV recovery af05 Apple Core Storage be00 Solaris boot
bf00 Solaris root bf01 Solaris /usr & Mac Z bf02 Solaris swap
bf03 Solaris backup bf04 Solaris /var bf05 Solaris /home
bf06 Solaris alternate se bf07 Solaris Reserved 1 bf08 Solaris Reserved 2
bf09 Solaris Reserved 3 bf0a Solaris Reserved 4 bf0b Solaris Reserved 5
c001 HP-UX data c002 HP-UX service ea00 Freedesktop $BOOT
eb00 Haiku BFS ed00 Sony system partitio ed01 Lenovo system partit
Press the <Enter> key to see more codes: 8e00
ef00 EFI System ef01 MBR partition scheme ef02 BIOS boot partition
fb00 VMWare VMFS fb01 VMWare reserved fc00 VMWare kcore crash p
fd00 Linux RAID
Hex code or GUID (L to show codes, Enter = 8300): 8e00
Changed type of partition to ‘Linux LVM’

Command (? for help): w

Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING
PARTITIONS!!

Do you want to proceed? (Y/N): y
OK; writing new GUID partition table (GPT) to /dev/xvdf.
The operation has completed successfully.

Please repeat the steps for /dev/xvdj

Please create physical volume group and logical volume group

[root@LinuxAcademy linuxacademy]# pvcreate /dev/xvdf1 /dev/xvdj1
Physical volume “/dev/xvdf1” successfully created
Physical volume “/dev/xvdj1” successfully created
[root@LinuxAcademy linuxacademy]# pvdisplay
“/dev/xvdf1” is a new physical volume of “20.00 GiB”
— NEW Physical volume —
PV Name /dev/xvdf1
VG Name
PV Size 20.00 GiB
Allocatable NO
PE Size 0
Total PE 0
Free PE 0
Allocated PE 0
PV UUID 5E12Tu-E1tn-8U7d-OK2g-pw3N-B8tC-9K1eeU

“/dev/xvdj1” is a new physical volume of “20.00 GiB”
— NEW Physical volume —
PV Name /dev/xvdj1
VG Name
PV Size 20.00 GiB
Allocatable NO
PE Size 0
Total PE 0
Free PE 0
Allocated PE 0
PV UUID xIc4BG-SXXf-RTYs-4xX8-Z4Ye-wT7n-z7vDJh

[root@LinuxAcademy linuxacademy]# vgcreate log_vg /dev/xvdf1 /dev/xvdj1
Volume group “log_vg” successfully created
[root@LinuxAcademy linuxacademy]# vgdisplay
— Volume group —
VG Name log_vg
System ID
Format lvm2
Metadata Areas 2
Metadata Sequence No 1
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 0
Open LV 0
Max PV 0
Cur PV 2
Act PV 2
VG Size 39.99 GiB
PE Size 4.00 MiB
Total PE 10238
Alloc PE / Size 0 / 0
Free PE / Size 10238 / 39.99 GiB
VG UUID ojwPwH-OhHT-vKCi-EIqc-K5g4-1Aom-7272Qn

[root@LinuxAcademy linuxacademy]# lvcreate -n log_lv -L 10GB log_vg
Logical volume “log_lv” created.
[root@LinuxAcademy linuxacademy]# lvdisplay
— Logical volume —
LV Path /dev/log_vg/log_lv
LV Name log_lv
VG Name log_vg
LV UUID ES1qhb-fIUg-Jvho-lviK-temg-ppD0-b9AEp4
LV Write Access read/write
LV Creation host, time LinuxAcademy, 2017-10-16 04:13:39 -0400
LV Status available
# open 0
LV Size 10.00 GiB
Current LE 2560
Segments 1
Allocation inherit
Read ahead sectors auto
– currently set to 256
Block device 253:0

Create file system

[root@LinuxAcademy linuxacademy]# mkfs -t ext4 /dev/log
mke2fs 1.41.12 (17-May-2010)
/dev/log is not a block special device.
Proceed anyway? (y,n) n
[root@LinuxAcademy linuxacademy]# mkfs -t ext4 /dev/log_vg/log_lv
mke2fs 1.41.12 (17-May-2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
655360 inodes, 2621440 blocks
131072 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=2684354560
80 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632

Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 34 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.

Mount the filesystem

[root@LinuxAcademy linuxacademy]# mkdir /mnt/log_files

[root@LinuxAcademy linuxacademy]# mount /dev/log_vg/log_lv /mnt/log_files
[root@LinuxAcademy linuxacademy]# vgdisplay
— Volume group —
VG Name log_vg
System ID
Format lvm2
Metadata Areas 2
Metadata Sequence No 2
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 1
Open LV 1
Max PV 0
Cur PV 2
Act PV 2
VG Size 39.99 GiB
PE Size 4.00 MiB
Total PE 10238
Alloc PE / Size 2560 / 10.00 GiB
Free PE / Size 7678 / 29.99 GiB
VG UUID ojwPwH-OhHT-vKCi-EIqc-K5g4-1Aom-7272Qn

Please make following entry in /etc/fstab to auto mount

/dev/log_vg/log_lv /mnt/log_files ext4 defaults 0 0

How to remove Volume group

[root@LinuxAcademy linuxacademy]# vgremove log_vg
Do you really want to remove volume group “log_vg” containing 1 logical volumes? [y/n]: y
Logical volume log_vg/log_lv contains a filesystem in use.
[root@LinuxAcademy linuxacademy]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/xvde1 5.8G 1.5G 4.1G 26% /
tmpfs 299M 0 299M 0% /dev/shm
/dev/mapper/log_vg-log_lv
9.9G 151M 9.2G 2% /mnt/log_files
[root@LinuxAcademy linuxacademy]# umount /mnt/log_files
[root@LinuxAcademy linuxacademy]# vgremove log_vg
Do you really want to remove volume group “log_vg” containing 1 logical volumes? [y/n]: y
Do you really want to remove active logical volume log_lv? [y/n]: y
Logical volume “log_lv” successfully removed
Volume group “log_vg” successfully removed
[root@LinuxAcademy linuxacademy]# lvdisplay
[root@LinuxAcademy linuxacademy]# vgdisplay

[root@LinuxAcademy linuxacademy]# pvremove /dev/xvdf1 /dev/xvdj1
Labels on physical volume “/dev/xvdf1” successfully wiped
Labels on physical volume “/dev/xvdj1” successfully wiped

Unix basic command and shell script for DBA and SYSADMIN

DIRECTORY STRUCTURE IN UNIX

 

Directory Contains
bin/ Required Boot-time binaries
boot/ Boot configuration files for the OS loader and kernel image
dev/ Device files
etc/ System configuration files and scripts
home/ User/Sub branch directories
lib/ Main OS shared libraries and kernel modules
Lost+found/ Storage directory for “recovered” files
mnt/ Temporary point to connect devices to
proc/ Pseudo directory structure containing information about the kernel, currently running processes and resource allocation
root/ Linux (non-standard) home directory for the root user. Alternate location being the / directory itself
sbin/ System administration binaries and tools
tmp/ Location of temporary files
usr/ Difficult to define – it contains almost everything else including local binaries, libraries, applications and packages (including X Windows)
var/ Variable data, usually machine specific. Includes spool directories for mail and news

 

GENERAL UNIX COMMANDS

 

TO SEE WHERE COMMANDS ARE LOCATED

echo $PATH

/bin:/usr/bin:/usr/local/bin:$HOME/bin

which ls

SEEING HELP COMMAND

date –help

info date

man date

whatis date

Extended Documents:=/usr/share/doc

TO CHANGE CONSOLE

Ctrl+alt+(F1-F7)

 

CHANGING DIRECTORY

cd /home/deb/work

cd ..[One level up]

LISTING DIRECTORIES AND FILES

ls /

ls –al [All hidden files]

ls –l [long listing]

ls –R[recurses through sub-directories]

ls -d[Listing directories only]

ls -ltr|grep ^d[Listing directories only]

List of command options

Option Action

 

-a list hidden files

-d list the name of the current directory

-F show directories with a trailing ’/’ executable

files with a trailing ’*’

-g show group ownership of file in long listing

-i print the inode number of each file

-l long listing giving details about files and direc-tories

-R list all subdirectories encountered

-t sort by time modified instead of name

COPYING FILES

cp source destination

cp –P source destination [Preserves permissions]

cp –R source destination [Recursively copies the files]

cp ~helper/tmp/for_john tmp/comments[To copy a file from another user’s directory using a relative

pathname.]

cp /usr/lib/more.help .[To copy a file from another directory to the current working

directory, preserving the file name:]

MOVING AND RENAMING

mv file1 file2 [ Renaming]

mv source destination [ Move]

REMOVING FILES AND DIRECTORIES

rm chap[1-2][wildcard is used]

rm -i chap1[Removing interactively]

rm -rf[Deleting forefully file or directories]

rmdir deb[removing empty directory]

TO CHECK FILE TYPES

file deb.sh

Displaying the first few lines of a file

head /usr/local/doc/vi.txt[To display the first 10 lines of a file]

head -5 *.xdh[To display the first 5 lines of several files]

 

Displaying the last part of a file

 

tail fred[To display the last 10 lines of a text file]

tail -50c help.txt[To display the last 50 characters in a file]

tail -c -512 foo[Copies last 512 bytes from foo]

tail -c +512 foo[Copies everything after skippiong 512 bytes]

 

HANDLING PRINTERS

 

lp deb.lis[Printing file]

lp -d laser deb.lis[If there are more printers]

lp -n3 deb.lis[Printing 3 copies]

lpstat[To view print queue]

cancel laser[To cancel printing of current printer]

cancel pr1-320[Cancel job with request id pr1-320]

lpq[To see print que job number]

lprm 31[Removes job number 31]

lpr -p laser deb.lis[Prints on printer laser]

lpstat -c[To see pending jobs]

 

COUNTING LINES

 

wc deb.txt

wc -c[No. of characters]

wc -l[No. of lines]

 

COMPRESSING FILES

 

compress deb.txt

uncompress deb.txt

gzip deb

gunzip deb.z

 

USING TAR COMMAND

 

# display contents of a file
tar tvf myfile.tar

# display contents of a diskette (Solaris)
volcheck
tar tvf /vol/dev/rdiskette0/unnamed_floppy

# copy files to a tar file
tar cvf myfile.tar *.sql

# format floppy, and copy files to it (Solaris)
fdformat -U -b floppy99
tar cvf /vol/dev/rdiskette0/floppy99 *.sql

# append files to a tar file
tar rvfn myfile.tar *.txt

# extract files from a tar filem to current dir
tar xvf myfile.tar

 

SORTING

 

sort /etc/passwd
sort in order based on the whole line

sort -r /etc/passwd
reverse the order of the sort

sort +2n -t: /etc/passwd
sort on third field, where field delimiter is : (skip the first two fields)

sort +2n -t: -n /etc/passwd
same sort but treat the field as numbers not ASCII characters

sort –o telnos telnos(sorts telnos and replaces its content with the sorted output)

 

DUPLICATE ROW BY UNIQUE COMMAND

 

uniq names
remove duplicate lines from names and display them on the screen

uniq names uniq.names
remove duplicates lines from names and put them into uniq.names

uniq -d names
display all duplicate lines

 

Using symbolic links

 

ln -s source linkname

ln -s reports/reportA publications/my_report

 

TO DETERMINE DISK USAGE

 

# display disk free, in KB
df -kt

# display disk usage, in KB for directory
du -k mydir

# display directory disk usage, sort by largest first
du -ahx .|sort -rh|head -20

 

 

FIND Command

 Find files by text search

$ find ~ -name “*.txt” -print

To find all ‘*.txt’ files in the current directory you are in and any sub-directories:

$ find . -name “*.txt” -print

To find all filenames with at least one upper case character, in your current directory

and any sub-directories:

$ find . -name “[A-Z]*” -print

To find files in /etc directory that begin with host, and any other characters after

that:

$ find /etc -name “host*” -print

To find all files in your $HOME directory:

$ find ~ -name “*” -print or find . -print

 

To bring the system to zero per cent response time, start at the root level and drill

through all directories listing ‘all’ files. If you want to remain on good terms with

your sys admin be very careful with this option!

$ find / -name “*” -print

Find all files that begin with two lower case characters, followed by two numbers,

followed by .txt. The following find command could return a file called ax37.txt.

$ find . -name “[a-z][a-z][0–9][0–9].txt” -print

 

Find files by perm mode

 

To find files with a permission of 755 which is read, write and executable by

owner and read and execute by everyone else,

$ find . -perm 755 -print

 

To find files that are read, write and executable for everyone (watch out for these) use

this find command. Put a dash in front of the octal number. The perm stands for

permissions and 007 is based on the notation you use for the chmod (absolute) mode.

$ find . -perm -007 -print

 

Ignoring directories

 

To display the files in the apps directory when you do not want find to search

in /apps/bin:

$ find /apps -name “/apps/bin” -prune -o -print

 

Find files by user and nouser

 

To find files by owner, you supply the actual login name of that user. To find files

owned by user dave in your $HOME directory:

$ find ~ -user dave -print

 

To find files owned by uucp in /etc:

$ find /etc -user uucp -print

 

To find files of a user that has been deleted, use the -nouser option. This

finds files that have no valid userid in the /etc/passwd file. You do not have to

supply a user name with this option; find will go off and do all the work. To find

all files not owned by a current user in the /home directory tree:

$ find /home -nouser -print

 

 

Find files by group and nogroup

 

Like the user and nouser option, find has the same options for groups. To find all

files with the group membership of ‘accts’ that are in the /apps directory:

$ find /apps -group accts -print

To find all files that have no valid group membership just supply the -nogroup

option. Here find starts at the root of the file system.

$ find / -nogroup -print

 

Find files by modification times

 

To find all files that have been modified in the last five days:

$ find / -mtime -5 -print

To find files in /var/adm directory that have not been modified in the last three

days:

$ find /var/adm -mtime +3 -print

 

Finding files that are newer or older in days or minutes

 

$ find . -newer age.awk ! -newer belts.awk -exec ls -l {} \;

  

Delete old trace files from DIAG location of Oracle Database

find /u01/app/oracle -type f -name “*.trc” -mtime +1 -exec rm {} \;

find /u01/app/oracle -type f -name “*.trm” -mtime +0.5 -exec rm {}  \;

find /u01/app/oracle -type f -name “*.aud” -mtime +0.5 -exec rm {} \;

  VI EDITOR

List of insert commands

 

To do this … Command

Insert text after the cursor                             a

Insert text before the cursor                           i

Append text at the end of  the current line     A

Insert text at the start of the current line        I

Open a new line above the  current line        o

Open a new line below the current line         O

 

Recovering files after a system crash

 

vi -r help.xdh

 

Viewing a file

view filename

 

Moving the cursor along a line

Press the ESC key to enter command mode before using these com-mands.

To move to Do this …

 

next character                        l

previous character                 h

next word                              w

next n words                         wn

previous word                       b

previous n words                  bn

end of current word              e

start of current line                0 (zero)

end of current line                  $

Moving the cursor between lines

Press the ESC key to enter command mode before using these com-mands.

To move to Do this …

 

next line down                        j

(same column)

start of next line down           +

previous line                           k

(same column)

start of previous line

Deleting characters

Press the ESC key to enter command mode before using these com-mands.

To delete Do this

 

current character                   x

previous character                dh

Deleting words and lines

Press the ESC key to enter command mode before using these com-mands.

To delete Do this

current word                         dw

previous word                       db

entire line                               dd

to end of line                         d$

to start of line                        d0 (zero)

next n lines                           ndd

 

To get back a word or line that you have just deleted enter the command:

p

Searching for text

Press the ESC key to enter command mode before using these com-mands.

To search Do this …

forward for a pattern /pattern

backward for a pattern ?pattern

repeat previous search n

repeat previous search in N

reverse direction

Examples of replacing text

To replace one word in the current line with another:

:s/that/which

 

To replace every instance of one word in the current line with

another:

:s/destroy/delete/g

This replaces every occurrence of the word “destroy” with the

word “delete” in the current line.

 

To replace every occurrence of a word throughout the file, with

another:

:g/insert/s//add/g

This replaces every occurrence of “insert” with “add”.

To replace ^M from text file

:%s/^v^M/ /g

 

UNIX Scheduler and Process

CRONTAB

 1st column Minutes 1–59

2nd column Hour 1–23 (0 is midnight)

3rd column Day_of_month 1–31

4th column Month 1–12

5th column Weekday 0–6 (0 is Sunday)

6th column Command to run

 

Here are some examples of crontab entries:

 

30 21 * * * /apps/bin/cleanup.sh

 

runs a script called cleanup.sh in /apps/bin at 9:30 every night.

Cron and crontab 33

45 4 1,10,22 * * /apps/bin/backup.sh

runs a script called backup.sh in /apps/bin at 4:45 a.m., on the 1st, 10th and 22nd

of each month.

10 1 * * 6,0 /bin/find -name “core” -exec rm {} \;

runs a find command at 1:10 a.m. only on Saturday and Sunday.

0,30 18-23 * * * /apps/bin/dbcheck.sh

runs a script called dbcheck.sh in /apps/bin at every 30 minutes past the hour,

between 18:00 and 23:00.

0 23 * * 6 /apps/bin/qtrend.sh

runs a script called qtrend.sh in /apps/bin at 11:00 p.m. every Saturday.

 

The general format of the crontab is:

Crontab [-u user] -e -l -r

where:

-u is the user login name

-e edits the crontab file

-l lists the crontab file

-r deletes the crontab file

 

Submitting a command to the background

 

$ find /etc -name “srm.conf” -print >find.dt 2>&1 &

 

Checking the process using ps

 

$ ps x|grep 28305

28305 p1 S 0:00 sh /root/ps1

28350 p1 S 0:00 grep 28305

 

Killing a background job

 

kill -signal [process_number]

 

$ kill 28305

$ kill -9 28305

 

Submitting a job using nohup

 

nohup command >myout.file 2>&1

 

$ nohup ps1 &

 

 CONTROLLING ACCESS TO FILE SYSTEM

 

1.To display the permissions on a single file:

ls -l file1

-rw-r–r– 2 unixjohn 3287 Apr 8 12:10 file1

2.This displays the following information about the file file1.

 

-rw-r–r– – access permissions

2 – number of links to this file

unixjohn – owner

3287 – size in bytes

Apr 8 12:10 – date and time last modified

3.There are three types of permissions:

r – read the file or list files in the directory

w – write to the file or directory

x – execute the file or search the directory

4.Each of these permissions can be set for any one of three types of

user:

u – the user who owns the file (usually you)

g – members of the group the owner belongs to

o – all other users

The access permissions for all three types of user can be given as a

string of nine characters:

user      group      others

r w x         r w x            r w x

5.Changing permission

  1. To give yourself permission to execute a file that you own:

chmod u+x file1

This gives you execute permission for the file file1.

  1. To give members of your group permission to read a file:

chmod g+r file2

This gives the group permission to read the file file2.

  1. To give read permission to everyone for a particular type of file:

chmod a+r *.pub

This gives everyone permission to read all files with the extension

.pub.

  1. To give the group write and execute permission:

chmod g+wx $HOME/SCCS

This gives all members of the group permission to place files in

5.Setting access permissions numerically

There is a shorthand way of setting permissions by using octal num-bers.

Read permission is given the value 4, write permission the value

2 and execute permission 1.

r w x

4 2 1

These values are added together for any one user category:

1 = execute only

2 = write only

3 = write and execute (1+2)

4 = read only

5 = read and execute (4+1)

6 = read and write (4+2)

7 = read and write and execute (4+2+1)

So any access permission can be expressed as three digits. For exam-ple:

user group others

chmod 640 file1 rw- r– —

chmod 754 file1 rwx r-x r–

chmod 664 file1 rw- rw- r–

6.Creating user and groups

useradd -u 21- -g dba -c “THE RDBMS” -d /home/oracle -s /bin/ksh -m oracle

groupadd -g 241 dba

Informations are stored in /etc/passwd annd /etc/shadow

7.Setting limits on file size

ulimit 234567

8.Modifying and removing users

usermod -s /bin/bash oracle

userdel oracle

9.Changing password

passwd

# change group to staff for this file
chgrp staff myfile

# change owner to jsmith for this file
chown jsmith myfile

 

Shell input and output

 

ECHO

 

$ echo “What is your name :\c”

$ read name

 

You can put variables within echo statements as well as escape codes. In this

example, the terminal bell is rung, your $HOME directory is displayed, and the

command tty is evaluated by the shell.

$ echo “\007your home directory is $HOME, you are connected on ‘tty‘”

your home directory is /home/dave, you are connected on /dev/ttyp1

 

If it’s LINUX then . . .

You have to put a ‘-n’ after the echo to suppress the new line:

$ echo -n “What is your name :”

You have to put a ‘-e’ after echo for the escape code to work:

$ echo -e “\007your home directory is $HOME, you are connected on

‘tty‘”

your home directory is /home/dave, you are connected on /dev/ttyp1

 

READ

 

read variable1 variable2 . . .

 

$ read name

Hello I am superman

$ echo $name

Hello I am superman

 

$ read name surname

John Doe

$ echo $name $surname

John Doe

 

$ pg var_test

#!/bin/sh

# var_test

echo “First Name :\c”

read 51

read name

echo “Middle Name :\c”

read middle

echo “Last name :\c”

read surname

 

If it’s LINUX then . . .

Remember to use the ‘-n’ echo option.

$ pg var_test

#!/bin/sh

# var_test

echo “First Name :\c”

read name

echo “Middle Name :\c”

read middle

echo “Last name :\c”

read surname

 

PIPES

 

$ who | awk ‘{print $1″\t”$2}’

matthew pts/0

louise pts/1

 

$ df -k | awk ‘{print $1}’| grep -v “Filesystem”

 

$ df -k | awk ‘{print $1}’| grep -v “Filesystem”|sed s’/\/dev\///g’

hda5

hda8

hda6

hdb5

hdb1

hda7

hda1

 

TEE

 

The tee command acts the way it is pronounced. It takes the input and sends

one copy to the standard output and another copy to a file. If you want to see

your output and save it to a file at the same time, then this is the command for

you.

The general format is:

tee -a files

 

$ who | tee who.out

louise pts/1 May 20 12:58 (193.132.90.9)

matthew pts/0 May 20 10:18 (193.132.90.1)

cat who.out

louise pts/1 May 20 12:58 (193.132.90.9)

matthew pts/0 May 20 10:18 (193.132.90.1)

 

$ find etc usr/local home -depth -print | cpio -ovC65536 -O \

 

$ echo ” myscript is now running, check out any errors…in

myscript.log” | tee -a myscript.log

$ myscript | tee -a myscript.log

 

$ echo “stand-by disk cleanup starting in 1 minute” | tee/dev/console

$ sort myfile | tee -a accounts.log

$ myscript | tee -a accounts.log

 

 

Standard input, output and errors

 

Redirecting standard output

 

$ cat passwd | awk -F: ‘{print $1}’| sort 1>sort.out

$ ls -l | grep ^d >>files.out

$ ls account* >> files.out

 

Redirecting standard input

sort < names.txt

sort <names.txt > names.out

 

$ cat >> myfile <<MAYDAY

> Hello there I am using a $TERM terminal

> and my user name is $LOGNAME

> bye…

> MAYDAY

$ pg myfile

Hello there I am using a vt100 terminal

and my user name is dave

 

Redirecting standard error

 

To redirect standard errors you specify the file descriptor ‘2’. Let’s first look at an

example, since they always explain better. Here grep searches for a pattern called

‘trident’ on a file called missiles.

$ grep “trident” missiles

grep: missiles: No such file or directory

 

 

grep reports no such file, and sends its errors to the terminal which is the default.

Let’s now send all errors to a file, in fact to the system dustbin, ‘/dev/null’.

$ grep “trident” missiles 2>/dev/null

 

Combining standard output and error

 

$ cat account_qtr.doc account_end.doc 1> accounts.out 2> accounts.err

 

Merging standard output and standard error

 

When merging output and errors, the shell evaluates the command from left to

right, which is really all you need to know when putting mergers together. Here’s an

example.

$ cleanup >cleanup.out 2>&1

In the above example the script cleanup directs all output (>) to a file called

cleanup.out, and all errors (2), are directed to (>) the same place as the output

(&1), which is cleanup.out.

$ grep “standard” * > grep.out 2>&1

In the above example all output from the grep command is put into the output file

grep.out. When you use here documents, you will probably need to capture all the

output to a file in case errors are encountered. To do this you need to use 2>&1 as

part of the command. Here’s how to do it:

$ cat>> filetest 2>&1 <<MAYDAY

> This is my home $HOME directory

> MAYDAY

 

 

Command execution order

Using &&

 

Here’s the general format of &&:

command1 && command2

 

The operation of these commands is fairly straightforward. The command on

the left of the && (command1) must be returned true (successfully executed) if the

command on the right (command2) is to be executed; or to put it another way, ‘If

this command works’ && ‘then execute this command’.

 

$ cp justice.doc justice.bak && echo “if you are seeing this then cp was

OK”

if you are seeing this then cp was OK

 

$ mv /apps/bin /apps/dev/bin && rm -r /apps/bin

In the above example, the directory /apps/bin is to be moved to /apps/dev/bin;

if this fails then the deletion of the directory /apps/bin will not happen.

 

In the following example, a file is to be sorted with the output going to a file

called quarter.sort; if this is successful, the file will be printed.

$ sort quarter_end.txt > quarter.sorted && lp quarter.sorted

 

Using ||

 

command1 || command2

The operation of the || is slightly different. If the command on the left of the ||

(command1) fails then execute the command on the right of the || (command2); or

to put it another way, ‘If this command fails’ || ‘then execute this command’.

Here’s another simple example that illustrates the use of the ||:

$ cp wopper.txt oops.txt || echo “if you are seeing this cp failed”

cp: wopper.txt: No such file or directory

if you are seeing this cp failed

The copy has failed so now the command on the right-hand side of the || is

executed.

A more practical example is this. I want to extract the first and fifth fields

from an accounts file and stick the output into a temp file. If the extraction does not

work, I want to be mailed.

$ awk ‘{print$1,$5}’ acc.qtr >qtr.tmp || echo “Sorry the payroll

extraction didn’t work” | mail dave

 

You don’t always have to use system commands; here I run the script comet on a file

called month_end.txt. If the script bombs out, the shell should terminate (exit).

$ comet month_end.txt || exit.

 

 

Using GREP

 

Searching more than one file

If I wanted to search for the string “sort” in all my .doc files in my directory I

would do this:

$ grep “sort” * .doc

Or I could search through all the files that have the word “sort it”.

$ grep “sort it” *

 

Line matches

 

$ grep -c “48” data.f

$ 4

 

Line numbers

 

$ grep -n “48” data.f

1:48 Dec 3BC1997 LPSX 68.00 LVX2A 138

2:483 Sept 5AP1996 USP 65.00 LVX2C 189

5:484 Nov 7PL1996 CAD 49.00 PLV2C 234

6:483 May 5PA1998 USP 37.00 KVM9D 644

 

Do not match

 

$ grep -v “48” data.f

47 Oct 3ZL1998 LPSX 43.00 KVM9D 512

219 Dec 2CC1999 CAD 23.00 PLV2C 68

216 Sept 3ZL1998 USP 86.00 KVM9E 234

 

Getting an exact match

 

$ grep ’48\>’ data.f

48 Dec 3BC1997 LPSX 68.00 LVX2A 138

 

Being case-sensitive

 

$ grep -i “sept” data.f

483 Sept 5AP1996 USP 65.00 LVX2C 189

216 sept 3ZL1998 USP 86.00 KVM9E 234

 

Pattern ranges

 

$ grep ’48[34]’ data.f

483 Sept 5AP1996 USP 65.00 LVX2C 189

484 nov 7PL1996 CAD 49.00 PLV2C 234

483 may 5PA1998 USP 37.00 KVM9D 644

 

Don’t match at the beginning of a line

 

$ grep ‘^[^48]’ data.f

219 dec 2CC1999 CAD 23.00 PLV2C 68

216 sept 3ZL1998 USP 86.00 KVM9E 234

 

Trapping upper and lower cases

 

$ grep ‘[Ss]ept’ data.f

483 Sept 5AP1996 USP 65.00 LVX2C 189

216 sept 3ZL1998 USP 86.00 KVM9E 234

 

Matching any characters

 

$ grep ‘K…D’ data.f

47 Oct 3ZL1998 LPSX 43.00 KVM9D 512

483 may 5PA1998 USP 37.00 KVM9D 644

 

$ grep ‘[A-Z][A-Z]..C’ data.f

483 Sept 5AP1996 USP 65.00 LVX2C 189

219 dec 2CC1999 CAD 23.00 PLV2C 68

484 nov 7PL1996 CAD 49.00 PLV2C 234

 

Date searching

 

$ grep ‘5..199[6,8]’ data.f

483 Sept 5AP1996 USP 65.00 LVX2C 189

483 may 5PA1998 USP 37.00 KVM9D 644

 

$ grep ‘[0-9]\ {3\}[8]’ data.f

47 Oct 3ZL1998 LPSX 43.00 KVM9D 512

483 may 5PA1998 USP 37.00 KVM9D 644

216 sept 3ZL1998 USP 86.00 KVM9E 234

 

Combining ranges

 

Staying with the use of the [ ] brackets to extract information, suppose we want to

get city codes where the first character could be any number, the second character

between 0 and 5 and the third between 0 and 6. If we use this pattern we’ll get the

following output:

$ grep ‘[0-9][0-5][0-6]’ data.f

48 Dec 3BC1997 LPSX 68.00 LVX2A 138

483 Sept 5AP1996 USP 65.00 LVX2C 189

47 Oct 3ZL1998 LPSX 43.00 KVM9D 512

219 dec 2CC1999 CAD 23.00 PLV2C 68

484 nov 7PL1996 CAD 49.00 PLV2C 234

483 may 5PA1998 USP 37.00 KVM9D 644

216 sept 3ZL1998 USP 86.00 KVM9E 234

 

Well, we certainly got a lot of information back. What we want is included but we

also got other records we didn’t want. However, looking at our pattern, all records

that have been returned are the correct ones according to our rules. We need to

specify that the pattern must start at the beginning of each line. We can use the ^ for

that.

$ grep ‘^[0-9][0-5][0-6]’ data.f

216 sept 3ZL1998 USP 86.00 KVM9E 234

 

Occurrences in a pattern

 

If we want to extract any row that has a number 4 repeated at least twice, we could

use this:

$ grep ‘4\{2,\}’ data.f

483 may 5PA1998 USP 37.00 KVM9D 644

 

$ grep ‘9\{3,\}’ data.f

219 dec 2CC1999 CAD 23.00 PLV2C 68

 

If you want to search for only so many occurrences then take the comma out.

This will search for only two occurrences of the number 9.

$ grep ‘9\{2\}’ data.f

88 The grep family

 

There may be a need to match between, say, two and six occurrences of a

number or maybe a letter. This will match between two and six occurrences of the

number 8 that ends with 3:

 

$ grep ‘6\{2,6}3’ myfile

83 – no match

888883 – match

8884 – no match

88883 – match

 

Blank lines

$ grep ‘^$’ myfile

 

Matching special characters

 

If you wish to search for characters that have special meanings, like one of the

following, $ . ‘ ” * [ ] ^ ( ) | \ + ?, then you must place a \ in front of them. Suppose

you want to search for all lines that contain a period (.), you would do this:

$ grep ‘\.’ myfile

 

Searching for ip addresses

Part of my job is looking after our DNS servers, which means maintaining a lot of

ip addresses that cover different networks. Our address ip file can contain over 200

addresses. Sometimes I want to look at just, say, the ‘nnn.nnn’ network addresses,

and forget about the rest that have only two digits in the second part, i.e. nnn.nn..

To extract all these nnn.nnn. addresses, use [0-9]\{3\}\.[0-0\[3\}\. This expression is

saying any number repeated three times followed by a period, any number repeated

three times followed by a period.

$ grep ‘[0-9]\{3\}\.[0-0\[3\}\.’ Ipfile

 

Pattern matching with wildcards

Let’s take a look at the use of wildcards in using grep. Suppose we have a file like

this:

$ pg testfile

looks

likes

looker

long

Here is what is displayed when using the following grep pattern:

$ grep ‘l.*s’ testfile

looks

likes

$ grep ‘l.*k.’ testfile

looks

likes

$ grep ‘ooo*’ testfile

looks

If you want to find a word only at the end of a line, try this:

$ grep ‘device$’ *

That will search all files for lines that have the word ‘device’ at the end of each line.

Class names 91

 

All this is saying is that you have typed a filename that does not exist. If we

use the -s switch in grep we can silence these error messages.

$ grep -s “louise” /etc/password

 

Using grep on a string

grep is not only reserved for files; you can also use grep on strings. All you need to

do is echo the string then pipe it through to grep.

$ STR=”Mary Joe Peter Pauline”

$ echo $STR | grep “Mary”

Mary Joe Peter Pauline

 

 

egrep

 

egrep stands for expression or extended grep depending on who you listen

Egrep accepts the full range of regular expressions. One of the nice features of egrep is that you can store your strings in a file and pass them into egrep. We do this with the -f switch. If we create a file called grep strings and then type 484 and 47 into it:

$ pg grepstrings

484

47

$ egrep -f grepstrings data.f

this would match all records with 484 or 47 in them. The -f switch really becomes

useful if you want to match a lot of patterns, and typing them in on the command

line becomes awkward.

If we want to search for store codes 32L or 2CC we can use the bar sign (|),

which means one or the other or both (all), separated by the bar sign.

$ egrep ‘(3ZL|2CC)’ data.f

47 Oct 3ZL1998 LPSX 43.00 KVM9D 512

219 dec 2CC1999 CAD 23.00 PLV2C 68

216 sept 3ZL1998 USP 86.00 KVM9E 234

You can use as many bars as you want. If we wanted to see if users louise,

matty or pauline were logged into the system we could use the who command and

pipe the output through to egrep.

$ who | egrep (louise|matty|pauline)

louise pty8

matty tty02

pauline pty2

You can also exclude certain strings using the caret sign (^). If I wanted to see who

was on the system, but I did not want to know if users matty and pauline were on I

could do this:

$ who | egrep -v ‘^(matty|pauline)’

If you want to search a directory listing for files that were called shutdown,

shutdowns, reboot or reboots, this is easily accomplished with egrep.

$ egrep ‘(shutdown | reboot) (s)?’ *

 

USING AWK

 

$ pg grade.txt

M.Tansley 05/99 48311 Green 8 40 44

J.Lulu 06/99 48317 green 9 24 26

P.Bunny 02/99 48 Yellow 12 35 28

J.Troll 07/99 4842 Brown-3 12 26 26

L.Tansley 05/99 4712 Brown-2 12 30 28

 

 

Printing all records

$ awk ‘{print $0}’ grade.txt

 

Printing individual records

 

$ awk ‘{print $1,$4}’ grade.txt

 

Printing report headers

 

$ awk ‘BEGIN {print “Name Belt\n——————————–“}

{print $1″\t”$4}’ grade.txt

Name Belt

—————————————

M.Tansley Green

J.Lulu green

P.Bunny Yellow

J.Troll Brown-3

L.Tansley Brown-3

 

Printing report trailers

$ awk ‘BEGIN {print “Name\n——-“} {print $1} END {“end-of-report”}’

grade.txt

 

Matching

 

$ awk {if($4~/Brown/) print $0}’ grade.txt

J.Troll 07/99 4842 Brown-3 12 26 26

L.Tansley 05/99 4712 Brown-2 12 30 28

 

$ awk ‘$0 ~ /Brown/’ grade.txt

J.Troll 07/99 4842 Brown-3 12 26 26

L.Tansley 05/99 4712 Brown-2 12 30 28

 

Exact match

 

$ awk ‘{if($3~/48/) print $0}’ grade.txt

M.Tansley 05/99 48311 Green 8 40 44

J.Lulu 06/99 48317 green 9 24 26

P.Bunny 02/99 48 Yellow 12 35 28

J.Troll 07/99 4842 Brown-3 12 26 26

 

Not matched

 

$ awk ‘$0 !~ /Brown/’ grade.txt

M.Tansley 05/99 48311 Green 8 40 44

J.Lulu 06/99 48317 green 9 24 26

P.Bunny 02/99 48 Yellow 12 35 28

We could have targeted just the belt grade field ‘field-4’ and done the test this way:

$ awk ‘{if($4!~/Brown/) print $0}’ grade.txt

M.Tansley 05/99 48311 Green 8 40 44

J.Lulu 06/99 48317 green 9 24 26

P.Bunny 02/99 48 Yellow 12 35 28

 

Less than

 

$ awk ‘{if ($6 < $7) print $0 “$1 Try better at the next comp”}’

grade.txt

M.Tansley Try better at the next comp

J.Lulu Try better at the next comp

 

Any characters

$ awk ‘$1 ~/^ . . . a/’ grade.txt

M.Tansley 05/99 48311 Green 8 40 44

L.Tansley 05/99 4712 Brown-2 12 30 28

 

Match either

 

$ awk ‘$0~/(Yellow|Brown)/’ grade.txt

P.Bunny 02/99 48 Yellow 12 35 28

J.Troll 07/99 4842 Brown-3 12 26 26

L.Tansley 05/99 4712 Brown-2 12 30 28

 

AND

$ awk ‘{if ($1==”P.Bunny” && $4==”Yellow”)print $0}’ grade.txt

P.Bunny 02/99 48 Yellow 12 35 28

 

OR

$ awk ‘{if ($4==”Yellow” || $4~/Brown/) print $0}’ grade.txt

P.Bunny 02/99 48 Yellow 12 35 28

J.Troll 07/99 4842 Brown-3 12 26 26

L.Tansley 05/99 4712 Brown-2 12 30 28

 

awk built-in variables

 

ARGC The number of command-line arguments

ARGV The array of command-line arguments

ENVIRON Holds the current system environment variables in the array

FILENAME The name of the current file awk is scanning

FNR The record number in the current file

FS Sets the input field separator; same as the command-line -F option

NF Number of fields in the current record

NR The number of records read so far

OFS Output field separator

ORS Output record separator

RS Controls the record separator

 

NF, NR and FILENAME

 

$ awk ‘{print NF,NR,$0}END{print FILENAME}’ grade.txt

7 1 M.Tansley 05/99 48311 Green 8 40 44

7 2 J.Lulu 06/99 48317 green 9 24 26

7 3 P.Bunny 02/99 48 Yellow 12 35 28

7 4 J.Troll 07/99 4842 Brown-3 12 26 26

7 5 L.Tansley 05/99 4712 Brown-2 12 30 28

grade.txt

 

$ awk ‘{if (NR >0 && $4~/Brown/)print $0}’ grade.txt

J.Troll 07/99 4842 Brown-3 12 26 26

L.Tansley 05/99 4712 Brown-2 12 30 28

 

Assigning input fields to field variable names

 

$ awk ‘{name=$1;belts=$4; if(belts ~/Yellow/)print name” is belt

“belts}’ grade.txt

P.Bunny is belt Yellow

 

Comparing fields with values

 

$ awk ‘{if($6 < 27)print$0}’ grade.txt

J.Lulu 06/99 48317 green 9 24 26

J.Troll 07/99 4842 Brown-3 12 26 26

 

 

$ awk ‘BEGIN {BASELINE=”27 “}{if($6 < BASELINE)print$0}’ grade.txt

J.Lulu 06/99 48317 green 9 24 26

J.Troll 07/99 4842 Brown-3 12 26 26

 

SED COMMAND

 

[root@debasis cls]# cat deb

debasis 1

debraj  1

debajyoti 1

debasis 1

 

[root@debasis cls]# sed ‘s/debasis/deb/g’ deb

deb 1

debraj  1

debajyoti 1

deb 1

 

[root@debasis cls]# sed ‘s/[^ ]*/M&/’ deb

Mdebasis 1

Mdebraj  1

Mdebajyoti 1

Mdebasis 1

 

[root@debasis cls]# sed ‘s/[^ ]*/&M/’ deb

debasisM 1

debrajM  1

debajyotiM 1

debasisM 1

 

[root@debasis cls]# sed ‘s/[^ ]*/&M/’ deb

debasisM 1

debrajM  1

debajyotiM 1

debasisM 1

 

[root@debasis cls]# sed ‘/debasis/ s/1/2/’ deb

debasis 2

debraj  1

debajyoti 1

debasis 2

 

[root@debasis cls]# sed ‘1 s/1/2/’ deb

debasis 2

debraj  1

debajyoti 1

debasis 1

 

[root@debasis cls]# sed ‘1,2 s/1/2/’ deb

debasis 2

debraj  2

debajyoti 1

debasis 1

 

[root@debasis cls]# sed -e ‘s/debasis/deb/’ -e ‘s/debraj/deba/’ deb

deb 1

deba  1

debajyoti 1

deb 1

 

[cls@debasis cls]$ sed ‘

> /debraj/ s/1/2/

> /debajyoti/ s/1/3/’ deb

debasis 1

debraj  2

debajyoti 3

debasis 1

 

[cls@debasis cls]$ sed ‘/debraj/ d’ deb

debasis 1

debajyoti 1

debasis 1

 

[cls@debasis cls]$ sed ‘1,3 d’ deb

 

[cls@debasis cls]$ sed ‘/^deb/ d’ deb

 

[cls@debasis cls]$ sed ‘/1$/ d’ deb

 

[cls@debasis cls]$ sed ‘/debasis/ s/1/2/;/debraj/ d’ deb

debasis 2

debajyoti 1

debasis 2

 

[cls@debasis cls]$ sed ‘$a\

> stop!!!\

> I am lost’ deb

debasis 1

debraj  1

debajyoti 1

debasis 1

stop!!!

I am lost

 

[cls@debasis cls]$ sed ‘3a\

stop!!!\

I am lost’ deb

debasis 1

debraj  1

debajyoti 1

stop!!!

I am lost

debasis 1

 

[cls@debasis cls]$ sed ‘3i\

stop!!!\

I am lost’ deb

debasis 1

debraj  1

stop!!!

I am lost

debajyoti 1

debasis 1

 

[cls@debasis cls]$ sed ‘/debasis/ c\

> no use’ deb

no use

debraj  1

debajyoti 1

no use

 

 

 ENVIRONMENT AND SHELL

 

Local variables

$variable_name=value or ${variable_name = value}

 

Displaying a variable

 

$ GREAT_PICTURE=”die hard”

$ echo ${GREAT_PICTURE}

die hard

$ DOLLAR=99

$ echo ${DOLLAR}

99

$ LAST_FILE=ZLPSO.txt

$ echo ${LAST_FILE}

ZLPSO.txt

 

Clearing a variable

 

unset variable_name

 

$ PC=enterprise

$ echo ${PC}

enterprise

$ unset PC

$ echo ${PC}

 

Displaying all local shell variables

 

$ set

PWD=/root

SHELL=/bin/sh

SHLVL=1

TERM=vt100

UID=7

USER=dave

dollar=99

great_picture=die hard

last_file=ZLPSO.txt

 

Using variables to hold arguments for system commands

 

$ SOURCE=”/etc/passwd”

$ DEST=”/tmp/passwd.bak”

$ cp ${SOURCE} ${DEST}

 

Making a variable read-only

 

$ TAPE_DEV=”/dev/rmt/0n”

$ echo ${TAPE_DEV}

/dev/rmt/0n

$ readonly TAPE_DEV

$ TAPE_DEV=”/dev/rmt/1n”

sh: TAPE_DEV: read-only variable

 SHELL SCRIPTING

 

File status tests

-d This is a directory

-f This is a regular file

-L This is a symbolic link

-r This file is readable

-s This file has a size greater than zero, not empty

-w This file is writeable

-u This file has the suid bit set

-x This is executable

 

We will use both test methods to test if the file scores.txt is writeable. We will use

the last status command to test it. Remember a zero status is OK, anything else is

an error.

$ ls -l scores.txt

-rw-r–r– 1 dave admin 0 May 15 11:29 scores.txt

$ [ -w scores.txt ]

$ echo $?

0

$ test -w scores.txt

$ echo $?

0

 

cutting columns(-c)

 

 cut -c 1-20 deb [show columns from c1-c10]

cut -c1 /etc/passwd
get the first character from every line

cut -c1,5,10-20 /etc/passwd
get the first, fifth character and every character between 10 and 20

cut -d: -f2 /etc/passwd
get the second field

cut -d: -f3- /etc/passwd
get all fields from the third on

cut –f1,3- telneos > deb

cut –d’ ‘ –f2- file (delimiter as space)

cut –f1,3 telnos address>telnos.all(selecting particular columns from two files and pasting in one)

colrm 8 12 twinkle (deleting 8 and 12 column)

 

pasting columns

 

paste adc efg>>abcefg

paste –d: abc efg>>abcefg

paste telos – > telnos.new

 

using cut and paste to reorganize a file

 

cut –f1,3 telnos > temp

cut –f4- telnos>temp2

cut –f2 telos|paste temp-temp2>telnos.new

 

 

Using logical operators with tests

 

-a Logical AND, true, if both sides of the operator are true

-o Logical OR, true, if either sides of the operator can be true

! Logical NOT, true, if the condition is false

 

-rw-r–r– 1 root root 0 May 15 11:29 scores.txt

-rwxr-xr– 1 root root 0 May 15 11:49 results.txt

The following example tests whether both files are readable.

$ [ -w results.txt -a -w scores.txt ]

$ echo $?

0

 

Testing strings

 

= The two strings are equal

!= The two strings are not equal

-z This string is null

-n This string is not null

 

To test if the environment variable EDITOR is empty:

$ [ -z $EDITOR ]

$ echo $?

1

No, it isn’t. Is it set to vi?

$ [ $EDITOR = “vi” ]

$ echo $?

0

 

Testing numbers

 

-eq The two numbers are equal

-ne The two numbers are not equal

-gt The first number is greater than the second number

-lt The first number is less than the second number

-le The first number is less than or equal to the second number

-gt The first number is greater than or equal to the second number

 

$ NUMBER=130

$ [ “$NUMBER” -eq “130” ]

$ echo $?

0

 

Using expr

 

$ expr 900 + 600

1500

$ expr 30 / 3

10

$ expr 30 / 3 / 2

5

 

Incrementing counters

 

Expr does the incrementing of values when using loops. First the loop is initialized

to zero. Then one is added to the variable loop. The use of the quotes means

command substitution, which basically means take the output from the command

(expr) and put it in the variable loop.

$ LOOP=0

$ LOOP=‘expr $LOOP + 1‘

 

 

Pattern matching

 

$ VALUE=accounts.doc

$ expr $VALUE : October 8, ‘.*’

12

 

Control flow structures

 

$ pg iftest

#!/bin/sh

# iftest

# this is a comment line, all comment lines start with a #

if [ “10” -lt “12” ]

then

# yes 10 is less than 12

echo “Yes, 10 is less than 12”

fi

 

Testing values of variables

 

$ pg iftest2

#!/bin/sh

# if test2

echo -n “Enter your name :”

read NAME

# did the user just hit return ????

if [ “$NAME” = “” ] ; then

echo “You did not enter any information”

fi