Storage
See also: Administrative, Hardware, Virtualization
Partitioning
cfdisk
A command line user interface (CLUI) wrapper for fdisk.
fdisk
Usage |
Explanation |
---|---|
-l |
lists all partitions, sector and disk label information |
m |
shows menu options |
p |
show current partition layout |
d |
deletes partition |
Example |
Explanation |
---|---|
/dev/sdc |
modify partitions on the third SATA drive |
parted
Partition manager that supports creating, reading, updating, and deleting MBR and GPT partitions.
Usage |
Explanation |
---|---|
help |
show the help guide |
help mkpart |
shows how to make a new partition |
select |
select a partition |
print all |
shows all the partitions; you will need to know where the last one ends to create a new one |
mktable |
create either a “”msdos”” (MBR) or “”gpt”” partition table on the selected drive |
align-check optimal |
check a partition to see if it is properly aligned |
mkfs.btrfs
Package: btrfs-progs
Create Btrfs file systems.
Usage |
Explanation |
---|---|
-d |
specify a RAID type (raid0, raid1, raid5, raid6, raid10) and drives for creating a RAID for daa |
-m |
the same as “”-d”” but instead specifying where the RAID filesystem metadata should be spread out to |
btrfs
Manage Btrfs partitions.
Usage |
Explanation |
---|---|
filesystem show |
list all BTRFS partitions and their usage |
filesystem df / |
show disk usage on the root partition; this should be used instead of just running “”df”” outside of the btrfs command |
filesystem resize {+|-}<SIZE>{m|g} |
resize the filesystem on an active/live mount |
subvolume list |
list all of the snapshots and subvolumes under a specified mount directory |
snapshot create |
create a snapshot of the partition |
subvolume snapshot |
create a snapshot of a subvolume |
filesystem defragment -r -v -c lzo |
defragment the system files, recursively, verbosely, and compress them using the high performance “lzo” algorithm; be sure to add the “compress=lzo” mount option in /etc/fstab for the partition |
balance start / -v |
rebalance/defragment the repartition at the block level, on the root partition, verbosely; this will help free up space |
Example |
Explanation |
---|---|
subvolume create /var/ /var/var-snapshot/ |
create a snapshot of the /var/ folder |
mkfs.ext4
Example |
Explanation |
---|---|
/dev/sdc3 |
creates a ext4 formatted partition on /dev/sdc3 |
mkswap
Package: util-linux
Example |
Explanation |
---|---|
/dev/sda2 |
creates swap partition format on /dev/sda2 |
swapon
Package: util-linux
Configure Linux to swap on certain partitions.
Example |
Explanation |
---|---|
/dev/sda2 |
tells Linux swap on /dev/sda2 |
partclone
An efficient partition backup utility.
Usage |
Explanation |
---|---|
-c |
clone a partition |
-r |
restore a cloned partition |
-s |
specify the source partition |
-o |
save the partition clone to a specified file |
mount
Mount a partition onto a folder.
Usage |
Explanation |
---|---|
-l |
lists all current mounts |
-r |
read-only |
-t |
file system to use |
-o loop |
for loop devices such as ISO images |
-o remount |
remount a directory |
-a |
remounts all entries in the /etc/fstab |
Example |
Explanation |
---|---|
/dev/sda1 /var |
mounts the first SCSI drive’s (sda) first partition onto the folder /var |
umount
Usage |
Explanation |
---|---|
-l |
lazy unmount; unmount the file system and let the running processes on it complete |
-f |
force dismount now |
Example |
Explanation |
---|---|
/var |
dismount the partition that is on /var |
lsblk
Lists partition, their size, and mount point information.
Usage |
Explanation |
---|---|
–scsi |
list SCSI devices like iSCSI, SATA, SAS, etc. |
-f |
show file systems |
Usage |
Explanation |
---|---|
/dev/sda |
show information only about the sda drive |
blkid
Shows UUIDs and labels for all of the partitions.
Usage |
Explanation |
---|---|
-c /dev/null |
regenerate the partition UUIDs for the /etc/blkid.tab file |
touch /forcefsck
Forces a files system check on the next boot.
dd
Usage |
Explanation |
---|---|
if= |
specify an input device to read from |
of= |
specify an output device to write to |
bs= |
specify the byte size |
count= |
number of times it should copy each byte size |
conv=fdatasync |
flush data to the disk; do not cache it in memory |
Example |
Explanation |
---|---|
if=/dev/zero of=/tmp/50MB.img bs=1M count=50 |
create a blank file that is 50MB |
if=/dev/sda1 of=file.img |
copy blocks from a partition to a file |
df
Show disk space usage.
Usage |
Explanation |
---|---|
-h |
human readable space usage |
-i |
inonde/file count |
-T |
also display the filesystem types |
-t |
only show filesystems of this type |
Example |
Explanation |
---|---|
-h -t xfs |
show human readable sizes for only XFS file systems |
du
Show folder disk space usage.
Usage |
Explanation |
---|---|
-h |
human readable size format |
–max-depth= |
the maximum number of subdirectories to find their disk space |
Example |
Explanation |
---|---|
-h –max-depth=2 |
show sizes up to two directories deep |
ncdu
Interactively show and navigate through “du” disk space usage reports.
testdisk
Data recovery tool.
fsck
File system check utility for helping fixing corrupt file systems.
Usage |
Explanation |
---|---|
-y |
automatically attempt to repair any problems |
-t |
the file system type |
-C |
show the progress |
-V |
verbosely show the specific file system commands that are being run and their output |
Example |
Explanation |
---|---|
-C -t ext4 -y /dev/sdb2 |
automatically fix any problems with the ext4 file system on /dev/sdb2 |
dumpe2fs
Lookup UUIDs of partitions. These can be used in /etc/fstab.
dmesg
Kernel logs for hardware devices. After plugging in a USB device, check this to find it’s device name and partition to mount.
Example |
Explanation |
---|---|
dmesg | grep SCSI |
shows all connected SCSI devices |
mkisofs
Create ISO images.
Usage |
Explanation |
---|---|
-o <FILE>.iso <DIRECTORY> |
creates an ISO file from the contents of a directory |
lsof
List open files is used to show what processes are using a certain file or directory.
Usage |
Explanation |
---|---|
+D <DIRECTORY> |
shows what PIDs are using the drive |
fuser
Usage |
Explanation |
---|---|
-uz <DIRECTORY> |
forcefully unmount a a file system |
-mv <DIRECTORY> |
used to see what user’s are using the file or directory |
iostat
Usage |
Explanation |
---|---|
-x <INTEGER> |
monitor every specified number of seconds |
Example |
Explanation |
---|---|
-x 5 |
report on I/O usage every 5 seconds |
partprobe
Package: parted
Rescan for the latest information about available partitions. Sometimes required after updating or changing a partition.
smartctl
Package: smartmontools
Monitor drive health using SMART firmware (standard on modern storage devices).
Usage |
Explanation |
---|---|
-a |
shows all SMART information |
-i |
shows detailed drive information |
-H |
shows health status |
-t |
test for any errors, this can be “short” or “long” |
-l selftest |
shows the results of a test |
-s {on|off} |
turn SMART recording on or off for a specified drive |
Example |
Explanation |
---|---|
-t short /dev/sdb |
run a quick test on the drive /dev/sdb |
kpartx
Find and attach partitions from storage devices.
Usage |
Explanation |
---|---|
-a |
find all partitions in a given LV or image file and create loop devices for them |
-d |
detach the loop devices for a given LV or image file |
-u |
update the partition mapping |
-v |
verbose |
losetup
Controls loop devices.
Usage |
Explanation |
---|---|
-a |
view all loop devices created by losetup |
-d <LOOP_DEVICE> |
detach a loop device |
–detach-all |
detach all loop devices created by losetup |
xfs_admin
Manage XFS partitions.
Usage |
Explanation |
---|---|
-L |
creates a label |
e2label
Create labels on EXT file systems.
Usage |
Explanation |
---|---|
<DEVICE> |
shows any label names |
<DEVICE> <LABLE> |
create a label for a device |
Example |
Explanation |
---|---|
/dev/sda1 BackupDrive |
label sda1 as BackupDrive |
mdadm
Create and manage software RAIDs.
Usage |
Explanation |
---|---|
–create |
create a RAID… |
–level=<INTEGER> |
…specify the RAID level |
–raid-device=<INTEGER> |
…and specify the amount of drives to be used. |
–detail |
shows details about a current RAID |
–detail –scan |
automatically scan for all RAIDs in use and show details |
–examine –scan |
look for inactive RAIDs |
–assemble |
recreate RAID with specified drives |
–assemble –scan |
automatically recreate existing RAIDs |
–stop /dev/md<NUMBER> |
disable a RAID device |
Example |
Explanation |
---|---|
mdam –create –level=0 –raid-device=2 /dev/md0 /dev/sda /dev/sdb |
use sda and sdb to create a software RAID0 |
ssm
Create and manage encrypted partitions.
Usage |
Explanation |
---|---|
create |
create a new volume |
open |
specify a partition and a mapper to map it to |
close |
remove the mapper device |
–fstype |
specify filesystem |
–encrypt |
specify encryption |
cryptsetup
Usage |
Explanation |
---|---|
-y |
verify password; ask for it to be input twice |
luksFormat |
specify a partition to format with LUKS encryption |
open –type luks /dev/<DEVICE> <MAP> |
mount the encrypted partition |
close |
remove the mapper device |
open –type luks /dev/sda4 mydata |
mount the /dev/sda4 partition as /dev/mapper/mydata |
od
Octal dump is used to read 8 bits (1 byte) at a time of data directly from a storage device.
Usage |
Explanation |
---|---|
-c |
shows special characters |
-N<INTEGER> |
read the first specified number of bytes |
-j<INTEGER> |
skip the first specified number of bytes |
-v |
output duplicate information |
Example |
Explanation |
---|---|
-N32 |
read the first 32 bytes |
-j512 |
skip the first 512 bytes |
gddrescue
GNU ddrescue.
Usage |
Explanation |
---|---|
-r 1 |
try to recover bad sectors |
-n |
copy non-error sectors |
ddrescue
Not to be confused with GNU ddrescue.
guestfish
Package: libguestfs-tools-c
Mount QCOW2 images interactively.
Usage |
Explanation |
---|---|
–rw |
mount in read/write mode |
–ro |
mount in read-only mode |
-a |
specify the image file |
>run |
run a search of information on the used disk |
>list-filesystems |
show all of the filesystems |
>mount |
mount a partition onto a mountpoint |
>ls |
list a directory |
>edit |
open up the “”vi”” editor |
>touch |
create a new file |
>exit |
properly close the image and exit out of the prompt |
Example |
Explanation |
---|---|
echo -e “runnlist-filesystems” | guestfish -a centos-7.qcow |
use guestfish to non-interactively view the file systems on a QCOW2 image |
guestmount
Mount QCOW2 images non-interactively.
Usage |
Explanation |
---|---|
-a |
specify the image file |
-m |
specify the partition to use and then a mount point |
–rw |
mount in read-write mode |
–ro |
mount in read-only mode |
Example |
Explanation |
---|---|
guestmount -a image.qcow2 -m /dev/sda1 –rw /mnt |
mount the sda1 partition from a QCOW2 image in a writable mode to the /mnt directory |
guestunmount
Unmount QCOW2 image partitions mounted by guestmount.
LVM
The Logical Volume Manager (LVM) is an implementation to help easily configure and setup partitions and drives.
pvcreate
Create physical volumes.
Example |
Explanation |
---|---|
pvcreate /dev/sdb1 /dev/sdc1 /dev/sdc2 |
create a physical volume from these three partitions |
pvremove
Remove physical volumes.
Example |
Explanation |
---|---|
/dev/sdb1 /dev/sdc1 /dev/sdc2 |
remove these three partitions from physical volumes |
pvdisplay
Show all physical of the LVM partitions.
vgcreate
Create logical volume groups.
Example |
Explanation |
---|---|
fileserver /dev/sdb1 /dev/sdc1 |
creates a volume group of LVMs that can be partitioned for actual use |
vgdisplay
Show information about the volume groups.
vgrename
Rename volume groups.
Example |
Explanation |
---|---|
fileserver fileserver_new |
change volume group name to fileserver_new |
lvcreate
Create a logical volume.
Usage |
Explanation |
---|---|
–name <NAME> –size <SIZE>G <VOLUME_GROUP> |
create a new logical volume |
-l 100%FREE –thinpool <LOGICAL_VOLUME> <VOLUME_GROUP> |
create a thin provisioning pool to allow for more efficient snapshots |
-s <VOLUME_GROUP>/<LOGICAL_NAME> |
create a snapshot of a logical volume (this can be restored at a later point in time) |
Example |
Explanation |
---|---|
–name media –size 30G fileserver |
create a logical volume to be used as a virtual drive/partition |
-L30G -s -n newsnapshot /dev/fileserver/media |
create a 30GB snapshot of the media logical volume |
-V 4G –thin -n <NEW_LOGICAL_VOLUME> <VOLUME_GROUP>/<LOGICAL_VOLUME> |
create a logical thin volume within a logical thin pool (does not fully allocate the space, allows for over-allocating resources) |
lvrename
Rename logical volumes.
Usage |
Explanation |
---|---|
<VOLUME_GROUP> <LOGICAL_VOLUME_NAME> <NEW_LOGICAL_VOLUME_NAME> |
lvresize
Resize logical volumes.
Usage |
Explanation |
---|---|
-r |
resize the filesystem |
-L |
size in M(B) or G(B) |
-l 100%FREE |
expand to use all available free space |
lvextend
Increase the size of logical volumes.
Usage |
Explanation |
---|---|
-L |
size in M(B) or G(B) |
Example |
Explanation |
---|---|
-L55.5G /dev/fileserver/media |
increases LV size of media by 55.5 gigabytes |
–extents +100%FREE lv_example |
extend the LV lv_example to utilize all of the available space in VG |
lvreduce
Decrease the size of a logical volume.
Usage |
Explanation |
---|---|
-L |
size in M(B) or G(B) |
lvremove
Remove logical volumes.
lvconvert
Restore a snapshot.
Usage |
Explanation |
---|---|
–merge <LOGICAL_VOLUME_SNAPSHOT> |
the original logical volume will be restored to this specified snapshot |
lvchange
Usage |
Explanation |
---|---|
-ay <VOLUME_GROUP>/<LOGICAL_VOLUME> |
activate a logical volume |
-an <VOLUME_GROUP>/<LOGICAL_VOLUME> |
deactivate a logical volume |
Boot Loaders
grub-customizer
A GUI for modifying GRUB boot entries.
gnome-disks
Package: gnome-disk-utility
Provides a GUI for modifying disk partitions
grub2-mkconfig
A utility to dynamically rebuild the GRUB 2 configuration based on the installed kernels and GRUB settings. On Fedora, the utility is grub2-mkconfig
. On other operating systems it is grub-mkconfig
.
Usage |
Explanation |
---|---|
-o |
output the GRUB configuration to a specified file |
Example |
Explanation |
---|---|
-o /boot/grub2/grub.cfg |
rebuild the GRUB2 BIOS configuration on RHEL 7 and Fedora |
-o /boot/grub/grub.cfg |
rebuild the GRUB2 BIOS or UEFI configuration on Arch Linux |
-o /boot/efi/EFI/redhat/grub.cfg |
rebuild the GRUB2 UEFI config on RHEL 7 |
-o /boot/efi/EFI/fedora/grub.cfg |
rebuild the GRUB2 UEFI config on Fedora |
grub-update
Update the GRUB bootloader data on the start of the storage device.
grubby
Update the GRUB configuration with new settings.
Example |
Explanation |
---|---|
–info=ALL |
show information about all of the GRUB menu entries |
–default-kernel |
show the default kernel that will boot |
–set-default <KERNEL_PATH_OR_MENU_ENTRY_NAME_OR_MENU_ENTRY_INDEX> |
set a new default kernel |
–add-kernel <KERNEL_PATH> |
add a GRUB entry for the new kernel |
–remove-kernel <KERNEL_PATH_OR_MENU_ENTRY_NAME_OR_MENU_ENTRY_INDEX> |
remove the GRUB entry for the specified kernel |
–args=<ARGS> |
add new kernel boot arguments |
–remove-args=<ARGS> |
remove existing kernel boot arguments if found |
–update-kernel=ALL |
make changes to all GRUB entries |
–update-kernel <KERNEL_PATH_OR_MENU_ENTRY_NAME_OR_MENU_ENTRY_INDEX> |
make changes to a specific kernel GRUB entry |
–[grub|lilo|silo|yaboot|zipl] |
specify the bootloader configuration to modify (default is GRUB) |
(GRUB Rescue Prompt)
If GRUB fails to boot, a grub rescue>
prompt is presented to the end-user.
Usage |
Explanation |
---|---|
insmod normal; normal |
this will load the kernel module for the GUI GRUB prompt |
blkid |
list partitions and their UUIDs |
ls |
list the partitions; this can also be used to view files in the partition |
busybox |
busybox is sometimes provided to have common shell utilities available |
dracut
Rebuild the initramfs on Fedora.
Usage |
Explanation |
---|---|
rebuild the initramfs for the current running kernel; optionally specify a kernel version |
|
-f, –force |
replace the initramfs if it already exists |
–add-drivers |
a list of kernel modules to append to the default modules |
–drivers |
a specific list of kernel modules to compile into the initramfs |
–list-modules |
show all Dracut modules |
–add |
Dracut modules to add, appending the default modules |
–driver |
a specific list of Dracut modules to add |
–omit-drivers |
specify a list of kernel modules to exclude |
–install |
a list of files to add |
–compress {gzip|bzip2|lzma|xz|lzo|lz4} |
specify the type of compression to use; default is gzip |
–kernel-image |
the kernel image file to use |
Ceph
ceph-deploy
Install and manage Ceph.
Usage |
Explanation |
---|---|
install –release <RELEASE> <SERVER1> <SERVER2> |
install or upgrade the release version of Ceph (see http://docs.ceph.com/docs/master/releases/) on the specified servers |
mon create-initial |
install the Ceph monitor services on the local node |
disk list <SERVER1> |
show all available disks on a specified server |
disk zap <SERVER1>:<DRIVE1> |
remove the partition table off of the specified server’s drives |
osd create <SERVER1>:<DRIVE1> |
prepare the drives; two partitions are created, a (1) data and (2) journal partition |
Example |
Explanation |
---|---|
disk zap node1:sdc |
wipe the sdc drive from node1 |
ceph
Usage |
Explanation |
---|---|
auth list |
show users and their permissions |
status |
show information about the Ceph cluster |
mon {stat|dump} |
show the status of the Ceph monitoring services |
tell osd.* version |
show the Ceph version on the OSD nodes |
tell mon.* version |
show the Ceph version on the monitor nodes |
osd pool ls |
list all created pools |
health |
show the health status of the cluster |
health detail |
show a more detailed report of any issues |
pg repair <PG> |
fix inconsistencies within a placement group |
osd <POOL> set data size <COUNT> |
set the total number of all objects (replicas and the original file) that should be created per object |
osd pool get <POOL> <KEY> |
get the current value for the key for a specific pool |
osd pool get rbd size |
show the replica count |
pg dump |
display the placement group map |
osd dump |
display the OSD map |
osd tree |
show OSD weights and brief status |
osd pool create <NAME> <PG_NUM> <PGP_NUM> |
create a new pool in Ceph |
osd pool get volumes size |
show volume sizes |
osd pool set volumes size <INTEGER> |
set volumes size |
ceph-osd
Usage |
Explanation |
---|---|
-i |
specify the OSD drive number |
–flush-journal |
flush the journal to the disk |
rbd
Usage |
Explanation |
---|---|
{ls|list} |
show block devices in a specified pool |
rm <DEVICE> -p <POOL> |
delete a device from a pool |
export <POOL>/<VOLUME> –path <FILE>.img |
download the block image |
snap ls <POOL>/<VOLUME> |
show all snapshots for a block device |
snap purge |
remove all snapshots |
snap protect |
prevent a snapshot from being deletable |
snap unprotect |
allow a snapshot to be deleted |
create <POOL>/<IMAGE> –size <SIZE_IN_MB> |
create a new RBD image |
map <POOL>/<IMAGE> |
map an RBD image to a client server as a block device |
feature disable imagename deep-flatten fast-diff object-map exclusive-lock |
disable all of the new RBD features for an image that are new to the Jewel release; these require the Linux 4.8 kernel or newer |
rbd snap purge <POOL>/<VOLUME_ID>@<SNAPSHOT> |
remove a snapshot |
rados
Usage |
Explanation |
---|---|
lspools |
view RADOS pools |
df |
show Ceph disk usage |
-p <POOL> ls |
show raw PG files that are part of a pool |
iSCSI
targetcli (interactive)
Create iSCSI targets.
Usage |
Explanation |
---|---|
ls |
view the current configuration tree |
cd |
change to a different path |
saveconfig |
save the configuration |
restoreconfig |
restore a specified configuration file |
clearconfig |
delete the current configuration |
help |
view help output |
Variables |
Explanation |
---|---|
auto_save_on_exit |
run saveconfig on exit; default: True |
confirm |
allow the configuration to be cleared; default: False |
iscsiadm
Package: iscsi-initiator-utils
Usage |
Explanation |
---|---|
–mode discoverydb –type sendtargets –portal <IPADDRESS> –discover |
search for iSCSI targets at a given address |
iscsiadm –mode node –targetname <TPG> –portal <IPADDRESS> –login |
attach the target portal group |
Windows Partitioning
These are utilities to manage Windows file systems from UNIX-like operating systems.
fatresize
Resize FAT file systems.
mkdosfs
Package: dosfstools
Manage DOS filesystems.
Usage |
Explanation |
---|---|
-F 16 |
format a partition to FAT16 |
mkfs.ntfs
Create NTFS file systems.
Usage |
Explanation |
---|---|
-Q |
quickly formats 1 partition to NTFS on a device |
mount -t ntfs-3g
Package: ntfs-3g
Mount NTFS file systems. The Windows operating system had to have been cleanly and fully shutdown first by running shutdown /s /f /t 0
from Windows.
ms-sys
Utility for creating Windows Master Boot Records (MBR).
Usage |
Explanation |
---|---|
–fat32 |
create a FAT32 DOS MBR |
-7 |
create a Windows 7 MBR |