edgecase
8. Item, that he the said abbot hath lived viciously, and kept to concubines divers and many women that is openly known.
9. Item, that the said abbot doth yet continue his vicious living, as it is known, openly.
10. Item, that the said abbot hath spent and wasted much of the goods of the said monastery upon the aforesaid women.
Author: StJohn Piano
Published: 2018-01-25
Datafeed Article 36
This article has been digitally signed by Edgecase Datafeed.
3204 words - 993 lines - 25 pages



GOAL:



Find a way to unmount a USB memory stick from Kalkin that preserves the data that has been written to it but does not require extra mount-unmount steps.



CONTENTS:



- Goal
- Contents
- Brief Summary
- Representative Sample of Commands Used
- Project Log



BRIEF SUMMARY:



I discovered that the problem was due to a faulty memory stick.

Using another memory stick, this command sequence worked without any trouble:

[root@kalkin work]# mount /dev/sdg1 /mnt/test1


[root@kalkin work]# cp hello.txt /mnt/test1/hello.txt


[root@kalkin work]# umount /mnt/test1


Before running the command sequence above, insert the memory stick into the computer and run the command
blkid
to identify its device name. Use this device name in the
mount
command.

Note: It is good practice to use the mountpoint directory as the argument for the
umount
command (instead of the device name), as the device may be mounted on multiple mountpoint directories.



REPRESENTATIVE SAMPLE OF COMMANDS USED:



[root@kalkin work]# mount /dev/sdg1 /mnt/test1


[root@kalkin work]# ls /mnt/test1


[root@kalkin work]# cp work.log /mnt/test1/work.log


[root@kalkin work]# umount /mnt/test1


[root@kalkin ~]# mount


[root@kalkin ~]# blkid


[root@kalkin ~]# cd /home/work


[root@kalkin work]# rm work.log


[root@kalkin work]# echo "hello world" > hello.txt


[root@kalkin work]# cat hello.txt


[root@kalkin work]# ls -a /mnt/test1


aineko:~ stjohnpiano$ ls -1 /Volumes/


aineko:HP v150w stjohnpiano$ ls -l


aineko:HP v150w stjohnpiano$ ls -lh


aineko:HP v150w stjohnpiano$ wc -l *


aineko:HP v150w stjohnpiano$ head -1 kjv_bible.txt


[root@kalkin work]# rm -f /mnt/test1/*




PROJECT LOG:



I encountered a problem with saving data to a USB memory stick while listing available tools on Centos 6.9 Minimal.


I had inserted a USB memory stick into Kalkin and mounted it to a mountpoint directory (
/mnt/test1
).

I then attempted to write a file (
work.log
) to this directory, which contained data that I wanted to transfer to another computer (Aineko).

When the memory stick was unmounted and removed from Kalkin, and then inserted into Aineko and mounted, the file was 0 KB. No data had been preserved.



Here is a relevant excerpt from the earlier project:


Remove the memory stick from Kalkin.

Insert the stick into Aineko.

It mounts automatically and an icon appears on the Desktop.


work.log is again 0 KB.


Looking back at this log, it appears that when I followed this sequence:
- unmount memory stick on Kalkin
- mount memory stick on Kalkin
- unmount memory stick on Kalkin
and only then removed the memory stick and inserted it into Aineko, the data was preserved. Odd.

Let's try that again.


Eject and remove memory stick from Aineko.

Insert the stick into Kalkin.


[root@kalkin work]#
sd 11:0:0:0: [sdg] Assuming drive cache: write through
sd 11:0:0:0: [sdg] Assuming drive cache: write through
sd 11:0:0:0: [sdg] Assuming drive cache: write through


Output hangs. Press enter. New prompt appears.


[root@kalkin work]# mount /dev/sdg1 /mnt/test1


[root@kalkin work]# ls /mnt/test1

work.log

[root@kalkin work]# cat /mnt/test1/work.log


[root@kalkin work]# cp work.log /mnt/test1/work.log

cp: overwrite `/mnt/test1/work.log'? y

[root@kalkin work]# cat /mnt/test1/work.log | wc -l

9072

[root@kalkin work]# umount /mnt/test1


[root@kalkin work]# mount /dev/sdg1 /mnt/test1


[root@kalkin work]# ls /mnt/test1

work.log

[root@kalkin work]# cat /mnt/test1/work.log | wc -l

9072

[root@kalkin work]# umount /mnt/test1



Remove the memory stick from Kalkin.

Insert the stick into Aineko.

It mounts automatically and an icon appears on the Desktop.


work.log is present and its size is 131 KB.

The unmount and remount step seems to do something that causes the data to be preserved on the memory stick.



Key sentence: "The unmount and remount step seems to do something that causes the data to be preserved on the memory stick."

I would like to remove these extra unmount and mount steps from my save-data-to-USB-stick sequence. If possible, I would also like to discover a good explanation for why the single unmount step did not work by itself.



Google "unmount file not saved centos".


Excerpt from:
www.ibm.com/developerworks/library/l-lpic1-104-3/index.html
Author: Ian Shields
Published on October 20, 2010
Updated: January 27, 2016


Control mounting and unmounting of filesystems

[...]

The Linux filesystem is one big tree rooted at /, and yet we have filesystems on different devices and partitions. How do we resolve this apparent incongruity? The root (/) filesystem is mounted as part of the initialization process. Each of the other filesystems that you create is not usable by your Linux system until it is mounted at a mount point.

[...]

Unless otherwise noted, the examples in this tutorial use CentOS 6, with a 2.6.32-504 kernel.

[...]

A mount point is simply a directory where the filesystem on a device is grafted into the tree. Mounting is the process of making the filesystem on the device accessible. For example, you might mount filesystems on hard drive partitions as /boot, /tmp, or /home, and you might mount the filesystem on a floppy drive as /mnt/floppy and the filesystem on a CD-ROM as /media/cdrom1. As you can see, mount points may be in the root directory, or in a subdirectory farther down the tree.

Besides filesystems on partitions, floppy disks, and CDs, there are other types of filesystems. The tmpfs filesystem is a virtual memory filesystem. You can also mount filesystems from one system on another system using a networked filesystem such as NFS or AFS. You can even create a file in an existing filesystem and format that as a, possibly different, kind of filesystem and mount that too. This is often done with images of optical media, where you perhaps download an ISO CD or DVD image, then mount that file rather than burning it to real media. Swap space in a file rather than a dedicated swap partition is another example.

While the mount process actually mounts the filesystem on some device (or other resource), it is common to simply say that you "mount the device", which is understood to mean "mount the filesystem on the device".

[...]

When you mount a filesystem over an existing directory, the files on the filesystem you are mounting become the files and subdirectories of the mount point. If the mount point directory already contained files or subdirectories, they are not lost, but are no longer visible until the mounted filesystem is unmounted, at which point they become visible again. It is a good idea to avoid this problem by using only empty directories as mount points.

[...]

Usually, the
mount
command will automatically detect the type of filesystem being mounted. Occasionally you may need to specify the filesystem type explicitly using the
-t
option as shown in Listing 2.

Listing 2. Mounting with explicit filesystem type
[root@attic4-cent ~]# mount -t vfat /dev/sda3 /dos


To see what filesystems are mounted, use the
mount
command with no parameters.

[...]

You can also view similar information by displaying /proc/mounts or /etc/mtab, both of which contain information about mounted filesystems.

[...]

All mounted filesystems are usually unmounted automatically by the system when it is rebooted or shut down. When a filesystem is unmounted, any cached filesystem data in memory is flushed to the device.

You may also unmount filesystems manually. Indeed, you should do this when removing writable media such as diskettes or USB drives or memory keys.

Use the
unmount
command to unmount the filesystem, specifying either the device name or mount point as an argument. This listing shows how to unmount /dos, then remount it and unmount again using the device name.

Listing 8. Unmounting filesystems
[root@attic4-cent ~]# umount /dos
[root@attic4-cent ~]# mount /dev/sda3 /dos
[root@attic4-cent ~]# umount /dev/sda3


After a filesystem is unmounted, any files in the directory used for the mount point are visible again.

If you attempt to unmount a filesystem while a process has open files on that filesystem, you will see an error message.

[...]

Note: You should always unmount removable drives or media before disconnecting the drive or attempting to remove the media. Failure to do so may result in loss of data that has not yet been written to the device.



Key sentence: "When a filesystem is unmounted, any cached filesystem data in memory is flushed to the device."

I also note that either the device name or the mountpoint directory can be used as an argument for
umount
.


Excerpt from:
linux.die.net/man/8/umount

The umount command detaches the file system(s) mentioned from the file hierarchy. A file system is specified by giving the directory where it has been mounted. Giving the special device on which the file system lives may also work, but is obsolete, mainly because it will fail in case this device was mounted on more than one directory.



Ok. I'll make a habit of using the mountpoint directory as an argument for
umount
.



I think that my problem might have been caused by a flush that didn't happen.

When I wrote the file to the memory stick, this operation was completed in system memory (RAM). Checking the contents of the file would cause a query to the filesystem in RAM, which would contain the file at the right location and return its contents without any trouble. Then, when I ran the unmount command, the stick was unmounted, but the data had not been flushed to it from the RAM first. When I inserted the memory stick into Aineko, the file existed, but was empty.

Running the mount-unmount steps after the initial unmount probably caused the flush to happen properly, for some reason that I don't understand.


Let's try using a different memory stick.


Press the power button to boot Kalkin.

Centos boots.

Log in.


[root@kalkin ~]# mount

/dev/mapper/vg_kalkin-lv_root on / type ext4 (rw)
proc on /proc type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
tmpfs on /dev/shm type tmpfs (rw,rootcontext="system_u:object_r:tmpfs_t:s0")
/dev/sda1 on /boot type ext4 (rw)
/dev/mapper/vg_kalkin-lv_home on /home type ext4 (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)



New memory stick: HP, 32 GB.

Insert memory stick into Aineko. It mounts automatically and an icon appears on the Desktop.

Open Disk Utility.

Disk Utility shows that the format of the disk is MS-DOS (FAT32) and its partition map scheme is MBR (Master Boot Record). The name of the single partition is "HP V150W".

Eject and remove the disk from Aineko.

Insert the disk into Kalkin.


[root@kalkin ~]#
sd 7:0:0:0: [sdg] Assuming drive cache: write through
sd 7:0:0:0: [sdg] Assuming drive cache: write through
sd 7:0:0:0: [sdg] Assuming drive cache: write through


Output hangs. Press Enter. New prompt appears.


[root@kalkin ~]# blkid

/dev/sda1: UUID="e0b18dd5-f392-42ca-bbf1-89a1a0229e38" TYPE="ext4"
/dev/sda2: UUID="rWbUY4-wwiQ-J2Vp-n0F5-bngF-G8yb-HWQyEv" TYPE="LVM2_member"
/dev/mapper/vg_kalkin-lv_root: UUID="508692c4-54da-4a93-b2e2-6f1584e74afe" TYPE="ext4"
/dev/mapper/vg_kalkin-lv_swap: UUID="707ee542-6bfe-453d-b43c-bcfc58c59b6f" TYPE="swap"
/dev/mapper/vg_kalkin-lv_home: UUID="6134733c-158e-4d62-a74d-e7d00e2ed8fd" TYPE="ext4"
/dev/sdg1: LABEL="HP v150w" UUID="81A8-0559" TYPE="vfat"



The device name of the second memory stick is
/dev/sdg1
.

Running the
mount
command produces the same output as before. The device is visible to the operating system but is not mounted onto the filesystem tree.


Let's move to my work directory, which I made earlier.

[root@kalkin ~]# cd /home/work


[root@kalkin work]# ls

work.log

[root@kalkin work]# rm work.log

rm: remove regular file `work.log'? y

[root@kalkin work]# echo "hello world" > hello.txt


[root@kalkin work]# ls

hello.txt

[root@kalkin work]# cat hello.txt

hello world



I removed work.log because it was used in a previous project and I no longer need it.

I'm going to use hello.txt as my test file.


Let's check the existing mountpoint directory and confirm that it's empty.

[root@kalkin work]# ls /mnt

test1

[root@kalkin work]# ls /mnt/test1


[root@kalkin work]# ls -a /mnt/test1

. ..



I'll mount the second memory stick at the mountpoint directory, copy the test file to it, and unmount it.

[root@kalkin work]# mount /dev/sdg1 /mnt/test1


[root@kalkin work]# ls /mnt/test1


[root@kalkin work]# cp hello.txt /mnt/test1/hello.txt


[root@kalkin work]# umount /mnt/test1



Remove the memory stick from Kalkin.

Insert it into Aineko. It mounts automatically and an icon appears on the Desktop.


On Aineko:

aineko:~ stjohnpiano$ ls /Volumes/

HP v150w Macintosh HD Untitled


aineko:~ stjohnpiano$ ls -1 /Volumes/

HP v150w
Macintosh HD
Untitled

aineko:~ stjohnpiano$ cd /Volumes/HP\ v150w/


aineko:HP v150w stjohnpiano$ ls

hello.txt

aineko:HP v150w stjohnpiano$ cat hello.txt

hello world




Hm. It worked. Perhaps the original memory stick was faulty in some way.


Perhaps the large size of work.log was a problem. It was 9072 lines and 131 KB.

I should have kept work.log to use as a test file.


I have a text file copy of the KJV Bible (downloaded from Project Gutenberg) on Aineko.

Copy it to the memory stick.

aineko:HP v150w stjohnpiano$ ls -l

total 8544 -rwxrwxrwx 1 stjohnpiano staff 12 Jan 24 2018 hello.txt -rwxrwxrwx@ 1 stjohnpiano staff 4352552 Apr 27 2016 kjv_bible.txt


aineko:HP v150w stjohnpiano$ ls -lh

total 8544 -rwxrwxrwx 1 stjohnpiano staff 12B Jan 24 2018 hello.txt -rwxrwxrwx@ 1 stjohnpiano staff 4.2M Apr 27 2016 kjv_bible.txt


aineko:HP v150w stjohnpiano$ wc -l *

1 hello.txt 100304 kjv_bible.txt 100305 total


aineko:HP v150w stjohnpiano$ head -1 kjv_bible.txt

The Project Gutenberg EBook of The King James Bible


It's 100304 lines and 4.2 MB.


Eject and remove the memory stick from Aineko (close Terminal window first to allow ejection).

I'm going to copy the large bible text file to Kalkin, delete it from the memory stick, and then copy it back to the memory stick in order to test the
umount
flush.


Insert memory stick into Kalkin.

[root@kalkin work]#
sd 8:0:0:0: [sdg] Assuming drive cache: write through
sd 8:0:0:0: [sdg] Assuming drive cache: write through
sd 8:0:0:0: [sdg] Assuming drive cache: write through


Output hangs. Press Enter. New prompt appears.


[root@kalkin work]# mount /dev/sdg1 /mnt/test1


[root@kalkin work]# ls -1 /mnt/test1

hello.txt
kjv_bible.txt

[root@kalkin work]# cp /mnt/test1/kjv_bible.txt bible.txt


[root@kalkin work]# ls -1

bible.txt
hello.txt

[root@kalkin work]# rm -f /mnt/test1/*


[root@kalkin work]# ls -1 /mnt/test1


[root@kalkin work]# umount /mnt/test1



Unplug and reinsert memory stick into Kalkin.

[root@kalkin work]#
sd 9:0:0:0: [sdg] Assuming drive cache: write through
sd 9:0:0:0: [sdg] Assuming drive cache: write through
sd 9:0:0:0: [sdg] Assuming drive cache: write through


Output hangs. Press Enter. New prompt appears.


[root@kalkin work]# mount /dev/sdg1 /mnt/test1


[root@kalkin work]# ls -1 /mnt/test1


[root@kalkin work]# cp bible.txt /mnt/test1/bible.txt


[root@kalkin work]# ls -1 /mnt/test1

bible.txt

[root@kalkin work]# head -1 /mnt/test1/bible.txt

The Project Gutenberg EBook of The King James Bible

[root@kalkin work]# wc -l /mnt/test1/bible.txt

100304 /mnt/test1/bible.txt

[root@kalkin work]# ls -lh /mnt/test1/bible.txt

-rwxr-xr-x. 1 root root 4.2M Jan 24 12:47 /mnt/test1/bible.txt

[root@kalkin work]# umount /mnt/test1



Remove memory stick from Kalkin.


Insert it into Aineko. It mounts automatically and an icon appears on the Desktop.


On Aineko:

aineko:~ stjohnpiano$ cd /Volumes/HP\ v150w/


aineko:HP v150w stjohnpiano$ ls

bible.txt

aineko:HP v150w stjohnpiano$ ls -lh

total 8512
-rwxrwxrwx 1 stjohnpiano staff 4.2M Jan 24 2018 bible.txt


aineko:HP v150w stjohnpiano$ head -1 bible.txt

The Project Gutenberg EBook of The King James Bible

aineko:HP v150w stjohnpiano$ wc -l bible.txt

100304 bible.txt



Hm. No problem copying a large file.

Close the Terminal window. Eject and remove the memory stick from Aineko.


Ok. Let's try again with the original memory stick (Kingston, 32 GB). I'll try to use it to transfer the hello.txt file.


Insert the first memory stick into Aineko.

It doesn't contain any files.

Open Disk Utility.

Disk Utility shows that the format of the disk is MS-DOS (FAT32) and its partition map scheme is MBR (Master Boot Record). The name of the single partition is "KINGSTON".

Eject and remove the disk from Aineko.

Insert the disk into Kalkin.


[root@kalkin work]#
sd 10:0:0:0: [sdg] Assuming drive cache: write through
sd 10:0:0:0: [sdg] Assuming drive cache: write through
sd 10:0:0:0: [sdg] Assuming drive cache: write through


Output hangs. Press Enter. New prompt appears.


[root@kalkin work]# blkid

/dev/sda1: UUID="e0b18dd5-f392-42ca-bbf1-89a1a0229e38" TYPE="ext4"
/dev/sda2: UUID="rWbUY4-wwiQ-J2Vp-n0F5-bngF-G8yb-HWQyEv" TYPE="LVM2_member"
/dev/mapper/vg_kalkin-lv_root: UUID="508692c4-54da-4a93-b2e2-6f1584e74afe" TYPE="ext4"
/dev/mapper/vg_kalkin-lv_swap: UUID="707ee542-6bfe-453d-b43c-bcfc58c59b6f" TYPE="swap"
/dev/mapper/vg_kalkin-lv_home: UUID="6134733c-158e-4d62-a74d-e7d00e2ed8fd" TYPE="ext4"
/dev/sdg1: LABEL="KINGSTON" UUID="7195-020C" TYPE="vfat"

[root@kalkin work]# mount /dev/sdg1 /mnt/test1


[root@kalkin work]# ls -1 /mnt/test1


[root@kalkin work]# cp hello.txt /mnt/test1/hello.txt


[root@kalkin work]# ls -1 /mnt/test1

hello.txt

[root@kalkin work]# cat /mnt/test1/hello.txt

hello world

[root@kalkin work]# umount /mnt/test1



Remove memory stick from Kalkin.


Insert it into Aineko. It mounts automatically and an icon appears on the Desktop.


On Aineko:

aineko:~ stjohnpiano$ ls -1 /Volumes/

KINGSTON
Macintosh HD
Untitled

aineko:~ stjohnpiano$ cd /Volumes/KINGSTON/


aineko:KINGSTON stjohnpiano$ ls

hello.txt

aineko:KINGSTON stjohnpiano$ cat hello.txt


aineko:KINGSTON stjohnpiano$ ls -lh

total 32
-rwxrwxrwx 1 stjohnpiano staff 12B Jan 24 2018 hello.txt



Hm. 12 KB but it contains no text data.


Let's try reformatting the memory stick.


Close the Terminal window.


Open Disk Utility.

In Disk Utility, select the memory stick (it shows up as "Kingston DataTraveler 2.0 Media").

Click Erase tab.
- Format = MS-DOS (FAT)
- Name = KINGSTON
Click Erase.

Pop-up dialogue box opens with this message:
Are you sure you want to erase the disk "Kingston DataTraveler 2.0 Media"?
Erasing a disk deletes all data on all its partitions.
This disk has 1 partition:
- "Kingston"

and two buttons: Cancel and Erase.
Click Erase.

Click Partition tab. Under Volume Scheme, choose 1 Partition. Click Options and choose Master Boot Record partition scheme.
Under Volume Information, set these values:
- Name = KINGSTON
- Format = MS-DOS (FAT)
- Size = 31.05
Click Apply.

Pop-up dialogue box opens with this message:
Are you sure you want to partition the disk "Kingston DataTraveler 2.0 Media"?
Partitioning this disk will delete all data on this disk.
This partition will be erased: "KINGSTON"
This partition will be added: "KINGSTON"

and two buttons: Cancel and Partition.
Click Partition.

Quit Disk Utility.

Eject KINGSTON disk (right-click icon on Desktop and choose Eject).

Remove the KINGSTON memory stick from Aineko.


Insert the disk into Kalkin.


[root@kalkin work]#
sd 11:0:0:0: [sdg] Assuming drive cache: write through
sd 11:0:0:0: [sdg] Assuming drive cache: write through
sd 11:0:0:0: [sdg] Assuming drive cache: write through


Output hangs. Press Enter twice. New prompt appears.

Note: I had to press Enter twice (the first press had no visible effect). This happened before in this project but I had thought I hadn't been pressing the key hard enough the first time.


[root@kalkin work]# blkid

/dev/sda1: UUID="e0b18dd5-f392-42ca-bbf1-89a1a0229e38" TYPE="ext4"
/dev/sda2: UUID="rWbUY4-wwiQ-J2Vp-n0F5-bngF-G8yb-HWQyEv" TYPE="LVM2_member"
/dev/mapper/vg_kalkin-lv_root: UUID="508692c4-54da-4a93-b2e2-6f1584e74afe" TYPE="ext4"
/dev/mapper/vg_kalkin-lv_swap: UUID="707ee542-6bfe-453d-b43c-bcfc58c59b6f" TYPE="swap"
/dev/mapper/vg_kalkin-lv_home: UUID="6134733c-158e-4d62-a74d-e7d00e2ed8fd" TYPE="ext4"
/dev/sdg1: LABEL="KINGSTON" UUID="6CE6-14FA" TYPE="vfat"

[root@kalkin work]# mount /dev/sdg1 /mnt/test1


[root@kalkin work]# ls -1 /mnt/test1


[root@kalkin work]# cp hello.txt /mnt/test1/hello.txt


[root@kalkin work]# ls -1 /mnt/test1

hello.txt

[root@kalkin work]# cat /mnt/test1/hello.txt

hello world

[root@kalkin work]# umount /mnt/test1



I note that the Kingston memory stick UUID has changed from 7195-020C to 6CE6-14FA.


Remove memory stick from Kalkin.


Insert it into Aineko. It mounts automatically and an icon appears on the Desktop.



On Aineko:

aineko:~ stjohnpiano$ cd /Volumes/KINGSTON/


aineko:KINGSTON stjohnpiano$ ls

hello.txt

aineko:KINGSTON stjohnpiano$ cat hello.txt

Mac

aineko:KINGSTON stjohnpiano$ ls -lh

total 32
-rwxrwxrwx 1 stjohnpiano staff 12B Jan 24 2018 hello.txt



Strange.


Download display_bytes.py (one of the downloadable assets included in Listing available tools on Centos 6.9 Minimal) to my Downloads directory.

Copy hello.txt from the Kingston memory stick to my Downloads directory.

Edit this value of the
file_path
variable in display_bytes.py so that the script targets hello.txt.

file_path = "hello.txt"


aineko:Downloads stjohnpiano$ python display_bytes.py

[ord_0][ord_5][ord_22][ord_7][ord_0][ord_2][ord_0][ord_0]Mac



Note: There is a space at the end of the output line.

Looking at my ASCII reference table, I find that:
- byte 0 is NUL (null)
- byte 5 is ENQ (enquiry)
- byte 22 is SYN (synchronous idle)
- byte 7 is BEL (bell)
- byte 2 is STX (start of text)


I think at this point I can conclude that the problem is with the Kingston memory stick, not with Kalkin or the usage of the
umount
command.

Solution: Put the Kingston memory stick into the disks-to-be-disposed-of pile.


Tidy up:
- delete display_bytes.py and hello.txt from Downloads directory.