CPS100 • INTRODUCTION TO COMPUTERS

Volumes and Folders

Filesystems keep volumes organized by placing their files into directories. You have seen a directory before, many times: you call them folders. "Directories" and "folders" are the same thing. The name "directory" goes back to the days of the CLI.

Like volumes, folders contain data. However, while volumes are physical devices, folders are imaginary locations. You cannot touch a data folder. Because of this, folders are always inside volumes, but volumes are never actually inside folders.

However, what controls this? What controls the rules regarding filenames? What system is used to organize drives? How is the address of any data point decided and stored? This and many other points are decided by the filesystem.

A filesystem is a set of rules about how storage works. The filesystem dictates how the volume is structured, how data is saved to which locations on the disk; how that data's location is recorded; how information about the data is kept; how the data is marked for deletion, and how it is deleted.

Modern computers use filesystems that fit the needs of modern computing. For example, older filesystems only allowed very short filenames, using limited character encodings. Newer filesystems allow very long filenames (255 characters long) using almost any language. Older filesystems could only allow files up to a few gigabytes in size; newer filesystems allow for huge file sizes. As computers grow more powerful and flexible, filesystems must change to allow for this growth. In addition, filesystems are always improving.

There are many different filesystems. Each operating system has its own favorite filesystem. This is similar to the way people organize their own files, or even their kitchens.

If you visit another person's house and try to use their kitchen, you will quickly become confused. Some things may be obvious—for example, you will probably find silverware in a flat drawer close to the sink, and you may find cleaning supplies under the sink. However, most items will be stored in ways different from your own kitchen. You will not be able to find anything without a search. This is because each person has their own system and preferences for how things are arranged.

The same is true for filesystems: each one has its own way of handling files.

Here are the major filesystems in use over time:

Each operating system has it's own major filesystem. Windows has NTFS. MacOS has APFS. Linux has ext4.

These primary filesystems are exclusive: NTFS is only used by Windows; APFS is only used by Macs; ext4 is only used by Linux. These could be called native filesystems. Each filesystem is developed to fit the features and needs of each operating system.

While a computer can read and use multiple filesystems, each one "prefers" the native filesystem, which was created especially for that operating system. The native filesystem will work better and faster with that computer.

Without special assistance, most of the operating systems cannot read disks formatted with the main filesystem of another operating system. Macs can read NTFS but not write to them; Windows cannot read ext3, and Linux cannot read APFS. However, it is possible to buy software which allows one operating system to read disks formatted by other major filesystems (for example, a program called "Paragon NTFS for Mac" allows Macs to read and write disks formatted with Windows' NTFS filesystem).

A volume, such as a hard drive, may be blank—it has no data, not even a filesystem. If you try to connect a blank disk to a computer, the computer will report that the disk cannot be read—and it asks you if you wish to format the drive (it also does this when it cannot read the formatted file system that exists on a drive). When you format the drive, you can choose a filesystem.

The computer's main HDD and SSD drives use the operating system's main filesystems. However, external storage might be shared between computers. So, which filesystems do they use?

The answer is usually FAT—either FAT32, exFAT, or a similar version of the FAT filesystem. The FAT filesystems are older ones used previously by Microsoft. These filesystems can be read and understood by any operating system. Therefore, they serve as popular filesystems for external and portable drives, especially USB Flash Drives.

If you will use an external hard drive with only one kind of computer, you should format the drive as NTFS if you use Windows, or as APFS if you use a Mac. USB Flash drives will probably work best using ExFAT, unless you also use the USB drive with only one type of computer.

Blocks and Fragmentation

One example of how file system works is how it chops up and stores the data. Large data files are not saved as one giant chunk of data. It is divided into blocks, often 4 KB in size.

Here you can see a file which is only 30 Bytes in size, but the computer reports that it is taking up 4 KB (4000 Bytes) in space on the disk.

This of the hard drive as being like a wall with cubbyholes, like those shown in a classroom here:

Each cubbyhole can hold a certain amount, but each person gets one cubbyhole—meaning that if one person only has a few small books, their cubbyhole will remain mostly empty.

Data is stored on computers in much the same way; the data is divided into and stored in blocks; each block may be 4 kilobytes in size.

When data is stored, erased, re-written, and erased again many times, the data for one file may become scattered across the disk, making it much slower to retrieve the data. This is called fragmentation, and was often a serious problem with older file systems. Newer file systems are designed to automatically re-arrange data in the blocks to avoid that problem.

Below is a small example of how fragmented data appears and then is repaired on a disk. Each color is a different file; each box is a block.

The Filesystem and Files

The filesystem must keep certain information about each file. Each file has the following information, called metadata, which is about the file:

• file size: the filesystem much know how many bits of data are in any one particular file.
• location on the disk: the filesystem must keep track of where all the data in the file is located, so it can be accessed. Some files are divided into parts in different locations on the disk.
• dates associated with the file: the filesystem should know when the file was created, when it was last modified, and when it was backed up.

There are many, many other possible metadata; a common digital photograph might have dozens of kinds of metadata, including the camera name, the shutter speed, the aperture, the exposure levels, and more.

Very basic metadata is stored in a directory table. However, many types of metadata are stored within the file, and are carried with that file when it is moved to another computer. In fact, this can cause problems. For example, many years ago, when you posted an image on Facebook, the original image was displayed on the web site. However, that original image had the image's metadata, such as the date and time it was taken—but sometimes even the GPS coordinates were included!

This caused severe privacy issues. For example, if a parent took a photo of their child at school, at home, or in a playground, the metadata would show exactly where and when the child was. This could allow criminals to know where to find the child at various times of the day.

As a result, Facebook now removes the metadata from files before displaying them.

Why Else Should You Care? Deleting Files, That's Why.

This is important to know because of how your computer erases (or does not erase) data.

Consider how your computer saves and deletes data. If you save a 3 GB file on your hard drive, it may take 1 minute or longer for the copy to be completed. That's the amount of time necessary to write all the 1s and 0s in the file.

However, when you empty your recycle bin / trash, the file seems to disappear almost instantly.

If it took more than a minute to write the 1s and 0s, why did it take no time to delete the 1s and 0s?

The answer is simple: the data was not really deleted! The filesystem is designed to prevent accidental data loss. As a result, the computer tries to not delete the information until it is necessary.

At first, all the filesystem does is mark the file for deletion. In one example, the filesystem goes into the metadata in the directory table, and replaces the first character of the filename with a "sigma" (σ or Σ).

The file's data is still there. The metadata is still there. All the computer did was to tag the file, essentially saying, "This space is free to use, but we'll keep the data as long as we can."

In some filesystems, the data can easily be recovered. In other filesystems, you must use special recovery software to get the data back.

As time goes by, and you need more disk space, the filesystem will eventually use the file's space, over-writing the data. When that happens, then the data is erased.

It is important to understand that data still exists on a computer even after the files have been "erased."

How Can I REALLY Get Rid of the Data?

Some people imagine that all they have to do is re-format the disk. This could work, but only if you do it the right way. Sometimes, reformatting the disk only takes a few moments. If that is the case, then the data is not being erased!

Data is only truly erased when it is over-written, when you write new data on top of the old data. This is done automatically by writing different data in the same location, but that might take some time. Another way to truly erase data is to zero out the data's location: to write all zeroes in the data location.

To zero out a whole hard drive could take hours. Therefore, if you re-formatted the drive in just a few moments, the data was clearly not erased. The computer just erased the filesystem and created a new one. But the data is still on the disk.

Have you ever sold or given away a computer you owned to someone else without zeroing out the hard drive? If you did, then that person could have uncovered all your old files—even files you thought you had erased months before!

Neither macOS nor Windows has a zero-out feature built in, but you can get various utilities that let you do this for Windows or Mac.

What If I Accidentally Erase Data?

People sometimes delete files and folders without meaning to, or sometimes it happens because of a bug in the system. I once had my entire User account folder disappear!

How can you get it back once it has disappeared?

Well, the good news is, if you did not go to the trouble of zeroing out the data, it is still there on the disk. However, you may need to go to some trouble to recover it.

The best solution is if you use backup software so that all of the data on your computer is saved, usually to an external hard drive. Such software is included in your operating system. In Windows, it is simply called Backup, and on a Mac, it is called Time Machine. These are the best solutions—and they can help save you from ransomware attacks! Just follow the instructions given for your operating system, and when you accidentally delete a file or folder, use your backup to restore it.

If you do not have backups, then you may need to use Recovery software. There are several programs available, but they tend to be a bit pricey.

If you accidentally lose or erase data from your main hard drive and it is gone from the recycle bin or trash, you should immediately stop using your computer until you can set up a recovery solution. Recovery software is often installed onto an alternate disk drive, because you cannot reliably recover data from the same disk that the recovery software is on. If you lose data from an external drive, then remove the drive and do not write any new data on it until you can run the recovery software.

Using recovery software can be difficult; very often, files are incomplete, are mislabeled, or have no labels at all. You may be presented with a list of hundreds, or even thousands of files, and may have to sort through most or all of them to recover the most important data. It can be a very tiresome task.

That's why it is much better to back up! Buy an external drive (maybe something like one of these), and then back up your files every week or so.

Partitions

A filesystem can create something called a disk partition.

A partition can divide one volume so it appears to be two or more partitions.

In this case, we can take a USB flash drive (which is a volume) and create partitions.

As you can see above, there is a flash drive called "USB Disk." It is a 4GB volume with a single partition, using the FAT filesystem.

In this case, I want two partitions. Under "Partition Layout," I choose "2 Partitions," at which time I can adjust the size of either one. Note that I have chosen a filesystem better for the Mac:

I can make various kinds of adjustments. For example, I can set up the device so it can be a "boot" volume—that is, I could install an operating system on it and use it to start up a computer (as an emergency disk).

Once I am finished, the computer confirms what I have set up:

After I have finished, the one USB flash device now appears as two volumes on the computer!

You might wonder, why do this? There are many possible reasons. Maybe you want one partition formatted best for Windows, and the other formatted for Mac. On a hard drive, you might divide the disk so that one partition is running Windows 7 in English, and the other partition is running Windows 8 in Japanese (having two operating systems on separate partitions is called dual boot).

Another use is for a restore partition, used to keep the operating system installer.