C Shell Build in Command Read Data From File

Working With Files and Directories

Overview

Teaching: 30 min
Exercises: 20 min

Questions

• How can I create, copy, and delete files and directories?

• How can I edit files?

Objectives

• Create a directory hierarchy that matches a given diagram.

• Create files in that hierarchy using an editor or past copying and renaming existing files.

• Delete, copy and move specified files and/or directories.

Creating directories

We at present know how to explore files and directories, but how do we create them in the first place?

Footstep one: run across where nosotros are and what we already accept

Let'south get back to our vanquish-lesson-data directory on the Desktop and use ls -F to see what it contains:

            /Users/nelle/Desktop/trounce-lesson-data                      

            creatures/  molecules/           notes.txt    pizza.cfg  writing/ data/       northward-pacific-gyre/  numbers.txt  solar.pdf                      

Create a directory

Let'due south create a new directory called thesis using the control mkdir thesis (which has no output):

As yous might guess from its name, mkdir ways 'make directory'. Since thesis is a relative path (i.e., does not have a leading slash, like /what/ever/thesis), the new directory is created in the electric current working directory:

            creatures/  molecules/           notes.txt    pizza.cfg  thesis/ data/       north-pacific-curl/  numbers.txt  solar.pdf  writing/                      

Since we've merely created the thesis directory, in that location's aught in it yet:

Annotation that mkdir is not limited to creating unmarried directories one at a fourth dimension. The -p option allows mkdir to create a directory with nested subdirectories in a single performance:

                          $                            mkdir              -p              project/data project/results                      

The -R choice to the ls command volition list all nested subdirectories inside a directory. Allow's utilize ls -FR to recursively list the new directory hierarchy we just created in the project directory:

            project/: data/  results/  projection/information:  project/results:                      

Ii ways of doing the same thing

Using the beat out to create a directory is no different than using a file explorer. If you open the current directory using your operating arrangement's graphical file explorer, the thesis directory will appear there too. While the shell and the file explorer are two different ways of interacting with the files, the files and directories themselves are the same.

Skilful names for files and directories

Complicated names of files and directories can brand your life painful when working on the control line. Here we provide a few useful tips for the names of your files and directories.

1. Don't use spaces.

   Spaces can make a name more meaningful, but since spaces are used to split arguments on the command line information technology is better to avoid them in names of files and directories. You tin apply - or _ instead (e.g. due north-pacific-gyre/ rather than north pacific gyre/). To test this out, effort typing mkdir north pacific ringletand see what directory (or directories!) are made when you cheque with ls -F.

2. Don't begin the name with - (nuance).

   Commands treat names starting with - as options.

3. Stick with letters, numbers, . (period or 'full stop'), - (dash) and _ (underscore).

   Many other characters have special meanings on the control line. We will learn near some of these during this lesson. There are special characters that can crusade your control to not work every bit expected and can even result in information loss.

If y'all demand to refer to names of files or directories that have spaces or other special characters, you should surround the name in quotes ("").

Create a text file

Let's change our working directory to thesis using cd, and so run a text editor called Nano to create a file chosen draft.txt:

                          $                            cd              thesis              $              nano draft.txt                      

Which Editor?

When we say, 'nano is a text editor' we really do mean 'text': information technology can only work with plain character data, not tables, images, or any other human-friendly media. We use it in examples considering it is i of the least complex text editors. However, considering of this trait, it may non exist powerful enough or flexible enough for the work yous need to do later this workshop. On Unix systems (such every bit Linux and macOS), many programmers use Emacs or Vim (both of which crave more than fourth dimension to learn), or a graphical editor such as Gedit. On Windows, you may wish to use Notepad++. Windows also has a built-in editor called notepad that tin can be run from the command line in the same mode as nano for the purposes of this lesson.

No affair what editor you apply, yous will demand to know where information technology searches for and saves files. If you start it from the crush, information technology will (probably) use your current working directory as its default location. If you apply your computer'due south beginning carte, it may want to salve files in your desktop or documents directory instead. You can alter this by navigating to another directory the first time you 'Salvage As…'

Permit'due south type in a few lines of text. Once we're happy with our text, we can press Ctrl+O (printing the Ctrl or Control key and, while holding information technology downwards, printing the O central) to write our data to disk (we'll be asked what file we want to salvage this to: press Render to accept the suggested default of draft.txt).

Once our file is saved, we tin use Ctrl+X to quit the editor and render to the shell.

Control, Ctrl, or ^ Key

The Control key is also called the 'Ctrl' key. There are various ways in which using the Control primal may be described. For example, you lot may see an pedagogy to printing the Control key and, while holding it down, printing the X fundamental, described as any of:

• Command-10
• Control+10
• Ctrl-X
• Ctrl+X
• ^X
• C-x

In nano, along the bottom of the screen yous'll see ^G Get Help ^O WriteOut. This means that yous tin can utilize Control-1000 to get help and Control-O to save your file.

nano doesn't leave whatsoever output on the screen afterward it exits, but ls now shows that we accept created a file chosen typhoon.txt:

Creating Files a Different Way

We have seen how to create text files using the nano editor. Now, try the post-obit command:

1. What did the touch command do? When y'all look at your current directory using the GUI file explorer, does the file show up?

2. Use ls -l to audit the files. How large is my_file.txt?

3. When might you want to create a file this way?

Solution

1. The touch control generates a new file called my_file.txt in your current directory. Yous can observe this newly generated file by typing ls at the command line prompt. my_file.txt tin can likewise be viewed in your GUI file explorer.

2. When you inspect the file with ls -50, note that the size of my_file.txt is 0 bytes. In other words, it contains no data. If y'all open my_file.txt using your text editor information technology is blank.

3. Some programs practise non generate output files themselves, but instead crave that empty files have already been generated. When the program is run, information technology searches for an existing file to populate with its output. The affect command allows you to efficiently generate a blank text file to be used by such programs.

What's In A Name?

Yous may have noticed that all of Nelle's files are named 'something dot something', and in this role of the lesson, we e'er used the extension .txt. This is but a convention: we can telephone call a file mythesis or almost anything else nosotros want. However, most people use two-part names most of the fourth dimension to help them (and their programs) tell different kinds of files apart. The second function of such a name is called the filename extension and indicates what type of data the file holds: .txt signals a plainly text file, .pdf indicates a PDF document, .cfg is a configuration file full of parameters for some program or other, .png is a PNG prototype, and then on.

This is just a convention, albeit an important one. Files contain bytes: it's up to u.s. and our programs to interpret those bytes co-ordinate to the rules for plain text files, PDF documents, configuration files, images, and and so on.

Naming a PNG paradigm of a whale equally whale.mp3 doesn't somehow magically plow information technology into a recording of whale song, though information technology might cause the operating system to try to open it with a music player when someone double-clicks information technology.

Moving files and directories

Returning to the shell-lesson-data directory,

                          $                            cd              ~/Desktop/shell-lesson-data/                      

In our thesis directory nosotros accept a file draft.txt which isn't a peculiarly informative proper name, then permit'due south change the file's proper noun using mv, which is brusk for 'move':

                          $                            mv              thesis/draft.txt thesis/quotes.txt                      

The first statement tells mv what nosotros're 'moving', while the 2nd is where it's to go. In this case, we're moving thesis/draft.txt to thesis/quotes.txt, which has the same effect as renaming the file. Certain enough, ls shows united states that thesis at present contains one file called quotes.txt:

I must be careful when specifying the target file name, since mv will silently overwrite any existing file with the same name, which could lead to data loss. An additional option, mv -i (or mv --interactive), can be used to make mv ask you for confirmation before overwriting.

Note that mv as well works on directories.

Allow'south move quotes.txt into the electric current working directory. We apply mv once again, simply this time nosotros'll employ just the proper name of a directory every bit the second argument to tell mv that we want to keep the filename but put the file somewhere new. (This is why the command is called 'move'.) In this case, the directory name we employ is the special directory proper name . that nosotros mentioned earlier.

The effect is to move the file from the directory it was in to the current working directory. ls now shows us that thesis is empty:

Alternatively, we can confirm the file quotes.txt is no longer nowadays in the thesis directory by explicitly trying to list it:

            ls: cannot access 'thesis/quotes.txt': No such file or directory                      

ls with a filename or directory every bit an argument simply lists the requested file or directory. If the file given as the argument doesn't be, the crush returns an error as we saw above. Nosotros can utilise this to see that quotes.txt is at present nowadays in our current directory:

Moving Files to a new folder

After running the following commands, Jamie realizes that she put the files sucrose.dat and maltose.dat into the wrong folder. The files should have been placed in the raw binder.

                              $                                ls                -F                analyzed/ raw/                $                                ls                -F                analyzed fructose.dat glucose.dat maltose.dat sucrose.dat                $                                cd                analyzed                          

Fill in the blanks to move these files to the raw/ folder (i.e. the one she forgot to put them in)

                              $                                mv                sucrose.dat maltose.dat ____/____                          

Solution

                                  $                                    mv                  sucrose.dat maltose.dat ../raw                              

Recall that .. refers to the parent directory (i.eastward. one in a higher place the electric current directory) and that . refers to the current directory.

Copying files and directories

The cp control works very much like mv, except information technology copies a file instead of moving it. We can check that information technology did the right thing using ls with 2 paths equally arguments — similar most Unix commands, ls tin be given multiple paths at once:

                          $                            cp              quotes.txt thesis/quotations.txt              $                            ls              quotes.txt thesis/quotations.txt                      

            quotes.txt   thesis/quotations.txt                      

We tin also copy a directory and all its contents by using the recursive option -r, due east.k. to back up a directory:

                          $                            cp              -r              thesis thesis_backup                      

Nosotros tin can check the result by list the contents of both the thesis and thesis_backup directory:

                          $                            ls              thesis thesis_backup                      

            thesis: quotations.txt  thesis_backup: quotations.txt                      

Renaming Files

Suppose that y'all created a plain-text file in your current directory to contain a list of the statistical tests you volition need to do to analyze your data, and named it: statstics.txt

After creating and saving this file y'all realize you misspelled the filename! You desire to right the mistake, which of the following commands could you use to practise then?

1. cp statstics.txt statistics.txt
2. mv statstics.txt statistics.txt
3. mv statstics.txt .
4. cp statstics.txt .

Solution

1. No. While this would create a file with the correct name, the incorrectly named file still exists in the directory and would need to be deleted.
2. Yes, this would work to rename the file.
3. No, the period(.) indicates where to move the file, but does not provide a new file proper name; identical file names cannot exist created.
4. No, the menses(.) indicates where to re-create the file, simply does not provide a new file name; identical file names cannot be created.

Moving and Copying

What is the output of the closing ls command in the sequence shown below?

                              $                                mkdir                recombined                $                                mv                proteins.dat recombined/                $                                cp                recombined/proteins.dat ../proteins-saved.dat                $                                ls                          

1. proteins-saved.dat recombined
2. recombined
3. proteins.dat recombined
4. proteins-saved.dat

Solution

We start in the /Users/jamie/data directory, and create a new folder called recombined. The 2d line moves (mv) the file proteins.dat to the new folder (recombined). The 3rd line makes a copy of the file we simply moved. The tricky part hither is where the file was copied to. Recall that .. means 'go up a level', so the copied file is now in /Users/jamie. Notice that .. is interpreted with respect to the current working directory, not with respect to the location of the file beingness copied. So, the only thing that will testify using ls (in /Users/jamie/data) is the recombined folder.

1. No, run across explanation above. proteins-saved.dat is located at /Users/jamie
2. Yes
3. No, come across explanation above. proteins.dat is located at /Users/jamie/data/recombined
4. No, see explanation above. proteins-saved.dat is located at /Users/jamie

Removing files and directories

Returning to the beat-lesson-information directory, let'southward tidy upwards this directory by removing the quotes.txt file we created. The Unix control we'll employ for this is rm (short for 'remove'):

We tin ostend the file has gone using ls:

            ls: cannot access 'quotes.txt': No such file or directory                      

Deleting Is Forever

The Unix shell doesn't have a trash bin that we can recover deleted files from (though most graphical interfaces to Unix practice). Instead, when we delete files, they are unlinked from the file system so that their storage infinite on deejay can be recycled. Tools for finding and recovering deleted files do exist, but there'due south no guarantee they'll work in any particular situation, since the calculator may recycle the file'southward disk infinite right abroad.

Using rm Safely

What happens when we execute rm -i thesis_backup/quotations.txt? Why would we desire this protection when using rm?

Solution

                rm: remove regular file 'thesis_backup/quotations.txt'? y                              

The -i option will prompt earlier (every) removal (utilize Y to confirm deletion or N to keep the file). The Unix vanquish doesn't have a trash bin, so all the files removed will disappear forever. By using the -i choice, we take the chance to check that we are deleting only the files that we desire to remove.

If we try to remove the thesis directory using rm thesis, we get an error message:

            rm: cannot remove `thesis': Is a directory                      

This happens because rm by default only works on files, non directories.

rm can remove a directory and all its contents if we use the recursive option -r, and it will practise so without any confirmation prompts:

Given that there is no manner to retrieve files deleted using the trounce, rm -r should be used with great caution (y'all might consider calculation the interactive choice rm -r -i).

Operations with multiple files and directories

Oftentimes one needs to re-create or move several files at once. This tin can be done by providing a listing of individual filenames, or specifying a naming design using wildcards.

Copy with Multiple Filenames

For this do, you can exam the commands in the beat-lesson-data/information directory.

In the example beneath, what does cp do when given several filenames and a directory name?

                              $                                mkdir                fill-in                $                                cp                amino-acids.txt animals.txt backup/                          

In the example beneath, what does cp do when given three or more than file names?

              amino-acids.txt  animals.txt  elements/  pdb/         salmon.txt animate being-counts/   backup/      morse.txt  planets.txt  sunspot.txt                          

                              $                                cp                amino-acids.txt animals.txt morse.txt                          

Solution

If given more than ane file proper name followed by a directory proper noun (i.e. the destination directory must be the last argument), cp copies the files to the named directory.

If given 3 file names, cp throws an mistake such as the one below, because it is expecting a directory name as the last argument.

                cp: target 'morse.txt' is not a directory                              

Using wildcards for accessing multiple files at one time

Wildcards

* is a wildcard, which matches zip or more characters. Let'southward consider the shell-lesson-data/molecules directory: *.pdb matches ethane.pdb, propane.pdb, and every file that ends with '.pdb'. On the other paw, p*.pdb merely matches pentane.pdb and propane.pdb, because the 'p' at the front only matches filenames that begin with the letter 'p'.

? is also a wildcard, just it matches exactly i character. So ?ethane.pdb would match marsh gas.pdb whereas *ethane.pdb matches both ethane.pdb, and marsh gas.pdb.

Wildcards can exist used in combination with each other east.g. ???1.pdb matches three characters followed by one.pdb, giving cubane.pdb ethane.pdb octane.pdb.

When the shell sees a wildcard, it expands the wildcard to create a list of matching filenames before running the command that was asked for. As an exception, if a wildcard expression does not friction match whatever file, Bash will pass the expression as an argument to the control as information technology is. For example, typing ls *.pdf in the molecules directory (which contains only files with names catastrophe with .pdb) results in an fault message that there is no file called *.pdf. Even so, generally commands like wc and ls see the lists of file names matching these expressions, just not the wildcards themselves. It is the shell, not the other programs, that deals with expanding wildcards.

Listing filenames matching a pattern

When run in the molecules directory, which ls control(s) will produce this output?

ethane.pdb methane.pdb

1. ls *t*ane.pdb
2. ls *t?ne.*
3. ls *t??ne.pdb
4. ls ethane.*

Solution

The solution is iii.

i. shows all files whose names contain zero or more characters (*) followed by the letter t, then null or more characters (*) followed by i.pdb. This gives ethane.pdb methane.pdb octane.pdb pentane.pdb.

2. shows all files whose names outset with zero or more characters (*) followed by the letter t, then a unmarried graphic symbol (?), then ne. followed by zilch or more characters (*). This will give united states of america octane.pdb and pentane.pdb only doesn't match anything which ends in thane.pdb.

three. fixes the problems of option ii by matching two characters (??) between t and ne. This is the solution.

iv. only shows files starting with ethane..

More on Wildcards

Sam has a directory containing calibration information, datasets, and descriptions of the datasets:

                              .                ├── 2015-10-23-calibration.txt ├── 2015-10-23-dataset1.txt ├── 2015-10-23-dataset2.txt ├── 2015-ten-23-dataset_overview.txt ├── 2015-10-26-calibration.txt ├── 2015-x-26-dataset1.txt ├── 2015-ten-26-dataset2.txt ├── 2015-10-26-dataset_overview.txt ├── 2015-eleven-23-calibration.txt ├── 2015-11-23-dataset1.txt ├── 2015-11-23-dataset2.txt ├── 2015-11-23-dataset_overview.txt ├── fill-in │   ├── calibration │   └── datasets └── send_to_bob     ├── all_datasets_created_on_a_23rd     └── all_november_files                          

Before heading off to another field trip, she wants to back up her data and send some datasets to her colleague Bob. Sam uses the following commands to get the job washed:

                              $                                cp                *dataset*                backup/datasets                $                                cp                ____calibration____ backup/calibration                $                                cp                2015-____-____ send_to_bob/all_november_files/                $                                cp                ____ send_to_bob/all_datasets_created_on_a_23rd/                          

Help Sam past filling in the blanks.

The resulting directory structure should await like this

                              .                ├── 2015-x-23-scale.txt ├── 2015-x-23-dataset1.txt ├── 2015-10-23-dataset2.txt ├── 2015-10-23-dataset_overview.txt ├── 2015-10-26-scale.txt ├── 2015-x-26-dataset1.txt ├── 2015-10-26-dataset2.txt ├── 2015-10-26-dataset_overview.txt ├── 2015-11-23-calibration.txt ├── 2015-xi-23-dataset1.txt ├── 2015-11-23-dataset2.txt ├── 2015-11-23-dataset_overview.txt ├── backup │   ├── calibration │   │   ├── 2015-10-23-scale.txt │   │   ├── 2015-10-26-calibration.txt │   │   └── 2015-11-23-calibration.txt │   └── datasets │       ├── 2015-10-23-dataset1.txt │       ├── 2015-ten-23-dataset2.txt │       ├── 2015-10-23-dataset_overview.txt │       ├── 2015-10-26-dataset1.txt │       ├── 2015-10-26-dataset2.txt │       ├── 2015-ten-26-dataset_overview.txt │       ├── 2015-11-23-dataset1.txt │       ├── 2015-11-23-dataset2.txt │       └── 2015-11-23-dataset_overview.txt └── send_to_bob     ├── all_datasets_created_on_a_23rd     │   ├── 2015-x-23-dataset1.txt     │   ├── 2015-ten-23-dataset2.txt     │   ├── 2015-ten-23-dataset_overview.txt     │   ├── 2015-11-23-dataset1.txt     │   ├── 2015-11-23-dataset2.txt     │   └── 2015-eleven-23-dataset_overview.txt     └── all_november_files         ├── 2015-11-23-scale.txt         ├── 2015-xi-23-dataset1.txt         ├── 2015-11-23-dataset2.txt         └── 2015-xi-23-dataset_overview.txt                          

Solution

                                  $                                    cp                  *calibration.txt backup/calibration                  $                                    cp                  2015-11-*                  send_to_bob/all_november_files/                  $                                    cp                  *                  -23-dataset                  *                  send_to_bob/all_datasets_created_on_a_23rd/                              

Organizing Directories and Files

Jamie is working on a project and she sees that her files aren't very well organized:

              analyzed/  fructose.dat    raw/   sucrose.dat                          

The fructose.dat and sucrose.dat files contain output from her data analysis. What command(south) covered in this lesson does she need to run so that the commands below will produce the output shown?

Solution

Jamie needs to move her files fructose.dat and sucrose.dat to the analyzed directory. The shell will expand *.dat to match all .dat files in the current directory. The mv command so moves the list of .dat files to the 'analyzed' directory.

Reproduce a folder construction

Y'all're starting a new experiment and would like to duplicate the directory structure from your previous experiment so you tin can add new data.

Presume that the previous experiment is in a binder called 2016-05-18, which contains a data binder that in turn contains folders named raw and processed that contain data files. The goal is to re-create the folder structure of the 2016-05-18 binder into a folder called 2016-05-20 and so that your final directory construction looks similar this:

              2016-05-20/ └── data    ├── processed    └── raw                          

Which of the following set of commands would achieve this objective? What would the other commands do?

                              $                                mkdir                2016-05-20                $                                mkdir                2016-05-20/data                $                                mkdir                2016-05-twenty/information/processed                $                                mkdir                2016-05-xx/data/raw                          

                              $                                mkdir                2016-05-twenty                $                                cd                2016-05-xx                $                                mkdir                data                $                                cd                information                $                                mkdir                raw processed                          

                              $                                mkdir                2016-05-twenty/information/raw                $                                mkdir                2016-05-twenty/data/processed                          

                              $                                mkdir                -p                2016-05-twenty/data/raw                $                                mkdir                -p                2016-05-twenty/information/processed                          

                              $                                mkdir                2016-05-20                $                                cd                2016-05-twenty                $                                mkdir                information                $                                mkdir                raw processed                          

Solution

The beginning ii sets of commands accomplish this objective. The first set uses relative paths to create the top-level directory before the subdirectories.

The third set of commands volition give an mistake because the default behavior of mkdir won't create a subdirectory of a non-existent directory: the intermediate level folders must be created offset.

The 4th prepare of commands attain this objective. Remember, the -p option, followed by a path of one or more directories, will cause mkdir to create any intermediate subdirectories every bit required.

The concluding set of commands generates the 'raw' and 'processed' directories at the aforementioned level every bit the 'data' directory.

Key Points

• cp [old] [new] copies a file.

• mkdir [path] creates a new directory.

• mv [old] [new] moves (renames) a file or directory.

• rm [path] removes (deletes) a file.

• * matches null or more characters in a filename, so *.txt matches all files ending in .txt.

• ? matches any unmarried character in a filename, and then ?.txt matches a.txt merely not any.txt.

• Use of the Command primal may exist described in many ways, including Ctrl-X, Control-Ten, and ^X.

• The shell does not have a trash bin: one time something is deleted, it's really gone.

• Most files' names are something.extension. The extension isn't required, and doesn't guarantee annihilation, but is normally used to indicate the type of data in the file.

• Depending on the blazon of work yous do, you lot may demand a more powerful text editor than Nano.

catoninfichice1998.blogspot.com

Source: https://swcarpentry.github.io/shell-novice/03-create/index.html

Share this post