Tutorial for bash, git, python

Search for documentation for any and all term you do not understand. If you find usefull links add them to the text below in markdown syntax, like this: [markdown syntax](<https://www.markdownguide.org/basic-syntax>)

Finding bash

Find the console app on your computer and start it. You will get a window displaying lines of characters. There will be a command prompt and behind that the text cursor where you are expected to type commandlines. The app is running the programm bash , which is patiently waiting for you to type a command and run it for you.

Finding where you are

Type the commandline pwd . Bash will run the program /bin/pwd . The output will be printed into the window, followed by a new prompt. The program will Print your current Working Directory.

Any filename you type or that a program you start tries to open will be searched in (relative to) this current working directory. Unless the filename is typed with a full path, starting with a / .

Kill a program

Run the command cat. You observe that you will not get a new prompt after entering that command. That is because the command is still running. It waits for your input. Type hello and «enter». The program will print what you entered. Type «Ctrl-D», which means End Of File. The programm will terminate and you get a new prompt.

Start the program cat again. This time you type «Ctrl-C» which means Interrupt. The programm will be kindly asked to terminate. Most programs will obey such requests.

Create a project directory

Use the command mkdir to MaKe a new DIRectory. Look into the manual of that program to figure out how. You can read the MANual by typing the man mkdir. Type q when you are done reading.

You need to think of a nice name for that directory.

When you succeed, you will have a new empty directory somewhere where you choose to create it. Use the command cd to Change your current working Directory to that new place. Use pwd to verify that you are indeed inside that new directory.

The first thing you do there is to run the command git init. We may come back to git later. If you do not have git installed (MacOS) get help installing it, or skip all git commands below. On Ubuntu or Debian, do: sudo apt install git if you have the necessary permissions. Or you postpone the git business until you learned how to do that on a remote computer.

Now you type ls -lA --color to verify that the current working directory contains nothing but a subdirectory named .git. Read the manual of the command ls to learn what that command does and what the options -l and -A mean. You use the program man as before. If you want to learn how to move around in the manual, read the manual of the programm less.

Create a README

Use the programm echo to print your name and a description of the project into the console window. Like:

• echo "Commandline tutorial by $USER"

When bash starts a program, it connects the input and output of that program to the console application window. You can instruct bash to connect those differently. For example, you want to connect the output of the program to a file, so that all output is written into that (new) file. If you use > the file already exists its contents will be overwritten:

• echo "Commandline tutorial by $USER" > README.txt

If you do not want to overwrite the file, but append the output of the program at the end, use >> like

• echo started on >> README.txt
• date >> README.txt

Verify the contents of your README using the programs cat and/or less.

When you try to type the name of the file, it is sufficient to type R followed by «Tab». bash will look for a unique filename beginning with R abd type the rest for you. Except for the first command above, when that file does not yet exist.

Commit your README into git

Run the command git status. The programm git will tell you that it does not know about the file README.txt. To change that, type

• git add README.txt
• git status
• git commit -m "new README"
• git status
• git log

git may complain that it does not know who you are. You must provide a name and an email. Follow the instructions.

bash remembers the commands you typed recently. Instead of typing them again, you can browse through the list with the arrow keys until you found something appropriate, optionally make changes to the line, and hit «enter» to run the program again. If the command you want to repeat was not so recently, you can type «Ctrl-R» to search for it.

Shell

bash is a shell. That name is from the origins of Unix. A shell is a program that is used to start other programs. A shell can start another shell. Type bash. Now you have two bash processes.

Type ps -f. The program ps prints a list of processes running on your computer. Without certain options, it only lists processes connected to your console application window. You will see two bash processes and one ps process. The column UID tells you whom those processes belong, that is you. The column PID is the Process IDentifier. The column PPID is the Parent Process ID. That is the process that started it. You will see that the second bash was created by the first one, and ps was started from the second.

Now type PS1="bash $$$ ". The prompt of the inner bash will be different now. Type exit to terminate the inner shell, you will be back to the outer shell with the old prompt. The outer shell enclosed the inner shell like a shell, or something.

Starting a shell from a shell like this i not very useful unless you want to temporarily configure a shell differently and then return to the old one.

It is not necessary for the inner shell to run on the same computer, though.

Remote shell

The program ssh provides a Secure remote SHell on a different computer.

You will have been provided with an account on the institute computers, and somebody will have helped you to install a public/private key pair of your computer to authenticate for this secure connection. Anything encrypted with one of the keys can only be decrypted with the other key. One key is public. The remote computer must know your public key and be told to let you in if you can prove that you know the private key. For that purpose the remote computer will give you a puzzle and your computer needs to encrypt that puzzle and return it to the remote computer. Before that happens the remote computer needs to prove its identity by solving a similar puzzle from your computer. The first time you connect to that particular computer your computer does not know its identity. You may know that is the first time and accept the identity. Next time that computer must show the same id. That is called TOFU, Trust On First Use.

The user name on the institute computers is probably not the same as your username at home. Let's assume your username is meatwork.. Substitute your real username.

• ssh meatwork@asterix.ieap.uni-kiel.de

If that succeeds you will be getting the prompt of a bash at the remote computer. Type

• hostname
• pwd
• whoami
• ls
• who

to find out about that new place. The last command tells you who else is using that computer right now.

Pushing your git

On the remote computer asterix, create a new directory with the same name as you used before for this project, change your working directory there and type pwd to make sure you got that right. Now type git init. You created a new empty git repository on asterix. Let us assume the directory is called tuti. Substitute as appropriate.

It is time to leave the remote computer for now. Type exit.

Back on you own computer, make sure your current working directory is the project you created. pwd should print something like /home/meathome/tuti.

Now we connect the git project at home with the git project at the institute. If you started the project at the institute, we could just git clone the remote git to your computer at home. The other direction is more difficult, because you cannot use ssh to connect from the institute to you home computer.

In git you have multiple versions of your project. The versions are developed in branches. You need to find out what the name of your branch is. It may be master or main, depending on the version of git used and how you reacted to the message that git init gave you. Type git branch to find out. Let us assume the branch is called master. Type:

• git remote add asterix meatwork@asterix.ieap.uni-kiel.de:tuti
• git push asterix master:home

If that succeeds, you configured a remote URL to the project directory on asterix. Your local branch master was then pushed to branch home in the remote repository. git will not allow you to push into the remote branch master because it is the working branch there.

Connect to asterix again. Eventually it is smarter to have two windows, one conected to the remote shell and one at home. You may also want to learn about ssh-add and ssh-agent to avoid typing the password of your key each time.

On asterix, cd to the project, and type

• git merge home
• ls -l Your changes to the branch home where applied to the branch master. You will now find your README in the remote project directory.

Now type

• date +'Pushed to asterix on %FT%T%Z' >> README.txt
• git status
• git diff

See how git observes that the file now has an extra line.

• git commit -am "Update to README"
• git status
• git log

There are now two commits. Go back to your own computer. There you type

• git pull asterix master
• cat README.txt

The new line should now also be in the local version.

The pushing, pulling, and cloning also works with project repositories on codeberg or the university gitlab server.

Moving files

In case you cannot use git at home, you will need to do that remotely. You omitted all the git commands at home, but you could still create a project directory on asterix. You copy your files into there and commit the changes. To copy files to another computer you use rsync or scp if the former is not available.

• rsync -pt README.txt meatwork@asterix.ieap.uni-kiel.de:tuti/
• scp -p README.txt meatwork@asterix.ieap.uni-kiel.de:tuti/

Then you login to asterix and commit the changes.

Hello World

Write a python script. Type

• echo '#! /usr/bin/python3' > world.py
• echo 'print("Hallo World")' >> world.py
• python3 world.py

You start the program python3 to read your script world.py and execute the python statements inside. You can transfer the file world.py into a program by itself. All that needs to be done is changing some permission bits in the metadata for the file.

• ls -l world.py
• chmod a+x world.py
• ls -l world.py

Observe how there are now x bits in the permissions, meaning that everybody is allowed to execute this file as a program. Now you can type

• ./world.py

This works because we put the hash bang line at the top of the file, to tell the operating system how to interpret this file as a program. It will not work if your python3 program is not at the location entered there. In that case you can find out where python3 is with the command which python3.

When the script works, check it in, so that others can review it.

• git status
• git add world.py
• git commit -m "Add script world.py"
• git push asterix master:home

Change your Program

Now you need to find some application to edit your script, so it becomes more useful. There are simple text editors, some not exactly simple as emacs or vim, some very basic like nano. There are python IDEs, Integrated Development Environments. Use whatever pleases you to edit the file world.py. Try to avoid an IDE that insists on poluting the project directory with its own files.

Use the python module getot or argparse to implement a commandline option -n NAME that allows to print a different greeting like print(f"Hello {name}!"). Do not forget to also add an option -h to explain the options and arguments of the program, argparse will provide infrastructure for that purpose. argparse is the recommended modern solution. getopt is the traditional, simpler and more transparent solution. Often, modern is also a lot less transparent. There may be reasons to prefer traditional.

Check in the new version and push it to the institute. Announce the change to get review and feedback.

Implement cat

Make a copy of your program world.py with the program cp. Name the copy cat.py or similar. Edit the copy, remove the print() statement. Iterate over fileinput.input(files) to read all lines from all files named on the commandline of from sys.stdin id no files are given. Print the lines to sys.stdout. Use the option parser from world.py to implement the options -n, --number, and --help described in man cat.

Process your data

Make a copy of your program cat.py. Name the copy appropriately for the work you need to do. Change the body of your code to do the required processing on the input lines. Add commandline options to the argument parser as appropriate for your needs.

During processing, print diagnostics to sys.stderr. Only the primary output of your processing is printed to sys.stdout. Add options -v, --verbose=n and -q, --quiet. Implement a variable verbosity with default value of one. The options -q, --quiet set the verbosity to zero. Each invocatiopn of the option -v increments verbosity. With --verbose=n set the verbosity to n. When verbosity is zero only emit serious error messages. When verbosity is one also emit warnings. With higher values of verbosity emit informational and debug messages as appropriate.

Print the output of the processing in text format to sys.stdout. Use options to select between different levels of processing to output. Do not emit graphical output on the screen unless explicitly seleted by an option.