What are some distributed shell implementations

Jörg Schilling Bourne Shell, Korn Shell, POSIX and further focus Fraunhofer


1 Jörg Schilling Bourne Shell, Korn Shell, POSIX and further focus on Fraunhofer

2 Old shells under UNIX Around 1970: Thompson Shell If goto as external commands, with lseek (0) No variables $ 0, $ 1, / etc / glob for wildchart characters in file names 1975 Mashey Shell as a patch on Thompson Shell (osh) 1976 first version des Bourne Shell also as Patch Shell as Language Variable Subst., Signal Mgt., control flow

3 shells based on BourneShell code Bourne Shell since 1976, 1979 first distributed version Bourne Shell 1981 #, [! ...], $ {var: [+? =] Val}, set Bourne Shell 1983 shift Bourne Shell 1984 functions, unset / echo / type, buitlin redirect Bourne Shell bit clean, getopts, recursive functions Bourne Shell 1989 Job control Bourne Shell 2006 Schily Bourn Shell (bosh) many extensions Korn Shell 1983 based on the Bourne Shell Source Korn Shell 1986 with built-in history editor Korn Shell 1988 expanded version, noticeable changes

4 shells based on Bourne / Kon Shell code Korn Shell 1993 New version Further changes Korn Shell 1997 first version with source Korn Shell 2001 first real open source version Dtksh Korn Shell '93 with CDE builtins can build GUI as shell script Tksh Korn Shell '93 with TCL / TK builtins, GUI can build as a shell script

5 shells based on the BourneShell syntax Bash 1989 Ash (Almquist Shell) 1989 BSD replacement for BourneShell Dash (Debian Almquist Shell) 2004 Pdksh (Public Domain ksh) 1988 MirKorn Shell mksh (pdksh variant) 2003 Zsh (as zsh not Bourne Shell / POSIX Compatible) Busybox by Bruce Perens Many builtins, similar to Bourne Shell based on ash, but only emerged after ksh with many builtins

6 differences Bourne Shell, Schily, ksh init scripts, e.g. $ HOME / .kshrc for interactive shells pipe structure order and parent child relationship variables assignment order variable assignment (lifetime) before builtins syntax error handling with builtins history editor aliases umask parameters S / chmod like mask $ PWD and other variables

7 differences Bourne Shell, Schily, ksh $ {. Sh. *} Variables built-in time measurements pushd / popd / dirs (only bosh) dosh builtin (only bosh) export / readonly features Killpg (only bosh) set o longopt test features ~ (tilde Expansion)

8 basic features Bourne Shell for name [in word ...] do list done case word in [pattern [pattern]) list ;; ] ... esac if list then list [elif list then list] ... [else list] fi while list do list done until list do list done (list) {list;} name () {list;}

9 Syntax extensions ksh for (([expr1]; [expr2]; [expr3])); do list; done (not POSIX) select vname [in word ...]; do list; done (not POSIX) ((expression) ) (not POSIX) [[expression]] (not POSIX) time [pipeline] (not POSIX as part of the ksh syntax)! cmd $ (cmd) $ ((expression))

10 init scripts Bourne Shell: / etc / profile + ~ / .profile at login Bosh: / etc / profile + ~ / .profile (login), /etc/sh.rc + /.shrc (i) Ksh93: / etc / profile + ~ / .profile (login), /etc/ksh.kshrc + ~ / .kshrc (i) For all: no .logout script, as there is a trap cmd EXIT

11 Pipe structure, parent-child relationship Command interpretation not specified in the standard Simple example: cmd1 cmd2 Bourne Shell: cmd1 is a child of cmd2, cmd2 is a child of sh Ksh93: both commands are children of ksh Example with Builtin: cmd1 read var Bourne Shell: read in the child of sh, cmd1 is the child of read Ksh93: read runs in the ksh, cmd is the child of ksh Schily Bourne Shell: similar to ksh93 Consequences? Answer from the audience

12 Variable assignments var1 = val1 var2 = val2 cmd command with temporary (private) environment variables No problem in the static case ... But: var1 = val1 var2 = $ var1 cmd With the Bourne shell, var2 is empty here! Fix is ​​in ksh and bosh

13 Temporary variables + builtin commands var = val cmd creates a child that assigns var and calls cmd CDPATH = val cd is obvious but problematic cd is a built-in command and must be The variable assignment is done in the shell itself. With the Bourne shell, all variable assignments have an effect Builtin commands on the whole shell after With ksh and bosh this is prevented with most commands. Exception: POSIX special built-ins POSIX: with special built-ins, vars enter the shell

14 Nomenclature of builtins Some commands have to be implemented in the shell in order to work (e.g. cd) The original Bourne shell had few builtins Later, many were added.Unusual behavior is not acceptable for commands that you cannot tell that they are implemented in the shell (e.g. that ksh builtin uname) POSIX introduces special builtins, different behavior OK List of special builtins does not contain cd, alias, New SUSv7 tc2: intrinsic commands without different behavior

15 Syntax error handling with builtins Changes to the shell to make the behavior understandable: External commands <> Builtins With external commands, each error leads to a maximum of one exit code! = 0 Classic method in the Bourne shell: fatal errors in builtins lead to longjmp in the interactive shell () before prompt output, otherwise (if not interactive) to exit Ksh: extensive alignment of the builtins with external commands except special builtins Bosh: similar to ksh POSIX: command no special behavior.

16 History The original Bourne Shell has no history Ksh: Originally built-in extensions for list processing in the Bourne Shell, therefore no history Ksh from ca History can be edited by external vi using the fc command. Later from 1986: built-in history editor in ksh Schily Bourne Shell (bosh) from 2006 with the history editor bsh (Berthold) concept / prototype 1982, first implementation under UNOS in bsh emulated: ved

17 Aliases Bourne Shell originally without aliases Ksh introduces aliases similar to csh around 1984 Schily Bourne Shell: from 2012 aliases similar to the command interpreter from UNOS (1980) with persistent alises and directory-specific aliases (POSIX ++) Ksh: Parameterized aliases as with csh are not possible Bosh: Parameterized aliases using the dosh builtin that emulates mini shell scripts

18 export / readonly The Bourne Shell understands export VAR or readonly VAR Ksh, bosh and POSIX also allow: as well as: export VAR = value ... readonly VAR = value ... export -p readonly -p To get an output, which can be used again as a shell command export -p> file. file

19 umask The builtin umask in the Bourne Shell only understands octal values. Natch part: The mask is the inverse of the access rights and is therefore difficult to understand. Ksh introduces umask -S, output like chmod, similar to ls Bosh follows the example, because it is also supported by POSIX required Umask u = rwx, g = rx, o = rx is also possible as a replacement for umask 022

20 Shell Variables The original Bourne Shell knows only a few shell variables: SHELL, PATH, HOME, IFS, MAIL, MAILCHECK, MAILPATH, PS1, PS2, SHACCT Ksh introduces many new variables, some of which are carried over into POSIX: ENV, LINENO, PPID, PS4, PWD Bosh takes over all POSIX extensions Ksh and bosh: TIMEFORMAT for time formatting Bosh: additionally built-in timing for all commands with set -o time

21 Extended Variables Bourne Shell: Use of variables that start with a dot lead to a longjmp () in front of the interactive prompt, or to the exit of the whole shell Ksh and bosh: $ {. Sh. *} Variables for special purposes Bosh: Introduction of .sh. * to deliver 32 bit exit code All other shells: only the lower 8 bits of the children's exit code are visible although waitid () has existed since 1989

22 Time measurements Classic UNIX method / usr / bin / time command Ksh implements time as part of the syntax Bosh implements built-in timing for all commands and time as reserved word like ksh Ksh and bosh deactivate the built-in time when it is called with an option

23 Directory management Bourne Shell (historical): only cd is available Bourne Shell 1989 from Svr4: CDPATH variable is supported Ksh: similar to Bourne Shell from Svr4 but cd is logical and options L / P (default is L) Bosh: additionally pushd / popd and dirs manage a directory stack options L / P (default is P)

24 Parameterized Alises Csh understands: alias lm 'ls -l \! * More' \! * Is expanded by the current argument list Bourne Shell knows no aliases Ksh only understands simple aliases without parameters Bosh knows the builtin dosh, which is from the UNOS command Interpreter from 1980: eg: alias lm = 'dosh' \ '' ls -l more '\' 'lm' A one-line built-in script dosh enables the parameterization

25 Editing complex aliases in an understandable way As can be seen in the previous example: the ksh / posix syntax for aliases can become illegible Bosh therefore knows a method for editing aliases in RAW mode: alias -R Displays aliases in RAW mode Our example shown in RAW mode: #pb lm suddenly becomes readable dosh 'ls -l more' lm To edit: set -o switch on hashcmds Then enter #lh lm and cursor up ...

26 Long options The Bourne Shell knows only a few options such as -v Ksh introduces long options and POSIX takes over listing the long options with: set -o / set + o Instead of switching set -v on now also with set -o verbose Bosh does this, since it is in POSIX Bosh can also use long options with getopts (1) e.g. long as an alias to l with optstring: l (long) but also long without a corresponding short option and long if optstring begins with ()

27 Test Features Since test has been a built-in command since the mid-1980s, the shell determines the features of test Bourne Shell had hardly any new features built in Ksh knows a lot more (e.g. e file) so be careful if it is to be portable Bosh knows all ksh test features

28 Abbreviations for the home directory Bourne Shell only knows $ HOME Ksh also introduces ~ / own home directory (POSIX) ~ joe / home directory from user joe (POSIX) ~ + current directory ($ PWD) ~ previous directory ($ OLDPWD) Bosh implemented this like ksh

29 Ksh for loop The ksh extension for ((i = 0; i <10, i ++)); do list; done is available in Bash but not in POSIX, therefore not in Bosh

30 Select command The select command of the ksh is not in POSIX but in many other ksh alike shells Bourne shell does not know it Bosh does not yet know it

31 Expressions at any position ((expression)) is in ksh and bash but not POSIX Bourne shell does not know it. Other POSIX compatible shells have: $ ((expression)) Warning: ((expression)) of ksh is part of the syntax $ ((expression)) is a macro expansion POSIX: $ ((expression)) or $ ((cmd)), ksh recognizes it by the context (arithmetic vs. sub shell)

32 Test extensions [[expression]] is in ksh and bash but not POSIX Bourne Shell does not know it With other POSIX compatible shells there are: test and [Caution: [[expression]] of ksh is part of the syntax [expression] is a built-in command

33! cmd! cmd Negates the exit code (POSIX) Therefore available in many shells. Not in the Bourne shell but in Bosh

34 Command replacement Bourne Shell knows: `cmd` to include the output of cmd in the command line This, however, is nested as complex as the ksh alias definitions Ksh therefore introduced $ (cmd) and POSIX takes over This is very difficult to parse and required a recursive parser correctly implemented only in ksh93, bosh, bash 4, mksh

35 Arithmetic replacement Not available in the Bourne Shell Ksh: $ ((expression)) in the command line is replaced Not yet available in bosh, but soon ... Warning spaces between the brackets, if a command replacement in a subshell is meant. Ksh makes plausibility considerations, however this is not portable and cannot find everything: $ ((ls)) is a valid arithmetic expression! (0)

36 Further ksh extensions When the Bourne Shell was created in 1979, there was no pattern matching in libc. The Bourne Shell uses its own function gmatch () which * ,? and [...] supported It is used for filenames and for case Ksh it is now partly regular expressions Ksh93 supported ** for an entire directory tree, if the shell option G (also set o globstar) is set Ksh knows many string processing functions

37 Further ksh extensions Ksh knows many string processing functions: $ {parameter% word} Remove smallest suffix $ {parameter %% word} Remove largest suffix $ {parameter # word} Remove smallest prefix $ {parameter ## word} Remove largest prefix $ {# parameter} Number of letters in parameter and other non POSIX functions for substrings

38 Exercise 1 Explain why $ shell -c 'date read VAR; echo $ VAR 'Outputs the date only with $ shell == ksh, bosh and zsh, but not with bash, dash, sh Develop a command that assigns the date to the variable VAR for all shells

39 Answer 1 With ksh the read does not run in a subprocess The portable command is: sh c 'VAR = `date`; echo $ VAR '

40 Exercise 2 The command VAR1 = bla VAR2 = $ VAR1 / usr / bin / env lists an empty variable VAR2 if a classic Bourne shell is used. Why does this happen and is there a solution with which the expected output can be used for all shells?

41 Answer 2 Portable you can only set VAR1 and VAR2 on separate command lines. But then VAR1 and VAR2 become part of the main shell process

42 Exercise 3 What is the value of the variable FOO after executing the command FOO = bar pwd in the various shells? Why are the results different?

43 Answer 3 For all shells that implement builtins in conformity with POSIX, VAR = val pwd has no effect on the actual shell process. Reason: Historically, this has always had an effect on the shell with all builtins, but it was changed with ksh88 and from there to POSIX adopted

44 Exercise 4 Read the Bourne Shell, bash, ksh, bosh man pages and look for a way to create the following text when expanding the variable FOO: set only if $ FOO is not empty, otherwise $ FOO is empty only if $ FOO is empty, otherwise $ FOO is missing only if $ FOO does not exist, otherwise $ FOO Note: the explanation can be found in the section Parameter substitution. Is this possible with all shells? Do all shells provide usable man pages?

45 Answer 4 FOO = something $ {FOO: + set} FOO = $ {FOO: $ {FOO + empty}} unset FOO $ {FOO missing} Expansion is possible with all shells if they are at least compatible with the 1981 Bourne Shell version

46 Exercise 5 The command sh -c 'test -R && echo bla' outputs bla Explain why The command sh -c 'test 2 -GT 3; echo foo 'prints foo with some shells, but not all. Explain why

47 Answer 5 Because R is a string, the length of which is> 0 Because GT is an illegal operator, the historical Bourne shell has an exit () of the whole shell, but with PO SIX compatibility only exit () with exit code! = 0 for the test command

48 Exercise 6 The command: sh -c 'exit'; echo $? Outputs 210, Explain why not from which shell and under what circumstances do you manage to achieve the expected result when you invoke the specified command there? With: ksh -c 'exit'; echo $? does it never work, what could be the reason?

49 Answer is & 0xFF If you are working from the bosh with set o fullexitcode and you are on a modern OS, it works. Ksh masks the parameter of the built-in exit command with 0xFF before the actual exit () is executed

50 Exercise 7 Write a small ksh script that displays a menu with 3 options.

51 Answer 7 select i in who where what; do echo Selected: $ i from number: $ REPLY break done

52 Exercise 8 Write this ksh script: color = brown-yellow-blue if [[$ color == * yellow *]]; then fi echo with yellow in such a way that it also delivers the desired result with the Bourne Shell

53 Answer 8 color = brown-yellow-blue case $ color in * yellow *) echo with yellow ;; esac

54 How POSIX standardization works Observe the development of UNIX variants Recognize trends Document existing implementations Features that contradict what was previously available must not be standardized Previous exceptions: POSIX: getpgrp () / setpgrp () is in conflict with BSD Drepper: fexec ( ) and getline () conflict with UNOS and libschily

55 Problems with POSIX and the Shell Normally, POSIX standardizes what is sufficiently uniformly available on the market. However, there are many shell implementations with many mutually incompatible extensions. Therefore, the initiative here must come from standardization and influence all shells. The normal method, everything To designate partially incompatible ones as unspecified would be unusable. Therefore, sensible shell extensions are taken up and reformulated by the POSIX committee

56 Planning for POSIX Issue 8 Support for the ksh $ '...' expansion C similar string escape representation, e.g. $ '\ n' Unicode characters with $ '\ uxxxx' Reserved new POSIX namespace List the incompatible builtins POSIX flags for set (1) Check or activate extended features 32 bit exit codes using waitid () Presumably with the help of a new variable $ / siginfo_t make values ​​available in trap commands

57 Search paths for built-in commands All shells have their own built-in commands Many of the commands correspond to normal UNIX commands with slight deviations Normal shell rule: a built-in always has priority PATH enables a selection with separate binaries For the ksh93 integration in OpenSolaris, PATH was required and implemented for built-ins With ksh93 with the builtin command: Change PATH assignment for each builtin Deactivate or reactivate individual builtins

58 Search paths for built-in commands Most of the ksh93 built-ins are assigned to the path / usr / ast / bin./ usr / ast / bin is not in the normal PATH. A few commands are assigned to / usr / bin. However, only on Solaris One such command on Solaris is getconf because of POSIX getconf PATH. An inexperienced user does not see these built-in commands

59 Thank you! Url: