runawk is a small wrapper for the AWK interpreter that helps one
write standalone AWK scripts. Its main feature is to provide a
module/library system for AWK which is somewhat similar to Perl's
"use" command. It also allows you to select a preferred AWK
interpreter and to setup the environment for your scripts. It also
provides other helpful features, for example it includes
numerous useful of modules.
Major Changes IN RUNAWK-0.19.0
fix in runawk.c: \n was missed in "running '%s' failed: %s" error
message. The problem was seen on ancient (12 years old) HP-UX
fix in teets/test.mk: "diff -u" is not portable (SunOS, HP-UX),
DIFF_PROG variable is introduced to fix the problem
fix in modules/power_getopt.awk: after printing help message we
should exit immediately not running END section, s/exit/exitnow/
new function heapsort_values in heapsort.awk module
new function quicksort_values in quicksort.awk module
runawk is a small wrapper for the AWK interpreter that helps one
write standalone AWK scripts. Its main feature is to provide a
module/library system for AWK which is somewhat similar to Perl's
"use" command. It also allows you to select a preferred AWK
interpreter and to setup the environment for your scripts. It also
provides other helpful features, for example it includes
numerous useful of modules.
Major Changes IN RUNAWK-0.18.0
Makefile:
"install-dirs" target has been renamed to "installdirs"
At compile time MODULESDIR can contain a *list* of
colon-separated directories,
e.g. /usr/local/share/runawk:/usr/local/share/awk
Support for multiply applied options, e.g. -vvv for increasing
verbosity level. If option without arguments is multiply
applied, getarg() function returns a number of times it was
applied, not just 0 or 1.
New modules:
init_getopt.awk using alt_getopt.awk and used by power_getopt.awk.
Its goal is to initialize `long_opts' and `long_opts' variables
but not run `getopt' function.
heapsort.awk : heapsort :-)
quicksort.awk : quicksort :-)
sort.awk : either heapsort or quicksort,
the default is heapsort.
Unfortunately GAWK's asort() and asorti() functions
do *not* satisfy my needs. Another (and more important) reason
is a portability.
Improvements, clean-ups and fixes in regression tests.
Also,
runawk-0-18-0 was successfully tested on the following platforms:
NetBSD-5.0/x86, NetBSD-2.0/alpha, OpenBSD-4.5/x86,
FreeBSD-7.1/x86, FreeBSD-7.1/spark, Linux/x86 and
Darwin/ppc.
RUNAWK is a small wrapper for the AWK interpreter that helps one write
standalone AWK scripts. Its main feature is to provide a
module/library system for AWK which is somewhat similar to Perl's
"use" command. It also allows you to select a preferred AWK
interpreter and to setup the environment for your scripts. RUNAWK
makes programming AWK easy and efficient. RUNAWK also provides many
useful AWK modules.
In a multilined code passed to runawk using option -e, spaces
are allowed before #directives.
After inventing alt_getopt.awk module there is no reason for
heuristics that detects whether to add `-' to AWK arguments or
not. So I've removed this heuristics. Use alt_getopt.awk module
or other "smart" module for handling options correctly!
alt_getopt.awk and power_getopt.awk:
FIX: for "abc:" short options specifier BSD and GNU getopt(3)
accept "-acb" and understand it as "-a -cb", they also accept
"-ac b" and also translate it to "-a -cb". Now alt_getopt.awk
and power_getopt.awk work the same way.
power_getopt.awk:
-h option doesn't print usage information, --help (and its short
synonym) does.
`shquote' transforms the string `str' by adding shell
escape and quoting characters to include it to the
system() and popen() functions as an argument, so that
the arguments will have the correct values after being
evaluated by the shell.
Inspired by NetBSD's shquote(3) from libc.
runcmd.awk, implementing functions runcmd1() and xruncmd1() runcmd1(CMD, OPTS, FILE):
wrapper for function system() that runs a command CMD
with options OPTS and one filename FILE. Unlike
system(CMD " " OPTS " " FILE) the function runcmd1()
handles correctly FILE and CMD containing spaces, single
quote, double quote, tilde etc.
xruncmd1(FILE):
safe wrapper for 'runcmd(1)'.
awk exits with error if running command failed.
isnum.awk, implementing trivial isnum() function,
see the source code.
alt_join.awk, implementing the following functions: join_keys(HASH, SEP):
returns string consisting of all keys from HASH separated
by SEP.
join_values(HASH, SEP):
returns string consisting of all values from HASH separated
by SEP.
join_by_numkeys (ARRAY, SEP [, START [, END]]):
returns string consisting of all values from ARRAY
separated by SEP. Indices from START (default: 1) to END
(default: +inf) are analysed. Collecting values is stopped
on index absent in ARRAY.
RUNAWK is a small wrapper for AWK interpreter that helps to write the
standalone programs in AWK. It provides MODULES for AWK similar to
PERL's "use" command and other powerful features. Dozens of ready to
use modules are also provided.
Lots of demo programs for most runawk modules were created and
they are in examples/ subdirectory now.
New MEGA module ;-) power_getopt.awk
See the documentation and demo program examples/demo_power_getopt.
It makes options handling REALLY easy (see below).
New modules:
embed_str.awk
has_suffix.awk
has_prefix.awk
readfile.awk
modinfo.awk
Minor fixes and improvements in dirname.awk and basename.awk.
Now they are fully compatible with dirname(1) and basename(1)
RUNAWK sets the following environment variables for the child awk
subprocess:
RUNAWK_MODC - A number of modules (-f filename) passed to AWK
RUNAWK_MODV_<n> - Full path to the module #n,
where n is in [0..RUNAWK_MODC) range.
RUNAWK sets RUNAWK_ART_STDIN environment variable for the child awk
subprocess to 1 if additional/artificial `-' was added to the list
to awk's arguments.
Makefile:
bmake-ism were removed. Now Makefile is fully compatible with
FreeBSD make.
CLEANFILES target is used instead of hand-made rules
Minor fix in 'test_all' target
Power_GetOpt.awk
The most powerful feature of this release is power_getopt.awk module.
It provides a very powerful and very easy way to handle options.
Everything is in the usage message, you should do anything at all.
I think example below is easy.
Example Code
% cat 1.awk
#!/usr/bin/env runawk
#use "power_getopt.awk"
#.begin-str help
# power_getopt - program demonstrating a power of power_getopt.awk module
# usage: power_getopt [OPTIONS]
# OPTIONS:
# -h|--help display this screen
# -f|--flag flag
# --long-flag long flag only
# -s short flag only
# =F|--FLAG flag with value
#.end-str
BEGIN {
print "f --- " getarg("f")
print "flag --- " getarg("flag")
print "long-flag --- " getarg("long-flag")
print "s --- " getarg("s")
print "F --- " getarg("F", "default1")
print "FLAG --- " getarg("FLAG", "default2")
exit 0
}
./1.awk
% ./1.awk
f --- 0
flag --- 0
long-flag --- 0
s --- 0
F --- default1
FLAG --- default2
./1.awk -h
% ./1.awk -h
power_getopt - program demonstrating a power of power_getopt.awk module
usage: power_getopt [OPTIONS]
OPTIONS:
-h|--help display this screen
-f|--flag flag
--long-flag long flag only
-s short flag only
-F|--FLAG flag with value
./1.awk -f
% ./1.awk -f
f --- 1
flag --- 1
long-flag --- 0
s --- 0
F --- default1
FLAG --- default2
./1.awk -F value
% ./1.awk -F value
f --- 0
flag --- 0
long-flag --- 0
s --- 0
F --- value
FLAG --- value
./1.awk --FLAG=value
% ./1.awk --FLAG=value
f --- 0
flag --- 0
long-flag --- 0
s --- 0
F --- value
FLAG --- value
After years of using AWK for programming I've found that despite of
its simplicity and limitations AWK is good enough for scripting a wide
range of different tasks. AWK is not as poweful as their bigger
counterparts like Perl, Ruby, TCL and others but it has their own
advantages like compactness, simplicity and availability on almost all
UNIX-like systems. I personally also like its data-driven nature and
token orientation, very useful technique for simple text processing
utilities.
But! Unfortunately awk interpreters lacks some important features and
sometimes work not as good as it whould be.
Problems I see (some of them, of course)
AWK lacks support for modules. Even if I create small programs, I
often want to use the functions created earlier and already used in
other scripts. That is, it whould great to orginise functions into
so called libraries (modules).
In order to pass arguments to #!/usr/bin/awk -f script (not to awk
interpreter), it is necessary to prepand a list of
arguments with -- (two minus signes). In my view, this looks badly.
Example:
awk_program:
#!/usr/bin/awk -f
BEGIN {
for (i=1; i < ARGC; ++i){
printf "ARGV [%d]=%s\n", i, ARGV [i]
}
}
When #!/usr/bin/awk -f script handles arguments (options) and wants
to read from stdin, it is necessary to add
/dev/stdin (or `-') as a last argument explicitly.
Example:
awk_program:
#!/usr/bin/awk -f
BEGIN {
if (ARGV [1] == "--flag"){
flag = 1
ARGV [1] = "" # to not read file named "--flag"
}
}
{
print "flag=" flag " $0=" $0
}
Shell session:
% echo test | awk_program -- --flag
% echo test | awk_program -- --flag /dev/stdin
flag=1 $0=test
%
Ideally awk_program should work like this
% echo test | awk_program --flag
flag=1 $0=test
%
runawk was created to solve all these problems
OPTIONS
-h|--help
Display help information.
-V|--version
Display version information.
-d|--debug
Turn on a debugging mode in which runawk prints argument list
with which real awk interpreter will be run.
-i|--with-stdin
Always add stdin file name to a list of awk arguments
-I|--without-stdin
Do not add stdin file name to a list of awk arguments
-e|--executeprogram
Specify program. If -e is not specified program is read from
program_file.
DETAILS/INTERNALS
Standalone script
Under UNIX-like OS-es you can use runawk
by beginning your script with
#!/usr/local/bin/runawk
line or something like this instead of
#!/usr/bin/awk -f
or similar.
AWK modules
In order to activate modules you should add them into awk script like this
#use "module1.awk"
#use "module2.awk"
that is the line that specifies module name is treated as a comment line
by normal AWK interpreter but is processed by runawk especially.
Note that #use should begin with column 0,
no spaces are allowed before it and no spaces are allowed between
# and use.
Also note that AWK modules can also "use" another modules and so forth.
All them are collected in a depth-first order
and each one is added to the list of
awk interpreter arguments prepanded with -f option.
That is #use directive is *NOT* similar to #include in
C programming language,
runawk's module code is not inserted into the place of #use.
Runawk's modules are closer to Perl's "use" command.
In case some module is mentioned more than once, only one -f
will be added for it, i.e duplications are removed automatically.
Position of #use directive in a source file does matter, i.e.
the earlier module is mentioned, the earlier -f will be generated for it.
Modules are first searched in a directory where main
program (or module in which #use directive is specified) is placed.
If it is not found there, then
AWKPATH environment variable is
checked. AWKPATH keeps a colon separated
list of search directories.
Finally, module is searched in system runawk modules directory,
by default PREFIX/share/runawk but this can be changed at build time.
An absolute path of the module can also be specified.
AWK interpreter and its arguments
In order to pass arguments to AWK script correctly, runawk
treats their arguments beginning with `-' sign (minus) especially.
The following command
runawk prog2 -x -f=file -o=output file1 file2
or
/path/to/prog2 -x -f=file -o=output file1 file2
will actually run
awk -f prog2 -- -x -f=file -o=output file1 file2
therefore -s, -f, -o options will be passed to ARGV/ARGC awk's variables
together with file1 and file2. If all arguments begin with `-' (minus),
runawk will add stdin filename to the end of argument list,
(unless -I option is specified) i.e. running
runawk prog3 --value=value
or
/path/to/prog3 --value=value
will actually run the following
awk -f prog3 -- --value=value /dev/stdin
Program as an argument
Like some other interpreters
runawk can obtain the script from a command line like this
For some reason you may prefer one AWK interpreter or another with a help of
#interp command like this
file prog:
#!/usr/local/bin/runawk
#use "A.awk"
#use "B.awk"
#interp "/usr/pkg/bin/nbawk"
# your code here
...
The reason may be efficiency for a particular task, useful but
not standard extensions or enything else.
Note that #interp directive should also begin with column 0,
no spaces are allowed before it and between # and interp.
Setting environment
In some cases you may want to run AWK interpreter with a
specific environment. For example, your script may be oriented to
process ASCII text only. In this case you can run AWK with LC_CTYPE=C
environment and use regexp ranges.
runawk provides #env directive for this. Strings inside double quotes
is passed to putenv(3) libc function.
Example:
file prog:
#!/usr/local/bin/runawk
#env "LC_ALL=C"
$1 ~ /^[A-Z]+$/ { # A-Z is valid if LC_CTYPE=C
print $1
}
EXIT STATUS
If AWK interpreter exits normally, runawk exits with its exit
status. If AWK interpreter was killed by signal, runawk
exits with exit status 128+signal.
ENVIRONMENT
AWKPATH
Colon separated list of directories where awk modules are searched.
RUNAWK_AWKPROG
Sets the path to the AWK interpreter, used by default,
i.e. this variable overrides the compile-time default.
Note that #interp directive overrides this.
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
BUGS/FEEDBACK
Please send any comments, questions, bug reports etc. to me by e-mail
or (even better) register them at sourceforge project home. Feature
requests are also welcomed.
