Awk.Info

"Cause a little auk awk
goes a long way."

About awk.info
 »  table of contents
 »  featured topics
 »  page tags


About Awk
 »  advocacy
 »  learning
 »  history
 »  Wikipedia entry
 »  mascot
Implementations
 »  Awk (rarely used)
 »  Nawk (the-one-true, old)
 »  Gawk (widely used)
 »  Mawk
 »  Xgawk (gawk + xml + ...)
 »  Spawk (SQL + awk)
 »  Jawk (Awk in Java JVM)
 »  QTawk (extensions to gawk)
 »  Runawk (a runtime tool)
 »  platform support
Coding
 »  one-liners
 »  ten-liners
 »  tips
 »  the Awk 100
Community
 »  read our blog
 »  read/write the awk wiki
 »  discussion news group

Libraries
 »  Gawk
 »  Xgawk
 »  the Lawker library
Online doc
 »  reference card
 »  cheat sheet
 »  manual pages
 »  FAQ

Reading
 »  articles
 »  books:

WHAT'S NEW?

Mar 01: Michael Sanders demos an X-windows GUI for AWK.

Mar 01: Awk100#24: A. Lahm and E. de Rinaldis' patent search, in AWK

Feb 28: Tim Menzies asks this community to write an AWK cookbook.

Feb 28: Arnold Robbins announces a new debugger for GAWK.

Feb 28: Awk100#23: Premysl Janouch offers a IRC bot, In AWK

Feb 28: Updated: the AWK FAQ

Feb 28: Tim Menzies offers a tiny content management system, in Awk.

Jan 31: Comment system added to awk.info. For example, see discussion bottom of ?keys2awk

Jan 31: Martin Cohen shows that Gawk can handle massively long strings (300 million characters).

Jan 31: The AWK FAQ is being updated. For comments/ corrections/ extensions, please mail tim@menzies.us

Jan 31: Martin Cohen finds Awk on the Android platform.

Jan 31: Aleksey Cheusov released a new version of runawk.

Jan 31: Hirofumi Saito contributes a candidate Awk mascot.

Jan 31: Michael Sanders shows how to quickly build an AWK GUI for windows.

Jan 31: Hyung-Hwan Chung offers QSE, an embeddable Awk Interpreter.

[More ...]

Bookmark and Share

categories: Jan,2010,HyungC

QSE: an Embeddable Awk Interpreter

Hyung-Hwan Chung offers QSE, an embeddable Awk.

Download

See QSE.

About QSE

QSE is a code library that implements various Unix utilities in an embeddable form and provides a set of APIs to embed them into an application. The APIs have been designed to be flexible enough to access various aspects of an embedding application and an embedded object from each other.

By embedding a Unix utility into an application, a developer is relieved of problems caused by interacting with external programs and can have tighter control over it. Currently the library contains the following utilities:

  • AWK Interpreter
  • CUT Text Cutter
  • SED Stream Editor

QSEAWK is an embeddable AWK interpreter and is a part of the QSE library. The interpreter implements the language described in the book the AWK Proramming Language, with some extensions. Its design focuses on building a flexible and robust embedding API with minimal platform dependency. An embedding application is capable of:

  • adding new global variables and functions.
  • getting and set the value of a global variable.
  • calling a function with or without parameters and getting its return value.
  • customizing I/O handlers for file, pipe, console I/O.
  • creating multiple interpreters independent of each other.
  • running a single script with different I/O streams independently.
  • changing language features by setting options.
  • and more

But Why QSE?

An advantage of embedding a scripting language into an application is that you can extend an application by changing scripts instead of recompiling the whole application. As an AWK lover, I was a bit disappointed that I could not find any embedded implementations of the AWK programming language that I could squeeze into my applications.

QSE is designed to embedded Awk into other applications, rather than being used as a standalone tool (though it is not impossible). Why did I choose AWK as an embedded language? Simple. Both I and my clients liked it and were too lazy to learn a new scripting language.

Also, an embedded solution is a better solution that calling an external AWK interpreter:

  • There is the extra overhead of forking an external process.
  • There was an absense of any AWK interpreters on the target platform
  • I found version issues of an AWK interpreter.
  • I an unable to extend the interpreter itself. (e.g. adding an application specific builtin function like hash_passwor).

Hence, my conclusion was to implement an embeddable awk interpreter myself.

Example

One of the applications I wrote implements password change policy in an AWK script. The application calls the "is_password_acceptable" function with the password entered by a user, before having accepted the user-entered password. It checks its return value and determines to accept the password.

Of course, the engine is prearranged with global variables PASSWD_HISTORY_SIZE, and PASSWORD_HISTORY_FILE, and a buitin function hash_password() using flexiable QSEAWK API functions upon application start-up.

For example, here is the sample AWK function below.

function is_password_acceptable(passwd)
{
    # check the password length
    if (length(passwd) < 8) return 0;
    # check if the password is composed of alphabets or digits only
    if (passwd ~ /^([[:alpha:]]+|[[:digit:]]+)$/) 
	return 0;
    if (PASSWD_HISTORY_SIZE > 0)
    {
	hashed = hash_passwd(passwd);
	# check if the password is found in the history file
	while ((getline entry < PASSWD_HISTORY_FILE) > 0)
	{
	    if (hashed == entry)
	    {
		# an entry is found in the history.
		# reject the password
		close (PASSWD_HISTORY_FILE);
		return 0;
	    }
	}
	close (PASSWD_HISTORY_FILE);
    }
    return 1;
}

The C application's password policy function is roughly shown below also. Note that this application utilized the embedded QSEAWK interprerter in an event(password change)-driven way, not entering the BEGIN, pattern-action blocks, END loops.

int is_password_acceptable (qse_awk_rtx_t* rtx, const char* passwd)
{
    qse_awk_val_t* ret, * arg[1];
    qse_bool_t ok;
    
    ... abbreviated ...
    
    /* transform a character string to an AWK value */
    arg[0] = qse_awk_rtx_makestrval0 (rtx, passwd);
    
    ... abbreviated ...
    
    /* increment the reference counter of arg[0] */
    qse_awk_rtx_refupval (rtx, arg[0]);
    /* call "is_password_acceptable" */
    ret = qse_awk_rtx_call (rtx, "is_password_acceptable", arg, 1);
    /* decrement the reference counter of arg[0] */
    qse_awk_rtx_refdownval (rtx, arg[0]);
    
    ... abbreviated ...
    
    /* get the boolean value from the return value */
    ok = qse_awk_rtx_valtobool (awk_rtx, ret);
    /* decrement the reference counter of the return value */
    qse_awk_rtx_refdownval (rtx, ret);
    /* accept or reject? */
    return ok? 0: -1;
}
After all, I managed to get rid of any needs to recompile the whole application and redeploy it whenever a client asks for password policy change.
blog comments powered by Disqus