newLISP® v.10.0.2 Release Notes March 9th, 2009

Versions 10.0.1 and 10.0.2 are maintenance updates to release 10.0.0. Some minor feature enhancements and changes are noted in the next (search for 10.0.1 and 10.0.2).

newLISP version 10.0 marks a new generation of newLISP with new abilities, such as reference returns instead of copies for many built-in functions and a generalized method to modify lists, arrays and strings in-place. Version 10.0 is also a cleanup and streamlining of functionality added during the 8.x and 9.x series of releases.

Other details in the built-in functions API have been changed to make functions work better together. Use the conversion guide at the end of these release notes to help you convert existing files for use with newLISP 10.0.

The changes introduce incompatibilities with the previous versions of releases. Any production source running on the new generation of newLISP should be carefully checked against these release notes.

Many thanks to the community of newLISP users for their ideas and contributions.

Reference returns

Any built-in function returning a list, array or string, or an element of a list, array or string, will return a reference and not a copy if the list or array is bound to a symbol. Destructive functions can now work on the return value of other functions modifying the original list, array or string. The new setf can be used to modify an element of a list, array or string referencing it via one of the many available list, array and string accessors:

(set 'lst '(a b c))
(setf (first lst) 99)
lst → (99 b c)
  
(setf (lst -1) 'C)
lst →  (99 b C)
  
(set 'A '((a 1) (b 2) (c 3)))
(push 'z (assoc 'c A))
A → ((a 1) (b 2) (z c 3))
      
(set 'L '(a b (c d e f g)))
(= (replace 'f (nth (+ 1 1) L) 'z) '(c d e z g))
(= L '(a b (c d e z g)))

(setq s "abc")
(setf (s -1) "C")
(setf (nth 1 s) "B")
(setf (first s) "The A")
s → "The ABC"

The following functions return a reference to the list, array or string or element of a list or array, which can be modified by other functions. Previously these functions returned a copy (with the exception of set):

assoc, eval (index references v.10.0.2), first, last, lookup, ntn, replace, reverse, rotate, set, setf, setq, set-ref, set-ref-all, sort, swap

All control structures without local variables maintain a reference for the last returned value in the statement block, if that value was a reference:

amb, begin, if, if-not, when, unless, while, until, do-while, do-until, cond and case.
... and added in v.10.0.1:
for, doargs, dolist, dostring, dotimes, dotree.

Now all built-in functions except let, letn, letx and local can return references.

To force the return of a copy instead of a reference, enclose the expression in the new copy function:

; sort a copy not the original list
(sort (copy myList))

Normally there are few places in program code that rely on returning copies. In fact, the new possibility to work directly on returned references will give many opportunities to write terser and speedier code.

Functions setf and copy

setf is a new function working not only on symbol references like setq but also on any reference returned from an expression. The function is well known under the same name in other Lisp dialects and works in a similar way. setq and the new setf both point to the same internal code and are interchangeable. It is recommended to use setq for setting symbol variables and use setf whenever a sublist, sub array or substring is referenced, Note that any list, array or string changed must be anchored in a symbol referencing it. If no reference is found, an error message is returned: 

(setq l1 '(a b c))
(setq l2 '(d e f))

(setf 1 (append l1 l2) 99)
ERR: no symbol reference found in function setf

Allthough the partial lists are anchored in l1 and l2 the return value from the append function is not referenced by any symbol and cannot be used by setf.

To make destructive functions non-destructive, a new copy function has been introduced:

(replace elmnt (copy aList) newElmnt)

In the example aList will not be changed although replace is destructive.

Anaphoric system variable $it

The new system variable $it is used when doing self-referential assignments in setf and hashes. It can also be used in many cases as an alternative choice where the system variable $0 is used.

(setq lst '(1 2 3 4))
(setf (lst 2) (* $it 10)) 
lst → (1 2 30 4)

(setq str "abc")
(setf (str -1) (dup $it))
str → "abcc"

(MyHash "var" "hello")
(MyHash "var" (upper-case $it))
(MyHash → "HELLO")

The functions find-all, replace, set-ref and set-ref-all use $it as an alternative choice to $0.

The difference between $it and $0 is that $it will not retain the value beyond the evaluation of the self-referential expression it is part of, while $0 retains the value until used in another expression. $it is read-only, while $0 can be set by the user. Neither $it nor $0 - $15 are reentrant. Another subexpression using these variables will change them too.

Removed functions

The new setf function makes the functions set-assoc, assoc-set, set-nth, nth-set and ref-set obsolete. These functions have been removed. The integer function, deprecated since a few years, has been removed, and the shorter writing int should be used instead. The last section in this release notes gives examples of how to convert code to use the new or modified functions.

Changes in indexing syntax

The parenthesized syntax for assoc nth ref set-ref and set-ref-all has been eliminated. Only the traditional flat syntax is allowed. Note that the parenthesized syntax should not be confused with the syntax used in implicit indexing. In the abolished parenthesized syntax, the parentheses were syntactical sugar similar to parentheses used in the dotimes or for forms. The following example illustrates this:

(nth (theList idx)) ; parenthesized syntax

(setf (theList idx) value) ; implicit indexing syntax

In the first line the inner parentheses are part of a special form syntax. In the second example all parentheses are part of s-expressions.

When multiple indices or keys are used, a list is specified instead of a single index or key:

  
(nth i myList)             ; single index, flat form
(myList i)                 ; single index implicit index form
(nth idxList myList)       ; multiple indices, flat form
(myList idx1 idx2 ... )    ; multiple indices, implicit index, flat indices
(myList idxList)           ; multiple indices, inplicit index, index vector

(assoc key aList)          ; single key
(assoc list-of-keys aList) ; multiple key

Index list vectors are the same as those returned by ref and ref-all. Index vectors have been supported by pop and push for several years. Index vectors tie together nth, ref, ref-all, push and pop. Using index vectors in impicit indexing and the syntax of 'nth' gives 'nth' a uniform parameter arity regardless of the number of indices involved.

Other changes in functions

Changed Lisp source files

Although all old code should be thoroughly investigated for incompatibilities many newLISP source files will be compatible with version 10.0 without any changes. The following files needed minor changes.

exmples/client TCP/IP client example script
exmples/finger TCP/IP finger example script
exmples/server TCP/IP server example script
exmples/sqlite.cgi SQLite3 CGI example script
example/tcltk.lsp TclTk example
example/udp-client.lsp UDP client example
example/udp-server.lsp UDP server example
example/upload.cgi CGI file upload example
modules/ftp.lsp ftp module
modules/postscript.lsp postscript module
modules/stat.lsp statistics module (v.10.0.1)
modules/xmlrpc-client.lsp XML RPC client module
util/newlispdoc documentation generator (v.10.0.1)
util/syntax.lsp syntax highlighter
guiserver/newlisp-edit.lsp The editor in newLISP-GS
guiserver/pinballs-demo.lsp A demo file in newLISP-GS

The following files needed more changes to conform to v.10.0 based versions. Changes were made in v.10.0.2.

modules/ftp.lsp ftp module
modules/pop3.lsp pop3 module
modules/smtp.lsp smtp module

§

Conversion Guide

Conversion of eliminated and changed functions

set-nth, nth-set, set-assoc, assoc-set must be replaced by a combination of the new setf and nth, assoc. 

   
(nth-set (data idx) value) 
(nth-set idx data value)
(set-nth (data idx) value)
(set-nth idx data value)

; change to one of the following
(setf (nth idx data) value)
(setf (data idx) value)

(set-assoc (aList key) value)
; change to
(setf (assoc key aList) value)

data is either a list or array or a string. idx is either a single number or a list of numbers.

Multiple indices have to be put in a vector, and/or use implicit indexing:

   
; old obsolete indexing
(nth i j k data)
(nth (data i j k))

; change to one of the following
(nth (list i j k) data) 
(data (list i j k))
(data i j k)

data is either a list or an array, because strings are one-dimensional.

The conversion of eliminated functions will typically not be overlooked, because programs will throw errors when running into undefined functions. More subtle are the following changes which could escape detection if the code is not tested carefully

Nested expressions of built-in destructive functions must not rely on functions to return a copy. E.g. in previous versions, after (pop (sort myList)), myList was sorted, but the pop of the first element happened on a copy:

   
(set 'myList '(a b c d))
       
; in 9.4.5 and earlier
(pop (sort myList)) → a
myList → (a b c d) 
   
; in 10.0
(pop (sort myList)) → a
myList → (b c d)

To restore the old behavior and make functions non-destructive, use copy: 

   
; restore old non-destructive behavior in nested expressions
(set 'myList '(b d c a))
(pop (copy (sort myList)))
L → '(b d c a)

dec take away the quote before the symbol.

env without any parameters now returns an association list. To get the old behavior, use (exec "env") returning a flat list of environment strings:

 
; new return value
(env) → (("PATH" "/usr/bin:/bin:/usr/sbin:/sbin") ...)

; return old format
(exec "env") → ("PATH=/usr/bin:/bin:/usr/sbin:/sbin" ...)

eval-string has swapped the order of the error-handler and context parameters:

 
; in 9.4.5 and earlier
(eval-string theString error-handler the-context)

; in 10.0
(eval-string theString the-context error-handler)

Both the error-handler and context parameter are optional. The context parameter is usually the one used more frequently.

inc take away the quote before the symbol.

integer replace with the shorter written int.

net-receive take away the quote before the buffer symbol.

push now returns the modified list instead of the element pushed. Little code seems to rely on the element pushed as the return value. In many circumstances explicit list returns in user-defined functions can be eliminated:

 
(define (foo ...)
    ...
   (push x aList)
    aList)
    
; can be modified to

(define (foo ...)
    ...
    (push x aList))

set-locale now returns a list of locale string and decimal point character. Previously only the local string was returned. Code expecting the old return value can wrap the return value as in (first (set-locale ... )).

read-buffer take away the quote before the buffer symbol.

read-expr has changed the order of parameters and also changed functionality. Please consult the reference manual for details.

sys-error used to return an error number only. Now the function returns a list of error number and error text (v.10.0.2).

unless has lost the alternate clause. If there was no alternate clause, conversion is not necessary, but all instances of unless containing an alternate clause should be converted to if-not, which was introduced in version 9.4.5.

write-line has swapped parameters to (write-line <device> <string>) order, the same as other write- functions. The behavior has changed when the second parameter is missing, see the reference manual for details.

The order of loading $HOME/.init.lsp and $NEWLISPDIR/init.lsp has changed. The new version will look first for $HOME/.init.lsp and not load $NEWLISPDIR/init.lsp if $HOME/.init.lsp could be loaded. $NEWLISPDIR/init.lsp is only loaded is $HOME/.init.lsp does not exist.

Shared libraries newlisp.so or newlisp.dll only load a file specified with its full path in the new environment variable NEWLISPLIB_INIT.

∂